This piece of code works as expected:
for var in a 'b c' d;
do
echo $var;
done
The bash script loops through 3 arguments printing
a
b c
d
However, if this string is read in via jq , and then looped over like so:
JSON_FILE=path/to/jsonfile.json
ARGUMENTS=$(jq -r '.arguments' "${JSON_FILE}")
for var in ${ARGUMENTS};
do
echo $var;
done
The result is 4 arguments as follows:
a
'b
c'
d
Example json file for reference:
{
"arguments" : "a 'b c' d"
}
What is the reason for this? I tried putting quotes around the variable like suggested in other SO answers but that caused everything to just be handled as 1 argument.
What can I do to get the behavior of the first case (3 arguments)?
What is the reason for this?
The word splitting expansion is run over unquoted results of other expansions. Because ${ARGUMENTS} expansion in for var in ${ARGUMENTS}; is unquoted, word splitting is performed. No, word splitting ignores quotes resulted from variable expansion - it only cares about whitespaces.
What can I do to get the behavior of the first case (3 arguments)?
The good way™ would be to write your own parser, to parse the quotes inside the strings and split the argument depending on the quotes.
I advise to use xargs, it (by default, usually a confusing behavior) parses quotes in the input strings:
$ arguments="a 'b c' d"
$ echo "${arguments}" | xargs -n1 echo
a
b c
d
# convert to array
$ readarray -d '' arr < <(<<<"${arguments}" xargs printf "%s\0")
As presented in the other answer, you may use eval, but please do not, eval is evil and will run expansions over the input string.
Change IFS to a new line to make it work:
...
IFS='\n'; for var in $ARGUMENTS;
do
echo $var;
done
Related
Linux bash: why the two shell script as follow had different result?
[root#yumserver ~]# data="a,b,c";IFS=",";for i in $data;do echo $i;done
a
b
c
[root#yumserver ~]# IFS=",";for i in a,b,c;do echo $i;done
a b c
expect output: the second script also output:
a
b
c
I should understood what #M.NejatAydin means。Thanks also #EdMorton,#HaimCohen!
[root#k8smaster01 ~]# set -x;data="a,b,c";IFS=",";echo $data;echo "$data";for i in $data;do echo $i;done
+ data=a,b,c
+ IFS=,
+ echo a b c
a b c
+ echo a,b,c
a,b,c
+ for i in '$data'
+ echo a
a
+ for i in '$data'
+ echo b
b
+ for i in '$data'
+ echo c
c
[root#k8smaster01 ~]# IFS=",";for i in a,b,c;do echo $i;done
+ IFS=,
+ for i in a,b,c
+ echo a b c
a b c
Word splitting is performed on the results of unquoted expansions (specifically, parameter expansions, command substitutions, and arithmetic expansions, with a few exceptions which are not relevant here). The literal string a,b,c in the
second for loop is not an expansion at all. Thus, word splitting is not performed on that literal string. But note that, in the second example, word splitting is still performed on $i (an unquoted expansion) in the command echo $i.
It seems the point of confusion is where and when the IFS is used. It is used in the word splitting phase following an (unquoted) expansion. It is not used when the shell reads its input and breaks the input into words, which is an earlier phase.
Note: IFS is also used in other contexts (eg, by the read builtin command) which are not relevant to this question.
#HaimCohen explained in detail why you get a different result with those two approaches. Which is what you asked. His answer is correct, it should get upvoted and accepted.
Just a trivial addition from my side: you can easily modify the second of your approaches however if you define the variable on the fly:
IFS=",";for i in ${var="a,b,c"};do echo $i;done
I'm trying to join input $* which is one parameter consisting of all the parameters added together.
This works.
#!/bin/bash
foo() {
params="${*}"
echo "${params//[[:space:]]/-}"
}
foo 1 2 3 4
1-2-3-4
However, is it possible to skip the assignment of variable?
"${"${*}"//[[:space:]]/-}"
I'm getting bad substitution error.
I can also do
: "${*}"
echo "${_//[[:space:]]/-}"
But it feels hacky.
One option could be to set bash's internal field separator, IFS, to - locally and just echo "$*":
foo() {
local IFS=$'-'
echo "$*"
}
To answer your question, you can do global pattern substitutions on the positional parameters like this:
${*//pat/sub}
${#//pat/sub}
And also arrays like this:
${arr[*]//pat/sub}
${arr[#]//pat/sub}
This won’t join the parameters, but substitute inside them.
Setting IFS to dash adds a dash in between each parameter for echo "$*", or p=$*, but won’t replace anything inside a parameter.
Eg:
$ set -- aa bb 'cc cc'
$ IFS=-
$ echo "$*"
aa-bb-cc cc
To remove all whitespace, including inside a parameter, you can combine them:
IFS=-
echo "${*//[[:space:]]/-}"
Or just assign to a name first, like you were doing:
no_spaces=$*
echo "${no_spaces//[[:space:]]/-}"
when I write something like this:
x = `grep "#include $1 | cut -f2"`
or any use with grep, cut like:
x = `grep string file.c`
I don't understand if x is an array or one long string? because when I write
echo ${#x[*]}
it prints 1, but I can write:
for d in `grep....`
as it was an array, please explain.
It is giving you a single long string. This is not called the "return value" of grep or cut, but rather the "standard output" (the text they print which you capture with backticks or perhaps clearer with $(...)).
What's happening here is that you get one single string, perhaps even with newlines inside, and then you iterate over it with your for d in .... In Bash, iterating over a string splits on spaces, so you get one d value for each word. Try this to see it in action, plus a way to avoid it:
x="foo bar baz"
for d in $x; do echo $d; done
for d in "$x"; do echo $d; done
If you quote in the loop, splitting on spaces will not occur.
x is a string.
In your example for loops through words.
Someone please help to explain how this work? About the single quote it should not interpret anything but it is not working as what i expected. I expect to get echo $testvar value exactly '"123b"'.
a="testvar"
b="'"123b"'"
eval $a='$b'
echo $testvar
'123b'
a="testvar"
b='"123b"'
eval $a='$b'
echo $testvar
"123b"
a="testvar"
b='"123b"'
eval $a=$b
echo $testvar
123b
Guessing that you want testvar to be <single-quote><double-quote>123b<double-quote><single-quote>:
testvar=\'\"123b\"\'
Consider this in C or Java:
char* str = "123b";
printf("%s\n", str);
String str = "123b";
System.out.println(str);
Why does this write 123b when we clearly used double quotes? Why doesn't it write "123b", with quotes?
The answer is that the quotes are not part of the data. The quotes are used by the programming language to determine where strings start and stop, but they're not in any way part of the string. This is just as true for Bash as for C and Java.
Just like there's no way in Java to differentiate Strings created with "123" + "b" and "123b", there's no way in Bash to tell that b='"123b"' used single quotes in its definition, as opposed to e.g. b=\"123b\".
If given a variable you want to assign its value surrounded by single quotes, you can use e.g.
printf -v testvar "'%s'" "$b"
But this just adds new literal single quotes around a string. It doesn't and cannot care how b was originally quoted, because that information is stored.
To instead add a layer of escaping to a variable, so that when evaluated once it turns into a literal string identical to your input, you can use:
printf -v testvar "%q" "$b"
This will produce a value which is quoted equivalently but not necessarily identically to your original definition. For "value" (a literal with double quotes in it), it may produce \"value\" or '"value"' or '"'value'"' which all evaluate exactly to "value".
I have this test variable in ZSH:
test_str='echo "a \" b c"'
I'd like to parse this into an array of two strings ("echo" "a \" b c").
i.e. Read test_str as the shell itself would and give me back an array of
arguments.
Please note that I'm not looking to split on white space or anything like that. This is really about parsing arbitrarily complex strings into shell arguments.
Zsh has (z) modifier:
ARGS=( ${(z)test_str} )
. But this will produce echo and "a \" b c", it won’t unquote string. To unquote you have to use Q modifier:
ARGS=( ${(Q)${(z)test_str}} )
: results in having echo and a " b c in $ARGS array. Neither would execute code in … or $(…), but (z) will split $(false true) into one argument.
that is to say:
% testfoo=${(z):-'blah $(false true)'}; echo $testfoo[2]
$(false true)
A simpler (?) answer is hinted at by the wording of the question. To set shell argument, use set:
#!/bin/sh
test_str='echo "a \" b"'
eval set $test_str
for i; do echo $i; done
This sets $1 to echo and $2 to a " b. eval certainly has risks, but this is portable sh. It does not assign to an array, of course, but you can use $# in the normal way.