This question already has answers here:
Variable containing multiple args with quotes in Bash
(4 answers)
Closed 4 years ago.
My goal is to call a script with an variable as it's arguments. For example, I have the following two files:
print_args.sh:
echo "Length: $#"
for i in "$#"; do echo "$i"; done
caller.sh:
ARGS="foo \"Hello, World\" bar"
./test.sh $ARGS
When I run:
./print_args.sh foo "Hello, World" bar
print_args.sh get's called with 3 arguments:
Length 3
foo
Hello, World
bar
However, when running it via caller.sh instead I get 4 args:
Length: 4
foo
"Hello,
World"
bar
What's going on here? How can I get caller.sh to perform as expected?
Note: I don't have control over ARGS it's passed in as an environment variable.
How can I get test_caller.sh to perform as expected?
Use an array:
#!/bin/bash
args=( foo "Hello, World" bar )
./test.sh "${args[#]}"
If you pass an unquoted variable, then word splitting occurs on its contents before it is passed to ./test.sh. You are essentially calling:
'./test.sh' 'foo' '"Hello,' 'World"' 'bar'
Where the single quotes are used to indicate the start and end of words. The double quotes have no syntactic meaning, they are just characters in a string at this point.
If you can convince the users of your script to change the format of the environment variable, then you could split it based on a different separator:
ARGS="foo:Hello World:bar"
set -f # disable glob expansion
IFS=: args=( $ARGS )
set +f # re-enable it
Now you have an array args which you can use as above.
One (really nasty, dangerous) way to get your string into an array would be to use the notorious eval:
readarray -t args < <(eval printf '%s\\n' $ARGS)
Now you have an array args which you can use as in the previous examples.
But this is a big security problem:
ARGS="foo \"Hello, World\" bar \$(ls -l)"
Replace ls -l with any command that you want to execute...
Related
This question already has answers here:
Script fails with spaces in directory names
(1 answer)
How do I split a string on a delimiter in Bash?
(37 answers)
Closed last month.
#!/bin/bash
get_data() {
echo foo
echo "bar baz"
}
for i in $(get_data); do
echo "got: $i"
done
actual output is
got: foo
got: bar
got: baz
what I want is
got: foo
got: bar baz
How do I get the variable i in for i in $(...) to be filled per-line?
Here is how I would do it:
get_data | while IFS= read -r i ; do
echo "got $i"
done
The reason why you are not getting what you want with for is that $(get_data) gets expanded by bash to foo bar baz; by default, new lines are treated as word boundaries like space and nothing more. Similarly, using for to loop over contents of a file (for line in $(cat file)) will not work as expected.
P.S. You could modify the IFS (field separator) like this:
IFS=$'\n'
as suggested in one of the comments; however I prefer my solution as more explicit and less error-prone.
I need to insert an array of arguments into an eval string executed via bash -c.
In this particular case, it's not possible to pass them separately as proper arguments (it's for an flock invocation which doesn't accept arguments for a -c script).
Smth like this:
flock f.lock -c 'do_stuff '"${ARGS[#]}"'; do_other_stuff'
How do I quote them properly so that they are correctly parsed into a sequence of arguments, even if they contain spaces or Bash special syntax?
Don't! It is going to be error prone and give you pain. Just:
{
flock 9
do_stuff "${ARGS[#]}"
do_other_stuff
} 9>f.lock
Anyway, split the operation into two:
first, safely transfer the environment to the subshell
then execute what you want to execute in a normal way
And it's just:
bash -c "$(declare -p ARGS)"'; do_stuff "${ARGS[#]}"; do_other_stuff'
Use Parameter transformation (new in Bash 4.4) with the Q (quote) operator which is specifically designed for this:
Q The expansion is a string that is the value of parameter quoted
in a format that can be reused as input.
$ ARGS=("foo bar" 'baz\n'\' '$xyzzy')
$ echo 'do_stuff '"${ARGS[*]#Q}"'; do_other_stuff'
do_stuff 'foo bar' 'baz\n'\''' '$xyzzy'; do_other_stuff
Note the use of * instead of the usual # since the code needs to be a single argument. Using # leads to erroneous behavior in some cases:
$ bash -xc "echo ${ARGS[#]#Q}; do_other_stuff"
+ echo 'foo bar'
foo bar
$ bash -xc "echo ${ARGS[*]#Q}; do_other_stuff"
+ echo 'foo bar' 'baz\n'\''' '$xyzzy'
foo bar baz\n' $xyzzy
+ do_other_stuff
You can even use this syntax for $*:
'do_stuff '"${*#Q}"' other args; do other stuff'
This question already has an answer here:
Passing argument containing space in shell script
(1 answer)
Closed 4 years ago.
How to solve below problem.
myscript.sh
echo First argument: $1
echo Second argument: $2
for i in $* do
echo $i
done
now executing the script:
$./myscript.sh "this is" value
First argument: this is
Second argument: value
this
is
value
When I am printing $1 I am getting the value "this is"
but inside loop first argument coming as "this"
I want when the loop execute it will print
this is
value
This is due to the usage of $* instead of "$#" to get all of the arguments. From ShellCheck:
$*, unquoted, is subject to word splitting and globbing.
Let's say you have three arguments: baz, foo bar and *
"$#" will expand into exactly that: baz, foo bar and *
$* will expand into multiple other arguments: baz, foo, bar, file.txt
and otherfile.jpg
For the desired behavior you can do this:
echo "First argument: $1"
echo "Second argument: $2"
for i in "$#"
do echo "$i"
done
I've got a program that I want to call by passing parameters from a shell variable. Throughout this question, I am going to assume that it is given by
#!/bin/sh
echo $#
i.e. that it prints out the number of arguments that are passed to it. Let's call it count-args.
I call my program like this:
X="arg1 arg2"
count-args $X
This works quite well. But now one of my arguments has a whitespace in it and I can't find a way to escape it, e.g. the following things do not work:
X="Hello\ World"
X="Hello\\ World"
X="'Hello World'"
In all of the cases, my program count-args prints out 2. I want to find a way so I can pass the string Hello World and that it returns 1 instead. How?
Just for clarification: I do not want to pass all parameters as a single string, e.g.
X="Hello World"
count-args $X
should print out 2. I want a way to pass parameters that contain whitespaces.
Use an array to store multiple, space-containing arguments.
$ args=("first one" "second one")
$ count-args "${args[#]}"
2
This can be solved with xargs. By replacing
count-args $X
with
echo $X | xargs count-args
I can use backslashes to escape whitespaces in $X, e.g.
X="Hello\\ World"
echo $X | xargs count-args
prints out 1 and
X="Hello World"
echo $X | xargs count-args
prints out 2.
count-args "$X"
The quotes ensure in bash, that the whole content of variable X is passed as a single parameter.
Your Counting script:
$ cat ./params.sh
#!/bin/sh
echo $#
For completeness here is what happens with various arguments:
$ ./params.sh
0
$ ./params.sh 1 2
2
$ ./params.sh
0
$ ./params.sh 1
1
$ ./params.sh 1 2
2
$ ./params.sh "1 2"
1
And here is what you get with variables:
$ XYZ="1 2" sh -c './params.sh $XYZ'
2
$ XYZ="1 2" sh -c './params.sh "$XYZ"'
1
Taking this a bit further:
$ cat params-printer.sh
#!/bin/sh
echo "Count: $#"
echo "1 : '$1'"
echo "2 : '$2'"
We get:
$ XYZ="1 2" sh -c './params-printer.sh "$XYZ"'
Count: 1
1 : '1 2'
2 : ''
This looks like what you wanted to do.
Now: If you have a script you cannot control and neither can you control the way the script is invoked. Then there is very little you can do to prevent a variable with spaces turning into multiple arguments.
There are quite a few questions around this on StackOverflow which indicate that you need the ability to control how the command is invoked else there is little you can do.
Passing arguments with spaces between (bash) script
Passing a string with spaces as a function argument in bash
Passing arguments to a command in Bash script with spaces
And wow! this has been asked so many times before:
How to pass argument with spaces to a shell script function
This question already has answers here:
What is the difference between $# and $* in shell scripts?
(3 answers)
Closed 8 years ago.
It seems to me that they both store all the command-line arguments.
So is there a difference between the two?
The difference is subtle; "$*" creates one argument separated by the $IFS variable, while "$#" will expand into separate arguments. As an example, consider:
for i in "$#"; do echo "# '$i'"; done
for i in "$*"; do echo "* '$i'"; done
When run with multiple arguments:
./testvar foo bar baz 'long arg'
# 'foo'
# 'bar'
# 'baz'
# 'long arg'
* 'foo bar baz long arg'
For more details:
http://www.gnu.org/software/bash/manual/bashref.html#Special-Parameters
$*
Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, it expands to a single word with the value of each parameter separated by the first character of the IFS special variable. That is, "$*" is equivalent to "$1c$2c...", where c is the first character of the value of the IFS variable. If IFS is unset, the parameters are separated by spaces. If IFS is null, the parameters are joined without intervening separators.
$#
Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is, "$#" is equivalent to "$1" "$2" .... If the double-quoted expansion occurs within a word, the expansion of the first parameter is joined with the beginning part of the original word, and the expansion of the last parameter is joined with the last part of the original word. When there are no positional parameters, "$#" and $# expand to nothing (i.e., they are removed).
A key difference from my POV is that "$#" preserves the original number
of arguments. It's the only form that does.
For example, if file my_script contains:
#!/bin/bash
main()
{
echo 'MAIN sees ' $# ' args'
}
main $*
main $#
main "$*"
main "$#"
### end ###
and I run it like this:
my_script 'a b c' d e
I will get this output:
MAIN sees 5 args
MAIN sees 5 args
MAIN sees 1 args
MAIN sees 3 args