I need to insert an array of arguments into an eval string executed via bash -c.
In this particular case, it's not possible to pass them separately as proper arguments (it's for an flock invocation which doesn't accept arguments for a -c script).
Smth like this:
flock f.lock -c 'do_stuff '"${ARGS[#]}"'; do_other_stuff'
How do I quote them properly so that they are correctly parsed into a sequence of arguments, even if they contain spaces or Bash special syntax?
Don't! It is going to be error prone and give you pain. Just:
{
flock 9
do_stuff "${ARGS[#]}"
do_other_stuff
} 9>f.lock
Anyway, split the operation into two:
first, safely transfer the environment to the subshell
then execute what you want to execute in a normal way
And it's just:
bash -c "$(declare -p ARGS)"'; do_stuff "${ARGS[#]}"; do_other_stuff'
Use Parameter transformation (new in Bash 4.4) with the Q (quote) operator which is specifically designed for this:
Q The expansion is a string that is the value of parameter quoted
in a format that can be reused as input.
$ ARGS=("foo bar" 'baz\n'\' '$xyzzy')
$ echo 'do_stuff '"${ARGS[*]#Q}"'; do_other_stuff'
do_stuff 'foo bar' 'baz\n'\''' '$xyzzy'; do_other_stuff
Note the use of * instead of the usual # since the code needs to be a single argument. Using # leads to erroneous behavior in some cases:
$ bash -xc "echo ${ARGS[#]#Q}; do_other_stuff"
+ echo 'foo bar'
foo bar
$ bash -xc "echo ${ARGS[*]#Q}; do_other_stuff"
+ echo 'foo bar' 'baz\n'\''' '$xyzzy'
foo bar baz\n' $xyzzy
+ do_other_stuff
You can even use this syntax for $*:
'do_stuff '"${*#Q}"' other args; do other stuff'
Related
I have to write a simple bash script for my programming class. The idea is to use a for loop with $* (names of Files as Command Line Parameters). The task is to reverse and print out the Command Line parameters while still using the for inFile in $*; do loop.
I have no idea how to do this.
#!/bin/bash
for inFile in $*;do
echo $inFile
done
I know this doesn't work it just prints out the command line parameters in order.
The idea to loop over $* to reverse command line arguments is broken,
when any command line argument contains a white space. For example when the command line arguments are foo and "bar baz", the output of the script in the question will be:
foo
bar
baz
When the correct output should be:
foo
bar baz
The exact wording of the task is important.
For example, if the task is to print the arguments in reverse, and it doesn't mention $*, then you can use a counting loop in reverse, and ${!i} to expand to the value of the numbered positional parameters:
# nice clean solution
for ((i = $#; i > 0; i--)); do
echo "${!i}"
done
Another example, if the task insists that you must use $* and accepts that the command line arguments will only have supported characters, then you could collect the parameters into an array, and then print the content of the array in reverse, again with a counting loop:
args=()
# not recommended, unsafe due to shell expansion of $*
for arg in $*; do
args+=("$arg")
done
for ((i = ${#args[#]} - 1; i >= 0; i--)); do
echo "${args[i]}"
done
If you are not allowed to use arrays, then you can prepend values to a string, and then iterate over that string:
# dirtiest solution, with unsafe expansions and unquoted variables, not recommended
args=
for arg in $*; do
args="$arg $args"
done
for arg in $args; do
echo "$arg"
done
This question already has answers here:
Variable containing multiple args with quotes in Bash
(4 answers)
Closed 4 years ago.
My goal is to call a script with an variable as it's arguments. For example, I have the following two files:
print_args.sh:
echo "Length: $#"
for i in "$#"; do echo "$i"; done
caller.sh:
ARGS="foo \"Hello, World\" bar"
./test.sh $ARGS
When I run:
./print_args.sh foo "Hello, World" bar
print_args.sh get's called with 3 arguments:
Length 3
foo
Hello, World
bar
However, when running it via caller.sh instead I get 4 args:
Length: 4
foo
"Hello,
World"
bar
What's going on here? How can I get caller.sh to perform as expected?
Note: I don't have control over ARGS it's passed in as an environment variable.
How can I get test_caller.sh to perform as expected?
Use an array:
#!/bin/bash
args=( foo "Hello, World" bar )
./test.sh "${args[#]}"
If you pass an unquoted variable, then word splitting occurs on its contents before it is passed to ./test.sh. You are essentially calling:
'./test.sh' 'foo' '"Hello,' 'World"' 'bar'
Where the single quotes are used to indicate the start and end of words. The double quotes have no syntactic meaning, they are just characters in a string at this point.
If you can convince the users of your script to change the format of the environment variable, then you could split it based on a different separator:
ARGS="foo:Hello World:bar"
set -f # disable glob expansion
IFS=: args=( $ARGS )
set +f # re-enable it
Now you have an array args which you can use as above.
One (really nasty, dangerous) way to get your string into an array would be to use the notorious eval:
readarray -t args < <(eval printf '%s\\n' $ARGS)
Now you have an array args which you can use as in the previous examples.
But this is a big security problem:
ARGS="foo \"Hello, World\" bar \$(ls -l)"
Replace ls -l with any command that you want to execute...
I've got a program that I want to call by passing parameters from a shell variable. Throughout this question, I am going to assume that it is given by
#!/bin/sh
echo $#
i.e. that it prints out the number of arguments that are passed to it. Let's call it count-args.
I call my program like this:
X="arg1 arg2"
count-args $X
This works quite well. But now one of my arguments has a whitespace in it and I can't find a way to escape it, e.g. the following things do not work:
X="Hello\ World"
X="Hello\\ World"
X="'Hello World'"
In all of the cases, my program count-args prints out 2. I want to find a way so I can pass the string Hello World and that it returns 1 instead. How?
Just for clarification: I do not want to pass all parameters as a single string, e.g.
X="Hello World"
count-args $X
should print out 2. I want a way to pass parameters that contain whitespaces.
Use an array to store multiple, space-containing arguments.
$ args=("first one" "second one")
$ count-args "${args[#]}"
2
This can be solved with xargs. By replacing
count-args $X
with
echo $X | xargs count-args
I can use backslashes to escape whitespaces in $X, e.g.
X="Hello\\ World"
echo $X | xargs count-args
prints out 1 and
X="Hello World"
echo $X | xargs count-args
prints out 2.
count-args "$X"
The quotes ensure in bash, that the whole content of variable X is passed as a single parameter.
Your Counting script:
$ cat ./params.sh
#!/bin/sh
echo $#
For completeness here is what happens with various arguments:
$ ./params.sh
0
$ ./params.sh 1 2
2
$ ./params.sh
0
$ ./params.sh 1
1
$ ./params.sh 1 2
2
$ ./params.sh "1 2"
1
And here is what you get with variables:
$ XYZ="1 2" sh -c './params.sh $XYZ'
2
$ XYZ="1 2" sh -c './params.sh "$XYZ"'
1
Taking this a bit further:
$ cat params-printer.sh
#!/bin/sh
echo "Count: $#"
echo "1 : '$1'"
echo "2 : '$2'"
We get:
$ XYZ="1 2" sh -c './params-printer.sh "$XYZ"'
Count: 1
1 : '1 2'
2 : ''
This looks like what you wanted to do.
Now: If you have a script you cannot control and neither can you control the way the script is invoked. Then there is very little you can do to prevent a variable with spaces turning into multiple arguments.
There are quite a few questions around this on StackOverflow which indicate that you need the ability to control how the command is invoked else there is little you can do.
Passing arguments with spaces between (bash) script
Passing a string with spaces as a function argument in bash
Passing arguments to a command in Bash script with spaces
And wow! this has been asked so many times before:
How to pass argument with spaces to a shell script function
How do you write a function in bash that executes the command that it is given as an argument, where
The given command may be an alias
Arguments must be passed on exactly as given; no evaluating may be done
In other words, how to write an as-transparent-as-possible wrapper function.
The goal of the wrapper function could for example be to set the current directory before and after the given command, and/or set environment variables, or time how long the given command takes,... As a simple example here I take a function that just prints a line and then executes the given command.
A first attempt:
function wrap1 {
echo Starting: "$#"
"$#"
}
You could use it like wrap1 echo hello. But the problem is you cannot do alias myalias echo and then call wrap1 myalias hello: it wouldn't resolve the alias.
Another attempt using eval:
function wrap2 {
echo Starting: "$#"
eval "$#"
}
Now calling an alias works. But the problem is it evaluates the arguments too. For example wrap2 echo "\\a" prints just a instead of \a because the arguments are evaluated twice.
shopt -s expand_aliases doesn't seem to help here either.
Is there a way to both evaluate aliases like wrap2, but still pass on the arguments directly like wrap1?
You (uh, I) can use printf %q to escape the arguments.
At first sight, escaping with printf and then doing eval always gives the same result as passing the arguments directly.
wrap() {
echo Starting: "$#"
eval $(printf "%q " "$#")
}
It seems to be possible with a double eval:
eval "eval x=($(alias y | cut -s -d '=' -f 2))"
# now the array x contains the split expansion of alias y
"${x[#]}" "${other_args[#]}"
So maybe your function could be written as follows:
wrap() {
eval "eval prefix=($(alias $1 | cut -s -d '=' -f 2))"
shift
"${prefix[#]}" "$#"
}
However, eval is evil, and double eval is double evil, and aliases are not expanded in scripts for a reason.
Is there a way to implement/use lambda functions in bash? I'm thinking of something like:
$ someCommand | xargs -L1 (lambda function)
I don't know of a way to do this, however you may be able to accomplish what you're trying to do using:
somecommand | while read -r; do echo "Something with $REPLY"; done
This will also be faster, as you won't be creating a new process for each line of text.
[EDIT 2009-07-09]
I've made two changes:
Incorporated litb's suggestion of using -r to disable backslash processing -- this means that backslashes in the input will be passed through unchanged.
Instead of supplying a variable name (such as X) as a parameter to read, we let read assign to its default variable, REPLY. This has the pleasant side-effect of preserving leading and trailing spaces, which are stripped otherwise (even though internal spaces are preserved).
From my observations, together these changes preserve everything except literal NUL (ASCII 0) characters on each input line.
[EDIT 26/7/2016]
According to commenter Evi1M4chine, setting $IFS to the empty string before running read X (e.g., with the command IFS='' read X) should also preserve spaces at the beginning and end when storing the result into $X, meaning you aren't forced to use $REPLY.
if you want true functions, and not just pipes or while loops (e.g. if you want to pass them around, as if they were data) I’d just not do lambdas, and define dummy functions with a recurring dummy name, to use right away, and throw away afterwards. Like so:
# An example map function, to use in the example below.
map() { local f="$1"; shift; for i in "$#"; do "$f" "$i"; done; }
# Lambda function [λ], passed to the map function.
λ(){ echo "Lambda sees $1"; }; map λ *
Like in proper functional languages, there’s no need to pass parameters, as you can wrap them in a closure:
# Let’s say you have a function with three parameters
# that you want to use as a lambda:
# (As in: Partial function application.)
trio(){ echo "$1 Lambda sees $3 $2"; }
# And there are two values that you want to use to parametrize a
# function that shall be your lambda.
pre="<<<"
post=">>>"
# Then you’d just wrap them in a closure, and be done with it:
λ(){ trio "$pre" "$post" "$#"; }; map λ *
I’d argue that it’s even shorter than all other solutions presented here.
What about this?
somecommand | xargs -d"\n" -I{} echo "the argument is: {}"
(assumes each argument is a line, otherwise change delimiter)
#!/bin/bash
function customFunction() {
eval $1
}
command='echo Hello World; echo Welcome;'
customFunction "$command"
GL
Source
if you want only xargs (due parallel -P N option for example), and only bash as function code, then bash -c can be used as parameter for xargs.
seq 1 10 | tr '\n' '\0' | xargs -0 -n 1 bash -c 'echo any bash code $0'
tr and -0 option are used here to disable any xargs parameters substitutions.
Yes. One can pass around a string variable representing a command call, and then execute the command with eval.
Example:
command='echo howdy'
eval "$command"
The eval trick has been already mentioned but here's my extended example of bash closures:
#!/usr/bin/env bash
set -e
function multiplyBy() {
X="$1"
cat <<-EOF
Y="\$1"
echo "$X * \$Y = \$(( $X * \$Y ))"
EOF
}
function callFunc() {
CODE="$1"
shift
eval "$CODE"
}
MULT_BY_2=`multiplyBy 2`
MULT_BY_4=`multiplyBy 4`
callFunc "$MULT_BY_2" 10
callFunc "$MULT_BY_4" 10
PS I've just came up with this for a completely different purpose and was just searching google to see if sb is using that. I actually needed to evaluate a reusable function in the context (shell) of main script.