I'm trying to write a script notify-finish that can be prepended to any command. When done, it will run the command given by the arguments following, then email the user when the command is complete. Here's what I have:
PROG=$1
# Run command given by arguments
$#
ECODE=$?
echo -e "Subject: `hostname`: $PROG finished\r\nTo: <$USER>\r\n\r\nExited with $ECODE\r\n" | sendmail $USER
This works most of the time, but when arguments contain spaces, the quoting is stripped off.
Working example:
notify-finished rsync -avz source/ user#remote:dest/
Failing example:
notify-finished rsync -avz -e 'ssh -c blowfish' source/ user#remote:dest/
In the second case, $# is expanded out to rsync -avz -e ssh -c blowfish source user#remote:dest/, missing the single quotes. It does not work with double-quotes either, nor with $*.
After reading other posts I tried putting the command in an array, but I get the exact same issue:
CMD=(notify-finished rsync -avz -e 'ssh -c blowfish' source/ user#remote:dest/)
${CMD[#]}
How do I make this work for all arguments?
Use "$#" with quotes:
prog="$1"
"$#"
ecode="$?"
echo "$prog exited with $ecode"
This will pass each argument exactly as it was received. If you don't include the quotes, each element will be split according to $IFS:
"$#" is like "$1" "$2" "$3" ..., passing each element as a separate argument.
"$*" is like "$1 $2 $3 ...", passing all elements concatenated as a single argument
$* and $# is like $1 $2 $3 ..., breaking up each element on whitespace, expanding all globs, and passing each resulting word as a separate element ($IFS).
The same is true for arrays, such as "${array[#]}" and "${array[*]}"
Put double-quotes around your variable substitutions to keep them from being parsed (note that this applies to all variables: $#, $1, and $PROG). Also: don't put a $ before a variable name when assigning to it; use # for comments; and, on the last line, the single-quotes will prevent variables from being substituted at all.
PROG="$1"
shift
# Run program below
"$PROG" "$#"
ECODE=$? # note: this will always be a number, so it doesn't have to be protected with double-quotes
echo -e "Subject: $(hostname): $PROG finished\r\nTo: <$USER>\r\n\r\nExited with $ECODE\r\n' | sendmail "$USER"
Related
I want to run this cmd line script
$ script.sh lib/* ../test_git_thing
I want it to process all the files in the /lib folder.
FILES=$1
for f in $FILES
do
echo "Processing $f file..."
done
Currently it only prints the first file. If I use $#, it gives me all the files, but also the last param which I don't want. Any thoughts?
The argument list is being expanded at the command line when you invoke "script.sh lib/*" your script is being called with all the files in lib/ as args. Since you only reference $1 in your script, it's only printing the first file. You need to escape the wildcard on the command line so it's passed to your script to perform the globbing.
As correctly noted, lib/* on the command line is being expanded into all files in lib. To prevent expansion, you have 2 options. (1) quote your input:
$ script.sh 'lib/*' ../test_git_thing
Or (2), turn file globbing off. However, the option set -f will disable pathname expansion within the shell, but it will disable all pathname expansion (setting it within the script doesn't help as expansion is done by the shell before passing arguments to your script). In your case, it is probably better to quote the input or pass the first arguments as a directory name, and add the expansion in the script:
DIR=$1
for f in "$DIR"/*
In bash and ksh you can iterate through all arguments except the last like this:
for f in "${#:1:$#-1}"; do
echo "$f"
done
In zsh, you can do something similar:
for f in $#[1,${#}-1]; do
echo "$f"
done
$# is the number of arguments and ${#:start:length} is substring/subsequence notation in bash and ksh, while $#[start,end] is subsequence in zsh. In all cases, the subscript expressions are evaluated as arithmetic expressions, which is why $#-1 works. (In zsh, you need ${#}-1 because $#- is interpreted as "the length of $-".)
In all three shells, you can use the ${x:start:length} syntax with a scalar variable, to extract a substring; in bash and ksh, you can use ${a[#]:start:length} with an array to extract a subsequence of values.
This answers the question as given, without using non-POSIX features, and without workarounds such as disabling globbing.
You can find the last argument using a loop, and then exclude that when processing the list of files. In this example, $d is the directory name, while $f has the same meaning as in the original answer:
#!/bin/sh
if [ $# != 0 ]
then
for d in "$#"; do :; done
if [ -d "$d" ]
then
for f in "$#"
do
if [ "x$f" != "x$d" ]
then
echo "Processing $f file..."
fi
done
fi
fi
Additionally, it would be a good idea to also test if "$f" is a file, since it is common for shells to pass the wildcard character through the argument list if no match is found.
How to escape quotes and pipe?
#!/bin/bash
set -x
MYCMD="VBoxManage showvminfo --machinereadable $1 \| grep \'VMState=\"poweroff\"\'"
echo "`$MYCMD`"
Executed command :
++ VBoxManage showvminfo --machinereadable d667 '|' grep '\'\''VMState="poweroff"\'\'''
And finally getting this error:
Syntax error: Invalid parameter '|'
You don't; you would need to use eval to embed an arbitrary pipeline in a regular string parameter.
MYCMD="VBoxManage showvminfo --machinereadable \"$1\" | grep 'VMState=\"poweroff\"'"
eval "$MYCMD"
However, this is not recommended unless you are certain that the value of $1 will not cause problems. (If you need an explanation of what those risks might be, then you should not be using eval.)
Instead, define a shell function:
mycmd () {
VBoxManage showvminfo --machinereadable "$1" | grep 'VMState="poweroff"'
}
mycmd "$1"
One really simple way to do it is by using an array or positional parameters.
Array-based solution :
# Build command
declare -a CMD_AND_ARGS=(command and args with normal quoting)
# Append arguments
CMD_AND_ARGS+=(more arguments quoted the normal way)
# Execute command
"${CMD_AND_ARGS[#]}"
Positional parameter-based solution:
# Create command
set -- command and args with normal quoting
# Append arguments
set -- "$#" more arguments quoted the normal way
# Execute command
"$#"
The nice thing about both solutions is you do not need to put quotes inside quotes, because expanding positional parameters or an array surrounded by double quotes does not cause word-splitting and expansion to be performed again.
Examples:
declare -a CMD=()
CMD=(ls "/dir/with spaces/in its name")
"$CMD"
set -- ls "/dir/with spaces/in its name"
"$#"
Note that in both cases, you get to build your command incrementally, for instance having conditional expressions (e.g. if/case) choosing to add different arguments depending on the flow of your script.
If you want to pipe a command to another, you will have to build each command separately (e.g. two arrays), as the | symbol cannot be used inside an array declaration unquoted, and once quoted will be treated as a string argument and will not cause piping to occur.
I have the following script (example):
#!/bin/bash
while getopts a: opt; do
case "$opt" in
a) val="$OPTARG";;
?) echo "use the flag \"-a\""
exit 2;;
esac
done
echo "a specified with: ${val}"
When I now call this script with test.sh -a "here is a string" the output is: a specified with: here but not as I would like to have a specified with: here is a string.
I know that I can call the script with test.sh -a here\ is\ a\ string or test.sh -a "here\ is\ a\ string" and it will work. But in my case I can not manipulate the string I want to pass.
So how can I change my getopts function to make it work?
I also tried getopt, but I worked even more wors:
commandsShort="a:"
commandsLong="aval:"
TEMP=`getopt \
-o $commandsShort \
-l $commandsLong \
-q \
-n "$0" -- "$#"`
What am I doing wrong?
This got solved in comments on your question. :-)
You're calling the script with:
eval "test.sh $#"
The effect of this "eval" line, if "here is a string" is your option, is to create the command line that is in the quotes:
test.sh here is a string
and evaluate it.
Per the additional comments, if you can avoid eval, you should.
That said, if you need it, you could always quote the string within the eval:
eval "test.sh \"$#\""
Or if you don't like escaping quotes, use singles, since your $# will be expanded due to the outer quotes being double:
eval "test.sh '$#'"
And finally, as you mentioned in comments, just running directly may be the best option:
test.sh "$#"
Note that if your $# includes the -a option, you may have a new problem. Consider the command line:
test.sh "-a here is a string"
In this case, your entire string, starting with -a, is found in $1, and you will have no options for getopts and no OPTARG.
I would like to run recursively myscript.sh, to execute all files in the directory:
It has been discussed here that I could do like this:
#!/bin/bash
for file in * ; do
echo $file
done
But I would like myscript.sh to execute with this syntax, so that I could select only certain filetypes to be executed:
./myscript.sh *.dat
Thus I modify the script above:
#!/bin/bash
for file in $1 ; do
echo $file
done
In which when executing, it only executes first occurrence, not all files with *.dat extensions.
What is wrong here?
The wildcard *.dat is expanded by the shell before your script ever sees it. So the filenames show up in your script as $1, $2, $3, etc.
You can work with them all at once by using the special $# variable:
for file in "$#"; do
echo $file
done
Note that the double quotes around "$#" is special. From man bash:
# Expands to the positional parameters, starting from one. When the expansion
occurs within double quotes, each parameter expands to a separate word. That
is, "$#" is equivalent to "$1" "$2" ... If the double-quoted expansion occurs
within a word, the expansion of the first parameter is joined with the begin-
ning part of the original word, and the expansion of the last parameter is
joined with the last part of the original word. When there are no positional
parameters, "$#" and $# expand to nothing (i.e., they are removed).
You could do the same thing with just: ls *.dat | xargs echo
You need to get the actual contents of *, like so:
for file in $* ; do
echo $file
done
This question already has answers here:
How can I preserve quotes in printing a bash script's arguments
(7 answers)
Closed 3 years ago.
I have a Bash script where I want to keep quotes in the arguments passed.
Example:
./test.sh this is "some test"
then I want to use those arguments, and re-use them, including quotes and quotes around the whole argument list.
I tried using \"$#\", but that removes the quotes inside the list.
How do I accomplish this?
using "$#" will substitute the arguments as a list, without re-splitting them on whitespace (they were split once when the shell script was invoked), which is generally exactly what you want if you just want to re-pass the arguments to another program.
Note that this is a special form and is only recognized as such if it appears exactly this way. If you add anything else in the quotes the result will get combined into a single argument.
What are you trying to do and in what way is it not working?
There are two safe ways to do this:
1. Shell parameter expansion: ${variable#Q}:
When expanding a variable via ${variable#Q}:
The expansion is a string that is the value of parameter quoted in a format that can be reused as input.
Example:
$ expand-q() { for i; do echo ${i#Q}; done; } # Same as for `i in "$#"`...
$ expand-q word "two words" 'new
> line' "single'quote" 'double"quote'
word
'two words'
$'new\nline'
'single'\''quote'
'double"quote'
2. printf %q "$quote-me"
printf supports quoting internally. The manual's entry for printf says:
%q Causes printf to output the corresponding argument in a format that can be reused as shell input.
Example:
$ cat test.sh
#!/bin/bash
printf "%q\n" "$#"
$
$ ./test.sh this is "some test" 'new
>line' "single'quote" 'double"quote'
this
is
some\ test
$'new\nline'
single\'quote
double\"quote
$
Note the 2nd way is a bit cleaner if displaying the quoted text to a human.
Related: For bash, POSIX sh and zsh: Quote string with single quotes rather than backslashes
Yuku's answer only works if you're the only user of your script, while Dennis Williamson's is great if you're mainly interested in printing the strings, and expect them to have no quotes-in-quotes.
Here's a version that can be used if you want to pass all arguments as one big quoted-string argument to the -c parameter of bash or su:
#!/bin/bash
C=''
for i in "$#"; do
i="${i//\\/\\\\}"
C="$C \"${i//\"/\\\"}\""
done
bash -c "$C"
So, all the arguments get a quote around them (harmless if it wasn't there before, for this purpose), but we also escape any escapes and then escape any quotes that were already in an argument (the syntax ${var//from/to} does global substring substitution).
You could of course only quote stuff which already had whitespace in it, but it won't matter here. One utility of a script like this is to be able to have a certain predefined set of environment variables (or, with su, to run stuff as a certain user, without that mess of double-quoting everything).
Update: I recently had reason to do this in a POSIX way with minimal forking, which lead to this script (the last printf there outputs the command line used to invoke the script, which you should be able to copy-paste in order to invoke it with equivalent arguments):
#!/bin/sh
C=''
for i in "$#"; do
case "$i" in
*\'*)
i=`printf "%s" "$i" | sed "s/'/'\"'\"'/g"`
;;
*) : ;;
esac
C="$C '$i'"
done
printf "$0%s\n" "$C"
I switched to '' since shells also interpret things like $ and !! in ""-quotes.
If it's safe to make the assumption that an argument that contains white space must have been (and should be) quoted, then you can add them like this:
#!/bin/bash
whitespace="[[:space:]]"
for i in "$#"
do
if [[ $i =~ $whitespace ]]
then
i=\"$i\"
fi
echo "$i"
done
Here is a sample run:
$ ./argtest abc def "ghi jkl" $'mno\tpqr' $'stu\nvwx'
abc
def
"ghi jkl"
"mno pqr"
"stu
vwx"
You can also insert literal tabs and newlines using Ctrl-V Tab and Ctrl-V Ctrl-J within double or single quotes instead of using escapes within $'...'.
A note on inserting characters in Bash: If you're using Vi key bindings (set -o vi) in Bash (Emacs is the default - set -o emacs), you'll need to be in insert mode in order to insert characters. In Emacs mode, you're always in insert mode.
I needed this for forwarding all arguments to another interpreter.
What ended up right for me is:
bash -c "$(printf ' %q' "$#")"
Example (when named as forward.sh):
$ ./forward.sh echo "3 4"
3 4
$ ./forward.sh bash -c "bash -c 'echo 3'"
3
(Of course the actual script I use is more complex, involving in my case nohup and redirections etc., but this is the key part.)
Like Tom Hale said, one way to do this is with printf using %q to quote-escape.
For example:
send_all_args.sh
#!/bin/bash
if [ "$#" -lt 1 ]; then
quoted_args=""
else
quoted_args="$(printf " %q" "${#}")"
fi
bash -c "$( dirname "${BASH_SOURCE[0]}" )/receiver.sh${quoted_args}"
send_fewer_args.sh
#!/bin/bash
if [ "$#" -lt 2 ]; then
quoted_last_args=""
else
quoted_last_args="$(printf " %q" "${#:2}")"
fi
bash -c "$( dirname "${BASH_SOURCE[0]}" )/receiver.sh${quoted_last_args}"
receiver.sh
#!/bin/bash
for arg in "$#"; do
echo "$arg"
done
Example usage:
$ ./send_all_args.sh
$ ./send_all_args.sh a b
a
b
$ ./send_all_args.sh "a' b" 'c "e '
a' b
c "e
$ ./send_fewer_args.sh
$ ./send_fewer_args.sh a
$ ./send_fewer_args.sh a b
b
$ ./send_fewer_args.sh "a' b" 'c "e '
c "e
$ ./send_fewer_args.sh "a' b" 'c "e ' 'f " g'
c "e
f " g
Just use:
"${#}"
For example:
# cat t2.sh
for I in "${#}"
do
echo "Param: $I"
done
# cat t1.sh
./t2.sh "${#}"
# ./t1.sh "This is a test" "This is another line" a b "and also c"
Param: This is a test
Param: This is another line
Param: a
Param: b
Param: and also c
Changed unhammer's example to use array.
printargs() { printf "'%s' " "$#"; echo; }; # http://superuser.com/a/361133/126847
C=()
for i in "$#"; do
C+=("$i") # Need quotes here to append as a single array element.
done
printargs "${C[#]}" # Pass array to a program as a list of arguments.
My problem was similar and I used mixed ideas posted here.
We have a server with a PHP script that sends e-mails. And then we have a second server that connects to the 1st server via SSH and executes it.
The script name is the same on both servers and both are actually executed via a bash script.
On server 1 (local) bash script we have just:
/usr/bin/php /usr/local/myscript/myscript.php "$#"
This resides on /usr/local/bin/myscript and is called by the remote server. It works fine even for arguments with spaces.
But then at the remote server we can't use the same logic because the 1st server will not receive the quotes from "$#". I used the ideas from JohnMudd and Dennis Williamson to recreate the options and parameters array with the quotations. I like the idea of adding escaped quotations only when the item has spaces in it.
So the remote script runs with:
CSMOPTS=()
whitespace="[[:space:]]"
for i in "$#"
do
if [[ $i =~ $whitespace ]]
then
CSMOPTS+=(\"$i\")
else
CSMOPTS+=($i)
fi
done
/usr/bin/ssh "$USER#$SERVER" "/usr/local/bin/myscript ${CSMOPTS[#]}"
Note that I use "${CSMOPTS[#]}" to pass the options array to the remote server.
Thanks for eveyone that posted here! It really helped me! :)
Quotes are interpreted by bash and are not stored in command line arguments or variable values.
If you want to use quoted arguments, you have to quote them each time you use them:
val="$3"
echo "Hello World" > "$val"
As Gary S. Weaver shown in his source code tips, the trick is to call bash with parameter '-c' and then quote the next.
e.g.
bash -c "<your program> <parameters>"
or
docker exec -it <my docker> bash -c "$SCRIPT $quoted_args"
If you need to pass all arguments to bash from another programming language (for example, if you'd want to execute bash -c or emit_bash_code | bash), use this:
escape all single quote characters you have with '\''.
then, surround the result with singular quotes
The argument of abc'def will thus be converted to 'abc'\''def'. The characters '\'' are interpreted as following: the already existing quoting is terminated with the first first quote, then the escaped singular single quote \' comes, then the new quoting starts.
Yes, seems that it is not possible to ever preserve the quotes, but for the issue I was dealing with it wasn't necessary.
I have a bash function that will search down folder recursively and grep for a string, the problem is passing a string that has spaces, such as "find this string". Passing this to the bash script will then take the base argument $n and pass it to grep, this has grep believing these are different arguments. The way I solved this by using the fact that when you quote bash to call the function it groups the items in the quotes into a single argument. I just needed to decorate that argument with quotes and pass it to the grep command.
If you know what argument you are receiving in bash that needs quotes for its next step you can just decorate with with quotes.
Just use single quotes around the string with the double quotes:
./test.sh this is '"some test"'
So the double quotes of inside the single quotes were also interpreted as string.
But I would recommend to put the whole string between single quotes:
./test.sh 'this is "some test" '
In order to understand what the shell is doing or rather interpreting arguments in scripts, you can write a little script like this:
#!/bin/bash
echo $#
echo "$#"
Then you'll see and test, what's going on when calling a script with different strings