combining variables onto one line with added around some parameters [duplicate] - bash

Sorry if the question is very straight forward but am a newbie to shell scripting.
I am trying to write something like this :
for i in {1..20}
do
curl "something $i ........ -d 'something "$i" something' "
done
The problem is that the second $i inside the single quotes part '' is not being replaced. What should be done to get it working ?

As said above, parameters are not expanded inside single quotes, you have to use double quotes. The only point is that since it occurs in a already double-quoted string, you have to escape them with a backslash (\), like this:
$ foo=bar
$ eval "echo \"something \\\"$foo\\\"\""
something "bar"
Note that there are three \ before the innermost ", as this will be expanded twice (once when evaluating the argument of eval and once when evaluating the argument of echo)

This is because variables inside the single quotes '' are not being replaced. If you want variable substitution, you need to get rid of the single quotes.
You coul maybe pre-initialize that variable
foo="something $i bla bla bla"
curl "something ... ${foo}something"

Your quoting is not correct. You don't need double quotes around the second $i because the whole thing is already surrounded in double-quotes.
Change it to the following:
for i in {1..20}
do
curl "something $i ........ -d 'something $i something'"
done

Related

How does Bash deal with brackets inside of double quoting?

The question is basically, how does bash deal with double quotes and brackets?
I couldn't come up with a better example, so apologies about that, but imagine something like this:
"$(echo "${foo}")"
What happens there? Is ${foo} technically out of the quotation scope? Is bash smart enough to also consider that it is inside of the parentheses and work inside-out? Would \"${foo}\" work better?
I've tried Googling this, but no luck. Please note that this is different from just nested quoting questions, as it includes expansion.
Thanks!
Each command substitution establishes a new quoting context, so the correct way to avoid word splitting inside nested command substitutions is to use double quotes.
In this example, white space is preserved:
$ echo "$(echo "$(echo 'foo bar')")" # outer echo sees 'foo bar'
foo bar
However, missing any of the double quotes means that the string is split by the shell:
$ echo $(echo "$(echo 'foo bar')") # outer echo sees 'foo' 'bar'
foo bar
$ echo "$(echo $(echo 'foo bar'))" # middle echo sees 'foo' 'bar'
foo bar
echo outputs each argument, separated by a spaces, which is where the single space between "foo" and "bar" comes from.
The example you gave does save you from word splitting completely:
"$(echo "${foo}")"
while,
"$(echo ${foo})"
would save you from word splitting on the output but not on the variable $foo.
$() opens a subshell and you need to quote your vars in this subshell too. Consider the following example:
"$(echo "$(echo "$foo")")"
None of the pair of quotes above is optional, if you miss any of them, you will be subject to word splitting.
Consider another example:
"$foo var in same shell, $(cmd) - this output is also in the same shell"
In above, $foo and the output of cmd are safe, but if the cmd includes another variable inside $(), then that variable will need to be quoted.
What is foo? Is it a value? try
"$(echo "$\{foo\}")"

Why won't bash escape whitespace or honor quotes in filepaths?

I have the following script saved as 'lspkg' in my path:
#!/bin/bash
args=("$#")
for item in ${args[#]}; do
echo "$item"
done
When I run lspkg /absolute/path/to/file, it works as expected, printing the path. But, the following behaviors are causing me a good deal of trouble:
Escaping the spaces in the path does not really escape the spaces:
$ lspkg /absolute/path/to/file\ with\ spaces
/absolute/path/to/file\
with\
spaces
Putting the path in quotes does not make bash see it as a single string:
$ lspkg "/absolute/path/to/file with spaces"
"/absolute/path/to/file
with
spaces"
Why is this, and how can this problem be solved?
Important quotes are missing from your script, use:
#!/bin/bash
args=("$#")
for item in "${args[#]}"; do
echo "$item"
done
Without quotes in "${args[#]}" shell is expanding and treating it as multiple arguments.

How to have both semicolons and variables in a URL with cURL?

My question is how can I make this bash code run:
#!/bin/sh
actionString = printpage
curl 'www.example.com/index.php?action=$actionString;post=5'
My problem is that if I do not escape the URL with quotation marks, then it will stop processing the URL after the ";", however if I do have it in quotation marks it won't recognize the variable. Is there some trick to getting past this? Thanks for the help.
Use the quotes that don't inhibit parameter substitution.
curl "http://www.example.com/index.php?action=$actionString;post=5"
Variables are NOT evaluated within single quotes:
greeting=hello
echo 'say $greeting; ...' # outputs literally: say $greeting; ...
Variables ARE evaluated within double quotes:
echo "say $greeting" # outputs: say hello; ...
You can quote only the part of parameters that needs quoting, and you can use double quotes for some parts and single quotes for others. All of these are equivalent:
echo say $greeting"; ..."
echo say $greeting'; ...'
echo "say $greeting"'; ...'
echo say $greeting\; ...
So in your case, these would all work:
curl "http://www.example.com/index.php?action=$actionString;post=5"
curl http://www.example.com/index.php?action="$actionString;post=5"
curl http://www.example.com/index.php?action="$actionString;"post=5
and if actionString doesn't have special characters, this would work too:
curl http://www.example.com/index.php?action=$actionString';post=5'

Which form is preferable to use in Bash?

I'm studyng Bash, and I see that the form
C=example
echo "$C"
give the same result of the form
C="example"
echo $C
I'd like to know if is better put the " " in the assignment of the variable or after the $. Or if it is indifferent. or if one is consider "more beautiful" than the other.
If you're certain that a variable's value is a single word (no white space) then it's OK to use $varname or ${varname}. If you can't guarantee this, then you should use "$varname" or "${varname}". Note that bash does word-splitting before interpreting your command, so you may actually get a syntax error if you don't quote the expression, for example
C="white space"
if [ -z $C ]
then
...
fi
will result in syntax error:
-bash: [: white: binary operator expected
while this works fine:
C="white space"
if [ -z "$C" ]
then
...
fi
This is due to the fact after variable expansion in the first, unquoted case bash sees this:
if [ -z white space ]
then
...
fi
and the -z operator expects just one, not two arguments. In the second, quoted case bash sees this:
if [ -z "white space" ]
then
...
fi
i.e. just a single argument as required. Note also that quotes were used in assignment
C="white space"
as it would also produce an error if you wrote
C=white space
since this would mean: execute command space with environment containing an added variable C=white.
So, in general you should quote these expressions to ensure your code is more robust against unforeseen variable values. This is especially true if the variable value comes from input, file etc. It is usually safe to drop the quotes for integer variables or when you just want to display the value of a variable as in echo $C.
It matters when the string contains whitespace characters. Without the quotes, whitespace characters are treated as token delimiters and bash tries to interpret the substituted string as an expression.
Always put quotes to be safe, when you don't intend to evaluate the variable as a part of the expression.
Imagine you change the input from "example" to "two words", then you could encounter strange behaviour or even syntax errors when executing the script, in case you have overlooked the above.
In other words,
C="abc def"
# the echo command receives one argument: "abc def"
echo "$C"
# echo receives two arguments: "abc" and "def"
echo $C
# bash tries to execute the program "abc" with a first argument "def"
$C
# bash tries to execute the program "abc def"
"$C"
A good documentation about quotes and word-spliting :
"USE MORE QUOTES!" They are vital. Also, learn the difference between ' and " and `. See http://mywiki.wooledge.org/Quotes and http://wiki.bash-hackers.org/syntax/words
greybot sample from IRC freenode #bash is talking to the world =)
If it's a one-word constant, it's irrelevant.
However, you should read about the two kinds of quoting. Try this article and this documentation. There is also a SO question.
Try with a real example with whitespace. For the string example you do not need any quoting at all. So create a file called This is an example.txt and then retry. Substitute echo with ls...

How to keep quotes in Bash arguments? [duplicate]

This question already has answers here:
How can I preserve quotes in printing a bash script's arguments
(7 answers)
Closed 3 years ago.
I have a Bash script where I want to keep quotes in the arguments passed.
Example:
./test.sh this is "some test"
then I want to use those arguments, and re-use them, including quotes and quotes around the whole argument list.
I tried using \"$#\", but that removes the quotes inside the list.
How do I accomplish this?
using "$#" will substitute the arguments as a list, without re-splitting them on whitespace (they were split once when the shell script was invoked), which is generally exactly what you want if you just want to re-pass the arguments to another program.
Note that this is a special form and is only recognized as such if it appears exactly this way. If you add anything else in the quotes the result will get combined into a single argument.
What are you trying to do and in what way is it not working?
There are two safe ways to do this:
1. Shell parameter expansion: ${variable#Q}:
When expanding a variable via ${variable#Q}:
The expansion is a string that is the value of parameter quoted in a format that can be reused as input.
Example:
$ expand-q() { for i; do echo ${i#Q}; done; } # Same as for `i in "$#"`...
$ expand-q word "two words" 'new
> line' "single'quote" 'double"quote'
word
'two words'
$'new\nline'
'single'\''quote'
'double"quote'
2. printf %q "$quote-me"
printf supports quoting internally. The manual's entry for printf says:
%q Causes printf to output the corresponding argument in a format that can be reused as shell input.
Example:
$ cat test.sh
#!/bin/bash
printf "%q\n" "$#"
$
$ ./test.sh this is "some test" 'new
>line' "single'quote" 'double"quote'
this
is
some\ test
$'new\nline'
single\'quote
double\"quote
$
Note the 2nd way is a bit cleaner if displaying the quoted text to a human.
Related: For bash, POSIX sh and zsh: Quote string with single quotes rather than backslashes
Yuku's answer only works if you're the only user of your script, while Dennis Williamson's is great if you're mainly interested in printing the strings, and expect them to have no quotes-in-quotes.
Here's a version that can be used if you want to pass all arguments as one big quoted-string argument to the -c parameter of bash or su:
#!/bin/bash
C=''
for i in "$#"; do
i="${i//\\/\\\\}"
C="$C \"${i//\"/\\\"}\""
done
bash -c "$C"
So, all the arguments get a quote around them (harmless if it wasn't there before, for this purpose), but we also escape any escapes and then escape any quotes that were already in an argument (the syntax ${var//from/to} does global substring substitution).
You could of course only quote stuff which already had whitespace in it, but it won't matter here. One utility of a script like this is to be able to have a certain predefined set of environment variables (or, with su, to run stuff as a certain user, without that mess of double-quoting everything).
Update: I recently had reason to do this in a POSIX way with minimal forking, which lead to this script (the last printf there outputs the command line used to invoke the script, which you should be able to copy-paste in order to invoke it with equivalent arguments):
#!/bin/sh
C=''
for i in "$#"; do
case "$i" in
*\'*)
i=`printf "%s" "$i" | sed "s/'/'\"'\"'/g"`
;;
*) : ;;
esac
C="$C '$i'"
done
printf "$0%s\n" "$C"
I switched to '' since shells also interpret things like $ and !! in ""-quotes.
If it's safe to make the assumption that an argument that contains white space must have been (and should be) quoted, then you can add them like this:
#!/bin/bash
whitespace="[[:space:]]"
for i in "$#"
do
if [[ $i =~ $whitespace ]]
then
i=\"$i\"
fi
echo "$i"
done
Here is a sample run:
$ ./argtest abc def "ghi jkl" $'mno\tpqr' $'stu\nvwx'
abc
def
"ghi jkl"
"mno pqr"
"stu
vwx"
You can also insert literal tabs and newlines using Ctrl-V Tab and Ctrl-V Ctrl-J within double or single quotes instead of using escapes within $'...'.
A note on inserting characters in Bash: If you're using Vi key bindings (set -o vi) in Bash (Emacs is the default - set -o emacs), you'll need to be in insert mode in order to insert characters. In Emacs mode, you're always in insert mode.
I needed this for forwarding all arguments to another interpreter.
What ended up right for me is:
bash -c "$(printf ' %q' "$#")"
Example (when named as forward.sh):
$ ./forward.sh echo "3 4"
3 4
$ ./forward.sh bash -c "bash -c 'echo 3'"
3
(Of course the actual script I use is more complex, involving in my case nohup and redirections etc., but this is the key part.)
Like Tom Hale said, one way to do this is with printf using %q to quote-escape.
For example:
send_all_args.sh
#!/bin/bash
if [ "$#" -lt 1 ]; then
quoted_args=""
else
quoted_args="$(printf " %q" "${#}")"
fi
bash -c "$( dirname "${BASH_SOURCE[0]}" )/receiver.sh${quoted_args}"
send_fewer_args.sh
#!/bin/bash
if [ "$#" -lt 2 ]; then
quoted_last_args=""
else
quoted_last_args="$(printf " %q" "${#:2}")"
fi
bash -c "$( dirname "${BASH_SOURCE[0]}" )/receiver.sh${quoted_last_args}"
receiver.sh
#!/bin/bash
for arg in "$#"; do
echo "$arg"
done
Example usage:
$ ./send_all_args.sh
$ ./send_all_args.sh a b
a
b
$ ./send_all_args.sh "a' b" 'c "e '
a' b
c "e
$ ./send_fewer_args.sh
$ ./send_fewer_args.sh a
$ ./send_fewer_args.sh a b
b
$ ./send_fewer_args.sh "a' b" 'c "e '
c "e
$ ./send_fewer_args.sh "a' b" 'c "e ' 'f " g'
c "e
f " g
Just use:
"${#}"
For example:
# cat t2.sh
for I in "${#}"
do
echo "Param: $I"
done
# cat t1.sh
./t2.sh "${#}"
# ./t1.sh "This is a test" "This is another line" a b "and also c"
Param: This is a test
Param: This is another line
Param: a
Param: b
Param: and also c
Changed unhammer's example to use array.
printargs() { printf "'%s' " "$#"; echo; }; # http://superuser.com/a/361133/126847
C=()
for i in "$#"; do
C+=("$i") # Need quotes here to append as a single array element.
done
printargs "${C[#]}" # Pass array to a program as a list of arguments.
My problem was similar and I used mixed ideas posted here.
We have a server with a PHP script that sends e-mails. And then we have a second server that connects to the 1st server via SSH and executes it.
The script name is the same on both servers and both are actually executed via a bash script.
On server 1 (local) bash script we have just:
/usr/bin/php /usr/local/myscript/myscript.php "$#"
This resides on /usr/local/bin/myscript and is called by the remote server. It works fine even for arguments with spaces.
But then at the remote server we can't use the same logic because the 1st server will not receive the quotes from "$#". I used the ideas from JohnMudd and Dennis Williamson to recreate the options and parameters array with the quotations. I like the idea of adding escaped quotations only when the item has spaces in it.
So the remote script runs with:
CSMOPTS=()
whitespace="[[:space:]]"
for i in "$#"
do
if [[ $i =~ $whitespace ]]
then
CSMOPTS+=(\"$i\")
else
CSMOPTS+=($i)
fi
done
/usr/bin/ssh "$USER#$SERVER" "/usr/local/bin/myscript ${CSMOPTS[#]}"
Note that I use "${CSMOPTS[#]}" to pass the options array to the remote server.
Thanks for eveyone that posted here! It really helped me! :)
Quotes are interpreted by bash and are not stored in command line arguments or variable values.
If you want to use quoted arguments, you have to quote them each time you use them:
val="$3"
echo "Hello World" > "$val"
As Gary S. Weaver shown in his source code tips, the trick is to call bash with parameter '-c' and then quote the next.
e.g.
bash -c "<your program> <parameters>"
or
docker exec -it <my docker> bash -c "$SCRIPT $quoted_args"
If you need to pass all arguments to bash from another programming language (for example, if you'd want to execute bash -c or emit_bash_code | bash), use this:
escape all single quote characters you have with '\''.
then, surround the result with singular quotes
The argument of abc'def will thus be converted to 'abc'\''def'. The characters '\'' are interpreted as following: the already existing quoting is terminated with the first first quote, then the escaped singular single quote \' comes, then the new quoting starts.
Yes, seems that it is not possible to ever preserve the quotes, but for the issue I was dealing with it wasn't necessary.
I have a bash function that will search down folder recursively and grep for a string, the problem is passing a string that has spaces, such as "find this string". Passing this to the bash script will then take the base argument $n and pass it to grep, this has grep believing these are different arguments. The way I solved this by using the fact that when you quote bash to call the function it groups the items in the quotes into a single argument. I just needed to decorate that argument with quotes and pass it to the grep command.
If you know what argument you are receiving in bash that needs quotes for its next step you can just decorate with with quotes.
Just use single quotes around the string with the double quotes:
./test.sh this is '"some test"'
So the double quotes of inside the single quotes were also interpreted as string.
But I would recommend to put the whole string between single quotes:
./test.sh 'this is "some test" '
In order to understand what the shell is doing or rather interpreting arguments in scripts, you can write a little script like this:
#!/bin/bash
echo $#
echo "$#"
Then you'll see and test, what's going on when calling a script with different strings

Resources