I need some help with the following simple bash script, where the variable i does not seem to get substituted when running curl (causing an error).
(This is just a simple abstraction of the actual script)
for i in {1..3}
do
HTML=$(curl -s 'http://example.com/index.php?id=$i')
done;
Variables are not substituted within single quotes. You have to use double quotes in this case:
for i in {1..3}; do
HTML=$( curl -s "http://example.com/index.php?id=$i" )
done
From http://tldp.org/LDP/abs/html/varsubn.html
Enclosing a referenced value in double quotes (" ... ") does not
interfere with variable substitution. This is called partial quoting,
sometimes referred to as "weak quoting." Using single quotes (' ... ')
causes the variable name to be used literally, and no substitution
will take place. This is full quoting, sometimes referred to as
'strong quoting.'
A
Related
When I write export X="test" in a file test.sh and do $(cat test.sh) in shell quote stay, i.e. echo $X gives "test" whereas export X="test" directly in shell makes quotes disappear, i.e. echo $X gives test why ?
I seem to be executing the same code and it has been messing with my paths :)
The difference between the two is quote removal. From the man page,
Quote Removal
After the preceding expansions, all unquoted occurrences of the characters \, ', and " that did not result from
one of the above expansions are removed.
In $(cat test.sh), the eventual command contains unquoted " characters that did result from a command substitution. The command substitution produced export X="test", which splits into the command word export and its literal argument X="test".
In export X="test", the command is again export with an argument X="test", but those quotes are unquoted and were not produced by any expansions, so they are removed. The result is the command export receiving X=test (not X="test") as its argument.
Each export command then splits its argument on the =, seeing X as the variable to define. One sees a right-hand side of test to use as a value, the other sees "test" to use as the value, resulting in the different assignments that you observed.
Given a file test.sh that contains the phrase export X="test" the command cat test.sh outputs the string with escaped double-quotes export X=\"test\" which evaluates when printed in the bash console as export X="test".
Which is why I was under the impression that I executed the same code, however in the first case I was affecting \"test\" to X while in the second I was affecting test to it.
The solution that I first used and then saw elsewhere is to pipe the output of cat to sed 's/"//g', note the single quotes ' instead of double quotes ".
Thanks to Hendrik Prinsloo for the reference above.
What do those two assignations (i and C omitting the first one to void) do? Is it some kind of regex for the variable? I tried with bash, but so far there were no changes in the output of my strings after instantiating them with "${i//\\/\\\\}" or "\"${i//\"/\\\"}\""
C=''
for i in "$#"; do
i="${i//\\/\\\\}"
C="$C \"${i//\"/\\\"}\""
done
${i//\\/\\\\} is a slightly complicated-looking parameter expansion:
It expands the variable $i in the following way:
${i//find/replace} means replace all instances of "find" with "replace". In this case, the thing to find is \, which itself needs escaping with another \.
The replacement is two \, which each need escaping.
For example:
$ i='a\b\c'
$ echo "${i//\\/\\\\}"
a\\b\\c
The next line performs another parameter expansion:
find " (which needs to be escaped, since it is inside a double-quoted string)
replace with \" (both the double quote and the backslash need to be escaped).
It looks like the intention of the loop is to build a string C, attempting to safely quote/escape the arguments passed to the script. This type of approach is generally error-prone, and it would probably be better to work with the input array directly. For example, the arguments passed to the script can be safely passed to another command like:
cmd "$#" # does "the right thing" (quotes each argument correctly)
if you really need to escape the backslashes, you can do that too:
cmd "${#//\\/\\\\}" # replaces all \ with \\ in each argument
It's bash parameter expansions
it replace all backslashes by double backslashes :"${i//\\/\\\\}
it replace all \" by \\" : ${i//\"/\\\"}
Check http://wiki.bash-hackers.org/syntax/pe
VALUE1=123
VALUE2=456
S_ID=123456789
G_ID=34634
/opt/customtools/custom_cmd -a -g $G_ID -m custom -c '=value9 $S_ID $VALUE1 $VALUE2'
Never worked with bash scripting before and I'm looking through some code now and I've spotted this, and for some reason I can't find anywhere on the internet that describes what the end of this line does (the '=value9...part). Concatenate? Add? Please advise.
Content within single quotes is literal -- it's not modified by the shell in any way before invoking the program that's being run.
Thus, =value9 $S_ID $VALUE1 $VALUE2 is passed to the command being run exactly as-is (without $VALUE1 being replaced with 123 and without $VALUE2 being replaced with 456); its meaning, then, depends on how that command interprets it.
' ' - supress all expansions
" " - supress all expansion, but parameter expansion, command substitution, and arithmetic expansion. Basically $ ` \ are expanded in double quotes and also ! If history is enabled. # * has also special meaning when in double quotes (check parameter expansion for more details).
Sample bash script
QRY="select * from mysql"
CMD="mysql -e \"$QRY\""
`$CMD`
I get errors because the * is getting evaluated as a glob (enumerating) files in my CWD.
I Have seen other posts that talk about quoting the "$CMD" reference for purposes of echo output, but in this case
"$CMD"
complains the whole literal string as a command.
If I
echo "$CMD"
And then copy/paste it to the command line, things seems to work.
You can just use:
qry='select * from db'
mysql -e "$qry"
This will not subject to * expansion by shell.
If you want to store mysql command line also then use BASH arrays:
cmd=(mysql -e "$qry")
"${cmd[#]}"
Note: anubhava's answer has the right solution.
This answer provides background information.
As for why your approach didn't work:
"$CMD" doesn't work, because bash sees the entire value as a single token that it interprets as a command name, which obviously fails.
`$CMD`
i.e., enclosing $CMD in backticks, is pointless in this case (and will have unintended side effects if the command produces stdout output[1]); using just:
$CMD
yields the same - broken - result (only more efficiently - by enclosing in backticks, you needlessly create a subshell; use backticks - or, better, $(...) only when embedding one command in another - see command substitution).
$CMD doesn't work,
because unquoted use of * subjects it to pathname expansion (globbing) - among other shell expansions.
\-escaping glob chars. in the string causes the \ to be preserved when the string is executed.
While it may seem that you've enclosed the * in double quotes by placing it (indirectly) between escaped double quotes (\"$QRY\") inside a double-quoted string, the shell does not see what's between these escaped double quotes as a single, double-quoted string.
Instead, these double quotes become literal parts of the tokens they abut, and the shell still performs word splitting (parsing into separate arguments by whitespace) on the string, and expansions such as globbing on the resulting tokens.
If we assume for a moment that globbing is turned off (via set -f), here is the breakdown of the arguments passed to mysql when the shell evaluates (unquoted) $CMD:
-e # $1 - all remaining arguments are the unintentionally split SQL command.
"select # $2 - note that " has become a literal part of the argument
* # $3
from # $4
mysql" # $5 - note that " has become a literal part of the argument
The only way to get your solution to work with the existing, single string variable is to use eval as follows:
eval "$CMD"
That way, the embedded escaped double-quoted string is properly parsed as a single, double-quoted string (to which no globbing is applied), which (after quote removal) is passed as a single argument to mysql.
However, eval is generally to be avoided due to its security implications (if you don't (fully) control the string's content, arbitrary commands could be executed).
Again, refer to anubhava's answer for the proper solution.
[1] A note re using `$CMD` as a command by itself:
It causes bash to execute stdout output from $CMD as another command, which is rarely the intent, and will typically result in a broken command or, worse, a command with unintended effects.
Try running `echo ha` (with the backticks - same as: $(echo ha)); you'll get -bash: ha: command not found, because bash tries to execute the command's output - ha - as a command, which fails.
If I have a variable containing an unescaped dollar sign, is there any way I can echo the entire contents of the variable?
For example something calls a script:
./script.sh "test1$test2"
and then if I want to use the parameter it gets "truncated" like so:
echo ${1}
test1
Of course single-quoting the varaible name doesn't help. I can't figure out how to quote it so that I can at least escape the dollar sign myself once the script recieves the parameter.
The problem is that script receives "test1" in the first place and it cannot possibly know that there was a reference to an empty (undeclared) variable. You have to escape the $ before passing it to the script, like this:
./script.sh "test1\$test2"
Or use single quotes ' like this:
./script.sh 'test1$test2'
In which case bash will not expand variables from that parameter string.
The variable is replaced before the script is run.
./script.sh 'test1$test2'
by using single quotes , meta characters like $ will retain its literal value. If double quotes are used, variable names will get interpolated.
As Ignacio told you, the variable is replaced, so your scripts gets ./script.sh test1 as values for $0 and $1.
But even in the case you had used literal quotes to pass the argument, you shoudl always quote "$1" in your echo "${1}". This is a good practice.