This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
Closed 6 years ago.
I can do
bash -x mysellh.sh
to see the commands as they are being executed, but I don't see the variables replaced with their values. For instance, if I have in the shell script:
screen -d -m sh -c 'RAILS_ENV="$R_ENV" bundle exec rake sunspot:solr:reindex[500] > "$PROJECT_DIR"/solr/indexing_out.txt'
I will see:
+ screen -d -m sh -c 'RAILS_ENV="$R_ENV" bundle exec rake sunspot:solr:reindex[500] > "$PROJECT_DIR"/solr/indexing_out.txt'
Even though I've already declared earlier:
R_ENV=test
Is there an option to do what I want?
In this case, it is doing what you asked: the command line "word" that $R_ENV is in is what you have enclosed in single quotes, which inhibit expansion. The -x output is showing you the non-expansion. If you want those variables to be expanded, enclose that first "word" in double quotes and use single quotes in the contents, like this:
screen -d -m sh -c "RAILS_ENV='$R_ENV' bundle exec rake sunspot:solr:reindex[500] > '$PROJECT_DIR'/solr/indexing_out.txt"
Then the single quotes are around the expanded text and the double quotes allow the variables to expand.
Related
This question already has answers here:
Execute command containing quotes from shell variable [duplicate]
(2 answers)
Bash - Variable with variables and commands
(1 answer)
Why does shell ignore quoting characters in arguments passed to it through variables? [duplicate]
(3 answers)
Closed 4 months ago.
I'm trying to running a command inside a container with a redirect that should work inside the container, the command should be stored in a variable.
This is the command that I run inside the container
consul kv export > /consul/data/backup.json
And this is the command that I tried from the docker host
docker exec f0a57e0592e5 consul kv export > /consul/data/backup.json
This is not working because the redirect happens on the host and not inside the container
For working I had to use shell for executing the redirect inside the container
docker exec f0a57e0592e5 sh -c 'consul kv export > /consul/data/backup.json'
Until here is working, what I need is to be able to pass all this command as a variable
command="docker exec f0a57e0592e5 sh -c 'consul kv export > /consul/data/backup.json'"
But when executing $command I receive kv: 1: Syntax error: Unterminated quoted string
Why
Quoting bash manual
Expansion is performed on the command line after it has been split into tokens. There are seven kinds of expansion performed:
brace expansion
tilde expansion
parameter and variable expansion
command substitution
arithmetic expansion
word splitting
filename expansion
So typing this
command="docker exec f0a57e0592e5 sh -c 'consul kv export > /consul/data/backup.json'"
does what you think it does. Double-quotes prevent spiting. So it is just command=something, with no other tokens. So far so good.
Now, when you type
$command
$command is expanded into
docker exec f0a57e0592e5 sh -c 'consul kv export > /consul/data/backup.json'
And this line is processed with the remaining of the expansion chain. But not restarting the whole evaluation chain.
So, after variable substitution comes command substitution ($(...) or `...`). There is none of that.
Then arithmetic expansion $((...)). None neither.
Then word slitting.
So we spit into words, that are docker exec f0a57e0592e5 sh -c 'consul kv export > /consul/data/backup.json' The quotes here are not interpreted as preventing splitting. It is too late for finding the quotes. That happens at "tokens" phase, the very first one. That was done before those quotes were there. So those quotes are just chars here.
And so docker is executed with the said arguments. Including one 'consul argument. That does not what you expect.
Experiments
If you had
command="printf (%s)"
Then
$command 1 2 3
gives (1)(2)(3)
While
$command "1 2 3"
gives (1 2 3)
As expected, I think. Because " were there at the first stage, token splitting, and interpreted to prevent, later, word splittng.
But try this variant
command="printf (%s) 1 2"
$command 3 4 5
Still nothing strange → (1)(2)(3)(4)(5)
command="printf (%s) '1 2'"
$command '3 4 5'
→ ('1)(2')(3 4 5)
Which may be strange. But it's normal. See what happens:
$command '3 4 5' tokens $command (variable reference) 3\ 4\ 5 (string — spaces in it are found to be literal spaces by tokenisation, because surrounded by quotes. I represent that with \) are found
no brace, no tilde to expand
we substitute variable, so now line to evaluate is printf (%s) '1 2' 3\ 4\ 5
no $(...) no $((...) to expand
split into words: printf (%s) '1 2' 3 4 5. Note that ' at this stages are just characters. It is too late for them to influence tokenization, as the ones around 3 4 5 did. So space between 1 and 2 does split words, contrarily to the ones between 3 4 and 5 that were transformed into literal spaces by tokenization, because of the presence of the '.
exec
→ ('1)(2')(3 4 5)
Exactly what we got. Even the strangest result are completely predictable when we understand what exactly occurs to our command.
Solution
You could use the infamous eval
command="eval docker exec f0a57e0592e5 sh -c 'consul kv export > /consul/data/backup.json'"
Which forces the expansion chain to start over from the beginning with the the rest. Including tokenization.
But, as always, eval must be handled with care, especially if there are user input part in this. Which seems not to be the case here. Because if anything in the chain expands to ; rm -fr / you know what happens...
Or you could try to create a script /path/bin/myscript
#!/bin/bash
docker exec f0a57e0592e5 sh -c 'consul kv export > /consul/data/backup.json'
and then have command=/path/bin/myscript
so $command just execute that script (this is a controlled eval. Since launching a script is forcing to evaluate its content).
Or, depending on the context, same with functions or alias.
This question already has answers here:
How to escape single quotes within single quoted strings
(25 answers)
Closed 2 years ago.
i have a file test.txt with data init=6'b000000; and i want to replace it with init=6'b110111; by using vim script in a bash file. I'm getting error
I'm using following command:
vim -c '%s/init=6'b000000;/init=6'b110111;/g | write | quit' test.txt
this works perfectly in vim, not in bash .
This has nothing to do with Vim, you cannot embed a single quote inside a pair of other single quotes. The shell parses the command line arguments before passing them to the invoked command and it just cannot deal with the inner single quote in the way you have defined.
The literal single quote inside need to be preserved before passing to the command. So use double quote and escape the inner quote
vim -c "%s/init=6\'b000000;/init=6\'b110111;/g | write | quit" file
or use the single quote, but include a multi-level double quote inside
vim -c '%s/init=6'"\'"'b000000;/init=6'"\'"'b110111;/g | write | quit' file
This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
Closed 2 years ago.
I have a requirement where I need to replace line feed characters appearing as part of a data field in a CSV files. Fortunately all the unnecessary linefeeds are followed by an '_' character. So I decided to used sed to preprocess the file. The following command in sed works if run it interactively.
sed -i -e ':a;N;$!ba;s/\n_/_/g' file
But in server, the command will get executed with sh -c "<command>"
To test it in local, I ran the same command with sh and it is not working. The command looks as follows.
sh -c "sed -i -e ':a;N;$!ba;s/\n_/_/g' file"
Not sure what I'm missing. Please help.
Between double-quotes, $! expands to the PID of most recent background command (or the empty string if there is not one). Pass sed script as a positional parameter to sh to avoid dealing with quotation issues/escaping every special character:
sh -c 'sed -i -e "$1" file' _ ':a;N;$!ba;s/\n_/_/g'
See also: Difference between single and double quotes in Bash
This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
Closed 2 years ago.
I'm trying to execute a list of commands through the command:
bash -l -c "commands"
However, when I define a variable an then try to use them, the variable is undefined (empty). To be clear:
bash -l -c "var=MyVariable; echo $var"
Bash expansion (as explained here) will expand the variable value inside the double quotes before actually executing the commands. To avoid so you can follow either of these options:
Option 1:
Avoid the expansion using single quotes
bash -l -c 'var=MyVariable; echo $var'
Option 2:
Avoid the expansion inside the double quotes by escaping the desired variables
bash -l -c "var=MyVariable; echo \$var"
The second option allows you to expand some variables and some others not. For example:
expandVar=MyVariable1
bash -l -c "var=MyVariable; echo $expandVar; echo \$var"
Bash expands variables inside double quotes. So in effect in your command $var is replaced by the current value of var before the command is executed. What you want can be accomplished by using single quotes:
bash -l -c 'var=MyVariable; echo $var'
Please note that it is rather unusual to invoke Bash as a login shell (-l) when passing a command string with -c, but then you may have your reasons.
I have an inclusion file test.inc:
export XXX=xxx
I use it when call bash to interpret a string:
bash -c ". test.inc; echo $XXX"
But the variable is not set at the point of echo command. If I do 'export' I can see it though:
bash -c ". test.inc; export"
Shows
declare -x XXX="XXX"
How do I make my first command see the exported variables from sourced files when I use bash -c syntax?
You are using double quotes. Therefore your current shell expands $XXX long before the bash -c instance sees it. Switch to single quotes, or escape the dollar sign.