I have a bash variable:
A="test; ls"
I want to use it as part of a call:
echo $A
I'm expecting it to be expanded into:
echo test; ls
However, it is expanded:
echo "test;" "ls"
How is it possible to achieve what I want? The only solution I can think of is this:
bash -c "echo $A"
Maybe there is something more elegant?
If you're building compound commands, running redirections, etc., then you need to use eval:
A="test; ls"
eval "$A"
However, it's far better not to do this. A typical use case that follows good practices follows:
my_cmd=( something --some-arg "another argument with spaces" )
if [[ $foo ]]; then
my_cmd+=( --foo="$foo" )
fi
"${my_cmd[#]}"
Unlike the eval version, this can only run a single command line, and will run it exactly as-given -- meaning that even if foo='$(rm -rf /)' you won't get your hard drive wiped. :)
If you absolutely must use eval, or are forming a shell command to be used in a context where it will necessarily be shell-evaluated (for instance, passed on a ssh command line), you can achieve a hybrid approach using printf %q to form command lines safe for eval:
printf -v cmd_str 'ls -l %q; exit 1' "some potentially malicious string"
eval "$cmd_str"
See BashFAQ #48 for more details on the eval command and why it should be used only with great care, or BashFAQ #50 for a general discussion of pitfalls and best practices around programmatically constructing commands. (Note that bash -c "$cmd_str" is equivalent to eval in terms of security impact, and requires the same precautions to use safely).
dont use echo, just have the var
A="echo test; ls"
$A
Related
In bash, in this specific case, echo behaves like so:
$ bash -c 'echo "a\nb"'
a\nb
but in zsh the same thing turns out very differently...:
$ zsh -c 'echo "a\nb"'
a
b
and fwiw in fish, because I was curious:
$ fish -c 'echo "a\nb"'
a\nb
I did realize that I can run:
$ zsh -c 'echo -E "a\nb"'
a\nb
But now I am worried that I'm about to stumble into more gotchas on such a basic operation. (Thus my investigation into fish: if I'm going to have to make changes at such a low level for zsh, why not go all the way and switch up to something that is blatant about being so different?)
I did not myself find any documentation to help clarify this echo difference in bash vs zsh or pages directly listing the differences, so can someone here list them out? And maybe direct me to any broader set of potentially impactful gotchas when making the switch, that would cover this case?
Usually prefer printf for consistent results.
If you need predictable consistent echo implementation, you can override it with your own function.
This will behave the same, regardless of the shell.
echo(){ printf %s\\n "$*";}
echo "a\nb"
echo is good and portable only for printing literal strings that end
with a newline but it's not good for anything more complex, you can
read more about it in this
answer and in
Shellcheck documentation
here.
Even though according to
POSIX
each implementation has to understand character sequences without any
additional options you cannot rely on that. As you already noticed, in
Bash for example echo 'a\nb' produces a\nb but it can be changed
with xpg_echo shell option:
$ echo 'a\nb'
a\nb
$ shopt -s xpg_echo
$ echo 'a\nb'
a
b
And maybe direct me to any broader set of potentially impactful
gotchas when making the switch, that would cover this case?
Notice that the inconsistency between different echo implementations
can manifest itself not only in shell but also in other places where
shell is used indirectly, for example in Makefile. I've once come
across Makefile that looked like this:
all:
#echo "target\tdoes this"
#echo "another-target\tdoes this"
make uses /bin/sh to run these commands so if /bin/sh is a symlink
to bash on your system what you get is:
$ make
target\tdoes this
another-target\tdoes this
If you want portability in shell use printf. This:
printf 'a\nb\n'
should produce the same output in most shells.
echo provides the same output, regardless of the shell interpreter it's called from.
The difference lies in the way that each shell will print the standard output buffer to the screen. zsh interprets escape characters/sequences (i.e., "\n\t\v\r..." or ANSI escape sequences) automatically where bash does not.
Using bash, you'll have to supply the -e flag to print newlines:
#Input:
echo "[text-to-print]\n[text-to-print-on-newline]"
#Output:
[text-to-print]\n[text-to-print-on-newline]
#Input:
echo -e "[text-to-print]\n[text-to-print-on-newline]"
#Output:
[text-to-print]
[text-to-print-on-newline]
Using zsh, the interpreter does the escape sequence interpretation by itself, and the output is the same regardless of the -e flag:
#Input:
echo "[text-to-print]\n[text-to-print-on-newline]"
#Output:
[text-to-print]
[text-to-print-on-newline]
#Input:
echo -e "[text-to-print]\n[text-to-print-on-newline]"
#Output:
[text-to-print]
[text-to-print-on-newline]
This should fix the discrepancy that you're seeing between shell interpreters.
I came across a script that is supposed to set up postgis in a docker container, but it references this "${psql[#]}" command in several places:
#!/bin/sh
# Perform all actions as $POSTGRES_USER
export PGUSER="$POSTGRES_USER"
# Create the 'template_postgis' template db
"${psql[#]}" <<- 'EOSQL'
CREATE DATABASE template_postgis;
UPDATE pg_database SET datistemplate = TRUE WHERE datname = 'template_postgis';
EOSQL
I'm guessing it's supposed to use the psql command, but the command is always empty so it gives an error. Replacing it with psql makes the script run as expected. Is my guess correct?
Edit: In case it's important, the command is being run in a container based on postgres:11-alpine.
$psql is supposed to be an array containing the psql command and its arguments.
The script is apparently expected to be run from here, which does
psql=( psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --no-password )
and later sources the script in this loop:
for f in /docker-entrypoint-initdb.d/*; do
case "$f" in
*.sh)
# https://github.com/docker-library/postgres/issues/450#issuecomment-393167936
# https://github.com/docker-library/postgres/pull/452
if [ -x "$f" ]; then
echo "$0: running $f"
"$f"
else
echo "$0: sourcing $f"
. "$f"
fi
;;
*.sql) echo "$0: running $f"; "${psql[#]}" -f "$f"; echo ;;
*.sql.gz) echo "$0: running $f"; gunzip -c "$f" | "${psql[#]}"; echo ;;
*) echo "$0: ignoring $f" ;;
esac
echo
done
See Setting an argument with bash for the reason to use an array rather than a string.
The #!/bin/sh and the [#] are incongruous. This is a bash-ism, where the psql variable is an array. This literal quote dollarsign psql bracket at bracket quote is expanded into "psql" "array" "values" "each" "listed" "and" "quoted" "separately." It's the safer way, e.g., to accumulate arguments to a command where any of them might have spaces in them.
psql=(/foo/psql arg arg arg) is the best way to define the array you need there.
It might look obscure, but it would work like so...
Let's say we have a bash array wc, which contains a command wc, and an argument -w, and we feed that a here document with some words:
wc=(wc -w)
"${wc[#]}" <<- words
one
two three
four
words
Since there are four words in the here document, the output is:
4
In the quoted code, there needs to be some prior point, (perhaps a calling script), that does something like:
psql=(psql -option1 -option2 arg1 arg2 ... )
As to why the programmer chose to invoke a command with an array, rather than just invoke the command, I can only guess... Maybe it's a crude sort of operator overloading to compensate for different *nix distros, (i.e. BSD vs. Linux), where the local variants of some necessary command might have different names from the same option, or even use different commands. So one might check for BSD or Linux or a given version, and reset psql accordingly.
The answer from #Barmar is correct.
The script was intended to be "sourced" and not "executed".
I faced the same problem and came to the same answer after I read that it had been reported here and fixed by "chmod".
https://github.com/postgis/docker-postgis/issues/119
Therefore, the fix is to change the permissions.
This can be done either in your git repository:
chmod -x initdb-postgis.sh
or add a line to your docker file.
RUN chmod -x /docker-entrypoint-initdb.d/10_postgis.sh
I like to do both so that it is clear to others.
Note: if you are using git on windows then permission can be lost. Therefore, "chmod" in the docker file is needed.
I am working on an option driven bash script that will use getopts. The script has cases where it can accept multiple options and specific cases where only one option is accepted. While testing a few cases out I ran into this issue which I have reduced down to pseudo-code for this question.
for arg in "$#"; do
echo ${arg}
done
echo "end"
Running below returns:
$ ./test.sh -a -b
-a
end
I am running bash 4.1.2, why isn't the -b returned on the empty line? I assume this has to do with the '-'.
I cannot reproduce your exact error, but this is the risk of using echo: if $arg is a valid option, it will be treated as such, not a string to print. Use printf instead:
printf '%s\n' "$arg"
Also check if you have applied any "shift" commands that might remove the arguments before you test then (typical in a argument collection block that might include a case statement)
I have read a line of bash code from the file, and I want to send it to log. To make it more useful, I'd like to send the variable-expanded version of the line.
I want to expand only shell variables. I don't want the pipes to be interpreted, nor I don't want to spawn any side processes, like when expanding a line with $( rm -r /).
I know that variable expansion is very deeply woven into the bash. I hope there is a way to perform just expansion, without any side effects, that would come from pipes, external programs and - perhaps - here-documents.
Maybe there is something like eval?
#!/bin/bash
linenr=42
line=`sed "${linenr}q;d" my-other-script.sh`
shell-expand $line >>mylog.log
I want a way to expand only the shell variables.
if x=a, a="example" then I want the following expansions:
echo $x should be echo a.
echo ${a} should be echo example
touch ${!x}.txt should be touch example.txt
if [ (( ${#a} - 6 )) -gt 10 ]; then echo "Too long string" should be if [ 1 -gt 10 ]; then echo "Too long string"
echo "\$a and \$x">/dev/null should be echo "\$a and \$x>dev/null"
For those arriving five years later, and, although it is not the best answer to the OP's problem, an answer to the question is as follows. Bash can do indirect parameter expansion:
some_param=a
a=b
echo ${!some_param}
echo $BASH_VERSION
# 5.0.18(1)-release
You are absolutely right, it is very dangerous to use the bash.
In fact your command suffers from your problem.
Let us discuss your script in detail:
#!/bin/bash
line=`sed "${42}q;d" my-other-script.sh`
shell-expand $line >>mylog.log
The sed may produce many lines of output, so it is misleading to use the name line.
Then you did not quote $line, this may have obscure effects:
$ x='| grep x'
$ ls -l $x
ls: cannot access |: No such file or directory
ls: cannot access grep: No such file or directory
-rw-rw-r-- 1 foo bar 34493 Nov 19 18:51 x
$
In this case the pipe is not executed, but passed to the program ls.
If you have untrusted input, it is very hard to program a robust shell script.
Using eval is evil - I would never suggest using it, especially for such a purpose!
An alternative way would be in perl, iterate over the $ENV array and replace all env keys by the env values. This way you have more control over the things, which may happen.
I have two variables in bash that complete the name of another one, and I want to expand it but don't know how to do it
I have:
echo $sala
a
echo $i
10
and I want to expand ${a10} in this form ${$sala$i} but apparently the {} scape the $ signs.
There are a few ways, with different advantages and disadvantages. The safest way is to save the complete parameter name in a single parameter, and then use indirection to expand it:
tmp="$sala$i" # sets $tmp to 'a10'
echo "${!tmp}" # prints the parameter named by $tmp, namely $a10
A slightly simpler way is a command like this:
eval echo \${$sala$i}
which will run eval with the arguments echo and ${a10}, and therefore run echo ${a10}. This way is less safe in general — its behavior depends a bit more chaotically on the values of the parameters — but it doesn't require a temporary variable.
Use the eval.
eval "echo \${$sala$i}"
Put the value in another variable.
result=$(eval "echo \${$sala$i}")
The usual answer is eval:
sala=a
i=10
a10=37
eval echo "\$$sala$i"
This echoes 37. You can use "\${$sala$i}" if you prefer.
Beware of eval, especially if you need to preserve spaces in argument lists. It is vastly powerful, and vastly confusing. It will work with old shells as well as Bash, which may or may not be a merit in your eyes.
You can do it via indirection:
$ a10=blah
$ sala=a
$ i=10
$ ref="${sala}${i}"
$ echo $ref
a10
$ echo ${!ref}
blah
However, if you have indexes like that... an array might be more appropriate:
$ declare -a a
$ i=10
$ a[$i]="test"
$ echo ${a[$i]}
test