I'm trying to understand -c option for bash better. The man page says:
-c: If the -c option is present, then commands are read from the first non-option argument command_string. If there are arguments after the command_string, they are assigned to the positional parameters, starting with $0.
I'm having trouble understanding what this means.
If I do the following command with and without bash -c, I get the same result (example from http://www.tldp.org/LDP/abs/html/abs-guide.html):
$ set w x y z; IFS=":-;"; echo "$*"
w:x:y:z
$ bash -c 'set w x y z; IFS=":-;"; echo "$*"'
w:x:y:z
bash -c isn't as interesting when you're already running bash. Consider, on the other hand, the case when you want to run bash code from a Python script:
#!/usr/bin/env python
import subprocess
fileOne='hello'
fileTwo='world'
p = subprocess.Popen(['bash', '-c', 'diff <(sort "$1") <(sort "$2")',
'_', # this is $0 inside the bash script above
fileOne, # this is $1
fileTwo, # and this is $2
])
print p.communicate() # run that bash interpreter, and print its stdout and stderr
Here, because we're using bash-only syntax (<(...)), you couldn't run this with anything that used POSIX sh by default, which is the case for subprocess.Popen(..., shell=True); using bash -c thus provides access to capabilities that wouldn't otherwise be available without playing with FIFOs yourself.
Incidentally, this isn't the only way to do that: One could also use bash -s, and pass code in on stdin. Below, that's being done not from Python but POSIX sh (/bin/sh, which likewise is not guaranteed to have <(...) available):
#!/bin/sh
# ...this is POSIX sh code, not bash code; you can't use <() here
# ...so, if we want to do that, one way is as follows:
fileOne=hello
fileTwo=world
bash -s "$fileOne" "$fileTwo" <<'EOF'
# the inside of this heredoc is bash code, not POSIX sh code
diff <(sort "$1") <(sort "$2")
EOF
The -c option finds its most important uses when bash is launched by another program, and especially when the code to be executed may or does include redirections, pipelines, shell built-ins, shell variable assignments, and / or non-trivial lists. On POSIX systems that have /bin/sh being an alias for bash, it specifically supports the C library's system() function.
Equivalent behavior is much trickier to implement on top of fork / exec without using -c, though not altogether impossible.
How to execute BASH code from outside the BASH shell?
The answer is, using the -c option, which makes BASH execute whatever that has been passed as an argument to -c.
So, yeah, this is the purpose of this option, to execute BASH code arbitrarily, but just in another way.
Related
I'm running a bash script using
wget -O - https://myserver/install/Setup.sh | bash
How can I pass a parameter to the above script so it runs? something like
wget -O - https://myserver/install/Setup.sh parameter1 | bash
You can also run your script with:
wget -qO - 'https://myserver/install/Setup.sh' | bash -s parameter1
See: man bash OPTIONS -s
-s If the -s option is present, or if no arguments remain
after option processing, then commands are read from the
standard input. This option allows the positional
parameters to be set when invoking an interactive shell or
when reading input through a pipe.
or alternatively use the -c option.
bash -c "$(wget -qO - 'https://myserver/install/Setup.sh')" '' parameter1
the '' defines the parameter $0 to be empty string. In a normal file based script invocation, the parameter $0 contains the caller script name.
See: man bash OPTIONS -c
-c If the -c option is present, then commands are read from
the first non-option argument command_string. If there are
arguments after the command_string, the first argument is
assigned to $0 and any remaining arguments are assigned to
the positional parameters. The assignment to $0 sets the
name of the shell, which is used in warning and error
messages.
The standard format for the bash (or sh or similar) command is bash scriptfilename arg1 arg2 .... If you leave off all the first argument (the name or path of the script to run), it reads the script from stdin. Unfortunately, there's no way to leave off the firs argument but pass the others. Fortunately, you can pass /dev/stdin as the first argument and get the same effect (at least on most unix systems):
wget -O - https://myserver/install/Setup.sh | bash /dev/stdin parameter1
If you're on a system that doesn't have /dev/stdin, you might have to look around for an alternative way to specify stdin explicitly (/dev/fd/0 or something like that).
Edit: Léa Gris suggestion of bash -s arg1 arg2 ... is probably a better way to do this.
Consider the following code:
#!/bin/bash -x
VAR='1 2 3'
bash -c "echo "\$$VAR""
eval "echo "\$$VAR""
bash -c "echo \"\$$VAR\""
eval "echo \"\$$VAR\""
Which outputs:
+ VAR='1 2 3'
+ bash -c 'echo $1' 2 3
3
+ eval 'echo $1' 2 3
++ echo 2 3
2 3
+ bash -c 'echo "$1 2 3"'
2 3
+ eval 'echo "$1 2 3"'
++ echo ' 2 3'
2 3
It seems both eval and bash -c interprets the codes the same way i.e "echo "\$$VAR"" to 'echo $1' 2 3 and "echo \"\$$VAR\"" to 'echo "$1 2 3"'.
The only difference I seem to notice is that bash -c opens a subshell and thus varies the results from eval. For example, in
bash -c 'echo $1' 2 3
2 and 3 are the positional parameters for the subshell. On the other hand, in
eval 'echo $1' 2 3
they are just another arguments for echo.
So my question is, is the -c option (bash -c, sh -c or other shells' equivalents) safe to use or is it evil like eval?
Yes, you should avoid using sh -c and equivalents in your shell scripts, just as you avoid eval.
eval "$foo"
...is, when one boils it down, a security risk because it treats data as code, restarting the parsing process at the very beginning (thus, running expansions, redirections, etc). Moreover, because quoting contexts are considered in this process, content inside of the data being evaluated is able to escape quotes, terminate commands, and otherwise attempt to evade any efforts made at security.
sh -c "$foo"
does the same -- running expansions, redirections, and the like -- only in a completely new shell (not sharing non-exported variables or other state).
Both of these mean that content such as $(rm -rf /) is prone to being expanded unless great care is taken, rather than ensuring that -- as is generally the case -- data will only ever be treated as data, which is a foundational element of writing secure code.
Now, the happy thing here is that when you're using bash (or zsh, or ksh93) rather than sh, you can avoid using eval in almost all cases.
For instance:
value=$(eval "echo \$$varname")
...can be replaced with:
value=${!varname}
...and in bash,
eval "$varname="'$value'
...can be replaced with...
printf -v varname %s "$value"
...and so forth (with ksh93 and zsh having direct equivalents to the latter); more advanced formulations involving associative maps and the like are best addressed with the new bash 4.3 (and ksh93) namevar support.
Notably, bash -c doesn't replace eval effectively in most of the above examples, because it doesn't run in the same context: Changes made to shell state are thrown away when the child process exits; thus, not only does bash -c not buy safety, but it doesn't work as an eval replacement to start with.
As far as I am aware, there is no reason to use bash -c from a bash shell. Using it will start a new process, which is expensive.
You can use eval, which doesn't start a new process. If you want a sub-shell (for example, to preserve the environment) you can use parentheses.
Normally, bash -c (or other shells with -c) is used to execute a command from another non-shell environment (or perhaps a DSL interpreted by a shell) where shell expansion of the arguments is required. In the old days, you might use it with execvp in a C program, for example. These days, there are usually ways of running a command using a shell in most environments.
I am looking for the quoting/splitting rules for a command passed to script -c command. The man pages just says
-c, --command command: Run the command rather than an interactive shell.
but I want to make sure "command" is properly escaped.
The COMMAND argument is just a regular string that is processed by the shell as if it were an excerpt of a file. We may think of -c COMMAND as being functionally equivalent of
printf '%s' COMMAND > /tmp/command_to_execute.sh
sh /tmp/command_to_execute.sh
The form -c COMMAND is however superior to the version relying of an auxiliary file because it avoids race conditions related to using an auxiliary file.
In the typical usage of the -c COMMAND option we pass COMMAND as a single-quoted string, as in this pseudo-code example:
sh -c '
do_some_complicated_tests "$1" "$2";
if something; then
proceed_this_way "$1" "$2";
else
proceed_that_way "$1" "$2";
fi' ARGV0 ARGV1 ARGV2
If command must contain single-quoted string, we can rely on printf to build the COMMAND string, but this can be tedious. An example of this technique is illustrated
by the overcomplicated grep-like COMMAND defined here:
% AWKSCRIPT='$0 ~ expr {print($0)}'
% COMMAND=$(printf 'awk -v expr="$1" \047%s\047' "$AWKSCRIPT")
% sh -c "$COMMAND" print_matching 'tuning' < /usr/share/games/fortune/freebsd-tips
"man tuning" gives some tips how to tune performance of your FreeBSD system.
Recall that 047 is octal representation of the ASCII code for the single quote character.
As a side note, these constructions are quite command in Makefiles where they can replace shell functions.
This command works fine:
$ bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
However, I don't understand how exactly stable is passed as a parameter to the shell script that is downloaded by curl. That's the reason why I fail to achieve the same functionality from within my own shell script - it gives me ./foo.sh: 2: Syntax error: redirection unexpected:
$ cat foo.sh
#!/bin/sh
bash -s stable < <(curl -s https://raw.github.com/wayneeseguin/rvm/master/binscripts/rvm-installer)
So, the questions are: how exactly this stable param gets to the script, why are there two redirects in this command, and how do I change this command to make it work inside my script?
Regarding the "redirection unexpected" error:
That's not related to stable, it's related to your script using /bin/sh, not bash. The <() syntax is unavailable in POSIX shells, which includes bash when invoked as /bin/sh (in which case it turns off nonstandard functionality for compatibility reasons).
Make your shebang line #!/bin/bash.
Understanding the < <() idiom:
To be clear about what's going on -- <() is replaced with a filename which refers to the output of the command which it runs; on Linux, this is typically a /dev/fd/## type filename. Running < <(command), then, is taking that file and directing it to your stdin... which is pretty close the behavior of a pipe.
To understand why this idiom is useful, compare this:
read foo < <(echo "bar")
echo "$foo"
to this:
echo "bar" | read foo
echo "$foo"
The former works, because the read is executed by the same shell that later echoes the result. The latter does not, because the read is run in a subshell that was created just to set up the pipeline and then destroyed, so the variable is no longer present for the subsequent echo.
Understanding bash -s stable:
bash -s indicates that the script to run will come in on stdin. All arguments, then, are fed to the script in the $# array ($1, $2, etc), so stable becomes $1 when the script fed in on stdin is run.
I want to inject a transparent wrappering command on each shell command in a make file. Something like the time shell command. ( However, not the time command. This is a completely different command.)
Is there a way to specify some sort of wrapper or decorator for each shell command that gmake will issue?
Kind of. You can tell make to use a different shell.
SHELL = myshell
where myshell is a wrapper like
#!/bin/sh
time /bin/sh "$0" "$#"
However, the usual way to do that is to prefix a variable to all command calls. While I can't see any show-stopper for the SHELL approach, the prefix approach has the advantage that it's more flexible (you can specify different prefixes for different commands, and override prefix values on the command line), and could be visibly faster.
# Set Q=# to not display command names
TIME = time
foo:
$(Q)$(TIME) foo_compiler
And here's a complete, working example of a shell wrapper:
#!/bin/bash
RESULTZ=/home/rbroger1/repos/knl/results
if [ "$1" == "-c" ] ; then
shift
fi
strace -f -o `mktemp $RESULTZ/result_XXXXXXX` -e trace=open,stat64,execve,exit_group,chdir /bin/sh -c "$#" | awk '{if (match("Process PID=\d+ runs in (64|32) bit",$0) == 0) {print $0}}'
# EOF
I don't think there is a way to do what you want within GNUMake itself.
I have done things like modify the PATH env variable in the Makefile so a directory with my script linked to all name the bins I wanted wrapped was executed rather than the actual bin. The script would then look at how it was called and exec the actual bin with the wrapped command.
ie. exec time "$0" "$#"
These days I usually just update the targets in the Makefile itself. Keeping all your modifications to one file is usually better IMO than managing a directory of links.
Update
I defer to Gilles answer. It's a better answer than mine.
The program that GNU make(1) uses to run commands is specified by the SHELL make variable. It will run each command as
$SHELL -c <command>
You cannot get make to not put the -c in, since that is required for most shells. -c is passed as the first argument ($1) and <command> is passed as a single argument string as the second argument ($2).
You can write your own shell wrapper that prepends the command that you want, taking into account the -c:
#!/bin/sh
eval time "$2"
That will cause time to be run in front of each command. You need eval since $2 will often not be a single command and can contain all sorts of shell metacharacters that need to be expanded or processed.