Why does bash do what I'd expect here with a compound command in a subshell:
$ bash -x -c 'set -e; (false && true; echo hi); echo here'
+ set -e
+ false
+ echo hi
hi
+ echo here
here
But NOT do what I'd expect here:
$ bash -x -c 'set -e; (eval "false && true"; echo hi); echo here'
+ set -e
+ eval 'false && true'
++ false
Basically, the difference is between 'eval'-uating a compound command and just executing a compound command. When the shell executes a compound command, non-terminal commands in the compound command that fail do not cause the entire compound command to fail, they simply terminate the command. But when eval runs the compound command and any non-terminal sub-command terminates the command with an error, eval terminates the command with an error.
I guess I need to format my eval statement like this:
eval "false && true" || :
so that the eval command doesn't exit my subshell with an error, because this works as I'd expect it to:
$ bash -x -c 'set -e; (eval "false && true" || :; echo hi); echo here'
+ set -e
+ false
+ echo hi
hi
+ echo here
here
The problem I have with this is that I've written a function:
function execute() {
local command="$1"
local remote="$2"
if [ ! -z "$remote" ]; then
$SSH $remote "$command" || :
else
eval "$command" || :
fi
}
I'm using set -e in my script. The same problem occurs with ssh in this function - if the last command in the ssh script is a compound command that terminates early, the entire command terminates with an error. I want commands like this to behave as if they were executing locally - early terminating compound commands should not cause ssh or eval to return 1, failing the entire command. If I tack || : on the end of my eval statement or my ssh statement, then all such commands will succeed, even if they shouldn't because the last command in the eval'd or ssh'd command failed.
Any ideas would be much appreciated.
I should also mention that set -e is terribly error-prone; see http://mywiki.wooledge.org/BashFAQ/105 for a bunch of examples. So the best solution might be to dispense with it, and write your own logic to detect errors and abort.
That out of the way . . .
The problem here is that eval "false && true" is a single command, and evaluates to false (nonzero), so set -e aborts after that command runs.
If you were instead to run eval "false && true; true", you would not see this behavior, because then eval evaluates to true (zero). (Note that, although eval does implement the set -e behavior, it obeys the rule that false && true is non-aborting.)
This is not actually specific to eval, by the way. A subshell would give the same result, for the same reason:
$ bash -x -c 'set -e; (false && true); echo here'
+ set -e
+ false
The simplest fix for your problem is probably just to run an extra true if the end is reached:
$SSH $remote "set -e; $command; true"
eval "$command; true"
eval counts as its own command with its own exit code.
Since eval "false && true" returns an exit code of 1, it triggers set -e.
Related
I have a function that runs a set of scripts that set variables, functions, and aliases in the current shell.
reloadVariablesFromScript() {
for script in "${scripts[#]}"; do
. "$script"
done
}
If one of the scripts has an error, I want to exit the script and then exit the function, but not to kill the shell.
reloadVariablesFromScript() {
for script in "${scripts[#]}"; do
{(
set -e
. "$script"
)}
if [[ $? -ne 0 ]]; then
>&2 echo $script failed. Skipping remaining scripts.
return 1
fi
done
}
This would do what I want except it doesn't set the variables in the script whether the script succeeds or fails.
Without the subshell, set -e causes the whole shell to exit, which is undesirable.
Is there a way I can either prevent the called script from continuing on an error without killing the shell or else set/export variables, aliases, and functions from within a subshell?
The following script simulates my problem:
test() {
{(
set -e
export foo=bar
false
echo Should not have gotten here!
export bar=baz
)}
local errorCode=$?
echo foo="'$foo'". It should equal 'bar'.
echo bar="'$bar'". It should not be set.
if [[ $errorCode -ne 0 ]]; then
echo Script failed correctly. Exiting function.
return 1
fi
echo Should not have gotten here!
}
test
If worst comes to worse, since these scripts don't actually edit the filesystem, I can run each script in a subshell, check the exit code, and if it succeeds, run it outside of a subshell.
Note that set -e has a number of surprising behaviors -- relying on it is not universally considered a good idea. That caveat being give, though: We can shuffle environment variables, aliases, and shell functions out as text:
envTest() {
local errorCode newVars
newVars=$(
set -e
{
export foo=bar
false
echo Should not have gotten here!
export bar=baz
} >&2
# print generate code which, when eval'd, recreates our functions and variables
declare -p | egrep -v '^declare -[^[:space:]]*r'
declare -f
alias -p
); errorCode=$?
if (( errorCode == 0 )); then
eval "$newVars"
fi
printf 'foo=%q. It should equal %q\n' "$foo" "bar"
printf 'bar=%q. It should not be set.\n' "$bar"
if [[ $errorCode -ne 0 ]]; then
echo 'Script failed correctly. Exiting function.'
return 1
fi
echo 'Should not have gotten here!'
}
envTest
Note that this code only evaluates either export should the entire script segment succeed; the question text and comments appear to indicate that this is acceptable if not desired.
This question already has an answer here:
Why doesn't set -e cause a failure with `false || false && true`? [duplicate]
(1 answer)
Closed 6 years ago.
Why is the third case returning success with exit code 0?
case 1 ~$ bash -c 'set -e; false || true; echo success'; echo $?
success
0
case 2 ~$ bash -c 'set -e; true || false; echo success'; echo $?
success
0
case 3 ~$ bash -c 'set -e; false && true; echo success'; echo $?
success
0
case 4 ~$ bash -c 'set -e; true && false; echo success'; echo $?
1
case 5 ~$ bash -c 'set -e; false || false; echo success'; echo $?
1
case 6 ~$ bash -c 'set -e; false && false; echo success'; echo $?
success
0
The bash documentation for set -e says:
The shell does not exit if the command that fails is [...] part of any command executed in a && or || list except the command following the final && or ||, [...]
The command list in question is false && true. The failing command is false, which is not the last command in the list, so the shell does not exit. The 0 you're seeing is the exit status of echo success.
set -e is a bit subtle.
From the reference:
-e
When this option is on, when any command fails (for any of the reasons listed in Consequences of Shell Errors or by returning an exit
status greater than zero), the shell immediately shall exit, as if by
executing the exit special built-in utility with no arguments, with
the following exceptions:
The failure of any individual command in a multi-command pipeline shall not cause the shell to exit. Only the failure of the
pipeline itself shall be considered.
The -e setting shall be ignored when executing the compound list following the while, until, if, or elif reserved word, a pipeline
beginning with the ! reserved word, or any command of an AND-OR list
other than the last.
If the exit status of a compound command other than a subshell command was the result of a failure while -e was being ignored, then
-e shall not apply to this command.
This requirement applies to the shell environment and each subshell environment separately. For example, in:
set -e; (false; echo one) | cat; echo two
the false command causes the subshell to exit without executing echo one; however, echo two is executed because the exit status of the
pipeline (false; echo one) | cat is zero.
Since false && true is part of an AND or OR list and false isn't the last, the shell doesn't exit immediately.
So echo success is executed, and it's return code is 0.
By the way, have you noticed case 6?
true && false; echo success returns the return code from echo which is 0.
bash -c 'false && true'; echo $?
yields 1 as expected. Pheew!
Aside: bash -c 'set -e; false || false; echo success'; echo $? yields return code 1 because second command does not execute echo (stops at false because errors stop the current command), note that success is not printed.
Let's imagine we have this code (script.sh):
#!/bin/bash
set -e
f() {
echo "[f] Start" >&2
echo "f:before-false1"
echo "f:before-false2"
false
echo "f:after-false"
echo "[f] Fail! I don't want this executed" >&2
}
out=$(f)
The output:
$ bash myscript.sh
[f] Start
[f] Fail! I don't want this executed
I understand that $(...) starts a sub-shell where set -e is not propagated, so my question is: what's the idiomatic way to make this run as expected without too much clutter? I can see 3 solutions, none of which I like (nor I am actually sure they indeed work): 1) Add set -e to the start of f (and every other function in the app). 2) Run $(set -e && f). 3) Add ... || return 1 to every command that may fail.
It's not the prettiest solution, but it does allow you to emulate set -e for the current shell as well as any functions and subshells:
#!/bin/bash
# Set up an ERR trap that unconditionally exits with a nonzero exit code.
# Similar to -e, this trap is invoked when a command reports a nonzero
# exit code (outside of a conditional / test).
# Note: This trap may be called *multiple* times.
trap 'exit 1' ERR
# Now ensure that the ERR trap is called not only by the current shell,
# but by any subshells too:
# set -E (set -o errtrace) causes functions and subshells to inherit
# ERR traps.
set -E
f() {
echo "[f] Start" >&2
echo "f:before-false1"
echo "f:before-false2"
false
echo "f:after-false"
echo "[f] Fail! I don't want this executed" >&2
}
out=$(f)
Output (to stderr) if you call this script (exit code afterward will be 1) - note how the 2nd echo to stderr (>&2) is not printed, proving that the execution of false aborted the command substitution:
[f] Start
Note:
By design, set -e / trap ERR only respond to failures that aren't part of conditionals (see man bash, under the description of set (search for literal " set ["), for the exact rules, which changed slightly between Bash 3.x and 4.x).
Thus, for instance, f does NOT trigger the trap in the following commands: if ! f; then ..., f && echo ok; the following triggers the trap in the subshell (command substitution $(...), but not in the enclosing conditional ([[ ... ]]): [[ $(f) == 'foo' ]] && echo ok, so the script as a whole doesn't abort.
To exit a function / subshell explicitly in such cases, use something like || return 1 / || exit 1, or call the function / subshell separately, outside of a conditional first; e.g., in the [[ $(f) == 'foo' ]] case: res=$(f); [[ $res == 'foo' ]] - res=$(f) will then trigger the trap for the current shell too.
As for why the trap code may be invoked multiple times: In the case at hand, false inside f() first triggers the trap, and then, because the trap's exit 1 exits the subshell ($(f)), the trap is triggered again for the current shell (the one running the script).
Instead of command substitution, you should use process substitution to call your function so that set -e remains in effect:
mapfile arr < <(f) # call function f using process substitution
out="${arr[*]}" # convert array content into a string
declare -p out # check output
Output:
[f] Start
declare -- out="f:before-false1
f:before-false2
"
According to this accepted answer using the set -e builtin should suffice for a bash script to exit on the first error. Yet, the following script:
#!/usr/bin/env bash
set -e
echo "a"
echo "b"
echo "about to fail" && /bin/false && echo "foo"
echo "c"
echo "d"
prints:
$ ./foo.sh
a
b
about to fail
c
d
removing the echo "foo" does stop the script; but why?
To simplify EtanReisner's detailed answer, set -e only exits on an 'uncaught' error. In your case:
echo "about to fail" && /bin/false && echo "foo"
The failing code, /bin/false, is followed by && which tests its exit code. Since && tests the exit code, the assumption is that the programmer knew what he was doing and anticipated that this command might fail. Ergo, the script does not exit.
By contrast, consider:
echo "about to fail" && /bin/false
The program does not test or branch on the exit code of /bin/false. So, when /bin/false fails, set -e will cause the script to exit.
Alternative that exits when /bin/false fails
Consider:
set -e
echo "about to fail" && /bin/false ; echo "foo"
This version will exit if /bin/false fails. As in the case where && was used, the final statement echo "foo" would therefore only be executed if /bin/false were to succeed.
Because that answer is not sufficiently specific enough.
It should say (bolded text is my addition):
# Any subsequent simple commands which fail will cause the shell script to exit immediately
Since the man page reads thusly:
-e Exit immediately if a simple command (see SHELL GRAMMAR
above) exits with a non-zero status. The shell does not
exit if the command that fails is part of the command
list immediately following a while or until keyword,
part of the test in an if statement, part of a && or ││
list, or if the command’s return value is being inverted
via !. A trap on ERR, if set, is executed before the
shell exits.
And SHELL GRAMMAR expands thusly:
SHELL GRAMMAR
Simple Commands
A simple command is a sequence of optional variable assignments fol-
lowed by blank-separated words and redirections, and terminated by a
control operator. The first word specifies the command to be executed,
and is passed as argument zero. The remaining words are passed as
arguments to the invoked command.
The return value of a simple command is its exit status, or 128+n if
the command is terminated by signal n.
I came across set -e for Bash scripts but had problems understanding what happens regarding the evaluation of the command following the last && or || in a && or || list. I know of the following quote from http://man7.org/linux/man-pages/man1/bash.1.html about set -e:
The shell does not exit if the command that fails is (...) part of
any command executed in a && or || list except the command
following the final && or || (...)
To test this, I wrote a small Bash script:
#!/bin/bash
bash -c "set -e ; true ; echo -n A"
bash -c "set -e ; false ; echo -n B"
bash -c "set -e ; true && true ; echo -n C"
bash -c "set -e ; true && false ; echo -n D"
bash -c "set -e ; false && true ; echo -n E"
bash -c "set -e ; false && false ; echo -n F"
bash -c "set -e ; true || true ; echo -n G"
bash -c "set -e ; true || false ; echo -n H"
bash -c "set -e ; false || true ; echo -n I"
bash -c "set -e ; false || false ; echo -n J"
echo ""
It prints:
ACEFGHI
About A:
true does not have a non-zero status. Therefore, the shell doesn't exit.
About B:
false does have a non-zero status and is not part of a && or || list. Therefore, the shell exits.
About C:
This is a && or || list. We will have to look at the command following the last && or ||. The command is true which does not have a non-zero status. So it doesn't matter if the command is evaluated or not - the shell doesn't exit either way.
About D:
This is a && or || list. We will have to look at the command following the last && or ||. This time, the command is false which does have a non-zero status. So we have to check if that false is being evaluated - which is indeed the case since && is following the true. Therefore, the shell exits.
About E:
Same reasoning as with C: true is the command following the last && or ||. Therefore, the shell doesn't exit.
About F:
Similar to D: This is a && or || list. We will have to look at the command following the last && or ||. Again the command is false which does have a non-zero status. But this time it doesn't matter, because the first command is false as well. Since it's a && list, the second false won't be evaluated. Therefore, the shell doesn't exit.
About G:
Same reasoning as with C or E: true is the command following the last && or ||. Therefore, the shell doesn't exit.
About H:
This is a && or || list. We will have to look at the command following the last && or ||. This command is false which does have a non-zero status, but it won't be evaluated since || is preceded by true. Therefore, the shell doesn't exit.
About I:
Same reasoning as with C, E or G: true is the command following the last && or ||. Therefore, the shell doesn't exit.
About J:
This is a && or || list. We will have to look at the command following the last && or ||. This command is false which does have a non-zero status. Since || is preceded by false the second false will be evaluated. Therefore, the shell does exit.
You should be able to apply these test cases to your case: true && false && true. Since the command following the last && or || is true which doesn't have a non-zero status, it doesn't matter what precedes && or ||, the shell won't exit either way.
I generally have -e set in my Bash scripts, but occasionally I would like to run a command and get the return value.
Without doing the set +e; some-command; res=$?; set -e dance, how can I do that?
From the bash manual:
The shell does not exit if the command that fails is [...] part of any command executed in a && or || list [...].
So, just do:
#!/bin/bash
set -eu
foo() {
# exit code will be 0, 1, or 2
return $(( RANDOM % 3 ))
}
ret=0
foo || ret=$?
echo "foo() exited with: $ret"
Example runs:
$ ./foo.sh
foo() exited with: 1
$ ./foo.sh
foo() exited with: 0
$ ./foo.sh
foo() exited with: 2
This is the canonical way of doing it.
as an alternative
ans=0
some-command || ans=$?
Maybe try running the commands in question in a subshell, like this?
res=$(some-command > /dev/null; echo $?)
Given behavior of shell described at this question it's possible to use following construct:
#!/bin/sh
set -e
{ custom_command; rc=$?; } || :
echo $rc
Another option is to use simple if. It is a bit longer, but fully supported by bash, i.e. that the command can return non-zero value, but the script doesn't exit even with set -e. See it in this simple script:
#! /bin/bash -eu
f () {
return 2
}
if f;then
echo Command succeeded
else
echo Command failed, returned: $?
fi
echo Script still continues.
When we run it, we can see that script still continues after non-zero return code:
$ ./test.sh
Command failed, returned: 2
Script still continues.
Use a wrapper function to execute your commands:
function __e {
set +e
"$#"
__r=$?
set -e
}
__e yourcommand arg1 arg2
And use $__r instead of $?:
if [[ __r -eq 0 ]]; then
echo "success"
else
echo "failed"
fi
Another method to call commands in a pipe, only that you have to quote the pipe. This does a safe eval.
function __p {
set +e
local __A=() __I
for (( __I = 1; __I <= $#; ++__I )); do
if [[ "${!__I}" == '|' ]]; then
__A+=('|')
else
__A+=("\"\$$__I\"")
fi
done
eval "${__A[#]}"
__r=$?
set -e
}
Example:
__p echo abc '|' grep abc
And I actually prefer this syntax:
__p echo abc :: grep abc
Which I could do with
...
if [[ ${!__I} == '::' ]]; then
...