Given the following contrived code:
#!/usr/bin/env bash
set -Eeuo pipefail
shopt -s inherit_errexit
echo 'before'
mapfile -t tuples < <(exit 1)
# ^ what option do I need to enable so the error exit code from this is not ignored
echo 'after'
Which produces:
before
after
Is there a set or shopt option that can be turned on such that <(exit 1) will cause the caller to inherit the failure, and thus preventing after from being executed? Such as what inherit_errexit and pipefail do in other contexts.
In bash 4.4 or later, process substitutions will set $!, which means you can wait on that process to get its exit status.
#!/usr/bin/env bash
set -Eeuo pipefail
shopt -s inherit_errexit
echo 'before'
mapfile -t tuples < <(exit 1)
wait $!
echo 'after'
mapfile itself (in general) won't have a non-zero status, because it's perfectly happy read what, if anything, the process substitution produces.
You can assign a variable with the output of the command. The variable assignment propagates errors from the command substitution.
t=$(exit 1)
echo 'after'
mapfile -t tuples <<<"$t"
If you have Bash 4.2 or later, since you are already setting errexit and pipefail, you can avoid the problem by using:
...
shopt -s lastpipe
exit 1 | mapfile -t tuples
shopt -s lastpipe causes the last command in a pipeline to be run in the current shell. See how does shopt -s lastpipe affect bash script behavior?. In this case it means that a tuples value read by mapfile can be accessed later in the code.
Related
Consider the following Bash script:
#!/usr/bin/env bash
set -o errexit pipefail
shopt -s inherit_errexit
( echo hello ; exit 1 ) | cat
echo world
Running with version 5.0.17, the output is as follows:
hello
world
However, the subshell responsible for printing hello would have failed, with a nonzero exit status. Part of a pipe, with enabling the optionpipefail, the entire pipe should similarly fail, with the same status (the subsequent item in the pipe of course returning a zero status by itself). Thus, the pipe should evaluate to a nonzero status, which due to errexit (if not inherit_errexit as well) would prompt the immediate termination of the script, without reaching the final statement printing world.
As seen, the prediction is not accurate that the final print statement is not reached.
Why?
You're only setting the errexit setting, not pipefail. You can't put multiple options after -o, you need to repeat the -o option. Everything else is being put into the positional arguments.
So change
set -o errexit pipefail
to
set -o errexit -o pipefail
Problem
The execution of a Bash script fails with the following error message when the 'extglob' option is set inside a subshell:
/tmp/foo.sh: line 7: syntax error near unexpected token `('
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
)
It fails in the same manner inside a function:
#!/usr/bin/env bash
set -euo pipefail
list_no_vendor () {
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
}
list_no_vendor
Investigation
In both cases, the script executes successfully when the option is set globally, outside of the subshell or function.
Surprisingly, when set locally, the 'extglob' option appears to be effectively enabled in both the subshell and function:
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
echo 'In the subshell:' "$(shopt extglob)"
)
list_no_vendor () {
shopt -s extglob
echo 'In the function:' "$(shopt extglob)"
}
echo 'In the main shell:' "$(shopt extglob)"
list_no_vendor
Output:
In the subshell: extglob on
In the main shell: extglob off
In the function: extglob on
This makes the syntax error extremely puzzling to me.
Workaround
Passing a heredoc to the bash command works.
#!/usr/bin/env bash
set -euo pipefail
bash <<'EOF'
shopt -s extglob
echo 'In the child:' "$(shopt extglob)"
EOF
echo 'In the parent:' "$(shopt extglob)"
Output:
In the child: extglob on
In the parent: extglob off
However I would be curious to understand the gist of the problem here.
extglob is a flag used by the parser. Functions, compound commands, &c. are parsed in entirety ahead of execution. Thus, extglob must be set before that content is parsed; setting it at execution time but after parse time does not have any effect for previously-parsed content.
This is also why you can't run shopt -s extglob; ls !(*.txt) as a one-liner (when extglob is previously unset), but must have a newline between the two commands.
Not as an example of acceptable practice, but as an example demonstrating the behavior, consider the following:
#!/usr/bin/env bash
(
shopt -s extglob
# Parse of eval'd code is deferred, so this succeeds
eval '
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
'
)
No such error takes place here, because parsing of the content passed to eval happens only after the shopt -s extglob was executed, rather than when the block of code to be run in a subshell is parsed.
Consider the following bash code:
#!/usr/bin/env bash
set -euo pipefail
Test() {
grep 'XXX' data
echo 'test message'
}
data is an empty file existing in the same directory as the above bash script.
If Test function is invoked with the following code, it produces nothing to stdout.
Test
The reason is that grep command return exit status 1 since data does not contains XXX. So set -euo pipefail causes the bash script to exit immediately.
But if it is invoked with:
output=$(eval 'Test')
echo "output: $output"
It produes the following message to stdout:
output: test message
It seems that set -euo pipefail does not have an effect on eval. So grep command does not cause the bash script to exit.
As long as bash is invoked as bash, you're mostly correct, except that it's the $(...), not the eval, that suppresses -e. Beware though if it may be called as sh (or bash --posix): this is item #40 on the list of changes between POSIX and non-POSIX mode:
Enabling POSIX mode has the effect of setting the
'inherit_errexit' option, so subshells spawned to execute command
substitutions inherit the value of the '-e' option from the parent
shell. When the 'inherit_errexit' option is not enabled, Bash
clears the '-e' option in such subshells.
I am using this:
$ uname -a
CYGWIN_NT-6.1 bassoon 1.7.15(0.260/5/3) 2012-05-09 10:25 i686 Cygwin
$ bash --version
GNU bash, version 4.1.10(4)-release (i686-pc-cygwin)
$ cat myexpr.sh
#!/bin/sh
echo "In myexpr, Before expr"
ac_optarg=`expr x--with-gnu-as : 'x[^=]*=\(.*\)'`
echo "ac_optarg=$ac_optarg"
echo "In myexpr, After expr"
$ cat myexpr2.sh
#!/bin/sh
set -e
echo "In myexpr, Before expr"
ac_optarg=`expr x--with-gnu-as : 'x[^=]*=\(.*\)'`
echo "ac_optarg=$ac_optarg"
echo "In myexpr, After expr"
The only difference between the two scripts is that myexpr2.sh uses "set -e"
$ echo $$
2880
$ ./myexpr.sh
In myexpr, Before expr
ac_optarg=
In myexpr, After expr
$ ./myexpr2.sh
In myexpr, Before expr
Expected behavior, so far.
If I do this in the parent shell (PID 2880, above):
$ set -e
$ ./myexpr.sh
The parent shell exits! That is pID 2880 above where I did the "set -e"
This is not the behavior on Linux or cygwin 1.5.12. Is this a bug in cygwin or BASH on cygwin?
This is not a bug, it's a feature of the Bash environment. This happens when you don't have the Bash shell environment variable execfail set, and/or the Shell environment variable errexit.
execfail - (is a BASHOPTS)
If set, a non-interactive shell will not exit if it cannot execute
the file specified as an argument to the exec builtin command.
An interactive shell does not exit if exec fails.
errexit - (is a SHELLOPTS)
Exit immediately if a pipeline (see Pipelines), which may consist of a
single simple command (see Simple Commands), a subshell command enclosed
in parentheses (see Command Grouping), or one of the commands executed as
part of a command list enclosed by braces (see Command Grouping) returns a
non-zero status. The shell does not exit if the command that fails is part
of the command list immediately following a while or until keyword, part
of the test in an if statement, part of any command executed in a && or ||
list except the command following the final && or ||, any command in a
pipeline but the last, or if the command’s return status is being inverted
with !. A trap on ERR, if set, is executed before the shell exits.
This option applies to the shell environment and each subshell environment
separately (see Command Execution Environment), and may cause subshells to
exit before executing all the commands in the subshell.
Different Linux versions have different defaults for these.
You can check which are enabled with:
echo "SHELLOPTS=$SHELLOPTS"
echo "BASHOPTS=$BASHOPTS"
and you can see all of them using:
set -o && echo -e "\n" && shopt -p
So, you need to enable yours with:
shopt -s execfail
If that doesn't work, you may also have to unset (off) the errexit of $SHELLOPTS with:
set -o errexit
For further info, see: The GNU Bash Manual!
PS. "set" is using reverse logic so if you wanna use the 'e' flag you have to use a "+": set +e
This question already has answers here:
Aborting a shell script if any command returns a non-zero value
(10 answers)
Closed 3 years ago.
I've been writing some shell script and I would find it useful if there was the ability to halt the execution of said shell script if any of the commands failed. See below for an example:
#!/bin/bash
cd some_dir
./configure --some-flags
make
make install
So in this case, if the script can't change to the indicated directory, then it would certainly not want to do a ./configure afterwards if it fails.
Now I'm well aware that I could have an if check for each command (which I think is a hopeless solution), but is there a global setting to make the script exit if one of the commands fails?
Use the set -e builtin:
#!/bin/bash
set -e
# Any subsequent(*) commands which fail will cause the shell script to exit immediately
Alternatively, you can pass -e on the command line:
bash -e my_script.sh
You can also disable this behavior with set +e.
You may also want to employ all or some of the the -e -u -x and -o pipefail options like so:
set -euxo pipefail
-e exits on error, -u errors on undefined variables, -x prints commands before execution, and -o (for option) pipefail exits on command pipe failures. Some gotchas and workarounds are documented well here.
(*) Note:
The shell does not exit if the command that fails is part of the
command list immediately following a while or until keyword,
part of the test following the if or elif reserved words, part
of any command executed in a && or || list except the command
following the final && or ||, any command in a pipeline but
the last, or if the command's return value is being inverted with
!
(from man bash)
To exit the script as soon as one of the commands failed, add this at the beginning:
set -e
This causes the script to exit immediately when some command that is not part of some test (like in a if [ ... ] condition or a && construct) exits with a non-zero exit code.
Use it in conjunction with pipefail.
set -e
set -o pipefail
-e (errexit): Abort the script at the first error, when a command exits with non-zero status (except in until or while loops, if-tests, and list constructs)
-o pipefail: Causes a pipeline to return the exit status of the last command in the pipe that returned a non-zero return value.
Chapter 33. Options
Here is how to do it:
#!/bin/sh
abort()
{
echo >&2 '
***************
*** ABORTED ***
***************
'
echo "An error occurred. Exiting..." >&2
exit 1
}
trap 'abort' 0
set -e
# Add your script below....
# If an error occurs, the abort() function will be called.
#----------------------------------------------------------
# ===> Your script goes here
# Done!
trap : 0
echo >&2 '
************
*** DONE ***
************
'
An alternative to the accepted answer that fits in the first line:
#!/bin/bash -e
cd some_dir
./configure --some-flags
make
make install
One idiom is:
cd some_dir && ./configure --some-flags && make && make install
I realize that can get long, but for larger scripts you could break it into logical functions.
I think that what you are looking for is the trap command:
trap command signal [signal ...]
For more information, see this page.
Another option is to use the set -e command at the top of your script - it will make the script exit if any program / command returns a non true value.
One point missed in the existing answers is show how to inherit the error traps. The bash shell provides one such option for that using set
-E
If set, any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment. The ERR trap is normally not inherited in such cases.
Adam Rosenfield's answer recommendation to use set -e is right in certain cases but it has its own potential pitfalls. See GreyCat's BashFAQ - 105 - Why doesn't set -e (or set -o errexit, or trap ERR) do what I expected?
According to the manual, set -e exits
if a simple commandexits with a non-zero status. The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test in a if statement, part of an && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command's return value is being inverted via !".
which means, set -e does not work under the following simple cases (detailed explanations can be found on the wiki)
Using the arithmetic operator let or $((..)) ( bash 4.1 onwards) to increment a variable value as
#!/usr/bin/env bash
set -e
i=0
let i++ # or ((i++)) on bash 4.1 or later
echo "i is $i"
If the offending command is not part of the last command executed via && or ||. For e.g. the below trap wouldn't fire when its expected to
#!/usr/bin/env bash
set -e
test -d nosuchdir && echo no dir
echo survived
When used incorrectly in an if statement as, the exit code of the if statement is the exit code of the last executed command. In the example below the last executed command was echo which wouldn't fire the trap, even though the test -d failed
#!/usr/bin/env bash
set -e
f() { if test -d nosuchdir; then echo no dir; fi; }
f
echo survived
When used with command-substitution, they are ignored, unless inherit_errexit is set with bash 4.4
#!/usr/bin/env bash
set -e
foo=$(expr 1-1; true)
echo survived
when you use commands that look like assignments but aren't, such as export, declare, typeset or local. Here the function call to f will not exit as local has swept the error code that was set previously.
set -e
f() { local var=$(somecommand that fails); }
g() { local var; var=$(somecommand that fails); }
When used in a pipeline, and the offending command is not part of the last command. For e.g. the below command would still go through. One options is to enable pipefail by returning the exit code of the first failed process:
set -e
somecommand that fails | cat -
echo survived
The ideal recommendation is to not use set -e and implement an own version of error checking instead. More information on implementing custom error handling on one of my answers to Raise error in a Bash script