Many of my test scripts begin:
set -e
test -n "$V" && set -x
Rather than putting those lines ( or sourcing a common script ) in each script, I'd
like to get that functionality through the environment. Is there a portable way to use environment settings to cause sh to behave as if "set -e" or "set -x" has been called? Is there a non-portable (ie shell-specific) way to do the same?
(I've tagged this question as automake because that's the framework I'm in at the moment and would like to be able to put something in TESTS_ENVIRONMENT that will allow me to omit those lines from each script, but clearly the question is not automake specific.)
Just add this to your scripts:
eval "${ENABLE_DEBUG}"
Now you can set the env variable ENABLE_DEBUG to set -e ; test -n "$V" && set -x to enable debugging or you can leave it unset.
Note that this fails if you have the option "fail for undefined variables" active (set -u or set -o nounset). If that is the case, you either need to check that the variable is set or use bash with:
eval "${ENABLE_DEBUG}"
that sets the variable to :, the "do nothing" command.
Answering the automake part.
If you have
TESTS = foo.test bar.test baz.test
the Makefile generated will have a test target roughly like
test:
...
$(TEST_ENVIRONMENT) $(srcdir)/foo.test
$(TEST_ENVIRONMENT) $(srcdir)/bar.test
$(TEST_ENVIRONMENT) $(srcdir)/baz.test
...
You can actually set TEST_ENVIRONMENT to a command that will start your shell scripts with sh -xe or sh -e.
If all tests are shell scripts, this can be a simple as setting
TEST_ENVIRONMENT = $(SHELL) -e $${V+-x}
if not all tests are shell scripts, you can have
TEST_ENVIRONMENT = $(srcdir)/run
and write a script run such as:
#!/bin/sh
case $1 in
*.py)
exec python "$#";;
*.test)
exec sh -x ${V+-x} "$#";;
*)
echo "Unknown extension" >&2
exit 2;;
esac
Related
I am trying to check in the script, that if that script is executed with -x flag which is for the debugging for shell scripts.
Is there any way to check that in script itself that -x is set.
I want to conditionally check that and do something if that is set.
Use:
if [[ $- == *x* ]]; then
echo "debug"
else
echo "not debug"
fi
From Bash manual:
($-, a hyphen.) Expands to the current option flags as specified upon invocation, by the set builtin command, or those set by the shell itself (such as the -i option).
The portable way to do this (without bashisms like [[ ]]) would be
case $- in
(*x*) echo "under set -x"
esac
You can trap the DEBUG signal, like so:
trap "do_this_if_it_is_being_debugged" DEBUG
function do_this_if_it_is_being_debugged() {
...
}
Note this needs to be executed before the set -x is being executed
Look for xtrace in $SHELLOPTS.
For example:
if grep -q xtrace <<<"$SHELLOPTS"; then
DO_SOMETHING;
fi
I would like to have set -o pipefail set "always" (.kshrc) but consider the following trivial example:
#!/bin/ksh
function fun {
return 99
}
if [[ $1 == 'pipe' ]] ; then
set -o pipefail
fi
if fun | tee /tmp/bork.txt ; then
print "fun returned 0"
else
print "fun nonzero"
fi
This results in:
/home/khb>./b pipe
fun nonzero GOOD what we want
/home/khb>./b
fun returned 0 What we expect without pipefail!
/home/khb>set -o pipefail
/home/khb>./b
fun returned 0 BAD: expected the set to impact inferior shells
No doubt this should be obvious, but other than creating an environment variable and having every script reference it ... or sourcing a common set of definitions ... what other options are there to arrange for this option to be "found" in each script?
Thanks in advance.
A slightly awkward solution would be to have set -o pipefail in your ~/.profile file and then write scripts that always invoke ksh as a login shell, i.e. by using #!/bin/ksh -l as the hash-bang line.
A less (?) awkward solution would be to put set -o pipe fail in the file pointed to by $ENV and then invoke ksh with -E (instead of -l above). However, shells parsing $ENV are usually interactive shells...
I'm writing a shell script that loops over some values and run a long command line for each value. I'd like to print out these commands along the way, just like make does when running a makefile. I know I could just "echo" all commands before running them, but it feels inelegant. So I'm looking at set -x and similar mechanisms instead :
#!/bin/sh
for value in a long list of values
do
set -v
touch $value # imagine a complicated invocation here
set +v
done
My problem is: at each iteration, not only is the interresting line printed out, but also the set +x line as well. Is it somehow possible to prevent that ? If not, what workaround do you recommend ?
PS: the MWE above uses sh, but I also have bash and zsh installed in case that helps.
Sandbox it in a subshell:
(set -x; do_thing_you_want_traced)
Of course, changes to variables or the environment made in that subshell will be lost.
If you REALLY care about this, you could also use a DEBUG trap (using set -T to cause it to be inherited by functions) to implement your own set -x equivalent.
For instance, if using bash:
trap_fn() {
[[ $DEBUG && $BASH_COMMAND != "unset DEBUG" ]] && \
printf "[%s:%s] %s\n" "$BASH_SOURCE" "$LINENO" "$BASH_COMMAND"
return 0 # do not block execution in extdebug mode
}
trap trap_fn DEBUG
DEBUG=1
# ...do something you want traced...
unset DEBUG
That said, emitting BASH_COMMAND (as a DEBUG trap can do) is not fully equivalent of set -x; for instance, it does not show post-expansion values.
You want to try using a single-line xtrace:
function xtrace() {
# Print the line as if xtrace was turned on, using perl to filter out
# the extra colon character and the following "set +x" line.
(
set -x
# Colon is a no-op in bash, so nothing will execute.
: "$#"
set +x
) 2>&1 | perl -ne 's/^[+] :/+/ and print' 1>&2
# Execute the original line unmolested
"$#"
}
The original command executes in the same shell under an identity transformation. Just prior to running, you get a non-recursive xtrace of the arguments. This allows you to xtrace the commands you care about without spamming stederr with duplicate copies of every "echo" command.
# Example
for value in $long_list; do
computed_value=$(echo "$value" | sed 's/.../...')
xtrace some_command -x -y -z $value $computed_value ...
done
Next command disables 'xtrace' option:
$ set +o xtrace
I thought of
set -x >/dev/null 2>1; echo 1; echo 2; set +x >/dev/null 2>&1
but got
+ echo 1
1
+ echo 2
2
+ 1> /dev/null 2>& 1
I'm surprised by these results. .... But
set -x ; echo 1; echo 2; set +x
+ echo 1
1
+ echo 2
2
looks to meet your requirement.
I saw similar results when I put each statement on its only line (excepting the set +x)
IHTH.
I am using this:
$ uname -a
CYGWIN_NT-6.1 bassoon 1.7.15(0.260/5/3) 2012-05-09 10:25 i686 Cygwin
$ bash --version
GNU bash, version 4.1.10(4)-release (i686-pc-cygwin)
$ cat myexpr.sh
#!/bin/sh
echo "In myexpr, Before expr"
ac_optarg=`expr x--with-gnu-as : 'x[^=]*=\(.*\)'`
echo "ac_optarg=$ac_optarg"
echo "In myexpr, After expr"
$ cat myexpr2.sh
#!/bin/sh
set -e
echo "In myexpr, Before expr"
ac_optarg=`expr x--with-gnu-as : 'x[^=]*=\(.*\)'`
echo "ac_optarg=$ac_optarg"
echo "In myexpr, After expr"
The only difference between the two scripts is that myexpr2.sh uses "set -e"
$ echo $$
2880
$ ./myexpr.sh
In myexpr, Before expr
ac_optarg=
In myexpr, After expr
$ ./myexpr2.sh
In myexpr, Before expr
Expected behavior, so far.
If I do this in the parent shell (PID 2880, above):
$ set -e
$ ./myexpr.sh
The parent shell exits! That is pID 2880 above where I did the "set -e"
This is not the behavior on Linux or cygwin 1.5.12. Is this a bug in cygwin or BASH on cygwin?
This is not a bug, it's a feature of the Bash environment. This happens when you don't have the Bash shell environment variable execfail set, and/or the Shell environment variable errexit.
execfail - (is a BASHOPTS)
If set, a non-interactive shell will not exit if it cannot execute
the file specified as an argument to the exec builtin command.
An interactive shell does not exit if exec fails.
errexit - (is a SHELLOPTS)
Exit immediately if a pipeline (see Pipelines), which may consist of a
single simple command (see Simple Commands), a subshell command enclosed
in parentheses (see Command Grouping), or one of the commands executed as
part of a command list enclosed by braces (see Command Grouping) returns a
non-zero status. The shell does not exit if the command that fails is part
of the command list immediately following a while or until keyword, part
of the test in an if statement, part of any command executed in a && or ||
list except the command following the final && or ||, any command in a
pipeline but the last, or if the command’s return status is being inverted
with !. A trap on ERR, if set, is executed before the shell exits.
This option applies to the shell environment and each subshell environment
separately (see Command Execution Environment), and may cause subshells to
exit before executing all the commands in the subshell.
Different Linux versions have different defaults for these.
You can check which are enabled with:
echo "SHELLOPTS=$SHELLOPTS"
echo "BASHOPTS=$BASHOPTS"
and you can see all of them using:
set -o && echo -e "\n" && shopt -p
So, you need to enable yours with:
shopt -s execfail
If that doesn't work, you may also have to unset (off) the errexit of $SHELLOPTS with:
set -o errexit
For further info, see: The GNU Bash Manual!
PS. "set" is using reverse logic so if you wanna use the 'e' flag you have to use a "+": set +e
This question already has answers here:
Aborting a shell script if any command returns a non-zero value
(10 answers)
Closed 3 years ago.
I've been writing some shell script and I would find it useful if there was the ability to halt the execution of said shell script if any of the commands failed. See below for an example:
#!/bin/bash
cd some_dir
./configure --some-flags
make
make install
So in this case, if the script can't change to the indicated directory, then it would certainly not want to do a ./configure afterwards if it fails.
Now I'm well aware that I could have an if check for each command (which I think is a hopeless solution), but is there a global setting to make the script exit if one of the commands fails?
Use the set -e builtin:
#!/bin/bash
set -e
# Any subsequent(*) commands which fail will cause the shell script to exit immediately
Alternatively, you can pass -e on the command line:
bash -e my_script.sh
You can also disable this behavior with set +e.
You may also want to employ all or some of the the -e -u -x and -o pipefail options like so:
set -euxo pipefail
-e exits on error, -u errors on undefined variables, -x prints commands before execution, and -o (for option) pipefail exits on command pipe failures. Some gotchas and workarounds are documented well here.
(*) Note:
The shell does not exit if the command that fails is part of the
command list immediately following a while or until keyword,
part of the test following the if or elif reserved words, part
of any command executed in a && or || list except the command
following the final && or ||, any command in a pipeline but
the last, or if the command's return value is being inverted with
!
(from man bash)
To exit the script as soon as one of the commands failed, add this at the beginning:
set -e
This causes the script to exit immediately when some command that is not part of some test (like in a if [ ... ] condition or a && construct) exits with a non-zero exit code.
Use it in conjunction with pipefail.
set -e
set -o pipefail
-e (errexit): Abort the script at the first error, when a command exits with non-zero status (except in until or while loops, if-tests, and list constructs)
-o pipefail: Causes a pipeline to return the exit status of the last command in the pipe that returned a non-zero return value.
Chapter 33. Options
Here is how to do it:
#!/bin/sh
abort()
{
echo >&2 '
***************
*** ABORTED ***
***************
'
echo "An error occurred. Exiting..." >&2
exit 1
}
trap 'abort' 0
set -e
# Add your script below....
# If an error occurs, the abort() function will be called.
#----------------------------------------------------------
# ===> Your script goes here
# Done!
trap : 0
echo >&2 '
************
*** DONE ***
************
'
An alternative to the accepted answer that fits in the first line:
#!/bin/bash -e
cd some_dir
./configure --some-flags
make
make install
One idiom is:
cd some_dir && ./configure --some-flags && make && make install
I realize that can get long, but for larger scripts you could break it into logical functions.
I think that what you are looking for is the trap command:
trap command signal [signal ...]
For more information, see this page.
Another option is to use the set -e command at the top of your script - it will make the script exit if any program / command returns a non true value.
One point missed in the existing answers is show how to inherit the error traps. The bash shell provides one such option for that using set
-E
If set, any trap on ERR is inherited by shell functions, command substitutions, and commands executed in a subshell environment. The ERR trap is normally not inherited in such cases.
Adam Rosenfield's answer recommendation to use set -e is right in certain cases but it has its own potential pitfalls. See GreyCat's BashFAQ - 105 - Why doesn't set -e (or set -o errexit, or trap ERR) do what I expected?
According to the manual, set -e exits
if a simple commandexits with a non-zero status. The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test in a if statement, part of an && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command's return value is being inverted via !".
which means, set -e does not work under the following simple cases (detailed explanations can be found on the wiki)
Using the arithmetic operator let or $((..)) ( bash 4.1 onwards) to increment a variable value as
#!/usr/bin/env bash
set -e
i=0
let i++ # or ((i++)) on bash 4.1 or later
echo "i is $i"
If the offending command is not part of the last command executed via && or ||. For e.g. the below trap wouldn't fire when its expected to
#!/usr/bin/env bash
set -e
test -d nosuchdir && echo no dir
echo survived
When used incorrectly in an if statement as, the exit code of the if statement is the exit code of the last executed command. In the example below the last executed command was echo which wouldn't fire the trap, even though the test -d failed
#!/usr/bin/env bash
set -e
f() { if test -d nosuchdir; then echo no dir; fi; }
f
echo survived
When used with command-substitution, they are ignored, unless inherit_errexit is set with bash 4.4
#!/usr/bin/env bash
set -e
foo=$(expr 1-1; true)
echo survived
when you use commands that look like assignments but aren't, such as export, declare, typeset or local. Here the function call to f will not exit as local has swept the error code that was set previously.
set -e
f() { local var=$(somecommand that fails); }
g() { local var; var=$(somecommand that fails); }
When used in a pipeline, and the offending command is not part of the last command. For e.g. the below command would still go through. One options is to enable pipefail by returning the exit code of the first failed process:
set -e
somecommand that fails | cat -
echo survived
The ideal recommendation is to not use set -e and implement an own version of error checking instead. More information on implementing custom error handling on one of my answers to Raise error in a Bash script