I would like to have set -o pipefail set "always" (.kshrc) but consider the following trivial example:
#!/bin/ksh
function fun {
return 99
}
if [[ $1 == 'pipe' ]] ; then
set -o pipefail
fi
if fun | tee /tmp/bork.txt ; then
print "fun returned 0"
else
print "fun nonzero"
fi
This results in:
/home/khb>./b pipe
fun nonzero GOOD what we want
/home/khb>./b
fun returned 0 What we expect without pipefail!
/home/khb>set -o pipefail
/home/khb>./b
fun returned 0 BAD: expected the set to impact inferior shells
No doubt this should be obvious, but other than creating an environment variable and having every script reference it ... or sourcing a common set of definitions ... what other options are there to arrange for this option to be "found" in each script?
Thanks in advance.
A slightly awkward solution would be to have set -o pipefail in your ~/.profile file and then write scripts that always invoke ksh as a login shell, i.e. by using #!/bin/ksh -l as the hash-bang line.
A less (?) awkward solution would be to put set -o pipe fail in the file pointed to by $ENV and then invoke ksh with -E (instead of -l above). However, shells parsing $ENV are usually interactive shells...
Related
All of my scripts have errexit turned on; that is, I run set -o errexit. However, sometimes I want to run commands like grep, but want to continue execution of my script even if the command fails.
How do I do this? That is, how can I get the exit code of a command into a variable without killing my whole script?
I could turn errexit off, but I'd prefer not to.
Your errexit will only cause the script to terminate if the command that fails is "untested". Per man sh on FreeBSD:
Exit immediately if any untested command fails in non-interactive
mode. The exit status of a command is considered to be explic-
itly tested if the command is part of the list used to control an
if, elif, while, or until; if the command is the left hand oper-
and of an ``&&'' or ``||'' operator; or if the command is a pipe-
line preceded by the ! keyword.
So .. if you were thinking of using a construct like this:
grep -q something /path/to/somefile
retval=$?
if [ $retval -eq 0 ]; then
do_something # found
else
do_something_else # not found
fi
you should instead use a construct like this:
if grep -q something /path/to/somefile; then
do_something # found
else
do_something_else # not found
fi
The existence of the if keyword makes the grep command tested, thus unaffected by errexit. And this way takes less typing.
Of course, if you REALLY need the exit value in a variable, there's nothing stopping you from using $?:
if grep -q something /path/to/somefile; then
do_something # found
else
unnecessary=$?
do_something $unnecessary # not found
fi
Here's a way to achieve this: you can "turn off" set -o errexit for some lines of code, and then turn it on again when you decide:
set +e #disables set -o errexit
grep -i something file.txt
rc="$?" #capturing the return code for last command
set -e #reenables set -o errexit
Another option would be the following:
grep -i something file.txt || rc="$?"
That would allow you to capture the return code on the variable rc, without interrupting your script. You could even extend this last option to capture and process the return code on the same line without risking to trigger an exit:
grep -i something file.txt || rc="$?" && echo rc="$?" > somefile.txt && command || :
The last bit ||: will guarantee that the line above always returns a return code = 0 (true).
The most compact and least repetitive form I can think of:
<command> && rc=$? || rc=$?
A form without variable name repetition -- rc=$(<command> && echo $? || echo $?) -- has an expression rvalue but would also capture the stdout of <command>1. So, it's only safe if you "know" that <command> has no normal output.
Using the a && b || c construct is safe here because rc=$? and $(echo $?) can never fail.
1Sure, you can work around this by fiddling with file descriptors, but that'd make the construct long-winded and inconvenient as a standard
I'm new to Bash and I had a hard time to figure out why when I was using set -o errexit and a command fail, the script was not exiting.
It seems because I declared a local variable!
Please tell me how I can use local variable and set -o errexit at the same time.
Example:
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail
function test {
local output=$(ls --badoption)
echo "error code is $?"
echo "output=$output"
}
test
Result:
./test.sh
ls: illegal option -- -
usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...]
error code is 0
output=
But:
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail
function test {
output=$(ls --badoption)
echo "error code is $?"
echo "output=$output"
}
test
Result:
ls: illegal option -- -
usage: ls [-ABCFGHLOPRSTUWabcdefghiklmnopqrstuwx1] [file ...]
It's because set -o errexit (or running bash with -e) only affects simple commands (as defined in the bash man page). cmd in local output=$(cmd) isn't considered a simple command, so -e doesn't have any affect (and $? doesn't have any relation to how cmd exited). A simple workaround is to split the line by replacing:
local output=$(ls --badoption)
with:
local output
output=$(ls --badoption)
In my testing that does everything you want, it will exit immediately, and if you replace the ls --badoption with just ls, it works, and setting output before calling test then echoing it afterwords shows output really is local to test.
The documentation of local says:
The return status is zero unless local is used outside a function, an invalid name is supplied, or name is a readonly variable.
So the return status of the local command is not affected by whether the command executed in the command substitution succeeded.
As blm indicated in his answer, you can get the effect you want by separating the declaration and assignment of the local variable. The return status of an ordinary assignment that uses command substitution is the return status of the subshell that executes the command.
I'm writing a shell script that loops over some values and run a long command line for each value. I'd like to print out these commands along the way, just like make does when running a makefile. I know I could just "echo" all commands before running them, but it feels inelegant. So I'm looking at set -x and similar mechanisms instead :
#!/bin/sh
for value in a long list of values
do
set -v
touch $value # imagine a complicated invocation here
set +v
done
My problem is: at each iteration, not only is the interresting line printed out, but also the set +x line as well. Is it somehow possible to prevent that ? If not, what workaround do you recommend ?
PS: the MWE above uses sh, but I also have bash and zsh installed in case that helps.
Sandbox it in a subshell:
(set -x; do_thing_you_want_traced)
Of course, changes to variables or the environment made in that subshell will be lost.
If you REALLY care about this, you could also use a DEBUG trap (using set -T to cause it to be inherited by functions) to implement your own set -x equivalent.
For instance, if using bash:
trap_fn() {
[[ $DEBUG && $BASH_COMMAND != "unset DEBUG" ]] && \
printf "[%s:%s] %s\n" "$BASH_SOURCE" "$LINENO" "$BASH_COMMAND"
return 0 # do not block execution in extdebug mode
}
trap trap_fn DEBUG
DEBUG=1
# ...do something you want traced...
unset DEBUG
That said, emitting BASH_COMMAND (as a DEBUG trap can do) is not fully equivalent of set -x; for instance, it does not show post-expansion values.
You want to try using a single-line xtrace:
function xtrace() {
# Print the line as if xtrace was turned on, using perl to filter out
# the extra colon character and the following "set +x" line.
(
set -x
# Colon is a no-op in bash, so nothing will execute.
: "$#"
set +x
) 2>&1 | perl -ne 's/^[+] :/+/ and print' 1>&2
# Execute the original line unmolested
"$#"
}
The original command executes in the same shell under an identity transformation. Just prior to running, you get a non-recursive xtrace of the arguments. This allows you to xtrace the commands you care about without spamming stederr with duplicate copies of every "echo" command.
# Example
for value in $long_list; do
computed_value=$(echo "$value" | sed 's/.../...')
xtrace some_command -x -y -z $value $computed_value ...
done
Next command disables 'xtrace' option:
$ set +o xtrace
I thought of
set -x >/dev/null 2>1; echo 1; echo 2; set +x >/dev/null 2>&1
but got
+ echo 1
1
+ echo 2
2
+ 1> /dev/null 2>& 1
I'm surprised by these results. .... But
set -x ; echo 1; echo 2; set +x
+ echo 1
1
+ echo 2
2
looks to meet your requirement.
I saw similar results when I put each statement on its only line (excepting the set +x)
IHTH.
The usual example for using getopt in bash is as follow
args=`getopt abo: $*`
errcode=$?
set -- $args
What does that last line achieve?
This explains it very well. Essentially, it is to break a single argument with multiple flags into multiple arguments each with single flag:
Whether you call your script as
script -ab
or as
script -a -b
after the set -- $args, $1 will be -a and $2 will be -b. It makes processing easier.
BTW, getopts is much better
set updates the positional parameters of the script.
#! /bin/bash
echo "$*"
set -- $1 baz
echo "$*"
If this script is invoked with /path/to/script foo bar, the output is:
foo bar
foo baz
Many of my test scripts begin:
set -e
test -n "$V" && set -x
Rather than putting those lines ( or sourcing a common script ) in each script, I'd
like to get that functionality through the environment. Is there a portable way to use environment settings to cause sh to behave as if "set -e" or "set -x" has been called? Is there a non-portable (ie shell-specific) way to do the same?
(I've tagged this question as automake because that's the framework I'm in at the moment and would like to be able to put something in TESTS_ENVIRONMENT that will allow me to omit those lines from each script, but clearly the question is not automake specific.)
Just add this to your scripts:
eval "${ENABLE_DEBUG}"
Now you can set the env variable ENABLE_DEBUG to set -e ; test -n "$V" && set -x to enable debugging or you can leave it unset.
Note that this fails if you have the option "fail for undefined variables" active (set -u or set -o nounset). If that is the case, you either need to check that the variable is set or use bash with:
eval "${ENABLE_DEBUG}"
that sets the variable to :, the "do nothing" command.
Answering the automake part.
If you have
TESTS = foo.test bar.test baz.test
the Makefile generated will have a test target roughly like
test:
...
$(TEST_ENVIRONMENT) $(srcdir)/foo.test
$(TEST_ENVIRONMENT) $(srcdir)/bar.test
$(TEST_ENVIRONMENT) $(srcdir)/baz.test
...
You can actually set TEST_ENVIRONMENT to a command that will start your shell scripts with sh -xe or sh -e.
If all tests are shell scripts, this can be a simple as setting
TEST_ENVIRONMENT = $(SHELL) -e $${V+-x}
if not all tests are shell scripts, you can have
TEST_ENVIRONMENT = $(srcdir)/run
and write a script run such as:
#!/bin/sh
case $1 in
*.py)
exec python "$#";;
*.test)
exec sh -x ${V+-x} "$#";;
*)
echo "Unknown extension" >&2
exit 2;;
esac