Context
While setting up a basic unit testing system, I ran into an odd issue.
My goal was to make sure all individual test scripts:
were run with set -e to detect errors, without needing to explicitly set this in each file;
knew right away about the functions to be tested (stored in another file) without needing to explicitly source those in each test file.
Observations
Let this be a dummy test file called to-be-sourced.sh. We want to be able to know if a command in it fails:
# Failing command!
false
# Last command is OK:
true
And here is a dummy test runner, which must run the test file:
#! /usr/bin/env bash
if (
set -e
. to-be-sourced.sh
)
then
echo 'Via set: =0'
else
echo 'Via set: ≠0'
fi
This yields Via set: =0, meaning that the runner is happy. But it should not!
My hypothesis was:
set -e is not propagated within . sourcing, and as explained in the help for . and source, the exit status is the one of the last command.
But then I came up with a workaround that works, but also relies on .:
if bash -ec '. "$0"' to-be-sourced.sh
then
echo 'Via bash: =0'
else
echo 'Via bash: ≠0'
fi
This yields ≠0 whenever a command in the test file fails, regardless of whether that command was the last one of the test file. As a bonus, I can toss any number of . a/library/file.sh within the -c command, so each test file can use all of my functions out of the box. I should therefore be happy, but:
Why does this work, considering that the -c command also relies on . to load the test file (and I thought bash’s -e was equivalent to set’s -e)?
I also thought about using bash’s --init-file, but it appeared to be skipped when a script is passed as a parameter. And anyway my question is not so much about what I was trying to achieve, but rather about the observed difference of behavior.
Edit
Sounds like if is tempering with the way set -e is handled.
This halts execution, indicating failure:
. to-be-sourced.sh
… while this goes into the then (not the else), indicating success:
if . to-be-sourced.sh
then
echo =0
else
echo ≠0
fi
(This may not be precisely correct, but I think it captures what happens.)
In your first example, set -e sets the option in a command that is lexically in the scope of an if statement, and so even though it is set, it is ignored. (You can confirm it is set by running echo $- inside to-be-sourced.sh. Note, too, that . itself has a 0 exit status, which you can confirm by replacing true with an echo statement; it's not that it fails but the failure is ignored.)
In your second example, -e sets the errexit option in a new process, which knows nothing about the if statement and therefore it is not ignored.
Related
For some reason after running the main script:
sudo bash main.sh
-> execution stops at the first diff redirected to file.
However, when I comment out the function name and parentheses and call the patching.sh directly - it works.
What is wrong with my script that when calling it in a form of a function from another file - it stops, but when called directly it works?
main.sh:
set -e
source $(dirname $0)/Scripts/patching.sh
# Overwrite files
update_files
patching.sh:
#!/bin/bash
function update_files() {
declare -r SW_DIR='Source/packages/'
CMP_FILE='file1.c'
diff -u ./$SW_DIR/examples/$CMP_FILE ./Source/$CMP_FILE > file.diff
cp -v ./Source/$CMP_FILE ./$SW_DIR/examples/$CMP_FILE
}
During my debugging - I added the -x option to set. This is what I see now:
+ declare -r SW_DIR=Source/packages
+ CMP_FILE=file1.c
+ diff -u ./Source/packages/examples/file1.c ./Source/file1.c
And that's the last line. If I omit the redirection operator - the diff is simply shown in the console and that's it. It does not proceed further, with no error message.
See What does set -e mean in a bash script? and BashFAQ/105 (Why doesn't set -e (or set -o errexit, or trap ERR) do what I expected?). Execution stops after the diff when set -e is in effect because diff exits with non-zero status when the files that it is comparing are different. This kind of behaviour is one of the downsides of using set -e. Follow the links for more, and useful, information.
I have a bash script that has set -ex, which means the running script will exit once any command in it hits an error.
My use case is that there's a subcommand in this script for which I want to catch its error, instead of making the running script shutdown.
E.g., here's myscript.sh
#!/bin/bash
set -ex
# sudo code here
error = $( some command )
if [[ -n $error ]] ; then
#do something
fi
Any idea how I can achieve this?
You can override the output of a single command
this_will_fail || true
Or for an entire block of code
set +e
this_will_fail
set -e
Beware, however, that if you decide you don't want to use set -e in the script anymore this won't work.
If you want to handle a particular command's error status yourself, you can use as the condition in an if statement:
if ! some command; then
echo 'An error occurred!' >&2
# handle error here
fi
Since the command is part of a condition, it won't exit on error. Note that other than the ! (which negates it, so the then clause will run if the command fails rather than it succeeds), you just include the command directly in the if statement (no brackets, parentheses, etc).
BTW, in your pseudocode example, you seem to be treating it as an error if the command produces any output; usually that's not what you want, and I'm assuming you actually want to test the exit status to detect errors.
If I put set -e in a Bash script, the script will exit on future errors. I'm confused about how this works with functions. Consider the following, which will only print one to standard out:
set -e # Exit on error
fun(){
echo one
non_existing_command
echo two
}
fun
Clearly, the non_existing_command is an error and so the script exits before the second echo. Usually one can use the or operator || to run another command if and only if the first command fails. That is, I would suspect the following to print out both one and three, but not two:
set -e # Exit on error
fun(){
echo one
non_existing_command
echo two
}
fun || echo three
What I get however is one and two. That is, the || operator prevents the exit (as it should) but it chooses to continue with the function body and disregard the right-hand command.
Any explanation?
It appears to be documented in the set builtin command
If a compound command or shell function executes in a context where -e is being ignored [such as on the left-hand of a ||], none of the commands executed within the compound command or function body will be affected by the -e setting, even if -e is set and a command returns a failure status.
Emphasis and comment are mine.
Also, if you try to set -e within the function, don't bother: the next sentence:
If a compound command or shell function sets -e while executing in a context where -e is ignored, that setting will not have any effect until the compound command or the command containing the function call completes.
I have this bash script (actually a part of https://github.com/ddollar/heroku-buildpack-multi/blob/master/bin/compile with echo I've added myself):
echo "[DEBUG] chmod done"
framework=$($dir/bin/detect $1)
echo "[DEBUG] $framework done"
And I see in the log:
[DEBUG] chmod done
Staging failed: Buildpack compilation step failed
And I do not see the second echo in the logs at all.
I do not know much bash unfortunately. Could anybody explain to me in what case the first echo performs and second do not? I always thought that both echo should always work no matter whether the second line succeeds or not.
It's not visible in your question, but clicking on your link, it says in the third line
set -e
This means to stop processing the script immediately whenever an error occurs. Comment that line, and the script should run through and also print the second echo statement.
Note that I didn't inspect what the script actually does and I cannot tell you if commenting out set -e is actually good advice or not.
From man set:
−e: When this option is on, when any command fails (for any of the reasons listed
in Section 2.8.1, Consequences of Shell Errors or by returning an exit status
greater than zero), the shell immediately shall exit with the following excep‐
tions:
1. The failure of any individual command in a multi-command pipeline shall not
cause the shell to exit. Only the failure of the pipeline itself shall be
considered.
2. The −e setting shall be ignored when executing the compound list following
the while, until, if, or elif reserved word, a pipeline beginning with the
! reserved word, or any command of an AND-OR list other than the last.
3. If the exit status of a compound command other than a subshell command was
the result of a failure while −e was being ignored, then −e shall not apply
to this command.
This requirement applies to the shell environment and each subshell environment
separately. For example, in:
set -e; (false; echo one) | cat; echo two
the false command causes the subshell to exit without executing echo one; how‐
ever, echo two is executed because the exit status of the pipeline (false; echo
one) | cat is zero.
I have a simple bash script which calls a php script every 10 minutes thats performs some maintenance. Every once in a while this php script terminates while it's running and when this happens the bash script exits.
I'd like to make it so the bash script keeps on looping even if the php script falters. Can anyone point me in the right direction? I've been searching for a while but I can't seem to find the answer, maybe I'm not using the right search terms.
#!/bin/sh
set -e
while :
do
/usr/bin/php /path/to/maintenance/script.php
sleep 600
done
Rjz's comment is correct, you should use cron. To do that, run crontab -e and add this line:
*/10 * * * * /usr/bin/php /path/to/maintenance/script.php
If it's set up properly, cron will email you any output (including error messages).
The:
set -e
line sets the shell's "exit on error" flag, which tells it that if a program it runs exits with a non-zero status, the shell should also exit:
set -e
false
echo if this prints, your shell is not honoring "set -e"
There are exceptions for programs whose status is being tested, of course, so that:
set -e
if prog; then
echo program succeeded
else
echo program failed
fi
echo this will still print
will work correctly (one or or the other echo will occur, and then the last one will as well).
Back in the Dim Time, when /bin/sh was non-POSIX and was written in Bournegol, there was a bug in some versions of sh that broke || expressions:
set -e
false || true
echo if this prints, your shell is OK
(The logic bug applied to && expressions internally as well, but was harmless there, since false && anything is itself false which means the whole expression fails anyway!) Ever since then, I've been wary of "-e".