How to use detect failure in command substitution - bash

As you know set -e is very useful to discover any failure when command is executed. However I found it to be delicate to use and I don't know how to use it in the following scenarios:
==============First example================================
set -e
function main() {
local X1="$(exp::notdefined)"
echo "Will reach here : ${LINENO}"
X2="$(exp::notdefined)"
echo "Will not reach here : ${LINENO}"
}
main
==============Second example================================
set -e
function exp::tmp() {
echo "Now '$-' is "$-" : ${LINENO}"
false
return 0
}
function main() {
X1="$(exp::tmp)"
echo "Will reach here : ${LINENO}. '\$X1' : ${X1}"
X2="$(set -e ; exp::tmp)"
echo "Will not reach here : ${LINENO}"
}
main
===============================
The first example shows that, if we use command substitution on a local variable, then it will not fail even if the command substituted is not found. I don't know how to detect these kinds of failures.
The second example shows that, if the bash options (-e) will not propagate unless we call set -e inside the command braces. Is there any better way to do this?

You request immediate exit on pipeline failure with -e, e.g.:
-e Exit immediately if a pipeline (which may consistof a single simple
command), a list, or a compound command (see SHELL GRAMMAR above),
exits with a non-zero status.
The reason the bad command substitution does not cause failure within the function is because local provides its own return status.
local [option] [name[=value] ...]
... The return status is 0 unless local is used outside a function, an
invalid name is supplied, or name is a readonly variable.
The assignment of a failed command substitution does not cause local to return non-zero. Therefore, no immediate-exit is triggered.
As far as checking for a failure of command substitution following local, since the output is assigned to the variable, and the return will not be non-zero in the event of command substitution failure, you would have to validate by checking the variable contents itself for expected values for success/failure.

From the bash manual:
Subshells spawned to execute command substitutions inherit the value of
the -e option from the parent shell. When not in posix mode, bash
clears the -e option in such subshells.
Example 2 behaves differently when bash runs with --posix; however for example 1 I can't find any documentation explaining why local would cause this.

Related

Bash exit status not caught despite set -e and/or trap being active

Can someone explain the bash/set -e behaviour on the code snippet below ?
#!/bin/bash
# Comment if you want to test the trap only
set -e -o pipefail -u -E
# Comment if you want to test the set -e only
trap "echo ERROR CAUGHT; exit 12" ERR
function reproduce() {
# Trigger arithmetic error on purpose
a=$((1109962735 - hello=12272 + 1))
}
reproduce
# The script is expected to trigger the trap and/or activate the set -e. In both cases it should stop and exit here on error.
status_code=$?
echo "STATUS ${status_code}"
if [[ "${status_code}" != "0" ]];then
echo "FIXME: why was status code not caught by set -e ?"
echo "Executing false to prove set -e is still active"
false
# If the following is not executed then it proves -e is still active
echo "set -e not active !!!!!"
exit 2
fi
Here is what is obtained when executing it:
$ bash reproduce.sh
reproduce.sh: line 8: 1109962735 - hello=12272 + 1: attempted assignment to non-variable (error token is "=12272 + 1")
STATUS 1
FIXME: why was status code it not caught by set -e ?
Executing false to prove set -e is still active
ERROR CAUGHT
Checking the exit code
$ echo $?
1
Bash version
bash --version
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)
Reproduced as well with
GNU bash, version 4.4.12(1)-release (x86_64-pc-linux-gnu)
Additional notes, related to comments (thanks to all suggestions anyway):
Commenting trap does not change the weird behaviour observed
Removing set -e to only keep the trap does trigger the trap either
Let's simplify it; minimum amount of code required for
reproducing the issue you're dealing with is
set -e
: $((+)) # arithmetic expansion error
echo survived
According to the standard, this should never print survived, it says a POSIX shell running non-interactively shall immediately exit upon an expansion error. But seemingly Bash doesn't think so. Although this difference isn't explicitly documented in the man page, in description of POSIX mode it says
Non-interactive shells exit if a syntax error in an arithmetic
expansion results in an invalid expression.
We can say this means in its default operating mode, a non-interactive Bash session doesn't exit upon such error, but as you realized, it doesn't trigger errexit mechanism, or ERR trap either. Instead, it assigns a non-zero value to $? an moves on.
To overcome this and get the expected behavior, you should define reproduce as follows
function reproduce() (
# Trigger arithmetic error on purpose
a=$((1109962735 - hello=12272 + 1))
)
This way the expansion error will take place in a subshell and cause it to exit with a non-zero status, thus, errexit and trap will be able to catch it.
Upon dash-o's request, here is a version that sets a for the current execution environment when the expression is valid
function reproduce() {
if : $((expression)); then
a=$((expression))
fi
}
On surface, it looks as if bash will not trigger the trap on various SYNTAX errors. Only when it a command (external, built-in) is executed (and return non-zero), the ERR trap will be trigered.
From the man page:
If a sigspec is ERR, the command arg is executed whenever a pipeline
(which may consist of a single simple command), a list, or a compound
command returns a non-zero exit status, subject to the following
conditions ...
The ERR trap only applies to PIPELINE. If bash identifies a syntax error, it aborts before executing the pipeline, therefore NO trap. Even though he documentation for '-e' specify the same condition (if a pipeline ... exit with non-zero status), the observed behavior is different.
If you try other expansions - e.g. command expansion- trap is triggered, as there is pipeline execution:
a=$(bad-commands)
a=$([)
If use try various syntax errors in arithmetic expansion, trap not triggered - there was no pipeline.
a=$((2+))
a=$((2 #))
Also, other bash Syntax error do not trigger the trap: (), [[ ]].
I could not find a solution that does not require extensive changes to the source script. May be file a bug/feature request with the bash team ?

Bash control flow using || on function, with set -e

If I put set -e in a Bash script, the script will exit on future errors. I'm confused about how this works with functions. Consider the following, which will only print one to standard out:
set -e # Exit on error
fun(){
echo one
non_existing_command
echo two
}
fun
Clearly, the non_existing_command is an error and so the script exits before the second echo. Usually one can use the or operator || to run another command if and only if the first command fails. That is, I would suspect the following to print out both one and three, but not two:
set -e # Exit on error
fun(){
echo one
non_existing_command
echo two
}
fun || echo three
What I get however is one and two. That is, the || operator prevents the exit (as it should) but it chooses to continue with the function body and disregard the right-hand command.
Any explanation?
It appears to be documented in the set builtin command
If a compound command or shell function executes in a context where -e is being ignored [such as on the left-hand of a ||], none of the commands executed within the compound command or function body will be affected by the -e setting, even if -e is set and a command returns a failure status.
Emphasis and comment are mine.
Also, if you try to set -e within the function, don't bother: the next sentence:
If a compound command or shell function sets -e while executing in a context where -e is ignored, that setting will not have any effect until the compound command or the command containing the function call completes.

Bash functions ignore errors when called in a boolean expression?

The following bash script prints "This should never run!":
#!/bin/bash
set -e
func() {
echo "Bailing out..."
false # Should bail out because of set -e
echo "This should never run!"
}
func || true
What's going on here?
A simple conclusion to make is that set -e is not propagated inside functions. However, this is not the case. Changing the last line (func || true) to just func causes the function to stop at the false statement as expected.
Another hypothesis is that bash will start executing the function, and on the first error (false) evaluate the rest of the line it was called on (func || true), and if that retuns 0 (true), then it continues executing the function as if nothing happened. However, replacing true with echo "Oops!" disproves that, since no "Oops!" is printed.
Is set -e somehow ignored when executing functions as part of a boolean expression?
From man bash regarding set -e:
Exit immediately if a pipeline (which may consist of a single simple command), a list, or a compound command (see SHELL GRAMMAR above), exits with a non-zero status. The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test following the if or elif reserved words, part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command's return value is being inverted with !. If a compound command other than a subshell returns a non-zero status because a command failed while -e was being ignored, the shell does not exit. A trap on ERR, if set, is executed before the shell exits. This option applies to the shell environment and each subshell environment separately (see COMMAND EXECUTION ENVIRONMENT above), and may cause subshells to exit before executing all the commands in the subshell.

Bash script failing of no reason

I have this bash script (actually a part of https://github.com/ddollar/heroku-buildpack-multi/blob/master/bin/compile with echo I've added myself):
echo "[DEBUG] chmod done"
framework=$($dir/bin/detect $1)
echo "[DEBUG] $framework done"
And I see in the log:
[DEBUG] chmod done
Staging failed: Buildpack compilation step failed
And I do not see the second echo in the logs at all.
I do not know much bash unfortunately. Could anybody explain to me in what case the first echo performs and second do not? I always thought that both echo should always work no matter whether the second line succeeds or not.
It's not visible in your question, but clicking on your link, it says in the third line
set -e
This means to stop processing the script immediately whenever an error occurs. Comment that line, and the script should run through and also print the second echo statement.
Note that I didn't inspect what the script actually does and I cannot tell you if commenting out set -e is actually good advice or not.
From man set:
−e: When this option is on, when any command fails (for any of the reasons listed
in Section 2.8.1, Consequences of Shell Errors or by returning an exit status
greater than zero), the shell immediately shall exit with the following excep‐
tions:
1. The failure of any individual command in a multi-command pipeline shall not
cause the shell to exit. Only the failure of the pipeline itself shall be
considered.
2. The −e setting shall be ignored when executing the compound list following
the while, until, if, or elif reserved word, a pipeline beginning with the
! reserved word, or any command of an AND-OR list other than the last.
3. If the exit status of a compound command other than a subshell command was
the result of a failure while −e was being ignored, then −e shall not apply
to this command.
This requirement applies to the shell environment and each subshell environment
separately. For example, in:
set -e; (false; echo one) | cat; echo two
the false command causes the subshell to exit without executing echo one; how‐
ever, echo two is executed because the exit status of the pipeline (false; echo
one) | cat is zero.

Exit status of a Command in Bash Scripting is always true

I'm trying to run a command ( gerrit query ) in bash and assign that to a variable.
I'm using this is a bash script file & I want to handle the case that if the command throws an error( i.e if the gerrit query fails), I should be able to handle the same.
For example:
var=`ssh -p $GERRIT_PORT_NUMBER $GERRIT_SERVER_NAME gerrit query --current-patch-set $PATCHSET_ID`
I do know that I can check the last exit status using $? in bash, but for the above case, the assignment to the variable over-rides the earlier exit status ( i.e the gerrit query failure status) and the above command never fails. It is always true.
Can you let me know if there is a way to handle the exit status of a command even when it is assigned to a variable in bash.
Update:
My assumption was wrong here that an assignment was causing the overriding of the exit status and Charles example and explanation in his answer are correct.
The real reason for the exit status being overridden was I was piping the output of the above command to a sed script which was the culprit in overriding the exit status. I found the following which helped me to resolve the issue. https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another/73180#73180 Pipe output and capture exit status in Bash
Complete command that I was trying.
var=ssh -p $GERRIT_PORT_NUMBER $GERRIT_SERVER_NAME gerrit query --current-patch-set $PATCHSET_ID | sed 's/message//'
The assertion made in this question is untrue; assignments do not modify exit status. You can check this yourself:
var=$(false); echo $?
...will correctly emit 1.
That said, if an assignment is done in the context of a local, declare, or similar keyword, this may no longer hold true:
f() { local var=$(false); echo $?; }; f
...will emit 0, and is worked around by separating out the local from the assignment:
f() { local var; var=$(false); echo $?; }; f
...which correctly returns 1.
SSH itself also returns exit status correctly, as you can similarly test yourself:
ssh localhost false; echo $?
...correctly returns 1.
The reasonable conclusion, then, is that gerrit itself is failing to convey a non-successful exit status. This bug should be addressed through gerrit's support mechanisms, rather than as a bash question.

Resources