Being trying to properly guard for non-zero exits on a bash script.
What is the difference between -e, -u and -o pipefail?
-o pipefail is not sufficient to exit with an error code?
set -e: Exit immediately if a command exits with a non-zero status.
set -u: If you try to access an undefined variable, that is an error.
set -o pipefail: If any command in a pipeline returns a non-zero exit code, the return code of the entire pipeline is the exit code of the last failed command.
Related
#!/usr/bin/env bash
set -eo pipefail
sha256sum \
Dockerfile-ci \
frontend/pnpm-lock.yaml \
| sha256sum
If frontend/pnpm-lock.yaml does not exist, and the script above is run
sha256sum: frontend/pnpm-lock.yaml: No such file or directory
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
It correctly logs that the file doesn't exist, but it continues piping that into the next sha256sum. Shouldn't set -eo pipefail immediately exit the script on the first sha256sum command and not pipe into the second sha256sum?
pipefail doesn't cause the pipeline to abort early if a command fails. The pipeline still runs to completion, until all the commands have exited. That's true with or without pipefail.
What pipefail does do is ensure the return status is a failure if any of the commands fail. Without pipefail the pipeline fails only if the final command fails.
From the bash manual (emphasis added):
The exit status of a pipeline is the exit status of the last command in the pipeline, unless the pipefail option is enabled (see The Set Builtin). If pipefail is enabled, the pipeline’s return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully. If the reserved word ! precedes the pipeline, the exit status is the logical negation of the exit status as described above. The shell waits for all commands in the pipeline to terminate before returning a value.
Consider the following Bash script:
#!/usr/bin/env bash
set -o errexit pipefail
shopt -s inherit_errexit
( echo hello ; exit 1 ) | cat
echo world
Running with version 5.0.17, the output is as follows:
hello
world
However, the subshell responsible for printing hello would have failed, with a nonzero exit status. Part of a pipe, with enabling the optionpipefail, the entire pipe should similarly fail, with the same status (the subsequent item in the pipe of course returning a zero status by itself). Thus, the pipe should evaluate to a nonzero status, which due to errexit (if not inherit_errexit as well) would prompt the immediate termination of the script, without reaching the final statement printing world.
As seen, the prediction is not accurate that the final print statement is not reached.
Why?
You're only setting the errexit setting, not pipefail. You can't put multiple options after -o, you need to repeat the -o option. Everything else is being put into the positional arguments.
So change
set -o errexit pipefail
to
set -o errexit -o pipefail
I am using $PIPESTATUS to print the exit code for each pipe command. I now know that pipes run in parallel but once the exit code is <>0, how do I get the script to exit instead of progressing to the next command? Thanks.
I can't put set -e at the top because once the error is detected, the script exits but $PIPESTATUS isn't displayed because that echo command is after any failed command in the pipeline.
set -o pipefail
true | false | true || { declare -p PIPESTATUS; exit; }
echo "whoops"
output:
declare -a PIPESTATUS=([0]="0" [1]="1" [2]="0")
From Bash Reference Manual:
pipefail
If set, the return value of a pipeline is the value of the
last (rightmost) command to exit with a non-zero status, or zero if
all commands in the pipeline exit successfully. This option is
disabled by default.
Below code is the part of my shell script.
But I am not able to understand why exit status(sshStatus) always coming 0?
I want ssh output as well as exit status.
Please help me to find the solution.
local output="$(ssh -q -o ConnectTimeout=10 \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
$user#$host "$command" 2>&1)"
local sshStatus=$?
command can be :
command="[ ! -d /home/upendra/dfs ]"
command="cat /home/upendra/a.txt"
command="sh /home/upendra/dfs/bin/start-datanode.sh"
Whenever i'm calling command like below directly on shell prompt:
ssh -q -o ConnectTimeout=10 \
-o BatchMode=yes \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
upendra#172.20.20.2 "[ ! -d /home/upendra/dfs ]" 2>&1
Then exit status(echo $?) is coming 1. This is correct because this directory not exists on host.
I got solution on this page :bash shell - ssh remote script capture output and exit code? it is due to "local output". – Upendra
You are always getting exot status as 0 as your command is excecuted successfully
The exit status you obtain from the local machine is the exit status of the last command in ssh session
for example
$ ssh localhost
$ exit 5
$ echo $? #on local system
5
Consider a case without any command
$ ssh localhost
$ ls
#will list commands and exit succussfully
ctrl+d
$ echo $? #on local system
0
Here the exit status of ls command is 0 which is printed.
Every command returns an exit status (sometimes referred to as a return status or exit code). A successful command returns a 0, while an unsuccessful command returns a non-zero value that usually can be interpreted as an error code. Well-behaved UNIX commands, programs, and utilities return a 0 exit code upon successful completion, though there are some exceptions.
Likewise, functions within a script and the script itself return an exit status. The last command executed in the function or script determines the exit status. Within a script, an exit nnn command may be used to deliver an nnn exit status to the shell (nnn must be an integer in the 0 - 255 range).
#!/bin/bash
echo hello
echo $? # Exit status 0 returned because command executed successfully.
lskdf # Unrecognized command. echo $? # Non-zero exit status
returned -- command failed to execute.
echo
exit 113 # Will return 113 to shell.
# To verify this, type "echo $?" after script terminates.
Your code return exit code 0 which means your shell script execute successfully.
I have a bash script having the following command
rm ${thefile}
In order to ensure the command is execute successfully, I use $? variable to check on the status, but this variable doesn't show the exact error? To do this, I redirect the standard error output to a log file using following command:
rm ${file} >> ${LOG_FILE} 2>&1
With this command I can't use $? variable to check status on the rm command because the command behind the rm command is executed successfully, thus $? variable is kind of useless here.
May I know is there a solution that could combine both features where I'm able to check on the status of rm command and at mean time I'm allow to redirect the output?
With this command I can't use $? variable to check status on the rm command because the command behind the rm command is executed successfully, thus $? variable is kind of useless here.
That is simply not true. All of the redirections are part of a single command, and $? contains its exit status.
What you may be thinking of is cases where you have multiple commands arranged in a pipeline:
command-1 | command-2
When you do that, $? is set to the exit status of the last command in the pipeline (in this case command-2), and you need to use the PIPESTATUS array to get the exit status of other commands. (In this example ${PIPESTATUS[0]} is the exit status of command-1 and ${PIPESTATUS[1]} is equivalent to $?.)
What you probably need is the shell option pipefail in bash (from man bash):
The return status of a pipeline is the exit status of the last command, unless
the pipefail option is enabled. If pipefail is enabled, the pipeline's return
status is the value of the last (rightmost) command to exit with a non-zero sta‐
tus, or zero if all commands exit successfully. If the reserved word ! precedes
a pipeline, the exit status of that pipeline is the logical negation of the exit
status as described above. The shell waits for all commands in the pipeline to
terminate before returning a value.
> shopt -s -o pipefail
> true | false
> echo $?
1
> false | true
> echo $?
1
true | true
echo $?
0