I am using $PIPESTATUS to print the exit code for each pipe command. I now know that pipes run in parallel but once the exit code is <>0, how do I get the script to exit instead of progressing to the next command? Thanks.
I can't put set -e at the top because once the error is detected, the script exits but $PIPESTATUS isn't displayed because that echo command is after any failed command in the pipeline.
set -o pipefail
true | false | true || { declare -p PIPESTATUS; exit; }
echo "whoops"
output:
declare -a PIPESTATUS=([0]="0" [1]="1" [2]="0")
From Bash Reference Manual:
pipefail
If set, the return value of a pipeline is the value of the
last (rightmost) command to exit with a non-zero status, or zero if
all commands in the pipeline exit successfully. This option is
disabled by default.
Related
I am curious about bash's behavior and the exit status of the situation when I enter the command
exit [exit status] | exit [exit status] | .. [repetition of exit and exit status]
it gives me output below. and, then doesn't exits.
Is this an undefined behavior?
bash-3.2$ exit 1 | exit 2
bash-3.2$ echo $?
2
From the bash man page:
Each command in a pipeline is executed as a separate process (i.e., in a subshell).
So, even the first exit will not exit your shell, as it only exits the subshell.
As for the exit codes:
The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully.
You can activate pipefail like this:
$ set -o pipefail
$ exit 1 | exit 2 | exit 0
$ echo $?
2
exit 1 | exit 2 is not sequential but concurrent.
Even if the last command takes STDOUT output from the first command.
What is a simple explanation for how pipes work in Bash?
Moreover, each one of those commands is executed in a subshell.
So your main shell, where you type commands is not exited.
A piped command is like a whole composition of commands instead of one command after another.
If you want to exit, you can make it sequential exit 1 || exit 2.
Finally, by default, $? is the most recent foreground pipeline exit status.
What are the special dollar sign shell variables?
This question already has answers here:
Aborting a shell script if any command returns a non-zero value
(10 answers)
Closed 1 year ago.
I am a noob in shell-scripting. I want to print a message and exit my script if a command fails. I've tried:
my_command && (echo 'my_command failed; exit)
but it does not work. It keeps executing the instructions following this line in the script. I'm using Ubuntu and bash.
Try:
my_command || { echo 'my_command failed' ; exit 1; }
Four changes:
Change && to ||
Use { } in place of ( )
Introduce ; after exit and
spaces after { and before }
Since you want to print the message and exit only when the command fails ( exits with non-zero value) you need a || not an &&.
cmd1 && cmd2
will run cmd2 when cmd1 succeeds(exit value 0). Where as
cmd1 || cmd2
will run cmd2 when cmd1 fails(exit value non-zero).
Using ( ) makes the command inside them run in a sub-shell and calling a exit from there causes you to exit the sub-shell and not your original shell, hence execution continues in your original shell.
To overcome this use { }
The last two changes are required by bash.
The other answers have covered the direct question well, but you may also be interested in using set -e. With that, any command that fails (outside of specific contexts like if tests) will cause the script to abort. For certain scripts, it's very useful.
If you want that behavior for all commands in your script, just add
set -e
set -o pipefail
at the beginning of the script. This pair of options tell the bash interpreter to exit whenever a command returns with a non-zero exit code. (For more details about why pipefail is needed, see http://petereisentraut.blogspot.com/2010/11/pipefail.html)
This does not allow you to print an exit message, though.
Note also, each command's exit status is stored in the shell variable $?, which you can check immediately after running the command. A non-zero status indicates failure:
my_command
if [ $? -eq 0 ]
then
echo "it worked"
else
echo "it failed"
fi
I've hacked up the following idiom:
echo "Generating from IDL..."
idlj -fclient -td java/src echo.idl
if [ $? -ne 0 ]; then { echo "Failed, aborting." ; exit 1; } fi
echo "Compiling classes..."
javac *java
if [ $? -ne 0 ]; then { echo "Failed, aborting." ; exit 1; } fi
echo "Done."
Precede each command with an informative echo, and follow each command with that same
if [ $? -ne 0 ];... line. (Of course, you can edit that error message if you want to.)
Provided my_command is canonically designed, ie returns 0 when succeeds, then && is exactly the opposite of what you want. You want ||.
Also note that ( does not seem right to me in bash, but I cannot try from where I am. Tell me.
my_command || {
echo 'my_command failed' ;
exit 1;
}
You can also use, if you want to preserve exit error status, and have a readable file with one command per line:
my_command1 || exit
my_command2 || exit
This, however will not print any additional error message. But in some cases, the error will be printed by the failed command anyway.
The trap shell builtin allows catching signals, and other useful conditions, including failed command execution (i.e., a non-zero return status). So if you don't want to explicitly test return status of every single command you can say trap "your shell code" ERR and the shell code will be executed any time a command returns a non-zero status. For example:
trap "echo script failed; exit 1" ERR
Note that as with other cases of catching failed commands, pipelines need special treatment; the above won't catch false | true.
Using exit directly may be tricky as the script may be sourced from other places (e.g. from terminal). I prefer instead using subshell with set -e (plus errors should go into cerr, not cout) :
set -e
ERRCODE=0
my_command || ERRCODE=$?
test $ERRCODE == 0 ||
(>&2 echo "My command failed ($ERRCODE)"; exit $ERRCODE)
I have a short script like this:
#!/bin/bash
<some_process> | tee -a /tmp/some.log &
wait $(pidof <some_process_name>)
echo $?
The result is always 0, irrespective of the exit status of some_process.
I know PIPESTATUS can be used here, but why does tee break wait?
Well, this is something that, for some peculiar reason, the docs don't mention. The code, however, does:
int wait_for (pid) { /*...*/
/* If this child is part of a job, then we are really waiting for the
job to finish. Otherwise, we are waiting for the child to finish. [...] */
if (job == NO_JOB)
job = find_job (pid, 0, NULL);
So it's actually waiting for the whole job, which, as we know, normally yields the exit status of the last command in the chain.
To make matters worse, $PIPESTATUS can only be used with the last foreground command.
You can, however, utilize $PIPESTATUS in a subshell job, like this:
(<some_process> | tee -a /tmp/some.log; exit ${PIPESTATUS[0]}) &
# somewhere down the line:
wait %%<some_process>
The trick here is to use $PIPESTATUS but also wait -n:
Check this example:
#!/bin/bash
# start background task
(sleep 5 | false | tee -a /tmp/some.log ; exit ${PIPESTATUS[1]}) &
# foreground code comes here
echo "foo"
# wait for background task to finish
wait -n
echo $?
The reason you always get an exit status of 0 is that the exit status returned is that of the last command in the pipeline, which is tee. By using a pipe, you eliminate the normal detection of exit status of <some_command>.
From the bash man page:
The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully.
So... The following might be what you want:
#!/usr/bin/env bash
set -o pipefail
<some_command> | tee -a /tmp/some.log &
wait %1
echo $?
Is there any way to a piped commands to replicate its previous command exit status?
For example:
#/bin/bash
(...)
function customizedLog() {
# do something with the piped command output
exit <returned value from the last piped comand/script (script.sh)>
}
script.sh | customizedLog
echo ${?} # here I wanna show the script exit value
(...)
I know I could simply check the return using ${PIPESTATUS[0]}, but I really want to do this like the customizedLog function wasn't there.
Any thoughts?
In bash:
set -o pipefail
This will return the last non-zero exit status in a pipeline, or zero if all commands in the pipeline succeed.
set -o pipefail
script.sh | customizedLog
echo ${?}
Just make sure customizedLog succeeds (return 0), and you should pick up the exit status of script.sh. Test with false | customizedLog and true | customizedLog.
script.sh | customizedLog
The above will run in two separate processes (or 3, actually -- customizedLog will run in a bash fork as you can verify with something like ps -T --forest). As far as I know, with the UNIX process model, the only process that has access to a process's return information is its parent so there's no way customized log will be able to retrieve it.
So no, unless the previous command is run from a wrapper command that passes the exit status through the pipe (e.g., as the last line):
( command ; echo $? ) | piped_command_that_is_aware_of_such_an_arrangement
I have a bash script having the following command
rm ${thefile}
In order to ensure the command is execute successfully, I use $? variable to check on the status, but this variable doesn't show the exact error? To do this, I redirect the standard error output to a log file using following command:
rm ${file} >> ${LOG_FILE} 2>&1
With this command I can't use $? variable to check status on the rm command because the command behind the rm command is executed successfully, thus $? variable is kind of useless here.
May I know is there a solution that could combine both features where I'm able to check on the status of rm command and at mean time I'm allow to redirect the output?
With this command I can't use $? variable to check status on the rm command because the command behind the rm command is executed successfully, thus $? variable is kind of useless here.
That is simply not true. All of the redirections are part of a single command, and $? contains its exit status.
What you may be thinking of is cases where you have multiple commands arranged in a pipeline:
command-1 | command-2
When you do that, $? is set to the exit status of the last command in the pipeline (in this case command-2), and you need to use the PIPESTATUS array to get the exit status of other commands. (In this example ${PIPESTATUS[0]} is the exit status of command-1 and ${PIPESTATUS[1]} is equivalent to $?.)
What you probably need is the shell option pipefail in bash (from man bash):
The return status of a pipeline is the exit status of the last command, unless
the pipefail option is enabled. If pipefail is enabled, the pipeline's return
status is the value of the last (rightmost) command to exit with a non-zero sta‐
tus, or zero if all commands exit successfully. If the reserved word ! precedes
a pipeline, the exit status of that pipeline is the logical negation of the exit
status as described above. The shell waits for all commands in the pipeline to
terminate before returning a value.
> shopt -s -o pipefail
> true | false
> echo $?
1
> false | true
> echo $?
1
true | true
echo $?
0