I am curious about bash's behavior and the exit status of the situation when I enter the command
exit [exit status] | exit [exit status] | .. [repetition of exit and exit status]
it gives me output below. and, then doesn't exits.
Is this an undefined behavior?
bash-3.2$ exit 1 | exit 2
bash-3.2$ echo $?
2
From the bash man page:
Each command in a pipeline is executed as a separate process (i.e., in a subshell).
So, even the first exit will not exit your shell, as it only exits the subshell.
As for the exit codes:
The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully.
You can activate pipefail like this:
$ set -o pipefail
$ exit 1 | exit 2 | exit 0
$ echo $?
2
exit 1 | exit 2 is not sequential but concurrent.
Even if the last command takes STDOUT output from the first command.
What is a simple explanation for how pipes work in Bash?
Moreover, each one of those commands is executed in a subshell.
So your main shell, where you type commands is not exited.
A piped command is like a whole composition of commands instead of one command after another.
If you want to exit, you can make it sequential exit 1 || exit 2.
Finally, by default, $? is the most recent foreground pipeline exit status.
What are the special dollar sign shell variables?
Related
I found in some bash scripts, usually at the end of the file, there is one line of code, like below:
: exit 0
What's the meaning of it? Can I remove it directly?
The bash builtin : is basically a command that returns zero (success) after all its arguments are expanded by the shell(a). In this case, the expansion doesn't really do anything so it's effectively a null operation. I suspect it's there just to indicate the effect of the :(b).
And the effect of that : has to do with what bash scripts return. They basically return the exit status of the last command that was run in the script. The : will therefore force the exit status of the script as a whole to be zero regardless of what the command before it returned.
You can see the effect with the following script:
ls /tmp/nosuchfile 2> /dev/null
If you run that followed by echo $?, you'll see an error code:
pax> ./script.sh ; echo $?
2
If you then change the script to:
ls /tmp/nosuchfile 2> /dev/null
: some arbitrary text
then you will see a success code from the script:
pax> ./script.sh ; echo $?
0
(a) I often use it for infinite loops, such as:
while : ; do somePeriodicThing ; sleep 60 ; done
(b) Of course, it's not quite the same as exit 0 since the exit will exit your current shell which will act differently depending on whether you ran it or sourced it:
./script.sh # runs it, exit will exit that script.
. ./script.sh # sources it, exit will exit your shell.
The : will not exit your current shell in either of those two cases.
I have a bash script where I would like to run two processes in parallel, and have the script fail if either of the processes return non-zero. A minimal example of my initial attempt is:
#!/bin/bash
set -e
(sleep 3 ; true ) &
(sleep 4 ; false ) &
wait %1 && wait %2
echo "Still here, exit code: $?"
As expected this doesn't print the message because wait %1 && wait %2 fails and the script exits due to the set -e. However, if the waits are reversed such that the first one has the non-zero status (wait %2 && wait %1), the message is printed:
$ bash wait_test.sh
Still here, exit code: 1
Putting each wait on its own line works as I want and exits the script if either of the processes fail, but the fact that it doesn't work with && makes me suspect that I'm misunderstanding something here.
Can anyone explain what's going on?
You can achieve what you want quite elegantly with GNU Parallel and its "fail handling".
In general, it will run as many jobs in parallel as you have CPU cores.
In your case, try this, which says "exit with failed status if one or more jobs failed":
#!/bin/bash
cat <<EOF | parallel --halt soon,fail=1
echo Job 1; exit 0
echo Job 2; exit 1
EOF
echo GNU Parallel exit status: $?
Sample Output
Job 1
Job 2
parallel: This job failed:
echo Job 2; exit 1
GNU Parallel exit status: 1
Now run it such that no job fails:
#!/bin/bash
cat <<EOF | parallel --halt soon,fail=1
echo Job 1; exit 0
echo Job 2; exit 0
EOF
echo GNU Parallel exit status: $?
Sample Output
Job 1
Job 2
GNU Parallel exit status: 0
If you dislike the heredoc syntax, you can put the list of jobs in a file called jobs.txt like this:
echo Job 1; exit 0
echo Job 2; exit 0
Then run with:
parallel --halt soon,fail=1 < jobs.txt
From bash manual section about usage of set
-e Exit immediately if a pipeline (which may consist of a single simple command), a list, or a compound command (see SHELL GRAMMAR above), exits with a non-zero status. The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test following the if or elif reserved words, part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command's return value is being inverted with !. If a compound command other than a subshell returns a non- zero status because a command failed while -e was being ignored, the shell does not exit. A trap on ERR, if set, is executed before the shell exits. This option applies to the shell environment and each subshell environment separately (see COMMAND EXECUTION ENVIRONMENT above), and may cause subshells to exit before executing all the commands in the subshell.
tl;dr
In a bash script, for a command list like this
command1 && command2
command1 is run in a separate environment, so it cannot affect the script's execution environment. but command2 is run in the current environment, so it can affect
I am using $PIPESTATUS to print the exit code for each pipe command. I now know that pipes run in parallel but once the exit code is <>0, how do I get the script to exit instead of progressing to the next command? Thanks.
I can't put set -e at the top because once the error is detected, the script exits but $PIPESTATUS isn't displayed because that echo command is after any failed command in the pipeline.
set -o pipefail
true | false | true || { declare -p PIPESTATUS; exit; }
echo "whoops"
output:
declare -a PIPESTATUS=([0]="0" [1]="1" [2]="0")
From Bash Reference Manual:
pipefail
If set, the return value of a pipeline is the value of the
last (rightmost) command to exit with a non-zero status, or zero if
all commands in the pipeline exit successfully. This option is
disabled by default.
I have a short script like this:
#!/bin/bash
<some_process> | tee -a /tmp/some.log &
wait $(pidof <some_process_name>)
echo $?
The result is always 0, irrespective of the exit status of some_process.
I know PIPESTATUS can be used here, but why does tee break wait?
Well, this is something that, for some peculiar reason, the docs don't mention. The code, however, does:
int wait_for (pid) { /*...*/
/* If this child is part of a job, then we are really waiting for the
job to finish. Otherwise, we are waiting for the child to finish. [...] */
if (job == NO_JOB)
job = find_job (pid, 0, NULL);
So it's actually waiting for the whole job, which, as we know, normally yields the exit status of the last command in the chain.
To make matters worse, $PIPESTATUS can only be used with the last foreground command.
You can, however, utilize $PIPESTATUS in a subshell job, like this:
(<some_process> | tee -a /tmp/some.log; exit ${PIPESTATUS[0]}) &
# somewhere down the line:
wait %%<some_process>
The trick here is to use $PIPESTATUS but also wait -n:
Check this example:
#!/bin/bash
# start background task
(sleep 5 | false | tee -a /tmp/some.log ; exit ${PIPESTATUS[1]}) &
# foreground code comes here
echo "foo"
# wait for background task to finish
wait -n
echo $?
The reason you always get an exit status of 0 is that the exit status returned is that of the last command in the pipeline, which is tee. By using a pipe, you eliminate the normal detection of exit status of <some_command>.
From the bash man page:
The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully.
So... The following might be what you want:
#!/usr/bin/env bash
set -o pipefail
<some_command> | tee -a /tmp/some.log &
wait %1
echo $?
I have a bash script having the following command
rm ${thefile}
In order to ensure the command is execute successfully, I use $? variable to check on the status, but this variable doesn't show the exact error? To do this, I redirect the standard error output to a log file using following command:
rm ${file} >> ${LOG_FILE} 2>&1
With this command I can't use $? variable to check status on the rm command because the command behind the rm command is executed successfully, thus $? variable is kind of useless here.
May I know is there a solution that could combine both features where I'm able to check on the status of rm command and at mean time I'm allow to redirect the output?
With this command I can't use $? variable to check status on the rm command because the command behind the rm command is executed successfully, thus $? variable is kind of useless here.
That is simply not true. All of the redirections are part of a single command, and $? contains its exit status.
What you may be thinking of is cases where you have multiple commands arranged in a pipeline:
command-1 | command-2
When you do that, $? is set to the exit status of the last command in the pipeline (in this case command-2), and you need to use the PIPESTATUS array to get the exit status of other commands. (In this example ${PIPESTATUS[0]} is the exit status of command-1 and ${PIPESTATUS[1]} is equivalent to $?.)
What you probably need is the shell option pipefail in bash (from man bash):
The return status of a pipeline is the exit status of the last command, unless
the pipefail option is enabled. If pipefail is enabled, the pipeline's return
status is the value of the last (rightmost) command to exit with a non-zero sta‐
tus, or zero if all commands exit successfully. If the reserved word ! precedes
a pipeline, the exit status of that pipeline is the logical negation of the exit
status as described above. The shell waits for all commands in the pipeline to
terminate before returning a value.
> shopt -s -o pipefail
> true | false
> echo $?
1
> false | true
> echo $?
1
true | true
echo $?
0