shell write to file without using last command status - shell

If I have the following command
do_stuff -pram somepram
and then typically I do the following to get the status
if [ $OUT -eq 0 ]; then
# do some stuff
else
# do other stuff
fi
However, I need to write the output of my do_stuff command to a file.
So the command looks like
do_stuff -pram somepram 2>&1 | tee someFile
Which means the result of $? will be the exit code on tee and no the command proceeding it.
Is there another way to write to a file that doesn't interfere with this or allows me to obtain the exit code

The command exit status is unrelated to the output it produces. If the command is well behaved, then you can write:
if do_stuff -pram somepram > someFile
then
: OK - it worked
else
: Oops - it failed
fi
The error messages go to standard error still. The standard output goes to the file. You can even check whether the output file is empty on success, treating an empty file as a failure, but if the command is well-behaved, that won't be necessary.
If the command is ill-behaved, then all bets are off. If at all possible, file bugs and get it fixed so that it returns a reliable exit status.

If you want to keep using tee, you can do this at the top of your Bash script:
set -o pipefail
Then the exit status of the pipeline will be non-zero if any of its stages is non-zero.

Related

In Bash, how to capture exit status code from a command nested within a complex statement

I'm relatively new to working with bash. I've inherited this bit of code to run a command via SLURM on an HPC system:
CMD="srun -srunParam1 ... -srunParamN ./scriptToRun.sh -scriptParam1"
exec 5>&1
results=$(eval "${CMD}" | tee - >&5))
That all works just fine.
But, I need to capture the exit status of just eval "${CMD}", and don't know how to do it.
Initially, I put exitStatus=$? after the results=... command; but, I believe that's catching the status of assigning a value to the results variable, and not of eval ${CMD}
The script goes on to process the output that is in $results. I don't really understand why the file descriptor has been opened here (or, how to properly use file descriptors). But, I'll save that for further research/a separate question.
EDIT: I commented out the bits with the file descriptor, and observed that the script still works, but $results does not contain the output from running $CMD - it only contains the post-processing of $CMD.
Bash has PIPESTATUS:
results=$(eval "${CMD}" | tee - >&5; exit ${PIPESTATUS[0]})
exitStatus=$?
Note that in the code above we are examining the exit status of a command that was run inside a subshell ($(...)); it would not work to access PIPESTATUS in the parent.
One way to get status is to save it in a file :
results=$( { eval "${CMD}"; echo $? > /tmp/cmd_status.txt; } | tee - >&5))
# Process /tmp/cmd_status.txt here

Wrong exit code for variable function call assignment in subshell

I have a script which quite often fails. It is crucial that the correct exit code is passed on.
This code works as expected:
#!/usr/bin/env bash
function my_function {
echo "Important output"
exit 1
}
function cleanup {
MY_EXIT_CODE=$?
echo "MY_EXIT_CODE: ${MY_EXIT_CODE}"
}
trap cleanup EXIT
my_function
MY_EXIT_CODE is 1 as expected and running echo $? after the script gives me 1 as well (as expected)
However, I need to get the complete output of my_function both to a variable and the console output. In order to do so, as advised in this answer (which seems itself based on this answer) I changed my code into
#!/usr/bin/env bash
function my_function {
echo "Important output"
exit 1
}
function cleanup {
MY_EXIT_CODE=$?
echo "MY_EXIT_CODE: ${MY_EXIT_CODE}"
}
trap cleanup EXIT
exec 5>&1
FF=$(my_function|tee /dev/fd/5)
And now the exit code is wrong. Is it 0 while it should be 1. I know this is somewhat connected to subshell handling but I couldn't figure out how to solve it.
And now the exit code is wrong. Is it 0 while it should be 1
No, your assumption is wrong. Assuming tee succeeded, the exit code should be 0. From the posix shell manual:
If the reserved word ! does not precede the pipeline, the exit status shall be the exit status of the last command specified in the pipeline.
The "last command" in a pipeline is the rightmost command. Because in your case tee exits with 0, the exit status of FF=$(.... | tee) is zero.
how to solve it.
That depends on the behavior you want to achieve. Usually, in bash you may just set -o pipefail to always catch errors.
It seems that simply stating
set -o pipefail
at the beginning of the script file did the trick.
Yes it's subshell and pipe, try like this
FF=$(my_function|tee /dev/fd/5; exit $PIPESTATUS)

How can I create timestamped logs and error handle in BASH at the same time?

I am writing a BASH script and two of the things I need it to do is:
Provide a timestamped log file.
Handle errors.
I am finding that these two objectives are clashing.
First of all, I am using the ts command to timestamp log entries, e.g. <a command/subscript> 2>&1 | ts '%H:%M:%S ' >> log. Note that I need all the lines output of any subscripts to be timestamped too. This works great... until I try to handle errors using exit codes.
Any command that fails (exits with a code of 1) is immediately followed with the ts command which executes successfully (exits with a code of 0). This means that I am unable to use the exit codes of the commands to handle errors with the $? environment variable because ts is always the last command to run and always has an exit code of 0.
Here is the case statement I am using:
<command> 2>&1 | ts '%H:%M:%S ' >> log
case $? in
0)
echo "Success"
;;
*)
echo "Failure"
esac
When a foreground pipeline returns, bash saves exit status values of its components to an array variable named PIPESTATUS. In this case, you can use ${PIPESTATUS[0]} (or just $PIPESTATUS; as you're interested in the first component) instead of $? to get <command>'s exit status value.
Proof of concept:
$ false | true | false | true
$ declare -p PIPESTATUS
declare -a PIPESTATUS=([0]="1" [1]="0" [2]="1" [3]="0")

Output redirection in a bash script doesn't work

I have a script which ensures that the process keeps on running and the output of the process are being redirected to corresponding process_name.out files.
Mysteriously (for me atleast), the output doesn't get redirected to the file unless I close down the script though the error does.
Any clues in this regard would be really helpful.
#!/bin/bash
until $1 >> "/root/$1.out" 2>>"/root/$1.log" ; do
echo "Server $1 crashed with exit code $?. Respawning.." >> "/root/procees.log"
sleep 1
done
The solution as #shelter mentioned in one of the comments is to use
until unbuffer $1
instead

In a Bash script, how can I exit the entire script if a certain condition occurs?

I'm writing a script in Bash to test some code. However, it seems silly to run the tests if compiling the code fails in the first place, in which case I'll just abort the tests.
Is there a way I can do this without wrapping the entire script inside of a while loop and using breaks? Something like a dun dun dun goto?
Try this statement:
exit 1
Replace 1 with appropriate error codes. See also Exit Codes With Special Meanings.
Use set -e
#!/bin/bash
set -e
/bin/command-that-fails
/bin/command-that-fails2
The script will terminate after the first line that fails (returns nonzero exit code). In this case, command-that-fails2 will not run.
If you were to check the return status of every single command, your script would look like this:
#!/bin/bash
# I'm assuming you're using make
cd /project-dir
make
if [[ $? -ne 0 ]] ; then
exit 1
fi
cd /project-dir2
make
if [[ $? -ne 0 ]] ; then
exit 1
fi
With set -e it would look like:
#!/bin/bash
set -e
cd /project-dir
make
cd /project-dir2
make
Any command that fails will cause the entire script to fail and return an exit status you can check with $?. If your script is very long or you're building a lot of stuff it's going to get pretty ugly if you add return status checks everywhere.
A SysOps guy once taught me the Three-Fingered Claw technique:
yell() { echo "$0: $*" >&2; }
die() { yell "$*"; exit 111; }
try() { "$#" || die "cannot $*"; }
These functions are *NIX OS and shell flavor-robust. Put them at the beginning of your script (bash or otherwise), try() your statement and code on.
Explanation
(based on flying sheep comment).
yell: print the script name and all arguments to stderr:
$0 is the path to the script ;
$* are all arguments.
>&2 means > redirect stdout to & pipe 2. pipe 1 would be stdout itself.
die does the same as yell, but exits with a non-0 exit status, which means “fail”.
try uses the || (boolean OR), which only evaluates the right side if the left one failed.
$# is all arguments again, but different.
If you will invoke the script with source, you can use return <x> where <x> will be the script exit status (use a non-zero value for error or false). But if you invoke an executable script (i.e., directly with its filename), the return statement will result in a complain (error message "return: can only `return' from a function or sourced script").
If exit <x> is used instead, when the script is invoked with source, it will result in exiting the shell that started the script, but an executable script will just terminate, as expected.
To handle either case in the same script, you can use
return <x> 2> /dev/null || exit <x>
This will handle whichever invocation may be suitable. That is assuming you will use this statement at the script's top level. I would advise against directly exiting the script from within a function.
Note: <x> is supposed to be just a number.
I often include a function called run() to handle errors. Every call I want to make is passed to this function so the entire script exits when a failure is hit. The advantage of this over the set -e solution is that the script doesn't exit silently when a line fails, and can tell you what the problem is. In the following example, the 3rd line is not executed because the script exits at the call to false.
function run() {
cmd_output=$(eval $1)
return_value=$?
if [ $return_value != 0 ]; then
echo "Command $1 failed"
exit -1
else
echo "output: $cmd_output"
echo "Command succeeded."
fi
return $return_value
}
run "date"
run "false"
run "date"
Instead of if construct, you can leverage the short-circuit evaluation:
#!/usr/bin/env bash
echo $[1+1]
echo $[2/0] # division by 0 but execution of script proceeds
echo $[3+1]
(echo $[4/0]) || exit $? # script halted with code 1 returned from `echo`
echo $[5+1]
Note the pair of parentheses which is necessary because of priority of alternation operator. $? is a special variable set to exit code of most recently called command.
I have the same question but cannot ask it because it would be a duplicate.
The accepted answer, using exit, does not work when the script is a bit more complicated. If you use a background process to check for the condition, exit only exits that process, as it runs in a sub-shell. To kill the script, you have to explicitly kill it (at least that is the only way I know).
Here is a little script on how to do it:
#!/bin/bash
boom() {
while true; do sleep 1.2; echo boom; done
}
f() {
echo Hello
N=0
while
((N++ <10))
do
sleep 1
echo $N
# ((N > 5)) && exit 4 # does not work
((N > 5)) && { kill -9 $$; exit 5; } # works
done
}
boom &
f &
while true; do sleep 0.5; echo beep; done
This is a better answer but still incomplete a I really don't know how to get rid of the boom part.
You can close your program by program name on follow way:
for soft exit do
pkill -9 -x programname # Replace "programmname" by your programme
for hard exit do
pkill -15 -x programname # Replace "programmname" by your programme
If you like to know how to evaluate condition for closing a program, you need to customize your question.
#!/bin/bash -x
# exit and report the failure if any command fails
exit_trap () { # ---- (1)
local lc="$BASH_COMMAND" rc=$?
echo "Command [$lc] exited with code [$rc]"
}
trap exit_trap EXIT # ---- (2)
set -e # ---- (3)
Explanation:
This question is also about how to write clean code. Let's divide the above script into multiple parts:
Part - 1:
exit_trap is a function that gets called when any step failed and captures the last executed step using $BASH_COMMAND and captures the return code of that step. This is the function that can be used for any clean-up, similar to shutdownhooks
The command currently being executed or about to be executed, unless the shell is executing a command as the result of a trap, in which case it is the command executing at the time of the trap.
Doc.
Part - 2:
trap [action] [signal]
Register the trap action (here exit_trap function) in case of EXIT signal.
Part - 3:
Exit immediately if a sequence of one or more commands returns a non-zero status. The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test in an if statement, part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command’s return status is being inverted with !. If a compound command other than a subshell returns a non-zero status because a command failed while -e was being ignored, the shell does not exit. A trap on ERR, if set, is executed before the shell exits.
Doc.
Part - 4:
You can create a common.sh file and source it in all of your scripts.
source common.sh

Resources