How can I get exit code of wget from the subshell process?
So, main problem is that $? is equal 0. Where can $?=8 be founded?
$> OUT=$( wget -q "http://budueba.com/net" | tee -a "file.txt" ); echo "$?"
0
It works without tee, actually.
$> OUT=$( wget -q "http://budueba.com/net" ); echo "$?"
8
But ${PIPESTATUS} array (I'm not sure it's related to that case) also does not contain that value.
$> OUT=$( wget -q "http://budueba.com/net" | tee -a "file.txt" ); echo "${PIPESTATUS[1]}"
$> OUT=$( wget -q "http://budueba.com/net" | tee -a "file.txt" ); echo "${PIPESTATUS[0]}"
0
$> OUT=$( wget -q "http://budueba.com/net" | tee -a "file.txt" ); echo "${PIPESTATUS[-1]}"
0
So, my question is - how can I get wget's exit code through tee and subshell?
If it could be helpful, my bash version is 4.2.20.
By using $() you are (effectively) creating a subshell. Thus the PIPESTATUS instance you need to look at is only available inside your subshell (i.e. inside the $()), since environment variables do not propagate from child to parent processes.
You could do something like this:
OUT=$( wget -q "http://budueba.com/net" | tee -a "file.txt"; exit ${PIPESTATUS[0]} );
echo $? # prints exit code of wget.
You can achieve a similar behavior by using the following:
OUT=$(wget -q "http://budueba.com/net")
rc=$? # safe exit code for later
echo "$OUT" | tee -a "file.txt"
Beware of this when using local variables:
local OUT=$(command; exit 1)
echo $? # 0
OUT=$(command; exit 1)
echo $? # 1
Copy the PIPESTATUS array first. Any reads destroy the current state.
declare -a PSA
cmd1 | cmd2 | cmd3
PSA=( "${PIPESTATUS[#]}" )
I used fifos to solve the sub-shell/PIPESTATUS problem. See
bash pipestatus in backticked command?
I also found these useful:
bash script: how to save return value of first command in a pipeline?
and https://unix.stackexchange.com/questions/14270/get-exit-status-of-process-thats-piped-to-another/70675#70675
Related
I want to retrieve the return status of a command which is being executed in a subshell.
I am running the below script from Unix Box A which has a passwordless SSH access to Box B whose IP is mentioned in the script as ip_addr.
I want to get the return status of command which is ran in subshell in my current environment.
That is if the below command fails:
echo "cmd" | system_program> 2>> /dev/null
then echo $? should print non-zero value and I should be able to use that value to decide further action.
Snippet of my script is:
sample.sh :
ip_addr="xxx.xxx.xx.xx"
status=$(ssh -q -T $ip_addr << EOF
rm /tmp/program.log; echo "cmd" | system_program> 2>> /dev/null; echo $?
EOF
)
You don't need the here-doc, or the echo. Try:
ssh -q -T $ip_addr 'rm /tmp/program.log; echo "cmd" | system_program> 2>> /dev/null'
Or if you want to use here-doc set errexit:
status=$(ssh -q -T $ip_addr << EOF
set -o errexit
rm /tmp/program.log; echo "cmd" | system_program> 2>> /dev/null
EOF
)
I have some code that I would like to have the $? variable of.
VARIABLE=`grep "searched_string" test.log | sed 's/searched/found/'`
Is there any way to test if this entire line (rather than just the sed command) was completed successfully? If I try the following code right after it:
if [ "$?" -ne 0 ]
then
echo 1
exit
fi
it doesn't run even if the grep part of the statement fails.
Could someone show how to resolve this issue?
Use the
echo ${PIPESTATUS[#]}
will print out the array of exit-statuses of all commands.
$ ls | grep . | wc -l
28
$ echo ${PIPESTATUS[#]}
0 0 0
but
$ ls | grep nonexistentfilename | wc -l
0
$ echo ${PIPESTATUS[#]}
0 1 0 #the grep returns 1 - pattern not found
or
$ ls nonexistentfilename | grep somegibberish | wc -l
ls: nonexistentfilename: No such file or directory
0
$ echo ${PIPESTATUS[#]}
1 1 0 #ls and grep fails
for exact command status
echo ${PIPESTATUS[1]} #for the grep
also here is the
set -o pipefail
from the docs
pipefail
If set, the return value of a pipeline is the value of the
last (rightmost) command to exit with a non-zero status, or zero if
all commands in the pipeline exit successfully. This option is
disabled by default.
$ ls nonexistentfile | wc -c
ls: nonexistentfile: No such file or directory
0
$ echo $?
0
$ set -o pipefail
$ ls nonexistentfile | wc -c
ls: nonexistentfile: No such file or directory
0
$ echo $?
1
EDIT based on the comment
Youre probably tried the next:
VARIABLE=$(grep "searched_string" test.log | sed 's/searched/found/')
echo "${PIPESTATUS[#]}"
Of course, this can't work because the whole $(...) part runs in the subshell (another process) and therefore any variable what is created is lost when the subshell exits. (at the ))
You should put the whole PIPESTATUS mechanism into $(...) like next:
variable=$(
grep "searched_string" test.log | sed 's/searched/found/'
# do something with PIPESTATUS
# you should not echo anythig to stdout (because will be captured into $variable)
# you can echo on stderr - e.g.
echo "=${PIPESTATUS[#]}=" >&2
)
Also, the second line of the comment is an solution, eg:
var_with_status=$(command | commmand2 ; echo ":DELIMITER:${PIPESTATUS[#]}")
now, the $var_with_status will contain not only the result of the command | command2 but the PIPESTATUS too, delimited with some unique delimiter, so you can extract it...
Also, the set -o pipefail will indicate the result - if you don't need exact place of the fail.
Also you can write the PIPESTATUS in some temp-file (in the subshell) and the parent can read it and delete the temp-file...
Also is possible print the PIPESTATUS into different file-descriptors in the subshell and read this descriptor in the parent shell, but....
... beware do not fall into the XY problem, where you will make extremelly complicated script, only because you don't want change the logic of the processing.
e.g. you can always break you script into safe parts, like:
var1=$(grep 'str' test.log)
#check the `$var1` and do something with the error indicated with `$?`
var2=(sed '....' <<<"$var1")
#check the `$var2` and do something with the error indicated with `$?`
#and so on
simple enough?
So, ask yourself - do you really need mungling with how to get the PIPESTATUS form an subshell?
Ps: don't use uppercase variable names. could interfere with some environment variables and causes hard-to-debug problems..
I have following code in my build script:
if [ -z "$1" ]; then
make -j10 $1 2>&1 | tee log.txt && notify-send -u critical -t 7 "BUILD DONE"
else
make -j10 $1 2>&1 | tee log.txt | grep -i --color "Error" && notify-send -u critical -t 7 "BUILD DONE"
fi
I tried to optimize it to:
local GREP=""
[[ ! -z "$1" ]] && GREP="| grep -i --color Error" && echo "Grepping for ERRORS"
make -j10 $1 2>&1 | tee log.txt "$GREP" && notify-send -u critical -t 7 "BUILD DONE"
But error thrown in make line if $1 isn't empty. I just can't figure out how to pass command with grep pipe through the variable.
Like others have already pointed out, you cannot, in general, expect a command in a variable to work. This is a FAQ.
What you can do is execute commands conditionally. Like this, for example:
( make -j10 $1 2>&1 && notify-send -u critical -t 7 "BUILD DONE" ) |
tee log.txt |
if [ -z "$1" ]; then
grep -i --color "Error"
else
cat
fi
This has the additional unexpected benefit that the notify-send is actually conditioned on the exit code of make (which is probably what you intended) rather than tee (which I would expect to succeed unless you run out of disk or something).
(Or if you want the notification regardless of the success status, change && to just ; -- I think this probably makes more sense.)
This is one of those rare Useful Uses of cat (although I still feel the urge to try to get rid of it!)
You can't put pipes in command variables:
$ foo='| cat'
$ echo bar $foo
bar | cat
The linked article explains how to do such things very well.
As mentioned in #l0b0's answer, the | will not be interpreted as you are hoping.
If you wanted to cut down on repetition, you could do something like this:
if [ $(make -j10 "$1" 2>&1 > log.txt) ]; then
[ "$1" ] && grep -i --color "error" log.txt
notify-send -u critical -t 7 "BUILD DONE"
fi
The inside of the test is common to both branches. Instead of using tee so that the output can be piped, you can just indirect the output to log.txt. If "$1" isn't empty, grep for any errors in log.txt. Either way, do the notify-send.
GNU bash, version 1.14.7(1)
I have a script is called "abc.sh"
I have to check this from abc.sh script only...
inside it I have written following statement
status=`ps -efww | grep -w "abc.sh" | grep -v grep | grep -v $$ | awk '{ print $2 }'`
if [ ! -z "$status" ]; then
echo "[`date`] : abc.sh : Process is already running"
exit 1;
fi
I know it's wrong because every time it exits as it found its own process in 'ps'
how to solve it?
how can I check that script is already running or not from that script only ?
An easier way to check for a process already executing is the pidof command.
if pidof -x "abc.sh" >/dev/null; then
echo "Process already running"
fi
Alternatively, have your script create a PID file when it executes. It's then a simple exercise of checking for the presence of the PID file to determine if the process is already running.
#!/bin/bash
# abc.sh
mypidfile=/var/run/abc.sh.pid
# Could add check for existence of mypidfile here if interlock is
# needed in the shell script itself.
# Ensure PID file is removed on program exit.
trap "rm -f -- '$mypidfile'" EXIT
# Create a file with current PID to indicate that process is running.
echo $$ > "$mypidfile"
...
Update:
The question has now changed to check from the script itself. In this case, we would expect to always see at least one abc.sh running. If there is more than one abc.sh, then we know that process is still running. I'd still suggest use of the pidof command which would return 2 PIDs if the process was already running. You could use grep to filter out the current PID, loop in the shell or even revert to just counting PIDs with wc to detect multiple processes.
Here's an example:
#!/bin/bash
for pid in $(pidof -x abc.sh); do
if [ $pid != $$ ]; then
echo "[$(date)] : abc.sh : Process is already running with PID $pid"
exit 1
fi
done
I you want the "pidof" method, here is the trick:
if pidof -o %PPID -x "abc.sh">/dev/null; then
echo "Process already running"
fi
Where the -o %PPID parameter tells to omit the pid of the calling shell or shell script. More info in the pidof man page.
Here's one trick you'll see in various places:
status=`ps -efww | grep -w "[a]bc.sh" | awk -vpid=$$ '$2 != pid { print $2 }'`
if [ ! -z "$status" ]; then
echo "[`date`] : abc.sh : Process is already running"
exit 1;
fi
The brackets around the [a] (or pick a different letter) prevent grep from finding itself. This makes the grep -v grep bit unnecessary. I also removed the grep -v $$ and fixed the awk part to accomplish the same thing.
Working solution:
if [[ `pgrep -f $0` != "$$" ]]; then
echo "Another instance of shell already exist! Exiting"
exit
fi
Edit: I checked out some comments lately, so I tried attempting same with some debugging. I will also will explain it.
Explanation:
$0 gives filename of your running script.
$$ gives PID of your running script.
pgrep searches for process by name and returns PID.
pgrep -f $0 searches by filename, $0 being the current bash script filename and returns its PID.
So, pgrep checks if your script PID ($0) is equal to current running script ($$). If yes, then the script runs normally. If no, that means there's another PID with same filename running, so it exits. The reason I used pgrep -f $0 instead of pgrep bash is that you could have multiple instances of bash running and thus returns multiple PIDs. By filename, its returns only single PID.
Exceptions:
Use bash script.sh not ./script.sh as it doesn't work unless you have shebang.
Fix: Use #!/bin/bash shebang at beginning.
The reason sudo doesn't work is that it returns pgrep returns PID of both bash and sudo, instead of returning of of bash.
Fix:
#!/bin/bash
pseudopid="`pgrep -f $0 -l`"
actualpid="$(echo "$pseudopid" | grep -v 'sudo' | awk -F ' ' '{print $1}')"
if [[ `echo $actualpid` != "$$" ]]; then
echo "Another instance of shell already exist! Exiting"
exit
fi
while true
do
echo "Running"
sleep 100
done
The script exits even if the script isn't running. That is because there's another process having that same filename. Try doing vim script.sh then running bash script.sh, it'll fail because of vim being opened with same filename
Fix: Use unique filename.
Someone please shoot me down if I'm wrong here
I understand that the mkdir operation is atomic, so you could create a lock directory
#!/bin/sh
lockdir=/tmp/AXgqg0lsoeykp9L9NZjIuaqvu7ANILL4foeqzpJcTs3YkwtiJ0
mkdir $lockdir || {
echo "lock directory exists. exiting"
exit 1
}
# take pains to remove lock directory when script terminates
trap "rmdir $lockdir" EXIT INT KILL TERM
# rest of script here
Here's how I do it in a bash script:
if ps ax | grep $0 | grep -v $$ | grep bash | grep -v grep
then
echo "The script is already running."
exit 1
fi
This allows me to use this snippet for any bash script. I needed to grep bash because when using with cron, it creates another process that executes it using /bin/sh.
I find the answer from #Austin Phillips is spot on. One small improvement I'd do is to add -o (to ignore the pid of the script itself) and match for the script with basename (ie same code can be put into any script):
if pidof -x "`basename $0`" -o $$ >/dev/null; then
echo "Process already running"
fi
pidof wasn't working for me so I searched some more and came across pgrep
for pid in $(pgrep -f my_script.sh); do
if [ $pid != $$ ]; then
echo "[$(date)] : my_script.sh : Process is already running with PID $pid"
exit 1
else
echo "Running with PID $pid"
fi
done
Taken in part from answers above and https://askubuntu.com/a/803106/802276
Use the PS command in a little different way to ignore child process as well:
ps -eaf | grep -v grep | grep $PROCESS | grep -v $$
I create a temporary file during execution.
This is how I do it:
#!/bin/sh
# check if lock file exists
if [ -e /tmp/script.lock ]; then
echo "script is already running"
else
# create a lock file
touch /tmp/script.lock
echo "run script..."
#remove lock file
rm /tmp/script.lock
fi
I have found that using backticks to capture command output into a variable, adversly, yeilds one too many ps aux results, e.g. for a single running instance of abc.sh:
ps aux | grep -w "abc.sh" | grep -v grep | wc -l
returns "1". However,
count=`ps aux | grep -w "abc.sh" | grep -v grep | wc -l`
echo $count
returns "2"
Seems like using the backtick construction somehow temporarily creates another process. Could be the reason why the topicstarter could not make this work. Just need to decrement the $count var.
I didn't want to hardcode abc.sh in the check, so I used the following:
MY_SCRIPT_NAME=`basename "$0"`
if pidof -o %PPID -x $MY_SCRIPT_NAME > /dev/null; then
echo "$MY_SCRIPT_NAME already running; exiting"
exit 1
fi
This is compact and universal
# exit if another instance of this script is running
for pid in $(pidof -x `basename $0`); do
[ $pid != $$ ] && { exit 1; }
done
The cleanest fastest way:
processAlreadyRunning () {
process="$(basename "${0}")"
pidof -x "${process}" -o $$ &>/dev/null
}
For other variants (like AIX) that don't have pidof or pgrep. Reliability is greatly improved by getting a "static" view of the process table as opposed to piping it directly to grep. Setting IFS to null will preserve the carriage returns when the ps output is assigned to a variable.
#!/bin/ksh93
IFS=""
script_name=$(basename $0)
PSOUT="$(ps ax)"
ANY_TEXT=$(echo $PSOUT | grep $script_name | grep -vw $$ | grep $(basename $SHELL))
if [[ $ANY_TEXT ]]; then
echo "Process is already running"
echo "$ANY_TEXT"
exit
fi
[ "$(pidof -x $(basename $0))" != $$ ] && exit
https://github.com/x-zhao/exit-if-bash-script-already-running/blob/master/script.sh
This question already has answers here:
Pipe output and capture exit status in Bash
(16 answers)
Closed 5 years ago.
How do I get the correct return code from a unix command line application after I've piped it through another command that succeeded?
In detail, here's the situation :
$ tar -cEvhf - -I ${sh_tar_inputlist} | gzip -5 -c > ${sh_tar_file} -- when only the tar command fails $?=0
$ echo $?
0
And, what I'd like to see is:
$ tar -cEvhf - -I ${sh_tar_inputlist} 2>${sh_tar_error_file} | gzip -5 -c > ${sh_tar_file}
$ echo $?
1
Does anyone know how to accomplish this?
Use ${PIPESTATUS[0]} to get the exit status of the first command in the pipe.
For details, see http://tldp.org/LDP/abs/html/internalvariables.html#PIPESTATUSREF
See also http://cfajohnson.com/shell/cus-faq-2.html for other approaches if your shell does not support $PIPESTATUS.
Look at $PIPESTATUS which is an array variable holding exit statuses. So ${PIPESTATUS[0]} holds the exit status of the first command in the pipe, ${PIPESTATUS[1]} the exit status of the second command, and so on.
For example:
$ tar -cEvhf - -I ${sh_tar_inputlist} | gzip -5 -c > ${sh_tar_file}
$ echo ${PIPESTATUS[0]}
To print out all statuses use:
$ echo ${PIPESTATUS[#]}
Here is a general solution using only POSIX shell and no temporary files:
Starting from the pipeline:
foo | bar | baz
exec 4>&1
error_statuses=`((foo || echo "0:$?" >&3) |
(bar || echo "1:$?" >&3) |
(baz || echo "2:$?" >&3)) 3>&1 >&4`
exec 4>&-
$error_statuses contains the status codes of any failed processes, in random order, with indexes to tell which command emitted each status.
# if "bar" failed, output its status:
echo $error_statuses | grep '1:' | cut -d: -f2
# test if all commands succeeded:
test -z "$error_statuses"
# test if the last command succeeded:
echo $error_statuses | grep '2:' >/dev/null
As others have pointed out, some modern shells provide PIPESTATUS to get this info. In classic sh, it's a bit more difficult, and you need to use a fifo:
#!/bin/sh
trap 'rm -rf $TMPDIR' 0
TMPDIR=$( mktemp -d )
mkfifo ${FIFO=$TMPDIR/fifo}
cmd1 > $FIFO &
cmd2 < $FIFO
wait $!
echo The return value of cmd1 is $?
(Well, you don't need to use a fifo. You can have the commands early in the pipe echo a status variable and eval that in the main shell, redirecting file descriptors all over the place and basically bending over backwards to check things, but using a fifo is much, much easier.)