I have a bash shell script doing sth like
#!/bin/bash
# construct regex from input
# set FILE according to input file
egrep "${regex}" "${FILE}" | doing stuff | sort
I want this script to write the output (a list of new line separated matches) of the command to stdout if matches are found (which it is doing). If no matches are found it needs to write out an error message to stderr and exit with exit status 3.
I tried this
#!/bin/bash
# construct regex from input
# set FILE according to input file
function check () {
if ! read > /dev/null
then
echo "error message" 1>&2
exit 3
fi
}
egrep "${regex}" "${FILE}" | doing stuff |
sort | tee >(check)
Now the correct error message is written out but the exit status "cannot escape the subshell"; the outer script is still exiting with exit status 0.
I also tried sth like
#!/bin/bash
# construct regex from input
# set FILE according to input file
if ! egrep "${regex}" "${FILE}" | doing stuff | sort
then
echo "error message" 1>&2
exit 3
fi
But here I have the problem that one of the commands in the pipe (especially sort) exits with an exit status 0
How can I get my desired exit status 3 and error message while keeping the output for normal execution and without doing all the stuff twice?
EDIT:
I can solve the problem by using
#!/bin/bash
# construct regex from input
# set FILE according to input file
if ! egrep "${regex}" "${FILE}" | doing stuff | sort | grep .
then
echo "error message" 1>&2
exit 3
fi
However I am not sure this is the best way since pipes are working in parallel...
I would use the PIPESTATUS to check the exit code of egrep:
#!/bin/bash
# construct regex from input
# set FILE according to input file
egrep "${regex}" "${FILE}" | doing stuff | sort
if [[ ${PIPESTATUS[0] != 0 ]]; then
echo "error message" 1>&2
exit 3
fi
Some context:
${PIPESTATUS[#]} is just an array wich contains the exit code of every program you chained up. $? will just give you the exit code of the last command in the pipe.
Related
I have a pre-commit hook script to check for TaskID in log message. I'm unable to make out the logic for same. In Highlighted **if** statement i need a logic to check if first letter of first line is TaskID:(multiple digits)-(space)(log message)
Pre-commit hook:
REPOS="$1"
TXN="$2"
# Check log message for proper task/bug identification
if [ -x ${REPOS}/hooks/check_log_message.sh ]; then
${REPOS}/hooks/check_log_message.sh "${REPOS}" "${TXN}" 1>&2 || exit 1
fi
exit 0
=======>>>>>check_log_message.sh
#!/bin/bash
REPOS="${1}"
TXN="${2}"
SVNLOOK=/usr/bin/svnlook
LOG_MSG_LINE1=`${SVNLOOK} log -t "${TXN}" "${REPOS}" | head -n1`
**if (echo "${LOG_MSG_LINE1}" | egrep '^[T][a][s][k][I][D][:]?[-][1-9]
[\s]*.*$' > /dev/null;) \
|| (echo "${LOG_MSG_LINE1}" | egrep '^[a-zA-Z]+[-][1-9][0-9]*[:]?[\s]*.*$'
> /dev/null;)**
then
exit 0
else
echo ""
echo "Your log message does not contain a TaskID(or bad format used)"
echo "The TaskID must be the first item on the first line of the log
message."
echo ""
echo "Proper TaskID format--> TaskID:xxx- 'Your commit message' "
exit 1
fi
I guess your question concerns using conditionals in Bash.
You can work with the exit codes of programs directly.
For example, if egrep matches something, it exits with code 0 which means success,
otherwise it exits with non-zero,
which means failure.
And you can use this is in conditions, for example:
if command; then
echo success
else
echo failure
fi
Where command can be a pipeline, for example:
if ${SVNLOOK} log -t "${TXN}" "${REPOS}" | head -n1 | egrep -q ^TaskID:
then
exit 0
fi
This means if the first line of the log starts with TaskID:, then exit with 0. Instead of an if statement, you could also use a shorter form with && like this:
${SVNLOOK} log -t "${TXN}" "${REPOS}" | head -n1 | egrep -q ^TaskID: && exit 0
In both examples I used -q with egrep, to suppress the output (the matched line), as I guess you probably don't need it.
The full script with the more complete pattern:
#!/bin/bash
REPOS="${1}"
TXN="${2}"
SVNLOOK=/usr/bin/svnlook
${SVNLOOK} log -t "${TXN}" "${REPOS}" | head -n1 | egrep -q '^TaskID:[0-9][0-9]*- ' && exit 0
echo ""
echo "Your log message does not contain a TaskID(or bad format used)"
echo "The TaskID must be the first item on the first line of the log
message."
echo ""
echo "Proper TaskID format--> TaskID:xxx- 'Your commit message' "
exit 1
Alternatively, I modified my script with help of a colleague.
REPOS="$1"
TXN="$2"
# Make sure that the log message contains some text.
SVNLOOK=/usr/bin/svnlook
LOGMSG=$($SVNLOOK log -t "$TXN" "$REPOS")
# check if any comment has supplied by the commiter
if [ -z "$LOGMSG" ]; then
echo "Your commit was blocked because it have no comments." 1>&2
exit 1
fi
#check minimum size of text
if [ ${#LOGMSG} -lt 15 ]; then
echo "Your Commit was blocked because the comments does not meet minimum length requirements (15 letters)." 1>&2
exit 1
fi
# get TaskID by regex
TaskID=$(expr "$LOGMSG" : '\([#][0-9]\{1,9\}[:][" "]\)[A-Za-z0-9]*')
# Check if task id was found.
if [ -z "$TaskID" ]; then
echo "" 1>&2
echo "No Task id found in log message \"$LOGMSG\"" 1>&2
echo "" 1>&2
echo "The TaskID must be the first item on the first line of the log message." 1>&2
echo "" 1>&2
echo "Proper TaskID format--> #123- 'Your commit message' " 1>&2
exit 1
fi
The following script calls another program reading its output in a while loop (see Bash - How to pipe input to while loop and preserve variables after loop ends):
while read -r col0 col1; do
# [...]
done < <(other_program [args ...])
How can I check for the exit code of other_program to see if the loop was executed properly?
Note: ls -d / /nosuch is used as an example command below, because it fails (exit code 1) while still producing stdout output (/) (in addition to stderr output).
Bash v4.2+ solution:
ccarton's helpful answer works well in principle, but by default the while loop runs in a subshell, which means that any variables created or modified in the loop will not be visible to the current shell.
In Bash v4.2+, you can change this by turning the lastpipe option on, which makes the last segment of a pipeline run in the current shell;
as in ccarton's answer, the pipefail option must be set to have $? reflect the exit code of the first failing command in the pipeline:
shopt -s lastpipe # run the last segment of a pipeline in the current shell
shopt -so pipefail # reflect a pipeline's first failing command's exit code in $?
ls -d / /nosuch | while read -r line; do
result=$line
done
echo "result: [$result]; exit code: $?"
The above yields (stderr output omitted):
result: [/]; exit code: 1
As you can see, the $result variable, set in the while loop, is available, and the ls command's (nonzero) exit code is reflected in $?.
Bash v3+ solution:
ikkachu's helpful answer works well and shows advanced techniques, but it is a bit cumbersome.
Here is a simpler alternative:
while read -r line || { ec=$line && break; }; do # Note the `|| { ...; }` part.
result=$line
done < <(ls -d / /nosuch; printf $?) # Note the `; printf $?` part.
echo "result: [$result]; exit code: $ec"
By appending the value of $?, the ls command's exit code, to the output without a trailing \n (printf $?), read reads it in the last loop operation, but indicates failure (exit code 1), which would normally exit the loop.
We can detect this case with ||, and assign the exit code (that was still read into $line) to variable $ec and exit the loop then.
On the off chance that the command's output doesn't have a trailing \n, more work is needed:
while read -r line ||
{ [[ $line =~ ^(.*)/([0-9]+)$ ]] && ec=${BASH_REMATCH[2]} && line=${BASH_REMATCH[1]};
[[ -n $line ]]; }
do
result=$line
done < <(printf 'no trailing newline'; ls /nosuch; printf "/$?")
echo "result: [$result]; exit code: $ec"
The above yields (stderr output omitted):
result: [no trailing newline]; exit code: 1
At least one way would be to redirect the output of the background process through a named pipe. This would allow to pick up its PID and then get the exit status through waiting on the PID.
#!/bin/bash
mkfifo pipe || exit 1
(echo foo ; exit 19) > pipe &
pid=$!
while read x ; do echo "read: $x" ; done < pipe
wait $pid
echo "exit status of bg process: $?"
rm pipe
If you can use a direct pipe (i.e. don't mind the loop being run in a subshell), you could use Bash's PIPESTATUS, which contains the exit codes of all commands in the pipeline:
(echo foo ; exit 19) | while read x ; do
echo "read: $x" ; done;
echo "status: ${PIPESTATUS[0]}"
A simple way is to use the bash pipefail option to propagate the first error code from a pipeline.
set -o pipefail
other_program | while read x; do
echo "Read: $x"
done || echo "Error: $?"
Another way is to use coproc (requires 4.0+).
coproc other_program [args ...]
while read -r -u ${COPROC[0]} col0 col1; do
# [...]
done
wait $COPROC_PID || echo "Error exit status: $?"
coproc frees you from having to setup asynchronicity and stdin/stdout redirection that you'd otherwise need to do in an equivalent mkfifo.
I would like to be able to create a bash function that can read the exit code of the command before the pipe. I'm not sure it is possible to have access to that.
echo "1" | grep 2 returns a 1 status code
echo "1" | grep 1 returns a 0 status code
Now I would like to add a third command to read the status, with a pipe:
echo "1" | grep 2 | echo $? will echo "0", even if the status code is 1.
I know I can use the echo "1" | grep 2 && echo "0" || echo "1", but I would prefer to write it using a pipe.
Is they anyway to do that (it would be even better if it was working on most shells, like bash, sh, and zsh)
You're going to have to get the exit status before the next stage of the pipeline. Something like
exec 3> debug.txt
{ echo "1"; echo "$?" >&3; } | long | command | here
You can't (easily) encapsulate this in a function, since it would require passing a properly quoted string and executing it via eval:
debug () {
eval "$#"
echo $? >&3
}
# It looks easy in this example, but it won't take long to find
# an example that breaks it.
debug echo 1 | long | command | here
You have to write the exit status to a different file descriptor, otherwise it will interfere with the output sent to the next command in the pipeline.
In bash you can do this with the PIPESTATUS variable
echo "1" | grep 1
echo ${PIPESTATUS[0]} # returns 0
echo "1" | grep 2
echo ${PIPESTATUS[0]} # returns 0
echo "1" | grep 2
echo ${PIPESTATUS[1]} # returns 1
I am writing a script that will take in 3 outputs and then search all files within a predefined path. However, my grep command seems to be breaking the script with error code 123. I have been staring at it for a while and cannot really seem the error so I was hoping someone could point out my error. Here is the code:
#! /bin/bash -e
#Check if path exists
if [ -z $ARCHIVE ]; then
echo "ARCHIVE NOT SET, PLEASE SET TO PROCEED."
echo "EXITING...."
exit 1
elif [ $# -ne 3 ]; then
echo "Illegal number of arguments"
echo "Please enter the date in yyyy mm dd"
echo "EXITING..."
exit 1
fi
filename=output.txt
#Simple signal handler
signal_handler()
{
echo ""
echo "Process killed or interrupted"
echo "Cleaning up files..."
rm -f out
echo "Finsihed"
exit 1
}
trap 'signal_handler' KILL
trap 'signal_handler' TERM
trap 'signal_handler' INT
echo "line 32"
echo $1 $2 $3
#Search for the TimeStamp field and replace the / and : characters
find $ARCHIVE | xargs grep -l "TimeStamp: $2/$3/$1"
echo "line 35"
fileSize=`wc -c out.txt | cut -f 1 -d ' '`
echo $fileSize
if [ $fileSize -ge 1 ]; then
echo "no"
xargs -n1 basename < $filename
else
echo "NO FILES EXIST"
fi
I added the echo's to know where it was breaking. My program prints out line 32 and the args but never line 35. When I check the exit code I get 123.
Thanks!
Notes:
ARCHIVE is set to a test directory, i.e. /home/'uname'/testDir
$1 $2 $3 == yyyy mm dd (ie a date)
In testDir there are N number of directories. Inside these directories there are data files that have contain data as well as a time tag. The time tag is of the following format: TimeStamp: 02/02/2004 at 20:38:01
The scripts goal is to find all files that have the date tag you are searching for.
Here's a simpler test case that demonstrates your problem:
#!/bin/bash -e
echo "This prints"
true | xargs false
echo "This does not"
The snippet exits with code 123.
The problem is that xargs exits with code 123 if any command fails. When xargs exits with non-zero status, -e causes the script to exit.
The quickest fix is to use || true to effectively ignore xargs' status:
#!/bin/bash -e
echo "This prints"
true | xargs false || true
echo "This now prints too"
The better fix is to not rely on -e, since this option is misleading and unpredictable.
xargs makes the error code 123 when grep returns a nonzero code even just once. Since you're using -e (#!/bin/bash -e), bash would exit the script when one of its commands return a nonzero exit code. Not using -e would allow your code to continue. Just disabling it on that part can be a solution too:
set +e ## Disable
find "$ARCHIVE" | xargs grep -l "TimeStamp: $2/$1/$3" ## If one of the files doesn't match the pattern, `grep` would return a nonzero code.
set -e ## Enable again.
Consider placing your variables around quotes to prevent word splitting as well like "$ARCHIVE".
-d '\n' may also be required if one of your files' filename contain spaces.
find "$ARCHIVE" | xargs -d '\n' grep -l "TimeStamp: $2/$1/$3"
Never coded on bash but need something urgent. Sorry if this is not the norm but would really like to get some help.
I have some messages that are thrown to stdout, Depending on the message type (the message is a string with the word "found") I need the bash script to beep.
So far I've come up with this.
output=$(command 1) # getting stdout stream?
while [ true ]; do
if [ "$output" = "found" ]; then # if the stdout has the word "found"
echo $(echo -e '\a') # this makes the beep sound
fi
done
I'm not sure where/how to add grep or awk command to check for the string that has the word "found" and only return "found" so that in the if condition it can check against that word.
Thanks!
You can do something as simple as:
command | grep -q 'found' && echo -e '\a'
If the output of command contains the text "found", then grep will return with a zero exit status, so the echo command will be executed, causing the beep.
If the output does not contain "found", grep will exit with status 1, and will not result in the echo.
Depending on what you need to make the beep work, just replace anything after the &&. The general syntax would be something like:
command | grep -q "$SEARCH" && command_if_found || command_if_not_found