This question already has answers here:
Meaning of $? (dollar question mark) in shell scripts
(8 answers)
Closed 4 years ago.
There is this line in a shell script i have seen:
grep -e ERROR ${LOG_DIR_PATH}/${LOG_NAME} > /dev/null
if [ $? -eq 0 ]
$? is the exit status of the most recently-executed command; by convention, 0 means success and anything else indicates failure. That line is testing whether the grep command succeeded.
The grep manpage states:
The exit status is 0 if selected lines are found, and 1 if not found. If an error occurred the exit status is 2. (Note: POSIX error handling code should check for '2' or greater.)
So in this case it's checking whether any ERROR lines were found.
It's checking the return value ($?) of grep. In this case it's comparing it to 0 (success).
Usually when you see something like this (checking the return value of grep) it's checking to see whether the particular string was detected. Although the redirect to /dev/null isn't necessary, the same thing can be accomplished using -q.
It is an extremely overused way to check for the success/failure of a command. Typically, the code snippet you give would be refactored as:
if grep -e ERROR ${LOG_DIR_PATH}/${LOG_NAME} > /dev/null; then
...
fi
(Although you can use 'grep -q' in some instances instead of redirecting to /dev/null, doing so is not portable. Many implementations of grep do not support the -q option, so your script may fail if you use it.)
Related
What is the correct way to output an exit status in bash? As far as I know, the exit status called by $? corresponds to the status of the last command executed.
The script being worked on has a few conditional checks on the files fed as arguments, for example, a check on whether any files were named at all or if a file exists or not.
So I have conditional statements like this:
if [ $# -eq 0 ] ; then
echo "No file name(s) given! \nExit status=$?"
exit
if [ ! -e "$fName" ] ; then
echo "$fName does not exist! \nExit status=$?"
exit
But these return an exit status of 0. I'm not entirely sure even what exit codes would be appropriate for each of these situations, but I think both 1 would work for both based on this. Should I just hard-code the 1 into the conditional statement or change the logic so unix outputs an error code? Also, what command would the 0 that I get for the above example be the exit code for?
In both of these cases, the last command executed was [, which exited with 0 status ("success") to land you in the "true" block.
Yes, you need to specify a non-zero exit code for the exit command.
Pedantically "unix" does not output the error code: it's the shell running your script that exits with the specified status.
I want to protect most of my bash script with set -e, in order to fail early and loudly when an error is detected during the script's processing. However, I still want to be able to run some commands that are actually expected to fail, such as using grep to evaluate the presence/absence of some file content that is used to direct the control flow of the rest of the script. How can I run grep within a set -e context, such that A) grep is allowed to fail and B) grep's exist status is recorded for access by the rest of the script?
In ordinary POSIX sh, I would do something like:
grep 'needle' haystack >/dev/null
if [ "$?" -eq 0 ]; then
handle_grep_results
else
handle_grep_no_results
fi
However, when set -e is specified before this section, then the script exits early whenever grep fails to find the needle. One way to work around this is to temporarily disable the protections with set +e, and then re-enable them after the section, but I would prefer to leave the protections on, if that makes sense. Is this possible with bash?
You can simply check the return status of grep:
grep -q luck myfile.txt || no_luck=1
Shell utilities use the return status to communicate with the shell; what they are communicating is not necessarily an error condition. As the grep example shows, it can be a simple boolean. In fact, the [[ builtin (and its friends [ and test) do precisely that: use the status code to return a boolean result. That doesn't make those utilities "flakey".
set -e ignores non-zero status returns from commands executed within a conditional or on the left-hand side of a || or && connector, which makes it possible to use set -e.
Having said that, -e is a very blunt tool and its use in production code is not generally recommended. You can always explicitly fail using ||:
important_setup || exit 1
Your if command seems to contain a couple of typos (an extra # and a missing space before ]), but more generally, you should understand that the very purpose of if is to run a command and check its exit code. Anything which looks like
command
if [ $? = 0 ]; then
is more compactly and idiomatically written
if command; then
and in this context, a failure exit status from command is not a condition which causes set -e to terminate the script (because then you couldn't have else clauses in scripts with set -e!)
In this particular example where both the then and else blocks contain simple commands, you can simplify to the shorthand
grep -q 'needle' haystack && handle_grep_results || handle_grep_no_results
which also suggests
command || true
when you simply don't care whether command succeeded or not in a script with set -e.
(Notice also grep -q over grep >/dev/null - the former implies -m 1, i.e. grep can close the file and return success as soon as it finds the first match.)
Example script:
#!/bin/bash
printf '1\n1\n1\n1\n' | ./script2*.sh >/dev/null 2>/dev/null
Shellcheck returns the following:
In script1.sh line 3:
printf '1\n1\n1\n1\n' | ./script2*.sh >/dev/null 2>/dev/null
^-- SC2211: This is a glob used as a command name. Was it supposed to be in ${..}, array, or is it missing quoting?
According to https://github.com/koalaman/shellcheck/wiki/SC2211, there should be no exceptions to this rule.
Specifically, it suggests "If you want to specify a command name via glob, e.g. to not hard code version in ./myprogram-*/foo, expand to array or parameters first to allow handling the cases of 0 or 2+ matches."
The reason I'm using the glob in the first place is that I append or change the date to any script that I have just created or changed. Interestingly enough, when I use "bash script2*.sh" instead of "./script2*.sh" the complaint goes away.
Have I fixed the problem or I am tricking shellcheck into ignoring a problem that should not be ignored? If I am using bad bash syntax, how might I execute another script that needs to be referenced to using a glob the proper way?
The problem with this is that ./script2*.sh may end up running
./script2-20171225.sh ./script2-20180226.sh ./script2-copy.sh
which is a strange and probably unintentional thing to do, especially if the script is confused by such arguments, or if you wanted your most up-to-date file to be used. Your "fix" has the same fundamental problem.
The suggestion you mention would take the form:
array=(./script2*.sh)
[ "${#array[#]}" -ne 1 ] && { echo "Multiple matches" >&2; exit 1; }
"${array[0]}"
and guard against this problem.
Since you appear to assume that you'll only ever have exactly one matching file to be invoked without parameters, you can turn this into a function:
runByGlob() {
if (( $# != 1 ))
then
echo "Expected exactly 1 match but found $#: $*" >&2
exit 1
elif command -v "$1" > /dev/null 2>&1
then
"$1"
else
echo "Glob is not a valid command: $*" >&2
exit 1
fi
}
whatever | runByGlob ./script2*.sh
Now if you ever have zero or multiple matching files, it will abort with an error instead of potentially running the wrong file with strange arguments.
I am new to shell script. I have a scenario where i need to get the exit status of one shell script and pass its value as input(if exit code is zero/otherwise exit the main shell script file) to the next shell script and continue until it executes all the scripts. Below is the code i tried.but its not working as expected.
status=`run1.sh`
if [[ status -eq 0 ]]; then
status=`run2.sh`
else
exit 1
fi
if [[ status -eq 0 ]];then
status=`run3.sh`
else
exit 2
fi
Its running successfully for first 2 scripts. It's failing on second if block, even though the output value of run2.sh is 0. I searched in google, its suggesting to use {PIPESTATUS[#]}, I tried it by replacing exit 1 with pipestatus and pass pipestatus in the second if block like below.
status=`run1.sh`
if [[ status -eq 0 ]]; then
status=`run2.sh`
else
exit ${PIPESTATUS[1]} ## pipestatus[1]- is run2.sh output value
fi
if [[ ${PIPESTATUS[1]} -eq 0 ]];then
status=`run3.sh`
else
exit 2
fi
I think i am not clear on how to use pipestatus. I would appreciate if anyone can provide me some example to my scenario.
status is a static string; you are not examining the variable you created at all.
There is no pipe here so PIPESTATUS does not come into play. It's for code like
one | two
where traditionally the exit status of one wasn't available to the shell; Bash changed that by exposing results from every process in a pipeline.
Having your scripts print the number zero to signal success is not how it's usually done. A command produces both output and a status code (the number you give as the argument to exit, or in a function, to return), and code would usually examine the latter. It is exposed in $? but the shell's control structures implicitly check it under the hood, so your code would look like
if run1.sh; then
if run2.sh; then
if run3.sh; then
...
provided you change them to produce a useful exit code. Though this could be further simplified to
run1.sh || exit
run2.sh || exit
run3.sh || exit
or even to
set -e
run1.sh
run2.sh
run3.sh
what is the difference if I put test or without test in a .ksh file?
if test $1 -ne 0 ; then ....
and
if $1 -ne 0 ; then ....
Many Thanks
I actually think this is an important question, as it highlights some important rules in shell programming:
Every command in the shell returns an true (0) or false exit code
Control structures in the shell do not require comparisons
Every command in the shell returns an exit code
Any properly coded command in the shell will return 0 for success,
and non-zero for failure. While there is only one way to succeed, but always more than way to fail.
Example:
$ no-such-command || echo no $?
ksh[1]: no-such-command: not found [No such file or directory]
no 127
$
The exit status of a command is caught in the pseudo variable $? and is available until you complete another command.
This exit status is used in control structures like if ... then ... fi
or until ... do ... done.
failing(){ return 2; }
failing &&
echo "It works" ||
echo "It failed with exit code $?"
results in
It failed with exit code 2
Control structures in the shell do not require comparisons
Let's start with the simplest definition
of the if command:
if compound-list
then
compound-list
fi
For the full syntax, see Section 2.9.4 Compound Commands of Shell Command Language of The Open Group Base Specifications.
Between the keywords, if, then, and fi there are two sections of
code, named compound-list.
This is shorthand for any sequence of code that would be valid in a script. The exit status of the list will be equal to the exit status of the last command in the list.
The important difference for the two lists is that the firts will determine the flow of control, while the second determines the exit status of the entire expression, when executed.
Conclusion
Any command can be used as the test in an if/then/else/fi construct.
Because we often want to test things explicitly, we often use the actual test command or its derivatives [ ... ] and [[ ... ]].
if [[ -n $1 ]]; then
echo "'$1' is a non-empty string"
fi
For complex expressions it is always preferred to wrap them in a
function to apply some abstraction.
One more trivial example:
non_empty_files_present(){
(path=${1:?directory expected}
(find ${path} -type f -size +0 | read line) 2> /dev/null
)
}
if non_empty_files_present /var/tmp; then
echo "Some files have content"
fi