Why doesn't this Bash script error out? - bash

Here's my Bash script:
#!/bin/bash -e
if [ == "" ]; then
echo "BAD"
exit 1
fi
echo "OK"
And here's the output:
./test.sh: line 3: [: ==: unary operator expected
OK
The return code is 0.
There's an obvious syntax error at line 3. Rather than raise the syntax error and refuse to run the script, somehow the script just runs and reports the syntax error at run time. The -e flag hasn't protected me from this - apparently a syntax error in an if statement constitutes a false condition rather than a reason to immediately exit the program. BUT, somehow Bash has parsed that whole if ... fi block, so after ignoring the bad line, execution somehow resumes not at the next syntactically correct line but after the end of the block?
I have two questions:
What is going on?
How can I protect myself from this behaviour in future?

if runs the command [, and just examines its return code. Bash doesn't know nor care about the syntax for the [ command.
You can put some other command there, and Bash still won't know anything about its particular syntax.
Two things come to mind:
Using [[ instead of [: Bash does know and care about its syntax.
Using ShellCheck1; online, manually or within your favourite editor.
Both if and -e deal with exit codes: If it's non-zero if won't let you into the then block, and -e will exit. You can't really have both those behaviours at once. (Well, it seems [ exits with different codes for false results (1) and syntax errors (2), so it might be possible to ‘detect’ syntax errors.)
1Or some other tool, but that's the only one of which I know. Suggestions welcome.

You don't have a shell syntax error here.
You have an error in the arguments to the [ command/builtin.
The reason set -e doesn't help here is because that's explicitly not what it is supposed to do. set -e would become entirely useless if you couldn't have if statements in your code with it on. Just think about that.
If you look in the POSIX spec for what the -e/errexit flag does you see this description:
-e
When this option is on, when any command fails (for any of the reasons listed in Consequences of Shell Errors or by returning an exit status greater than zero), the shell immediately shall exit with the following exceptions:
The failure of any individual command in a multi-command pipeline shall not cause the shell to exit. Only the failure of the pipeline itself shall be considered.
The -e setting shall be ignored when executing the compound list following the while, until, if, or elif reserved word, a pipeline beginning with the ! reserved word, or any command of an AND-OR list other than the last.
If the exit status of a compound command other than a subshell command was the result of a failure while -e was being ignored, then -e shall not apply to this command.
This requirement applies to the shell environment and each subshell environment separately. For example, in:
set -e; (false; echo one) | cat; echo two
See point two there? That's your situation.
The reason the shell continues to execute is back to the "not a shell syntax error". You have a command error. The [ command/builtin attempted to parse its arguments and failed. It then returned an error return code. The if caught that, skipped its body and returned true (as per documented behavior of if when no conditions return true). So the shell script continued normally.
As I indicated in my comment, however, if you had used [[ (which is a bash-ism and a language construct) then your script would have had a syntax error and would have exited immediately on that line (at least in my tests).

From the bash man page
-e
Exit immediately if a pipeline (which may consist of a single simple command), a subshell command enclosed in parentheses, or one of the commands executed as part of a command list enclosed by braces (see SHELL GRAMMAR above) exits with a non-zero status. The shell does not exit if the command that fails is part of the command list immediately following a while or until keyword, part of the test following the if or elif reserved words, part of any command executed in a && or ││ list except the command following the final && or ││, any command in a pipeline but the last, or if the command's return value is being inverted with !. A trap on ERR, if set, is executed before the shell exits. This option applies to the shell environment and each subshell environment separately (see COMMAND EXECUTION ENVIRONMENT above), and may cause subshells to exit before executing all the commands in the subshell.
Emphasis mine.

First, that isn't a syntax error. You are simply providing bad arguments to the [ command.
Second, the exit status of a command in the list following the if keyword are ignored for the purpose of the -e option.

Related

Execute command that results from execution of a script whose name is in a variable

When posting this question originally, I totally misworded it, obtaining another, reasonable but different question, which was correctly answered here.
The following is the correct version of the question I originally wanted to ask.
In one of my Bash scripts, there's a point where I have a variable SCRIPT which contains the /path/to/an/exe which, when executed, outputs a line to be executed.
What my script ultimately needs to do, is executing that line to be executed. Therefore the last line of the script is
$($SCRIPT)
so that $SCRIPT is expanded to /path/to/an/exe, and $(/path/to/an/exe) executes the executable and gives back the line to be executed, which is then executed.
However, running shellcheck on the script generates this error:
In setscreens.sh line 7:
$($SCRIPT)
^--------^ SC2091: Remove surrounding $() to avoid executing output.
For more information:
https://www.shellcheck.net/wiki/SC2091 -- Remove surrounding $() to avoid e...
Is there a way I can rewrite that $($SCRIPT) in a more appropriate way? eval does not seem to be of much help here.
If the script outputs a shell command line to execute, the correct way to do that is:
eval "$("$SCRIPT")"
$($SCRIPT) would only happen to work if the command can be completely evaluated using nothing but word splitting and pathname expansion, which is generally a rare situation. If the program instead outputs e.g. grep "Hello World" or cmd > file.txt then you will need eval or equivalent.
You can make it simple by setting the command to be executed as a positional argument in your shell and execute it from the command line
set -- "$SCRIPT"
and now run the result that is obtained by expansion of SCRIPT, by doing below on command-line.
"$#"
This works in case your output from SCRIPT contains multiple words e.g. custom flags that needs to be run. Since this is run in your current interactive shell, ensure the command to be run is not vulnerable to code injection. You could take one step of caution and run your command within a sub-shell, to not let your parent environment be affected by doing ( "$#" ; )
Or use shellcheck disable=SCnnnn to disable the warning and take the occasion to comment on the explicit intention, rather than evade the detection by cloaking behind an intermediate variable or arguments array.
#!/usr/bin/env bash
# shellcheck disable=SC2091 # Intentional execution of the output
"$("$SCRIPT")"
By disabling shellcheck with a comment, it clarifies the intent and tells the questionable code is not an error, but an informed implementation design choice.
you can do it in 2 steps
command_from_SCRIPT=$($SCRIPT)
$command_from_SCRIPT
and it's clean in shellcheck

What is exclamation mark at beginning of line doing in this BASH snippet?

I understand history expansion in C-shell and BASH, but I don't understand why the exclamation mark on the line with the read command is being used in this snippet from some source I'm looking at:
#!/bin/bash
set -e
. "$(dirname "$0")/includes.sh"
! read -d '' SOME_VAR <<"EOT"
some ASCII art
EOT
echo -e "\033[95m$SOME_VAR\033[0m"
Why would you negate the return value of the read command (I think that is its effect, not history expansion)? Are there any possible error conditions, besides out-of-memory? The only one I can think of would be EINTR, which I think would be an abort condition (SIGINT or SIGHUP). And why would you double-quote the starting heredoc marker (EOT in this case), and not the ending marker?
It's from a major open source project, so I was guessing the author had some reason for doing this that I haven't been able to discern.
The set -e command tells bash to exit if a pipeline returns a non-zero exit status (basically if a command fails).
The ! negates the exit status, but it also inhibits the effect of set -e. The idea is that if a command is being executed as part of a condition, you don't want to terminate the shell. If it's preceded by !, the shell effectively assumes that it's being executed as a condition (even if the result is ignored).
Quoting the bash manual:
The shell does not exit if the command that fails is part of the
command list immediately following a 'while' or 'until' keyword, part
of the test in an 'if' statement, part of any command executed in a
'&&' or '||' list except the command following the final '&&' or '||',
any command in a pipeline but the last, or if the command's return
status is being inverted with '!'.
https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html
(You're correct that the ! has nothing to do with history substitution in this context.)
The return code for the read built-in is
zero, unless end-of-file is encountered, read times out (in which case it's
greater than 128), a variable assignment error occurs,
or an invalid file descriptor is supplied as the argument to -u.
The relevant case here is end-of-file. Redirecting the input of read using <<EOT (and disabling normal termination using -d '') means that the read command will encounter an end-of-file, which would cause it to return a non-zero status. The ! prevents this from aborting the script (but the value is still assigned to $SOME_VAR).

Abort issue when error/exception occurs?

I have a bash script which calls multiple scripts (bash & python) from some directories.
I would like to get it aborted when any of the script throws an error/exception.
#!/bin/bash
/usr/bin/test.sh /usr/1/sample.sh /usr/2/temp.py
exit 0
Any suggestion on how to achieve this ?
FYI : I'm a beginner in bash scripting.
You can put set -e at the top of the script:
-e errexit If not interactive, exit immediately if any
untested command fails. The exit status of a com‐
mand is considered to be explicitly tested if the
command is used to control an if, elif, while, or
until; or if the command is the left hand operand
of an “&&” or “||” operator.
This will only work if one of your commands exit with an exit code of non-zero on failure. Well-behaved programs should always exit with 0 only on success, and if yours don't, you probably want to fix that.
I'm not entirely sure what you expect this to do:
/usr/bin/test.sh /usr/1/sample.sh /usr/2/temp.py
Since this will run one command (/usr/bin/test.sh) with two arguments, you probably want to put them on separate lines.

Execute a line in Bash without aborting if it fails?

Is there a generic way in a bash script to "try" something but continue if it fails? The analogue in other languages would be wrapping it in a try/catch and ignoring the exception.
Specifically I am trying to source an optional satellite script file:
. $OPTIONAL_PATH
But when executing this, if $OPTIONAL_PATH doesn't exist, the whole script screeches to a halt.
I realize I could check to see if the file exists before sourcing it, but I'm curious if there is a generic reusable mechanism I can use that will ignore the error without halting.
Update: Apparently this is not normal behavior. I'm not sure why this is happening. I'm not explicitly calling set -e anywhere ($- is hB), yet it halts on the error. Here is the output I see:
./script.sh: line 36: projects/mobile.sh: No such file or directory
I added an echo "test" immediately after the source line, but it never prints, so it's not anything after that line that is exiting. I am running Mac OS 10.9.
Update 2: Nevermind, it was indeed shebanged as #!/bin/sh instead of #!/bin/bash. Thanks for the informative answer, Kaz.
Failed commands do not abort the script unless you explicitly configure that mode with set -e.
With regard to Bash's dot command, things are tricky. If we invoke bash as /bin/sh then it bails the script if the . command does not find the file. If we invoke bash as /bin/bash then it doesn't fail!
$ cat source.sh
#!/bin/sh
. nonexistent
echo here
$ ./source.sh
./source.sh: 3: .: nonexistent: not found
$ ed source.sh
35
1s/sh/bash/
wq
37
$ ./source.sh
./source.sh: line 3: nonexistent: No such file or directory
here
It does respond to set -e; if we have #!/bin/bash, and use set -e, then the echo is not reached. So one solution is to invoke bash this way.
If you want to keep the script maximally portable, it looks like you have to do the test.
The behavior of the dot command aborting the script is required by POSIX. Search for the "dot" keyword here. Quote:
If no readable file is found, a non-interactive shell shall abort; an interactive shell shall write a diagnostic message to standard error, but this condition shall not be considered a syntax error.
Arguably, this is the right thing to do, because dot is used for including pieces of the script. How can the script continue when a whole chunk of it has not been found?
Otherwise arguably, this is braindamaged behavior inconsistent with the treatment of other commands, and so Bash makes it consistent in its non-POSIX-conforming mode. If programmers want a command to fail, they can use set -e.
I tend to agree with Bash. The POSIX behavior is actually more broken than initially meets the eye, because this also doesn't work the way you want:
if . nonexistent ; then
echo loaded
fi
Even if the command is tested, it still aborts the script when it bails.
Thank GNU-deness we have alternative utilities, with source code.
You have several options:
Make sure set -e wasn't used, or turn it off with set +e. Your bash script should not exit by default simply because the . command failed.
Test that the file exists prior to sourcing.
[ -f "$OPTIONAL_PATH" ] && . "$OPTIONAL_PATH"
This option is complicated by the fact that if $OPTIONAL_PATH does not contain
any slashes, . will still try to find the file in your path.
If you want to keep set -e on, "hide" the failure like this:
. "$OPTIONAL_PATH" || true
Even if the source fails, the exit status of the command list as a whole will be 0, due to the || true.
(Much of this is covered [better] by Kaz's answer, especially the references to the POSIX standard, but I wasn't sure when or if he would undelete his answer.)
This is not the default behavior. Did you set -e or use #!/bin/bash -e anywhere in your script, to make it automatically exit on failure?
If so, you can use
. $OPTIONAL_PATH || true
to continue anyways.

What does $? mean in a shell if statement?

What is the meaning of the following statement in shell script?
if ($?REGRESS) then
....
endif
from where the given function is taking input from? This is a part of an old script which I am not able to understand.
From the csh the man page:
$?name
${?name}
Substitutes the string `1' if name is set, `0' if it is not.
But stop using csh (obligatory comment for future readers who may have somehow missed the memo that using csh is bad for you health).
$? is a special variable. It stores the exit status of last command. If the last command runs successfully then it will be 0 and some other value for failure.

Resources