I am trying to automate our application backup. Part of the process is to check the exit status of egrep in an if statement:
if [ ! -f /opt/apps/SiteScope_backup/sitescope_configuration.zip ] ||
[ egrep -i -q "error|warning|fatal|missing|critical" "$File" ]
then
echo "testing"
fi
I expected it to output testing because the file exists and egrep returns success, but instead I'm getting an error:
-bash: [: too many arguments
I tried with all kinds of syntax - additional brackets, quotes etc but error still persists.
Please help me in understanding where I am going wrong.
You are making the common mistake of assuming that [ is part of the if statement's syntax. It is not; the syntax of if is simply
if command; then
: # ... things which should happen if command's result code was 0
else
: # ... things which should happen otherwise
fi
One of the common commands we use is [ which is an alias for the command test. It is a simple command for comparing strings, numbers, and files. It accepts a fairly narrow combination of arguments, and tends to generate confusing and misleading error messages if you don't pass it the expected arguments. (Or rather, the error messages are adequate and helpful once you get used to it, but they are easily misunderstood if you're not used.)
Here, you want to check the result of the command egrep:
if [ ! -f /opt/apps/SiteScope_backup/sitescope_configuration.zip ] ||
egrep -i -q "error|warning|fatal|missing|critical" "$File"
then
echo "testing"
fi
In the general case, command can be a pipeline or a list of commands; then, the exit code from the final command is the status which if will examine, similarly to how the last command in a script decides the exit status from the script.
These compound commands can be arbitrarily complex, like
if read thing
case $thing in
'' | 'quit') false;;
*) true;;
esac
then ...
but in practice, you rarely see more than a single command in the if statement (though it's not unheard of; your compound statement with || is a good example!)
Just to spell this out,
if [ egrep foo bar ]
is running [ aka test on the arguments egrep foo bar. But [ without options only accepts a single argument, and then checks whether or not that argument is the empty string. (egrep is clearly not an empty string. Quoting here is optional, but would perhaps make it easier to see:
if [ "egrep" ]; then
echo "yes, 'egrep' is not equal to ''"
fi
This is obviously silly in isolation, but should hopefully work as an illustrative example.)
The historical reasons for test as a general kitchen sink of stuff the authors didn't want to make part of the syntax of if is one of the less attractive designs of the original Bourne shell. Bash and zsh offer alternatives which are less unwieldy (like the [[ double brackets in bash), and of course, POSIX test is a lot more well-tempered than the original creation from Bell Labs.
Related
I am trying to automate our application backup. Part of the process is to check the exit status of egrep in an if statement:
if [ ! -f /opt/apps/SiteScope_backup/sitescope_configuration.zip ] ||
[ egrep -i -q "error|warning|fatal|missing|critical" "$File" ]
then
echo "testing"
fi
I expected it to output testing because the file exists and egrep returns success, but instead I'm getting an error:
-bash: [: too many arguments
I tried with all kinds of syntax - additional brackets, quotes etc but error still persists.
Please help me in understanding where I am going wrong.
You are making the common mistake of assuming that [ is part of the if statement's syntax. It is not; the syntax of if is simply
if command; then
: # ... things which should happen if command's result code was 0
else
: # ... things which should happen otherwise
fi
One of the common commands we use is [ which is an alias for the command test. It is a simple command for comparing strings, numbers, and files. It accepts a fairly narrow combination of arguments, and tends to generate confusing and misleading error messages if you don't pass it the expected arguments. (Or rather, the error messages are adequate and helpful once you get used to it, but they are easily misunderstood if you're not used.)
Here, you want to check the result of the command egrep:
if [ ! -f /opt/apps/SiteScope_backup/sitescope_configuration.zip ] ||
egrep -i -q "error|warning|fatal|missing|critical" "$File"
then
echo "testing"
fi
In the general case, command can be a pipeline or a list of commands; then, the exit code from the final command is the status which if will examine, similarly to how the last command in a script decides the exit status from the script.
These compound commands can be arbitrarily complex, like
if read thing
case $thing in
'' | 'quit') false;;
*) true;;
esac
then ...
but in practice, you rarely see more than a single command in the if statement (though it's not unheard of; your compound statement with || is a good example!)
Just to spell this out,
if [ egrep foo bar ]
is running [ aka test on the arguments egrep foo bar. But [ without options only accepts a single argument, and then checks whether or not that argument is the empty string. (egrep is clearly not an empty string. Quoting here is optional, but would perhaps make it easier to see:
if [ "egrep" ]; then
echo "yes, 'egrep' is not equal to ''"
fi
This is obviously silly in isolation, but should hopefully work as an illustrative example.)
The historical reasons for test as a general kitchen sink of stuff the authors didn't want to make part of the syntax of if is one of the less attractive designs of the original Bourne shell. Bash and zsh offer alternatives which are less unwieldy (like the [[ double brackets in bash), and of course, POSIX test is a lot more well-tempered than the original creation from Bell Labs.
if is followed by then in bash but I don't understand why then cannot be used in the same line like if [...] then it has to be used in the next line. Does that remove some ambiguity from the code? or bash is designed like that? what is the underlying reason for it?
I tried to write if and then in the same line but it gave the error below:
./test: line 6: syntax error near unexpected token \`fi'
./test: line 6: \`fi'
the code is:
#!/bin/bash
if [ $1 -gt 0 ] then
echo "$1 is positive"
fi
It has to be preceded by a separator of some description, not necessarily on the next line(a). In other words, to achieve what you want, you can simply use:
if [[ $1 -gt 0 ]] ; then
echo "$1 is positive"
fi
As an aside, for one-liners like that, I tend to prefer:
[[ $1 -gt 0 ]] && echo "$1 is positive"
But that's simply because I prefer to see as much code on screen as possible. It's really just a style thing which you can freely ignore.
(a) The reason for this can be found in the Bash manpage (my emphasis):
RESERVED WORDS: Reserved words are words that have a special meaning to the shell. The following words are recognized as reserved when unquoted and either the first word of a simple command (see SHELL GRAMMAR below) or the third word of a case or for command:
! case coproc do done elif else esac fi for function if in select then until while { } time [[ ]]
Note that, though that section states it's the "first word of a simple command", the manpage seems to contradict itself in the referenced SHELL GRAMMAR section:
A simple command is a sequence of optional variable assignments followed by blank-separated words and redirections, and terminated by a control operator. The first word specifies the command to be executed, and is passed as argument zero.
So, whether you consider it part of the next command or a separator of some sort is arguable. What is not arguable is that it needs a separator of some sort (newline or semicolon, for example) before the then keyword.
The manpage doesn't go into why it was designed that way but it's probably to make the parsing of commands a little simpler.
Here's another way to explain the need for a line break or semicolon before then: the thing that goes between if and then is a command (or sequence of commands); if the then just came directly after the command without a delimiter, it'd be ambiguous whether it should be treated as a shell keyword or just an argument to the command.
For instance, this is a perfectly valid command:
echo This prints a phrase ending with then
...which prints "This prints a phrase ending with then". Now, consider this one:
if echo This prints a phrase ending with then
should that print "This prints a phrase ending with then" and look for a then keyword later on, or should it just print "This prints a phrase ending with" and treat the then as a keyword?
In order to settle this ambiguity, shell syntax says it should treat "then" as an argument to echo, and in order to get it treated as a keyword you need a command delimiter (line break or semicolon) to mark the end of the command.
Now, you might think that your if condition [ $1 -gt 0 ], already has a perfectly good delimiter, namely the ]. But in shell syntax, that's really just an argument to the [ command (yes, that's a command). Try this command:
[ 1 -gt 0 ] then
...and you'll probably get an error like "-bash: [: missing ']'", because the [ command checked its last argument to make sure it was "]", found that it was "then" instead, and panicked.
Perhaps it helps to understand why this is so by way of a few examples. The argument to if is a sequence of commands; so you can say e.g.
if read -r -p "What is your name?" name
[ "$name" -eq "tripleee" ]
then
echo "I kneel before thee"
fi
or even a complex compound like
while read -r -p "Favorite number?" number
case $number in
42) true; break;;
*) false;;
esac
do
echo "Review your preferences, then try again"
done
This extremely powerful but potentially confusing feature of the shell is probably one of its most misunderstood constructs. The ability to pass a sequence of commands to the flow control statements can make for very elegant scripts, but is often missed entirely (see e.g. Why is testing "$?" to see if a command succeeded or not, an anti-pattern?)
If it helps, you can use semi-colons
if [ $1 -gt 0 ]; then
echo "$1 is positive"
fi
# or even
if [ $1 -gt 0 ]; then echo "$1 is positive"; fi
As for why, it helps me to think of if, then, else, and fi as bash commands, and just like all other commands, they need to be at the start of a line (or after a semi-colon).
Example script:
#!/bin/bash
printf '1\n1\n1\n1\n' | ./script2*.sh >/dev/null 2>/dev/null
Shellcheck returns the following:
In script1.sh line 3:
printf '1\n1\n1\n1\n' | ./script2*.sh >/dev/null 2>/dev/null
^-- SC2211: This is a glob used as a command name. Was it supposed to be in ${..}, array, or is it missing quoting?
According to https://github.com/koalaman/shellcheck/wiki/SC2211, there should be no exceptions to this rule.
Specifically, it suggests "If you want to specify a command name via glob, e.g. to not hard code version in ./myprogram-*/foo, expand to array or parameters first to allow handling the cases of 0 or 2+ matches."
The reason I'm using the glob in the first place is that I append or change the date to any script that I have just created or changed. Interestingly enough, when I use "bash script2*.sh" instead of "./script2*.sh" the complaint goes away.
Have I fixed the problem or I am tricking shellcheck into ignoring a problem that should not be ignored? If I am using bad bash syntax, how might I execute another script that needs to be referenced to using a glob the proper way?
The problem with this is that ./script2*.sh may end up running
./script2-20171225.sh ./script2-20180226.sh ./script2-copy.sh
which is a strange and probably unintentional thing to do, especially if the script is confused by such arguments, or if you wanted your most up-to-date file to be used. Your "fix" has the same fundamental problem.
The suggestion you mention would take the form:
array=(./script2*.sh)
[ "${#array[#]}" -ne 1 ] && { echo "Multiple matches" >&2; exit 1; }
"${array[0]}"
and guard against this problem.
Since you appear to assume that you'll only ever have exactly one matching file to be invoked without parameters, you can turn this into a function:
runByGlob() {
if (( $# != 1 ))
then
echo "Expected exactly 1 match but found $#: $*" >&2
exit 1
elif command -v "$1" > /dev/null 2>&1
then
"$1"
else
echo "Glob is not a valid command: $*" >&2
exit 1
fi
}
whatever | runByGlob ./script2*.sh
Now if you ever have zero or multiple matching files, it will abort with an error instead of potentially running the wrong file with strange arguments.
Is there a list of all the if switches for use in Bash scripting? Sometimes I see someone using it and I wonder what the switch they're using actually does.
An example is the -z in this one. I know how to use it, but I don't know where it was derived from.
if [ -z "$BASH_VERSION" ]; then
echo -e "Error: this script requires the BASH shell!"
exit 1
fi
Any references, guides, posts, answers would be appreciated.
Look at the Bash man page (man bash). The options are specified in the CONDITIONAL EXPRESSIONS section:
CONDITIONAL EXPRESSIONS
Conditional expressions are used by the [[ compound command and the
test and [ builtin commands to test file attributes and perform string
and arithmetic comparisons. Expressions are formed from the following
unary or binary primaries. If any file argument to one of the pri-
maries is of the form /dev/fd/n, then file descriptor n is checked. If
the file argument to one of the primaries is one of /dev/stdin,
/dev/stdout, or /dev/stderr, file descriptor 0, 1, or 2, respectively,
is checked.
Unless otherwise specified, primaries that operate on files follow sym-
bolic links and operate on the target of the link, rather than the link
itself.
-a file
True if file exists.
... more options ...
It is also explained in the help:
$ help [
[: [ arg... ]
This is a synonym for the "test" builtin, but the last
argument must be a literal `]', to match the opening `['.
Yes. These are called conditional expressions and these are used by the [[ compound command and the test and [ builtin commands ([ is simply a synonym for test).
Read section 6.4 Bash Conditional Expressions of the Bash Reference Manual, which contains a list of all these switches and their usage.
The single square brackets ([ ... ]) is an synonym of the test command. If you look at the man page for test, you will see almost all (Bash might have a few extra not mentioned here) of the various if switches as you called them. All in one easy-to-find place.
If you use double square brackets ([[ ... ]]), you are using an extended Bash set of tests. These mainly have to do with regular expression matching, and glob matching (and extended glob matching if you have that set too). For that, you'll have to read that Bash man page.
You called them if switches, but that's not really correct. These are tests and really have nothing to do with the if command.
The if command merely executes the command you give it, and then if that command returns an exit code of 0, will run the if portion of the if statement. Otherwise, it will run the else portion (if that's present).
Let's look at this:
rm foo.test.txt # Hope this wasn't an important file
if ls foo.test.txt
> then
> echo "This file exists"
> else
> echo "I can't find it anywhere.."
> fi
ls: foo.test.txt: No such file or directory
I can't find it anywhere..
The if statement runs the ls foo.test.txt command and the ls command returns a non-zero because the file does not exist. This causes the if statement to execute the else clause.
Let's try that again...
touch foo.test.txt # Now this file exists.
if ls foo.test.txt # Same "if/else" statement as above
> then
> echo "This file exists"
> else
> echo "I can't find it anywhere.."
> fi
foo.test.txt
This file exists
Here, the ls command returned a 0 exit status (since the file exists and the file exists and can be stat'ed by the ls command.
Normally, you shouldn't use the ls command to test for a file. I merely used it here to show that the if statement executes the command, then executed the if or else clause depending upon the exit status of that command. If you want to test whether or not a file exists, you should use the test -e command instead of ls command:
if test -e foo.test.txt # The same as above, but using "test" instead of "ls"
then
echo "This file exists"
else
echo "I can't find it anywhere..."
fi
If the file exists, test -e will return an exit status of 0. Otherwise, it will return a non-zero exit status.
If you do this:
ls -i /bin/test /bin/[
10958 /bin/[ 10958 /bin/test
That 10958 is the inode. Files with the same inode are two different names for the same file. Thus [ and test command are soft linked1. This means you can use [ instead of test:
if [ -e foo.test.txt ]
then
echo "This file exists"
else
echo "I can't find it anywhere.."
fi
Does it look familiar?
1. In Bash, the test and [ are builtin, so when you run these commands in BASH, it isn't running /bin/test or /bin/[. However, they're still linked to each other.
They are not switches for the if statement, but for the test command ([ is a synonym for the test builtin). See help test in Bash for a complete list.
It's actually not if that's providing those — it's [, better known by the name of test. help test should give you a list of all options it can take. You could also look at the standard, if you care.
I'm learning shell scripting, and am finding it hard finding a good way to learn. I have created a script below which lets the user search various different Internet engines through options. I would be really grateful if someone could look through this and point out what I'm doing wrong, how to improve it, etc.
#!/bin/bash
## Get user search-engine option
while getopts aegwy: OPTIONS ; do
case "$OPTIONS" in
a) ENGINE="http://www.amazon.com/s/ref=nb_sb_noss/?field-keywords";;
e) ENGINE="http://www.ebay.com/sch/i.html?_nkw";;
g) ENGINE="http://www.google.com/search?q";;
w) ENGINE="http://en.wikipedia.org/wiki/?search";;
y) ENGINE="http://www.youtube.com/results?search_query";;
?) ERRORS=true;;
esac
done &>/dev/null
## Ensure correct command usage
[ $# -ne 2 ] || [ $ERRORS ] && printf "USAGE: $(basename $0) [-a Amazon] [-e eBay] [-g Google] [-w Wikipedia] [-y YouTube] \"search query\"\n" && exit 1
## Ensure user is connected to the Internet
ping -c 1 209.85.147.103 &>/dev/null ; [ $? -eq 2 ] && printf "You are not connected to the Internet!\n" && exit 1
## Reformat the search query
QUERY=`printf "$2" | sed 's/ /+/g'`
## Execute the search and exit program
which open &>/dev/null ; [ $? -eq 0 ] && open "$ENGINE"="$QUERY" &>/dev/null && exit 0 || xdg-open "$ENGINE"="$QUERY" &>/dev/null && exit 0 || printf "Command failed!\n" && exit 1
Thanks in advance everyone, means a lot!
Best posted in codereviews, as indicated above, but here are some mostly stylistic comments. I should stress that the script is pretty much fine as-is; these are just minor improvements that I think will help make the code easier to read/maintain, more robust in a couple cases, etc.
You don't need to use all-caps for variable names just because environment variables are all-caps; shell variables and environment variables aren't the same thing.
Since your $OPTIONS variable only holds one option at a time, a singular name would be better (e.g. $option). Or you could go with $opt, which is somewhat traditional here.
The : in your getopts string (aegwy:) indicates that the -y option expects an argument, as in -ysomething instead of just -y by itself. Since you aren't doing anything with $OPTARG, I'm guessing that's not intentional.
As others have said, an if/then/elif/else would probably be clearer than the chain of && and ||.
The test [ $ERRORS ] is somewhat unclear because it can mean a lot of different things depending on the content of the $ERRORS parameter. A more explicit indication that you only care about whether or not it's set would be [ -n "$ERRORS" ].
Comparisons like [ -ne ] and friends are mostly holdovers from before the shell had built-in integer arithmetic; the more modern idiom would be (( $# != 2 )).
Your usage message implies that the -a, -e, -g, -w, and -y options take arguments of the form Amazon, eBay, Google, etc. It would be clearer what the actual syntax of the command is without those additions; you can include an extra paragraph in the help text listing what each option stands for.
As a rule, error messages should go to stderr instead of stdout (>&2).
It's fine to use basename $0 for consistency of output, but there's something to be said for leaving $0 alone as it will reflect however the user actually invoked the command. Something to consider.
Not much point in using printf if you're not using a format string; just use echo, which automatically appends the newline. Usage messages traditionally don't include quotation marks, either; it's up to the user to quote the arg or not depending on whether it's needed.
Checking a command for success is exactly how if works, so there's no need to do explicit checks of $? unless you really care about the exact exit value. In the case of your connectivity ping, you probably don't care about why it failed, only that it did:
if ! ping -c 1 209.85.147.103 >/dev/null; then
echo >&2 "$0: You are not connected to the Internet!"
exit 1
fi
Your search query reformat might need to do more than just turn spaces into plus signs; what if it has an ampersand? But if you're just doing the spaces-to-pluses thing, you could use bash parameter expansion do it without sed: QUERY="${QUERY// /+}"
If your program relies on open/xdg-open etc, you should probably check for its availability at the top; no sense doing anything else if you know you can't possibly perform the requested operation anyway. And you can use a variable so you don't wind up repeating yourself in multiple clauses:
open=
for cmd in open xdg-open; do
if type -p "$cmd" >/dev/null; then
open="$cmd"
break
fi
done
if [ -z "$open" ]; then
echo >&2 "$0: open command not found."
exit 1
fi
And then later you can finish up with just this one line:
"$open" "$ENGINE=$QUERY" &>/dev/null
http://linuxcommand.org/ is an excellent resource to improve your bash scripting skills.
http://tldp.org/LDP/abs/html/ is another great document.
Hope this helps.