I am having shell script like this
while read -r LINE || [[ -n $LINE ]]; do
noStudent="false"
LINE=$(echo "$LINE" | tr '[:upper:]' '[:lower:]')
if [ $LINE == "myname" ]
then
noStudent="true"
fi
if [ $noStudent == "true" ]
then
#DO SOME STUFF
fi
done < Teachers
But I am getting error on each line when files is read :
+ [ yourname == myname ]
/tmp/hudson433507028658734743.sh: 20: [: yourname: unexpected operator
+ [ false == true ]
/tmp/hudson433507028658734743.sh: 26: [: false: unexpected operator
What can be the reason for it ? Please help. Am not able to figure out.
Your script isn't actually being executed by bash, but by some other shell linked to by /bin/sh (likely dash, see below). The POSIX standard does not recognize == as a valid operator with the [ command; you need to use = instead.
Using dash, for example, this can be reproduced with
$ [ foo == foo ] && echo same
dash: 5: [: foo: unexpected operator
$ [ foo = foo ] && echo same
same
$
You will almost certainly use [ because you are concerned about portability, in which case you must also use =. If you aren't concerned about portability, and your shell allows == inside [, you can almost certainly (and probably should) use [[ instead.
Related
I'm reading bash examples about if but some examples are written with single square brackets:
if [ -f $param ]
then
#...
fi
others with double square brackets:
if [[ $? -ne 0 ]]
then
start looking for errors in yourlog
fi
What is the difference?
Single [] are posix shell compliant condition tests.
Double [[]] are an extension to the standard [] and are supported by bash and other shells (e.g. zsh, ksh). They support extra operations (as well as the standard posix operations). For example: || instead of -o and regex matching with =~. A fuller list of differences can be found in the bash manual section on conditional constructs.
Use [] whenever you want your script to be portable across shells. Use [[]] if you want conditional expressions not supported by [] and don't need to be portable.
Behavior differences
Tested in Bash 4.3.11:
POSIX vs Bash extension:
[ is POSIX
[[ is a Bash extension inspired from Korn shell
regular command vs magic
[ is just a regular command with a weird name.
] is just the last argument of [.
Ubuntu 16.04 actually has an executable for it at /usr/bin/[ provided by coreutils, but the bash built-in version takes precedence.
Nothing is altered in the way that Bash parses the command.
In particular, < is redirection, && and || concatenate multiple commands, ( ) generates subshells unless escaped by \, and word expansion happens as usual.
[[ X ]] is a single construct that makes X be parsed magically. <, &&, || and () are treated specially, and word splitting rules are different.
There are also further differences like = and =~.
In Bashese: [ is a built-in command, and [[ is a keyword: https://askubuntu.com/questions/445749/whats-the-difference-between-shell-builtin-and-shell-keyword
<
[[ a < b ]]: lexicographical comparison
[ a \< b ]: Same as above. \ required or else does redirection like for any other command. Bash extension.
expr x"$x" \< x"$y" > /dev/null or [ "$(expr x"$x" \< x"$y")" = 1 ]: POSIX equivalents, see: How to test strings for lexicographic less than or equal in Bash?
&& and ||
[[ a = a && b = b ]]: true, logical and
[ a = a && b = b ]: syntax error, && parsed as an AND command separator cmd1 && cmd2
[ a = a ] && [ b = b ]: POSIX reliable equivalent
[ a = a -a b = b ]: almost equivalent, but deprecated by POSIX because it is insane and fails for some values of a or b like ! or ( which would be interpreted as logical operations
(
[[ (a = a || a = b) && a = b ]]: false. Without ( ), would be true because [[ && ]] has greater precedence than [[ || ]]
[ ( a = a ) ]: syntax error, () is interpreted as a subshell
[ \( a = a -o a = b \) -a a = b ]: equivalent, but (), -a, and -o are deprecated by POSIX. Without \( \) would be true because -a has greater precedence than -o
{ [ a = a ] || [ a = b ]; } && [ a = b ] non-deprecated POSIX equivalent. In this particular case however, we could have written just: [ a = a ] || [ a = b ] && [ a = b ] because the || and && shell operators have equal precedence unlike [[ || ]] and [[ && ]] and -o, -a and [
word splitting and filename generation upon expansions (split+glob)
x='a b'; [[ $x = 'a b' ]]: true, quotes not needed
x='a b'; [ $x = 'a b' ]: syntax error, expands to [ a b = 'a b' ]
x='*'; [ $x = 'a b' ]: syntax error if there's more than one file in the current directory.
x='a b'; [ "$x" = 'a b' ]: POSIX equivalent
=
[[ ab = a? ]]: true, because it does pattern matching (* ? [ are magic). Does not glob expand to files in current directory.
[ ab = a? ]: a? glob expands. So may be true or false depending on the files in the current directory.
[ ab = a\? ]: false, not glob expansion
= and == are the same in both [ and [[, but == is a Bash extension.
case ab in (a?) echo match; esac: POSIX equivalent
[[ ab =~ 'ab?' ]]: false, loses magic with '' in Bash 3.2 and above and provided compatibility to bash 3.1 is not enabled (like with BASH_COMPAT=3.1)
[[ ab? =~ 'ab?' ]]: true
=~
[[ ab =~ ab? ]]: true, POSIX extended regular expression match, ? does not glob expand
[ a =~ a ]: syntax error. No bash equivalent.
printf 'ab\n' | grep -Eq 'ab?': POSIX equivalent (single line data only)
awk 'BEGIN{exit !(ARGV[1] ~ ARGV[2])}' ab 'ab?': POSIX equivalent.
Recommendation: always use []
There are POSIX equivalents for every [[ ]] construct I've seen.
If you use [[ ]] you:
lose portability
force the reader to learn the intricacies of another bash extension. [ is just a regular command with a weird name, no special semantics are involved.
Thanks to Stéphane Chazelas for important corrections and additions.
Inside single brackets for condition test (i.e. [ ... ]), some operators such as single = is supported by all shells, whereas use of operator == is not supported by some of the older shells.
Inside double brackets for condition test (i.e. [[ ... ]]), there is no difference between using = or == in old or new shells.
Edit: I should also note that: In bash, always use double brackets [[ ... ]] if possible, because it is safer than single brackets. I'll illustrate why with the following example:
if [ $var == "hello" ]; then
if $var happens to be null / empty, then this is what the script sees:
if [ == "hello" ]; then
which will break your script. The solution is to either use double brackets, or always remember to put quotes around your variables ("$var"). Double brackets is better defensive coding practice.
[[ is a bash keyword similar to (but more powerful than) the [ command.
See
http://mywiki.wooledge.org/BashFAQ/031 and http://mywiki.wooledge.org/BashGuide/TestsAndConditionals
Unless you're writing for POSIX sh, we recommend [[.
[ is a builtin like printf. Bash syntax expect to see it at the same place as commands. And ] is nothing to Bash except the fact that it is expected by the [ builtin. (man bash / SHELL BUILTIN COMMANDS)
[[ is a keyword like if. Bash syntax starts also expect it at the same place as command but instead of executing it, it enters the conditional context. And ]] is also a keyword ending this context. (man bash / SHELL GRAMMAR / Compound Commands)
In order, bash tries to parse: Syntax Keywords > User Alias > Builtin Function > User Function > Command in $PATH
type [ # [ is a shell builtin
type [[ # [[ is a shell keyword
type ] # bash: type: ]: not found
type ]] # ]] is a shell keyword
compgen -k # Keywords: if then else ...
compgen -b # Builtins: . : [ alias bg bind ...
which [ # /usr/bin/[
[ is slower <= it executes more parsing code I guess. But I know that it calls the same number of syscall (tested with
[[ is syntactically easier to parse even for human as it starts a context. For arithmetical condition, think about using ((.
time for i in {1..1000000}; do [ 'a' = 'b' ] ; done # 1.990s
time for i in {1..1000000}; do [[ 'a' == 'b' ]] ; done # 1.371s
time for i in {1..1000000}; do if [ 'a' = 'a' ]; then if [ 'a' = 'b' ];then :; fi; fi ; done # 3.512s
time for i in {1..1000000}; do if [[ 'a' == 'a' ]]; then if [[ 'a' == 'b' ]];then :; fi; fi; done # 2.143s
strace -cf bash -c "for i in {1..100000}; do if [ 'a' = 'a' ]; then if [ 'a' = 'b' ];then :; fi; fi ; done;" # 399
strace -cf bash -c "for i in {1..100000}; do if [[ 'a' == 'a' ]]; then if [[ 'a' == 'b' ]];then :; fi; fi ; done;" # 399
I recommend using [[: If you do not explicitly care about posix compatibility, it means that you are not, so do no care about getting "more" compatible a script that is not.
you can use the double square brackets for light regex matching, e.g. :
if [[ $1 =~ "foo.*bar" ]] ; then
(as long as the version of bash you are using supports this syntax)
Bash manual says:
When used with [[, the ‘<’ and ‘>’ operators sort lexicographically
using the current locale. The test command uses ASCII
ordering.
(The test command is identical to [ ] )
I couldn't find any one simple straightforward resource spelling out the meaning of and fix for the following BASH shell error, so I'm posting what I found after researching it.
The error:
-bash: [: too many arguments
Google-friendly version: bash open square bracket colon too many arguments.
Context: an if condition in single square brackets with a simple comparison operator like equals, greater than etc, for example:
VARIABLE=$(/some/command);
if [ $VARIABLE == 0 ]; then
# some action
fi
If your $VARIABLE is a string containing spaces or other special characters, and single square brackets are used (which is a shortcut for the test command), then the string may be split out into multiple words. Each of these is treated as a separate argument.
So that one variable is split out into many arguments:
VARIABLE=$(/some/command);
# returns "hello world"
if [ $VARIABLE == 0 ]; then
# fails as if you wrote:
# if [ hello world == 0 ]
fi
The same will be true for any function call that puts down a string containing spaces or other special characters.
Easy fix
Wrap the variable output in double quotes, forcing it to stay as one string (therefore one argument). For example,
VARIABLE=$(/some/command);
if [ "$VARIABLE" == 0 ]; then
# some action
fi
Simple as that. But skip to "Also beware..." below if you also can't guarantee your variable won't be an empty string, or a string that contains nothing but whitespace.
Or, an alternate fix is to use double square brackets (which is a shortcut for the new test command).
This exists only in bash (and apparently korn and zsh) however, and so may not be compatible with default shells called by /bin/sh etc.
This means on some systems, it might work from the console but not when called elsewhere, like from cron, depending on how everything is configured.
It would look like this:
VARIABLE=$(/some/command);
if [[ $VARIABLE == 0 ]]; then
# some action
fi
If your command contains double square brackets like this and you get errors in logs but it works from the console, try swapping out the [[ for an alternative suggested here, or, ensure that whatever runs your script uses a shell that supports [[ aka new test.
Also beware of the [: unary operator expected error
If you're seeing the "too many arguments" error, chances are you're getting a string from a function with unpredictable output. If it's also possible to get an empty string (or all whitespace string), this would be treated as zero arguments even with the above "quick fix", and would fail with [: unary operator expected
It's the same 'gotcha' if you're used to other languages - you don't expect the contents of a variable to be effectively printed into the code like this before it is evaluated.
Here's an example that prevents both the [: too many arguments and the [: unary operator expected errors: replacing the output with a default value if it is empty (in this example, 0), with double quotes wrapped around the whole thing:
VARIABLE=$(/some/command);
if [ "${VARIABLE:-0}" == 0 ]; then
# some action
fi
(here, the action will happen if $VARIABLE is 0, or empty. Naturally, you should change the 0 (the default value) to a different default value if different behaviour is wanted)
Final note: Since [ is a shortcut for test, all the above is also true for the error test: too many arguments (and also test: unary operator expected)
Just bumped into this post, by getting the same error, trying to test if two variables are both empty (or non-empty). That turns out to be a compound comparison - 7.3. Other Comparison Operators - Advanced Bash-Scripting Guide; and I thought I should note the following:
I used -e thinking it means "empty" at first; but that means "file exists" - use -z for testing empty variable (string)
String variables need to be quoted
For compound logical AND comparison, either:
use two tests and && them: [ ... ] && [ ... ]
or use the -a operator in a single test: [ ... -a ... ]
Here is a working command (searching through all txt files in a directory, and dumping those that grep finds contain both of two words):
find /usr/share/doc -name '*.txt' | while read file; do \
a1=$(grep -H "description" $file); \
a2=$(grep -H "changes" $file); \
[ ! -z "$a1" -a ! -z "$a2" ] && echo -e "$a1 \n $a2" ; \
done
Edit 12 Aug 2013: related problem note:
Note that when checking string equality with classic test (single square bracket [), you MUST have a space between the "is equal" operator, which in this case is a single "equals" = sign (although two equals' signs == seem to be accepted as equality operator too). Thus, this fails (silently):
$ if [ "1"=="" ] ; then echo A; else echo B; fi
A
$ if [ "1"="" ] ; then echo A; else echo B; fi
A
$ if [ "1"="" ] && [ "1"="1" ] ; then echo A; else echo B; fi
A
$ if [ "1"=="" ] && [ "1"=="1" ] ; then echo A; else echo B; fi
A
... but add the space - and all looks good:
$ if [ "1" = "" ] ; then echo A; else echo B; fi
B
$ if [ "1" == "" ] ; then echo A; else echo B; fi
B
$ if [ "1" = "" -a "1" = "1" ] ; then echo A; else echo B; fi
B
$ if [ "1" == "" -a "1" == "1" ] ; then echo A; else echo B; fi
B
Another scenario that you can get the [: too many arguments or [: a: binary operator expected errors is if you try to test for all arguments "$#"
if [ -z "$#" ]
then
echo "Argument required."
fi
It works correctly if you call foo.sh or foo.sh arg1. But if you pass multiple args like foo.sh arg1 arg2, you will get errors. This is because it's being expanded to [ -z arg1 arg2 ], which is not a valid syntax.
The correct way to check for existence of arguments is [ "$#" -eq 0 ]. ($# is the number of arguments).
I also faced same problem. #sdaau answer helped me in logical way. Here what I was doing which seems syntactically correct to me but getting too many arguments error.
Wrong Syntax:
if [ $Name != '' ] && [ $age != '' ] && [ $sex != '' ] && [ $birthyear != '' ] && [ $gender != '' ]
then
echo "$Name"
echo "$age"
echo "$sex"
echo "$birthyear"
echo "$gender"
else
echo "Enter all the values"
fi
in above if statement, if I pass the values of variable as mentioned below then also I was getting syntax error
export "Name"="John"
export "age"="31"
export "birthyear"="1990"
export "gender"="M"
With below syntax I am getting expected output.
Correct syntax:
if [ "$Name" != "" -a "$age" != "" -a "$sex" != "" -a "$birthyear" != "" -a "$gender" != "" ]
then
echo "$Name"
echo "$age"
echo "$sex"
echo "$birthyear"
echo "$gender"
else
echo "it failed"
fi
There are few points which we need to keep in mind
use "" instead of ''
use -a instead of &&
put space before and after operator sign like [ a = b], don't use as [ a=b ] in if condition
Hence above solution worked for me !!!
Some times If you touch the keyboard accidentally and removed a space.
if [ "$myvar" = "something"]; then
do something
fi
Will trigger this error message. Note the space before ']' is required.
I have had same problem with my scripts. But when I did some modifications it worked for me. I did like this :-
export k=$(date "+%k");
if [ $k -ge 16 ]
then exit 0;
else
echo "good job for nothing";
fi;
that way I resolved my problem. Hope that will help for you too.
I would like to define a prompt which will indicate with colors whether the command executed properly and whether the command was found. As for now I have something like this but I does not work properly.
PS1="\`COMMAND_RESULT=\$\?;
if [ $COMMAND_RESULT -eq 127 ]; then echo \[\e[33m\] ---=== Command not found ===--- ;
elif [ $COMMAND_RESULT -ne 0 ]; then echo \[\e[33m\]---=== \[\e[31m\]Oh noes, bad command \[\e[33m\]==---;
fi\`
\n\[\e[0;37m\][\[\e[1;31m\]\#\[\e[0;37m\]]
\[\e[0;32m\]\u\[\033[1;33m\]#\[\033[0;32m\]\h
As for now I get this error on bash start :
-bash: [: -eq: unary operator expected
-bash: [: -ne: unary operator expected
Don't pollute your PS1 with functions. You should use the special PROMPT_COMMAND variable to do this. The value of PROMPT_COMMAND is executed as a command prior to issuing each primary prompt.
Here is an example:
_check_command(){
local COMMAND_RESULT=$?
if [ $COMMAND_RESULT -eq 127 ]
then
echo -e "\e[1;33m---=== Command not found ===---\e[m"
elif [ $COMMAND_RESULT -ne 0 ]
then
echo -e "\e[1;31m---=== Oh noes, bad command ===---\e[m"
fi
}
PROMPT_COMMAND='_check_command'
PS1="\[\e[0;37m\][\[\e[1;31m\]\#\[\e[0;37m\]] \[\e[0;32m\]\u\[\033[1;33m\]#\[\033[0;32m\]\h "
There are many bash prompts you can find online to guide you. Here is one good example.
You probably should not escape $? as \$\?. Looks like it gets interpreted literally.
Also you can check out the Arch Wiki article that shows how to implement something similar to what you want. Look at this line:
PS1="$(if [[ ${EUID} == 0 ]]; then echo '\[\033[01;31m\]\h'; else echo '\[\033[01;32m\]\u#\h'; fi)\[\033[01;34m\] \w \$([[ \$? != 0 ]] && echo \"\[\033[01;31m\]:(\[\033[01;34m\] \")\$\[\033[00m\] "
especially this part:
([[ \$? != 0 ]] && echo \"\[\033[01;31m\]:(\[\033[01;34m\] \")
if a script has
if [ $1 == "-?" ]; then #line 4
echo "usage: ...."
fi
when the script get runs without any parameter, it will complain that
./script.sh: line 4: [: ==: unary operator expected
but if instead
if [ "$1" == "-?" ]; then #line 4
echo "usage: ...."
fi
then everything's fine
why is that?
thanks
If the first argument is missing or empty, your first script evaluates to:
if [ == "-?" ] ; then
... which is a syntax error. As you noticed, to prevent that you need to make use of "", then it evaluates to:
if [ "" == "-?" ] ; then
AFAIK this is due to the way the original Bourne shell was working. You should make it a habit of enclosing variables in "" to also work correctly with arguments that have spaces in it. For example, if you would call your script like this:
./myScript "first argument has spaces"
Then your first script would evaluate to:
if [ first argument has spaces == "-?" ] ; then
which is also a syntax error. Also things like rm $1 will not do what you want if you pass filenames with spaces. Do rm "$1" instead.
Because [ replaces the values before executing. [[ doesn't, so will work as expected.
Usually work in Windows, but trying to setup RabbitMQ on my Mac. Can someone let me know what the line below does?
[ "x" = "x$RABBITMQ_NODE_IP_ADDRESS" ] && [ "x" != "x$NODE_IP_ADDRESS" ] && RABBITMQ_NODE_IP_ADDRESS=${NODE_IP_ADDRESS}
Specifically, I'm curious about the [ "x" = "x$RAB..."] syntax.
If the RABBITMQ_NODE_IP_ADDRESS variable is empty/doesn't exist, it'll evaluate as "x" = "x" , which is true.
So it basically says, if RABBITMQ_NODE_IP_ADDRESS isn't set and NODE_IP_ADDRESS is set, set RABBITMQ_NODE_IP_ADDRESS=NODE_IP_ADDRESS
The "x" is used (somewhat superstitiously*) to prevent errors if the variable is null or unset. Most of the time the quotes take care of that for you. By putting the literal first and the variable second you eliminate errors in cases where the variable contains a string that starts with a dash, since test (aka [) would think it is an operator. In the case of your example, it would be preferable to use the -z and -n operators that test whether a variable is empty (null or unset) or not empty, respectively.
POSIX shells, such as Bourne (works in Bash, too):
[ -z $RABBITMQ_NODE_IP_ADDRESS ] && [ -n $NODE_IP_ADDRESS" ] && RABBITMQ_NODE_IP_ADDRESS=${NODE_IP_ADDRESS}
Bash (and ksh and zsh):
[[ -z $RABBITMQ_NODE_IP_ADDRESS && -n $NODE_IP_ADDRESS" ]] && RABBITMQ_NODE_IP_ADDRESS=${NODE_IP_ADDRESS}
* There may be some shells that need the "x", but some people do that "because it's always been done that way".
The "x" is not always superstitious, even in my relatively new bash (4.0.33).
Let's put the operation between parens. Empty variables are fine:
$ a=""
$ b=""
$ if [ '(' "$a" = "$b" ')' ]; then echo both_equal; fi
both_equal
But the ! operator for instance is not:
$ a='!'
$ if [ '(' "$a" = "$b" ')' ]; then echo both_equal; fi
bash: [: `)' expected, found
This is not a problem if we write "x$a" = "x$b" instead of "$a" = "$b".
The bracket [ is a test operator, which you can think of as an if statement. This is checking to see if the shell variable RABBITMQ_NODE_IP_ADDRESS is empty. Unfortunately, if you try to compare to an empty string "", the shell eliminates it before it does the test and your binary comparison operator only gets one (or maybe zero) operands. To prevent that error, it is a common practice to concatenate an "x" on each side of the =. Thus, instead of
[ "" = "<variable>" ]
becoming
[ = value ]
and yielding an error,
[ "X" = "X<variable>" ]
becomes
[ X = Xvalue ]
and the comparison may continue