I'd like to have some explanation on how to best use variables in bash conditional expressions [[...]].
I usually write if statement in this way:
var=1;
# some code...
if [[ $var -eq 1 ]]; then echo "ok"; else echo "fail"; fi
and this return ok as I expected.
Now I saw in some script the same similar statement like:
var=1;
# some code...
if [[ var -eq 1 ]]; then echo "ok"; else echo "fail"; fi
The only difference is the missing parameter expansion character $ in the conditional expression [[...]].
I actually expected this statement to give an error, but this syntax is accepted and returns the ok string.
I tested this statement using bash (GNU bash, version 4.3.46), zsh (5.1.1), ksh (93u+ 2012-08-01) and busybox ash (BusyBox v1.23.2).
I only get an error with busybox shell:
ash: var: bad number
I saw in the bash man page, in the ARITHMETIC EVALUATION paragraph, that:
Within an expression, shell variables may also be referenced by name without using the parameter expansion syntax
But I didn't find anything special related to parameter expansion in the CONDITIONAL EXPRESSIONS paragraph.
So, should conditional expression contain $ when referring to variable or not? and why?
The trigger here is -eq; since it is defined to perform integer comparison, its operands are evaluated in an arithmetic context. This isn't explicitly documented, though.
You should use the $, though. [[ is an extension, so there is no guarantee that it will behave identically in every shell that defines such a construct. In fact, I wouldn't even assume that [[ var -eq 3 ]] will continue to behave this way in future versions of the same shell. (( var == 3 )) is, though, documented to perform expansion of var since you are in a explicit arithmetic context.
Check the bash man page's sections on Compound Commands. In particular, the following:
((expression))
The expression is evaluated according to the rules described below
under ARITHMETIC EVALUATION. If the value of the expression is non-zero,
the return status is 0; otherwise the return status is 1. This is exactly
equivalent to `let "expression"`.
[[ expression ]]
Return a status of 0 or 1 depending on the evaluation of the conditional
expression expression. Expressions are composed of the primaries
described below under CONDITIONAL EXPRESSIONS.
If you are evaluating things that require arithmetic, use arithmetic evaluation, and check the CONDITIONAL EXPRESSIONS section for the various things you can do with [[ ... ]]. Conditions in double-square-brackets can evaluate both strings and integers, and sometimes those work the same way .. sometimes not.
From the bash man page, under CONDITIONAL EXPRESSIONS:
string1 == string2
string1 = string2
True if the strings are equal. = should be used with the test command for POSIX conformance. When used with the [[ command, this performs pattern
matching as described above (Compound Commands).
...
arg1 OP arg2
OP is one of -eq, -ne, -lt, -le, -gt, or -ge. These arithmetic binary operators return true if arg1 is equal to, not equal to, less than, less than
or equal to, greater than, or greater than or equal to arg2, respectively. Arg1 and arg2 may be positive or negative integers.
Obviously, the string "1" is the string "1", so if n=1, and you compare $n against the string "1", you'll get a hit. But you should know what you're doing, and that this is not a numeric comparison. And similarly, < and > are NOT numeric comparisons, they are string comparisons. So while "1" < "2", you may be surprised that "11" < "2" as well.
That said, bash is forgiving about what kind of conditions you ask it to evaluate:
bash-4.4$ n=1
bash-4.4$ [[ n -eq 1 ]] && echo yes
yes
bash-4.4$ [[ $n -eq 1 ]] && echo yes
yes
bash-4.4$ (( n == 1 )) && echo yes
yes
bash-4.4$ (( n = 2 )) && echo yes
yes
bash-4.4$ echo "$n"
2
The first one works because n can't be anything but a variable in this context, so bash treats it as such. But you shouldn't rely on this behaviour. Use dollar signs for variables in conditional expressions, and stick with the bash mantra, "always quote your variables".
Inside a double-square-bracket expression in bash, you should use the arithmetic binary operators if you intend your comparison to be of integers.
Note that your busybox build appears to be using ash, which is NOT bash. Ash is the "Almquist shell", an older POSIX shell than bash, written in the late 1980s. Ash is the basis for /bin/sh in FreeBSD, in addition to being preferred often over bash in embedded systems (hence busybox) due to its smaller size.
You might want to consider writing POSIX shell scripts instead of Bash shell scripts. This will mean simplifying some things, as well as jumping through some hoops for others. POSIX does not include double-square-bracket expressions, but it does allow things like [ "$n" -eq 1 ] or [ $(( n + 1 )) -eq 2 ]. And it'll work in busybox. :)
Related
In ABS guide, I could see below snippet
var1=5
var2=4
if (( var1 > var2 ))
then
echo "$var1 is greater than $var2"
fi
I am not able to understand, why we don't need $ symbol. I added $ symbol, shellcheck shows "$ symbol is not necessary on arithmetic variables".
I am still not able to understand how that dereferencing of var1 and var2 works...
Expressions inside ((...)) are evaluated in arithmetic context. Strings which can be variable names are considered as variables whose values are integers, since evaluating these strings as literal strings makes no sense in arithmetic context. These considerations are also valid for C style for loops: in for ((i = 0; i < 10; ++i)), preceding i with $ is not necessary (but it may be necessary, depending on the context, within the body of the loop).
(( ... )) follows the same evaluation rules as defined in the POSIX specification for arithmetic expressions. (The main difference is ((...)) produces an exit status reflecting if the result is zero/nonzero, while $((...)) produces the result as a string.) In particular:
If the shell variable x contains a value that forms a valid integer constant, optionally including a leading <plus-sign> or <hyphen-minus>, then the arithmetic expansions "$((x))" and "$(($x))" shall return the same value.
The shell variable var1 contains an integer constant, so $((var1)) and $(($var1)) are equivalent. This holds recursively as well.
Various shells seem to treat variables that refer to variables differently. Nothing in the POSIX wording seems to require the following kind of recursive evaluation, though both bash and dash do:
$ foo=bar
$ bar=5
$ echo $((foo)) # foo/$foo evaluates to bar, which contains an integer constant
5
bash seems to take it a step further, allowing any string to expand, followed by an attempt to evaluate the result as an arithmetic expression.
$ foo="x + 3"
$ x=5
$ echo $((foo)) # foo evaluates to x + 3, which evaluates to 5 + 3
8
but in dash:
$ foo="x+3"
$ x=5
$ echo $((foo))
dash: 3: Illegal number: x+3
It's documented in the manual in 6.5 Shell Arithmetic. It says
Within an expression, shell variables may also be referenced by name without using the parameter expansion syntax.
This features makes arithmetic expressions so much easier to read.
Interestingly, you can also refer to array elements like this as well
values=(42 54) i=0 j=1
echo $(( values[i] + values[j] )) # => 96
Other places you don't need $ (i.e. other arithmetic "contexts"):
For numerically-indexed arrays, the index (${values[i]} not ${values[$i]})
the offset and length parts of the ${var:offset:length} parameter expansion.
You don't need the $ if it's inside the (( )) for doing arithmetic.
I am still not able to understand how that dereferencing of var1 and var2 works
bash parses things differently inside the (( )), undoubtedly because it makes it much easier to read complex arithmetic expressions without the $.
All of these options below work for setting i to count.
count=5
for (( i=count; i>=0; i-- )); do
echo "$i"
done
for (( i=$count; i>=0; i-- )); do
echo "$i"
done
for (( i=$((count)); i>=0; i-- )); do
echo "$i"
done
for (( i=${count}; i>=0; i-- )); do
echo "$i"
done
Which is correct or preferred?
Generally, in arithmetic contexts such as ((...)) and $((...)), reference variables by name only, without the $ prefix (as your commands are already doing with respect to variable $i):
for (( i=count; i>=0; i-- )); do
echo "$i"
done
Since ((...)) is an arithmetic context itself, there is no good reason to use a separate, expanding arithmetic context - $((count)) - inside of it.
Note that $count and ${count} are equivalent, and enclosing the variable name - count - in { and } after $ is only necessary to disambiguate the variable name from subsequent characters that can also legally be part of a variable name (which doesn't apply to your commands).
As Gordon Davisson points out, some people choose to always use the ${var} form for visual clarity.
While $-prefixed variable references in arithmetic contexts do work, there is rarely a good reason to use them: the use of $ introduces an extra expansion step before the arithmetic evaluation, which is not only unnecessary, but can result in different behavior, as explained in rici's helpful answer.
The only cases where you need the $ prefix:
To reference positional and special parameters (variables) that can only ever be referenced with $: Thanks, rici.
Positional parameters: $1, $2, ...
The count of positional parameters: $#
Special parameters: $?, $$, $! (there are others, but they are not generally numeric - see chapter Special Parameters in man bash).
If you need a nonzero default value that you provide via a parameter expansion; e.g., ${count:-2} defaults to 2 if $count is unset or empty.
If you want to use a variable value as an operator rather than operand; e.g.:
op='*'; echo $(( 2 $op 2 )) - this wouldn't work with just op.
There is a difference between using $x and just x in an arithmetic context:
x causes the value of x to be evaluated as a number. If x hasn't been defined, the result is 0. If the value of x is a valid arithmetic expression, that expression is evaluated. If it is an invalid expression, the result is a syntax error.
$x causes the value of x as a string to be interpolated into the arithmetic expression, which will then be evaluated.
This leads to different evaluations, particularly with uninitialized variables:
$ unset x
$ echo $((x/2))
0
$ echo $(($x/2))
bash: /2: syntax error: operand expected (error token is "/2")
# Also, with incomplete expressions
$ x=42+
$ echo $((x 7))
bash: 42+: syntax error: operand expected (error token is "+")
$ echo $(($x 7))
49
In both cases, I prefer the behaviour associated with the unadorned use of the variable name. Consequently, I recommend its consistent use in arithmetic expressions unless you have a really good reason not to (in which case, you probably should quote the expansion to make it clearer what your expectation is.)
I would avoid $((count)), as this creates an unnecessary arithmetic evaluation step. The other ones have different syntax, but the exact same semantics. I would use the first one, for no better reason than it being shorter.
I suggest the still equivalent (in its result), but shorter :
for (( i=count+1; i-->1 ;)); do
echo "$i"
done
I have always liked that one, because it visually says "i goes toward 0".
Recently I've got confused by the following situation.
What is the difference between two if usage:
Case 1
amount=10
if [[ $amount -eq 10 ]]
then
echo "something"
fi
script output:
$ ./1.sh
something
Case 2
if [[ amount -eq 10 ]]
This also works like this (note that the variable name doesn't contain the $).
So the question is how does it work even without dollar sign in the variable name.
P.S. I'm using a POSIX shell on HP-UX.
man bash
ARITHMETIC EVALUATION
...
Shell variables are allowed as operands; parameter expansion is per‐
formed before the expression is evaluated. Within an expression,
shell variables may also be referenced by name without using the
parameter expansion syntax.
In this context shell does not expect anything but numerics, so it expands strings as variables. That makes sense to me.
While learning a bit about bash, I come to see four types of ways of working with if statements:
Single Parenthesis - ( ... )
Double Parenthesis - (( ... ))
Single Square Bracket - [ ... ]
Double Square Brackets - [[ ... ]]
What is the difference between Parenthesis and Square Brackets in bash.
The tests you had listed :
Single Parenthesis - ( ... ) is creating a subshell
Double Parenthesis - (( ... )) is for arithmetic operation
Single Square Bracket - [ ... ] is the syntax for the POSIX test
Double Square Brackets - [[ ... ]] is the syntax for bash conditional expressions (similar to test but more powerful)
are not exhaustive, you can use boolean logic
if command; then ...
too, because the commands have exit status. In bash, 0 is true and > 0 is false.
You can see the exit status like this :
command
echo $?
See :
http://wiki.bash-hackers.org/syntax/basicgrammar
http://wiki.bash-hackers.org/syntax/arith_expr
http://mywiki.wooledge.org/BashGuide/TestsAndConditionals
The shell itself only runs the command and evaluates its exit code. A zero exit code signifies success; all other values indicate failure.
if command; then
: things to do if the exit code from command was 0
else
: things to do if it was not 0
fi
while command; do
: things to do if the exit code was 0
done
The command [ (aka test) is very commonly used in conditionals, because the original Bourne shell lacked built-in operators to check if a string was empty or a file existed. Modern shells have this command built in, and many shells have an extended and modernized version [[, but this is not properly portable to POSIX sh and should thus be avoided for portable scripts. This related question explains the differences between the two in more detail.
The notation (( ... )) introduces an arithmetic context. Again, this was something which was not part of the original Bourne shell (it had a dedicated external tool expr for these things) but modern shells have this built in. The result code of an arithmetic expression is 0 if the result of the arithmetic evaluation was not 0 (or an error).
The notation ( command ) creates a subshell and evaluates command in that. There are situations where this is actually necessary and useful, but if you are only just learning the syntax, you are unlikely to need this.
... In fact, in the majority of scripts I have seen this used in a conditional, it was clearly unnecessary.
Another antipattern to look out for is
command
if [ $? = 0 ]; then
: things
fi
You should almost never need to examine $? explicitly, and in particular, comparing it to zero is something if and while specifically do for you behind the scenes. This should simply be refactored to
if command; then
: ...
This question already has answers here:
Are double square brackets [[ ]] preferable over single square brackets [ ] in Bash?
(10 answers)
Closed 4 years ago.
I looked at bash man page and the [[ says it uses Conditional Expressions. Then I looked at Conditional Expressions section and it lists the same operators as test (and [).
So I wonder, what is the difference between [ and [[ in Bash?
[[ is bash's improvement to the [ command. It has several enhancements that make it a better choice if you write scripts that target bash. My favorites are:
It is a syntactical feature of the shell, so it has some special behavior that [ doesn't have. You no longer have to quote variables like mad because [[ handles empty strings and strings with whitespace more intuitively. For example, with [ you have to write
if [ -f "$file" ]
to correctly handle empty strings or file names with spaces in them. With [[ the quotes are unnecessary:
if [[ -f $file ]]
Because it is a syntactical feature, it lets you use && and || operators for boolean tests and < and > for string comparisons. [ cannot do this because it is a regular command and &&, ||, <, and > are not passed to regular commands as command-line arguments.
It has a wonderful =~ operator for doing regular expression matches. With [ you might write
if [ "$answer" = y -o "$answer" = yes ]
With [[ you can write this as
if [[ $answer =~ ^y(es)?$ ]]
It even lets you access the captured groups which it stores in BASH_REMATCH. For instance, ${BASH_REMATCH[1]} would be "es" if you typed a full "yes" above.
You get pattern matching aka globbing for free. Maybe you're less strict about how to type yes. Maybe you're okay if the user types y-anything. Got you covered:
if [[ $ANSWER = y* ]]
Keep in mind that it is a bash extension, so if you are writing sh-compatible scripts then you need to stick with [. Make sure you have the #!/bin/bash shebang line for your script if you use double brackets.
See also
Bash FAQ - "What is the difference between test, [ and [[ ?"
Bash Practices - Bash Tests
Server Fault - What is the difference between double and single brackets in bash?
[ is the same as the test builtin, and works like the test binary (man test)
works about the same as [ in all the other sh-based shells in many UNIX-like environments
only supports a single condition. Multiple tests with the bash && and || operators must be in separate brackets.
doesn't natively support a 'not' operator. To invert a condition, use a ! outside the first bracket to use the shell's facility for inverting command return values.
== and != are literal string comparisons
[[ is a bash
is bash-specific, though others shells may have implemented similar constructs. Don't expect it in an old-school UNIX sh.
== and != apply bash pattern matching rules, see "Pattern Matching" in man bash
has a =~ regex match operator
allows use of parentheses and the !, &&, and || logical operators within the brackets to combine subexpressions
Aside from that, they're pretty similar -- most individual tests work identically between them, things only get interesting when you need to combine different tests with logical AND/OR/NOT operations.
The most important difference will be the clarity of your code. Yes, yes, what's been said above is true, but [[ ]] brings your code in line with what you would expect in high level languages, especially in regards to AND (&&), OR (||), and NOT (!) operators. Thus, when you move between systems and languages you will be able to interpret script faster which makes your life easier. Get the nitty gritty from a good UNIX/Linux reference. You may find some of the nitty gritty to be useful in certain circumstances, but you will always appreciate clear code! Which script fragment would you rather read? Even out of context, the first choice is easier to read and understand.
if [[ -d $newDir && -n $(echo $newDir | grep "^${webRootParent}") && -n $(echo $newDir | grep '/$') ]]; then ...
or
if [ -d "$newDir" -a -n "$(echo "$newDir" | grep "^${webRootParent}")" -a -n "$(echo "$newDir" | grep '/$')" ]; then ...
In bash, contrary to [, [[ prevents word splitting of variable values.