Handling arithmetic expressions in shell scripting - shell

Kindly tell me that is it necessary to use "expr" keyword.
EG:-
echo `expr a*b`
And where we can simply handle arithmetic expressions using simple arithmetic operators.
EG:-
echo a*b
Thanks in advance.

In a Posix shell you can evaluate expressions directly in the shell when they are enclosed in
$(( ... ))
So:
a=12
b=34
echo $(($a + $b))
And although this wasn't always the case, all Posix shells you are likely to encounter will also deal with:
echo $((a + b))
This all happened because, a long time ago, the shell did not do arithmetic, and so the external program expr was written. These days, expr is usually a builtin (in addition to still being in /bin) and there is the Posix $((...)) syntax available. If $((...)) had been around from day one there would be no expr.
The shell is not exactly a normal computer language, and not exactly a macro processor: it's a CLI. It doesn't do inline expressions; an unquoted * is a wildcard for filename matching, because a CLI needs to reference files more often than it needs to do arithmetic.

The second form will almost surely never do what you want. In Bash, you have a built-in numeric expression handler, though:
A=4; B=6; echo $((A * B))

You can do arithmatic using $(())
echo $((2*3))
results in 6

Related

Different result from $((++n)) when running bash vs dash

I'm getting different outputs when running the program in bash and dash
#!/bin/sh
echo $SHELL
n=1
a=$((++n))
echo $n
Bash:
$ bash shell_test.sh
2
Dash:
$ dash shell_test.sh
1
dash is the Debian Almquist shell and an extreme light-weight version of a full POSIX-compliant shell-implementation of /bin/sh that aims to be as small as possible creating faster bootup times.
Operators such as $((n++)), $((--n)) and similar are features that are not required by POSIX and therefore not implemented.
To see how dash interprets these statements, see Chepner's answer
A nice page explaining how to make your script POSIX compliant, is here.
2.6.4 Arithmetic Expansion: Arithmetic expansion provides a mechanism for evaluating an arithmetic expression and substituting its value. The format for arithmetic expansion shall be as follows:
$((expression))
The expression shall be treated as if it were in double-quotes, except that a double-quote inside the expression is not treated specially. The shell shall expand all tokens in the expression for parameter expansion, command substitution, and quote removal.
Next, the shell shall treat this as an arithmetic expression and substitute the value of the expression. The arithmetic expression shall be processed according to the rules given in Arithmetic Precision and Operations, with the following exceptions:
Only signed long integer arithmetic is required.
Only the decimal-constant, octal-constant, and hexadecimal-constant constants specified in the ISO C standard, Section 6.4.4.1 are required to be recognized as constants.
The sizeof() operator and the prefix and postfix ++ and -- operators are not required.
Selection, iteration, and jump statements are not supported.
source: POSIX IEEE Std 1003.1-2017
Prefix ++ is not required by POSIX, and dash doesn't implement it. Instead, it's parsed as two unary + operators:
$ n=1
$ echo $((+(+n)))
1
$ echo $((++n))
1
$ echo $n
1

Expr and count in shell

I learn a shell code from a video,and there is a code:
#!/bin/sh
i=1
while true
do
echo $i
i='expr $i + 1'
done
The code above gives me output is:
1
expr $i + 1
expr $i + 1
expr $i + 1
......
But in the video the output are consistent numbers,why the same code shows different results?
As aready mentioned in a comment, you have the wrong quotes; the video will have had backticks (`, ASCII 96), not single quotes (', ASCII 39).
Additionally, anything which uses this obsolescent syntax for command substitution or expr is at least 20 years behind, and should be avoided in favor of modern POSIX constructs.
#!/bin/sh
i=1
while true
do
echo "$i" # Note proper quoting
i=$((i+1))
done
In Bash, you could use brace expansion or a C-style for loop, but these are not portable to modern sh.
If you need your code to be portable to pre-POSIX systems, my recommendation would still be to learn the modern constructs first, especially for simple code like this (where in a production system you would probably want to use Awk or Perl for a loop like this anyway).

A behavior when we parse non-number in arithmetic evaluation

I know from the manpage from bash that the variable that is null or unset is regarded as zero.
And I guess that non-number should be regarded as zero in arithmetic evaluation.
But without official ruling, it could be ambiguous like the second case of below example.
$ FOO=10
$ echo $((FOO))
10
$ FOO=10.abc
$ echo $((FOO))
bash: 10.abc: syntax error: invalid arithmetic operator (error token is ".abc")
atoi() from C parses the second example as 10.
What's the formal semantic of
parsing non-number in bash's arithmetic evaluation?
The $(( ... )) construct for arithmetic expansion was first introduced in the Korn shell back in the '80s, and then adopted for bash (which originally had a different syntax for arithmetic expansions) and POSIX.
Bash and ksh93 expect well-formed arithmetic expressions inside $(( ... )). They go way beyond atoi() in parsing them.
How the shells handle empty and unset variables is a red herring in this case. It's a sensible convenience that just happens to make sense in the context of the shell.

When do you use (( )) or /usr/bin/test

I have seen scripts using the test command and [ ] or [[ ]]. But when do we need to use /usr/bin/test and (( ))?
Are there any occasions when we need to go for the latter?
Regards,
John
To answer your question:
you want to use /usr/bin/test when you want to test something but not in a shell (for example find ... -exec test ...)
you want to use (( )) when you have an arithmetic expression to solve, AND you are using bash, because (( )) is bash specific.
Now for some background:
The command /usr/bin/test is required by the POSIX standard. POSIX also requires that [ is defined as an alias for test. The only difference between test and [ is that [ requires the final parameter to be a ].
Since test is used so frequently in shell scripts, most shells have a builtin version of test (and [). The advantage of a builtin is that it avoids context switches. Which, depending how you use test in your script, can be a measurable performance advantage.
I think it is safe to assume that under most circumstances it doesn't matter whether you use the system test or the shell's builtin test. But if you want to use test in a find -exec situation then of course you have to use the system test because find cannot use the shell test.
(( )) and [[ ]] were introduced by bash (and perhaps some other shells) as syntactic sugar. (( )) evaluates arithmetic expressions, while [[ ]] evaluates logical expressions. Both allow you to write the expressions in a "more natural syntax".
The decision to use [[ or [ depends on whether you want to use the "more natural syntax", and, since sh does not support [[, whether you want to depend on bash.
The decision to use (( )) depends on whether you need arithmetic expressions, and again, since sh does not support (( )), whether you want to depend on bash. The POSIX alternative to (( )) is $(( )). Note that there are some subtle differences in the behaviour.
The following links explain these topics in great detail:
http://mywiki.wooledge.org/BashFAQ/031 (difference between test, [ and [[)
http://mywiki.wooledge.org/ArithmeticExpression (let, (( )) and $(( )))
http://www.ibm.com/developerworks/library/l-bash-test/index.html (all of the above)
See also:
POSIX definition of test
POSIX definition of $(( ))
Bonus: Some debian developers once argued whether they should use the system test or the shell builtin test, because of some differences in the implementation of the builtin test. If you are interested in details of the differences of the system test and the shell builtin test then you can read the debian developer discussion here: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=267142.
(( )) evaluates an arithmetic expression in bash (see man bash).
[[ ]] evaluates a logic expression in bash (see man bash).
[ ] is the same as test, used to check file types and compare values (see man test).
You use /usr/bin/test when you want things to run more slowly. Modern shells (most shells released since about 1990, probably earlier) have test and its synonym [ as built-in commands. Formally invoking /usr/bin/test would be an act of desparation because the shell has a broken test command and the system standalone is OK - but it would be better to get a fixed shell.
You use (( ... )) to do arithmetic. The old-fashioned alternative was the expr command. That was tricky to use because it required a lot of escaping - it is/was a separate executable, and you had to get lots of shell metacharacters past the shell to expr. Hence:
x=$(expr $y '*' $z)
instead of
((x = y * z))
You don't even have to decorate the variables with $ in (( ... )).

Arithmetic expressions in Bash?

I had used several ways to do some simple integer arithmetic in BASH (3.2). But I can't figure out the best (preferred) way to do it.
result=`expr 1 + 2`
result=$(( 1 + 2 ))
let "result = 1 + 2"
What are the fundamental differences between those expressions?
Is there other ways to do the same?
Is the use of a tool like bc mandatory for floating point arithmetic?
result=`echo "7/354" | bc`
In Bash, let allows multiple assignments on a line:
let a=3 b=4 c=5
As you show in your question, quoting the argument to let allows you to put spaces around the operators. You can, however, omit the quotes if you avoid using spaces.
Another form using double parentheses at the beginning of the statement (instead of the i=$((j + 1)) form) allows you to include spaces around the equal sign or do post- or pre- increment or decrement and additional assignment operations:
(( a = ( b + c ) * 4 ))
(( count++ ))
(( d = --c**2 ))
(( e *= 2 ))
(( f = 3, g = 5 )) # multiple operations require a comma separator
If you do help "((" it says that the double parentheses is 'Equivalent to "let EXPRESSION".'
You can use the declare builtin to make assignments, including indirectly:
blue=2
name=blue
declare $name=4
echo $blue # result: 4
echo ${!name} # result: 4
Edit:
The $(()) construct is called "arithmetic expansion" and causes the contents to be evaluated as an integer expression. It's a syntax element of the shell.
If a variable is declared as an integer you don't need to use either form of double parentheses, you can omit the dollar sign from the variable name (as in the double-parentheses forms), but you can't add spaces around operators:
declare -i x=1 # set integer, initialize to 1
declare +i s=1 # clear integer, initialize to 1
x+=1 # could also be x=x+1
echo $x # result: 2 (addition)
s+=1 # could also be s=$s+1, requires a "$"
echo $s # result: 11 (string concatenation)
Unlike the forms above, calling expr involves spawning an external executable which can be quite expensive for a lot of calculations in a loop. The only time it should be used is in environments where the shell can't do its own arithmetic or for portability when a script may find its way into such an environment. POSIX shells have arithmetic capability so it would be a concern only with older systems.
Regarding the use of bc for floating point arithmetic, it or something similar is required when using Bash and many other shells. POSIX says that "Only signed long integer arithmetic is required."
Two shells that do support float math are ksh and zsh. In addition to bc, you can use dc, AWK, Python, Perl and others from within a Bash script.
One thing that Bash will do with floating point numbers is print them with the printf builtin (note that there is also an external printf, but builtins have priority).
printf "%'14.4f\n" 1234.56 # result " 1,234.5600" (in my locale)
I prefer your second option, since it doesn't need an external utility:
result=$(( 1 + 2 ))
The first option calls out to expr to do the math - I'm not familiar with let. Another alternative to bc is dc. Choose your favourite.
I can't say it's "mandatory" but bc is probably your best bet for general purpose arithmetic.
For something fancier, you can always pipe through Perl.
The downside of both thee approaches is that they both open a child process, so doing it in a tight loop will be slower than native bash expressions (same problem arises with use of backticks, in your first example). I'm not sure whether $(()) calls a child process.
Is the use of a tool like bc mandatory for floating point arithmetic?
No, if you are using a shell that supports floating point, eg zsh, ksh. Otherwise, if you want to do more advanced floating point maths, use either one of these, bc/awk/dc. Of course, Perl/Python etc as well.
The third option you have is a lot less readable as it does not look like an assignment operator. The first, as noted by others, calls an external command

Resources