I have a script named build_other:
count=0
while [ $count -lt 6 ]
do
./build
count+=1
done
From this script, you can see I am calling a second script, named build:
echo "building job"
sleep 30s
echo "wake after sleep"
echo "file build" >> output.txt
I expect that build should be triggered 5 times, because count increases by 1 after build runs. This means output.txt should have 5 lines saying file build. The following is the output I actually received.
building job
wake after sleep
building job
wake after sleep
The file output.txt has only two lines in it.
file build
file build
Why is the loop not running 5 times as I expected?
The += operator does not do what you think here; it is not an "add to" operator, it is acting as a string concatenation operator.
$ ./foo.sh
./build
count is now 01
./build
count is now 011
After two iterations, count is 011. Using the numeric less-than operator -lt, the string 011 is converted to the numeric 11. Since 11 is not less than 6, the loop ends.
Assuming you are using bash or another modern Bourne-family shell, you can solve this in a few ways.
# Using arithmetic expansion -- $(( expression ))
count=$((count + 1))
# Increment variable in an arithmetic evaluation context -- (( ))
(( count++ ))
# When declaring count, declare that it is an integer value. Declared an
# integer, the += operator will behave as you originally expected it would.
declare -i count=0
A few excerpts from the bash man page are below.
On the behavior of the += operator:
In the context where an assignment statement is assigning a value to a shell variable or array index, the += operator can be used to append to or add to the variable's previous value. When += is applied to a variable for which the integer attribute has been set, value is evaluated as an arithmetic expression and added to the variable's current value, which is also evaluated. When += is applied to an array variable using compound assignment (see Arrays below), the variable's value is not unset (as it is when using =), and new values are appended to the array beginning at one greater than the array's maximum index (for indexed arrays) or added as additional key-value pairs in an associative array. When applied to a string-valued variable, value is expanded and appended to the variable's value.
On arithmetic evaluation:
Arithmetic expansion allows the evaluation of an arithmetic expression and the substitution of the result. The format for arithmetic expansion is:
$((expression))
The old format $[expression] is deprecated and will be removed in upcoming versions of bash.
The expression is treated as if it were within double quotes, but a double quote inside the parentheses is not treated specially. All tokens in the expression undergo parameter and variable expansion, command substitution, and quote removal. The result is treated as the arithmetic expression to be evaluated. Arithmetic expansions may be nested.
On arithmetic evaluation context:
((expression))
The expression is evaluated according to the rules described below under ARITHMETIC EVALUATION. If the value of the expression is non-zero, the return status is 0; otherwise the return status is 1. This is exactly equivalent to let "expression".
Related
This question already has answers here:
How do I iterate over a range of numbers defined by variables in Bash?
(20 answers)
Variables in bash seq replacement ({1..10}) [duplicate]
(7 answers)
Closed 5 years ago.
#!/bin/sh
for i in {1..5}
do
echo "Welcome"
done
Would work, displays Welcome 5 times.
#!/bin/sh
howmany=`grep -c $1 /root/file`
for i in {1..$howmany}
do
echo "Welcome"
done
Doesn't work! howmany would equal 5 as that is what the output of grep -c would display. $1 is parameter 1 which is specific when running the script.
Any ideas?
Workarounds for not being able to use variables in a sequence brace expression:
If the intent is merely to iterate over numbers in a range - as in the OP's case - the best choice is not to use brace expansion, but instead use bash's C-style loop - see user000001's answer.
If the specific numbers aren't important and you simply need to execute a loop body a specified number of times, Cole Tierney's answer is an option.
If use of brace expansion is desired nonetheless:
If you do NOT need the numbers in the list to have a prefix or postfix, use the seq utility with an unquoted command substitution (small caveat: seq is NOT a POSIX utility, but it is widely available); e.g.
echo $(seq 3) -> 1 2 3; start number 1 implied
echo $(seq -f '%02.f' 3) -> 01 02 03 - zero-padded
echo $(seq 2 4) -> 2 3 4; explicit start and end numbers
echo $(seq 1 2 5) -> 1 3 5; custom increment (the 2 in the middle)
If you DO need the numbers in the list to have a prefix or postfix, you have several choices:
Use the seq utility with its -f option for providing a printf-style format string (as used above for zero-padding), or pure Bash workarounds based on eval (extra care needed!) or building an array in a loop, all of which are detailed in this answer.
You could also consider implementing the functionality generically, such as by writing a custom shell function or a custom script with utilities such as awk or perl.
Example of safe use of eval with variables driving a sequence brace expression:
The variables are validated beforehand, to make sure they contain decimal integers.
from=1 to=3 # sample values
# Ensure that $from and $to are decimal numbers and abort, if they are not.
(( 10#$from + 10#$to || 1 )) 2>/dev/null || { echo "Need decimal integers" >&2; exit 1; }
eval echo "A{$from..$to}" # -> 'A1 A2 A3'
General overview of brace expansion
The main purpose of brace expansion is to expand to a list of tokens with each token having an optional prefix and/or postfix; brace expansions must be unquoted and come in 2 flavors:
a fixed series (list) of comma-separated strings - variables supported
specifies and expands to a fixed number of tokens (2 or more); e.g.:
echo A{b,c,d} -> Ab Ac Ad, i.e., 3 tokens, as implied by the number of args.
echo {/,$HOME/}Library e.g., -> /Library /User/jdoe/Library
Variable references - and even globs - are supported, but note that they get expanded after brace expansion, in its result, in the course of normal evaluation.
a sequence expression (range) with .., typically numerical - variables NOT supported
expands to a variable number of tokens, driven by literal start and end points (for historical reasons, use of variables is NOT supported - see the comments on user000001's answer):
[rare] strings: only single English letters allowed; e.g. {a..c}
numbers: decimal integers only; e.g., {1..10}, {10..1}, {-1..2}
example with prefix and postfix: A{1..3}# -> A1# A2# A3#
broken example with variables: {$from..$to} # !! FAILS - $from and $to are interpreted as literals and therefore not recognized as either a single letter or a decimal integer - no brace expansion is performed (see below).
by contrast, using variables does work in zsh and ksh.
bash 4+ adds two features:
optional increment step value:
echo A{1..5..2} -> A1 A3 A5 - numbers incremented by 2
ability to zero-pad:
echo A{001..003} -> A001 A002 A003
An invalid brace expression is not expanded (treated like a regular unquoted string, with { and } treated as literals):
echo {} -> '{}' - invalid as a brace expr.: at least 2 ,-separated tokens needed
This allows the use of unquoted {} with find, for instance.
echo {1..$to} -> '{1..<value-of-$to>}' - invalid as a brace expr. in bash: variables not supported; however, valid in ksh and zsh.
(fish, by contrast, expands any {...} sequence; similarly, zsh has option BRACE_CCL (OFF by default) for expanding individual characters inside {..}, which effectively causes expansion of any nonempty {...} sequence.)
The brace expansion is evaluated before the variables are expanded. You need a c-style for loop instead:
for ((i=1;i<=howmany;i++))
do
echo "Welcome"
done
create a sequence to control your loop
for i in $(seq 1 $howmany); do
echo "Welcome";
done
The problem is that the "brace expansion" is performed before the "variable expansion"
for i in $(seq 1 $howmany)
works as #damienfrancois said, or, if you would like:
for i in $(eval echo "{$start..10}")
probably does, but don't use it for everyone's sanity.
You could also use a while loop:
while ((howmany--)); do
echo "Welcome"
done
We could also use eval in this case:
howmany=`grep -c $1 /root/file`
for i in $(eval echo {1..$howmany}); do
echo "Welcome"
done
In ABS guide, I could see below snippet
var1=5
var2=4
if (( var1 > var2 ))
then
echo "$var1 is greater than $var2"
fi
I am not able to understand, why we don't need $ symbol. I added $ symbol, shellcheck shows "$ symbol is not necessary on arithmetic variables".
I am still not able to understand how that dereferencing of var1 and var2 works...
Expressions inside ((...)) are evaluated in arithmetic context. Strings which can be variable names are considered as variables whose values are integers, since evaluating these strings as literal strings makes no sense in arithmetic context. These considerations are also valid for C style for loops: in for ((i = 0; i < 10; ++i)), preceding i with $ is not necessary (but it may be necessary, depending on the context, within the body of the loop).
(( ... )) follows the same evaluation rules as defined in the POSIX specification for arithmetic expressions. (The main difference is ((...)) produces an exit status reflecting if the result is zero/nonzero, while $((...)) produces the result as a string.) In particular:
If the shell variable x contains a value that forms a valid integer constant, optionally including a leading <plus-sign> or <hyphen-minus>, then the arithmetic expansions "$((x))" and "$(($x))" shall return the same value.
The shell variable var1 contains an integer constant, so $((var1)) and $(($var1)) are equivalent. This holds recursively as well.
Various shells seem to treat variables that refer to variables differently. Nothing in the POSIX wording seems to require the following kind of recursive evaluation, though both bash and dash do:
$ foo=bar
$ bar=5
$ echo $((foo)) # foo/$foo evaluates to bar, which contains an integer constant
5
bash seems to take it a step further, allowing any string to expand, followed by an attempt to evaluate the result as an arithmetic expression.
$ foo="x + 3"
$ x=5
$ echo $((foo)) # foo evaluates to x + 3, which evaluates to 5 + 3
8
but in dash:
$ foo="x+3"
$ x=5
$ echo $((foo))
dash: 3: Illegal number: x+3
It's documented in the manual in 6.5 Shell Arithmetic. It says
Within an expression, shell variables may also be referenced by name without using the parameter expansion syntax.
This features makes arithmetic expressions so much easier to read.
Interestingly, you can also refer to array elements like this as well
values=(42 54) i=0 j=1
echo $(( values[i] + values[j] )) # => 96
Other places you don't need $ (i.e. other arithmetic "contexts"):
For numerically-indexed arrays, the index (${values[i]} not ${values[$i]})
the offset and length parts of the ${var:offset:length} parameter expansion.
You don't need the $ if it's inside the (( )) for doing arithmetic.
I am still not able to understand how that dereferencing of var1 and var2 works
bash parses things differently inside the (( )), undoubtedly because it makes it much easier to read complex arithmetic expressions without the $.
I'm trying to understand a bash script and I'm having trouble with the following line:
result=${!#}
I found part of the solution (! within ${}) here:
If the first character of parameter is an exclamation point (!), it introduces a level of variable indirection. Bash uses the value of the variable formed from the rest of parameter as the name of the variable; this variable is then expanded and that value is used in the rest of the substitution, rather than the value of parameter itself. This is known as indirect expansion.
Another part of the solution (# within ${}) is here:
The length in characters of the expanded value of parameter is substituted. If parameter is ‘’ or ‘#’, the value substituted is the number of positional parameters. If parameter is an array name subscripted by ‘’ or ‘#’, the value substituted is the number of elements in the array. If parameter is an indexed array name subscripted by a negative number, that number is interpreted as relative to one greater than the maximum index of parameter, so negative indices count back from the end of the array, and an index of -1 references the last element.
But I don't know how these two are combined into result. What does this line do?
${#} is the number of arguments in the current shell/function:
$ set -- a b c
$ echo ${#}
3
The ! performs indirect parameter expansion, so the value of ${#} is used as the name of the parameter to expand.
$ echo ${!#} # same as echo ${3}
c
In short, ${!#} expands to the value of the last argument.
In the absence of such bash extensions, one might simply write a loop like
for result; do :; done # instead of result=${!#}
which would iterate over the positional arguments, setting result to each in turn, leaving result with the value of the last one once the loop completes.
This question already has answers here:
How do I iterate over a range of numbers defined by variables in Bash?
(20 answers)
Variables in bash seq replacement ({1..10}) [duplicate]
(7 answers)
Closed 5 years ago.
#!/bin/sh
for i in {1..5}
do
echo "Welcome"
done
Would work, displays Welcome 5 times.
#!/bin/sh
howmany=`grep -c $1 /root/file`
for i in {1..$howmany}
do
echo "Welcome"
done
Doesn't work! howmany would equal 5 as that is what the output of grep -c would display. $1 is parameter 1 which is specific when running the script.
Any ideas?
Workarounds for not being able to use variables in a sequence brace expression:
If the intent is merely to iterate over numbers in a range - as in the OP's case - the best choice is not to use brace expansion, but instead use bash's C-style loop - see user000001's answer.
If the specific numbers aren't important and you simply need to execute a loop body a specified number of times, Cole Tierney's answer is an option.
If use of brace expansion is desired nonetheless:
If you do NOT need the numbers in the list to have a prefix or postfix, use the seq utility with an unquoted command substitution (small caveat: seq is NOT a POSIX utility, but it is widely available); e.g.
echo $(seq 3) -> 1 2 3; start number 1 implied
echo $(seq -f '%02.f' 3) -> 01 02 03 - zero-padded
echo $(seq 2 4) -> 2 3 4; explicit start and end numbers
echo $(seq 1 2 5) -> 1 3 5; custom increment (the 2 in the middle)
If you DO need the numbers in the list to have a prefix or postfix, you have several choices:
Use the seq utility with its -f option for providing a printf-style format string (as used above for zero-padding), or pure Bash workarounds based on eval (extra care needed!) or building an array in a loop, all of which are detailed in this answer.
You could also consider implementing the functionality generically, such as by writing a custom shell function or a custom script with utilities such as awk or perl.
Example of safe use of eval with variables driving a sequence brace expression:
The variables are validated beforehand, to make sure they contain decimal integers.
from=1 to=3 # sample values
# Ensure that $from and $to are decimal numbers and abort, if they are not.
(( 10#$from + 10#$to || 1 )) 2>/dev/null || { echo "Need decimal integers" >&2; exit 1; }
eval echo "A{$from..$to}" # -> 'A1 A2 A3'
General overview of brace expansion
The main purpose of brace expansion is to expand to a list of tokens with each token having an optional prefix and/or postfix; brace expansions must be unquoted and come in 2 flavors:
a fixed series (list) of comma-separated strings - variables supported
specifies and expands to a fixed number of tokens (2 or more); e.g.:
echo A{b,c,d} -> Ab Ac Ad, i.e., 3 tokens, as implied by the number of args.
echo {/,$HOME/}Library e.g., -> /Library /User/jdoe/Library
Variable references - and even globs - are supported, but note that they get expanded after brace expansion, in its result, in the course of normal evaluation.
a sequence expression (range) with .., typically numerical - variables NOT supported
expands to a variable number of tokens, driven by literal start and end points (for historical reasons, use of variables is NOT supported - see the comments on user000001's answer):
[rare] strings: only single English letters allowed; e.g. {a..c}
numbers: decimal integers only; e.g., {1..10}, {10..1}, {-1..2}
example with prefix and postfix: A{1..3}# -> A1# A2# A3#
broken example with variables: {$from..$to} # !! FAILS - $from and $to are interpreted as literals and therefore not recognized as either a single letter or a decimal integer - no brace expansion is performed (see below).
by contrast, using variables does work in zsh and ksh.
bash 4+ adds two features:
optional increment step value:
echo A{1..5..2} -> A1 A3 A5 - numbers incremented by 2
ability to zero-pad:
echo A{001..003} -> A001 A002 A003
An invalid brace expression is not expanded (treated like a regular unquoted string, with { and } treated as literals):
echo {} -> '{}' - invalid as a brace expr.: at least 2 ,-separated tokens needed
This allows the use of unquoted {} with find, for instance.
echo {1..$to} -> '{1..<value-of-$to>}' - invalid as a brace expr. in bash: variables not supported; however, valid in ksh and zsh.
(fish, by contrast, expands any {...} sequence; similarly, zsh has option BRACE_CCL (OFF by default) for expanding individual characters inside {..}, which effectively causes expansion of any nonempty {...} sequence.)
The brace expansion is evaluated before the variables are expanded. You need a c-style for loop instead:
for ((i=1;i<=howmany;i++))
do
echo "Welcome"
done
create a sequence to control your loop
for i in $(seq 1 $howmany); do
echo "Welcome";
done
The problem is that the "brace expansion" is performed before the "variable expansion"
for i in $(seq 1 $howmany)
works as #damienfrancois said, or, if you would like:
for i in $(eval echo "{$start..10}")
probably does, but don't use it for everyone's sanity.
You could also use a while loop:
while ((howmany--)); do
echo "Welcome"
done
We could also use eval in this case:
howmany=`grep -c $1 /root/file`
for i in $(eval echo {1..$howmany}); do
echo "Welcome"
done
All of these options below work for setting i to count.
count=5
for (( i=count; i>=0; i-- )); do
echo "$i"
done
for (( i=$count; i>=0; i-- )); do
echo "$i"
done
for (( i=$((count)); i>=0; i-- )); do
echo "$i"
done
for (( i=${count}; i>=0; i-- )); do
echo "$i"
done
Which is correct or preferred?
Generally, in arithmetic contexts such as ((...)) and $((...)), reference variables by name only, without the $ prefix (as your commands are already doing with respect to variable $i):
for (( i=count; i>=0; i-- )); do
echo "$i"
done
Since ((...)) is an arithmetic context itself, there is no good reason to use a separate, expanding arithmetic context - $((count)) - inside of it.
Note that $count and ${count} are equivalent, and enclosing the variable name - count - in { and } after $ is only necessary to disambiguate the variable name from subsequent characters that can also legally be part of a variable name (which doesn't apply to your commands).
As Gordon Davisson points out, some people choose to always use the ${var} form for visual clarity.
While $-prefixed variable references in arithmetic contexts do work, there is rarely a good reason to use them: the use of $ introduces an extra expansion step before the arithmetic evaluation, which is not only unnecessary, but can result in different behavior, as explained in rici's helpful answer.
The only cases where you need the $ prefix:
To reference positional and special parameters (variables) that can only ever be referenced with $: Thanks, rici.
Positional parameters: $1, $2, ...
The count of positional parameters: $#
Special parameters: $?, $$, $! (there are others, but they are not generally numeric - see chapter Special Parameters in man bash).
If you need a nonzero default value that you provide via a parameter expansion; e.g., ${count:-2} defaults to 2 if $count is unset or empty.
If you want to use a variable value as an operator rather than operand; e.g.:
op='*'; echo $(( 2 $op 2 )) - this wouldn't work with just op.
There is a difference between using $x and just x in an arithmetic context:
x causes the value of x to be evaluated as a number. If x hasn't been defined, the result is 0. If the value of x is a valid arithmetic expression, that expression is evaluated. If it is an invalid expression, the result is a syntax error.
$x causes the value of x as a string to be interpolated into the arithmetic expression, which will then be evaluated.
This leads to different evaluations, particularly with uninitialized variables:
$ unset x
$ echo $((x/2))
0
$ echo $(($x/2))
bash: /2: syntax error: operand expected (error token is "/2")
# Also, with incomplete expressions
$ x=42+
$ echo $((x 7))
bash: 42+: syntax error: operand expected (error token is "+")
$ echo $(($x 7))
49
In both cases, I prefer the behaviour associated with the unadorned use of the variable name. Consequently, I recommend its consistent use in arithmetic expressions unless you have a really good reason not to (in which case, you probably should quote the expansion to make it clearer what your expectation is.)
I would avoid $((count)), as this creates an unnecessary arithmetic evaluation step. The other ones have different syntax, but the exact same semantics. I would use the first one, for no better reason than it being shorter.
I suggest the still equivalent (in its result), but shorter :
for (( i=count+1; i-->1 ;)); do
echo "$i"
done
I have always liked that one, because it visually says "i goes toward 0".