the following script with debug option 'set -e -v' fails at the increment operator only when the variable has a prior value of zero.
#!/bin/bash
set -e -v
i=1; let i++; echo "I am still here"
i=0; let i++; echo "I am still here"
i=0; ((i++)); echo "I am still here"
bash (GNU bash, version 4.0.33(1)-release (x86_64-apple-darwin10) but also GNU bash, version 4.2.4(1)-release (x86_64-unknown-linux-gnu))
any ideas?
the answer to my question is not to use let (or shift, or...) but to use
i=$((i+1))
when trying to check a bash script by setting 'exit on non-zero status code' with
set -e
The bash manual states that set -e has the effect of 'Exit immediately if a simple command exits with a non-zero status.'.
Unfortunately let (and shift and ...) return the result of the computation ('If the last arg evaluates to 0, let returns 1; 0 is returned otherwise'). So instead of a status code one gets a return value of some sort. And sometimes this return value will be zero and sometimes one depending on the computation. Therefore set -e will cause the script to exit depending on the result of your computation!!! and there is nothing to do about it unless either you don't use it ever or resort to
let i++ || true
as pointed by arnaud576875 which btw adds extra CPU burden.
Using
let ++i
works only for the specific case that i is not -1, as with let i++ which works only for when i is not 0. Therefore half-solutions.
I love Unix though, I wouldn't have it any other way.
If the last argument of let evaluates to 0, let returns 1 (so, a non-zero status):
From the manual:
let arg [arg ...]
Each arg is an arithmetic expression to be evaluated. If the last arg evaluates to 0, let returns 1; 0 is returned otherwise.
i++ evaluates to zero when i is 0 (because it's a post-increment, so the previous value of i is returned), so let returns 1, and due to set -e, bash exists.
Here are some solutions:
let ++i # pre-increment, if you expect `i` to never be -1
let i++ 1 # add an expression evaluating to non-zero
let i++ || true # call true if let returns non-zero
Looking at the BASH manpage on the set -e:
Exit immediately if a simple command (see SHELL GRAMMAR above) exits with a non-zero status. [...]
So, if any statement returns a non-zero exit code, the shell will exit.
Taking a look at the BASH manpage, on the let command:
If the last arg evaluates to 0, let returns 1; 0 is returned otherwise.
But wait! The answer to i++ is a one and not a zero! It should have worked!
Again, the answer is with the BASH manpage on the increment operator:
id++ id--: variable post-increment and post-decrement
Okay, not so clear. Try this shell script:
#!/bin/bash
set -e -v
i=1; let ++i; echo "I am still here"
i=0; let ++i; echo "I am still here"
i=0; ((++i)); echo "I am still here"
Hmmm... that works as expected, and all I did was change i++ to ++i in each line.
The i++ is a post-increment operator. That means, it increments i after the let statement returns a value. Since i was zero before being incremented, the let statement returns a non-zero value.
However, the ++i is a pre-increment operator. That means it increments i before returning the exit status. Since i is incremented to a 1, the exit status becomes a zero.
I hope this makes sense.
Related
I am running the following script using tcsh. In my while loop, I'm running a C++ program that I created and will return a different exit code depending on certain things. While it returns an exit code of 0, I want the script to increment counter and run the program again.
#!/bin/tcsh
echo "Starting the script."
set counter = 0
while ($? == 0)
# counter ++
./auto $counter
end
I have verified that my program is definitely returning with exit code = 1 after a certain point. However, the condition in the while loop keeps evaluating to true for some reason and running.
I found that if I stick the following line at the end of my loop and then replace the condition check in the while loop with this new variable, it works fine.
while ($return_code == 0)
# counter ++
./auto $counter
set return_code = $?
end
Why is it that I can't just use $? directly? Is another operation underneath the hood performed in between running my custom program and checking the loop condition that's causing $? to change value?
That is peculiar.
I've altered your example to something that I think illustrates the issue more clearly. (Note that $? is an alias for $status.)
#!/bin/tcsh -f
foreach i (1 2 3)
false
# echo false status=$status
end
echo Done status=$status
The output is
Done status=0
If I uncomment the echo command in the loop, the output is:
false status=1
false status=1
false status=1
Done status=0
(Of course the echo in the loop would break the logic anyway, because the echo command completes successfully and sets $status to zero.)
I think what's happening is that the end that terminates the loop is executed as a statement, and it sets $status ($?) to 0.
I see the same behavior with both tcsh and bsd-csh.
Saving the value of $status in another variable immediately after the command is a good workaround -- and arguably just a better way of doing it, since $status is extremely fragile, and will almost literally be clobbered if you look at it.
Note that I've add a -f option to the #! line. This prevents tcsh from sourcing your init file(s) (.cshrc or .tcshrc) and is considered good practice. (That's not the case for sh/bash/ksh/zsh, which assign a completely different meaning to -f.)
A digression: I used tcsh regularly for many years, both as my interactive login shell and for scripting. I would not have anticipated that end would set $status. This is not the first time I've had to find out how tcsh or csh behaves by trial and error and been surprised by the result. It is one of the reasons I switched to bash for interactive and scripting use. I won't tell you to do the same, but you might want to read Tom Christiansen's classic "csh.whynot".
Slightly shorter/simpler explanation:
Recall that with tcsh/csh EACH command (including shell builtin) return a status. Therefore $? (aliases to $status) is updated by 'if' statements, 'for' loops, assignments, ...
From practical point of view, better to limit the usage of direct use of $? to an if statement after the command execution:
do-something
if ( $status == 0 )
...
endif
In all other cases, capture the status in a variable, and use only that variable
do-something
something_status=$?
if ( $something_status == 0 )
...
endif
To expand on the $status, even a condition test in an if statement will modify the status, therefore the following repeated test on $status will not never hit the '$status == 5', even when do-something will return status of 5
do-something
if ( $status == 2 ) then
echo FOO
else if ( $status == 5 ) then
echo BAR
endif
Why doesn't the following snippet work ?
set -ue
inc=0
((inc++))
((inc++))
echo $inc
echo Here
[output nothing, return code 1]
But when disabling the '-e' it does work as intended? I ran this with "GNU bash, version 4.3.46(1)-release"
Because of this (from the bash man page):
((expression))
The expression is evaluated according to the rules described below
under ARITHMETIC EVALUATION. If the value of the expression is
non-zero, the return status is 0; otherwise the return status is 1.
This is exactly equivalent to let "expression".
So when inc is 0 and you run:
((inc++))
...the value of the expression is 0 (because you're using the postfix ++ operator), so the return value is 1, which means that your script exits when -e is in effect. The easiest way to solve this particular issue is to use the prefix ++ operator:
set -ue
inc=0
((++inc))
((++inc))
echo $inc
echo Here
Update
As #cdarke mentions, you can instead use the : command, which is a special shell command that means "do nothing but evaluate all the arguments". You will often encounter this in shell scripts where it is used to variable defaults like this:
: ${SOMEVAR:=somevalue}
Or in while loops, like this:
while :; do
...
done
So instead of:
((inc++))
You can do this:
: $((inc++))
But you'll notice two changes there (and this is why I didn't mention it in my original answer). Because : is itself a command, you can no longer use the ((...)) syntax by itself (which is exactly equivalent to the let command). Instead, you need to use the arithmetic expression syntax, $((...)).
You could also do something like this:
((inc++)) || true
Or even:
((inc++)) || :
Which similarly have the effect of suppressing the error return code from the expression.
I'm learning bash and I noticed something weird I can't (yet) explain. At school I learned that an if statement evaluates 0 as true and 1 as false, so it can be used with the status code from an other command. Now is my question: Why does this happen:
echo $((5>2)) #prints 1
echo $((5<2)) #prints 0
if ((5>2)) ; then echo "yes" ; else echo "no" ; fi #prints yes
if ((5<2)) ; then echo "yes" ; else echo "no" ; fi #prints no
This doesn't seem logical. How does bash know i'm using an arithmetic expression and not another command?
You're confusing the output of the command substitution and the return value of the arithmetic context.
The output of the command substitution is 1 for true and 0 for false.
The return value of (( )) is 0 (success) if the result is true (i.e. non-zero) and 1 if it is false.
if looks at the return value, not the output of the command.
$((...)) is an arithmetic expression; it expands to the value of the expression inside the parentheses. Since it is not a command, it does not have an exit status or return value of its own. A boolean expression evaluates to 1 if it is true, 0 if it is false.
((...)), on the other hand, is an arithmetic statement. It is a command in its own right, which works be evaluating its body as an arithmetic expression, then looking at the resulting value. If the value is true, the command succeeds and has an exit status of 0. If the value is false, the command fails, and has an exit status of 1.
It's a good idea when learning bash to stop thinking of the conditions in if statements, while loops, etc. as being true or false, but rather if the commands succeed or fail. After all, shell languages are not designed for data processing; they are a glue language for running other programs.
From the bash manual:
((expression))
The expression is evaluated according to the rules described below under ARITHMETIC EVALUATION. If the value of the expression is non-zero, the return status is 0; otherwise the return status is 1. (My emphasis.)
So essentially in a boolean context ((expression)) gives the inverse of what $((expression)) would give.
Can anyone explain what is going on here?
Why does an "IF" statement think the return is '1' (not Null) when i say 'Return 0' and the other way round.
I found that out while coding another script, so i developed this small script to test it:
#!/bin/bash
function testreturnzero()
{
echo $1
return 0
}
function testreturnone()
{
echo $1
return 1
}
if (testreturnzero 1) || (testreturnzero 2)
then
echo "zero returned '1'"
fi
if (testreturnone 1) || (testreturnone 2)
then
echo "one returned '1'"
fi
The IF which refers to the 'return 0' thinks its true (and doesn't process the second function), the IF which refers to 'return 1' thinks its false. Shouldn't it be the exact opposite?
1
zero returned '1'
1
2
I cant put the return value in a variable as I will have several of those checks.
In bash, a function returning 0 means success and returning a non-zero value means failure. Hence your testreturnzero succeeds and your testreturnone fails.
Does that help understanding why your ifs behave that way? (it should!).
The return code of the last executed command/function is stored in the special variable $?.
So:
testreturnzero 0
ret_testreturnzero=$?
testreturnone 1
ret_testreturnone=$?
echo "$ret_testreturnzero"
echo "$ret_testreturnone"
will output (the two last lines):
0
1
Now you may think of storing them in a variable (as here) and do your logic processing later. But there's a catch :). Because you didn't store true and false in variables, you stored 0 and 1 (bash can't store booleans in a variable). So to check success or failure later:
if ((ret_testreturnzero==0)); then
echo "testreturnzero succeeded"
fi
or
if ((ret_testreturnzero!=0)); then
echo "testreturnzero failed"
fi
In bash, the return code of a function is the same as a external program when you test the result.
So for test, a valid return code is 0 and an invalid is any other number
so, by doing
if ( testreturnone 1 ); then #it is ok
echo "error"; #it's supposed to happen, not an error
fi
You can explicitly test the value to the one you want to clear it up:
if [[ "$(testreturnzero 1)" = "1"); then #it is ok if you decide that 1 is the good value
echo "ok"; #But absolutly not the bash philosophy
fi
I find this behaviour counter-intuitive:
v=0; ((v)) && echo K; v=1; ((v)) && echo J
echoes:
J
but generally 0 is success:
f() { return 0; }; if f; then echo W; fi
Explain the reasoning?
man page for (( )) says:
If the value of the expression is non-zero, the return status is 0; otherwise the return status is 1.
How best to check if a variable is zero using Arithmetic Evaluation?
The canonical way to test for 0 is [ $something -eq 0 ], often written as [ x$something = x0 ]. (Using = is a string comparison but it is still valid syntax when the variable isn't set or is otherwise blank.) See man expr.
So,
q=0
if [ $q -eq 0 ]; then # or [ x$q = x0 ]
echo yes, zero
fi
The ((expr)) syntax is a shortcut for the let command, and that command is simply defined to return true status when the expression is non-zero and false otherwise.
And neither ((...)) nor let ... are Posix commands, so some amount of WTF?! may be expected. It's just something the authors of bash either thought up or pulled in from some now-obscure earlier shell. (It seems to be in zsh.) It is not in ash(1), aka dash(1), which is kind of the reference pure-Posix shell.
One good reason to "flip" the meaning to 0 sucess any value not 0 not sucess is that you might want to indicate different error conditions.
Using 0 as failure this wouldn't be possible at all as any value != 0 would have been wasted to indicate sucess.
Given a numeric variable the best way to check if the value is 0 is to do exactly that:
((v==0))