What is the use case of noop [:] in bash? - bash

I searched for noop in bash (:), but was not able to find any good information. What is the exact purpose or use case of this operator?
I tried following and it's working like this for me:
[mandy#root]$ a=11
[mandy#root]$ b=20
[mandy#root]$ c=30
[mandy#root]$ echo $a; : echo $b ; echo $c
10
30
Please let me know, any use case of this operator in real time or any place where it is mandatory to use it.

It's there more for historical reasons. The colon builtin : is exactly equivalent to true. It's traditional to use true when the return value is important, for example in an infinite loop:
while true; do
echo 'Going on forever'
done
It's traditional to use : when the shell syntax requires a command but you have nothing to do.
while keep_waiting; do
: # busy-wait
done
The : builtin dates all the way back to the Thompson shell, it was present in Unix v6. : was a label indicator for the Thompson shell's goto statement. The label could be any text, so : doubled up as a comment indicator (if there is no goto comment, then : comment is effectively a comment). The Bourne shell didn't have goto but kept :.
A common idiom that uses : is : ${var=VALUE}, which sets var to VALUE if it was unset and does nothing if var was already set. This construct only exists in the form of a variable substitution, and this variable substitution needs to be part of a command somehow: a no-op command serves nicely.
See also What purpose does the colon builtin serve?.

I use it for if statements when I comment out all the code. For example you have a test:
if [ "$foo" != "1" ]
then
echo Success
fi
but you want to temporarily comment out everything contained within:
if [ "$foo" != "1" ]
then
#echo Success
fi
Which causes bash to give a syntax error:
line 4: syntax error near unexpected token `fi'
line 4: `fi'
Bash can't have empty blocks (WTF). So you add a no-op:
if [ "$foo" != "1" ]
then
#echo Success
:
fi
or you can use the no-op to comment out the lines:
if [ "$foo" != "1" ]
then
: echo Success
fi

If you use set- e then || : is a great way to not exit the script if a failure happens (it explicitly makes it pass).

You would use : to supply a command that succeeds but doesn't do anything. In this example the "verbosity" command is turned off by default, by setting it to :. The 'v' option turns it on.
#!/bin/sh
# example
verbosity=:
while getopts v OPT ; do
case $OPT in
v)
verbosity=/bin/realpath
;;
*)
exit "Cancelled"
;;
esac
done
# `$verbosity` always succeeds by default, but does nothing.
for i in * ; do
echo $i $($verbosity $i)
done
$ example
file
$ example -v
file /home/me/file

One use is as multiline comments, or to comment out part of your code for testing purposes by using it in conjunction with a here file.
: << 'EOF'
This part of the script is a commented out
EOF
Don't forget to use quotes around EOF so that any code inside doesn't get evaluated, like $(foo). It also might be worth using an intuitive terminator name like NOTES, SCRATCHPAD, or TODO.

Ignoring alias arguments
Some times you want to have an alias that doesn't take any argument. You can do it using ::
> alias alert_with_args='echo hello there'
> alias alert='echo hello there;:'
> alert_with_args blabla
hello there blabla
> alert blabla
hello there

Two of mine.
Embed POD comments
A quite funky application of : is for embedding POD comments in bash scripts, so that man pages can be quickly generated. Of course, one would eventually rewrite the whole script in Perl ;-)
Run-time function binding
This is a sort of code pattern for binding functions at run-time.
F.i., have a debugging function to do something only if a certain flag is set:
#!/bin/bash
# noop-demo.sh
shopt -s expand_aliases
dbg=${DBG:-''}
function _log_dbg {
echo >&2 "[DBG] $#"
}
log_dbg_hook=':'
[ "$dbg" ] && log_dbg_hook='_log_dbg'
alias log_dbg=$log_dbg_hook
echo "Testing noop alias..."
log_dbg 'foo' 'bar'
You get:
$ ./noop-demo.sh
Testing noop alias...
$ DBG=1 ./noop-demo.sh
Testing noop alias...
[DBG] foo bar

Somewhat related to this answer, I find this no-op rather convenient to hack polyglot scripts. For example, here is a valid comment both for bash and for vimscript:
":" # this is a comment
":" # in bash, ‘:’ is a no-op and ‘#’ starts a comment line
":" # in vimscript, ‘"’ starts a comment line
Sure, we may have used true just as well, but : being a punctuation sign and not an irrelevant English word makes it clear that it is a syntax token.
As for why would someone do such a tricky thing as writing a polyglot script (besides it being cool): it proves helpful in situations where we would normally write several script files in several different languages, with file X referring to file Y.
In such a situation, combining both scripts in a single, polyglot file avoids any work in X for determining the path to Y (it is simply "$0"). More importantly, it makes it more convenient to move around or distribute the program.
A common example. There is a well-known, long-standing issue with shebangs: most systems (including Linux and Cygwin) allow only one argument to be passed to the interpreter. The following shebang:
#!/usr/bin/env interpreter --load-libA --load-libB
will fire the following command:
/usr/bin/env "interpreter --load-libA --load-libB" "/path/to/script"
and not the intended:
/usr/bin/env interpreter --load-libA --load-libB "/path/to/script"
Thus, you would end up writing a wrapper script, such as:
#!/usr/bin/env sh
/usr/bin/env interpreter --load-libA --load-libB "/path/to/script"
This is where polyglossia enters the stage.
A more specific example. I once wrote a bash script which, among other things, invoked Vim. I needed to give Vim additional setup, which could be done with the option --cmd "arbitrary vimscript command here". However, that setup was substantial, so that inlining it in a string would have been terrible (if ever possible). Hence, a better solution was to write it in extenso in some configuration file, then make Vim read that file with -S "/path/to/file". Hence I ended up with a polyglot bash/vimscript file.

suppose you have a command you wish to chain to the success of another:
cmd="some command..."
$cmd
[ $? -eq 0 ] && some-other-command
but now you want to execute the commands conditionally and you want to show the commands that would be executed (dry-run):
cmd="some command..."
[ ! -z "$DEBUG" ] && echo $cmd
[ -z "$NOEXEC" ] && $cmd
[ $? -eq 0 ] && {
cmd="some-other-command"
[ ! -z "$DEBUG" ] && echo $cmd
[ -z "$NOEXEC" ] && $cmd
}
so if you set DEBUG and NOEXEC, the second command never shows up. this is because the first command never executes (because NOEXEC is not empty) but the evaluation of that fact leaves you with a return of 1, which means the subordinate command never executes (but you want it to because it's a dry run). so to fix this you can reset the exit value left on the stack with a noop:
[ -z "$NOEXEC" ] && $cmd || :

Sometimes no-op clauses can make your code more readable.
That can be a matter of opinion, but here's an example. Let's suppose you've created a function that works by taking two unix paths. It calculates the 'change path' needed to cd from one path to another. You place a restriction on your function that the paths must both start with a '/' OR both must not.
function chgpath() {
# toC, fromC are the first characters of the argument paths.
if [[ "$toC" == / && "$fromC" == / ]] || [[ "$toC" != / && "$fromC" != / ]]
then
true # continue with function
else
return 1 # Skip function.
fi
Some developers will want to remove the no-op but that would mean negating the conditional:
function chgpath() {
# toC, fromC are the first characters of the argument paths.
if [[ "$toC" != / || "$fromC" == / ]] && [[ "$toC" == / || "$fromC" != / ]]
then
return 1 # Skip function.
fi
Now -in my opinion- its not so clear from the if-clause the conditions in which you'd want to skip doing the function. To eliminate the no-op and do it clearly, you would want to move the if-clause out of the function:
if [[ "$toC" == / && "$fromC" == / ]] || [[ "$toC" != / && "$fromC" != / ]]
then
cdPath=$(chgPath pathA pathB) # (we moved the conditional outside)
That looks better, but many times we can't do this; we want the check to be done inside the function.
So how often does this happen? Not very often. Maybe once or twice a year. It happens often enough, that you should be aware of it. I don't shy away from using it when I think it improves the readability of my code (regardless of the language).

I've also used in it scripts to define default variables.
: ${VARIABLE1:=my_default_value}
: ${VARIABLE2:=other_default_value}
call-my-script ${VARIABLE1} ${VARIABLE2}

I sometimes use it on Docker files to keep RUN commands aligned, as in:
RUN : \
&& somecommand1 \
&& somecommand2 \
&& somecommand3
For me, it reads better than:
RUN somecommand1 \
&& somecommand2 \
&& somecommand3
But this is just a matter of preference, of course

null command [:] is actually considered a synonym for the shell builtin true. The ":" command is itself a Bash builtin, and its exit status is true (0).
`
$ :
$ echo $? # 0
while :
do
operation-1
operation-2
...
operation-n
done
# Same as:
while true
do
...
done
Placeholder in if/then test:
if condition
then : # Do nothing and branch ahead
else # Or else ...
take-some-action
fi
$ : ${username=`whoami`}
$ ${username=`whoami`} #Gives an error without the leading :
Source: TLDP

I used the noop today when I had to create a mock sleep function to use in bats testing framework. This allowed me to create an empty function with no side effects:
function sleep() {
:
}

Related

Prevent bash completion from printing the reply twice

I am having difficulties making my bash completion script behave as expected.
Below is the most simplified version of what I am trying to do, and a screencast of the problem.
Everything works great, with the exception of one case:
When there is one match only in the completion, when I press Tab twice, it will print the same command twice, which is not the intended effect.
In my completion case, I am only interested in completing the first argument (i.e. the "subcommand"). I thought that the if [ "$#" -gt "2" ] line below should have prevented this case, but it does not - since COMP_LINE is still not updated after the first tab.
#!/usr/bin/env bash
send_completions() {
if [[ -z "$COMP_LINE" ]]; then
return
fi
set $COMP_LINE
if [ "$#" -gt "2" ]; then
return
fi
compgen -W "one two test" "$2"
}
send_completions
Enable the completion by running:
$ complete -C './complete.sh' asd
then type asd o<tab><tab> to see the problem.
Any help is appreciated.
Check the value of COMP_LINE at this very moment:
$ asd one <TAB>
You will notice that the value is:
declare -x COMP_LINE="asd one "
Which will result in two positional parameters after you issue set and your condition will not get triggered. Try typing an additional character: $ asd one o<TAB> and you will see that your condition will get triggered correctly and nothing will be completed.
Most completions I have seen were done using a function. In fact, checking all completions defined on my system now (most of them default), none of them are done using a script.
Using a function has some benefits, for example the COMP_WORDS array that respects quotes. Your script could be rewritten roughly like this:
_asd_comp() {
(( COMP_CWORD > 1 )) && return
local cur=${COMP_WORDS[COMP_CWORD]} IFS=$' \t\n'
COMPREPLY=( $(compgen -W "one two three" -- "$cur") )
}
complete -o bashdefault -o default -F _asd_comp asd
I added the -o options so that default completions are performed after the first argument. You can read more about that in manual.
The only thing that comes to mind as to how to change your script, I guess you could add rather unclean condition, such as:
(( $# > 1 )) && [[ ${COMP_LINE: -1} = " " ]] && return

BASH - never execute unless environment variable is defined and certain value [duplicate]

I've got a few Unix shell scripts where I need to check that certain environment variables are set before I start doing stuff, so I do this sort of thing:
if [ -z "$STATE" ]; then
echo "Need to set STATE"
exit 1
fi
if [ -z "$DEST" ]; then
echo "Need to set DEST"
exit 1
fi
which is a lot of typing. Is there a more elegant idiom for checking that a set of environment variables is set?
EDIT: I should mention that these variables have no meaningful default value - the script should error out if any are unset.
Parameter Expansion
The obvious answer is to use one of the special forms of parameter expansion:
: ${STATE?"Need to set STATE"}
: ${DEST:?"Need to set DEST non-empty"}
Or, better (see section on 'Position of double quotes' below):
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
The first variant (using just ?) requires STATE to be set, but STATE="" (an empty string) is OK — not exactly what you want, but the alternative and older notation.
The second variant (using :?) requires DEST to be set and non-empty.
If you supply no message, the shell provides a default message.
The ${var?} construct is portable back to Version 7 UNIX and the Bourne Shell (1978 or thereabouts). The ${var:?} construct is slightly more recent: I think it was in System III UNIX circa 1981, but it may have been in PWB UNIX before that. It is therefore in the Korn Shell, and in the POSIX shells, including specifically Bash.
It is usually documented in the shell's man page in a section called Parameter Expansion. For example, the bash manual says:
${parameter:?word}
Display Error if Null or Unset. If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard error and the shell, if it is not interactive, exits. Otherwise, the value of parameter is substituted.
The Colon Command
I should probably add that the colon command simply has its arguments evaluated and then succeeds. It is the original shell comment notation (before '#' to end of line). For a long time, Bourne shell scripts had a colon as the first character. The C Shell would read a script and use the first character to determine whether it was for the C Shell (a '#' hash) or the Bourne shell (a ':' colon). Then the kernel got in on the act and added support for '#!/path/to/program' and the Bourne shell got '#' comments, and the colon convention went by the wayside. But if you come across a script that starts with a colon, now you will know why.
Position of double quotes
blong asked in a comment:
Any thoughts on this discussion? https://github.com/koalaman/shellcheck/issues/380#issuecomment-145872749
The gist of the discussion is:
… However, when I shellcheck it (with version 0.4.1), I get this message:
In script.sh line 13:
: ${FOO:?"The environment variable 'FOO' must be set and non-empty"}
^-- SC2086: Double quote to prevent globbing and word splitting.
Any advice on what I should do in this case?
The short answer is "do as shellcheck suggests":
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
To illustrate why, study the following. Note that the : command doesn't echo its arguments (but the shell does evaluate the arguments). We want to see the arguments, so the code below uses printf "%s\n" in place of :.
$ mkdir junk
$ cd junk
$ > abc
$ > def
$ > ghi
$
$ x="*"
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
abc
def
ghi
$ unset x
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
bash: x: You must set x
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
bash: x: You must set x
$ x="*"
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
*
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
abc
def
ghi
$ x=
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$ unset x
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$
Note how the value in $x is expanded to first * and then a list of file names when the overall expression is not in double quotes. This is what shellcheck is recommending should be fixed. I have not verified that it doesn't object to the form where the expression is enclosed in double quotes, but it is a reasonable assumption that it would be OK.
Try this:
[ -z "$STATE" ] && echo "Need to set STATE" && exit 1;
Your question is dependent on the shell that you are using.
Bourne shell leaves very little in the way of what you're after.
BUT...
It does work, just about everywhere.
Just try and stay away from csh. It was good for the bells and whistles it added, compared the Bourne shell, but it is really creaking now. If you don't believe me, just try and separate out STDERR in csh! (-:
There are two possibilities here. The example above, namely using:
${MyVariable:=SomeDefault}
for the first time you need to refer to $MyVariable. This takes the env. var MyVariable and, if it is currently not set, assigns the value of SomeDefault to the variable for later use.
You also have the possibility of:
${MyVariable:-SomeDefault}
which just substitutes SomeDefault for the variable where you are using this construct. It doesn't assign the value SomeDefault to the variable, and the value of MyVariable will still be null after this statement is encountered.
Surely the simplest approach is to add the -u switch to the shebang (the line at the top of your script), assuming you’re using bash:
#!/bin/sh -u
This will cause the script to exit if any unbound variables lurk within.
${MyVariable:=SomeDefault}
If MyVariable is set and not null, it will reset the variable value (= nothing happens).
Else, MyVariable is set to SomeDefault.
The above will attempt to execute ${MyVariable}, so if you just want to set the variable do:
MyVariable=${MyVariable:=SomeDefault}
In my opinion the simplest and most compatible check for #!/bin/sh is:
if [ "$MYVAR" = "" ]
then
echo "Does not exist"
else
echo "Exists"
fi
Again, this is for /bin/sh and is compatible also on old Solaris systems.
bash 4.2 introduced the -v operator which tests if a name is set to any value, even the empty string.
$ unset a
$ b=
$ c=
$ [[ -v a ]] && echo "a is set"
$ [[ -v b ]] && echo "b is set"
b is set
$ [[ -v c ]] && echo "c is set"
c is set
I always used:
if [ "x$STATE" == "x" ]; then echo "Need to set State"; exit 1; fi
Not that much more concise, I'm afraid.
Under CSH you have $?STATE.
For future people like me, I wanted to go a step forward and parameterize the var name, so I can loop over a variable sized list of variable names:
#!/bin/bash
declare -a vars=(NAME GITLAB_URL GITLAB_TOKEN)
for var_name in "${vars[#]}"
do
if [ -z "$(eval "echo \$$var_name")" ]; then
echo "Missing environment variable $var_name"
exit 1
fi
done
We can write a nice assertion to check a bunch of variables all at once:
#
# assert if variables are set (to a non-empty string)
# if any variable is not set, exit 1 (when -f option is set) or return 1 otherwise
#
# Usage: assert_var_not_null [-f] variable ...
#
function assert_var_not_null() {
local fatal var num_null=0
[[ "$1" = "-f" ]] && { shift; fatal=1; }
for var in "$#"; do
[[ -z "${!var}" ]] &&
printf '%s\n' "Variable '$var' not set" >&2 &&
((num_null++))
done
if ((num_null > 0)); then
[[ "$fatal" ]] && exit 1
return 1
fi
return 0
}
Sample invocation:
one=1 two=2
assert_var_not_null one two
echo test 1: return_code=$?
assert_var_not_null one two three
echo test 2: return_code=$?
assert_var_not_null -f one two three
echo test 3: return_code=$? # this code shouldn't execute
Output:
test 1: return_code=0
Variable 'three' not set
test 2: return_code=1
Variable 'three' not set
More such assertions here: https://github.com/codeforester/base/blob/master/lib/assertions.sh
This can be a way too:
if (set -u; : $HOME) 2> /dev/null
...
...
http://unstableme.blogspot.com/2007/02/checks-whether-envvar-is-set-or-not.html
None of the above solutions worked for my purposes, in part because I checking the environment for an open-ended list of variables that need to be set before starting a lengthy process. I ended up with this:
mapfile -t arr < variables.txt
EXITCODE=0
for i in "${arr[#]}"
do
ISSET=$(env | grep ^${i}= | wc -l)
if [ "${ISSET}" = "0" ];
then
EXITCODE=-1
echo "ENV variable $i is required."
fi
done
exit ${EXITCODE}
Rather than using external shell scripts I tend to load in functions in my login shell. I use something like this as a helper function to check for environment variables rather than any set variable:
is_this_an_env_variable ()
local var="$1"
if env |grep -q "^$var"; then
return 0
else
return 1
fi
}
The $? syntax is pretty neat:
if [ $?BLAH == 1 ]; then
echo "Exists";
else
echo "Does not exist";
fi

Bash if statement not working properly

I have a bash statement to test a command line argument. If the argument passed to the script is "clean", then the script removes all .o files. Otherwise, it builds a program. However, not matter what is passed (if anything), the script still thinks that the argument "clean" is being passed.
#!/bin/bash
if test "`whoami`" != "root" ; then
echo "You must be logged in as root to build (for loopback mounting)"
echo "Enter 'su' or 'sudo bash' to switch to root"
exit
fi
ARG=$1
if [ $ARG == "clean" ] ; then
echo ">>> cleaning up object files..."
rm -r src/*.o
echo ">>> done. "
echo ">>> Press enter to continue..."
read
else
#Builds program
fi
Answer for first version of question
In bash, spaces are important. Replace:
[ $ARG=="clean" ]
With:
[ "$ARG" = "clean" ]
bash interprets $ARG=="clean" as a single-string. If a single-string is placed in a test statement, test returns false if the string is empty and true if it is non-empty. $ARG=="clean" will never be empty. Thus [ $ARG=="clean" ] will always return true.
Second, $ARG should be quoted. Otherwise, if it is empty, then the statement reduces to `[ == "clean" ] which is an error ("unary operator expected").
Third, it is best practices to use lower or mixed case for your local variables. The system uses upper-case shell variables and you don't want to accidentally overwrite one of them.
Lastly, with [...], the POSIX operator for equal, in the string sense, is =. Bash will accept either = or == but = is more portable.
first:
Every string must double quoted or will error absent argument.
second:
for string used only = or != not a == and also -n and -z commands.
third:
you may combine conditions by -a and -o commands but newer used enclose in () yous conditions so not to get error. Logical operators acts through operators presidence, fist calculate -o operator and then -a! For example
[ -n "$1" -a $1 = '-h' -o $1 = '--help' ] && { usage; exit 0; }
will work when passed to script at least 1 argument and is -h or --help. All spaces must be!!! Bush do short cycle logical evaluations. So don't trouble for case when $1 don't exist in second condition because of result of this expression is determined in first one. next don't calculate in this case. But if your argument may contains space symbols you need it double quote. You must do it also in command line too! Else you get error in script or split your arguments in two or more parts.
Operator == isn't used in test. For numbers(not siring) used -eq or -ne commands. See man 1 test for full descriptions. test EXPRESSION... equivalent of [ EXPRESSIONS... ]. More shirt form of test.

Operations on boolean variables

In this question it has been shown how to use neat boolean variables in bash. Is there a way of performing logic operations with such variables? E.g. how to get this:
var1=true
var2=false
# ...do something interesting...
if ! $var1 -a $var2; then <--- doesn't work correctly
echo "do sth"
fi
This does work:
if ! $var1 && $var2; then
echo "do sth"
fi
Maybe somebody can explain why -a and -o operators don't work and &&, ||, ! do?
Okay boys and girls, lesson time.
What's happening when you execute this line?
if true ; then echo 1 ; fi
What's happening here is that the if command is being executed. After that everything that happens is part of the if command.
What if does is it executes one or more commands (or rather, pipelines) and, if the return code from the last command executed was successful, it executes the commands after then until fi is reached. If the return code was not successful the then part is skipped and execution continues after fi.
if takes no switches, its behavior is not modifiable in anyway.
In the example above the command I told if to execute was true. true is not syntax or a keyword, it's just another command. Try executing it by itself:
true
It will print nothing, but it set its return code to 0 (aka "true"). You can more clearly see that it is a command by rewriting the above if statement like this:
if /bin/true ; then echo 1 ; fi
Which is entirely equivalent.
Always returning true from a test is not very useful. It is typical to use if in conjunction with the test command. test is sometimes symlinked to or otherwise known as [. On your system you probably have a /bin/[ program, but if you're using bash [ will be a builtin command. test is a more complex command than if and you can read all about it.
help [
man [
But for now let us say that test performs some tests according to the options you supply and returns with either a successful return code (0) or an unsuccessful one. This allows us to say
if [ 1 -lt 2 ] ; then echo one is less than two ; fi
But again, this is always true, so it's not very useful. It would be more useful if 1 and 2 were variables
read -p' Enter first number: ' first
read -p' Enter second number: ' second
echo first: $first
echo second: $second
if [ $first -lt $second ] ; then
echo $first is less than $second
fi
Now you can see that test is doing its job. Here we are passing test four arguments. The second argument is -lt which is a switch telling test that the first argument and third argument should be tested to see if the first argument is less than the third argument. The fourth argument does nothing but mark the end of the command; when calling test as [ the final argument must always be ].
Before the above if statement is executed the variables are evaluated. Suppose that I had entered 20 for first and 25 for second, after evaluation the script will look like this:
read -p' Enter first number: ' first
read -p' Enter second number: ' second
echo first: 20
echo second: 25
if [ 20 -lt 25 ] ; then
echo 20 is less than 25
fi
And now you can see that when test is executed it will be testing is 20 less than 25?, which is true, so if will execute the then statement.
Bringing it back to the question at hand: What's going on here?
var1=true
var2=false
if ! $var1 -a $var2 ; then
echo $var1 and $var2 are both true
fi
When the if command is evaluated it will become
if ! true -a false ; then
This is instructing if to execute true and passing the arguments -a false to the true command. Now, true doesn't take any switches or arguments, but it also will not produce an error if you supply them without need. This means that it will execute, return success and the -a false part will be ignored. The ! will reverse the success in to a failure and the then part will not be executed.
If you were to replace the above with a version calling test it would still not work as desired:
var1=true
var2=false
if ! [ $var1 -a $var2 ] ; then
echo $var1 and $var2 are both true
fi
Because the if line would be evaluated to
if ! [ true -a false ; ] then
And test would see true not as a boolean keyword, and not as a command, but as a string. Since a non-empty string is treated as "true" by test it will always return success to if, even if you had said
if ! [ false -a yourmom ] ; then
Since both are non-empty strings -a tests both as true, returns success which is reversed with ! and passed to if, which does not execute the then statement.
If you replace the test version with this version
if ! $var1 && $var2 ; then
Then it will be evaluated in to
if ! true && false ; then
And will be processed like this: if executes true which returns success; which is reversed by !; because the return code of the first command was failure the && statement short circuits and false never gets executed. Because the final command executed returned a failure, failure is passed back to if which does not execute the then clause.
I hope this is all clear.
It is perhaps worth pointing out that you can use constructs like this:
! false && true && echo 1
Which does not use if but still checks return codes, because that is what && and || are for.
There is kind of a black art to using test without making any mistakes. In general, when using bash, the newer [[ command should be used instead because it is more powerful and does away with lots of gotchas which must, for compatibility reasons, be kept in [.
Since the original poster did not supply a realistic example of what he's trying to accomplish it's hard to give any specific advice as to the best solution. Hopefully this has been sufficiently helpful that he can now figure out the correct thing to do.
You have mixed here two different syntaxes.
This will work:
if ! [ 1 -a 2 ]; then
echo "do sth"
fi
Note brackets around the expressions.
You need the test command ([ in newer syntax) to use these keys (-a, -o and so on).
But test does nut run commands itself.
If you want to check exit codes of commands you must not use test.

What's a concise way to check that environment variables are set in a Unix shell script?

I've got a few Unix shell scripts where I need to check that certain environment variables are set before I start doing stuff, so I do this sort of thing:
if [ -z "$STATE" ]; then
echo "Need to set STATE"
exit 1
fi
if [ -z "$DEST" ]; then
echo "Need to set DEST"
exit 1
fi
which is a lot of typing. Is there a more elegant idiom for checking that a set of environment variables is set?
EDIT: I should mention that these variables have no meaningful default value - the script should error out if any are unset.
Parameter Expansion
The obvious answer is to use one of the special forms of parameter expansion:
: ${STATE?"Need to set STATE"}
: ${DEST:?"Need to set DEST non-empty"}
Or, better (see section on 'Position of double quotes' below):
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
The first variant (using just ?) requires STATE to be set, but STATE="" (an empty string) is OK — not exactly what you want, but the alternative and older notation.
The second variant (using :?) requires DEST to be set and non-empty.
If you supply no message, the shell provides a default message.
The ${var?} construct is portable back to Version 7 UNIX and the Bourne Shell (1978 or thereabouts). The ${var:?} construct is slightly more recent: I think it was in System III UNIX circa 1981, but it may have been in PWB UNIX before that. It is therefore in the Korn Shell, and in the POSIX shells, including specifically Bash.
It is usually documented in the shell's man page in a section called Parameter Expansion. For example, the bash manual says:
${parameter:?word}
Display Error if Null or Unset. If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard error and the shell, if it is not interactive, exits. Otherwise, the value of parameter is substituted.
The Colon Command
I should probably add that the colon command simply has its arguments evaluated and then succeeds. It is the original shell comment notation (before '#' to end of line). For a long time, Bourne shell scripts had a colon as the first character. The C Shell would read a script and use the first character to determine whether it was for the C Shell (a '#' hash) or the Bourne shell (a ':' colon). Then the kernel got in on the act and added support for '#!/path/to/program' and the Bourne shell got '#' comments, and the colon convention went by the wayside. But if you come across a script that starts with a colon, now you will know why.
Position of double quotes
blong asked in a comment:
Any thoughts on this discussion? https://github.com/koalaman/shellcheck/issues/380#issuecomment-145872749
The gist of the discussion is:
… However, when I shellcheck it (with version 0.4.1), I get this message:
In script.sh line 13:
: ${FOO:?"The environment variable 'FOO' must be set and non-empty"}
^-- SC2086: Double quote to prevent globbing and word splitting.
Any advice on what I should do in this case?
The short answer is "do as shellcheck suggests":
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
To illustrate why, study the following. Note that the : command doesn't echo its arguments (but the shell does evaluate the arguments). We want to see the arguments, so the code below uses printf "%s\n" in place of :.
$ mkdir junk
$ cd junk
$ > abc
$ > def
$ > ghi
$
$ x="*"
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
abc
def
ghi
$ unset x
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
bash: x: You must set x
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
bash: x: You must set x
$ x="*"
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
*
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
abc
def
ghi
$ x=
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$ unset x
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$
Note how the value in $x is expanded to first * and then a list of file names when the overall expression is not in double quotes. This is what shellcheck is recommending should be fixed. I have not verified that it doesn't object to the form where the expression is enclosed in double quotes, but it is a reasonable assumption that it would be OK.
Try this:
[ -z "$STATE" ] && echo "Need to set STATE" && exit 1;
Your question is dependent on the shell that you are using.
Bourne shell leaves very little in the way of what you're after.
BUT...
It does work, just about everywhere.
Just try and stay away from csh. It was good for the bells and whistles it added, compared the Bourne shell, but it is really creaking now. If you don't believe me, just try and separate out STDERR in csh! (-:
There are two possibilities here. The example above, namely using:
${MyVariable:=SomeDefault}
for the first time you need to refer to $MyVariable. This takes the env. var MyVariable and, if it is currently not set, assigns the value of SomeDefault to the variable for later use.
You also have the possibility of:
${MyVariable:-SomeDefault}
which just substitutes SomeDefault for the variable where you are using this construct. It doesn't assign the value SomeDefault to the variable, and the value of MyVariable will still be null after this statement is encountered.
Surely the simplest approach is to add the -u switch to the shebang (the line at the top of your script), assuming you’re using bash:
#!/bin/sh -u
This will cause the script to exit if any unbound variables lurk within.
${MyVariable:=SomeDefault}
If MyVariable is set and not null, it will reset the variable value (= nothing happens).
Else, MyVariable is set to SomeDefault.
The above will attempt to execute ${MyVariable}, so if you just want to set the variable do:
MyVariable=${MyVariable:=SomeDefault}
In my opinion the simplest and most compatible check for #!/bin/sh is:
if [ "$MYVAR" = "" ]
then
echo "Does not exist"
else
echo "Exists"
fi
Again, this is for /bin/sh and is compatible also on old Solaris systems.
bash 4.2 introduced the -v operator which tests if a name is set to any value, even the empty string.
$ unset a
$ b=
$ c=
$ [[ -v a ]] && echo "a is set"
$ [[ -v b ]] && echo "b is set"
b is set
$ [[ -v c ]] && echo "c is set"
c is set
I always used:
if [ "x$STATE" == "x" ]; then echo "Need to set State"; exit 1; fi
Not that much more concise, I'm afraid.
Under CSH you have $?STATE.
For future people like me, I wanted to go a step forward and parameterize the var name, so I can loop over a variable sized list of variable names:
#!/bin/bash
declare -a vars=(NAME GITLAB_URL GITLAB_TOKEN)
for var_name in "${vars[#]}"
do
if [ -z "$(eval "echo \$$var_name")" ]; then
echo "Missing environment variable $var_name"
exit 1
fi
done
We can write a nice assertion to check a bunch of variables all at once:
#
# assert if variables are set (to a non-empty string)
# if any variable is not set, exit 1 (when -f option is set) or return 1 otherwise
#
# Usage: assert_var_not_null [-f] variable ...
#
function assert_var_not_null() {
local fatal var num_null=0
[[ "$1" = "-f" ]] && { shift; fatal=1; }
for var in "$#"; do
[[ -z "${!var}" ]] &&
printf '%s\n' "Variable '$var' not set" >&2 &&
((num_null++))
done
if ((num_null > 0)); then
[[ "$fatal" ]] && exit 1
return 1
fi
return 0
}
Sample invocation:
one=1 two=2
assert_var_not_null one two
echo test 1: return_code=$?
assert_var_not_null one two three
echo test 2: return_code=$?
assert_var_not_null -f one two three
echo test 3: return_code=$? # this code shouldn't execute
Output:
test 1: return_code=0
Variable 'three' not set
test 2: return_code=1
Variable 'three' not set
More such assertions here: https://github.com/codeforester/base/blob/master/lib/assertions.sh
This can be a way too:
if (set -u; : $HOME) 2> /dev/null
...
...
http://unstableme.blogspot.com/2007/02/checks-whether-envvar-is-set-or-not.html
None of the above solutions worked for my purposes, in part because I checking the environment for an open-ended list of variables that need to be set before starting a lengthy process. I ended up with this:
mapfile -t arr < variables.txt
EXITCODE=0
for i in "${arr[#]}"
do
ISSET=$(env | grep ^${i}= | wc -l)
if [ "${ISSET}" = "0" ];
then
EXITCODE=-1
echo "ENV variable $i is required."
fi
done
exit ${EXITCODE}
Rather than using external shell scripts I tend to load in functions in my login shell. I use something like this as a helper function to check for environment variables rather than any set variable:
is_this_an_env_variable ()
local var="$1"
if env |grep -q "^$var"; then
return 0
else
return 1
fi
}
The $? syntax is pretty neat:
if [ $?BLAH == 1 ]; then
echo "Exists";
else
echo "Does not exist";
fi

Resources