Related
I've got a few Unix shell scripts where I need to check that certain environment variables are set before I start doing stuff, so I do this sort of thing:
if [ -z "$STATE" ]; then
echo "Need to set STATE"
exit 1
fi
if [ -z "$DEST" ]; then
echo "Need to set DEST"
exit 1
fi
which is a lot of typing. Is there a more elegant idiom for checking that a set of environment variables is set?
EDIT: I should mention that these variables have no meaningful default value - the script should error out if any are unset.
Parameter Expansion
The obvious answer is to use one of the special forms of parameter expansion:
: ${STATE?"Need to set STATE"}
: ${DEST:?"Need to set DEST non-empty"}
Or, better (see section on 'Position of double quotes' below):
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
The first variant (using just ?) requires STATE to be set, but STATE="" (an empty string) is OK — not exactly what you want, but the alternative and older notation.
The second variant (using :?) requires DEST to be set and non-empty.
If you supply no message, the shell provides a default message.
The ${var?} construct is portable back to Version 7 UNIX and the Bourne Shell (1978 or thereabouts). The ${var:?} construct is slightly more recent: I think it was in System III UNIX circa 1981, but it may have been in PWB UNIX before that. It is therefore in the Korn Shell, and in the POSIX shells, including specifically Bash.
It is usually documented in the shell's man page in a section called Parameter Expansion. For example, the bash manual says:
${parameter:?word}
Display Error if Null or Unset. If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard error and the shell, if it is not interactive, exits. Otherwise, the value of parameter is substituted.
The Colon Command
I should probably add that the colon command simply has its arguments evaluated and then succeeds. It is the original shell comment notation (before '#' to end of line). For a long time, Bourne shell scripts had a colon as the first character. The C Shell would read a script and use the first character to determine whether it was for the C Shell (a '#' hash) or the Bourne shell (a ':' colon). Then the kernel got in on the act and added support for '#!/path/to/program' and the Bourne shell got '#' comments, and the colon convention went by the wayside. But if you come across a script that starts with a colon, now you will know why.
Position of double quotes
blong asked in a comment:
Any thoughts on this discussion? https://github.com/koalaman/shellcheck/issues/380#issuecomment-145872749
The gist of the discussion is:
… However, when I shellcheck it (with version 0.4.1), I get this message:
In script.sh line 13:
: ${FOO:?"The environment variable 'FOO' must be set and non-empty"}
^-- SC2086: Double quote to prevent globbing and word splitting.
Any advice on what I should do in this case?
The short answer is "do as shellcheck suggests":
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
To illustrate why, study the following. Note that the : command doesn't echo its arguments (but the shell does evaluate the arguments). We want to see the arguments, so the code below uses printf "%s\n" in place of :.
$ mkdir junk
$ cd junk
$ > abc
$ > def
$ > ghi
$
$ x="*"
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
abc
def
ghi
$ unset x
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
bash: x: You must set x
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
bash: x: You must set x
$ x="*"
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
*
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
abc
def
ghi
$ x=
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$ unset x
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$
Note how the value in $x is expanded to first * and then a list of file names when the overall expression is not in double quotes. This is what shellcheck is recommending should be fixed. I have not verified that it doesn't object to the form where the expression is enclosed in double quotes, but it is a reasonable assumption that it would be OK.
Try this:
[ -z "$STATE" ] && echo "Need to set STATE" && exit 1;
Your question is dependent on the shell that you are using.
Bourne shell leaves very little in the way of what you're after.
BUT...
It does work, just about everywhere.
Just try and stay away from csh. It was good for the bells and whistles it added, compared the Bourne shell, but it is really creaking now. If you don't believe me, just try and separate out STDERR in csh! (-:
There are two possibilities here. The example above, namely using:
${MyVariable:=SomeDefault}
for the first time you need to refer to $MyVariable. This takes the env. var MyVariable and, if it is currently not set, assigns the value of SomeDefault to the variable for later use.
You also have the possibility of:
${MyVariable:-SomeDefault}
which just substitutes SomeDefault for the variable where you are using this construct. It doesn't assign the value SomeDefault to the variable, and the value of MyVariable will still be null after this statement is encountered.
Surely the simplest approach is to add the -u switch to the shebang (the line at the top of your script), assuming you’re using bash:
#!/bin/sh -u
This will cause the script to exit if any unbound variables lurk within.
${MyVariable:=SomeDefault}
If MyVariable is set and not null, it will reset the variable value (= nothing happens).
Else, MyVariable is set to SomeDefault.
The above will attempt to execute ${MyVariable}, so if you just want to set the variable do:
MyVariable=${MyVariable:=SomeDefault}
In my opinion the simplest and most compatible check for #!/bin/sh is:
if [ "$MYVAR" = "" ]
then
echo "Does not exist"
else
echo "Exists"
fi
Again, this is for /bin/sh and is compatible also on old Solaris systems.
bash 4.2 introduced the -v operator which tests if a name is set to any value, even the empty string.
$ unset a
$ b=
$ c=
$ [[ -v a ]] && echo "a is set"
$ [[ -v b ]] && echo "b is set"
b is set
$ [[ -v c ]] && echo "c is set"
c is set
I always used:
if [ "x$STATE" == "x" ]; then echo "Need to set State"; exit 1; fi
Not that much more concise, I'm afraid.
Under CSH you have $?STATE.
For future people like me, I wanted to go a step forward and parameterize the var name, so I can loop over a variable sized list of variable names:
#!/bin/bash
declare -a vars=(NAME GITLAB_URL GITLAB_TOKEN)
for var_name in "${vars[#]}"
do
if [ -z "$(eval "echo \$$var_name")" ]; then
echo "Missing environment variable $var_name"
exit 1
fi
done
We can write a nice assertion to check a bunch of variables all at once:
#
# assert if variables are set (to a non-empty string)
# if any variable is not set, exit 1 (when -f option is set) or return 1 otherwise
#
# Usage: assert_var_not_null [-f] variable ...
#
function assert_var_not_null() {
local fatal var num_null=0
[[ "$1" = "-f" ]] && { shift; fatal=1; }
for var in "$#"; do
[[ -z "${!var}" ]] &&
printf '%s\n' "Variable '$var' not set" >&2 &&
((num_null++))
done
if ((num_null > 0)); then
[[ "$fatal" ]] && exit 1
return 1
fi
return 0
}
Sample invocation:
one=1 two=2
assert_var_not_null one two
echo test 1: return_code=$?
assert_var_not_null one two three
echo test 2: return_code=$?
assert_var_not_null -f one two three
echo test 3: return_code=$? # this code shouldn't execute
Output:
test 1: return_code=0
Variable 'three' not set
test 2: return_code=1
Variable 'three' not set
More such assertions here: https://github.com/codeforester/base/blob/master/lib/assertions.sh
This can be a way too:
if (set -u; : $HOME) 2> /dev/null
...
...
http://unstableme.blogspot.com/2007/02/checks-whether-envvar-is-set-or-not.html
None of the above solutions worked for my purposes, in part because I checking the environment for an open-ended list of variables that need to be set before starting a lengthy process. I ended up with this:
mapfile -t arr < variables.txt
EXITCODE=0
for i in "${arr[#]}"
do
ISSET=$(env | grep ^${i}= | wc -l)
if [ "${ISSET}" = "0" ];
then
EXITCODE=-1
echo "ENV variable $i is required."
fi
done
exit ${EXITCODE}
Rather than using external shell scripts I tend to load in functions in my login shell. I use something like this as a helper function to check for environment variables rather than any set variable:
is_this_an_env_variable ()
local var="$1"
if env |grep -q "^$var"; then
return 0
else
return 1
fi
}
The $? syntax is pretty neat:
if [ $?BLAH == 1 ]; then
echo "Exists";
else
echo "Does not exist";
fi
I have a bash statement to test a command line argument. If the argument passed to the script is "clean", then the script removes all .o files. Otherwise, it builds a program. However, not matter what is passed (if anything), the script still thinks that the argument "clean" is being passed.
#!/bin/bash
if test "`whoami`" != "root" ; then
echo "You must be logged in as root to build (for loopback mounting)"
echo "Enter 'su' or 'sudo bash' to switch to root"
exit
fi
ARG=$1
if [ $ARG == "clean" ] ; then
echo ">>> cleaning up object files..."
rm -r src/*.o
echo ">>> done. "
echo ">>> Press enter to continue..."
read
else
#Builds program
fi
Answer for first version of question
In bash, spaces are important. Replace:
[ $ARG=="clean" ]
With:
[ "$ARG" = "clean" ]
bash interprets $ARG=="clean" as a single-string. If a single-string is placed in a test statement, test returns false if the string is empty and true if it is non-empty. $ARG=="clean" will never be empty. Thus [ $ARG=="clean" ] will always return true.
Second, $ARG should be quoted. Otherwise, if it is empty, then the statement reduces to `[ == "clean" ] which is an error ("unary operator expected").
Third, it is best practices to use lower or mixed case for your local variables. The system uses upper-case shell variables and you don't want to accidentally overwrite one of them.
Lastly, with [...], the POSIX operator for equal, in the string sense, is =. Bash will accept either = or == but = is more portable.
first:
Every string must double quoted or will error absent argument.
second:
for string used only = or != not a == and also -n and -z commands.
third:
you may combine conditions by -a and -o commands but newer used enclose in () yous conditions so not to get error. Logical operators acts through operators presidence, fist calculate -o operator and then -a! For example
[ -n "$1" -a $1 = '-h' -o $1 = '--help' ] && { usage; exit 0; }
will work when passed to script at least 1 argument and is -h or --help. All spaces must be!!! Bush do short cycle logical evaluations. So don't trouble for case when $1 don't exist in second condition because of result of this expression is determined in first one. next don't calculate in this case. But if your argument may contains space symbols you need it double quote. You must do it also in command line too! Else you get error in script or split your arguments in two or more parts.
Operator == isn't used in test. For numbers(not siring) used -eq or -ne commands. See man 1 test for full descriptions. test EXPRESSION... equivalent of [ EXPRESSIONS... ]. More shirt form of test.
I searched for noop in bash (:), but was not able to find any good information. What is the exact purpose or use case of this operator?
I tried following and it's working like this for me:
[mandy#root]$ a=11
[mandy#root]$ b=20
[mandy#root]$ c=30
[mandy#root]$ echo $a; : echo $b ; echo $c
10
30
Please let me know, any use case of this operator in real time or any place where it is mandatory to use it.
It's there more for historical reasons. The colon builtin : is exactly equivalent to true. It's traditional to use true when the return value is important, for example in an infinite loop:
while true; do
echo 'Going on forever'
done
It's traditional to use : when the shell syntax requires a command but you have nothing to do.
while keep_waiting; do
: # busy-wait
done
The : builtin dates all the way back to the Thompson shell, it was present in Unix v6. : was a label indicator for the Thompson shell's goto statement. The label could be any text, so : doubled up as a comment indicator (if there is no goto comment, then : comment is effectively a comment). The Bourne shell didn't have goto but kept :.
A common idiom that uses : is : ${var=VALUE}, which sets var to VALUE if it was unset and does nothing if var was already set. This construct only exists in the form of a variable substitution, and this variable substitution needs to be part of a command somehow: a no-op command serves nicely.
See also What purpose does the colon builtin serve?.
I use it for if statements when I comment out all the code. For example you have a test:
if [ "$foo" != "1" ]
then
echo Success
fi
but you want to temporarily comment out everything contained within:
if [ "$foo" != "1" ]
then
#echo Success
fi
Which causes bash to give a syntax error:
line 4: syntax error near unexpected token `fi'
line 4: `fi'
Bash can't have empty blocks (WTF). So you add a no-op:
if [ "$foo" != "1" ]
then
#echo Success
:
fi
or you can use the no-op to comment out the lines:
if [ "$foo" != "1" ]
then
: echo Success
fi
If you use set- e then || : is a great way to not exit the script if a failure happens (it explicitly makes it pass).
You would use : to supply a command that succeeds but doesn't do anything. In this example the "verbosity" command is turned off by default, by setting it to :. The 'v' option turns it on.
#!/bin/sh
# example
verbosity=:
while getopts v OPT ; do
case $OPT in
v)
verbosity=/bin/realpath
;;
*)
exit "Cancelled"
;;
esac
done
# `$verbosity` always succeeds by default, but does nothing.
for i in * ; do
echo $i $($verbosity $i)
done
$ example
file
$ example -v
file /home/me/file
One use is as multiline comments, or to comment out part of your code for testing purposes by using it in conjunction with a here file.
: << 'EOF'
This part of the script is a commented out
EOF
Don't forget to use quotes around EOF so that any code inside doesn't get evaluated, like $(foo). It also might be worth using an intuitive terminator name like NOTES, SCRATCHPAD, or TODO.
Ignoring alias arguments
Some times you want to have an alias that doesn't take any argument. You can do it using ::
> alias alert_with_args='echo hello there'
> alias alert='echo hello there;:'
> alert_with_args blabla
hello there blabla
> alert blabla
hello there
Two of mine.
Embed POD comments
A quite funky application of : is for embedding POD comments in bash scripts, so that man pages can be quickly generated. Of course, one would eventually rewrite the whole script in Perl ;-)
Run-time function binding
This is a sort of code pattern for binding functions at run-time.
F.i., have a debugging function to do something only if a certain flag is set:
#!/bin/bash
# noop-demo.sh
shopt -s expand_aliases
dbg=${DBG:-''}
function _log_dbg {
echo >&2 "[DBG] $#"
}
log_dbg_hook=':'
[ "$dbg" ] && log_dbg_hook='_log_dbg'
alias log_dbg=$log_dbg_hook
echo "Testing noop alias..."
log_dbg 'foo' 'bar'
You get:
$ ./noop-demo.sh
Testing noop alias...
$ DBG=1 ./noop-demo.sh
Testing noop alias...
[DBG] foo bar
Somewhat related to this answer, I find this no-op rather convenient to hack polyglot scripts. For example, here is a valid comment both for bash and for vimscript:
":" # this is a comment
":" # in bash, ‘:’ is a no-op and ‘#’ starts a comment line
":" # in vimscript, ‘"’ starts a comment line
Sure, we may have used true just as well, but : being a punctuation sign and not an irrelevant English word makes it clear that it is a syntax token.
As for why would someone do such a tricky thing as writing a polyglot script (besides it being cool): it proves helpful in situations where we would normally write several script files in several different languages, with file X referring to file Y.
In such a situation, combining both scripts in a single, polyglot file avoids any work in X for determining the path to Y (it is simply "$0"). More importantly, it makes it more convenient to move around or distribute the program.
A common example. There is a well-known, long-standing issue with shebangs: most systems (including Linux and Cygwin) allow only one argument to be passed to the interpreter. The following shebang:
#!/usr/bin/env interpreter --load-libA --load-libB
will fire the following command:
/usr/bin/env "interpreter --load-libA --load-libB" "/path/to/script"
and not the intended:
/usr/bin/env interpreter --load-libA --load-libB "/path/to/script"
Thus, you would end up writing a wrapper script, such as:
#!/usr/bin/env sh
/usr/bin/env interpreter --load-libA --load-libB "/path/to/script"
This is where polyglossia enters the stage.
A more specific example. I once wrote a bash script which, among other things, invoked Vim. I needed to give Vim additional setup, which could be done with the option --cmd "arbitrary vimscript command here". However, that setup was substantial, so that inlining it in a string would have been terrible (if ever possible). Hence, a better solution was to write it in extenso in some configuration file, then make Vim read that file with -S "/path/to/file". Hence I ended up with a polyglot bash/vimscript file.
suppose you have a command you wish to chain to the success of another:
cmd="some command..."
$cmd
[ $? -eq 0 ] && some-other-command
but now you want to execute the commands conditionally and you want to show the commands that would be executed (dry-run):
cmd="some command..."
[ ! -z "$DEBUG" ] && echo $cmd
[ -z "$NOEXEC" ] && $cmd
[ $? -eq 0 ] && {
cmd="some-other-command"
[ ! -z "$DEBUG" ] && echo $cmd
[ -z "$NOEXEC" ] && $cmd
}
so if you set DEBUG and NOEXEC, the second command never shows up. this is because the first command never executes (because NOEXEC is not empty) but the evaluation of that fact leaves you with a return of 1, which means the subordinate command never executes (but you want it to because it's a dry run). so to fix this you can reset the exit value left on the stack with a noop:
[ -z "$NOEXEC" ] && $cmd || :
Sometimes no-op clauses can make your code more readable.
That can be a matter of opinion, but here's an example. Let's suppose you've created a function that works by taking two unix paths. It calculates the 'change path' needed to cd from one path to another. You place a restriction on your function that the paths must both start with a '/' OR both must not.
function chgpath() {
# toC, fromC are the first characters of the argument paths.
if [[ "$toC" == / && "$fromC" == / ]] || [[ "$toC" != / && "$fromC" != / ]]
then
true # continue with function
else
return 1 # Skip function.
fi
Some developers will want to remove the no-op but that would mean negating the conditional:
function chgpath() {
# toC, fromC are the first characters of the argument paths.
if [[ "$toC" != / || "$fromC" == / ]] && [[ "$toC" == / || "$fromC" != / ]]
then
return 1 # Skip function.
fi
Now -in my opinion- its not so clear from the if-clause the conditions in which you'd want to skip doing the function. To eliminate the no-op and do it clearly, you would want to move the if-clause out of the function:
if [[ "$toC" == / && "$fromC" == / ]] || [[ "$toC" != / && "$fromC" != / ]]
then
cdPath=$(chgPath pathA pathB) # (we moved the conditional outside)
That looks better, but many times we can't do this; we want the check to be done inside the function.
So how often does this happen? Not very often. Maybe once or twice a year. It happens often enough, that you should be aware of it. I don't shy away from using it when I think it improves the readability of my code (regardless of the language).
I've also used in it scripts to define default variables.
: ${VARIABLE1:=my_default_value}
: ${VARIABLE2:=other_default_value}
call-my-script ${VARIABLE1} ${VARIABLE2}
I sometimes use it on Docker files to keep RUN commands aligned, as in:
RUN : \
&& somecommand1 \
&& somecommand2 \
&& somecommand3
For me, it reads better than:
RUN somecommand1 \
&& somecommand2 \
&& somecommand3
But this is just a matter of preference, of course
null command [:] is actually considered a synonym for the shell builtin true. The ":" command is itself a Bash builtin, and its exit status is true (0).
`
$ :
$ echo $? # 0
while :
do
operation-1
operation-2
...
operation-n
done
# Same as:
while true
do
...
done
Placeholder in if/then test:
if condition
then : # Do nothing and branch ahead
else # Or else ...
take-some-action
fi
$ : ${username=`whoami`}
$ ${username=`whoami`} #Gives an error without the leading :
Source: TLDP
I used the noop today when I had to create a mock sleep function to use in bats testing framework. This allowed me to create an empty function with no side effects:
function sleep() {
:
}
I want to write a function for when I have something like the following
echo 1 2 3|pick
Pick will then take the arguments and I will do something with them.
How do I do this?
Are you looking for xargs?
pick() {
read -r arg1 arg2 remainder
echo first arg is $arg1
echo The remaining args are $remainder
}
--EDIT (response to question in comment)
One way to loop through the arguments:
pick() {
read args;
set $args;
while test $# -ne 0; do
echo $1
shift
done
}
On each iteration of the loop, $1 refers to an argument.
If I'm not mistaken, the OP wants the same thing I do: you feed it a string, and if the string containes multiple {words,lines}, it presents you a menu, and you pick one, and it returns the one you pick on stdout.
If there's only one item, it just returns it.
This is useful for--to use my particular use-case--a log file viewer script: you give it a substring of a filename, and it greps through find /var/log -name \*$arg\* -print to see what it can find. If it gets a unique hit, it hands it back to your script, which runs less against it. If it gets more than one hit, it shows you a menu, and lets you pick one.
ISTR that KSH has a builtin for this, but that I wasn't all that impressed with it; I don't recall if bash has one.
I am here because I was searching to see if someone had already written it before writing it myself. :-)
UPDATE: Nope; I wrote it myself:
Here's some example code:
/usr/local/bin/msg:
PATH=$PATH:/usr/local/bin
[ $UID = 0 ] || exec sudo su root -c "$0 $*"
FILE=/var/log/messages
[ $# -eq 1 ] &&
FILE=`find /var/log/ -name \*$1\* -print |
egrep -v '2011|.[0-9]$' |
pick`
echo "$FILE"
less +F $FILE
Since I'm piping the name to less +F I want to grep out archived log files; this is for interactive log viewing.
/usr/local/bin/pick:
# Present the user a bash Select menu, and let them pick
# Try to be smart about multi-line responses
# must take input on stdin if it might be multiline
# get multiline input from stdin
while read LINE </dev/stdin
do
CHOICES+=( $LINE )
done
# add on anything specified as arguments
while [ $# -gt 0 ]
do
CHOICES+=( $1 )
shift
done
# if only one thing to pick, just pick it
if [ ${#CHOICES[*]} -eq 1 ]
then
echo $CHOICES
exit
fi
# eval set $CHOICES
select CHOSEN in ${CHOICES[#]}
do
echo $CHOSEN
exit
done </dev/tty
I've got a few Unix shell scripts where I need to check that certain environment variables are set before I start doing stuff, so I do this sort of thing:
if [ -z "$STATE" ]; then
echo "Need to set STATE"
exit 1
fi
if [ -z "$DEST" ]; then
echo "Need to set DEST"
exit 1
fi
which is a lot of typing. Is there a more elegant idiom for checking that a set of environment variables is set?
EDIT: I should mention that these variables have no meaningful default value - the script should error out if any are unset.
Parameter Expansion
The obvious answer is to use one of the special forms of parameter expansion:
: ${STATE?"Need to set STATE"}
: ${DEST:?"Need to set DEST non-empty"}
Or, better (see section on 'Position of double quotes' below):
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
The first variant (using just ?) requires STATE to be set, but STATE="" (an empty string) is OK — not exactly what you want, but the alternative and older notation.
The second variant (using :?) requires DEST to be set and non-empty.
If you supply no message, the shell provides a default message.
The ${var?} construct is portable back to Version 7 UNIX and the Bourne Shell (1978 or thereabouts). The ${var:?} construct is slightly more recent: I think it was in System III UNIX circa 1981, but it may have been in PWB UNIX before that. It is therefore in the Korn Shell, and in the POSIX shells, including specifically Bash.
It is usually documented in the shell's man page in a section called Parameter Expansion. For example, the bash manual says:
${parameter:?word}
Display Error if Null or Unset. If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard error and the shell, if it is not interactive, exits. Otherwise, the value of parameter is substituted.
The Colon Command
I should probably add that the colon command simply has its arguments evaluated and then succeeds. It is the original shell comment notation (before '#' to end of line). For a long time, Bourne shell scripts had a colon as the first character. The C Shell would read a script and use the first character to determine whether it was for the C Shell (a '#' hash) or the Bourne shell (a ':' colon). Then the kernel got in on the act and added support for '#!/path/to/program' and the Bourne shell got '#' comments, and the colon convention went by the wayside. But if you come across a script that starts with a colon, now you will know why.
Position of double quotes
blong asked in a comment:
Any thoughts on this discussion? https://github.com/koalaman/shellcheck/issues/380#issuecomment-145872749
The gist of the discussion is:
… However, when I shellcheck it (with version 0.4.1), I get this message:
In script.sh line 13:
: ${FOO:?"The environment variable 'FOO' must be set and non-empty"}
^-- SC2086: Double quote to prevent globbing and word splitting.
Any advice on what I should do in this case?
The short answer is "do as shellcheck suggests":
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
To illustrate why, study the following. Note that the : command doesn't echo its arguments (but the shell does evaluate the arguments). We want to see the arguments, so the code below uses printf "%s\n" in place of :.
$ mkdir junk
$ cd junk
$ > abc
$ > def
$ > ghi
$
$ x="*"
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
abc
def
ghi
$ unset x
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
bash: x: You must set x
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
bash: x: You must set x
$ x="*"
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
*
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
abc
def
ghi
$ x=
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$ unset x
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$
Note how the value in $x is expanded to first * and then a list of file names when the overall expression is not in double quotes. This is what shellcheck is recommending should be fixed. I have not verified that it doesn't object to the form where the expression is enclosed in double quotes, but it is a reasonable assumption that it would be OK.
Try this:
[ -z "$STATE" ] && echo "Need to set STATE" && exit 1;
Your question is dependent on the shell that you are using.
Bourne shell leaves very little in the way of what you're after.
BUT...
It does work, just about everywhere.
Just try and stay away from csh. It was good for the bells and whistles it added, compared the Bourne shell, but it is really creaking now. If you don't believe me, just try and separate out STDERR in csh! (-:
There are two possibilities here. The example above, namely using:
${MyVariable:=SomeDefault}
for the first time you need to refer to $MyVariable. This takes the env. var MyVariable and, if it is currently not set, assigns the value of SomeDefault to the variable for later use.
You also have the possibility of:
${MyVariable:-SomeDefault}
which just substitutes SomeDefault for the variable where you are using this construct. It doesn't assign the value SomeDefault to the variable, and the value of MyVariable will still be null after this statement is encountered.
Surely the simplest approach is to add the -u switch to the shebang (the line at the top of your script), assuming you’re using bash:
#!/bin/sh -u
This will cause the script to exit if any unbound variables lurk within.
${MyVariable:=SomeDefault}
If MyVariable is set and not null, it will reset the variable value (= nothing happens).
Else, MyVariable is set to SomeDefault.
The above will attempt to execute ${MyVariable}, so if you just want to set the variable do:
MyVariable=${MyVariable:=SomeDefault}
In my opinion the simplest and most compatible check for #!/bin/sh is:
if [ "$MYVAR" = "" ]
then
echo "Does not exist"
else
echo "Exists"
fi
Again, this is for /bin/sh and is compatible also on old Solaris systems.
bash 4.2 introduced the -v operator which tests if a name is set to any value, even the empty string.
$ unset a
$ b=
$ c=
$ [[ -v a ]] && echo "a is set"
$ [[ -v b ]] && echo "b is set"
b is set
$ [[ -v c ]] && echo "c is set"
c is set
I always used:
if [ "x$STATE" == "x" ]; then echo "Need to set State"; exit 1; fi
Not that much more concise, I'm afraid.
Under CSH you have $?STATE.
For future people like me, I wanted to go a step forward and parameterize the var name, so I can loop over a variable sized list of variable names:
#!/bin/bash
declare -a vars=(NAME GITLAB_URL GITLAB_TOKEN)
for var_name in "${vars[#]}"
do
if [ -z "$(eval "echo \$$var_name")" ]; then
echo "Missing environment variable $var_name"
exit 1
fi
done
We can write a nice assertion to check a bunch of variables all at once:
#
# assert if variables are set (to a non-empty string)
# if any variable is not set, exit 1 (when -f option is set) or return 1 otherwise
#
# Usage: assert_var_not_null [-f] variable ...
#
function assert_var_not_null() {
local fatal var num_null=0
[[ "$1" = "-f" ]] && { shift; fatal=1; }
for var in "$#"; do
[[ -z "${!var}" ]] &&
printf '%s\n' "Variable '$var' not set" >&2 &&
((num_null++))
done
if ((num_null > 0)); then
[[ "$fatal" ]] && exit 1
return 1
fi
return 0
}
Sample invocation:
one=1 two=2
assert_var_not_null one two
echo test 1: return_code=$?
assert_var_not_null one two three
echo test 2: return_code=$?
assert_var_not_null -f one two three
echo test 3: return_code=$? # this code shouldn't execute
Output:
test 1: return_code=0
Variable 'three' not set
test 2: return_code=1
Variable 'three' not set
More such assertions here: https://github.com/codeforester/base/blob/master/lib/assertions.sh
This can be a way too:
if (set -u; : $HOME) 2> /dev/null
...
...
http://unstableme.blogspot.com/2007/02/checks-whether-envvar-is-set-or-not.html
None of the above solutions worked for my purposes, in part because I checking the environment for an open-ended list of variables that need to be set before starting a lengthy process. I ended up with this:
mapfile -t arr < variables.txt
EXITCODE=0
for i in "${arr[#]}"
do
ISSET=$(env | grep ^${i}= | wc -l)
if [ "${ISSET}" = "0" ];
then
EXITCODE=-1
echo "ENV variable $i is required."
fi
done
exit ${EXITCODE}
Rather than using external shell scripts I tend to load in functions in my login shell. I use something like this as a helper function to check for environment variables rather than any set variable:
is_this_an_env_variable ()
local var="$1"
if env |grep -q "^$var"; then
return 0
else
return 1
fi
}
The $? syntax is pretty neat:
if [ $?BLAH == 1 ]; then
echo "Exists";
else
echo "Does not exist";
fi