How do I set a variable in the parent shell, from a subshell?
a=3
(a=4)
echo $a
The whole point of a subshell is that it doesn't affect the calling session. In bash a subshell is a child process, other shells differ but even then a variable setting in a subshell does not affect the caller. By definition.
Do you need a subshell? If you just need a group then use braces:
a=3
{ a=4;}
echo $a
gives 4 (be careful of the spaces in that one). Alternatively, write the variable value to stdout and capture it in the caller:
a=3
a=$(a=4;echo $a)
echo $a
avoid using back-ticks ``, they are deprecated and can be difficult to read.
There is the gdb-bash-variable hack:
gdb --batch-silent -ex "attach $$" -ex 'set bind_variable("a", "4", 0)';
although that always sets a variable in the global scope, not just the parent scope
You don't. The subshell doesn't have access to its parent's environment. (At least within the abstraction that Bash provides. You could potentially try to use gdb, or smash the stack, or whatnot, to gain such access clandestinely. I wouldn't recommend that, though.)
One alternative is for the subshell to write assignment statements to a temporary file for its parent to read:
a=3
(echo 'a=4' > tmp)
. tmp
rm tmp
echo "$a"
If the problem is related to a while loop, one way to fix this is by using Process Substitution:
var=0
while read i;
do
# perform computations on $i
((var++))
done < <(find . -type f -name "*.bin" -maxdepth 1)
as shown here: https://stackoverflow.com/a/13727116/2547445
To change variables in a script called from a parent script, you can call the script preceded with a "."
(EDIT - for explanation)
In most shells "." is an alias for "source". the source command just inserts the text of another file at that position in the executing script. In the context of this question this answer avoids a sub-shell
a=3
echo $a
. ./calledScript.sh
echo $a
in calledScript.sh
a=4
Expected output
3
4
By reading the answer from #ruakh (thank you) with a temporary file approach and the comments asking for a file descriptors solution, I got the following idea:
a=3
. <(echo a=4; echo b=5)
echo $a
echo $b
It allows returning different variables at once (which could be an issue in the subshell variant of the accepted answer).
No iteration is needed,
No temporary file to take care of.
Close to the syntax proposed by the OP.
Result:
4
5
With xtrace enabled is visible that we are sourcing from the file descriptor created for the output of the subshell:
+ a=3
+ . /dev/fd/63 # <-- the file descriptor ;)
++ echo a=4
++ echo b=5
++ a=4
++ b=5
+ echo 4
4
+ echo 5
5
You can output the value in the subshell and assign the subshell output to a variable in the caller script:
# subshell.sh
echo Value
# caller
myvar=$(subshell.sh)
If the subshell has more to output you can separate the variable value and other messages by redirecting them into different output streams:
# subshell.sh
echo "Writing value" 1>&2
echo Value
# caller
myvar=$(subshell.sh 2>/dev/null) # or to somewhere else
echo $myvar
Alternatively, you can output variable assignments in the subshell, evaluate them in the caller script and avoid using files to exchange information:
# subshell.sh
echo "a=4"
# caller
# export $(subshell.sh) would be more secure, since export accepts name=value only.
eval $(subshell.sh)
echo $a
The last way I can think of is to use exit codes but this covers the integer values exchange only (and in a limited range) and breaks the convention for interpreting exit codes (0 for success non-0 for everything else).
Instead of accessing the variable from the parent shell, change the order of the commands and use the process substitution:
a=3
echo 5 | (read a)
echo $a
prints 3
a=3
read a < <(echo 5)
echo $a
prints 5
Another example:
let i=0
seq $RANDOM | while read r
do
let i=r
done
echo $i
vs
let i=0
while read r
do
let i=r
done < <(seq $RANDOM)
echo $i
Alternatively, when job control is inactive (e.g. in scripts) you can use the lastpipe shell option to achieve the same result without changing the order of the commands:
#!/bin/bash
shopt -s lastpipe
let i=0
seq $RANDOM | while read r
do
let i=r
done
echo $i
Unless you can apply all io to pipes and use file handles, basic variable updating is impossible within $(command) and any other sub-process.
Regular files, however, are bash's global variables for normal sequential processing. Note: Due to race conditions, this simple approach is not good for parallel processing.
Create an set/get/default function like this:
globalVariable() { # NEW-VALUE
# set/get/default globalVariable
if [ 0 = "$#" ]; then
# new value not given -- echo the value
[ -e "$aRam/globalVariable" ] \
&& cat "$aRam/globalVariable" \
|| printf "default-value-here"
else
# new value given -- set the value
printf "%s" "$1" > "$aRam/globalVariable"
fi
}
"$aRam" is the directory where values are stored. I like it to be a ram disk for speed and volatility:
aRam="$(mktemp -td $(basename "$0").XXX)" # temporary directory
mount -t tmpfs ramdisk "$aRam" # mount the ram disk there
trap "umount "$aRam" && rm -rf "$aRam"" EXIT # auto-eject
To read the value:
v="$(globalVariable)" # or part of any command
To set the value:
globalVariable newValue # newValue will be written to file
To unset the value:
rm -f "$aRam/globalVariable"
The only real reason for the access function is to apply a default value because cat will error given a non-existent file. It is also useful to apply other get/set logic. Otherwise, it would not be needed at all.
An ugly read method avoiding cat's non-existent file error:
v="$(cat "$aRam/globalVariable 2>/dev/null")"
A cool feature of this mess is that you can open another terminal and examine the contents of the files while the program is running.
While it's harder to get multiple variables out of a subshell, you can set multiple variables inside a function without using globals.
You can pass the name of a variable into a function that uses local -n to turn it into a special variable called a nameref:
myfunc() {
local -n OUT=$1
local -n SIDEEFFECT=$2
OUT='foo'
SIDEEFFECT='bar'
}
myfunc A B
echo $A
> foo
echo $B
> bar
This is the technique I ended up using instead of getting subshell FOO=$(myfunc) working setting multiple variables.
A very simple and practical method that allows multiple variables is as follows, eventually may add parameters to the call:
function ComplexReturn(){
# do your processing...
a=123
b=456
echo -n "AAA=${a}; BBB=${b};"
}
# ... this can be internal function or any subshell command
eval $(ComplexReturn)
echo $AAA $BBB
Related
I have a set of shell commands that look like this
if check-some-condition $a then;
do stuff
run-exit-code $a
fi
where check-some-condition and run-exit-code could be replaced by functions taking a single argument $a, while do stuff is a placeholder for possibly several shell commands. Is it possible to emulate the Lisp functionality of a macro where I could just write
(my-macro $a stuff)
and have it replaced by the code above? I am using Bash but I can use any other shell if they have features that make this easier. I thought at first of using functions but I don't think I can pass in a block of commands.
There isn't a macro definition system in the shell, and consequently shell syntax is not walked in order to expand macros. However, the shell has a textual eval command. You can write a function which synthesizes shell syntax, such as by inserting arguments it has been given, into a template. The function can print that syntax, which the caller can capture using $(...) command substitution syntax and pass to eval:
eval $(macro-like foo bar)
The expansion will happen every time that line of code is executed.
I've done something like this on a very small number of occasions. I don't remember all the details, but I remember that the code was also taking advantage of Bash local variables, which have dynamic scope, like ancient Lisp dialects and defvar variables in Common Lisp.
In Bash, eval takes place in a dynamic environment which sees the surrounding local variables, which is something you can exploit; it can help bring about some macro-like semantics. In a Lisp with lexically scoped local variables, eval-ed code has no access to those variables, but macro-substituted code does. Under dynamic scope, evaled code can access and assign surrounding locals.
Here is an example. Note that because eval is a command which has no control over expansions taking place in its argument space because it is called (analogously to eval in Lisp being a function which doesn't control argument evaluation), the client code is encumbered with quoting responsibilities.
# $1 = variable
# $2 = low
# $3 = high
# $4 = body
dofor()
{
cat <<!
$1=$2 ;
while [ \$$1 -lt $3 ] ; do
$4
$1=\$(( $1 + 1 ))
done
!
}
eval "$(dofor i 0 100 'printf "[%d]\n" $i')"
We could make it so that
eval $(dofor i 0 100 'printf "[%d]\n" $i')
works without the quotes, at the cost of more heaps of arcane escapery inside dofor.
Imagine we extended the shell with a built-in command evalcmd, which let us write this instead of the above:
evalcmd dofor i 0 100 'printf "[%d]\n" $i'
Can we write that as a shell function? It turns out, yes:
# run the command specified in the arguments
# capturing its output, which is evaled in quotes
evcmd()
{
eval "$("$#")"
}
evcmd dofor i 0 100 'printf "[%d]\n" $i'
Now, though still monstrously inefficient, it's substantially more ergonomic.
Finally, let's ask: could we split dofor into an a dofor_impl which generates the code, and a dofor command which calls dofor_impl and invokes the evcmd semantics? Also, yes:
dofor_impl()
{
cat <<!
$1=$2 ;
while [ \$$1 -lt $3 ] ; do
$4
$1=\$(( $1 + 1 ))
done
!
}
dofor()
{
# like evcmd, but inserting an operator into the left position
eval "$(dofor_impl "$#")"
}
dofor i 0 100 'printf "[%d]\n" $i'
This is not bad for some simple uses, but what we can't achieve is not having to put the $i into a quote so that the substitution doesn't take place before dofor is invoked.
In Bash you can define functions as follows:
function run_exit_code () {
echo "EXIT $1"
}
function check_some_condition () {
echo "CHECKING $1";
true
}
And your code can execute commands associated with variables:
function my_code () {
var=$1
stuff=$2
if check_some_condition $var; then
echo "OK";
$stuff;
run_exit_code $var
fi
}
So you can write, for example:
$ my_code /tmp 'ls /'
CHECKING /tmp
OK
bin boot cdrom dev etc home lib lib32 lib64 libx32 lost+found media mnt opt proc root run sbin srv swapfile sys tmp usr var
EXIT /tmp
If you want stuff to refer to $var, then you need to add eval:
function my_code () {
var=$1
stuff=$2
if check_some_condition $var; then
echo "OK";
eval $stuff; # <<< eval
run_exit_code $var
fi
}
This allows you to write a quoted bash expression and have it beeing evaluated in the context of your function:
$ my_code / 'ls $var'
CHECKING /
OK
bin boot cdrom dev etc home lib lib32 ...
EXIT /
I'm setting up my shell environments and I want to be able to use some of the same functions/aliases in zsh as in bash. One of these functions opens either .bashrc or .zshrc in an editor (whichever file is relevant), waits for the editor to close, then reloads the rc file.
# a very simplified version of this function
editrc() {
local rcfile=".$(basename $SHELL)rc"
code -w ~/$rcfile
. ~/$rcfile
}
I use the value of rcfile in a few other functions, so I've pulled it out of the function declaration.
_rc=".$(basename $SHELL)rc"
editrc() {
code -w ~/$_rc
. ~/$_rc
}
# ... other functions that use it ...
unset _rc
However, because I'm a neat freak, I want to unset _rc at the end of my script, but I still want my functions to run correctly. Is there a clever way to evaluate $_rc at the time the function is declared?
I know I could use eval and place everything except $_rc instances within single quotes, but that seems like a pain, since the full version of my function uses both single-quotes and double-quotes.
_rc=".$(basename $SHELL)rc"
eval 'editrc() {
echo Here'"'"'s a thing that uses single quotes. As you can see it'"'"'s a pain.
code -w ~/'$_rc'
. ~/'$_rc'
}'
# ... other functions using `_rc`
unset _rc
I'm guessing I could declare my functions, then do some magic with eval "$(declare -f editrc | awk)". It very well be more pain than it's worth, but I'm always interested in learning new things.
Note: I'd love to generalize this into a utility function that does this.
_myvar=foo
anothervar=bar
myfunc() {
echo $_myvar $anothervar
}
# redeclares myfunc with `$_myvar` expanded, but leaves `$anothervar` as-is
expandfunctionvars myfunc '$_myvar'
Is there a clever way to evaluate $_rc at the time the function is declared?
_rc=".$(basename "$SHELL")rc"
# while you could eval here, source lets you work with a stream
source <(
cat <<EOF
editrc() {
local _rc
# first safely trasfer context
$(declare -p _rc)
EOF
# use quoted here string to do anything inside without caring.
cat <<'EOF'
# do anything else
echo "Here's a thing that uses single quotes. As you can see it's not a pain, just choose proper quoting."
code -w "~/$_rc"
. "~/$_rc"
}
EOF
)
unset _rc
Generally first use declare -p to transfer variables as strings to be evaluated. Then after you "import" variables, use a quoted here document to do anything as in a normal script.
References to read:
<<EOF is a here document. Note the difference in parsing when the here delimiter is quoted vs unquoted.
<(..) is a process substitution
The source command reads a pipe created by process substitution. Inside the process subtitution I output the function to be sourced. With the first here document I output the function name definition, with a local of the variable so that it doesn't pollute global namespace. Then with declare -p I output the variable definition as a properly quoted string later to be sourced by source. Then with a quoted here document I output the rest of the function, so that I do not need to care about quoting.
The code is bash specific, I know nothing about zsh and don't use it.
You could do it with eval too:
eval '
editrc() {
local _rc
# first safely trasfer context
'"$(declare -p _rc)"'
# use quoted here string to do anything inside without caring.
# do anything else
echo "Here'\''s a thing that uses single quotes. As you can see it'\''s not a pain, just choose proper quoting."
code -w "~/$_rc"
. "~/$_rc"
}'
But for me using a quoted here document delimiter allows for easier writing.
While KamilCuck was working on their answer, I devised a function that will take in any function name and a set of variable names, expand just those variables, and redeclare the function.
expandFnVars() {
if [[ $# -lt 2 ]]; then
>&2 echo 'expandFnVars requires at least two arguments: the function name and the variable(s) to be expanded'
return 1
fi
local fn="$1"
shift
local vars=("$#")
if [[ -z "$(declare -F $fn 2> /dev/null)" ]]; then
>&2 echo $fn is not a function.
return 1
fi
foundAllVars=true
for v in $vars; do
if [[ -z "$(declare -p $v 2> /dev/null)" ]]; then
>&2 echo $v is not a declared value.
foundAllVars=false
fi
done
[[ $foundAllVars != true ]] && return 1
fn="$(declare -f $fn)"
for v in $vars; do
local val="$(eval 'echo $'$v)" # get the value of the varable represented by $v
val="${val//\"/\\\"}" # escape any double-quotes
val="${val//\\/\\\\\\}" # escape any backslashes
fn="$(echo "$fn" | sed -r 's/"?\$'$v'"?/"'"$val"'"/g')" # replace instances of "$$v" and $$v with $val
done
eval "$fn"
}
Usage:
foo="foo bar"
bar='$foo'
baz=baz
fn() {
echo $bar $baz
}
expandFnVars fn bar
declare -f fn
# prints:
# fn ()
# {
# echo "$foo" $baz
# }
expandFnVars fn foo
declare -f fn
# prints:
# fn ()
# {
# echo "foo bar" $baz
# }
Looking at it now, I see one flaw. Suppose $bar in the original function was in single-quotes. We probably would not want its value to be replaced. This could be fixed by some clever regex lookbehinds to count the number of unescaped 's, but I'm happy with it as-is.
I've got a few Unix shell scripts where I need to check that certain environment variables are set before I start doing stuff, so I do this sort of thing:
if [ -z "$STATE" ]; then
echo "Need to set STATE"
exit 1
fi
if [ -z "$DEST" ]; then
echo "Need to set DEST"
exit 1
fi
which is a lot of typing. Is there a more elegant idiom for checking that a set of environment variables is set?
EDIT: I should mention that these variables have no meaningful default value - the script should error out if any are unset.
Parameter Expansion
The obvious answer is to use one of the special forms of parameter expansion:
: ${STATE?"Need to set STATE"}
: ${DEST:?"Need to set DEST non-empty"}
Or, better (see section on 'Position of double quotes' below):
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
The first variant (using just ?) requires STATE to be set, but STATE="" (an empty string) is OK — not exactly what you want, but the alternative and older notation.
The second variant (using :?) requires DEST to be set and non-empty.
If you supply no message, the shell provides a default message.
The ${var?} construct is portable back to Version 7 UNIX and the Bourne Shell (1978 or thereabouts). The ${var:?} construct is slightly more recent: I think it was in System III UNIX circa 1981, but it may have been in PWB UNIX before that. It is therefore in the Korn Shell, and in the POSIX shells, including specifically Bash.
It is usually documented in the shell's man page in a section called Parameter Expansion. For example, the bash manual says:
${parameter:?word}
Display Error if Null or Unset. If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard error and the shell, if it is not interactive, exits. Otherwise, the value of parameter is substituted.
The Colon Command
I should probably add that the colon command simply has its arguments evaluated and then succeeds. It is the original shell comment notation (before '#' to end of line). For a long time, Bourne shell scripts had a colon as the first character. The C Shell would read a script and use the first character to determine whether it was for the C Shell (a '#' hash) or the Bourne shell (a ':' colon). Then the kernel got in on the act and added support for '#!/path/to/program' and the Bourne shell got '#' comments, and the colon convention went by the wayside. But if you come across a script that starts with a colon, now you will know why.
Position of double quotes
blong asked in a comment:
Any thoughts on this discussion? https://github.com/koalaman/shellcheck/issues/380#issuecomment-145872749
The gist of the discussion is:
… However, when I shellcheck it (with version 0.4.1), I get this message:
In script.sh line 13:
: ${FOO:?"The environment variable 'FOO' must be set and non-empty"}
^-- SC2086: Double quote to prevent globbing and word splitting.
Any advice on what I should do in this case?
The short answer is "do as shellcheck suggests":
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
To illustrate why, study the following. Note that the : command doesn't echo its arguments (but the shell does evaluate the arguments). We want to see the arguments, so the code below uses printf "%s\n" in place of :.
$ mkdir junk
$ cd junk
$ > abc
$ > def
$ > ghi
$
$ x="*"
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
abc
def
ghi
$ unset x
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
bash: x: You must set x
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
bash: x: You must set x
$ x="*"
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
*
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
abc
def
ghi
$ x=
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$ unset x
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$
Note how the value in $x is expanded to first * and then a list of file names when the overall expression is not in double quotes. This is what shellcheck is recommending should be fixed. I have not verified that it doesn't object to the form where the expression is enclosed in double quotes, but it is a reasonable assumption that it would be OK.
Try this:
[ -z "$STATE" ] && echo "Need to set STATE" && exit 1;
Your question is dependent on the shell that you are using.
Bourne shell leaves very little in the way of what you're after.
BUT...
It does work, just about everywhere.
Just try and stay away from csh. It was good for the bells and whistles it added, compared the Bourne shell, but it is really creaking now. If you don't believe me, just try and separate out STDERR in csh! (-:
There are two possibilities here. The example above, namely using:
${MyVariable:=SomeDefault}
for the first time you need to refer to $MyVariable. This takes the env. var MyVariable and, if it is currently not set, assigns the value of SomeDefault to the variable for later use.
You also have the possibility of:
${MyVariable:-SomeDefault}
which just substitutes SomeDefault for the variable where you are using this construct. It doesn't assign the value SomeDefault to the variable, and the value of MyVariable will still be null after this statement is encountered.
Surely the simplest approach is to add the -u switch to the shebang (the line at the top of your script), assuming you’re using bash:
#!/bin/sh -u
This will cause the script to exit if any unbound variables lurk within.
${MyVariable:=SomeDefault}
If MyVariable is set and not null, it will reset the variable value (= nothing happens).
Else, MyVariable is set to SomeDefault.
The above will attempt to execute ${MyVariable}, so if you just want to set the variable do:
MyVariable=${MyVariable:=SomeDefault}
In my opinion the simplest and most compatible check for #!/bin/sh is:
if [ "$MYVAR" = "" ]
then
echo "Does not exist"
else
echo "Exists"
fi
Again, this is for /bin/sh and is compatible also on old Solaris systems.
bash 4.2 introduced the -v operator which tests if a name is set to any value, even the empty string.
$ unset a
$ b=
$ c=
$ [[ -v a ]] && echo "a is set"
$ [[ -v b ]] && echo "b is set"
b is set
$ [[ -v c ]] && echo "c is set"
c is set
I always used:
if [ "x$STATE" == "x" ]; then echo "Need to set State"; exit 1; fi
Not that much more concise, I'm afraid.
Under CSH you have $?STATE.
For future people like me, I wanted to go a step forward and parameterize the var name, so I can loop over a variable sized list of variable names:
#!/bin/bash
declare -a vars=(NAME GITLAB_URL GITLAB_TOKEN)
for var_name in "${vars[#]}"
do
if [ -z "$(eval "echo \$$var_name")" ]; then
echo "Missing environment variable $var_name"
exit 1
fi
done
We can write a nice assertion to check a bunch of variables all at once:
#
# assert if variables are set (to a non-empty string)
# if any variable is not set, exit 1 (when -f option is set) or return 1 otherwise
#
# Usage: assert_var_not_null [-f] variable ...
#
function assert_var_not_null() {
local fatal var num_null=0
[[ "$1" = "-f" ]] && { shift; fatal=1; }
for var in "$#"; do
[[ -z "${!var}" ]] &&
printf '%s\n' "Variable '$var' not set" >&2 &&
((num_null++))
done
if ((num_null > 0)); then
[[ "$fatal" ]] && exit 1
return 1
fi
return 0
}
Sample invocation:
one=1 two=2
assert_var_not_null one two
echo test 1: return_code=$?
assert_var_not_null one two three
echo test 2: return_code=$?
assert_var_not_null -f one two three
echo test 3: return_code=$? # this code shouldn't execute
Output:
test 1: return_code=0
Variable 'three' not set
test 2: return_code=1
Variable 'three' not set
More such assertions here: https://github.com/codeforester/base/blob/master/lib/assertions.sh
This can be a way too:
if (set -u; : $HOME) 2> /dev/null
...
...
http://unstableme.blogspot.com/2007/02/checks-whether-envvar-is-set-or-not.html
None of the above solutions worked for my purposes, in part because I checking the environment for an open-ended list of variables that need to be set before starting a lengthy process. I ended up with this:
mapfile -t arr < variables.txt
EXITCODE=0
for i in "${arr[#]}"
do
ISSET=$(env | grep ^${i}= | wc -l)
if [ "${ISSET}" = "0" ];
then
EXITCODE=-1
echo "ENV variable $i is required."
fi
done
exit ${EXITCODE}
Rather than using external shell scripts I tend to load in functions in my login shell. I use something like this as a helper function to check for environment variables rather than any set variable:
is_this_an_env_variable ()
local var="$1"
if env |grep -q "^$var"; then
return 0
else
return 1
fi
}
The $? syntax is pretty neat:
if [ $?BLAH == 1 ]; then
echo "Exists";
else
echo "Does not exist";
fi
How can I optimize following Bash code?
if grep --quiet $pattern $fname; then
echo "==> "$fname" <=="
grep -n $pattern $fname
fi
Firstly it scans file for occurences of $pattern. If there were any results found, it prints file name and then all occurences.
You can see that it does same grep twice. If I could store results from first call and then reuse them, it would be perfect.
An assignment won't change the value of $?, so you can add one without otherwise modifying your logic:
if content=$(grep -n "$pattern" "$fname"); then
echo "==> $fname <=="
printf '%s\n' "$content"
fi
Note, here, that all the variable expansions are inside double quotes. For some reason your original was explicitly performing them only unquoted -- this causes both string-splitting and glob expansion to take place; you almost certainly don't want either.
By the way -- there are things you could do that would make an assignment modify the exit status of a command run in the subshell generating its value! Using declare, export, local, or the like to perform an assignment will cause that command's own exit status to replace that of the subshell being assigned.
# here, the "local" will replace $? with 0
$ f() {
> local foo=$(echo "bar"; exit 1)
> echo "$?"
> }
$ f
0
...whereas...
# here, the "local" is separate, so the subshell's exit status survives
$ f() {
> local foo
> foo=$(echo "bar"; exit 1)
> echo "$?"
> }
$ f
1
I've got a few Unix shell scripts where I need to check that certain environment variables are set before I start doing stuff, so I do this sort of thing:
if [ -z "$STATE" ]; then
echo "Need to set STATE"
exit 1
fi
if [ -z "$DEST" ]; then
echo "Need to set DEST"
exit 1
fi
which is a lot of typing. Is there a more elegant idiom for checking that a set of environment variables is set?
EDIT: I should mention that these variables have no meaningful default value - the script should error out if any are unset.
Parameter Expansion
The obvious answer is to use one of the special forms of parameter expansion:
: ${STATE?"Need to set STATE"}
: ${DEST:?"Need to set DEST non-empty"}
Or, better (see section on 'Position of double quotes' below):
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
The first variant (using just ?) requires STATE to be set, but STATE="" (an empty string) is OK — not exactly what you want, but the alternative and older notation.
The second variant (using :?) requires DEST to be set and non-empty.
If you supply no message, the shell provides a default message.
The ${var?} construct is portable back to Version 7 UNIX and the Bourne Shell (1978 or thereabouts). The ${var:?} construct is slightly more recent: I think it was in System III UNIX circa 1981, but it may have been in PWB UNIX before that. It is therefore in the Korn Shell, and in the POSIX shells, including specifically Bash.
It is usually documented in the shell's man page in a section called Parameter Expansion. For example, the bash manual says:
${parameter:?word}
Display Error if Null or Unset. If parameter is null or unset, the expansion of word (or a message to that effect if word is not present) is written to the standard error and the shell, if it is not interactive, exits. Otherwise, the value of parameter is substituted.
The Colon Command
I should probably add that the colon command simply has its arguments evaluated and then succeeds. It is the original shell comment notation (before '#' to end of line). For a long time, Bourne shell scripts had a colon as the first character. The C Shell would read a script and use the first character to determine whether it was for the C Shell (a '#' hash) or the Bourne shell (a ':' colon). Then the kernel got in on the act and added support for '#!/path/to/program' and the Bourne shell got '#' comments, and the colon convention went by the wayside. But if you come across a script that starts with a colon, now you will know why.
Position of double quotes
blong asked in a comment:
Any thoughts on this discussion? https://github.com/koalaman/shellcheck/issues/380#issuecomment-145872749
The gist of the discussion is:
… However, when I shellcheck it (with version 0.4.1), I get this message:
In script.sh line 13:
: ${FOO:?"The environment variable 'FOO' must be set and non-empty"}
^-- SC2086: Double quote to prevent globbing and word splitting.
Any advice on what I should do in this case?
The short answer is "do as shellcheck suggests":
: "${STATE?Need to set STATE}"
: "${DEST:?Need to set DEST non-empty}"
To illustrate why, study the following. Note that the : command doesn't echo its arguments (but the shell does evaluate the arguments). We want to see the arguments, so the code below uses printf "%s\n" in place of :.
$ mkdir junk
$ cd junk
$ > abc
$ > def
$ > ghi
$
$ x="*"
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
abc
def
ghi
$ unset x
$ printf "%s\n" ${x:?You must set x} # Careless; not recommended
bash: x: You must set x
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
bash: x: You must set x
$ x="*"
$ printf "%s\n" "${x:?You must set x}" # Careful: should be used
*
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
abc
def
ghi
$ x=
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$ unset x
$ printf "%s\n" ${x:?"You must set x"} # Not quite careful enough
bash: x: You must set x
$
Note how the value in $x is expanded to first * and then a list of file names when the overall expression is not in double quotes. This is what shellcheck is recommending should be fixed. I have not verified that it doesn't object to the form where the expression is enclosed in double quotes, but it is a reasonable assumption that it would be OK.
Try this:
[ -z "$STATE" ] && echo "Need to set STATE" && exit 1;
Your question is dependent on the shell that you are using.
Bourne shell leaves very little in the way of what you're after.
BUT...
It does work, just about everywhere.
Just try and stay away from csh. It was good for the bells and whistles it added, compared the Bourne shell, but it is really creaking now. If you don't believe me, just try and separate out STDERR in csh! (-:
There are two possibilities here. The example above, namely using:
${MyVariable:=SomeDefault}
for the first time you need to refer to $MyVariable. This takes the env. var MyVariable and, if it is currently not set, assigns the value of SomeDefault to the variable for later use.
You also have the possibility of:
${MyVariable:-SomeDefault}
which just substitutes SomeDefault for the variable where you are using this construct. It doesn't assign the value SomeDefault to the variable, and the value of MyVariable will still be null after this statement is encountered.
Surely the simplest approach is to add the -u switch to the shebang (the line at the top of your script), assuming you’re using bash:
#!/bin/sh -u
This will cause the script to exit if any unbound variables lurk within.
${MyVariable:=SomeDefault}
If MyVariable is set and not null, it will reset the variable value (= nothing happens).
Else, MyVariable is set to SomeDefault.
The above will attempt to execute ${MyVariable}, so if you just want to set the variable do:
MyVariable=${MyVariable:=SomeDefault}
In my opinion the simplest and most compatible check for #!/bin/sh is:
if [ "$MYVAR" = "" ]
then
echo "Does not exist"
else
echo "Exists"
fi
Again, this is for /bin/sh and is compatible also on old Solaris systems.
bash 4.2 introduced the -v operator which tests if a name is set to any value, even the empty string.
$ unset a
$ b=
$ c=
$ [[ -v a ]] && echo "a is set"
$ [[ -v b ]] && echo "b is set"
b is set
$ [[ -v c ]] && echo "c is set"
c is set
I always used:
if [ "x$STATE" == "x" ]; then echo "Need to set State"; exit 1; fi
Not that much more concise, I'm afraid.
Under CSH you have $?STATE.
For future people like me, I wanted to go a step forward and parameterize the var name, so I can loop over a variable sized list of variable names:
#!/bin/bash
declare -a vars=(NAME GITLAB_URL GITLAB_TOKEN)
for var_name in "${vars[#]}"
do
if [ -z "$(eval "echo \$$var_name")" ]; then
echo "Missing environment variable $var_name"
exit 1
fi
done
We can write a nice assertion to check a bunch of variables all at once:
#
# assert if variables are set (to a non-empty string)
# if any variable is not set, exit 1 (when -f option is set) or return 1 otherwise
#
# Usage: assert_var_not_null [-f] variable ...
#
function assert_var_not_null() {
local fatal var num_null=0
[[ "$1" = "-f" ]] && { shift; fatal=1; }
for var in "$#"; do
[[ -z "${!var}" ]] &&
printf '%s\n' "Variable '$var' not set" >&2 &&
((num_null++))
done
if ((num_null > 0)); then
[[ "$fatal" ]] && exit 1
return 1
fi
return 0
}
Sample invocation:
one=1 two=2
assert_var_not_null one two
echo test 1: return_code=$?
assert_var_not_null one two three
echo test 2: return_code=$?
assert_var_not_null -f one two three
echo test 3: return_code=$? # this code shouldn't execute
Output:
test 1: return_code=0
Variable 'three' not set
test 2: return_code=1
Variable 'three' not set
More such assertions here: https://github.com/codeforester/base/blob/master/lib/assertions.sh
This can be a way too:
if (set -u; : $HOME) 2> /dev/null
...
...
http://unstableme.blogspot.com/2007/02/checks-whether-envvar-is-set-or-not.html
None of the above solutions worked for my purposes, in part because I checking the environment for an open-ended list of variables that need to be set before starting a lengthy process. I ended up with this:
mapfile -t arr < variables.txt
EXITCODE=0
for i in "${arr[#]}"
do
ISSET=$(env | grep ^${i}= | wc -l)
if [ "${ISSET}" = "0" ];
then
EXITCODE=-1
echo "ENV variable $i is required."
fi
done
exit ${EXITCODE}
Rather than using external shell scripts I tend to load in functions in my login shell. I use something like this as a helper function to check for environment variables rather than any set variable:
is_this_an_env_variable ()
local var="$1"
if env |grep -q "^$var"; then
return 0
else
return 1
fi
}
The $? syntax is pretty neat:
if [ $?BLAH == 1 ]; then
echo "Exists";
else
echo "Does not exist";
fi