Related
In shell scripts, when do we use {} when expanding variables?
For example, I have seen the following:
var=10 # Declare variable
echo "${var}" # One use of the variable
echo "$var" # Another use of the variable
Is there a significant difference, or is it just style? Is one preferred over the other?
In this particular example, it makes no difference. However, the {} in ${} are useful if you want to expand the variable foo in the string
"${foo}bar"
since "$foobar" would instead expand the variable identified by foobar.
Curly braces are also unconditionally required when:
expanding array elements, as in ${array[42]}
using parameter expansion operations, as in ${filename%.*} (remove extension)
expanding positional parameters beyond 9: "$8 $9 ${10} ${11}"
Doing this everywhere, instead of just in potentially ambiguous cases, can be considered good programming practice. This is both for consistency and to avoid surprises like $foo_$bar.jpg, where it's not visually obvious that the underscore becomes part of the variable name.
Variables are declared and assigned without $ and without {}. You have to use
var=10
to assign. In order to read from the variable (in other words, 'expand' the variable), you must use $.
$var # use the variable
${var} # same as above
${var}bar # expand var, and append "bar" too
$varbar # same as ${varbar}, i.e expand a variable called varbar, if it exists.
This has confused me sometimes - in other languages we refer to the variable in the same way, regardless of whether it's on the left or right of an assignment. But shell-scripting is different, $var=10 doesn't do what you might think it does!
You use {} for grouping. The braces are required to dereference array elements. Example:
dir=(*) # store the contents of the directory into an array
echo "${dir[0]}" # get the first entry.
echo "$dir[0]" # incorrect
You are also able to do some text manipulation inside the braces:
STRING="./folder/subfolder/file.txt"
echo ${STRING} ${STRING%/*/*}
Result:
./folder/subfolder/file.txt ./folder
or
STRING="This is a string"
echo ${STRING// /_}
Result:
This_is_a_string
You are right in "regular variables" are not needed... But it is more helpful for the debugging and to read a script.
Curly braces are always needed for accessing array elements and carrying out brace expansion.
It's good to be not over-cautious and use {} for shell variable expansion even when there is no scope for ambiguity.
For example:
dir=log
prog=foo
path=/var/${dir}/${prog} # excessive use of {}, not needed since / can't be a part of a shell variable name
logfile=${path}/${prog}.log # same as above, . can't be a part of a shell variable name
path_copy=${path} # {} is totally unnecessary
archive=${logfile}_arch # {} is needed since _ can be a part of shell variable name
So, it is better to write the three lines as:
path=/var/$dir/$prog
logfile=$path/$prog.log
path_copy=$path
which is definitely more readable.
Since a variable name can't start with a digit, shell doesn't need {} around numbered variables (like $1, $2 etc.) unless such expansion is followed by a digit. That's too subtle and it does make to explicitly use {} in such contexts:
set app # set $1 to app
fruit=$1le # sets fruit to apple, but confusing
fruit=${1}le # sets fruit to apple, makes the intention clear
See:
Allowed characters in Linux environment variable names
The end of the variable name is usually signified by a space or newline. But what if we don't want a space or newline after printing the variable value? The curly braces tell the shell interpreter where the end of the variable name is.
Classic Example 1) - shell variable without trailing whitespace
TIME=10
# WRONG: no such variable called 'TIMEsecs'
echo "Time taken = $TIMEsecs"
# What we want is $TIME followed by "secs" with no whitespace between the two.
echo "Time taken = ${TIME}secs"
Example 2) Java classpath with versioned jars
# WRONG - no such variable LATESTVERSION_src
CLASSPATH=hibernate-$LATESTVERSION_src.zip:hibernate_$LATEST_VERSION.jar
# RIGHT
CLASSPATH=hibernate-${LATESTVERSION}_src.zip:hibernate_$LATEST_VERSION.jar
(Fred's answer already states this but his example is a bit too abstract)
Following SierraX and Peter's suggestion about text manipulation, curly brackets {} are used to pass a variable to a command, for instance:
Let's say you have a sposi.txt file containing the first line of a well-known Italian novel:
> sposi="somewhere/myfolder/sposi.txt"
> cat $sposi
Ouput: quel ramo del lago di como che volge a mezzogiorno
Now create two variables:
# Search the 2nd word found in the file that "sposi" variable points to
> word=$(cat $sposi | cut -d " " -f 2)
# This variable will replace the word
> new_word="filone"
Now substitute the word variable content with the one of new_word, inside sposi.txt file
> sed -i "s/${word}/${new_word}/g" $sposi
> cat $sposi
Ouput: quel filone del lago di como che volge a mezzogiorno
The word "ramo" has been replaced.
In shell scripts, when do we use {} when expanding variables?
For example, I have seen the following:
var=10 # Declare variable
echo "${var}" # One use of the variable
echo "$var" # Another use of the variable
Is there a significant difference, or is it just style? Is one preferred over the other?
In this particular example, it makes no difference. However, the {} in ${} are useful if you want to expand the variable foo in the string
"${foo}bar"
since "$foobar" would instead expand the variable identified by foobar.
Curly braces are also unconditionally required when:
expanding array elements, as in ${array[42]}
using parameter expansion operations, as in ${filename%.*} (remove extension)
expanding positional parameters beyond 9: "$8 $9 ${10} ${11}"
Doing this everywhere, instead of just in potentially ambiguous cases, can be considered good programming practice. This is both for consistency and to avoid surprises like $foo_$bar.jpg, where it's not visually obvious that the underscore becomes part of the variable name.
Variables are declared and assigned without $ and without {}. You have to use
var=10
to assign. In order to read from the variable (in other words, 'expand' the variable), you must use $.
$var # use the variable
${var} # same as above
${var}bar # expand var, and append "bar" too
$varbar # same as ${varbar}, i.e expand a variable called varbar, if it exists.
This has confused me sometimes - in other languages we refer to the variable in the same way, regardless of whether it's on the left or right of an assignment. But shell-scripting is different, $var=10 doesn't do what you might think it does!
You use {} for grouping. The braces are required to dereference array elements. Example:
dir=(*) # store the contents of the directory into an array
echo "${dir[0]}" # get the first entry.
echo "$dir[0]" # incorrect
You are also able to do some text manipulation inside the braces:
STRING="./folder/subfolder/file.txt"
echo ${STRING} ${STRING%/*/*}
Result:
./folder/subfolder/file.txt ./folder
or
STRING="This is a string"
echo ${STRING// /_}
Result:
This_is_a_string
You are right in "regular variables" are not needed... But it is more helpful for the debugging and to read a script.
Curly braces are always needed for accessing array elements and carrying out brace expansion.
It's good to be not over-cautious and use {} for shell variable expansion even when there is no scope for ambiguity.
For example:
dir=log
prog=foo
path=/var/${dir}/${prog} # excessive use of {}, not needed since / can't be a part of a shell variable name
logfile=${path}/${prog}.log # same as above, . can't be a part of a shell variable name
path_copy=${path} # {} is totally unnecessary
archive=${logfile}_arch # {} is needed since _ can be a part of shell variable name
So, it is better to write the three lines as:
path=/var/$dir/$prog
logfile=$path/$prog.log
path_copy=$path
which is definitely more readable.
Since a variable name can't start with a digit, shell doesn't need {} around numbered variables (like $1, $2 etc.) unless such expansion is followed by a digit. That's too subtle and it does make to explicitly use {} in such contexts:
set app # set $1 to app
fruit=$1le # sets fruit to apple, but confusing
fruit=${1}le # sets fruit to apple, makes the intention clear
See:
Allowed characters in Linux environment variable names
The end of the variable name is usually signified by a space or newline. But what if we don't want a space or newline after printing the variable value? The curly braces tell the shell interpreter where the end of the variable name is.
Classic Example 1) - shell variable without trailing whitespace
TIME=10
# WRONG: no such variable called 'TIMEsecs'
echo "Time taken = $TIMEsecs"
# What we want is $TIME followed by "secs" with no whitespace between the two.
echo "Time taken = ${TIME}secs"
Example 2) Java classpath with versioned jars
# WRONG - no such variable LATESTVERSION_src
CLASSPATH=hibernate-$LATESTVERSION_src.zip:hibernate_$LATEST_VERSION.jar
# RIGHT
CLASSPATH=hibernate-${LATESTVERSION}_src.zip:hibernate_$LATEST_VERSION.jar
(Fred's answer already states this but his example is a bit too abstract)
Following SierraX and Peter's suggestion about text manipulation, curly brackets {} are used to pass a variable to a command, for instance:
Let's say you have a sposi.txt file containing the first line of a well-known Italian novel:
> sposi="somewhere/myfolder/sposi.txt"
> cat $sposi
Ouput: quel ramo del lago di como che volge a mezzogiorno
Now create two variables:
# Search the 2nd word found in the file that "sposi" variable points to
> word=$(cat $sposi | cut -d " " -f 2)
# This variable will replace the word
> new_word="filone"
Now substitute the word variable content with the one of new_word, inside sposi.txt file
> sed -i "s/${word}/${new_word}/g" $sposi
> cat $sposi
Ouput: quel filone del lago di como che volge a mezzogiorno
The word "ramo" has been replaced.
For each of two examples below I'll try to explain what result I expected and what I got instead. I'm hoping for you to help me understand why I was wrong.
1)
VAR1=VAR2
$VAR1=FOO
result: -bash: VAR2=FOO: command not found
In the second line, $VAR1 gets expanded to VAR2, but why does Bash interpret the resulting VAR2=FOO as a command name rather than a variable assignment?
2)
'VAR=FOO'
result: -bash: VAR=FOO: command not found
Why do the quotes make Bash treat the variable assignment as a command name?
Could you please describe, step by step, how Bash processes my two examples?
How best to indirectly assign variables is adequately answered in other Q&A entries in this knowledgebase. Among those:
Indirect variable assignment in bash
Saving function output into a variable named in an argument
If that's what you actually intend to ask, then this question should be closed as a duplicate. I'm going to make a contrary assumption and focus on the literal question -- why your other approaches failed -- below.
What does the POSIX sh language specify as a valid assignment? Why does $var1=foo or 'var=foo' fail?
Background: On the POSIX sh specification
The POSIX shell command language specification is very specific about what constitutes an assignment, as quoted below:
4.21 Variable Assignment
In the shell command language, a word consisting of the following parts:
varname=value
When used in a context where assignment is defined to occur and at no other time, the value (representing a word or field) shall be assigned as the value of the variable denoted by varname.
The varname and value parts shall meet the requirements for a name and a word, respectively, except that they are delimited by the embedded unquoted equals-sign, in addition to other delimiters.
Also, from section 2.9.1, on Simple Commands, with emphasis added:
The words that are recognized as variable assignments or redirections according to Shell Grammar Rules are saved for processing in steps 3 and 4.
The words that are not variable assignments or redirections shall be expanded. If any fields remain following their expansion, the first field shall be considered the command name and remaining fields are the arguments for the command.
Redirections shall be performed as described in Redirection.
Each variable assignment shall be expanded for tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal prior to assigning the value.
Also, from the grammar:
If all the characters preceding '=' form a valid name (see the Base Definitions volume of IEEE Std 1003.1-2001, Section 3.230, Name), the token ASSIGNMENT_WORD shall be returned. (Quoted characters cannot participate in forming a valid name.)
Note from this:
The command must be recognized as an assignment at the very beginning of the parsing sequence, before any expansions (or quote removal!) have taken place.
The name must be a valid name. Literal quotes are not part of a valid variable name.
The equals sign must be unquoted. In your second example, the entire string was quoted.
Assignments are recognized before tilde expansion, parameter expansion, command substitution, etc.
Why $var1=foo fails to act as an assignment
As given in the grammar, all characters before the = in an assignment must be valid characters within a variable name for an assignment to be recognized. $ is not a valid character in a name. Because assignments are recognized in step 1 of simple command processing, before expansion takes place, the literal text $var1, not the value of that variable, is used for this matching.
Why 'var=foo' fails to act as an assignment
First, all characters before the = must be valid in variable names, and ' is not valid in a variable name.
Second, an assignment is only recognized if the = is not quoted.
1)
VAR1=VAR2
$VAR1=FOO
You want to use a variable name contained in a variable for the assignment. Bash syntax does not allow this. However, there is an easy workaround :
VAR1=VAR2
declare "$VAR1"=FOO
It works with local and export too.
2)
By using single quotes (double quotes would yield the same result), you are telling Bash that what is inside is a string and to treat it as a single entity. Since it is the first item on the line, Bash tries to find an alias, or shell builtin, or an executable file in its PATH, that would be named VAR=FOO. Not finding it, it tells you there is no such command.
An assignment is not a normal command. To perform an assignment contained in a quote, you would need to use eval, like so :
eval "$VAR1=FOO" # But please don't do that in real life
Most experienced bash programmers would probably tell you to avoid eval, as it has serious drawbacks, and I am giving it as an example just to recommend against its use : while in the example above it would not involve any security risk or error potential because the value of VAR1 is known and safe, there are many cases where an arbitrary (i.e. user-supplied) value could cause a crash or unexpected behavior. Quoting inside an eval statement is also more difficult and reduces readability.
You declare VAR2 earlier in the program, right?
If you are trying to assign the value of VAR2 to VAR1, then you need to make sure and use $ in front of VAR2, like so:
VAR1=$VAR2
That will set the value of VAR2 equal to VAR1, because when you utilize the $, you are saying that value that is stored in the variable. Otherwise it doesn't recognize it as a variable.
Basically, a variable that doesn't have a $ in front of it will be interpreted as a command. Any word will. That's why we have the $ to clarify "hey this is a variable".
In shell scripts, when do we use {} when expanding variables?
For example, I have seen the following:
var=10 # Declare variable
echo "${var}" # One use of the variable
echo "$var" # Another use of the variable
Is there a significant difference, or is it just style? Is one preferred over the other?
In this particular example, it makes no difference. However, the {} in ${} are useful if you want to expand the variable foo in the string
"${foo}bar"
since "$foobar" would instead expand the variable identified by foobar.
Curly braces are also unconditionally required when:
expanding array elements, as in ${array[42]}
using parameter expansion operations, as in ${filename%.*} (remove extension)
expanding positional parameters beyond 9: "$8 $9 ${10} ${11}"
Doing this everywhere, instead of just in potentially ambiguous cases, can be considered good programming practice. This is both for consistency and to avoid surprises like $foo_$bar.jpg, where it's not visually obvious that the underscore becomes part of the variable name.
Variables are declared and assigned without $ and without {}. You have to use
var=10
to assign. In order to read from the variable (in other words, 'expand' the variable), you must use $.
$var # use the variable
${var} # same as above
${var}bar # expand var, and append "bar" too
$varbar # same as ${varbar}, i.e expand a variable called varbar, if it exists.
This has confused me sometimes - in other languages we refer to the variable in the same way, regardless of whether it's on the left or right of an assignment. But shell-scripting is different, $var=10 doesn't do what you might think it does!
You use {} for grouping. The braces are required to dereference array elements. Example:
dir=(*) # store the contents of the directory into an array
echo "${dir[0]}" # get the first entry.
echo "$dir[0]" # incorrect
You are also able to do some text manipulation inside the braces:
STRING="./folder/subfolder/file.txt"
echo ${STRING} ${STRING%/*/*}
Result:
./folder/subfolder/file.txt ./folder
or
STRING="This is a string"
echo ${STRING// /_}
Result:
This_is_a_string
You are right in "regular variables" are not needed... But it is more helpful for the debugging and to read a script.
Curly braces are always needed for accessing array elements and carrying out brace expansion.
It's good to be not over-cautious and use {} for shell variable expansion even when there is no scope for ambiguity.
For example:
dir=log
prog=foo
path=/var/${dir}/${prog} # excessive use of {}, not needed since / can't be a part of a shell variable name
logfile=${path}/${prog}.log # same as above, . can't be a part of a shell variable name
path_copy=${path} # {} is totally unnecessary
archive=${logfile}_arch # {} is needed since _ can be a part of shell variable name
So, it is better to write the three lines as:
path=/var/$dir/$prog
logfile=$path/$prog.log
path_copy=$path
which is definitely more readable.
Since a variable name can't start with a digit, shell doesn't need {} around numbered variables (like $1, $2 etc.) unless such expansion is followed by a digit. That's too subtle and it does make to explicitly use {} in such contexts:
set app # set $1 to app
fruit=$1le # sets fruit to apple, but confusing
fruit=${1}le # sets fruit to apple, makes the intention clear
See:
Allowed characters in Linux environment variable names
The end of the variable name is usually signified by a space or newline. But what if we don't want a space or newline after printing the variable value? The curly braces tell the shell interpreter where the end of the variable name is.
Classic Example 1) - shell variable without trailing whitespace
TIME=10
# WRONG: no such variable called 'TIMEsecs'
echo "Time taken = $TIMEsecs"
# What we want is $TIME followed by "secs" with no whitespace between the two.
echo "Time taken = ${TIME}secs"
Example 2) Java classpath with versioned jars
# WRONG - no such variable LATESTVERSION_src
CLASSPATH=hibernate-$LATESTVERSION_src.zip:hibernate_$LATEST_VERSION.jar
# RIGHT
CLASSPATH=hibernate-${LATESTVERSION}_src.zip:hibernate_$LATEST_VERSION.jar
(Fred's answer already states this but his example is a bit too abstract)
Following SierraX and Peter's suggestion about text manipulation, curly brackets {} are used to pass a variable to a command, for instance:
Let's say you have a sposi.txt file containing the first line of a well-known Italian novel:
> sposi="somewhere/myfolder/sposi.txt"
> cat $sposi
Ouput: quel ramo del lago di como che volge a mezzogiorno
Now create two variables:
# Search the 2nd word found in the file that "sposi" variable points to
> word=$(cat $sposi | cut -d " " -f 2)
# This variable will replace the word
> new_word="filone"
Now substitute the word variable content with the one of new_word, inside sposi.txt file
> sed -i "s/${word}/${new_word}/g" $sposi
> cat $sposi
Ouput: quel filone del lago di como che volge a mezzogiorno
The word "ramo" has been replaced.
A contrived example... given
FOO="/foo/bar/baz"
this works (in bash)
BAR=$(basename $FOO) # result is BAR="baz"
BAZ=${BAR:0:1} # result is BAZ="b"
this doesn't
BAZ=${$(basename $FOO):0:1} # result is bad substitution
My question is which rule causes this [subshell substitution] to evaluate incorrectly? And what is the correct way, if any, to do this in 1 hop?
First off, note that when you say this:
BAR=$(basename $FOO) # result is BAR="baz"
BAZ=${BAR:0:1} # result is BAZ="b"
the first bit in the construct for BAZ is BAR and not the value that you want to take the first character of. So even if bash allowed variable names to contain arbitrary characters your result in the second expression wouldn't be what you want.
However, as to the rule that's preventing this, allow me to quote from the bash man page:
DEFINITIONS
The following definitions are used throughout the rest of this docu‐
ment.
blank A space or tab.
word A sequence of characters considered as a single unit by the
shell. Also known as a token.
name A word consisting only of alphanumeric characters and under‐
scores, and beginning with an alphabetic character or an under‐
score. Also referred to as an identifier.
Then a bit later:
PARAMETERS
A parameter is an entity that stores values. It can be a name, a num‐
ber, or one of the special characters listed below under Special Param‐
eters. A variable is a parameter denoted by a name. A variable has a
value and zero or more attributes. Attributes are assigned using the
declare builtin command (see declare below in SHELL BUILTIN COMMANDS).
And later when it defines the syntax you're asking about:
${parameter:offset:length}
Substring Expansion. Expands to up to length characters of
parameter starting at the character specified by offset.
So the rules as articulated in the manpage say that the ${foo:x:y} construct must have a parameter as the first part, and that a parameter can only be a name, a number, or one of the few special parameter characters. $(basename $FOO) is not one of the allowed possibilities for a parameter.
As for a way to do this in one assignment, use a pipe to other commands as mentioned in other responses.
Modified forms of parameter substitution such as ${parameter#word} can only modify a parameter, not an arbitrary word.
In this case, you might pipe the output of basename to a dd command, like
BAR=$(basename -- "$FOO" | dd bs=1 count=1 2>/dev/null)
(If you want a higher count, increase count and not bs, otherwise you may get fewer bytes than requested.)
In the general case, there is no way to do things like this in one assignment.
It fails because ${BAR:0:1} is a variable expansion. Bash expects to see a variable name after ${, not a value.
I'm not aware of a way to do it in a single expression.
As others have said, the first parameter of ${} needs to be a variable name. But you can use another subshell to approximate what you're trying to do.
Instead of:
BAZ=${$(basename $FOO):0:1} # result is bad substitution
Use:
BAZ=$(_TMP=$(basename $FOO); echo ${_TMP:0:1}) # this works
A contrived solution for your contrived example:
BAZ=$(expr $(basename $FOO) : '\(.\)')
as in
$ FOO=/abc/def/ghi/jkl
$ BAZ=$(expr $(basename $FOO) : '\(.\)')
$ echo $BAZ
j
${string:0:1},string must be a variable name
for example:
FOO="/foo/bar/baz"
baz="foo"
BAZ=eval echo '${'"$(basename $FOO)"':0:1}'
echo $BAZ
the result is 'f'