This question already has an answer here:
How can I preserve command line spaces in a linux application?
(1 answer)
Closed 3 years ago.
Invoking a shell script with command line arguments containing spaces is generally solved by enclosing the argument in quotes:
getParams.sh 'one two' 'foo bar'
Produces:
one two
foo bar
getParams.sh:
while [[ $# > 0 ]]
do
echo $1
shift
done
However, if a shell variable is first defined to hold the value of the arguments such as:
args="'one two' 'foo bar'"
then why does:
getParams.sh $args
not recognize the single quotes enclosing the grouped arguments? The output is:
'one
two'
'three
four'
How can I store command line arguments containing spaces into a variable so that when getParams is invoked, the arguments are grouped according to the quoted arguments just as in the original example?
Use an array:
args=('one two' 'foo bar')
getParams.sh "${args[#]}"
Using args="'one two' 'foo bar'" wont work, because the single quotes retain their literal value when inside double quotes.
To preserve multiple spaces in the arguments (and also handle special characters such as *), you should quote your variable:
while [[ $# -gt 0 ]]
do
echo "$1"
shift
done
I am looking for a bash one liner that appends a newline after each element of a list. If I call the script as:
./script arg1 arg2 arg3
I want the output to be
arg1
arg2
arg3
I tried different variations of the following. The newline does not get added. Any ordinary char gets added.
# pretty much works except for an extra space
list=${#/%/x}
echo "$list"
# appends 'n'
list=${#/%/\n}
echo "$list"
# appends nothing
list=${#/%/$'\n'}
echo "$list"
# appends nothing, \x078 would append 'x'
list=${#/%/$'\x0D'}
echo "$list"
# appends nothing
CR=$'\n'
list=${#/%/$CR}
echo "$list"
# same issues with arrays
tmp=($#)
list=${tmp/%/\n}
echo "$list"
What fix or alternative do you suggest? I obviously could write a loop or call tr but that's precisely what I thought I could avoid with a bash substitution.
You can use this function with "$#":
f() { printf "%s\n" "$#"; }
f arg1 arg2 arg3
arg1
arg2
arg3
As per man bash
# Expands to the positional parameters, starting from one. When the expansion occurs
within double quotes, each parameter expands to a separate word. That is, "$#" is
equivalent to "$1" "$2" ...
printf would have been my answer as well. Another technique is to use IFS:
$ IFS=$'\n'
$ list="$*"
$ echo "$list"
arg1
arg2
arg3
Notes:
uses ANSI-C quoting for the newline sequence
"$*" (with the quotes, crucial) joins the positional params using the first char of $IFS
quote the shell variable for the echo command to preserve the inner newlines.
That redefines the IFS value for the current shell. You can save the old value first and restore it after:
oldIFS=$IFS; IFS=$'\n'; list="$*"; IFS=$oldIFS
or you can use a subshell so the modification is discarded for you:
$ list=$( IFS=$'\n'; echo "$*" )
This question already has answers here:
Accessing bash command line args $# vs $*
(5 answers)
What is the difference between "$#" and "$*" in Bash? [duplicate]
(2 answers)
Closed 9 years ago.
I have wrote a simple script that takes in any number of parameters to demonstrate the difference between $# and $*:
#!/bin/bash
echo "double quoted $* $#"
echo 'single quoted $* $#'
On the CLI I did
$./stuff.sh a b c d e f dfs
And this is what prints out
double quoted a b c d e f dfs a b c d e f dfs
single quoted $* $#
Since they are identical does that mean $# equals to $*? Or is there a point I am missing?
From Special Parameters in Bash Reference Manual
*
Expands to the positional parameters, starting from one. When the
expansion occurs within double quotes, it expands to a single word
with the value of each parameter separated by the first character of
the IFS special variable. That is, "$*" is equivalent to "$1c$2c…",
where c is the first character of the value of the IFS variable. If
IFS is unset, the parameters are separated by spaces. If IFS is null,
the parameters are joined without intervening separators.
#
Expands to the positional parameters, starting from one. When the
expansion occurs within double quotes, each parameter expands to a
separate word. That is, "$#" is equivalent to "$1" "$2" …. If the
double-quoted expansion occurs within a word, the expansion of the
first parameter is joined with the beginning part of the original
word, and the expansion of the last parameter is joined with the last
part of the original word. When there are no positional parameters,
"$#" and $# expand to nothing (i.e., they are removed).
Better with an example:
$ d=(a b c)
$ for i in "${d[*]}"; do echo $i; done <---- it is a field all together
a b c
$ for i in "${d[#]}"; do echo $i; done <---- each item is a different field
a
b
c
$ set -- "hello world" foo bar
$ printf '<%s> ' "$#"; echo
<hello world> <foo> <bar> # "$#" keeps original separation
$ printf '<%s> ' "$*"; echo
<hello world foo bar> # "$*" combines everything into one string
$ printf '<%s> ' $*; echo
<hello> <world> <foo> <bar> # $* (no quotes) loses delimitation entirely.
$ printf '<%s> ' $#; echo
<hello> <world> <foo> <bar> # unquoted, $# is identical to unquoted $*.
Technically, $* isn't limited to space-delimiting; it uses the first character of $IFS. This can be used to your advantage:
jars=( *.jar )
IFS=:
export CLASSPATH=${jars[*]}
From http://linuxsig.org/files/bash_scripting.html:
$* Stores all the arguments that were entered on the command line
so
$1 $2 ...
"$#" Stores all the arguments that were entered on the command line, individually quoted
so
"$1" "$2" ...
What the title says: what does it mean to encapsulate a variable in {}, "", or "{}"? I haven't been able to find any explanations online about this - I haven't been able to refer to them except for using the symbols, which doesn't yield anything.
Here's an example:
declare -a groups
groups+=("CN=exampleexample,OU=exampleexample,OU=exampleexample,DC=example,DC=com")
groups+=("CN=example example,OU=example example,OU=example example,DC=example,DC=com")
This:
for group in "${groups[#]}"; do
echo $group
done
Proves to be much different than this:
for group in $groups; do
echo $group
done
and this:
for group in ${groups}; do
echo $group
done
Only the first one accomplishes what I want: to iterate through each element in the array. I'm not really clear on the differences between $groups, "$groups", ${groups} and "${groups}". If anyone could explain it, I would appreciate it.
As an extra question - does anyone know the accepted way to refer to these encapsulations?
Braces ($var vs. ${var})
In most cases, $var and ${var} are the same:
var=foo
echo $var
# foo
echo ${var}
# foo
The braces are only needed to resolve ambiguity in expressions:
var=foo
echo $varbar
# Prints nothing because there is no variable 'varbar'
echo ${var}bar
# foobar
Quotes ($var vs. "$var" vs. "${var}")
When you add double quotes around a variable, you tell the shell to treat it as a single word, even if it contains whitespaces:
var="foo bar"
for i in "$var"; do # Expands to 'for i in "foo bar"; do...'
echo $i # so only runs the loop once
done
# foo bar
Contrast that behavior with the following:
var="foo bar"
for i in $var; do # Expands to 'for i in foo bar; do...'
echo $i # so runs the loop twice, once for each argument
done
# foo
# bar
As with $var vs. ${var}, the braces are only needed for disambiguation, for example:
var="foo bar"
for i in "$varbar"; do # Expands to 'for i in ""; do...' since there is no
echo $i # variable named 'varbar', so loop runs once and
done # prints nothing (actually "")
var="foo bar"
for i in "${var}bar"; do # Expands to 'for i in "foo barbar"; do...'
echo $i # so runs the loop once
done
# foo barbar
Note that "${var}bar" in the second example above could also be written "${var}"bar, in which case you don't need the braces anymore, i.e. "$var"bar. However, if you have a lot of quotes in your string these alternative forms can get hard to read (and therefore hard to maintain). This page provides a good introduction to quoting in Bash.
Arrays ($var vs. $var[#] vs. ${var[#]})
Now for your array. According to the bash manual:
Referencing an array variable without a subscript is equivalent to referencing the array with a subscript of 0.
In other words, if you don't supply an index with [], you get the first element of the array:
foo=(a b c)
echo $foo
# a
Which is exactly the same as
foo=(a b c)
echo ${foo}
# a
To get all the elements of an array, you need to use # as the index, e.g. ${foo[#]}. The braces are required with arrays because without them, the shell would expand the $foo part first, giving the first element of the array followed by a literal [#]:
foo=(a b c)
echo ${foo[#]}
# a b c
echo $foo[#]
# a[#]
This page is a good introduction to arrays in Bash.
Quotes revisited (${foo[#]} vs. "${foo[#]}")
You didn't ask about this but it's a subtle difference that's good to know about. If the elements in your array could contain whitespace, you need to use double quotes so that each element is treated as a separate "word:"
foo=("the first" "the second")
for i in "${foo[#]}"; do # Expands to 'for i in "the first" "the second"; do...'
echo $i # so the loop runs twice
done
# the first
# the second
Contrast this with the behavior without double quotes:
foo=("the first" "the second")
for i in ${foo[#]}; do # Expands to 'for i in the first the second; do...'
echo $i # so the loop runs four times!
done
# the
# first
# the
# second
TL;DR
All the examples you give are variations on Bash Shell Expansions. Expansions happen in a particular order, and some have specific use cases.
Braces as Token Delimiters
The ${var} syntax is primarily used for delimiting ambiguous tokens. For example, consider the following:
$ var1=foo; var2=bar; var12=12
$ echo $var12
12
$ echo ${var1}2
foo2
Braces in Array Expansions
The braces are required to access the elements of an array and for other special expansions. For example:
$ foo=(1 2 3)
# Returns first element only.
$ echo $foo
1
# Returns all array elements.
$ echo ${foo[*]}
1 2 3
# Returns number of elements in array.
$ echo ${#foo[*]}
3
Tokenization
Most of the rest of your questions have to do with quoting, and how the shell tokenizes input. Consider the difference in how the shell performs word splitting in the following examples:
$ var1=foo; var2=bar; count_params () { echo $#; }
# Variables are interpolated into a single string.
$ count_params "$var1 $var2"
1
# Each variable is quoted separately, created two arguments.
$ count_params "$var1" "$var2"
2
The # symbol interacts with quoting differently than *. Specifically:
$# "[e]xpands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word."
In an array, "[i]f the word is double-quoted, ${name[*]} expands to a single word with the value of each array member separated by the first character of the IFS variable, and ${name[#]} expands each element of name to a separate word."
You can see this in action as follows:
$ count_params () { echo $#; }
$ set -- foo bar baz
$ count_params "$#"
3
$ count_params "$*"
1
The use of a quoted expansion matters a great deal when variables refer to values with spaces or special characters that might prevent the shell from word-splitting the way you intend. See Quoting for more on how quoting works in Bash.
You need to distinguish between arrays and simple variables — and your example is using an array.
For plain variables:
$var and ${var} are exactly equivalent.
"$var" and "${var}" are exactly equivalent.
However, the two pairs are not 100% identical in all cases. Consider the output below:
$ var=" abc def "
$ printf "X%sX\n" $var
XabcX
XdefX
$ printf "X%sX\n" "${var}"
X abc def X
$
Without the double quotes around the variable, the internal spacing is lost and the expansion is treated as two arguments to the printf command. With the double quotes around the variable, the internal spacing is preserved and the expansion is treated as one argument to the printf command.
With arrays, the rules are both similar and different.
If groups is an array, referencing $groups or ${groups} is tantamount to referencing ${groups[0]}, the zeroth element of the array.
Referencing "${groups[#]}" is analogous to referencing "$#"; it preserves the spacing in the individual elements of the array, and returns a list of values, one value per element of the array.
Referencing ${groups[#]} without the double quotes does not preserve spacing and can introduce more values than there are elements in the array if some of the elements contain spaces.
For example:
$ groups=("abc def" " pqr xyz ")
$ printf "X%sX\n" ${groups[#]}
XabcX
XdefX
XpqrX
XxyzX
$ printf "X%sX\n" "${groups[#]}"
Xabc defX
X pqr xyz X
$ printf "X%sX\n" $groups
XabcX
XdefX
$ printf "X%sX\n" "$groups"
Xabc defX
$
Using * instead of # leads to subtly different results.
See also How to iterate over the arguments in a bash script.
The second sentence of the first paragraph under Parameter Expansion in man bash says,
The parameter name or symbol to be expanded may be enclosed in braces, which are optional but serve to protect the variable to be expanded from characters immediately following it which could be interpreted as part of the name.
Which tells you that the name is simply braces, and the main purpose is to clarify where the name begins and ends:
foo='bar'
echo "$foobar"
# nothing
echo "${foo}bar"
barbar
If you read further you discover,
The braces are required when parameter is a positional parameter with more than one digit…
Let's test:
$ set -- {0..100}
$ echo $22
12
$ echo ${22}
20
Huh. Neat. I honestly didn't know that before writing this (I've never had more than 9 positional parameters before.)
Of course, you also need braces to do the powerful parameter expansion features like
${parameter:-word}
${parameter:=word}
${parameter:?word}
… [read the section for more]
as well as array expansion.
A related case not covered above. Quoting an empty variable seems to change things for test -n. This is specifically given as an example in the info text for coreutils, but not really explained:
16.3.4 String tests
-------------------
These options test string characteristics. You may need to quote
STRING arguments for the shell. For example:
test -n "$V"
The quotes here prevent the wrong arguments from being passed to
`test' if `$V' is empty or contains special characters.
I'd love to hear the detailed explanation. My testing confirms this, and I'm now quoting my variables for all string tests, to avoid having -z and -n return the same result.
$ unset a
$ if [ -z $a ]; then echo unset; else echo set; fi
unset
$ if [ -n $a ]; then echo set; else echo unset; fi
set # highly unexpected!
$ unset a
$ if [ -z "$a" ]; then echo unset; else echo set; fi
unset
$ if [ -n "$a" ]; then echo set; else echo unset; fi
unset # much better
Well, I know that encapsulation of a variable helps you to work with something like:
${groups%example}
or syntax like that, where you want to do something with your variable before returning the value.
Now, if you see your code, all the magic is inside
${groups[#]}
the magic is in there because you can't write just: $groups[#]
You're putting your variable inside the {} because you want to use special characters [] and #. You can't name or call your variable just: # or something[] because these are reserved characters for other operations and names.
$var and ${var} are the same, if var is the name of the variable.
The braces are required when parameter is a positional parameter with more than one digit, or when parameter is followed by a character that is not to be interpreted as part of its name.
Thus, "$var" and "${var}" are the same.
However, $var and "$var" are different.
Bash will do Word Splitting for $var but not for "$var".
The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.
Note: Word splitting won't be performed in variable assignment:
https://www.gnu.org/software/bash/manual/html_node/Shell-Parameters.html
A variable may be assigned to by a statement of the form
name=[value]
If value is not given, the variable is assigned the null string. All values undergo tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, and quote removal (see Shell Parameter Expansion). If the variable has its integer attribute set, then value is evaluated as an arithmetic expression even if the $((…)) expansion is not used (see Arithmetic Expansion). Word splitting and filename expansion are not performed.
This question already has answers here:
What is the difference between $# and $* in shell scripts?
(3 answers)
Closed 8 years ago.
It seems to me that they both store all the command-line arguments.
So is there a difference between the two?
The difference is subtle; "$*" creates one argument separated by the $IFS variable, while "$#" will expand into separate arguments. As an example, consider:
for i in "$#"; do echo "# '$i'"; done
for i in "$*"; do echo "* '$i'"; done
When run with multiple arguments:
./testvar foo bar baz 'long arg'
# 'foo'
# 'bar'
# 'baz'
# 'long arg'
* 'foo bar baz long arg'
For more details:
http://www.gnu.org/software/bash/manual/bashref.html#Special-Parameters
$*
Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, it expands to a single word with the value of each parameter separated by the first character of the IFS special variable. That is, "$*" is equivalent to "$1c$2c...", where c is the first character of the value of the IFS variable. If IFS is unset, the parameters are separated by spaces. If IFS is null, the parameters are joined without intervening separators.
$#
Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is, "$#" is equivalent to "$1" "$2" .... If the double-quoted expansion occurs within a word, the expansion of the first parameter is joined with the beginning part of the original word, and the expansion of the last parameter is joined with the last part of the original word. When there are no positional parameters, "$#" and $# expand to nothing (i.e., they are removed).
A key difference from my POV is that "$#" preserves the original number
of arguments. It's the only form that does.
For example, if file my_script contains:
#!/bin/bash
main()
{
echo 'MAIN sees ' $# ' args'
}
main $*
main $#
main "$*"
main "$#"
### end ###
and I run it like this:
my_script 'a b c' d e
I will get this output:
MAIN sees 5 args
MAIN sees 5 args
MAIN sees 1 args
MAIN sees 3 args