I want my script to perform the product of all its integer arguments. Instead of performing a loop I tried to replace blanks with * and then compute the operation. But I got the following result which I don't understand:
#!/bin/bash
# product.sh
echo $(( ${*// /*} )) # syntax error with ./product.sh 2 3 4
args=$*
echo $(( ${args// /*} )) # ./product.sh 2 3 4 => outputs 24
How is it that the first one produces an error while using an intermediate variable works fine?
How is it that the first one produces an error:
From the Bash Reference Manual:
If parameter is ‘#’ or ‘*’, the substitution operation is applied to each positional parameter in turn
(emphasis mine)
That is, the expression ${*// /*} replaces spaces inside positional parameters, not the spaces separating positional parameters. That expression expands to 2 3 4 (which gives a syntax error when used in an arithmetic context), since the parameters itself don't contain a space. Try with
./product '2 ' '3 ' 4
and you will see the difference.
In your example, the value $* does not actually contain any literal spaces, so ${*// /*} does not do anything.
If it did, those asterisks would be subject to wildcard expansion, so the idea of performing a substitution would seem to be rather brittle even if it worked.
I would simply create a function to process the arguments, instead of rely on trickery with substitutions -- these tend to have icky corner cases when one of the arguments is a variable or etc.
mul () {
case $# in
[01]) echo "$#";;
*) local n=$1; shift; echo $((n * $(mul "$#")));;
esac
}
You may utilize IFS:
#!/bin/bash
# product.sh
# set IFS to *
IFS='*'
# use IFS in output
echo "$*"
# perform arithmetic
echo "$(( $* ))";
Output:
2*3*4
24
Or use printf, like this:
echo $(( $(printf '%s*' $*)1 ))
What the title says: what does it mean to encapsulate a variable in {}, "", or "{}"? I haven't been able to find any explanations online about this - I haven't been able to refer to them except for using the symbols, which doesn't yield anything.
Here's an example:
declare -a groups
groups+=("CN=exampleexample,OU=exampleexample,OU=exampleexample,DC=example,DC=com")
groups+=("CN=example example,OU=example example,OU=example example,DC=example,DC=com")
This:
for group in "${groups[#]}"; do
echo $group
done
Proves to be much different than this:
for group in $groups; do
echo $group
done
and this:
for group in ${groups}; do
echo $group
done
Only the first one accomplishes what I want: to iterate through each element in the array. I'm not really clear on the differences between $groups, "$groups", ${groups} and "${groups}". If anyone could explain it, I would appreciate it.
As an extra question - does anyone know the accepted way to refer to these encapsulations?
Braces ($var vs. ${var})
In most cases, $var and ${var} are the same:
var=foo
echo $var
# foo
echo ${var}
# foo
The braces are only needed to resolve ambiguity in expressions:
var=foo
echo $varbar
# Prints nothing because there is no variable 'varbar'
echo ${var}bar
# foobar
Quotes ($var vs. "$var" vs. "${var}")
When you add double quotes around a variable, you tell the shell to treat it as a single word, even if it contains whitespaces:
var="foo bar"
for i in "$var"; do # Expands to 'for i in "foo bar"; do...'
echo $i # so only runs the loop once
done
# foo bar
Contrast that behavior with the following:
var="foo bar"
for i in $var; do # Expands to 'for i in foo bar; do...'
echo $i # so runs the loop twice, once for each argument
done
# foo
# bar
As with $var vs. ${var}, the braces are only needed for disambiguation, for example:
var="foo bar"
for i in "$varbar"; do # Expands to 'for i in ""; do...' since there is no
echo $i # variable named 'varbar', so loop runs once and
done # prints nothing (actually "")
var="foo bar"
for i in "${var}bar"; do # Expands to 'for i in "foo barbar"; do...'
echo $i # so runs the loop once
done
# foo barbar
Note that "${var}bar" in the second example above could also be written "${var}"bar, in which case you don't need the braces anymore, i.e. "$var"bar. However, if you have a lot of quotes in your string these alternative forms can get hard to read (and therefore hard to maintain). This page provides a good introduction to quoting in Bash.
Arrays ($var vs. $var[#] vs. ${var[#]})
Now for your array. According to the bash manual:
Referencing an array variable without a subscript is equivalent to referencing the array with a subscript of 0.
In other words, if you don't supply an index with [], you get the first element of the array:
foo=(a b c)
echo $foo
# a
Which is exactly the same as
foo=(a b c)
echo ${foo}
# a
To get all the elements of an array, you need to use # as the index, e.g. ${foo[#]}. The braces are required with arrays because without them, the shell would expand the $foo part first, giving the first element of the array followed by a literal [#]:
foo=(a b c)
echo ${foo[#]}
# a b c
echo $foo[#]
# a[#]
This page is a good introduction to arrays in Bash.
Quotes revisited (${foo[#]} vs. "${foo[#]}")
You didn't ask about this but it's a subtle difference that's good to know about. If the elements in your array could contain whitespace, you need to use double quotes so that each element is treated as a separate "word:"
foo=("the first" "the second")
for i in "${foo[#]}"; do # Expands to 'for i in "the first" "the second"; do...'
echo $i # so the loop runs twice
done
# the first
# the second
Contrast this with the behavior without double quotes:
foo=("the first" "the second")
for i in ${foo[#]}; do # Expands to 'for i in the first the second; do...'
echo $i # so the loop runs four times!
done
# the
# first
# the
# second
TL;DR
All the examples you give are variations on Bash Shell Expansions. Expansions happen in a particular order, and some have specific use cases.
Braces as Token Delimiters
The ${var} syntax is primarily used for delimiting ambiguous tokens. For example, consider the following:
$ var1=foo; var2=bar; var12=12
$ echo $var12
12
$ echo ${var1}2
foo2
Braces in Array Expansions
The braces are required to access the elements of an array and for other special expansions. For example:
$ foo=(1 2 3)
# Returns first element only.
$ echo $foo
1
# Returns all array elements.
$ echo ${foo[*]}
1 2 3
# Returns number of elements in array.
$ echo ${#foo[*]}
3
Tokenization
Most of the rest of your questions have to do with quoting, and how the shell tokenizes input. Consider the difference in how the shell performs word splitting in the following examples:
$ var1=foo; var2=bar; count_params () { echo $#; }
# Variables are interpolated into a single string.
$ count_params "$var1 $var2"
1
# Each variable is quoted separately, created two arguments.
$ count_params "$var1" "$var2"
2
The # symbol interacts with quoting differently than *. Specifically:
$# "[e]xpands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word."
In an array, "[i]f the word is double-quoted, ${name[*]} expands to a single word with the value of each array member separated by the first character of the IFS variable, and ${name[#]} expands each element of name to a separate word."
You can see this in action as follows:
$ count_params () { echo $#; }
$ set -- foo bar baz
$ count_params "$#"
3
$ count_params "$*"
1
The use of a quoted expansion matters a great deal when variables refer to values with spaces or special characters that might prevent the shell from word-splitting the way you intend. See Quoting for more on how quoting works in Bash.
You need to distinguish between arrays and simple variables — and your example is using an array.
For plain variables:
$var and ${var} are exactly equivalent.
"$var" and "${var}" are exactly equivalent.
However, the two pairs are not 100% identical in all cases. Consider the output below:
$ var=" abc def "
$ printf "X%sX\n" $var
XabcX
XdefX
$ printf "X%sX\n" "${var}"
X abc def X
$
Without the double quotes around the variable, the internal spacing is lost and the expansion is treated as two arguments to the printf command. With the double quotes around the variable, the internal spacing is preserved and the expansion is treated as one argument to the printf command.
With arrays, the rules are both similar and different.
If groups is an array, referencing $groups or ${groups} is tantamount to referencing ${groups[0]}, the zeroth element of the array.
Referencing "${groups[#]}" is analogous to referencing "$#"; it preserves the spacing in the individual elements of the array, and returns a list of values, one value per element of the array.
Referencing ${groups[#]} without the double quotes does not preserve spacing and can introduce more values than there are elements in the array if some of the elements contain spaces.
For example:
$ groups=("abc def" " pqr xyz ")
$ printf "X%sX\n" ${groups[#]}
XabcX
XdefX
XpqrX
XxyzX
$ printf "X%sX\n" "${groups[#]}"
Xabc defX
X pqr xyz X
$ printf "X%sX\n" $groups
XabcX
XdefX
$ printf "X%sX\n" "$groups"
Xabc defX
$
Using * instead of # leads to subtly different results.
See also How to iterate over the arguments in a bash script.
The second sentence of the first paragraph under Parameter Expansion in man bash says,
The parameter name or symbol to be expanded may be enclosed in braces, which are optional but serve to protect the variable to be expanded from characters immediately following it which could be interpreted as part of the name.
Which tells you that the name is simply braces, and the main purpose is to clarify where the name begins and ends:
foo='bar'
echo "$foobar"
# nothing
echo "${foo}bar"
barbar
If you read further you discover,
The braces are required when parameter is a positional parameter with more than one digit…
Let's test:
$ set -- {0..100}
$ echo $22
12
$ echo ${22}
20
Huh. Neat. I honestly didn't know that before writing this (I've never had more than 9 positional parameters before.)
Of course, you also need braces to do the powerful parameter expansion features like
${parameter:-word}
${parameter:=word}
${parameter:?word}
… [read the section for more]
as well as array expansion.
A related case not covered above. Quoting an empty variable seems to change things for test -n. This is specifically given as an example in the info text for coreutils, but not really explained:
16.3.4 String tests
-------------------
These options test string characteristics. You may need to quote
STRING arguments for the shell. For example:
test -n "$V"
The quotes here prevent the wrong arguments from being passed to
`test' if `$V' is empty or contains special characters.
I'd love to hear the detailed explanation. My testing confirms this, and I'm now quoting my variables for all string tests, to avoid having -z and -n return the same result.
$ unset a
$ if [ -z $a ]; then echo unset; else echo set; fi
unset
$ if [ -n $a ]; then echo set; else echo unset; fi
set # highly unexpected!
$ unset a
$ if [ -z "$a" ]; then echo unset; else echo set; fi
unset
$ if [ -n "$a" ]; then echo set; else echo unset; fi
unset # much better
Well, I know that encapsulation of a variable helps you to work with something like:
${groups%example}
or syntax like that, where you want to do something with your variable before returning the value.
Now, if you see your code, all the magic is inside
${groups[#]}
the magic is in there because you can't write just: $groups[#]
You're putting your variable inside the {} because you want to use special characters [] and #. You can't name or call your variable just: # or something[] because these are reserved characters for other operations and names.
$var and ${var} are the same, if var is the name of the variable.
The braces are required when parameter is a positional parameter with more than one digit, or when parameter is followed by a character that is not to be interpreted as part of its name.
Thus, "$var" and "${var}" are the same.
However, $var and "$var" are different.
Bash will do Word Splitting for $var but not for "$var".
The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.
Note: Word splitting won't be performed in variable assignment:
https://www.gnu.org/software/bash/manual/html_node/Shell-Parameters.html
A variable may be assigned to by a statement of the form
name=[value]
If value is not given, the variable is assigned the null string. All values undergo tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, and quote removal (see Shell Parameter Expansion). If the variable has its integer attribute set, then value is evaluated as an arithmetic expression even if the $((…)) expansion is not used (see Arithmetic Expansion). Word splitting and filename expansion are not performed.
In bash $# contains all the arguments used to call the script but I am looking for a solution to remove the first one
./wrapper.sh foo bar baz ...:
#!/bin/bash
# call `cmd` with bar baz ... (withouyt foo one)
I just want to call cmd bar baz ...
You can use shift to shift the argument array. For instance, the following code:
#!/bin/bash
echo $#
shift
echo $#
produces, when called with 1 2 3 prints 1 2 3 and then 2 3:
$ ./example.sh 1 2 3
1 2 3
2 3
shift removes arguments from $#.
shift [n]
Shift positional parameters.
Rename the positional parameters $N+1,$N+2 ... to $1,$2 ... If N is
not given, it is assumed to be 1.
Exit Status:
Returns success unless N is negative or greater than $#.
Environment-variable-expansion! Is a very portable solution.
Remove the first argument: with $#
${##"$1"}
Remove the first argument: with $*
${*#"$1"}
Remove the first and second argument: with $#
${##"$1$2"}
Both $# or $* will work because the result of expansion is a string.
links:
Remove a fixed prefix/suffix from a string in Bash
http://www.tldp.org/LDP/abs/html/abs-guide.html#ARGLIST
Variable expansion is portable because it is defined under gnu core-utils
Search for "Environment variable expansion" at this link:
https://www.gnu.org/software/coreutils/manual/html_node/
I want to check if an input parameter ends with ".c"? How do I check that? Here is what I got so far (Thanks for your help):
#!/bin/bash
for i in $#
do
if [$i ends with ".c"]
then
echo "YES"
fi
done
A classical case for case!
case $i in *.c) echo Yes;; esac
Yes, the syntax is arcane, but you get used to it quickly. Unlike various Bash and POSIX extensions, this is portable all the way back to the original Bourne shell.
$ [[ foo.c = *.c ]] ; echo $?
0
$ [[ foo.h = *.c ]] ; echo $?
1
for i in $#; do
if [ -z ${i##*.c} ]; then
echo "YES: $i"
fi
done
$ ./test.sh .c .c-and-more before.c-and-after foo.h foo.c barc foo.C
YES: .c
YES: foo.c
$
Explanation (thanks to jpaugh):
Iterate over command line arguments: for i in $#; do
Main trick is here: if [ -z ${i##*.c} ]; then. Here we check if length of string ${i##*.c} is zero. ${i##*.c} means: take $i value and remove substring by template "*.c". If result is empty string, then we have ".c" suffix.
Here if some additional info from man bash, section Parameter Expasion
${parameter#word}
${parameter##word}
Remove matching prefix pattern. The word is expanded to produce a pat‐
tern just as in pathname expansion. If the pattern matches the begin‐
ning of the value of parameter, then the result of the expansion is the
expanded value of parameter with the shortest matching pattern (the
``#'' case) or the longest matching pattern (the ``##'' case) deleted.
If parameter is # or *, the pattern removal operation is applied to
each positional parameter in turn, and the expansion is the resultant
list. If parameter is an array variable subscripted with # or *, the
pattern removal operation is applied to each member of the array in
turn, and the expansion is the resultant list.
This question already has answers here:
What is the difference between $# and $* in shell scripts?
(3 answers)
Closed 8 years ago.
It seems to me that they both store all the command-line arguments.
So is there a difference between the two?
The difference is subtle; "$*" creates one argument separated by the $IFS variable, while "$#" will expand into separate arguments. As an example, consider:
for i in "$#"; do echo "# '$i'"; done
for i in "$*"; do echo "* '$i'"; done
When run with multiple arguments:
./testvar foo bar baz 'long arg'
# 'foo'
# 'bar'
# 'baz'
# 'long arg'
* 'foo bar baz long arg'
For more details:
http://www.gnu.org/software/bash/manual/bashref.html#Special-Parameters
$*
Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, it expands to a single word with the value of each parameter separated by the first character of the IFS special variable. That is, "$*" is equivalent to "$1c$2c...", where c is the first character of the value of the IFS variable. If IFS is unset, the parameters are separated by spaces. If IFS is null, the parameters are joined without intervening separators.
$#
Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. That is, "$#" is equivalent to "$1" "$2" .... If the double-quoted expansion occurs within a word, the expansion of the first parameter is joined with the beginning part of the original word, and the expansion of the last parameter is joined with the last part of the original word. When there are no positional parameters, "$#" and $# expand to nothing (i.e., they are removed).
A key difference from my POV is that "$#" preserves the original number
of arguments. It's the only form that does.
For example, if file my_script contains:
#!/bin/bash
main()
{
echo 'MAIN sees ' $# ' args'
}
main $*
main $#
main "$*"
main "$#"
### end ###
and I run it like this:
my_script 'a b c' d e
I will get this output:
MAIN sees 5 args
MAIN sees 5 args
MAIN sees 1 args
MAIN sees 3 args