I want to do this:
for i in {1.."$((2**3))"}; do echo "$i"; done
But that would output {1..8}, which I want to execute, not output.
How to?
You could to use seq instead of range braces:
for i in $(seq 1 $((2**3))); do echo "$i"; done
You can't do in like that in bash, brace expansion happens before variable does.
A c-style for loop can be an alternative.
for ((i = 1; i <= 2**3; i++)); do printf '%d ' "$i"; done
... Or if you really want to do the brace expansion use eval which is not advised to use but it is the only way...
eval echo {1..$((2**3))}
See the local bash manual for the order of expansion PAGER='less +/^EXPANSION' man bash and the online manual (thanks to #Freddy) https://www.gnu.org/software/bash/manual/html_node/Shell-Expansions.html
See eval in the local bash manual PAGER='less +/^[[:blank:]]*eval\ ' man bash
Related
Seems that the recommended way of doing indirect variable setting in bash is to use eval:
var=x; val=foo
eval $var=$val
echo $x # --> foo
The problem is the usual one with eval:
var=x; val=1$'\n'pwd
eval $var=$val # bad output here
(and since it is recommended in many places, I wonder just how many scripts are vulnerable because of this...)
In any case, the obvious solution of using (escaped) quotes doesn't really work:
var=x; val=1\"$'\n'pwd\"
eval $var=\"$val\" # fail with the above
The thing is that bash has indirect variable reference baked in (with ${!foo}), but I don't see any such way to do indirect assignment -- is there any sane way to do this?
For the record, I did find a solution, but this is not something that I'd consider "sane"...:
eval "$var='"${val//\'/\'\"\'\"\'}"'"
A slightly better way, avoiding the possible security implications of using eval, is
declare "$var=$val"
Note that declare is a synonym for typeset in bash. The typeset command is more widely supported (ksh and zsh also use it):
typeset "$var=$val"
In modern versions of bash, one should use a nameref.
declare -n var=x
x=$val
It's safer than eval, but still not perfect.
Bash has an extension to printf that saves its result into a variable:
printf -v "${VARNAME}" '%s' "${VALUE}"
This prevents all possible escaping issues.
If you use an invalid identifier for $VARNAME, the command will fail and return status code 2:
$ printf -v ';;;' '%s' foobar; echo $?
bash: printf: `;;;': not a valid identifier
2
eval "$var=\$val"
The argument to eval should always be a single string enclosed in either single or double quotes. All code that deviates from this pattern has some unintended behavior in edge cases, such as file names with special characters.
When the argument to eval is expanded by the shell, the $var is replaced with the variable name, and the \$ is replaced with a simple dollar. The string that is evaluated therefore becomes:
varname=$value
This is exactly what you want.
Generally, all expressions of the form $varname should be enclosed in double quotes, to prevent accidental expansion of filename patterns like *.c.
There are only two places where the quotes may be omitted since they are defined to not expand pathnames and split fields: variable assignments and case. POSIX 2018 says:
Each variable assignment shall be expanded for tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal prior to assigning the value.
This list of expansions is missing the parameter expansion and the field splitting. Sure, that's hard to see from reading this sentence alone, but that's the official definition.
Since this is a variable assignment, the quotes are not needed here. They don't hurt, though, so you could also write the original code as:
eval "$var=\"the value is \$val\""
Note that the second dollar is escaped using a backslash, to prevent it from being expanded in the first run. What happens is:
eval "$var=\"the value is \$val\""
The argument to the command eval is sent through parameter expansion and unescaping, resulting in:
varname="the value is $val"
This string is then evaluated as a variable assignment, which assigns the following value to the variable varname:
the value is value
The main point is that the recommended way to do this is:
eval "$var=\$val"
with the RHS done indirectly too. Since eval is used in the same
environment, it will have $val bound, so deferring it works, and since
now it's just a variable. Since the $val variable has a known name,
there are no issues with quoting, and it could have even been written as:
eval $var=\$val
But since it's better to always add quotes, the former is better, or
even this:
eval "$var=\"\$val\""
A better alternative in bash that was mentioned for the whole thing that
avoids eval completely (and is not as subtle as declare etc):
printf -v "$var" "%s" "$val"
Though this is not a direct answer what I originally asked...
Newer versions of bash support something called "parameter transformation", documented in a section of the same name in bash(1).
"${value#Q}" expands to a shell-quoted version of "${value}" that you can re-use as input.
Which means the following is a safe solution:
eval="${varname}=${value#Q}"
Just for completeness I also want to suggest the possible use of the bash built in read. I've also made corrections regarding -d'' based on socowi's comments.
But much care needs to be exercised when using read to ensure the input is sanitized (-d'' reads until null termination and printf "...\0" terminates the value with a null), and that read itself is executed in the main shell where the variable is needed and not a sub-shell (hence the < <( ... ) syntax).
var=x; val=foo0shouldnotterminateearly
read -d'' -r "$var" < <(printf "$val\0")
echo $x # --> foo0shouldnotterminateearly
echo ${!var} # --> foo0shouldnotterminateearly
I tested this with \n \t \r spaces and 0, etc it worked as expected on my version of bash.
The -r will avoid escaping \, so if you had the characters "\" and "n" in your value and not an actual newline, x will contain the two characters "\" and "n" also.
This method may not be aesthetically as pleasing as the eval or printf solution, and would be more useful if the value is coming in from a file or other input file descriptor
read -d'' -r "$var" < <( cat $file )
And here are some alternative suggestions for the < <() syntax
read -d'' -r "$var" <<< "$val"$'\0'
read -d'' -r "$var" < <(printf "$val") #Apparently I didn't even need the \0, the printf process ending was enough to trigger the read to finish.
read -d'' -r "$var" <<< $(printf "$val")
read -d'' -r "$var" <<< "$val"
read -d'' -r "$var" < <(printf "$val")
Yet another way to accomplish this, without eval, is to use "read":
INDIRECT=foo
read -d '' -r "${INDIRECT}" <<<"$(( 2 * 2 ))"
echo "${foo}" # outputs "4"
Seems that the recommended way of doing indirect variable setting in bash is to use eval:
var=x; val=foo
eval $var=$val
echo $x # --> foo
The problem is the usual one with eval:
var=x; val=1$'\n'pwd
eval $var=$val # bad output here
(and since it is recommended in many places, I wonder just how many scripts are vulnerable because of this...)
In any case, the obvious solution of using (escaped) quotes doesn't really work:
var=x; val=1\"$'\n'pwd\"
eval $var=\"$val\" # fail with the above
The thing is that bash has indirect variable reference baked in (with ${!foo}), but I don't see any such way to do indirect assignment -- is there any sane way to do this?
For the record, I did find a solution, but this is not something that I'd consider "sane"...:
eval "$var='"${val//\'/\'\"\'\"\'}"'"
A slightly better way, avoiding the possible security implications of using eval, is
declare "$var=$val"
Note that declare is a synonym for typeset in bash. The typeset command is more widely supported (ksh and zsh also use it):
typeset "$var=$val"
In modern versions of bash, one should use a nameref.
declare -n var=x
x=$val
It's safer than eval, but still not perfect.
Bash has an extension to printf that saves its result into a variable:
printf -v "${VARNAME}" '%s' "${VALUE}"
This prevents all possible escaping issues.
If you use an invalid identifier for $VARNAME, the command will fail and return status code 2:
$ printf -v ';;;' '%s' foobar; echo $?
bash: printf: `;;;': not a valid identifier
2
eval "$var=\$val"
The argument to eval should always be a single string enclosed in either single or double quotes. All code that deviates from this pattern has some unintended behavior in edge cases, such as file names with special characters.
When the argument to eval is expanded by the shell, the $var is replaced with the variable name, and the \$ is replaced with a simple dollar. The string that is evaluated therefore becomes:
varname=$value
This is exactly what you want.
Generally, all expressions of the form $varname should be enclosed in double quotes, to prevent accidental expansion of filename patterns like *.c.
There are only two places where the quotes may be omitted since they are defined to not expand pathnames and split fields: variable assignments and case. POSIX 2018 says:
Each variable assignment shall be expanded for tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal prior to assigning the value.
This list of expansions is missing the parameter expansion and the field splitting. Sure, that's hard to see from reading this sentence alone, but that's the official definition.
Since this is a variable assignment, the quotes are not needed here. They don't hurt, though, so you could also write the original code as:
eval "$var=\"the value is \$val\""
Note that the second dollar is escaped using a backslash, to prevent it from being expanded in the first run. What happens is:
eval "$var=\"the value is \$val\""
The argument to the command eval is sent through parameter expansion and unescaping, resulting in:
varname="the value is $val"
This string is then evaluated as a variable assignment, which assigns the following value to the variable varname:
the value is value
The main point is that the recommended way to do this is:
eval "$var=\$val"
with the RHS done indirectly too. Since eval is used in the same
environment, it will have $val bound, so deferring it works, and since
now it's just a variable. Since the $val variable has a known name,
there are no issues with quoting, and it could have even been written as:
eval $var=\$val
But since it's better to always add quotes, the former is better, or
even this:
eval "$var=\"\$val\""
A better alternative in bash that was mentioned for the whole thing that
avoids eval completely (and is not as subtle as declare etc):
printf -v "$var" "%s" "$val"
Though this is not a direct answer what I originally asked...
Newer versions of bash support something called "parameter transformation", documented in a section of the same name in bash(1).
"${value#Q}" expands to a shell-quoted version of "${value}" that you can re-use as input.
Which means the following is a safe solution:
eval="${varname}=${value#Q}"
Just for completeness I also want to suggest the possible use of the bash built in read. I've also made corrections regarding -d'' based on socowi's comments.
But much care needs to be exercised when using read to ensure the input is sanitized (-d'' reads until null termination and printf "...\0" terminates the value with a null), and that read itself is executed in the main shell where the variable is needed and not a sub-shell (hence the < <( ... ) syntax).
var=x; val=foo0shouldnotterminateearly
read -d'' -r "$var" < <(printf "$val\0")
echo $x # --> foo0shouldnotterminateearly
echo ${!var} # --> foo0shouldnotterminateearly
I tested this with \n \t \r spaces and 0, etc it worked as expected on my version of bash.
The -r will avoid escaping \, so if you had the characters "\" and "n" in your value and not an actual newline, x will contain the two characters "\" and "n" also.
This method may not be aesthetically as pleasing as the eval or printf solution, and would be more useful if the value is coming in from a file or other input file descriptor
read -d'' -r "$var" < <( cat $file )
And here are some alternative suggestions for the < <() syntax
read -d'' -r "$var" <<< "$val"$'\0'
read -d'' -r "$var" < <(printf "$val") #Apparently I didn't even need the \0, the printf process ending was enough to trigger the read to finish.
read -d'' -r "$var" <<< $(printf "$val")
read -d'' -r "$var" <<< "$val"
read -d'' -r "$var" < <(printf "$val")
Yet another way to accomplish this, without eval, is to use "read":
INDIRECT=foo
read -d '' -r "${INDIRECT}" <<<"$(( 2 * 2 ))"
echo "${foo}" # outputs "4"
This question already has answers here:
How do I iterate over a range of numbers defined by variables in Bash?
(20 answers)
Closed 8 years ago.
If I do this, I get the result as expected.
for i in {125..129}; do echo $i; done
125
126
127
128
129
But when I do this? I get something weired.
for i in {$((1+(25-1)*500))..$((25*500))}; do echo $i; done
{12001..12500}
I wish to pass a variable inside the loop variables like $((1+($j-1)*500))
Bash's brace expansion has limitations. What you want is seq:
for i in $( seq $((1+(25-1)*500)) $((25*500)) ); do echo $i; done
The above will loop over all numbers from 12001 to 12500.
Discussion
seq is similar to bash's braces:
$ echo {2..4}
2 3 4
$ echo $(seq 2 4)
2 3 4
The key advantage of seq is that its arguments can include not just arithmetic expressions, as shown above, but also shell variables:
$ x=4; echo $(seq $((x-2)) $x)
2 3 4
By contrast, the brace notation will accept neither.
seq is a GNU utility and is available on all linux systems as well as recent versions of OSX. Older BSD systems can use a similar utility called jot.
Brace expansion is the very first expansion that occurs, before parameter, variable, and arithmetic expansion. Brace expansion with .. only occurs if the values before and after the .. are integers or single characters. Since the arithmetic expansion in your example has not yet occurred, they aren't single characters or integers, so no brace expansion occurs.
You can force reexpansion to occur after arithmetic expansion with eval:
for i in $(eval echo {$((1+(25-1)*500))..$((25*500))}); do echo $i;
You query is very similar to :
shell script "for" loop syntax
brace expansion, {x..y} is performed before other expansions, so you cannot use that for variable length sequences.
Instead try
for i in seq $((1+(25-1)*500)) $((25*500)); do echo $i; done
It is just echoing the text which is exactly as it says:
{12001..12500}
That is "{"+12001+"..."+12500+"}"
Don't do this with a for loop. The {..} notation is not that flexible:
i=$((1+(25-1)*500)); while test $i -le $((25*500)); do echo $((i++)); done
Try this
for (( i= $((1+(25-1)*500)); i<=$((25*500)); i++ )); do echo $i; done
or this
for i in $(seq $(( 1+(25-1)*500 )) $(( 25*500 )) ); do echo $i; done
i'm doing a script to download one file in multiple parts, i'm in the part of joining that files. I can join the parts outside of the script with
cat name.part{0..4} > name.ext
But, if i use this in the script
cat $filename.part{0..$N} > $filename
i get:
cat: Bitcoin-960x623.jpg.part{0..5}: No such file or
{0..4} is a brace expansion; $N is a variable expansion. Your expression doesn't work as you expect because brace expansion happens before variable expansion:
The order of expansions is: brace expansion, tilde expansion, parameter, variable, and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and filename expansion.
http://www.gnu.org/software/bash/manual/bashref.html#Shell-Expansions
You can use the seq command instead of a brace expansion to generate the required filenames up to arbitrary N:
$ filename=myfile
$ N=4
$ cat $(seq -f "$filename.part%g" 0 $N) > name.ext
$
In the unlikely event seq -f is unavailable, you can use a for loop one-liner instead:
$ cat $(for ((i=0; i<N; i++)); do echo "$filename.part$i"; done) > name.ext
$
I solved with this.
for (( k=0 ; k<=$N ; k++ ))
do
cat $filename.part$k >> $filename
done
You cannot use variables inside curly braces in shell. eval can be used here:
eval "$filename.part{0..$N} > $filename"
Assertion: I'm going to argue that this answer is better. (Something I don't do often or lightly.) My rationale is that all of the other answers here at the time of this edit just expand to a range of integers, whether the files matching those integers exist or not. By using glob expansions you guarantee you will not encounter No such file or directory from cat. In all other cases with brace expansion, eval, seq and a for loop you'd have to explicitly check.
If there are fewer than 10 total filenames:
cat "$filename".part[0-9]
will expand as a glob to however many names there are. This is a fast and portable solution, but obviously limited. If there are more than 10 then that pattern will only match the ones with one digit, so you have to get a bit wilier and use extglob:
shopt -s extglob
cat "$filename".part+([0-9])
should do it, but this will only work for the Bourne-Again SHell. (That said, several other modern shells have expansion capabilities similar to extglob.)
The following bash script works fine to print numbers from 1 to 10:
for i in {1..10}
do
echo "$i"
done
But if I want to make the upper limit a variable, then this script does not works.
i=10
for j in {1..$i}
do
echo "$j"
done
Can anyone suggest please how to make the second script work?
Brace expansion happens before any other expansion.
You can say:
for j in $(seq 1 $i); do echo "$j"; done
Quoting from the link above:
Brace expansion is performed before any other expansions, and any
characters special to other expansions are preserved in the result. It
is strictly textual. Bash does not apply any syntactic interpretation
to the context of the expansion or the text between the braces. To
avoid conflicts with parameter expansion, the string ‘${’ is not
considered eligible for brace expansion.
You cannot use variables in {..} directive of Bash. Use BASH arithmetic operator ((...)) like this:
i=10
for ((j=1; j<=i; j++)); do
echo "$j"
done