This question already has answers here:
Brace expansion with variable? [duplicate]
(6 answers)
Closed 4 years ago.
Consider the following script:
#! /bin/bash -e
echo {foo,bar}
EX={foo,bar}
echo ${EX}
The output of this script is:
foo bar
{foo,bar}
I would like the the echo command to perform brace expansion on ${EX}. Thus, I would like to see an output of
foo bar
foo bar
I want to create a script where the user can supply a path with curly brackets where every expanded version of it is copied.
Something like this:
#! /bin/bash -e
$SOURCES=$1
$TARGET=$2
cp -r ${SOURCES} ${TARGET}
How can I achieve this?
This is a way:
ex=({foo,bar,baz})
echo ${ex[#]}
foo bar baz
See man bash:
The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion.
As you see, variable expansion happens later than brace expansion.
Fortunately, you don't need it at all: let the user specify the braced paths, let the shell expand them. You can then just
mv "$#"
If you need to separate the arguments, use an array and parameter expansion:
sources=("${#:1:$#-1}")
target=${#: -1}
mv "${sources[#]}" "$target"
Brace expansion does not work the way you are attempting to use it. Brace expansion is basically used to generate lists to be applied within the context of the present command. You have two primary modes where brace expansion is used directly (and many more where brace expansion is used a part of another operator.) The two direct uses are to expand a list of items within a comma-separated pair of braces. e.g.
$ touch file_{a,b,c,d}.txt
After executing the command, brace expansion creates all four files with properly formatted file names in the present directory:
$ ls -1 file*.txt
file_a.txt
file_b.txt
file_c.txt
file_d.txt
You may also use brace-expansion in a similar manner to generate lists for loop iteration (or wherever a generated system/range of numbers in needed). The syntax for using brace expansion here is similar, but with .. delimiters within the braces (instead of ',' separation). The syntax is {begin..end..increment} (whereincrement can be both positive and negative) e.g.
$ for i in {20..-20..-4}; do echo $i; done)
20
16
12
8
4
0
-4
-8
-12
-16
-20
(note: using variables for begin, end or increment is not allowed without some horrible eval trickery -- avoid it.).
Related
I seem to be able to create environment variables that execute commands; like this:
$ cat ./src
FOO="echo"
$ . ./src
$ echo $FOO
echo
$ $FOO hello
hello
$
Is there a way I can modify that environment variable so that it prefixes the setting of another environment variable before the command? I.e. is there a way to work around the following problem?
$ cat ./src
FOO="MY_DIR=/tmp echo"
$ . ./src
$ echo $FOO
MY_DIR=/tmp echo
$ $FOO hello
-bash: MY_DIR=/tmp: No such file or directory
$
I.e. what I'd like to happen is to have an environment variable that does the equivalent of the following manually typed in the shell:
$ MY_DIR=/tmp echo hello
hello
$
...similar to how sans envvar-prefix, $FOO effectively had the same effect as typing echo at the shell.
/tmp/ exists of course, btw:
$ ls -ld /tmp/
drwxrwxrwt. 25 root root 500 May 19 11:35 /tmp/
$
Update:
I have a constraint that "FOO" must be invoked like $FOO hello and not FOO hello. So unfortunately a function like in #John Kugelman's (current) answer can't be a solution, even if it's more proper.
It's best to put data into variables, code into functions. Functions are more natural, expressive, and flexible than variables holding code. They look just like any other command but can take arbitrary actions, including but not limited to prepending commands and variable assignments.
foo() {
MY_DIR=/tmp echo "$#"
}
foo hello
Here "$#" is a placeholder for the arguments passed to foo().
I have a constraint that "FOO" must be invoked like $FOO hello and not FOO hello.
That constraint is impossible, I'm afraid.
I am curious about the mechanics of what's going on here: i.e. why can you make an environment variable that's sort of "aliased" to a command (I know true aliasing is something else), but that mechanism doesn't accommodate the seemingly small change to prefix "stuff" to the command?
Bash expands commands in several passes in a fixed, prescribed order. Very early on it splits the command into words and then marks the variable assignments with invisible flags. It expands $variable references in a later pass. It doesn't look at the results to see if they look like additional variable expansions. The equal signs are effectively ignored.
If you want to know the nitty gritty details, open up the Bash man page. It's incredibly long and the details are scattered throughout. Let me pull out the key sections and some choice quotes to help you digest it:
Shell Grammar, Simple Commands
A simple command is a sequence of optional variable assignments followed by blank-separated words and redirections, and terminated by a control operator.
Simple Command Expansion
When a simple command is executed, the shell performs the following expansions, assignments, and redirections, from left to right.
The words that the parser has marked as variable assignments (those preceding the command name) and redirections are saved for later processing.
The words that are not variable assignments or redirections are expanded. If any words remain after expansion, the first word is taken to be the name of the command and the remaining words are the arguments.
...
If no command name results, the variable assignments affect the current shell environment. Otherwise, the variables are added to the environment of the executed command and do not affect the current shell environment.
Expansion
Expansion is performed on the command line after it has been split into words. There are seven kinds of expansion performed: brace expansion, tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, word splitting, and pathname expansion.
The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion.
Expansion, Parameter Expansion
The $ character introduces parameter expansion, command substitution, or arithmetic expansion.
Assignments are marked in step 1 and variables (AKA parameters) are expanded in step 4.
The only things that happen after variable expansion are:
Word splitting. A variable can expand to multiple words if it contains whitespace. (Or to be more precise, if it contains any of the characters in the inter-field separator variable $IFS.)
Pathname expansion. Also known as globbing, or wildcards. If a variable contains *, ?, or [ they'll be expanded to the names of matching files, if there are any.
Quote removal. This pass happens after variable expansion, but it specifically does not apply to the results of any previous expansion step. So quotes the user typed are removed, but quotes that were the results of a substitution are retained.
Neither word splitting nor pathname expansion are what you need, so that's why it's not possible to store an assignment in a variable.
This question already has answers here:
Brace expansion with variable? [duplicate]
(6 answers)
Closed 4 years ago.
Consider the following script:
#! /bin/bash -e
echo {foo,bar}
EX={foo,bar}
echo ${EX}
The output of this script is:
foo bar
{foo,bar}
I would like the the echo command to perform brace expansion on ${EX}. Thus, I would like to see an output of
foo bar
foo bar
I want to create a script where the user can supply a path with curly brackets where every expanded version of it is copied.
Something like this:
#! /bin/bash -e
$SOURCES=$1
$TARGET=$2
cp -r ${SOURCES} ${TARGET}
How can I achieve this?
This is a way:
ex=({foo,bar,baz})
echo ${ex[#]}
foo bar baz
See man bash:
The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion.
As you see, variable expansion happens later than brace expansion.
Fortunately, you don't need it at all: let the user specify the braced paths, let the shell expand them. You can then just
mv "$#"
If you need to separate the arguments, use an array and parameter expansion:
sources=("${#:1:$#-1}")
target=${#: -1}
mv "${sources[#]}" "$target"
Brace expansion does not work the way you are attempting to use it. Brace expansion is basically used to generate lists to be applied within the context of the present command. You have two primary modes where brace expansion is used directly (and many more where brace expansion is used a part of another operator.) The two direct uses are to expand a list of items within a comma-separated pair of braces. e.g.
$ touch file_{a,b,c,d}.txt
After executing the command, brace expansion creates all four files with properly formatted file names in the present directory:
$ ls -1 file*.txt
file_a.txt
file_b.txt
file_c.txt
file_d.txt
You may also use brace-expansion in a similar manner to generate lists for loop iteration (or wherever a generated system/range of numbers in needed). The syntax for using brace expansion here is similar, but with .. delimiters within the braces (instead of ',' separation). The syntax is {begin..end..increment} (whereincrement can be both positive and negative) e.g.
$ for i in {20..-20..-4}; do echo $i; done)
20
16
12
8
4
0
-4
-8
-12
-16
-20
(note: using variables for begin, end or increment is not allowed without some horrible eval trickery -- avoid it.).
This question already has answers here:
"~/Desktop/test.txt: No such file or directory"
(2 answers)
Closed 4 years ago.
I have a very surprising problem while trying to execute a diff command inside a bash script.
Here is a working code illustrating the point:
#!/bin/bash
cd
mkdir foo bar
head -c 1024 /dev/urandom >foo/qux
head -c 1024 /dev/urandom >bar/qux
# works properly as expected
diff ~/{foo,bar}/qux
folder="~"
# this fails with the ~ inside a variable
diff $folder/{foo,bar}/qux
# cleaning the mess
rm -rf foo bar
So my question is:
Why ? :-)
Don't quote the ~ when assigning it to a variable. The ~ is only expanded by bash when you don't quote it.
~ is a feature of shell expansion.
Double quotes limit the expansion to only three features:
Command substitution: $(some command here) or `some command here`
Variable substitution: $VAR or ${VAR}
Arithmetic: $((2+2))
so when put inside double quotes, the ~ is not expanded
Tilde expansion only applies to unquoted tildes. The tilde must be expanded at the time you perform the assignment to folder, because tilde expansion is not applied to parameter expansions, only word splitting and pathname expansion.
folder=~ # ~ is expanded, and the result is assigned to folder
This question already has answers here:
Brace expansion with variable? [duplicate]
(6 answers)
Closed 4 years ago.
Consider the following script:
#! /bin/bash -e
echo {foo,bar}
EX={foo,bar}
echo ${EX}
The output of this script is:
foo bar
{foo,bar}
I would like the the echo command to perform brace expansion on ${EX}. Thus, I would like to see an output of
foo bar
foo bar
I want to create a script where the user can supply a path with curly brackets where every expanded version of it is copied.
Something like this:
#! /bin/bash -e
$SOURCES=$1
$TARGET=$2
cp -r ${SOURCES} ${TARGET}
How can I achieve this?
This is a way:
ex=({foo,bar,baz})
echo ${ex[#]}
foo bar baz
See man bash:
The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion.
As you see, variable expansion happens later than brace expansion.
Fortunately, you don't need it at all: let the user specify the braced paths, let the shell expand them. You can then just
mv "$#"
If you need to separate the arguments, use an array and parameter expansion:
sources=("${#:1:$#-1}")
target=${#: -1}
mv "${sources[#]}" "$target"
Brace expansion does not work the way you are attempting to use it. Brace expansion is basically used to generate lists to be applied within the context of the present command. You have two primary modes where brace expansion is used directly (and many more where brace expansion is used a part of another operator.) The two direct uses are to expand a list of items within a comma-separated pair of braces. e.g.
$ touch file_{a,b,c,d}.txt
After executing the command, brace expansion creates all four files with properly formatted file names in the present directory:
$ ls -1 file*.txt
file_a.txt
file_b.txt
file_c.txt
file_d.txt
You may also use brace-expansion in a similar manner to generate lists for loop iteration (or wherever a generated system/range of numbers in needed). The syntax for using brace expansion here is similar, but with .. delimiters within the braces (instead of ',' separation). The syntax is {begin..end..increment} (whereincrement can be both positive and negative) e.g.
$ for i in {20..-20..-4}; do echo $i; done)
20
16
12
8
4
0
-4
-8
-12
-16
-20
(note: using variables for begin, end or increment is not allowed without some horrible eval trickery -- avoid it.).
The shell has a great feature, where it'll preserve argument quoting across variable expansion when you use "$#", such that the script:
for f in "$#"; do echo "$f"; done
when invoked with arguments:
"with spaces" '$and $(metachars)'
will print, literally:
with spaces
$and $(metachars)
This isn't the normal behaviour of expansion of a quoted string, it seems to be a special case for "$#".
Is there any way to get this behaviour for other variables? In the specific case I'm interested in, I want to safely expand $SSH_ORIGINAL_COMMAND in a command= specifier in a restricted public key entry, without having to worry about spaces in arguments, metacharacters, etc.
"$SSH_ORIGINAL_COMMAND" expands like "$*" would, i.e. a naïve expansion that doesn't add any quoting around separate arguments.
Is the information required for "$#" style expansion simply not available to the shell in this case, by the time it gets the env var SSH_ORIGINAL_COMMAND? So I'd instead need to convince sshd to quote the arguments?
The answer to this question is making me wonder if it's possible at all.
You can get similar "quoted dollar-at" behavior for arbitrary arrays using "${YOUR_ARRAY_HERE[#]}" syntax for bash arrays. Of course, that's no complete answer, because you still have to break the string into multiple array elements according to the quotes.
One thought was to use bash -x, which renders expanded output, but only if you actually run the command; it doesn't work with -n, which prevents you from actually executing the commands in question. Likewise you could use eval or bash -c along with set -- to manage the quote removal, performing expansion on the outer shell and quote removal on the inner shell, but that would be extremely hard to bulletproof against executing arbitrary code.
As an end run, use xargs instead. xargs handles single and double quotes. This is a very imperfect solution, because xargs treats backslash-escaped characters very differently than bash does and fails entirely to handle semicolons and so forth, but if your input is relatively predictable it gets you most of the way there without forcing you to write a full shell parser.
SSH_ORIGINAL_COMMAND='foo "bar baz" $quux'
# Build out the parsed array.
# Bash 4 users may be able to do this with readarray or mapfile instead.
# You may also choose to null-terminate if newlines matter.
COMMAND_ARRAY=()
while read line; do
COMMAND_ARRAY+=("$line")
done < <(xargs -n 1 <<< "$SSH_ORIGINAL_COMMAND")
# Demonstrate working with the array.
N=0
for arg in "${COMMAND_ARRAY[#]}"; do
echo "COMMAND_ARRAY[$N]: $arg"
((N++))
done
Output:
COMMAND_ARRAY[0]: foo
COMMAND_ARRAY[1]: bar baz
COMMAND_ARRAY[2]: $quux