I am trying to write a bash script. I am not sure why in my script:
ls {*.xml,*.txt}
works okay, but
name="{*.xml,*.txt}"
ls $name
doesn't work. I get
ls: cannot access {*.xml,*.txt}: No such file or directory
The expression
ls {*.xml,*.txt}
results in Brace expansion and shell passes the expansion (if any) to ls as arguments. Setting shopt -s nullglob makes this expression evaluate to nothing when there are no matching files.
Double quoting the string suppresses the expansion and shell stores the literal contents in your variable name (not sure if that is what you wanted). When you invoke ls with $name as the argument, shell does the variable expansion but no brace expansion is done.
As #Cyrus has mentioned, eval ls $name will force brace expansion and you get the same result as that of ls {\*.xml,\*.txt}.
The reason your expansion doesn't work is that brace expansion is performed before variable expansion, see Shell expansions in the manual.
I'm not sure what it is you're trying to do, but if you want to store a list of file names, use an array:
files=( {*.txt,*.xml} ) # these two are the same
files=(*.txt *.xml)
ls -l "${files[#]}" # give them to a command
for file in "${files[#]}" ; do # or loop over them
dosomething "$file"
done
"${array[#]}" expands to all elements of the array, as separate words. (remember the quotes!)
Related
Say I have a folder called Foo located in /home/user/ (my /home/user also being represented by ~).
I want to have a variable
a="~/Foo" and then do
cd $a
I get
-bash: cd: ~/Foo: No such file or directory
However if I just do cd ~/Foo it works fine. Any clue on how to get this to work?
You can do (without quotes during variable assignment):
a=~/Foo
cd "$a"
But in this case the variable $a will not store ~/Foo but the expanded form /home/user/Foo. Or you could use eval:
a="~/Foo"
eval cd "$a"
You can use $HOME instead of the tilde (the tilde is expanded by the shell to the contents of $HOME).
Example:
dir="$HOME/Foo";
cd "$dir";
Although this question is merely asking for a workaround, this is listed as the duplicate of many questions that are asking why this happens, so I think it's worth giving an explanation. According to https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06:
The order of word expansion shall be as follows:
Tilde expansion, parameter expansion, command substitution, and arithmetic expansion shall be performed, beginning to end.
When the shell evaluates the string cd $a, it first performs tilde expansion (which is a no-op, since $a does not contain a tilde), then it expands $a to the string ~/Foo, which is the string that is finally passed as the argument to cd.
A much more robust solution would be to use something like sed or even better, bash parameter expansion:
somedir="~/Foo/test~/ing";
cd "${somedir/#\~/$HOME}"
or if you must use sed,
cd $(echo "$somedir" | sed "s#^~#$HOME#")
If you use double quotes the ~ will be kept as that character in $a.
cd $a will not expand the ~ since variable values are not expanded by the shell.
The solution is:
eval "cd $a"
Say I have a folder called Foo located in /home/user/ (my /home/user also being represented by ~).
I want to have a variable
a="~/Foo" and then do
cd $a
I get
-bash: cd: ~/Foo: No such file or directory
However if I just do cd ~/Foo it works fine. Any clue on how to get this to work?
You can do (without quotes during variable assignment):
a=~/Foo
cd "$a"
But in this case the variable $a will not store ~/Foo but the expanded form /home/user/Foo. Or you could use eval:
a="~/Foo"
eval cd "$a"
You can use $HOME instead of the tilde (the tilde is expanded by the shell to the contents of $HOME).
Example:
dir="$HOME/Foo";
cd "$dir";
Although this question is merely asking for a workaround, this is listed as the duplicate of many questions that are asking why this happens, so I think it's worth giving an explanation. According to https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06:
The order of word expansion shall be as follows:
Tilde expansion, parameter expansion, command substitution, and arithmetic expansion shall be performed, beginning to end.
When the shell evaluates the string cd $a, it first performs tilde expansion (which is a no-op, since $a does not contain a tilde), then it expands $a to the string ~/Foo, which is the string that is finally passed as the argument to cd.
A much more robust solution would be to use something like sed or even better, bash parameter expansion:
somedir="~/Foo/test~/ing";
cd "${somedir/#\~/$HOME}"
or if you must use sed,
cd $(echo "$somedir" | sed "s#^~#$HOME#")
If you use double quotes the ~ will be kept as that character in $a.
cd $a will not expand the ~ since variable values are not expanded by the shell.
The solution is:
eval "cd $a"
My question is very similar to How to copy multiple files from a different directory using cp?
I don't want to use an explicit loop. Here is what I do:
$ FILES_TOOLS="fastboot,fastboot-HW.sh"
$ cp $HOME/tools/{$FILES_TOOLS} $TOP_DIR/removeme
cp: cannot stat `/home/johndoe/tools/{fastboot,fastboot-HW.sh}': No such file or directory
The files are present and destination is valid, because:
$ cp $HOME/tools/{fastboot,fastboot-HW.sh} $TOP_DIR/removeme
$ echo $?
0
I tried to remove the double quote from FILES_TOOLS, no luck.
I tried to quote and double quote {...}, no luck
I tried to backslash the brackets, no luck
I guess this is a problem of when the shell expansion actually occurs.
This answer is limited to the bash.
Prepend an echo to see what your cp command turns into:
echo cp $HOME/tools/{$FILES_TOOLS} $TOP_DIR/removeme
You have to insert an eval inside a sub-shell to make it work:
cp $( eval echo $HOME/tools/{$FILES_TOOLS} ) $TOP_DIR/removeme
I guess this is a problem of when the shell expansion actually occurs.
Yes. Different shells have different rules about brace expansion in relation to variable expansion. Your way works in ksh, but not in zsh or bash. {1..$n} works in ksh and zsh but not in bash. In bash, variable expansion always happens after brace expansion.
The closest you'll get to this in bash is with eval.
As long as the contents of the braces are literals, you can use brace expansion to populate an array with the full path names of the files to copy, then expand the contents of the array in your cp command.
$ FILES_TOOLS=( $HOME/tools/{fastboot,fastboot-HW.sh} )
$ cp "${FILES_TOOLS[#]}" $TOP_DIR/removeme
Update: I realized you might have a reason for having the base names alone in the variable. Here's another array-based solution that lets you prefix each element of the array with a path, again without an explicit loop:
$ FILES_TOOLS=( fastboot fastboot-HW.sh )
$ cp "${FILES_TOOLS[#]/#/$HOME/tools/}" $TOP_DIR/removeme
In this case, you use the pattern substitution operator to replace the empty string at the beginning of each array element with the directory name.
I have some files that look like:
/path/with spaces/{a,b,c}/*.gz
And I need all files matching the glob under a subset of the a,b,c dirs to end up as arguments to a single command:
mycmd '/path/with spaces/a/1.gz' '/path/with spaces/a/2.gz' '/path/with spaces/c/3.gz' ...
The directories I care about come in as command line params and I have them in an array:
dirs=( "$#" )
And I want to do something like:
IFS=,
mycmd "/path/with spaces/{${dirs[*]}}/"*.gz
but this doesn't work, because bash expands braces before variables. I have tried tricks with echo and ls and even eval (*shudder*) but it's tough to make them work with spaces in filenames. find doesn't seem to be much help because it doesn't do braces. I can get a separate glob for each dir in an array with:
dirs=( "${dirs[#]/#//path/with spaces/}" )
dirs=( "${dirs[#]/%//*.gz}" )
but then bash quotes the wildcards on expansion.
So: is there an elegant way to get all the files matching a variable brace and glob pattern, properly handling spaces, or am I stuck doing for loops? I'm using Bash 3 if that makes a difference.
To perform brace expansion and globbing on a path with spaces, you can quote the portions of the path that contain spaces, e.g.
mycmd '/path/with spaces/'{a,b,c}/*.gz
Doing brace expansion using a list of values from a variable is a little tricky since brace expansion is done before any other expansion. I don't see any way but to use the dreaded eval.
eval mycmd "'/path/with spaces/'{a,b,c}/*.gz"
P.S. In such a case however, I would personally opt for a loop to build the argument list rather than the approach shown above. While more verbose, a loop will be a lot easier to read for the uninitiated and will avoid the need to use eval (especially when the expansion candidates are derived from user input!).
Proof of concept:
Using a dummy command (x.sh) which prints out the number of arguments and prints out each argument:
[me#home]$ shopt -s nullglob # handle case where globbing returns no match
[me#home]$ ./x.sh 'path with space'/{a,b}/*.txt
Number of arguments = 3
- path with space/a/1.txt
- path with space/b/2.txt
- path with space/b/3.txt
[me#home]:~/temp$ dirs="a,b"
[me#home]k:~/temp$ eval ./x.sh "'path with space'/{$dirs}/*.txt"
Number of arguments = 3
- path with space/a/1.txt
- path with space/b/2.txt
- path with space/b/3.txt
Okay, so here is one using bash for the "braces" and find for the globs:
find "${dirs[#]/#//path/with spaces/}" -name '*.gz' -print0 | xargs -0 mycmd
Useful with this if you need the results in an array.
Here's one for the GNU Parallel fans:
parallel -Xj1 mycmd {}/*.gz ::: "${dirs[#]/#//path/with spaces/}"
I am trying achieve the same effect as typing
mv ./images/*.{pdf,eps,jpg,svg} ./images/junk/
at the command line, from inside a bash script. I have:
MYDIR="./images"
OTHERDIR="./images/junk"
SUFFIXES='{pdf,eps,jpg,svg}'
mv "$MYDIR/"*.$SUFFIXES "$OTHERDIR/"
which, when run, gives the not unexpected error:
mv: rename ./images/*.{pdf,eps,jpg,svg} to ./images/junk/*.{pdf,eps,jpg,svg}:
No such file or directory
What is the correct way to quote all this so that mv will actually do the desired expansion? (Yes, there are plenty of files that match the pattern in ./images/.)
A deleted answer was on the right track. A slight modification to your attempt:
shopt -s extglob
MYDIR="./images"
OTHERDIR="./images/junk"
SUFFIXES='#(pdf|eps|jpg|svg)'
mv "$MYDIR/"*.$SUFFIXES "$OTHERDIR/"
Brace expansion is done before variable expansion, but variable expansion is done before pathname expansion. So the braces are still braces when the variable is expanded in your original, but when the variable instead contains pathname elements, they have already been expanded when the pathname expansion gets done.
You'll need to eval that line in order for it to work, like so:
MYDIR="./images"
OTHERDIR="./images/junk"
SUFFIXES='{pdf,eps,jpg,svg}'
eval "mv \"$MYDIR\"/*.$SUFFIXES \"$OTHERDIR/\""
Now, this has problems, in particular, if you don't trust $SUFFIXES, it might contain an injection attack, but for this simple case it should be alright.
If you are open to other solutions, you might want to experiment with find and xargs.
You can write a function:
function expand { for arg in "$#"; do [[ -f $arg ]] && echo $arg; done }
then call it with what you want to expand:
expand "$MYDIR/"*.$SUFFIXES
You can also make it a script expand.sh if you like.