In bash, if you do this:
mkdir /tmp/empty
array=(/tmp/empty/*)
you find that array now has one element, "/tmp/empty/*", not zero as you'd like. Thankfully, this can be avoided by turning on the nullglob shell option using shopt -s nullglob
But nullglob is global, and when editing an existing shell script, may break things (e.g., did someone check the exit code of ls foo* to check if there are files named starting with "foo"?). So, ideally, I'd like to turn it on only for a small scope—ideally, one filename expansion. You can turn it off again using shopt -u nullglob But of course only if it was disabled before:
old_nullglob=$(shopt -p | grep 'nullglob$')
shopt -s nullglob
array=(/tmp/empty/*)
eval "$old_nullglob"
unset -v old_nullglob
makes me think there must be a better way. The obvious "put it in a subshell" doesn't work as of course the variable assignment dies with the subshell. Other than waiting for the Austin group to import ksh93 syntax, is there?
Unset it when done:
shopt -u nullglob
And properly (i.e. storing the previous state):
shopt -u | grep -q nullglob && changed=true && shopt -s nullglob
... do whatever you want ...
[ $changed ] && shopt -u nullglob; unset changed
With mapfile in Bash 4, you can load an array from a subshell with something like: mapfile array < <(shopt -s nullglob; for f in ./*; do echo "$f"; done). Full example:
$ shopt nullglob
nullglob off
$ find
.
./bar baz
./qux quux
$ mapfile array < <(shopt -s nullglob; for f in ./*; do echo "$f"; done)
$ shopt nullglob
nullglob off
$ echo ${#array[#]}
2
$ echo ${array[0]}
bar baz
$ echo ${array[1]}
qux quux
$ rm *
$ mapfile array < <(shopt -s nullglob; for f in ./*; do echo "$f"; done)
$ echo ${#array[#]}
0
Be sure to glob with ./* instead of a bare * when using echo to print the file name
Doesn't work with newline characters in the filename :( as pointed out by derobert
If you need to handle newlines in the filename, you will have to do the much more verbose:
array=()
while read -r -d $'\0'; do
array+=("$REPLY")
done < <(shopt -s nullglob; for f in ./*; do printf "$f\0"; done)
But by this point, it may be simpler to follow the advice of one of the other answers.
This is just a tiny bit better than your original suggestion:
local nullglob=$(shopt -p nullglob) ; shopt -s nullglob
... do whatever you want ...
$nullglob ; unset nullglob
This may be close to what you want; as is, it requires executing a command to expand the glob.
$ ls
file1 file2
$ array=( $(shopt -s nullglob; ls foo*) )
$ ls foo*
ls: foo*: No such file or directory
$ echo ${array[*]}
file1 file2
Instead of setting array in the subshell, we create a subshell using $() whose output is captured by array.
This is the simplest solution I've found:
For example, to expand the literal **/*.mp3 into a glob for only a particular variable, you can use
VAR=**/*.mp3(N)
Source: https://unix.stackexchange.com/a/204944/56160
Related
I want to excluse a specific filename (say, fubar.log) from a shell (bash) globbing string, *.log. Nothing of what I tried seems to work, because globbing doesn't use the standard RE set.
Test case : the directory contains
fubar.log
fubaz.log
barbaz.log
text.txt
and only fubaz.log barbaz.log must be expanded by the glob.
if you are using bash
#!/bin/bash
shopt -s extglob
ls !(fubar).log
or without extglob
shopt -u extglob
for file in !(fubar).log
do
echo "$file"
done
or
for file in *log
do
case "$file" in
fubar* ) continue;;
* ) echo "do your stuff with $file";;
esac
done
Why don't you use grep? For example:
ls |grep -v fubar|while read line; do echo "reading $line"; done;
And here is the output:
reading barbaz.log
reading fubaz.log
reading text.txt
How to have Bash to do case insensitive operation on test:
$ n=Foo
$ [ -e "$n" ] && echo $n exist
foo exist
if it is:
$ ls
foo bar baz
How the correct setting ?
You can also use bash case insensitive global matching pattern :
$ shopt -s nocaseglob
$ shopt -s nocasematch
$ n=Foo
$ f=("$n"*)
$ [ "${f[0]/$n/}" = "" ] && echo ${f[0]} exist
foo exist
But take care that to have a conditional test matching a pattern and not directly a filename with f=("$n"*), we use here a *. With that, you'll get a wrong result if there is pattern symbols like * or ? in $n.
You have also to carefully consider the global effect of shell options (shopt -s nocaseglob, shopt -s nocasematch). Generally, to avoid unexpected behaviors in any following shell commands / scripts, you have to restore initial states of theses options. Check and store the initial state with shopt option to restore it later.
Use ipath in find.
if tmp=$(find . -maxdepth 1 -ipath "./$n") && [[ -n "$tmp" ]]; then
echo "$n exists and there are:" $tmp
fi
Some days ago I started a little bash script that should sum up the number of pages and file size of all PDF's in a folder. It's working quite well now but there's still one thing I don't understand.
Why is sed always failing if shopt -s nullglob is set? Does somebody know why this happens?
I'm working with GNU Bash 4.3 and sed 4.2.2 in Ubuntu 14.04.
set -u
set -e
folder=$1
overallfilesize=0
overallpages=0
numberoffiles=0
#If glob fails nothing should be returned
shopt -s nullglob
for file in $folder/*.pdf
do
# Disable empty string if glob fails
# (Necessary because otherwise sed fails ?:|)
#shopt -u nullglob
# This command is allowed to fail
set +e
pdfinfo="$(pdfinfo "$file" 2> /dev/null)"
ret=$?
set -e
if [[ $ret -eq 0 ]]
then
#Remove every non digit in the result
sedstring='s/[^0-9]//g'
filesize=$(echo -e "$pdfinfo" | grep -m 1 "File size:" | sed $sedstring)
pages=$(echo -e "$pdfinfo" | grep -m 1 "Pages:" | sed $sedstring)
overallfilesize=$(($overallfilesize + $filesize))
overallpages=$(($overallpages+$pages))
numberoffiles=$(($numberoffiles+1))
fi
done
echo -e "Processed files: $numberoffiles"
echo -e "Pagesum: $overallpages"
echo -e "Filesizesum [Bytes]: $overallfilesize"
Here's a simpler test case for reproducing your problem:
#!/bin/bash
shopt -s nullglob
pattern='s/[^0-9]//g'
sed $pattern <<< foo42
Expected output:
42
Actual output:
Usage: sed [OPTION]... {script-only-if-no-other-script} [input-file]...
(sed usage follows)
This happens because s/[^0-9]//g is a valid glob (matching a dir structure like like s/c/g), and you asked bash to interpret it. Since you don't have a matching file, nullglob kicks in and removes the pattern entirely.
Double quoting prevents word splitting and glob interpretation, which is almost always what you want:
#!/bin/bash
shopt -s nullglob
pattern='s/[^0-9]//g'
sed "$pattern" <<< foo42
This produces the expected output.
You should always double quote all your variable references, unless you have a specific reason not to.
From the Bash manual:
Bash performs the expansion by
executing command and replacing the
command substitution with the standard
output of the command, with any
trailing newlines deleted.
That means obscure bugs are possible when handling output with meaningful trailing newlines. A contrived example:
user#host:~$ path='test
'
user#host:~$ touch -- "$path"
user#host:~$ readlink -fn -- "$path"
/home/user/test
user#host:~$ full_path="$(readlink -fn -- "$path")"
user#host:~$ ls -- "$full_path"
ls: cannot access /home/user/test: No such file or directory
Any tips on how to assign the value of a command to a variable without losing semantically useful data?
The adventures of Bash continue another day!
You could use quoting and eval to work around this. Change your last two commands to:
full_path="'$(readlink -fn -- "$path"; echo \')"
eval ls -- "$full_path"
If you want the result with trailing newlines in a variable you could first add a bogus character (in this case underscore) and then remove that.
full_path="$(readlink -fn -- "$path"; echo _)"
full_path=${full_path%_}
ls -- "$full_path"
I have no good answers. However, this hack will work for both files with and without newlines in the name.
path='test
'
touch -- "$path"
readlink -fn -- "$path"
full_path=
if [[ $path =~ $'\n' ]] ; then
while IFS=$'\n' read fn ; do
full_path+="$fn"$'\n'
done < <(readlink -fn -- "$path")
else
full_path="$(readlink -fn -- "$path")"
fi
ls -l -- "$full_path"
One (deprecated) way could be to use back-quotes instead of $(); try:
full_path=`readlink -fn -- $path`
$ readlink -f "$path"
This return the good path. I don't know why you used the -n option with readlink because it removes newlines.
Unfortunatly when I store the result of this command the newline seems to be removed
$ fullpath=`readlink -f "$path"`
$ echo "$fullpath"
/home/test
No newlines. I don't know if it's "echo" or the way to store that remove the newline at the end of the path.
$ ls
test?
$ls test?
test?
Newline seems to be replaced by ?. May be there is something to do with it.
I want to excluse a specific filename (say, fubar.log) from a shell (bash) globbing string, *.log. Nothing of what I tried seems to work, because globbing doesn't use the standard RE set.
Test case : the directory contains
fubar.log
fubaz.log
barbaz.log
text.txt
and only fubaz.log barbaz.log must be expanded by the glob.
if you are using bash
#!/bin/bash
shopt -s extglob
ls !(fubar).log
or without extglob
shopt -u extglob
for file in !(fubar).log
do
echo "$file"
done
or
for file in *log
do
case "$file" in
fubar* ) continue;;
* ) echo "do your stuff with $file";;
esac
done
Why don't you use grep? For example:
ls |grep -v fubar|while read line; do echo "reading $line"; done;
And here is the output:
reading barbaz.log
reading fubaz.log
reading text.txt