I want to excluse a specific filename (say, fubar.log) from a shell (bash) globbing string, *.log. Nothing of what I tried seems to work, because globbing doesn't use the standard RE set.
Test case : the directory contains
fubar.log
fubaz.log
barbaz.log
text.txt
and only fubaz.log barbaz.log must be expanded by the glob.
if you are using bash
#!/bin/bash
shopt -s extglob
ls !(fubar).log
or without extglob
shopt -u extglob
for file in !(fubar).log
do
echo "$file"
done
or
for file in *log
do
case "$file" in
fubar* ) continue;;
* ) echo "do your stuff with $file";;
esac
done
Why don't you use grep? For example:
ls |grep -v fubar|while read line; do echo "reading $line"; done;
And here is the output:
reading barbaz.log
reading fubaz.log
reading text.txt
Related
I want to excluse a specific filename (say, fubar.log) from a shell (bash) globbing string, *.log. Nothing of what I tried seems to work, because globbing doesn't use the standard RE set.
Test case : the directory contains
fubar.log
fubaz.log
barbaz.log
text.txt
and only fubaz.log barbaz.log must be expanded by the glob.
if you are using bash
#!/bin/bash
shopt -s extglob
ls !(fubar).log
or without extglob
shopt -u extglob
for file in !(fubar).log
do
echo "$file"
done
or
for file in *log
do
case "$file" in
fubar* ) continue;;
* ) echo "do your stuff with $file";;
esac
done
Why don't you use grep? For example:
ls |grep -v fubar|while read line; do echo "reading $line"; done;
And here is the output:
reading barbaz.log
reading fubaz.log
reading text.txt
How to escape specialcharacter # in search filename template?
this code doesn't work (empty result)
for file in *_#2.png; do echo "$file"; done
also tried this:
for i in *_#2.png; do mv "$i" "${i/_#2.png}"#2x.png; done
I need to rename files *_#2.png to *#2x.png
Check the file exists
for file in *_#2.png; do
[[ -e $file ]] || continue # to skip file which not exists
...
done
Using shell options.
If failglob option is not set when not glob match the glob *_#2.png expand to itself
shopt globfail # to check value
shopt -s globfail # to turn on
shopt -u globfail # to turn off
solved
for i in *.png; do mv "$i" "${i/_#2.png/#2x.png}" ; done
Some days ago I started a little bash script that should sum up the number of pages and file size of all PDF's in a folder. It's working quite well now but there's still one thing I don't understand.
Why is sed always failing if shopt -s nullglob is set? Does somebody know why this happens?
I'm working with GNU Bash 4.3 and sed 4.2.2 in Ubuntu 14.04.
set -u
set -e
folder=$1
overallfilesize=0
overallpages=0
numberoffiles=0
#If glob fails nothing should be returned
shopt -s nullglob
for file in $folder/*.pdf
do
# Disable empty string if glob fails
# (Necessary because otherwise sed fails ?:|)
#shopt -u nullglob
# This command is allowed to fail
set +e
pdfinfo="$(pdfinfo "$file" 2> /dev/null)"
ret=$?
set -e
if [[ $ret -eq 0 ]]
then
#Remove every non digit in the result
sedstring='s/[^0-9]//g'
filesize=$(echo -e "$pdfinfo" | grep -m 1 "File size:" | sed $sedstring)
pages=$(echo -e "$pdfinfo" | grep -m 1 "Pages:" | sed $sedstring)
overallfilesize=$(($overallfilesize + $filesize))
overallpages=$(($overallpages+$pages))
numberoffiles=$(($numberoffiles+1))
fi
done
echo -e "Processed files: $numberoffiles"
echo -e "Pagesum: $overallpages"
echo -e "Filesizesum [Bytes]: $overallfilesize"
Here's a simpler test case for reproducing your problem:
#!/bin/bash
shopt -s nullglob
pattern='s/[^0-9]//g'
sed $pattern <<< foo42
Expected output:
42
Actual output:
Usage: sed [OPTION]... {script-only-if-no-other-script} [input-file]...
(sed usage follows)
This happens because s/[^0-9]//g is a valid glob (matching a dir structure like like s/c/g), and you asked bash to interpret it. Since you don't have a matching file, nullglob kicks in and removes the pattern entirely.
Double quoting prevents word splitting and glob interpretation, which is almost always what you want:
#!/bin/bash
shopt -s nullglob
pattern='s/[^0-9]//g'
sed "$pattern" <<< foo42
This produces the expected output.
You should always double quote all your variable references, unless you have a specific reason not to.
For example, try following command in an empty directory:
$ for i in *; do echo $i; done
*
Is there a way to suppress the printout of *?
Set nullglob
shopt -s nullglob
for i in *; do echo "$i"; done
Basic idea is use ls command, but if filename has space, ls will split file name by space. In order to handle space in filename, you can do like this:
ls|while read i; do echo $i; done
Aleks-Daniel Jakimenko said "Do not parse ls". Which is good, so how about this if we don't want to change nullglob:
for i in *; do [ -e "$i" ] && echo $i; done
To avoid the *, you can try something like
for i in `ls`; do echo $i; done
Tried now on an empty directory, no output given...
In bash, if you do this:
mkdir /tmp/empty
array=(/tmp/empty/*)
you find that array now has one element, "/tmp/empty/*", not zero as you'd like. Thankfully, this can be avoided by turning on the nullglob shell option using shopt -s nullglob
But nullglob is global, and when editing an existing shell script, may break things (e.g., did someone check the exit code of ls foo* to check if there are files named starting with "foo"?). So, ideally, I'd like to turn it on only for a small scope—ideally, one filename expansion. You can turn it off again using shopt -u nullglob But of course only if it was disabled before:
old_nullglob=$(shopt -p | grep 'nullglob$')
shopt -s nullglob
array=(/tmp/empty/*)
eval "$old_nullglob"
unset -v old_nullglob
makes me think there must be a better way. The obvious "put it in a subshell" doesn't work as of course the variable assignment dies with the subshell. Other than waiting for the Austin group to import ksh93 syntax, is there?
Unset it when done:
shopt -u nullglob
And properly (i.e. storing the previous state):
shopt -u | grep -q nullglob && changed=true && shopt -s nullglob
... do whatever you want ...
[ $changed ] && shopt -u nullglob; unset changed
With mapfile in Bash 4, you can load an array from a subshell with something like: mapfile array < <(shopt -s nullglob; for f in ./*; do echo "$f"; done). Full example:
$ shopt nullglob
nullglob off
$ find
.
./bar baz
./qux quux
$ mapfile array < <(shopt -s nullglob; for f in ./*; do echo "$f"; done)
$ shopt nullglob
nullglob off
$ echo ${#array[#]}
2
$ echo ${array[0]}
bar baz
$ echo ${array[1]}
qux quux
$ rm *
$ mapfile array < <(shopt -s nullglob; for f in ./*; do echo "$f"; done)
$ echo ${#array[#]}
0
Be sure to glob with ./* instead of a bare * when using echo to print the file name
Doesn't work with newline characters in the filename :( as pointed out by derobert
If you need to handle newlines in the filename, you will have to do the much more verbose:
array=()
while read -r -d $'\0'; do
array+=("$REPLY")
done < <(shopt -s nullglob; for f in ./*; do printf "$f\0"; done)
But by this point, it may be simpler to follow the advice of one of the other answers.
This is just a tiny bit better than your original suggestion:
local nullglob=$(shopt -p nullglob) ; shopt -s nullglob
... do whatever you want ...
$nullglob ; unset nullglob
This may be close to what you want; as is, it requires executing a command to expand the glob.
$ ls
file1 file2
$ array=( $(shopt -s nullglob; ls foo*) )
$ ls foo*
ls: foo*: No such file or directory
$ echo ${array[*]}
file1 file2
Instead of setting array in the subshell, we create a subshell using $() whose output is captured by array.
This is the simplest solution I've found:
For example, to expand the literal **/*.mp3 into a glob for only a particular variable, you can use
VAR=**/*.mp3(N)
Source: https://unix.stackexchange.com/a/204944/56160