Prevent "mv" command from raising error if no file matches the glob. eg" mv *.json /dir/ - bash

I want to move all JSON files created within a jenkins job to a different folder.
It is possible that the job does not create any json file.
In that case the mv command is raising an error and so that job is failing.
How do I prevent mv command from raising error in case no file is found?

Welcome to SO.
Why do you not want the error?
If you just don't want to see the error, then you could always just throw it away with 2>/dev/null, but PLEASE don't do that. Not every error is the one you expect, and this is a debugging nightmare. You could write it to a log with 2>$logpath and then build in logic to read that to make certain it's ok, and ignore or respond accordingly --
mv *.json /dir/ 2>$someLog
executeMyLogParsingFunction # verify expected err is the ONLY err
If it's because you have set -e or a trap in place, and you know it's ok for the mv to fail (which might not be because there is no file!), then you can use this trick -
mv *.json /dir/ || echo "(Error ok if no files found)"
or
mv *.json /dir/ ||: # : is a no-op synonym for "true" that returns 0
see https://www.gnu.org/software/bash/manual/html_node/Conditional-Constructs.html
(If it's failing simply because the mv is returning a nonzero as the last command, you could also add an explicit exit 0, but don't do that either - fix the actual problem rather than patching the symptom. Any of these other solutions should handle that, but I wanted to point out that unless there's a set -e or a trap that catches the error, it shouldn't cause the script to fail unless it's the very last command.)
Better would be to specifically handle the problem you expect without disabling error handling on other problems.
shopt -s nullglob # globs with no match do not eval to the glob as a string
for f in *.json; do mv "$f" /dir/; done # no match means no loop entry
c.f. https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html
or if you don't want to use shopt,
for f in *.json; do [[ -e "$f" ]] && mv "$f" /dir/; done
Note that I'm only testing existence, so that will include any match, including directories, symlinks, named pipes... you might want [[ -f "$f" ]] && mv "$f" /dir/ instead.
c.f. https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html

This is expected behavior -- it's why the shell leaves *.json unexpanded when there are no matches, to allow mv to show a useful error.
If you don't want that, though, you can always check the list of files yourself, before passing it to mv. As an approach that works with all POSIX-compliant shells, not just bash:
#!/bin/sh
# using a function here gives us our own private argument list.
# that's useful because minimal POSIX sh doesn't provide arrays.
move_if_any() {
dest=$1; shift # shift makes the old $2 be $1, the old $3 be $2, etc.
# so, we then check how many arguments were left after the shift;
# if it's only one, we need to also check whether it refers to a filesystem
# object that actually exists.
if [ "$#" -gt 1 ] || [ -e "$1" ] || [ -L "$1" ]; then
mv -- "$#" "$dest"
fi
}
# put destination_directory/ in $1 where it'll be shifted off
# $2 will be either nonexistent (if we were really running in bash with nullglob set)
# ...or the name of a legitimate file or symlink, or the string '*.json'
move_if_any destination_directory/ *.json
...or, as a more bash-specific approach:
#!/bin/bash
files=( *.json )
if (( ${#files[#]} > 1 )) || [[ -e ${files[0]} || -L ${files[0]} ]]; then
mv -- "${files[#]}" destination/
fi

Loop over all json files and move each of them, if it exists, in a oneliner:
for X in *.json; do [[ -e $X ]] && mv "$X" /dir/; done

Related

How to check if files with a certain extension exist with wildcard in Bash [duplicate]

If I want to check for the existence of a single file, I can test for it using test -e filename or [ -e filename ].
Supposing I have a glob and I want to know whether any files exist whose names match the glob. The glob can match 0 files (in which case I need to do nothing), or it can match 1 or more files (in which case I need to do something). How can I test whether a glob has any matches? (I don't care how many matches there are, and it would be best if I could do this with one if statement and no loops (simply because I find that most readable).
(test -e glob* fails if the glob matches more than one file.)
Bash-specific solution:
compgen -G "<glob-pattern>"
Escape the pattern or it'll get pre-expanded into matches.
Exit status is:
1 for no-match,
0 for 'one or more matches'
stdout is a list of files matching the glob.
I think this is the best option in terms of conciseness and minimizing potential side effects.
Example:
if compgen -G "/tmp/someFiles*" > /dev/null; then
echo "Some files exist."
fi
The nullglob shell option is indeed a bashism.
To avoid the need for a tedious save and restore of the nullglob state, I'd only set it inside the subshell that expands the glob:
if test -n "$(shopt -s nullglob; echo glob*)"
then
echo found
else
echo not found
fi
For better portability and more flexible globbing, use find:
if test -n "$(find . -maxdepth 1 -name 'glob*' -print -quit)"
then
echo found
else
echo not found
fi
Explicit -print -quit actions are used for find instead of the default implicit -print action so that find will quit as soon as it finds the first file matching the search criteria. Where lots of files match, this should run much faster than echo glob* or ls glob* and it also avoids the possibility of overstuffing the expanded command line (some shells have a 4K length limit).
If find feels like overkill and the number of files likely to match is small, use stat:
if stat -t glob* >/dev/null 2>&1
then
echo found
else
echo not found
fi
I like
exists() {
[ -e "$1" ]
}
if exists glob*; then
echo found
else
echo not found
fi
This is both readable and efficient (unless there are a huge number of files).
The main drawback is that it's much more subtle than it looks, and I sometimes feel compelled to add a long comment.
If there's a match, "glob*" is expanded by the shell and all the matches are passed to exists(), which checks the first one and ignores the rest.
If there's no match, "glob*" is passed to exists() and found not to exist there either.
Edit: there may be a false positive, see comment
#!/usr/bin/env bash
# If it is set, then an unmatched glob is swept away entirely --
# replaced with a set of zero words --
# instead of remaining in place as a single word.
shopt -s nullglob
M=(*px)
if [ "${#M[*]}" -ge 1 ]; then
echo "${#M[*]} matches."
else
echo "No such files."
fi
If you have globfail set you can use this crazy (which you really should not)
shopt -s failglob # exit if * does not match
( : * ) && echo 0 || echo 1
or
q=( * ) && echo 0 || echo 1
test -e has the unfortunate caveat that it considers broken symbolic links to not exist. So you may want to check for those, too.
function globexists {
test -e "$1" -o -L "$1"
}
if globexists glob*; then
echo found
else
echo not found
fi
I have yet another solution:
if [ "$(echo glob*)" != 'glob*' ]
This works nicely for me. There may be some corner cases I missed.
Based on flabdablet's answer, for me it looks like easiest (not necessarily fastest) is just to use find itself, while leaving glob expansion on shell, like:
find /some/{p,long-p}ath/with/*globs* -quit &> /dev/null && echo "MATCH"
Or in if like:
if find $yourGlob -quit &> /dev/null; then
echo "MATCH"
else
echo "NOT-FOUND"
fi
To simplify miku's answer somewhat, based on his idea:
M=(*py)
if [ -e ${M[0]} ]; then
echo Found
else
echo Not Found
fi
In Bash, you can glob to an array; if the glob didn't match, your array will contain a single entry that doesn't correspond to an existing file:
#!/bin/bash
shellglob='*.sh'
scripts=($shellglob)
if [ -e "${scripts[0]}" ]
then stat "${scripts[#]}"
fi
Note: if you have nullglob set, scripts will be an empty array, and you should test with [ "${scripts[*]}" ] or with [ "${#scripts[*]}" != 0 ] instead. If you're writing a library that must work with or without nullglob, you'll want
if [ "${scripts[*]}" ] && [ -e "${scripts[0]}" ]
An advantage of this approach is that you then have the list of files you want to work with, rather than having to repeat the glob operation.
If you want to test if the files exist before iterating over them, you can use this pattern:
for F in glob*; do
if [[ ! -f $F ]]; then break; fi
...
done
if the glob does not does not match anything, $F will be the non-expanded glob ('glob*' in this case) and if a file with the same name does not exist, it will skip the rest of the loop.
#!/bin/bash
set nullglob
touch /tmp/foo1 /tmp/foo2 /tmp/foo3
FOUND=0
for FILE in /tmp/foo*
do
FOUND=$((${FOUND} + 1))
done
if [ ${FOUND} -gt 0 ]; then
echo "I found ${FOUND} matches"
else
echo "No matches found"
fi
set -- glob*
if [ -f "$1" ]; then
echo "It matched"
fi
Explanation
When there isn't a match for glob*, then $1 will contain 'glob*'. The test -f "$1" won't be true because the glob* file doesn't exist.
Why this is better than alternatives
This works with sh and derivates: KornShell and Bash. It doesn't create any sub-shell. $(..) and `...` commands create a sub-shell; they fork a process, and therefore are slower than this solution.
Like this in Bash (test files containing pattern):
shopt -s nullglob
compgen -W *pattern* &>/dev/null
case $? in
0) echo "only one file match" ;;
1) echo "more than one file match" ;;
2) echo "no file match" ;;
esac
It's far better than compgen -G: because we can discriminates more cases and more precisely.
It can work with only one wildcard *.
This abomination seems to work:
#!/usr/bin/env bash
shopt -s nullglob
if [ "`echo *py`" != "" ]; then
echo "Glob matched"
else
echo "Glob did not match"
fi
It probably requires bash, not sh.
This works because the nullglob option causes the glob to evaluate to an empty string if there are no matches. Thus any non-empty output from the echo command indicates that the glob matched something.
A solution for extended globs (extglob) in Bash:
bash -c $'shopt -s extglob \n /bin/ls -1U <ext-glob-pattern>'
Exit status is 0 if there is at least one match, and non-zero (2) when there is no match. Standard output contains a newline-separated list of matching files (and file names containing spaces they are quoted).
Or, slightly different:
bash -c $'shopt -s extglob \n compgen -G <ext-glob-pattern>'
Differences to the ls-based solution: probably faster (not measured), file names with spaces not quoted in output, exit code 1 when there is no match (not 2 :shrug:).
Example usage:
No match:
$ bash -c $'shopt -s extglob \n /bin/ls -1U #(*.foo|*.bar)'; echo "exit status: $?"
/bin/ls: cannot access '#(*.foo|*.bar)': No such file or directory
exit status: 2
At least one match:
$ bash -c $'shopt -s extglob \n /bin/ls -1U #(*.ts|*.mp4)'; echo "exit status: $?"
'video1 with spaces.mp4'
video2.mp4
video3.mp4
exit status: 0
Concepts used:
ls' exit code behavior (adds -U for efficiency, and -1 for output control).
Does not enable extglob in current shell (often not desired).
Makes use of the $ prefix so that the \n is interpreted, so that the extended glob pattern is on a different line than the shopt -s extglob -- otherwise the extended glob pattern would be a syntax error!
Note 1: I worked towards this solution because the compgen -G "<glob-pattern>" approach suggested in other answers does not seem to work smoothly with brace expansion; and yet I needed some more advanced globbing features.
Note 2: lovely resource for the extended glob syntax: extglob
Both nullglob and compgen are useful only on some bash shells.
A (non-recursive) solution that works on most shells is:
set -- ./glob* # or /path/dir/glob*
[ -f "$1" ] || shift # remove the glob if present.
if [ "$#" -lt 1 ]
then echo "at least one file found"
fi
Assuming you may want to do something with the files if they exist:
mapfile -t exists < <(find "$dirName" -type f -iname '*.zip'); [[ ${#exists} -ne 0 ]] && { echo "Zip files found" ; } || { echo "Zip files not found" ; }
You can then loop through the exists array if you need to do something with the files.
(ls glob* &>/dev/null && echo Files found) || echo No file found
if ls -d $glob > /dev/null 2>&1; then
echo Found.
else
echo Not found.
fi
Note that this can be very time cosuming if there are a lot of matches or file access is slow.
ls | grep -q "glob.*"
Not the most efficient solution (if there's a ton of files in the directory it might be slowish), but it's simple, easy to read and also has the advantage that regexes are more powerful than plain Bash glob patterns.
[ `ls glob* 2>/dev/null | head -n 1` ] && echo true

How can I find the exist file in shell script with * argument [duplicate]

If I want to check for the existence of a single file, I can test for it using test -e filename or [ -e filename ].
Supposing I have a glob and I want to know whether any files exist whose names match the glob. The glob can match 0 files (in which case I need to do nothing), or it can match 1 or more files (in which case I need to do something). How can I test whether a glob has any matches? (I don't care how many matches there are, and it would be best if I could do this with one if statement and no loops (simply because I find that most readable).
(test -e glob* fails if the glob matches more than one file.)
Bash-specific solution:
compgen -G "<glob-pattern>"
Escape the pattern or it'll get pre-expanded into matches.
Exit status is:
1 for no-match,
0 for 'one or more matches'
stdout is a list of files matching the glob.
I think this is the best option in terms of conciseness and minimizing potential side effects.
Example:
if compgen -G "/tmp/someFiles*" > /dev/null; then
echo "Some files exist."
fi
The nullglob shell option is indeed a bashism.
To avoid the need for a tedious save and restore of the nullglob state, I'd only set it inside the subshell that expands the glob:
if test -n "$(shopt -s nullglob; echo glob*)"
then
echo found
else
echo not found
fi
For better portability and more flexible globbing, use find:
if test -n "$(find . -maxdepth 1 -name 'glob*' -print -quit)"
then
echo found
else
echo not found
fi
Explicit -print -quit actions are used for find instead of the default implicit -print action so that find will quit as soon as it finds the first file matching the search criteria. Where lots of files match, this should run much faster than echo glob* or ls glob* and it also avoids the possibility of overstuffing the expanded command line (some shells have a 4K length limit).
If find feels like overkill and the number of files likely to match is small, use stat:
if stat -t glob* >/dev/null 2>&1
then
echo found
else
echo not found
fi
I like
exists() {
[ -e "$1" ]
}
if exists glob*; then
echo found
else
echo not found
fi
This is both readable and efficient (unless there are a huge number of files).
The main drawback is that it's much more subtle than it looks, and I sometimes feel compelled to add a long comment.
If there's a match, "glob*" is expanded by the shell and all the matches are passed to exists(), which checks the first one and ignores the rest.
If there's no match, "glob*" is passed to exists() and found not to exist there either.
Edit: there may be a false positive, see comment
#!/usr/bin/env bash
# If it is set, then an unmatched glob is swept away entirely --
# replaced with a set of zero words --
# instead of remaining in place as a single word.
shopt -s nullglob
M=(*px)
if [ "${#M[*]}" -ge 1 ]; then
echo "${#M[*]} matches."
else
echo "No such files."
fi
If you have globfail set you can use this crazy (which you really should not)
shopt -s failglob # exit if * does not match
( : * ) && echo 0 || echo 1
or
q=( * ) && echo 0 || echo 1
test -e has the unfortunate caveat that it considers broken symbolic links to not exist. So you may want to check for those, too.
function globexists {
test -e "$1" -o -L "$1"
}
if globexists glob*; then
echo found
else
echo not found
fi
I have yet another solution:
if [ "$(echo glob*)" != 'glob*' ]
This works nicely for me. There may be some corner cases I missed.
Based on flabdablet's answer, for me it looks like easiest (not necessarily fastest) is just to use find itself, while leaving glob expansion on shell, like:
find /some/{p,long-p}ath/with/*globs* -quit &> /dev/null && echo "MATCH"
Or in if like:
if find $yourGlob -quit &> /dev/null; then
echo "MATCH"
else
echo "NOT-FOUND"
fi
To simplify miku's answer somewhat, based on his idea:
M=(*py)
if [ -e ${M[0]} ]; then
echo Found
else
echo Not Found
fi
In Bash, you can glob to an array; if the glob didn't match, your array will contain a single entry that doesn't correspond to an existing file:
#!/bin/bash
shellglob='*.sh'
scripts=($shellglob)
if [ -e "${scripts[0]}" ]
then stat "${scripts[#]}"
fi
Note: if you have nullglob set, scripts will be an empty array, and you should test with [ "${scripts[*]}" ] or with [ "${#scripts[*]}" != 0 ] instead. If you're writing a library that must work with or without nullglob, you'll want
if [ "${scripts[*]}" ] && [ -e "${scripts[0]}" ]
An advantage of this approach is that you then have the list of files you want to work with, rather than having to repeat the glob operation.
If you want to test if the files exist before iterating over them, you can use this pattern:
for F in glob*; do
if [[ ! -f $F ]]; then break; fi
...
done
if the glob does not does not match anything, $F will be the non-expanded glob ('glob*' in this case) and if a file with the same name does not exist, it will skip the rest of the loop.
#!/bin/bash
set nullglob
touch /tmp/foo1 /tmp/foo2 /tmp/foo3
FOUND=0
for FILE in /tmp/foo*
do
FOUND=$((${FOUND} + 1))
done
if [ ${FOUND} -gt 0 ]; then
echo "I found ${FOUND} matches"
else
echo "No matches found"
fi
set -- glob*
if [ -f "$1" ]; then
echo "It matched"
fi
Explanation
When there isn't a match for glob*, then $1 will contain 'glob*'. The test -f "$1" won't be true because the glob* file doesn't exist.
Why this is better than alternatives
This works with sh and derivates: KornShell and Bash. It doesn't create any sub-shell. $(..) and `...` commands create a sub-shell; they fork a process, and therefore are slower than this solution.
Like this in Bash (test files containing pattern):
shopt -s nullglob
compgen -W *pattern* &>/dev/null
case $? in
0) echo "only one file match" ;;
1) echo "more than one file match" ;;
2) echo "no file match" ;;
esac
It's far better than compgen -G: because we can discriminates more cases and more precisely.
It can work with only one wildcard *.
This abomination seems to work:
#!/usr/bin/env bash
shopt -s nullglob
if [ "`echo *py`" != "" ]; then
echo "Glob matched"
else
echo "Glob did not match"
fi
It probably requires bash, not sh.
This works because the nullglob option causes the glob to evaluate to an empty string if there are no matches. Thus any non-empty output from the echo command indicates that the glob matched something.
A solution for extended globs (extglob) in Bash:
bash -c $'shopt -s extglob \n /bin/ls -1U <ext-glob-pattern>'
Exit status is 0 if there is at least one match, and non-zero (2) when there is no match. Standard output contains a newline-separated list of matching files (and file names containing spaces they are quoted).
Or, slightly different:
bash -c $'shopt -s extglob \n compgen -G <ext-glob-pattern>'
Differences to the ls-based solution: probably faster (not measured), file names with spaces not quoted in output, exit code 1 when there is no match (not 2 :shrug:).
Example usage:
No match:
$ bash -c $'shopt -s extglob \n /bin/ls -1U #(*.foo|*.bar)'; echo "exit status: $?"
/bin/ls: cannot access '#(*.foo|*.bar)': No such file or directory
exit status: 2
At least one match:
$ bash -c $'shopt -s extglob \n /bin/ls -1U #(*.ts|*.mp4)'; echo "exit status: $?"
'video1 with spaces.mp4'
video2.mp4
video3.mp4
exit status: 0
Concepts used:
ls' exit code behavior (adds -U for efficiency, and -1 for output control).
Does not enable extglob in current shell (often not desired).
Makes use of the $ prefix so that the \n is interpreted, so that the extended glob pattern is on a different line than the shopt -s extglob -- otherwise the extended glob pattern would be a syntax error!
Note 1: I worked towards this solution because the compgen -G "<glob-pattern>" approach suggested in other answers does not seem to work smoothly with brace expansion; and yet I needed some more advanced globbing features.
Note 2: lovely resource for the extended glob syntax: extglob
Both nullglob and compgen are useful only on some bash shells.
A (non-recursive) solution that works on most shells is:
set -- ./glob* # or /path/dir/glob*
[ -f "$1" ] || shift # remove the glob if present.
if [ "$#" -lt 1 ]
then echo "at least one file found"
fi
Assuming you may want to do something with the files if they exist:
mapfile -t exists < <(find "$dirName" -type f -iname '*.zip'); [[ ${#exists} -ne 0 ]] && { echo "Zip files found" ; } || { echo "Zip files not found" ; }
You can then loop through the exists array if you need to do something with the files.
(ls glob* &>/dev/null && echo Files found) || echo No file found
if ls -d $glob > /dev/null 2>&1; then
echo Found.
else
echo Not found.
fi
Note that this can be very time cosuming if there are a lot of matches or file access is slow.
ls | grep -q "glob.*"
Not the most efficient solution (if there's a ton of files in the directory it might be slowish), but it's simple, easy to read and also has the advantage that regexes are more powerful than plain Bash glob patterns.
[ `ls glob* 2>/dev/null | head -n 1` ] && echo true

Find multiple file formats in IF condition in Shell Script [duplicate]

If I want to check for the existence of a single file, I can test for it using test -e filename or [ -e filename ].
Supposing I have a glob and I want to know whether any files exist whose names match the glob. The glob can match 0 files (in which case I need to do nothing), or it can match 1 or more files (in which case I need to do something). How can I test whether a glob has any matches? (I don't care how many matches there are, and it would be best if I could do this with one if statement and no loops (simply because I find that most readable).
(test -e glob* fails if the glob matches more than one file.)
Bash-specific solution:
compgen -G "<glob-pattern>"
Escape the pattern or it'll get pre-expanded into matches.
Exit status is:
1 for no-match,
0 for 'one or more matches'
stdout is a list of files matching the glob.
I think this is the best option in terms of conciseness and minimizing potential side effects.
Example:
if compgen -G "/tmp/someFiles*" > /dev/null; then
echo "Some files exist."
fi
The nullglob shell option is indeed a bashism.
To avoid the need for a tedious save and restore of the nullglob state, I'd only set it inside the subshell that expands the glob:
if test -n "$(shopt -s nullglob; echo glob*)"
then
echo found
else
echo not found
fi
For better portability and more flexible globbing, use find:
if test -n "$(find . -maxdepth 1 -name 'glob*' -print -quit)"
then
echo found
else
echo not found
fi
Explicit -print -quit actions are used for find instead of the default implicit -print action so that find will quit as soon as it finds the first file matching the search criteria. Where lots of files match, this should run much faster than echo glob* or ls glob* and it also avoids the possibility of overstuffing the expanded command line (some shells have a 4K length limit).
If find feels like overkill and the number of files likely to match is small, use stat:
if stat -t glob* >/dev/null 2>&1
then
echo found
else
echo not found
fi
I like
exists() {
[ -e "$1" ]
}
if exists glob*; then
echo found
else
echo not found
fi
This is both readable and efficient (unless there are a huge number of files).
The main drawback is that it's much more subtle than it looks, and I sometimes feel compelled to add a long comment.
If there's a match, "glob*" is expanded by the shell and all the matches are passed to exists(), which checks the first one and ignores the rest.
If there's no match, "glob*" is passed to exists() and found not to exist there either.
Edit: there may be a false positive, see comment
#!/usr/bin/env bash
# If it is set, then an unmatched glob is swept away entirely --
# replaced with a set of zero words --
# instead of remaining in place as a single word.
shopt -s nullglob
M=(*px)
if [ "${#M[*]}" -ge 1 ]; then
echo "${#M[*]} matches."
else
echo "No such files."
fi
If you have globfail set you can use this crazy (which you really should not)
shopt -s failglob # exit if * does not match
( : * ) && echo 0 || echo 1
or
q=( * ) && echo 0 || echo 1
test -e has the unfortunate caveat that it considers broken symbolic links to not exist. So you may want to check for those, too.
function globexists {
test -e "$1" -o -L "$1"
}
if globexists glob*; then
echo found
else
echo not found
fi
I have yet another solution:
if [ "$(echo glob*)" != 'glob*' ]
This works nicely for me. There may be some corner cases I missed.
Based on flabdablet's answer, for me it looks like easiest (not necessarily fastest) is just to use find itself, while leaving glob expansion on shell, like:
find /some/{p,long-p}ath/with/*globs* -quit &> /dev/null && echo "MATCH"
Or in if like:
if find $yourGlob -quit &> /dev/null; then
echo "MATCH"
else
echo "NOT-FOUND"
fi
To simplify miku's answer somewhat, based on his idea:
M=(*py)
if [ -e ${M[0]} ]; then
echo Found
else
echo Not Found
fi
In Bash, you can glob to an array; if the glob didn't match, your array will contain a single entry that doesn't correspond to an existing file:
#!/bin/bash
shellglob='*.sh'
scripts=($shellglob)
if [ -e "${scripts[0]}" ]
then stat "${scripts[#]}"
fi
Note: if you have nullglob set, scripts will be an empty array, and you should test with [ "${scripts[*]}" ] or with [ "${#scripts[*]}" != 0 ] instead. If you're writing a library that must work with or without nullglob, you'll want
if [ "${scripts[*]}" ] && [ -e "${scripts[0]}" ]
An advantage of this approach is that you then have the list of files you want to work with, rather than having to repeat the glob operation.
If you want to test if the files exist before iterating over them, you can use this pattern:
for F in glob*; do
if [[ ! -f $F ]]; then break; fi
...
done
if the glob does not does not match anything, $F will be the non-expanded glob ('glob*' in this case) and if a file with the same name does not exist, it will skip the rest of the loop.
#!/bin/bash
set nullglob
touch /tmp/foo1 /tmp/foo2 /tmp/foo3
FOUND=0
for FILE in /tmp/foo*
do
FOUND=$((${FOUND} + 1))
done
if [ ${FOUND} -gt 0 ]; then
echo "I found ${FOUND} matches"
else
echo "No matches found"
fi
set -- glob*
if [ -f "$1" ]; then
echo "It matched"
fi
Explanation
When there isn't a match for glob*, then $1 will contain 'glob*'. The test -f "$1" won't be true because the glob* file doesn't exist.
Why this is better than alternatives
This works with sh and derivates: KornShell and Bash. It doesn't create any sub-shell. $(..) and `...` commands create a sub-shell; they fork a process, and therefore are slower than this solution.
Like this in Bash (test files containing pattern):
shopt -s nullglob
compgen -W *pattern* &>/dev/null
case $? in
0) echo "only one file match" ;;
1) echo "more than one file match" ;;
2) echo "no file match" ;;
esac
It's far better than compgen -G: because we can discriminates more cases and more precisely.
It can work with only one wildcard *.
This abomination seems to work:
#!/usr/bin/env bash
shopt -s nullglob
if [ "`echo *py`" != "" ]; then
echo "Glob matched"
else
echo "Glob did not match"
fi
It probably requires bash, not sh.
This works because the nullglob option causes the glob to evaluate to an empty string if there are no matches. Thus any non-empty output from the echo command indicates that the glob matched something.
A solution for extended globs (extglob) in Bash:
bash -c $'shopt -s extglob \n /bin/ls -1U <ext-glob-pattern>'
Exit status is 0 if there is at least one match, and non-zero (2) when there is no match. Standard output contains a newline-separated list of matching files (and file names containing spaces they are quoted).
Or, slightly different:
bash -c $'shopt -s extglob \n compgen -G <ext-glob-pattern>'
Differences to the ls-based solution: probably faster (not measured), file names with spaces not quoted in output, exit code 1 when there is no match (not 2 :shrug:).
Example usage:
No match:
$ bash -c $'shopt -s extglob \n /bin/ls -1U #(*.foo|*.bar)'; echo "exit status: $?"
/bin/ls: cannot access '#(*.foo|*.bar)': No such file or directory
exit status: 2
At least one match:
$ bash -c $'shopt -s extglob \n /bin/ls -1U #(*.ts|*.mp4)'; echo "exit status: $?"
'video1 with spaces.mp4'
video2.mp4
video3.mp4
exit status: 0
Concepts used:
ls' exit code behavior (adds -U for efficiency, and -1 for output control).
Does not enable extglob in current shell (often not desired).
Makes use of the $ prefix so that the \n is interpreted, so that the extended glob pattern is on a different line than the shopt -s extglob -- otherwise the extended glob pattern would be a syntax error!
Note 1: I worked towards this solution because the compgen -G "<glob-pattern>" approach suggested in other answers does not seem to work smoothly with brace expansion; and yet I needed some more advanced globbing features.
Note 2: lovely resource for the extended glob syntax: extglob
Both nullglob and compgen are useful only on some bash shells.
A (non-recursive) solution that works on most shells is:
set -- ./glob* # or /path/dir/glob*
[ -f "$1" ] || shift # remove the glob if present.
if [ "$#" -lt 1 ]
then echo "at least one file found"
fi
Assuming you may want to do something with the files if they exist:
mapfile -t exists < <(find "$dirName" -type f -iname '*.zip'); [[ ${#exists} -ne 0 ]] && { echo "Zip files found" ; } || { echo "Zip files not found" ; }
You can then loop through the exists array if you need to do something with the files.
(ls glob* &>/dev/null && echo Files found) || echo No file found
if ls -d $glob > /dev/null 2>&1; then
echo Found.
else
echo Not found.
fi
Note that this can be very time cosuming if there are a lot of matches or file access is slow.
ls | grep -q "glob.*"
Not the most efficient solution (if there's a ton of files in the directory it might be slowish), but it's simple, easy to read and also has the advantage that regexes are more powerful than plain Bash glob patterns.
[ `ls glob* 2>/dev/null | head -n 1` ] && echo true

if condition to assign variable for file check with wildcard - shell script [duplicate]

If I want to check for the existence of a single file, I can test for it using test -e filename or [ -e filename ].
Supposing I have a glob and I want to know whether any files exist whose names match the glob. The glob can match 0 files (in which case I need to do nothing), or it can match 1 or more files (in which case I need to do something). How can I test whether a glob has any matches? (I don't care how many matches there are, and it would be best if I could do this with one if statement and no loops (simply because I find that most readable).
(test -e glob* fails if the glob matches more than one file.)
Bash-specific solution:
compgen -G "<glob-pattern>"
Escape the pattern or it'll get pre-expanded into matches.
Exit status is:
1 for no-match,
0 for 'one or more matches'
stdout is a list of files matching the glob.
I think this is the best option in terms of conciseness and minimizing potential side effects.
Example:
if compgen -G "/tmp/someFiles*" > /dev/null; then
echo "Some files exist."
fi
The nullglob shell option is indeed a bashism.
To avoid the need for a tedious save and restore of the nullglob state, I'd only set it inside the subshell that expands the glob:
if test -n "$(shopt -s nullglob; echo glob*)"
then
echo found
else
echo not found
fi
For better portability and more flexible globbing, use find:
if test -n "$(find . -maxdepth 1 -name 'glob*' -print -quit)"
then
echo found
else
echo not found
fi
Explicit -print -quit actions are used for find instead of the default implicit -print action so that find will quit as soon as it finds the first file matching the search criteria. Where lots of files match, this should run much faster than echo glob* or ls glob* and it also avoids the possibility of overstuffing the expanded command line (some shells have a 4K length limit).
If find feels like overkill and the number of files likely to match is small, use stat:
if stat -t glob* >/dev/null 2>&1
then
echo found
else
echo not found
fi
I like
exists() {
[ -e "$1" ]
}
if exists glob*; then
echo found
else
echo not found
fi
This is both readable and efficient (unless there are a huge number of files).
The main drawback is that it's much more subtle than it looks, and I sometimes feel compelled to add a long comment.
If there's a match, "glob*" is expanded by the shell and all the matches are passed to exists(), which checks the first one and ignores the rest.
If there's no match, "glob*" is passed to exists() and found not to exist there either.
Edit: there may be a false positive, see comment
#!/usr/bin/env bash
# If it is set, then an unmatched glob is swept away entirely --
# replaced with a set of zero words --
# instead of remaining in place as a single word.
shopt -s nullglob
M=(*px)
if [ "${#M[*]}" -ge 1 ]; then
echo "${#M[*]} matches."
else
echo "No such files."
fi
If you have globfail set you can use this crazy (which you really should not)
shopt -s failglob # exit if * does not match
( : * ) && echo 0 || echo 1
or
q=( * ) && echo 0 || echo 1
test -e has the unfortunate caveat that it considers broken symbolic links to not exist. So you may want to check for those, too.
function globexists {
test -e "$1" -o -L "$1"
}
if globexists glob*; then
echo found
else
echo not found
fi
I have yet another solution:
if [ "$(echo glob*)" != 'glob*' ]
This works nicely for me. There may be some corner cases I missed.
Based on flabdablet's answer, for me it looks like easiest (not necessarily fastest) is just to use find itself, while leaving glob expansion on shell, like:
find /some/{p,long-p}ath/with/*globs* -quit &> /dev/null && echo "MATCH"
Or in if like:
if find $yourGlob -quit &> /dev/null; then
echo "MATCH"
else
echo "NOT-FOUND"
fi
To simplify miku's answer somewhat, based on his idea:
M=(*py)
if [ -e ${M[0]} ]; then
echo Found
else
echo Not Found
fi
In Bash, you can glob to an array; if the glob didn't match, your array will contain a single entry that doesn't correspond to an existing file:
#!/bin/bash
shellglob='*.sh'
scripts=($shellglob)
if [ -e "${scripts[0]}" ]
then stat "${scripts[#]}"
fi
Note: if you have nullglob set, scripts will be an empty array, and you should test with [ "${scripts[*]}" ] or with [ "${#scripts[*]}" != 0 ] instead. If you're writing a library that must work with or without nullglob, you'll want
if [ "${scripts[*]}" ] && [ -e "${scripts[0]}" ]
An advantage of this approach is that you then have the list of files you want to work with, rather than having to repeat the glob operation.
If you want to test if the files exist before iterating over them, you can use this pattern:
for F in glob*; do
if [[ ! -f $F ]]; then break; fi
...
done
if the glob does not does not match anything, $F will be the non-expanded glob ('glob*' in this case) and if a file with the same name does not exist, it will skip the rest of the loop.
#!/bin/bash
set nullglob
touch /tmp/foo1 /tmp/foo2 /tmp/foo3
FOUND=0
for FILE in /tmp/foo*
do
FOUND=$((${FOUND} + 1))
done
if [ ${FOUND} -gt 0 ]; then
echo "I found ${FOUND} matches"
else
echo "No matches found"
fi
set -- glob*
if [ -f "$1" ]; then
echo "It matched"
fi
Explanation
When there isn't a match for glob*, then $1 will contain 'glob*'. The test -f "$1" won't be true because the glob* file doesn't exist.
Why this is better than alternatives
This works with sh and derivates: KornShell and Bash. It doesn't create any sub-shell. $(..) and `...` commands create a sub-shell; they fork a process, and therefore are slower than this solution.
Like this in Bash (test files containing pattern):
shopt -s nullglob
compgen -W *pattern* &>/dev/null
case $? in
0) echo "only one file match" ;;
1) echo "more than one file match" ;;
2) echo "no file match" ;;
esac
It's far better than compgen -G: because we can discriminates more cases and more precisely.
It can work with only one wildcard *.
This abomination seems to work:
#!/usr/bin/env bash
shopt -s nullglob
if [ "`echo *py`" != "" ]; then
echo "Glob matched"
else
echo "Glob did not match"
fi
It probably requires bash, not sh.
This works because the nullglob option causes the glob to evaluate to an empty string if there are no matches. Thus any non-empty output from the echo command indicates that the glob matched something.
A solution for extended globs (extglob) in Bash:
bash -c $'shopt -s extglob \n /bin/ls -1U <ext-glob-pattern>'
Exit status is 0 if there is at least one match, and non-zero (2) when there is no match. Standard output contains a newline-separated list of matching files (and file names containing spaces they are quoted).
Or, slightly different:
bash -c $'shopt -s extglob \n compgen -G <ext-glob-pattern>'
Differences to the ls-based solution: probably faster (not measured), file names with spaces not quoted in output, exit code 1 when there is no match (not 2 :shrug:).
Example usage:
No match:
$ bash -c $'shopt -s extglob \n /bin/ls -1U #(*.foo|*.bar)'; echo "exit status: $?"
/bin/ls: cannot access '#(*.foo|*.bar)': No such file or directory
exit status: 2
At least one match:
$ bash -c $'shopt -s extglob \n /bin/ls -1U #(*.ts|*.mp4)'; echo "exit status: $?"
'video1 with spaces.mp4'
video2.mp4
video3.mp4
exit status: 0
Concepts used:
ls' exit code behavior (adds -U for efficiency, and -1 for output control).
Does not enable extglob in current shell (often not desired).
Makes use of the $ prefix so that the \n is interpreted, so that the extended glob pattern is on a different line than the shopt -s extglob -- otherwise the extended glob pattern would be a syntax error!
Note 1: I worked towards this solution because the compgen -G "<glob-pattern>" approach suggested in other answers does not seem to work smoothly with brace expansion; and yet I needed some more advanced globbing features.
Note 2: lovely resource for the extended glob syntax: extglob
Both nullglob and compgen are useful only on some bash shells.
A (non-recursive) solution that works on most shells is:
set -- ./glob* # or /path/dir/glob*
[ -f "$1" ] || shift # remove the glob if present.
if [ "$#" -lt 1 ]
then echo "at least one file found"
fi
Assuming you may want to do something with the files if they exist:
mapfile -t exists < <(find "$dirName" -type f -iname '*.zip'); [[ ${#exists} -ne 0 ]] && { echo "Zip files found" ; } || { echo "Zip files not found" ; }
You can then loop through the exists array if you need to do something with the files.
(ls glob* &>/dev/null && echo Files found) || echo No file found
if ls -d $glob > /dev/null 2>&1; then
echo Found.
else
echo Not found.
fi
Note that this can be very time cosuming if there are a lot of matches or file access is slow.
ls | grep -q "glob.*"
Not the most efficient solution (if there's a ton of files in the directory it might be slowish), but it's simple, easy to read and also has the advantage that regexes are more powerful than plain Bash glob patterns.
[ `ls glob* 2>/dev/null | head -n 1` ] && echo true

Test -d directory true - subdirectory false (POSIX)

I'm trying to print all directories/subdirectories from a given start directory.
for i in $(ls -A -R -p); do
if [ -d "$i" ]; then
printf "%s/%s \n" "$PWD" "$i"
fi
done;
This script returns all of the directories found in the . directory and all of the files in that directory, but for some reason the test fails for subdirectories. All of the directories end up in $i and the output looks exactly the same.
Let's say I have the following structure:
foo/bar/test
echo $i prints
foo/
bar/
test/
While the contents of the folders are listed like this:
./foo:
file1
file2
./bar:
file1
file2
However the test statement just prints:
PWD/TO/THIS/DIRECTORY/foo
For some reason it returns true for the first level directories, but false for all of the subdirectories.
(ls is probably not a good way of doing this and I would be glad for a find statement that solves all of my issues, but first I want to know why this script doesn't work the way you'd think.)
As pointed out in the comments, the issue is that the directory names include a :, so -d is false.
I guess that this command gives you the output you want (although it requires Bash):
# enable globstar for **
# disabled in non-interactive shell (e.g. a script)
shopt -s globstar
# print each path ending in a / (all directories)
# ** expands recursively
printf '%s\n' **/*/
The standard way would either to do the recursion yourself, or to use find:
find . -type d
Consider your output:
dir1:
dir1a
Now, the following will be true:
[ -d dir1/dir1a ]
but that's not what your code does; instead, it runs:
[ -d dir1a ]
To avoid this, don't attempt to parse ls; if you want to implement recursion in baseline POSIX sh, do it yourself:
callForEachEntry() {
# because calling this without any command provided would try to execute all found files
# as commands, checking for safe/correct invocation is essential.
if [ "$#" -lt 2 ]; then
echo "Usage: callForEachEntry starting-directory command-name [arg1 arg2...]" >&2
echo " ...calls command-name once for each file recursively found" >&2
return 1
fi
# try to declare variables local, swallow/hide error messages if this fails; code is
# defensively written to avoid breaking if recursing changes either, but may be faulty if
# the command passed as an argument modifies "dir" or "entry" variables.
local dir entry 2>/dev/null ||: "not strict POSIX, but available in dash"
dir=$1; shift
for entry in "$dir"/*; do
# skip if the glob matched nothing
[ -e "$entry" ] || [ -L "$entry" ] || continue
# invoke user-provided callback for the entry we found
"$#" "$entry"
# recurse last for if on a baseline platform where the "local" above failed.
if [ -d "$entry" ]; then
callForEachEntry "$entry" "$#"
fi
done
}
# call printf '%s\n' for each file we recursively find; replace this with the code you
# actually want to call, wrapped in a function if appropriate.
callForEachEntry "$PWD" printf '%s\n'
find can also be used safely, but not as a drop-in replacement for the way ls was used in the original code -- for dir in $(find . -type d) is just as buggy. Instead, see the "Complex Actions" and "Actions In Bulk" section of Using Find.

Resources