I would like to loop through files in a directory that match a particular filename pattern. At the moment, I am using the following piece of code as a point of reference:
shopt -s nullglob
while :; do
files=("/home/methuselah"/${filePattern})
if [ ${#files} -gt 0 ]; then
for file in "${files[#]}"; do
# something to be done here
done
fi
done
The issue with this is that it will loop indefinitely. I want it to stop as soon as its gone through the files available in the directory. How do I impose this restriction?
Simplify it to just the inner loop.
shopt -s nullglob
files=( /home/methuselah/$filePattern )
for file in "${files[#]}"; do
# something to be done here
done
Related
I have a loop and I need it to ignore empty directories.
for i in */*/
do
cd "$i"
mv ./*.py ..
cd -
rm -r "$i"
done
What can i add on to make it ignore empty directories?
I have this but I would like something simpler
x=$(shopt -s nullglob dotglob; echo "$i"/*)
(( ${#x} )) || continue
What can i add on to make it ignore empty directories?
Bash does not have a primitive operator for testing whether a directory is empty. The best alternative in your case is probably to test whether pathname expansion matches any files within. That's what you are already considering, though I would write it differently.
As a general rule, I would also avoid changing the working directory. If you must change directory then consider doing it in a subshell, so that you need only let the subshell terminate to revert to the original working directory. Using a subshell is also a good approach when different parts of your script want different shell options.
I would probably write your script like this:
#!/bin/bash
shopt -s nullglob dotglob
for i in */*/; do
anyfiles=( "$i"/* )
if [[ ${#anyfiles[#]} -ne 0 ]]; then
# Process nonempty directory "$i"
# If there are any Python files within then move them to the parent directory
pyfiles=( "$i"/*.py )
if [[ ${#pyfiles[#]} -ne 0 ]]; then
mv "${pyfiles[#]}" "$(dirname "$i")"
fi
# Remove directory "$i" and any remaining contents
rm -r "$i"
fi
done
If you want that as part of a larger script, then you could put everything from the shopt to the end in a subshell to limit the scope of the shopt.
Alternatively, you could simplify that slightly at the cost of some clarity by using loops to skip the capture of the directory contents into explicit variables:
#!/bin/bash
shopt -s nullglob dotglob
for i in */*/; do
for anyfile in "$i"/*; do
# Process nonempty directory "$i"
# If there are any Python files within then move them to the parent directory
for pyfile in "$i"/*.py; do
mv "$i"/*.py "$(dirname "$i")"
break
done
# Remove directory "$i" and any remaining contents
rm -r "$i"
break
done
done
In that case, each inner loop contains an unconditional break at the end, so at most one iteration will be performed.
I want to move all JSON files created within a jenkins job to a different folder.
It is possible that the job does not create any json file.
In that case the mv command is raising an error and so that job is failing.
How do I prevent mv command from raising error in case no file is found?
Welcome to SO.
Why do you not want the error?
If you just don't want to see the error, then you could always just throw it away with 2>/dev/null, but PLEASE don't do that. Not every error is the one you expect, and this is a debugging nightmare. You could write it to a log with 2>$logpath and then build in logic to read that to make certain it's ok, and ignore or respond accordingly --
mv *.json /dir/ 2>$someLog
executeMyLogParsingFunction # verify expected err is the ONLY err
If it's because you have set -e or a trap in place, and you know it's ok for the mv to fail (which might not be because there is no file!), then you can use this trick -
mv *.json /dir/ || echo "(Error ok if no files found)"
or
mv *.json /dir/ ||: # : is a no-op synonym for "true" that returns 0
see https://www.gnu.org/software/bash/manual/html_node/Conditional-Constructs.html
(If it's failing simply because the mv is returning a nonzero as the last command, you could also add an explicit exit 0, but don't do that either - fix the actual problem rather than patching the symptom. Any of these other solutions should handle that, but I wanted to point out that unless there's a set -e or a trap that catches the error, it shouldn't cause the script to fail unless it's the very last command.)
Better would be to specifically handle the problem you expect without disabling error handling on other problems.
shopt -s nullglob # globs with no match do not eval to the glob as a string
for f in *.json; do mv "$f" /dir/; done # no match means no loop entry
c.f. https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html
or if you don't want to use shopt,
for f in *.json; do [[ -e "$f" ]] && mv "$f" /dir/; done
Note that I'm only testing existence, so that will include any match, including directories, symlinks, named pipes... you might want [[ -f "$f" ]] && mv "$f" /dir/ instead.
c.f. https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html
This is expected behavior -- it's why the shell leaves *.json unexpanded when there are no matches, to allow mv to show a useful error.
If you don't want that, though, you can always check the list of files yourself, before passing it to mv. As an approach that works with all POSIX-compliant shells, not just bash:
#!/bin/sh
# using a function here gives us our own private argument list.
# that's useful because minimal POSIX sh doesn't provide arrays.
move_if_any() {
dest=$1; shift # shift makes the old $2 be $1, the old $3 be $2, etc.
# so, we then check how many arguments were left after the shift;
# if it's only one, we need to also check whether it refers to a filesystem
# object that actually exists.
if [ "$#" -gt 1 ] || [ -e "$1" ] || [ -L "$1" ]; then
mv -- "$#" "$dest"
fi
}
# put destination_directory/ in $1 where it'll be shifted off
# $2 will be either nonexistent (if we were really running in bash with nullglob set)
# ...or the name of a legitimate file or symlink, or the string '*.json'
move_if_any destination_directory/ *.json
...or, as a more bash-specific approach:
#!/bin/bash
files=( *.json )
if (( ${#files[#]} > 1 )) || [[ -e ${files[0]} || -L ${files[0]} ]]; then
mv -- "${files[#]}" destination/
fi
Loop over all json files and move each of them, if it exists, in a oneliner:
for X in *.json; do [[ -e $X ]] && mv "$X" /dir/; done
This a small bash program that is tasked with looking through a directory and counting how many files are in the directory. It's to ignore other directories and only count the files.
Below is my bash code, which seems to fail to count the files specifically in the directory, I say this because if I remove the if statement and just increment the counter the for loop continues to iterate and prints 4 in the counter (this is including directories though). With the if statement it prints this to the console.
folder1 has files
Looking at other questions I think the expression in my if statement is right and I am getting no compilation errors for syntax or another problems.
So I just simply dumbfounded as to why it is not counting the files.
#!/bin/bash
folder=$1
if [ $1 = empty ]; then
folder=empty
counter=0
echo $folder has $counter files
exit
fi
for d in $(ls $folder); do
if [[ -f $d ]]; then
let 'counter++'
fi
done
echo $folder has $counter files
Thank you.
Your entire script could be very well simplified as below with enhancements made. Never use output of ls programmatically. It should be used only in the command-line. The -z construct allows to you assert if the parameter following it is empty or non-empty.
For looping over files, use the default glob expansion provided by the shell. Note the && is a short-hand to do a action when the left-side of the operand returned a true condition, in a way short-hand equivalent of if <condition>; then do <action>; fi
#!/usr/bin/env bash
[ -z "$1" ] && { printf 'invalid argument passed\n' >&2 ; exit 1 ; }
shopt -s nullglob
for file in "$1"/*; do
[ -f "$file" ] && ((count++))
done
printf 'folder %s had %d files\n' "$1" "$count"
This question already has answers here:
How to skip the for loop when there are no matching files?
(2 answers)
Closed 3 years ago.
I'm trying to iterate through all zip files in a bash script (I'm using Cygwin, but I kind of doubt this is a bug with Cygwin).
It looks like this right now:
for z in *.zip
do
echo $z
done
which works well when there are zip files in the folder, and it echos exactly the zip files and nothing but the zip files. However, when I do it on a folder that's empty, it echos *.zip, when I'd rather it echo nothing.
What should I be doing? I don't think the right solution is if [ $z != "*.zip ]... but is it?
This is the expected behavior. From the documentation:
If no matching filenames are found, and the shell option nullglob is disabled, the word is left unchanged. If the nullglob option is set, and no matches are found, the word is removed.
So the solution is to set the nullglob option before the loop:
shopt -s nullglob
for z in *.zip
...
As one of its steps for executing a command, the shell may perform path expansion in which case it will replace a string, such as *.zip with the list of files that match that glob. If there are no such files, then the string is left unchanged. A reasonable solution is:
for z in *.zip
do
[ -f "$z" ] && echo $z
done
[ -f "$z" ] verifies that the file exists and is a regular file. The && means that echo will be executed only if that test is passed.
Either that or turning on the nullglob option.
$ set -x
$ echo *.none
+ echo '*.none'
*.none
$ shopt -s nullglob
+ shopt -s nullglob
$ echo *.none
$ echo *.none
+ echo
I'd like to check of the existence of one or more directories in a Bash script using a wildcard.
I've tried this;
if [ -d app/*management ]
then
for mscript in `ls -d app/*management`
do
...
done
fi
Which works if there is one match but throws the error "binary operator expected".
Any suggestion on a good way to do this?
You can't use -d to check multiple directories at the same time without && (and) in your expressions. I would use this:
for dir in app/*management; do
if [[ -d $dir ]]; then
...
fi
done
You should use globs instead of parsing the output of ls. See the following link for more information: http://mywiki.wooledge.org/ParsingLs
After the glob expands to your list of management directories, the if statement looks like
if [ -d app/1management app/2management ]
Since -d takes only one argument, bash doesn't know what to do with the remaining directories. To bash, it looks like you forgot to include a binary operator.
You can do just do the following:
for mscript in app/*management; do
if [ ! -d $mscript ]; then
continue
fi
...
done
EDIT: As jordanm commented, the following probably isn't necessary, but I'll leave it here for reference, as nullglob is good to know about.
One caveat. If there is a possibility that app/*management won't expand to anything, you need to set the shell option nullglob before your loop, or else "app/*management" will be treated as a literal string, not a shell glob.
if ! shopt nullglob; then
setting_nullglob=1
shopt -qs nullglob
fi
for mscript in app/*management; do
...
done
if [ ${setting_nullglob:-0} = 1 ]; then
unset setting_nullglob
shopt -qu nullglob
fi
Why don't you do it like this:
for mscript in app/*management ; do
if [ -d "$myscript" ] ; then
...
fi
done