What to do to make loop ignore empty directories - bash

I have a loop and I need it to ignore empty directories.
for i in */*/
do
cd "$i"
mv ./*.py ..
cd -
rm -r "$i"
done
What can i add on to make it ignore empty directories?
I have this but I would like something simpler
x=$(shopt -s nullglob dotglob; echo "$i"/*)
(( ${#x} )) || continue

What can i add on to make it ignore empty directories?
Bash does not have a primitive operator for testing whether a directory is empty. The best alternative in your case is probably to test whether pathname expansion matches any files within. That's what you are already considering, though I would write it differently.
As a general rule, I would also avoid changing the working directory. If you must change directory then consider doing it in a subshell, so that you need only let the subshell terminate to revert to the original working directory. Using a subshell is also a good approach when different parts of your script want different shell options.
I would probably write your script like this:
#!/bin/bash
shopt -s nullglob dotglob
for i in */*/; do
anyfiles=( "$i"/* )
if [[ ${#anyfiles[#]} -ne 0 ]]; then
# Process nonempty directory "$i"
# If there are any Python files within then move them to the parent directory
pyfiles=( "$i"/*.py )
if [[ ${#pyfiles[#]} -ne 0 ]]; then
mv "${pyfiles[#]}" "$(dirname "$i")"
fi
# Remove directory "$i" and any remaining contents
rm -r "$i"
fi
done
If you want that as part of a larger script, then you could put everything from the shopt to the end in a subshell to limit the scope of the shopt.
Alternatively, you could simplify that slightly at the cost of some clarity by using loops to skip the capture of the directory contents into explicit variables:
#!/bin/bash
shopt -s nullglob dotglob
for i in */*/; do
for anyfile in "$i"/*; do
# Process nonempty directory "$i"
# If there are any Python files within then move them to the parent directory
for pyfile in "$i"/*.py; do
mv "$i"/*.py "$(dirname "$i")"
break
done
# Remove directory "$i" and any remaining contents
rm -r "$i"
break
done
done
In that case, each inner loop contains an unconditional break at the end, so at most one iteration will be performed.

Related

Prevent "mv" command from raising error if no file matches the glob. eg" mv *.json /dir/

I want to move all JSON files created within a jenkins job to a different folder.
It is possible that the job does not create any json file.
In that case the mv command is raising an error and so that job is failing.
How do I prevent mv command from raising error in case no file is found?
Welcome to SO.
Why do you not want the error?
If you just don't want to see the error, then you could always just throw it away with 2>/dev/null, but PLEASE don't do that. Not every error is the one you expect, and this is a debugging nightmare. You could write it to a log with 2>$logpath and then build in logic to read that to make certain it's ok, and ignore or respond accordingly --
mv *.json /dir/ 2>$someLog
executeMyLogParsingFunction # verify expected err is the ONLY err
If it's because you have set -e or a trap in place, and you know it's ok for the mv to fail (which might not be because there is no file!), then you can use this trick -
mv *.json /dir/ || echo "(Error ok if no files found)"
or
mv *.json /dir/ ||: # : is a no-op synonym for "true" that returns 0
see https://www.gnu.org/software/bash/manual/html_node/Conditional-Constructs.html
(If it's failing simply because the mv is returning a nonzero as the last command, you could also add an explicit exit 0, but don't do that either - fix the actual problem rather than patching the symptom. Any of these other solutions should handle that, but I wanted to point out that unless there's a set -e or a trap that catches the error, it shouldn't cause the script to fail unless it's the very last command.)
Better would be to specifically handle the problem you expect without disabling error handling on other problems.
shopt -s nullglob # globs with no match do not eval to the glob as a string
for f in *.json; do mv "$f" /dir/; done # no match means no loop entry
c.f. https://www.gnu.org/software/bash/manual/html_node/The-Shopt-Builtin.html
or if you don't want to use shopt,
for f in *.json; do [[ -e "$f" ]] && mv "$f" /dir/; done
Note that I'm only testing existence, so that will include any match, including directories, symlinks, named pipes... you might want [[ -f "$f" ]] && mv "$f" /dir/ instead.
c.f. https://www.gnu.org/software/bash/manual/html_node/Bash-Conditional-Expressions.html
This is expected behavior -- it's why the shell leaves *.json unexpanded when there are no matches, to allow mv to show a useful error.
If you don't want that, though, you can always check the list of files yourself, before passing it to mv. As an approach that works with all POSIX-compliant shells, not just bash:
#!/bin/sh
# using a function here gives us our own private argument list.
# that's useful because minimal POSIX sh doesn't provide arrays.
move_if_any() {
dest=$1; shift # shift makes the old $2 be $1, the old $3 be $2, etc.
# so, we then check how many arguments were left after the shift;
# if it's only one, we need to also check whether it refers to a filesystem
# object that actually exists.
if [ "$#" -gt 1 ] || [ -e "$1" ] || [ -L "$1" ]; then
mv -- "$#" "$dest"
fi
}
# put destination_directory/ in $1 where it'll be shifted off
# $2 will be either nonexistent (if we were really running in bash with nullglob set)
# ...or the name of a legitimate file or symlink, or the string '*.json'
move_if_any destination_directory/ *.json
...or, as a more bash-specific approach:
#!/bin/bash
files=( *.json )
if (( ${#files[#]} > 1 )) || [[ -e ${files[0]} || -L ${files[0]} ]]; then
mv -- "${files[#]}" destination/
fi
Loop over all json files and move each of them, if it exists, in a oneliner:
for X in *.json; do [[ -e $X ]] && mv "$X" /dir/; done

Stop indefinite loop of files through directory

I would like to loop through files in a directory that match a particular filename pattern. At the moment, I am using the following piece of code as a point of reference:
shopt -s nullglob
while :; do
files=("/home/methuselah"/${filePattern})
if [ ${#files} -gt 0 ]; then
for file in "${files[#]}"; do
# something to be done here
done
fi
done
The issue with this is that it will loop indefinitely. I want it to stop as soon as its gone through the files available in the directory. How do I impose this restriction?
Simplify it to just the inner loop.
shopt -s nullglob
files=( /home/methuselah/$filePattern )
for file in "${files[#]}"; do
# something to be done here
done

Can't iterate correctly over specific files in folder with endings {*-enterprise.ipa,*.apk}

I have a folder containing these app files:
mles:fairs-ionic mles$ ls build
build/com.solutions.enterprise.fairs.dev_v2.2.0.17_20170720_1423-enterprise.ipa
build/com.solutions.enterprise.fairs_v2.2.0.17_20170720_1423-enterprise.ipa
build/de.fairs.dev_v2.2.0.17_20170720_1423-enterprise.apk
build/de.fairs_v2.2.0.17_20170720_1423-appstore.ipa
build/de.fairs_v2.2.0.17_20170720_1423-enterprise.apk
I need to uppload all of them except the *-appstore.ipa one. More specifically these one:
mles:fairs-ionic mles$ ls build/{*-enterprise.ipa,*.apk}
build/com.solutions.enterprise.fairs.dev_v2.2.0.17_20170720_1423-enterprise.ipa
build/com.solutions.enterprise.fairs_v2.2.0.17_20170720_1423-enterprise.ipa
build/de.fairs.dev_v2.2.0.17_20170720_1423-enterprise.apk
build/de.fairs_v2.2.0.17_20170720_1423-enterprise.apk
In my bash script I've tried:
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
appFiles=($(cd ${DIR}/build;ls {*-enterprise.ipa,*.apk}))
echo ${appFiles};
echo "loop"
for appFile in "${appFiles[#]}"
do
echo ${appfile}
#app_upload "${appfile}"
done
this yields:
mles:fairs-ionic ben$ ./test.sh
com.solutions.enterprise.fairs.dev_v2.2.0.17_20170720_1423-enterprise.ipa
loop
appFiles only contains one row, and the appfile variable in the loop is always empty.
How can I iterate over all the files in the build folder except the .ipa files with appstore in the filename (build/de.fairs_v2.2.0.17_20170720_1423-appstore.ipa) ?
Variable appfile is not defined only appFile, So change
echo ${appfile}
to
echo ${appFile}
Edit the line displaying all the data
echo ${appFiles};
To
echo ${appFiles[*]};
You can just do this simply with extended glob features (see Options which change globbing behavior) provided by bash, turn the options on and stash the values to an array. Run it outside the build/ folder.
shopt -s extglob nullglob
fileList=( build/!(*-appstore.ipa) )
now loop over the array and do whatever you want to do with it.
for file in "${fileList[#]}"; do
printf "%s\n" "$file"
done
You don't need to create an array of selected files as you can directly iterate them using a glob with brace expansion like this:
for file in build/*{-enterprise.ipa,.apk}; do
echo "$file"
# app_upload "$file"
done

Iterating through files gives odd results when no files [duplicate]

This question already has answers here:
How to skip the for loop when there are no matching files?
(2 answers)
Closed 3 years ago.
I'm trying to iterate through all zip files in a bash script (I'm using Cygwin, but I kind of doubt this is a bug with Cygwin).
It looks like this right now:
for z in *.zip
do
echo $z
done
which works well when there are zip files in the folder, and it echos exactly the zip files and nothing but the zip files. However, when I do it on a folder that's empty, it echos *.zip, when I'd rather it echo nothing.
What should I be doing? I don't think the right solution is if [ $z != "*.zip ]... but is it?
This is the expected behavior. From the documentation:
If no matching filenames are found, and the shell option nullglob is disabled, the word is left unchanged. If the nullglob option is set, and no matches are found, the word is removed.
So the solution is to set the nullglob option before the loop:
shopt -s nullglob
for z in *.zip
...
As one of its steps for executing a command, the shell may perform path expansion in which case it will replace a string, such as *.zip with the list of files that match that glob. If there are no such files, then the string is left unchanged. A reasonable solution is:
for z in *.zip
do
[ -f "$z" ] && echo $z
done
[ -f "$z" ] verifies that the file exists and is a regular file. The && means that echo will be executed only if that test is passed.
Either that or turning on the nullglob option.
$ set -x
$ echo *.none
+ echo '*.none'
*.none
$ shopt -s nullglob
+ shopt -s nullglob
$ echo *.none
$ echo *.none
+ echo

bash - recursive script can't see files in sub directory

I got a recursive script which iterates a list of names, some of which are files and some are directories.
If it's a (non-empty) directory, I should call the script again with all of the files in the directory and check if they are legal.
The part of the code making the recursive call:
if [[ -d $var ]] ; then
if [ "$(ls -A $var)" ]; then
./validate `ls $var`
fi
fi
The part of code checking if the files are legal:
if [[ -f $var ]]; then
some code
fi
But, after making the recursive calls, I can no longer check any of the files inside that directory, because they are not in the same directory as the main script, the -f $var if cannot see them.
Any suggestion how can I still see them and use them?
Why not use find? Simple and easy solution to the problem.
Always quote variables, you never known when you will find a file or directory name with spaces
shopt -s nullglob
if [[ -d "$path" ]] ; then
contents=( "$path"/* )
if (( ${#contents[#]} > 0 )); then
"$0" "${contents[#]}"
fi
fi
you're re-inventing find
of course, var is a lousy variable name
if you're recursively calling the script, you don't need to hard-code the script name.
you should consider putting the logic into a function in the script, and the function can recursively call itself, instead of having to spawn an new process to invoke the shell script each time. If you do this, use $FUNCNAME instead of "$0"
A few people have mentioned how find might solve this problem, I just wanted to show how that might be done:
find /yourdirectory -type f -exec ./validate {} +;
This will find all regular files in yourdirectory and recursively in all its sub-directories, and return their paths as arguments to ./validate. The {} is expanded to the paths of the files that find locates within yourdirectory. The + at the end means that each call to validate will be on a large number of files, instead of calling it individually on each file (wherein the + is replaced with a \), this provides a huge speedup sometimes.
One option is to change directory (carefully) into the sub-directory:
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec ./validate $(ls))
fi
fi
The outer parentheses start a new shell so the cd command does not affect the main shell. The exec replaces the original shell with (a new copy of) the validate script. Using $(...) instead of back-ticks is sensible. In general, it is sensible to enclose variable names in double quotes when they refer to file names that might contain spaces (but see below). The $(ls) will list the files in the directory.
Heaven help you with the ls commands if any file names or directory names contain spaces; you should probably be using * glob expansion instead. Note that a directory containing a single file with a name such as -n would trigger a syntax error in your script.
Corrigendum
As Jens noted in a comment, the location of the shell script (validate) has to be adjusted as you descend the directory hierarchy. The simplest mechanism is to have the script on your PATH, so you can write exec validate or even exec $0 instead of exec ./validate. Failing that, you need to adjust the value of $0 — assuming your shell leaves $0 as a relative path and doesn't mess around with converting it to an absolute path. So, a revised version of the code fragment might be:
# For validate on PATH or absolute name in $0
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec $0 $(ls))
fi
fi
or:
# For validate not on PATH and relative name in $0
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec ../$0 $(ls))
fi
fi

Resources