This question already has answers here:
How to skip the for loop when there are no matching files?
(2 answers)
Closed 3 years ago.
I'm trying to iterate through all zip files in a bash script (I'm using Cygwin, but I kind of doubt this is a bug with Cygwin).
It looks like this right now:
for z in *.zip
do
echo $z
done
which works well when there are zip files in the folder, and it echos exactly the zip files and nothing but the zip files. However, when I do it on a folder that's empty, it echos *.zip, when I'd rather it echo nothing.
What should I be doing? I don't think the right solution is if [ $z != "*.zip ]... but is it?
This is the expected behavior. From the documentation:
If no matching filenames are found, and the shell option nullglob is disabled, the word is left unchanged. If the nullglob option is set, and no matches are found, the word is removed.
So the solution is to set the nullglob option before the loop:
shopt -s nullglob
for z in *.zip
...
As one of its steps for executing a command, the shell may perform path expansion in which case it will replace a string, such as *.zip with the list of files that match that glob. If there are no such files, then the string is left unchanged. A reasonable solution is:
for z in *.zip
do
[ -f "$z" ] && echo $z
done
[ -f "$z" ] verifies that the file exists and is a regular file. The && means that echo will be executed only if that test is passed.
Either that or turning on the nullglob option.
$ set -x
$ echo *.none
+ echo '*.none'
*.none
$ shopt -s nullglob
+ shopt -s nullglob
$ echo *.none
$ echo *.none
+ echo
Related
I have a loop and I need it to ignore empty directories.
for i in */*/
do
cd "$i"
mv ./*.py ..
cd -
rm -r "$i"
done
What can i add on to make it ignore empty directories?
I have this but I would like something simpler
x=$(shopt -s nullglob dotglob; echo "$i"/*)
(( ${#x} )) || continue
What can i add on to make it ignore empty directories?
Bash does not have a primitive operator for testing whether a directory is empty. The best alternative in your case is probably to test whether pathname expansion matches any files within. That's what you are already considering, though I would write it differently.
As a general rule, I would also avoid changing the working directory. If you must change directory then consider doing it in a subshell, so that you need only let the subshell terminate to revert to the original working directory. Using a subshell is also a good approach when different parts of your script want different shell options.
I would probably write your script like this:
#!/bin/bash
shopt -s nullglob dotglob
for i in */*/; do
anyfiles=( "$i"/* )
if [[ ${#anyfiles[#]} -ne 0 ]]; then
# Process nonempty directory "$i"
# If there are any Python files within then move them to the parent directory
pyfiles=( "$i"/*.py )
if [[ ${#pyfiles[#]} -ne 0 ]]; then
mv "${pyfiles[#]}" "$(dirname "$i")"
fi
# Remove directory "$i" and any remaining contents
rm -r "$i"
fi
done
If you want that as part of a larger script, then you could put everything from the shopt to the end in a subshell to limit the scope of the shopt.
Alternatively, you could simplify that slightly at the cost of some clarity by using loops to skip the capture of the directory contents into explicit variables:
#!/bin/bash
shopt -s nullglob dotglob
for i in */*/; do
for anyfile in "$i"/*; do
# Process nonempty directory "$i"
# If there are any Python files within then move them to the parent directory
for pyfile in "$i"/*.py; do
mv "$i"/*.py "$(dirname "$i")"
break
done
# Remove directory "$i" and any remaining contents
rm -r "$i"
break
done
done
In that case, each inner loop contains an unconditional break at the end, so at most one iteration will be performed.
And I started playing with dmenu and it seems such an automation for almost every thing. Unfortunately I'm not familiar with bash and it should be on my list.
I have a folder for my markdowns with subfolders containing my files. I'm trying to have a script to show them in dmenu while using an alias.
If the path to a file is
/home/user/docs/markdown/practice01/rmd/network.rmd
I would like to have
network
as an option in my dmenu. So when I choose
network -----> /home/user/docs/markdown/practice01/rmd/network.rmd
Here is my broken script. There are a few things I'm missing.
This way I get full path on my dmenu which i don't need. I tried to read about associative arrays but I can't figure it out in bash.
This script works but in case I decide to ESC and exit, still it opens up an empty vim in my directory. Hence, I should know if statements huh!
#!/bin/bash
DMenu=("dmenu -l 10 -i -nb "#eaeaea" -sb "#E53935" -nf "#474747"")
cd ~/docs/markdown/
target=$(find -type f -name '*.rmd' | $DMenu)
st vim "$target"
I made a little example. But the problem is that it is a manual work to add each file, which definitely we don't wanna do right!
#!/bin/bash
declare -A dotfiles
dotfiles[i3]="/home/user/dotfiles/i3/.config/i3/config"
dotfiles[vimrc]="/home/user/dotfiles/vim/.vimrc"
list=("i3\nvimrc")
target=$(echo -e $list | dmenu -i -nb "#eaeaea" -sb "#E53935" -nf "#474747")
st vim "${dotfiles["$target"]}"
Thank you
Associative arrays can be weird... but returning output to a variable makes it easier to manipulate as any other string in bash, as shown in the example below:
prefix="$HOME/git/notes"
suffix=".md"
shopt -s nullglob globstar
item=( "$prefix"/**/*${suffix}) # Search *.md in all dirs/subdirs
item=( "${item[#]#"$prefix"/}" )
item=( "${item[#]%${suffix}}" ) # Removes '.md' string from item name
result=$(printf '%s\n' "${item[#]}" | dmenu)
[[ -n $result ]] || exit # exit if nothing is found
gedit "${prefix}/${result}.md" # Open file by adding again '.md'
When the percent sign (%) is used in the pattern ${variable%substring}, it will return content of the variable with the shortest occurrence of substring deleted from the back of the variable.
Listed below for reference are 2 examples I wrote, one in Bash and the other in Python, for managing pass and markdown notes with dmenu:
dmenu-pass.sh
dmenu-launch.py
Also, listed below are a couple nice articles that might help you out:
The weird, wondrous world of Bash arrays
Advanced Bash-Scripting Guide: Manipulating Strings
Instead of putting some code in an array, use a function!
my_dmenu() {
dmenu -l 10 -i -nb "#eaeaea" -sb "#e53935" -nf "#474747"
}
If your markdown files are all in the same folder (and not in subfolders), you certainly don't need find: use a glob instead! and if your files are in subfolders, use a glob instead (with the globstar shell option).
All in all:
#!/bin/bash
my_dmenu() {
dmenu -l 10 -i -nb "#eaeaea" -sb "#e53935" -nf "#474747"
}
base_dir=~/docs/markdown
# Also, check the return code of cd!
cd "$base_dir" || { echo >&2 "Can't cd to $base_dir. Exiting"; exit 1; }
# Using a glob: use the shell option nullglob
shopt -s nullglob
files=( *.rmd )
# Check that there are some files found:
if (( ${#files[#]} == 0 )); then
echo "No files found. Exiting."
exit 1
fi
# Now we're ready to send the files to dmenu:
chosen_file=$(printf '%s\n' "${files[#]}" | my_dmenu)
# If dmenu returns nothing: don't launch vim!
if [[ ! $chosen_file ]]; then
echo "No files selected. Exiting."
exit 1
fi
# Now you can launch vim!
st vim "$chosen_file"
If you also want to find the *.rmd files in subfolders: use instead:
shopt -s nullglob globstar
files=( **/*.rmd )
Edit to address the requirement in your comment (and the edit of your question):
If you want to strip the .rmd suffix to show in dmenu, use:
chosen_file=$(printf '%s\n' "${files[#]%.rmd}" | my_dmenu)
# ...
st vim "$chosen_file.rmd"
The expansion ${files[#]%.rmd} will strip the suffix .rmd from each field of the array files. Don't forget to add this suffix back when you edit the file (as shown in the last line).
dmenuoptions="-l 10 -i -nb '#eaeaea' -sb '#E53935' -nf '#474747'"
st -e vim $(find ~/docs/markdown -type f -name '*.rmd' | dmenu $dmenuoptions)
I have a folder containing these app files:
mles:fairs-ionic mles$ ls build
build/com.solutions.enterprise.fairs.dev_v2.2.0.17_20170720_1423-enterprise.ipa
build/com.solutions.enterprise.fairs_v2.2.0.17_20170720_1423-enterprise.ipa
build/de.fairs.dev_v2.2.0.17_20170720_1423-enterprise.apk
build/de.fairs_v2.2.0.17_20170720_1423-appstore.ipa
build/de.fairs_v2.2.0.17_20170720_1423-enterprise.apk
I need to uppload all of them except the *-appstore.ipa one. More specifically these one:
mles:fairs-ionic mles$ ls build/{*-enterprise.ipa,*.apk}
build/com.solutions.enterprise.fairs.dev_v2.2.0.17_20170720_1423-enterprise.ipa
build/com.solutions.enterprise.fairs_v2.2.0.17_20170720_1423-enterprise.ipa
build/de.fairs.dev_v2.2.0.17_20170720_1423-enterprise.apk
build/de.fairs_v2.2.0.17_20170720_1423-enterprise.apk
In my bash script I've tried:
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
appFiles=($(cd ${DIR}/build;ls {*-enterprise.ipa,*.apk}))
echo ${appFiles};
echo "loop"
for appFile in "${appFiles[#]}"
do
echo ${appfile}
#app_upload "${appfile}"
done
this yields:
mles:fairs-ionic ben$ ./test.sh
com.solutions.enterprise.fairs.dev_v2.2.0.17_20170720_1423-enterprise.ipa
loop
appFiles only contains one row, and the appfile variable in the loop is always empty.
How can I iterate over all the files in the build folder except the .ipa files with appstore in the filename (build/de.fairs_v2.2.0.17_20170720_1423-appstore.ipa) ?
Variable appfile is not defined only appFile, So change
echo ${appfile}
to
echo ${appFile}
Edit the line displaying all the data
echo ${appFiles};
To
echo ${appFiles[*]};
You can just do this simply with extended glob features (see Options which change globbing behavior) provided by bash, turn the options on and stash the values to an array. Run it outside the build/ folder.
shopt -s extglob nullglob
fileList=( build/!(*-appstore.ipa) )
now loop over the array and do whatever you want to do with it.
for file in "${fileList[#]}"; do
printf "%s\n" "$file"
done
You don't need to create an array of selected files as you can directly iterate them using a glob with brace expansion like this:
for file in build/*{-enterprise.ipa,.apk}; do
echo "$file"
# app_upload "$file"
done
I have a bash script where I loop over all files in a directory using
for x in "$path/"{.,}*; do
do some stuff here
done
This does loop over all the files and directories (this is what I want),
but it also gives me the file .* which does not exist.
I am using a terminal emulator on android, so it could be an error there.
I am looking for a shell solution as I don't have most of the "normal Linux" commands such as sed.
If Your Shell Is Really Bash (as the question originally stated)
Running the following command:
shopt -s nullglob
will disable the default behavior of leaving globs with no matches unexpanded. Note that this can have surprising side effects: With nullglob set, ls -l *.NotFilenamesHaveThisSuffix will be exactly the same as ls -l.
That said, you can do even better:
shopt -s dotglob
for x in "$path/"*; do
printf 'Processing file: %q\n' "$x"
done
will, on account of setting dotglob, find all hidden files without needing the {.,} idiom at all, and will also have the benefit of consistently ignoring . and ...
If Your Shell Is ash Or mksh
It's actually quite usual to find bash on an Android device. If you have a baseline POSIX shell, the following will be more robust.
for x in "$path"/* "$path"/.*; do
[ -e "$x" ] || continue # skip files that don't exist
basename=${x##*/} # find filename w/o path
case $basename in "."|"..") continue ;; esac # skip "." and ".."
echo "Processing $x" >&2
: ...put your logic here...
done
I got a recursive script which iterates a list of names, some of which are files and some are directories.
If it's a (non-empty) directory, I should call the script again with all of the files in the directory and check if they are legal.
The part of the code making the recursive call:
if [[ -d $var ]] ; then
if [ "$(ls -A $var)" ]; then
./validate `ls $var`
fi
fi
The part of code checking if the files are legal:
if [[ -f $var ]]; then
some code
fi
But, after making the recursive calls, I can no longer check any of the files inside that directory, because they are not in the same directory as the main script, the -f $var if cannot see them.
Any suggestion how can I still see them and use them?
Why not use find? Simple and easy solution to the problem.
Always quote variables, you never known when you will find a file or directory name with spaces
shopt -s nullglob
if [[ -d "$path" ]] ; then
contents=( "$path"/* )
if (( ${#contents[#]} > 0 )); then
"$0" "${contents[#]}"
fi
fi
you're re-inventing find
of course, var is a lousy variable name
if you're recursively calling the script, you don't need to hard-code the script name.
you should consider putting the logic into a function in the script, and the function can recursively call itself, instead of having to spawn an new process to invoke the shell script each time. If you do this, use $FUNCNAME instead of "$0"
A few people have mentioned how find might solve this problem, I just wanted to show how that might be done:
find /yourdirectory -type f -exec ./validate {} +;
This will find all regular files in yourdirectory and recursively in all its sub-directories, and return their paths as arguments to ./validate. The {} is expanded to the paths of the files that find locates within yourdirectory. The + at the end means that each call to validate will be on a large number of files, instead of calling it individually on each file (wherein the + is replaced with a \), this provides a huge speedup sometimes.
One option is to change directory (carefully) into the sub-directory:
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec ./validate $(ls))
fi
fi
The outer parentheses start a new shell so the cd command does not affect the main shell. The exec replaces the original shell with (a new copy of) the validate script. Using $(...) instead of back-ticks is sensible. In general, it is sensible to enclose variable names in double quotes when they refer to file names that might contain spaces (but see below). The $(ls) will list the files in the directory.
Heaven help you with the ls commands if any file names or directory names contain spaces; you should probably be using * glob expansion instead. Note that a directory containing a single file with a name such as -n would trigger a syntax error in your script.
Corrigendum
As Jens noted in a comment, the location of the shell script (validate) has to be adjusted as you descend the directory hierarchy. The simplest mechanism is to have the script on your PATH, so you can write exec validate or even exec $0 instead of exec ./validate. Failing that, you need to adjust the value of $0 — assuming your shell leaves $0 as a relative path and doesn't mess around with converting it to an absolute path. So, a revised version of the code fragment might be:
# For validate on PATH or absolute name in $0
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec $0 $(ls))
fi
fi
or:
# For validate not on PATH and relative name in $0
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec ../$0 $(ls))
fi
fi