bash - recursive script can't see files in sub directory - bash

I got a recursive script which iterates a list of names, some of which are files and some are directories.
If it's a (non-empty) directory, I should call the script again with all of the files in the directory and check if they are legal.
The part of the code making the recursive call:
if [[ -d $var ]] ; then
if [ "$(ls -A $var)" ]; then
./validate `ls $var`
fi
fi
The part of code checking if the files are legal:
if [[ -f $var ]]; then
some code
fi
But, after making the recursive calls, I can no longer check any of the files inside that directory, because they are not in the same directory as the main script, the -f $var if cannot see them.
Any suggestion how can I still see them and use them?

Why not use find? Simple and easy solution to the problem.

Always quote variables, you never known when you will find a file or directory name with spaces
shopt -s nullglob
if [[ -d "$path" ]] ; then
contents=( "$path"/* )
if (( ${#contents[#]} > 0 )); then
"$0" "${contents[#]}"
fi
fi
you're re-inventing find
of course, var is a lousy variable name
if you're recursively calling the script, you don't need to hard-code the script name.
you should consider putting the logic into a function in the script, and the function can recursively call itself, instead of having to spawn an new process to invoke the shell script each time. If you do this, use $FUNCNAME instead of "$0"

A few people have mentioned how find might solve this problem, I just wanted to show how that might be done:
find /yourdirectory -type f -exec ./validate {} +;
This will find all regular files in yourdirectory and recursively in all its sub-directories, and return their paths as arguments to ./validate. The {} is expanded to the paths of the files that find locates within yourdirectory. The + at the end means that each call to validate will be on a large number of files, instead of calling it individually on each file (wherein the + is replaced with a \), this provides a huge speedup sometimes.

One option is to change directory (carefully) into the sub-directory:
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec ./validate $(ls))
fi
fi
The outer parentheses start a new shell so the cd command does not affect the main shell. The exec replaces the original shell with (a new copy of) the validate script. Using $(...) instead of back-ticks is sensible. In general, it is sensible to enclose variable names in double quotes when they refer to file names that might contain spaces (but see below). The $(ls) will list the files in the directory.
Heaven help you with the ls commands if any file names or directory names contain spaces; you should probably be using * glob expansion instead. Note that a directory containing a single file with a name such as -n would trigger a syntax error in your script.
Corrigendum
As Jens noted in a comment, the location of the shell script (validate) has to be adjusted as you descend the directory hierarchy. The simplest mechanism is to have the script on your PATH, so you can write exec validate or even exec $0 instead of exec ./validate. Failing that, you need to adjust the value of $0 — assuming your shell leaves $0 as a relative path and doesn't mess around with converting it to an absolute path. So, a revised version of the code fragment might be:
# For validate on PATH or absolute name in $0
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec $0 $(ls))
fi
fi
or:
# For validate not on PATH and relative name in $0
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec ../$0 $(ls))
fi
fi

Related

Bash script that checks for parts of current folderpath

Clean and simple: how do I check with bash for certain parts of the folder I'm currently in?
#!/usr/bin/sh
CURRENTFOLDER=$(pwd)
echo "${CURRENTFOLDER}"
CHECKFOLDER="/home/*/domains/*/public_html"
if [ $CURRENTFOLDER ! $CHECKFOLDER ]
then
echo "Current folder is not /home/user/domains/domain.com/public_html"
exit
fi
User and domain are variable, I don't need to know them for this checkup, just the 3 pre-defined folders in the variable CHECKFOLDER
There's a problem with this approach.
For example in bash the following expression evaluates to true:
[[ /www/user/domains/local/public_html == /www/*/public_html ]]
It is more accurate to use a bash regex:
[[ /www/user/domains/local/public_html =~ ^/www/[^/]+/public_html$ ]]
So your code would become:
#!/bin/bash
current_folder=$PWD
check_folder='^/home/[^/]+/domains/[^/]+/public_html$'
if ! [[ $current_folder =~ $check_folder ]]
then
echo "Current folder is not /home/user/domains/domain.com/public_html"
exit
fi
BTW, the shebang needs to be a bash, not sh. And it's kind of dangerous to capitalize your variables.
Try this (almost) Shellcheck-clean code:
#! /usr/bin/sh
curr_phpath=''
for phpath in /home/*/domains/*/public_html/; do
if [ "$phpath" -ef . ]; then
curr_phpath=$phpath
break
fi
done
if [ -z "$curr_phpath" ]; then
echo "Current folder is not /home/user/domains/domain.com/public_html" >&2
exit 1
fi
Because of aliasing mechanisms (e.g. symbolic links, bind mounts) it is very difficult in general to determine if two paths reference the same file or directory by comparing them textually. See How to check if two paths are equal in Bash? for more information. This solution uses a more reliable mechanism to determine if the current directory is one of the valid ones.
Since the shebang line references sh instead of bash, the code avoids Bashisms. It's been tested with both bash and dash (probably the most common non-Bash sh).
See Correct Bash and shell script variable capitalization for an explanation of why the code does not use ALL_UPPERCASE variable names.
The [ "$phpath" -ef . ] test is true if the .../public_html path being checked is the same directory as the current directory. The -ef operator is not in POSIX so it is not guaranteed to be supported by an sh shell, and Shellcheck (correctly) warns about it. However, it is supported in both bash and dash, and sh is usually one of those (on Linux at least).
You can save a step just by changing to the directory instead of checking.
Check your glob matches only one file first.
Then, cd to check it's a dir.
#! /bin/bash
IFS="$(printf '\n\t')"
files=( $(compgen -G '/home/*/domains/*/public_html') )
if [[ "${#files[#]}" != 1 ]]
then
printf 'Multiple matches\n' >&2
exit 1
fi
if ! cd "${files[0]}"
then
printf 'Cannot chdir\n'
exit 1
fi

Test -d directory true - subdirectory false (POSIX)

I'm trying to print all directories/subdirectories from a given start directory.
for i in $(ls -A -R -p); do
if [ -d "$i" ]; then
printf "%s/%s \n" "$PWD" "$i"
fi
done;
This script returns all of the directories found in the . directory and all of the files in that directory, but for some reason the test fails for subdirectories. All of the directories end up in $i and the output looks exactly the same.
Let's say I have the following structure:
foo/bar/test
echo $i prints
foo/
bar/
test/
While the contents of the folders are listed like this:
./foo:
file1
file2
./bar:
file1
file2
However the test statement just prints:
PWD/TO/THIS/DIRECTORY/foo
For some reason it returns true for the first level directories, but false for all of the subdirectories.
(ls is probably not a good way of doing this and I would be glad for a find statement that solves all of my issues, but first I want to know why this script doesn't work the way you'd think.)
As pointed out in the comments, the issue is that the directory names include a :, so -d is false.
I guess that this command gives you the output you want (although it requires Bash):
# enable globstar for **
# disabled in non-interactive shell (e.g. a script)
shopt -s globstar
# print each path ending in a / (all directories)
# ** expands recursively
printf '%s\n' **/*/
The standard way would either to do the recursion yourself, or to use find:
find . -type d
Consider your output:
dir1:
dir1a
Now, the following will be true:
[ -d dir1/dir1a ]
but that's not what your code does; instead, it runs:
[ -d dir1a ]
To avoid this, don't attempt to parse ls; if you want to implement recursion in baseline POSIX sh, do it yourself:
callForEachEntry() {
# because calling this without any command provided would try to execute all found files
# as commands, checking for safe/correct invocation is essential.
if [ "$#" -lt 2 ]; then
echo "Usage: callForEachEntry starting-directory command-name [arg1 arg2...]" >&2
echo " ...calls command-name once for each file recursively found" >&2
return 1
fi
# try to declare variables local, swallow/hide error messages if this fails; code is
# defensively written to avoid breaking if recursing changes either, but may be faulty if
# the command passed as an argument modifies "dir" or "entry" variables.
local dir entry 2>/dev/null ||: "not strict POSIX, but available in dash"
dir=$1; shift
for entry in "$dir"/*; do
# skip if the glob matched nothing
[ -e "$entry" ] || [ -L "$entry" ] || continue
# invoke user-provided callback for the entry we found
"$#" "$entry"
# recurse last for if on a baseline platform where the "local" above failed.
if [ -d "$entry" ]; then
callForEachEntry "$entry" "$#"
fi
done
}
# call printf '%s\n' for each file we recursively find; replace this with the code you
# actually want to call, wrapped in a function if appropriate.
callForEachEntry "$PWD" printf '%s\n'
find can also be used safely, but not as a drop-in replacement for the way ls was used in the original code -- for dir in $(find . -type d) is just as buggy. Instead, see the "Complex Actions" and "Actions In Bulk" section of Using Find.

How do I check whether a file or file directory exist in bash?

I currently have this bash script (which is located in my home directory, i.e., /home/username/ and I am running it as root as it's necessary for the icon copying lines):
cd /home/username/Pictures/Icon*
declare -a A={Arch,Debian,Fedora,Mageia,Manjaro,OpenSUSE}
declare -a B={Adwaita,Faenza,gnome,Humanity}
for i in $A; do
for j in $B; do
if test -e /usr/share/icons/$j/scalable ; else
mkdir /usr/share/icons/$j/scalable/
fi
if test -e /usr/share/icons/$j/scalable/$i.svg ; else
cp -a $i*.svg /usr/share/icons/$j/scalable/$i.svg
fi
done
done
What I want this script to do is to copy icons from my Pictures/Icons and logos directory to the scalable theme (specified in $B) subdirectories in /usr/share/icons. Before it does this, however, I'd like it to create a scalable directory in these theme subdirectories if it does not already exist. The problem is that the else part of the conditionals is not being read properly, as I keep receiving this error:
./copyicon.sh: line 8: syntax error near unexpected token `else'
./copyicon.sh: line 8: ` if test -e /usr/share/icons/$j/scalable ; else'
If you're wondering why the test -e ... in the conditional it's based on a textbook on bash scripting I've been following.
Checking file and/or directory existence
To check whether a file exists in bash, you use the -f operator. For directories, use -d. Example usage:
$ mkdir dir
$ [ -d dir ] && echo exists!
exists!
$ rmdir dir
$ [ -d dir ] && echo exists!
$ touch file
$ [ -f file ] || echo "doesn't exist..."
$ rm file
$ [ -f file ] || echo "doesn't exist..."
doesn't exist...
For more information simply execute man test.
A note on -e, this test operator checks whether a file exists. While this may seem like a good choice, it's better to use -f which will return false if the file isn't a regular file. /dev/null for example is a file but nor a regular file. Having the check return true is undesired in this case.
A note on variables
Be sure to quote variables too, once you have a space or any other special character contained in a variable it can have undesired side effects. So when you test for existence of files and directories, wrap the file/dir in double quotes. Something like [ -f "/path/to/some/${dir}/" ] will work while the following would fail if there is a space in dir: [ -f /path/to/some/${dir}/ ].
Fixing the syntax error
You are experiencing a syntax error in the control statements. A bash if clause is structured as following:
if ...; then
...
fi
Or optional with an else clause:
if ...; then
...
else
...
fi
You cannot omit the then clause. If you wish to only use the else clause you should negate the condition. Resulting in following code:
if [ ! -f "/usr/share/icons/$j/scalable" ]; then
mkdir "/usr/share/icons/$j/scalable/"
fi
Here we add an exclamation point (!) to flip the expression's evaluation. If the expression evaluates to true, the same expression preceded by ! will return false and the other way around.
You can't skip the then part of the if statement, easiest solution would be to just negate the test
if [[ ! -e /usr/share/icons/${j}/scalable ]] ; then
mkdir /usr/share/icons/${j}/scalable/
fi
if [[ ! -e /usr/share/icons/${j}/scalable/${i}.svg ]] ; then
cp -a ${i}*.svg /usr/share/icons/${j}/scalable/${i}.svg
fi
I left it with -e (exists), but you might consider using -d for directories or -f for files and some error handling to catch stuff (e.g. /usr/share/icons/$j/scalable/ exists, but is a file and not a directory for whatever reason.)
I also noticed that in your original code you are potentially trying to copy multiple files into one:
cp -a $i*.svg /usr/share/icons/$j/scalable/$i.svg
I left it that way in my example in case you are sure that it is always only one file and are intentionally renaming it. If not I'd suggest only specifying a target directory.

Shell script to browse one or more directories passed as parameters

I made this script that should receive one or more parameter, and those parameter are all directories, and it has to browse those directories (one by one) and do some operations.
The operations work fine if the parameter is 1 (only one directory),
How should I modify my script to make it works if more than 1 parameter is passed
Example if I want it to do the same operations in 2 or 3 directories at the same time?
Thanks
#!/bin/sh
cd $1
for file in ./* # */
do
if [[ -d $file ]]
then
ext=dir
else
ext="${file##*.}"
fi
mv "${file}" "${file}.$ext"
done
First, if you are using bash use bash shebang (#! /bin/bash).
Then use
#! /bin/bash
for d in "$#"
do
echo "Do something with $d"
done
to iterate over the command line arguments (dirs in your case)
#!/bin/sh
for dir in "$#"; do
for file in "$dir"/*; do
echo "Doing something with '$file'"
done
done

List directory and find files with certain strings in the name

I am fairly new to shell scripting, so go easy on me please as I know this is most likely something real basic. My question is this, I need to write a script that will look at a directory and tell me if it finds a match for a string that I specify in the filenames. Here would be an example.
I have a directory named tmp. Inside that directory are files named tmp-a, temp-a, temporary-a, etc. If the script looks at the directory and sees that there is a filename with the string of 'tmp', or 'temp' it should continue with the script, but if it does not see any filenames matching a string specified in the shell script it should quit. I am basically looking for a conditional 'if [ -f filename ]' statement that can apply 'or'.
I hope that made sense and as always, thanks in advance.
Tim
The pattern tmp* expands to the list of files whose name begins with tmp, or to the single-word list consisting of the literal pattern itself if there is no matching file.
set --
for pattern in 'tmp*' 'temp*'; do
set -- $pattern "$#"
if [ ! -e "$1" ]; then shift; fi
done
if [ $# -eq 0 ]; then
echo "No matching file"
else
for x in "$#"; do …; done
fi
In bash, you can request the expansion of a pattern that matches no file to be the empty list, which simplifies matters a lot.
shopt -s nullglob
set -- tmp* temp*
if [ $# -eq 0 ]; then …
The same thing goes for zsh, which allows this to be set per-pattern.
set -- tmp*(N) temp*(N)
if [ $# -eq 0 ]; then …
If you wish to search recursively inside directories, you can use the find command.
if [ -n "$(find -name 'tmp*' -o -name 'temp*' | head -c 1)" ]; then
# there are matching files
fi

Resources