I'm currently writing a bash script for executing test suites. Besides passing the suites directly to this script, like
./bash-specs test.suite
it should also be able to execute all scripts in a given directory if no suite is passed to it, like so
./bash-specs # executes all tests in the directory, namely test.suite
This is implemented like this
(($# == 0)) && set -- *.suite
So, if no suite is passed, all the files ending on .suite are executed. This works fine but fails if the directory contains no such files.
That means I will also need a check to test if there actually are files with that ending.
How would I do this in bash?
I thought a test like
[[ -f *.suite ]]
should work but it seems to fail when there are more than one file in the directory.
The reason -f is failing is because -f only takes a single parameter. When you do [[ -f *.suite ]], it expands to:
[[ -f test.suite test2.suite test3.suite ]]
... which is not valid.
Instead, do this:
shopt -s nullglob
FILES=`echo *.suite`
if [[ -z $FILES ]]; then
echo "No suites found"
exit
fi
for i in $FILES; do
# Run your test on file $i
done
nullglob is a shell option that makes wildcard patterns that aren't found expand to nothing, rather than expanding to the wildcard pattern itself. Once $FILES is set to either a list of files or nothing, we can use -z to test for emptiness, and display the appropriate error message.
ls -al | grep "\.suite";echo $?
This will show 0 if file is present and 1 if file is not present
I would iterate over every suite file like this:
for i in *.suite ; do
if [ -x $i ] ; then
echo running $i
fi
done
Right after :
(($# == 0)) && set -- *.suite
Test if $1 is empty (with -z), then it means that there's no file named *.suite.
Related
This a small bash program that is tasked with looking through a directory and counting how many files are in the directory. It's to ignore other directories and only count the files.
Below is my bash code, which seems to fail to count the files specifically in the directory, I say this because if I remove the if statement and just increment the counter the for loop continues to iterate and prints 4 in the counter (this is including directories though). With the if statement it prints this to the console.
folder1 has files
Looking at other questions I think the expression in my if statement is right and I am getting no compilation errors for syntax or another problems.
So I just simply dumbfounded as to why it is not counting the files.
#!/bin/bash
folder=$1
if [ $1 = empty ]; then
folder=empty
counter=0
echo $folder has $counter files
exit
fi
for d in $(ls $folder); do
if [[ -f $d ]]; then
let 'counter++'
fi
done
echo $folder has $counter files
Thank you.
Your entire script could be very well simplified as below with enhancements made. Never use output of ls programmatically. It should be used only in the command-line. The -z construct allows to you assert if the parameter following it is empty or non-empty.
For looping over files, use the default glob expansion provided by the shell. Note the && is a short-hand to do a action when the left-side of the operand returned a true condition, in a way short-hand equivalent of if <condition>; then do <action>; fi
#!/usr/bin/env bash
[ -z "$1" ] && { printf 'invalid argument passed\n' >&2 ; exit 1 ; }
shopt -s nullglob
for file in "$1"/*; do
[ -f "$file" ] && ((count++))
done
printf 'folder %s had %d files\n' "$1" "$count"
I want to know if my file exists in any of the sub directories below. The sub directories are created in the steps above in my shell script, the below code always tells me the file do not exist (even if it does) and I want the path to be printed as well.
#!/bin/bash
....
if ! [[ -e [ **/**/somefile.txt && -s **/**/somefile.txt ]]; then
echo "===> Warn: somefile.txt was not created in the following path: "
# I want to be able to print the path in which file is not generated
exit 1
fi
I know the file name is somefile.txt which is to be created in all sub-directories, but the subdirectory names change a lot.. Hence globbing.
#!/bin/bash
shopt -s extglob ## enable **, which by default has no special behavior
for d in **/; do
if ! [[ -s "$d/somefile.txt" ]]; then
echo "===> WARN: somefile.txt was not created (or is empty) in $d" >&2
exit 1
fi
done
The issue that I have is with the line: "does not work" - below. The last line does indeed work - but I need to understand why the second to last line does not. I need to check for file existence on the remote server.
Have a need to check for existence for files at the following location:
/home/remoteuser/files/
and when the files are processed, they are moved to:
/home/remoteuser/logs/archive/
Would like to create an alert if the files exist at - in other words, the files were not processed:
/home/remoteuser/logs/
Found the following page and seems to be what I am looking for:
http://www.cyberciti.biz/tips/find-out-if-file-exists-with-conditional-expressions.html
Testing this and I know there are files there, but does not work:
ssh remoteuser#1.2.3.4 [ ! -f /home/remoteuser/logs/archive/*.* ] && echo "File does not exist in the root" >> /home/localuser/files/dirlist.txt
Because we know this works and does indeed list files on the local server:
ssh remoteuser#1.2.3.4 ls -l /home/remoteuser/logs/archive/*.* >> /home/localuser/files/dirlist.txt
Wildcards and test construct in Bash
You cannot use the wildcards in the [ command to test the existence of multiple files. In fact, the wildcards will be expanded and all the files will be passed to the test. Te results is that it would complain that "-f" is given too many arguments.
Try this in any non empty directory to see the output:
[ ! -f *.* ]
The only situation in which the above command does not fail is when there is only one file matching the expression, in your case a non hidden file of the form "*.*" in /home/remoteuser/logs/archive/
Using Find
A possible solution is to use find in combination with grep:
ssh name#server find /path/to/the/files -type f -name "\*.\*" 2>/dev/null | grep -q . && echo "Not Empty" || echo "Empty"
find search for regular files (-type f) whose names are in the form . (-name) and return false if nothing is found, then "grep -q ." return 1 or 0 if something is found or not.
Your goal can be accomplished with only shell builtins -- and without any uses of those builtins which depend on their behavior when passed invalid syntax (as the [ ! -e *.* ] approach does). This removes the dependency on having an accessible, working find command on your remote system.
Consider:
rmtfunc() {
set -- /home/remoteuser/logs/*.* # put contents of directory into $# array
for arg; do # ...for each item in that array...
[ -f "$arg" ] && exit 0 # ...if it's a file that exists, success
done
exit 1 # if nothing matched above, failure
}
# emit text that defines that function into the ssh command, then run same
if ssh remoteuser#host "$(declare -f rmtfunc); rmtfunc"; then
echo "Found remote logfiles"
else
echo "No remote logfiles exist"
fi
ANSWER:
Did find the following about the use of -e for a regular file.
http://www.cyberciti.biz/faq/unix-linux-test-existence-of-file-in-bash/
Even though it says "too many arguments" it does seem to test out OK.
ssh remoteuser#1.2.3.4 [ ! -e /home/remoteuser/logs/archive/*.zip ] && echo "File does not exists in the root" >> /home/localuser/files/dirlist.txt || echo "File does exists in the root" >> /home/localuser/files/dirlist.txt
Your script will work simply using double parenthesis:
ssh remoteuser#1.2.3.4 [[ ! -f /home/remoteuser/logs/archive/*.* ]] && echo "File does not exist in the root" >> /home/localuser/files/dirlist.txt
From man bash
Word splitting and pathname expansion are not performed on the words between the [[ and ]].
I currently have this bash script (which is located in my home directory, i.e., /home/username/ and I am running it as root as it's necessary for the icon copying lines):
cd /home/username/Pictures/Icon*
declare -a A={Arch,Debian,Fedora,Mageia,Manjaro,OpenSUSE}
declare -a B={Adwaita,Faenza,gnome,Humanity}
for i in $A; do
for j in $B; do
if test -e /usr/share/icons/$j/scalable ; else
mkdir /usr/share/icons/$j/scalable/
fi
if test -e /usr/share/icons/$j/scalable/$i.svg ; else
cp -a $i*.svg /usr/share/icons/$j/scalable/$i.svg
fi
done
done
What I want this script to do is to copy icons from my Pictures/Icons and logos directory to the scalable theme (specified in $B) subdirectories in /usr/share/icons. Before it does this, however, I'd like it to create a scalable directory in these theme subdirectories if it does not already exist. The problem is that the else part of the conditionals is not being read properly, as I keep receiving this error:
./copyicon.sh: line 8: syntax error near unexpected token `else'
./copyicon.sh: line 8: ` if test -e /usr/share/icons/$j/scalable ; else'
If you're wondering why the test -e ... in the conditional it's based on a textbook on bash scripting I've been following.
Checking file and/or directory existence
To check whether a file exists in bash, you use the -f operator. For directories, use -d. Example usage:
$ mkdir dir
$ [ -d dir ] && echo exists!
exists!
$ rmdir dir
$ [ -d dir ] && echo exists!
$ touch file
$ [ -f file ] || echo "doesn't exist..."
$ rm file
$ [ -f file ] || echo "doesn't exist..."
doesn't exist...
For more information simply execute man test.
A note on -e, this test operator checks whether a file exists. While this may seem like a good choice, it's better to use -f which will return false if the file isn't a regular file. /dev/null for example is a file but nor a regular file. Having the check return true is undesired in this case.
A note on variables
Be sure to quote variables too, once you have a space or any other special character contained in a variable it can have undesired side effects. So when you test for existence of files and directories, wrap the file/dir in double quotes. Something like [ -f "/path/to/some/${dir}/" ] will work while the following would fail if there is a space in dir: [ -f /path/to/some/${dir}/ ].
Fixing the syntax error
You are experiencing a syntax error in the control statements. A bash if clause is structured as following:
if ...; then
...
fi
Or optional with an else clause:
if ...; then
...
else
...
fi
You cannot omit the then clause. If you wish to only use the else clause you should negate the condition. Resulting in following code:
if [ ! -f "/usr/share/icons/$j/scalable" ]; then
mkdir "/usr/share/icons/$j/scalable/"
fi
Here we add an exclamation point (!) to flip the expression's evaluation. If the expression evaluates to true, the same expression preceded by ! will return false and the other way around.
You can't skip the then part of the if statement, easiest solution would be to just negate the test
if [[ ! -e /usr/share/icons/${j}/scalable ]] ; then
mkdir /usr/share/icons/${j}/scalable/
fi
if [[ ! -e /usr/share/icons/${j}/scalable/${i}.svg ]] ; then
cp -a ${i}*.svg /usr/share/icons/${j}/scalable/${i}.svg
fi
I left it with -e (exists), but you might consider using -d for directories or -f for files and some error handling to catch stuff (e.g. /usr/share/icons/$j/scalable/ exists, but is a file and not a directory for whatever reason.)
I also noticed that in your original code you are potentially trying to copy multiple files into one:
cp -a $i*.svg /usr/share/icons/$j/scalable/$i.svg
I left it that way in my example in case you are sure that it is always only one file and are intentionally renaming it. If not I'd suggest only specifying a target directory.
I got a recursive script which iterates a list of names, some of which are files and some are directories.
If it's a (non-empty) directory, I should call the script again with all of the files in the directory and check if they are legal.
The part of the code making the recursive call:
if [[ -d $var ]] ; then
if [ "$(ls -A $var)" ]; then
./validate `ls $var`
fi
fi
The part of code checking if the files are legal:
if [[ -f $var ]]; then
some code
fi
But, after making the recursive calls, I can no longer check any of the files inside that directory, because they are not in the same directory as the main script, the -f $var if cannot see them.
Any suggestion how can I still see them and use them?
Why not use find? Simple and easy solution to the problem.
Always quote variables, you never known when you will find a file or directory name with spaces
shopt -s nullglob
if [[ -d "$path" ]] ; then
contents=( "$path"/* )
if (( ${#contents[#]} > 0 )); then
"$0" "${contents[#]}"
fi
fi
you're re-inventing find
of course, var is a lousy variable name
if you're recursively calling the script, you don't need to hard-code the script name.
you should consider putting the logic into a function in the script, and the function can recursively call itself, instead of having to spawn an new process to invoke the shell script each time. If you do this, use $FUNCNAME instead of "$0"
A few people have mentioned how find might solve this problem, I just wanted to show how that might be done:
find /yourdirectory -type f -exec ./validate {} +;
This will find all regular files in yourdirectory and recursively in all its sub-directories, and return their paths as arguments to ./validate. The {} is expanded to the paths of the files that find locates within yourdirectory. The + at the end means that each call to validate will be on a large number of files, instead of calling it individually on each file (wherein the + is replaced with a \), this provides a huge speedup sometimes.
One option is to change directory (carefully) into the sub-directory:
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec ./validate $(ls))
fi
fi
The outer parentheses start a new shell so the cd command does not affect the main shell. The exec replaces the original shell with (a new copy of) the validate script. Using $(...) instead of back-ticks is sensible. In general, it is sensible to enclose variable names in double quotes when they refer to file names that might contain spaces (but see below). The $(ls) will list the files in the directory.
Heaven help you with the ls commands if any file names or directory names contain spaces; you should probably be using * glob expansion instead. Note that a directory containing a single file with a name such as -n would trigger a syntax error in your script.
Corrigendum
As Jens noted in a comment, the location of the shell script (validate) has to be adjusted as you descend the directory hierarchy. The simplest mechanism is to have the script on your PATH, so you can write exec validate or even exec $0 instead of exec ./validate. Failing that, you need to adjust the value of $0 — assuming your shell leaves $0 as a relative path and doesn't mess around with converting it to an absolute path. So, a revised version of the code fragment might be:
# For validate on PATH or absolute name in $0
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec $0 $(ls))
fi
fi
or:
# For validate not on PATH and relative name in $0
if [[ -d "$var" ]] ; then
if [ "$(ls -A $var)" ]; then
(cd "$var"; exec ../$0 $(ls))
fi
fi