I'm writing a bash script that checks the number of files in a directory and if it's over a certain number, do something. The code is here:
DIR=/home/inventory
waiting="$((ls ${DIR}/waiting/ -1 | wc -l))"
echo $waiting
if [ $waiting -gt 3 ]
then
(DO SOME STUFF HERE)
fi
The error I am getting is this line....
waiting="$((ls ${DIR}/waiting/ -1 | wc -l))"
Specifically the error is ....
division by 0 (error token is "/home/inventory/waiting/ -1 | wc -l")
I thought trying to put the number of files in this directory into a variable would work using $(()).
Does anyone have an idea why this is failing?
Many TIA.....
Jane
Use single parenthesis:
waiting="$(ls ${DIR}/waiting/ -1 | wc -l)"
$(( ... )) is used to perform arithmetic calculations.
From the man page:
((expression))
The expression is evaluated according to the rules described below under ARITHMETIC EVALUATION. If the value of the expression is non-zero, the return status is 0; otherwise the return status is 1. This is exactly equivalent to let "expression".
It is not recommended to use output of ls to count number of files/directories in a sub -directory as file names may contain newlines or glob characters as well.
Here is one example of doing it safely using gnu find:
dir=/home/inventory
waiting=$(find "$dir" -mindepth 1 -maxdepth 1 -printf '.' | wc -c)
If you don't have gnu find then use:
waiting=$(find "$dir" -mindepth 1 -maxdepth 1 -exec printf '.' \; | wc -c)
Another alternative is to pipe the waiting variable through awk to check for a value greater than 3 so:
DIR=/home/inventory
waiting="$((ls ${DIR}/waiting/ -1 | wc -l))"
echo $waiting
res=$(echo $waiting | awk '{ if ( $0 > 3 ) print "OK" }')
if [ $res == "OK" ]
then
(DO SOME STUFF HERE)
fi
In case someone has a similar problem even without using the compound command ((..))
Be sure your variables are not defined using let keyword, as it would force shell arithmetic evaluation too.
From the referece:
The shell allows arithmetic expressions to be evaluated, as one of the
shell expansions or by using the (( compound command, the let builtin,
or the -i option to the declare builtin.
Related
I am attempting to find multiple files with .py extension and grep to see if any of these files contain the string nn. Then return only the directory name (uniques), afterwards, if the last folder of the path ends in nn, then select this.
For example:
find `pwd` -iname '*.py' | xargs grep -l 'nn' | xargs dirname | sort -u | while read files; do if [[ sed 's|[\/](.*)*[\/]||g' == 'nn' ]]; then echo $files; fi; done
However, I cannot use sed in if-else expression, how can I use it for this case?
[[ ]] is not bracket syntax for an if statement like in other languages such as C or Java. It's a special command for evaluating a conditional expression. Depending on your intentions you need to either exclude it or use it correctly.
If you're trying to test a command for success or failure just call the command:
if command ; then
:
fi
If you want to test the output of the command is equal to some value, you need to use a command substitution:
if [[ $( command ) = some_value ]] ; then
:
fi
In your case though, a simple parameter expansion will be easier:
# if $files does not contain a trailing slash
if [[ "${files: -2}" = "nn" ]] ; then
echo "${files}"
fi
# if $files does contain a trailing slash
if [[ "${files: -3}" = "nn/" ]] ; then
echo "${files%/}"
fi
Shell loop and the [[ is superfluous here, since you use the sed anyway. This task could be accomplished by:
find "$PWD" -type f -name '*.py' -exec grep -l 'nn' {} + |
sed -n 's%\(.*nn\)/[^/]*$%\1%p' | sort -u
assuming pathnames don't contain a newline character.
This question already has answers here:
Variables as commands in Bash scripts
(5 answers)
Closed 3 years ago.
I'm trying to count the number of files with different extensions in /foo/.
case 1 works as expected, but more flexible situations such as case 2 or case 3 don't work as expected.
File test.sh
# case 1
vista=$(find /foo/*.zip -atime -1)
echo "$vista" | wc -l
# case 2
vista=$(find /foo/*)
echo "$vista.zip -atime -1" | wc -l
# case 3
echo "$vista.xz -atime -1" | wc -l
Output:
./test.sh
187
4566
4566
I suspect the problem is that for example echo "$vista.zip -atime -1" from case 2 runs first find /foo/* before appending the string zip -atime -1, but I don't know how to do it right.
Code should never be stored in strings (unless using printf %q to generate eval-safe versions of variables, and then using eval at runtime). Use either an array (for dynamically-constructed content) or a function.
The former:
find_cmd=( find /foo/* )
"${find_cmd[#]}" -atime -1 | wc -l
The latter:
find_under_foo() { find /foo/* "$#"; }
find_under_foo -atime -1 | wc -l
I want to have a conditional behavior depending on the number of files found:
found=$(find . -type f -name "$1")
numfiles=$(printf "%s\n" "$found" | wc -l)
if [ $numfiles -eq 0 ]; then
echo "cannot access $1: No such file" > /dev/stderr; exit 2;
elif [ $numfiles -gt 1 ]; then
echo "cannot access $1: Duplicate file found" > /dev/stderr; exit 2;
else
echo "File: $(ls $found)"
head $found
fi
EDITED CODE (to reflect more precisely what I need)
Though, numfiles isn't equal to 2(or more) when there are duplicate files found...
All the filenames are on one line, separated by a space.
On the other hand, this works correctly:
find . -type f -name "$1" | wc -l
but I don't want to do twice the recursive search in the if/then/else construct...
Adding -print0 doesn't help either.
What would?
PS- Simplifications or improvements are always welcome!
You want to find files and count the files with a name "$1":
grep -c "/${1}$" $(find . 2>/dev/null)
And store the result in a var. In one command:
numfiles=$(grep -c "/${1}$" <(find . 2>/dev/null))
Using $() to store data to a variable trims tailing whitespace. Since the final newline does not appear in the variable numfiles, wc miscounts by one. You can recover the trailing newline with:
numfiles=$(printf "%s\n" "$found" | wc -l)
This miscounts if found is empty (and if any filenames contain a newline), emphasizing the fact that this entire approach is faulty. If you really want to go this way, you can try:
numfiles=$(test -z "$numfiles" && echo 0 || printf "%s\n" "$found" | wc -l)
or pipe the output of find to a script that counts the output and prints a count along with the first filename:
find . -type f -name "$1" | tr '\n' ' ' |
awk '{c=NF; f=$1 } END {print c, f; exit c!=1}' c=0 |
while read count name; do
case $count in
0) echo no files >&2;;
1) echo 1 file $name;;
*) echo Duplicate files >&2;;
esac;
done
All of these solutions fail miserably if any pathnames contain whitespace. If that matters, you could change the awk to a perl script to make it easier to handle null separators and use -print0, but really I think you should stop worrying about special cases. (find -exec and find | xargs both fail to handle to 0 files matching case cleanly. Arguably this awk solution also doesn't handle it cleanly.)
I have a bash script where I'm trying to find out the number of files in a directory and perform an addition operation on it as well.
But while doing the same I'm getting the error as follows:
admin> ./fileCount.sh
1
./fileCount.sh: line 6: 22 + : syntax error: operand expected (error token is " ")
My script is as shown:
#!/usr/bin/bash
Var1=22
Var2= ls /stud_data/Input_Data/test3 | grep ".txt" | wc -l
Var3= $(($Var1 + $Var2))
echo $Var3
Can anyone point out where is the error.
A little away
As #devnull already answered to the question point out where is the error,
Just some more ideas:
General unix
To make this kind of browsing, there is a very powerfull command find that let you find recursively, exactly what you're serching for:
Var2=`find /stud_data/Input_Data/test3 -name '*.txt' | wc -l`
If you won't this to be recursive:
Var2=`find /stud_data/Input_Data/test3 -maxdepth 1 -name '*.txt' | wc -l`
If you want files only (meaning no symlink, nor directories)
Var2=`find /stud_data/Input_Data/test3 -maxdepth 1 -type f -name '*.txt' | wc -l`
And so on... Please read the man page: man find.
Particular bash solutions
As your question stand for bash, there is some bashism you could use to make this a lot quicker:
#!/bin/bash
Var1=22
VarLs=(/stud_data/Input_Data/test3/*.txt)
[ -e $VarLs ] && Var2=${#VarLs[#]} || Var2=0
Var3=$(( Var1 + Var2 ))
echo $Var3
# Uncomment next line to see more about current environment
# set | grep ^Var
Where bash expansion will translate /path/*.txt in an array containing all filenames matching the jocker form.
If there is no file matching the form, VarLs will only contain the jocker form himself.
So the test -e will correct this: If the first file of the returned list exist, then assing the number of elements in the list (${#VarLs[#]}) to Var2 else, assign 0 to Var2.
Can anyone point out where is the error.
You shouldn't have spaces around =.
You probably wanted to use command substitution to capture the result in Var2.
Try:
Var1=22
Var2=$(ls /stud_data/Input_Data/test3 | grep ".txt" | wc -l)
Var3=$(($Var1 + $Var2))
echo $Var3
Moreover, you could also say
Var3=$((Var1 + Var2))
What is the right way to test if several variables are all equal?
if [[ $var1 = $var2 = $var3 ]] # syntax error
Is it necessary to write something like the following?
if [[ $var1 = $var2 && $var1 = $var3 && $var2 = $var3 ]] # cumbersome
if [[ $var1 = $var2 && $var2 = $var3 && $var3 = $var4 ]] # somewhat better
Unfortunately, the otherwise excellent Advanced Bash Scripting Guide and other online sources I could find don't provide such an example.
My particular motivation is to test if several directories all have the same number of files, using ls -1 $dir | wc -l to count files.
Note
"var1" etc. are example variables. I'm looking for a solution for arbitrary variable names, not just those with a predictable numeric ending.
Update
I've accepted Richo's answer, as it is the most general. However, I'm actually using Kyle's because it's the simplest and my inputs are guaranteed to avoid the caveat.
Thanks for the suggestions, everyone.
if you want to test equality of an arbitrary number of items (let's call them $item1-5, but they could be an array
st=0
for i in $item2 $item3 $item4 $item5; do
[ "$item1" = "$i" ]
st=$(( $? + st ))
done
if [ $st -eq 0 ]; then
echo "They were all the same"
fi
If they are single words you can get really cheap about it.
varUniqCount=`echo "${var1} ${var2} ${var3} ${var4}" | sort -u | wc -l`
if [ ${varUniqCount} -gt 1 ]; then
echo "Do not match"
fi
Transitive method of inspection.
#!/bin/bash
var1=10
var2=10
var3=10
if [[ ($var1 == $var2) && ($var2 == $var3) ]]; then
echo "yay"
else
echo "nay"
fi
Output:
[jaypal:~/Temp] ./s.sh
yay
Note:
Since you have stated in your question that your objective is to test several directories that have same number of files, I thought of the following solution. I know this isn't something you had request so please feel free to disregard it.
Step1:
Identify number of files in a given directory. This command will look inside sub-dirs too but that can be controlled using -depth option of find.
[jaypal:~/Temp] find . -type d -exec sh -c "printf {} && ls -1 {} | wc -l " \;
. 9
./Backup 7
./bash 2
./GTP 22
./GTP/ParserDump 11
./GTP/ParserDump/ParserDump 1
./perl 7
./perl/p1 2
./python 1
./ruby 0
./scripts 22
Step2:
This can be combined with Step1 as we are just redirecting the content to a file.
[jaypal:~/Temp] find . -type d -exec sh -c "printf {} && ls -1 {} | wc -l " \; > file.temp
Step3:
Using the following command we will look in the file.temp twice and it will give us a list of directories that have same number of files.
[jaypal:~/Temp] awk 'NR==FNR && a[$2]++ {b[$2];next} ($2 in b)' file.temp file.temp | sort -k2
./GTP/ParserDump/ParserDump 1
./python 1
./bash 2
./perl/p1 2
./Backup 7
./perl 7
./GTP 22
./scripts 22
(edited to include delimiters to fix the problem noted by Keith Thompson)
Treating the variable values as strings, you can concatenate them along with a suitable delimiter and do one comparison:
if [[ "$var1|$var2|$var3" = "$var1|$var1|$var1" ]]
I used = instead == because == isn't an equality comparison inside [[ ]], it is a pattern match.
For your specific case, this should work:
distinct_values=$(for dir in this_dir that_dir another_dir ; do ls -l "$dir" | wc -l ; done | uniq | wc -l)
if [ $distinct_values -eq 1 ] ; then
echo All the same
else
echo Not all the same
fi
Explanation:
ls -l "$dir" lists the files and subdirectories in the directory, one per line (omitting dot files).
Piping the output through wc -l gives you the number of files in the directory.
Doing that consecutively for each directory in the list gives you a list consisting of the number of files in each directory; if there are 7 in each, this gives 3 lines each consisting of the number 7
Piping that through uniq eliminates consecutive duplicate lines.
Piping that through wc -l gives you the number of distinct lines, which will be 1 if and only if all the directories contain the same number of files.
Note that the output of the 4th stage doesn't necessarily give you the number of distinct numbers of files in the directories; uniq only removes adjacent duplicates, so if the inputs are 7 6 7, the two 7s won't be merged. But it will merge all lines into 1 only if they're all the same.
This is the power of the Unix command line: putting small tools together to do interesting and useful things. (Show me a GUI that can do that!)
For values stored in variables, replace the first line by:
distinct_values=$(echo "$this_var" "$that_var" "$another_var" | fmt -1 | uniq | wc -l)
This assumes that the values of the variables don't contain spaces.