I'm new in ShellScripting and have the following script that i created based on a simpler one, i want to pass it an argument with the path to count files. Cannot find my logical mistake to make it work right, the output is always "1"
#!/bin/bash
i=0
for file in $0/*
do
let i=$i+1
done
echo $i
To execute the code i use
sh scriptname.sh /path/to/folder/to/count/files
$0 is the name with which your script was invoked (roughly, subject to several exceptions that aren't pertinent here). The first argument is $1, and so it's $1 that you want to use in your glob expression.
#!/bin/bash
i=0
for file in "$1"/*; do
i=$(( i + 1 )) ## $(( )) is POSIX-compliant arithmetic syntax; let is deprecated.
done
echo "$i"
That said, you can get this number more directly:
#!/bin/bash
shopt -s nullglob # allow globs to expand to an empty list
files=( "$1"/* ) # put list of files into an array
echo "${#files[#]}" # count the number of items in the array
...or even:
#!/bin/sh
set -- "$1"/* # override $# with the list of files matching the glob
if [ -e "$1" ] || [ -L "$1" ]; then # if $1 exists, then it had matches
echo "$#" # ...so emit their number.
else
echo 0 # otherwise, our result is 0.
fi
If you want to count the number of files in a directory, you can run something like this:
ls /path/to/folder/to/count/files | wc -l
Related
subjects_list=$(ls -l /Volumes/Backup_Plus/PPMI_10 | awk '{ print $NF }')
filepath="/Volumes/Backup_Plus/PPMI_10/$subjects/*/*/S*/"
for subjects in $subjects_list; do
if [[ -f "${filepath}/*.bval" && -f "${filepath}/*.bvec" && -f "${filepath}/*.json" && -f "${filepath}/*.nii.gz" ]]; then
echo "${subjects}" >> /Volumes/Backup_Plus/PPMI_10/keep_subjects.txt
else
echo "${subjects}" >> /Volumes/Backup_Plus/PPMI_10/not_keep_subjects.txt
fi
done
problem is supposedly in the if statement, I tried this...
bvalfile = (*.bval)
bvecfile =(*.bvec)
jsonfile =(*.json)
niigzfile =(*.nii.gz)
if [[ -f "$bvalfile" && -f "$bvecfile" && -f "$jsonfile" && -f "$niigzfile" ]]; then
however that didn't work. Any help with syntax or errors or does it need to be changed completely. Trying to separate the files that have .^file types from those that don't by making two lists.
thanks
You're assigning filepath outside the for-subject loop but using the unset variable $subjects in it. You want to move that inside the loop.
Double-quoted wildcards aren't expanded, so both $filepath and your -f test will be looking for filenames with literal asterisks in them.
-f only works on a single file, so even if you fix the quotes, you'll have a syntax error if there's more than one file matching the pattern.
So I think what you want is something like this:
# note: array assignment -
# shell does the wildcard expansion, no ls required
prefix_list=( /Volumes/Backup_Plus/PPMI_10/* )
# and array expansion
for prefix in "${prefix_list[#]}"; do
# the subject is just the last component of the path
subject=${prefix##*/}
# start by assuming we're keeping this one
decision=keep
# in case filepath pattern matches more than one directory, loop over them
for filepath in "$prefix"/*/*/S*/; do
# if any of the files don't exist, switch to not keeping it
for file in "$filepath"/{*.bval,*.bvec,*.json,*.nii.gz}; do
if [[ ! -f "$file" ]]; then
decision=not_keep
# we have our answer and can stop looping now
break 2
fi
done
done
# now append to the correct list
printf '%s\n' "$subject" >>"/Volumes/Backup_Plus/PPMI_10/${decision}_subjects.txt"
done
As an exercise I have set myself the task of recursively listing files using bash builtins. I particularly don't want to use ls or find and I would prefer not to use setopt extendedglob. The following appears to work but I cannot see how to extend it with /.* to list hidden files. Is there a simple workaround?
g() { for k in "$1"/*; do # loop through directory
[[ -f "$k" ]] && { echo "$k"; continue; }; # echo file path
[[ -d "$k" ]] && { [[ -L "$k" ]] && { echo "$k"; continue; }; # echo symlinks but don't follow
g "$k"; }; # start over with new directory
done; }; g "/Users/neville/Desktop" # original directory
Added later: sorry - I should have said: 'bash-3.2 on OS X'
Change
for k in "$1"/*; do
to
for k in "$1"/* "$1"/.[^.]* "$1"/..?*; do
The second glob matches all files whose names start with a dot followed by anything other than a dot, while the third matches all files whose names start with two dots followed by something. Between the two of them, they will match all hidden files other than the entries . and ...
Unfortunately, unless the shell option nullglob is set, those (like the first glob) could remain as-is if there are no files whose names match (extremely likely in the case of the third one) so it is necessary to verify that the name is actually a file.
An alternative would be to use the much simpler glob "$1"/.*, which will always match the . and .. directory entries, and will consequently always be substituted. In that case, it's necessary to remove the two entries from the list:
for k in "$1"/* "$1"/.*; do
if ! [[ $k =~ /\.\.?$ ]]; then
# ...
fi
done
(It is still possible for "$1"/* to remain in the list, though. So that doesn't help as much as it might.)
Set the GLOBIGNORE file to exclude . and .., which implicitly turns on "shopt -u dotglob". Then your original code works with no other changes.
user#host [/home/user/dir]
$ touch file
user#host [/home/user/dir]
$ touch .dotfile
user#host [/home/user/dir]
$ echo *
file
user#host [/home/user/dir]
$ GLOBIGNORE=".:.."
user#host [/home/user/dir]
$ echo *
.dotfile file
Note that this is bash-specific. In particular, it does not work in ksh.
You can specify multiple arguments to for:
for k in "$1"/* "$1"/.*; do
But if you do search for .* in directories , you should be aware that it also gives you the . and .. files. You may also be given a nonexistent file if the "$1"/* glob matches, so I would check that too.
With that in mind, this is how I would correct the loop:
g() {
local k subdir
for k in "$1"/* "$1"/.*; do # loop through directory
[[ -e "$k" ]] || continue # Skip missing files (unmatched globs)
subdir=${k##*/}
[[ "$subdir" = . ]] || [[ "$subdir" = .. ]] && continue # Skip the pseudo-directories "." and ".."
if [[ -f "$k" ]] || [[ -L "$k" ]]; then
printf %s\\n "$k" # Echo the paths of files and symlinks
elif [[ -d "$k" ]]; then
g "$k" # start over with new directory
fi
done
}
g ~neville/Desktop
Here the funky-looking ${k##*/} is just a fast way to take the basename of the file, while local was put in so that the variables don't modify any existing variables in the shell.
One more thing I've changed is echo "$k" to printf %s\\n "$k", because echo is irredeemably flawed in its argument handling and should be avoided for the purpose of echoing an unknown variable. (See Rich's sh tricks for an explanation of how; it boils down to -n and -e throwing a spanner in the works.)
By the way, this will NOT print sockets or fifos - is that intentional?
I've been trying for hours on this problem, and cannot set it straight.
This minimal script works as it should:
#!/bin/bash
wipe_thumbs=1
if (( wipe_thumbs )); then
src_dir=$1
thumbs="$src_dir"/*/t1*.jpg
echo $thumbs
fi
Invoke with ./script workdir and a lot of filenames starting with t1* in all the sub-dirs of workdir are shown.
When putting the above if-case in the bigger script, the globbing is not executed:
SRC: -- workdir/ --
THUMBS: -- workdir//*/t1*.jpg --
ls: cannot access workdir//*/t1*.jpg: No such file or directory
The only difference with the big script and the minimal script is that the big script has a path-validator and getopts-extractor. This code is immediately above the if-case:
#!/bin/bash
OPTIONS=":ts:d:"
src_dir=""
dest_dir=""
wipe_thumbs=0
while getopts $OPTIONS opt ; do
case "$opt" in
t) wipe_thumbs=1
;;
esac
done
shift $((OPTIND - 1))
src_dir="$1"
dest_dir="${2:-${src_dir%/*}.WORK}"
# Validate source
echo -n "Validating source..."
if [[ -z "$src_dir" ]]; then
echo "Can't do anything without a source-dir."
exit
else
if [[ ! -d "$src_dir" ]]; then
echo "\"$src_dir\" is really not a directory."
exit
fi
fi
echo "done"
# Validate dest
echo -n "Validating destination..."
if [[ ! -d "$dest_dir" ]]; then
mkdir "$dest_dir"
(( $? > 0 )) && exit
else
if [[ ! -w "$dest_dir" ]]; then
echo "Can't write into the specified destination-dir."
exit
fi
fi
echo "done"
# Move out the files into extension-named directories
echo -n "Moving files..."
if (( wipe_thumbs )); then
thumbs="$src_dir"/*/t1*.jpg # not expanded
echo DEBUG THUMBS: -- "$thumbs" --
n_thumbs=$(ls "$thumbs" | wc -l)
rm "$thumbs"
fi
...rest of script, never reached due to error...
Can anyone shed some lights on this? Why is the glob not expanded in the big script, but working fine in the minimalistic test script?
EDIT: Added the complete if-case.
The problem is that wildcards aren't expanded in assignment statements (e.g. thumbs="$src_dir"/*/t1*.jpg), but are expanded when variables are used without double-quotes. Here's an interactive example:
$ src_dir=workdir
$ thumbs="$src_dir"/*/t1*.jpg
$ echo $thumbs # No double-quotes, wildcards will be expanded
workdir/sub1/t1-1.jpg workdir/sub1/t1-2.jpg workdir/sub2/t1-1.jpg workdir/sub2/t1-2.jpg
$ echo "$thumbs" # Double-quotes, wildcards printed literally
workdir/*/t1*.jpg
$ ls $thumbs # No double-quotes, wildcards will be expanded
workdir/sub1/t1-1.jpg workdir/sub2/t1-1.jpg
workdir/sub1/t1-2.jpg workdir/sub2/t1-2.jpg
$ ls "$thumbs" # Double-quotes, wildcards treated as literal parts of filename
ls: workdir/*/t1*.jpg: No such file or directory
...so the quick-n-easy fix is to remove the double-quotes from the ls and rm commands. But this isn't safe, as it'll also cause parsing problems if $src_dir contains any whitespace or wildcard characters (this may not be an issue for you, but I'm used to OS X where spaces in filenames are everywhere, and I've learned to be careful about these things). The best way to do this is to store the list of thumb files as an array:
$ src="work dir"
$ thumbs=("$src_dir"/*/t1*.jpg) # No double-quotes protect $src_dir, but not the wildcard portions
$ echo "${thumbs[#]}" # The "${array[#]}" idiom expands each array element as a separate word
work dir/sub1/t1-1.jpg work dir/sub1/t1-2.jpg work dir/sub2/t1-1.jpg work dir/sub2/t1-2.jpg
$ ls "${thumbs[#]}"
work dir/sub1/t1-1.jpg work dir/sub2/t1-1.jpg
work dir/sub1/t1-2.jpg work dir/sub2/t1-2.jpg
You might also want to set nullglob in case there aren't any matches (so it'll expand to a zero-length array).
In your script, this'd come out something like this:
if (( wipe_thumbs )); then
shopt -s nullglob
thumbs=("$src_dir"/*/t1*.jpg) # expanded as array elements
shopt -u nullglob # back to "normal" to avoid unexpected behavior later
printf 'DEBUG THUMBS: --'
printf ' "%s"' "${thumbs[#]}"
printf ' --\n'
# n_thumbs=$(ls "${thumbs[#]}" | wc -l) # wrong way to do this...
n_thumbs=${#thumbs[#]} # better...
if (( n_thumbs == 0 )); then
echo "No thumb files found" >&2
exit
fi
rm "${thumbs[#]}"
fi
I'm beginner in bash programming. I want to display head -n $1 results of sorting files
by size in /etc/*. The problem is that at final search, I must know how many directories and files has processed.
I compose following code:
#!/bash/bin
let countF=0;
let countD=0;
for file in $(du -sk /etc/* |sort +0n | head $1); do
if [ -f "file" ] then
echo $file;
let countF=countF+1;
else if [ -d "file" ] then
let countD=countD+1;
fi
done
echo $countF
echo $countD
I have errors at execution. How use find with du, because I must search recursively?
#!/bin/bash # directory and program reversed
let countF=0 # semicolon not needed (several more places)
let countD=0
while read -r file; do
if [ -f "$file" ]; then # missing dollar sign and semicolon
echo $file
let countF=countF+1 # could also be: let countF++
else if [ -d "$file" ]; then # missing dollar sign and semicolon
let countD=countD+1
fi
done < <(du -sk /etc/* |sort +0n | head $1) # see below
echo $countF
echo $countD
Changing the loop from a for to a while allows it to work properly in case filenames contain spaces.
I'm not sure what version of sort you have, but I'll take your word for it that the argument is correct.
It's #!/bin/bash not #!/bash/bin.
I don't know what that argument to sort is supposed to be. Maybe you meant sort -r -n?
Your use of head is wrong. Giving head file arguments causes it to ignore its standard input, so in general it's an error to both pipe something to head and give it a file argument. Besides that, "$1" refers to the script's first argument. Did you maybe mean head -n 1, or were you trying to make the number of lines processed configurable from an argument to the script: head -n"$1".
In your if tests, you're not referencing your loop variable: it should read "$file", not "file".
Not that the bash parser cares, but you should try to indent sanely.
#!/bin/bash # directory and program reversed
let countF=0 # semicolon not needed (several more places)
let countD=0
while read -r file; do
if [ -f "$file" ]; then # missing dollar sign and semicolon
echo $file
let countF=countF+1 # could also be: let countF++
else if [ -d "$file" ]; then # missing dollar sign and semicolon
let countD=countD+1
fi
done < <(du -sk /etc/* |sort +0n | head $1) # see below
echo $countF
echo $countD
I tried instead of file variable the /etc/* but I don't see a result. the idea is to sort all files by size from a directories and subdirectories and display $1 results ordered by
size of the files. In this process I must know how many files and dirs contains the directory where
I did the search.
Ruby(1.9+)
#!/usr/bin/env ruby
fc=0
dc=0
a=Dir["/etc/*"].inject([]) do |x,f|
fc+=1 if File.file?(f)
dc+=1 if File.directory?(f)
x<<f
end
puts a.sort
puts "number of files: #{fc}"
puts "number of directories: #{dc}"
I want to check whether a directory has files or not in bash.
My code is here.
for d in {,/usr/local}/etc/bash_completion.d ~/.bash/completion.d
do
[ -d "$d" ] && [ -n "${d}/*" ] &&
for f in $d/*; do
[ -f "$f" ] && echo "$f" && . "$f"
done
done
The problem is that "~/.bash/completion.d" has no file.
So, $d/* is regarded as simple string "~/.bash/completion.d/*", not empty string which is result of filename expansion.
As a result of that code, bash tries to run
. "~/.bash/completion.d/*"
and of course, it generates error message.
Can anybody help me?
If you set the nullglob bash option, through
shopt -s nullglob
then globbing will drop patterns that don't match any file.
# NOTE: using only bash builtins
# Assuming $d contains directory path
shopt -s nullglob
# Assign matching files to array
files=( "$d"/* )
if [ ${#files[#]} -eq 0 ]; then
echo 'No files found.'
else
# Whatever
fi
Assignment to an array has other benefits, including desirable (correct!) handling of filenames/paths containing white-space, and simple iteration without using a sub-shell, as the following code does:
find "$d" -type f |
while read; do
# Process $REPLY
done
Instead, you can use:
for file in "${files[#]}"; do
# Process $file
done
with the benefit that the loop is run by the main shell, meaning that side-effects (such as variable assignment, say) made within the loop are visible for the remainder of script. Of course, it's also way faster, if performance is an issue.
Finally, an array can also be inserted in command line arguments (without splitting arguments containing white-space):
$ md5sum fileA "${files[#]}" fileZ
You should always attempt to correctly handle files/paths containing white-space, because one day, they will happen!
You could use find directly in the following way:
for f in $(find {,/usr/local}/etc/bash_completion.d ~/.bash/completion.d -maxdepth 1 -type f);
do echo $f; . $f;
done
But find will print a warning if some of the directory isn't found, you can either put a 2> /dev/null or put the find call after testing if the directories exist (like in your code).
find() {
for files in "$1"/*;do
if [ -d "$files" ];then
numfile=$(ls $files|wc -l)
if [ "$numfile" -eq 0 ];then
echo "dir: $files has no files"
continue
fi
recurse "$files"
elif [ -f "$files" ];then
echo "file: $files";
:
fi
done
}
find /path
Another approach
# prelim stuff to set up d
files=`/bin/ls $d`
if [ ${#files} -eq 0 ]
then
echo "No files were found"
else
# do processing
fi