counting files, directories and subdirectories in a given path - bash

I am trying to figure how to run a simple script "count.sh" that is called together with a path, e.g.:
count.sh /home/users/documents/myScripts
The script will need to iterate over the directories in this path and print how many files and folders (including hidden) in each level of this path.
For example:
7
8
9
10
(myScripts - 7, documents - 8, users -9, home - 10)
And by the way, can I run this script using count.sh pwd?

More or less something like that:
#!/bin/sh
P="$1"
while [ "/" != "$P" ]; do
echo "$P `find \"$P\" -maxdepth 1 -type f | wc -l`"
P=`dirname "$P"`;
done
echo "$P `find \"$P\" -maxdepth 1 -type f | wc -l`"
You can use it from the current directory with script.sh `pwd`

Another approach is to handle separation or tokenizing the path using an array and controlling word-splitting with the internal field separator (IFS). You can include the root directory if desired (you would need to trim the additional leading '/' in the printout in that case)
#!/bin/bash
[ -z "$1" -o ! -d "$1" ] && {
printf "error: directory argument required.\n"
exit 1
}
p="$1" ## remove two lines to include /
[ "${p:0:1}" = "/" ] && p="${p:1}"
oifs="$IFS" ## save internal field separator value
IFS=$'/' ## set to break on '/'
array=( $p ) ## tokenize given path into array
IFS="$oifs" ## restore original IFS
## print out path level info using wc
for ((i=0; i<${#array[#]}; i++)); do
dirnm="${dirnm}/${array[i]}"
printf "%d. %s -- %d\n" "$((i+1))" "$dirnm" $(($(ls -Al "$dirnm" | wc -l)-1))
done
Example Output
$ bash filedircount.sh /home/david/tmp
1. /home -- 5
2. /home/david -- 132
3. /home/david/tmp -- 113
As an alternative, you could use a for loop to loop through and count the items in the directory at each level instead of using wc if desired.

You could try the following
#!/bin/bash
DIR=$(cd "$1" ; pwd)
PREFIX=
until [ "$DIR" = / ] ; do
echo -n "$PREFIX"$(basename "$DIR")" "$(ls -Ab "$DIR" | wc -l)
DIR=$(dirname "$DIR")
PREFIX=", "
done
echo
(ls -Ab lists all files and folders except . and .. and escapes special characters so that only one line is printed per file even if filenames include newline characters. wc -l counts lines.)
You can invoke the script using
count.sh `pwd`

Related

How to replace the position of a directory containing spaces

I made two pipelines; this, x="$(ls -1p | grep "/$" | tr -d "/")" get all the sub directories from the working directory, and this, y="$(ls -1p | grep "/$"| grep \ | tr -d "/")" gets the sub directories that contain spaces in the working directory.
So now what I've trying to do is to replace the position of the directory that contains spaces and puts it at the very top, ie., say below are my sub dirs:
Dir1
Dir2
Dir 3
Now Dir 3 goes to the top:
Dir 3
Dir1
Dir1
for I in $x; do
for X in $y; do
if [[ $I == $X ]];then
sed "/"$X"/d" "$I"
fi
done
echo "$I"
done
Above is my loop to do that task. It prints all the sub dirs that contains no spaces but prints it as:
Dir1
Dir2
sed: Dir: No such file or directory
Dir
sed: 3: No such file or directory
3
If anyone can help out that will be greatly appreciated.
If you prefer for loop to the find command, how about:
#!/bin/bash
# 1st loop to print the dirnames containing space character
for d in */; do # loops over subdirectories under current directory
if [[ $d =~ [[:space:]] ]]; then # if the dirname contains a space character
echo "${d%/}" # then print the name removing the trailing slash
fi
done
# 2nd loop to print the dirnames without space character
for d in */; do
if [[ ! $d =~ [[:space:]] ]]; then # if the dirname does not contain a space character
echo "${d%/}"
fi
done
Output with the provided example:
Dir 3
Dir1
Dir2
Using GNU find:
find . -mindepth 1 -type d -name '*[[:space:]]*' # spaces
find . -mindepth 1 -type d -regex '.*/[^/[:space:]]+$' # no spaces
This is recursive.

Bash Script to Prepend a Single Random Character to All Files In a Folder

I have an audio sample library with thousands of files. I would like to shuffle/randomize the order of these files. Can someone provide me with a bash script/line that would prepend a single random character to all files in a folder (including files in sub-folders). I do not want to prepend a random character to any of the folder names though.
Example:
Kickdrum73.wav
Kickdrum SUB.wav
Kick808.mp3
Renamed to:
f_Kickdrum73.wav
!_Kickdrum SUB.wav
4_Kick808.mp3
If possible, I would like to be able to run this script more than once, but on subsequent runs, it just changes the randomly prepended character instead of prepending a new one.
Some of my attempts:
find ~/Desktop/test -type f -print0 | xargs -0 -n1 bash -c 'mv "$0" "a${0}"'
find ~/Desktop/test/ -type f -exec mv -v {} $(cat a {}) \;
find ~/Desktop/test/ -type f -exec echo -e "Z\n$(cat !)" > !Hat 15.wav
for file in *; do
mv -v "$file" $RANDOM_"$file"
done
Note: I am running on macOS.
Latest attempt using code from mr. fixit:
find . -type f -maxdepth 999 -not -name ".*" |
cut -c 3- - |
while read F; do
randomCharacter="${F:2:1}"
if [ $randomCharacter == '_' ]; then
new="${F:1}"
else
new="_$F"
fi
fileName="`basename $new`"
newFilename="`jot -r -c $fileName 1 A Z`"
filePath="`dirname $new`"
newFilePath="$filePath$newFilename"
mv -v "$F" "$newFilePath"
done
Here's my first answer, enhanced to do sub-directories.
Put the following in file randomize
if [[ $# != 1 || ! -d "$1" ]]; then
echo "usage: $0 <path>"
else
find $1 -type f -not -name ".*" |
while read F; do
FDIR=`dirname "$F"`
FNAME=`basename "$F"`
char2="${FNAME:1:1}"
if [ $char2 == '_' ]; then
new="${FNAME:1}"
else
new="_$FNAME"
fi
new=`jot -r -w "%c$new" 1 A Z`
echo mv "$F" "${FDIR}/${new}"
done
fi
Set the permissions with chmod a+x randomize.
Then call it with randomize your/path.
It'll echo the commands required to rename everything, so you can examine them to ensure they'll work for you. If they look right, you can remove the echo from the 3rd to last line and rerun the script.
cd ~/Desktop/test, then
find . -type f -maxdepth 1 -not -name ".*" |
cut -c 3- - |
while read F; do
char2="${F:2:1}"
if [ $char2 == '_' ]; then
new="${F:1}"
else
new="_$F"
fi
new=`jot -r -w "%c$new" 1 A Z`
mv "$F" "$new"
done
find . -type f -maxdepth 1 -not -name ".*" will get all the files in the current directory, but not the hidden files (names starting with '.')
cut -c 3- - will strip the first 2 chars from the name. find outputs paths, and the ./ gets in the way of processing prefixes.
while read VAR; do <stuff>; done is a way to deal with one line at a time
char2="${VAR:2:1} sets a variable char2 to the 2nd character of the variable VAR.
if - then - else sets new to the filename, either preceded by _ or with the previous random character stripped off.
jot -r -w "%c$new" 1 A Z tacks random 1 character from A-Z onto the beginning of new
mv old new renames the file
You can also do it all in bash and there are several ways to approach it. The first is simply creating an array of letters containing whatever letters you want to use as a prefix and then generating a random number to use to choose the element of the array, e.g.
#!/bin/bash
letters=({0..9} {A..Z} {a..z}) ## array with [0-9] [A-Z] [a-z]
for i in *; do
num=$(($RANDOM % 63)) ## generate number
## remove echo to actually move file
echo "mv \"$i\" \"${letters[num]}_$i\"" ## move file
done
Example Use/Output
Current the script outputs the changes it would make, you must remove the echo "..." surrounding the mv command and fix the escaped quotes to actually have it apply changes:
$ bash ../randprefix.sh
mv "Kick808.mp3" "4_Kick808.mp3"
mv "Kickdrum SUB.wav" "h_Kickdrum SUB.wav"
mv "Kickdrum73.wav" "l_Kickdrum73.wav"
You can also do it by generating a random number representing the ASCII character between 48 (character '0') through 126 (character '~'), excluding 'backtick'), and then converting the random number to an ASCII character and prefix the filename with it, e.g.
#!/bin/bash
for i in *; do
num=$((($RANDOM % 78) + 48)) ## generate number for '0' - '~'
letter=$(printf "\\$(printf '%03o' "$num")") ## letter from number
while [ "$letter" = '`' ]; do ## exclude '`'
num=$((($RANDOM % 78) + 48)) ## generate number
letter=$(printf "\\$(printf '%03o' "$num")")
done
## remove echo to actually move file
echo "mv \"$i\" \"${letter}_$i\"" ## move file
done
(similar output, all punctuation other than backtick is possible)
In each case you will want to place the script in your path or call it from within the directory you want to move the file in (you split split dirname and basename and join them back together to make the script callable passing the directory to search as an argument -- that is left to you)

Separate folders into subfolders according to numbering in bash

I have the following directory tree:
1_loc
2_buzdfg
4_foodga
5_bardfg
6_loc
8_buzass
9_foossd
12_bardaf
There may be numbers missing in the folder ordering.
I want to separate these folders into subfolders according to their numbers, so that all folders with a number smaller than 6 (before the second _loc folder) would go to folder1 and all folders with a number equal or greater than 6 with go to folder2.
I can solve the problem very easily using the mouse, of course, but I wanted a suggestion of how to do this automatically from the terminal.
Any ideas?
while read -r line; do
# Regex match the beginning of the string for a character between 1 and 5 - change this as you'd please to any regex
FOLDERNUMBER=""
[[ "$line" ~= "^[1-5]" ]] && FOLDERNUMBER="1" || FOLDERNUMBER="2"
# So FOLDERPATH = "./folder1", if FOLDERNUMBER=1
FOLDERPATH="./folder$FOLDERNUMBER"
# Check folder exists, if not create it
[[ ! -d "$FOLDERPATH" ]] && mkdir "$FOLDERPATH"
# Finally, move the file to FOLDERPATH
mv "$line" "$FOLDERPATH/$(basename $line)"
done < <(find . -type f)
# This iterates through each line of the command in the brackets - in this case, each file path from the `find` command.
I think the solution is to loop through the files and check the number before the first _.
Firstly, let's check how to get the number before _:
$ d="1_loc_b"
$ echo "${d%%_*}"
1
OK, so this works. Then, let's loop:
for file in *
do
echo "$file"
(( ${file%%_*} > 5)) && echo "moving to dir2/" || echo "moving to dir1/"
done
Suppose folder1 and folder2 exists in the same directory, I will do it like this:
for d in *_*; do # to avoid folder1 and folder2
# check if the first field seperated by _ is less than 5
if ((`echo $d | cut -d"_" -f1` < 6)); then
mv $d folder1/$d;
else
mv $d folder2/$d;
fi;
done
(more about cut)
You can go to the current directory and run these simple commands:
mv {1,2,3,4}_* folder1/
mv {5,6,7,8}_* folder2/
This assumes no other files/directory starting with these prefixes (i.e. 1-8).
Another pure bash, parameter-expansion solution:-
#!/bin/bash
# 'find' returns folders having a '_' in their names, the flag -print0 option to
# preserve special characters in names.
# the folders are names as './1_folder', './2_folder', bash magic is done
# to remove those special characters.
# '-v' flag in 'mv' for verbose action
while IFS= read -r -d '' folder; do
folderName="${folder%_*}" # To strip the characters after the '_'
finalName="${folderName##*/}" # To strip the everything before '/'
((finalName > 5)) && mv -v "$folder" folder1 || mv -v "$folder" folder2
done < <(find . -maxdepth 1 -mindepth 1 -name "*_*" -type d -print0)
You can create the a script with the following code and when you run it, the folders will be moved as desired..
#seperate the folders into 2 folders
#this is a generic solution for any folder that start with a number
#!/bin/bash
for file in *
do
prefix=`echo $file | awk 'BEGIN{FS="_"};{print $1}'`
if [[ $prefix != ?(-)+([0-9]) ]]
then continue
fi
if [ $prefix -le 4 ]
then mv "$file" folder1
elif [ $prefix -ge 5 ]
then mv "$file" folder2
fi
done

counting the total numbers of files and directories in a provided folder including subdirectories and their files

I want to count all the files and directories from a provided folder including files and directories in a subdirectory. I have written a script which will count accurately the number of files and directory but it does not handle the subdirectories any ideas ???
I want to do it without using FIND command
#!/bin/bash
givendir=$1
cd "$givendir" || exit
file=0
directories=0
for d in *;
do
if [ -d "$d" ]; then
directories=$((directories+1))
else
file=$((file+1))
fi
done
echo "Number of directories :" $directories
echo "Number of file Files :" $file
Use find:
echo "Number of directories: $(find "$1" -type d | wc -l)"
echo "Number of files/symlinks/sockets: $(find "$1" ! -type d | wc -l)"
Using plain shell and recursion:
#!/bin/bash
countdir() {
cd "$1"
dirs=1
files=0
for f in *
do
if [[ -d $f ]]
then
read subdirs subfiles <<< "$(countdir "$f")"
(( dirs += subdirs, files += subfiles ))
else
(( files++ ))
fi
done
echo "$dirs $files"
}
shopt -s dotglob nullglob
read dirs files <<< "$(countdir "$1")"
echo "There are $dirs dirs and $files files"
find "$1" -type f | wc -l will give you the files, find "$1" -type d | wc -l the directories
My quick-and-dirty shellscript would read
#!/bin/bash
test -d "$1" || exit
files=0
# Start with 1 to count the starting dir (as find does), else with 0
directories=1
function docount () {
for d in $1/*; do
if [ -d "$d" ]; then
directories=$((directories+1))
docount "$d";
else
files=$((files+1))
fi
done
}
docount "$1"
echo "Number of directories :" $directories
echo "Number of file Files :" $files
but mind it: On my build folder for a project, there were quite some differences:
find: 6430 dirs, 74377 non-dirs
my script: 6032 dirs, 71564 non-dirs
#thatotherguy's script: 6794 dirs, 76862 non-dirs
I assume that has to do with the legions of links, hidden files etc., but I am too lazy to investigate: find is the tool of choice.
Here are some one-line commands that work without find:
Number of directories: ls -Rl ./ | grep ":$" | wc -l
Number of files: ls -Rl ./ | grep "[0-9]:[0-9]" | wc -l
Explanation:
ls -Rl lists all files and directories recursively, one line each.
grep ":$" finds just the results whose last character is ':'. These are all of the directory names.
grep "[0-9]:[0-9]" matches on the HH:MM part of the timestamp. The timestamp only shows up on file, not directories. If your timestamp format is different then you will need to pick a different grep.
wc -l counts the number of lines that matched from the grep.

How to locate the directory where the sum of the number of lines of regular file is greatest (in bash)

Hi i'm new in Unix and bash and I'd like to ask q. how can i do this
The specified directory is given as arguments. Locate the directory
where the sum of the number of lines of regular file is greatest.
Browse all specific directories and their subdirectories. Amounts
count only for files that are directly in the directory.
I try somethnig but it's not working properly.
while [ $# -ne 0 ];
do case "$1" in
-h) show_help ;;
-*) echo "Error: Wrong arguments" 1>&2 exit 1 ;;
*) directories=("$#") break ;;
esac
shift
done
IFS='
'
amount=0
for direct in "${directories[#]}"; do
for subdirect in `find $direct -type d `; do
temp=`find "$subdirect" -type f -exec cat {} \; | wc -l | tr -s " "`
if [ $amount -lt $temp ]; then
amount=$temp
subdirect2=$subdirect
fi
done
echo Output: "'"$subdirect2$amount"'"
done
the problem is here when i use as arguments this dirc.(just example)
/home/usr/first and there are this direct.
/home/usr/first/tmp/first.txt (50 lines)
/home/usr/first/tmp/second.txt (30 lines)
/home/usr/first/tmp1/one.txt (20 lines)
it will give me on Output /home/usr/first/tmp1 100 and this is wrong it should be /home/usr/first/tmp 80
I'd like to scan all directories and all its subdirectories in depth. Also if multiple directories meets the maximum should list all.
Given your sample files, I'm going to assume you only want to look at the immediate subdirectories, not recurse down several levels:
max=-1
# the trailing slash limits the wildcard to directories only
for dir in */; do
count=0
for file in "$dir"/*; do
[[ -f "$file" ]] && (( count += $(wc -l < "$file") ))
done
if (( count > max )); then
max=$count
maxdir="$dir"
fi
done
echo "files in $maxdir have $max lines"
files in tmp/ have 80 lines
In the spirit of Unix (caugh), here's an absolutely disgusting chain of pipes that I personally hate, but it's a lot of fun to construct :):
find . -mindepth 1 -maxdepth 1 -type d -exec sh -c 'find "$1" -maxdepth 1 -type f -print0 | wc -l --files0-from=- | tail -1 | { read a _ && echo "$a $1"; }' _ {} \; | sort -nr | head -1
Of course, don't use this unless you're mentally ill, use glenn jackman's nice answer instead.
You can have great control on find's unlimited filtering possibilities, too. Yay. But use glenn's answer!

Resources