I made two pipelines; this, x="$(ls -1p | grep "/$" | tr -d "/")" get all the sub directories from the working directory, and this, y="$(ls -1p | grep "/$"| grep \ | tr -d "/")" gets the sub directories that contain spaces in the working directory.
So now what I've trying to do is to replace the position of the directory that contains spaces and puts it at the very top, ie., say below are my sub dirs:
Dir1
Dir2
Dir 3
Now Dir 3 goes to the top:
Dir 3
Dir1
Dir1
for I in $x; do
for X in $y; do
if [[ $I == $X ]];then
sed "/"$X"/d" "$I"
fi
done
echo "$I"
done
Above is my loop to do that task. It prints all the sub dirs that contains no spaces but prints it as:
Dir1
Dir2
sed: Dir: No such file or directory
Dir
sed: 3: No such file or directory
3
If anyone can help out that will be greatly appreciated.
If you prefer for loop to the find command, how about:
#!/bin/bash
# 1st loop to print the dirnames containing space character
for d in */; do # loops over subdirectories under current directory
if [[ $d =~ [[:space:]] ]]; then # if the dirname contains a space character
echo "${d%/}" # then print the name removing the trailing slash
fi
done
# 2nd loop to print the dirnames without space character
for d in */; do
if [[ ! $d =~ [[:space:]] ]]; then # if the dirname does not contain a space character
echo "${d%/}"
fi
done
Output with the provided example:
Dir 3
Dir1
Dir2
Using GNU find:
find . -mindepth 1 -type d -name '*[[:space:]]*' # spaces
find . -mindepth 1 -type d -regex '.*/[^/[:space:]]+$' # no spaces
This is recursive.
Related
I have an audio sample library with thousands of files. I would like to shuffle/randomize the order of these files. Can someone provide me with a bash script/line that would prepend a single random character to all files in a folder (including files in sub-folders). I do not want to prepend a random character to any of the folder names though.
Example:
Kickdrum73.wav
Kickdrum SUB.wav
Kick808.mp3
Renamed to:
f_Kickdrum73.wav
!_Kickdrum SUB.wav
4_Kick808.mp3
If possible, I would like to be able to run this script more than once, but on subsequent runs, it just changes the randomly prepended character instead of prepending a new one.
Some of my attempts:
find ~/Desktop/test -type f -print0 | xargs -0 -n1 bash -c 'mv "$0" "a${0}"'
find ~/Desktop/test/ -type f -exec mv -v {} $(cat a {}) \;
find ~/Desktop/test/ -type f -exec echo -e "Z\n$(cat !)" > !Hat 15.wav
for file in *; do
mv -v "$file" $RANDOM_"$file"
done
Note: I am running on macOS.
Latest attempt using code from mr. fixit:
find . -type f -maxdepth 999 -not -name ".*" |
cut -c 3- - |
while read F; do
randomCharacter="${F:2:1}"
if [ $randomCharacter == '_' ]; then
new="${F:1}"
else
new="_$F"
fi
fileName="`basename $new`"
newFilename="`jot -r -c $fileName 1 A Z`"
filePath="`dirname $new`"
newFilePath="$filePath$newFilename"
mv -v "$F" "$newFilePath"
done
Here's my first answer, enhanced to do sub-directories.
Put the following in file randomize
if [[ $# != 1 || ! -d "$1" ]]; then
echo "usage: $0 <path>"
else
find $1 -type f -not -name ".*" |
while read F; do
FDIR=`dirname "$F"`
FNAME=`basename "$F"`
char2="${FNAME:1:1}"
if [ $char2 == '_' ]; then
new="${FNAME:1}"
else
new="_$FNAME"
fi
new=`jot -r -w "%c$new" 1 A Z`
echo mv "$F" "${FDIR}/${new}"
done
fi
Set the permissions with chmod a+x randomize.
Then call it with randomize your/path.
It'll echo the commands required to rename everything, so you can examine them to ensure they'll work for you. If they look right, you can remove the echo from the 3rd to last line and rerun the script.
cd ~/Desktop/test, then
find . -type f -maxdepth 1 -not -name ".*" |
cut -c 3- - |
while read F; do
char2="${F:2:1}"
if [ $char2 == '_' ]; then
new="${F:1}"
else
new="_$F"
fi
new=`jot -r -w "%c$new" 1 A Z`
mv "$F" "$new"
done
find . -type f -maxdepth 1 -not -name ".*" will get all the files in the current directory, but not the hidden files (names starting with '.')
cut -c 3- - will strip the first 2 chars from the name. find outputs paths, and the ./ gets in the way of processing prefixes.
while read VAR; do <stuff>; done is a way to deal with one line at a time
char2="${VAR:2:1} sets a variable char2 to the 2nd character of the variable VAR.
if - then - else sets new to the filename, either preceded by _ or with the previous random character stripped off.
jot -r -w "%c$new" 1 A Z tacks random 1 character from A-Z onto the beginning of new
mv old new renames the file
You can also do it all in bash and there are several ways to approach it. The first is simply creating an array of letters containing whatever letters you want to use as a prefix and then generating a random number to use to choose the element of the array, e.g.
#!/bin/bash
letters=({0..9} {A..Z} {a..z}) ## array with [0-9] [A-Z] [a-z]
for i in *; do
num=$(($RANDOM % 63)) ## generate number
## remove echo to actually move file
echo "mv \"$i\" \"${letters[num]}_$i\"" ## move file
done
Example Use/Output
Current the script outputs the changes it would make, you must remove the echo "..." surrounding the mv command and fix the escaped quotes to actually have it apply changes:
$ bash ../randprefix.sh
mv "Kick808.mp3" "4_Kick808.mp3"
mv "Kickdrum SUB.wav" "h_Kickdrum SUB.wav"
mv "Kickdrum73.wav" "l_Kickdrum73.wav"
You can also do it by generating a random number representing the ASCII character between 48 (character '0') through 126 (character '~'), excluding 'backtick'), and then converting the random number to an ASCII character and prefix the filename with it, e.g.
#!/bin/bash
for i in *; do
num=$((($RANDOM % 78) + 48)) ## generate number for '0' - '~'
letter=$(printf "\\$(printf '%03o' "$num")") ## letter from number
while [ "$letter" = '`' ]; do ## exclude '`'
num=$((($RANDOM % 78) + 48)) ## generate number
letter=$(printf "\\$(printf '%03o' "$num")")
done
## remove echo to actually move file
echo "mv \"$i\" \"${letter}_$i\"" ## move file
done
(similar output, all punctuation other than backtick is possible)
In each case you will want to place the script in your path or call it from within the directory you want to move the file in (you split split dirname and basename and join them back together to make the script callable passing the directory to search as an argument -- that is left to you)
I have the following directory tree:
1_loc
2_buzdfg
4_foodga
5_bardfg
6_loc
8_buzass
9_foossd
12_bardaf
There may be numbers missing in the folder ordering.
I want to separate these folders into subfolders according to their numbers, so that all folders with a number smaller than 6 (before the second _loc folder) would go to folder1 and all folders with a number equal or greater than 6 with go to folder2.
I can solve the problem very easily using the mouse, of course, but I wanted a suggestion of how to do this automatically from the terminal.
Any ideas?
while read -r line; do
# Regex match the beginning of the string for a character between 1 and 5 - change this as you'd please to any regex
FOLDERNUMBER=""
[[ "$line" ~= "^[1-5]" ]] && FOLDERNUMBER="1" || FOLDERNUMBER="2"
# So FOLDERPATH = "./folder1", if FOLDERNUMBER=1
FOLDERPATH="./folder$FOLDERNUMBER"
# Check folder exists, if not create it
[[ ! -d "$FOLDERPATH" ]] && mkdir "$FOLDERPATH"
# Finally, move the file to FOLDERPATH
mv "$line" "$FOLDERPATH/$(basename $line)"
done < <(find . -type f)
# This iterates through each line of the command in the brackets - in this case, each file path from the `find` command.
I think the solution is to loop through the files and check the number before the first _.
Firstly, let's check how to get the number before _:
$ d="1_loc_b"
$ echo "${d%%_*}"
1
OK, so this works. Then, let's loop:
for file in *
do
echo "$file"
(( ${file%%_*} > 5)) && echo "moving to dir2/" || echo "moving to dir1/"
done
Suppose folder1 and folder2 exists in the same directory, I will do it like this:
for d in *_*; do # to avoid folder1 and folder2
# check if the first field seperated by _ is less than 5
if ((`echo $d | cut -d"_" -f1` < 6)); then
mv $d folder1/$d;
else
mv $d folder2/$d;
fi;
done
(more about cut)
You can go to the current directory and run these simple commands:
mv {1,2,3,4}_* folder1/
mv {5,6,7,8}_* folder2/
This assumes no other files/directory starting with these prefixes (i.e. 1-8).
Another pure bash, parameter-expansion solution:-
#!/bin/bash
# 'find' returns folders having a '_' in their names, the flag -print0 option to
# preserve special characters in names.
# the folders are names as './1_folder', './2_folder', bash magic is done
# to remove those special characters.
# '-v' flag in 'mv' for verbose action
while IFS= read -r -d '' folder; do
folderName="${folder%_*}" # To strip the characters after the '_'
finalName="${folderName##*/}" # To strip the everything before '/'
((finalName > 5)) && mv -v "$folder" folder1 || mv -v "$folder" folder2
done < <(find . -maxdepth 1 -mindepth 1 -name "*_*" -type d -print0)
You can create the a script with the following code and when you run it, the folders will be moved as desired..
#seperate the folders into 2 folders
#this is a generic solution for any folder that start with a number
#!/bin/bash
for file in *
do
prefix=`echo $file | awk 'BEGIN{FS="_"};{print $1}'`
if [[ $prefix != ?(-)+([0-9]) ]]
then continue
fi
if [ $prefix -le 4 ]
then mv "$file" folder1
elif [ $prefix -ge 5 ]
then mv "$file" folder2
fi
done
I am trying to figure how to run a simple script "count.sh" that is called together with a path, e.g.:
count.sh /home/users/documents/myScripts
The script will need to iterate over the directories in this path and print how many files and folders (including hidden) in each level of this path.
For example:
7
8
9
10
(myScripts - 7, documents - 8, users -9, home - 10)
And by the way, can I run this script using count.sh pwd?
More or less something like that:
#!/bin/sh
P="$1"
while [ "/" != "$P" ]; do
echo "$P `find \"$P\" -maxdepth 1 -type f | wc -l`"
P=`dirname "$P"`;
done
echo "$P `find \"$P\" -maxdepth 1 -type f | wc -l`"
You can use it from the current directory with script.sh `pwd`
Another approach is to handle separation or tokenizing the path using an array and controlling word-splitting with the internal field separator (IFS). You can include the root directory if desired (you would need to trim the additional leading '/' in the printout in that case)
#!/bin/bash
[ -z "$1" -o ! -d "$1" ] && {
printf "error: directory argument required.\n"
exit 1
}
p="$1" ## remove two lines to include /
[ "${p:0:1}" = "/" ] && p="${p:1}"
oifs="$IFS" ## save internal field separator value
IFS=$'/' ## set to break on '/'
array=( $p ) ## tokenize given path into array
IFS="$oifs" ## restore original IFS
## print out path level info using wc
for ((i=0; i<${#array[#]}; i++)); do
dirnm="${dirnm}/${array[i]}"
printf "%d. %s -- %d\n" "$((i+1))" "$dirnm" $(($(ls -Al "$dirnm" | wc -l)-1))
done
Example Output
$ bash filedircount.sh /home/david/tmp
1. /home -- 5
2. /home/david -- 132
3. /home/david/tmp -- 113
As an alternative, you could use a for loop to loop through and count the items in the directory at each level instead of using wc if desired.
You could try the following
#!/bin/bash
DIR=$(cd "$1" ; pwd)
PREFIX=
until [ "$DIR" = / ] ; do
echo -n "$PREFIX"$(basename "$DIR")" "$(ls -Ab "$DIR" | wc -l)
DIR=$(dirname "$DIR")
PREFIX=", "
done
echo
(ls -Ab lists all files and folders except . and .. and escapes special characters so that only one line is printed per file even if filenames include newline characters. wc -l counts lines.)
You can invoke the script using
count.sh `pwd`
I have a file having the list of directory name I want to keep. Say file1 and its contents are names of directories like
dir1
dir2
dir3
My directory (actual directories) on the other hand has directories like
dir1
dir2
dir3
dir4
dirs
What I want to do is delete dir4, dirs and other directories of which their name doesn't exist on file1 from My directory. file1 has a directory name per line. There might be sub directories or files under dir4 and dirs which needs a recursive deletion.
I can use xargs to delete the files in the list within My directory
xargs -a file1 rm -r
But instead of removing, I want to keep them and remove the others which are not on file1. Can do
xargs -a file1 mv -t /home/user1/store/
And delete the remaining directories in my directory but I am wandering if there is a better way?
Thanks.
find . -maxdepth 1 -type d -path "./*" -exec sh -c \
'for f; do f=${f#./}; grep -qw "$f" file1 || rm -rf "$f"; done' sh {} +
Anish has a great one-liner answer for you. If you wanted something verbose that can help you in the future with data manipulation or such, here's a verbose version:
#!/bin/bash
# send this function the directory name
# it compares that name with all entries in
# file1. If entry is found, 0 is returned
# That means...do not delete directory
#
# Otherwise, 1 is returned
# That means...delete the directory
isSafe()
{
# accept the directory name parameter
DIR=$1
echo "Received $DIR"
# assume that directory will not be found in file list
IS_SAFE=1 # false
# read file line by line
while read -r line; do
echo "Comparing $DIR and $line."
if [ $DIR = $line ]; then
IS_SAFE=0 # true
echo "$DIR is safe"
break
fi
done < file1
return $IS_SAFE
}
# find all files in current directory
# and loop through them
for i in $(find * -type d); do
# send each directory name to function and
# capture the output with $?
isSafe $i
SAFETY=$?
# decide whether to delete directory or not
if [ $SAFETY -eq 1 ]; then
echo "$i will be deleted"
# uncomment below
# rm -rf $i
else
echo "$i will NOT be deleted"
fi
echo "-----"
done
you can exclude your directories using grep:
find . -mindepth 1 -maxdepth 1 -type d -printf '%P\n' | grep -f file1 -Fx -v | xargs rm -r
-printf '%P\n' is used in order to remove leading './' from directory names.
From man find, description of -printf format:
%P File's name with the name of the starting-point under which it was found removed.
grep parameters:
-f FILE Obtain patterns from FILE, one per line.
-F Interpret PATTERNS as fixed strings, not regular expressions.
-x Select only those matches that exactly match the whole line. For a regular expression pattern, this is like parenthesizing the pattern and then surrounding it with ^ and $.
-v Invert the sense of matching, to select non-matching lines.
I want to count all the files and directories from a provided folder including files and directories in a subdirectory. I have written a script which will count accurately the number of files and directory but it does not handle the subdirectories any ideas ???
I want to do it without using FIND command
#!/bin/bash
givendir=$1
cd "$givendir" || exit
file=0
directories=0
for d in *;
do
if [ -d "$d" ]; then
directories=$((directories+1))
else
file=$((file+1))
fi
done
echo "Number of directories :" $directories
echo "Number of file Files :" $file
Use find:
echo "Number of directories: $(find "$1" -type d | wc -l)"
echo "Number of files/symlinks/sockets: $(find "$1" ! -type d | wc -l)"
Using plain shell and recursion:
#!/bin/bash
countdir() {
cd "$1"
dirs=1
files=0
for f in *
do
if [[ -d $f ]]
then
read subdirs subfiles <<< "$(countdir "$f")"
(( dirs += subdirs, files += subfiles ))
else
(( files++ ))
fi
done
echo "$dirs $files"
}
shopt -s dotglob nullglob
read dirs files <<< "$(countdir "$1")"
echo "There are $dirs dirs and $files files"
find "$1" -type f | wc -l will give you the files, find "$1" -type d | wc -l the directories
My quick-and-dirty shellscript would read
#!/bin/bash
test -d "$1" || exit
files=0
# Start with 1 to count the starting dir (as find does), else with 0
directories=1
function docount () {
for d in $1/*; do
if [ -d "$d" ]; then
directories=$((directories+1))
docount "$d";
else
files=$((files+1))
fi
done
}
docount "$1"
echo "Number of directories :" $directories
echo "Number of file Files :" $files
but mind it: On my build folder for a project, there were quite some differences:
find: 6430 dirs, 74377 non-dirs
my script: 6032 dirs, 71564 non-dirs
#thatotherguy's script: 6794 dirs, 76862 non-dirs
I assume that has to do with the legions of links, hidden files etc., but I am too lazy to investigate: find is the tool of choice.
Here are some one-line commands that work without find:
Number of directories: ls -Rl ./ | grep ":$" | wc -l
Number of files: ls -Rl ./ | grep "[0-9]:[0-9]" | wc -l
Explanation:
ls -Rl lists all files and directories recursively, one line each.
grep ":$" finds just the results whose last character is ':'. These are all of the directory names.
grep "[0-9]:[0-9]" matches on the HH:MM part of the timestamp. The timestamp only shows up on file, not directories. If your timestamp format is different then you will need to pick a different grep.
wc -l counts the number of lines that matched from the grep.