Currently I have multiple directories
Directory1 Directory2 Directory3 Directory4
each of these directories contain files (the files are somewhat cryptic)
what i wish to do is scan files within the folders to see if certain files are present, if they are then leave that folder alone, if the certain files are not present then just delete the entire directory. here is what i mean:
im searching for the files that have the word .pass. in the filename.
Say Directory 4 has that file that im looking for
Direcotry4:
file1.temp.pass.exmpl
file1.temp.exmpl
file1.tmp
and the rest of the Directories do not have that specific file:
file.temp
file.exmp
file.tmp.other
so i would like to delete Directory1,2 and3 But only keep Directory 4...
So far i have come up with this code
(arr is a array of all the directory names)
for x in ${arr[#]}
do
find $x -type f ! -name "*pass*" -exec rd {} $x\;
done
another way i have thought of doing this is like this:
for x in ${arr[#]}
do
cd $x find . -type f ! -name "*Pass*" | xargs -i rd {} $x/
done
SO far these don't seem to work, and im scared that i might do something wrong and have all my files deleted.....(i have backed up)
is there any way that i can do this? remember i want Directory 4 to be unchanged, everything in it i want to keep
To see if your directory contains a pass file:
if [ "" = "$(find directory -iname '*pass*' -type f | head -n 1)" ]
then
echo notfound
else
echo found
fi
To do that in a loop:
for x in "${arr[#]}"
do
if [ "" = "$(find "$x" -iname '*pass*' -type f | head -n 1)" ]
then
rm -rf "$x"
fi
done
Try this:
# arr is a array of all the directory names
for x in ${arr[#]}
do
ret=$(find "$x" -type f -name "*pass*" -exec echo "0" \;)
# expect zero length $ret value to remove directory
if [ -z "$ret" ]; then
# remove dir
rm -rf "$x"
fi
done
Related
Looking for a bash script:
Here's the situation:
I have 1000's folders and subfolders on my Backup Directory Drive
lets say.....
/backup
/backup/folderA
/backup/folderA/FolderAA
/backup/folderB
/backup/folderB/FolderBB
I have Dozens of similar folders in a secondary location (with files in them) and the Folder names will match one of the folders or subfolders in the main backup drive.
I would like to move all contents of specific extension types from my secondary location $FolderName to the Backup location + matching subfolder ONLY if the $FolderName matches exactly and remove the folders from my secondary location!
If there is no corrosponding folder or subfolder in the backup location then leave the source folders & files alone.
looking forward to getting some help/guidance.
Mike
Additional info requested.Expected input and ouput
Lets say i have the following:
Backup Folder
/backup/test/file.bak
And for my secondary folder location:
/secondarylocation/mike/test/hello/john.bak
/secondarylocation/mike/test/hello/backup.zip
i would like this as the end result
/backup/test/file.bak
/backup/test/john.bak
/backup/test/backup.zip
and /secondarylocation/mike/test *and sub folders and files removed
run this script with quoted folders and file types:
./merge.sh "backup" "secondarylocation/mike" "*.zip" "*.bak"
replace -iname with -name if you want to search for suffix case sensitive
replace mv -fv with mv -nv when you don't want to overwrite duplicate file names
add -mindepth 1 to last find if you want to keep empty folder test
merge.sh
#!/bin/bash
# read folders from positional parameters
[ -d "$1" ] && targetf="$1" && shift
[ -d "$1" ] && sourcef="$1" && shift
if [ -z "$targetf" ] || [ -z "$sourcef" ]
then
echo -e "usage: ./merge.sh <targetfolder> <sourcefolder> [PATTERN]..."
exit 1
fi
# add prefix -iname for each pattern
while [ ${pattern:-1} -le $# ]
do
set -- "$#" "-iname \"$1\""
shift
pattern=$((${pattern:-1}+1))
done
# concatenate all prefix+patterns with -o and wrap in parentheses ()
if (( $# > 1 ))
then
pattern="\( $1"
while (( $# > 1 ))
do
pattern="$pattern -o $2"
shift
done
pattern="$pattern \)"
else
pattern="$1"
fi
# move files from searchf to destf
find "$targetf" -mindepth 1 -type d -print0 | sort -z | while IFS=$'\0' read -r -d $'\0' destf
do
find "$sourcef" -mindepth 1 -type d -name "${destf##*/}" -print0 | sort -z | while IFS=$'\0' read -r -d $'\0' searchf
do
if (( $# ))
then
# search with pattern
eval find "\"$searchf\"" -depth -type f "$pattern" -exec mv -fv {} "\"$destf\"" \\\;
else
# all files
find "$searchf" -depth -type f -exec mv -fv {} "$destf" \;
fi
# delete empty folders
find "$searchf" -depth -type d -exec rmdir --ignore-fail-on-non-empty {} +
done
done
exit 0
this will merge hello into test (earn the fruits and cut the tree)
I have been trying to recursively rename folders whose names ends in four digits.
For example, I have a folder name like this:
this is the name 2004
and I'm trying to rename it to:
this is the name (2004)
I've tried to split the prefix and digit parts of the name however I cannot mv as rename these folder.
Here is the code I've tried so far:
#!/bin/bash
F=$(find . -name '*[0-9]' -type d)
for i in "$F";
do
R2=$(echo "$i" | awk '{print $NF}')
R1=$(echo "$i" | sed 's/.\{4\}$//')
R3=$(echo "$R2" | sed -r "s/(^[0-9]+$)/(\1)/g")
mv "$i" "$R1 $R3"
# Even tried:
mv "\"$i"\" "\"$R2 $R3"\"
done
Does anyone can review or/and suggest some guidance to allow mv to find the initial folder and its destination?
following command:
find -name '*[0-9][0-9][0-9][0-9]' -type d -exec bash -c 'for dir; do mv "$dir" "${dir%[0-9][0-9][0-9][0-9]}(${dir#${dir%[0-9][0-9][0-9][0-9]}})"; done' - {} + -prune
should work.
double quote arround variable expansion
${dir%[0-9][0-9][0-9][0-9]} to remove last 4 digits suffix
${dir#${dir%[0-9][0-9][0-9][0-9]}} to remove previous prefix
-exec bash -c '..' - {} + the - to skip the first argument after -c command which is taken for $0, see man bash /-c
-prune at the end to prevent to search in sub tree when matched, (suppose 2004/2004 then mv 2004/2004 "2004/(2004)" or mv 2004/2004 (2004)/2004' would fail)
I found Bash annoying when it comes to find and rename files for all the escaping one needs to make. This is a cleaner Ruby solution :
#!/usr/bin/ruby
require 'fileutils'
dirs = Dir.glob('./**/*').select {|x| x =~ / [0-9]*/ }
dirs.sort().reverse().each do |dir|
new_name=dir.gsub(/(.*)( )([0-9]{4})/, '\1\2(\3)')
FileUtils.mv dir,new_name
end
When $F has more than one directoy, the for loop will consider it as a one long entry with newlines (try echo "F=[$F]").
Also use -depth, you might have topdir 2004/subdir 2004. So first rename the subdir.
When the directories don't have newlines, you can try
#!/bin/bash
while IFS= read -r orgdir; do
mv "${orgdir}" "$(sed -r "s/([0-9]+$)/(\1)/g" <<< "${orgdir}")"
done < <(find . -depth -name '*[0-9]' -type d)
Prompt the user for a file name, without the path (Ex: xyz.out)
- Using the find command, provide the full path to the file
- At the end, “print number of locations of that file found”
- If it’s not found, then display “not found
and this is my script
#! /bin /bash
echo "please enter your file name"
read filename
if [ -f $filename ];
then
echo "file $filename found"
find $PWD -type f | grep $filename
#find "$(cd ..; pwd)" -name $filename
else
echo "file $filename was not found"
fi
but the thing is At the end, i need to “print number of locations of that file found”
help me out with this
Something like this to get the count:
find $PWD -type f -name $filename 2>/dev/null | wc -l
This should work:
find "$PWD" -type f -name "$fname" |grep "" -c
In trying to keep it as short as possible, one approach with Posix shell would be to fill a temporary file with the file names returned by find, cat the file to provide your output, and use wc to provide the line count (note: you use your own pattern instead of "*cmpf*" shown below):
$ find . -name "*cmpf*" -printf "%P\n" >tmp; cat tmp; c=$(wc -l <tmp); \
rm tmp; printf "[%s] files found\n" $c
cmpf1f2.c
cmpf1f2_2.c
bin/cmpf1f2_2
bin/cmpf1f2
snip/cmpf1f2_notes.txt
cmpf1f2_for.c
[6] files found
If bash is available, another approach is to read the matching files into an array and then use the number of elements as your count. Example:
$ a=( $(find . -name "*cmpf*" -printf "%P\n") ); printf "%s\n" ${a[#]}; \
printf -- "--\n'%s' files found.\n" ${#a[#]}
cmpf1f2.c
cmpf1f2_2.c
bin/cmpf1f2_2
bin/cmpf1f2
snip/cmpf1f2_notes.txt
cmpf1f2_for.c
--
'6' files found.
Both approaches give you both the files and directories in which they reside as well as the count of the files returned.
Note: if you would like ./ before each file and directory names, use the %p format instead of %P above.
I have a folder with 320G images, I want to move the images to 5 sub folders randomly(just need to move to 5 sub folders). But I know nothing on bash scripts.Please could someone help? thanks!
You could move the files do different directories based on their first letter:
mv [A-Fa-f]* dir1
mv [F-Kf-k]* dir2
mv [^A-Ka-k]* dir3
Here is my take on this. In order to use it place the script somewhere else (not in you folder) but run it from your folder. If you call your script file rmove.sh, you can place it in, say ~/scripts/, then cd to your folder and run:
source ~/scripts/rmove.sh
#/bin/bash
ndirs=$((`find -type d | wc -l` - 1))
for file in *; do
if [ -f "${file}" ]; then
rand=`dd if=/dev/random bs=1 count=1 2>/dev/null | hexdump -b | head -n1 | cut -d" " -f2`
rand=$((rand % ndirs))
i=0
for directory in `find -type d`; do
if [ "${directory}" = . ]; then
continue
fi
if [ $i -eq $rand ]; then
mv "${file}" "${directory}"
fi
i=$((i + 1))
done
fi
done
Here's my stab at the problem:
#!/usr/bin/env bash
sdprefix=subdir
dirs=5
# pre-create all possible sub dirs
for n in {1..5} ; do
mkdir -p "${sdprefix}$n"
done
fcount=$(find . -maxdepth 1 -type f | wc -l)
while IFS= read -r -d $'\0' file ; do
subdir="${sdprefix}"$(expr \( $RANDOM % $dirs \) + 1)
mv -f "$file" "$subdir"
done < <(find . -maxdepth 1 -type f -print0)
Works with huge numbers of files
Does not beak if a file is not moveable
Creates subdirectories if necessary
Does not break on unusual file names
Relatively cheap
Any scripting language will do so I'll write in Python here:
#!/usr/bin/python
import os
import random
new_paths = ['/path1', '/path2', '/path3', '/path4', '/path5']
image_directory = '/path/to/images'
for file_path in os.listdir(image_directory):
full_path = os.path.abspath(os.path.join(image_directory, file_path))
random_subdir = random.choice(new_paths)
new_path = os.path.abspath(os.path.join(random_subdir, file_path))
os.rename(full_path, new_path)
mv `ls | while read x; do echo "`expr $RANDOM % 1000`:$x"; done \
| sort -n| sed 's/[0-9]*://' | head -1` ./DIRNAME
run it in your current image directory, this command will select one file at a time and move it to ./DIRNAME, iterate this command until there are no more files to move.
Pay attention that ` is backquotes and not just quotes characters.
I need to loop through a directory recursively and remove all files with extension .pdf and .doc. I'm managing to loop through a directory recursively but not managing to filter the files with the above mentioned file extensions.
My code so far
#/bin/sh
SEARCH_FOLDER="/tmp/*"
for f in $SEARCH_FOLDER
do
if [ -d "$f" ]
then
for ff in $f/*
do
echo "Processing $ff"
done
else
echo "Processing file $f"
fi
done
I need help to complete the code, since I'm not getting anywhere.
As a followup to mouviciel's answer, you could also do this as a for loop, instead of using xargs. I often find xargs cumbersome, especially if I need to do something more complicated in each iteration.
for f in $(find /tmp -name '*.pdf' -or -name '*.doc'); do rm $f; done
As a number of people have commented, this will fail if there are spaces in filenames. You can work around this by temporarily setting the IFS (internal field seperator) to the newline character. This also fails if there are wildcard characters \[?* in the file names. You can work around that by temporarily disabling wildcard expansion (globbing).
IFS=$'\n'; set -f
for f in $(find /tmp -name '*.pdf' -or -name '*.doc'); do rm "$f"; done
unset IFS; set +f
If you have newlines in your filenames, then that won't work either. You're better off with an xargs based solution:
find /tmp \( -name '*.pdf' -or -name '*.doc' \) -print0 | xargs -0 rm
(The escaped brackets are required here to have the -print0 apply to both or clauses.)
GNU and *BSD find also has a -delete action, which would look like this:
find /tmp \( -name '*.pdf' -or -name '*.doc' \) -delete
find is just made for that.
find /tmp -name '*.pdf' -or -name '*.doc' | xargs rm
Without find:
for f in /tmp/* tmp/**/* ; do
...
done;
/tmp/* are files in dir and /tmp/**/* are files in subfolders. It is possible that you have to enable globstar option (shopt -s globstar).
So for the question the code should look like this:
shopt -s globstar
for f in /tmp/*.pdf /tmp/*.doc tmp/**/*.pdf tmp/**/*.doc ; do
rm "$f"
done
Note that this requires bash ≥4.0 (or zsh without shopt -s globstar, or ksh with set -o globstar instead of shopt -s globstar). Furthermore, in bash <4.3, this traverses symbolic links to directories as well as directories, which is usually not desirable.
If you want to do something recursively, I suggest you use recursion (yes, you can do it using stacks and so on, but hey).
recursiverm() {
for d in *; do
if [ -d "$d" ]; then
(cd -- "$d" && recursiverm)
fi
rm -f *.pdf
rm -f *.doc
done
}
(cd /tmp; recursiverm)
That said, find is probably a better choice as has already been suggested.
Here is an example using shell (bash):
#!/bin/bash
# loop & print a folder recusively,
print_folder_recurse() {
for i in "$1"/*;do
if [ -d "$i" ];then
echo "dir: $i"
print_folder_recurse "$i"
elif [ -f "$i" ]; then
echo "file: $i"
fi
done
}
# try get path from param
path=""
if [ -d "$1" ]; then
path=$1;
else
path="/tmp"
fi
echo "base path: $path"
print_folder_recurse $path
This doesn't answer your question directly, but you can solve your problem with a one-liner:
find /tmp \( -name "*.pdf" -o -name "*.doc" \) -type f -exec rm {} +
Some versions of find (GNU, BSD) have a -delete action which you can use instead of calling rm:
find /tmp \( -name "*.pdf" -o -name "*.doc" \) -type f -delete
For bash (since version 4.0):
shopt -s globstar nullglob dotglob
echo **/*".ext"
That's all.
The trailing extension ".ext" there to select files (or dirs) with that extension.
Option globstar activates the ** (search recursivelly).
Option nullglob removes an * when it matches no file/dir.
Option dotglob includes files that start wit a dot (hidden files).
Beware that before bash 4.3, **/ also traverses symbolic links to directories which is not desirable.
This method handles spaces well.
files="$(find -L "$dir" -type f)"
echo "Count: $(echo -n "$files" | wc -l)"
echo "$files" | while read file; do
echo "$file"
done
Edit, fixes off-by-one
function count() {
files="$(find -L "$1" -type f)";
if [[ "$files" == "" ]]; then
echo "No files";
return 0;
fi
file_count=$(echo "$files" | wc -l)
echo "Count: $file_count"
echo "$files" | while read file; do
echo "$file"
done
}
This is the simplest way I know to do this:
rm **/#(*.doc|*.pdf)
** makes this work recursively
#(*.doc|*.pdf) looks for a file ending in pdf OR doc
Easy to safely test by replacing rm with ls
The following function would recursively iterate through all the directories in the \home\ubuntu directory( whole directory structure under ubuntu ) and apply the necessary checks in else block.
function check {
for file in $1/*
do
if [ -d "$file" ]
then
check $file
else
##check for the file
if [ $(head -c 4 "$file") = "%PDF" ]; then
rm -r $file
fi
fi
done
}
domain=/home/ubuntu
check $domain
There is no reason to pipe the output of find into another utility. find has a -delete flag built into it.
find /tmp -name '*.pdf' -or -name '*.doc' -delete
The other answers provided will not include files or directories that start with a . the following worked for me:
#/bin/sh
getAll()
{
local fl1="$1"/*;
local fl2="$1"/.[!.]*;
local fl3="$1"/..?*;
for inpath in "$1"/* "$1"/.[!.]* "$1"/..?*; do
if [ "$inpath" != "$fl1" -a "$inpath" != "$fl2" -a "$inpath" != "$fl3" ]; then
stat --printf="%F\0%n\0\n" -- "$inpath";
if [ -d "$inpath" ]; then
getAll "$inpath"
#elif [ -f $inpath ]; then
fi;
fi;
done;
}
I think the most straightforward solution is to use recursion, in the following example, I have printed all the file names in the directory and its subdirectories.
You can modify it according to your needs.
#!/bin/bash
printAll() {
for i in "$1"/*;do # for all in the root
if [ -f "$i" ]; then # if a file exists
echo "$i" # print the file name
elif [ -d "$i" ];then # if a directroy exists
printAll "$i" # call printAll inside it (recursion)
fi
done
}
printAll $1 # e.g.: ./printAll.sh .
OUTPUT:
> ./printAll.sh .
./demoDir/4
./demoDir/mo st/1
./demoDir/m2/1557/5
./demoDir/Me/nna/7
./TEST
It works fine with spaces as well!
Note:
You can use echo $(basename "$i") # print the file name to print the file name without its path.
OR: Use echo ${i%/##*/}; # print the file name which runs extremely faster, without having to call the external basename.
Just do
find . -name '*.pdf'|xargs rm
If you can change the shell used to run the command, you can use ZSH to do the job.
#!/usr/bin/zsh
for file in /tmp/**/*
do
echo $file
done
This will recursively loop through all files/folders.
The following will loop through the given directory recursively and list all the contents :
for d in /home/ubuntu/*;
do
echo "listing contents of dir: $d";
ls -l $d/;
done