I have two folders each which contain a lot of subfolders. Most of the files and folders should be the same, but I want to be able to figure out which folders are missing.
For example:
Folder1/
A/
1.jpg
B/
2.jpg
C/
3.jpg
D/
4.jpg
and
Folder2/
A/
1.jpg
E/
2.jpg
C/
3.jpg
D/
4.jpg
Is there any way to know that "B" got deleted? I'm running windows, but I have cygwin installed so bash scripts, diff, or python/perl would work.
I know I can just "diff -q -r Folder1 Folder2" everything in both folders, but that takes FOREVER and spits out everything that's changed, including files in those folders, where I just need the folders themselves.
Any suggestions?
Thanks!
diff -u <(cd Folder1 ; find | sort) <(cd Folder2 ; find | sort)
Some notes:
This would include files that are added/removed, but not files that are merely modified. If you don't even want to include files that are added/removed, change find to find -type d, as herby suggests.
If a given directory is added/removed, this will also list out all the files and directories within that directory. If that's a problem, you can address it by appending something like | perl -ne 'print unless m/^\Q$previous\E\//; $previous = $_;'.
Barron's answer makes me realize that you didn't actually specify that you need the folders to be examined recursively. If you just need the very top level, you can change find to either find -maxdepth 1 (or find -maxdepth 1 -type d) or ls -d * (or ls -d */), as you prefer.
(cd Folder1 && find . -type d >/tmp/$$.1)
(cd Folder2 && find . -type d >/tmp/$$.2)
diff /tmp/$$.1 /tmp/$$.2
rm /tmp/$$.1 /tmp/$$.2
This is how I hacked it together in bash:
dirs=`ls $PWD/Folder1`
for dir in ${dirs[*]}; do
if [ ! -e $PWD/Folder2/$dir ]; then
echo "$dir missing"
fi
done
I make no claim that this is an ideal solution, but since I'm also learning bash, I'd be interested to hear why this is a particularly good or bad way to go about it.
If you really want only one level of nesting, you can do this:
(cd Folder1 && find -type d -mindepth 1 -maxdepth 1) >list1
(cd Folder2 && find -type d -mindepth 1 -maxdepth 1) >list2
while read dir; do
fgrep -qx "$dir" list2 || echo "\"$dir\" has been deleted"
done <list1
If you are sure only to have directories in both folders, replace the find commands with a simple ls.
Related
I have many folders with files in them, in each of them I have a subfolder called wvfm . What I want to do is move all the files into each of the wvfm folder.
I tried doing the line below but it is not working
for i in "./*"; mv $i "./wvfm"; done
but that didn't work quite right
Use a for loop (for i in */; do) to list the files, then move to the folders to list and save all files in to a variable (F=$(ls)).
Then move all your files with excluding your folder (mv ${F/wvfm/} wvfm), like this:
#!/bin/bash
subdirectory="wvfm"
for i in */; do
cd $i
F=$(ls)
mv ${F/$subdirectory/} $subdirectory
cd ../
done
find * -maxdepth 0 -type d | xargs -n 1 -I {} mv {}/* {}/wvfm
should do the trick; quick and dirty. Not a MacOS user, but works in bash.
Explanation:
find * -maxdepth 0 -type d find all directories at depth 0 (ie: do not descend dirs),
pipe to xargs, using options -n 1, operate on one value at a time, -I replace-str, string replacement (see man xargs).
action command: mv {}/* {}/wvfm substitues to mv dirA/* dirA/wvfm for each dir match.
You will get an "error", mv: cannot move /wvfm' to a subdirectory of itself, 'wvfm/wvfm', but you can ignore / take advantage of it (quick and dirty).
You could cover all bases with:
for entry in $(find * -maxdepth 0 -type d); do
(cd $entry; [[ -d wvfm ]] && \
find * -maxdepth 0 ! -name wvfm -exec mv {} wvfm \;
)
done
find * -maxdepth 0 -type d, again, find only the top-level directories,
in a subshell, change to the input directory, if there's a directory wvfm,
look for all contents except the wvfm directory and -exec the mv command
exiting the subshell leaves you back in the starting (root) directory, ready for next input.
I have some folders which needs my regular attention:
dir1
dir2
dir3
I know that I can safely remove them if anything inside has not been changed for 10 days. But how to do it?
I wanted to use "find /basedir/ -maxdepth 1 -mtime +10 -print | xargs -1 rm -f"
But this deletes those dirs which has not been modyfied but even if inside those folders were modyfied. Incomplete content of any of dir1,dir2 or dir3 is useless so I need to decide if delete whole dir1-3 or leave it based on how old complete content is.
Does anyone know easy way to do it?
Don't use -maxdepth. Iterate over the directories in a for loop:
for dir in /basedir/dir{1,2,3} ; do
if ! find "$dir" -mtime -10 | grep -q ^ ; then
rm -rf "$dir"
fi
done
I had been playing around with mv, and now I have a situation.
Earlier, say
Folder1 had file1,2,3.
Now Folder1 has Folder2 which has Folder3 which has Folder4 which contains file1,2,3.
I am trying to write a bash script such that it identifies intermediate folders containing only 1 directory and moves all its contents up one level, ultimately giving back only Folder1->file1,2,3, and rest folders deleted.
I tried to write something like below, but I am :
1.unable to distinguish between file and folder
2.unable to find the file/directory name stored inside current folder
3.Not sure how to do recursively.
#!/bin/bash
echo "Directory Name?"
read dir_name
no_files=`ls -A| wc -l`
if [ $no_file==1 ] && [ itisaDirectory()];
then `mv folder_name/* dir_name`
fi
When you do not care for error messages and want to move all files in subdirs to the current dir and remove the remaining empty dir, do something like
find . -type f -exec mv {} "${dir_name}" \; 2>/dev/null
rm -r */
You ask for something else, only move files where an intermediate directory is unique. That is the case if exactly one subdir has that dir as a parent. The parent of a dir can be found with dirname.
When a dir has one subdir, only one subdir will have it as a parent. You can list all dirs, look for the parent and select the unique paths.
find . -type d -exec dirname {} \; | sort | uniq -u | while read dir; do
echo "${dir} has exactly one subdir"
done
The problem is that the dir can have files as well. We try to improve the above solution:
find . -exec dirname {} \; | sort | uniq -u | while read dir; do
echo "${dir} has exactly one subdir or one file"
done
You can test the content of the dir with if [ -d "${dir}/*" ] but I do not need to know:
find . -exec dirname {} \; | sort | uniq -u | while read dir; do
echo "${dir} has exactly one subdir or one file"
find "${dir}"/*/ -type f -exec mv {} "${dir_name}" \; 2>/dev/null
done
The path ${dir}/*/ will only exist when ${dir} has a subdirectory in it, and will move the files beneath. When $dir only has one file, the find command will find nothing.
I have a directory structure
Dir_1
Dir_2
Dir_3
Source
. The directory Source contains the files File_1.txt and File_2.txt.
I want to copy all the files from the directory Source to all the remaining directories, in this case Dir_1, Dir_2 and Dir_3.
For this, I used
for i in $(ls -d */ | grep -v 'Source'); do echo $i | xargs -n 1 cp ./Source/*; done
. I, however, keep getting the message
cp: target ‘5’ is not a directory
It seems cp has problems with the directory names which have spaces in them. How do I resolve this (keeping the spaces in the directory names, obviously)?
Using find you could do something like this:
find . -mindepth 1 -maxdepth 1 -type d ! -name Source -exec cp Source/*.txt {} \;
This command searches the current directory for all subdirectories one level deep, excluding Source and then copies the text files into each.
Hope this helps :)
I have two separate folder directories, which mostly contain the same files, but the directory structure is completely different between the two folders. The filenames do not correspond either
So, for example:
FOLDER 1
--- Subfolder A
-file1
-file2
--- Subfolder B
-file3
-file4
FOLDER 2
--- Subfolder C
-Subfolder C1
-file5
-file6
-file7
-Subfolder C2
-file8
-file9
Let's suppose that file1=file5, file2=file6, file3=file7, file4=file8
And file9 is unmatched.
Is there some combination of options to the diff command that will identify the matches? Doing a recursive diff with -r doesn't seem to do the job.
This is a way to get the different and/or identical files with find and xargs:
find FOLDER1 -type f -print0 |
xargs -0 -I % find FOLDER2 -type f -exec diff -qs --from-file="%" '{}' \+
Sample output:
Files FOLDER1/SubfolderB/file3 and FOLDER2/SubfolderC/SubfolderC1/file5 differ
Files FOLDER1/SubfolderB/file3 and FOLDER2/SubfolderC/SubfolderC1/file7 are identical
So, you can filter the ones you want with grep (see example).
Notice this solution supports filenames with spaces and special characters (e.g.: newlines) embedded, so you don't have to worry about it
Explanation
For every file in FOLDER1 (find FOLDER1 -type f -print0), executes:
find FOLDER2 -type f -exec diff -qs --from-file="%" '{}' \+
That line calls find again to get all the files in FOLDER2 and executes the following (processed):
diff -qs --from-file="<a file from FOLDER1>" <all the files from FOLDER2>
From man diff:
--from-file=FILE1
Compare FILE1 to all operands. FILE1 can be a directory.
Example
This is the directory tree and the file content:
$ find FOLDER1 FOLDER2 -type f -exec sh -c 'echo "$0": && cat "$0"' '{}' \;
FOLDER1/SubfolderA/file1:
1=5
FOLDER1/SubfolderA/file2:
2=6
FOLDER1/SubfolderB/file3:
3=7
FOLDER1/SubfolderB/file4:
4=8
FOLDER2/SubfolderC/SubfolderC1/file5:
1=5
FOLDER2/SubfolderC/SubfolderC1/file6:
2=6
FOLDER2/SubfolderC/SubfolderC1/file7:
3=7
FOLDER2/SubfolderC/SubfolderC2/file8:
4=8
FOLDER2/SubfolderC/SubfolderC2/file9:
anything
And this is the command (pipeline) getting just the identical ones:
$ find FOLDER1 -type f -print0 |
> xargs -0 -I % find FOLDER2 -type f -exec diff -qs --from-file="%" '{}' \+ |
> grep "identical$"
Files FOLDER1/SubfolderA/file1 and FOLDER2/SubfolderC/SubfolderC1/file5 are identical
Files FOLDER1/SubfolderA/file2 and FOLDER2/SubfolderC/SubfolderC1/file6 are identical
Files FOLDER1/SubfolderB/file3 and FOLDER2/SubfolderC/SubfolderC1/file7 are identical
Files FOLDER1/SubfolderB/file4 and FOLDER2/SubfolderC/SubfolderC2/file8 are identical
Enhanced solution with bash's Process Substitution and Arrays
If you're using bash, you can first save all the FOLDER2 filenames in an array to avoid calling find for each file in FOLDER1:
# first of all, we save all the FOLDER2 filenames (recursively) in an array
while read -d $'\0' file; do
folder2_files=("${folder2_files[#]}" "$file")
done < <(find FOLDER2 -type f -print0)
# now we compare each file in FOLDER1 with the files in the array
find FOLDER1 -type f -exec diff -qs --from-file='{}' "${folder2_files[#]}" \; |
grep "identical$"
Create a temporary Git repository. Add the first directory tree to it, and commit.
Remove all the files and add the second directory tree to it. Do the second commit.
The git diff between those two commits will turn on rename detection and you will probably see something more englightening.