Destroy Hierarchy of directory? - bash

Question

I have a folder with many folders in it and many folders below that and so on and so forth. In those final folders are are small clusters of files. I am attempting to move those files to the main folder and delete the now empty folder hierarchy. This is what I have so far.
#!/bin/bash
NAME=`whoami`
DEST="/Users/"$NAME"/Desktop/Music 2"
FILES=`find "$DEST" -type f`
for F in "$FILES"
do
mv "${F}" "${DEST}"
done
If I replace the mv command with "echo" it will catch all the right names but when I run this it gives me an error saying that the name is too long. Help will be greatly appreciated.
So say I have
/foo/bar/in/side/test1.txt
/foo/bar/in/down/test2.doc
/foo/bar/last/dog/test3.mp3
I want test1.txt, test2.doc, and test3.mp3 to be in /foo, and for each of the (now empty) directories /foo/bar, /foo/bar/in, /foo/bar/in/side, /foo/bar/in/down, /foo/bar/last, and /foo/bar/last/dog to be deleted.
End result:
/foo/test1.txt
/foo/test2.doc
/foo/test3.mp3

Answer 1

Try doing this :
find "$DEST" -type f -exec bash -c '
mv "$1" "$DEST"; rmdir "${1%/*}" &>/dev/null
' -- {} \;

Answer 2

Especially when the path names have spaces in them, using FILES=$(find ...) really doesn't work. You got the file name too long message because "$FILES" in the for loop treats all the names as a single file name; ${F} contains everything, and the mv command is trying to move a single file to ${DEST}.
The rmdir -p command removes directories that are empty (think of -p for 'prune'), working depth first.
GNU mv has a very useful option -t target for use in this context:
DEST=/foo
find /foo/bar -type f -exec mv -t "${DEST}" {} +
find /foo/bar -type d -depth -exec rmdir -p {} +
Given that you're on Mac OS X, you don't have quite that convenience, so your best bet is a slower (but equally effective):
DEST=/foo
find /foo/bar -type f -exec mv {} "${DEST}" ';'
find /foo/bar -type d -depth -exec rmdir -p {} +
This executes the mv command once per file (whereas with GNU mv, many files may be moved with a single invocation). Otherwise, it is equivalent.
Both sets of commands avoid the issues with spaces in file names.
Note that if $DEST is the same as the directory you're searching in, you'll run into problems moving the files already in $DEST over themselves. As written, the code does not avoid that problem. If necessary, you can avoid that with:
find "$DEST"/*/ -type f ...
The trailing slash enforces 'only directories' (think of it as equivalent to "$DEST"/*/.).
Proof of concept script
Remember: always test destructive scripts (scripts that delete stuff) on copies of the live material, unless you've got good backups on hand. Actually, make them on copies anyway; it is almost invariably quicker to make a copy than to recover from a backup (but you should have a backup anyway if the data is crucial).
echo "Before"
du -a .
filelist="./foo/bar/in/side/test1.txt
./foo/bar/in/down/test2.doc
./foo/bar/last/dog/test3.mp3"
for file in $filelist
do
mkdir -p $(dirname $file)
cp script $file
done
FIFO=./foo/bar/first/installment
mkdir $(dirname $FIFO)
mkfifo $FIFO
echo "Created"
du -a .
echo "Clean up"
DEST=./foo
find ./foo/bar -type f -exec mv {} "${DEST}" ';'
find ./foo/bar -depth -type d -exec rmdir -p {} + 2>/dev/null
echo "After"
du -a .
rm -fr ./foo
The rmdir -p process is noisy. It reports on directories it can't remove. The GNU version of rmdir provides an option to suppress some errors (--ignore-fail-on-non-empty), but in the context, you end up with some errors about non-existent directories, too (they were removed by the pruning process before the entry that listed the directory on its own). So, after putting up with the noise for a bit, I redirected all errors from rmdir to /dev/null. Remove that redirection until you're satisfied things work as intended.
This script should be run in an empty directory you've just created and made your current directory:
mkdir junk
cd junk
cp ../script .
sh -x ./script
Sample output:
$ sh -x script
+ echo Before
Before
+ du -a .
4 ./script
8 .
+ filelist='./foo/bar/in/side/test1.txt
./foo/bar/in/down/test2.doc
./foo/bar/last/dog/test3.mp3'
+ for file in '$filelist'
++ dirname ./foo/bar/in/side/test1.txt
+ mkdir -p ./foo/bar/in/side
+ cp script ./foo/bar/in/side/test1.txt
+ for file in '$filelist'
++ dirname ./foo/bar/in/down/test2.doc
+ mkdir -p ./foo/bar/in/down
+ cp script ./foo/bar/in/down/test2.doc
+ for file in '$filelist'
++ dirname ./foo/bar/last/dog/test3.mp3
+ mkdir -p ./foo/bar/last/dog
+ cp script ./foo/bar/last/dog/test3.mp3
+ FIFO=./foo/bar/first/installment
++ dirname ./foo/bar/first/installment
+ mkdir ./foo/bar/first
+ mkfifo ./foo/bar/first/installment
+ echo Created
Created
+ du -a .
4 ./foo/bar/in/side/test1.txt
8 ./foo/bar/in/side
4 ./foo/bar/in/down/test2.doc
8 ./foo/bar/in/down
20 ./foo/bar/in
4 ./foo/bar/last/dog/test3.mp3
8 ./foo/bar/last/dog
12 ./foo/bar/last
0 ./foo/bar/first/installment
4 ./foo/bar/first
40 ./foo/bar
44 ./foo
4 ./script
52 .
+ echo 'Clean up'
Clean up
+ DEST=./foo
+ find ./foo/bar -type f -exec mv '{}' ./foo ';'
+ find ./foo/bar -depth -type d -exec rmdir -p '{}' +
+ echo After
After
+ du -a .
0 ./foo/bar/first/installment
4 ./foo/bar/first
8 ./foo/bar
4 ./foo/test1.txt
4 ./foo/test3.mp3
4 ./foo/test2.doc
24 ./foo
4 ./script
32 .
+ rm -fr ./foo
Note that this script carefully creates a non-file (a FIFO) in a separate directory under ./foo/bar and shows that it is left behind. Comment out the mkfifo line that create the FIFO and the run looks like:
$ sh -x script
+ echo Before
Before
+ du -a .
4 ./script
8 .
+ filelist='./foo/bar/in/side/test1.txt
./foo/bar/in/down/test2.doc
./foo/bar/last/dog/test3.mp3'
+ for file in '$filelist'
++ dirname ./foo/bar/in/side/test1.txt
+ mkdir -p ./foo/bar/in/side
+ cp script ./foo/bar/in/side/test1.txt
+ for file in '$filelist'
++ dirname ./foo/bar/in/down/test2.doc
+ mkdir -p ./foo/bar/in/down
+ cp script ./foo/bar/in/down/test2.doc
+ for file in '$filelist'
++ dirname ./foo/bar/last/dog/test3.mp3
+ mkdir -p ./foo/bar/last/dog
+ cp script ./foo/bar/last/dog/test3.mp3
+ FIFO=./foo/bar/first/installment
++ dirname ./foo/bar/first/installment
+ mkdir ./foo/bar/first
+ echo Created
Created
+ du -a .
4 ./foo/bar/in/side/test1.txt
8 ./foo/bar/in/side
4 ./foo/bar/in/down/test2.doc
8 ./foo/bar/in/down
20 ./foo/bar/in
4 ./foo/bar/last/dog/test3.mp3
8 ./foo/bar/last/dog
12 ./foo/bar/last
4 ./foo/bar/first
40 ./foo/bar
44 ./foo
4 ./script
52 .
+ echo 'Clean up'
Clean up
+ DEST=./foo
+ find ./foo/bar -type f -exec mv '{}' ./foo ';'
+ find ./foo/bar -depth -type d -exec rmdir -p '{}' +
+ echo After
After
+ du -a .
4 ./foo/test1.txt
4 ./foo/test3.mp3
4 ./foo/test2.doc
16 ./foo
4 ./script
24 .
+ rm -fr ./foo
$
This strongly suggests that if written properly and handled carefully, the code above does work correctly without damaging the system. But you should still be cautious before using any variant of this in production (and even in test).
Tests run on an Ubuntu 12.04 derivative.