What is causing "Directory not empty" errors? - bash

I've written a script that is mostly working for me. However, the two rmdir commands at the end are sometimes returning Directory not empty errors. How do I beat this? I tried adding r and then -rf which returned Illegal option errors.
I'm a rookie, and I'd be grateful for any help. Here is the full shell script, which, again, is mostly working beautifully:
if [[ -z "${1}" ]]; then
die "FolderName Required"
fi
newDirName="DirectoryName"
newBaseDir="/Users/JSG/Desktop/DataFarm/$1/"
/bin/mkdir -p $newBaseDir/{ProtectedOrig,Data}
echo -n "---Data Folder Setup
---Data Introduction
---Data Audit/Manipulation
---Data Queries" > $newBaseDir/Data/$1_DataJournal.txt
ditto NewData/ NewDataCopy
fab deploy_data_to_s3:data=*
mv NewData/ $newBaseDir/ProtectedOrig/NewData
mv NewDataCopy/ $newBaseDir/Data/NewDataCopy
mv $newBaseDir/Data/NewDataCopy/* $newBaseDir/Data/
rmdir $newBaseDir/Data/NewDataCopy
mv $newBaseDir/ProtectedOrig/NewData/* $newBaseDir/ProtectedOrig/
rmdir $newBaseDir/ProtectedOrig/NewData
chflags -R uchg $newBaseDir/ProtectedOrig/
mkdir NewData
What am I missing? And thanks in advance!

For the rmdir command, you need to add the --ignore-fail-on-non-empty flag so it deletes the directory even if files are in there like so:
rmdir --ignore-fail-on-non-empty $newBaseDir/Data/NewDataCopy
You could also just use rm -r too:
rm -r $newBaseDir/Data/NewDataCopy
From the Wikipedia Entry:
rmdir will not remove a directory if it is not empty in UNIX. The correct way to remove a directory and all its contents recursively is with the rm command.

Check for any files in the directory that start with .. I note you're moving *, but if there's a file called, for example, .hello, then * will not match this file and as a result the directory will not be empty when you come to do an rmdir.

Related

Create folders with filenames and move files into folder

I have a bunch of files that end with .EHZ and I am trying to create a folder for each of these files and then move them into the corresponding folders. Also the files have a Z attached to the name that I would like to remove. So ideally i would have, for example these files
AAAAZBBBBZ.EHZ
CCCCZDDDDZ.EHZ
EEEEZFFFFZ.EHZ
Turn into the folders
AAAABBBB
CCCCDDDD
EEEEFFFF
What I have written so far is
dir0=pwd
for file in `ls *.EHZ`
do
name=echo $file | head 10
mkdir -p $name
mv "$file" "$name"
done
P.S. I have found many answers in stackoverflow that were addressed to this issue but not on shell so I figured I would ask.
Could you please try following. I have used cp command for safer side in my code, you could remove it and put it as mv in there to actually move files to folders.
for file in *.EHZ
do
name="${file%.*}"
actual_directory_name="${name::-1}"
if [[ ! -d "$actual_directory_name" ]]
then
mkdir "$actual_directory_name"
fi
cp "$file" "$actual_directory_name"
if [[ -s "$actual_directory_name/$file" ]]
then
echo "$file is moved Successfully to directory name $actual_directory_name now.."
else
echo "Please check seems $file is NOT moved to directory $actual_directory_name."
fi
done

How to move files from subfolders to their parent directory (unix, terminal)

I have a folder structure like this:
A big parent folder named Photos. This folder contains 900+ subfolders named a_000, a_001, a_002 etc.
Each of those subfolders contain more subfolders, named dir_001, dir_002 etc. And each of those subfolders contain lots of pictures (with unique names).
I want to move all these pictures contained in the subdirectories of a_xxx inside a_xxx. (where xxx could be 001, 002 etc)
After looking in similar questions around, this is the closest solution I came up with:
for file in *; do
if [ -d $file ]; then
cd $file; mv * ./; cd ..;
fi
done
Another solution I got is doing a bash script:
#!/bin/bash
dir1="/path/to/photos/"
subs= `ls $dir1`
for i in $subs; do
mv $dir1/$i/*/* $dir1/$i/
done
Still, I'm missing something, can you help?
(Then it would be nice to discard the empty dir_yyy, but not much of a problem at the moment)
You could try the following bash script :
#!/bin/bash
#needed in case we have empty folders
shopt -s nullglob
#we must write the full path here (no ~ character)
target="/path/to/photos"
#we use a glob to list the folders. parsing the output of ls is baaaaaaaddd !!!!
#for every folder in our photo folder ...
for dir in "$target"/*/
do
#we list the subdirectories ...
for sub in "$dir"/*/
do
#and we move the content of the subdirectories to the parent
mv "$sub"/* "$dir"
#if you want to remove subdirectories once the copy is done, uncoment the next line
#rm -r "$sub"
done
done
Here is why you don't parse ls in bash
Make sure the directory where the files exist is correct (and complete) in the following script and try it:
#!/bin/bash
BigParentDir=Photos
for subdir in "$BigParentDir"/*/; do # Select the a_001, a_002 subdirs
for ssdir in "$subdir"/*/; do # Select dir_001, … sub-subdirs
for f in "$ssdir"/*; do # Select the files to move
if [[ -f $f ]]; do # if indeed are files
echo \
mv "$ssdir"/* "$subdir"/ # Move the files.
fi
done
done
done
No file will be moved, just printed. If you are sure the script does what you want, comment the echo line and run it "for real".
You can try this
#!/bin/bash
dir1="/path/to/photos/"
subs= `ls $dir1`
cp /dev/null /tmp/newscript.sh
for i in $subs; do
find $dir1/$i -type f -exec echo mv \'\{\}\' $dir1/$i \; >> /tmp/newscript.sh
done
then open /tmp/newscript.sh with an editor or less and see if looks like what you are trying to do.
if it does then execute it with sh -x /tmp/newscript.sh

OS X Shell - 'Clickable' script to erase files?

I use Latex to write my documents. Latex creates MANY auxiliary files to compile a document. I often times want to clean my working directory.
While I worked on Windows, I used to keep a .bat file in the working directory that looked like this:
del *.aux
del *.pdf
del *.log
del *.bak
del *.gz
del *.bbl
del *.blg
which I could just click on to get rid of all auxiliary files in the current directory.
Now, I want to do the same on my Mac. I have created a .sh file like this:
#!/bin/bash
echo "Cleaning files..."
rm *.aux
rm *.bak
rm *.bbl
rm *.blg
rm *.gz
rm *.log
rm *.pdf
echo "Done!"
which I know I can run (i.e. invoke from command line), but I cannot click on - which is more convenient because not always I will be using Terminal.
I should stress the fact that the script should delete the files in the directory where it was clicked from!
How can I convert this script into a "clickable" one?
I appreciate any input!
Based on this reference (Getting the source directory of a Bash script from within), I ended up solving my problem with the following code:
#!/bin/bash
DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )
echo "Cleaning files..."
cd $DIR
rm *.aux
rm *.bak
rm *.bbl
rm *.blg
rm *.gz
rm *.log
rm *.pdf
echo "Done!"
#read -p "Press [Enter] to continue..."
It works very well to clean all the nasty files left behind by Texmaker!
See this answer for making a shell script double-clickable, but note that there is no concept of a "current directory" when you launch a script from the Finder.

Collapse nested directories in bash

Often after unzipping a file I end up with a directory containing nothing but another directory (e.g., mkdir foo; cd foo; tar xzf ~/bar.tgz may produce nothing but a bar directory in foo). I wanted to write a script to collapse that down to a single directory, but if there are dot files in the nested directory it complicates things a bit.
Here's a naive implementation:
mv -i $1/* $1/.* .
rmdir $1
The only problem here is that it'll also try to move . and .. and ask overwrite ./.? (y/n [n]). I can get around this by checking each file in turn:
IFS=$'\n'
for file in $1/* $1/.*; do
if [ "$file" != "$1/." ] && [ "$file" != "$1/.." ]; then
mv -i $file .
fi
done
rmdir $1
But this seems like an inelegant workaround. I tried a cleaner method using find:
for file in $(find $1); do
mv -i $file .
done
rmdir $1
But find $1 will also give $1 as a result, which gives an error of mv: bar and ./bar are identical.
While the second method seems to work, is there a better way to achieve this?
Turn on the dotglob shell option, which allows the your pattern to match files beginning with ..
shopt -s dotglob
mv -i "$1"/* .
rmdir "$1"
First, consider that many tar implementations provide a --strip-components option that allows you to strip off that first path. Not sure if there is a first path?
tar -tf yourball.tar | awk -F/ '!s[$1]++{print$1}'
will show you all the first-level contents. If there is only that one directory, then
tar --strip-components=1 -tf yourball.tar
will extract the contents of that directory in tar into the current directory.
So that's how you can avoid the problem altogether. But it's also a solution to your immediate problem. Having extracted the files already, so you have
foo/bar/stuff
foo/bar/.otherstuff
you can do
tar -cf- foo | tar --strip-components=2 -C final_destination -xf-
The --strip-components feature is not part of the POSIX specification for tar, but it is on both the common GNU and OSX/BSD implementations.

Remove files from one folder that contained in another folder

I'am trying to write simple script that will get files name from one folder and search them in another folder and remove if found them in that folder.
Got two folder like
/home/install/lib
/home/install/bin
/home/install/include
and
/usr/local/lib
/usr/local/bin
/usr/local/include
I want to remove all file's from /usr/local/lib{bin,include} that contains in /home/install/lib{bin,include}. For example having
/home/install/lib/test1
/usr/local/lib/test1
scritp will remove /usr/local/lib/test1. I tried to do it from each separate directory
/home/install/lib:ls -f -exec rm /usr/local/lib/{} \;
but nothing. Can you help me to manage with this simple script?
Create script rmcomm
#!/bin/bash
a="/home/install/$1"
b="/usr/local/$1"
comm -12 <(ls "$a") <(ls "$b") | while read file; do
rm "$b/$file"
done
Then call this script for every pair:
for dir in lib bin include; do rmcomm "$dir"; done
Here's something simple. Remove the echo from the line containing rm to run it after you've ensured it's doing what you want:
#!/bin/bash
dirs[0]=lib
dirs[1]=bin
dirs[2]=include
pushd /home/install
for dir in "${dirs[#]}"
do
for file in $(find $dir -type f)
do
# Remove 'echo' below once you're satisfied the correct files
# are being removed
echo rm /usr/local/$file
done
done
popd

Resources