Writing a loop for removing files from subdirectories - bash

Im trying to figure out how to write a loop to remove specific files from multiple subdirectories which are all very similar in structure.
Within /main_directory/ there are 50 sub directories all with different names.
So /main_directory/sub1 /main_directory/testsub1 /main_directory/sub1_practice and so on. Each subdirectory has different file types and I want to remove a specific type, call it .txt.
I want to write a loop something like this:
for file in ./main_directory/*; cd $file; rm *.txt; done
So that within each subdirectory in main_directory, all .txt files are removed.
Apologies for the basic question, but I can only find solutions for subdirectories that have the same name or prefix, not all differently named subdirectories.
I tried for file in ./main_directory/*; cd $file; rm *.txt; done
and received the error that there is no such file or directory for all the subdirectories

If you cd into the directory, you have to remember to return to your original working directory afterward otherwise every future command will be executed relative to whatever directory you last successfully changed to.
Some examples that should work:
# don't change directories at all
for dir in ./main_directory/* ; do
rm "$dir"/*.txt
done
# explicitly change back to the original directory
for dir in ./main_directory/* ; do
cd "$dir" && { rm *.txt ; cd - ; }
done
# change directories in a subshell, so that your
# original working directory is restored afterward
for dir in ./main_directory/* ; do
(
cd "$dir" && rm *.txt
)
done
# without a loop, using the find command
find ./main_directory -type f -name '*.txt' -delete

After descending in the first directory, you don't return to the original location. The second directory will be searched for in the first one. An improvement of your code will be adding cd ..
for directory in ./main_directory/*; do
cd "$directory" || continue
rm *.txt
cd ..
done
This code is wrong without the || continue part.
When the cd "$directory" fails, it will start removing files from the wrong location and next put you in the parent directory. You can change it into cd "$directory" || continue.
An alternative is starting it in a subprocess:
for directory in ./main_directory/*; do
(
cd "$directory" || continue
rm *.txt
)
done
You still need the || continue, because you don't want to remove files from the wrong directory.
In this example, you don't need a loop. You can use
rm main_directory/*.txt

Related

Recursively read folders and executes command on each of them if a file exists in that folder

I have this command which is fully functional and really like it.
for d in ./*/ ; do (cd "$d" && ../Process.sh); done
It goes through all the subfolders and runs the Process.sh inside each folder.
What I need is, it just run the Process.sh if the "kill_by_pid" file exists in the folder. Unless, if the "kill_by_pid" does not exist in that specific folder, the folder be skipped and it just move to the next folder.
What about using find and execdir, for example:
find . -type f -name "kill_by_pid" -execdir ../Process.sh \;
This will only call the Process.sh the script on the parent directory where the file kill_by_pid exists.
Try this,
for d in ./*/ ;
do
if [ -f "$d"/kill_by_pid ]; then
(cd "$d" && ../Process.sh);
fi
done
A slightly different approach:
for f in ./*/kill_by_pid; do
pushd -- "${f#/*}"
../Process.sh
popd
done
This finds each kill_by_pid file, then determines the name of the directory that contains it. (Using pushd/popd is just to demonstrate
a way of changing and restoring the current directory that doesn't require a subshell, in the event Process.sh needs to execute in the current shell session.)

How to move files from subfolders to their parent directory (unix, terminal)

I have a folder structure like this:
A big parent folder named Photos. This folder contains 900+ subfolders named a_000, a_001, a_002 etc.
Each of those subfolders contain more subfolders, named dir_001, dir_002 etc. And each of those subfolders contain lots of pictures (with unique names).
I want to move all these pictures contained in the subdirectories of a_xxx inside a_xxx. (where xxx could be 001, 002 etc)
After looking in similar questions around, this is the closest solution I came up with:
for file in *; do
if [ -d $file ]; then
cd $file; mv * ./; cd ..;
fi
done
Another solution I got is doing a bash script:
#!/bin/bash
dir1="/path/to/photos/"
subs= `ls $dir1`
for i in $subs; do
mv $dir1/$i/*/* $dir1/$i/
done
Still, I'm missing something, can you help?
(Then it would be nice to discard the empty dir_yyy, but not much of a problem at the moment)
You could try the following bash script :
#!/bin/bash
#needed in case we have empty folders
shopt -s nullglob
#we must write the full path here (no ~ character)
target="/path/to/photos"
#we use a glob to list the folders. parsing the output of ls is baaaaaaaddd !!!!
#for every folder in our photo folder ...
for dir in "$target"/*/
do
#we list the subdirectories ...
for sub in "$dir"/*/
do
#and we move the content of the subdirectories to the parent
mv "$sub"/* "$dir"
#if you want to remove subdirectories once the copy is done, uncoment the next line
#rm -r "$sub"
done
done
Here is why you don't parse ls in bash
Make sure the directory where the files exist is correct (and complete) in the following script and try it:
#!/bin/bash
BigParentDir=Photos
for subdir in "$BigParentDir"/*/; do # Select the a_001, a_002 subdirs
for ssdir in "$subdir"/*/; do # Select dir_001, … sub-subdirs
for f in "$ssdir"/*; do # Select the files to move
if [[ -f $f ]]; do # if indeed are files
echo \
mv "$ssdir"/* "$subdir"/ # Move the files.
fi
done
done
done
No file will be moved, just printed. If you are sure the script does what you want, comment the echo line and run it "for real".
You can try this
#!/bin/bash
dir1="/path/to/photos/"
subs= `ls $dir1`
cp /dev/null /tmp/newscript.sh
for i in $subs; do
find $dir1/$i -type f -exec echo mv \'\{\}\' $dir1/$i \; >> /tmp/newscript.sh
done
then open /tmp/newscript.sh with an editor or less and see if looks like what you are trying to do.
if it does then execute it with sh -x /tmp/newscript.sh

How to cd into multiple directories, zip it (naming it after the directory name)?

So I have a bash script that cds into a directory, executes a command and then exits and enters a new directory again:
for d in ./*/ ; do (cd "$d" && somecommand); done
From here.
Unfortunately I'm not sure how to zip up the directory it is in (maybe using something such as 7z). It was a long shot but I tried this command and it didn't work (I didn't expect the asterisk to take the name of the directory...but I hoped):
7z a -r *.zip
I don't suppose anyone has any suggestions?
The variable $d contains the name of the directory (among other things):
for d in ./*/ ; do (
cd "$d"
dirname=${d%/} # remove trailing /
dirname=${dirname##*/} # remove everything up to the last /
7z a -r "$dirname".zip
)
done
I'm assuming that your 7z command was correct.
Perhaps you're looking for something like this:
for d in *; do
test -d "$d" && zip -r "$d.zip" "$d"
done
That examines all files in the working directory whose names do not begin with '.' (for d in *). For those that are directories (test -d $d) it zips the directory contents, recursively, as members of a directory. The zip files are left in the original working directory (the parent of all the directories that get zipped), but they could as easily be put into the subdirectories.

bash script to remove files matching those in another directory

I'm trying to create a script that retrieves files (including subfolders) from CVS and stores them into a temporary directory /tmp/projectdir/ (OK), then removes copies of those files from my project directory /home/projectdir/ (not OK) without touching any other files in the project directory or the folder structure itself.
I've been attempting two methods, but I'm running into problems with both. Here's my script so far:
#!/usr/bin/bash
cd /tmp/
echo "removing /tmp/projectdir/"
rm -rf /tmp/projectdir
# CVS login goes here, code redacted
# export files to /tmp/projectdir/dir_1/file_1 etc
cvs export -kv -r $1 projectdir
# method 1
for file in /tmp/projectdir/*
do
# check for zero-length string
if [-n "$file"]; then
echo "removing $file"
rm /home/projectdir/"$file"
fi
done
# method 2
find /tmp/projectdir/ -exec rm -i /home/projectdir/{} \;
Neither method works as intended, because I need some way of stripping /tmp/projectdir/ from the filename (to be replaced with /home/projectdir/) and to prevent them from executing rm /home/projectdir/dir_1 (i.e. the directory and not a specific file), but I'm not sure how to achieve this.
(In case anybody is wondering, the zero-length string bit was an attempt to avoid rm'ing the directory, before I realised /tmp/projectdir/ would also be a part of the string)
You can use:
cd /tmp/projectdir/
find . -type f -exec rm -i /home/projectdir/{} \;

Remove files from one folder that contained in another folder

I'am trying to write simple script that will get files name from one folder and search them in another folder and remove if found them in that folder.
Got two folder like
/home/install/lib
/home/install/bin
/home/install/include
and
/usr/local/lib
/usr/local/bin
/usr/local/include
I want to remove all file's from /usr/local/lib{bin,include} that contains in /home/install/lib{bin,include}. For example having
/home/install/lib/test1
/usr/local/lib/test1
scritp will remove /usr/local/lib/test1. I tried to do it from each separate directory
/home/install/lib:ls -f -exec rm /usr/local/lib/{} \;
but nothing. Can you help me to manage with this simple script?
Create script rmcomm
#!/bin/bash
a="/home/install/$1"
b="/usr/local/$1"
comm -12 <(ls "$a") <(ls "$b") | while read file; do
rm "$b/$file"
done
Then call this script for every pair:
for dir in lib bin include; do rmcomm "$dir"; done
Here's something simple. Remove the echo from the line containing rm to run it after you've ensured it's doing what you want:
#!/bin/bash
dirs[0]=lib
dirs[1]=bin
dirs[2]=include
pushd /home/install
for dir in "${dirs[#]}"
do
for file in $(find $dir -type f)
do
# Remove 'echo' below once you're satisfied the correct files
# are being removed
echo rm /usr/local/$file
done
done
popd

Resources