I have a small bash script which goes through some directories and deletes contents within them like so:
#!/bin/bash
echo "Start job at: $(date)"
rm -rf /my/location/log_folder1/*
exit 0
unfortunately, I have these log_folder from 1 --> 10. So, log_folder1, log_folder2, log_folder3 etc..
How can I efficiently loop through this rather than write a separate line for each rm?
I do not want to use the find command to do this - I would like to learn how to loop through.
You can do it with brace expansion:
rm -rf /my/location/log_folder{1..10}/*
See man bash - Brace Expansion for details.
Brace Expansion Example:
$ echo "asd"{1..10}
asd1 asd2 asd3 asd4 asd5 asd6 asd7 asd8 asd9 asd10
Given:
$ find . -type d
.
./log_folder1
./log_folder1/sub_log_folder1
./log_folder1/sub_log_folder2
./log_folder1/sub_log_folder3
./log_folder10
./log_folder2
./log_folder3
./log_folder4
./log_folder5
./log_folder6
./log_folder7
./log_folder8
./log_folder9
Use a filename expansion (a 'glob'), and a for loop:
$ for fn in log_folder*/*; do
> rm -rf "$fn"
> done
Resulting in:
$ find . -type d
.
./log_folder1
./log_folder10
./log_folder2
./log_folder3
./log_folder4
./log_folder5
./log_folder6
./log_folder7
./log_folder8
./log_folder9
Be sure to use quotes around the file name in "$fn" so you actually delete what your think and not the unintended result of word splits.
Or, you can just use a glob that targets subdirectories:
$ rm -rf log_folder*/*
Or, an expansion that only targets some of the sub directories, not all:
$ rm -rf log_folder{1..5}/*
Which can also be used in the for loop.
(I suggest being careful with rm -rf .... if you are just getting started with scripting and expansions however. Perhaps practice with something more reversible...)
Use find
find log_folder* -delete
You can use this loop:
# using a common base directory makes it a little safer with 'rm -rf'
fbase=/my/location/log_folder
for n in {1..10}; do
f="$fbase$n"
echo "Cleaning up folder $f"
rm -rf "$f"/* # this will empty everything under $f, but not $f itself will remain as an empty folder
# use 'rm -rf "$f"' to remove everything including the top folder
done
Related
I have these test files:
ABCD1234__12_Ab2_Hh_3P.mp4
ABCD1234__12_Ab2_Lw_3P.wmv
ABCD1234__12_Ab2_SSV.mov
EFGH56789__13_Mn1_SSV.avi
EFGH56789__13_Mn1_Ve1_3P.mp4
EFGH56789__13_Mn1_Ve2_3P.webm
I want to create a context service in Automator that makes directories based on the filename prefixes above like so:
ABCD1234__12_Ab2
EFGH56789__13_Mn1
...and move the files into those two accordingly. The only consistent variables in the names are underscores, so I was thinking I could delineate by those, preferably capturing the name before the fourth one.
I originally started with this very simple script:
for file in "$#"
do
mkdir "${file%.*}" && mv "$file" "${file%.*}"
done
Which makes a folder for every file and moves each file into its own folder.
I tried adding variables, various if/thens, etc. but to no avail (not a programmer by trade).
I also wrote another script to do it in a slightly different way, but with the same results to mess around with:
for folder in "$#"
do
cd "$1"
find . -type f -maxdepth 1 -exec bash -c 'mkdir -p "${0%.*}"' {} \; \
-exec bash -c 'mv "$0" "${0%.*}"' {} \;
done
I feel like there's something obvious I am missing.
Your script is splitting on dot, but you say you want to split on underscore. If the one you want to split on is the last one, the fix is trivial:
for file in "$#"
do
mkdir -p "${file%_*}" && mv "$file" "${file%_*}"
done
To get precisely the fourth, try
for file in "$#"
do
tail=${file#*_*_*_*_}
dir=${file%_"$tail"}
mkdir -p "$dir" && mv "$file" "$dir"
done
The addition of the -p option is a necessary bug fix if you want to use && here; mkdir without this option will fail if the directory already exists.
Perhaps see also the section about parameter expansions in the Bash Reference Manual which explains this syntax and its variations.
You could do something like this:
#!/bin/bash
shopt -s extglob
for file
do
if [[ $file =~ ^(([^_]*_){3}[^_]*) ]]
then
echo "${BASH_REMATCH[0]}"
else
echo "${file%.*}"
fi
done
Okey, my question seems to be related to this one:
Trouble in script with spaces in filename
but the answer doesn't seem to tackle the problem when you have to deal with a bunch of those files.
Let us say that I get some filenames of files that I want to erase with find, and that I put them in a text file. Then I want to delete them, with rm, say. rm seems not to be able to interpret the spaces of the filenames, even after I put the names inside quotes manually!
What would you suggests?
It depends on how you're actually processing the file list (which you haven't yet shown). Assuming you have them one per line in the file:
:: cat list
thishasnospaces
but this does
then even something like this won't work if they have spaces:
:: for fspec in $(cat list) ; do echo "rm -f \"${fspec}\"" ; done
rm -rf "thishasnospaces"
rm -rf "but"
rm -rf "this"
rm -rf "does"
That's because this treats all white space as identical. However, you can do it with a while loop:
:: cat list | while read ; do echo "rm -f \"$REPLY\""; done
rm -f "thishasnospaces"
rm -f "but this does"
You'll see that preserves the one per line aspect. Just remove the echo when you're happy it will work.
:: cat list | while read ; do rm -f "$REPLY"; done
But just keep in mind this may all be unnecessary. The find command already has the capability to delete files that it finds, bu use of the -delete option. If you can use that, there'll be far less messing about with spaces in the filenames.
Why do you need a text file?
Try something like:
touch "a b"
touch "c d"
find . -name "* *" -exec rm "{}" \;
Good luck!
rm -fr *
won't delete .files
On the other hand,
rm -fr * .*
will delete too much!
Is there a reliable way to recursively delete all contents of a directory in Bash?
One way I can think of is:
rm -fr $PWD
mkdir $PWD
cd $PWD
This has the side-effect of deleting $PWD temporarily.
I suggest to use first:
shopt -s dotglob
dotglob: If set, bash includes filenames beginning with a . in the results of pathname expansion
rm -fr * .*
is relatively "safe". rm is forbidden by POSIX from acting on . and ...
rm -rf . ..
will be a no-op, though it will return 1. If you don't want the error return, you can do:
rm -rf .[!.]*
which is POSIX standardized, no bash extension required.
You can also use find:
find . -delete
You could use find with -delete and -maxdepth:
find . -name "*" -delete -maxdepth 2
So let's say you are in the directory temp which looks like this:
./temp
|_____dir1
| |_____subdir1
X|_file X|_file |_file
|
X|_____dir2
X|_file
Looking at the tree the files and directories which have an X next to them would be deleted using the command above. subdir1 is spared, since the maximum depth at which find will delete a file is set at 2 and there is a file residing within it. find will delete files starting with . — however, it doesn't work for symbolic links.
-delete
Delete found files and/or directories. Always returns true.
This executes from the current working directory as find recurses
down the tree. It will not attempt to delete a filename with a
``/'' character in its pathname relative to ``.'' for security
reasons. Depth-first traversal processing is implied by this
option. Following symlinks is incompatible with this option.
The usual wisdom for UNIX is to use something like:
rm -rf * .[!.]* ..?*
That will list all files that start with a dot or even double dot (without including the plain double dot (./..).
But that globbing expansion will keep the asterisk if files of that type do not exist.
Let's test:
$ mkdir temp5; cd temp5
$ touch {,.,..}{aa,bb,cc}
$ echo $(find .)
. ./aa ./cc ./..bb ./..aa ./.cc ./.bb ./..cc ./.aa ./bb
And, as indicated, this will include all files:
$ echo * .[!.]* ..?*
aa bb cc .aa .bb .cc ..aa ..bb ..cc
But if one of the types doesn't exist, the asterisk will stay:
$ rm ..?*
$ echo * .[!.]* ..?*
aa bb cc .aa .bb .cc ..?*
We need to avoid arguments that contain an asterisk to workaround this issue.
I'm trying to pass parameter to rm in bash script to clean my system automatically. For example, I want to remove everything except the *.doc files. So I wrote the following codes.
#!/bin/bash
remove_Target="!*.txt"
rm $remove_Target
However, the output always say
rm: cannot remove ‘!*.txt’: No such file or directory
It is obviously that bash script add single quotes for me when passing the variable to rm. How can I remove the single quotes?
Using Bash
Suppose that we have a directory with three files
$ ls
a.py b.py c.doc
To delete all except *.doc:
$ shopt -s extglob
$ rm !(*.doc)
$ ls
c.doc
!(*.doc) is an extended shell glob, or extglob, that matches all files except those ending in .doc.
The extglob feature requires a modern bash.
Using find
Alternatively:
find . -maxdepth 1 -type f ! -name '*.doc' -delete
Is there some elegant/simple way to delete the folder's contents in such a way there's no error output if it is empty?
The following command
$ rm -r $dir/*
doesn't work if the directory is empty, since in such a case, the wilcard * is not expanded and you get an error saying that rm cannot find file *.
Of course, the standard way is check if it is empty (with ls $dir | wc -w or find $dir -link 2 or any other related command), and deleting its contents otherwise.
Is there an alternative way not to check folder contents and only "truncate" the directory instead?
Bash
Simply,
$ rm -rf dir/*
(By default I believe) Bash doesn't complain about not finding anything with the glob. It just passes your literal glob through to your command:
$ echo dir/*
dir/*
When rm doesn't find a filename that has the literal glob character, it complains about not finding the file it's been asked to delete:
$ rm "dir/*"
rm: cannot remove ‘dir/*’: No such file or directory
$ echo $?
1
But if you force it, it won't complain:
$ rm -f "dir/*"
$ echo $?
0
I don't know if that refrain-from-complain is POSIX.
Do note, however, that if you don't have the shell option "dotglob" set, you'll miss files that start with a dot, AKA "hidden" files.
Generally
Zsh doesn't pass the literal glob through by default. You have to ask for it with "set -o nonomatch".
$ echo dir/*
zsh: no matches found: dir/*
$ echo $?
1
$ set -o nonomatch
$ echo dir/*
dir/*
For compatibility, I wouldn't use the above modern-Bash-specific "rm -rf dir/*", but would use the more general, widely-compatible solution:
$ find dir -mindepth 1 -delete
Find all files in "dir" at a minimum depth of 1 ("dir" itself is at depth 0), and delete them.
You can use rm -rf:
rm -rf "$dir"/*
As per man bash:
-f, --force
ignore nonexistent files and arguments, never prompt
rm -rf dir/*
does not delete hidden files which name starts with dot.
This is quite weird, when bash glob the *, it does not include .* files.
mkdir -p dir
touch dir/.a
rm -fr dir/*
ls dir/.a && echo I am not deleted
output is
dir/.a
I am not deleted
Besides, the rm -fr dir/* has another disadvantage: when there are too many files in the dir, the rm command will get too many arguments and results in error too many arguments. Also, it is very slow in that case.
Seems that the most reliable and fastest way is
find dir -mindepth 1 -delete