Recursively unzip files and then delete original file, leaving unzipped files in place from shell - bash

I've so far figured out how to use find to recursively unzip all the files:
find . -depth -name `*.zip` -exec /usr/bin/unzip -n {} \;
But, I can't figure out how to remove the zip files one at a time after the extraction. Adding rm *.zip in an -a -exec ends up deleting most of the zip files in each directory before they are extracted. Piping through a script containing the rm command (with -i enabled for testing) causes find to not find any *.zips (or at least that's what it complains). There is, of course, whitespace in many of the filenames but at this point syntaxing in a sed command to add _'s is a bit beyond me. Thank for your help!

have you tried:
find . -depth -name '*.zip' -exec /usr/bin/unzip -n {} \; -exec rm {} \;
or
find . -depth -name '*.zip' -exec /usr/bin/unzip -n {} \; -delete
or running a second find after the unzip one
find . -depth -name '*.zip' -exec rm {} \;

thx for the 2nd command with -delete! helped me a lot..
just 2 (maybe helpful) remarks from my side:
-had to use '.zip' instead of `.zip` on my debian system
-use -execdir instead of -exec > this will extract each zip file within its current folder, otherwise you end up with all extracted content in the dir you invoked the find cmd.
find . -depth -name '*.zip' -execdir /usr/bin/unzip -n {} \; -delete
THX & Regards,
Nord

As mentioned above, this should work.
find . -depth -name '*.zip' -execdir unzip -n {} \; -delete
However, note two things:
The -n option instructs unzip to not overwrite existing files. You may not know if the zip files differ from the similarly named target files. Even so, the -delete will remove the zip file.
If unzip can't unzip the file--say because of an error--it might still delete it. The command will certainly remove it if -exec rm {} \; is used in place of -delete.
A safer solution might be to move the files following the unzip to a separate directory that you can trash when you're sure you have extracted all the files successfully.

Unzip archives in subdir based on the file name (../file.zip -> ../file/..):
for F in $(find . -depth -name *.zip); do unzip "$F" -d "${F%.*}/" && rm "$F"; done

I have a directory filling up with zipped csv files. External processes are writing new zipped files to it often. I wish to bulk unzip and remove the originals as you do.
To do that I use:
unzip '*.zip'
find . | sed 's/$/\.zip/g' | xargs -n 1 rm
It works by searching and expanding all zip files presently in the directory. Later, after it finishes there are potentially new unzipped new files mixed in there too that are not to be deleted yet.
So I delete by finding successfully unzipped *.csv files, and using sed to regenerate the original filenames for deletion which is then fed to rm via the xargs command.

Related

How to print the deleted file names along with path in shell script

I am deleting the files in all the directories and subdirectories using the command below:
find . -type f -name "*.txt" -exec rm -f {} \;
But I want to know which are the files deleted along with their paths. How can I do this?
Simply add a -print argument to your find.
$ find . -type f -name "*.txt" -print -exec rm -f {} \;
As noted by #JonathanRoss below, you can achieve an equivalent result with the -v option to rm.
It's not the scope of your question, but more generally it gets more interesting if you want to delete directories recursively. Then:
a simple -exec rm -r argument keeps it silent
a -print -exec rm -r argument reports the toplevel directories you're operating on
a -exec rm -rv argument reports all you're removing

rename folder/directory recursively

I want to rename folder/directory names recursively and found this solution on SO. However this command has no effect
find . -type f -exec rename 's/old/new/' '{}' \;
Is that a correct command?
find . -depth -name '*a_*' -execdir bash -c 'mv "$0" "${0//a_/b_}"' {} \;
The -depth switch is important so that the directory content is processed before the directory itself! otherwise you'll run into problems :).
100% safe regarding filenames with spaces or other funny symbols.

Move only files recursively from multiple directories into one directory with mv

I currently have ~40k RAW images that are in a nested directory structure. (Some folders have as many as 100 subfolders filled with files.) I would like to move them all into one master directory, with no subfolders. How could this be accomplished using mv? I know the -r switch will copy recursively, but this copies folders as well, and I do not wish to have subdirectories in the master folder.
If your photos are in /path/to/photos/ and its subdirectories, and you want to move then in /path/to/master/, and you want to select them by extension .jpg, .JPG, .png, .PNG, etc.:
find /path/to/photos \( -iname '*.jpg' -o -iname '*.png' \) -type f -exec mv -nv -t '/path/to/master' -- {} +
If you don't want to filter by extension, and just move everything (i.e., all the files):
find /path/to/photos -type f -exec mv -nv -t '/path/to/master' -- {} +
The -n option so as to not overwrite existing files (optional if you don't care) and -v option so that mv shows what it's doing (very optional).
The -t option to mv is to specify the target directory, so that we can stack all the files to be moved at the end of the command (see the + delimiter of -exec). If your mv doesn't support -t:
find /path/to/photos \( -iname '*.jpg' -o -iname '*.png' \) -type f -exec mv -nv -- {} '/path/to/master' \;
but this will be less efficient, as one instance of mv will be created for each file.
Btw, this moves the files, it doesn't copy them.
Remarks.
The directory /path/to/master must already exist (it will not be created by this command).
Make sure the directory /path/to/master is not in /path/to/photos. It would make the thing awkward!
Make use of -execdir option of find:
find /path/of/images -type f -execdir mv '{}' /master-dir \;
As per man find:
-execdir utility [argument ...] ;
The -execdir primary is identical to the -exec primary with the exception that
utility will be executed from the directory that holds the current
file. The filename substituted for the string ``{}'' is not qualified.
Since -execdir makes find execute given command from each directory therefore only base filename is moved without any parent path of the file.
find <base location of files> -type -f -name \*\.raw -exec mv {} master \;
If your hierachy is only one level deep, here is another way using the automated tools of StringSolver:
mv -a firstfolder/firstfile.raw firstfile.raw
The -a options immediately applies the similar transformation to all similar files at a nesting level 1 (i.e. for all other subfolders).
If you do not trust the system, you can use other options such as -e to explain the transformation or -t to test it on all files.
DISCLAIMER: I am a co-author of this work for academic purposes, and working on a bash script renderer. But the system is already available for testing purposes.

Recursive erase hidden files

I'm trying to recursive erase all files that begin with "._" (aka mac dot files) on my server using SSH.
The files are listed with a ls -a but this won't work:
rm -rf ._*
I know there's a way. Mind to share?
Cheers!
find . -name ._\* -print0 | xargs -0 rm -f
find . -name ._\* -type f -delete
Specify that it's files and just call the find-delete on item directly.
find . -name ._\* -exec rm -f {} \;
by the way rm -rf is for removing directories recursively

How do I remove all .pyc files from a project?

I've renamed some files in a fairly large project and want to remove the .pyc files they've left behind. I tried the bash script:
rm -r *.pyc
But that doesn't recurse through the folders as I thought it would. What am I doing wrong?
find . -name "*.pyc" -exec rm -f {} \;
find . -name '*.pyc' -type f -delete
Surely the simplest.
Add to your ~/.bashrc:
pyclean () {
find . -type f -name "*.py[co]" -delete
find . -type d -name "__pycache__" -delete
}
This removes all .pyc and .pyo files, and __pycache__ directories. It's also very fast.
Usage is simply:
$ cd /path/to/directory
$ pyclean
In current version of debian you have pyclean script which is in python-minimal package.
Usage is simple:
pyclean .
If you're using bash >=4.0 (or zsh)
rm **/*.pyc
Note that */*.pyc selects all .pyc files in the immediate first-level subdirectories while **/*.pyc recursively scans the whole directory tree. As an example, foo/bar/qux.pyc will be deleted by rm **/*.pyc but not by */*.pyc.
The globstar shell options must be enabled. To enable globstar:
shopt -s globstar
and to check its status:
shopt globstar
For windows users:
del /S *.pyc
I used to use an alias for that:
$ which pycclean
pycclean is aliased to `find . -name "*.pyc" | xargs -I {} rm -v "{}"'
find . -name '*.pyc' -print0 | xargs -0 rm
The find recursively looks for *.pyc files. The xargs takes that list of names and sends it to rm. The -print0 and the -0 tell the two commands to seperate the filenames with null characters. This allows it to work correctly on file names containing spaces, and even a file name containing a new line.
The solution with -exec works, but it spins up a new copy of rm for every file. On a slow system or with a great many files, that'll take too long.
You could also add a couple more args:
find . -iname '*.pyc' -print0 | xargs -0 --no-run-if-empty rm
iname adds case insensitivity, like *.PYC . The no-run-if-empty keeps you from getting an error from rm if you have no such files.
$ find . -name '*.pyc' -delete
This is faster than
$ find . -name "*.pyc" -exec rm -rf {} \;
Further, people usually want to remove all *.pyc, *.pyo files and __pycache__ directories recursively in the current directory.
Command:
find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf
Django Extension
Note: This answer is very specific to Django project that have already been using Django Extension.
python manage.py clean_pyc
The implementation can be viewed in its source code.
Just to throw another variant into the mix, you can also use backquotes like this:
rm `find . -name *.pyc`
full recursive
ll **/**/*.pyc
rm **/**/*.pyc
Now there is a package pyclean on PyPI, which is easy to use, and cross-platform. User just need a simple command line to clean all __pycache__ files in current dir:
pyclean .
if you don't want .pyc anymore you can use this single line in a terminal:
export PYTHONDONTWRITEBYTECODE=1
if you change your mind:
unset PYTHONDONTWRITEBYTECODE
First run:
find . -type f -name "*.py[c|o]" -exec rm -f {} +
Then add:
export PYTHONDONTWRITEBYTECODE=1
To ~/.profile
rm -r recurses into directories, but only the directories you give to rm. It will also delete those directories. One solution is:
for i in $( find . -name *.pyc )
do
rm $i
done
find will find all *.pyc files recursively in the current directory, and the for loop will iterate through the list of files found, removing each one.
find . -name "*.pyc"|xargs rm -rf
If you want to delete all the .pyc files from the project folder.
First, you have
cd <path/to/the/folder>
then find all the .pyc file and delete.
find . -name \*.pyc -delete
You can run find . -name "*.pyc" -type f -delete.
But use it with precaution. Run first find . -name "*.pyc" -type f to see exactly which files you will remove.
In addition, make sure that -delete is the last argument in your command. If you put it before the -name *.pyc argument, it will delete everything.
To delete all the python compiled files in current directory.
find . -name "__pycache__"|xargs rm -rf
find . -name "*.pyc"|xargs rm -rf
If you want remove all *.pyc files and __pycache__ directories recursively in the current directory:
with python:
import os
os.popen('find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf')
or manually with terminal or cmd:
find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf
py3clean works for me!
cd /usr/local/lib/python3.9
sudo py3clean -v .
Had to add a few ignore params on M1:
pyclean --verbose . --ignore "Library",".Trash"

Resources