A bash script to rename files from different directories at once - bash

If I have a directory named /all_images, and inside this directory there's a ton of directories, all the directories named dish_num as shown below. and inside each dish directory, there's one image named rgb.png. How can i rename all the image files to be the name of its directory.
Before
|
├── dish_1
│ └── rgb.png
├── dish_2
│ └── rgb.png
├── dish_3
│ └── rgb.png
├── dish_4
│ └── rgb.png
└── dish_5
└── rgb.png
After
|
├── dish_1
│ └── dish_1.png
├── dish_2
│ └── dish_2.png
├── dish_3
│ └── dish_3.png
├── dish_4
│ └── dish_4.png
└── dish_5
└── dish_5.png

WARNING: Make sure you have backups before running code you got someplace on the Internet!
find /all_images -name rgb.png -execdir sh -c 'mv rgb.png $(basename $PWD).png' \;
where
find /all_images will start looking from the directory "/all_images"
-name rbg.png will look anywhere for anything named "rbg.png"
optionally use -type f to restrict results to only files
-exedir in every directory where you got a hit, execute the following:
sh -c shell script
mv move, or "rename" in this case
rgb.png file named "rgb.png"
$(basename $PWD).png output of "basename $PWD", which is the last section of the $PWD - the current directory - and append ".png" to it
\; terminating string for the find loop

If you want to benefited from your multi-core processors, consider using xargs instead of find -execdir to process files concurrently.
Here is a solution composed of find, xargs, mv, basename and dirname.
find all_images -type f -name rgb.png |
xargs -P0 -I# sh -c 'mv # $(dirname #)/$(basename $(dirname #)).png'
find all_images -type f -name rgb.png prints a list of file paths whose filename is exactly rgb.png.
xargs -P0 -I# CMD... executes CMD in a parallel mode with # replaced by path names from find command. Please refer to man xargs for more information.
-P maxprocs
Parallel mode: run at most maxprocs invocations of utility at once. If maxprocs is set to 0, xargs will run as many processes as possible.
dirname all_images/dash_4/rgb.png becomes all_images/dash_4
basename all_images/dash_4 becomes dash_4
Demo
mkdir all_images && seq 5 |
xargs -I# sh -c 'mkdir all_images/dash_# && touch all_images/dash_#/rgb.png'
tree
find all_images -type f -name rgb.png |
xargs -P0 -I# sh -c 'mv # $(dirname #)/$(basename $(dirname #)).png'
tree
Output
.
└── all_images
├── dash_1
│   └── rgb.png
├── dash_2
│   └── rgb.png
├── dash_3
│   └── rgb.png
├── dash_4
│   └── rgb.png
└── dash_5
└── rgb.png
.
└── all_images
├── dash_1
│   └── dash_1.png
├── dash_2
│   └── dash_2.png
├── dash_3
│   └── dash_3.png
├── dash_4
│   └── dash_4.png
└── dash_5
└── dash_5.png
6 directories, 5 files

Related

Formatting Unix password store ls output

I just want to automate some process, and I need to be able to format the output from pass ls
Which is the listing command for https://www.passwordstore.org
Current output looks like so:
➜ ~ pass ls
Password Store
├── README.md
├── folder
│   └── subfolder
│   └── subfolder2
│   └── key1
│   └── key2
│   └── key3
├── anotherfolder
│   ├── subfolder
│   │   └── subfolder2
│   │   └── subfolder3
│   │ └── key1
│   │ └── key2
│   │ └── key3
I want the output to look like:
➜ ~ pass ls | some magic sed/grep/replace/etc
folder/subfolder/subfolder2/key1
folder/subfolder/subfolder2/key2
folder/subfolder/subfolder2/key3
anotherfolder/subfolder/subfolder2/subfolder3/key1
anotherfolder/subfolder/subfolder2/subfolder3/key2
anotherfolder/subfolder/subfolder2/subfolder3/key3
I am trying to use sed to do so, but couldn't replace the increasing spaces/tabs as the subfolder levels gets deeper (for example folder/subfolder/subfolder/subfolder/key)
Here is what I am trying so far:
pass ls | sed -e 's/├──[ \t]*/\\/g' | sed -e 's/│   └──[ \t]*/\\/g'
EDIT AFTER COMMENTS:
Seems like pass ls is just a simple tree command on the password store directory, so I can run find on my directory to achieve the above format I want
I will try that, and the scope of the question can be changed to:
What's the proper listing command that can produce the above format?
find . -type d -name '.git' -prune -o -name '*.gpg' -type f -print
Did the trick for me, it excludes all files in .git directory, and only prints files with .gpg extension (which are the actual keys)

Duplicating a nested set of folders in bash/fish

I have a folder full of other folders. Within each of these folders, there also exists another folder that I want to duplicate, but with a new name (same for all copies).
For example:
├── application
│   └── foo
│ └── bar
│   └── redacted.txt
│
├── something_different
│   └── foo
│ └── bar
│   └── RobotoMono.ttf
So every top level folder has a "foo/bar/" folder. I'd like to clone the "bar" folder (and the contents) so there is a "bar2" folder under each "foo" folder.
Then it would look like this:
├── application
│ └── foo
│ └── bar
│ └── redacted.txt
│ └── bar2
│ └── redacted.txt
│
├── something_different
│ └── foo
│ └── bar
│ └── RobotoMono.ttf
│ └── bar2
│ └── RobotoMono.ttf
I can successfully get the list with "find". Here is what I have tried:
find . -name bar -exec cp -r '{}' '{}'/bar2 \;
find . -name bar | xargs cp -r /bar2
And of course theses don't work and leave some nice looping that was fun to clean up. Thank you for reading and explaining what I'm doing incorrectly, or if I'm even close to what I should be doing.
First of all the single quotes around the parentheses will give you a string not the actual target (the directory) that you want. Removing these and copying to one level above "bar" seems to work (bash):
#> mkdir -p application/foo/bar && touch application/foo/bar/redacted.txt
#> find application -name bar -exec cp -r {} {}/../bar2 \;
#> ls -l application/foo/ | awk '{ print $NF }'
./
../
bar/
bar2/
#> ls -l application/foo/bar2/ | awk '{ print $NF }'
./
../
redacted.txt
find . \
-path '.*/foo/bar' \
-type d \
-exec sh -c 'p="${0%/*}"; n="${0##*/}"; cp -rp -- "$p/$n" "$p/${n}2"' {} \;
-exec sh -c Executes the inline shell script
Here is the content of the inline shell script with added comments:
# Extract the path before /bar from argument 0
p="${0%/*}"
# Extract the trailing name bar or anything else from argument 0
n="${0##*/}"
# Perform a recursive copy with preserved permissions
# from the source to the destination with suffix2
cp -rp -- "$p/$n" "$p/${n}2"

Changing names (version numbers) in nested folders with find and mv

TL;DR: I want all the 2.6s to say 2.7
lib
└── python2.6
└── site-packages
├── x
│   ├── x.py
│   ├── x.pyc
│   ├── __init__.py
│   ├── __init__.pyc
│   └── test
│   ├── __init__.py
│   └── __init__.pyc
└── x-0.2.0-py2.6.egg-info
├── dependency_links.txt
├── entry_points.txt
├── PKG-INFO
├── requires.txt
├── SOURCES.txt
└── top_level.txt
What I've tried:
find . -type d -name "*2.6*" -exec bash -c 'mv "$1" "${1/2.6/2.7}"' -- {} \;
Obviously this doesn't work because it sees the main folder, moves that, and then sees the nested folder and tries to move it, but it no longer exists in that spot and says no such file or directory
Is there a good way to do nested find and moves? In this case, I can just run the command twice and that would technically work, but it feels dirty.
Also, I know this could screw up the versioning of the package, or that I could do
find . -type d -name "*python2.6*" -exec bash -c 'mv "$1" "${1/2.6/2.7}"' -- {} \;
find . -type d -name "*py2.6*" -exec bash -c 'mv "$1" "${1/2.6/2.7}"' -- {} \;
But I'm more interested in learning if bash has a method to solve this in general than how to deal with this narrow scenario.
You can go depth first and substitute only in the basename:
find lib -depth -type d -name "*2.6*" -exec \
bash -c 'basename="${1##*/}" && mv "$1" "${1%/*}/${basename/2.6/2.7}"' -- {} \;
If you run it with an echo as:
find lib -depth -type d -name "*2.6*" -exec \
bash -c 'bn="${1##*/}" && echo mv "$1" "${1%/*}/${bn/2.6/2.7}"' -- {} \;
on a tree created with:
mkdir -p lib/python2.6/site-packages/{x/test,x-0.20-py2.6.egg-info}
i.e., on:
lib/
└── python2.6
└── site-packages
├── x
│   └── test
└── x-0.20-py2.6.egg-info
You get:
mv lib/python2.6/site-packages/x-0.20-py2.6.egg-info lib/python2.6/site-packages/x-0.20-py2.7.egg-info
mv lib/python2.6 lib/python2.7
Remove the echos, and the moves should proceed error-free.

Bash - Combine files in separated sub folders

So I'm looking for a way to cat .html files in multiple subfolders, but by keeping them in their place.
Actual situation:
$ Folder1
.
├── Subfolder1
│ └── File1.html
└── File2.html
├── Subfolder2
│ └── File1.html
└── File2.html
Desired outcome:
$ Folder1
.
├── Subfolder1
│ └── Mergedfile1.html
└── File1.html
└── File2.html
├── Subfolder2
│ └── Mergedfile2.html
└── File1.html
└── File2.html
So far I've came up with this:
find . -type f -name *.html -exec cat {} + > Mergedfile.html
But this combines all the files of all the subfolders of Folder1, while I want to keep them separated.
Thanks a lot!
You can loop on all subfolders with a for statement:
for i in Folder1/SubFolder*; do
cat "$i"/File*.html > MergeFile$(echo "$i" | sed 's,.*\([0-9]\+\)$,\1,').html
done
Like told by AK_ , you can use find with exec.
find Folder1/ -mindepth 1 -maxdepth 1 -type d -exec sh -c "rep='{}';cat "'"$rep"'"/*.html > "'"$rep"'"/Mergedfile.html" \;

Recursively deleting all "*.foo" files with corresponding "*.bar" files

How can I recursively delete all files ending in .foo which have a sibling file of the same name but ending in .bar? For example, consider the following directory tree:
.
├── dir
│   ├── dir
│   │   ├── file4.bar
│   │   ├── file4.foo
│   │   └── file5.foo
│   ├── file2.foo
│   ├── file3.bar
│   └── file3.foo
├── file1.bar
└── file1.foo
In this example file.foo, file3.foo, and file4.foo would be deleted since there are sibling file{1,3,4}.bar files. file{2,5}.foo should be left alone leaving this result:
.
├── dir
│   ├── dir
│   │   ├── file4.bar
│   │   └── file5.foo
│   ├── file2.foo
│   ├── file3.bar
└── file1.bar
Remember to first take a backup before you try this find and rm command.
Use this find:
find . -name "*.foo" -execdir bash -c '[[ -f "${1%.*}.bar" ]] && rm "$1"' - '{}' \;
while IFS= read -r FILE; do
rm -f "${FILE%.bar}".foo
done < <(exec find -type f -name '*.bar')
Or
find -type f -name '*.bar' | sed -e 's|.bar$|.foo|' | xargs rm -f
In bash 4.0 and later, and in zsh:
shopt -s globstar # Only needed by bash
for f in **/*.foo; do
[[ -f ${f%.foo}.bar ]] && rm ./"$f"
done
In zsh, you can define a selective pattern that matches files ending in .foo only if there is a corresponding .bar file, so that rm is invoked only once, rather than once per file.
rm ./**/*.foo(e:'[[ -f ${REPLY%.foo}.bar ]]':)

Resources