I have a large number of files in sub-folders that I need to rename. For example, I have:
ParentFolder/sub-Folders/*.jpg
How can I copy the files with a new naming convention as follows?
ParentFolder1.jpg
ParentFolder2.jpg
One way to do this is via GNU parallel. Tutorial here:
find ./ParentFolder -name "*.jpg" | parallel "mv {} DESTINATION/ParentFolder{#}.jpg"
To view the commands to be run before executing them, try:
find ./ParentFolder -name "*.jpg" | parallel --dryrun "mv {} DESTINATION/ParentFolder{#}.jpg"
Use rename over the full path using file globbing :
*/*
If you don't understand, you can test it with :
echo */*
First * is your directory, second * is your file name. Catch them in a regular expression :
(.*)/(.*)
Now $1 is your parent folder name and $2 is your file name. You can easily build your solution like this :
rename -n "s/(.*)\/(.*)/\$1\/\$1\$2/" */*
It keeps directory structure and adds directory name as a prefix to each of its files. You could move your files up by simply changing \$1\/\$1\$2 to \$1\$2. You then just have to delete empty directories using rmdir.
I voluntarily added the option -n so you don't do a mess if you copy-paste. Simply remove the option when you think it's good.
adrien#adrienLT:~/Documents/PEV$ cp -r Holiday/ Holiday_copy/
adrien#adrienLT:~/Documents/PEV$ tree Holiday*
Holiday
├── France
│ ├── 1.jpg
│ ├── 2.jpg
│ └── 3.jpg
├── Italy1
│ ├── 1.jpg
│ ├── 2.jpg
│ └── 3.jpg
└── Italy2
├── 1.jpg
├── 2.jpg
└── 3.jpg
Holiday_copy
├── France
│ ├── 1.jpg
│ ├── 2.jpg
│ └── 3.jpg
├── Italy1
│ ├── 1.jpg
│ ├── 2.jpg
│ └── 3.jpg
└── Italy2
├── 1.jpg
├── 2.jpg
└── 3.jpg
6 directories, 18 files
adrien#adrienLT:~/Documents/PEV$ cd Holiday_copy/
adrien#adrienLT:~/Documents/PEV/Holiday_copy$ rename "s/(.*)\/(.*)/\$1\/\$1\$2/" */*
adrien#adrienLT:~/Documents/PEV/Holiday_copy$ tree .
.
├── France
│ ├── France1.jpg
│ ├── France2.jpg
│ └── France3.jpg
├── Italy1
│ ├── Italy11.jpg
│ ├── Italy12.jpg
│ └── Italy13.jpg
└── Italy2
├── Italy21.jpg
├── Italy22.jpg
└── Italy23.jpg
3 directories, 9 files
Related
What is the easiest way to get a list of all hidden and non-hidden directories (only directories) in a single call?
Obviously I can do it by connecting 2 different commands with && like this:
ls -d ./.*/ && ls -d ./*/
but shouldn't there be a better way?
EDIT: I do not want the current directory to be included in the list.
Also, ls -d ./.*/ ./*/ is a better alternative to what I have up there.
In bash you don't need to call any expternal utility to list all directories (including hidden ones). Just use:
# make sure we match entries starting with dot and return empty when list is empty
shopt -s dotglob nullglob
# print all the directories including ones starting with dot
printf '%s\n' */
Simple find should do it:
find /some/path -mindepth 1 -maxdepth 1 -type d
-maxdepth 1 ensures you don't descend into subdirectories. However, if that's what you want, you can just remove it.
-mindepth 1 ensures find does not include the directory itself.
If you have the tree command, you can get a nice view of your directories:
tree -a -d -L 3
where:
a -> shows hidden and nonhidden
d -> only shows directories
L 3 -> limit depth (if you want infinite depth, simply remove this option)
Output example:
.
├── bin
├── Common
├── .git
│ ├── branches
│ ├── hooks
│ ├── info
│ ├── logs
│ │ └── refs
│ ├── objects
│ │ ├── 08
│ │ ├── 38
│ │ ├── 62
│ │ ├── 6f
│ │ ├── 8c
│ │ ├── 9e
│ │ ├── a0
│ │ ├── cb
│ │ ├── d9
│ │ ├── info
│ │ └── pack
│ └── refs
│ ├── heads
│ ├── remotes
│ └── tags
├── Setup
├── test
└── tools
27 directories
You also get the number of directories. If you do not want it, add the --noreport option.
It is also possible to exclude pattern etc... man tree is your friend there.
Another example: a flat list of directories, excluding obj* and refs
tree -a -d -L 3 -if -I "obj*|refs" --noreport
returns:
.
./bin
./Common
./.git
./.git/branches
./.git/hooks
./.git/info
./.git/logs
./Setup
./test
./tools
In Linux I have a folder with the following structure:
.
├── a
│ ├── 00000400000000030bfd.dat
│ ├── 10000400000000030bfd.dat
│ ├── 20000400000000030bfd.dat
│ ├── etc....
├── b
│ ├── 00000401000000030bfd.dat
│ ├── 10000401000000030bfd.dat
│ ├── 20000401000000030bfd.dat
│ ├── etc....
├── c
│ ├── 00000402000000030bfd.dat
│ ├── 10000402000000030bfd.dat
│ ├── 20000402000000030bfd.dat
│ ├── etc....
├── d
│ ├── etc....
├── e
│ ├── etc....
├── f
And so on until the "p" folder. I want to rename every .dat file in every directory to .html file with a bash script. How i can do it?
Use a loop.
for file in {a..p}/*.dat ; do
mv "$file" "${file%.dat}.html"
done
${file%.dat} removes .dat from the end of the value of the $file.
Here is a version that uses all traditional commands:
find log -name "*.dat" |
sed 's/.log$//;s/mv &.dat &.html/' |
bash
Essentially, the find creates the target name list, the sed makes the names generic and then generates a mv command that does the rename and then pipes the results to bash for execution.
The bash command can be omitted to merely get a list of the mv commands for eyeball testing. You can also add a -x to the bash command to get a log of each mv command executed.
This will no doubt be simple for someone, but I'm not a bash wizard by any means. I tend to learn as I go when I need to do something. I've got a script that optimizes images and requires only that it be called with the image folder as an argument. It will determine which files can be used and travel down each subfolder as well.
I've created a simple loop that reads lines (paths to the image folders) from a txt file and feeds them to the optimization script one by one. My problem is that some of the folder trees contain a folder and subfolder branch that I want to skip, say a cache which contains a mirror of the image folder structure. What I need is a way to easily add that folder name, say to an exclude txt file, and have the bash loop ignore that and everything under it.
Here is the loop:
#!/bin/bash
filename='imgopt_sites.txt'
while read p; do
nice -n 10 bash /usr/local/bin/imgopt.sh $p > /dev/null 2>&1
done < $filename
Here is the txt file with folders to run on:
/srv/www/domain1.com/htdocs/wp-content/uploads
/srv/www/domain2.com/htdocs/wp-content/uploads
/srv/www/domain3.com/htdocs/wp-content/uploads
/srv/www/domain4.com/htdocs/wp-content/uploads
/srv/www/domain5.com/htdocs/wp-content/uploads
/srv/www/domain6.com/htdocs/image
An example of what would be excluded would be:
/srv/www/domain6.com/htdocs/image/cache (and all under that)
This could be in another txt file and would need to be looped as there will be more. Or, if there is an easy way to put them all in the same file and distinguish the exclude lines in some way that would be fine too. I've searched and nothing I've found really hits this. Thanks.
The OS is Ubuntu 14.04 and the loop is called by a cron job.
Last, here is a sample directory tree to give perspective:
/srv/www/domain6.com/htdocs/image
├── cache
│ ├── catalog
│ │ ├── apparel
│ │ ├── art
│ │ ├── banners
│ │ ├── commission
│ │ ├── demo
│ │ │ ├── banners
│ │ │ └── manufacturer
│ │ ├── illustrations
│ │ ├── jewelry
│ │ ├── misc
│ │ ├── news
│ │ ├── pen_ink
│ │ └── sample_data
│ │ ├── gallery
│ │ ├── logos
│ │ ├── patterns
│ │ ├── payments
│ │ └── slider
│ └── tb
├── catalog
│ ├── apparel
│ ├── art
│ ├── banners
│ ├── commission
│ ├── illustrations
│ ├── jewelry
│ ├── misc
│ ├── news
│ ├── pen_ink
│ └── sample_data
│ ├── banners
│ ├── features
│ ├── gallery
│ ├── logos
│ ├── patterns
│ │ └── _original
│ ├── payments
│ ├── slider
│ └── wallpaper
├── data
│ └── productimage
├── flags
└── templates
Any suggestions are appreciated.
Update: This is the section of code in the script itself that handles the input directories.
do_dir () {
# $1 is name of dir
local ret_val="0"
while IFS= read -r -u 3 -d $'\0' file; do
do_file "${file}"
ret_val=$?
done 3< <(find "${1}" -type f \( -iregex '.*\.[jp][pn]g$' -o -iregex '.*\.jpeg$' \) -print0)
return $ret_val
}
I'm a little confused over what the "-u 3" does.
Update 2: New version of loop with errors in response to #shellter
#!/bin/bash
filename='imgopt_sites.txt'
excludes='imgopt_excludes.txt'
set -vx; while read p ; do nice -n 10 bash /usr/local/bin/imgopt/imgopt-ng.sh -new -progressive -sqlite $p; done <( grep -v -F -f "$excludes" "$filename" )
Error:
/usr/local/bin/imgopt/imgopt_loop.sh: line 6: syntax error near unexpected token `<( grep -v -F -f "$excludes" "$filename" )'
/usr/local/bin/imgopt/imgopt_loop.sh: line 6: `set -vx; while read p ; do nice -n 10 bash /usr/local/bin/imgopt/imgopt-ng.sh -new -progressive -sqlite $p; done <( grep -v -F -f "$excludes" "$filename" )'
I have a bunch of video files that I want to put into directories that have the same name as the video file minus the extension (.mov)
For example, given the following directory, how would I write a script to create a directory for each .mov file with the same name as the file (minus the .mov) and then place the .mov file into that directory?
/Volumes/06a_SD_Video_01/_COLLECTIONS/TR2014_295_Walter_Reed/DER_01/
├── TR2014_295_10_1a_PM.mov
├── TR2014_295_11_1a_DER_01.mov
├── TR2014_295_12_1a_DER_01.mov
├── TR2014_295_13_1a_DER_01.mov
├── TR2014_295_14_1a_PM.mov
├── TR2014_295_15_1a_DER_01.mov
├── TR2014_295_16_1a_DER_01.mov
├── TR2014_295_17_1a_DER_01.mov
├── TR2014_295_18_1a_DER_01.mov
├── TR2014_295_19_1a_DER_01.mov
├── TR2014_295_19_1b_DER_01.mov
├── TR2014_295_1_1a_PM.mov
├── TR2014_295_20_1a_DER_01.mov
├── TR2014_295_21_1a_PM.mov
├── TR2014_295_22_1a_DER_01.mov
├── TR2014_295_22_1b_DER_01.mov
├── TR2014_295_23_1a_DER_01.mov
├── TR2014_295_2_1a_PM.mov
├── TR2014_295_3_1a_DER_01.mov
├── TR2014_295_4_1a_DER_01.mov
├── TR2014_295_5_1a_DER_01.mov
├── TR2014_295_6_1a_DER_01.mov
├── TR2014_295_7_1a_DER_01.mov
├── TR2014_295_8_1a_DER_01.mov
├── TR2014_295_9_1a_DER_01.mov
here is a way:
script:
cd "$1"
for m in *.mov; do
d=`echo "$m" | sed 's/\.mov$//'`
mkdir "$d"
mv "$m" "$d"
done
usage:
sh <scriptname.sh> "<directory>"
e.g.,
sh mov2dir.sh "/Volumes/06a_SD_Video_01/_COLLECTIONS/TR2014_295_Walter_Reed/DER_01/"
Much research has turned almost similar questions yet nothing close enough to give me an idea of how to accomplish part my task. I'll try to keep this clear and short, while explaining the situation and desired result. My structure would be as follows:
-mobile
--Docs
--Downloads
--SomeFile
----this.is.crazy_0.0.1-1_named-silly.txt
----dont.touch.me.pdf
----leave.me.alone.png
----this.is.crazy_0.0.1-2_named-silly.txt
----this.is.crazy_0.0.1-3_named-silly.txt <---- file to keep
--SomeFileA
----this.is.crazy_0.0.1-1_also-silly.txt
----this.is.crazy_0.0.1-2_also-silly.txt
----dont.touch.me.either.pdf
----leave.me.alone.too.png
----this.is.crazy_0.0.1-3_also-silly.txt
----this.is.crazy_0.0.1-11_also-silly.txt <----file to keep
The first part of my script to find the .txt files ignores every directory that is constant in this working directory and prints them to a list (which is a completely ugly hack and most likely a hinder to the way most would accomplish this task) "SomeFileB and SomeFileC" could come along with the same file structure and I'd like to catch them in this script as well.
The idea is to keep the newest .txt file in each directory according to its time stamp which obviously isn't in the filename. The files to keep will continue to change of course. To clarify the question again, how to go about keeping the newest .txt file in each variable directory with variable crazy name, according to timestamp which isn't in the filename? Hopefully I've been clear enough for help. This script should be in bash.
I'm not with the current code right now, as i said its ugly but heres a snippet of what I have find /path/to/working/directory -maxdepth 0 -not -path "*Docs*" -not -path "*Downloads* -name "*.txt" >list
Assuming the question was understood correctly, the task could be expressed as:
Recursively remove all files *.txt except the newest in each respective directory
#!/bin/bash
# Find all directories from top of tree
find a -type d | while read -r dir; do
# skip $dir if doesn't contain any files *.txt
ls "$dir"/*.txt &>/dev/null || continue
# list *.txt by timestamp, skipping the newest file
ls -t "$dir"/*.txt | awk 'NR>1' | while read -r file; do
rm "$file"
done
done
Assuming this directory tree, where a.txt is always the newest:
$ tree -t a
a
├── otherdir
├── b
│ ├── d e
│ │ ├── a.txt
│ │ ├── b.txt
│ │ ├── c.txt
│ │ ├── bar.txt
│ │ └── foo.pdf
│ ├── c
│ │ ├── a.txt
│ │ ├── b.txt
│ │ └── c.txt
│ ├── a.txt
│ ├── b.txt
│ ├── c.txt
│ └── foo.pdf
├── a.txt
├── b.txt
└── c.txt
This is the result after running the script:
$ tree -t a
a
├── b
│ ├── c
│ │ └── a.txt
│ ├── d e
│ │ ├── a.txt
│ │ └── foo.pdf
│ ├── a.txt
│ └── foo.pdf
├── otherdir
└── a.txt
Change rm "$file" to echo rm "$file" to check what would be removed before running "for real"