I'm writing a script that will perform some actions, and one of those actions is to find all occurrences of a string in both file names and directory names, and replace it with another string.
I have this so far
find . -name "*foo*" -type f -depth | while read file; do
newpath=${file//foo/bar}
mv "$file" "$newpath"
done
This works fine as long as the path to the file doesn't also contain foo, but that isn't guaranteed.
I feel like the way to approach this is to ONLY change the file names first, then go back through and change the directory names, but even then, if you have a structure that has more than one directory with foo in it, it will not work properly.
Is there a way to do this with built in macOS tools? (I say built-in, because this script is going to be distributed to some other folks in our organization and it can't rely on any packages to be installed).
Separating the path_name from the file_name, something like.
#!/usr/bin/env bash
while read -r file; do
path_name="${file%/*}"; printf 'Path is %s\n' "$path_name"
file_name="${file#"$path_name"}"; printf 'Filename is %s\n' "$file_name"
newpath="$path_name${file_name//foo/bar}"
echo mv -v "$file" "$newpath"
done < <(find . -name "*foo*" -type f)
Have a look at basename and dirname as well.
The printf's is just there to show which is the path and the filename.
The script just replace foo to bar from the file_name, It can be done with the path_name as well, just use the same syntax.
newpath="${path_name//bar/more}${file_name//foo/bar}"
So renaming both path_name and file_name.
Or renaming the path_name and then the file_name like your idea is an option also.
path_name="${file%/*}"
file_name="${file#"$path_name"}"
new_pathname="${path_name//bar/more}"
mv -v "$path_name" "$new_pathname"
new_filename="${file_name//foo/bar}"
mv -v "${new_pathname%/*}$file_name" "$new_pathname$new_filename"
There are no additional external tool/utility used, except from the ones being used by your script.
Remove the echo If you're satisfied with the result/output.
You can use -execdir to run a command on just the filename (basename) in the relevant directory:
find . -depth -name '*foo*' -execdir bash -c 'mv -- "${1}" "${1//foo/bar}"' _ {} \;
Related
Update 01/12/2022
With triplee's helpful suggestions, I resolved it to take both files & directories by adding a comma in between f and d, the final code now looks like this:
while read -r old new;
do echo "replacing ${old} by ${new}" >&2
find '/path/to/dir' -depth -type d,f -name "$old" -exec rename
"s/${old}/${new}/" {} ';'
done <input.txt
Thank you!
Original request:
I am trying to rename a list of files (from $old to $new), all present in $homedir or in subdirectories in $homedir.
In the command line this line works to rename files in the subfolders:
find ${homedir}/ -name ${old} -exec rename "s/${old}/${new}/" */${old} ';'
However, when I want to implement this line in a simple bash script getting the $old and $new filenames from input.txt, it doesn't work anymore...
input.txt looks like this:
name_old name_new
name_old2 name_new2
etc...
the script looks like this:
#!/bin/bash
homedir='/path/to/dir'
cat input.txt | while read old new;
do
echo 'replacing' ${old} 'by' ${new}
find ${homedir}/ -name ${old} -exec rename "s/${old}/${new}/" */${old} ';'
done
After running the script, the text line from echo with $old and $new filenames being replaced is printed for the entire loop, but no files are renamed. No error is printed either. What am I missing? Your help would be greatly appreaciated!
I checked whether the $old and $new variables were correctly passed to the find -exec rename command, but because they are printed by echo that doesn't seem to be the issue.
If you add an echo, like -exec echo rename ..., you'll see what actually gets executed. I'd say that both the path to $old is wrong (you're not using the result of find in the -exec clause), and */$old isn't quoted and might be expanded by the shell before find ever gets to see it.
You're also having most other expansions unquoted, which can lead to all sorts of trouble.
You could do it in pure Bash (drop echo when output looks good):
shopt -s globstar
for f in **/"$old"; do echo mv "$f" "${f/%*/$new}"; done
Or with rename directly, though this would run into trouble if too many files match (drop -n when output looks good):
rename -n "s/$old\$/$new/" **/"$old"
Or with GNU find, using -execdir to run in the same directory as the matching file (drop echo when output looks good):
find -type f -name "$old" -execdir echo mv "$old" "$new" \;
And finally, a version with find that spawns just a single subshell (drop echo when output looks right):
find -type f -name "$old" -exec bash -c '
new=$1
shift
for f; do
echo mv "$f" "${f/%*/$new}"
done
' bash "$new" {} +
The argument to rename should be the file itself, not */${old}. You also have a number of quoting errors, and a useless cat).
#!/bin/bash
while read -r old new;
do
echo "replacing ${old} by ${new}" >&2
find /path/to/dir -name "$old" -exec rename "s/${old}/${new}/" {} ';'
done <input.txt
Running find multiple times on the same directory is hugely inefficient, though. Probably a better solution is to find all files in one go, and abort if it's not one of the files on the list.
find /path/to/dir -type f -exec sh -c '
for f in "$#"; do
awk -v f="$f" "f==\$1 { print \"s/\" \$1 \"/\" \$2 \"/\" }" "$0" |
xargs -I _ -r rename _ "$f"
done' input.txt {} +
(Untested; probably try with echo before you run this live.)
The scenario is that I want to convert all of my music files from .mp3 to .ogg. They are in a folder called "Music". In this folder there are folders and files. The files are .mp3s. The directories may contain .mp3s or directories which further contain .mp3s or directories, and so on. This is because some artists have albums which have parts and some do not, etc.
I want to write a script that converts each file using avconv.
Basically, what I am going to do is manually cd into every directory and run the following:
for file in $(ls); do avconv -i $file `echo \`basename $file .mp3\`.ogg`; done
This successfully gets me what I want. However, this is not great as I have a lot of folders, and manually going into each of them and executing this is slow.
My question, then, is how do I write a script that runs this in any directory that has .mp3s, and then goes into any subdirectory it finds and recursively calls itself? My intuition tells me to use Perl or Python because of the complex nature of this.
Thanks for any suggestions!
I'm not familiar with avconv but assuming your command is:
avconv -i inputname outputname
And you want to convert all inputname.mp3 to inputname.ogg in their original directories below Music, then the following should work in bash:
#!/bin/bash
while read -r fname; do
avconv -i "$fname" "${fname%.mp3}.ogg"
done < <(find /path/to/Music -type f -name "*.mp3")
Note: this does not remove the original .mp3, and the space between < < is required. Also note, for file in $(ls) is filled with potential for errors.
You can do it with bash in one liner:
First you find all files (of type file (-type f) ) that match next pattern "*.mp3". To read each one you use 'while' and invoke avconf.
For exchange extension I prefer 'sed' command, that keep folder so you don't need the 'cd' command.
Notice that you must put quotes on $FN variable because it can contain spaces.
find -type f -iname "*.mp3" | while read "FN" ; do avconf -i "$FN" $(echo "$FN" | sed 's/\.mp3/\.ogg/g') ; done
find <music-folder> -type f -name '*.mp3' | \
xargs -I{} bash -c 'mp3="$0"; ogg="${mp3%.mp3}.ogg"; avconv -i "$mp3" "$ogg";' {}
This should survive in cases of "weird" filenames with spaces, quotes and other strange symbols within.
You can list directories with absolute paths and recursively cd into every directory using find $PWD -type d syntax:
Just inside from Music directory run:
for d in $(find $PWD -type d)
do
cd $d
for file in $(find . -maxdepth 1 -type f)
do
echo $file
avconv -i $file `echo \`basename $file .mp3\`.ogg`
done
done
The history of this problem is:
I have millions of files and directories on a NAS system. I found a count of 1,095,601 empty (0 byte) files. These files used to have data but were destroyed by a predecessor not using the correct toolsets to migrate data between an XSAN and this Isilon NAS.
The files were media production data, like fonts, pdfs and image files. They are no longer useful beyond the history of their existence. Before I proceed to delete them, the production user's need a record of which files used to exist, so when they browse a project folder, they can use the unaffected files but then refer to a text file in the same directory which records which files used to also be there and thus provide reason as to why certain reference files are broken.
So how do I find files across multiple directories and delete them but first output their filename to a text file which would be saved to each relevant path location?
I am thinking along the lines of:
for file in $(find . -type f -size 0); do
echo "$file" >> /PATH/TO/FOUND/FILE/PARENT/DIR/deletedFiles.txt -print0 |
xargs -0 rm ;
done
To delete each empty file while leaving behind a file called deletedFiles.txt which contains the names of the deleted files, try:
PATH=/bin:/usr/bin find . -empty -type f -execdir bash -c 'printf "%s\n" "$#" >>deletedFiles.txt' none {} + -delete
How it works
PATH=/bin:/usr/bin
This sets a temporary but secure path.
find .
This starts find looking in the current directory
-empty
This tells find to only look for empty files
-type f
This restricts find to looking for regular files.
-execdir bash -c 'printf "%s\n" "$#" >>deletedFiles.txt' none {} +
In each directory that contains an empty file, this adds the name of each empty file to the file deletedFiles.txt.
Notice the peculiar use of none in the command:
bash -c 'printf "%s\n" "$#" >>deletedFiles.txt' none {} +
When this command is run, bash will execute the string printf "%s\n" "$#" >>deletedFiles.txt and the arguments that follow that string are assigned to the positional parameters: $0, $1, $2, etc. When we use $#, it does not include $0. It, as is usual, expands to $1, $2, .... Thus, we add the placeholder none so that the placeholder is assigned is the $0, which we will ignore, and the complete list of file names are assigned to "$#".
-delete
This deletes each empty file.
Why not simply
find . -type f -size 0 -exec rm -v + |
sed -e 's%^removed .\./%%' -e 's/.$//' >deletedFiles.txt
If your find is too old to support -exec ... + you'll need to revert to -exec rm -v {} \; or refactor to
find . -type f -size 0 -print0 |
xargs -r -0 rm -v |
sed -e 's%^removed .\./%%' -e 's/.$//' >deletedFiles.txt
The brief sed script is to postprocess the output from rm -v which looks like
removed ‘./bar’
removed ‘./foo’
(with some funny quote characters around the file name) on my system. If you are fine with that output, of course, just omit the sed script from the pipeline.
If you know in advance which directories contain empty files, you can run the above snippet individually in those directories. Assuming you saved the snippet above as a script (with a proper shebang and execute permissions) named find-empty, you could simply use
for path in /path/to/first /path/to/second/directory /path/to/etc; do
cd "$path" && find-empty
done
This will only work if you have absolute paths (if not, you can run the body of the loop in a subshell by adding parentheses around it).
If you want to inspect all the directories in a tree, change the script to print to standard output instead (remove >deletedFiles.txt from the script) and try something like
find /path/to/tree -type d -exec sh -c '
t=$(mktemp -t find-emptyXXXXXXXX)
cd "$1" &&
find-empty | grep . >"$t" &&
mv "$t" deletedFiles.txt ||
rm "$t"' _ {} \;
This uses a temporary file so as to avoid updating the timestamp of directories which do not contain any empty files. The grep . is used purely for side effect; if any (non-empty) lines are printed, it will return success, whereas otherwise, it will report failure; this way, we know whether or not to move the temporary file to the target directory.
With prompting from #JonathanLeffler I have succeeded with the following:
#!/bin/bash
## call this script with: find . -type f -empty -exec handleEmpty.sh {} +
for file in "$#"
do
file2="$(basename "$file")"
echo "$file2" >> "$(dirname "$file")"/deletedFiles.txt
rm "$file"
done
This means I retain a trace of the removed files in a deletedFiles.txt flag file in each respective directory for the users to see when files are missing. That way, they can pursue going back to archive CD's to retrieve these deleted files, which are hopefully not 0 byte files.
Thanks to #John1024 for the suggestion of using the empty flag rather than size.
This question already has an answer here:
Closed 10 years ago.
Possible Duplicate:
makefile: how to add a prefix to the basename?
I have a lit of files (which I get from find bla -name "*.so") such as:
/bla/a1.so
/bla/a2.so
/bla/blo/a3.so
/bla/blo/a4.so
/bla/blo/bli/a5.so
and I want to rename them such as it becomes:
/bla/liba1.so
/bla/liba2.so
/bla/blo/liba3.so
/bla/blo/liba4.so
/bla/blo/bli/liba5.so
... i.e. add the prefix 'lib' to the basename
any idea on how to do that in bash ?
Something along the lines of:
for a in /bla/a1.so /bla/a2.so /bla/blo/a4.so
do
dn=$(dirname $a)
fn=$(basename $a)
mv "$a" "${dn}/lib${fn}"
done
should do it. You might want to add code to read the list of filenames from a file, rather than listing them verbatim in the script, of course.
find . -name "*.so" -printf "mv '%h/%f' '%h/lib%f'\n" | bash
The code will rename files in current directory and subdirectories to append "lib" in front of .so filenames.
No looping needed, as find already does its recursive work to list the files. The code builds the "mv" commands one by one and executes them. To see the "mv" commands without executing them, simply remove the piping to shell part "| bash".
find's printf command understands many variables which makes it pretty scalable. I only needed to use two here:
%h: directory
%f: filename
How to test it:
Run this first (will perform nothing yet, only print lines on the screen):
find . -name "*.so" -printf "mv '%h/%f' '%h/lib%f'\n" | less -S
This will show you all the commands that your script will execute. If you're satisfied with the result, simply execute it afterwards by piping it into bash instead of less.
find . -name "*.so" -printf "mv '%h/%f' '%h/lib%f'\n" | bash
while multiliner
A slightly more robust and generalized solution based on $nfm (maybe more than you really need) would be
while IFS= read -r -u3 -d $'\0' FILE; do
DIR=`dirname $FILE`;
FILENAME=`basename $FILE`;
mv $FILE ${DIR}/lib${FILENAME};
done 3< <(find bla -name *.so -print0 | sort -rz)
This is quite robust:
read -u3 and 3< does not interfere with stdin
-print0 + IFS= + -d $'/0' allows for newlines in filenames
sort -rz renames deeper paths first, so that you can even rename directories and the files inside them at once
find -execdir + rename
This would be perfect if it weren't for the PATH annoyances, see: Find multiple files and rename them in Linux
Try mmv:
cd /bla/
mmv "*.so" "lib#1.so"
(mmv "*" "lib#1" would also work but it's less safe).
If you don't have mmv installed, get it.
basename and dirname are your friends :)
You want something like this (excuse my bash syntax - it's a little rusty):
for FILE in `find bla -name *.so` do
DIR=`dirname $FILE`;
FILENAME=`basename $FILE`;
mv $FILE ${DIR}/lib${FILENAME};
done
Beaten to the punch!
Note I've commented out the mv command to prevent any accidental mayhem
for f in *
do
dir=`dirname "$f"`
fname=`basename "$f"`
new="$dir/lib$fname"
echo "new name is $new"
# only uncomment this if you know what you are doing
# mv "$f" "$new"
done
In a directory, I have a bunch of *.html files. I'd like to rename them all to *.txt
How can I do that? I use the bash shell.
If using bash, there's no need for external commands like sed, basename, rename, expr, etc.
for file in *.html
do
mv "$file" "${file%.html}.txt"
done
For an better solution (with only bash functionality, as opposed to external calls), see one of the other answers.
The following would do and does not require the system to have the rename program (although you would most often have this on a system):
for file in *.html; do
mv "$file" "$(basename "$file" .html).txt"
done
EDIT: As pointed out in the comments, this does not work for filenames with spaces in them without proper quoting (now added above). When working purely on your own files that you know do not have spaces in the filenames this will work but whenever you write something that may be reused at a later time, do not skip proper quoting.
rename 's/\.html$/\.txt/' *.html
does exactly what you want.
This worked for me on OSX from .txt to .txt_bak
find . -name '*.txt' -exec sh -c 'mv "$0" "${0%.txt}.txt_bak"' {} \;
You want to use rename :
rename -S <old_extension> <new_extension> <files>
rename -S .html .txt *.html
This does exactly what you want - it will change the extension from .html to .txt for all files matching *.html.
Note: Greg Hewgill correctly points out this is not a bash builtin; and is a separate Linux command. If you just need something on Linux this should work fine; if you need something more cross-platform then take a look at one of the other answers.
On a Mac...
Install rename if you haven't: brew install rename
rename -S .html .txt *.html
For Ubuntu Users :
rename 's/\.html$/\.txt/' *.html
This is the slickest solution I've found that works on OSX and Linux, and it works nicely with git too!
find . -name "*.js" -exec bash -c 'mv "$1" "${1%.js}".tsx' - '{}' \;
and with git:
find . -name "*.js" -exec bash -c 'git mv "$1" "${1%.js}".tsx' - '{}' \;
This question explicitly mentions Bash, but if you happen to have ZSH available it is pretty simple:
zmv '(*).*' '$1.txt'
If you get zsh: command not found: zmv then simply run:
autoload -U zmv
And then try again.
Thanks to this original article for the tip about zmv.
Here is an example of the rename command:
rename -n ’s/\.htm$/\.html/’ *.htm
The -n means that it's a test run and will not actually change any files. It will show you a list of files that would be renamed if you removed the -n. In the case above, it will convert all files in the current directory from a file extension of .htm to .html.
If the output of the above test run looked ok then you could run the final version:
rename -v ’s/\.htm$/\.html/’ *.htm
The -v is optional, but it's a good idea to include it because it is the only record you will have of changes that were made by the rename command as shown in the sample output below:
$ rename -v 's/\.htm$/\.html/' *.htm
3.htm renamed as 3.html
4.htm renamed as 4.html
5.htm renamed as 5.html
The tricky part in the middle is a Perl substitution with regular expressions, highlighted below:
rename -v ’s/\.htm$/\.html/’ *.htm
One line, no loops:
ls -1 | xargs -L 1 -I {} bash -c 'mv $1 "${1%.*}.txt"' _ {}
Example:
$ ls
60acbc4d-3a75-4090-85ad-b7d027df8145.json ac8453e2-0d82-4d43-b80e-205edb754700.json
$ ls -1 | xargs -L 1 -I {} bash -c 'mv $1 "${1%.*}.txt"' _ {}
$ ls
60acbc4d-3a75-4090-85ad-b7d027df8145.txt ac8453e2-0d82-4d43-b80e-205edb754700.txt
The command mmv seems to do this task very efficiently on a huge number of files (tens of thousands in a second). For example, to rename all .xml files to .html files, use this:
mmv ";*.xml" "#1#2.html"
the ; will match the path, the * will match the filename, and these are referred to as #1 and #2 in the replacement name.
Answers based on exec or pipes were either too slow or failed on a very large number of files.
In Linux or window git bash or window's wsl, try below command to change every file's extension in current directory or sub-directories or even their sub-directories with just one line of code
find . -depth -name "*.html" -exec sh -c 'mv "$1" "${1%.html}.txt"' _ {} \;
Try this
rename .html .txt *.html
usage:
rename [find] [replace_with] [criteria]
After someone else's website crawl, I ended up with thousands of files missing the .html extension, across a wide tree of subdirectories.
To rename them all in one shot, except the files already having a .html extension (most of them had none at all), this worked for me:
cd wwwroot
find . -xtype f \! -iname *.html -exec mv -iv "{}" "{}.html" \; # batch rename files to append .html suffix IF MISSING
In the OP's case I might modify that slightly, to only rename *.txt files, like so:
find . -xtype f -iname *.txt -exec filename="{}" mv -iv ${filename%.*}.{txt,html} \;
Broken down (hammertime!):
-iname *.txt
- Means consider ONLY files already ending in .txt
mv -iv "{}.{txt,html}"
- When find passes a {} as the filename, ${filename%.*} extracts its basename without any extension to form the parameters to mv. bash takes the {txt,html} to rewrite it as two parameters so the final command runs as: mv -iv "filename.txt" "filename.html"
Fix needed though: dealing with spaces in filenames
This is a good way to modify multiple extensions at once:
for fname in *.{mp4,avi}
do
mv -v "$fname" "${fname%.???}.mkv"
done
Note: be careful at the extension size to be the same (the ???)
Rename file extensions for all files under current directory and sub directories without any other packages (only use shell script):
Create a shell script rename.sh under current directory with the following code:
#!/bin/bash
for file in $(find . -name "*$1"); do
mv "$file" "${file%$1}$2"
done
Run it by ./rename.sh .old .new.
Eg. ./rename.sh .html .txt
A bit late to the party. You could do it with xargs:
ls *.html | xargs -I {} sh -c 'mv $1 `basename $1 .html`.txt' - {}
Or if all your files are in some folder
ls folder/*.html | xargs -I {} sh -c 'mv $1 folder/`basename $1 .html`.txt' - {}
Similarly to what was suggested before, this is how I did it:
find . -name '*OldText*' -exec sh -c 'mv "$0" "${0/OldText/NewText}"' {} \;
I first validated with
find . -name '*OldText*' -exec sh -c 'echo mv "$0" "${0/OldText/NewText}"' {} \;
Nice & simple!
find . -iname *.html -exec mv {} "$(basename {} .html).text" \;
If you prefer PERL, there is a short PERL script (originally written by Larry Wall, the creator of PERL) that will do exactly what you want here:
tips.webdesign10.com/files/rename.pl.txt.
For your example the following should do the trick:
rename.pl 's/html/txt/' *.html
The easiest way is to use rename.ul it is present in most of the Linux distro
rename.ul -o -v [oldFileExtension] [newFileExtension] [expression to search for file to be applied with]
rename.ul -o -v .oldext .newext *.oldext
Options:
-o: don't overwrite preexisting .newext
-v: verbose
-n: dry run
Unfortunately it's not trivial to do portably. You probably need a bit of expr magic.
for file in *.html; do echo mv -- "$file" "$(expr "$file" : '\(.*\)\.html').txt"; done
Remove the echo once you're happy it does what you want.
Edit: basename is probably a little more readable for this particular case, although expr is more flexible in general.
Here is what i used to rename .edge files to .blade.php
for file in *.edge; do mv "$file" "$(basename "$file" .edge).blade.php"; done
Works like charm.
You can also make a function in Bash, add it to .bashrc or something and then use it wherever you want.
change-ext() {
for file in *.$1; do mv "$file" "$(basename "$file" .$1).$2"; done
}
Usage:
change-ext css scss
Source of code in function: https://stackoverflow.com/a/1224786/6732111
Here is a solution, using AWK. Make sure the files are present in the working directory. Else, cd to the directory where the html files are located and then execute the below command:
for i in $(ls | grep .html); do j=$(echo $i | grep -oh "^\w*." | awk '{print $1"txt"}'); mv $i $j; done
I wrote this code in my .bashrc
alias find-ext='read -p "Path (dot for current): " p_path; read -p "Ext (unpunctured): " p_ext1; find $p_path -type f -name "*."$p_ext1'
alias rename-ext='read -p "Path (dot for current): " p_path; read -p "Ext (unpunctured): " p_ext1; read -p "Change by ext. (unpunctured): " p_ext2; echo -en "\nFound files:\n"; find $p_path -type f -name "*.$p_ext1"; find $p_path -type f -name "*.$p_ext1" -exec sh -c '\''mv "$1" "${1%.'\''$p_ext1'\''}.'\''$p_ext2'\''" '\'' _ {} \;; echo -en "\nChanged Files:\n"; find $p_path -type f -name "*.$p_ext2";'
In a folder like "/home/<user>/example-files" having this structure:
/home/<user>/example-files:
file1.txt
file2.txt
file3.pdf
file4.csv
The commands would behave like this:
~$ find-text
Path (dot for current): example-files/
Ext (unpunctured): txt
example-files/file1.txt
example-files/file2.txt
~$ rename-text
Path (dot for current): ./example-files
Ext (unpunctured): txt
Change by ext. (unpunctured): mp3
Found files:
./example-files/file1.txt
./example-files/file1.txt
Changed Files:
./example-files/file1.mp3
./example-files/file1.mp3
~$
You could use a tool designed for renaming files in bulk, e.g. renamer.
To rename all file extensions in the current folder:
$ renamer --find ".html" --replace ".txt" --dry-run *
Many more usage examples here.