I need to find all files recursively with the name 'config.xml' and set them aside for analysis. The paths have spaces in them just to keep it interesting. However, I need them to be unique or they will collide in the same folder. What I would like to do is basically copy them off but using the name of the directory they were found in. The command I want is something like from this question except I need it to do something like $(dirname {}). When I do that, nothing gets moved (but I get no error)
Sample, but non-functional command:
find . -name 'config.xml' -exec sh -c 'cp "$1" "$2.xml"' -- {} "$HOME/data/$(dirname {})" \;
To do this with just one shell, not one per file found (as used by prior answers):
while IFS= read -r -d '' filename; do
outFile="$HOME/data/${filename%/*}.xml"
mkdir -p -- "${outFile%/*}"
cp -- "$filename" "$outFile"
done < <(find . -name 'config.xml' -print0)
This way your find emits a NUL-delimited stream of filenames, consumed one-by-one by the while read loop in the parent shell.
(You could use "$HOME/data/$(dirname "$filename").xml", but from a performance perspective that's really silly: $() fork()s off a subshell, and dirname is an external executable that needs to be exec'd, linked and loaded; no point to all that overhead when you can just do the string manipulation internal to the shell itself).
You may use it like this:
find . -name 'config.xml' -exec bash -c \
'd="$HOME/data/${1%/*}/"; mkdir -p "$d"; command cp -p "$1" "$d"' - {} \;
-exec sh is a little hard to handle, but not impossible. The $(dirname ...) is expanded prior sh is run, so it's equal dirname {} - the dirname of file {}. Do something like -exec sh -c ' .... ' -- {} and put the $(dirname ... ) inside sh script using $1.
find . -name 'config.xml' -exec sh -c 'cp "$1" "$2/data/$(dirname "$1").xml"' -- {} "$HOME" \;
Whole story: I am writing the script that will link all files from one directory to another. New file name will contain an original directory name. I use find at this moment with -execdir option.
This is how I want to use it:
./linkPictures.sh 2017_wien 2017/10
And it will create a symbolic link 2017_wien_picture.jpg in 2017/10 pointing to a file 2017_wien/picture.jpg.
This is my current script:
#!/bin/bash
UPLOAD="/var/www/wordpress/wp-content/uploads"
SOURCE="$UPLOAD/photo-gallery/$1/"
DEST="$UPLOAD/$2/"
find $SOURCE -type f -execdir echo ln -s {} $DEST/"$1"_{} ";"
It prints:
ln -s ./DSC03278.JPG /var/www/wordpress/wp-content/uploads/2017/10/pokus_./DSC03278.JPG
This is what I want:
ln -s ./DSC03278.JPG /var/www/wordpress/wp-content/uploads/2017/10/pokus_DSC03278.JPG
How to implement it? I do not know how to incorporate basename into to strip ./.
To run basename on {} you would need to execute a command through sh:
find "$SOURCE" -type f -execdir sh -c "echo ln -s '{}' \"$DEST/${1}_\$(basename \"{}\")\"" ";"
This won't win any speed contests (because of the sh for every file), but it will work.
All the quoting may look a bit crazy, but it's necessary to make it safe for files that may contain spaces.
You can use this find with bash -c:
find $SOURCE -type f -execdir bash -c 'echo ln -s "$2" "/$DEST/$1"_${2#./}' - "$1" '{}' \;
${2#./} will strip starting ./ from each entry of find command's output.
$1 will be passed as is to bash -c command line.
If you have large number of files to process I suggest using this while loop using a process substitution for faster execution since it doesn't spawn a new bash for every file. Moreover it will also handle filenames with white-spaces and other special characters:
while IFS= read -r file; do
echo ln -s "$file" "/$DEST/${1}_${file#./}"
done < <(find "$SOURCE" -type f -print0)
I know you can do this to rename all filenames in a single folder with something like this:
for file in 1_*; do
mv "$file" "${file/1_/}"
done
However, is there a way to do this across multiple folders? For example, it will search through all the folders in the current directory and change them.
I have bash version 4.3
A robust solution, assuming you have GNU or BSD/OSX find:
find . -type f -name '1_*' -execdir sh -c 'echo mv -- "$1" "${1#1_}"' _ {} \;
Note:
- This will only echo the mv commands, to be safe; remove the echo to perform actual renaming.
- The OP's substitution, "${file/1_/}" was changed to the POSIX-compliant "${file#1_}", which is actually closer to the intent.
- If you truly need a substitution such as "${file/1_/}", which the sh on your system may or may not support, it is better to explicitly invoke a shell known to support it, such as bash.
- Symlinks are ignored (both files and directories); use find -L ... to include them (both as potential files to be renamed and to make find descend into symlinks to directories).
find . -type f -name '1_*' finds all files (-type f) with names matching 1_* (-name '1_*') in the current dir.'s (.) subtree.
-execdir executes the command passed to it in the subdirectory in which the file at hand is located.
sh -c 'echo mv -- "$1" "${1#1_}"' _ {} \; invokes the default shell (sh):
with a command string (passed to -c)
mv -- "$1" "${1#1_}" effectively removes prefix 1_ from the filename represented by the first positional parameter ($1).
and dummy parameter _ (which sh will assign to $0, which is not of interest here)
and the path of the file at hand, {}, which the shell will bind to $1;
\; simply terminates -execdir's argument.
Note that -- ensures that any filename that happens to start with - isn't mistaken for an option by mv (applies analogously below).
-execdir is not POSIX-compliant; if a POSIX-compliant variant is therefore more cumbersome:
find . -type f -name '1_*' -exec sh -c \
'cd -- "${1%/*}" && f=${1##*/} && echo mv -- "$f" "${f#1_}"' _ {} \;
cd -- "${1%/*}" changes to the directory in which the file at hand is located.
Note: cd -- "$(dirname -- "$1")" is generally more robust, but less efficient; since we know that $1 is always a path rather than a mere filename in this scenario, we can use the more efficient cd -- "${1%/*}".
f=${1##*/} extracts the mere filename from the file path at hand.
The remainder of the command line then works as above, analogously.
Performance note:
The above solutions are concise, but inefficient, because a shell instance has to be spawned for each matching file.
A potential speed-up is to use a variant of peak's approach, but only if you avoid calling external utilities in the while loop (except for mv):
find . -type f -name '1_*' | while IFS= read -r f; do
new_name=${f##*/} # extract filename
new_name=${new_name#1_} # perform substitution
d=${f%/*} # extract dir
echo mv -- "$f" "$d/$new_name" # call mv to rename
done
The above bears the hypothetical risk of breaking with filenames with embedded newlines (very rare); with GNU or BSD find, this problem could be solved.
With this approach, only a single shell instance is spawned, which processes all matching filenames in a loop - as long as the only external utility that is called in the loop is mv, this will generally be faster than the find-only solutions with -exec[dir].
If you don't want to depend on too many subtleties, consider this very pedestrian approach, which assumes a bash-like shell and that all the usual suspects (find, sed, ....) are directly available:
find . -type f -name "1_*" | while read -r file ; do
x=$(basename "$file")
y=$(sed 's/1_//' <<< "$x")
d=$(dirname "$file")
mv "$file" "$d/$y"
done
(You might want to try this using "mv -i" or "echo mv ....". You might also want to use find with the -follow option.)
When I am using xargs sometimes I do not need to explicitly use the replacing string:
find . -name "*.txt" | xargs rm -rf
In other cases, I want to specify the replacing string in order to do things like:
find . -name "*.txt" | xargs -I '{}' mv '{}' /foo/'{}'.bar
The previous command would move all the text files under the current directory into /foo and it will append the extension bar to all the files.
If instead of appending some text to the replace string, I wanted to modify that string such that I could insert some text between the name and extension of the files, how could I do that? For instance, let's say I want to do the same as in the previous example, but the files should be renamed/moved from <name>.txt to /foo/<name>.bar.txt (instead of /foo/<name>.txt.bar).
UPDATE: I manage to find a solution:
find . -name "*.txt" | xargs -I{} \
sh -c 'base=$(basename $1) ; name=${base%.*} ; ext=${base##*.} ; \
mv "$1" "foo/${name}.bar.${ext}"' -- {}
But I wonder if there is a shorter/better solution.
The following command constructs the move command with xargs, replaces the second occurrence of '.' with '.bar.', then executes the commands with bash, working on mac OSX.
ls *.txt | xargs -I {} echo mv {} foo/{} | sed 's/\./.bar./2' | bash
It is possible to do this in one pass (tested in GNU) avoiding the use of the temporary variable assignments
find . -name "*.txt" | xargs -I{} sh -c 'mv "$1" "foo/$(basename ${1%.*}).new.${1##*.}"' -- {}
In cases like this, a while loop would be more readable:
find . -name "*.txt" | while IFS= read -r pathname; do
base=$(basename "$pathname"); name=${base%.*}; ext=${base##*.}
mv "$pathname" "foo/${name}.bar.${ext}"
done
Note that you may find files with the same name in different subdirectories. Are you OK with duplicates being over-written by mv?
If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:
find . -name "*.txt" | parallel 'ext={/} ; mv -- {} foo/{/.}.bar."${ext##*.}"'
Watch the intro videos for GNU Parallel to learn more:
https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
If you're allowed to use something other than bash/sh, AND this is just for a fancy "mv"... you might try the venerable "rename.pl" script. I use it on Linux and cygwin on windows all the time.
http://people.sc.fsu.edu/~jburkardt/pl_src/rename/rename.html
rename.pl 's/^(.*?)\.(.*)$/\1-new_stuff_here.\2/' list_of_files_or_glob
You can also use a "-p" parameter to rename.pl to have it tell you what it WOULD HAVE DONE, without actually doing it.
I just tried the following in my c:/bin (cygwin/windows environment). I used the "-p" so it spit out what it would have done. This example just splits the base and extension, and adds a string in between them.
perl c:/bin/rename.pl -p 's/^(.*?)\.(.*)$/\1-new_stuff_here.\2/' *.bat
rename "here.bat" => "here-new_stuff_here.bat"
rename "htmldecode.bat" => "htmldecode-new_stuff_here.bat"
rename "htmlencode.bat" => "htmlencode-new_stuff_here.bat"
rename "sdiff.bat" => "sdiff-new_stuff_here.bat"
rename "widvars.bat" => "widvars-new_stuff_here.bat"
the files should be renamed/moved from <name>.txt to /foo/<name>.bar.txt
You can use rename utility, e.g.:
rename s/\.txt$/\.txt\.bar/g *.txt
Hint: The subsitution syntax is similar to sed or vim.
Then move the files to some target directory by using mv:
mkdir /some/path
mv *.bar /some/path
To do rename files into subdirectories based on some part of their name, check for:
-p/--mkpath/--make-dirs Create any non-existent directories in the target path.
Testing:
$ touch {1..5}.txt
$ rename --dry-run "s/.txt$/.txt.bar/g" *.txt
'1.txt' would be renamed to '1.txt.bar'
'2.txt' would be renamed to '2.txt.bar'
'3.txt' would be renamed to '3.txt.bar'
'4.txt' would be renamed to '4.txt.bar'
'5.txt' would be renamed to '5.txt.bar'
Adding on that the wikipedia article is surprisingly informative
for example:
Shell trick
Another way to achieve a similar effect is to use a shell as the launched command, and deal with the complexity in that shell, for example:
$ mkdir ~/backups
$ find /path -type f -name '*~' -print0 | xargs -0 bash -c 'for filename; do cp -a "$filename" ~/backups; done' bash
Inspired by an answer by #justaname above, this command which incorporates Perl one-liner will do it:
find ./ -name \*.txt | perl -p -e 's/^(.*\/(.*)\.txt)$/mv $1 .\/foo\/$2.bar.txt/' | bash
In a directory, I have a bunch of *.html files. I'd like to rename them all to *.txt
How can I do that? I use the bash shell.
If using bash, there's no need for external commands like sed, basename, rename, expr, etc.
for file in *.html
do
mv "$file" "${file%.html}.txt"
done
For an better solution (with only bash functionality, as opposed to external calls), see one of the other answers.
The following would do and does not require the system to have the rename program (although you would most often have this on a system):
for file in *.html; do
mv "$file" "$(basename "$file" .html).txt"
done
EDIT: As pointed out in the comments, this does not work for filenames with spaces in them without proper quoting (now added above). When working purely on your own files that you know do not have spaces in the filenames this will work but whenever you write something that may be reused at a later time, do not skip proper quoting.
rename 's/\.html$/\.txt/' *.html
does exactly what you want.
This worked for me on OSX from .txt to .txt_bak
find . -name '*.txt' -exec sh -c 'mv "$0" "${0%.txt}.txt_bak"' {} \;
You want to use rename :
rename -S <old_extension> <new_extension> <files>
rename -S .html .txt *.html
This does exactly what you want - it will change the extension from .html to .txt for all files matching *.html.
Note: Greg Hewgill correctly points out this is not a bash builtin; and is a separate Linux command. If you just need something on Linux this should work fine; if you need something more cross-platform then take a look at one of the other answers.
On a Mac...
Install rename if you haven't: brew install rename
rename -S .html .txt *.html
For Ubuntu Users :
rename 's/\.html$/\.txt/' *.html
This is the slickest solution I've found that works on OSX and Linux, and it works nicely with git too!
find . -name "*.js" -exec bash -c 'mv "$1" "${1%.js}".tsx' - '{}' \;
and with git:
find . -name "*.js" -exec bash -c 'git mv "$1" "${1%.js}".tsx' - '{}' \;
This question explicitly mentions Bash, but if you happen to have ZSH available it is pretty simple:
zmv '(*).*' '$1.txt'
If you get zsh: command not found: zmv then simply run:
autoload -U zmv
And then try again.
Thanks to this original article for the tip about zmv.
Here is an example of the rename command:
rename -n ’s/\.htm$/\.html/’ *.htm
The -n means that it's a test run and will not actually change any files. It will show you a list of files that would be renamed if you removed the -n. In the case above, it will convert all files in the current directory from a file extension of .htm to .html.
If the output of the above test run looked ok then you could run the final version:
rename -v ’s/\.htm$/\.html/’ *.htm
The -v is optional, but it's a good idea to include it because it is the only record you will have of changes that were made by the rename command as shown in the sample output below:
$ rename -v 's/\.htm$/\.html/' *.htm
3.htm renamed as 3.html
4.htm renamed as 4.html
5.htm renamed as 5.html
The tricky part in the middle is a Perl substitution with regular expressions, highlighted below:
rename -v ’s/\.htm$/\.html/’ *.htm
One line, no loops:
ls -1 | xargs -L 1 -I {} bash -c 'mv $1 "${1%.*}.txt"' _ {}
Example:
$ ls
60acbc4d-3a75-4090-85ad-b7d027df8145.json ac8453e2-0d82-4d43-b80e-205edb754700.json
$ ls -1 | xargs -L 1 -I {} bash -c 'mv $1 "${1%.*}.txt"' _ {}
$ ls
60acbc4d-3a75-4090-85ad-b7d027df8145.txt ac8453e2-0d82-4d43-b80e-205edb754700.txt
The command mmv seems to do this task very efficiently on a huge number of files (tens of thousands in a second). For example, to rename all .xml files to .html files, use this:
mmv ";*.xml" "#1#2.html"
the ; will match the path, the * will match the filename, and these are referred to as #1 and #2 in the replacement name.
Answers based on exec or pipes were either too slow or failed on a very large number of files.
In Linux or window git bash or window's wsl, try below command to change every file's extension in current directory or sub-directories or even their sub-directories with just one line of code
find . -depth -name "*.html" -exec sh -c 'mv "$1" "${1%.html}.txt"' _ {} \;
Try this
rename .html .txt *.html
usage:
rename [find] [replace_with] [criteria]
After someone else's website crawl, I ended up with thousands of files missing the .html extension, across a wide tree of subdirectories.
To rename them all in one shot, except the files already having a .html extension (most of them had none at all), this worked for me:
cd wwwroot
find . -xtype f \! -iname *.html -exec mv -iv "{}" "{}.html" \; # batch rename files to append .html suffix IF MISSING
In the OP's case I might modify that slightly, to only rename *.txt files, like so:
find . -xtype f -iname *.txt -exec filename="{}" mv -iv ${filename%.*}.{txt,html} \;
Broken down (hammertime!):
-iname *.txt
- Means consider ONLY files already ending in .txt
mv -iv "{}.{txt,html}"
- When find passes a {} as the filename, ${filename%.*} extracts its basename without any extension to form the parameters to mv. bash takes the {txt,html} to rewrite it as two parameters so the final command runs as: mv -iv "filename.txt" "filename.html"
Fix needed though: dealing with spaces in filenames
This is a good way to modify multiple extensions at once:
for fname in *.{mp4,avi}
do
mv -v "$fname" "${fname%.???}.mkv"
done
Note: be careful at the extension size to be the same (the ???)
Rename file extensions for all files under current directory and sub directories without any other packages (only use shell script):
Create a shell script rename.sh under current directory with the following code:
#!/bin/bash
for file in $(find . -name "*$1"); do
mv "$file" "${file%$1}$2"
done
Run it by ./rename.sh .old .new.
Eg. ./rename.sh .html .txt
A bit late to the party. You could do it with xargs:
ls *.html | xargs -I {} sh -c 'mv $1 `basename $1 .html`.txt' - {}
Or if all your files are in some folder
ls folder/*.html | xargs -I {} sh -c 'mv $1 folder/`basename $1 .html`.txt' - {}
Similarly to what was suggested before, this is how I did it:
find . -name '*OldText*' -exec sh -c 'mv "$0" "${0/OldText/NewText}"' {} \;
I first validated with
find . -name '*OldText*' -exec sh -c 'echo mv "$0" "${0/OldText/NewText}"' {} \;
Nice & simple!
find . -iname *.html -exec mv {} "$(basename {} .html).text" \;
If you prefer PERL, there is a short PERL script (originally written by Larry Wall, the creator of PERL) that will do exactly what you want here:
tips.webdesign10.com/files/rename.pl.txt.
For your example the following should do the trick:
rename.pl 's/html/txt/' *.html
The easiest way is to use rename.ul it is present in most of the Linux distro
rename.ul -o -v [oldFileExtension] [newFileExtension] [expression to search for file to be applied with]
rename.ul -o -v .oldext .newext *.oldext
Options:
-o: don't overwrite preexisting .newext
-v: verbose
-n: dry run
Unfortunately it's not trivial to do portably. You probably need a bit of expr magic.
for file in *.html; do echo mv -- "$file" "$(expr "$file" : '\(.*\)\.html').txt"; done
Remove the echo once you're happy it does what you want.
Edit: basename is probably a little more readable for this particular case, although expr is more flexible in general.
Here is what i used to rename .edge files to .blade.php
for file in *.edge; do mv "$file" "$(basename "$file" .edge).blade.php"; done
Works like charm.
You can also make a function in Bash, add it to .bashrc or something and then use it wherever you want.
change-ext() {
for file in *.$1; do mv "$file" "$(basename "$file" .$1).$2"; done
}
Usage:
change-ext css scss
Source of code in function: https://stackoverflow.com/a/1224786/6732111
Here is a solution, using AWK. Make sure the files are present in the working directory. Else, cd to the directory where the html files are located and then execute the below command:
for i in $(ls | grep .html); do j=$(echo $i | grep -oh "^\w*." | awk '{print $1"txt"}'); mv $i $j; done
I wrote this code in my .bashrc
alias find-ext='read -p "Path (dot for current): " p_path; read -p "Ext (unpunctured): " p_ext1; find $p_path -type f -name "*."$p_ext1'
alias rename-ext='read -p "Path (dot for current): " p_path; read -p "Ext (unpunctured): " p_ext1; read -p "Change by ext. (unpunctured): " p_ext2; echo -en "\nFound files:\n"; find $p_path -type f -name "*.$p_ext1"; find $p_path -type f -name "*.$p_ext1" -exec sh -c '\''mv "$1" "${1%.'\''$p_ext1'\''}.'\''$p_ext2'\''" '\'' _ {} \;; echo -en "\nChanged Files:\n"; find $p_path -type f -name "*.$p_ext2";'
In a folder like "/home/<user>/example-files" having this structure:
/home/<user>/example-files:
file1.txt
file2.txt
file3.pdf
file4.csv
The commands would behave like this:
~$ find-text
Path (dot for current): example-files/
Ext (unpunctured): txt
example-files/file1.txt
example-files/file2.txt
~$ rename-text
Path (dot for current): ./example-files
Ext (unpunctured): txt
Change by ext. (unpunctured): mp3
Found files:
./example-files/file1.txt
./example-files/file1.txt
Changed Files:
./example-files/file1.mp3
./example-files/file1.mp3
~$
You could use a tool designed for renaming files in bulk, e.g. renamer.
To rename all file extensions in the current folder:
$ renamer --find ".html" --replace ".txt" --dry-run *
Many more usage examples here.