I have a large directory of folders, each of which has only one file:
directory/folder1/208hasdfasdf.jpg
directory/folder2/f230fsdf.gif
directory/folder3/23fsdbfasf.jpg
I'd like to rename this to:
directory2/folder1/folder1.jpg
directory2/folder2/folder2.gif
directory3/folder3/folder3.jpg
How can I do that?
For the path and filenames shown, you can use a loop and combination of find and sed to make the substitutions, e.g.
for f in $(find directory -type f -wholename "*folder*"); do
mv "$f" $(sed -E 's|^([^/]+)/([^/]+)/([^.]+)[.](.*)$|\1/\2/\2.\4|' <<< "$f")
done
Where sed -E 's|^([^/]+)/([^/]+)/([^.]+)[.](.*)$|\1/\2/\2.\4| uses the alternative delimiter '|' instead of '/' to ease dealing with pathnames, and then separates and captures the "directory" with ^([^/]+) and then the "folderX" with ([^/]+), followed by the filename without the extension ([^.]+) and lastly the extension (.*)$ making each component available through the numbered backreferences \1, \2, \3, and \4, respectively.
Then to form the new filename, you just duplicate the \2 foldername in place of the \3 filename, for a new filename of \1/\2/\2.\4
Example Use/Output
$ find tmp-david -type f -wholename "*folder*"
tmp-david/folder3/23fsdbfasf.jpg
tmp-david/folder2/f230fsdf.gif
tmp-david/folder1/208hasdfasdf.jpg
And the replacement of the filenames with
$ for f in $(find tmp-david -type f -wholename "*folder*"); do
> mv "$f" $(sed -E 's|^([^/]+)/([^/]+)/([^.]+)[.](.*)$|\1/\2/\2.\4|' <<< "$f")
> done
Resulting in:
$ find tmp-david -type f -wholename "*folder*"
tmp-david/folder3/folder3.jpg
tmp-david/folder2/folder2.gif
tmp-david/folder1/folder1.jpg
You could try something like this, assuming you're using bash:
find directory/ \( -name '*.gif' -o -name '*.jpg' \) -print |
while read old; do
parent=${old%/*}
base=${parent##*/}
ext=${old##*.}
mv $old $parent/$base.$ext
done
If you're dealing with filenames that contain whitespace you're going
to need to massage this a bit.
Before running this script:
$ find directory -type f -print
directory/folder2/f230fsdf.gif
directory/folder1/208hasdfasdf.jpg
directory/folder3/23fsdbfasf.jpg
After running this script:
$ find directory -type f -print
directory/folder2/folder2.gif
directory/folder1/folder1.jpg
directory/folder3/folder3.jpg
Here is my current code, my goal is to find every file in a given directory (recursively) and replace "FIND" with "REPLACEWITH" and overwrite the files.
FIND='ALEX'
REPLACEWITH='<strong>ALEX</strong>'
DIRECTORY='/some/directory/'
find $DIRECTORY -type f -name "*.html" -print0 |
LANG=C xargs -0 sed -i "s|$FIND|$REPLACEWITH|g"
The error I am getting is:
sed: 1: "/some/directory ...": command a expects \ followed by text
As given in BashFAQ #21, you can use perl to perform search-and-replace operations with no potential for data being treated as code:
in="$FIND" out="$REPLACEWITH" find "$DIRECTORY" -type f -name '*.html' \
-exec perl -pi -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' '{}' +
If you want to include only files matching the FIND string, find can be told to only pass files which grep flags on to perl:
in="$FIND" out="$REPLACEWITH" find "$DIRECTORY" -type f -name '*.html' \
-exec grep -F -q -e "$FIND" '{}' ';' \
-exec perl -pi -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' '{}' +
Because grep is being used to evaluate individual files, it's necessary to use one grep call per file so its exit status can be evaluated on a per-file basis; thus, the use of the less efficient -exec ... {} ';' action. For perl, it's possible to put multiple files to process on one command, hence the use of -exec ... {} +.
Note that fgrep is line-oriented; if your FIND string contains multiple lines, then files with any one of those lines will be passed to perl for replacements.
You can have find invoke sed directly although I think all the modification times on your files will be affected (which might matter or not):
find $DIRECTORY -type f -name "*.html" -exec sed -i "s|$FIND|$REPLACEWITH|g" '{}' ';'
I am trying to loop my project folder recursively, grep every PHP file and find any string that matches xdebug_break();.
Then I want to remove that xdebug_break() (I will accept replacing with a space as well).
Here's what I got so far:
#!/bin/bash
FILES=$(find ../../Dev/projects/api -type f -name *.php)
for f in $FILES
do
if grep -nr "xdebug_break();" $f
then
sed -e '/xdebug_break();/d' -i $f
echo "xdebug_break(); has been deleted."
fi
done
Everything works, except the replace part. I keep getting this error:
sed: -i may not be used with stdin
I do not care if its sed or awk or whatever (but I do use a mac).
Thanks,
SOLUTION (FOR FUTURE READERS)
Thanks for the help everyone (ESP #anubhava). This one line trick did it for me:
find ../../Dev/projects/api -type f -name "*.php" -exec sed -i '' '/xdebug_break();/d' {} +
Also you can do it by loop (if you really really want to) like this:
#!/bin/bash
FILES=$(find ../../Dev/projects/api -type f -name *.php)
for f in $FILES
do
if grep -nr "xdebug_break();" $f
then
sed -i '' '/xdebug_break();/d' "$f"
echo "xdebug_break(); has been deleted."
fi
done
On OSX, your sed command should be:
sed -i.bak '/xdebug_break();/d' "$f"
Here .bak is the name of extension to create a backup of input file in inline editing.
You can avoid loop and do it in one find like this:
find ../../Dev/projects/api -type f -name "*.php" \
-exec sed -i.bak '/xdebug_break();/d' {} +
You can combine find and sed:
find Dev/projects/api -type f -name '*.php' -exec sed -i.bak '/xdebug_break();/d' {} \;
I would like to replace the part of each file path, which will be find by find linux command.
My approach is attached below:
find . -type f -name "*.txt" -exec echo {} | sed "s/f/u/g" {} \;
I expect the replacement of each letter "f" with "u" in file path. Unfortunately I got this error:
find: missing argument to `-exec'
sed: can't read {}: No such file or directory
sed: can't read ;: No such file or directory
What I did wrong? Thank you for your help.
I would like to replace the part of each file path
If you want to change just the file names/paths then use:
find . -type f -name "*.txt" -exec bash -c 'echo "$1" | sed "s/f/u/g"' - {} \;
or a bit more efficient with xargs (since it avoids spawning subshell for each found file):
find . -type f -name "*.txt" -print0 |
xargs -0 bash -c 'for f; do sed "s/f/u/g" <<< "$f"; done'
find . -type f -name "*.txt" | while read files
do
newname=$(echo "${files}" | sed s"#f#u#"g)
mv -v "${files}" "${newname}"
done
I don't completely understand what you meant by file path. If you weren't talking about the file name, please clarify further.
How can you remove all of the trailing whitespace of an entire project? Starting at a root directory, and removing the trailing whitespace from all files in all folders.
Also, I want to to be able to modify the file directly, and not just print everything to stdout.
Here is an OS X >= 10.6 Snow Leopard solution.
It Ignores .git and .svn folders and their contents. Also it won't leave a backup file.
(export LANG=C LC_CTYPE=C
find . -not \( -name .svn -prune -o -name .git -prune \) -type f -print0 | perl -0ne 'print if -T' | xargs -0 sed -Ei 's/[[:blank:]]+$//'
)
The enclosing parenthesis preserves the L* variables of current shell – executing in subshell.
Use:
find . -type f -print0 | xargs -0 perl -pi.bak -e 's/ +$//'
if you don't want the ".bak" files generated:
find . -type f -print0 | xargs -0 perl -pi -e 's/ +$//'
as a zsh user, you can omit the call to find, and instead use:
perl -pi -e 's/ +$//' **/*
Note: To prevent destroying .git directory, try adding: -not -iwholename '*.git*'.
Two alternative approaches which also work with DOS newlines (CR/LF) and do a pretty good job at avoiding binary files:
Generic solution which checks that the MIME type starts with text/:
while IFS= read -r -d '' -u 9
do
if [[ "$(file -bs --mime-type -- "$REPLY")" = text/* ]]
then
sed -i 's/[ \t]\+\(\r\?\)$/\1/' -- "$REPLY"
else
echo "Skipping $REPLY" >&2
fi
done 9< <(find . -type f -print0)
Git repository-specific solution by Mat which uses the -I option of git grep to skip files which Git considers to be binary:
git grep -I --name-only -z -e '' | xargs -0 sed -i 's/[ \t]\+\(\r\?\)$/\1/'
In Bash:
find dir -type f -exec sed -i 's/ *$//' '{}' ';'
Note: If you're using .git repository, try adding: -not -iwholename '.git'.
Ack was made for this kind of task.
It works just like grep, but knows not to descend into places like .svn, .git, .cvs, etc.
ack --print0 -l '[ \t]+$' | xargs -0 -n1 perl -pi -e 's/[ \t]+$//'
Much easier than jumping through hoops with find/grep.
Ack is available via most package managers (as either ack or ack-grep).
It's just a Perl program, so it's also available in a single-file version that you can just download and run. See: Ack Install
This worked for me in OSX 10.5 Leopard, which does not use GNU sed or xargs.
find dir -type f -print0 | xargs -0 sed -i.bak -E "s/[[:space:]]*$//"
Just be careful with this if you have files that need to be excluded (I did)!
You can use -prune to ignore certain directories or files. For Python files in a git repository, you could use something like:
find dir -not -path '.git' -iname '*.py'
ex
Try using Ex editor (part of Vim):
$ ex +'bufdo!%s/\s\+$//e' -cxa **/*.*
Note: For recursion (bash4 & zsh), we use a new globbing option (**/*.*). Enable by shopt -s globstar.
You may add the following function into your .bash_profile:
# Strip trailing whitespaces.
# Usage: trim *.*
# See: https://stackoverflow.com/q/10711051/55075
trim() {
ex +'bufdo!%s/\s\+$//e' -cxa $*
}
sed
For using sed, check: How to remove trailing whitespaces with sed?
find
Find the following script (e.g. remove_trail_spaces.sh) for removing trailing whitespaces from the files:
#!/bin/sh
# Script to remove trailing whitespace of all files recursively
# See: https://stackoverflow.com/questions/149057/how-to-remove-trailing-whitespace-of-all-files-recursively
case "$OSTYPE" in
darwin*) # OSX 10.5 Leopard, which does not use GNU sed or xargs.
find . -type f -not -iwholename '*.git*' -print0 | xargs -0 sed -i .bak -E "s/[[:space:]]*$//"
find . -type f -name \*.bak -print0 | xargs -0 rm -v
;;
*)
find . -type f -not -iwholename '*.git*' -print0 | xargs -0 perl -pi -e 's/ +$//'
esac
Run this script from the directory which you want to scan. On OSX at the end, it will remove all the files ending with .bak.
Or just:
find . -type f -name "*.java" -exec perl -p -i -e "s/[ \t]$//g" {} \;
which is recommended way by Spring Framework Code Style.
I ended up not using find and not creating backup files.
sed -i '' 's/[[:space:]]*$//g' **/*.*
Depending on the depth of the file tree, this (shorter version) may be sufficient for your needs.
NOTE this also takes binary files, for instance.
Instead of excluding files, here is a variation of the above the explicitly white lists the files, based on file extension, that you want to strip, feel free to season to taste:
find . \( -name *.rb -or -name *.html -or -name *.js -or -name *.coffee -or \
-name *.css -or -name *.scss -or -name *.erb -or -name *.yml -or -name *.ru \) \
-print0 | xargs -0 sed -i '' -E "s/[[:space:]]*$//"
I ended up running this, which is a mix between pojo and adams version.
It will clean both trailing whitespace, and also another form of trailing whitespace, the carriage return:
find . -not \( -name .svn -prune -o -name .git -prune \) -type f \
-exec sed -i 's/[:space:]+$//' \{} \; \
-exec sed -i 's/\r\n$/\n/' \{} \;
It won't touch the .git folder if there is one.
Edit: Made it a bit safer after the comment, not allowing to take files with ".git" or ".svn" in it. But beware, it will touch binary files if you've got some. Use -iname "*.py" -or -iname "*.php" after -type f if you only want it to touch e.g. .py and .php-files.
Update 2: It now replaces all kinds of spaces at end of line (which means tabs as well)
This works well.. add/remove --include for specific file types :
egrep -rl ' $' --include *.c * | xargs sed -i 's/\s\+$//g'
Ruby:
irb
Dir['lib/**/*.rb'].each{|f| x = File.read(f); File.write(f, x.gsub(/[ \t]+$/,"")) }
1) Many other answers use -E. I am not sure why, as that's undocumented BSD compatibility option. -r should be used instead.
2) Other answers use -i ''. That should be just -i (or -i'' if preffered), because -i has the suffix right after.
3) Git specific solution:
git config --global alias.check-whitespace \
'git diff-tree --check $(git hash-object -t tree /dev/null) HEAD'
git check-whitespace | grep trailing | cut -d: -f1 | uniq -u -z | xargs -0 sed --in-place -e 's/[ \t]+$//'
The first one registers a git alias check-whitespace which lists the files with trailing whitespaces.
The second one runs sed on them.
I only use \t rather than [:space:] as I don't typically see vertical tabs, form feeds and non-breakable spaces. Your measurement may vary.
I use regular expressions. 4 steps:
Open the root folder in your editor (I use Visual Studio Code).
Tap the Search icon on the left, and enable the regular expression mode.
Enter " +\n" in the Search bar and "\n" in the Replace bar.
Click "Replace All".
This removes all trailing spaces at the end of each line in all files. And you can exclude some files that don't fit with this need.
This is what works for me (Mac OS X 10.8, GNU sed installed by Homebrew):
find . -path ./vendor -prune -o \
\( -name '*.java' -o -name '*.xml' -o -name '*.css' \) \
-exec gsed -i -E 's/\t/ /' \{} \; \
-exec gsed -i -E 's/[[:space:]]*$//' \{} \; \
-exec gsed -i -E 's/\r\n/\n/' \{} \;
Removed trailing spaces, replaces tabs with spaces, replaces Windows CRLF with Unix \n.
What's interesting is that I have to run this 3-4 times before all files get fixed, by all cleaning gsed instructions.