A bunch of Word & Excel documents were being moved on the server when the process terminated before it was complete. As a result, we're left with several perfectly fine files that have a .tmp extension, and we need to rename these files back to the appropriate .xlsx or .docx extension.
Here's my current code to do this in Bash:
#!/bin/sh
for i in "$(find . -type f -name *.tmp)"; do
ft="$(file "$i")"
case "$(file "$i")" in
"$i: Microsoft Word 2007+")
mv "$i" "${i%.tmp}.docx"
;;
"$i: Microsoft Excel 2007+")
mv "$i" "${i%.tmp}.xlsx"
;;
esac
done
It seems that while this does search recursively, it only does 1 file. If it finds an initial match, it doesn't go on to rename the rest of the files. How can I get this to loop correctly through the directories recursively without it doing just 1 file at a time?
Try find command like this:
while IFS= read -r -d '' i; do
ft="$(file "$i")"
case "$ft" in
"$i: Microsoft Word 2007+")
mv "$i" "${i%.tmp}.docx"
;;
"$i: Microsoft Excel 2007+")
mv "$i" "${i%.tmp}.xlsx"
;;
esac
done < <(find . -type f -name '*.tmp' -print0)
Using <(...) is called process substitution to run find command here
Quote filename pattern in find
Use -print0 to get find output delimited by a null character to allow space/newline characters in file names
Use IFS= and -d '' to read null separated filenames
I too would recommend using find. I would do this in two passes of find:
find . -type f -name \*.tmp \
-exec sh -c 'file "{}" | grep -q "Microsoft Word 2007"' \; \
-exec sh -c 'f="{}"; echo mv "$f" "${f%.tmp}.docx"' \;
find . -type f -name \*.tmp \
-exec sh -c 'file "{}" | grep -q "Microsoft Excel 2007"' \; \
-exec sh -c 'f="{}"; echo mv "$f" "${f%.tmp}.xlsx"' \;
Lines are split for readability.
Each instance of find will search for tmp files, then use -exec to test the output of find. This is similar to how you're doing it within the while loop in your shell script, only it's launched from within find itself. We're using the pipe to grep instead of your case statement.
The second -exec only gets run if the first one returned "true" (i.e. grep -q ... found something), and executes the rename in a tiny shell instance.
I haven't profiled this to see whether it would be faster or slower than a loop in a shell script. Just another way to handle things.
Related
Update 01/12/2022
With triplee's helpful suggestions, I resolved it to take both files & directories by adding a comma in between f and d, the final code now looks like this:
while read -r old new;
do echo "replacing ${old} by ${new}" >&2
find '/path/to/dir' -depth -type d,f -name "$old" -exec rename
"s/${old}/${new}/" {} ';'
done <input.txt
Thank you!
Original request:
I am trying to rename a list of files (from $old to $new), all present in $homedir or in subdirectories in $homedir.
In the command line this line works to rename files in the subfolders:
find ${homedir}/ -name ${old} -exec rename "s/${old}/${new}/" */${old} ';'
However, when I want to implement this line in a simple bash script getting the $old and $new filenames from input.txt, it doesn't work anymore...
input.txt looks like this:
name_old name_new
name_old2 name_new2
etc...
the script looks like this:
#!/bin/bash
homedir='/path/to/dir'
cat input.txt | while read old new;
do
echo 'replacing' ${old} 'by' ${new}
find ${homedir}/ -name ${old} -exec rename "s/${old}/${new}/" */${old} ';'
done
After running the script, the text line from echo with $old and $new filenames being replaced is printed for the entire loop, but no files are renamed. No error is printed either. What am I missing? Your help would be greatly appreaciated!
I checked whether the $old and $new variables were correctly passed to the find -exec rename command, but because they are printed by echo that doesn't seem to be the issue.
If you add an echo, like -exec echo rename ..., you'll see what actually gets executed. I'd say that both the path to $old is wrong (you're not using the result of find in the -exec clause), and */$old isn't quoted and might be expanded by the shell before find ever gets to see it.
You're also having most other expansions unquoted, which can lead to all sorts of trouble.
You could do it in pure Bash (drop echo when output looks good):
shopt -s globstar
for f in **/"$old"; do echo mv "$f" "${f/%*/$new}"; done
Or with rename directly, though this would run into trouble if too many files match (drop -n when output looks good):
rename -n "s/$old\$/$new/" **/"$old"
Or with GNU find, using -execdir to run in the same directory as the matching file (drop echo when output looks good):
find -type f -name "$old" -execdir echo mv "$old" "$new" \;
And finally, a version with find that spawns just a single subshell (drop echo when output looks right):
find -type f -name "$old" -exec bash -c '
new=$1
shift
for f; do
echo mv "$f" "${f/%*/$new}"
done
' bash "$new" {} +
The argument to rename should be the file itself, not */${old}. You also have a number of quoting errors, and a useless cat).
#!/bin/bash
while read -r old new;
do
echo "replacing ${old} by ${new}" >&2
find /path/to/dir -name "$old" -exec rename "s/${old}/${new}/" {} ';'
done <input.txt
Running find multiple times on the same directory is hugely inefficient, though. Probably a better solution is to find all files in one go, and abort if it's not one of the files on the list.
find /path/to/dir -type f -exec sh -c '
for f in "$#"; do
awk -v f="$f" "f==\$1 { print \"s/\" \$1 \"/\" \$2 \"/\" }" "$0" |
xargs -I _ -r rename _ "$f"
done' input.txt {} +
(Untested; probably try with echo before you run this live.)
I want to rename a file present in several subdirectories using bash script.
my files are in folders:
./FolderA/ABCD/ABCD_Something.ctl
./FolderA/EFGH/EFGH_Something.ctl
./FolderA/WXYZ/WXYZ_Something.ctl
I want to rename all of the .ctl file with the same name (name.ctl).
I tried several command using mv or rename but didnt work.
Working from FolderA:
find . -name '*.ctl' -exec rename *.ctl name.ctl '{}' \;
or
for f in ./*/*.ctl; do mv "$f" "${f/*.ctl/name .ctl}"; done
or
for f in $(find . -type f -name '*.ctl'); do mv $f $(echo "$f" | sed 's/*.ctl/name.ctl/'); done
Can you help me using bash?
thanks
You can do this with one line with:
find . -name *.ctl -exec sh -c 'mv "$1" `dirname "$1"`/name.ctl' x {} \;
The x just allows the filename to be positional character 1 rather than 0 which (in my opinion) wrong to use as a parameter.
Try this:
find . -name '*.ctl' | while read f; do
dn=$(dirname "${f}")
# remove the echo after you sanity check the output
echo mv "${f}" "${dn}/name.ctl"
done
find should get all the files you want, dirname will get just the directory name, and mv will perform the rename. You can remove the quotes if you're sure that you'll never have spaces in the names.
for subj in `cat dti_list.txt`; do
echo $subj
find . -type f -iname '*306.nii' -execdir bash -c 'rename.ul "$subj" DTI_MAIN_AP.nii *.nii' \+
done
I have some trouble with a small bash script, which adds the name instead of replacing when I use the rename.ul function.
Currently, the code adds DTI_MAIN_AP.nii in front of the old name.
My goal is to replace the name from the subj list and using the find to search up any directory with a *306.nii file, and then using execdir to execute the rename.ul function to rename the file from the dti_list.txt.
Any solution, or correction to get the code working, will be appreciated.
If you just want to rename the first file matching *306.nii in each directory to DTI_MAIN_AP.nii, that might look like:
find . -type f -iname '*306.nii' \
-execdir sh -c '[[ -e DTI_MAIN_AP.nii ]] || mv "$1" DTI_MAIN_AP.nii' _ {} +
If instead of matching on *306.nii you want to iterate over names from dti_list.txt, that might instead look like:
while IFS= read -r -d '' filename <&3; do
find . -type f -name "$filename" \
-execdir sh -c '[[ -e DTI_MAIN_AP.nii ]] || mv "$1" DTI_MAIN_AP.nii' _ {} +
done <dti_list.txt
References of note:
BashFAQ #1 (on reading files line-by-line)
Using Find
I'd like to run the following bash command for every file in a folder (outputting a unique JSON file for each processed .csv), via a Makefile:
csvtojson ./file/path.csv > ./file/path.json
Here's what I've managed, I'm struggling with the stdin/out syntax and arguments:
find ./ -type f -name "*.csv" -exec csvtojson {} > {}.json \;
Help much appreciated!
You're only passing a single argument to csvtojson -- the filename to run.
The > outputfile isn't an argument at all; instead, it's an instruction to the shell that parses and invokes the relevant command to connect the command's stdout to the given filename before actually starting that command.
Thus, above, that redirection is parsed before the find command is run -- because that's the only place a shell is involved at all.
If you want to involve a shell, consider doing so as follows:
find ./ -type f -name "*.csv" \
-exec sh -c 'for arg; do csvtojson "$arg" >"${arg}.json"; done' _ {} +
...or, as follows:
find ./ -type f -name '*.csv' -print0 |
while IFS= read -r -d '' filename; do
csvtojson "$filename" >"$filename.json"
done
...or, if you want to be able to set shell variables inside the loop and have them persist after its exit, you can use a process substitution to avoid the issues described in BashFAQ #24:
bad=0
good=0
while IFS= read -r -d '' filename; do
if csvtojson "$filename" >"$filename.json"; then
(( ++good ))
else
(( ++bad ))
fi
done < <(find ./ -type f -name '*.csv' -print0)
echo "Converting CSV files to JSON: ${bad} failures, ${good} successes" >&2
See UsingFind, particularly the Complex Actions section and the section on Actions In Bulk.
I have a directory named as assets, which in further has a set of directories.
user_images
|-content_images
|-original
|-cropped
|-resize
|-gallery_images
|-slider_images
|-logo
These folders can have folders like original, cropped, resize. And these folders further will have images. These images are named something like this – 14562345+Image.jpeg. I need to replace all the images/files that have + to _.
for f in ls -a;
do
if [[ $f == + ]]
then
cp "$f" "${f//+/_}"
fi
done
I was able to do this in the current directory. But I need to iterate this to other many other directories. How can I do that?
You can use this loop using find in a process substitution:
cd user_images
while IFS= read -r -d '' f; do
echo "$f"
mv "$f" "${f//+/_}"
done < <(find . -name '*+*' -type f -print0)
With find -exec:
find user_images -type f \
-exec bash -c '[[ $0 == *+* ]] && mv "$0" "${0//+/_}"' {} \;
Notice that this uses mv and not cp as the question states "rename", but simply replace by cp if you want to keep the original files.
The bash -c is required to be able to manipulate the file names, otherwise we could use {} directly in the -exec action.
The following will run recursively and will rename all files replacing + with a _ :
find . -name '*+*' -type f -execdir bash -c 'for f; do mv "$f" "${f//+/_}"' _ {} +
Notice the use of -execdir :
Like -exec, but the specified command is run from the subdirectory containing the matched file, which is not normally the directory in which you started find -- Quoted from man find.
Which will protect us in case of directory names matching the pattern *+* which you do not want to rename.