I'm trying to find a bash script that will recursively look for files with a .bx extension, and remove this extension. The filenames are in no particular format (some are hidden files with "." prefix, some have spaces in the name, etc.), and not all files have this extension.
I'm not sure how to find each file with the .bx extension (in and below my cwd) and remove it. Thanks for the help!
find . -name '*.bx' -type f | while read NAME ; do mv "${NAME}" "${NAME%.bx}" ; done
find -name "*.bx" -print0 | xargs -0 rename 's/\.bx//'
Bash 4+
shopt -s globstar
shopt -s nullglob
shopt -s dotglob
for file in **/*.bx
do
mv "$file" "${file%.bx}"
done
Assuming you are in the folder from where you want to do this
find . -name "*.bx" -print0 | xargs -0 rename .bx ""
for blah in *.bx ; do mv ${blah} ${blah%%.bx}
Here is another version which does the following:
Finds out files based on $old_ext variable (right now set to .bx) in and below cwd, stores them in $files
Replaces those files' extension to nothing (or something new depending on $new_ext variable, currently set to .xyz)
The script uses dirname and basename to find out file-path and file-name respectively.
#!/bin/bash
old_ext=".bx"
new_ext=".xyz"
files=$(find ./ -name "*${old_ext}")
for file in $files
do
file_name=$(basename $file $old_ext)
file_path=$(dirname $file)
new_file=${file_path}/${file_name}${new_ext}
#echo "$file --> $new_file"
mv "$file" "$new_file"
done
Extra: How to remove any extension from filenames
find -maxdepth 1 -type f | sed 's/.\///g'| grep -E [.] | while read file; do mv $file ${file%.*}; done
will cut starting from last dot, i.e. pet.cat.dog ---> pet.cat
find -maxdepth 1 -type f | sed 's/.\///g'| grep -E [.] | while read file; do mv $file ${file%%.*}; done
will cut starting from first dot, i.e. pet.cat.dog ---> pet
"-maxdepth 1" limits operation to current directory, "-type f" is used to select files only. Sed & grep combination is used to pick only filenames with dot. Number of percent signs in "mv" command will define actual cut point.
Related
I have a lot of files that are named as: MM-DD-YYYY.pdf. I want to rename them as YYYY-MM-DD.pdf I’m sure there is some bash magic to do this. What is it?
For files in the current directory:
for name in ./??-??-????.pdf; do
if [[ "$name" =~ (.*)/([0-9]{2})-([0-9]{2})-([0-9]{4})\.pdf ]]; then
echo mv "$name" "${BASH_REMATCH[1]}/${BASH_REMATCH[4]}-${BASH_REMATCH[3]}-${BASH_REMATCH[2]}.pdf"
fi
done
Recursively, in or under the current directory:
find . -type f -name '??-??-????.pdf' -exec bash -c '
for name do
if [[ "$name" =~ (.*)/([0-9]{2})-([0-9]{2})-([0-9]{4})\.pdf ]]; then
echo mv "$name" "${BASH_REMATCH[1]}/${BASH_REMATCH[4]}-${BASH_REMATCH[3]}-${BASH_REMATCH[2]}.pdf"
fi
done' bash {} +
Enabling the globstar shell option in bash lets us do the following (will also, like the above solution, handle all files in or below the current directory):
shopt -s globstar
for name in **/??-??-????.pdf; do
if [[ "$name" =~ (.*)/([0-9]{2})-([0-9]{2})-([0-9]{4})\.pdf ]]; then
echo mv "$name" "${BASH_REMATCH[1]}/${BASH_REMATCH[4]}-${BASH_REMATCH[3]}-${BASH_REMATCH[2]}.pdf"
fi
done
All three of these solutions uses a regular expression to pick out the relevant parts of the filenames, and then rearranges these parts into the new name. The only difference between them is how the list of pathnames is generated.
The code prefixes mv with echo for safety. To actually rename files, remove the echo (but run at least once with echo to see that it does what you want).
A direct approach example from the command line:
$ ls
10-01-2018.pdf 11-01-2018.pdf 12-01-2018.pdf
$ ls [0-9]*-[0-9]*-[0-9]*.pdf|sed -r 'p;s/([0-9]{2})-([0-9]{2})-([0-9]{4})/\3-\1-\2/'|xargs -n2 mv
$ ls
2018-10-01.pdf 2018-11-01.pdf 2018-12-01.pdf
The ls output is piped to sed , then we use the p flag to print the argument without modifications, in other words, the original name of the file, and s to perform and output the conversion.
The ls + sed result is a combined output that consist of a sequence of old_file_name and new_file_name.
Finally we pipe the resulting feed through xargs to get the effective rename of the files.
From xargs man:
-n number Execute command using as many standard input arguments as possible, up to number arguments maximum.
You can use the following command very close to the one of klashxx:
for f in *.pdf; do echo "$f"; mv "$f" "$(echo "$f" | sed 's#\(..\)-\(..\)-\(....\)#\3-\2-\1#')"; done
before:
ls *.pdf
12-01-1998.pdf 12-03-2018.pdf
after:
ls *.pdf
1998-01-12.pdf 2018-03-12.pdf
Also if you have other pdf files that does not respect this format in your folder, what you can do is to select only the files that respect the format: MM-DD-YYYY.pdf to do so use the following command:
for f in `find . -maxdepth 1 -type f -regextype sed -regex './[0-9]\{2\}-[0-9]\{2\}-[0-9]\{4\}.pdf' | xargs -n1 basename`; do echo "$f"; mv "$f" "$(echo "$f" | sed 's#\(..\)-\(..\)-\(....\)#\3-\2-\1#')"; done
Explanations:
find . -maxdepth 1 -type f -regextype sed -regex './[0-9]\{2\}-[0-9]\{2\}-[0-9]\{4\}.pdf this find command will look only for files in the current working directory that respect your syntax and extract their basename (remove the ./ at the beginning, folders and other type of files that would have the same name are not taken into account, other *.pdf files are also ignored.
for each file you do a move and the resulting file name is computed using sed and back reference to the 3 groups for MM,DD and YYYY
For these simple filenames, using a more verbose pattern, you can simplify the body of the loop a bit:
twodigit=[[:digit:]][[:digit:]]
fourdigit="$twodigit$twodigit"
for f in $twodigit-$twodigit-$fourdigit.pdf; do
IFS=- read month day year <<< "${f%.pdf}"
mv "$f" "$year-$month-$day.pdf"
done
This is basically #Kusalananda's answer, but without the verbosity of regular-expression matching.
I have a folder with lots of images. In this folder are subfolders containing high resolution images. Images can be .png, .jpg or .gif.
Some images are duplicates called a.jpg and a.hi.jpg or a.b.c.gif and a.b.c.hi.gif. File names are always different, the will be never a.gif, a.jpg or a.png. I guess i have not to take care of extension.
These are the same images with different resolution.
Now i want to write a script to delete all lower resolution images. But there are files that do not have high resolution like b.png. So i want to delete only if there is a high resolution image too.
I guess i have to do something like this, but can't figure out how exactly.
find . -type f -name "*" if {FILENAME%hi*} =2 --delete smallest else keep file
Could anyone help? Thanks
Something like the following could do the job:
#!/bin/bash
while IFS= read -r -d '' hi
do
d=$(dirname "$hi")
b=$(basename "$hi")
low="${b//.hi./}"
[[ -e "$d/$low" ]] && echo rm -- "$d/$low" #dry run - if satisfied, remove the echo
done < <(find /some/path -type f -name \*.hi.\* -print0)
how it works:
finds all files with .hi. in their names. (not only images, you can extend the find be more restrictive
for all found images
get the directory, where is he
and get the name of the file (without directory)
in the name, remove all occurences of the string .hi. (aka make the "lowres" name
check the existence of the lowres image
delete if exists.
You can use bash extended glob features for this, which you can enable first by
shopt -s extglob
and using the pattern
!(pattern-list)
Matches anything except one of the given patterns.
Now to store the files not containing the string hi
shopt -s extglob
fileList=()
fileList+=( !(*hi*).jpg )
fileList+=( !(*hi*).gif )
fileList+=( !(*hi*).png )
You can print once the array to see if it lists all the files you need as
printf "%s\n" "${fileList[#]}"
and to delete those files do
for eachfile in "${fileList[#]}"; do
rm -v -- "$eachfile"
done
(or) as Benjamin.W suggested in comments below, do
rm -v -- "#{fileList[#]}"
Now i want to write a script to delete all lower resolution images
This script could be used for that:
find /path/to/dir -type f -iname '*.hi.png' -or -iname '*.hi.gif' -or -iname '*.hi.jpg' | while read F; do LOWRES="$(echo "$F" | rev | cut -c7- | rev)$(echo "$F" | rev | cut -c 1-3 | rev)"; if [ -f "$LOWRES" ]; then echo rm -fv -- "$LOWRES"; fi; done
You can run it to see what files will be removed first. If you're ok with results then remove echo before rm command.
Here is the non-one line version, but a script:
#!/bin/sh
find /path/to/dir -type f -iname '*.hi.png' -or -iname '*.hi.gif' -or -iname '*.hi.jpg' |
while read F; do
NAME="$(echo "$F" | rev | cut -c7- | rev)"
EXTENSION="$(echo "$F" | rev | cut -c 1-3 | rev)"
LOWRES="$NAME$EXTENSION"
if [ -f "$LOWRES" ]; then
echo rm -fv -- "$LOWRES"
fi
done
i have several directories which i want to rename:
etc:
"duedate-year" directory to "duedate" (just removing -year)
"start-year" directory to "start"
This is what i've tried:
for CACHE in `find ${DESTINATION_REPO} -maxdepth 1 -type d -name "*year" ` ;
do
set UPDATE="awk -F"-year" '{print $1}' $CACHE" ;
mv $CACHE $UPDATE
done
However it doesn't succeed. Is there away to rename directory using "sed" command?
You're assigning the result of awk incorrectly. It should be inside backticks or $(...). And to process a variable, you need to pipe echo $CACHE to it, not use $CACHE as the filename argument (that will process the contents of the file). So that line should be:
And variables aren't assigned using set, you just write var=value.
So that line should be:
UPDATE=$(echo "$CACHE" | awk -F-year '{print $1}`)
But there's no need to use awk for this at all, you can use shell variable expansion operators:
UPDATE=${CACHE%%-year*}
%%year* means to remove the longest trailing part of the value that matches the wildcard -year*.
Many shell solutions will "work" for a given sample input set and then blow up disastrously later, usually due to unquoted variables, incorrect processing of blanks, etc. This should be safe unless your file name contains newlines (in which case see find -print0 and xargs -0):
find "$DESTINATION_REPO" -maxdepth 1 -type d -name "*-year" |
while IFS= read -r CACHE
do
mv -- "$CACHE" "${CACHE%-year}"
done
Or use the rename command
rename 's/-year//' *year
Yes you can use a pipe. I do this:
for DIR in $(find ${DESTINATION_REPO} -maxdepth 1 -type d -name "*year"); do
mv "${DIR}" $(echo "${DIR}" | sed -E 's/year//')
done
It should be noted that I am very much self taught and sometimes have bad habits...
After consulting gniourf_gniourf I am posting a more robust version (read more) which is a "code lift" of Ed Morton's answer below.
find ${DESTINATION_REPO} -maxdepth 1 -type d -name "*-year"|
while IFS= read -r DIR
do
mv "${DIR}" $(echo "${DIR}" | sed -E 's/year//')
done
So I would like to do a simple find in a dir with:
find /HOME/ | grep .properties
Then with this list I want to weed out certain files, lets say one is server.properties and another is testing.properties.
After those have been taken out, I want to do a quick for loop that will pass each remaning file that didn't get filtered out into a function one by one. The function call is just something like
extractHash FILE OUTPUTFILE
I hope this makes sense, I'll try to be more clear if it's not.
Thanks
for file in "`find ~ -name \*.properties |grep -v -e server.properties -e testfile.properties`"; do
extractHash $file output
done
Use while, not for, for iterating over files: for will not work as you expect for iterating over the output of a backtick-ed program if there is extraneous whitespace:
find /HOME -name \*.properties \! -name server.propertiees \! -name testing.properties` |
while read -r file; do
extractHash "$file" OUTPUTFILE
done
If all your files are in the current directory, use an extended globbing pattern, and for is appropriate to iterate over filename wildcards:
shopt -s extglob
for file in !(server|testing).properties; do
extractHash "$file" out
done
In csh you would use foreach:
#!/bin/csh
set files=`find /HOME/ | grep .properties`
foreach file ($files)
set outfile = $file.out
extractHash $file $outfile
end
not sure about bash - it has a similar for loop but I never learned it :)
First, I would recommend using the -name argument for find instead of piping every filename through grep. Then you can do something like:
for file in `find /HOME -name \*.properties \! -name server.propertiees \! -name testing.properties`; do
extractHash "$file" OUTPUTFILE
done
I know there are a lot of things like this around, but either they don't work recursively or they are huge.
This is what I got:
find . -name "*.so" -exec mv {} `echo {} | sed s/.so/.dylib/` \;
When I just run the find part it gives me a list of files. When I run the sed part it replaces any .so with .dylib. When I run them together they don't work.
I replaced mv with echo to see what happened:
./AI/Interfaces/C/0.1/libAIInterface.so ./AI/Interfaces/C/0.1/libAIInterface.so
Nothing is replaced at all!
What is wrong?
This will do everything correctly:
find -L . -type f -name "*.so" -print0 | while IFS= read -r -d '' FNAME; do
mv -- "$FNAME" "${FNAME%.so}.dylib"
done
By correctly, we mean:
1) It will rename just the file extension (due to use of ${FNAME%.so}.dylib). All the other solutions using ${X/.so/.dylib} are incorrect as they wrongly rename the first occurrence of .so in the filename (e.g. x.so.so is renamed to x.dylib.so, or worse, ./libraries/libTemp.so-1.9.3/libTemp.so is renamed to ./libraries/libTemp.dylib-1.9.3/libTemp.so - an error).
2) It will handle spaces and any other special characters in filenames (except double quotes).
3) It will not change directories or other special files.
4) It will follow symbolic links into subdirectories and links to target files and rename the target file, not the link itself (the default behaviour of find is to process the symbolic link itself, not the file pointed to by the link).
for X in `find . -name "*.so"`
do
mv $X ${X/.so/.dylib}
done
A bash script to rename file extensions generally
#/bin/bash
find -L . -type f -name '*.'$1 -print0 | while IFS= read -r -d '' file; do
echo "renaming $file to $(basename ${file%.$1}.$2)";
mv -- "$file" "${file%.$1}.$2";
done
Credits to aps2012.
Usage
Create a file e.g. called ext-rename (no extension, so you can run it like a command) in e.g. /usr/bin (make sure /usr/bin is added to your $PATH)
run ext-rename [ext1] [ext2] anywhere in terminal, where [ext1] is renaming from and [ext2] is renaming to. An example use would be: ext-rename so dylib, which will rename any file with extension .so to same name but with extension .dylib.
What is wrong is that
echo {} | sed s/.so/.dylib/
is only executed once, before the find is launched, sed is given {} on its input, which doesn't match /.so/ and is left unchanged, so your resulting command line is
find . -name "*.so" -exec mv {} {}
if you have Bash 4
#!/bin/bash
shopt -s globstar
shopt -s nullglob
for file in /path/**/*.so
do
echo mv "$file" "${file/%.so}.dylib"
done
He needs recursion:
#!/bin/bash
function walk_tree {
local directory="$1"
local i
for i in "$directory"/*;
do
if [ "$i" = . -o "$i" = .. ]; then
continue
elif [ -d "$i" ]; then
walk_tree "$i"
elif [ "${i##*.}" = "so" ]; then
echo mv $i ${i%.*}.dylib
else
continue
fi
done
}
walk_tree "."