I'm trying to rename over 1700 videos for a emulator I'm putting together,
Some of the files can look like the following examples:
romfilename1!!! (Japan) [SLUS-01005].mp4
romfilename2 (USA) [SLUS-28605] (Disc 1).mp4
romfilename3 (USA) [SLUS-28605] (Disc 2).mp4
I'm trying to achieve the following results:
romfilename1.mp4
romfilename2 (Disc 1).mp4
romfilename3 (Disc 2).mp4
So far I've been able to remove (USA) & (Japan) by using:
for i in *.mp4
do
mv "$i" "`echo $i | sed 's/ (USA)//'`"
done
So now I'm stuck on how I could go about removing the Exclamation Marks,
I've spent much time trying to search for an answer but havnt had much luck.
I am also stuck on how I got about removing these code thingys "[SLUS-28605]"
Mostly because of the brackets "[" and "]", the code inside is not important.
I've triend the following but the these particular characters mess things up.
for i in *.mp4
do
mv "$i" "`echo $i | sed 's/!!//'`"
done
and...
for i in *.mp4
do
mv "$i" "`echo $i | sed 's/[SLUS-28605]//'`"
done
and..
for i in *.mp4
do
mv "$i" "`echo $i | sed -i 's/[]"[]//g'
done
Thanks in advance for any assistance, Nem
You don't need sed for any of this.
shopt -s extglob
for i in *.mp4
do
# Remove all !; the ! doesn't need to be escaped if history
# expansion is disabled.
new_i=${i//\!}
# Remove the *first* parenthesized group (which contains the country)
new_i=${new_i/ (+([!)]))}
# Remove the bracketed group
new_i=${new_i// \[*]}
#mv "$i" "$new_i"
echo "mv \"$i\" \"$new_i\""
done
You can remove the echo once you verify that the mv commands are correct.
You can substitute multiple patterns in one line using sed and should escape special chars like spaces and square braces:
#!/bin/bash
for i in *.mp4
do
mv "$i" "$(echo $i | sed 's/!!!//; s/\ (USA)\ //; s/\ (Japan)\ //; s/\[SLUS-[^][]*\]//')"
done
You can use rename command for that. It supports regexes. So the command will looks like:
rename 's/[![]]//g' *
or
rename 's/[!]*\|\[[^]]*\]\| *(Japan) *\| *(USA) *//g' *
Though please double check man page of rename available in your system. E.g. deb-based and rpm-based distributives use different versions and regex will vary depending on your local rename version.
Regex should be adjusted to your complete requirement, as it is not really clear from the question.
It will also save from possible issues with special symbols in filename like \n and others.
Remove the ! :
for i in *.mp4
do
name=`echo $i | sed 's/!//g'`
mv "$i" "$name"
done
Eemove the [???] :
for i in *.mp4
do
name=`echo $i | sed 's/\[[^][]*\]//g'`
mv "$i" "$name"
done
Remove the (???) :
for i in *.mp4
do
name=`echo $i | sed 's/([^)(]*)//g'`
mv "$i" "$name"
done
If you want to remove all in once :
for i in *.mp4
do
name=`echo $i | sed 's/!//g' | sed 's/([^)(]*)//g' | sed 's/\[[^][]*\]//g' `
mv "$i" "$name"
done
Related
Answer #1: I ended up starting from scratch, and I was able to piece something together that works! Might not be the most efficient, but it does the job.
#!/bin/bash
for file in *.php
do
#Check to see if the filename contains any uppercase characters
iscap=`echo $file | awk '{if ($0 ~ /[[:upper:]]/) print }'`
if [[ -n $iscap ]]
then
#If the filename contains upper case characters convert them to lower case
newname=`echo $file | tr '[A-Z]' '[a-z]'` #make lower case
#Perform various search/replaces on the file name to clean things up
newname=$(echo "$newname" | sed 's/---/-/')
newname=$(echo "$newname" | sed 's/--/-/')
newname=$(echo "$newname" | sed 's/-\./\./')
newname=$(echo "$newname" | sed 's/there-s-/theres-/')
#Rename file
echo "Moving $file\n To $newname\n\n"
mv $file $newname
#Update all references to the new filename in all php files
`sed -i "s/$file/$newname/g" *.php`
fi
done
Answer #2: Although it doesn't check for upper case characters and just tries to convert everything (giving an error on files that are already lower case), if you know you're only going to be processing files with upper case characters you can use the version that aqua submitted.
#!/bin/bash
for file in *.php
do
newname=`echo $file | tr '[A-Z]' '[a-z]'`
newname=$(echo "$newname" | sed 's/---/-/')
newname=$(echo "$newname" | sed 's/--/-/')
newname=$(echo "$newname" | sed 's/-\./\./')
newname=$(echo "$newname" | sed 's/there-s-/theres-/')
mv "$file" "$newname"
for f in *.php
do
sed -i "s/$file/$newname/g" *.php
done
done
Thanks for everyone's help on this!
The Original Question & Additional Information
I'm using some software to create documentation for an open source software project, except it outputs the HTML files with upper case characters, which isn't recommended for SEO. So, I'm trying to write a bash script that will change all .html files to lower case, and then search/replace
So, I want to turn this:
This-Is-The-Output.html
Into this:
this-is-the-output.html
And then search all the files in the current directory for "This-Is-The-Output.html" and replace it with "this-is-the-output.html".
I've been trying to piece something together based on what I've found online, but I just can't get it to work. I'm about as inexperienced with writing bash scripts as someone can be, so I figured I would turn to SO to see if someone could help out.
This is what I'm currently working with...
#!/bin/sh
for i in *.php;
icleaned=`echo $i | sed s/./\\./g`;
inew="$(tr [A-Z] [a-z] <<< "$icleaned")";
sed -i 's/$i/$inew/g' *.php;
do mv $i $inew;
done
Line 3: Do a search and replace for special characters, like "."
Line 4: Assign the clean, lower case string to $inew
Line 5: Search/replace all references of the old filenames to the new filenames
Line 6: Move the files from their old filename to their new filename
The errors I'm getting are:
line 3: syntax error near unexpected token `icleaned=`echo $i | sed s/./\\./g`'
line 3: ` icleaned=`echo $i | sed s/./\\./g`;'
I tried just echoing icleaned to see if the first step was working properly, but I get the exact same error.
I even tried commenting out everything except for the inew= line:
#!/bin/sh
for i in *.php;
# icleaned=`echo $i | sed s/./\\./g`;
inew="$(tr [A-Z] [a-z] <<< "$i")";
echo $inew;
# sed -i 's/$i/$inew/g' *.php;
# do mv $i $inew;
done
And I'm still getting the "syntax error near unexpected token" error.
line 4: syntax error near unexpected token `inew="$(tr [A-Z] [a-z] <<< "$i")"'
line 4: ` inew="$(tr [A-Z] [a-z] <<< "$i")";'
So obvious the script is messed up in many, many ways.
Any help someone could offer up would be greatly appreciated.
Updated as per comments from #JS to use -exec:
find . -name '*[A-Z]*' -type f -exec bash -c 'echo "{}" | mv "{}" "$(tr A-Z a-z)"' \;
You can try rename as well if you have the correct version installed. Check your man page for rename:
rename 'y/A-Z/a-z/' *
A one-liner with mv to search and rename files with uppercase in filename only using find.
for i in $(find . -name '*[A-Z]*' -type f); do mv "$i" "$(echo $i|tr A-Z a-z)"; done
Something like this should work:
#!/bin/bash
for file in *.html
do
lowercase=`echo $file | tr '[A-Z]' '[a-z]'`
mv "$file" "$lowercase"
for f in *.html
do
sed -i "s/$file/$lowercase/g" "$f"
done
done
Change *.html to *.<extension> if you're working with something other than html files.
This is really the answer the OP came up with after some work, but I thought I would separate it here so it doesn't clutter up the question.
I ended up starting from scratch, and I was able to piece something together that works! Might not be the most efficient, but it does the job.
#!/bin/bash
for file in *.php
do
#Check to see if the filename contains any uppercase characters
iscap=`echo $file | awk '{if ($0 ~ /[[:upper:]]/) print }'`
if [[ -n $iscap ]]
then
#If the filename contains upper case characters convert them to lower case
newname=`echo $file | tr '[A-Z]' '[a-z]'` #make lower case
#Perform various search/replaces on the file name to clean things up
newname=$(echo "$newname" | sed 's/---/-/')
newname=$(echo "$newname" | sed 's/--/-/')
newname=$(echo "$newname" | sed 's/-\./\./')
newname=$(echo "$newname" | sed 's/there-s-/theres-/')
#Rename file
echo "Moving $file\n To $newname\n\n"
mv $file $newname
#Update all references to the new filename in all php files
`sed -i "s/$file/$newname/g" *.php`
fi
done
Although it doesn't check for upper case characters and just tries to convert everything (giving an error on files that are already lower case), if you know you're only going to be processing files with upper case characters you can use the version that aqua submitted.
#!/bin/bash
for file in *.php
do
newname=`echo $file | tr '[A-Z]' '[a-z]'`
newname=$(echo "$newname" | sed 's/---/-/')
newname=$(echo "$newname" | sed 's/--/-/')
newname=$(echo "$newname" | sed 's/-\./\./')
newname=$(echo "$newname" | sed 's/there-s-/theres-/')
mv "$file" "$newname"
for f in *.php
do
sed -i "s/$file/$newname/g" *.php
done
done
Thanks for everyone's help on this!
I'm trying to build a script where a portion of it utilizes 'sed' to tag the filename onto the end of each line in that file, then dumps the output to a master list.
The part of the script giving me trouble is sed here:
DIR=/var/www/flatuser
FILES=$DIR/*
for f in $FILES
do
echo "processing $f file...."
sed -i "s/$/:$f/" $f
cat $f >> $DIR/master.txt
done
The issue is that the 'sed' statement works fine outside of the for loop, but when I place it in the script, I believe it's having issues interpreting the dollar signs. I've tried nearly every combo of " and ' that I can think of to get it to interpret the variable and it continuously either puts "$f" at the end of each line, or it fails outright.
Thanks for any input!
You just need to escape the dollar sign:
sed -i "s/\$/:$f/" "$f"
so that the shell passes it literally to sed.
To expand on Charles Duffy's point about quoting variables:
DIR=/var/www/flatuser
for f in "$DIR"/*
do
echo "processing $f file...."
sed -i "s/\$/:${f##*/}/" "$f"
cat "$f" >> "$DIR/master.txt"
done
If any file names contain a space, it's too late to do anything about it if you assign the list of file names to $FILES; you can no longer distinguish between spaces that belong to file names and spaces that separate file names. You could use an array instead, but it's simpler to just put the glob directly in the for loop. Here's how you would use an array:
DIR=/var/www/flatuser
FILES=( "$DIR"/* )
for f in "${FILES[#]}"
do
echo "processing $f file...."
sed -i "s/\$/:${f##*/}/" "$f"
cat "$f" >> "$DIR/master.txt"
done
For versions of sed that don't use -i, here's a way to explicitly handle the temp file needed to simulate in-place editing:
t=$(mktmp sXXXX); sed "s/\$/:$f/" "$f" > "$t"; mv "$t" "$f" && rm "$t"
Personally, I'd do this like so:
dir=/var/www/flatuser
for f in "$dir"/*; do
[[ $f = */master.txt ]] && continue
while read -r; do printf '%s:%s\n' "$REPLY" "${f##*/}"; done <"$f"
done >/var/www/flatuser/master.txt
It doesn't modify your files in-place the way sed -i does, so it's safe to run more than one time (the sed -i version will add the names to your files in-place every time it runs, so you'll end up with each line having more than one copy of the filename on it).
Also, sed -i isn't specified by POSIX, so not all operating systems will have it.
The problem is NOT the dollar sign. It's that the variable $f contains a "/" character, and sed is using that to separate expressions. Try using "#" as the separator.
DIR=/var/www/flatuser
FILES=$DIR/*
for f in $FILES
do
echo "processing $f file...."
sed -i s#"$"#:"$f"# $f
cat $f >> $DIR/master.txt
done
it's old, but maybe it helps someone.
Why not basename the file to get rid of leading directory
DIR=/var/www/flatuser
FILES=( "$DIR"/* )
for f in "${FILES[#]}"
do
echo "processing $f file...."
b=`basename $f`
sed -i "s/\$/:${b##*/}/" "$b"
cat "$f" >> "$DIR/master.txt"
done
not tested ...
I would like to change lowercase filenames to uppercase with awk/sed/bash
your help would be appreciated
aaaa.txt
vvjv.txt
acfg.txt
desired output
AAAA.txt
VVJV.txt
ACFG.txt
PREFACE:
If you don't care about the case of your extensions, simply use the 'tr' utility in a shell loop:
for i in *.txt; do mv "$i" "$(echo "$i" | tr '[a-z]' '[A-Z]')"; done
If you do care about the case of the extensions, then you should be aware that there is more than one way to do it (TIMTOWTDI). Personally, I believe the Perl solution, listed here, is probably the simplest and most flexible solution under Linux. If you have multiple file extensions, simply specify the number you wish to keep unchanged. The BASH4 solution is also a very good one, but you must be willing to write out the extension a few times, or alternatively, use another variable to store it. But if you need serious portability then I recommend the last solution in this answer which uses octals. Some flavours of Linux also ship with a tool called rename that may also be worth checking out. It's usage will vary from distro to distro, so type man rename for more info.
SOLUTIONS:
Using Perl:
# single extension
perl -e 's/\.[^\.]*$/rename $_, uc($`) . $&/e for #ARGV' *.txt
# multiple extensions
perl -e 's/(?:\.[^\.]*){2}$/rename $_, uc($`) . $&/e for #ARGV' *.tar.gz
Using BASH4:
# single extension
for i in *.txt; do j="${i%.txt}"; mv "$i" "${j^^}.txt"; done
# multiple extensions
for i in *.tar.gz; do j="${i%.tar.gz}"; mv "$i" "${j^^}.tar.gz"; done
# using a var to store the extension:
e='.tar.gz'; for i in *${e}; do j="${i%${e}}"; mv "$i" "${j^^}${e}"; done
Using GNU awk:
for i in *.txt; do
mv "$i" $(echo "$i" | awk '{ sub(/.txt$/,""); print toupper($0) ".txt" }');
done
Using GNU sed:
for i in *.txt; do
mv "$i" $(echo "$i" | sed -r -e 's/.*/\U&/' -e 's/\.TXT$/\u.txt/');
done
Using BASH3.2:
for i in *.txt; do
stem="${i%.txt}";
for ((j=0; j<"${#stem}"; j++)); do
chr="${stem:$j:1}"
if [[ "$chr" == [a-z] ]]; then
chr=$(printf "%o" "'$chr")
chr=$((chr - 40))
chr=$(printf '\'"$chr")
fi
out+="$chr"
done
mv "$i" "$out.txt"
out=
done
In general for lowercase/upper case modifications "tr" ( translate characters ) utility is often used, it's from the set of command line utilities used for character replacement.
dtpwmbp:~ pwadas$ echo "xxx" | tr '[a-z]' '[A-Z]'
XXX
dtpwmbp:~ pwadas$
Also, for renaming files there's "rename" utility, delivered with perl ( man rename ).
SYNOPSIS
rename [ -v ] [ -n ] [ -f ] perlexpr [ files ]
DESCRIPTION
"rename" renames the filenames supplied according to the rule specified as the first argument. The perlexpr argument is a Perl expression which is expected to modify the $_ string in
Perl for at least some of the filenames specified. If a given filename is not modified by the expression, it will not be renamed. If no filenames are given on the command line,
filenames will be read via standard input.
For example, to rename all files matching "*.bak" to strip the extension, you might say
rename 's/\.bak$//' *.bak
To translate uppercase names to lower, you'd use
rename 'y/A-Z/a-z/' *
I would suggest using rename, if you only want to uppercase the filename and not the extension, use something like this:
rename -n 's/^([^.]*)\.(.*)$/\U$1\E.$2/' *
\U uppercases everything until \E, see perlreref(1). Remove the -n when your happy with the output.
Bash 4 parameter expansion can perform case changes:
for i in *.txt; do
i="${i%.txt}"
mv "$i.txt" "${i^^?}.txt"
done
bash:
for f in *.txt; do
no_ext=${f%.txt}
mv "$f" "${no_ext^^}.txt"
done
for f in *.txt; do
mv "$f" "`tr [:lower:] [:upper:] <<< "${f%.*}"`.txt"
done
An easier, lightweight and portable approach would be:
for i in *.txt
do
fname=$(echo $i | cut -d"." -f1 | tr [a-z] [A-Z])
ext=$(echo $i | cut -d"." -f2)
mv $i $fname.$ext
done
This would work on almost every version of BASH since we are using most common external utilities (cut, tr) found on every Unix flavour.
Simply use (on terminal):
for i in *.txt; do mv $i `echo ${i%.*} | tr [:lower:] [:upper:]`.txt; done;
This might work for you (GNU sed):
printf "%s\n" *.txt | sed 'h;s/[^.]*/\U&/;H;g;s/\(.*\)\n/mv -v \1 /' | sh
or more simply:
printf "%s\n" *.txt | sed 'h;s/[^.]*/\U&/;H;g;s/\(.*\)\n/mv -v \1 /e'
for i in *.jar; do mv $i `echo ${i%} | tr [:upper:] [:lower:]`; done;
this works for me.
EDIT: Ok, I'm sorry, I should have specified that I was on Windows, and using win-bash, which is based on bash 1.14.2, along with the gnuwin32 tools. This means all of the solutions posted unfortunately didn't help out. It doesn't contain many of the advanced features. I have however figured it out finally. It's an ugly script, but it works.
#/bin/bash
function readdir
{
cd "$1"
for infile in *
do
if [ -d "$infile" ]; then
readdir "$infile"
else
renamer "$infile"
fi
done
cd ..
}
function renamer
{
#replace " - " with a single underscore.
NEWFILE1=`echo "$1" | sed 's/\s-\s/_/g'`
#replace spaces with underscores
NEWFILE2=`echo "$NEWFILE1" | sed 's/\s/_/g'`
#replace "-" dashes with underscores.
NEWFILE3=`echo "$NEWFILE2" | sed 's/-/_/g'`
#remove exclamation points
NEWFILE4=`echo "$NEWFILE3" | sed 's/!//g'`
#remove commas
NEWFILE5=`echo "$NEWFILE4" | sed 's/,//g'`
#remove single quotes
NEWFILE6=`echo "$NEWFILE5" | sed "s/'//g"`
#replace & with _and_
NEWFILE7=`echo "$NEWFILE6" | sed "s/&/_and_/g"`
#remove single quotes
NEWFILE8=`echo "$NEWFILE7" | sed "s/’//g"`
mv "$1" "$NEWFILE8"
}
for infile in *
do
if [ -d "$infile" ]; then
readdir "$infile"
else
renamer "$infile"
fi
done
ls
I'm trying to create a bash script to recurse through a directory and rename files, to remove spaces, dashes and other characters. I've gotten the script working fine for what I need, except for the recursive part of it. I'm still new to this, so it's not as efficient as it should be, but it works. Anyone know how to make this recursive?
#/bin/bash
for infile in *.*;
do
#replace " - " with a single underscore.
NEWFILE1=`echo $infile | sed 's/\s-\s/_/g'`;
#replace spaces with underscores
NEWFILE2=`echo $NEWFILE1 | sed 's/\s/_/g'`;
#replace "-" dashes with underscores.
NEWFILE3=`echo $NEWFILE2 | sed 's/-/_/g'`;
#remove exclamation points
NEWFILE4=`echo $NEWFILE3 | sed 's/!//g'`;
#remove commas
NEWFILE5=`echo $NEWFILE4 | sed 's/,//g'`;
mv "$infile" "$NEWFILE5";
done;
find is the command able to display all elements in a filesystem hierarchy. You can use it to execute a command on every found file or pipe the results to xargs which will handle the execution part.
Take care that for infile in *.* does not work on files containing whitespaces. Check the -print0 option of find, coupled to the -0 option of xargs.
All those semicolons are superfluous and there's no reason to use all those variables. If you want to put the sed commands on separate lines and intersperse detailed comments you can still do that.
#/bin/bash
find . | while read -r file
do
newfile=$(echo "$file" | sed '
#replace " - " with a single underscore.
s/\s-\s/_/g
#replace spaces with underscores
s/\s/_/g
#replace "-" dashes with underscores.
s/-/_/g
#remove exclamation points
s/!//g
#remove commas
s/,//g')
mv "$infile" "$newfile"
done
This is much shorter:
#/bin/bash
find . | while read -r file
do
# replace " - " or space or dash with underscores
# remove exclamation points and commas
newfile=$(echo "$file" | sed 's/\s-\s/_/g; s/\s/_/g; s/-/_/g; s/!//g; s/,//g')
mv "$infile" "$newfile"
done
Shorter still:
#/bin/bash
find . | while read -r file
do
# replace " - " or space or dash with underscores
# remove exclamation points and commas
newfile=$(echo "$file" | sed 's/\s-\s/_/g; s/[-\s]/_/g; s/[!,]//g')
mv "$infile" "$newfile"
done
In bash 4, setting the globstar option allows recursive globbing.
shopt -s globstar
for infile in **
...
Otherwise, use find.
while read infile
do
...
done < <(find ...)
or
find ... -exec ...
I've used 'find' in the past to locate files then had it execute another application.
See '-exec'
rename 's/pattern/replacement/' glob_pattern
I would like to rename files numbering: I have a files with '???' format I need to put them in '????'.
myfile_100_asd_4 to myfile_0100_asd_4
Thanks
Arman.
Not so elegant SOLUTION:
#/bin/bash
snap=`ls -t *_???`
c=26
for k in $snap
do
end=${k}
echo mv $k ${k%_*}_0${k##*_}_asd_4
(( c=c-1 ))
done
This works for me because I have myfile_100 files as well.
Use rename, a small script that comes with perl:
rename 's/(\d{3})/0$1/g' myfile_*
If you pass it the -n parameter before the expression it only prints what renames it would have done, no action is taken. This way you can verify it works ok before you rename your files:
rename -n 's/(\d{3})/0$1/g' myfile_*
just use the shell,
for file in myfile*
do
t=${file#*_}
f=${file%%_*}
number=$(printf "%04d" ${t%%_*})
newfile="${f}_${number}_${t#*_}"
echo mv "$file" "$newfile"
done
There's a UNIX app called ren (manpage) which supports renaming multiple files using search and substitution patterns. You should be able to cobble together a pattern that will inject that extra 0 into the filename.
Edit: Project page w/ download link can be found at Freshmeat.
Try:
for file in `ls my*`
do
a=`echo $file | cut -d_ -f1`
b=`echo $file | cut -d_ -f2`
c=`echo $file | cut -d_ -f3,4`
new=${a}_0${b}_${c}
mv $file $new
done