I have a directory of files with names formatted like
01-Peterson#2x.png
15-Consolidated#2x.png
03-Brady#2x.png
And I would like to format them like
PETERSON.png
CONSOLIDATED.png
BRADY.png
But my bash scripting skills are pretty weak right now. What is the best way to go about this?
Edit: my bash version is 3.2.57(1)-release
This will work for files that contains spaces (including newlines), backslashes, or any other character, including globbing chars that could cause a false match on other files in the directory, and it won't remove your home file system given a particularly undesirable file name!
for old in *.png; do
new=$(
awk 'BEGIN {
base = sfx = ARGV[1]
sub(/^.*\./,"",sfx)
sub(/^[^-]+-/,"",base)
sub(/#[^#.]+\.[^.]+$/,"",base)
print toupper(base) "." sfx
exit
}' "$old"
) &&
mv -- "$old" "$new"
done
If the pattern for all your files are like the one you posted, I'd say you can do something as simple as running this on your directory:
for file in `ls *.png`; do new_file=`echo $file | awk -F"-" '{print $2}' | awk -F"#" '{n=split($2,a,"."); print toupper($1) "." a[2]}'`; mv $file $new_file; done
If you fancy learning other solutions, like regexes, you can also do:
for file in `ls *.png`; do new_file=`echo $file | sed "s/.*-//g;s/#.*\././g" | tr '[:lower:]' '[:upper:]'`; mv $file $new_file; done
Testing it, it does for example:
mv 01-Peterson#2x.png PETERSON.png
mv 02-Bradley#2x.png BRADLEY.png
mv 03-Jacobs#2x.png JACOBS.png
mv 04-Matts#1x.png MATTS.png
mv 05-Jackson#4x.png JACKSON.png
Related
I have a folder contain daily rainfall data in geotiff format from 1981-2019 with naming convention chirps-v2.0.yyyymmdd.1days.tif
I would like to arrange all the files based on MONTH information, and move into a new folder, ie all files with Month = January will move to Month01 folder.
Is there any one-liner solution for that, I am using terminal on macos.
This should do it:
for i in $(seq -f "%02g" 1 12); do mkdir -p "Month$i"; mv chirps-v2.0.????$i*.tif "Month$i"; done
Explanation:
For each number in the range 1, 12 (padded with 0 if necessary)...
Make the directories Month01, Month02, etc. If the directory already exists, continue.
Move all files that include the current month number in the relevant part of the filename to the appropriate folder. The question marks in chirps-v2.0.????$i*.tif represent single-character wildcards.
Note: If there is any chance there will be spaces in your .tif filenames, you can use "chirps-v2.0."????"$i"*".tif" instead.
I don't think there is a simple way to do this. You can, however, do a "one-liner" solution if you use pipes and for loops, things like that:
for file in $(ls *.tif); do sed -r 's/(.*\.[0-9]{4})([0-9]{2})(.*)/\1 \2 \3/' <<< "$file" | awk '{print "mkdir -p dstDir/Month" $2 "; cp", $1 $2 $3, "dstDir/Month" $2}' | bash ; done
Formatting this a bit:
for file in $(ls *.tif); do \
sed -r 's/(.*\.[0-9]{4})([0-9]{2})(.*)/\1 \2 \3/' <<< "$file" \
| awk '{print "mkdir -p dstDir/Month" $2 "; cp", $1 $2 $3, "dstDir/Month" $2}' \
| bash
done
This needs to be executed from the directory containing your files (see "ls *.tif). You will also need to replace "dstDir" with the name of the parent directory where "Month01" will be created.
This may not be perfect, but you can edit it, if required. Also, if you don't have bash, only zsh, replace the "bash" bit by "zsh" should still work.
I have three files in a directory that has the structure like this:
file.exe.trace, file.exe.trace.functions and file.exe.trace.netlog
I want to know how can I get file.exe as file name?
In other world I need to get file name that has the .trace extension? I should note that as you can see all the files has the .trace part.
If $FILENAME has the name, the root part can be gotten from ${FILENAME%%.trace*}
for FILENAME in *.trace; do
echo ${FILENAME%%.trace*}
done
You can also use basename:
for f in *.trace; do
basename "$f" ".trace"
done
Update: The previous won't process files with extra extensions besides .trace like .trace.functions, but the following sed will do:
sed -r 's_(.*)\.trace.*_\1_' <(ls -c1)
You can also use it in a for loop instead:
for f in *.trace*; do
sed -r 's_(.*)\.trace.*_\1_' <<< "$f"
done
Try:
for each in *exe*trace* ; do echo $each | awk -F. '{print $1"."$2}' ; done | sort | uniq
I had written an awk code for deleting all the lines ending in a colon from a file. But now I want to run this particular awk action on a whole folder containing similar files.
awk '!/:$/' qs.txt > fin.txt
awk '{print $3 " " $4}' fin.txt > out.txt
You could wrap your awk command in a loop in your shell such as bash.
myfiles=mydirectory/*.txt
for file in $myfiles
do
b=$(basename "$file" .txt)
awk '!/:$/' "$b.txt" > "$b.out"
done
EDIT: improved quoting as commenters suggested
If you like it better, you can use "${file%.txt}" instead of $(basename "$file" .txt).
Aside: My own preference runs to basename just because man basename is easier for me than man -P 'less -p "^ Param"' bash (when that is the relevant heading on the particular system). Please accept this quirk of mine and let's not discuss info and http://linux.die.net/man/ and whatever.
You could use sed. Just run the below command on the directory in which the files you want to change was actually stored.
sed -i '/:$/d' *.*
This will create new files in an empty directory, with the same name.
mkdir NEWFILES
for file in `find . -name "*name_pattern*"`
do
awk '!/:$/' $file > fin.txt
awk '{print $3 " " $4}' fin.txt > NEWFILES/$file
done
After that you just need to
cp -fr NEWFILES/* .
I would like to change lowercase filenames to uppercase with awk/sed/bash
your help would be appreciated
aaaa.txt
vvjv.txt
acfg.txt
desired output
AAAA.txt
VVJV.txt
ACFG.txt
PREFACE:
If you don't care about the case of your extensions, simply use the 'tr' utility in a shell loop:
for i in *.txt; do mv "$i" "$(echo "$i" | tr '[a-z]' '[A-Z]')"; done
If you do care about the case of the extensions, then you should be aware that there is more than one way to do it (TIMTOWTDI). Personally, I believe the Perl solution, listed here, is probably the simplest and most flexible solution under Linux. If you have multiple file extensions, simply specify the number you wish to keep unchanged. The BASH4 solution is also a very good one, but you must be willing to write out the extension a few times, or alternatively, use another variable to store it. But if you need serious portability then I recommend the last solution in this answer which uses octals. Some flavours of Linux also ship with a tool called rename that may also be worth checking out. It's usage will vary from distro to distro, so type man rename for more info.
SOLUTIONS:
Using Perl:
# single extension
perl -e 's/\.[^\.]*$/rename $_, uc($`) . $&/e for #ARGV' *.txt
# multiple extensions
perl -e 's/(?:\.[^\.]*){2}$/rename $_, uc($`) . $&/e for #ARGV' *.tar.gz
Using BASH4:
# single extension
for i in *.txt; do j="${i%.txt}"; mv "$i" "${j^^}.txt"; done
# multiple extensions
for i in *.tar.gz; do j="${i%.tar.gz}"; mv "$i" "${j^^}.tar.gz"; done
# using a var to store the extension:
e='.tar.gz'; for i in *${e}; do j="${i%${e}}"; mv "$i" "${j^^}${e}"; done
Using GNU awk:
for i in *.txt; do
mv "$i" $(echo "$i" | awk '{ sub(/.txt$/,""); print toupper($0) ".txt" }');
done
Using GNU sed:
for i in *.txt; do
mv "$i" $(echo "$i" | sed -r -e 's/.*/\U&/' -e 's/\.TXT$/\u.txt/');
done
Using BASH3.2:
for i in *.txt; do
stem="${i%.txt}";
for ((j=0; j<"${#stem}"; j++)); do
chr="${stem:$j:1}"
if [[ "$chr" == [a-z] ]]; then
chr=$(printf "%o" "'$chr")
chr=$((chr - 40))
chr=$(printf '\'"$chr")
fi
out+="$chr"
done
mv "$i" "$out.txt"
out=
done
In general for lowercase/upper case modifications "tr" ( translate characters ) utility is often used, it's from the set of command line utilities used for character replacement.
dtpwmbp:~ pwadas$ echo "xxx" | tr '[a-z]' '[A-Z]'
XXX
dtpwmbp:~ pwadas$
Also, for renaming files there's "rename" utility, delivered with perl ( man rename ).
SYNOPSIS
rename [ -v ] [ -n ] [ -f ] perlexpr [ files ]
DESCRIPTION
"rename" renames the filenames supplied according to the rule specified as the first argument. The perlexpr argument is a Perl expression which is expected to modify the $_ string in
Perl for at least some of the filenames specified. If a given filename is not modified by the expression, it will not be renamed. If no filenames are given on the command line,
filenames will be read via standard input.
For example, to rename all files matching "*.bak" to strip the extension, you might say
rename 's/\.bak$//' *.bak
To translate uppercase names to lower, you'd use
rename 'y/A-Z/a-z/' *
I would suggest using rename, if you only want to uppercase the filename and not the extension, use something like this:
rename -n 's/^([^.]*)\.(.*)$/\U$1\E.$2/' *
\U uppercases everything until \E, see perlreref(1). Remove the -n when your happy with the output.
Bash 4 parameter expansion can perform case changes:
for i in *.txt; do
i="${i%.txt}"
mv "$i.txt" "${i^^?}.txt"
done
bash:
for f in *.txt; do
no_ext=${f%.txt}
mv "$f" "${no_ext^^}.txt"
done
for f in *.txt; do
mv "$f" "`tr [:lower:] [:upper:] <<< "${f%.*}"`.txt"
done
An easier, lightweight and portable approach would be:
for i in *.txt
do
fname=$(echo $i | cut -d"." -f1 | tr [a-z] [A-Z])
ext=$(echo $i | cut -d"." -f2)
mv $i $fname.$ext
done
This would work on almost every version of BASH since we are using most common external utilities (cut, tr) found on every Unix flavour.
Simply use (on terminal):
for i in *.txt; do mv $i `echo ${i%.*} | tr [:lower:] [:upper:]`.txt; done;
This might work for you (GNU sed):
printf "%s\n" *.txt | sed 'h;s/[^.]*/\U&/;H;g;s/\(.*\)\n/mv -v \1 /' | sh
or more simply:
printf "%s\n" *.txt | sed 'h;s/[^.]*/\U&/;H;g;s/\(.*\)\n/mv -v \1 /e'
for i in *.jar; do mv $i `echo ${i%} | tr [:upper:] [:lower:]`; done;
this works for me.
I run a script which generated about 10k files in a directory. I just discovered that there is a bug in the script which causes some filenames to have a carriage return (presumably a '\n' character).
I want to run a sed command to remove the carriage return from the filenames.
Anyone knows which params to pass to sed to clean up the filenames in the manner described?
I am running Linux (Ubuntu)
I don't know how sed would do this, but this python script should do the trick:.
This isn't sed, but I find python a lot easier to use when doing things like these:
#!/usr/bin/env python
import os
files = os.listdir('.')
for file in files:
os.rename(file, file.replace('\r', '').replace('\n', ''))
print 'Processed ' + file.replace('\r', '').replace('\n', '')
It strips any occurrences of both \r and \n from all of the filenames in a given directory.
To run it, save it somewhere, cd into your target directory (with the files to be processed), and run python /path/to/the/file.py.
Also, if you plan on doing more batch renaming, consider Métamorphose. It's a really nice and powerful GUI for this stuff. And, it's free!
Good luck!
Actually, try this: cd into the directory, type in python, and then just paste this in:
exec("import os\nfor file in os.listdir('.'):\n os.rename(file, file.replace('\\r', '').replace('\\n', ''))\n print 'Processed ' + file.replace('\\r', '').replace('\\n', '')")
It's a one-line version of the previous script, and you don't have to save it.
Version 2, with space replacement powers:
#!/usr/bin/env python
import os
for file in os.listdir('.'):
os.rename(file, file.replace('\r', '').replace('\n', '').replace(' ', '_')
print 'Processed ' + file.replace('\r', '').replace('\n', '')
And here's the one-liner:
exec("import os\nfor file in os.listdir('.'):\n os.rename(file, file.replace('\\r', '').replace('\\n', '')replace(' ', '_'))\n print 'Processed ' + file.replace('\\r', '').replace('\\n', '');")
If there are no spaces in your filenames, you can do:
for f in *$'\n'; do mv "$f" $f; done
It won't work if the newlines are embedded, but it will work for trailing newlines.
If you must use sed:
for f in *$'\n'; do mv "$f" "$(echo "$f" | sed '/^$/d')"; done
Using the rename Perl script:
rename 's/\n//g' *$'\n'
or the util-linux-ng utility:
rename $'\n' '' *$'\n'
If the character is a return instead of a newline, change the \n or ^$ to \r in any places they appear above.
The reason you aren't getting any pure-sed answers is that fundamentally sed edits file contents, not file names; thus the answers that use sed all do something like echo the filename into a pipe (pseudo file), edit that with sed, then use mv to turn that back into a filename.
Since sed is out, here's a pure-bash version to add to the Perl, Python, etc scripts you have so far:
killpattern=$'[\r\n]' # remove both carriage returns and linefeeds
for f in *; do
if [[ "$f" == *$killpattern* ]]; then
mv "$f" "${f//$killpattern/}"
fi
done
...but since ${var//pattern/replacement} isn't available in plain sh (along with [[...]]), here's a version using sh-only syntax, and tr to do the character replacement:
for f in *; do
new="$(printf %s "$f" | tr -d "\r\n")"
if [ "$f" != "$new" ]; then
mv "$f" "$new"
fi
done
EDIT: If you really want it with sed, take a look at this:
http://www.linuxquestions.org/questions/programming-9/merge-lines-in-a-file-using-sed-191121/
Something along these lines should work similar to the perl below:
for i in *; do echo mv "$i" `echo "$i"|sed ':a;N;s/\n//;ta'`; done
With perl, try something along these lines:
for i in *; do mv "$i" `echo "$i"|perl -pe 's/\n//g'`; done
This will rename all files in the current folder by removing all newline characters from them. If you need to go recursive, you can use find instead - be aware of the escaping in that case, though.
In fact there is a way to use sed:
carr='\n' # specify carriage return
files=( $(ls -f) ) # array of files in current dir
for i in ${files[#]}
do
if [[ -n $(echo "$i" | grep $carr) ]] # filenames with carriage return
then
mv "$i" "$(echo "$i" | sed 's/\\n//g')" # move!
fi
done
This actually works.