I have been trying to write a script that will take the current working directory, scan every file and check if it is a .txt file. Then take every file (that's a text file), and check to see if it contains an underscore anywhere in its name and if it does to change the underscore to a hyphen.
I know that this is a tall order, but here is the rough code I have so far:
#!/bin/bash
count=1
while((count <= $#))
do
case $count in
"*.txt") sed 's/_/-' $count
esac
((count++))
done
What I was thinking is that this would take the files in the current working directory as the arguments and check every file(represented by $count or the file at "count"). Then for every file, it would check if it ended in .txt and if it did it would change every underscore to a hyphen using sed. I think one of the main problems I am having is that the script is not reading the files from the current working directory. I tried included the directory after the command to run the script, but I think it took each line instead of each file (since there are 4 or so files on every line).
Anyway, any help would be greatly appreciated! Also, I'm sorry that my code is so bad, I am very new to UNIX.
for fname in ./*_*.txt; do
new_fname=$(printf '%s' "$fname" | sed 's,_,-,')
mv "$fname" "$new_fname"
done
why not:
rename 's/_/-/' *.txt
$ ls *.txt | while read -r file; do echo $file |
grep > /dev/null _ && mv $file $(echo $file | tr _ -); done
(untested)
Thanks for all your input guys! All in all, I think the solution I found was the most appropriate for my skill level was:
ls *.txt | while read -r file; do echo file |
mv $file $(echo $file | sed 's,_,-,');
done
This got what I needed done, and for my purposes I am not too worried about the spaces. But thanks for all your wonderful suggestions, you are all very intelligent!
Related
I have three files in a directory that has the structure like this:
file.exe.trace, file.exe.trace.functions and file.exe.trace.netlog
I want to know how can I get file.exe as file name?
In other world I need to get file name that has the .trace extension? I should note that as you can see all the files has the .trace part.
If $FILENAME has the name, the root part can be gotten from ${FILENAME%%.trace*}
for FILENAME in *.trace; do
echo ${FILENAME%%.trace*}
done
You can also use basename:
for f in *.trace; do
basename "$f" ".trace"
done
Update: The previous won't process files with extra extensions besides .trace like .trace.functions, but the following sed will do:
sed -r 's_(.*)\.trace.*_\1_' <(ls -c1)
You can also use it in a for loop instead:
for f in *.trace*; do
sed -r 's_(.*)\.trace.*_\1_' <<< "$f"
done
Try:
for each in *exe*trace* ; do echo $each | awk -F. '{print $1"."$2}' ; done | sort | uniq
The script works as far as renaming the files but it moves the renamed files out of their respective folders.
I would like it to not move them but only rename them and I have failed after a few days of trying. I know this code is a mess and there is unneeded code in it but it nearly works.
Also the renamed file isn’t getting an extension of .txt but that isn't really an issue for me. I just want to see the "Dynamic Range Value" that is taken from inside the file as the file name so I don’t have to open every file (a couple thousand albums worth) to see what the DR is. Here is the code:
#!/bin/bash
cd /media/Storage/MusicWorks/Processing
find . -name 'dr14.txt' | while IFS=$'\n' read -r i
do mv -n "$i" `egrep -m1 -e 'Official DR value:' "$i" | sed -e 's/Official DR value://'`;
echo "Done"
done
I run this script from the terminal with a bash alias.
I have reservations about the egrep | sed part of your script, but if they work for you, so be it. You need to preserve the pathname of the file, for example like this:
find . -name 'dr14.txt' |
while IFS=$'\n' read -r i
do
newname="${i%/*}"/$(egrep -m1 -e 'Official DR value:' "$i" | sed -e 's/Official DR value://');
mv -n "$i" "$newname"
echo "Done $i ($newname)"
done
The ${i%/*} notation removes anything from the last slash to the end of the name in $i. Since all the names from find will start with ./, this is secure enough; it would not work well on absolute names such as / and /unix (the output would be the empty string, but /usr/bin/sh would be fine).
Under a little prompting by tripleee in a comment, it is possible to simplify the egrep | sed part of the code to:
newname="${i%/*}"/$(sed -n -e '/Official DR value:/{s///p;q;}' "$i");
The second semicolon is needed with BSD sed but not with GNU sed.
I have the following files in the following format:
$ ls CombinedReports_LLL-*'('*.csv
CombinedReports_LLL-20140211144020(Untitled_1).csv
CombinedReports_LLL-20140211144020(Untitled_11).csv
CombinedReports_LLL-20140211144020(Untitled_110).csv
CombinedReports_LLL-20140211144020(Untitled_111).csv
CombinedReports_LLL-20140211144020(Untitled_12).csv
CombinedReports_LLL-20140211144020(Untitled_13).csv
CombinedReports_LLL-20140211144020(Untitled_14).csv
CombinedReports_LLL-20140211144020(Untitled_15).csv
CombinedReports_LLL-20140211144020(Untitled_16).csv
CombinedReports_LLL-20140211144020(Untitled_17).csv
CombinedReports_LLL-20140211144020(Untitled_18).csv
CombinedReports_LLL-20140211144020(Untitled_19).csv
I would like this part removed:
20140211144020 (this is the timestamp the reports were run so this will vary)
and end up with something like:
CombinedReports_LLL-(Untitled_1).csv
CombinedReports_LLL-(Untitled_11).csv
CombinedReports_LLL-(Untitled_110).csv
CombinedReports_LLL-(Untitled_111).csv
CombinedReports_LLL-(Untitled_12).csv
CombinedReports_LLL-(Untitled_13).csv
CombinedReports_LLL-(Untitled_14).csv
CombinedReports_LLL-(Untitled_15).csv
CombinedReports_LLL-(Untitled_16).csv
CombinedReports_LLL-(Untitled_17).csv
CombinedReports_LLL-(Untitled_18).csv
CombinedReports_LLL-(Untitled_19).csv
I was thinking simply along the lines of the mv command, maybe something like this:
$ ls CombinedReports_LLL-*'('*.csv
but maybe a sed command or other would be better
rename is part of the perl package. It renames files according to perl-style regular expressions. To remove the dates from your file names:
rename 's/[0-9]{14}//' CombinedReports_LLL-*.csv
If rename is not available, sed+shell can be used:
for fname in Combined*.csv ; do mv "$fname" "$(echo "$fname" | sed -r 's/[0-9]{14}//')" ; done
The above loops over each of your files. For each file, it performs a mv command: mv "$fname" "$(echo "$fname" | sed -r 's/[0-9]{14}//')" where, in this case, sed is able to use the same regular expression as the rename command above. s/[0-9]{14}// tells sed to look for 14 digits in a row and replace them with an empty string.
Without using an other tools like rename or sed and sticking strictly to bash alone:
for f in CombinedReports_LLL-*.csv
do
newName=${f/LLL-*\(/LLL-(}
mv -i "$f" "$newName"
done
for f in CombinedReports_LLL-* ; do
b=${f:0:20}${f:34:500}
mv "$f" "$b"
done
You can try line by line on shell:
f="CombinedReports_LLL-20140211144020(Untitled_11).csv"
b=${f:0:20}${f:34:500}
echo $b
You can use the rename utility for this. It uses syntax much like sed to change filenames. The following example (from the rename man-page) shows how to remove the trailing '.bak' extension from a list of backup files in the local directory:
rename 's/\.bak$//' *.bak
I'm using the advice given in the top response and have put the following line into a shell script:
ls *.nii | xargs rename 's/[f_]{2}//' f_0*.nii
In terminal, this line works perfectly, but in my script it will not execute and reads * as a literal part of the file name.
I am trying to modify a bash script to remove a glob of malicious code from a large number of files.
The community will benefit from this, so here it is:
#!/bin/bash
grep -r -l 'var createDocumentFragm' /home/user/Desktop/infected_site/* > /home/user/Desktop/filelist.txt
for i in $(cat /home/user/Desktop/filelist.txt)
do
cp -f $i $i.bak
done
for i in $(cat /home/user/Desktop/filelist.txt)
do
$i | sed 's/createDocumentFragm.*//g' > $i.awk
awk '/<\/SCRIPT>/{p=1;print}/<\/script>/{p=0}!p'
This is where the script bombs out with this message:
+ for i in '$(cat /home/user/Desktop/filelist.txt)'
+ sed 's/createDocumentFragm.*//g'
+ /home/user/Desktop/infected_site/index.htm
I get 2 errors and the script stops.
/home/user/Desktop/infected_site/index.htm: line 1: syntax error near unexpected token `<'
/home/user/Desktop/infected_site/index.htm: line 1: `<html><head><script>(function (){ '
I have the first 2 parts done.
The files containing createDocumentfragm have been enumerated in a text file correctly.
The files in the textfile.txt have been duplicated, in their original location with a .bak added to them IE: infected_site/some_directory/infected_file.htm and infected_file.htm.bak
effectively making sure we have a backup.
All I need to do now is write an AWK command that will use the list of files in filelist.txt, use the entire glob of malicious text as a pattern, and remove it from the files. Using just the uppercase script as the starting point, and the lower case script is too generic and could delete legitimate text
I suspect this may help me, but I don't know how to use it correctly.
http://backreference.org/2010/03/13/safely-escape-variables-in-awk/
Once I have this part figured out, and after you have verified that the files weren't mangled you can do this to clean out the bak files:
for i in $(cat /home/user/Desktop/filelist.txt)
do
rm -f $i.bak
done
Several things:
You have:
$i | sed 's/var createDocumentFragm.*//g' > $i.awk
You should probably meant this (using your use of cat which we'll talk about in a moment):
cat $i | sed 's/var createDocumentFragm.*//g' > $i.awk
You're treating each file in your file list as if it was a command and not a file.
Now, about your use of cat. If you're using cat for almost anything but concatenating multiple files together, you probably are doing something not quite right. For example, you could have done this:
sed 's/var createDocumentFragm.*//g' "$i" > $i.awk
I'm also a bit confused about the awk statement. Exactly what file are you using awk on? Your awk statement is using STDIN and STDOUT, so it's reading file names from the for loop and then printing the output on the screen. Is the sed statement suppose to feed into the awk statement?
Note that I don't have to print out my file to STDOUT, then pipe that into sed. The sed command can take the file name directly.
You also want to avoid for loops over a list of files. That is very inefficient, and can cause problems with the command line getting overloaded. Not a big issue today, but can affect you when you least suspect it. What happens is that your $(cat /home/user/Desktop/filelist.txt) must execute first before the for loop can even start.
A little rewriting of your program:
cd ~/Desktop
grep -r -l 'var createDocumentFragm' infected_site/* > filelist.txt
while read file
do
cp -f "$file" "$file.bak"
sed 's/var createDocumentFragm.*//g' "$file" > "$i.awk"
awk '/<\/SCRIPT>/{p=1;print}/<\/script>/{p=0}!p'
done < filelist.txt
We can use one loop, and we made it a while loop. I could even feed the grep into that while loop:
grep -r -l 'var createDocumentFragm' infected_site/* | while read file
do
cp -f "$file" "$file.bak"
sed 's/var createDocumentFragm.*//g' "$file" > "$i.awk"
awk '/<\/SCRIPT>/{p=1;print}/<\/script>/{p=0}!p'
done < filelist.txt
and then I don't even have to create a temporary file.
Let me know what's going on with the awk. I suspect you wanted something like this:
grep -r -l 'var createDocumentFragm' infected_site/* | while read file
do
cp -f "$file" "$file.bak"
sed 's/var createDocumentFragm.*//g' "$file" \
| awk '/<\/SCRIPT>/{p=1;print}/<\/script>/{p=0}!p' > "$i.awk"
done < filelist.txt
Also note I put quotes around file names. This helps prevent problems if file name has a space in it.
I would like to rename files numbering: I have a files with '???' format I need to put them in '????'.
myfile_100_asd_4 to myfile_0100_asd_4
Thanks
Arman.
Not so elegant SOLUTION:
#/bin/bash
snap=`ls -t *_???`
c=26
for k in $snap
do
end=${k}
echo mv $k ${k%_*}_0${k##*_}_asd_4
(( c=c-1 ))
done
This works for me because I have myfile_100 files as well.
Use rename, a small script that comes with perl:
rename 's/(\d{3})/0$1/g' myfile_*
If you pass it the -n parameter before the expression it only prints what renames it would have done, no action is taken. This way you can verify it works ok before you rename your files:
rename -n 's/(\d{3})/0$1/g' myfile_*
just use the shell,
for file in myfile*
do
t=${file#*_}
f=${file%%_*}
number=$(printf "%04d" ${t%%_*})
newfile="${f}_${number}_${t#*_}"
echo mv "$file" "$newfile"
done
There's a UNIX app called ren (manpage) which supports renaming multiple files using search and substitution patterns. You should be able to cobble together a pattern that will inject that extra 0 into the filename.
Edit: Project page w/ download link can be found at Freshmeat.
Try:
for file in `ls my*`
do
a=`echo $file | cut -d_ -f1`
b=`echo $file | cut -d_ -f2`
c=`echo $file | cut -d_ -f3,4`
new=${a}_0${b}_${c}
mv $file $new
done