I have thousands of files, I want to remove part of files name.
For example:
Agu-1_9942_quant.sf
Aiell-1_9646_quant.sf
Should be converted to
9942_quant.sf
9646_quant.sf
I am trying rename function (rename 's/_//g' *.sf) but unable to get desired results. Any help will be highly appreciated
Assuming this is bash scripting:
for elem in ./*
do
if echo $elem | grep '_'; then
elem_new=$(echo $elem | cut -d '_' -f 1 --complement)
mv $elem $elem_new
fi
done
Related
In my current directory there are many files. Some of the files share part of their filename.
e.g.:
XGAE_537493_GSR.FITS
TGFE_537493_RRF.FITS
EGRE_537497_HDR.FITS
TRTE_537497_YUH.FITS
TRXX_537499_YDF.FITS
.
.
Files 1 & 2 would be a match, as would files 3 & 4. File 5 has no match. Therefore, files 1,2,3 and 4 would be moved.
I want to move the files which share part of their filename, in order to separate them from the ones that don't.
I was attempting to do this using bash. I googled but couldn't locate websites that were quite describing the process I need. So far in pseudo-code I have:
FOR F IN *
IF ${FILE:5:10} MATCHES ANY OTHER ${FILE:5:10}
MOVE ALL MATCHES TO ANOTHER DIRECTORY
Any information to help me move in the right direction would be appreciated.
Try this:
for f in ./*.FITS ; do
middleBit=$(echo $f| cut -d'_' -f 1)
count=$(ls *middleBit*.FITS | wc -l)
if [ $count -ge 1 ]
then
for match in *middleBit*.FITS ; do
mv $match ./somewhere
done
fi
done
Using associative array in BASH 4 you can do it easily:
#!/bin/bash
declare -A arr
for f in *.FITS; do
k="${f:5:6}"
[[ ${arr[$k]} ]] && mv "$f" /dest/ || arr["$k"]=1
done
if your file structure is fixed, you can scan them and find duplicates in sub fields of the file name in awk.
for example
$ ls -1 | awk -F_ 'NF==3{f[$2]=(a[$2]++?f[$2] OFS $0:$0)}
END{for(k in f) if(a[k]>1) print f[k]} '
TGFE_537493_RRF.FITS
XGAE_537493_GSR.FITS
you can then pipe the results to a cp command
$ ... | xargs -I file cp file file.DUP
adds suffix DUP to duplicate file names, or
$ ... | xargs -I file mv file anotherlocation/
moves to anotherlocation.
I have a file input.txt that looks as follows.
abas_1.txt
abas_2.txt
abas_3.txt
1fgh.txt
3ghl_1.txt
3ghl_2.txt
I have a folder ff. The filenames of this folder are abas.txt, 1fgh.txt, 3ghl.txt. Based on the input file, I would like to create and rename the multiple copies in ff folder.
For example in the input file, abas has three copies. In the ff folder, I need to create the three copies of abas.txt and rename it as abas_1.txt, abas_2.txt, abas_3.txt. No need to copy and rename 1fgh.txt in ff folder.
Your valuable suggestions would be appreciated.
You can try something like this (to be run from within your folder ff):
#!/bin/bash
while IFS= read -r fn; do
[[ $fn =~ ^(.+)_[[:digit:]]+\.([^\.]+)$ ]] || continue
fn_orig=${BASH_REMATCH[1]}.${BASH_REMATCH[2]}
echo cp -nv -- "$fn_orig" "$fn"
done < input.txt
Remove the echo if you're happy with it.
If you don't want to run from within the folder ff, just replace the line
echo cp -nv -- "$fn_orig" "$fn"
with
echo cp -nv -- "ff/$fn_orig" "ff/$fn"
The -n option to cp so as to not overwrite existing files, and the -v option to be verbose. The -- tells cp that there are no more options beyond this point, so that it will not be confused if one of the files starts with a hyphen.
using for and grep :
#!/bin/bash
for i in $(ls)
do
x=$(echo $i | sed 's/^\(.*\)\..*/\1/')"_"
for j in $(grep $x in)
do
cp -n $i $j
done
done
Try this one
#!/bin/bash
while read newFileName;do
#split the string by _ delimiter
arr=(${newFileName//_/ })
extension="${newFileName##*.}"
fileToCopy="${arr[0]}.$extension"
#check for empty : '1fgh.txt' case
if [ -n "${arr[1]}" ]; then
#check if file exists
if [ -f $fileToCopy ];then
echo "copying $fileToCopy -> $newFileName"
cp "$fileToCopy" "$newFileName"
#else
# echo "File $fileToCopy does not exist, so it can't be copied"
fi
fi
done
You can call your script like this:
cat input.txt | ./script.sh
If you could change the format of input.txt, I suggest you adjust it in order to make your task easier. If not, here is my solution:
#!/bin/bash
SRC_DIR=/path/to/ff
INPUT=/path/to/input.txt
BACKUP_DIR=/path/to/backup
for cand in `ls $SRC_DIR`; do
grep "^${cand%.*}_" $INPUT | while read new
do
cp -fv $SRC_DIR/$cand $BACKUP_DIR/$new
done
done
I have created a list of files using find, foundlist.lst.
The find command is simply find . -type f -name "<search_pattern>" > foundlist.lst
I would now like to use this list to find copies of these files in other directories.
The 'twist' in my requirements is that I want to search only for the 'base' of the file name. I don't want to include the extension in the search.
Example:
./sort.cc is a member of the list. I want to look for all files of the pattern sort.*
Here is what I wrote. It works. It seems to me that there is a more efficient way to do this.
./findfiles.sh foundfiles.lst /usr/bin/temp
#!/bin/bash
# findfiles.sh
if [ $# -ne 2 ]; then
echo "Need two arguments"
echo "usage: findfiles <filelist> <dir_to_search>"
else
filename=$1
echo "$filename"
while read -r line; do
name=$line
# change './file.ext' to 'file.*'
search_base=$( echo ${name} | sed "s%\.\/%%" | sed "s/\..*/\.\*/" )
find $2 -type f -name $search_base
done < $filename
fi
For stripping the file, I'd use the following (instead of awk)
search_base=`basename ${name} | cut -d'.' -f1`
I have a list of files like so :
10_I_am_here_001.jpg
20_I_am_here_003.jpg
30_I_am_here_008.jpg
40_I_am_here_004.jpg
50_I_am_here_009.jpg
60_I_am_here_002.jpg
70_I_am_here_005.jpg
80_I_am_here_006.jpg
How can I rename all the files in a directory, so that I can drop ^[0-9]+_ from the filename ?
Thank you
Using pure BASH:
s='10_I_am_here_001.jpg'
echo "${s#[0-9]*_}"
I_am_here_001.jpg
You can then write a simple for loop in that directory like this:
for s in *; do
f="${s#[0-9]*_}" && mv "$s" "$f"
done
Using rename :
rename 's/^[0-9]+_//' *
Here's another bash idea based on files ending .jpg as shown above or whatever>
VonBell
#!/bin/bash
ls *.jpg |\
while read FileName
do
NewName="`echo $FileName | cut -f2- -d "_"`"
mv $FileName $NewName
done
With bash extended globbing
shopt -s extglob
for f in *
do
[[ $f == +([0-9])_*.jpg ]] && mv "$f" "${f#+([0-9])_}"
done
I'm trying to take a list of files and rename them, incrementing a number in their filenames. The directory contains a bunch of files named like:
senreg1.csv senreg2.csv senreg10.csv
senreg1.csv.1 senreg2.csv.1 senreg10.csv.1
senreg1.csv.2 senreg2.csv.2 senreg10.csv.2
senreg1.csv.3 senreg2.csv.3 ... senreg10.csv.3
senreg1.csv.4 senreg2.csv.4 senreg10.csv.4
... ... ...
senreg1.csv.10 senreg2.csv.10 senreg10.csv.10
senreg1.csv.11 senreg2.csv.11 senreg10.csv.11
I want to increment all of the files that end in 3 or higher so I can insert a new file with suffix 3, so I made a text file called 'renames.txt' containing all the filenames that I want to rename. Then, I tried using a for loop to do the actual renaming.
for f in `cat renames.txt`
do
newfile=`echo $f | awk 'BEGIN { FS = "."}; { printf $1 "." $2 "." $3+1 }'`
mv "$f" "$newfile"
done
I want to end up with something like:
senreg1.csv senreg2.csv senreg10.csv
senreg1.csv.1 senreg2.csv.1 senreg10.csv.1
senreg1.csv.2 senreg2.csv.2 senreg10.csv.2
senreg1.csv.4 senreg2.csv.4 ... senreg10.csv.4
senreg1.csv.5 senreg2.csv.5 senreg10.csv.5
... ... ...
senreg1.csv.11 senreg2.csv.11 senreg10.csv.11
senreg1.csv.12 senreg2.csv.12 senreg10.csv.12
But instead I get:
senreg1.csv senreg2.csv senreg10.csv
senreg1.csv.1 senreg2.csv.1 senreg10.csv.1
senreg1.csv.2 senreg2.csv.2 ... senreg10.csv.2
senreg1.csv.12 senreg2.csv.12 senreg10.csv.12
The contents of senregX.csv.12 are the same as the original senregX.csv.3. Hope this explanation made sense. Anybody know what's going on here?
You need to rename the files in reverse.
11 -> 12
10 -> 11
9 -> 10
and so on.
This script do what you want, without temporary files, only to have diversity of solutions:
#!/bin/bash
for file in $(ls -1 *[0-9]) # list files ending with a number
do
# get file name and id
name=$(echo $file | sed 's/\(.*\)\.\([0-9]\+\)$/\1/g');
id=$(echo $file | sed 's/.*\.\([0-9]\+\)$/\1/g');
if [ $id -ge 3 ]
then
((id += 1))
# We need to backup the files because we may override some files
cp $file "_$name.$id"
fi
done
# remove old files
for file in $(ls -1 [!_]*[0-9])
do
id=$(echo $file | sed 's/.*\.\([0-9]\+\)$/\1/g');
if [ $id -ge 3 ]
then
rm $file;
fi
done
# finish
for file in $(ls -1 _*[0-9])
do
name=$(echo $file | tr -d '_');
mv "$file" "$name";
done