rename numbering within filename using shell - bash

My files have the following pattern:
a0015_random_name.txt
a0016_some_completely_different_name.txt
a0017_and_so_on.txt
...
I would like to rename only the numbering using the shell, so that they are going two numbers down:
a0015_random_name.txt ---> a0013_random_name.txt
a0016_some_completely_different_name.txt ---> a0014_some_completely_different_name.txt
a0017_and_so_on.txt ---> a0015_and_so_on.txt
I've tried already this:
let n=15; for i in *.txt; do let n=n-2; b=`printf a00`$n'*'.txt; echo "mv $i $b"; done
(I use echo first, in order to see what would happen)
but this gave me:
mv a0015_random_name.txt a0013*.txt
mv a0016_some_completely_different_name.txt a0014*.txt
mv a0017_and_so_on.txt a0015*.txt
Also I've tried to find the command, which would set the rest of the name right, but I couldn't find it. Does someone know it, or have a better idea how to do this?

Your code is almost correct. Try this:
let n=15; for i in *.txt; do let n=n-2; b=`echo $i | sed "s/a[0-9]*/a$n/g`; echo "mv $i $b"; done
Better yet, to make it more robust, use the following modification:
let n=15; for i in *.txt; do let t=n-2; let n=n+1; b=`echo $i | sed "s/a00$n/a00$t/g`; echo "mv $i $b"; done

If you have the Perl rename.pl script, this is a one-liner:
rename 's/\d+/sprintf "%0${\(length $&)}d", $&-2/e' *.txt
Otherwise, it's a bit wordier. Here's one way:
for f in *.txt; do
number=$(expr "$f" : '^[^0-9]*\([0-9]*\)') # extract the first number from the filename
prefix=${f%%$number*} # remember the part before
suffix=${f#*$number} # and after the number
let n=10#$number-2 # subtract 2
nf=$(printf "%s%0${#number}d%s" \
"$prefix" "$n" "$suffix") # build new filename
echo "mv '$f' '$nf'" # echo the rename command
# mv "$f" "$nf" # uncomment to actually do the rename
done
Note the 10# on the let line - that forces the number to be interpreted in base 10 even if it has leading zeroes, which would otherwise cause it to be interpreted in base 8. Also, the %0${#number}d format tells printf to format the new number with enough leading zeroes to be the same length as the original number.
On your example, the above script produces this output:
mv 'a0015_random_name.txt' 'a0013_random_name.txt'
mv 'a0016_some_completely_different_name.txt' 'a0014_some_completely_different_name.txt'
mv 'a0017_and_so_on.txt' 'a0015_and_so_on.txt'

Related

How to split large *.csv files with headers in Bash?

I need split big *.csv file for several smaller. Currently there is 661497 rows, I need each file with max. 40000. I've tried solution that I found on Github but with no success:
FILENAME=/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/products-files/my_file.csv
HDR=$(head -1 ${FILENAME})
split -l 40000 ${FILENAME} xyz
n=1
for f in xyz*
do
if [[ ${n} -ne 1 ]]; then
echo ${HDR} > part-${n}-${FILENAME}.csv
fi
cat ${f} >> part-${n}-${FILENAME}.csv
rm ${f}
((n++))
done
The error I get:
/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/download.sh: line 23: part-1-/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/products-files/my_file.csv.csv: No such file or directory
thanks for help!
Keep in mind FILENAME contains both a directory and a file so later in the script when you build the new filename you get something like:
part-1-/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/products-files/tyre_8.csv.csv
One quick-n-easy fix would be split the directory and filename into 2 separate variables, eg:
srcdir='/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/products-files'
filename='tyre_8.csv'
hdr=$(head -1 ${srcdir}/${filename})
split -l 40000 "${srcdir}/${filename}" xyz
n=1
for f in xyz*
do
if [[ ${n} -ne 1 ]]; then
echo ${hdr} > "${srcdir}/part-${n}-${filename}"
fi
cat "${f}" >> "${srcdir}/part-${n}-${filename}"
rm "${f}"
((n++))
done
NOTES:
consider using lowercase variables (using uppercase variables raises the possibility of problems if there's an OS variable of the same name)
wrap variable references in double quotes in case string contains spaces
don't need to add a .csv extension on the new filename since it's already part of $filename

Replace numbers in a file name in unix (BASH)

I have multiple files approximately 150 and there names do not match a requirement of a vendor. example file names are:
company_red001.p12
company_red002.p12
.
.
.
.
company_red150.p12
I need to rename all files so that 24 is added to each number sequentially and that there are no preceding zero's and that the company_ component is removed.
red25.p12
red26.p12
red27.p12
.
.
.
red150.p12
I have used a for loop in bash to remove the company_ component but would like something that executes all changes simultaneously as I have to perform this at a moments notice.
example:
#!/bin/bash
n = 24
for file in company_red*
do
new_name=$file$n
n=$(($+1))
mv -i $file $new_name
done
example 2
#!/bin/bash
for f in company_red*
do mv "$f" "${f/company_red/red}";
done
Most probably this one could be fine :)
# printf is used to emulate a lot of files
for f in $( printf "company_red%03d.p12\n" {1..150} )
do
# get the filename
n="$f"
# remove extension
n="${n%.*}"
# remove leading letters
n="${n##*[[:alpha:]]}"
# add 24, 10# is used to consider the 10-based number
n="$(( 10#$n + 24 ))"
# construct new filename
g="red${n}.p12"
echo mv "$f" "$g"
done
And this could be simplified a bit
for f in $( printf "company_red%03d.p12\n" {1..150} )
do
# take the number from the specific, fixed position
n="${f:11:3}"
# everything below is the same as in the previous example
n="$(( 10#$n + 24 ))"
g="red${n}.p12"
echo mv "$f" "$g"
done
And finally, this could be simplified yet twice -- just escape of using $n and $g:
for f in $( printf "company_red%03d.p12\n" {1..150} )
do
echo mv "$f" "red$(( 10#${f:11:3} + 24 )).p12"
done
But this could complicate understanding and supporting of the code.
Do:
for file in *.p12; do
name=${file#*_} ## Extracts the portion after `_` from filename, save as variable "name"
pre=${name%.*} ## Extracts the portion before extension, save as "pre"
num=${pre##*[[:alpha:]]} ## Extracts number from variable "pre"
pre=${pre%%[0-9]*} ## Extracts the alphabetic portion from variable "pre"
suf=${name##*.} ## Extracts the extension from variable "name"
echo mv -i "$file" "${pre}""$(printf '%d' $((10#$num+24)))"."${suf}" ## Doing arithmetic expansion for addition, and necessary formatting to get desired name
done
Outputs:
mv -i company_red001.p12 red25.p12
mv -i company_red002.p12 red26.p12
The above is dry-run, remove echo if you are satisfied with the renaming to be done:
for file in *.p12; do
name=${file#*_}
pre=${name%.*}
num=${pre##*[[:alpha:]]}
pre=${pre%%[0-9]*}
suf=${name##*.}
mv -i "$file" "${pre}""$(printf '%d' $((10#$num+24)))"."${suf}"
done

How do you name output files using an increment after a bash file loop?

I'm trying to treat a bunch of files (five) with an awk command and name the output files using an incrementation.
The input files have complicated names. I know how to reuse the files' basenames to rename the outputs but I want to simplify the file names.
this is my code:
for f in *.txt; do
for i in {1..5}; do
echo processing file $f
awk '
{ if ($1=="something" && ($5=="60" || $5=="61"|| $5=="62"|| $5=="63"|| $5=="64"|| $5=="65"|| $5=="66"|| $5=="67"|| $5=="68"|| $5=="69"|| $5=="70"))
print }' $b.txt>"file_treated"$i.txt
echo processing file $f over
done
done
I understand that the error is in the second line because what I wrote runs the second loop for each value of the first one. I want each value of the first loop to correspond to one value of the second one.
Hope this was clear enough
How about:
i=0
for f in *.txt; do
let i++;
awk '$1=="something" && ($5 >= 60 && $5 <=70)' "$f" > file_treated"${i}".txt
done
I simplified your awk command and straightened out your various quoting issues. I also removed the $b.txt since you were simply recreating $f. I left the echo $b etc in case you actually wanted that but it could just as easily be replaced with echo "$f".
Use a counter:
i=1
for f in *.txt
do
echo "$f is number $((i++))"
done

Why String concatenation of extracted file names does not work?

I have a file "a.txt", that contains file names with paths.
a.txt:
/root/chan/properties.lo
/root/attributes.cc
/root/chan/eagle/bath.ear
I would like to extract these file names and put them in one variable this way:
#!/bin/bash
for i in $(cat a.txt); do
o+=$(basename $i)
done
echo $o
But it does not work.
I am geting:
feedBackMailConfiguration.xmltiess
Please, help.
while read -r i;
do
o=$o" "$(basename $i)
done < a.txt
echo $o
The above will do it.
(edits for copy and paste errors)
More edits: just tried this (terminator, ubuntu) and it give the right result:
while read -r i; do o="$o $(basename $i)";done < a.txt
You just need to add a space concatenation to your script, like this:
#!/bin/bash
for i in $(cat a.txt); do
# Check the " " at the end of the following line
o+=$(basename $i)" "
done
echo $o
And it would work
"o+=..." is not really portable (which version of bash allows it? I never saw it until your mention of it? It looks like perl mechanisms)
Edit2 : following vladintok remarks on what the output is, I add DEBUG infos to try to pinpoints the problem
Edit3 : another step of trying to take out of the way every possible explanation for the weird output reported... I unalias, and unset any function named "o" and "basename" ...
Try:
#!/bin/bash
unalias o 2>/dev/null
unalias basename 2>/dev/null
unset o 2>/dev/null
unset basename 2>/dev/null
for i in $(cat a.txt); do
printf 'DEBUG: i is: %s\n' "$i"
o="${o} $(basename $i)"
done
printf 'DEBUG: final o is: %s\n' "$o"
echo $o
#end of the script.
(of course the above is a SINGLE file. Name it 'test.bash", make it executable: chmod +x test.bash, and then execute it : ./test.bash)
Edit1: I corrected the o=$o" "$(basename $i) into o="${o} $(basename $i)" as pointed out by Pantoine. Otherwise, toto=$toto" "titi : would assign to toto everything until the first space (which will be somewhere inside $toto, after the first iteration or if a basename contained spaces)...

How sort recursively by maximum file size and count files?

I'm beginner in bash programming. I want to display head -n $1 results of sorting files
by size in /etc/*. The problem is that at final search, I must know how many directories and files has processed.
I compose following code:
#!/bash/bin
let countF=0;
let countD=0;
for file in $(du -sk /etc/* |sort +0n | head $1); do
if [ -f "file" ] then
echo $file;
let countF=countF+1;
else if [ -d "file" ] then
let countD=countD+1;
fi
done
echo $countF
echo $countD
I have errors at execution. How use find with du, because I must search recursively?
#!/bin/bash # directory and program reversed
let countF=0 # semicolon not needed (several more places)
let countD=0
while read -r file; do
if [ -f "$file" ]; then # missing dollar sign and semicolon
echo $file
let countF=countF+1 # could also be: let countF++
else if [ -d "$file" ]; then # missing dollar sign and semicolon
let countD=countD+1
fi
done < <(du -sk /etc/* |sort +0n | head $1) # see below
echo $countF
echo $countD
Changing the loop from a for to a while allows it to work properly in case filenames contain spaces.
I'm not sure what version of sort you have, but I'll take your word for it that the argument is correct.
It's #!/bin/bash not #!/bash/bin.
I don't know what that argument to sort is supposed to be. Maybe you meant sort -r -n?
Your use of head is wrong. Giving head file arguments causes it to ignore its standard input, so in general it's an error to both pipe something to head and give it a file argument. Besides that, "$1" refers to the script's first argument. Did you maybe mean head -n 1, or were you trying to make the number of lines processed configurable from an argument to the script: head -n"$1".
In your if tests, you're not referencing your loop variable: it should read "$file", not "file".
Not that the bash parser cares, but you should try to indent sanely.
#!/bin/bash # directory and program reversed
let countF=0 # semicolon not needed (several more places)
let countD=0
while read -r file; do
if [ -f "$file" ]; then # missing dollar sign and semicolon
echo $file
let countF=countF+1 # could also be: let countF++
else if [ -d "$file" ]; then # missing dollar sign and semicolon
let countD=countD+1
fi
done < <(du -sk /etc/* |sort +0n | head $1) # see below
echo $countF
echo $countD
I tried instead of file variable the /etc/* but I don't see a result. the idea is to sort all files by size from a directories and subdirectories and display $1 results ordered by
size of the files. In this process I must know how many files and dirs contains the directory where
I did the search.
Ruby(1.9+)
#!/usr/bin/env ruby
fc=0
dc=0
a=Dir["/etc/*"].inject([]) do |x,f|
fc+=1 if File.file?(f)
dc+=1 if File.directory?(f)
x<<f
end
puts a.sort
puts "number of files: #{fc}"
puts "number of directories: #{dc}"

Resources