I am trying to write a bash script to create some playlists of music. The part that has me stuck is the while loop for read line. I figure I am over thinking this so I turned to stackoverflow for assistance.
# The first while loop is how many playlists I want to create
i=1
while [ $i -le $plist ]
do
echo -e "iteration $i"
i=$[$i + 1]
z=0
# This while loop is for the length of time I want the playlist to be
while [ $z -le $TOTAL ]
do
echo -e "Count $z"
z=$[$z + xxx]
# This while loop is for reading the track list previously generated.
# It would read the line, calculate the track length,
# add to $z, cp the track to a folder
while read line
do
secs=$(metaflac --show-total-samples --show-sample-rate "$line" | tr '\n' ' '
| awk '{print $1/$2}' -)
z=$[$z + $secs]
cp $line to destination folder
done
done
done
Related
I have issue with an if statement. In WEDI_RC is saved log file in the following format:
name_of_file date number_of_starts
I want to compare first argument $1 with first column and if it is true than increment number of starts. When I start my script it works but just with one file, eg:
file1.c 11:23:07 1
file1.c 11:23:14 2
file1.c 11:23:17 3
file1.c 11:23:22 4
file2.c 11:23:28 1
file2.c 11:23:35 2
file2.c 11:24:10 3
file2.c 11:24:40 4
file2.c 11:24:53 5
file1.c 11:25:13 1
file1.c 11:25:49 2
file2.c 11:26:01 1
file2.c 11:28:12 2
Every time when I change file it begin counts from 1. I need to continue with counting when it ends.
Hope you understand me.
while read -r line
do
echo "line:"
echo $line
if [ "$1"="$($line | grep ^$1)" ]; then
number=$(echo $line | grep $1 | awk -F'[ ]' '{print $3}')
else
echo "error"
fi
done < $WEDI_RC
echo "file"
((number++))
echo $1 `date +"%T"` $number >> $WEDI_RC
There are at least two ways to resolve the problem. The most succinct is probably:
echo "$1 $(date +"%T") $(($(grep -c "^$1 " "$WEDI_RC") + 1))" >> "$WEDI_RC"
However, if you want to have counts for each file separately, you can do that using an associative array, assuming you have Bash version 4.x (not 3.x as is provided on Mac OS X, for example). This code assumes the file is correctly formatted (so that the counts do not reset to 1 each time the file name changes).
declare -A files # Associative array
while read -r file time count # Split line into three variables
do
echo "line: $file $time $count" # One echo - not two
files[$file]="$count" # Record the current maximum for file
done < "$WEDI_RC"
echo "$1 $(date +"%T") $(( ${files[$1]} + 1 ))" >> "$WEDI_RC"
The code uses read to split the line into three separate variables. It echoes what it read and records the current count. When the loop's done, it echoes the data to append to the file. If the file is new (not mentioned in the file yet), then you will get a 1 added.
If you need to deal with the broken file as input, then you can amend the code to count the number of entries for a file, instead of trusting the count value. The bare-array reference notation used in the (( … )) operation is necessary when incrementing the variable; you can't use ${array[sub]}++ with the increment (or decrement) operator because that evaluates to the value of the array element, not its name!
declare -A files # Associative array
while read -r file time count # Split line into three variables
do
echo "line: $file $time $count" # One echo - not two
((files[$file]++)) # Count the occurrences of file
done < "$WEDI_RC"
echo "$1 $(date +"%T") $(( ${files[$1]} + 1 ))" >> "$WEDI_RC"
You can even detect whether the format is in the broken or fixed style:
declare -A files # Associative array
while read -r file time count # Split line into three variables
do
echo "line: $file $time $count" # One echo - not two
if [ $((files[$file]++)) != "$count" ]
then echo "$0: warning - count out of sync: ${files[$file]} vs $count" >&2
fi
done < "$WEDI_RC"
echo "$1 $(date +"%T") $(( ${files[$1]} + 1 ))" >> "$WEDI_RC"
I don't get exactly what you want to achieve with your test [ "$1"="$($line | grep ^$1)" ] but it seems you are checking that the line start with the first argument.
If it is so, I think you can either:
provide the -o option to grep so that it print just the matched output (so $1)
use [[ "$line" =~ ^"$1" ]] as test.
I am making one script to delete all printer jobs older than one day and no of jobs older than one day has to be print also the jobid which has been canceled
Below are few conditions which I have to follow
I do not have access to cups directory , I can not create any temporary file .
I have to use Bash shell.
I tried to use a variable, declared outside while loop but the variable value remains unchanged ~came to know it is because of child process
Tried to use export variable but that also does not work in Bash shell , same is working in ksh shell.
I have tried below logic:
count=0
currDate=`date +%Y%m%d`
lpstat -o|while read line
do
jobid=`echo $line|awk '{print $1}'|cut -d"-" -f2`
jobDate=`echo $line|awk -F ' ' 'BEGIN{OFS="-";} {print $5,$6,$7;}'`
formattedDate=`date -d"${jobDate}" +%Y%m%d`
if [ `expr $currDate - $formattedDate` -gt 1 ]
then
count=`expr $count + 1`
echo " cancelling printer job with jobid $jobid "
cancel $jobid
fi;
done
[ $count -gt 0 ] ; echo "NO of Printer jobs pending more than 1 days are $count";
Not getting how to handle the count variable as the above mentioned way it will not work,
Export is not working in bash shell , creating tmp file is not allowed.
Any suggestion to get the solution.
The problem with count is that you process it inside a new bash child process (bash creates it automatically when you pipe input into while). This is why you get back the initial value.
The solution for you would be to output the value in that child process. This should work (braces are forcing bash to create a child process that includes the output part):
count=0
currDate=`date +%Y%m%d`
lpstat -o| (while read line
do
jobid=`echo $line|awk '{print $1}'|cut -d"-" -f2`
jobDate=`echo $line|awk -F ' ' 'BEGIN{OFS="-";} {print $5,$6,$7;}'`
formattedDate=`date -d"${jobDate}" +%Y%m%d`
if [ `expr $currDate - $formattedDate` -gt 1 ]
then
count=`expr $count + 1`
echo " cancelling printer job with jobid $jobid "
cancel $jobid
fi;
done
[ $count -gt 0 ] && echo "NO of Printer jobs pending more than 1 days are $count")
The simplest solution is to reference the variable in the same shell you are setting it. Syntactically, the easiest thing to do is to add a block:
count=0
currDate=`date +%Y%m%d`
lpstat -o| { while read line # Add opening brace here
do
jobid=`echo $line|awk '{print $1}'|cut -d"-" -f2`
jobDate=`echo $line|awk -F ' ' 'BEGIN{OFS="-";} {print $5,$6,$7;}'`
formattedDate=`date -d"${jobDate}" +%Y%m%d`
if [ `expr $currDate - $formattedDate` -gt 1 ]
then
count=`expr $count + 1`
echo " cancelling printer job with jobid $jobid "
cancel $jobid
fi;
done
echo "Number of Printer jobs pending more than 1 day: $count"
} # Add closing brace here
This is not a code review site, but I feel compelled to address other elements of this code. echo $line | awk '{print $1}' | cut -d"-" -f2 is an atrocity. Rather than reading in an entire line and then parsing it with echo/awk/cut, let read do the parsing for you by assigning IFS appropriately. Stop using backticks. $() notation is better. The last line of this code [ $count -gt 0 ]; echo ... completely ignores the evaluation of $count. It would be better written [ $count -gt 0 ] || echo ... >&2 or test "$count" = 0 && echo ... >&2 (redirect the error message to file descriptor 2, since error belong on stderr).
I'm trying to write a small script that will count entries in a log file, and I'm incrementing a variable (USCOUNTER) which I'm trying to use after the loop is done.
But at that moment USCOUNTER looks to be 0 instead of the actual value. Any idea what I'm doing wrong? Thanks!
FILE=$1
tail -n10 mylog > $FILE
USCOUNTER=0
cat $FILE | while read line; do
country=$(echo "$line" | cut -d' ' -f1)
if [ "US" = "$country" ]; then
USCOUNTER=`expr $USCOUNTER + 1`
echo "US counter $USCOUNTER"
fi
done
echo "final $USCOUNTER"
It outputs:
US counter 1
US counter 2
US counter 3
..
final 0
You are using USCOUNTER in a subshell, that's why the variable is not showing in the main shell.
Instead of cat FILE | while ..., do just a while ... done < $FILE. This way, you avoid the common problem of I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?:
while read country _; do
if [ "US" = "$country" ]; then
USCOUNTER=$(expr $USCOUNTER + 1)
echo "US counter $USCOUNTER"
fi
done < "$FILE"
Note I also replaced the `` expression with a $().
I also replaced while read line; do country=$(echo "$line" | cut -d' ' -f1) with while read country _. This allows you to say while read var1 var2 ... varN where var1 contains the first word in the line, $var2 and so on, until $varN containing the remaining content.
Always use -r with read.
There is no need to use cut, you can stick with pure bash solutions.
In this case passing read a 2nd var (_) to catch the additional "fields"
Prefer [[ ]] over [ ].
Use arithmetic expressions.
Do not forget to quote variables! Link includes other pitfalls as well
while read -r country _; do
if [[ $country = 'US' ]]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
minimalist
counter=0
((counter++))
echo $counter
You're getting final 0 because your while loop is being executed in a sub (shell) process and any changes made there are not reflected in the current (parent) shell.
Correct script:
while read -r country _; do
if [ "US" = "$country" ]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
I had the same $count variable in a while loop getting lost issue.
#fedorqui's answer (and a few others) are accurate answers to the actual question: the sub-shell is indeed the problem.
But it lead me to another issue: I wasn't piping a file content... but the output of a series of pipes & greps...
my erroring sample code:
count=0
cat /etc/hosts | head | while read line; do
((count++))
echo $count $line
done
echo $count
and my fix thanks to the help of this thread and the process substitution:
count=0
while IFS= read -r line; do
((count++))
echo "$count $line"
done < <(cat /etc/hosts | head)
echo "$count"
USCOUNTER=$(grep -c "^US " "$FILE")
Incrementing a variable can be done like that:
_my_counter=$[$_my_counter + 1]
Counting the number of occurrence of a pattern in a column can be done with grep
grep -cE "^([^ ]* ){2}US"
-c count
([^ ]* ) To detect a colonne
{2} the colonne number
US your pattern
Using the following 1 line command for changing many files name in linux using phrase specificity:
find -type f -name '*.jpg' | rename 's/holiday/honeymoon/'
For all files with the extension ".jpg", if they contain the string "holiday", replace it with "honeymoon". For instance, this command would rename the file "ourholiday001.jpg" to "ourhoneymoon001.jpg".
This example also illustrates how to use the find command to send a list of files (-type f) with the extension .jpg (-name '*.jpg') to rename via a pipe (|). rename then reads its file list from standard input.
Script:
#!/bin/bash
IFS=','
i=0
for j in `cat database | head -n 1`; do
variables[$i]=$j
i=`expr $i + 1`
done
k=0
for l in `cat database | tail -n $(expr $(cat database | wc -l) - 1)`; do
echo -n $k
k=`expr $k + 1`
if [ $k -eq 3 ]; then
k=0
fi
done
Input file
a,b,c
d,e,f
g,e,f
Output
01201
Expected output
012012
The question is why the for skips last echo? It is weird, because if I change $k to $l echo will run 6 times.
Update:
#thom's analysis is correct. You can fix the problem by changing IFS=',' to IFS=$',\n'.
My original statements below may be of general interest, but do not address the specific problem.
If accidental shell expansions were a concern, here's how the loop could be rewritten (assuming it's practical to read everything into an array variable first):
IFS=$',\n' read -d '' -r -a fields < <(echo $'*,b,c\nd,e,f\ng,h,i')
for field in "${fields[#]}"; do
# $field is '*' in 1st iteration, then 'b', 'c', 'd',...
done
Original statements:
Just a few general pointers:
You should use a while loop rather than for to read command output - see http://mywiki.wooledge.org/BashFAQ/001; the short of it: with for, the input lines are subject to various shell expansions.
A missing iteration typically stems from the last input line missing a terminating \n (or a separator as defined in $IFS). With a while loop, you can use the following approach to address this: while read -r line || [[ -n $line ]]; do …
For instance, your 2nd for loop could be rewritten as (using process substitution as input to avoid creating a subshell with a separate variable scope):
while read -r l || [[ -n $l ]]; do …; done < <(cat database | tail -n $(expr $(cat database | wc -l) - 1))
Finally, you could benefit from using modern bashisms: for instance,
k=`expr $k + 1`
could be rewritten much more succinctly as (( ++k )) (which will run faster, too).
Your code expects after EVERY read variable a comma but you only give this:
a,b,c
d,e,f
g,e,f
instead of this:
a,b,c,
d,e,f,
g,e,f,
so it reads:
d,e,f'\n'g,e,f
and that is equal to 5 values, not 6
I am using while loop to read xyz.txt file and file which contains contents like below:
2 - info1
4 - info2
6 - info3
9 - info4
Further I am using if condition to match the count -gt then y value so it will send an email. The problem I am facing every time it matches the if condition it is sending an email which I want once, it should read the file till end and if condition matches store the next line output to a file and then send that file with all information. At present I am receiving number of email.
Hope my question is clear I think I am looking for return function once condition matches it continue reading file till the end and store the info.
count=`echo $line | awk '{print $3}'`
cnt=o
while read line
do
if [ "$count" -gt "$x" ]; then ---> This logic is working fine
cnt=$(( $cnt + 1)) --- > This logic is working fine
echo $line > info.txt -----> In info.txt I want to store info in 1 go which ever matches condition.
export info.txt=$info.txt
${PERL_BIN}/perl $send_mail
fi
done < file.txt
If you only want to send email once, don't put the invocation of Perl which sends mail inside the loop; put it outside the loop (after the end of the loop). Use append (>>) to build the file up piecemeal.
count=`echo $line | awk '{print $3}'`
cnt=0 # 0 not o!
while read line
do
if [ "$count" -gt "$x" ]; then
cnt=$(($cnt + 1))
echo $line >> info.txt
fi
done < file.txt
if [ $cnt -gt 0 ]
then
export info_txt=$info.txt
${PERL_BIN}/perl $send_mail
fi
Okay. I've tried to grasp what you want, I think it is this:
First, before the loop, remove any old info.txt file.
rm info.txt
Then, each time through the loop, append new lines to it like so:
echo $line >> info.txt
Notice the double arrows >>. This means append, instead of overwrite.
Finally, do the email sending after the loop.