I am using while loop to read xyz.txt file and file which contains contents like below:
2 - info1
4 - info2
6 - info3
9 - info4
Further I am using if condition to match the count -gt then y value so it will send an email. The problem I am facing every time it matches the if condition it is sending an email which I want once, it should read the file till end and if condition matches store the next line output to a file and then send that file with all information. At present I am receiving number of email.
Hope my question is clear I think I am looking for return function once condition matches it continue reading file till the end and store the info.
count=`echo $line | awk '{print $3}'`
cnt=o
while read line
do
if [ "$count" -gt "$x" ]; then ---> This logic is working fine
cnt=$(( $cnt + 1)) --- > This logic is working fine
echo $line > info.txt -----> In info.txt I want to store info in 1 go which ever matches condition.
export info.txt=$info.txt
${PERL_BIN}/perl $send_mail
fi
done < file.txt
If you only want to send email once, don't put the invocation of Perl which sends mail inside the loop; put it outside the loop (after the end of the loop). Use append (>>) to build the file up piecemeal.
count=`echo $line | awk '{print $3}'`
cnt=0 # 0 not o!
while read line
do
if [ "$count" -gt "$x" ]; then
cnt=$(($cnt + 1))
echo $line >> info.txt
fi
done < file.txt
if [ $cnt -gt 0 ]
then
export info_txt=$info.txt
${PERL_BIN}/perl $send_mail
fi
Okay. I've tried to grasp what you want, I think it is this:
First, before the loop, remove any old info.txt file.
rm info.txt
Then, each time through the loop, append new lines to it like so:
echo $line >> info.txt
Notice the double arrows >>. This means append, instead of overwrite.
Finally, do the email sending after the loop.
Related
I am trying to write this code so that if the process reads map finished in the pipe it increments a variable by 1 so that it eventually breaks out of the while loop. Otherwise it will add unique parameters to a keys file. However it goes into an infinite loop and never breaks out of the loop.
while [ $a -le 5 ]; do
read input < map_pipe;
if [ $input = "map finished" ]; then
((a++))
echo $a
else
sort -u map_pipe >> keys.txt;
fi
done
I decided to fix it for you, not sure if this is what you wanted, but I think I am close:
#!/bin/bash
a=0 #Initialize your variable to something
while [ $a -le 5 ]; do
read input < map_pipe;
if [ "$input" = "map finished" ]; then #Put double quotes around variables to allow values with spaces
a=$(($a + 1)) #Your syntax was off, use spaces and do something with the output
else
echo $input >> keys.txt #Don't re-read the pipe, it's empty by now and sort will wait for the next input
sort -u keys.txt > tmpfile #Instead sort your file, don't save directly into the same file it will break
mv tmpfile keys.txt
#sort -u keys.txt | sponge keys.txt #Will also work instead of the other sort and mv, but sponge is not installed on most machines
fi
done
I have a data file that is in this format:
name=...
phone=...
address=..
name=...
phone=..
address=...
name=...
phone=...
address=...
and I've tried to use while loop for that to split it into chunks where there is a blank line that is "\n\n". But this approach below fails.
cat mydatafile.txt | while read row; do
echo $row
# process the data
done
Wanted end state is a variable that contains three lines of content from file, that is row="name=...\nphone=...\naddress=..." in every iteration of the loop.
Well, if you're 100% certain there will always be 3 lines that you want and then 1 unneeded line you could do something like this:
cat mydatafile.txt | while read row1; do
read row2
read row3
read junk
row="$row1 $row2 $row3"
echo $row
# process the data
done
I think that will continue to read from the same stdin, but I'm not 100% certain.
Or you could create your own finite state automaton (sorry - I just love the sound of that):
recno=0
cat mydatafile.txt | while read foo; do
let recno=recno+1
if [ $recno -lt 4 ]
then
row="$row $foo"
fi
if [ $recno == 4 ]
then
echo $row
# process the data
recno=0
row=''
fi
done
# Here you might want to check that you've processed the last data...
If you want to use a blank line to determine the start of a new row it would look something like this (modifying the 2nd solution):
cat mydatafile.txt | while read foo; do
if [ -z "$foo" ]
then
echo $row
# process the data
row=''
else
row="$row $foo"
fi
done
# Here you NEED to process the last row unless the file ended in a blank line
#!/bin/bash
i=1
str=""
while read row
do
if (($i % 4 == 0 ))
then
echo $str
# process $str
str=""
else
str="$str\n$row"
fi
i=$(($i+1))
done < "mydatafile.txt"
This is more suited to awk with a custom record separator:
awk -v RS='\n\n' 'NF{printf "row=[%s]\n", $0}' file
row=[name=...
phone=...
address=..]
row=[name=...
phone=..
address=...]
row=[name=...
phone=...
address=...]
-v RS='\n\n' sets record separator as 2 new lines and then $0 gives you all the data of a block.
Working Demo
I'm trying to write a small script that will count entries in a log file, and I'm incrementing a variable (USCOUNTER) which I'm trying to use after the loop is done.
But at that moment USCOUNTER looks to be 0 instead of the actual value. Any idea what I'm doing wrong? Thanks!
FILE=$1
tail -n10 mylog > $FILE
USCOUNTER=0
cat $FILE | while read line; do
country=$(echo "$line" | cut -d' ' -f1)
if [ "US" = "$country" ]; then
USCOUNTER=`expr $USCOUNTER + 1`
echo "US counter $USCOUNTER"
fi
done
echo "final $USCOUNTER"
It outputs:
US counter 1
US counter 2
US counter 3
..
final 0
You are using USCOUNTER in a subshell, that's why the variable is not showing in the main shell.
Instead of cat FILE | while ..., do just a while ... done < $FILE. This way, you avoid the common problem of I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?:
while read country _; do
if [ "US" = "$country" ]; then
USCOUNTER=$(expr $USCOUNTER + 1)
echo "US counter $USCOUNTER"
fi
done < "$FILE"
Note I also replaced the `` expression with a $().
I also replaced while read line; do country=$(echo "$line" | cut -d' ' -f1) with while read country _. This allows you to say while read var1 var2 ... varN where var1 contains the first word in the line, $var2 and so on, until $varN containing the remaining content.
Always use -r with read.
There is no need to use cut, you can stick with pure bash solutions.
In this case passing read a 2nd var (_) to catch the additional "fields"
Prefer [[ ]] over [ ].
Use arithmetic expressions.
Do not forget to quote variables! Link includes other pitfalls as well
while read -r country _; do
if [[ $country = 'US' ]]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
minimalist
counter=0
((counter++))
echo $counter
You're getting final 0 because your while loop is being executed in a sub (shell) process and any changes made there are not reflected in the current (parent) shell.
Correct script:
while read -r country _; do
if [ "US" = "$country" ]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
I had the same $count variable in a while loop getting lost issue.
#fedorqui's answer (and a few others) are accurate answers to the actual question: the sub-shell is indeed the problem.
But it lead me to another issue: I wasn't piping a file content... but the output of a series of pipes & greps...
my erroring sample code:
count=0
cat /etc/hosts | head | while read line; do
((count++))
echo $count $line
done
echo $count
and my fix thanks to the help of this thread and the process substitution:
count=0
while IFS= read -r line; do
((count++))
echo "$count $line"
done < <(cat /etc/hosts | head)
echo "$count"
USCOUNTER=$(grep -c "^US " "$FILE")
Incrementing a variable can be done like that:
_my_counter=$[$_my_counter + 1]
Counting the number of occurrence of a pattern in a column can be done with grep
grep -cE "^([^ ]* ){2}US"
-c count
([^ ]* ) To detect a colonne
{2} the colonne number
US your pattern
Using the following 1 line command for changing many files name in linux using phrase specificity:
find -type f -name '*.jpg' | rename 's/holiday/honeymoon/'
For all files with the extension ".jpg", if they contain the string "holiday", replace it with "honeymoon". For instance, this command would rename the file "ourholiday001.jpg" to "ourhoneymoon001.jpg".
This example also illustrates how to use the find command to send a list of files (-type f) with the extension .jpg (-name '*.jpg') to rename via a pipe (|). rename then reads its file list from standard input.
I am trying to write a bash script to create some playlists of music. The part that has me stuck is the while loop for read line. I figure I am over thinking this so I turned to stackoverflow for assistance.
# The first while loop is how many playlists I want to create
i=1
while [ $i -le $plist ]
do
echo -e "iteration $i"
i=$[$i + 1]
z=0
# This while loop is for the length of time I want the playlist to be
while [ $z -le $TOTAL ]
do
echo -e "Count $z"
z=$[$z + xxx]
# This while loop is for reading the track list previously generated.
# It would read the line, calculate the track length,
# add to $z, cp the track to a folder
while read line
do
secs=$(metaflac --show-total-samples --show-sample-rate "$line" | tr '\n' ' '
| awk '{print $1/$2}' -)
z=$[$z + $secs]
cp $line to destination folder
done
done
done
The following program reads a file and it intends to store the all values (each line) into a variable but doesn't store the last line. Why?
file.txt :
1
2
.
.
.
n
Code :
FileName=file.txt
if test -f $FileName # Check if the file exists
then
while read -r line
do
fileNamesListStr="$fileNamesListStr $line"
done < $FileName
fi
echo "$fileNamesListStr" // 1 2 3 ..... n-1 (but it should print up to n.)
Instead of reading line-by-line, why not read the whole file at once?
[ -f $FileName ] && fileNameListStr=$( tr '\n' ' ' < $FileName )
One probable cause is that there misses a newline after the last line n.
Use the following command to check it:
tail -1 file.txt
And the following fixes:
echo >> file.txt
If you really need to keep the last line without newline, I reorganized the while loop here.
#!/bin/bash
FileName=0
if test -f $FileName ; then
while [ 1 ] ; do
read -r line
if [ -z $line ] ; then
break
fi
fileNamesListStr="$fileNamesListStr $line"
done < $FileName
fi
echo "$fileNamesListStr"
The issue is that when the file does not end in a newline, read returns non-zero and the loop does not proceed. The read command will still read the data, but it will not process the loop. This means that you need to do further processing outside of the loop. You also probably want an array instead of a space separated string.
FileName=file.txt
if test -f $FileName # Check if the file exists
then
while read -r line
do
fileNamesListArr+=("$line")
done < $FileName
[[ -n $line ]] && fileNamesListArr+=("$line")
fi
echo "${fileNameListArr[#]}"
See the "My text files are broken! They lack their final newlines!" section of this article:
http://mywiki.wooledge.org/BashFAQ/001
As a workaround, before reading from the text file a newline can be appended to the file.
echo "\n" >> $file_path
This will ensure that all the lines that was previously in the file will be read. Now the file can be read line by line.