I have file1.txt as
data 1
data 2
data 3
data 4
and similarly i have another file2.txt as
record 1
record 2
record 3
record 4
In both the cases i have same number of records.
i can access each line from file1.txt by
for line in file1.txt
echo $line
I want to access the 1st records from both the files from single loop and so on.The reason i want first record from both the files in a single loop because i will generate html code based on both the data. If not in this way can anyone help the other way if possible.
Please Help.
% cat data
data 1
data 2
data 3
% cat record
record 1
record 2
record 3
% paste data record
data 1 record 1
data 2 record 2
data 3 record 3
%
The paste command might help you, if what you want to do is join the files line by line. It isn't part of bash, it's a tool that has been installed on almost every unix system since about 1979. :-)
$ cat record
ONE
TWO
THREE
$ cat data
one
two
three
$ paste record data
ONE one
TWO two
THREE three
Reading from multiple files at the same time is tricky in bash. Using multiple file handles is considered by some to be "advanced".
A bash script like this might be the way to go.
#!/usr/bin/env bash
exec 3< data
while read left; do
read right <&3
echo "$left /// $right" # or whatever you need to do
done < record
This opens the file data on file handle 3 (leaving stdin, file handle 0, for the file record), and steps through each file, reading from both.
Alternately, if you want text processing in a different flavour, you could use awk (which is not bash, but usually installed anywhere that bash is installed):
awk '{getline B < "data"; print $0 "\t" B;}' record > combined.txt
This will walk through each file line by line, opening both files and reading a line from each. It has the advantage of not taking a bunch of memory just to store your files.
Alternately, a higher performance solution would be to store one file in memory in an array, then process the other file line by line:
awk 'NR==FNR{a[NR]=$0;next;} {print $0 "\t" a[FNR];}' record data
In either case, replace the print function with whatever processing you need.
exec 3< file2
while read -r line1; do read -r line2 <&3; echo "$line1 $line2"; done < file1
Output:
data 1 record 1
data 2 record 2
data 3 record 3
data 4 record 4
Similar advice that others have given you, but I'd read from each file in the condition of a while loop:
while IFS= read -r -u3 data && IFS= read -r -u4 record; do
echo "$data => $record"
done 3< file1.txt 4< file2.txt
outputs
data 1 => record 1
data 2 => record 2
data 3 => record 3
data 4 => record 4
Related
Here is what I have. which works..... but
#!/bin/bash
echo -n > ~/sites/received.txt
while :
do
(stty raw; cat > ~/sites/received.txt | tail -n 2 ~/sites/received.txt > ~/sites/received.txt) < /dev/cu.usbmodem1431
done
the file gets bigger every 10 seconds or so... Its like there is data leakage. The received.txt file never stays to only 2 lines.
Is there a better way to temporarily hold serial data in a txt file?
Your approach can't work for several reasons, primarily because cat reads the input until end-of-file is reached (which won't happen with stty raw).
To just store the two most recent input lines in the file, do in your loop
line2="$line"; read line </dev/cu.usbmodem1431; { echo $line2; echo $line; } >received.txt
I am using a while do loop to read in from a file that contains a list of hostnames, run a command against the host list, and input specific data from the results into a second file. I need the output to be from line 33 column 3 and line 224 column 7, output to a single line in the second file. I can do it for either one or the other but having trouble getting it to work for both. example:
while read i; do
/usr/openv/netbackup/bin/admincmd/bpgetconfig -M $i |\
awk -v j=33 -v k=3 'FNR == j {print $k}' > /tmp/clientversion.txt
done < /tmp/clientlist.txt
Any hints or help is greatly appreciated!
You could use something like this:
awk 'NR==33{a=$3}NR==224{print a,$7}'
This saves the value in the third column of line 33 to the variable a, then prints it out along with the seventh column of line 224.
However, you're currently overwriting the file /tmp/clientversion.txt every iteration of the while loop. Assuming you want the file to contain all of the output once the loop has run, you should move the redirection outside the loop:
while read -r i; do
/usr/openv/netbackup/bin/admincmd/bpgetconfig -M $i |\
awk 'NR==33{a=$3}NR==224{print a,$7}'
done < /tmp/clientlist.txt > /tmp/clientversion.txt
As a bonus, I have added the -r switch to read (see related discussion here). Depending on the contents of your input file, you might also want to use double quotes around "$i" as well.
I have multiple (1086) files (.dat) and in each file I have 5 columns and 6384 lines.
I have a single file named "info.txt" which contains 2 columns and 6883 lines. First column gives the line numbers (to delete in .dat files) and 2nd column gives a number.
1 600
2 100
3 210
4 1200
etc...
I need to read in info.txt, find every-line number corresponding to values less than 300 in 2nd column (so it is 2 and 3 in above example). Then I need to read these values into sed-awk or grep and delete these #lines from each .dat file. (So I will delete every 2nd and 3rd row of dat files in the above example).
More general form of the question would be (I suppose):
How to read numbers as input from file, than assign them to the rows to be deleted from multiple files.
I am using bash but ksh help is also fine.
sed -i "$(awk '$2 < 300 { print $1 "d" }' info.txt)" *.dat
The Awk script creates a simple sed script to delete the selected lines; the script it run on all the *.dat files.
(If your sed lacks the -i option, you will need to write to a temporary file in a loop. On OSX and some *BSD you need -i "" with an empty argument.)
This might work for you (GNU sed):
sed -rn 's/^(\S+)\s*([1-9]|[1-9][0-9]|[12][0-9][0-9])$/\1d/p' info.txt |
sed -i -f - *.dat
This builds a script of the lines to delete from the info.txt file and then applies it to the .dat files.
N.B. the regexp is for numbers ranging from 1 to 299 as per OP request.
# create action list
cat info.txt | while read LineRef Index
do
if [ ${Index} -lt 300 ]
then
ActionReq="${ActionReq};${Index} b
"
fi
done
# apply action on files
for EachFile in ( YourListSelectionOf.dat )
do
sed -i -n -e "${ActionReq}
p" ${EachFile}
done
(not tested, no linux here). Limitation with sed about your request about line having the seconf value bigger than 300. A awk is more efficient in this operation.
I use sed in second loop to avoid reading/writing each file for every line to delete. I think that the second loop could be avoided with a list of file directly given to sed in place of file by file
This should create a new dat files with oldname_new.dat but I havent tested:
awk 'FNR==NR{if($2<300)a[$1]=$1;next}
!(FNR in a)
{print >FILENAME"_new.dat"}' info.txt *.dat
I have multiple files which have the same structure but not the same data. Say their names are values_#####.txt (values_00001.txt, values_00002.txt, etc.).
I want to extract a specific line from each file and copy it in another file. For example, I want to extract the 8th line from values_00001.txt, the 16th line from values_00002.txt, the 24th line from values_00003.txt and so on (increment = 8 each time), and copy them line by line in a new file (say values.dat).
I am new to shell scripting, I tried to use sed, but I didn't figure out how to do that.
Thank you in advance for your answers !
I believe ordering of files is also important to make sure you get output in desired sequence.
Consider this script:
n=8
while read f; do
sed $n'q;d' "$f" >> output.txt
((n+=8))
done < <(printf "%s\n" values_*.txt|sort -t_ -nk2,2)
This can make it:
for var in {1..NUMBER}
do
awk -v line=$var 'NR==8*line' values_${var}.txt >> values.dat
done
Explanation
The for loop is basic.
-v line=$var "gives" the $var value to awk, so it can be used with the variable line.
'NR==8*line' prints the line number 8*{value we are checking}.
values_${var}.txt gets the file values_1.txt, values_2.txt, and so on.
>> values.dat redirects to values.dat file.
Test
I created 3 equal files a1, a2, a3. They contain 30 lines, being each one the line number:
$ cat a1
1
2
3
4
...
Executing the one liner:
$ for var in {1..3}; do awk -v line=$var 'NR==8*line' a${var} >> values.dat; done
$ cat values.dat
8
16
24
I have some lines like the following saved into a txt file:
Mike Tyson 1 2 3 4 5
Alì 1 2 3 4 5
every different fields are separated with a tab, but in the first field I could have 2 words separated only by a space.
how can I have a correct interpretation by awk? I want only the values separated by tabs like this:
$a=mike tyson
$b=1
$c=2
etc etc....
now, i'm using a while cycle to read each line, finished by
done < <(awk 'NR>0' file.txt)
but this command sees the value "mike tyson" as two different fields.
Try to change to:
done < <(awk -F"\t" 'NR>0' file.txt)
awk does see any space (blanks and tabs) as filed separators.
Setting it to only tab, prevents it divide files with space.
The problem is not with awk, as you are interpreting the columns in bash.
Maybe you're looking for something like this:
IFS=$'\t'
while read a b; do
echo a=$a
echo b=$b
done < file.txt
In your sample code awk doesn't seem to play any role btw.