I have a series of data files that I need to get a few specific values from. I know the line in the file that the data is on. My data files look like this
x y z
1 0.2 0.3
2 0.1 0.2
3 0.5 0.6
etc.
I am using a shell script to access the files, collect the desired data from each file, and output the collected data in one file.
For example, I need the y value in line 3, 0.1. I have tried the following
let dataLine=3
let yVal=`head -n $dataLine dataFile | tail -n 1 | awk '{print $2}'`
but I get the following error
let: yVal=0.1: syntax error: invalid arithmetic operator (error token is ".1")
I have tried adding | bc after awk '{print $2}' but then it did not even register the correct value for what should be assigned to yVal. When I do it as shown above, it does show that it is recognizing the value in the correct line and column.
Thanks for the help,
$ dataLine=3
$ yVal=$(awk -v dataLine="$dataLine" 'NR==dataLine{print $2}' data)
$ echo $yVal
0.1
If you want to get 3rd line's 2nd field then following may help you in same.
Solution 1st: If you want to pass any shell variable's value to any awk variable then following may help you in same.
line_number=3
awk -v line="$line_number" 'FNR==line{print $2}' Input_file
Solution 2nd: If you want to directly print 3rd line's 2nd field then following may help you in same.
awk 'FNR==3{print $2}' Input_file
Related
I want to extract a value using "awk subtring" which should also count the number of spaces without any separator.
For example, below is the input, and I want to extract the "29611", including space,
201903011232101029 2961104E3021 223 0 12113 5 15 8288 298233 0 45 0 39 4
I used this method, but it used space as a separator:
more abbas.dat | awk '{print substr($1,1,16),substr($1,17,25)}'
Expected output should be :
201903011232101029 2961
But it prints only
201903011232101029
My question is how can we print using "substr" which count spaces?
I know, I can use this command to get the desired output but it is not helpful for my objective
more abbas.dat | awk '{print substr($1,1,16),substr($2,1,5)}'
1st solution: With your shown samples, please try following awk code. Written and tested in GNU awk. Using match function of awk here to get required output.
To print 1st field followed by varying spaces followed by 5 digits from 2nd field then use following:
awk 'match($0,/^[0-9]+[[:space:]]+[0-9]{5}/){print substr($0,RSTART,RLENGTH)}' Input_file
OR To print 16 letters in 1st field and 5 from second field including varying length of spaces between 1st and 2nd fields:
awk 'match($0,/^([0-9]{16})[^[:space:]]+([[:space:]]+)([0-9]{5})/,arr){print arr[1] arr[2] arr[3]}' Input_file
2nd solution: Using GNU grep please try following, considering that your 2nd column first 4 needed values can be anything(eg: digits, alphabets etc).
grep -oP '^\S+\s+.{5}' Input_file
OR to only match 4 digits in 2nd field have a minor change in above grep.
grep -oP '^\S+\s+\d{5}' Input_file
If there is always one space you can use the following command which will print the first group, plus the first 5 character of the second group.
N.B. It's not clear in the question whether you want 4 or 5 characters but that can be adjusted easily.
more abbas.dat | awk '{print $1" "substr($2,1,5) }'
I think the simplest way is to include "Fs" in your command.
more abbas.dat | awk -Fs '{print substr($1,1,16),substr($1,17,25)}'
$ awk '{print substr($0,1,24)}' file
201903011232101029 29611
If that's not all you need then edit your question to clarify your requirements.
For example i have the following file:
4:Oscar:Ferk:Florida
14:Steven:Pain:Texas
7:Maya:Ross:California
and so on...
It has an unknown number of lines because you can keep adding more to it.
I'm writing a script where you can edit the name by giving in the id of the person and the new name you want to give it as parameters.
What i am trying to do is use awk to find the line and then change the name on that specific line and update the file. I'm having trouble because my code updates every single column value to the given one.
My current code is:
getID=$1
getName=$2
awk -v varID="$getID" -F':' '$0~varID' file.dat | awk -v varName="$getName" -F':' '$2=varName' file.dat > tmp && mv tmp file.dat
Help is really appreciated, thank you kindly in advance.
You may use this awk:
getID=14 # change to getID="$1"
getName='John K' # change to getName="$2"
awk -v id="$getID" -v name="$getName" 'BEGIN{FS=OFS=":"} $1==id{$2=name} 1' file
4:Oscar:Ferk:Florida
14:John K:Pain:Texas
7:Maya:Ross:California
cat TEXT | awk -v var=$i -v varB=$j '$1~var , $1~varB {print $1}' > PROBLEM HERE
I am passing two variables from an array to parse a very large text file by range. And it works, kind of.
if I use ">" the output to the file will ONLY be the last three lines as verified by cat and a text editor.
if I use ">>" the output to the file will include one complete read of TEXT and then it will divide the second read into the ranges I want.
if I let the output go through to the shell I get the same problem as above.
Question:
It appears awk is reading every line and printing it. Then it goes back and selects the ranges from the TEXT file. It does not do this if I use constants in the range pattern search.
I undestand awk must read all lines to find the ranges I request.
why is it printing the entire document?
How can I get it to ONLY print the ranges selected?
This is the last hurdle in a big project and I am beating my head against the table.
Thanks!
give this a try, you didn't assign varB in right way:
yours: awk -v var="$i" -varB="$j" ...
mine : awk -v var="$i" -v varB="$j" ...
^^
Aside from the typo, you can't use variables in //, instead you have to specify with regular ~ match. Also quote your shell variables (here is not needed obviously, but to set an example). For example
seq 1 10 | awk -v b="3" -v e="5" '$0 ~ b, $0 ~ e'
should print 3..5 as expected
It sounds like this is what you want:
awk -v var="foo" -v varB="bar" '$1~var{f=1} f{print $1} $1~varB{f=0}' file
e.g.
$ cat file
1
2
foo
3
4
bar
5
foo
6
bar
7
$ awk -v var="foo" -v varB="bar" '$1~var{f=1} f{print $1} $1~varB{f=0}' file
foo
3
4
bar
foo
6
bar
but without sample input and expected output it's just a guess and this would not address the SHELL behavior you are seeing wrt use of > vs >>.
Here's what happened. I used an array to input into my variables. I set the counter for what I thought was the total length of the array. When the final iteration of the array was reached, there was a null value returned to awk for the variable. This caused it to print EVERYTHING. Once I correctly had a counter with the correct number of array elements the printing oddity ended.
As far as the > vs >> goes, I don't know. It did stop, but I wasn't as careful in documenting it. I think what happened is that I used $1 in the print command to save time, and with each line it printed at the end it erased the whole file and left the last three identical matches. Something to ponder. Thanks Ed for the honest work. And no thank you to Robo responses.
I have following data file in bash. I want to search if the user entered webserver is present in the data file, if present it should return the Phase and Managed server name.
1 K1 tvtw1 tvtm1
1 K1 tvtw2 tvtw2
2 K2 tvtw26 tvtw26
3 k5 tvtw29 tvtm29
I tried grep "$webserver" serverList.lst | awk '{print $1}' but it returns multiple values for tvtw2. Is there any way to find exact server name from the list ?
If I understand correctly, if column 3 matches exactly, then you want to get the value of column 1:
awk -v serv=tvtw2 '$3 == serv {print $1}' serverList.lst
That is, we put the string you want to match in variable serv, and then use that as a filter expression in awk to match column 3 exactly.
you need add word boundary in your grep regex, so that tvtw26 won't be selected.
e.g. grep '\btvtw2\b' file ...
However since you have already used awk, you can consider to use awk for all. #janos 's answer showed how could it be done.
Try grep -w (-w stands for word regex)
However, grep | awk is useless-use-of-grep. See janos's answer for more optimal solution.
I have created simple script:
#!/bin/sh
column=${1:-1}
awk '{colawk='$column'+1; print colawk}'
But when I run:
ls -la | ./Column.sh 4
I receive output:
5
5
5
5
But I have expected receive 5th column. Why this error?
I believe this will do what you've attempted in your example:
#!/bin/sh
let "column=${1:-1} + 1"
awk "{print \$$column}"
However, I don't see why you're adding one to the column index? You'll then not be able to intuitively access the first column.
I'd to it this way instead:
#!/bin/sh
let "column=${1:-1}"
awk "{print \$$column}"
The argument to ./Column.sh will be the column number you want, 0 will give you all columns, while a call without arguments will default the column index to 1.
I know bash. I would like make arithmetic with AWK
In that case, how about:
#!/bin/sh
column=${1:-1}
awk 'BEGIN{colawk='$column'+1} {print $colawk}'
Or, simply:
#!/bin/sh
awk 'BEGIN{colawk='${1:-1}'+1} {print $colawk}'
Two things I changed in your script:
put the arithmetic in a BEGIN{} block since it only needs to be done once and not repeated for every input line.
"print $colawk" instead of "print colawk" so we're printing the column indexed by colawk instead of its value.