I have a .dat file with | separator and I want to change the value of the column which is defined by a number passed as argument and stored in a var. My code is
awk -v var="$value" -F'|' '{ FS = OFS = "|" } $1=="$id" {$"\{$var}"=8}1'
myfile.dat > tmp && mv tmp myfiletemp.dat
This changes the whole line to 8, obviously doesn't work. I was wondering what is the right way to write this part
{$"\{$var}"=8}1
For example, if I want to change the fourth column to 8 and I have value=4, how do I get {$4=8}?
The other answer is mostly correct, but just wanted to add a couple of notes, in case it wasn't totally clear.
Referring to a variable with a $ in front of it turns it in to a reference to the column. So i=3; print $i; print i will print the third column and then the number 3.
Putting all your variables in the command line will avoid any problems with trying to include bash variables inside your single-quoted awk code, which won't work.
You can let awk do the output to the specific file instead of relying on bash to redirect output and move files.
The -F option on the command line specifies FS for you, so no need to redeclare it in your code.
Here's how I would do this:
#!/bin/bash
column=4
value=8
id=1
awk -v col="$column" -v val="$value" -v id="$id" -F"|" '
BEGIN {OFS="|"}
{$1==id && $col=val; print > "myfiletemp.dat"}
' myfile.dat
you can refer to the awk variable directly by it's name, slight rewrite of your script with correct reference to column number var...
awk -F'|' -v var="$value" 'BEGIN{OFS=FS} $1=="$id"{$var=8}1'
should work as long as $value is a number. If id is another bash variable, pass it the same way as an awk variable
awk -F'|' -v var="$value" -v id="$id" 'BEGIN{OFS=FS} $1==id{$var=8}1'
Not only can you use a number in a variable by putting a $ in front of it, you can also use put a $ in front of an expression!
$ date | tee /dev/stderr | awk '{print $(2+2)}'
Mon Aug 3 12:47:39 CDT 2020
12:47:39
Related
I have this variable:
a='/08/OPT/imaginary/N/08_i_N.out'
I want to use "/" as a field separator.
Then, I want to extract the first pattern.
I have tried:
awk -F/ '{print $1}' "$a"
But I get:
awk: cannot open /08/OPT/imaginary/N/08_i_N.out (No such file or directory)
I do not want the file, only to work on the path of that file.
Same way as any other command, either of these (or other alternatives, e.g. within "here-documents" or passed as awk variables or...):
printf '%s\n' "$a" | command
command <<<"$a"
I have this awk command:
echo www.host.com |awk -F. '{$1="";OFS="." ; print $0}' | sed 's/^.//'
which what it does is to get the domain from the hostname:
host.com
that command works on CentOS 7 (awk v 4.0.2), but it does not work on ubuntu 19.04 (awk 4.2.1) nor alpine (gawk 5.0.1), the output is:
host com
How could I fix that awk expression so it works in recent awk versions ?
For your provided samples could you please try following. This will try to match regex from very first . to till last of the line and then prints after first dot to till last of line.
echo www.host.com | awk 'match($0,/\..*/){print substr($0,RSTART+1,RLENGTH-1)}'
OP's code fix: In case OP wants to use his/her own tried code then following may help. There are 2 points here: 1st- We need not to use any other command along with awk to processing. 2nd- We need to set values of FS and OFS in BEGIN section which you are doing in everyline.
echo www.host.com | awk 'BEGIN{FS=OFS="."} {$1="";sub(/\./,"");print}'
To get the domain, use:
$ echo www.host.com | awk 'BEGIN{FS=OFS="."}{print $(NF-1),$NF}'
host.com
Explained:
awk '
BEGIN { # before processing the data
FS=OFS="." # set input and output delimiters to .
}
{
print $(NF-1),$NF # then print the next-to-last and last fields
}'
It also works if you have arbitrarily long fqdns:
$ echo if.you.have.arbitrarily.long.fqdns.example.com |
awk 'BEGIN{FS=OFS="."}{print $(NF-1),$NF}'
example.com
And yeah, funny, your version really works with 4.0.2. And awk version 20121220.
Update:
Updated with some content checking features, see comments. Are there domains that go higher than three levels?:
$ echo and.with.peculiar.fqdns.like.co.uk |
awk '
BEGIN {
FS=OFS="."
pecs["co\034uk"]
}
{
print (($(NF-1),$NF) in pecs?$(NF-2) OFS:"")$(NF-1),$NF
}'
like.co.uk
You got 2 very good answers on awk but I believe this should be handled with cut because of simplicity it offers in getting all fields starting for a known position:
echo 'www.host.com' | cut -d. -f2-
host.com
Options used are:
-d.: Set delimiter as .
-f2-: Extract all the fields starting from position 2
What you are observing was a bug in GNU awk which was fixed in release 4.2.1. The changlog states:
2014-08-12 Arnold D. Robbins
OFS being set should rebuild $0 using previous OFS if $0 needs to be
rebuilt. Thanks to Mike Brennan for pointing this out.
awk.h (rebuild_record): Declare.
eval.c (set_OFS): If not being called from var_init(), check if $0 needs rebuilding. If so, parse the record fully and rebuild it. Make OFS point to a separate copy of the new OFS for next time, since OFS_node->var_value->stptr was
already updated at this point.
field.c (rebuild_record): Is now extern instead of static. Use OFS and OFSlen instead of the value of OFS_node.
When reading the code in the OP, it states:
awk -F. '{$1="";OFS="." ; print $0}'
which, according to POSIX does the following:
-F.: set the field separator FS to represent the <dot>-character
read a record
Perform field splitting with FS="."
$1="": redefine field 1 and rebuild record $0 using OFS. At this time, OFS is set to be a single space. If the record $0 was www.foo.com it now reads _foo_com (underscores represent spaces). Recompute the number of fields which are now only one as there is no FS available anymore.
OFS=".": redefine the output field separator OFS to be the <dot>-character. This is where the bug happens. The Gnu awk knew that a rebuild needed to happend, but did this already with the new OFS and not the old OFS.
**print $0':** print the record $0 which is now_foo_com`.
The minimal change to your program would be:
awk -F. '{OFS="."; $1=""; print $0}'
The clean change would be:
awk 'BEGIN{FS=OFS="."}{$1="";print $0}'
The perfect change would be to replace the awk and sed by the cut solution of Anubahuva
If you have a variable with that name in there, you could use:
var=www.foo.com
echo ${var#*.}
I'm on a Mac, and I want to find a field in a CSV file adjacent to a search string
This is going to be a single file with a hard path; here's a sample of it:
84:a5:7e:6c:a6:b0, AP-ATC-151g84
84:a5:7e:6c:a6:b1, AP-A88-131g84
84:a5:7e:73:10:32, AP-AG7-133g56
84:a5:7e:73:10:30, AP-ADC-152g81
84:a5:7e:73:10:31, AP-D78-152e80
so if my search string is "84:a5:7e:73:10:32"
I want to get returned "AP-AG7-133g56"
I had been working within an Applescript, but maybe a shell script will do.
I just need the proper syntax for opening the file and having awk search it. Again, I'm weak conceptually on how shell commands run, how they must be executed, etc
This errors, gives me ("command not found"):
set the_file to "/Users/Paw/Desktop/AP-Decoder 3.app/Contents/Resources/BSSIDtable.csv"
set the_val to "70:56:81:cb:a2:dc"
do shell script "'awk $1 ~ the_val {print $2} the_file'"
Thank you for coddling me...
This is a relatively simple:
awk '$1 == "70:56:81:cb:a2:dc," {print "The answer is "$2}' 'BSSIDtable.csv'
(the "The answer is " text can be omitted if you only wish to see only the data, but this shows you how to get more user-friendly output if desired).
The comma is included since awk uses white space for separators so the comma becomes part of column 1.
If the thing you're looking for is in a shell variable, you can use -v to provide that to awk as an awk variable:
lookfor="70:56:81:cb:a2:dc,"
awk -v mac=$lookfor '$1 == mac {print "The answer is "$2}' 'BSSIDtable.csv'
As an aside, your AppleScript solution is probably not working because the $1/$2 are being interpreted as shell variable rather than awk variables. If you insist on using AppleScript, you will have to figure out how to construct a shell command that quotes the awk commands correctly.
My advice is to just use the shell directly, the number of people proficient in that almost certainly far outnumber those proficient in AppleScript :-)
if sed is available (normaly on mac, event if not tagged in OP)
simple but read all the file
sed -n 's/84:a5:7e:73:10:32,[[:blank:]]*//p' YourFile
quit after first occurence (so average of 50% faster on huge file)
sed -n -e '/84:a5:7e:73:10:32,[[:blank:]]*/!b' -e 's///p;q' YourFile
awk
awk '/^84:a5:7e:73:10:32/ {print $2}'
# OR using a variable for batch interaction
awk -v Src='84:a5:7e:73:10:32' '$1 == Src {print $2}'
# OR assuming that case is unknow
awk -v Src='84:a5:7e:73:10:32' 'BEGIN{IGNORECASE=1} $1 == Src {print $2}'
by default it take $0 as compare test if a regex is present, just add the ^ to take first field content
The variable in awk does not return the result.
I am trying to get the next line of the matched value from file by using awk. It works fine without the variable. Thanks.
$ cat file
name=bobk
snm=sahh
emp=bklc
jdate=879
$
$ awk '/name/{getline; print}' file
snm=sahh ---------> Got the result
$
$ export MYVAR=name
$
$ echo $MYVAR
name
$
$ awk -v AVAR=${MYVAR} '/AVAR/{getline; print}' file
$ ---------> No result
You need to use the regexp match operator ~ against the whole line $0 as /AVAR/ is match for the string AVAR not the variable AVAR:
$ awk -v AVAR=${MYVAR} '$0~AVAR{getline; print}' file
snm=sahh
Using AWK variables to pass parameters is almost always cleaner as to intent; however, in this specific case, the resulting script is a bit more complex than it needs to be -- and breaking the rules a bit may actually be easier to understand and communicate to others.
With use of double-quotes to escape the script there is no need to use an additional AWK variable. That, and if we place the shell variable within /'s there is no need for the {}:
$ awk "/$MYVAR/ {getline; print}" file
snm=sahh
$
I have a file with fields separated by pipe characters and I want to print only the second field. This attempt fails:
$ cat file | awk -F| '{print $2}'
awk: syntax error near line 1
awk: bailing out near line 1
bash: {print $2}: command not found
Is there a way to do this?
Or just use one command:
cut -d '|' -f FIELDNUMBER
The key point here is that the pipe character (|) must be escaped to the shell. Use "\|" or "'|'" to protect it from shell interpertation and allow it to be passed to awk on the command line.
Reading the comments I see that the original poster presents a simplified version of the original problem which involved filtering file before selecting and printing the fields. A pass through grep was used and the result piped into awk for field selection. That accounts for the wholly unnecessary cat file that appears in the question (it replaces the grep <pattern> file).
Fine, that will work. However, awk is largely a pattern matching tool on its own, and can be trusted to find and work on the matching lines without needing to invoke grep. Use something like:
awk -F\| '/<pattern>/{print $2;}{next;}' file
The /<pattern>/ bit tells awk to perform the action that follows on lines that match <pattern>.
The lost-looking {next;} is a default action skipping to the next line in the input. It does not seem to be necessary, but I have this habit from long ago...
The pipe character needs to be escaped so that the shell doesn't interpret it. A simple solution:
$ awk -F\| '{print $2}' file
Another choice would be to quote the character:
$ awk -F'|' '{print $2}' file
Another way using awk
awk 'BEGIN { FS = "|" } ; { print $2 }'
And 'file' contains no pipe symbols, so it prints nothing. You should either use 'cat file' or simply list the file after the awk program.