For a lab, I wrote a shell script that used awk to do some stuff. Rereading the lab's directions, it seems that I was supposed to write a self-contained awk script. I'm working on translating my bash script into awk, and I'm having a problem right now:
I want to save the output of an awk command to a new file, and then I want to use that output as input for another awk command.
In my bash script, I have this:
awk '/Blocked SPAM/' maillog > spamlog
cat spamlog | awk '{print $0}' RS=' '
It takes all the lines from maillog that contain the string "Blocked SPAM" and saves this to a new file titled spamlog. Then it opens spamlog and replaces every space character ' ' with a new line.
For my awk script, maillog is the file that is passed to the script from shell. My attempt at writing analogous code:
/Blocked SPAM/ > spamlog`
-f spamlog {print $0} RS=' '
I don't really know what I'm doing with my awk script since I'm having trouble finding useful resources for self-contained awk scripts.
awk '/Blocked SPAM/{ print > "spamlog"; gsub( " ","\n"); print }' maillog
Personally, I prefer to invoke that directly from a shell script, but you can easily make it an awk script by writing:
#!/usr/bin/awk -f
/Blocked SPAM/{ print > "spamlog"; gsub( " ","\n"); print }
Invoke that script with 'maillog' as an argument.
Related
I have a large csv files , where few columns values are in hex. I need to convert them into decimal. The CSV files are very big. So If I process each row , then it takes a lot of time to execute the script. So I want to know how this can be done parallely by using awk command
If I process the code line by line it works.
I process the files like this.
while read -r line;
do
start_time=`echo "$line"|awk -F "," '{ print $1 }'`
end_time=`echo "$line"|awk -F "," '{ print $2 }'`
st_time=$((16#$start_time))
en_time=$((16#$end_time))
Then I echo the required fields to output file.
Sample Input file:
16a91f90539,16a91f931a9,e,0
16a91f90bab,16a91f931a9,e,0
Expected output:
1557227177273,1557227188649,e,0
1557227178923,1557227188649,e,0
I need to know how the statement "((16#$start_time))" , can be used in awk.
I tried
awk -F',' '{OFS=",";}{print '"(($1#16))"','$en_time',$3'
But this syntax does not work.
With GNU awk for strtonum() you don't need to spawn multiple shells on each input line:
$ awk 'BEGIN{FS=OFS=","} {for (i=1;i<=2;i++) $i=strtonum("0x"$i)} 1' file
1557227177273,1557227188649,e,0
1557227178923,1557227188649,e,0
You can execute system calls from withnig awk with system(...). Don't forget to close the command afterwards.
awk -F "," '{ cmd=sprintf("echo $((0x%s))\n", $1); system(cmd); close(cmd); }' input
(for some reason the system call does not work with $((16#...)) on my system, but does work with $((0x...)))
With getline you can assign the echo'ed output to a variable. See https://www.gnu.org/software/gawk/manual/html_node/Getline-Notes.html to get you started.
I'm on a Mac, and I want to find a field in a CSV file adjacent to a search string
This is going to be a single file with a hard path; here's a sample of it:
84:a5:7e:6c:a6:b0, AP-ATC-151g84
84:a5:7e:6c:a6:b1, AP-A88-131g84
84:a5:7e:73:10:32, AP-AG7-133g56
84:a5:7e:73:10:30, AP-ADC-152g81
84:a5:7e:73:10:31, AP-D78-152e80
so if my search string is "84:a5:7e:73:10:32"
I want to get returned "AP-AG7-133g56"
I had been working within an Applescript, but maybe a shell script will do.
I just need the proper syntax for opening the file and having awk search it. Again, I'm weak conceptually on how shell commands run, how they must be executed, etc
This errors, gives me ("command not found"):
set the_file to "/Users/Paw/Desktop/AP-Decoder 3.app/Contents/Resources/BSSIDtable.csv"
set the_val to "70:56:81:cb:a2:dc"
do shell script "'awk $1 ~ the_val {print $2} the_file'"
Thank you for coddling me...
This is a relatively simple:
awk '$1 == "70:56:81:cb:a2:dc," {print "The answer is "$2}' 'BSSIDtable.csv'
(the "The answer is " text can be omitted if you only wish to see only the data, but this shows you how to get more user-friendly output if desired).
The comma is included since awk uses white space for separators so the comma becomes part of column 1.
If the thing you're looking for is in a shell variable, you can use -v to provide that to awk as an awk variable:
lookfor="70:56:81:cb:a2:dc,"
awk -v mac=$lookfor '$1 == mac {print "The answer is "$2}' 'BSSIDtable.csv'
As an aside, your AppleScript solution is probably not working because the $1/$2 are being interpreted as shell variable rather than awk variables. If you insist on using AppleScript, you will have to figure out how to construct a shell command that quotes the awk commands correctly.
My advice is to just use the shell directly, the number of people proficient in that almost certainly far outnumber those proficient in AppleScript :-)
if sed is available (normaly on mac, event if not tagged in OP)
simple but read all the file
sed -n 's/84:a5:7e:73:10:32,[[:blank:]]*//p' YourFile
quit after first occurence (so average of 50% faster on huge file)
sed -n -e '/84:a5:7e:73:10:32,[[:blank:]]*/!b' -e 's///p;q' YourFile
awk
awk '/^84:a5:7e:73:10:32/ {print $2}'
# OR using a variable for batch interaction
awk -v Src='84:a5:7e:73:10:32' '$1 == Src {print $2}'
# OR assuming that case is unknow
awk -v Src='84:a5:7e:73:10:32' 'BEGIN{IGNORECASE=1} $1 == Src {print $2}'
by default it take $0 as compare test if a regex is present, just add the ^ to take first field content
I am new to awk.I just try to write some thing that to exchange my text file.but I failed.
I want to output like 'hello'.
I used command awk '{print "'hello'"}' filename to do it.but failed:
output like: hello
but I used command awk '{print "\'hello\'"}' filename to do it.failed again:
output like: >
ok.it seems that the awk command do not get what I mean.
So I am confused about that .how to solve the problem.
guys thanks.
Using the ascii code:
awk '{print "\x27" "hello" "\x27"}' filename
Using a variable:
awk -v q="'" '{print q "hello" q}' filename
Example:
$ seq 2 > filename
$ awk '{print "\x27" "hello" "\x27"}' filename
'hello'
'hello'
$ awk -v q="'" '{print q "hello" q}' filename
'hello'
'hello'
Simply use double quotes:
awk "{print \"'hello'\"}" filename
Although that won't really modify your file.
awk '{print "'"'"'hello'"'"'"}' filename
clyfish's answer works, if you must have it output single quotes and you must use scripts that you pass on the command line.
What I usually do in cases like these, though, when I need to do quoting but I don't want to write a 'real' awk script, is this:
awk 'function q(word) { return "\"" word "\"" }
{ printf("mv %s SomeDir/;", q($0)) }'
What I've done is to define a function that returns whatever you pass it in double quotes. Then use printf to actually use it. Without doing that, I would have had to do:
awk '{ print("mv \"" $0 "\" SomeDir/;") }';
It gets pretty nasty. For more complicated examples, this can be a life saver.
However, suppose you really do need to output something with actual single quotes. In that case dealing with odd shell quoting rules while trying to pass scripts like this on the command line is going to drive you completely insane, so I would suggest you just write a simple throwaway file.
#!/usr/bin/awk
# hi.awk
{ print("'hello'") }
then call it:
awk -f ./hi.awk
You don't really even need the #! line in the file if you do it that way, but neither does it hurt.
I need a simple working example that makes me understand
1, awk command in shell script.
2, awk withing awk command in shell script.
I'm not really sure what the point is, but here's awk calling awk from within a shell script...
#!/bin/sh
cmd='BEGIN {print \"foo\"}'
echo foo |
awk "{ system( \"awk '$cmd'\" )}"
I know I can redirect awk's print output to another file from within a script, like this:
awk '{print $0 >> "anotherfile" }' 2procfile
(I know that's dummy example, but it's just an example...)
But what I need is to redirect output to another file, which has a dynamic name like this
awk -v MYVAR"somedinamicdata" '{print $0 >> "MYWAR-SomeStaticText" }' 2procfile
And the outpus should be redirected to somedinamicdata-SomeStaticText.
I know I can do it via:
awk '{print $0 }' 2procfile >> "$MYVAR-somedinamicdata"
But the problem is that it's a bigger awk script, and I have to output to several files depending on certain conditions (and this awk script is called from another bash, and it passes some dynamic variable via the -v switch... and son on.
Is it possible anyhow?
Thanks in advance.
i think
awk -v MYVAR="somedinamicdata" '{print $0 >> (MYVAR "-SomeStaticText") }' 2procfile
should do it. String concatenation in awk is just put one after another.