I am trying to evaluate two different files (.tf files) to check if the resource names are the same in a bash script.
I know that in those files resources are declared as such:
resource "service_name" "resource_name" {
#resource config
}
One approach would be something like:
while read line
do
if word_file1 == "resource"; then
#save string in array1 before "{" character
fi
while read line
do
if word_file2 == "resource"; then
#save string in array2 before "{" character
if array1 == array2; then
break
else
# write diff to another file, including all config
# info between {} for the missing or different resource
fi
fi
done < filename2
done < filename1
From a test file (file1) an example input would be:
resource "service_name" "resource_name_AA" {
#resource config
# policy_config = << POLICY
{
policy_definition
} POLICY
}
From a test file (file2) an example input would be:
resource "service_name" "resource_name_AA" {
#resource config
# policy_config = << POLICY
{
policy_definition
} POLICY
}
resource "service_name" "resource_name_BB" {
#resource config
# policy_config = << POLICY
{
policy_definition
} POLICY
}
The desired output would be (diff_file):
resource "service_name" "resource_name_BB" {
#resource config
# policy_config = << POLICY
{
policy_definition
} POLICY
}
I think I would try to keep it simpler than that:
grep 'resource' file1 > file1_resources.txt
grep 'resource' file2 > file2_resources.txt
diff file{1,2}_resources.txt
if the word "resource" shows up in different contexts, then you could use a regexp grep instead:
egrep "resource.*\{" fileX
This might do the job. I guess as per the code you showed, the contents of the resource also need to to printed in cases of mismatch. If only difference needs to be pointed out, diff is better and enough. Anyways I still like writing awk(still a bit new, learning) scripts, so wrote one.
#! /bin/bash
awk '{
if (FNR == NR)
{
if ($1 == "resource")
resource_name=$3
else
contents[resource_name]=contents[resource_name]"\n"
contents[resource_name]=contents[resource_name]$0
}
else
{
if (($1 == "}") && (flag == 1))
{
flag=0
next
}
else
if (($1 == "resource") && (contents[$3] != ""))
{
flag=1
contents[$3]=""
next
}
if (flag == 1)
next
print
}
}
END {
for (resource in contents)
{
if (contents[resource] != "")
print contents[resource]
}
}
' file2 file1 > otherfile
UPDATE:
#! /bin/bash
awk '{
if (FNR == NR)
{
if ($1 == "resource")
resource_name=$3
else
contents[resource_name]=contents[resource_name]"\n"
contents[resource_name]=contents[resource_name]$0
}
else
{
if (($1 == "}") && (flag == 1))
{
flag=0
next
}
else
if (($1 == "resource") && (contents[$3] == ""))
{
flag=1
contents[$3]=""
next
}
if (flag == 1)
next
print
}
}' file1 file2 > same_resources
UPDATE-2:
#! /bin/bash
awk '{
if (FNR == NR)
{
if ($1 == "resource")
resource_name=$3
else
contents[resource_name]=contents[resource_name]"\n"
contents[resource_name]=contents[resource_name]$0
}
else
{
if ($1 == "resource")
{
if (flag == 1)
{
flag=0
}
if (contents[$3] != "")
{
flag=1
contents[$3]=""
}
}
if (flag == 1)
next
print
}
}
END {
for (resource in contents)
{
if (contents[resource] != "")
print contents[resource]
}
}' file2 file1 > someotherfile
File 1:
A|sam|2456|8901
B|kam|5678|9000
C|pot|4567|8000
File 2:
X|ter|2456|8901
Y|mar|5678|9000
Z|poi|4567|8000
File 3:
Column1|Column2|Column3|Coumn4
Now i want this 3 Files to be passed as parameters to the GNU Awk Script as below -
awk -f script.awk file1 file2 file3
Script i have written are able to handle only 2 Files but not able to handle the 3rd file. Pleas help.
script.awk
BEGIN { # setup file separator and sorting:
FS=OFS="|"
PROCINFO["sorted_in"]="#ind_str_asc"
}
# skip header lines
FNR == 1 { next }
# store first file
(FNR==NR) { f1[$5]=$0
# skip processing of other rules and
# read the next line from input
next
}
# store second file
{ f2[$5]=$0
if( ! ($5 in f1)) {
f1[$5] = ""
}
}
END {
for( k in f1) {
split( f1[k], arr1, "|")
for( c = 1; c <= length( f1[ k ] ); c++ ) {
print arr1[c]
}
}
for( k in f2) {
split( f2[k], arr2, "|")
for( c = 1; c <= length( f2[ k ] ); c++ ) {
print arr2[c]
}
}
}
}
My Objective is the Read the 3rd File also in the same code in the print in the similar way as the printing is handled in the above code.
Note : Would be good if anyone can keep the similar Code structure as above and just include the reading and printing of the 3rd File.
Your existing code is more complicated than it has to be. It could be written as just:
BEGIN { # setup file separator and sorting:
...
}
# skip header lines
FNR == 1 { next }
ARGIND==1 { f1[$5]=$0; next }
ARGIND==2 { f2[$5]=$0; f1[$5] }
END {
...
}
I assume you can see the obvious extension to add a 3rd file. The above requires GNU awk for ARGIND and PROCINFO[] which you're already using.
You can use ARGV array to process multiple files like this:
function disp() {
for (i=1; i<=NF; i++)
print FILENAME " :: " FNR " :: " $i
print ""
}
BEGIN { # setup file separator and sorting:
FS=OFS="|"
PROCINFO["sorted_in"]="#ind_str_asc"
}
# process first file
ARGV[1] == FILENAME {
disp()
}
# process second file
ARGV[2] == FILENAME {
disp()
}
# process third file
ARGV[3] == FILENAME {
disp()
}
I'm trying to add line breaks to a text file each time a subsequent number is smaller than the immediately preceding number (e.g. a break between "72.774" and "7.009") in a text file with this structure:
7.007 28.929 50.851 72.774 7.009 28.932 50.854 72.777 7.015 32.939 54.862 76.784
I want the output to be in this format:
7.007 28.929 50.851 72.774
7.009 28.932 50.854 72.777
7.015 32.939 54.862 76.784
Files do not always have the same number of numerical entries (either in total or before the series begins counting up again) nor are the same number of line breaks required in all text files.
I've been trying to use conditionals in awk or sed but haven't had any luck.
Thank you in advance for any suggestions/solutions.
note: edited to reflect 1st comment.
This may be what you want:
$ awk -v RS=' ' '{printf "%s%s", (NR>1?($0<p?ORS:OFS):""), $0; p=$0}' file
7.007 28.929 50.851 72.774
7.009 28.932 50.854 72.777
7.015 32.939 54.862 76.784
Here's one solution using awk:
{
for (i=1; i<=NF; ++i) {
if ($i < last) {
printf "\n"
last=-1
} else if (last > 0) {
printf " "
}
printf "%s", $i
last = $i
}
}
END { printf "\n" }
Example run:
$ awk -f foo.awk bar.txt
7.007 28.929 50.851 72.774 94.696 116.619 138.542 160.464 182.387 204.309 226.232 248.155 270.077 292 313.922 335.845 357.768 379.69 401.613 423.535 445.458 467.381
7.009 28.932 50.854 72.777 94.699 116.622 138.545 160.467 182.39 204.312 226.235 248.158 270.08 292.003 313.925 335.848 357.771 379.693 401.616 423.538 445.461 467.384 489.306
7.015 32.939 54.862 76.784 102.708 124.631 146.553 168.476 190.398 212.321 234.244 260.167 282.09 308.013 333.937 355.86 377.782 403.706
7.005 28.928 50.85 72.773 94.696 116.618 138.541 160.463 186.387 212.311 234.233 256.156 278.079 300.001 321.924 347.847
Another AWK solution:
awk 'BEGIN { RS=" "; ORS=" "; prev=-999 }
{ if ( $1<prev ) { printf "\n%.3f", $1 }
else { printf "%.3f ", $1 } prev=$1
}
END { print }'
In awk, how can I do this:
1303361997;15;67.067014
1303361997;5;51.529837
1303361997;14;47.036197
1303361997;3;44.064681
1303361997;6;37.632831
1303361997;23;24.990078
1303361997;24;26.750984
1303361998;15;67.074100
1303361998;5;51.522981
1303361998;14;47.028185
1303361998;3;44.056715
1303361998;6;37.638584
1303361998;23;24.987800
1303361998;24;26.756648
When number in second columns is absent this date should be replace by zero in output file.
First place is the number of the first column. The values of the second column of data to determine the position of the third column in the output file. The first column each time it may begin with different values. Desired output, by sorting first and second columns:
1303361997;0;0;44.064681;0;51.529837;37.632831;0;0;0;0;0;0;0;47.036197;67.067014;0;0;0;0;0;0;0;24.990078;26.750984;
1303361998;0;0;44.056715;0;51.522981;37.638584;0;0;0;0;0;0;0;47.028185;67.074100;0;0;0;0;0;0;0;24.987800;26.756648;
$ cat tst.awk
BEGIN { FS=";" }
NR == 1 {
for (i=1;i<=2;i++) {
min[i] = max[i] = $i
}
}
{
val[$1,$2] = $3
keys[$1]
for (i=1;i<=2;i++) {
min[i] = ($i < min[i] ? $i : min[i])
max[i] = ($i > max[i] ? $i : max[i])
}
}
END {
for (r=min[1];r<=max[1];r++) {
if (r in keys) {
printf "%d",r
for (c=1;c<=max[2];c++) {
printf ";%s", ((r,c) in val ? val[r,c] : 0)
}
print ";"
}
}
}
$
$ cat file
1303361997;15;67.067014
1303361997;5;51.529837
1303361997;14;47.036197
1303361997;3;44.064681
1303361997;6;37.632831
1303361997;23;24.990078
1303361997;24;26.750984
1303361998;15;67.074100
1303361998;5;51.522981
1303361998;14;47.028185
1303361998;3;44.056715
1303361998;6;37.638584
1303361998;23;24.987800
1303361998;24;26.756648
$
$ awk -f tst.awk file
1303361997;0;0;44.064681;0;51.529837;37.632831;0;0;0;0;0;0;0;47.036197;67.067014;0;0;0;0;0;0;0;24.990078;26.750984;
1303361998;0;0;44.056715;0;51.522981;37.638584;0;0;0;0;0;0;0;47.028185;67.074100;0;0;0;0;0;0;0;24.987800;26.756648;
I wrote wlan manager script to handle open/ad-hoc/wep/wpa2 networks. Now im trying to parse iw wlan0 scan output to get nice scan feature to my script. My goal is to get output like this :
SSID channel signal encryption
wlan-ap 6 70% wpa2-psk
test 1 55% wep
What i have achived already is output like this :
$ iw wlan0 scan | grep 'SSID\|freq\|signal\|capability' | tac
SSID: Koti783
signal: -82.00 dBm
capability: ESS Privacy ShortPreamble SpectrumMgmt ShortSlotTime (0x0531)
freq: 2437
I have been trying to study bash/sed/awk but havent found yet a way to achieve what im trying. So what is good way to achieve that?
Here is my final solution based of Sudo_O answer:
$1 == "BSS" {
MAC = $2
wifi[MAC]["enc"] = "Open"
}
$1 == "SSID:" {
wifi[MAC]["SSID"] = $2
}
$1 == "freq:" {
wifi[MAC]["freq"] = $NF
}
$1 == "signal:" {
wifi[MAC]["sig"] = $2 " " $3
}
$1 == "WPA:" {
wifi[MAC]["enc"] = "WPA"
}
$1 == "WEP:" {
wifi[MAC]["enc"] = "WEP"
}
END {
printf "%s\t\t%s\t%s\t\t%s\n","SSID","Frequency","Signal","Encryption"
for (w in wifi) {
printf "%s\t\t%s\t\t%s\t%s\n",wifi[w]["SSID"],wifi[w]["freq"],wifi[w]["sig"],wifi[w]["enc"]
}
}'
Output:
$ sudo iw wlan0 scan | awk -f scan.awk
SSID Frequency Signal Encryption
netti 2437 -31.00 dBm Open
Koti783 2437 -84.00 dBm WPA
WLAN-AP 2462 -85.00 dBm WPA
it's generally bad practice to try parsing complex output of programs intended for humans to read (rather than machines to parse).
e.g. the output of iw might change depending on the language settings of the system and/or the version of iw, leaving you with a "manager" that only works on your development machine.
instead you might use the same interface that iw uses to get it's information: the library backend libnl
you might also want to have a look at the wireless-tools (iwconfig, iwlist,...) that use the libiw library.
Here is an GNU awk script to get you going that grabs the SSIDs and the channel for each unique BSS:
/^BSS / {
MAC = $2
}
/SSID/ {
wifi[MAC]["SSID"] = $2
}
/primary channel/ {
wifi[MAC]["channel"] = $NF
}
# Insert new block here
END {
printf "%s\t\t%s\n","SSID","channel"
for (w in wifi) {
printf "%s\t\t%s\n",wifi[w]["SSID"],wifi[w]["channel"]
}
}
It should be easy for you to add the new blocks for signal and encryption considering all the studying you have been doing.
Save the script to file such as wifi.awk and run like:
$ sudo iw wlan0 scan | awk -f wifi.awk
The output will be in the formatted requested:
SSID channel
wlan-ap 6
test 1
Here is a simple Bash function which uses exclusively Bash internals and spawns only one sub-shell:
#!/bin/bash
function iwScan() {
# disable globbing to avoid surprises
set -o noglob
# make temporary variables local to our function
local AP S
# read stdin of the function into AP variable
while read -r AP; do
## print lines only containing needed fields
[[ "${AP//'SSID: '*}" == '' ]] && printf '%b' "${AP/'SSID: '}\n"
[[ "${AP//'signal: '*}" == '' ]] && ( S=( ${AP/'signal: '} ); printf '%b' "${S[0]},";)
done
set +o noglob
}
iwScan <<< "$(iw wlan0 scan)"
Output:
-66.00,FRITZ!Box 7312
-56.00,ALICE-WLAN01
-78.00,o2-WLAN93
-78.00,EasyBox-7A2302
-62.00,dlink
-74.00,EasyBox-59DF56
-76.00,BELAYS_Network
-82.00,o2-WLAN20
-82.00,BPPvM
The function can be easily modified to provide additional fields by adding a necessary filter into the while read -r AP while-loop, eg:
[[ "${AP//'last seen: '*}" == '' ]] && ( S=( ${AP/'last seen: '} ); printf '%b' "${S[0]},";)
Output:
-64.00,1000,FRITZ!Box 7312
-54.00,492,ALICE-WLAN01
-76.00,2588,o2-WLAN93
-78.00,652,LN8-Gast
-72.00,2916,WHITE-BOX
-66.00,288,ALICE-WLAN
-78.00,800,EasyBox-59DF56
-80.00,720,EasyBox-7A2302
-84.00,596,ALICE-WLAN08
I am using such solution for openwrt:
wlan_scan.sh
#!/bin/sh
sudo iw dev wlan0 scan | awk -f wlan_scan.awk | sort
wlan_scan.awk
/^BSS/ {
mac = gensub ( /^BSS[[:space:]]*([0-9a-fA-F:]+).*?$/, "\\1", "g", $0 );
}
/^[[:space:]]*signal:/ {
signal = gensub ( /^[[:space:]]*signal:[[:space:]]*(\-?[0-9.]+).*?$/, "\\1", "g", $0 );
}
/^[[:space:]]*SSID:/ {
ssid = gensub ( /^[[:space:]]*SSID:[[:space:]]*([^\n]*).*?$/, "\\1", "g", $0 );
printf ( "%s %s %s\n", signal, mac, ssid );
}
result
-62.00 c8:64:c7:54:d9:05 a
-72.00 70:72:3c:1c:af:17 b
-81.00 78:f5:fd:be:33:cb c
There is a bug in the awk script above.
The following code will not work if the SSID has spaces in the name. The received result will be the first token of the SSID name only.
$1 == "SSID:" {
wifi[MAC]["SSID"] = $2
}
When printing $0, $1, $2:
$0: SSID: DIRECT-82-HP OfficeJet 8700
$1: SSID:
$2: DIRECT-82-HP
One possibly solution is to take a substr of $0 which contains leading spaces, the token "SSID: " and the provided multi-token network name.
Any other suggestions?
I've taken awk code from Ari Malinen and reworked it a bit, because iw output is not stable and changes, also there are other issues like spaces in SSID. I put it on github in case if I'll change it in the future.
#!/usr/bin/env awk -f
$1 ~ /^BSS/ {
if($2 !~ /Load:/) { #< Escape "BBS Load:" line
gsub("(\\(.*|:)", "", $2)
MAC = toupper($2)
wifi[MAC]["enc"] = "OPEN"
wifi[MAC]["WPS"] = "no"
wifi[MAC]["wpa1"] = ""
wifi[MAC]["wpa2"] = ""
wifi[MAC]["wep"] = ""
}
}
$1 == "SSID:" {
# Workaround spaces in SSID
FS=":" #< Changing field separator on ":", it should be
# forbidded sign for SSID name
$0=$0
sub(" ", "", $2) #< remove first whitespace
wifi[MAC]["SSID"] = $2
FS=" "
$0=$0
}
$1 == "capability:" {
for(i=2; i<=NF; i++) {
if($i ~ /0x[0-9]{4}/) {
gsub("(\\(|\\))", "", $i)
if (and(strtonum($i), 0x10))
wifi[MAC]["wep"] = "WEP"
}
}
}
$1 == "WPA:" {
wifi[MAC]["wpa1"] = "WPA1"
}
$1 == "RSN:" {
wifi[MAC]["wpa2"] = "WPA2"
}
$1 == "WPS:" {
wifi[MAC]["WPS"] = "yes"
}
$1 == "DS" {
wifi[MAC]["Ch"] = $5
}
$1 == "signal:" {
match($2, /-([0-9]{2})\.00/, m)
wifi[MAC]["Sig"] = m[1]
}
$1 == "TSF:" {
gsub("(\\(|d|,)", "", $4)
match($5, /([0-9]{2}):([0-9]{2}):/, m)
day = $4
hour = m[1]
min = m[2]
wifi[MAC]["TSF"] = day"d"hour"h"min"m"
}
END {
for (w in wifi) {
if (wifi[w]["wep"]) {
if (wifi[w]["wpa1"] || wifi[w]["wpa2"])
wifi[w]["enc"] = wifi[w]["wpa1"]wifi[w]["wpa2"]
else
wifi[w]["enc"] = "WEP"
}
printf "%s:%s:%s:%s:%s:%s:%s\n", w, wifi[w]["SSID"], wifi[w]["enc"], \
wifi[w]["WPS"], wifi[w]["Ch"], wifi[w]["Sig"], wifi[w]["TSF"]
}
}
Output:
A5FEF2C499BB:test-ssid2:OPEN:no:9:43:0d00h00m
039EFACA9A8B:test-ssid2:WPA1:no:9:33:0d00h00m
038BF3C1988B:test-ssid2:WPA2:no:9:35:0d00h00m
028EF3C2997B:test-ssid2:WPA1:no:9:35:0d00h03m
if you wonder what if($2 !~ /Load:/) does, well on some routers there might be "BSS Load:" string.