This is a difficult one to extrapolate what I need, so bear with me. Let's say we have this subnet 10.4.x.0/28 which iterates itself across multiple subnets that look the same, i.e
10.4.0.0/28
10.4.0.16/28
10.4.0.32/28 etc...
and we have a single address in these subnets that we need to extract that will always be +12 of the current subnet. i.e
10.4.0.0/28 -> 10.4.0.12
10.4.0.16/28 -> 10.4.0.28
10.4.0.32/28 -> 10.4.0.44
How would I go about writing a shell script to loop through all of the subnets from
10.4.0.0/28 to 10.4.255.240 to get the single address in each subnet?
Hard to tell what format you want exactly, but ...
for I in $(seq 12 16 256); do echo 10.4.{0..255}.$I; done
Gives 16 lines with 4096 values like this...
10.4.0.12 10.4.1.12 10.4.2.12 10.4.3.12 10.4.4.12 10.4.5.12 10.4.6.12 10.4.7.12 10.4.8.12 10.4.9.12 10.4.10.12 10.4.11.12 10.4.12.12 10.4.13.12 10.4.14.12 10.4.15.12 10.4.16.12 10.4.17.12 ...
10.4.0.28 10.4.1.28 10.4.2.28 10.4.3.28 10.4.4.28 10.4.5.28 10.4.6.28 10.4.7.28 10.4.8.28 10.4.9.28 10.4.10.28 10.4.11.28 10.4.12.28 10.4.13.28 10.4.14.28 10.4.15.28 10.4.16.28 10.4.17.28 ...
[...]
10.4.235.252 10.4.236.252 10.4.237.252 10.4.238.252 10.4.239.252 10.4.240.252 10.4.241.252 10.4.242.252 10.4.243.252 10.4.244.252 10.4.245.252 10.4.246.252 10.4.247.252 10.4.248.252 10.4.249.252 10.4.250.252 10.4.251.252 10.4.252.252 10.4.253.252 10.4.254.252 10.4.255.252
If you want to change the top-level addresses on the fly, you could make a function...
iterateIP(){
for I in $(seq 12 16 256); do echo $1.{0..255}.$I; done
}
And call like this:
iterateIP 10.4
Related
I'm currently experimenting with a little bash script.
I have an IPv6 address (via mysql query) in a variable and want to increment it by 1 (by also honoring the IPv6 rules [hex]). Any hint how to achieve this?
Is there a simple way in bash to increment a given address by 1? (this is to avoid duplicates).
Basically:
2001::0000:fe04 is given and I want the script to manipulate the variable to output 2001::0000:fe05, 2001::0000:fe06 and after 2001::0000:fe09 it should be 2001::0000:fe0a - 2001::0000:fe0f followed by 2001::0000:fe10 (hex count). If all ip's are full (2001::0:ffff) it should use the next IP subnet by honoring IPv6 subnetting.
So after 2001::0:ffff the next address would be 2001::0001:0001
Hope you understand what I mean ;)
This will add 1 to your ip6 address using the Net::IP perl package.
#!/usr/bin/env perl
use strict;
use Net::IP qw(ip_inttobin ip_bintoip);
my $ip = Net::IP->new('2001::0000:fe05') || die;
print ("IP : ".$ip->ip()."\n");
print ("Sho : ".$ip->short()."\n");
print ("Int : ".$ip->intip()."\n");
# add 1 to the address
my $next_bin = ip_inttobin($ip->intip + 1, $ip->version);
my $next = Net::IP->new(ip_bintoip($next_bin, $ip->version));
print "\nAfter incrementing by 1\n";
print ("IP : ".$next->ip()."\n");
print ("Sho : ".$next->short()."\n");
$ perl ip6_test.pl
IP : 2001:0000:0000:0000:0000:0000:0000:fe05
Sho : 2001::fe05
Int : 42540488161975842760550356425300311557
After incrementing by 1
IP : 2001:0000:0000:0000:0000:0000:0000:fe06
Sho : 2001::fe06
I have created a snmp manager which will calculate the traffic of a list of ip address.
Here is the snmp walk cammand :
walk snmpwalk = new walk(" 192.168.30.172" ,". 1.3.6.1.2.1.2.2.1.10", "");
System.out.println(",,,,,,,,,,,,,,,," + snmpwalk.doSnmpwalk());
Here is the output:
1.3.6.1.2.1.2.2.1.10.1=0
1.3.6.1.2.1.2.2.1.10.2=0
1.3.6.1.2.1.2.2.1.10.3=0
1.3.6.1.2.1.2.2.1.10.4=0
1.3.6.1.2.1.2.2.1.10.5=0
1.3.6.1.2.1.2.2.1.10.6=0
1.3.6.1.2.1.2.2.1.10.7=0
1.3.6.1.2.1.2.2.1.10.8=0
1.3.6.1.2.1.2.2.1.10.9=0
1.3.6.1.2.1.2.2.1.10.10=0
1.3.6.1.2.1.2.2.1.10.11=0
1.3.6.1.2.1.2.2.1.10.12=0
1.3.6.1.2.1.2.2.1.10.13=0
1.3.6.1.2.1.2.2.1.10.14=163292301
1.3.6.1.2.1.2.2.1.10.15=0
1.3.6.1.2.1.2.2.1.10.16=0
1.3.6.1.2.1.2.2.1.10.17=0
1.3.6.1.2.1.2.2.1.10.18=0
1.3.6.1.2.1.2.2.1.10.19=0
1.3.6.1.2.1.2.2.1.10.20=0
1.3.6.1.2.1.2.2.1.10.21=0
1.3.6.1.2.1.2.2.1.10.22=0
1.3.6.1.2.1.2.2.1.10.23=0
1.3.6.1.2.1.2.2.1.10.24=163292301
1.3.6.1.2.1.2.2.1.10.25=163292301
1.3.6.1.2.1.2.2.1.10.26=163292301
1.3.6.1.2.1.2.2.1.10.27=163292301
1.3.6.1.2.1.2.2.1.10.28=163292301
1.3.6.1.2.1.2.2.1.10.29=0
1.3.6.1.2.1.2.2.1.10.30=0
1.3.6.1.2.1.2.2.1.10.31=0
1.3.6.1.2.1.2.2.1.10.32=0
1.3.6.1.2.1.2.2.1.10.33=0
1.3.6.1.2.1.2.2.1.10.34=0
1.3.6.1.2.1.2.2.1.10.35=0
1.3.6.1.2.1.2.2.1.10.36=0
1.3.6.1.2.1.2.2.1.10.37=0
1.3.6.1.2.1.2.2.1.10.38=0
1.3.6.1.2.1.2.2.1.10.39=0
Now my question is it possible to calculate the traffic(monthly data usage) at the end of the month of this IP device?
if possible then how can i do that? What is the formula and calculation process?
On Linux I would use awk, e.g.:
awk 'BEGIN{RS="\r"} /::if(In|Out)Octets/{i=$1; sub(/^.*\./,"",i); t[i]+=$4} /::ifDescr/{i=$1; sub(/^.*\./,"",i); d[i]=$4}; END{for(i in t) if(t[i]>0) printf "%s %.2fMB\n",d[i],t[i]/1048576}' Cisco_3750_snmpwalk.txt
It sums ifInOctets and ifOutOctets counters, and displays the sum in MB next to ifDescr string if it is greater than 0. Output for your file looks like this:
Vlan66 0.00MB
Vlan100 2829.20MB
Vlan117 780.88MB
Vlan230 40.42MB
Vlan237 28.77MB
Vlan242 0.01MB
Vlan302 1902.10MB
Vlan312 181.48MB
Vlan325 738.01MB
Vlan328 253.37MB
Vlan330 0.08MB
Vlan331 590.89MB
Vlan337 320.17MB
Vlan356 3247.89MB
Vlan358 1360.38MB
Vlan362 45.38MB
Vlan364 2179.56MB
Vlan400 0.00MB
Vlan404 105.89MB
Vlan413 157.37MB
Vlan466 3487.25MB
Vlan467 0.00MB
Vlan510 2547.83MB
Vlan548 3033.33MB
Vlan603 0.01MB
Vlan624 0.00MB
Vlan667 0.01MB
Vlan725 1.58MB
Vlan776 103.93MB
Vlan875 29.90MB
Vlan908 15.19MB
Vlan910 27.36MB
Vlan936 13.78MB
Vlan956 13.91MB
Vlan965 68.12MB
Vlan966 298.39MB
Vlan967 167.47MB
Vlan1000 0.00MB
GigabitEthernet1/0/1 5629.52MB
GigabitEthernet1/0/2 4220.78MB
GigabitEthernet1/0/3 6369.15MB
GigabitEthernet1/0/9 3866.67MB
GigabitEthernet1/0/15 5511.63MB
GigabitEthernet1/0/16 6030.57MB
GigabitEthernet1/0/18 4251.62MB
I am working with Graphchi's pagerank example: https://github.com/GraphChi/graphchi-cpp/wiki/Example-Apps#pagerank-easy
The example app writes a binary file with vertex information that I would like to read/convert to a plan text file (to later call into R or some other language).
The documentation states that:
"GraphChi will write the values of the edges in a binary file, which is easy to handle in other programs. Name of the file containing vertex values is GRAPH-NAME.4B.vout. Here "4B" refers to the vertex-value being a 4-byte type (float)."
The 'easy to handle' part is what I'm struggling with - I have experience with high level languages but not C++ or dealing with binary files. I have found a few things through searching stackoverflow but no luck yet in reading this file. Ideally this would be done through bash or python.
thanks very much for your help on this.
Update: hexdump graph-name.4B.vout | head -5 gives:
0000000 999a 3e19 7468 3e7f 7d2a 3e93 d8e0 3ec4
0000010 cec6 3fe4 d551 3f08 eff2 3e54 999a 3e19
0000020 999a 3e19 3690 3e8c 0080 3f38 9ea3 3ef5
0000030 b7d6 3f66 999a 3e19 10e3 3ee1 400c 400d
0000040 a3df 3e7c 999a 3e19 979c 3e91 5230 3f18
Here is example code how you can use GraphCHi to write the output out as a string:
https://github.com/GraphChi/graphchi-cpp/wiki/Vertex-Aggregators
But the array is simple byte array. Here is example how to read it in python:
import struct
from array import array as binarray
import sys
inputfile = sys.argv[1]
data = open(inputfile).read()
a = binarray('c')
a.fromstring(data)
s = struct.Struct("f")
l = len(a)
print "%d bytes" %l
n = l / 4
for i in xrange(0, n):
x = s.unpack_from(a, i * 4)[0]
print ("%d %f" % (i, x))
I was having the same trouble. Luckily I work with a bunch of network engineers who helped me out! On Mac Linux, the following command works to print the 4B.vout data one line per node, with the integer values the same as is given in the summary file. If your file is called eg, filename.4B.vout, then some command line perl gets you:
cat filename.4B.vout | LANG= perl -0777 -e '$,=\"\n\"; print unpack(\"L*\",<>),\"\";'
Edited to add: this is for the assignments of connected component ID and community ID, written implicitly the 1st line is the ID of the node labeled 0, the 2nd line is the node labeled 1 etc. But I am copypasting here so I'm not sure how it would need to change for floats. It works great for the integer values per node.
I am just new to scripting and I need some help. I have something like a bazillion files that look like this.
Assign F2 Height
3IleN 2.34025e+07
4PheN 2.05028e+07
6LysN 1.43672e+07
7ThrN 1.49120e+07
8LeuN 1.30838e+07
9ThrN 1.44298e+07
And i want it to look like this + save it in another file with the same name as the previous file however, with a "MOD" written at the beginning.
Number AA Height
3 IleN 6.20756e+07
4 PheN 5.26499e+07
7 ThrN 3.00216e+07
8 LeuN 3.26377e+07
9 ThrN 4.03901e+07
10 GlyN 2.73659e+07
12 ThrN 3.16319e+07
13 IleN 5.94604e+07
If you could please describe and explain the parameters used, that would be of great help.
Thanks!
The following should work for you:
sed 's/^\([0-9]*\)/\1 /' filename
I have an input file that contains a list of ip addresses and the ip_counts(some parameter that I use internally.)The file looks somewhat like this.
202.124.127.26 2135869
202.124.127.25 2111217
202.124.127.17 2058082
202.124.127.16 2014958
202.124.127.20 1949323
202.124.127.24 1933773
202.124.127.27 1932076
202.124.127.22 1886466
202.124.127.18 1882955
202.124.127.21 1803528
202.124.127.23 1786348
119.224.129.200 1776592
119.224.129.211 1639325
202.124.127.19 1479198
119.224.129.201 1145426
202.49.175.110 1133354
119.224.129.210 1119525
68.232.45.132 1085491
119.224.129.209 1015078
131.203.3.8 857951
202.162.73.4 817197
207.123.58.125 785326
202.7.6.18 762603
117.121.253.254 718022
74.125.237.120 710448
68.232.44.219 693002
202.162.73.2 671559
205.128.75.126 611301
119.161.91.17 604393
119.224.129.202 559930
8.27.241.126 528862
74.125.237.152 517516
8.254.9.254 514341
As you can see the ip addresses themselves are unsorted.So I use the sort command on the file to sort the ip addresses as below
cat address_count.txt | sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n > sorted_address.txt
Which gives me an output with ip addresses in the sorted order.The partial output of that file is shown below.
4.23.63.126 15731
4.26.254.254 320705
4.27.8.254 25174
8.12.129.50 176141
8.12.223.125 11800
8.19.32.65 15854
8.19.240.53 11013
8.19.240.70 11915
8.19.240.72 31541
8.19.240.73 23304
8.20.213.28 96434
8.20.213.32 108191
8.20.213.34 170058
8.20.213.39 23512
8.20.213.41 10420
8.20.213.61 24809
8.26.195.253 28568
8.27.152.253 104446
8.27.233.125 115856
8.27.235.126 16102
8.27.235.254 25628
8.27.238.254 108485
8.27.240.125 169262
8.27.241.126 528862
8.27.241.252 197302
8.27.248.125 14926
8.254.9.254 514341
12.129.210.71 89663
15.192.45.21 20139
15.192.45.26 35265
15.193.0.148 10313
15.193.113.29 40318
15.201.49.136 14243
15.240.238.52 57163
17.250.248.95 28166
23.33.125.13 19179
23.33.125.37 17953
31.151.163.60 72709
38.99.42.37 192356
38.99.68.180 41251
38.99.68.181 10272
38.104.237.74 74012
38.108.112.103 37034
38.108.112.115 69698
38.108.112.121 92173
38.108.112.122 99230
38.112.63.238 39958
38.119.130.62 42159
46.4.28.22 19769
Now I want to parse the file given above and convert it to aaa.bbb.ccc.0/8 format and
aaa.bbb.0.0/16 format and I also want to count the number of occurences of the ip's in each subnet.I want to do this using bash.I am open to using sed or awk.How do I achieve this.
For example
8.19.240.53 11013
8.19.240.70 11915
8.19.240.72 31541
8.19.240.73 23304
8.20.213.28 96434
8.20.213.32 108191
8.20.213.34 170058
8.20.213.39 23512
8.20.213.41 10420
8.20.213.61 24809
The about input portion should produce 8.19.240.0/8 and 8.20.213.0/8 and similarly for /16 domains.I also want to count the occurences of machines in the subnet.
For example In the above output this subnet should have the count 4 in the next column beside it.It should also add the already displayed count.i.e (11013 + 11915 + 31541 + 23304) in another column.
8.19.240.0/8 4 (11013 + 11915 + 31541 + 23304)
8.20.213.0/8 6 (96434 + 108191 + 170058 + 23512 + 10420 + 24809)
It would be great if someone could suggest some way to achieve this.
The main problem here is that without having the routing table from the individual moments the packets arrived, you have no idea what netblock they were originally in. Sure, you can put them in the class-full blocks they would be in, in a class-full routing situation, but all that will give you is a nice presentation (and, admittedly, a shorter file).
Furthermore, your example looks a bit broken. You have a bunch of IP addresses in 8.0.0.0/8 and you are aggregating them into what looks like /24 routes and presenting them with a /8 at the end.
Nonetheless, in awk you can use sub() to do text replacement (or you can use index to find occurrences of ., or you can use split to split at dots). It should be relatively easy to go from that to "drop last digit, add the string "0/24" and use that as a key to update an IP-count and a hit-count dictionary, then drop the last two octets and the slash, replace with "0.0/16" and do the same" (all arrays in awk are associative arrays, so essentially dicts). No need to sort in advance, when you loop through the result, you'll get the keys in a random order, but on average there will be fewer of them, so sorting afterwards will be cheaper.
I seem to not have an awk at hand, so I cannot give you a code example.
This might work for you:
awk '{a=$1;sub(/\.[^.]*$/,"",a);ac[a]++;at[a]+=$2};END{for(x in ac)print x".0/8",ac[x],at[x]}' file
This prints the '0/8 addresses to get the 0/16 duplicate the code i.e. b=a;sub(/\.[^.]*$/,"",b);ba[b]++ etc, etc.