how can i convert the total number of hosts in a network to mask?
for example, 1024 to /22
ip calc can do the opposite
$ ipcalc 89.150.4.0/22
Address: 89.150.4.0 01011001.10010110.000001 00.00000000
Netmask: 255.255.252.0 = 22 11111111.11111111.111111 00.00000000
Wildcard: 0.0.3.255 00000000.00000000.000000 11.11111111
=>
Network: 89.150.4.0/22 01011001.10010110.000001 00.00000000
HostMin: 89.150.4.1 01011001.10010110.000001 00.00000001
HostMax: 89.150.7.254 01011001.10010110.000001 11.11111110
Broadcast: 89.150.7.255 01011001.10010110.000001 11.11111111
Hosts/Net: 1022 Class A
background information:
i have a text file with content like
AD|85.94.160.0|8192
AD|89.150.2.0|512
AD|89.150.4.0|1024
AD|89.150.8.0|2048
and i need to convert the 3rd column (total number of hosts) to network mask
if it's not possible using shell tools, i don't mind piping it to a python/perl/ruby script
EDIT
very concise answer and comments from #KamilCuk helped me understand the logic behind it.
i'm not even using ipcalc for that anymore :)
THE ANSWER
from hosts to mask
$: HOSTS=1024
$: LC_NUMERIC=C printf "%.0f" $(bc -l <<< "32-(l($HOSTS)/l(2))")
22
or
$: HOSTS=1024
$: echo | awk -v HOSTS=$HOSTS '{print int(32-(log(HOSTS)/log(2)))}'
22
and from mask to hosts
$: MASK=22
$: echo "2^(32-$MASK)" | bc
1024
or
$: MASK=22
$: echo | awk -v MASK=$MASK '{print 2^(32-MASK)}'
1024
You can calculate in bc or awk the logarithm. Then just substract from 32.
For your input file that would be:
awk -F'|' '{print $2"/"32 - log($3)/log(2)}'
Related
I am trying to write a function that compares values in 2 files.
I have a logfile from which I have extracted unique IP addresses.
I have another file that has a list of domains in them which are "bad domians".
The objective is to print the list of IP addresses and also compare with the bad domains list and if a match is found, we need to prefix "bad address".
Conceptually, I can save the IP result to a file, and the domain result to a variable, use the while read loop on the IP file, a for loop on the domain variable and use grep to see if a pattern is found. If yes, add the prefix, else print normally. Yes, seems a bit time consuming but that is the idea.
list=`dig +short -f dns.blacklist.txt`
awk '{ print $1 }' $logfile | sort | uniq -c | sort -nr | awk '{print $2 "\t" $1}' >> response
while read -r listed
do
for x in $list
do
if [ "$(echo $listed | grep -F $x )" ]; then
echo $listed "*bad domain!*"
else
#echo $listed
fi
done
done < response | uniq
It does find the bad IP and adds the prefix, problem is, it creates a duplicate.
What it should be
213.64.237.230 2438
213.64.225.123 1202 *bad domain!*
213.64.141.89 731
213.64.214.124 480
.
.
.
What it shows
213.64.237.230 2438
213.64.225.123 1202
213.64.225.123 1202 *bad domain!*
213.64.225.123 1202
213.64.141.89 731
213.64.214.124 480
.
.
.
I fail to see why a duplicate is being made. If I remove the else condition and push the normal print to after, it still prints the IP below while it shouldnt.
Do note that the results are being piped to the uniq command.
I need a hint on where I am wrong and how I can mitigate this error.
Since I did not have access to your logfile or your dns.blacklist.txt files. I created dummy files using some of the ip addresses you listed. I refactored your loops and your if check and was able to solve your duplicate output issue.
#!/bin/bash
# used instead of your list=`dig +short -f dns.blacklist.txt`
bad=/tmp/bad.txt
# used instead of your logfile parsing/sorting/ etc...
all=/tmp/all.txt
# read blacklisted ips into a bash variable
list=$(<${bad})
# for each ip parsed from your logfile count number of times
# it is found in blacklisted ips, if greater than 0 then flag
# as a bad domain
while read -r listed
do
if [[ $(grep -c "${listed}" <<<${list}) -gt 0 ]]; then
echo "${listed} *bad domain!*"
else
echo "${listed}"
fi
done <${all}
Contents of bad.txt:
213.64.225.123 1202
Contents of all.txt:
213.64.237.230 2438
213.64.225.123 1202
213.64.141.89 731
213.64.214.124 480
Example output:
$ ./script.sh
213.64.237.230 2438
213.64.225.123 1202 *bad domain!*
213.64.141.89 731
213.64.214.124 480
This question was recently asked in an interview.
Question: Write a bash script to subdivide an given subnet into a pre-defined number of smaller subnets.
After division IP addresses shouldn't be wasted, i.e. accumulation of your subdivisions should make up the divided subnet.
Every subnet has 3 IP addresses reserved and not usable by hosts: network, broadcast, gateway.
Show network/broadcast address, number of hosts and assign gateway. Gateway should be first IP available in divided subnet. Examples:
INPUT: ./subnetter.sh 192.168.0.0/24 3
OUTPUT:
subnet=192.168.0.0/25 network=192.168.0.0 broadcast=192.168.0.127 gateway=192.168.0.1 hosts=125
subnet=192.168.0.128/26 network=192.168.0.128 broadcast=192.168.0.191 gateway=192.168.0.129 hosts=61
subnet=192.168.0.192/26 network=192.168.0.192 broadcast=192.168.0.255 gateway=192.168.0.193 hosts=61
INPUT: ./subnetter.sh 192.168.0.0/24 4
OUTPUT:
subnet=192.168.0.0/26 network=192.168.0.0 broadcast=192.168.0.63 gateway=192.168.0.1 hosts=61
subnet=192.168.0.64/26 network=192.168.0.64 broadcast=192.168.0.127 gateway=192.168.0.65 hosts=61
subnet=192.168.0.128/26 network=192.168.0.128 broadcast=192.168.0.191 gateway=192.168.0.129 hosts=61
subnet=192.168.0.192/26 network=192.168.0.192 broadcast=192.168.0.255 gateway=192.168.0.193 hosts=61
INPUT: ./subnetter.sh 10.55.10.64/28 2
OUTPUT:
subnet=10.55.10.64/29 network=10.55.10.64 broadcast=10.55.10.71 gateway=10.55.10.65 hosts=5
subnet=10.55.10.72/29 network=10.55.10.72 broadcast=10.55.10.79 gateway=10.55.10.73 hosts=5
First of all, I am trying to analyse what logic is used to divide the subnets.
Secondly, I am trying to use the ipcalc command to get outputs but no luck.
Thanks
Here is the bash script, I have tried my hands on:
Should work fine. The script takes 2 arguments, CIDR block and number of subnets to divide into.
#!/bin/bash
cidr=$1
total_subnet=$2
nw_addr=`echo $cidr | awk -F'/' '{ print $1 }'` # retrieving network IP from input 1
nw_mask=`echo $cidr | awk -F'/' '{ print $2 }'` # retrieving network mask from input 1
dbit=`echo $nw_addr | awk -F'.' '{ print $4 }'` # retrieving the D-bit from network ( A.B.C.D )
significant_bit=`echo $nw_addr | awk -F'.' 'BEGIN {OFS = ""}{print $1,".",$2,".",$3 }'` # retrieving A.B.C bits from n/w address
change_bit=$(( 32 - $nw_mask))
max_addr=$(( 2 ** $change_bit)) # calculating maximum addresses available in the network that can be subdivided
dbit_max=$dbit
# A Funtion to calculate the least power of 2 that is equal or greater than the argument
least_greater_power_of_two()
{
next_power=2
power=1
subnet_range=$1
while [ $next_power -lt $subnet_range ]; do
power=$(($power+1))
next_power=$(( 2 ** $power))
done
}
#initialising Loop Variables
remaining_addr=$max_addr
max_subnet_dbit=$dbit
total_subnet_addr=0
starting_dbit=$dbit
i=$total_subnet
while [ $i -gt 0 ]; do
starting_dbit=$(( $starting_dbit + $total_subnet_addr )) #Finding the starting D bit of the subnet
#finding the total number of addresses in the subnet
subnet_range=$(( $remaining_addr / $i ))
least_greater_power_of_two $subnet_range
total_subnet_addr=$(( 2 ** $power ))
max_subnet_dbit=$(( $max_subnet_dbit + $total_subnet_addr ))
remaining_addr=$(( $remaining_addr - $total_subnet_addr )) # Remaining addresses left to be assigned to the other subnets
last_dbit=$(( $max_subnet_dbit - 1)) #calculating last D bit in the subnet range
subnet_mask=$(( $change_bit - $power + $nw_mask )) #calculating the subnet mask
gateway_dbit=$(( $starting_dbit + 1 )) # calculating the Gateway D bit
total_hosts=$(( $total_subnet_addr - 3 )) # calculating the Total-hosts in the network
echo "Subnet= $significant_bit.$starting_dbit/$subnet_mask Network=$significant_bit.$starting_dbit Broadcast=$significant_bit.$last_dbit Gateway= $significant_bit.$gateway_dbit Hosts=$total_hosts"
i=$(($i-1)) # updating loop variable
done
I believe I have done 70% of the stuff that you require off the script. Since you were using ipcalc, dediced to use a similar binary. To start off, do the following:
yum install sipcalc if you have a RPM based OS or apt-get install sipcalc depending on the distro of your Linux OS.
Then write the following script and save it as subnetter.sh and give it 'x' permissions so that it can be executed.
#!/bin/bash
if [ $# == 0 ]; then
echo "Usage: ./subnetter.sh IP/SUBNET RANGE"
exit
fi
subnet=$1
network=`echo $subnet | cut -d / -f1`
broadcast=`/usr/bin/sipcalc $1 | grep -i broadcast | cut -d '-' -f2`
gateway=`/usr/bin/sipcalc $1 | grep -i usable | awk '{print $4}'`
hosts=`/usr/bin/sipcalc $1 | grep -i addresses | cut -d '-' -f2`
echo "subnet =" $subnet "network =" $network "broadcast =" $broadcast "gateway =" $gateway "hosts =" $hosts
Output of my script:
[root#puppet ~]# ./subnetter.sh 192.168.0.0/24
subnet = 192.168.0.0/24 network = 192.168.0.0 broadcast = 192.168.0.255 gateway = 192.168.0.1 hosts = 256
Please note that the requirement of the third argument is very simple and can be simply done using a for loop. I expect you to do that.
You can use the following tool to make sure that your output is correct: http://www.subnet-calculator.com/subnet.php?net_class=C
I have gone through the above requirment and below is what i have programmed to achieve the result .
Python Code integrated with the above shell script to achieve the result as users are expecting
Below code will create sub subnet for the existing subnet and then will call the shell script to perform the operation in loop and will provide records based on the users request.
#
from netaddr import *
import ipaddress
import csv
import subprocess
import os
import shlex
import sys
import numpy as np
from itertools import islice
if os.path.exists('hst.txt'):
os.remove('hst.txt')
else:
print("Sorry, I can not remove " )
if os.path.exists('hst_refined.txt'):
os.remove('hst_refined.txt')
else:
print("Sorry, I can not remove " )
fd = open('store_subnet',"w")
enter_subnet = raw_input('Enter subnet please: ')
fd.write(enter_subnet)
fd = open('store_sub_subnet',"w")
sub_subnet = raw_input('Enter nested_subet_count: ')
fd.write(sub_subnet)
ip=ipaddress.ip_network(unicode(enter_subnet))
list_demo = list(ip.subnets(int(sub_subnet)))
for i in list_demo:
hs = open("hst.txt","a")
hs.write(str(i))
hs.write("\n")
hs.close()
p = subprocess.Popen([ "/home/ramkumar5/network_cal/report_demo.sh" ], stdout=subprocess.PIPE).communicate()[0]
for i in p:
hs = open("hst_refined.txt","a")
hs.write(i)
hs.close()
print(sub_subnet)
records_req = raw_input('Enter Number of Records needed: ')
f=open("hst_refined.txt")
for i in xrange(int(records_req)):
line=f.next().strip()
print line
f.close()
#
#
Code2
#!/bin/bash
for res in `cat hst.txt` ; do
subnet=$res
network=`echo $subnet | cut -d / -f1`
broadcast=`/usr/bin/sipcalc $res | grep -i broadcast | cut -d '-' -f2`
gateway=`/usr/bin/sipcalc $res | grep -i usable | awk '{print $4}'`
hosts=`/usr/bin/sipcalc $res | grep -i addresses | cut -d '-' -f2`
echo "subnet =" $subnet "network =" $network "broadcast =" $broadcast "gateway =" $gateway "hosts =" $hosts
done
#
Sample Out from the result###
Trying to use grep to find some information. Evaluate the information. Then perform a function.
Heres what I have, any help is appreciated.
FIXED TO:
#! /bin/bash
UT=$(/usr/sbin/system_profiler SPSoftwareDataType | grep "Time since boot" | grep "days")
if [ "$UT" -ge "5 days" ]; then
echo this
else
echo that
fi
SPSoftwareDataType looks like this:
System Software Overview:
System Version: OS X 10.9.5
Kernel Version: Darwin 13.4.0
Boot Volume: Macintosh HD
Boot Mode: Normal
Computer Name: xxxxxxxxxxxx
User Name: xxxxxxxxxx (xxxxxxxxxx)
Secure Virtual Memory: Enabled
Time since boot: 8 days 3:25
Trying using sysctl
#! /bin/bash
UT=$(awk -F":" ' $4 > 200 ' sysctl -n kern.boottime)
echo $UT
if [ “$UT” -ge “1430315296” ]; then
echo this
else
echo that
fi
I can't see how you expect awk to compare anything specified in days and hours with anything else... nor do I know why you would choose to parse system_profiler output.
Have you considered:
sysctl -n kern.boottime
{ sec = 1431023230, usec = 0 } Thu May 7 19:27:10 201
which will give the boot time in seconds since the epoch which is just a simple integer you can compare with other times?
So, you can parse out the seconds like this
UT=$(sysctl -n kern.boottime | awk -F"[ ,]+" '{print $4}')
the -F"[ ,]+" says to treat multiple spaces or commas as field separators.
You can do all in awk
awk '/Time since boot.*days/ {print ($4>5?"This":"That")}' file
This
awk '/Time since boot.*days/ {print ($4>8?"This":"That")}' file
That
Edit:
#!/bin/bash
/usr/sbin/system_profiler SPSoftwareDataType | awk '/Time since boot.*days/ {print ($4>8?"This":"That")}'
#!/bin/bash
CURRENT=$(df -h / | grep / | awk '{ print $4}')
THRESHOLD=75
if (( "$CURRENT" >= "$THRESHOLD" )); then
mail -s "CENTOS-6 localhost 10.10.1.238 Disk Space Alert" sss#abc.net << EOF
Your root partition remaining free space is critically low. Used: $CURRENT%
EOF
fi
I got the following error when i run the script, syntax error: operand expected (error token is ">= 75 ")
It's because CURRENT will contain a percent sign, so it won't be a valid operand for the comparison operation.
You can remove the last character like this :
CURRENT=${CURRENT%?};
Also make sure that df -h / | grep / | awk '{ print $4}' is correctly returning the usage ratio, on most systems you have to use print $5.
A couple of things:
you don't need grep at all, awk is quite capable of doing it's own regex stuff.
if you search for / in the df output, you'll probably get most lines as most mounts have a / somewhere in them. If you just want the root mountpoint, you can use <space>/$.
Check that 4 is the correct field number, on my box it's 5.
In any case, that field is of the form 55% which will not be considered numeric. You can use gsub to get rid of it.
With that in mind, the following snippet can be used to get the percentage:
df -h | awk '$0 ~ / \/$/ { gsub("%","",$5); print $5 }'
And, just as an aside, I'm not that big a fan of here-docs in shell scripts since it either (1) screws up my nicely indented files; or (2) makes me burn half an hour while I try to remember the various syntax options which will allow indented EOF strings :-)
I prefer something like:
(
echo Your root partition remaining free space is critically low: Used: ${CURRENT}%
) | mail -s "CENTOS-6 localhost 10.10.1.238 Disk Space Alert" sss#abc.net
Especially since that means I can put arbitrarily complex commands in the sub-shell to generate whatever info I want in the mail message (rather than just simple text substitutions).
So, bottom line, I'd be looking at something more like:
#!/usr/bin/env bash
# Config section.
LIMIT=75
# Code section.
CURR=$(df -h | awk '$0 ~ / \/$/ { gsub("%","",$5); print $5 }')
if [[ ${CURR} -ge ${LIMIT} ]] ; then
(
echo "Your root partition remaining free space is critically low: Used: ${CURR}%"
) | mail -s "CENTOS-6 localhost 10.10.1.238 Disk Space Alert" sss#abc.net
fi
Just try:
CURRENT=$(df -h |awk '{print $4}' |sort -n |tail -n1 |sed 's/%//g')
THRESHOLD=90
if [ $THRESHOLD -gt $CURRENT ]
I am working on a script which does subnet calculations. So far, it looks like this (in part):
echo "Subnet Address : "$sn1.$sn2.$sn3.$sn4
echo "BCast Address : "$br1.$br2.$br3.$br4
echo -e "\nSubnet address in binary" :
echo "obase=2;$ip1"+"obase=2;$ip2"+"obase=2;$ip3"+"obase=2;$ip4" \
| bc | awk '{printf("%08d",$ip1)}'
echo -e "\nBroadcast address in binary" :
echo "obase=2;$br1"+"obase=2;$br2"+"obase=2;$br3"+"obase=2;$br4" \
| bc | awk '{printf("%08d",$br1)}'
which gives me this output:
Subnet address in binary :
11000010101010100000001100000000
Broadcast address in binary :
11000010101010100000001100011111
I tried '{printf("%08d.",$br1)}' and '{printf(".%08d",$br1)}' to separate the octets but I get an extra dot on the beginning or at the end.
I want to calculate how many aces the netmask has, but I really cant find a way to get the output of the echo "obase=2;$br1"+"obase=2;$br2"+"obase=2;$br3"+"obase=2;$br4"| bc | awk to a string so I can count them.
Any suggestions?
So you want the output to be in the format of: 01111111.00000000.00000000.00000001?
Well, the cheat method, I used when I banged my head against this was to wrap around ipcalc:
%ipcalc 127.0.0.1
Address: 127.0.0.1 01111111.00000000.00000000. 00000001
Netmask: 255.255.255.0 = 24 11111111.11111111.11111111. 00000000
Wildcard: 0.0.0.255 00000000.00000000.00000000. 11111111
=>
Network: 127.0.0.0/24 01111111.00000000.00000000. 00000000
HostMin: 127.0.0.1 01111111.00000000.00000000. 00000001
HostMax: 127.0.0.254 01111111.00000000.00000000. 11111110
Broadcast: 127.0.0.255 01111111.00000000.00000000. 11111111
Hosts/Net: 254 Class A, Loopback
And then extract what I needed. Runs MUCH faster than parsing multiple times through 'bc'. Ie, no sense re-inventing the wheel if you don't need to.
If you do feel like re-inventing the wheel a bit:
$ echo "obase=2;200" + "obase=2;150" + "obase=2;200" + "obase=2;150" | \
bc | awk '{printf "%08d\." ,$1}' | \
sed -e 's/[.]*$//'
11001010.10011000.11001010.10010110
That will get you the 8 digit binary output format you want.
To separate the octets I would use printf and paste instead, e.g.:
ip1=127; ip2=0; ip3=0; ip4=1
printf "%08d\n" $(bc <<<"obase=2; $ip1; $ip2; $ip3; $ip4") | paste -sd.
Or if the ip is in one variable:
ip=127.0.0.1
printf "%08d\n" $(bc <<<"obase=2; ${ip//./;}") | paste -sd.
Output:
00001010.00101010.01100110.11111100
To count the number of 1s in a netmask just add them:
netmask=255.255.255.252
printf "%08d" $(bc <<<"obase=2; ${netmask//./;}") | grep -o . | paste -sd+ | bc
Output:
30
Two quick solutions:
$ echo $br1.$br2.$br3.$br4 |
perl -F\\\. -anE 'say join ".", map { sprintf "%08b", $_ } #F'
$ perl -e 'printf( "%08b.%08b.%08b.%08b\n", '$br1,$br2,$br3,$br4')'