Count IP addresses from list - bash

My goal is to take a list of IPs, sometimes the list is just a single IP or it could be a range of IPs and count how many IPs there are in a list. We have another product that generates a list of IPs and we have been manually counting it using an Excel spreadsheet.
Using an existing post here on Stack, I have been attempting to incorporate it into a script that will accept a list. https://codegolf.stackexchange.com/questions/28760/how-many-ip-addresses-are-in-a-given-range (See Pure Bash, 66 bytes).
Instead of grabbing two arguments on the command line $1 and $2, I pull a list into an array then enumerate through the array. If it is a single IP, it just adds 1 to the counter variable, if it is a range it uses the logic to convert the IPs to hex and counts them. I'm running into an issue where I am receiving errors.
I can't seem to figure out why this says "invalid number: printf: 229". I've read up on the expansion principle and I cannot seem to get my head around why it keeps throwing this error and calculating it improperly.
I've used this site for years, this is my first post. Any help would be greatly appreciated!
Thank you!
This is what I have so far:
#!/bin/bash
if [ $# -lt 1 ]; then
echo "Please supply a list of IP addresses"
echo "Example: " $0 "list.txt"
exit
fi
#Set some variables
LIST=($(cat ./$1))
COUNT=0
# Function for removing dots and converting to hex
p()(printf %02x ${1//./ })
# Enumerate the array of IPs
for RANGE in ${LIST[#]};do
IFS=- read IP1 IP2 <<< $RANGE
if [ -z $IP2 ]; then
COUNT=$[COUNT + 1]
else
r=$[0x`p $IP1`-0x`p $IP2`]
COUNT=$[COUNT + $[1+${r/-}]]
fi
done
echo "The count is" $COUNT
sample_range.txt:
192.168.47.11
192.168.48.10
192.168.65.228-192.168.65.229
192.168.65.234
192.168.65.239
192.168.65.241
192.168.65.244
192.168.80.220
192.168.93.231-192.168.93.235
192.168.93.237-192.168.93.239
192.168.103.222
This should result in 18, instead it gives me this output:
# ./rc.sh sample_range.txt
: invalid number: printf: 229
: invalid number: printf: 235
: invalid number: printf: 239
The count is 707

IPs are numbers base 256.
#!/bin/bash
ipdiff() {
declare -i dec1 dec2 diff # set integer attribute
dec1=$1*256*256*256+$2*256*256+$3*256+$4
dec2=$5*256*256*256+$6*256*256+$7*256+$8
diff=$dec2-$dec1+1
counter=counter+$diff
}
declare -i counter
# read IP(s) from file, use . and - as separator
while IFS=".-" read -r a1 a2 a3 a4 b1 b2 b3 b4; do
if [[ -z $b1 ]]; then # $b1 is empty (line with one IP)
counter=counter+1
else # $b1 is not empty (line with 2 IPs)
ipdiff $a1 $a2 $a3 $a4 $b1 $b2 $b3 $b4
fi
done < file
echo $counter
Output:
18

Related

Error in IPv6 increment script (bash) - can't find the mistake

per my last question, I managed to increment a given IPv6 Address by 1 to have the next possible address - hooray it works :)
But only until I hit the :ffff IP - the next would obviously be :1:0000 but my script is not "behaving" in a sane way - and I cannot figure out why :(
#!/bin/bash
initialipv6address="2001:0000:0000:0000:0000:0000:0000:ffff"
echo $initialipv6address
function ipv6_increment {
IFS=':' read -r -a fields <<< "$initialipv6address"
carry=1
for ((i=${#fields[#]}-1; i>=0; i--)); do
fields[$i]=$((16#${fields[i]}+carry))
if [[ ${fields[$i]} -ge 65536 ]]; then
carry=1
fields[$i]=$((16#${fields[i]}-65536))
else
carry=0
fi
done
printf "%04x:%04x:%04x:%04x:%04x:%04x:%04x:%04x\n" "${fields[#]}"
}
newipv6address=$(ipv6_increment "$initialipv6address")
echo "$newipv6address"
Output expected:
2001:0000:0000:0000:0000:0000:0000:ffff
2001:0000:0000:0000:0000:0000:0001:0000
Output is in reality:
2001:0000:0000:0000:0000:0000:0000:ffff
2001:0000:0000:0000:0000:0000:0001:55536 <-- WTF?????
It seems like it correctly sets the value to 1 but I cannot figure out where the 55536 is coming from :(

Bash: checking substring increments with modular arithmetic

I have a list of files with file names that contain a substring of 6 numbers that represents HHMMSS, HH: 2 digits hour, MM: 2 digits minutes, SS: 2 digits seconds.
If the list of files is ordered, the increments should be in steps of 30 minutes, that is, the first substring should be 000000, followed by 003000, 010000, 013000, ..., 233000.
I want to check that no file is missing iterating the list of files and checking that neither of these substrings is missing. My approach:
string_check=000000
for file in ${file_list[#]}; do
if [[ ${file:22:6} == $string_check ]]; then
echo "Ok"
else
echo "Problem: an hour (file) is missing"
exit 99
fi
string_check=$((string_check+3000)) #this is the key line
done
And the previous to the last line is the key. It should be formatted to 6 digits, I know how to do that, but I want to add time like a clock, or, in more specific words, modular arithmetic modulo 60. How can that be done?
Assumptions:
all 6-digit strings are of the format xx[03]0000 (ie, has to be an even 00 or 30 minutes and no seconds)
if there are strings like xx1529 ... these will be ignored (see 2nd half of answer - use of comm - to address OP's comment about these types of strings being an error)
Instead of trying to do a bunch of mod 60 math for the MM (minutes) portion of the string, we can use a sequence generator to generate all the desired strings:
$ for string_check in {00..23}{00,30}00; do echo $string_check; done
000000
003000
010000
013000
... snip ...
230000
233000
While OP should be able to add this to the current code, I'm thinking we might go one step further and look at pre-parsing all of the filenames, pulling the 6-digit strings into an associative array (ie, the 6-digit strings act as the indexes), eg:
unset myarray
declare -A myarray
for file in ${file_list}
do
myarray[${file:22:6}]+=" ${file}" # in case multiple files have same 6-digit string
done
Using the sequence generator as the driver of our logic, we can pull this together like such:
for string_check in {00..23}{00,30}00
do
[[ -z "${myarray[${string_check}]}" ]] &&
echo "Problem: (file) '${string_check}' is missing"
done
NOTE: OP can decide if the process should finish checking all strings or if it should exit on the first missing string (per OP's current code).
One idea for using comm to compare the 2 lists of strings:
# display sequence generated strings that do not exist in the array:
comm -23 <(printf "%s\n" {00..23}{00,30}00) <(printf "%s\n" "${!myarray[#]}" | sort)
# OP has commented that strings not like 'xx[03]000]` should generate an error;
# display strings (extracted from file names) that do not exist in the sequence
comm -13 <(printf "%s\n" {00..23}{00,30}00) <(printf "%s\n" "${!myarray[#]}" | sort)
Where:
comm -23 - display only the lines from the first 'file' that do not exist in the second 'file' (ie, missing sequences of the format xx[03]000)
comm -13 - display only the lines from the second 'file' that do not exist in the first 'file' (ie, filenames with strings not of the format xx[03]000)
These lists could then be used as input to a loop, or passed to xargs, for additional processing as needed; keeping in mind the comm -13 output will display the indices of the array, while the associated contents of the array will contain the name of the original file(s) from which the 6-digit string was derived.
Doing this easy with POSIX shell and only using built-ins:
#!/usr/bin/env sh
# Print an x for each glob matched file, and store result in string_check
string_check=$(printf '%.0sx' ./*[0-2][0-9][03]000*)
# Now string_check length reflects the number of matches
if [ ${#string_check} -eq 48 ]; then
echo "Ok"
else
echo "Problem: an hour (file) is missing"
exit 99
fi
Alternatively:
#!/usr/bin/env sh
if [ "$(printf '%.0sx' ./*[0-2][0-9][03]000*)" \
= 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' ]; then
echo "Ok"
else
echo "Problem: an hour (file) is missing"
exit 99
fi

Increase the number each time the script is run from 0 to 999 and start over

I have a scripts that needs to change the number each time is run. There can be max 3 digits from 0 to 999, it needs to repeat after it reaches 999.
#!/bin/sh
#Ncat is a part of the nmap, install nmap to use ncat
#Define variables
i=poland.aprs2.net
port=14580
lat=3439.94N
lon=02517.67E_ #_ is a symbol for WX station
user=APRS
password=99999
#Generate authentication data
aprsauth="user $user pass $password filter m/50"
#Generate Weather data
xastir="$user>APX206,TCPIP*:!$lat/$lon".../...g...t...h41X210""
#Telemetry data
t1="$user>APX206,TCPIP*:T#672,060,000,000,000,000,00000000"
#Send data to the APRS server
printf "%s\n" "$aprsauth" "$xastir" "$t1"| ncat --send-only $i $port
#Output control
printf "%s\n" "$aprsauth" "$xastir" "$t1"
Code that needs to be changed is T#672
from:
t1="$user>APX206,TCPIP*:T#672,060,000,000,000,000,00000000"
T#xxx is a sequence number in APRS protocol, and it only supports 3 digits. Each telemetry report needs a different sequence number.
You need to maintain some sort of external state: write the last number used to a file, and read from that file on startup so you can incremented t it.
read num < /var/run/myscript/last_value
num=$((num + 1))
if (( num == 1000 )); then
num=0
fi
echo "$num" > /var/run/myscript/last_value
...
printf -v t1 "%s>APX206,TCPIP*:T#%03d,060,000,000,000,000,00000000" "$user" "$num"

Storing multiple columns of data from a file in a variable

I'm trying to read from a file the data that it contains and get 2 important pieces of data from the file and use it in a bash script. A string and then a number for example:
Box 12
Toy 85
Dog 13
Bottle 22
I was thinking I could write a while loop to loop through the file and store the data into a variable. However I need two different variables, one for the number and one for the word. How do I get them separated into two variables?
Example code:
#!/bin/bash
declare -a textarr numarr
while read -r text num;do
textarr+=("$text")
numarr+=("$num")
done <file
echo ${textarr[1]} ${numarr[1]} #will print Toy 85
data are stored into two array variables: textarr numarr.
You can access each one of them using index ${textarr[$index]} or all of them at once with ${textarr[#]}
To read all the data into a single associative array (in bash 4.0 or newer):
#!/bin/bash
declare -A data=( )
while read -r key value; do
data[$key]=$value
done <file
With that done, you can retrieve a value by key efficiently:
echo "${data[Box]}"
...or iterate over all keys:
for key in "${!data[#]}"; do
value=${data[$key]}
echo "Key $key has value $value"
done
You'll note that read takes multiple names on its argument list. When given more than one argument, it splits fields by IFS, putting columns into their respective variables (with the entire rest of the line going into the last variable named, if more columns exist than variables are named).
Here I provide my own solution which should be discussed. I am not sure this is a good solution or not. Using while read construct has the drawback of starting a new shell and it will not be able to update a variable outside the loop. Here is an example code which you can modify to suite your own need. If you have more column data to use, then slight adjustment is need.
#!/bin/sh
res=$(awk 'BEGIN{OFS=" "}{print $2, $3 }' mytabularfile.tab)
n=0
for x in $res; do
row=$(expr $n / 2)
col=$(expr $n % 2)
#echo "row: $row column: $col value: $x"
if [ $col -eq 0 ]; then
if [ $n -gt 0 ]; then
echo "row: $row "
echo col1=$col1 col2=$col2
fi
col1=$x
else
col2=$x
fi
n=$(expr $n + 1)
done
row=$(expr $row + 1)
echo "last row: $row col1=$col1 col2=$col2"

bash split a variable into two or more based on character length

I built a script for SMS autoresponder, my goal is that when an sms content has more than 160 of character length, it splits the content into two or more variables then send them separately.
myvar="this variable has more than ten character length"
That variable has 48 of character length, how do I print that variable from length 1 to length 25 and length 26 to 48 ? So i'll have 2 variables in the end and send those variables with sms:
firstvar="this variable has more th"
secondvar="an ten character length"
I know there's a command split but my openwrt doesn't support that command, so I have to find another way to do that.
Bash can split a variable into substrings using it's substitution rules.
echo ${variable:4:8}
Will display eight characters starting at offset four. The offset starts at zero.
In general:
${parameter:offset:length}
this snippet should help you:
myvar="this variable has more than ten character length"
size=${#myvar}
if [ $size -gt 25 ]; then
firstvar=${myvar:0:25}
secondvar=${myvar:26:size}
echo "$firstvar"
echo "$secondvar"
fi
A pure Bash possibility, no external tools (hence only depends on Bash and no other specific third-party tools) and no subshells:
#!/bin/bash
mysms="this variable has more than ten character length"
maxlength=25
sms_tmp=$mysms
sms_ary=()
while [[ $sms_tmp ]]; do
sms_ary+=( "${sms_tmp::maxlength}" )
sms_tmp=${sms_tmp:maxlength}
done
# At this point, you have your sms split in the array sms_ary:
# You can print them, one per line:
printf '%s\n' "${sms_ary[#]}"
# You can print them, one per line, with header:
printf -- '--START SMS-- %s --END SMS--\n' "${sms_ary[#]}"
# You can print them, space padded (spaces on the right):
printf -- "--START SMS-- %-$(maxlength}s --END SMS--\n" "${sms_ary[#]}"
# You can print them, space padded (spaces on the left):
printf -- "--START SMS-- %${maxlength}s --END SMS--\n" "${sms_ary[#]}"
# You can loop through them:
for sms in "${sms_ary[#]}"; do
printf 'Doing stuff with SMS: %s\n' "$sms"
done
# You can loop through them with index (C-style loop):
for ((i=0;i<${#sms_ary[#]};++i)); do
printf 'This is SMS #%d at index %d: %s\n' "$((i+1))" "$i" "${sms_ary[i]}"
done
# You can loop through them (using array key as variable):
n=1
for i in "${!sms_ary[#]}"; do
printf 'This is SMS #%d at index %d: %s\n' "$((n++))" "$i" "${sms_ary[i]}"
done
# Here's the number of SMS:
printf 'That was fun. There were %d chunks of SMS\n' "${#sms_ary[#]}"
Another way to split your string:
#!/bin/bash
mysms="this variable has more than ten character length"
maxlength=25
sms_ary=()
for ((i=0;i<${#mysms};i+=maxlength)); do
sms_ary+=( "${mysms:i:maxlength}" )
done
# Same as before, at this point you have your chunks in array sms_ary

Resources