Trying to pass some output of lsof -Pi to geoiplookup - bash

Looking for a way to pass the second column of output to geoiplookup, ideally on the same line, but not necessarily. This is the best I can muster. It's usable, but the geoiplookup results are unfortunately below the list of connections. I wanted more integrated results. If anyone can suggest improvements, they would be welcome.
ns () {
echo ""
while sleep 1; do
lsof -Pi |
grep ESTABLISHED |
sed "s/[^:]*$//g" |
sed "s/^[^:]*//g" |
sed "s/://g" |
sed "s/->/\t/g" |
grep -v localdomain$ |
tee >(for x in `grep -o "\S*$"`; do geoiplookup $x | sed "s/GeoIP.*: /\t/g"; done)
done
}
The results currently look something like this:
<Port> <URL or IP if no reverse available #1>
<Port> <URL or IP if no reverse available #2>
<geoiplookup trimmed result #1>
<geoiplookup trimmed result #2>

I received an excellent answer here.

Related

How can I get the IP from arp command in shell

I've build this command:
arp -a | grep <mac here> | grep -P '\((.*?)\)' -o
This should return an IP address for a specific mac. The problem is that the IP get's returned but with () around them. I need them without the (). Any idea how I can fix this?
Currently: (192.168.187.136)
Should be 192.168.187.136
I believe arp is somewhat deprecated and ip neighbor is the proper replacement. Since ip n output doesn't have those parenthesis, you also avoid this issue
ip n | grep <mac here> | cut -d' ' -f1
You can remove the 1st and last character using sed:
arp -a | grep <mac here> | grep -P '\((.*?)\)' -o | sed 's/^|\(.*\)|$/\1/'
Also, on BASH 4.2 and newer:
ip=$(arp -a | grep <mac here> | grep -P '\((.*?)\)' -o)
echo "${ip:1:-1}"
using sed in one command
arp -a | grep <mac here> | grep -P '\((.*?)\)' -o | sed 's/^|\(.*\)|$/\1/'
Since you're already using grep -P :
arp -a | grep <mac here> | grep -oP '\(\K[^)]*'
\K is a PCRE meta-character meaning "drop the string matched so far", so you will check that there's a bracket before your result without outputting it, and the negated class [^)] will match up to the next bracket excluded.
You can also fuse both greps if the mac address appears before the IP :
arp -a | grep -oP '<mac here>.*\(\K[^)]*'
Using ip neighboor and (posix) awk:
ip n | awk -v mac='<mac here>' 'mac{print $1}'
or using GNU awk and arp:
arp -a | awk -F'[()]' -v mac='<mac here>' '$0 ~ mac{print $2}'

Display interface + ip list nice way

I have to display net interface and IP's attached to it.
I came up with this code:
if [ -f intf ]; then
rm -I intf
fi &&
if [ -f ipl ]; then
rm -I ipl
fi &&
ip ntable | grep dev | sort | uniq | sed -e 's/^.*dev //;/^lo/d' >> intf &&
ip a | grep -oP "inet\s+\K[\w./]+" | grep -v 127 >> ipl &&
paste <(cat intf) <(cat ipl)
It does the job but I believe it's ugly :), created files, IMHO a total mess :)
any one can suggest the nice way to get exact the same result but short and efficient way ?
If there are a few interfaces, right now I'm thinking about looping, but that will make this code even bigger and probably uglier :) What would you suggest?
As the first thing, you can eliminate the need for temporary files with process substitution:
paste <(ip ntable | grep dev | sort -u | sed -e 's/^.*dev //;/^lo/d') <(ip a | grep -oP "inet\s+\K[\w./]+" | grep -v 127)
sort -u does the same thing as sort | uniq
This oneliner outputs the interface name and its ip address:
ifconfig |\
grep -e 'Link' -A 1 |\
paste -d" " - - - |\
grep ' addr' |\
sed -e 's/ */ /g' -e 's/Link.*addr://' |\
cut -d" " -f1,2
Here an explanation of the commands:
Shows network configuration
Filters lines containing Link and the next line to it.
Joins three lines
Filters lines having an assigned address
Trim whitespaces and remove not relevant information
Splits remaining data and keeps only interface name and ip address.
Example output:
br-2065e5d2fc59 172.18.0.1
docker0 172.17.0.1
lo 127.0.0.1
wlp3s0

Process sed-(stream editor)-output further

I'm a bash-scripting newbie and don't even know how to formulate my question. I've skimmed this tutorial, but couldn't find an appropriate code example. This is what I want...
I have a list of hostnames, (hostname being google.com and such), which looks like:
1,hostname_1
2,hostname_2
...
n,hostname_n
I want to remove the number in the front, that can be easily done with:
originList="originList.txt"
preparedList="preparedList.txt"
ipv6list="ipv6list.txt"
sed 's/[0-9]*,//' <$originList >$preparedList
But instead of piping the output to preparedList.txt I'd like to use it in my dig command:
sed 's/[0-9]*,//' <$originList | dig **HERE** AAAA +short >> $ipv6List
use this:
sed -e 's/^[[:digit:]]*,//' FILE | xargs -I {} dig {} AAAA +short
Using cut & GNU parallel:
cut -d',' -f2 hostnames | parallel 'dig {} AAAA +short'

Get all running ports using UNIX command and form a new command based on result

I need to get all running ports from the server, like the unix command 'netstat -an | grep tcp46'
OUTPUT:
tcp46 0 0 *.8009 *.* LISTEN
tcp46 0 0 *.8080 *.* LISTEN
Then I need to iterate the ports and form a command like below.
curl http://serverhost.com:${iterative ports}/app/version
eg.
curl http://serverhost.com:8080/app/version
Can anyone please help me with the shell script or any easy commands available?
My netstat -an output looks different from yours, so I'm going in blind here:
for i in netstat -an | grep tcp46 | cut -d' ' -f18 | sed 's/*.//g'; do echo curl http://serverhost.com:$i/app/version; done
That is a one liner that should work, but it assumes the output is the same as you said, if the number of spaces change then it won't work correctly. Just remove the echo if you want to run the command directly.
You can use regular expressions (REGEX) in sed to get to the output you need from the grep input stream. Then in a bash for loop, execute your curl command for every port you find. Note: the following doesn't check for duplicate ports.
for port in $(netstat -an | grep tcp46 | sed 's/[a-zA-Z]\{1,3\}[ 0-9.]*:\{1,3\}//g' | sed 's/ \+.*//g');
do echo "curl http://serverhost.com:$port/app/version" >> commandFile.txt;
done
I invoke sed twice: once to remove the first portion of the string and a second time to remove the trailing portion.
The output of this script is sent to commandFile.txt in your current directory.
If you would rather run each curl command rather than send to a file, simply remove the echo as follows:
for port in $(netstat -an | grep tcp46 | sed 's/[a-zA-Z]\{1,3\}[ 0-9.]*:\{1,3\}//g' | sed 's/ \+.*//g');
do curl http://serverhost.com:$port/app/version;
done

Get open ports as an array

So, I'm using netstat -lt to get open ports. However, I'm not interested in certain values (like SSH or 22), so I want to be able to exclude them. I also want to get them as an array in bash. So far I have netstat -lt | sed -r 's/tcp[^:]+://g' | cut -d' ' -f1 but they're not an array, nor am I excluding anything.
Try using the ss command, which replaces netstat.
ss -atu | awk '{print $5}' | awk -F: '{print $NF}'
The ss command gives you all TCP and UDP ports on the local machine (the only sockets that would have ports). The first awk extracts the column containing the local address and port number. The second awk takes only the last field following a colon; this is necessary in case you have IPv6 sockets on your machine, whose IP address will also include colons.
Once you've done this, you can grep out the ports you don't want. Also, see the documentation referred to by the ss man page for information on filters, which may let you filter out unwanted sockets from the output of ss.
Add ($()) around your statement:
port=($(netstat -ltn | sed -rne '/^tcp/{/:(22|25)\>/d;s/.*:([0-9]+)\>.*/\1/p}'))
Filtering ports 22 and 25.
a=( `netstat -ltn --inet | sed -r -e '1,2d''s/tcp[^:]+://g' | cut -d' ' -f1 | sed -e '1,2d' | grep -v "22\|33\|25"` )
second sed command removes headers if your version of netstat prints such. I have "Active" and "Proto" as first two lines. Use grep to filter unwanted ports. add -n to netstat to see port numbers instead of names. --inet is to force ipv4, otherwise you may see IPv6 which may confuse your script.
btw not sure you need an array. usually arrays are needed only if you are going to work on a subset of values you have. If you work on all values there are simpler constructs but not sure what you're going to do.
Regards.
update: you can use a single sed command with two operations instead of two separate invocations:
sed -r -e '1,2d' -e 's/tcp[^:]+://g'

Resources