How do i redirect a list of IP addresses to a command line function? - bash

I want to see what countries are trying to access my VPS. I have installed a tool called "goiplookup", which was forked from another effort called "geoiplookup". If I type this at the command line:
goiplookup 8.8.8.8
It returns this:
US, United States
So I figured out how to get a list of IPs that are trying to access my server by using this:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}'
Which gives a long list of IPs like this:
1.1.1.1
2.2.2.2
3.3.3.3
I cannot figure out how to get this list of IPs to be processed by the "goiplookup" tool. I tried this:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | goiplookup
but that did not work. I also tried with no luck:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | xargs -0 goiplookup

Try this:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | sort | uniq | xargs -n 1 goiplookup
I added | sort | uniq to ensure each IP only appears once
and xargs -n 1 so that each found IP is processes by goiplookup

I would put it into a file and make a small utility to parse it:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | sort -u > ./file.txt
cat ./file.txt | while read -r line; do
temp$(echo $line)
goiplookup $temp
done
This will read through the file one line at a time and execute the goiplookup with each IP.

sudo grep disconnect /var/log/auth.log | awk '!/COMMAND/ && !seen[$0]++ {system("geoiplookup \""$9"\""}
Note that geoiplookup only allows one IP per invocation.
The whole thing can be done in awk, but using grep allows the rest to be run unprivileged.
Consider whether grep -w (match whole word) is appropriate, and in awk you can do a similar thing with !/(^|[^[:alnum:]_])COMMAND($|[^[:alnum:]_])/.

I just made a shell script, which works.
#!/bin/bash
readarray -t array < <(sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | sort | uniq)
for ip in "${array[#]}"
do
:
country=$(/usr/local/bin/goiplookup -c $ip)
echo "$ip $country"
done

Related

ssh remote command execution quoting and piping awk

I'm working on a script, that should find certain disks and add hostname to them.
I'm using this for 40 servers with a for loop in bash
#!/bin/bash
for i in myservers{1..40}
do ssh user#$i findmnt -o SIZE,TARGET -n -l |
grep '1.8T\|1.6T\|1.7T' |
sed 's/^[ \t]*//' |
cut -d ' ' -f 2 |
awk -v HOSTNAME=$HOSTNAME '{print HOSTNAME ":" $0}'; done |
tee sorted.log
can you help out with the quoting here? It looks like awk gets piped (hostname) from localhost, not the remote server.
Everything after the first pipe is running locally, not on the remote server.
Try quoting the entire pipeline to have it run on the remote server:
#!/bin/bash
for i in myservers{1..40}
do ssh user#$i "findmnt -o SIZE,TARGET -n -l |
sed 's/^[ \t]*//' |
cut -d ' ' -f 2 |
awk -v HOSTNAME=\$HOSTNAME '{print HOSTNAME \":\" \$0}'" ;
done | tee sorted.log
This is a shorter version of your stuff:
findmnt -o SIZE,TARGET -n -l |
awk -v HOSTNAME=$HOSTNAME '/M/{print HOSTNAME ":" $2}'
Applied to the above:
for i in myservers{1..40}
do ssh user#$i bash -c '
findmnt -o SIZE,TARGET -n -l |
awk -v HOSTNAME=$HOSTNAME '"'"'/M/{print HOSTNAME ":" $2}'"'"' '
done |
tee sorted.log
see: How to escape the single quote character in an ssh / remote bash command?

Command run via xargs fails but runs manually

I'm trying to create a command that will automatically attach to my existing python docker container, and trying to chain a bunch of commands together.
docker ps | grep "mypythoncontainer" | awk '{print $1}' | xargs docker attach
If I run
docker ps | grep "mypythoncontainer" | awk '{print $1}' | xargs echo
I get back a docker id string, as expected. And if I do docker attach {id string} (copied from the return of the statement right above this), it works. But when I run the full command at top, I get an error (the input device is not a TTY).
So docker ps | grep "mypythoncontainer" | awk '{print $1}' | xargs echo would echo out abc, but docker ps | grep "mypythoncontainer" | awk '{print $1}' | xargs docker attach would fail, while docker attach abc works. Not sure what about xargs I don't understand.
Try:
docker attach $(docker ps | grep "mypythoncontainer" | awk '{print $1}')
or simplier:
docker attach $(docker ps | awk '/mypythoncontainer/{print $1}')
Not sure what about xargs I don't understand.
Running: ...| ... docker ... will redirect docker's standard input to ... the ouput of awk, wich was already read by xargs. So docker abc will r
un with a broken (already closed) STDIN, then fail.

Ubuntu grep option error while executing a batch script

#!/bin/bash
vm1_MAC=`virsh -c qemu:///system domiflist instance-00000009 -e | grep virbr0 | awk '{print $5}'`
vm2_MAC=`virsh -c qemu:///system domiflist instance-0000000d -e | grep -i virbr0 | awk -e '{print $5}'`
vm1_IP=`arp -e | grep $vm1_MAC | awk '{print $1}'`
vm2_IP=`arp -e | grep $vm2_MAC | awk '{print $1}'`
echo "VM1 IP Address: $vm1_IP"
echo "VM2 IP Address: $vm2_IP"
The shell script was meant to display the IP addresses of my two openstack instances but I am receiving grep command option error:
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
VM1 IP Address:
VM2 IP Address:
Is there anyone here who can assist me as I am not a bash script expert, just need to do this to get some tasks done. Thank you
This message typically happens when you try to grep for a string which starts with a dash.
The immediate workaround is to use grep -e "$variable" but really, you want to avoid the useless use of grep here.
#!/bin/bash
vm1_MAC=$(virsh -c qemu:///system domiflist instance-00000009 -e | awk "/virbr0/"'{print $5}')
vm2_MAC=$(virsh -c qemu:///system domiflist instance-0000000d -e | awk -e 'tolower($0) ~ /vibr0/ {print $5}')
vm1_IP=$(arp -e | awk -v mac="$vm1_MAC" '$0 ~ mac {print $1}')
vm2_IP=$(arp -e | awk -v mac="$vm2_MAC" '$0 ~ mac {print $1}')
Incidentally, this also demonstrates three different ways to pass a regex to Awk. Notice as well how we prefer the modern $(command) substitution over the dinosaur `backtick` syntax.

Processing ifconfig and hostname output and frame it in single line for multiple hosts

I'm trying to filter remote machine mac address , IP and hostname from 100 machines. Initially I'm capturing the information using below command.
$for i in `cat IP`;do in ssh root#$IP "ifconfig eth0 && hostname " ;done
eth0 Link encap:Ethernet HWaddr 00:MN:77:TR:XX:ZZ
inet addr:192.168.122.25 Bcast:192.168.122.255
Mask:255.255.224.0
test.com-112304
eth0 Link encap:Ethernet HWaddr 00:TT:77:MM:XX:YY
inet addr:192.168.122.22 Bcast:192.168.122.255
Mask:255.255.224.0
test.com-11035
Initially the output redirect to one file. from their I need to process output which comes in the above format.
I can able to accomplish to print IP and mac by line as follows
$cat input | awk '/HWaddr/{printf $NF;printf " ";getline;print $2;}' | cut -c1-18,24-
00:MN:77:TR:XX:ZZ 192.168.122.25
00:TT:77:MM:XX:YY 192.168.122.22
Actually I want to include hostname along with the output as follows
00:MN:77:TR:XX:ZZ 192.168.122.25 test.com-112304
00:TT:77:MM:XX:YY 192.168.122.22 test.com-11035
test.com will be common name for all hostname. I tried redirect both output in two variable and called as follows , but no luck.
a=`cat input | awk '/HWaddr/{printf $NF;printf " ";getline;print $2;}' | cut -c1-18,24-`
b=`grep test.com input`
echo $a $b | xargs -n2
Please shed some views
I'm trying to filter remote machine mac address , IP and hostname from 100 machines.
Machine mac address, from here: cat /sys/class/net/eth0/address
You have hostname utility, why don't you just hostname -i? You can parse ifconfig | grep inet | awk '{print $2}' | sed 's/addr://' and hope ifconfig versions are the same across all machines (some ifconfig verions return inet <ip> some return inet addr:<ip> like yours).
Hostname with hostname
Putting together:
echo $(cat /sys/class/net/eth0/address) $(ifconfig eth0 | grep inet | awk '{print $2}') $(hostname)
Executing a command for each line in some file is a job for xargs and remember about properly escaping your arguments:
cat IP | xargs -n1 -I{} ssh root#{} 'echo $(cat /sys/class/net/eth0/address) $(ifconfig eth0 | grep inet | awk '\''{print $2}'\'' | sed '\''s/addr://'\'') $(hostname)'
If you really have 100 machines, ansible might interest you.
If you really have that "input" file and you can't change it, you may parse it like this:
sed 's/eth0/#eth0/' input \
| xargs -d'#' -n1 -- bash -c '{ echo "$1" | grep HWaddr | awk "{print \$5}"; echo "$1" | grep inet | awk "{print \$2}" | sed "s/addr://"; echo "$1" | grep "^test.com"; } | tr "\n" " "; echo' --
And this hurts my eyes:
for i in `cat IP`; then

Using awk inside ssh from ruby

When running the command:
puts `ssh -o StrictHostKeyChecking=no -i keyfile user#host "sudo cat file | awk '/^server/ {print \$2}' | sort -u"`
After running this command, its only counts the ^server, but it ignores the print $2 command.
i get the whole line instead of just the 2nd word.
You're running through a couple of shell processing layers, so you need an additional backslash:
puts `ssh -o StrictHostKeyChecking=no -i keyfile user#host "sudo cat file | awk '/^server/ {print \\$2}' | sort -u"`
What happens if you change this:
sudo cat file | awk '/^server/ {print \$2}' | sort -u
to this:
sudo cat file | grep '/^server' | awk '{ print \$2 }' | sort -u
If still not working, try not escaping the $ - try $2 instead of \$2
Might be more maintainable to split it up into pieces. Also using ruby's single quoting mechanism works to send the command as you intend.
cmd = %q(sudo cat file | awk '/^server/ {print $2}' | sort -u)
puts %x(ssh -o StrictHostKeyChecking=no -i keyfile user#host #{cmd})

Resources