I want to write a script to go through my UFW firewall logs for blocked connections, and pull out the source IP and port they were trying to connect to. I have managed to put together scripts to pull these out individually, and am having difficulty in getting the script now to output the IP and port on one line. Below is a sample of the firewall logs I have and the script. Currently it output all IPs, followed by all port numbers. What I want on each line is the source IP address and corresponding destination port from each log entry.
The ultimate aim is to see what ports each IP address is trying to connect to. My plan was to use uniq -c, once I can output each IP and port from the logs.
input
Nov 26 06:25:11 vps123456 kernel: [620802.845897] [UFW BLOCK] IN=ens3 OUT= MAC=fa:16:3e:9f:c7:5d:11:22:33:44:55:66:77:88 SRC=85.93.20.253 DST=10.20.30.40 LEN=40 TOS=0x00 PREC=0x00 TTL=245 ID=5830 PROTO=TCP SPT=51639 DPT=735 WINDOW=1024 RES=0x00 SYN URGP=0
Nov 26 06:27:44 vps123456 kernel: [620955.012996] [UFW BLOCK] IN=ens3 OUT= MAC=fa:16:3e:9f:c7:5d:11:22:33:44:55:66:77:88 SRC=51.15.51.140 DST=10.20.30.40 LEN=433 TOS=0x00 PREC=0x00 TTL=50 ID=42044 DF PROTO=UDP SPT=5088 DPT=5062 LEN=413
./script
file="input"
for line in $file; do
addr=$(awk '{match($0,/SRC=[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/); ip = substr($0,RSTART,RLENGTH); print ip}' $file)
port=$(awk '{match($0,/DPT=[0-9]{0,5}/); port = substr($0,RSTART,RLENGTH); print port}' $file)
echo $addr, $src >> output
done
output
SRC=85.93.20.253 SRC=51.15.51.140, DPT=735 DPT=5062
A quick php solution for you problem, that will print on each line the source ip address and the corresponding destination port. The script reads your input logfile line by line and process each row. Match ip address and port number into variables using regex.
<?php
$infile = $argv[1];
$handle = fopen($infile, "r");
if ($handle) {
while (($line = fgets($handle)) !== false) {
preg_match('/.*SRC=(?<ip>[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)(.*DPT=)(?<port>[0-9]+).*/', $line, $m);
if ($m) {
print $m['ip'] . ' ' . $m['port'] . "\n";
}
}
fclose($handle);
} else {
exit("Unable to open file ($finfile)");
}
After saving the above script into a file (ie.: get-uwf-ip-port.php), you can run in the following way:
php get-uwf-ip-port.php your_logfile
The output (source file was your two example lines):
85.93.20.253 735
51.15.51.140 5062
UPDATE:
And here is a native bash solution. The main idea here is to read the file line by line, read each line into a shell variable, extract the ip and the port number using shell parameter expansion.
#!/bin/bash
infile="$1"
while read line; do
ip="${line##*SRC=}"
ip="${ip%% *}"
port="${line##*DPT=}"
port="${port%% *}"
echo $ip $port
done < "$infile"
I slightly modified your awk code, putting the ip and port matching together it will produce the same result as above:
awk '{match($0,/SRC=[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+/); ip = substr($0,RSTART+4,RLENGTH-4); match($0,/DPT=[0-9]{0,5}/); port = substr($0,RSTART+4,RLENGTH-4); print ip,port}' your_logfile
Related
[SOLVED]
I'm pretty new tho bash-/shell-scripting and trying setup a check for ip address on a server which gets about once a week a new ip.
The script will then send the new ip to the users.
My problem is, that I'm getting a syntax-error in the last if-else statement for "unexpected" else and can wrap my head around why.
My first iteration didn't use functions, but instead one multi lined if-else which got me the same error. The functions on their own seem to work just fine.
#!/bin/bash
# script to send the new server ip to the users
# get the recent ip address of the system
new_ip=$(ip route get 8.8.8.8 | awk -F"src " 'NR==1{split($2,a," ");print a[1]}')
file=old_ip.txt
function ip_mail(){
source $file
if [ $new_ip != $old_ip ]
then
# email-address changed for obvious reasons
mail -s "New Server IP" [hidden]#[hidden].com <<< "$new_ip"
echo "old_ip=$new_ip" > old_ip.txt
exit 0
fi
exit 0
}
function set_old(){
touch old_ip.txt
echo "old_ip=$new_ip" > old_ip.txt
exit 0
}
if [ $file ]
then
ip_mail()
else
set_old()
fi
I'm writing a script that will check/open ports/protocols in the event any are blocked. What I have so far is below. The port/protocol names look strange to me. I would have expected IP addresses, but I've never done this before. Would the host be IP address of the DSLAM? Also, can I run nc without specifying host if it's the current machine? Otherwise, does this script do what is needed?
#!/bin/bash
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
echo -e "############################nnnPresent ports opened on this machine are
$(iptables -nL INPUT | grep ACCEPT | grep dpt)
nCompleted listing...nnn#########################"
#these look funny to me
PORTS=( 123 161 69 "UDP" 80 443 22 8443 8080 23 25 3307 "TCP" "HTTPS" "SNMP" "SFTP" "TFTP")
#modified ip's for public sharing
HOSTS=( "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x")
for HOST in "${HOSTS[#]}"
do
for PORT in "${PORTS[#]}"
do
#see which ones need opening...0 is pass (open), 1 fail, 5 timeout; need host still
#alternatively try nmap
nc -z -v -w5 ${HOST} ${PORT}
#if it's not open, then open it
if [ "$?" ne 0 ]; then #shellcheck err this line: Couldn't parse this test expression.
iptables -A INPUT -m tcp -p tcp --dport "$PORT" -j ACCEPT &&
{ service iptables save;
service iptables restart;
echo -e "Ports opened through iptables are n$(iptables -nL INPUT | grep ACCEPT | grep dpt)"; }
else
echo "Port $PORT already open"
fi
done
done
I've been referring to test if port is open, and also open port.
These lines seem odd, OP edit #6 adds an outer for loop which assigns the same value to $HOST on each go-round:
HOSTS=( "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x")
for HOST in "${HOSTS[#]}"
do
< stuff ... >
done
Assuming running < stuff ... > four times is not necessary, then
the seven lines above, as written, would be equivalent to:
HOST="10.x.x.x"
< stuff ... >
(Fixed.) Remove the commas from this line:
PORTS=( 123, 161, 69, UDP, 80, 443, 22, 8443, 8080, 23, 25,
3307, TCP, HTTPS, SNMP, SFTP, TFTP)
bash does not use commas to define arrays, and if commas are used
they become chars in the the array data. Example, given the array
exactly as it is above:
echo ${PORTS[0]}
Outputs:
123,
I have an issue changing an script I did in bash to powershell, the script is the following:
#! /bin/sh
for IPVAR in 172.27.41.202 172.27.41.203
do
TIEMPO=$(date +"%m-%d-%y")
FILENAME=${IPVAR}_${TIEMPO}
date +"%c" >> $FILENAME.txt
snmpget -v 2c -c public $IPVAR -mALL 1.3.6.1.4.1.41413.1.1.0 1.3.6.1.4.1.41413.1.4.0 1.3.6.1.4.1.41413.1.2.0 1.3.6.1.4.1.41413.1.3.0 1.3.6.1.4.1.41413.10.3.4.1.1.1 1.3.6.1.4.1.41413.10.3.4.1.2.1 1.3.6.1.4.1.41413.10.3.4.1.3.1 1.3.6.1.4.1.41413.10.3.4.1.4.1 1.3.6.1.4.1.41413.10.3.4.1.5.1 1.3.6.1.4.1.41413.10.3.4.1.6.1 1.3.6.1.4.1.41413.10.3.4.1.7.1 1.3.6.1.4.1.41413.10.3.4.1.8.1 1.3.6.1.4.1.41413.10.3.4.1.9.1 1.3.6.1.4.1.41413.10.3.4.1.10.1 >> $FILENAME.txt
done
In my Linux enviroment works fine but I installed NET-SNMP in a Windows Server because there is where we need the files to be but I can seem to make it work I did this:
$IPS = (10.96.90.2)
$TIEMPO = get-date -f yyyy-MM-dd
Foreach ($IPVAR in $IPS) {snmpget -v 2c -c public -m ALL $IPVAR 1.3.6.1.4.1.41413.1.1.0 1.3.6.1.4.1.41413.1.4.0 1.3.6.1.4.1.41413.1.2.0 1.3.6.1.4.1.41413.1.3.0 1.3.6.1.4.1.41413.10.3.4.1.1.1 1.3.6.1.4.1.41413.10.3.4.1.2.1 1.3.6.1.4.1.41413.10.3.4.1.3.1 1.3.6.1.4.1.41413.10.3.4.1.4.1 1.3.6.1.4.1.41413.10.3.4.1.5.1 1.3.6.1.4.1.41413.10.3.4.1.6.1 1.3.6.1.4.1.41413.10.3.4.1.7.1 1.3.6.1.4.1.41413.10.3.4.1.8.1 1.3.6.1.4.1.41413.10.3.4.1.9.1 1.3.6.1.4.1.41413.10.3.4.1.10.1 >> "$IPVAR_$TIEMPO".txt}
If I run only the "snmpget" command it works fine but I have troubles with the scripting part here.
Hope you can help me.
Regards,
Try the code below (this hasn't been tested as I don't have snmpget, but the method works with other command line apps):
$IPS = #('172.27.41.202', '172.27.41.203')
$IPS | ForEach-Object {
$snmpgetParams = #(
'-v', '2c' ,'-c' ,'public' ,'-m' ,'ALL', $_, '1.3.6.1.4.1.41413.1.1.0 1.3.6.1.4.1.41413.1.4.0 1.3.6.1.4.1.41413.1.2.0 1.3.6.1.4.1.41413.1.3.0 1.3.6.1.4.1.41413.10.3.4.1.1.1 1.3.6.1.4.1.41413.10.3.4.1.2.1 1.3.6.1.4.1.41413.10.3.4.1.3.1 1.3.6.1.4.1.41413.10.3.4.1.4.1 1.3.6.1.4.1.41413.10.3.4.1.5.1 1.3.6.1.4.1.41413.10.3.4.1.6.1 1.3.6.1.4.1.41413.10.3.4.1.7.1 1.3.6.1.4.1.41413.10.3.4.1.8.1 1.3.6.1.4.1.41413.10.3.4.1.9.1 1.3.6.1.4.1.41413.10.3.4.1.10.1'
)
$TIEMPO = Get-Date -f yyyy-MM-dd
$FILENAME="$_`_$TIEMPO`.txt"
snmpget #snmpgetParams | Set-Content $FILENAME -Force
}
Line 1 declares an array of IP addresses.
Line 2 starts a foreach loop which will iterate through each IP in the $IPS array.
Lines 3,4,5 create an array of parameters to pass to the snmpget command. The $_ parameter is the current IP address within the loop.
Line 7 sets the $TIEMPO variable with the date.
Line 8 sets the $FILENAME variable with the IP address, followed by an underscore, followed by the date. The backticks ` tell PowerShell to not treat the following characters as part of the preceding variable name. An example filename: 172.27.41.202_2016-08-31.txt
Line 10 calls the snmpget command. The #snmpgetParams 'splats' the parameter array. The output is piped into the Set-Content command, which, with the Force option creates or overwrites the file contents for that IP & date.
Line 11 closes the loop.
I am using this script to check a list of ips I own to see if they are on the spam block list.
auto.sh:
while read ip ; do
./blacklist.sh $ip
done < block.txt
blacklist.sh is the above linked script.
block.txt lists each of my ips one line at a time (I have several /22).
A typical output of a blocked ip scan looks like this:
Warning: PTR lookup failed
b.barracudacentral.org : 127.0.0.2
bb.barracudacentral.org : 127.0.0.2
black.junkemailfilter.com : 127.0.0.2
cbl.abuseat.org : 127.0.0.2
cidr.bl.mcafee.com : 127.0.0.4
dnsbl.justspam.org : 127.0.0.2
hostkarma.junkemailfilter.com : 127.0.0.2
----------------------------------------------------------
Results for <my ip>
Tested: 117
Passed: 110
Invalid: 0
Blacklisted: 7
----------------------------------------------------------
what I want to do is have the script spit out output to a file when the text above doesn't say "Blacklisted: 0".
I am not sure how to approach this, will this work?
sudo ./auto.sh "conditions where Blacklisted: is > 0" >> 12.txt
Thanks for any help
Put the output in a temporary file and
then check its content:
./auto.sh > 12_temp.txt
grep -q 'Blacklisted:[ \t]*0$' 12_temp.txt || cat 12_temp.txt >> 12.txt
rm -f 12_temp.txt
I am trying to parse rsyslog logs. For this i am sending all my logs to socat which is then sending them to Unix Domain Socket. That socket is created via perl script which is listening on that socket to parse logs.
My bash script to which rsyslog is sending all log is
if [ ! `pidof -x log_parser.pl` ]
then
./log_parser.pl & 1>&1
fi
if [ -S /tmp/sock ]
then
/usr/bin/socat -t0 -T0 - UNIX-CONNECT:/tmp/sock 2>> /var/log/socat.log
fi
/tmp/sock is created using perl script log_parser.pl which is
use IO::Socket::UNIX;
sub socket_create {
$socket_path = '/tmp/sock';
unlink($socket_path);
$listner = IO::Socket::UNIX->new(
Type => SOCK_STREAM,
Local => $socket_path,
Listen => SOMAXCONN,
Blocking => 0,
)
or die("Can't create server socket: $!\n");
$socket = $listner->accept()
or die("Can't accept connection: $!\n");
}
socket_create();
while(1) {
chomp($line=<$socket>);
print "$line\n";
}
There is this error i am getting from socat which is
2015/02/24 11:58:01 socat[4608] E connect(3, AF=1 "/tmp/sock", 11): Connection refused
I am no champion in sockets so i am not able to understand what is this. Please help. Thanks in advance.
The main issue is that when i kill my perl script then bash script is suppose to call it again and start it.
What actually happening is that sript is started but socat is not started instead it give this error and never start.
I can duplicate your error if I don't run your perl program before trying to use socat. Here is what works for me:
1) my_prog.pl:
use strict;
use warnings;
use 5.016;
use Data::Dumper;
use IO::Socket::UNIX;
my $socket_path = '/tmp/sock';
unlink $socket_path;
my $socket = IO::Socket::UNIX->new(
Local => $socket_path,
Type => SOCK_STREAM,
Listen => SOMAXCONN,
) or die "Couldn't create socket: $!";
say "Connected to $socket_path...";
my $CONN = $socket->accept()
or die "Whoops! Failed to open a connection: $!";
{
local $/ = undef; #local -> restore previous value when the enclosing scope, delimited by the braces, is exited.
#Setting $/ to undef puts file reads in 'slurp mode' => whole file is considered one line.
my $file = <$CONN>; #Read one line.
print $file;
}`
2) $ perl my_prog.pl
3) socat -u -v GOPEN:./data.txt UNIX-CONNECT:/tmp/sock
The -u and -v options aren't necessary:
-u Uses unidirectional mode. The first address is only used for
reading, and the second address is only used for writing (exam-
ple).
-v Writes the transferred data not only to their target streams,
but also to stderr. The output format is text with some conver-
sions for readability, and prefixed with "> " or "< " indicating
flow directions.
4) You can also do it like this:
cat data.txt | socat STDIN UNIX-CONNECT:/tmp/sock
Pipe stdout of cat command to socat, then list STDIN as one of socat's files.
Response to comment:
This bash script works for me:
#!/usr/bin/env bash
echo 'bash script'
../pperl_programs/my_prog.pl &
sleep 1s
socat GOPEN:./data.txt UNIX-CONNECT:/tmp/sock
It looks like the perl script doesn't have enough time to setup the socket before socat tries to transfer data.