Parsing iptables log to insert into mysql - bash

Question

My router sends its 'DROP' packets to my server via syslog to be logged to a file in the following manner:
Oct 30 13:01:02 192.168.1.1 kernel: DROP IN=vlan2 OUT=
MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
SRC=93.108.197.92 DST=192.168.2.10 LEN=60 TOS=0x00 PREC=0x00 TTL=51
ID=44828 DF PROTO=TCP SPT=55552 DPT=33248 WINDOW=7300 RES=0x00 SYN
URGP=0 OPT (020405840402080A0035BAC40000000001030300)
Oct 30 13:01:06
192.168.1.1 kernel: DROP IN=vlan2 OUT= MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
SRC=93.108.197.92 DST=192.168.2.10 LEN=60 TOS=0x00 PREC=0x00 TTL=51
ID=44829 DF PROTO=TCP SPT=55552 DPT=33248 WINDOW=7300 RES=0x00 SYN
URGP=0 OPT (020405840402080A0035BEAE0000000001030300)
Oct 30 13:01:07
192.168.1.1 kernel: DROP IN=vlan2 OUT= MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
SRC=189.175.171.76 DST=192.168.2.10 LEN=44 TOS=0x00 PREC=0x00 TTL=53
ID=260 PROTO=TCP SPT=14779 DPT=23 WINDOW=50523 RES=0x00 SYN URGP=0 OPT
(020405AC)
Oct 30 13:01:09 192.168.1.1 kernel:
DROP IN=vlan2 OUT=
MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
SRC=125.211.218.39 DST=192.168.1.1 LEN=88 TOS=0x00 PREC=0x00 TTL=48
ID=39896 DF PROTO=ICMP TYPE=8 CODE=0 ID=29389 SEQ=1
Oct 30 13:01:14 192.168.1.1 kernel: DROP IN=vlan2 OUT=
MAC=xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx
SRC=93.108.197.92 DST=192.168.2.10 LEN=60 TOS=0x00 PREC=0x00 TTL=51
ID=44830 DF PROTO=TCP SPT=55552 DPT=33248 WINDOW=7300 RES=0x00 SYN
URGP=0 OPT (020405840402080A0035C6800000000001030300)
I'd like to put each field into a mysql db so I view and analyze later. I'm trying to think of the best way to parse/filter/do this. I'd like to do it in bash but I'm open to other alternatives/langs if it makes it way more efficient or easier.
My iptables log files are rotated every so often and I was going to create a bash/sed/awk script to look through each line of the log(s) and create an sql file so I can use a 'LOAD DATA INFILE' command to load all data into one INSERT statement.
As you can see above, an ICMP and TCP type of packet will differ from the way it is written to the file (number of fields after ID)
I have a few different ways to complete this:
Search by PROTO and the remaining awk 'print' commands are used to grab all relevant information.
Search for all [PARAM]=[VALUE] in every line, regardless of PROTO and just shove them in mysql and analyze later.
So far I have (I know this is basic, but I'm wondering if I should approach it differently before investing more time into it):
cat "$fw_file" | while read line; do
type=$(grep -oP 'PROTO=\w+\s' | cut -d= -f2)
df=$(grep -oP 'ID=\w+\sDF\s' | cut -d' ' -f2)
# continuing on for all fields....
# ......
done
If there a better, more efficient, way for me to do this instead of just grabbing all fields?

Answer 1

Before your begin working on your script (Reinventing the Wheel), For parsing logs and analyzing log data there are opensource tools that do the job efficently. Please consider them!
For your exact usecase, You Better use Elasticsearch,Logstash, Kibana (ELK Stack).
Some Advantages of ELK Stack comparing to script and relational database approach:
Easier to store logs from multiple sources ( one can be your router)
Scalable ( you probably get slowed pretty soon depending on your logs input rate)
Very easy to visualize data in web interface with various charts using kibana.
Elasticsearch has REST API, So your developers can do their own thing too!
There are many tutorials online to get you going pretty fast.