Optimizing firewall rules processing - performance

Question

I'm using fail2ban to block failed login attempts on my server. The block is performed using IP tables with the following configuration:
actionstart = iptables -N fail2ban
iptables -A fail2ban -j RETURN
iptables -I <chain> -p tcp -m multiport --dports <port> -j fail2ban
actionstop = iptables -D <chain> -p tcp -m multiport --dports <port> -j fail2ban
iptables -F fail2ban
iptables -X fail2ban
actionban = iptables -I fail2ban 1 -s <ip> -j DROP
actionunban = iptables -D fail2ban -s <ip> -j DROP
What I'm concerned about is rules processing performance. The above rules are in stateful mode and I've been wondering if stateless mode would make the processing faster. To make things clear, I'm blocking the intruder IP address on a TCP port (e.g., 22 or 25).
I read somewhere that for TCP connection specialy, adding the ESTABLISHED,RELATED states would be better. But since each IP refers to a different connection, does it make sense to apply these states?
UPDATE:
Here is a sample iptables -L:
Chain INPUT (policy ACCEPT 399 packets, 36043 bytes)
pkts bytes target prot opt in out source destination
39 4230 fail2ban tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport dports 22,25,80,99,100,101
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 282 packets, 39686 bytes)
pkts bytes target prot opt in out source destination
Chain fail2ban (1 references)
pkts bytes target prot opt in out source destination
0 0 DROP all -- * * 192.168.0.1 0.0.0.0/0
0 0 DROP all -- * * 192.168.0.2 0.0.0.0/0
0 0 DROP all -- * * 192.168.0.3 0.0.0.0/0
0 0 DROP all -- * * 192.168.0.4 0.0.0.0/0
39 4230 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0

Answer 1

Despite what many performance apologists claim, IPtables CAN have significant overhead, but it won't be noticeable until you get some substantial traffic. Now how you do the tables, and which extensions you call, will be determining factor on CPU overhead per packet.
As for stateless vs statefull, yes the performance difference can be immense, but again it's at a very high throughput. In addition as you may have read, it is much more complexity to manage a stateless firewall. It should really only be done if the IPtable impact is measurable.
BUT good practice should always be followed, and IMO that includes the least amount of overhead without adding lots of complexity.
Now as for your situation, fail2ban is only considering a segment of your iptables overall, but the only thing that I would recommend looking at preventively is this part.
-p tcp -m multiport --dports
Using the multiport extension does have more overhead, except when doing so would make a solid reduction of rules. Since you are only doing 2, I would list them separately, to avoid the multiport extension... or better yet just one by range, if you don't care about blocking 23 and 24.
-p tcp -m tcp --dport 22:25
As for established tracking, yes you can use it with fail2ban, although it does have some considerations. To get the most impact, you'd want to place fail2ban chain below ESTABLISHED,RELATED. However this will allow already established connections, considering you'd need an already authenticated user, it seems reasonable enough.
This is a mini example of the tables I use for my servers, with some example rules, I commented it for you,
*raw
:PREROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
## Stateless on Loopback
## Remove everything before #filter if iptables chokes on #raw
-A OUTPUT -o lo -j NOTRACK
COMMIT
*filter
## Default Chains
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
## Proto Chains
:FWINPUT-TCP - [0:0]
:FWINPUT-UDP - [0:0]
:FWINPUT-ICMP - [0:0]
## FAIL2BAN Chain
:fail2ban - [0:0]
## Accept Established
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
## Accept Loopback
-A INPUT -i lo -j ACCEPT
## Seperate Proto
-A INPUT -p tcp -g FWINPUT-TCP
-A INPUT -p udp -g FWINPUT-UDP
-A INPUT -p icmp -g FWINPUT-ICMP
## Reject Anything Non-TCP/UDP/ICMP
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
## TCP Rules
-A FWINPUT-TCP -p tcp -m tcp --dport 80 -j ACCEPT
-A FWINPUT-TCP -p tcp -m tcp --dport 443 -j ACCEPT
## fail2ban Check
-A FWINPUT-TCP -p tcp -m tcp --dport 22:25 -g fail2ban
## fail2ban Return
-A FWINPUT-TCP -p tcp -m tcp --dport 22 -j ACCEPT
-A FWINPUT-TCP -p tcp -m tcp --dport 25 -j ACCEPT
## TCP-Reset Ident
-A FWINPUT-TCP -p tcp -m tcp --dport 113 -j REJECT --reject-with tcp-reset
## Reject Any Other TCP Traffic
-A FWINPUT-TCP -j REJECT --reject-with icmp-port-unreachable
## UDP Rules
-A FWINPUT-UDP -p udp --dport 5353 -d 224.0.0.251 -j ACCEPT
## Reject Any Other UDP Traffic
-A FWINPUT-UDP -j REJECT --reject-with icmp-port-unreachable
## ICMP Rules
-A FWINPUT-ICMP -p icmp -m icmp --icmp-type 8 -m limit --limit 5/s -j ACCEPT
-A FWINPUT-ICMP -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A FWINPUT-ICMP -p icmp -m icmp --icmp-type 4 -j ACCEPT
-A FWINPUT-ICMP -p icmp -m icmp --icmp-type 11 -j ACCEPT
-A FWINPUT-ICMP -p icmp -m icmp --icmp-type 12 -j ACCEPT
## Reject Any Other ICMP Types
-A FWINPUT-ICMP -j REJECT --reject-with icmp-host-prohibited
## fail2ban Inserted Rules
-A fail2ban -j RETURN
COMMIT
I would just blank out the action start and action stop, and let fail2ban just add the blocked IP rules when running. This would a pinch more manual considerations, like if you wanted to start using fail2ban to block more stuff... but if it's a set type of thing your trying to work on, then it shouldn't be a problem.
... On my home system, and not on servers, I usually just set an iptables limit and call it good enough.