I have two bash scripts. In the first one a user chain will be created like
#!/bin/bash
iptables -X STATS
iptables -N STATS
iptables -I INPUT -j STATS
In another bash script I will insert the rules like
#!/bin/bash
# HERE: If STATS-Chain not available end skript
iptables -A STATS --dport 80
How can I check at the position HERE: if the STATS-Chain is available?
Is it possible with iptables itself or only with iptables -L an some
sed/awk/grep... magic?
iptables-save output might be easier to grep for this task.
Check if chain STATUS is loaded, if it is exits:
/sbin/iptables-save | grep -q '\-A STATUS ' && exit 0
Related
I have found myself in several situations, where it would be handy to be able to feed a command arguments from an input file, on a per line basis. In the handful of times I've wanted to be able to do this, I've ended up finding a workaround or running the command multiple times manually.
I have a file input.txt which contains multiple lines, with an arbitrary number of arguments on each line. I am going to use my most recent need for this functionality as an example. I am trying to simply copy my iptables rules to ip6tables. I have run the command iptables -S > input.txt to generate the following file:
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -m state --state ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 53 -m state --state NEW -j ACCEPT
-A OUTPUT -p udp -m udp --dport 53 -m state --state NEW -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 80 -m state --state NEW -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 443 -m state --state NEW -j ACCEPT
For each line, I would like to run something like sudo ip6tables $line, so that ip6tables runs with all the arguments from a given line, and N commands are run, where N is the number of lines in input.txt.
The answer to this question should be applicable to this scenario in general. I am not looking for a solution that only works for copying iptables rules to the IPv6 counterpart (through some functionality in iptables for example). I would also prefer a one liner that I can execute in the terminal if possible, instead of writing a bash script. Though, if a bash script turns out to be much less cumbersome than a one liner, then I would accept that as an answer as well.
This sounds like the perfect job for xargs. For your iptables example it works like this:
xargs -L 1 sudo ip6tables < input.txt
xargs reads command arguments from stdin and executes the provided command with the arguments added to the command line. Here the arguments are piped in to stdin from the input.txt file. -L 1 limits the arguments to one line per execution, i.e. one line is added to the command line, the resulting command gets executed, continue with the next line etc.
If it is just for iptables your system may already have an appropriate iptables-restore command you just have to call with:
sudo iptables-restore input.txt
Now if input.txt is to be used to pass arguments to other commands, there i s an easy solution with Bash:
arguments is an array that is read (filled) from each line of input.txt
Iterate over each line of the input.txt file.
Pass the arguments array to your command like this:
#!/usr/bin/env bash
arguments=()
while read -r -a arguments; do
your_command "${arguments[#]}"
done <input.txt
for a one-liner of the above:
while read -ra a;do your_command "${a[#]}";done<input.txt
Or for a POSIX compatible version (no array):
while read -r a;do set -- $a; your_command "$#";done <input.txt
I am writing a shell script where i want to ssh to a server and get the cpu and memory details data of that displayed as a result. I’m using the help of top command here.
Script line:
ssh -q user#host -n “cd; top -n 1 | egrep ‘Cpu|Mem|Swap’”
But the result is
TERM environment variable is not set.
I had checked the same in the server by entering set | grep TERM and got result as TERM=xterm
Please someone help me on this. Many thanks.
Try using the top -b flag:
ssh -q user#host -n "cd; top -bn 1 | egrep 'Cpu|Mem|Swap'"
This tells top to run non-interactively, and is intended for this sort of use.
top need an environment. You have to add the parameter -t to get the result:
ssh -t user#host -n "top -n 1 | egrep 'Cpu|Mem|Swap'"
Got it..!! Need to make a small modification for the below script line.
ssh -t user#host -n "top -n 1 | egrep 'Cpu|Mem|Swap'"
Instead of -t we need to give -tt. It worked for me.
To execute command top after ssh’ing. It requires a tty to run. Using -tt it will enable a force pseudo-tty allocation.
Thanks stony for providing me a close enough answer!! :)
I am using Ubuntu Server 14.04 32bit for the following.
I am trying to use blocklists to add regional blocks (China, Russia...) to my firewall rules and am struggling with the length it takes my script to complete and understanding why a different script fails to work.
I had originally used http://whatnotlinux.blogspot.com/2012/12/add-block-lists-to-iptables-from.html as an example and tidied up / changed parts of the script to pretty close to what's below:
#!/bin/bash
# Blacklist's names & URLs array
declare -A blacklists
blacklists[china]="http://www.example.com"
#blacklists[key]="url"
for key in ${!blacklists[#]}; do
#Download blacklist
wget --output-document=/tmp/blacklist_$key.gz -w 3 ${blacklists[$key]}
iptables -D INPUT -j $key #Delete current iptables chain link
iptables -F $key #Flush current iptables chain
iptables -X $key #Delete current iptables chain
iptables -N $key #Create current iptables chain
iptables -A INPUT -j $key #Link current iptables chain to INPUT chain
#Read blacklist
while read line; do
#Drop description, keep only IP range
ip_range=`echo -n $line | sed -e 's/.*:\(.*\)-\(.*\)/\1-\2/'`
#Test if it's an IP range
if [[ $ip_range =~ ^[0-9].*$ ]]; then
# Add to the blacklist
iptables -A $key -m iprange --src-range $ip_range -j LOGNDROP
fi
done < <(zcat /tmp/blacklist_$key.gz | iconv -f latin1 -t utf-8 - | dos2unix)
done
# Delete files
rm /tmp/blacklist*
exit 0
This appears to work fine for short test lists, but manually adding many (200,000+) entries to iptables takes an EXORBITANT amount of time and I'm not sure why? Depending on the list I have calculated this taking upwards of 10 hours to complete which just seems silly.
After viewing the format of the iptables-save output I created a new script that uses iptables-save to save working iptables rules and then appends the expected format for blocks to this file, such as: -A bogon -m iprange --src-range 0.0.0.1-0.255.255.255 -j LOGNDROP, and eventually uses iptables-restore to load the file as seen below:
#!/bin/bash
# Blacklist's names & URLs arrays
declare -A blacklists
blacklists[china]="http://www.example.com"
#blacklists[key]="url"
iptables -F # Flush iptables chains
iptables -X # Delete all user created chains
iptables -P FORWARD DROP # Drop all forwarded traffic
iptables -N LOGNDROP # Create LOGNDROP chain
iptables -A LOGNDROP -p tcp -m limit --limit 5/min -j LOG --log-prefix "Denied TCP: " --log-level 7
iptables -A LOGNDROP -p udp -m limit --limit 5/min -j LOG --log-prefix "Denied UDP: " --log-level 7
iptables -A LOGNDROP -p icmp -m limit --limit 5/min -j LOG --log-prefix "Denied ICMP: " --log-level 7
iptables -A LOGNDROP -j DROP # Drop after logging
# Build first part of iptables-rules
for key in ${!blacklists[#]}; do
iptables -N $key # Create chain for current list
iptables -A INPUT -j $key # Link input to current list chain
done
iptables-save | sed '$d' | sed '$d' > /tmp/iptables-rules.rules # Save WORKING iptables-rules and remove last 2 liens (COMMIT & comment)
for key in ${!blacklists[#]}; do
#Download blacklist
wget --output-document=/tmp/blacklist_$key.gz -w 3 ${blacklists[$key]}
zcat /tmp/blacklist_$key.gz | sed '1,2d' | sed s/.*:/-A\ $key\ -m\ iprange\ --src-range\ / | sed s/$/\ -j\ LOGNDROP/ >> iptables-rules.rules
done
echo 'COMMIT' >> /tmp/iptables-rules.rules
iptables-restore < /tmp/iptables-rules.rules
# Delete files
rm /tmp/blacklist*
rm /tmp/iptables-rules.rules
exit 0
This works great for most lists on the testbed however there are specific lists that if included will produce the iptables-restore: line 389971 failed error, which is always the last line (COMMIT). I've read that due to the way iptables works whenever there is an issue reloading rules the error will always say the last line failed.
The truly odd thing is that testing these same lists on Ubuntu Desktop 14.04 64bit the second script works just fine. I have tried running the script on the Desktop machine, then using iptables-save to save a "properly" formatted version of the ruleset, and then loading this file to iptables on the server using iptables-restore and still receive the error.
I am at a loss as to how to troubleshoot this, why the initial script takes so long to add rules to iptables, and what could potentially be causing problems with the lists in the second script.
If you need to block a multitude of IP Addresses, use ipset instead.
Step 1: Create the IPset:
# Hashsize of 1024 is usually enough. Higher numbers might speed up the search,
# but at the cost of higher memory usage.
ipset create BlockAddress hash:ip hashsize 1024
Step 2: Add the addresses to block into that IPset:
# Put this in a loop, the loop reading a file containing list of addresses to block
ipset add BlockAddress $IP_TO_BLOCK
Finally, replace all those lines to block with just one line in netfilter:
iptables -t raw -A PREROUTING -m set --match-set BlockAddress src -j DROP
Done. iptables-restore will be mucho fasta.
IMPORTANT NOTE: I strongly suggest NOT using a domain name to be added into netfilter; netfilter needs to first do a DNS Resolve, and if DNS is not properly configured and/or too slow, it will fail. Rather, do a pre-resolve (or periodic resolve) of domain names to block, and feed the found IP addresses to the "file containing list of addresses to block". It should be an easy script, invoked from crontab every 5 minutes or so.
EDIT 1:
This is an example of a cronjob I use to get facebook.com's address, invoked every 5 minutes:
#!/bin/bash
fbookfile=/etc/iptables.d/facebook.ip
for d in www.facebook.com m.facebook.com facebook.com; do
dig +short "$d" >> "$fbookfile"
done
sort -n -u "$fbookfile" -o "$fbookfile"
Every half hour, another cronjob feeds those addresses to ipset:
#!/bin/bash
ipset flush IP_Fbook
while read ip; do
ipset add IP_Fbook "$ip"
done < /etc/iptables.d/facebook.ip
Note: I have to do this because doing dig +short facebook.com, for instance, returns exactly ONE IP address. After some observation, the IP address returned changed every ~5 minutes. Since I'm too lazy occupied to make an optimized version, I took the easy way out and do a flush/rebuild only every 30 minutes to minimize CPU spikes.
The following is how I ended up solving this using ipsets as well.
#!/bin/bash
# Blacklist names & URLs array
declare -A blacklists
blacklists[China]="url"
# blacklists[key]="url"
# etc...
for key in ${!blacklists[#]}; do
# Download blacklist
wget --output-document=/tmp/blacklist_$key.gz -w 3 ${blacklists[$key]}
# Create ipset for current blacklist
ipset create $key hash:net maxelem 400000
# TODO method for determining appropriate maxelem
while read line; do
# Add addresses from list to ipset
ipset add $key $line -quiet
done < <(zcat /tmp/blacklist_$key.gz | sed '1,2d' | sed s/.*://)
# Add rules to iptables
iptables -D INPUT -m set --match-set $key src -j $key # Delete link to list chain from INPUT
iptables -F $key # Flush list chain if existed
iptables -X $key # Delete list chain if existed
iptables -N $key # Create list chain
iptables -A $key -p tcp -m limit --limit 5/min -j LOG --log-prefix "Denied $key TCP: " --log-level 7
iptables -A $key -p udp -m limit --limit 5/min -j LOG --log-prefix "Denied $key UDP: " --log-level 7
iptables -A $key -p icmp -m limit --limit 5/min -j LOG --log-prefix "Denied $key ICMP: " --log-level 7
iptables -A $key -j DROP # Drop after logging
iptables -A INPUT -m set --match-set $key src -j $key
done
I'm not wildly familiar with ipsets but this makes for a much faster method of downloading, parsing and adding blocks.
I've added individual chains for each list for more verbose logging that will log which blocklist the dropped ip is coming from should you have multiples. On my actual box I'm using around 10 lists and have added several hundred thousand addresses with no problem!
Donwload Zones
#!/bin/bash
# http://www.ipdeny.com/ipblocks/
zone=/path_to_folder/zones
if [ ! -d $zone ]; then mkdir -p $zone; fi
wget -c -N http://www.ipdeny.com/ipblocks/data/countries/all-zones.tar.gz
tar -C $zone -zxvf all-zones.tar.gz >/dev/null 2>&1
rm -f all-zones.tar.gz >/dev/null 2>&1
Edit your Iptables bash script and add the following lines:
#!/bin/bash
ipset=/sbin/ipset
iptables=/sbin/iptables
route=/path_to_blackip/
$ipset -F
$ipset -N -! blockzone hash:net maxelem 1000000
for ip in $(cat $zone/{cn,ru}.zone $route/blackip.txt); do
$ipset -A blockzone $ip
done
$iptables -t mangle -A PREROUTING -m set --match-set blockzone src -j DROP
$iptables -A FORWARD -m set --match-set blockzone dst -j DROP
example: where "blackip.txt" is your own ip blacklist and "cn,ru" china-russia"
Source: blackip
I execute my bash script PLCCheck as process
./PLCCheck &
PLCCheck
while read -r line
do
...
def_host=192.168.100.110
def_port=6002
HOST=${2:-$def_host}
PORT=${3:-$def_port}
echo -n "OKConnection" | netcat -u -c $HOST $PORT
done < <(netcat -u -l -p 6001)
It listens on UDP Port 6001.
When I want to execute my second bash script SQLCheck as process that listens on UDP Port 4001
./SQLCheck &
SQLCheck
while read -r line
do
...
def_host=192.168.100.110
def_port=6002
HOST=${2:-$def_host}
PORT=${3:-$def_port}
echo -n "OPENEF1" | netcat -u -c $HOST $PORT
done < <(nc -l -p 4001)
I got this error:
Error: Couldn't setup listening socket (err=-3)
Port 6001 and 4001 are open in the iptables and both scripts work as a single process. Why do I get this error?
I have checked the man page of nc. I think it is used on a wrong way:
-l Used to specify that nc should listen for an incoming connection rather
than initiate a connection to a remote host. It is an error to use this
option in conjunction with the -p, -s, or -z options. Additionally,
any timeouts specified with the -w option are ignored.
...
-p source_port
Specifies the source port nc should use, subject to privilege restrictions
and availability. It is an error to use this option in conjunction with the
-l option.
According to this one should not use -l option with -p option!
Try to use without -p, just nc -l 4001. Maybe this is the error...
I have basically two lines of code which are:
tcpdump -i eth0 -s 65535 -w - >/tmp/Captures
tshark -i /tmp/Captures -T pdml >results.xml
if I run them both in separate terminals it works fine.
However I've been trying to create a simple bash script that will execute them at the same time, but have had no luck. Bash script is as follows:
#! /bin/bash
tcpdump -i eth0 -s 65535 -w - >/tmp/Captures &
tshark -i /tmp/Captures -T pdml >results.xml &
If anyone could possibly help in getting this to work or getting it to "run tcpdump until a key is pressed, then run tshark. then when a key is pressed again close."
I have only a little bash scripting experience.
Do you need to run tcpdump and tshark separately? Using a pipe command will feed the output of tcpdump to the input of tshark.
tcpdump -i eth0 -s 65535 | tshark -T -pdml > results.xml