I would like to block IP address from a log file, IPs are collected from access.log file in apache2.
The IPs are correctly collected in file ips.log, but while reading the file to ban collected IPs, the block is not done.
#!/bin/bash
# Store words to avoid an search for
BADWORDS=( '/etc/passwd' 'file?' 'w00tw00t' 'fckeditor' 'ifconfig' )
# Get number of elements in the backup_files array
entries=${#BADWORDS[#]}
for ((i=0; i<= entries-1; i++))
do
setBadWord=${BADWORDS[$i]}
tail -F /var/log/apache2/access.log | grep --line-buffered "$setBadWord" | while read -r a; do echo "$a" | awk '{ print $1 } ' >> ips.log; done
done # end for
while IFS= read -r ip; do
iptables -A INPUT -s "$ip" -j DROP
done < ips.log
Your code has many issues:
it runs a new copy of awk for every line selected (awk is not needed at all);
it tries to run the first loop multiple times (once for every element of "$BADWORDS")
the first loop never finishes (because of tail -F) and so the iptables loop never starts
the iptables command appends a new rule even if the IP has been seen before
it is simpler to write i<entries rather than i<=entries-1, and even simpler to just use for setBadword in "${BADWORDS[#]}"; do ...
If you really want to permanently loop reading the logfile, with GNU utilities you can do something like:
#!/bin/sh
log=/var/log/apache2/access.log
words=/my/list/of/badwords/one/per/line
banned=/my/list/of/already/banned/ips
tail -F "$log" |\
grep --line-buffered -Ff "$words" |\
while read ip junk; do
grep -qxF $ip "$banned" || (
iptables -A INPUT -s $ip -j DROP
echo $ip >> "$banned"
)
done
# we never get here because "tail -F" never finishes
To just process the logfile once and then finish, you can feed grep from "$log" directly:
grep --line-buffered -Ff "$words" "$log" | ...
but it is probably less error-prone to just use fail2ban which is explicitly designed for this sort of task.
I’m doing simple bash script that is basically working with files and basic bash functions. However I’ve crossed something I don’t know how to deal with. I need to convert 1st column of inputed file IPv4/v6 to their hostname and prints that out. w
I know, that to print out 1st column is good to use awk | ‘{print $1}’, and to find host of IP is good nslookup, but I can’t quite put those together to echo out that IP column as HOSTNAME column.
This is how I print out 1st column. Parameter passed into function is a filename.
function ip_echo(){
cat $1 | awk '{print $1}’
}
You can look here to get sense how to do it.
In short, your command should use dig and reverse lookup to translate the IP's into their hostnames.
Finally, your bash command should be look like this -
cat $1 | awk '{print $1}' | while read count ip; do printf "%d " $count; printf "%s " $ip; dig +noall +answer -x $ip; done
I need to create a Bash Script that will be able to ssh into a computer or Machine B, run a command and have the output piped back to a .txt file on machine A how do I go about doing this? Ultimately it will be list of computers that I will ssh to and run a command but all of the output will append to the same .txt file on Machine A.
UPDATE: Ok so I went and followed what That other Guy suggested and this is what seems to work:
File=/library/logs/file.txt
ssh -n username#<ip> "$(< testscript.sh)" > $File
What I need to do now is instead of manually entering an ip address, I need to have it read from a list of hostnames coming from a .txt file and have it place it in a variable that will substitute the ip address. An example would be: ssh username#Variable in which "Variable" will be changing each time a word is read from a file containing hostnames. Any ideas how to go about this?
This should do it
ssh userB#machineB "some command" | ssh userA#machineA "cat - >> file.txt"
With your commands:
ssh userB#machineB <<'END' | ssh userA#machineA "cat - >> file.txt"
echo Hostname=$(hostname) LastChecked=$(date)
ls -l /applications/utilities/Disk\ Utility.app/contents/Plugins/*Partition.dumodule* | awk '{printf "Username=%s DateModified=%s %s %s\n", $3, $6, $7, $8}'
END
You could replace the ls -l | awk pipeline with a single stat call, but it appears that the OSX stat does not have a way to return the user name, only the user id
I am working on a script that executes ssh to few systems (listed in lab.txt), run two commands, store the output of commands in two different variables and print them.
Here is the script used :
#!/bin/bash
while read host; do
ssh -n root#$host "$(STATUS=$(awk 'NR==1{print $1}' /etc/*release) \
OS=$(/opt/agent/bin/agent.sh status | awk 'NR==1{print $3 $4}'))"
echo $STATUS
echo $OS
done < lab.txt
The lab.txt file contains few Ips where I need to login, execute and print the command output.
~#] cat lab.txt
192.168.1.1
192.168.1.2
While executing the script, the ssh login prompt of 192.168.1.1 is shown and after entering the password, the output is shown blank. Same as for next ip 192.168.1.2
When I execute these command manually within 192.168.1.1, the following is returned.
~]# awk 'NR==1{print $1}' /etc/*release
CentOS
~]# /opt/agent/bin/agent.sh status | awk 'NR==1{print $3 $4}'
isrunning
What could be wrong with the script? Is there a better way of doing this?
As the comment says, you are setting the variables inside the bash session on the server side and trying to read them from the client side.
If you want to assign the variables in the client script you need to put the assignment in front of the ssh command, and separate the two assignments. Something like the following.
STATUS=`ssh -n root#$host 'awk \'NR==1{print $1}\' /etc/*release)`
OS=`ssh -n root#$host '/opt/agent/bin/agent.sh status | awk \'NR==1{print $3 $4}\''`
You need to do two ssh commands. It also simplifies things if you run awk on the client rather than the server, because quoting in the ssh command gets complicated.
while read host; do
STATUS=$(ssh -n root#$host 'cat /etc/*release' | awk 'NR==1{print $1}')
OS=$(ssh -n root#$host /opt/agent/bin/agent.sh status | awk 'NR==1{print $3 $4}')
echo $STATUS
echo $OS
done < lab.txt
with one ssh statement:
read STATUS OS < <(ssh -n root#$host "echo \
\$(awk 'NR==1{print \$1}' /etc/*release) \
\$(/opt/agent/bin/agent.sh status | awk 'NR==1{print \$3 \$4}')")
echo $STATUS
echo $OS
Explanation:
The <(command) syntax is called process substitution. You can use it anywhere where a file is expected.
Example:
sdiff <(echo -e "1\n2\n3") <(echo -e "1\n3")
The command sdiff expects two files as arguments. With the process substitution syntax you can use commands as arguments. ( e.g. fake files )
I'm trying to get the default gateway, using the destination 0.0.0.0.
I used the command netstat -rn | grep 0.0.0.0 and it returned this list:
Destination Gateway Genmask Flags MSS Window irtt Iface
10.9.9.17 0.0.0.0 255.255.255.255 UH 0 0 0 tun0
133.88.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 133.88.31.70 0.0.0.0 UG 0 0 0 eth0
My goal here is to ping the default gateway using the destination 0.0.0.0, which is 133.88.31.70, but this one returns a list because of using grep.
How do I get the default gateway only? I need it for my Bash script to determine whether the network connection is up or not.
You can get the default gateway using ip command like this:
IP=$(/sbin/ip route | awk '/default/ { print $3 }')
echo $IP
The ip route command from the iproute2 package can select routes without needing to use awk/grep, etc to do the selection.
To select the default route (from possibly many):
$ ip -4 route show default # use -6 instead of -4 for ipv6 selection.
default via 172.28.206.254 dev wlp0s20f3 proto dhcp metric 600
To select the next hop for a particular interface:
$ ip -4 route list type unicast dev eth0 exact 0/0 # Exact specificity
default via 172.29.19.1 dev eth0
In case of multiple default gateways, you can select which one gets chosen as the next hop to a particular destination address:
$ ip route get $(dig +short google.com | tail -1)
173.194.34.134 via 172.28.206.254 dev wlan0 src 172.28.206.66
cache
You can then extract the value using sed/awk/grep, etc. Here is one example using bash's read builtin:
$ read _ _ gateway _ < <(ip route list match 0/0); echo "$gateway"
172.28.206.254
works on any linux:
route -n|grep "UG"|grep -v "UGH"|cut -f 10 -d " "
This simple perl script will do it for you.
#!/usr/bin/perl
$ns = `netstat -nr`;
$ns =~ m/0.0.0.0\s+([0-9]+.[0-9]+.[0-9]+.[0-9]+)/g;
print $1
Basically, we run netstat, save it to $ns. Then find the line that starts off with 0.0.0.0. Then the parentheses in the regex saves everything inside it into $1. After that, simply print it out.
If it was called null-gw.pl, just run it on the command like:
perl null-gw.pl
or if you need it inside a bash expression:
echo $(perl null-gw.pl)
Good luck.
This is how I do it:
#!/bin/sh
GATEWAY_DEFAULT=$(ip route list | sed -n -e "s/^default.*[[:space:]]\([[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\).*/\1/p")
echo ${GATEWAY_DEFAULT}
For a list of all default gateways, use mezgani's answer, duplicated (and slightly simplified) here:
/sbin/ip route | awk '/^default/ { print $3 }'
If you have multiple network interfaces configured simultaneously, this will print multiple gateways. If you want to select a single known network interface by name (e.g. eth1), simply search for that in addition to filtering for the ^default lines:
/sbin/ip route |grep '^default' | awk '/eth1/ {print $3}'
You can make a script that takes a single network-interface name as an argument and prints the associated gateway:
#!/bin/bash
if [[ $# -ne 1 ]]; then
echo "ERROR: must specify network interface name!" >&2
exit 1
fi
# The third argument of the 'default' line associated with the specified
# network interface is the Gateway.
# By default, awk does not set the exit-code to a nonzero status if a
# specified search string is not found, so we must do so manually.
/sbin/ip route | grep '^default' | awk "/$1/ {print \$3; found=1} END{exit !found}"
As noted in the comments, this has the advantage of setting a sensible exit-code, which may be useful in a broader programmatic context.
There are a lot of answers here already. Some of these are pretty distro specific. For those who found this post looking for a way to find the gateway, but not needing to use it in code/batch utilization (as I did)... try:
traceroute www.google.com
the first hop is your default gateway.
The following command returns the default route gateway IP on a Linux host using only bash and awk:
printf "%d.%d.%d.%d" $(awk '$2 == 00000000 && $7 == 00000000 { for (i = 8; i >= 2; i=i-2) { print "0x" substr($3, i-1, 2) } }' /proc/net/route)
This should even work if you have more than one default gateway as long as their metrics are different (and they should be..).
Another perl thing:
$return = (split(" ", `ip route | grep default`))[2];<br>
Note: use these backticks before ip and after default
netstat -rn | grep 0.0.0.0 | awk '{print $2}' | grep -v "0.0.0.0"
#!/bin/bash
##################################################################3
# Alex Lucard
# June 13 2013
#
# Get the gateway IP address from the router and store it in the variable $gatewayIP
# Get the Router mac address and store it in the variable gatewayRouter
# Store your routers mac address in the variable homeRouterMacAddress
#
# If you need the gateway IP uncomment the next line to get the gateway address and store it in the variable gateWayIP
# gatewayIP=`sudo route -n | awk '/^0.0.0.0/ {print $2}'`
homeRouterMacAddress="20:aa:4b:8d:cb:7e" # Store my home routers mac address in homeRouterMac.
gatewayRouter=`/usr/sbin/arp -a`
# This if statement will search your gateway router for the mac address you have in the variable homeRouterMacAddress
if `echo ${gatewayRouter} | grep "${homeRouterMacAddress}" 1>/dev/null 2>&1`
then
echo "You are home"
else
echo "You are away"
fi
If you know that 0.0.0.0 is your expected output, and will be at the beginning of the line, you could use the following in your script:
IP=`netstat -rn | grep -e '^0\.0\.0\.0' | cut -d' ' -f2`
then reference the variable ${IP}.
It would be better to use awk instead of cut here... i.e.:
IP=`netstat -rn | grep -e '^0\.0\.0\.0' | awk '{print $2}'`
use command below:
route -n | grep '^0\.0\.\0\.0[ \t]\+[1-9][0-9]*\.[1-9][0-9]*\.[1-9][0-9]*\.[1-9][0-9]*[ \t]\+0\.0\.0\.0[ \t]\+[^ \t]*G[^ \t]*[ \t]' | awk '{print $2}'
/sbin/route |egrep "^default" |cut -d' ' -f2-12 #and 'cut' to taste...
To get the NIC name that it's the default gateway use this command:
netstat -rn | grep UG | awk '{print $8}'