I would like to scan multiple ports in multiple hosts. I used this script but it takes long time to show the result.
#!/bin/bash
hosts=(
"server1"
"server2"
)
for host in "${hosts[#]}"
do
echo "=========================================="
echo "Scanning $host"
echo "=========================================="
for port in {21,22,80}
do
echo "" > /dev/tcp/$host/$port && echo "Port $port is open"
done 2>/dev/null
done
Some people suggested to use telnet or NetCat instead but i prefer to do it without installing any new packages. So, are there any ways to speed it up by multithreading or other way.
You could use GNU Parallel to run all the checks in parallel. I am not the best at using it, and #OleTange (the author) normally has to correct me but I keep trying. So, let's try your case, by building up to it slowly:
parallel echo {1} {2} ::: 192.168.0.1 192.168.0.8 ::: 21 22 80
192.168.0.8 22
192.168.0.8 80
192.168.0.8 21
192.168.0.1 80
192.168.0.1 22
192.168.0.1 21
looks kind of hopeful to me. Then I add in -k to keep the results in order, and I supply a function that takes those IP addresses and ports as arguments:
parallel -k 'echo "" > /dev/tcp/{1}/{2} && echo {1}:{2} is open' ::: 192.168.0.1 192.168.0.8 ::: 21 22 80 2>/dev/null
192.168.0.1:80 is open
192.168.0.8:21 is open
192.168.0.8:22 is open
192.168.0.8:80 is open
This will run 8 jobs in parallel if your CPU has 8 cores, however echo is not very resource intensive so you can probably run 32 in parallel, so add -j 32 after the -k.
If you wanted to stick closer to your own script, you can do it like this:
#!/bin/bash
hosts=(
"192.168.0.1"
"192.168.0.8"
)
for host in "${hosts[#]}"
do
for port in {21,22,80}
do
echo "(echo > /dev/tcp/$host/$port) 2>/dev/null && echo Host:$host Port:$port is open"
done
done | parallel -k -j 32
Basically, instead of running your commands, I am just sending them to the stdin of parallel so it can do its magic with them.
You could run all three pokes in the background, then wait for them all to finish, and probably slash the running time to 1/3.
for port in 21 22 80; do
echo "" > /dev/tcp/$host/$port 2>/dev/null &
pid[$port]=$!
done
for port in 21 22 80; do
wait $pid[$port] && echo "Port $port" is open"
done
You could add parallelism by running multiple hosts in the background, too, but that should be an obvious extension.
#!/bin/bash
function alarm {
local timeout=$1; shift;
# execute command, store PID
bash -c "$#" &
local pid=$!
# sleep for $timeout seconds, then attempt to kill PID
{
sleep "$timeout"
kill $pid 2> /dev/null
} &
wait $pid 2> /dev/null
return $?
}
function scan {
if [[ -z $1 || -z $2 ]]; then
echo "Usage: ./scanner <host> <port, ports, or port-range>"
echo "Example: ./scanner google.com 79-81"
return
fi
local host=$1
local ports=()
# store user-provided ports in array
case $2 in
*-*)
IFS=- read start end <<< "$2"
for ((port=start; port <= end; port++)); do
ports+=($port)
done
;;
*,*)
IFS=, read -ra ports <<< "$2"
;;
*)
ports+=($2)
;;
esac
# attempt to write to each port, print open if successful, closed if not
for port in "${ports[#]}"; do
alarm 1 "echo >/dev/tcp/$host/$port" &&
echo "$port/tcp open" ||
echo "$port/tcp closed"
done
}
scan $1 $2
Related
I have a file in which I have given all the IP addresses. The file looks like following:
[asad.javed#tarts16 ~]#cat file.txt
10.171.0.201
10.171.0.202
10.171.0.203
10.171.0.204
10.171.0.205
10.171.0.206
10.171.0.207
10.171.0.208
I have been trying to loop over the IP addresses by doing the following:
launch_sipp () {
readarray -t sipps < file.txt
for i in "${!sipps[#]}";do
ip1=(${sipps[i]})
echo $ip1
sip=(${i[#]})
echo $sip
done
But when I try to access the array I get only the last IP address which is 10.171.0.208. This is how I am trying to access in the same function launch_sipp():
local sipp=$1
echo $sipp
Ip=(${ip1[*]})
echo $Ip
Currently I have IP addresses in the same script and I have other functions that are using those IPs:
launch_tarts () {
local tart=$1
local ip=${ip[tart]}
echo " ---- Launching Tart $1 ---- "
sshpass -p "tart123" ssh -Y -X -L 5900:$ip:5901 tarts#$ip <<EOF1
export DISPLAY=:1
gnome-terminal -e "bash -c \"pwd; cd /home/tarts; pwd; ./launch_tarts.sh exec bash\""
exit
EOF1
}
kill_tarts () {
local tart=$1
local ip=${ip[tart]}
echo " ---- Killing Tart $1 ---- "
sshpass -p "tart123" ssh -tt -o StrictHostKeyChecking=no tarts#$ip <<EOF1
. ./tartsenvironfile.8.1.1.0
nohup yes | kill_tarts mcgdrv &
nohup yes | kill_tarts server &
pkill -f traf
pkill -f terminal-server
exit
EOF1
}
ip[1]=10.171.0.10
ip[2]=10.171.0.11
ip[3]=10.171.0.12
ip[4]=10.171.0.13
ip[5]=10.171.0.14
case $1 in
kill) function=kill_tarts;;
launch) function=launch_tarts;;
*) exit 1;;
esac
shift
for ((tart=1; tart<=$1; tart++)); do
($function $tart) &
ips=(${ip[tart]})
tarts+=(${tart[#]})
done
wait
How can I use different list of IPs for a function created for different purpose from a file?
How about using GNU parallel? It's an incredibly powerful wonderful-to-know very popular free linux tool, easy to install.
Firstly, here's a basic parallel tool usage ex.:
$ parallel echo {} :::: list_of_ips.txt
# The four colons function as file input syntax.†
10.171.0.202
10.171.0.201
10.171.0.203
10.171.0.204
10.171.0.205
10.171.0.206
10.171.0.207
10.171.0.208
†(Specific to parallel; see parallel usage cheatsheet here]).
But you can replace echo with just about any as complex series of commands as you can imagine / calls to other scripts. parallel loops through the input it receives and performs (in parallel) the same operation on each input.
More specific to your question, you could replace echo simply with a command call to your script
Now you would no longer need to handle any looping through ip's itself, and instead be written designed for just a single IP input. parallel will handle running the program in parallel (you can custom set the number of concurrent jobs with option -j n for any int 'n')* .
*By default parallel sets the number of jobs to the number of vCPUs it automatically determines your machine has available.
$ parallel process_ip.sh :::: list_of_ips.txt
In pure Bash:
#!/bin/bash
while read ip; do
echo "$ip"
# ...
done < file.txt
Or in parallel:
#!/bin/bash
while read ip; do
(
sleep "0.$RANDOM" # random execution time
echo "$ip"
# ...
) &
done < file.txt
wait
This question already has answers here:
While loop stops reading after the first line in Bash
(5 answers)
Closed 2 years ago.
I've created a bash script to connect to a number of servers and execute a program. The ips and quantities per IP should be read from a config file that is structured like this:
127.0.0.1 10
127.0.0.1 1
127.0.0.1 3
etc
j=$((0))
while IFS=' ' read -r ip quantity; do
echo "${ip} x ${quantity}";
for (( i = 1; i <= quantity; i++ ))
do
echo "ssh root#${ip} cd test/libhotstuff && ./examples/hotstuff-app --conf ./hotstuff.gen-sec${j}.conf > log${j} 2>&1"
ssh root#"${ip}" "cd test/libhotstuff && ./examples/hotstuff-app --conf ./hotstuff.gen-sec${j}.conf > log${j} 2>&1" &
j=$((j+1))
done
sleep 1
done < ips
I noticed that this while loop breaks if the execution takes too long. If I put sleep for 1s here it will stop after the first execution. If I remove it, but the inner loop takes too long a subset of the lines will not be read.
What is the problem here?
Here's a version that starts your background processes with a 1 second delay between each, waits 6 minutes before killing them one by one, with a 1 second delay between each, to give them approximately the same running time.
You should also add some options to ssh to prevent it from interfering with stdin and terminate your loop prematurely while running.
-n
Prevents reading from stdin
-oBatchMode=yes
Passphrase/password querying will be disabled
-oStrictHostKeyChecking=no
Connect to host even if the host key has changed
#!/bin/bash
sshopts=(-n -oBatchMode=yes -oStrictHostKeyChecking=no)
j=0
pids=()
while IFS=$' \t\n' read -r ip quantity; do
echo "${ip} x ${quantity}";
for (( i = 0; i < quantity; ++i ))
do
remotecmd="cd test/libhotstuff && ./examples/hotstuff-app --conf ./hotstuff.gen-sec${j}.conf > log${j} 2>&1"
localcmd=(ssh ${sshopts[#]} root#${ip} "$remotecmd")
echo "${localcmd[#]}"
"${localcmd[#]}" &
# store the background pid
pids+=($!)
(( ++j ))
sleep 1
done
done < ips
seconds=360
echo "running ${pids[#]} in the background $seconds seconds"
sleep $seconds
echo "telling the background processes to terminate"
for pid in ${pids[#]}
do
echo killing $pid
kill $pid
sleep 1
done
echo "waiting for all the background processes to terminate"
wait
echo Done
Here is a version that offloads the loop and parallel processes to the remote shell script. Generate a remote shell script from a HereDocument with quantity, and wait for all the background processes to terminate before exiting.
#!/usr/bin/env sh
while IFS=$' \t\n\r' read -r ip quantity || [ -n "$quantity" ]
do
{
# When satisfied by the output:
# Ucomment the line below and delete its following line with the echo and cat
# ssh "root#$ip" <<EOF
echo ssh "root#$ip"; cat <<EOF
if cd test/libhotstuff
then
i=$quantity
until
i=\$((i - 1))
[ \$i -lt 0 ]
do
./examples/hotstuff-app \\
--conf "./hotstuff.gen-sec\$i.conf" >"log\$i" 2>&1 &
done
wait
fi
EOF
} &
done <ips
# Wait for all child processes to terminate
wait
echo "All child ssh done!"
Another way replacing the dynamic HereDocument by an inline shell script called with a quantity argument:
#!/usr/bin/env sh
while IFS=$' \t\n\r' read -r ip quantity || [ -n "$quantity" ]; do
echo ssh "root#$ip" sh -c '
if cd test/libhotstuff
then
i=0
while [ $i -lt "$1" ]; do
./examples/hotstuff-app --conf "./hotstuff.gen-sec$i.conf" >"log$i" 2>&1 &
i=$((i + 1))
done
wait
fi
' _ "$quantity" &
done <ips
# Wait for all child processes to terminate
wait
echo "All child ssh done!"
I want to ping in a for loop using dynamic names.
When i try ping -c1 -w1 10.0.0.10 > /dev/null it works perfectly, but when i try to change my static adress into a dynamic one, i'll get an error.
declare -A array=([piotr_pc]=10.0.0.10 )
for item in ${!array[*]}
do
file=$(grep $item homesystem/web/openvpn-status.log | tr 'm\n' 'p,')
IFS=','
for x in $file
do
eval $item+=\("$x"\)
done
eval echo \${$item[9]}
if ping -c1 -w1 eval echo \${$item[9]} > /dev/null; then
echo eval echo \${$item[9]} "ONLINE" $NOW
echo "UPDATE openvpn SET status='ONLINE', last_online='$NOW' , Common Name = '${piotr_pc[0]}', Real Address = '${piotr_pc[1]}', Bytes Received = '${piotr_pc[2]}', Bytes Sent = '${piotr_pc[3]}', Connected Since = '${piotr_pc[4]}', Virtual Address = '${piotr_pc[5]}' , Last Ref = '${piotr_pc[8]}' WHERE ip_vpn='${piotr_pc[5]}'"
#mysql -uphptest -pphphaslo openwrt -e "UPDATE openvpn SET status='ONLINE', last_online='$NOW' WHERE ip_vpn='$ip'"
else
eval echo \${$item[9]} "OFFLINE"
#mysql -uphptest -pphphaslo openwrt -e "UPDATE openvpn SET status='OFFLINE' WHERE ip_vpn='$ip'"
fi
done
Result:
root#VigoradoNetwork:/www1# ./ping.sh
10.0.0.10
BusyBox v1.28.3 () multi-call binary.
Usage: ping [OPTIONS] HOST
Send ICMP ECHO_REQUEST packets to network hosts
-4,-6 Force IP or IPv6 name resolution
-c CNT Send only CNT pings
-s SIZE Send SIZE data bytes in packets (default 56)
-t TTL Set TTL
-I IFACE/IP Source interface or IP address
-W SEC Seconds to wait for the first response (default 10)
(after all -c CNT packets are sent)
-w SEC Seconds until ping exits (default:infinite)
(can exit earlier with -c CNT)
-q Quiet, only display output at start
and when finished
-p HEXBYTE Pattern to use for payload
10.0.0.10 OFFLINE
I want to change this 10.0.0.10 with eval echo \${$item[9]} but i dont know how to do this.
I'm not sure if i understood you correctly but following may be helpful if you are running multiple pings to hosts using a hostlist file.
#!/bin/bash
echo "`date` Starting the Ping Check...."
function pingHost () {
hostTarget=${1}
# send stdout/stderr to /dev/null since all we need is the return code
ping -c2 ${hostTarget} >/dev/null 2>&1 &&
echo Host ${hostTarget} is SUCCESS||
echo Host ${hostTarget} is FAILED
}
data=$(<my_hostlist) # Place your hostlist here example "Master_hostlist"
for line in ${data}
do
# call our function and place the call in the background
pingHost ${line} &
done
# wait for all outstanding background jobs to complete before continuing
wait
# [optional] let operator know we're done.
echo "Completed #=> `date`"
I am currently writing the following script that logs into a remote server and runs couple of commands to verify the performance of the server and prints a message based on the output of those commands .But the ssh doesn't work and returns the stats of the server that hosts the script instead .
Script
#!/bin/bash
#######################
#Function to add hosts to the array
#the following function takes the ip addresses provided while the script is run and stores them in an array
#######################
Host_storing_func () {
HOST_array=()
for i in $# ;do
HOST_array+=(${i});
done
#echo ${HOST_array[*]}
}
#######################
#Calling above function
#######################
Host_storing_func "$#"
############################################################
#Collect Stats of Ping,memory,iowait time test function
############################################################
b=`expr ${#HOST_array[*]} - 1 `
for i in `seq 0 $b` ;do
sshpass -f /root/scripts/passwordFile.txt /usr/bin/ssh student35#${HOST_array[${i}]} << HERE
echo `hostname`
iowaittm=`sar 2 2|awk '/^Average/{print $5};'`
if [ $iowaittm > 10 ];then
echo "IO ==> BAD"
else
echo "IO ==> GOOD"
fi
memoryy=`free -m |grep Swap|awk '{if($2 == 0) print 0;else print (($4 / $2 ) * 100)}'`
if [ ${memoryy} < '10' ] ;then
echo "memory ==> good"
elif [[ "${memory}" -ge 0 ]] && [[ "${memory}" -le 10 ]];then
echo "No Swap"
else
echo "memory ==> bad"`enter code here`
fi
ping -w2 -c2 `hostname` | grep "packet loss"|awk -F, '{print $3}'|awk -F% '{print $1}'|sed 's/^ *//'|awk '{if ($1 == 0) print "Yes" ;else print "No"}'
HERE
done
Output : oc5610517603.XXX.com is the name of the source server
[root#oc5610517603 scripts]# ./big_exercise.sh 9.XXX.XXX.XXX 9.XXX.XXX.XXX
Pseudo-terminal will not be allocated because stdin is not a terminal.
oc5610517603.XXX.com
IO ==> GOOD
No Swap
ping: oc5610517603.ibm.com: Name or service not known
Pseudo-terminal will not be allocated because stdin is not a terminal.
oc5610517603.XXX.com
IO ==> GOOD
No Swap
ping: oc5610517603.XXX.com: Name or service not known
thanks for checking the script , I figured out a way to solve the problem
It is the sshpass command that is causing issue , you just have to put the opening HERE in single quotes if you want to use variables with in the HEREdoc but if the variables are calculated before ssh then you don't have to put opening HERE in single quotes
sshpass -f /root/scripts/passwordFile.txt /usr/bin/ssh -T student35#${i} << 'HERE'
after I changed the sshpass command as above my script worked
I have modified your script a bit.
As suggested by #chepner, I am not using the Host_storing_func.
Heredocs for sshpaas are somewhat tricky. You have to escape every back-tick and $ sign in the heredoc.
Notice the - before the heredoc start, it allows you to indent the heredoc body. Also, try to avoid back-ticks when you can. use $(command) instead.
Hope it helps.
#!/bin/bash
#######################
#Function to add hosts to the array
#the following function takes the ip addresses provided while the script is run and stores them in an array
#######################
array=( "$#" )
user="student35"
############################################################
#Collect Stats of Ping,memory,iowait time test function
############################################################
for host in ${array[#]}; do
sshpass -f /root/scripts/passwordFile.txt /usr/bin/ssh -l ${user} ${host} <<-HERE
thishost=\$(hostname)
echo "Current Host -> \$thishost";
iowaittm=\`sar 2 2|awk '/^Average/{print \$5}'\`
if [ \$iowaittm > 10 ]; then
echo "IO ==> BAD"
else
echo "IO ==> GOOD"
fi
memory=\$(free -m | grep Swap | awk '{if(\$2 == 0) print 0;else print ((\$4 / \$2 ) * 100)}')
if [ \${memory} < '10' ] ;then
echo "memory ==> good"
elif [[ "\${memory}" -ge 0 ]] && [[ "\${memory}" -le 10 ]]; then
echo "No Swap"
else
echo "memory ==> bad"\`enter code here\`
fi
ping -w2 -c2 \`hostname\` | grep "packet loss"|awk -F, '{print \$3}'|awk -F% '{print \$1}'|sed 's/^ *//'|awk '{if (\$1 == 0) print "Yes" ;else print "No"}'
HERE
done
I am required to test at least 130 ip addresses and ports.
I am hoping to write a bash script such that it reads the ip address and ports from an input file.
I have the following
while read line
do
telnet $line >> $2
done < $1
This is a crappy code as it cannot determine whether its connected or failed, and I have to rely on its auto escape character to disconnect from a connection.
How can I improvise this such that it updates $2 with the status quickly?
I am working on Redhat and do not have netcat or expect installed..
As other stackoverflower's said, I would recommend using nmap or netcat if avilable.
However, if you cannot use those software, you can use bash's builtin /dev/tcp/<host>/<port> instead.
http://www.gnu.org/software/bash/manual/bashref.html#Redirections
I could'nt figure out which version of bash you are using, but /dev/tcp/... seems to implemented since some old bash.
#!/bin/bash
echo "scanme.nmap.org 21
scanme.nmap.org 22
scanme.nmap.org 23
scanme.nmap.org 79
scanme.nmap.org 80
scanme.nmap.org 81" | \
while read host port; do
r=$(bash -c 'exec 3<> /dev/tcp/'$host'/'$port';echo $?' 2>/dev/null)
if [ "$r" = "0" ]; then
echo $host $port is open
else
echo $host $port is closed
fi
done
This produces
scanme.nmap.org 21 is closed
scanme.nmap.org 22 is open
scanme.nmap.org 23 is closed
scanme.nmap.org 79 is closed
scanme.nmap.org 80 is open
scanme.nmap.org 81 is closed
UPDATED: The following can do timeout.
Although it may seem little tricky, idea is just to kill the child process after some timeout.
Bash script that kills a child process after a given timeout
#!/bin/bash
echo "scanme.nmap.org 80
scanme.nmap.org 81
192.168.0.100 1" | (
TCP_TIMEOUT=3
while read host port; do
(CURPID=$BASHPID;
(sleep $TCP_TIMEOUT;kill $CURPID) &
exec 3<> /dev/tcp/$host/$port
) 2>/dev/null
case $? in
0)
echo $host $port is open;;
1)
echo $host $port is closed;;
143) # killed by SIGTERM
echo $host $port timeouted;;
esac
done
) 2>/dev/null # avoid bash message "Terminated ..."
this produces
scanme.nmap.org 80 is open
scanme.nmap.org 81 is closed
192.168.0.100 1 timeouted
since 192.168.100 does not exist in my local network.
A slight update to the accepted answer:
#!/bin/bash
# supertelnet 127.0.0.1:3306 10.10.10.45:22
(
TCP_TIMEOUT=3
for hostport in ${#}; do
a=(${hostport//:/ })
host=${a[0]}
port=${a[1]}
(CURPID=$BASHPID;
(sleep $TCP_TIMEOUT;kill $CURPID) &
exec 3<> /dev/tcp/$host/$port
) 2>/dev/null
case $? in
0)
echo $host $port is open;;
1)
echo $host $port is closed;;
143) # killed by SIGTERM
echo $host $port timeouted;;
esac
done
) 2>/dev/null # avoid bash message "Terminated ..."
I find this to be a lot more friendly as a script.
Pure bash nmap replacment:
Sorry for comming so late on this question.
Speed parallelized process
This could be a lot quicker if all probe are done together:
TargetList=(
scanme.nmap.org:21 scanme.nmap.org:22 scanme.nmap.org:23
scanme.nmap.org:79 scanme.nmap.org:80 scanme.nmap.org:81
)
checkTcpConn() {
local line testfd bpid=$BASHPID
( sleep 3 && kill -INT $bpid && echo $1 timeout) &
if exec {testfd}<>/dev/tcp/${1/:/\/};then
echo >&$testfd $'\r\n\r\n'
read -ru $testfd -t 1 line
[[ $line ]] &&
echo $1 open $line ||
echo $1 open
exec {testfd}<&-
else
echo $1 closed
fi
}
for target in ${TargetList[#]};do
checkTcpConn $target &
done 2>/dev/null | sort
will output quickly:
scanme.nmap.org:21 closed
scanme.nmap.org:22 open SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13
scanme.nmap.org:23 closed
scanme.nmap.org:79 closed
scanme.nmap.org:80 open HTTP/1.1 400 Bad Request
scanme.nmap.org:81 closed
or worst:
for target in scanme.nmap.org:{{1..1024},3128,3306,5432,5900,8080};do
checkTcpConn $target &
sleep .002
done 2>/dev/null | grep open
scanme.nmap.org:22 open SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13
scanme.nmap.org:80 open HTTP/1.1 400 Bad Request
And with a timeout:
for target in scanme.nmap.org:2{2,1} 192.168.210.123:1 ;do
checkTcpConn $target &
done 2>/dev/null |
sort
192.168.210.123:1 timeout
scanme.nmap.org:21 closed
scanme.nmap.org:22 open SSH-2.0-OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.13
Nota The last pipe done 2>/dev/null | sort is required in order to avoid job control messages. For showing raw output, use
...
done 2>/dev/null | cat