script to check if I can access multiple servers - bash

I made this script to check if I can connect to a list of many servers:
for SERVER in $(cat servers.txt); do
ssh root#$SERVER && echo OK $SERVER || echo ERR $SERVER
done
The problem is that if is the first time I’m connecting to the server, the server asks the classic question “The authenticity of host ‘x.x.x.x’ can't be established... bla bla bla” Then I have to respond yes or not and it losses the purpose of making it a script, is there any way to bypass that so I can add it to the script?
Also, There are some servers in which I don't have my keys in them but they have the option to enter a password. In that case it will wait until I try a password to continue with the script execution, so I was wondering if there is a way to improve this script so if the server asks for a password then set it to ERR $SERVER and continue with the script?
Thank you for your help.

Do you actually need to establish an SSH connection?
Or is simply opening a socket connection to the host on the SSH port enough for you to determine that the server is online?
servers.txt
hostA.example.com
hostB.example.com
hostC.example.com
port-probe.sh
#!/bin/bash
PORT=22
TIMEOUT=3
for SERVER in $(cat servers.txt); do
# Open a socket and send a char
echo "-" | nc -w $TIMEOUT $SERVER $PORT &> /dev/null
# Check exit code of NC
if [ $? -eq 0 ]; then
echo "$SERVER is Available."
else
echo "$SERVER is Unavailable."
fi
done

You can use the -o flag to set options in SSH:
for SERVER in $(cat servers.txt); do
ssh -o StrictHostKeyChecking=no -o BatchMode=yes root#$SERVER exit && echo OK $SERVER || echo ERR $SERVER
done
Check out the manpage for ssh_config: man ssh_config for all of the options available with the -o flag.

If you have quite a few servers what you may want to do is connect to all of them simultaneously. This way it only waits maximum 2 minutes (default TCP connect timeout) if any of the servers is unresponsive.
Connect to each server without allocating a terminal and execute echo . command, redirect the output into a named file. Issue these commands in a loop asynchronously. Then wait till all commands complete, iterate over the log files and check which ones have dots in it. Then report the servers whose log files do not have a dot in it.
E.g.:
#/bin/bash
servers="$#"
for server in $servers; do
ssh -o StrictHostKeyChecking=no -o PasswordAuthentication=no -Tn $server echo . >$server.log 2>$server.error.log &
done
wait # After 2 minutes all connection attempts timeout.
for server in $servers; do
[[ -s $server.log ]] || echo "Failed to connect to $server" >2
done

Related

How to check the SSH login status of routers in bash script with password prompts

I am running a task where i need to check the SSH login status on 400 remote routers. I have made scripts using expect in bash which SSH the remote routers and run some commands over it. However, there are some routers that are not responding to SSH. I am using if statement to avoid those routers which are failing on SSH. Sample code to check the status on remote router works only if we have password less entry or the private key saved. Could you please help how can I check the SSH status on the remote routers?
If I get the password prompt while doing SSH to the router, I can say that the server is able to SSH the router. There is no need to supply password to it.
#!/bin/bash
ssh -q -o BatchMode=yes -o ConnectTimeout=7 username#IP exit
echo $?
if [ $? -ne 0 ]
then
# Do stuff here if example.com SSH is down
echo "Can not connect to the device"
fi
Well,
If you are using expect package then there is timeout option there as well.
Else your shell code above is correct way of doing it except just a few corrections:
#!/bin/bash
ssh -q -o BatchMode=yes -o ConnectTimeout=7 username#IP date
ret=$?
echo $ret
if [ $ret -ne 0 ]
then
# Do stuff here if example.com SSH is down
echo "Can not connect to the device"
fi
You can see, we are assigning $? immediately to a var. If you don't, then $? will contain the return value of echo $? command which will be always 0. Hence giving you true for all ssh.
Also I suggest to run some other command rather than exit with ssh.
Hope this helps
===Edited====>>>
well since you don't have passwordless ssh enabled. You can try to telnet to port 22, if 22 port is open then it will show connected and if its not open then it won't you can grep on it.
Here is the modified code: (Provided that ssh is running on 22 port otherwise change the port in code.)
#!/bin/bash
echo "" | telnet $IP 22 | grep "Connected"
ret=$?
echo $ret
if [ $ret -ne 0 ]
then
# Do stuff here if example.com SSH is down
echo "Can not connect to the device"
fi

How to check ssh availability if check ssh OK then connect to server 1 and if ssh NOT OK then connect to server 2?

I have 2 servers.
The first server is always active and the second server is always in the stanby state.
I'm trying to write a bash script to check the status of these two servers before doing the task.
If the first host dies (or the connect SSH to server 1 not OK) then it will connect SSH to the second host for execution to task.
Any idea on how to do it?
EDIT2: since OP mentioned that there is a file named IP.list from which server names should be read then following may help you here.
while read server
do
if [[ -n $(ssh user_name#"$server" "ls -ld /tmp") ]]
then
echo "$server is up."
echo "Exiting from script now.."
exit;
else
echo "$server is NOT up, look for another server."
fi
done < "IP.txt"
EDIT: Since OP is saying ping is not an option to check server's availability then using ssh only for servers. Considering that your source server(from where you are running the script) and target server(whose status you are checking are having passwordless connection between them so that you will NOT be prompted for any password)
if [[ -n $(ssh user_name#server_name "ls -ld /tmp") ]]
then
echo "Server 1 is up."
else
echo "Server is NOT up, look for another server."
fi
You could simple do ping to SERVER1 and see if that is happening successfully in case it not it means server is not up and you could move your things to another stand up server.
if [[ $(ping -c5 SERVER1 | grep -q "5 packets transmitted, 5 received") -eq 0 ]]
then
echo "Server1 is up."
else
echo "Server is NOT up, look for another server."
fi

sshpass exit in automation

I have total of 6 IP addresses and out of the 6 only 2 IP addresses are valid. I wrote a shell script to use sshpass to test each IP.
The issue is when script reaches IP that is working it log's in the system (Cisco switch) and stays there and not continuing with the loop to test the remaining IPs. If i type "exit" on the system than it continues with the loop.
After a successful login how can script automatically get out, from logged system, and continue with testing remaining IP?
/usr/bin/sshpass -p $ADMINPASS ssh -oStrictHostKeyChecking=no -oCheckHostIP=no -t $ADMINLOGIN#$IP exit
i can use the exit status to figure out which IP worked and which on didn't work.
Test first if IP is alive, and then 'ssh' on it, could help you.I don't know if you are using a loop or not, but loop can be a good choice.Should look like : for f in ip-1 ip-2 ip-3 ip-4 ip-5 ip-6; do ping -c 1 -w 3 $f; if [ $? -eq 0 ]; then echo OK; ssh_pass $f your_command; else echo " IP is NOK"; fi; done
You can then also add 'exit' command, depending on what you test : 'exit 0' if it is OK, after you 'ssh' command, 'exit 1' if NOK.

Checking SSH failure in a script

Hi what is the best way to check to see if SSH fails for whatever reason?
Can I use a IF statement ( if it fails then do something)
I'm using the ssh command in a loop and passing my hosts names form a flat file.
so I do something like:
for i in `cat /tmp/hosts` ; do ssh $i 'hostname;sudo ethtool eth1'; done
I get sometime this error or I just cannot connect
ssh: host1 Temporary failure in name resolution
I want to skip the hosts that I cannot connect to is SSH fails. What is the best way to do this? Is there a runtime error I can trap to bypass the hosts that I cannot ssh into for whatever reason, perhaps ssh is not allowed or I do not have the right password ?
Thanking you in advance
Cheers
To check if there was a problem connecting and/or running the remote command:
if ! ssh host command
then
echo "SSH connection or remote command failed"
fi
To check if there was a problem connecting, regardless of success of the remote command (unless it happens to return status 255, which is rare):
if ssh host command; [ $? -eq 255 ]
then
echo "SSH connection failed"
fi
Applied to your example, this would be:
for i in `cat /tmp/hosts` ;
do
if ! ssh $i 'hostname;sudo ethtool eth1';
then
echo "Connection or remote command on $i failed";
fi
done
You can check the return value that ssh gives you as originally shown here:
How to create a bash script to check the SSH connection?
$ ssh -q user#downhost exit
$ echo $?
255
$ ssh -q user#uphost exit
$ echo $?
0
EDIT - I cheated and used nc
Something like this:
#!/bin/bash
ssh_port_is_open() { nc -z ${1:?hostname} 22 > /dev/null; }
for host in `cat /tmp/hosts` ; do
if ssh_port_is_open $host; then
ssh -o "BatchMode=yes" $i 'hostname; sudo ethtool eth1';
else
echo " $i Down"
fi
done

Bash script to set up a temporary SSH tunnel

On Cygwin, I want a Bash script to:
Create an SSH tunnel to a remote server.
Do some work locally that uses the tunnel.
Then shut down the tunnel.
The shutdown part has me perplexed.
Currently, I have a lame solution. In one shell I run the following to create a tunnel:
# Create the tunnel - this works! It runs forever, until the shell is quit.
ssh -nNT -L 50000:localhost:3306 jm#sampledomain.com
Then, in another shell window, I do my work:
# Do some MySQL stuff over local port 50000 (which goes to remote port 3306)
Finally, when I am done, I close the first shell window to kill the tunnel.
I'd like to do this all in one script like:
# Create tunnel
# Do work
# Kill tunnel
How do I keep track of the tunnel process, so I know which one to kill?
You can do this cleanly with an ssh 'control socket'. To talk to an already-running SSH process and get it's pid, kill it etc. Use the 'control socket' (-M for master and -S for socket) as follows:
$ ssh -M -S my-ctrl-socket -fNT -L 50000:localhost:3306 jm#sampledomain.com
$ ssh -S my-ctrl-socket -O check jm#sampledomain.com
Master running (pid=3517)
$ ssh -S my-ctrl-socket -O exit jm#sampledomain.com
Exit request sent.
Note that my-ctrl-socket will be an actual file that is created.
I got this info from a very RTFM reply on the OpenSSH mailing list.
You can tell SSH to background itself with the -f option but you won't get the PID with $!.
Also instead of having your script sleep an arbitrary amount of time before you use the tunnel, you can use -o ExitOnForwardFailure=yes with -f and SSH will wait for all remote port forwards to be successfully established before placing itself in the background. You can grep the output of ps to get the PID. For example you can use
...
ssh -Cfo ExitOnForwardFailure=yes -N -L 9999:localhost:5900 $REMOTE_HOST
PID=$(pgrep -f 'N -L 9999:')
[ "$PID" ] || exit 1
...
and be pretty sure you're getting the desired PID
You can tell ssh to go into background with & and not create a shell on the other side (just open the tunnel) with a command line flag (I see you already did this with -N).
Save the PID with PID=$!
Do your stuff
kill $PID
EDIT: Fixed $? to $! and added the &
I prefer to launch a new shell for separate tasks and I often use the following command combination:
$ sudo bash; exit
or sometimes:
$ : > sensitive-temporary-data.txt; bash; rm -f sensitive-temporary-data.txt; exit
These commands create a nested shell where I can do all my work; when I'm finished I hit CTRL-D and the parent shell cleans up and exits as well. You could easily throw bash; into your ssh tunnel script just before the kill part so that when you log out of the nested shell your tunnel will be closed:
#!/bin/bash
ssh -nNT ... &
PID=$!
bash
kill $PID
You could launch the ssh with a & a the end, to put it in the background and grab its id when doing. Then you just have to do a kill of that id when you're done.
A simple bash script to solve your problem.
# Download then put in $PATH
wget https://raw.githubusercontent.com/ijortengab/bash/master/commands/command-keep-alive.sh
mv command-keep-alive.sh -t /usr/local/bin
# open tunnel, put script in background
command-keep-alive.sh "ssh -fN -o ServerAliveInterval=10 -o ServerAliveCountMax=2 -L 33306:localhost:3306 myserver" /tmp/my.pid &
# do something
mysql --port 33306
# close tunnel
kill $(cat /tmp/my.pid)
https://github.com/aronpc/remina-ssh-tunnel
#!/usr/bin/env sh
scriptname="$(basename $0)"
actionname="$1"
tunnelname=$(echo "$2" | iconv -t ascii//TRANSLIT | sed -E 's/[^a-zA-Z0-9-]+/-/g' | sed -E 's/^-+|-+$//g' | tr A-Z a-z)
remotedata="$3"
tunnelssh="$4"
if [ $# -lt 4 ]
then
echo "Usage: $scriptname start | stop LOCAL_PORT:RDP_IP:RDP_PORT SSH_NODE_IP"
exit
fi
case "$actionname" in
start)
echo "Starting tunnel to $tunnelssh"
ssh -M -S ~/.ssh/sockets/$tunnelname.control -fnNT -L $remotedata $tunnelssh
ssh -S ~/.ssh/sockets/$tunnelname.control -O check $tunnelssh
;;
stop)
echo "Stopping tunnel to $tunnelssh"
ssh -S ~/.ssh/sockets/$tunnelname.control -O exit $tunnelssh
;;
*)
echo "Did not understand your argument, please use start|stop"
;;
esac
usage example
Edit or create new remmina server connection
schema
~/.ssh/rdp-tunnel.sh ACTION TUNNELNAME LOCAL_PORT:REMOTE_SERVER:REMOTE_PORT TUNNEL_PROXY
name
description
ACTION
start|stop
TUNNELNAME
"string identify socket" slugify to create socket file into ~/.ssh/sockets/string-identify-socket.control
LOCAL_PORT
the door that will be exposed locally if we use the same port for two connections it will crash
REMOTE_SERVER
the ip of the server that you would access if you had it on the proxy server that will be used
REMOTE_PORT
the service port that runs on the server
TUNNEL_PROXY
the connection you are going to use as a proxy, it needs to be in your ~/.ssh/config preferably using the access keys
I use the combination (% g-% p) of the remmina group name and connection name to be my TUNNELNAME (this needs to be unique, it will see the socket name)
pre-command
~/.ssh/rdp-tunnel.sh start "%g-%p" 63394:192.168.8.176:3389 tunnel-name-1
post-command
~/.ssh/rdp-tunnel.sh stop "%g-%p" 63394:192.168.8.176:3389 tunnel-name-1
you can and should use this script to access anything, I use it constantly to access systems and services that do not have a public ip going through 1,2,3,4,5 or more ssh proxies
see more into :
ssh config
ssh mach
ssh jump hosts
sshuttle python ssh
Refs:
https://remmina.org/remmina-rdp-ssh-tunnel/
https://kgibran.wordpress.com/2019/03/13/remmina-rdp-ssh-tunnel-with-pre-and-post-scripts/
Bash script to set up a temporary SSH tunnel
https://gist.github.com/oneohthree/f528c7ae1e701ad990e6

Resources