I have a bash script which is doing very plain sftp to transfer data to production and uat servers. See my code below.
if [ `ls -1 ${inputPath}|wc -l` -gt 0 ]; then
sh -x wipprod.sh >> ${sftpProdLog}
sh -x wipdev.sh >> ${sftpDevLog}
sh -x wipdevone.sh >> ${sftpDevoneLog}
fi
sometimes the UAT server may go down. In those cases the number of scripts hanged are getting increased. If it reaches the user max. number of process the other scripts also getting affected. So I am thinking before executing each of the above script i have to test the port 22 availability on the destination server. Then I can execute the script.
Is this the right way? If yes what is the optimistic way to do that? If no what else is the best approach to avoid unnecessary sftp connection when destination not available? Thanks in advance.
Use sftp in batch mode together with ConnectTimeout option explicitely set. Sftp will take care about up/down problems by itself.
Note, that ConnectTimeout should be slightly higher if your network is slow.
Then put sftp commands into your wip*.sh backup scripts.
If UAT host is up:
[localuser#localhost tmp]$ sftp -b - -o ConnectTimeout=1 remoteuser#this_host_is_up <<<"put testfile.xml /tmp/"; echo $?
sftp> put testfile.xml /tmp/
Uploading testfile.xml to /tmp/testfile.xml
0
File is uploaded, sftp exits with exit code 0.
If UAT host is down, sftp exits wihin 1 second with exit code 255.
[localuser#localhost tmp]$ sftp -b - -o ConnectTimeout=1 remoteuser#this_host_is_down <<<"put testfile.xml /tmp/"; echo $?
ssh: connect to host this_host_is_down port 22: Connection timed out
Couldn't read packet: Connection reset by peer
255
It sounds reasonable - if the server is inaccessible you want to immediately report an error and not try and block.
The question is - why does the SFTP command block if the server is unavailable? If the server is down, then I'd expect the port open to fail almost immediately and you need only detect that the SFTP copy has failed and abort early.
If you want to detect a closed port in bash, you can simply as bash to connect to it directly - for example:
(echo "" > /dev/tcp/remote-host/22) 2>/dev/null || echo "failed"
This will open the port and immediately close it, and report a failure if the port is closed.
On the other hand, if the server is inaccessible because the port is blocked (in a firewall, or something, that drops all packets), then it makes sense for your process to hang and the base TCP test above will also hang.
Again this is something that should probably be handled by your SFTP remote copy using a timeout parameter, as suggested in the comments, but a bash script to detect blocked port is also doable and will probably look something like this:
(
(echo "" > /dev/tcp/remote-host/22) &
pid=$!
timeout=3
while kill -0 $pid 2>/dev/null; do
sleep 1
timeout=$(( $timeout - 1 ))
[ "$timeout" -le 0 ] && kill $pid && exit 1
done
) || echo "failed"
(I'm going to ignore the ls ...|wc business, other than to say something like find and xargs --no-run-if-empty are generally more robust if you have GNU find, or possibly AIX has an equivalent.)
You can perform a runtime connectivity check, OpenSSH comes with ssh-keyscan to quickly probe an SSH server port and dump the public key(s), but sadly it doesn't provide a usable exit code, leaving parsing the output as a messy solution.
Instead you can do a basic check with a bash one-liner:
read -t 2 banner < /dev/tcp/127.0.0.1/22
where /dev/tcp/127.0.0.1/22 (or /dev/tcp/hostname/ssh) indicates the host and port to connect to.
This relies on the fact that the SSH server will return an identifying banner terminated with CRLF. Feel free to inspect $banner. If it fails after the indicated timeout read will receive SIGALARM (exit code 142), and connection refused will result in exit code 1.
(Support for /dev/tcp and network redirection is enabled by default since before bash-2.05, though it can be disabled explicitly with --disable-net-redirections or with --enable-minimal-config at build time.)
To prevent such problems, an alternative is to set a timeout: with any of the ssh, scp or sftp commands you can set a connection timeout with the option -o ConnectTimeout=15, or, implicitly via ~/.ssh/config:
Host 1.2.3.4 myserver1
ConnectionTimeout 15
The commands will return non-zero on timeout (though the three commands may not all return the same exit code on timeout). See this related question: how to make SSH command execution to timeout
Finally, if you have GNU parallel you may use its sem command to limit concurrency to prevent this kind of problem, see https://unix.stackexchange.com/questions/168978/limit-maximum-number-of-concurrent-scp-processes-running-on-a-host .
Related
I'm working on a bash script to connect to a server via SSH that is running sish (https://github.com/antoniomika/sish). This will essentially create a port forward on the internet like ngrok using only SSH. Here is what happens during normal usage.
The command:
ssh -i ./tun -o StrictHostKeyChecking=no -R 5900:localhost:5900 tun.domain.tld sleep 10
The response:
Starting SSH Forwarding service for tcp:5900. Forwarded connections can be accessed via the following methods:
TCP: tun.domain.tld:43345
Now I need to send the ssh command to the background and figure out some way of capturing the response from the server as a variable so that I can grab the port that sish has assigned and send that somewhere (probably a webhook). I've tried a few things like using -f and piping to a file or named pipe and trying to cat it, but the issue is that the piping to the file never works and although the file is created, it's always empty. Any assistance would be greatly appreciated.
If you're running a single instance of sish (and the tunnel you're attempting to define) you can actually have sish bind the specific part you want (in this case 5900).
You just set the --bind-random-ports=false flag on your server command in order to tell sish that it's okay to not use random ports.
If you don't want to do this (or you have multiple clients that will expose this same port), you can use a simple script like the following:
#!/bin/bash
ADDR=""
# Start the tunnel. Use a phony command to tell ssh to clean the output.
exec 3< <(ssh -R 5900:localhost:5900 tun.domain.tld foobar 2>&1 | grep --line-buffered TCP | awk '{print $2; system("")}')
# Get our buffered output that is now just the address sish has given to us.
for i in 1; do
read <&3 line;
ADDR="$line"
done
# Here is where you'd call the webhook
echo "Do something with $ADDR"
# If you want the ssh command to continue to run in the background
# you can omit the following. This is to wait for the ssh command to
# exit or until this script dies in order to kill the ssh command.
PIDS=($(pgrep -P $(pgrep -P $$)))
function killssh() {
kill ${PIDS[0]}
}
trap killssh EXIT
while kill -0 ${PIDS[0]} 2> /dev/null; do sleep 1; done;
sish also has an admin api which you can scrape. The information on that is available here.
References: I build and maintain sish and use it myself (as well as a similar type of script).
I need to print a count of a remote server which is written on '/REMOTE_DIR/DR_count'. But that remote server is not much reliable due to a network or OS failure. However I need to print the DR_count value from the local machine if that remote machine is not available. Here is my logic. Please correct me to how to check that if condition in correct way. I'm running this script on Solaris 11.3.
#!/bin/sh
if[check wether ssh user#host_name is possible]
then
op="cat /REMOTE_DIR/DR_count"
cmd="ssh user#host_name $op"
drlog=`$cmd`
else
drlog='cat /LOCAL_DIR/DR_count'
fi
echo $drlog
As I said in my comment, I would simply try to ssh, and if use its exit code to see whether it worked:
ssh -o ConnectTimeout=5 user#host_name cat /REMOTE_DIR/DR_count 2>/dev/null || cat /LOCAL_DIR/DR_count
you should check for exit code 255 to detect whether you have network error or not
#!/bin/bash
#EXIT STATUS
# ssh exits with the exit status of the remote command or with 255 if an error occurred.
cnt=`ssh -o ConnectTimeout=5 root#$host "cat /REMOTE_DIR/DR_count"`
exit_code=$?
if [ $exit_code -eq 255 ]; then
cnt=`cat /LOCAL_DIR/DR_count`
fi
it makes sense also to check for other (non 0 / 255) exit codes to check possible issues at remote side (like file missing on remote side)
so im trying to make an if statement to tell me if an sftp connection was sucessfull or failed, and if its a sucess i want to run a piece of code that automates an sftp download that ive already made.
My problem is that this if statement executes this sftp connection, and then prompts me for a password and stalls the rest of the code.
i wanted to do something like this
if ( sftp -oPort=23 user#server )
then
expect <<-EOF
spawn sftp -oPort=23 user#server
.....
I want to know if its possible for me to make the if statement not execute the sftp connection and then not prompt me , maybe execute it on the background or something.
I would appreciate if someone could tell me if what im asking is possible, or propose a better solution to what im trying to do, thanks
You cannot not-execute a command and then react on the return value of the executed command (because this is what you really want to do: check if you can run sftp successful, and if so do a "proper" run; but you'll never know whether it can run successfull without running it).
So the main question is, what it is what you actually want to test.
If you want to test whether you can do a full sftp connection (with all the handshaking and what not), you could try running sftp in batch-mode (which is handily non-interactive).
E.g. the following runs an sftp session, only to terminate it immediately with a bye command:
if echo bye | sftp -b - -oPort=23 user#server ; then
echo "sftp succeeded"
fi
This will only succeed if the entire sftp session works (that is: you pass any key checks; you can authenticate, ...).
If the server asks you for a password, it will fail to authenticate (being non-interactive), and you won't enter the then body.
If you only want to check whether something is listening on port 23, you can use netcat for this:
if netcat -z server 23; then
echo "port:32 is open"
fi
This will succeed whenever it can successfully bind to port 23 on the server. It doesn't care whether there's an sftp daemon running, or (more likely) a telnet daemon.
You could also do some minimal test whether the remote server looks like an SSH/SFTP server: ssh servers usually greet you with a string indicating that they indeed speak ssh: something like "SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.4".
With this information you can then run:
if echo QUIT | netcat server 23 | grep SSH; then
echo "found an ssh server"
fi
I have a BASH script that logs in to multiple servers and runs a series of commands. Occasionally I'll have instances where a server is unavailable (regular maintenance, etc). How can I exit out of the SSH session cleanly without passing an error like this?:
bash-3.2$ ssh myserver3
Disconnecting: Bad packet length.
You may be looking for -q option:
ssh -q user#host
And you can check the return code $? afterwards.
From man bash:
-q
Quiet mode. Causes all warning and diagnostic messages to be
suppressed
You can find more info about this topic in How to create a bash script to check the SSH connection?.
How can I run telnet inside a shell script and execute commands on the remote server?
I do not have expect installed on my solaris machine because of security reasons.
I also do not have the perl net::telnet module installed.
So with out using expect and perl how can I do it?
I tried the below thing but its not working.
#!/usr/bin/sh
telnet 172.16.69.116 <<!
user
password
ls
exit
!
When I execute it, this is what I am getting:
> cat tel.sh
telnet 172.16.69.116 <<EOF
xxxxxx
xxxxxxxxx
ls
exit
EOF
> tel.sh
Trying 172.16.69.116...
Connected to 172.16.69.116.
Escape character is '^]'.
Connection to 172.16.69.116 closed by foreign host.
>
Some of your commands might be discarded. You can achieve finer control with ordinary script constructs and then send required commands through a pipe with echo. Group the list of commands to make one "session":-
{
sleep 5
echo user
sleep 3
echo password
sleep 3
echo ls
sleep 5
echo exit
} | telnet 172.16.65.209
I had the same issue...however, at least in my environment it turned out being the SSL Certificate on the destination server was corrupted in some way and the server team took care of the issue.
Now, what I'm trying to do is to figure out how to get a script to run the exact same thing you're doing above except I want it to dump out the exact same scenario above into a file and then when it encounters a server in which it actually connects, I want it to provide the escape character (^]) and go on to the next server.