so im trying to make an if statement to tell me if an sftp connection was sucessfull or failed, and if its a sucess i want to run a piece of code that automates an sftp download that ive already made.
My problem is that this if statement executes this sftp connection, and then prompts me for a password and stalls the rest of the code.
i wanted to do something like this
if ( sftp -oPort=23 user#server )
then
expect <<-EOF
spawn sftp -oPort=23 user#server
.....
I want to know if its possible for me to make the if statement not execute the sftp connection and then not prompt me , maybe execute it on the background or something.
I would appreciate if someone could tell me if what im asking is possible, or propose a better solution to what im trying to do, thanks
You cannot not-execute a command and then react on the return value of the executed command (because this is what you really want to do: check if you can run sftp successful, and if so do a "proper" run; but you'll never know whether it can run successfull without running it).
So the main question is, what it is what you actually want to test.
If you want to test whether you can do a full sftp connection (with all the handshaking and what not), you could try running sftp in batch-mode (which is handily non-interactive).
E.g. the following runs an sftp session, only to terminate it immediately with a bye command:
if echo bye | sftp -b - -oPort=23 user#server ; then
echo "sftp succeeded"
fi
This will only succeed if the entire sftp session works (that is: you pass any key checks; you can authenticate, ...).
If the server asks you for a password, it will fail to authenticate (being non-interactive), and you won't enter the then body.
If you only want to check whether something is listening on port 23, you can use netcat for this:
if netcat -z server 23; then
echo "port:32 is open"
fi
This will succeed whenever it can successfully bind to port 23 on the server. It doesn't care whether there's an sftp daemon running, or (more likely) a telnet daemon.
You could also do some minimal test whether the remote server looks like an SSH/SFTP server: ssh servers usually greet you with a string indicating that they indeed speak ssh: something like "SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.4".
With this information you can then run:
if echo QUIT | netcat server 23 | grep SSH; then
echo "found an ssh server"
fi
Related
for my work I need to connect to a lot of different servers every day: telnet ti the host, enter username, enter password - commence work.
Now I wanted to make life easier by automatically entering the username - I managed to do that, but telnet quits afterwards, that's obviously not what I wanted.
I work from a system with BASH and I can't install any programs there, so please don't give answers like "Use expect, that solves your problem easily..."
My tries led me to this:
function tn() { (echo "user"
sleep 1) | telnet $1 23
}
Calling the function with tn 123.45.67.89 connects to the server at 123.45.67.89, where the username is asked, which is entered automatically - great!
But then the password is asked, and instead of letting me enter it and begin my work, the connection is closed.
I really hope someone knows a solution for this!
Thanks in advance!
You might want to look at the expect command to script interactions with telnet:
#!/usr/bin/env bash
function tn() {
expect -f <<EOF
spawn telnet $1
expect "login"
send "${2}\r"
interact
EOF
}
telnetis designed for interactive usage. Use netcat, ncat, nc, socat or any other tool of this family.
Example:
( echo "user"; sleep 1) | ncat $1 23
But if you want to simulate interactive behavior, use socat and redirect stdin+stdout to a script:
Example:
socat TCP:$1:23 EXEC:my-shell.sh
In this case, a TCP connection for address $1 port 23 is established and stdin+stdout are redirected to stdout+stdin of the script. See man socat for details and more options.
my-shell.sh look for example like:
#!/bin/sh
read line
do_domething "$line"
printf "reply\n"
read line
do_domething "$line"
printf "reply\n"
btw, I have tested nothing (just written down)
I have a bash script which is doing very plain sftp to transfer data to production and uat servers. See my code below.
if [ `ls -1 ${inputPath}|wc -l` -gt 0 ]; then
sh -x wipprod.sh >> ${sftpProdLog}
sh -x wipdev.sh >> ${sftpDevLog}
sh -x wipdevone.sh >> ${sftpDevoneLog}
fi
sometimes the UAT server may go down. In those cases the number of scripts hanged are getting increased. If it reaches the user max. number of process the other scripts also getting affected. So I am thinking before executing each of the above script i have to test the port 22 availability on the destination server. Then I can execute the script.
Is this the right way? If yes what is the optimistic way to do that? If no what else is the best approach to avoid unnecessary sftp connection when destination not available? Thanks in advance.
Use sftp in batch mode together with ConnectTimeout option explicitely set. Sftp will take care about up/down problems by itself.
Note, that ConnectTimeout should be slightly higher if your network is slow.
Then put sftp commands into your wip*.sh backup scripts.
If UAT host is up:
[localuser#localhost tmp]$ sftp -b - -o ConnectTimeout=1 remoteuser#this_host_is_up <<<"put testfile.xml /tmp/"; echo $?
sftp> put testfile.xml /tmp/
Uploading testfile.xml to /tmp/testfile.xml
0
File is uploaded, sftp exits with exit code 0.
If UAT host is down, sftp exits wihin 1 second with exit code 255.
[localuser#localhost tmp]$ sftp -b - -o ConnectTimeout=1 remoteuser#this_host_is_down <<<"put testfile.xml /tmp/"; echo $?
ssh: connect to host this_host_is_down port 22: Connection timed out
Couldn't read packet: Connection reset by peer
255
It sounds reasonable - if the server is inaccessible you want to immediately report an error and not try and block.
The question is - why does the SFTP command block if the server is unavailable? If the server is down, then I'd expect the port open to fail almost immediately and you need only detect that the SFTP copy has failed and abort early.
If you want to detect a closed port in bash, you can simply as bash to connect to it directly - for example:
(echo "" > /dev/tcp/remote-host/22) 2>/dev/null || echo "failed"
This will open the port and immediately close it, and report a failure if the port is closed.
On the other hand, if the server is inaccessible because the port is blocked (in a firewall, or something, that drops all packets), then it makes sense for your process to hang and the base TCP test above will also hang.
Again this is something that should probably be handled by your SFTP remote copy using a timeout parameter, as suggested in the comments, but a bash script to detect blocked port is also doable and will probably look something like this:
(
(echo "" > /dev/tcp/remote-host/22) &
pid=$!
timeout=3
while kill -0 $pid 2>/dev/null; do
sleep 1
timeout=$(( $timeout - 1 ))
[ "$timeout" -le 0 ] && kill $pid && exit 1
done
) || echo "failed"
(I'm going to ignore the ls ...|wc business, other than to say something like find and xargs --no-run-if-empty are generally more robust if you have GNU find, or possibly AIX has an equivalent.)
You can perform a runtime connectivity check, OpenSSH comes with ssh-keyscan to quickly probe an SSH server port and dump the public key(s), but sadly it doesn't provide a usable exit code, leaving parsing the output as a messy solution.
Instead you can do a basic check with a bash one-liner:
read -t 2 banner < /dev/tcp/127.0.0.1/22
where /dev/tcp/127.0.0.1/22 (or /dev/tcp/hostname/ssh) indicates the host and port to connect to.
This relies on the fact that the SSH server will return an identifying banner terminated with CRLF. Feel free to inspect $banner. If it fails after the indicated timeout read will receive SIGALARM (exit code 142), and connection refused will result in exit code 1.
(Support for /dev/tcp and network redirection is enabled by default since before bash-2.05, though it can be disabled explicitly with --disable-net-redirections or with --enable-minimal-config at build time.)
To prevent such problems, an alternative is to set a timeout: with any of the ssh, scp or sftp commands you can set a connection timeout with the option -o ConnectTimeout=15, or, implicitly via ~/.ssh/config:
Host 1.2.3.4 myserver1
ConnectionTimeout 15
The commands will return non-zero on timeout (though the three commands may not all return the same exit code on timeout). See this related question: how to make SSH command execution to timeout
Finally, if you have GNU parallel you may use its sem command to limit concurrency to prevent this kind of problem, see https://unix.stackexchange.com/questions/168978/limit-maximum-number-of-concurrent-scp-processes-running-on-a-host .
I could really use some help. I'm still pretty new with expect. I need to launch a scp command directly after I run sftp.
I got the first portion of this script working, my main concern is the bottom portion. I really need to launch a command after this command completes. I'd rather be able to spawn another command than, hack something up like piping this with a sleep command and running it after 10 s or something weird.
Any suggestions are greatly appreciated!
spawn sftp user#host
expect "password: "
send "123\r"
expect "$ "
sleep 2
send "cd mydir\r"
expect "$ "
sleep 2
send "get somefile\r"
expect "$ "
sleep 2
send "bye\r"
expect "$ "
sleep 2
spawn scp somefile user2#host2:/home/user2/
sleep 2
So i figured out I can actually get this to launch the subprocess if I use "exec" instead of spawn.. in other words:
exec scp somefile user2#host2:/home/user2/
the only problem? It prompts me for a password! This shouldn't happen, I already have the ssh-keys installed on both systems. (In other words, if I run the scp command from the host I'm running this expect script on, it will run without prompting me for a password). The system I'm trying to scp to, must be recognizing this newly spawned process as a new host, because its not picking up my ssh-key. Any ideas?
BTW, I apologize I haven't actually posted a "working" script, I can't really do that without comprimising the security of this server. I hope that doesn't detract from anyones ability to assist me.
I think the problem lies with me not terminating the initially spawned process. I don't understand expect enough to do it properly. If I try "close" or "eof", it simply kills the entire script, which I don't want to do just yet (because I still need to scp the file to the second host).
Ensure that your SSH private key is loaded into an agent, and that the environment variables pointing to that agent are active in the session where you're calling scp.
[[ $SSH_AUTH_SOCK ]] || { # if no agent already running...
eval "$(ssh-agent -s)" # ...then start one...
ssh-add /path/to/your/ssh/key # ...load your key...
started_ssh_agent=1 # and flag that we started it ourselves
}
# ...put your script here...
[[ $started_ssh_agent ]] && { # if we started the agent ourselves...
eval "$(ssh-agent -s -k)" # ...then clean up nicely when done.
}
As an aside, I'd strongly suggest replacing the code given in the question with something like the following:
lftp -u user,123 -e 'get /mydir/somefile -o localfile' sftp://host </dev/null
lftp scp://user2#host2 -e 'put localfile -o /home/user2/somefile' </dev/null
Each connection handled in one line, and no silliness messing around with expect.
Using the following shell code for remote telnet request:
{
sleep 5
echo admin
sleep 3
echo pass
sleep 3
echo ls
sleep 5
echo exit
} | telnet 172.16.1.1
I want to check if telnet connection was successful or not. Trying to use $?:
echo $?
But it always returns "1", even if telnet connection was OK.
Telnet is exceptionally difficult to script in this way, there is a high degree of asynchronicity with the time it takes to establish a connection and for your intended actions to complete. expect was created for exactly this kind of purpose. You launch a program, like telnet, then declare a series of expectations - eg, when 'username: ' is emitted from the program, and an action to trigger (eg: typing in the username).
There are also libraries or wrappers for expect in many languages:
python expect
ruby expect
perl expect
Here is an example that drives telnet to make an HTTP HEAD request:
set timeout 20
spawn telnet localhost 80
expect "Connected to "
send "HEAD / HTTP/1.0\r\n\r\n"
expect "HTTP 200 OK"
Given all of this, I feel I must point out that telnet is considered insecure. Ssh is a much better choice and supports better choices for authentication (eg: public/private key auth), restrictions for commands that can be run (via .ssh/authorized_keys). With ssh, and ssh-keys set up, your script reduces to a single shell command:
ssh user#hostname ls
ssh has great support for safe, secure remote command execution.
If I'm remembering correctly, this expect script does what you're doing above.
#!/usr/bin/expect
spawn telnet 172.16.1.1
expect username:
send admin
expect password:
send pass
expect "\$ "
send ls
expect "\$ "
send exit
Here's a useful link for getting started: http://oreilly.com/catalog/expect/chapter/ch03.html
How can I run telnet inside a shell script and execute commands on the remote server?
I do not have expect installed on my solaris machine because of security reasons.
I also do not have the perl net::telnet module installed.
So with out using expect and perl how can I do it?
I tried the below thing but its not working.
#!/usr/bin/sh
telnet 172.16.69.116 <<!
user
password
ls
exit
!
When I execute it, this is what I am getting:
> cat tel.sh
telnet 172.16.69.116 <<EOF
xxxxxx
xxxxxxxxx
ls
exit
EOF
> tel.sh
Trying 172.16.69.116...
Connected to 172.16.69.116.
Escape character is '^]'.
Connection to 172.16.69.116 closed by foreign host.
>
Some of your commands might be discarded. You can achieve finer control with ordinary script constructs and then send required commands through a pipe with echo. Group the list of commands to make one "session":-
{
sleep 5
echo user
sleep 3
echo password
sleep 3
echo ls
sleep 5
echo exit
} | telnet 172.16.65.209
I had the same issue...however, at least in my environment it turned out being the SSL Certificate on the destination server was corrupted in some way and the server team took care of the issue.
Now, what I'm trying to do is to figure out how to get a script to run the exact same thing you're doing above except I want it to dump out the exact same scenario above into a file and then when it encounters a server in which it actually connects, I want it to provide the escape character (^]) and go on to the next server.