I have a device with intermittent connectivity that "calls home" to open a reverse tunnel to allow me to SSH into it. This works very reliably started by systemd and automatically restarted on any exit:
ssh -R 1234:localhost:22 -N tunnel-user#reliable.host
Now however I want to run a script on the reliable-host on connect. This is easy enough with a simple change to the ssh ... cli: swap -N for a script name on the remote reliable-host:
ssh -R 1234:localhost:22 tunnel-user#reliable.host ./on-connect.sh
The problem is that once the script exits, it closes the tunnels if they're not in use.
One workaround I've found is to put a long sleep at the end of my script. This however leaves sleep processes around after the connection drops since sleep doesn't respond to SIGHUP. I could put a shorter sleep in an infinite loop (I think) but that feels hacky.
~/on-connect.sh
#!/bin/bash
# Do stuff...
sleep infinity
How can I get ssh to behave like -N has been used so that it stays connected with no activity but also runs a script on initial connection? Ideally without needing to have a special sleep (or equivalent) in the remote script but, if not possible, proper cleanup on the reliable-host when the connection drops.
I am creating a script to kickstart several servers. I am nearly finished, however I want the bash script to wait until the server kickstart is done.
When the kickstart is done and the server is rebooted a file will be created on the remote kickstarted server which is located under "/root/" and is called "kickstart-DONE"
Is it possible to make the bash script wait until it sees this file and then post something like "Done!"...?
I tried searching the forums and internet, but probably I am searching incorrectly, as I was unable to find something relevant to this issue. Heck, I don't even know if this is possible at all.
So in short; I run my script which kickstarts a server. After the kickstart is done it will create a file on the remote (- kickstarted) server called: kickstart-DONE. This would be an indication for the script that the kickstart is fully done and the server can be used. How do I make the script aware of this?
I hope someone understands what I mean and trying to achieve....
Thanks in advance.
//EDIT
SOLVED! Thanks to Cole Tierney!
Cole Tierney gave some good answers, however though it works it does not wait until the server is kickstarted. I ran the script to kickstart a server and in the end it was running the provided command:
ssh root#$HWNODEIP "while ! test -e /root/kickstart-DONE; do sleep 3; done; echo KICKSTART IS DONE...\!"
However since the kickstart can take some time (depending on server speed and such; ranging from 15 minutes to 1 hour). The command timed out:
ssh: connect to host 100.125.150.175 port 22: Connection timed out
Is there a way that the script does not time out at all and keeps it alive until the server gets back or until it takes more than 1 hour or so?
Maybe there is also a way to make it show that the script is still active? Like "Waiting... 5 minutes passed." "Waiting... 10 minutes passed." etc.
So it gives the current user some information that it not died?
You could call sleep until the file exists:
while ! test -e /root/kickstart-DONE; do sleep 3; done; echo kickstart done
Or sleep until the server is accepting ssh connections. Run the following netcat command locally to check when port 22 is open on the server (remove echo closed; if you don't want the extra feedback):
while ! nc -zw2 $HWNODEIP 22; do echo closed; sleep 3; done
On a side note, it's useful to setup a host entry in ~/.ssh/config. You can add all sorts of ssh options here without making your ssh command unwieldy. Options that are common to all host entries can be added outside of the host entries. See man ssh_config for other options. Here's an example (server1 can be anything, replace <server ip> with the server's ip address):
Host server1
Hostname <server ip>
User root
Then to use it:
ssh server1 'some command'
Note that many systems will not allow ssh connections from root for security reasons. You may want to consider adding another user for kickstart stuff. Add this user to sudoers if root access is needed.
I am using guestcontrol with Virtual Box with a Windows host and a Linux (RHEL7) guest. I want to do some config from the host to the guest by running a shell script on the guest (from a .bat on the host). This is fine and the script runs, however, it hangs when I call the reboot (I believe it is because nothing is returned). So when the following .sh is called:
#!/bin/bash
echo "here"
exit
The .bat file shows "here" and then exits (or if I use pause gives the correct message). However, when I add the reboot, the .bat never processes anything past where it calls the script. I think this would be because the guest never tells the host that the script is complete.
I have tried things like:
#!/bin/bash
{ sleep 1; reboot; } >/dev/null &
exit
or even:
#!/bin/bash
do_reboot(){
sleep 1
reboot
}
do_reboot() &
exit
but the .bat never gets past the line where it runs the .sh
How can I tell the host that the .sh script (on the guest) is complete so it can continue with the .bat script?
We need to make sure there are no sub processes running, so we want to do a no heads up using the nohup command. So the script simply becomes this:
#!/bin/bash
nohup reboot &> /tmp/nohup.out </dev/null &
exit
The stdin and stdout were causing the issues, so this just sends them into the void so that the script will not be waiting for any input from any other processes.
If you have any issues with this script, you could do something like:
#!/bin/bash
nohup /path/to/reboot_delay.sh &> /tmp/nohup.out </dev/null &
exit
And then in /path/to/reboot_delay.sh you would have:
#!/bin/bash
sleep 10 # or however many seconds you need to wait for something to happen
reboot
This way you could even allow some time for something to finish etc, yet the host machine (or ssh or wherever you are calling this from) would still know the script had finished and do what it needs to do.
I hope this can help people in future.
I'm writing a script which has several sets of commands that it needs to run on a remote server, with processing of results in between. Currently this is achieved by running ssh for each set of commands, however this requires a new connection to be made and authenticated each time, which is slow.
I recently read about the ControlMaster option in SSH, which seems like exactly what I need, namely the ability to run separate SSH sessions through a single SSH connection.
However, what I'm extremely unclear on is how exactly I would achieve this in my shell script. For example, I was thinking of constructing it like so:
#!/bin/sh
HOST="$1"
# Make sure we clean up after ourselves
on_complete() {
kill $ssh_control_master_id
rm -r "$tmp_dir"
}
trap 'on_complete 2> /dev/null' SIGINT SIGHUP SIGTERM EXIT
tmp_dir=$(mktemp -d "/tmp/$(basename "$0").XXXXXX")
ssh_control_socket="$tmp_dir/ssh_control_socket"
# Setup control master
ssh -o 'ControlMaster=yes' -S "$ssh_control_socket" "$HOST" &
ssh_control_master_id=$!
# Run initial commands
data=$(ssh -S "$ssh_control_socket" "$HOST" 'echo "Foo"')
# Process the data
echo "$data"
# Run some more commands
data=$(ssh -S "$ssh_control_socket" "$HOST" 'echo "Bar"')
# Process the second batch of data
echo "$data"
Just a simple example to give you an idea, but this doesn't seem to be the correct way to do this, as running it will either cause the second ssh command to hang, or each command will just run normally (create their own connection). I'm also not sure how to go about waiting for the master connection to be established, i.e - I'm probably running my actual commands while the remote connection is still being established.
Also on a related note, what is the correct way to close the control master once it's running, is killing it and/or deleting its socket fine?
Your code looks fine. I haven't tested it, but the first process that tries to use the master connection should probably block until the master connection has actually successfully been established. You can use the -N option to avoid running a spurious shell on the master connection:
ssh -N -o 'ControlMaster=yes' -S "$ssh_control_socket" "$HOST" &
It's perfectly fine to simply kill the ssh process once all the subordinate sessions have completed.
I'm trying to write a shell script that automates certain startup tasks based on my location (home/campusA/campusB). I go to University and take classes in two different campuses (hence campusA/campusB). My location is determined by which wireless network I'm connected to. For the purposes of this script, we can assume that I will be connected to one of these networks when the script is called and my script knows which one I'm connected to based on a call to iwconfig.
This is what I want it to do:
cat file1 > file2 # always do this, regardless of where I am
if Im at home:
start tweetdeck, thunderbird, skype
else if Im at campusA:
activate the login script # I need to login on a webform before I get internet access.
# I have written a script to automate this.
# Wait for this script to finish before doing anything else
myProg2 & # I want myProg2 running in the background until I shutdown my computer.
else if Im at campusB:
ssh username#domain # this is the problematic line
myProg2 & # I want myProg2 running in the background until I shutdown my computer.
start tweetdeck, thunderbird
close the terminal with the "exit" command
The problem is that campusB's wireless network is behind a firewall, which grants me internet access ONLY after I successfully ssh by username#domain. After a successful ssh, I need to keep the terminal window active in order to hold keep the internet access. If I close the terminal window, I lose internet access (this is bad).
When I try doing just ssh username#domain, the script stops because I don't exit the ssh command. I can't ^C out of it, which means that the rest of the script is never executed. I also have the same problem if I just close the terminal window in an attempt to kill the ssh session.
Some googling brought me to subshell, which I'm either using wrong or can't use to solve my problem. So how should I go about solving this problem? I'd appreciate any help - I've been at this for a while now and am unable to find anything helpful. If it makes a difference, I'd rather not store my ssh password in the script
Further, ampersanding the ssh call (ssh username#domain &) doesn't seem to do any good (can anyone explain why?)
Thank you in advance
EDIT
I must clarify, that the ssh connection has to be active in order for me to have internet access. Thus, when I close the terminal window, I need the ssh connection to still be active.
I had a script that looped on 6 servers, calling via ssh in the background. In 1 part of the script, there was a mis-behaving vendor application; the application didn't 'let go' of the connection properly. (other parts of the script using ssh in background worked fine).
I found that using ssh -t -t cured the problem. Maybe this can help you too.
(a teammate found this on the web, and we had spent so much time, I never went back to read the article that suggested this. The man page on our system gave no hint that such a thing was possible)
Hope this helps.
You may want to try to double background myProg2 to detach it from the tty:
# cf. "Wizard Boot Camp, Part Six: Daemons & Subshells",
# http://www.linux-mag.com/id/5981
(myProg2 &) &
Another option may be to use the daemon tool from the libslack package:
http://ingvar.blog.linpro.no/2009/05/18/todays-sysadmin-tip-using-libslack-daemon-to-daemonize-a-script/
Having a ssh with pseudy tty on background shell
In addition to #shellter's answer, I would like make some precision:
where #shelter said:
The man page on our system gave no hint that such a thing was possible
On my system (Debian 7 GNU/Linux), if I hit:
man -Pcol\ -b ssh| grep -A3 '^ *-t '
I could read:
-t Force pseudo-tty allocation. This can be used to execute arbi‐
trary screen-based programs on a remote machine, which can be
very useful, e.g. when implementing menu services. Multiple -t
options force tty allocation, even if ssh has no local tty.
Yes: Multiple -t options force tty allocation, even if ssh has no local tty.
This mean: If you remotely run a tool that require access to pseudo terminal ( pty like /dev/pts/0), you could run them by using -t switch.
But this would work only if ssh is run from a shell console (aka having his own pty). If you plan to run them is shell session without console, like background scripts, you may use Multiple -t to enforce pseudo tty allocation from ssh.
Multiple ssh shell on one ssh connection
In addition to answers from #tommy and #geekosaur, I would make some precision:
#tommy point to a very intersting feature of ssh. Not sure this have a lot to do with answer, but speaking around long time connection, this feature has to be clearly understood.
Once a connection is established, ssh could (and know how to) use them to drive a lot of thing in this one connection:
-L let you drive remote TCP connections to local machines/network. (full syntax is: -L localip:localport:distip:distport) where localip could be specified to permit other hosts from same local domain to access same tcp bind, and distip could by any host from distant network ( not only localhost ) sample: -L192.168.1.31:8443:google.com:443 permit any host from local domain to reach google through your host: http://192.168.1.31:8443
-R Same remarks in reverse way!
-M Tell ssh to open a local unix socket for bindind next ssh consoles. Simply open two terminal window. First in both window, hit: ssh somewhere than hit netstat -tan | grep :22 or netstat -tan | grep 192.168.1.31:22 (assuming 192.168.1.31 is your onw host's ip)
Than compare close all your ssh session and in first terminal, hit: ssh -M somewhere and in second, simply ssh somewhere. you may see in second terminal:
$ ssh somewhere
+ ssh somewhere
Last login: Mon Feb 3 08:58:01 2014 from elsewhere
If now you hit netstat -tan | grep 192.168.1.31:22 (on any of two oppened ssh session;) you must see that there is only one tcp connection.
This kind of features could be used in combination with -L and maybe some sleep 86399...
To work around a tcp killer router that close every inactive TCP connection from more than 120 seconds, I run:
ssh -M somewhere 'while :;do uptime;sleep 60;done'
This ensure connection stay up even if I dont hit a key for more than two minutes.
Here's a few thoughts that might help.
Sub-shells
Sub-shells fork new processes, but don't return control to the calling shell. If you want to fork a sub-shell to do the work for you, then you'll need to append a & to the line.
(ssh username#domain) &
But this doesn't look like a compelling reason to use a sub-shell. If you had a number commands you wanted to execute in order from each other, yet in parallel from the calling shell, then maybe it would be worth it. For example...
(dothis.sh; thenthis.sh; andthislastthingtoo.sh) &
Forking
I'm not sure why & isn't working for you, but it may be worth looking into nohup as well. This makes the command "immune" to hang up signals.
nohup ssh username#domain (try with and without the & at the end)
Passwords
Not storing passwords in the script is essential for any ssh automation. You can accomplish that using public key cryptography which is an inherent feature of ssh. I wont go into the details here because there are a number of great resources all across the interwebs on setting this up. I strongly suggest investigating this further.
HOWTO: set up ssh keys - Paul Keck, 2001
SSH Keys - archlinux.org
SSH with authentication key instead of password - Debian Administration
Secure Shell - Wikipedia, the free encyclopedia
If you do go this route, I also suggest running ssh in "batch mode" which will disable password querying and will automatically disconnect from the server if it becomes unresponsive after 5 minutes.
ssh -o 'BatchMode=yes' username#domain
Persistence
Then if you want to persist the connection, run some silly loop in bash! :)
ssh -o 'BatchMode=yes' username#domain "while (( 1 == 1 )); do sleep 60; done"
The problem with & is that ssh loses access to its standard input (the terminal), so when it goes to read something to send to the other side it either gets an error and exits, or is killed by the system with SIGTTIN which will implicitly suspend it. The -n and -f options are used to deal with this: -n tells it not to use standard input, -f tells it to set up any necessary tunnels etc., then close the terminal stream.
So the best way to do this is probably to do
ssh -L 9999:localhost:9999 -f host & # for some random unused port
and then manually kill the ssh before logout. Alternately,
ssh -L 9999:localhost:9999 -n host 'while :; do sleep 86400; done' </dev/null &
(The redirection is to make sure the SIGTTIN doesn't happen anyway.)
While you're at it, you may want to save the process ID and shut it down from your .logout/.bash_logout:
ssh -L 9999:localhost:9999 -n host 'while :; do sleep 86400; done' < /dev/null & echo $! >~.ssh_pid; chmod 0600 ~/.ssh_pid
and in .bash_logout:
if test -f ~/.ssh_pid; then
set -- $(sed -n 's/^\([0-9][0-9]*\)$/\1/p' ~/.ssh_pid)
if [ $# = 1 ]; then
kill $1 >/dev/null 2>&1
fi
rm ~/.ssh_pid
fi
The extra code there attempts to avoid someone sabotaging your ~/.ssh_pid, because I'm a professional paranoid.
(Code untested and may have typoes)
It's been a while since I've used ssh, and I can't test it right now, but have you tried the -f switch?
ssh -f username#domain
The man page says it backgrounds ssh. Not sure why & wouldn't work, but I guess it's interpreting it as a command to be run on the remote machine.
Maybe screen + ssh would fit the bill as well?
Something like:
screen -d -m -S sessionName cmd
screen -d -m -S sessionName cmd &
# reconnect with
screen -r sessionName