I have a loop that will connect to a server via ssh to execute a command. I want to save the output of that command.
o=$(ssh $s "$#")
This works fine. I can then do what I need with the output. However I have a lot of servers to run this against and I'm trying to speed up the process by backgrounding the ssh connection, basically to do all of the requests at once. If I wasn't saving the output I could do something like
ssh $s "$#" &
and this works fine
I haven't been able to get the correct combination to do both.
o=$(ssh $s "$#")&
This doesn't give me any output. Other combinations I've tried appear to try to execute the output. Suggestions?
Thanks!
A process going to the background gets its own copies of the file descriptors. The stdout (o=..) will not be available in the calling process. However, you can bind the stdout to a file and access the file.
ssh $s "$#" >outfile &
wait
o=$(cat outfile)
If you don't like files, you could also use named pipes. This way the 'wait' is done by the 'cat' command. The pipe can be reused and consumes no space on the disk.
mkfifo testpipe
ssh $s "$#" >testpipe &
o=$(cat testpipe)
I would just use a temporary file. You can't set a variable in a background process and access it from the shell that started it.
ssh "$s" "$#" > output.txt & ssh_pid=$!
...
wait "$ssh_pid"
o=$(<output.txt)
Related
I am reading a file through a script using the below method and storing it in myArray
while IFS=$'\t' read -r -a myArray
do
"do something"
done < file.txt
echo "ALL DONE"
Now in the "do something" area I am using some commands over ssh
ssh user#$SERVER "some command"
But the issue is after executing this for the 1st line of file.txt, the script stops reading the file further and skips to next step that is I get the output
ALL DONE
But instead of commands over ssh I use local commands the scripts run file. I am not sure why this is happening. Can someone please suggest what I need to do?
You'll have to try giving the -n flag to ssh, from the manpage:
-n Redirects stdin from /dev/null (actually, prevents reading from
stdin). This must be used when ssh is run in the background. A
common trick is to use this to run X11 programs on a remote
machine. For example, ssh -n shadows.cs.hut.fi emacs & will
start an emacs on shadows.cs.hut.fi, and the X11 connection will
be automatically forwarded over an encrypted channel. The ssh
program will be put in the background. (This does not work if
ssh needs to ask for a password or passphrase; see also the -f
option.)
I have a script I want to run on remote via ssh. It checks if there is a process running and should try to kill it, if it exists. Now, my code looks like this:
ssh my_prod_env << ENDSSH
...
pid=$(pgrep -f "node my_app.js")
echo $pid
# kill process with $pid
...
exit
ENDSSH
The problem lies here: I cannot capture output of pgrep command in variable. I tried with $(), backticks, pipe then read and maybe other approaches, but all without success.
I would like to do it all in one ssh session.
Now I am thinking the output of command goes to the output stream I cannot access in my script. I might be wrong, though.
Either way, help will be appreciated.
Ok, after you provided in comments more info what you want, I believe this is the correct answer to your question:
ssh my_prod_env -t 'pgrep -f "node my_app.js"'
This will call the command and leave you logged on the server
This is what fixes the thing - "escaping" the ENDSSH tag.
ssh my_prod_env << /ENDSSH
...
# capture output of remote commands in remote variables
...
ENDSSH
Problem was that my vars were local and I was trying to capture output of remote commands in them.
This question/answer helped me realize what is going on: How to assign local variable with a remote command result in bash script?
So, my question could be marked as duplicate or something similar, I guess.
I have a text file in which I have a list of servers. I'm trying to read the server one by one from the file, SSH in the server and execute ls to see the directory contents. My loop runs just once when I run the SSH command, however, for SCP it runs for all servers in the text file and exits, I want the loop to run till the end of text file for SSH. Following is my bash script, how can I make it run for all the servers in the text file while doing SSH?
#!/bin/bash
while read line
do
name=$line
ssh abc_def#$line "hostname; ls;"
# scp /home/zahaib/nodes/fpl_* abc_def#$line:/home/abc_def/
done < $1
I run the script as $ ./script.sh hostnames.txt
The problem with this code is that ssh starts reading data from stdin, which you intended for read line. You can tell ssh to read from something else instead, like /dev/null, to avoid eating all the other hostnames.
#!/bin/bash
while read line
do
ssh abc_def#"$line" "hostname; ls;" < /dev/null
done < "$1"
A little more direct is to use the -n flag, which tells ssh not to read from standard input.
Change your loop to a for loop:
for server in $(cat hostnames.txt); do
# do your stuff here
done
It's not parallel ssh but it works.
I open-sourced a command line tool called Overcast to make this sort of thing easier.
First you import your servers:
overcast import server.01 --ip=1.1.1.1 --ssh-key=/path/to/key
overcast import server.02 --ip=1.1.1.2 --ssh-key=/path/to/key
Once that's done you can run commands across them using wildcards, like so:
overcast run server.* hostname "ls -Al" ./scriptfile
overcast push server.* /home/zahaib/nodes/fpl_* /home/abc_def/
I have a couple of machines to update some script on. I can do this with a small bash script on my side, which consists of one while loop for reading IPs from a list and calling scp for them. It works fine, but when I try to run updated script in a loop, it break the loop, although runs quite fine itself.
#!/bin/bash
cat ip_list.txt | while read i; do
echo ${i}
scp the_script root#${i}:/usr/sbin/ # works ok
ssh root#${i} /usr/sbin/the_script # works for a first IP, then breaks
done
Is this how it suppose to work? If so, how can I run a script remotely via ssh without breaking the loop?
Use this:
ssh -n root#${i} /usr/sbin/the_script # works for a first IP, then breaks
The -n option tells ssh not to read from stdin. Otherwise, it reads stdin and passes it through to the network connection, and this consumes the rest of the input pipe.
You need to change the ssh line like this
ssh root#${i} /usr/sbin/the_script < /dev/null # works for a first IP, then breaks
I want to start a script remotely via ssh like this:
ssh user#remote.org -t 'cd my/dir && ./myscript data my#email.com'
The script does various things which work fine until it comes to a line with nohup:
nohup time ./myprog $1 >my.log && mutt -a ${1%.*}/`basename $1` -a ${1%.*}/`basename ${1%.*}`.plt $2 < my.log 2>&1 &
it is supposed to do start the program myprog, pipe its output to mylog and send an email with some datafiles created by myprog as attachment and the log as body. Though when the script reaches this line, ssh outputs:
Connection to remote.org closed.
What is the problem here?
Thanks for any help
Your command runs a pipeline of processes in the background, so the calling script will exit straight away (or very soon afterwards). This will cause ssh to close the connection. That in turn will cause a SIGHUP to be sent to any process attached to the terminal that the -t option caused to be created.
Your time ./myprog process is protected by a nohup, so it should carry on running. But your mutt isn't, and that is likely to be the issue here. I suggest you change your command line to:
nohup sh -c "time ./myprog $1 >my.log && mutt -a ${1%.*}/`basename $1` -a ${1%.*}/`basename ${1%.*}`.plt $2 < my.log 2>&1 " &
so the entire pipeline gets protected. (If that doesn't fix it it may be necessary to do something with file descriptors - for instance mutt may have other issues with the terminal not being around - or the quoting may need tweaking depending on the parameters - but give that a try for now...)
This answer may be helpful. In summary, to achieve the desired effect, you have to do the following things:
Redirect all I/O on the remote nohup'ed command
Tell your local SSH command to exit as soon as it's done starting the remote process(es).
Quoting the answer I already mentioned, in turn quoting wikipedia:
Nohuping backgrounded jobs is for example useful when logged in via SSH, since backgrounded jobs can cause the shell to hang on logout due to a race condition [2]. This problem can also be overcome by redirecting all three I/O streams:
nohup myprogram > foo.out 2> foo.err < /dev/null &
UPDATE
I've just had success with this pattern:
ssh -f user#host 'sh -c "( (nohup command-to-nohup 2>&1 >output.file </dev/null) & )"'
Managed to solve this for a use case where I need to start backgrounded scripts remotely via ssh using a technique similar to other answers here, but in a way I feel is more simple and clean (at least, it makes my code shorter and -- I believe -- better-looking), by explicitly closing all three streams using the stream-close redirection syntax (as discussed at the following locations:
https://unix.stackexchange.com/questions/131801/closing-a-file-descriptor-vs
https://unix.stackexchange.com/questions/70963/difference-between-2-2-dev-null-dev-null-and-dev-null-21
http://www.tldp.org/LDP/abs/html/io-redirection.html#CFD
https://www.gnu.org/software/bash/manual/html_node/Redirections.html
Rather than the more widely used but (IMHO) hackier "redirect to/from /dev/null", resulting in the deceptively simple:
nohup script.sh >&- 2>&- <&-&
2>&1 works just as well as 2>&-, but I feel the latter is ever-so-slightly more clear. ;) Most people might have a space preceding the final "background job" ampersand, but since it is not required (as the ampersand itself functions like a semicolon in normal usage), I prefer to omit it. :)