I am trying to execute an interactive shell program on a remote host from another ruby program. For the sake of simplicity let's suppose that the program I want to execute is something like this:
puts "Give me a number:"
number = gets.chomp()
puts "You gave me #{number}"
The approach that most successful has been so far is using the one I got from here. It is this one:
require 'open3'
Open3.popen3("ssh -tt root#remote 'ruby numbers.rb'") do |stdin, stdout, stderr|
# stdin = input stream
# stdout = output stream
# stderr = stderr stream
threads = []
threads << Thread.new(stderr) do |terr|
while (line = terr.gets)
puts "stderr: #{line}"
end
end
threads << Thread.new(stdout) do |terr|
while (line = terr.gets)
puts "stdout: #{line}"
end
end
sleep(2)
puts "Give me an answer: "
answer = gets.chomp()
stdin.puts answer
threads.each{|t| t.join()} #in order to cleanup when you're done.
end
The problem is that this is not "interactive" enough to me, and the program that I would like to execute (not the simple numbers.rb) has a lot more of input / output. You can think of it as an apt-get install that will ask you for some input to solve some problems.
I have read about net::ssh and pty, but couldn't see if they were going to be the (easy/elegant) solution I am looking for.
The ideal solution will be to make it in such a way that the user does not realize that the IO is being done on a remote host: the stdin goes to the remote host stdin, the stdout from the remote host comes to me and I show it.
If you have any ideas I could try I will be happy to hear them. Thank you!
Try this:
require "readline"
require 'open3'
Open3.popen3("ssh -tt root#remote 'ruby numbers.rb'") do |i, o, e, th|
Thread.new {
while !i.closed? do
input =Readline.readline("", true).strip
i.puts input
end
}
t_err = Thread.new {
while !e.eof? do
putc e.readchar
end
}
t_out = Thread.new {
while !o.eof? do
putc o.readchar
end
}
Process::waitpid(th.pid) rescue nil
# "rescue nil" is there in case process already ended.
t_err.join
t_out.join
end
I got it working, but don't ask me why it works. It was mainly trial/error.
Alternatives:
Using Net::SSH, you need to use :on_process and a Thread: ruby net/ssh channel dies? Don't forget to add session.loop(0.1). More info at the link. The Thread/:on_process idea inspired me to write a gem for my own use: https://github.com/da99/Chee/blob/master/lib/Chee.rb
If the last call in your Ruby program is SSH, then you can exec ssh -tt root#remote 'ruby numbers.rb'. But, if you still want interactivity between User<->Ruby<->SSH, then the previous alternative is the best.
Related
In ruby18 I sometimes did the following to get a subprocess with full control:
stdin, #stdin= IO.pipe
#stdout, stdout= IO.pipe
#stderr, stderr= IO.pipe
#pid= fork do
#stdin.close
STDIN.close
stdin.dup
#stdout.close
STDOUT.close
stdout.dup
#stderr.close
STDERR.close
stderr.dup
exec(...)
end
This does not work in ruby19. The close method for STDIN, STDOUT, STDERR does not close the underlying filedescriptor in ruby19. How do I do this in ruby19.
Check out Process.spawn, Open3, and the childprocess gem.
I can't tell exactly what you're trying to do there, but you can take control of a child process's IO in many ways.
Using Unix pipes:
readme, writeme = IO.pipe
pid = fork {
$stdout.reopen writeme
readme.close
exec(...)
}
Juggling the IOs with Process.spawn:
pid = spawn(command, :err=>:out)
Or wrapping the process in POpen3:
require 'open3'
include Open3
popen3(RUBY, '-r', THIS_FILE, '-e', 'hello("Open3", true)') do
|stdin, stdout, stderr|
stdin.write("hello from parent")
stdin.close_write
stdout.read.split("\n").each do |line|
puts "[parent] stdout: #{line}"
end
stderr.read.split("\n").each do |line|
puts "[parent] stderr: #{line}"
end
You might also consider Jesse Storimer's Working With Unix Processes. It has a lot of information and his writing style is very easy to read and understand. The book doubles as a reference guide that is somehow more useful than a lot of the actual documentation.
references:
http://pleac.sourceforge.net/pleac_ruby/processmanagementetc.html
http://rubydoc.info/stdlib/core/1.9.3/Process.spawn
http://devver.wordpress.com/2009/10/12/ruby-subprocesses-part_3/
This post shows one way to temporarily replace stdin in Ruby:
begin
save_stdin = $stdin # a dup by any other name
$stdin.reopen('/dev/null') # dup2, essentially
# do stuff
ensure
$stdin.reopen(save_stdin) # restore original $stdout
save_stdin.close # and dispose of the copy
end
Since this question is one of the top google hits for “ruby replace stdin,” I hope this will help others looking for how to do that.
I would like to use Sinatra's Streaming capability introduced in 1.3 coupled with some stdout redirection. It would basically be a live streaming output of a long running job. I looked into this question and the Sinatra streaming sample in the README.
Running 1.8.7 on OSX:
require 'stringio'
require 'sinatra'
$stdout.sync = true
module Kernel
def capture_stdout
out = StringIO.new
$stdout = out
yield out
ensure
$stdout = STDOUT
end
end
get '/' do
stream do |out|
out << "Part one of a three part series... <br>\n"
sleep 1
out << "...part two... <br>\n"
sleep 1
out << "...and now the conclusion...\n"
Kernel.capture_stdout do |stream|
Thread.new do
until (line = stream.gets).nil? do
out << line
end
end
method_that_prints_text
end
end
end
def method_that_prints_text
puts "starting long running job..."
sleep 3
puts "almost there..."
sleep 3
puts "work complete!"
end
So this bit of code prints out the first three strings properly, and blocks while the method_that_prints_text executes and does not print anything to the browser. My feeling is that stdout is empty on the first call and it never outputs to the out buffer. I'm not quite sure what the proper ordering would be and would appreciate any suggestions.
I tried a few of the EventMachine implementations mentioned in the question above, but couldn't get them to work.
UPDATE
I tried something slightly different to where I had the method run in a new thread, and override STDOUT for that thread as described here...
Instead of Kernel.capture_stdout above...
s = StringIO.new
Thread.start do
Thread.current[:stdout] = s
method_that_prints_text
end.join
while line = s.gets do
out << line
end
out << s.string
With the ThreadOut module listed in the link above, this seems to work a bit better. However it doesn't stream. The only time something is printed to the browser is on the final line out << s.string. Does StringIO not have the capability to stream?
I ended up solving this by discovering that s.string was updated periodically as time went on, so I just captured the output in a separate thread and grabbed the differences and streamed them out. It appears as though string redirection doesn't behave like a normal IO object.
s = StringIO.new
t = Thread.start do
Thread.current[:stdout] = s
method_that_prints_text
sleep 2
end
displayed_text = ''
while t.alive? do
current_text = s.string
unless (current_text.eql?(displayed_text))
new_text = current_text[displayed_text.length..current_text.length]
out << new_text
displayed_text = current_text * 1
end
sleep 2
end
I'm creating a script to wrap jdb (java debugger). I essentially want to wrap this process and proxy the user interaction. So I want it to:
start jdb from my script
send the output of jdb to stdout
pause and wait for input when jdb does
when the user enters commands, pass it to jdb
At the moment I really want a pass thru to jdb. The reason for this is to initialize the process with specific parameters and potentially add more commands in the future.
Update:
Here's the shell of what ended up working for me using expect:
PTY.spawn("jdb -attach 1234") do |read,write,pid|
write.sync = true
while (true) do
read.expect(/\r\r\n> /) do |s|
s = s[0].split(/\r\r\n/)
s.pop # get rid of prompt
s.each { |line| puts line }
print '> '
STDOUT.flush
write.print(STDIN.gets)
end
end
end
Use Open3.popen3(). e.g.:
Open3.popen3("jdb args") { |stdin, stdout, stderr|
# stdin = jdb's input stream
# stdout = jdb's output stream
# stderr = jdb's stderr stream
threads = []
threads << Thread.new(stderr) do |terr|
while (line = terr.gets)
puts "stderr: #{line}"
end
end
threads << Thread.new(stdout) do |terr|
while (line = terr.gets)
puts "stdout: #{line}"
end
end
stdin.puts "blah"
threads.each{|t| t.join()} #in order to cleanup when you're done.
}
I've given you examples for threads, but you of course want to be responsive to what jdb is doing. The above is merely a skeleton for how you open the process and handle communication with it.
The Ruby standard library includes expect, which is designed for just this type of problem. See the documentation for more information.
Similar to Getting output of system() calls in Ruby , I am running a system command, but in this case I need to output the STDOUT from the command as it runs.
As in the linked question, the answer is again not to use system at all as system does not support this.
However this time the solution isn't to use backticks, but IO.popen, which returns an IO object that you can use to read the input as it is being generated.
In case someone might want to read stdout and stderr:
It is important to read them in parallel, not first one then the other. Because programs are allowed to output to stdout and stderr by turns and even in parallel. So, you need threads. This fact isn't even Ruby-specific.
Stolen from here.
require 'open3'
cmd = './packer_mock.sh'
data = {:out => [], :err => []}
# see: http://stackoverflow.com/a/1162850/83386
Open3.popen3(cmd) do |stdin, stdout, stderr, thread|
# read each stream from a new thread
{ :out => stdout, :err => stderr }.each do |key, stream|
Thread.new do
until (raw_line = stream.gets).nil? do
parsed_line = Hash[:timestamp => Time.now, :line => "#{raw_line}"]
# append new lines
data[key].push parsed_line
puts "#{key}: #{parsed_line}"
end
end
end
thread.join # don't exit until the external process is done
end
here is my solution
def io2stream(shell, &block)
Open3.popen3(shell) do |_, stdout, stderr|
while line = stdout.gets
block.call(line)
end
while line = stderr.gets
block.call(line)
end
end
end
io2stream("ls -la", &lambda { |str| puts str })
With following you can capture stdout of a system command:
output = capture(:stdout) do
system("pwd") # your system command goes here
end
puts output
shortened version:
output = capture(:stdout) { system("pwd") }
Similarly we can also capture standard errors too with :stderr
capture method is provided by active_support/core_ext/kernel/reporting.rb
Looking at that library's code comments, capture is going to be deprecated, so not sure what is the current supported method name is.
I've got a script thats supposed to mimic ffmpeg on my local machine, by sending the command of to a remote machine, running it there and then returning the results.
(see previous stackoverflow question.)
#!/usr/bin/env ruby
require 'rubygems'
require 'net/ssh'
require 'net/sftp'
require 'highline/import'
file = ARGV[ ARGV.index( '-i' ) + 1] if ARGV.include?( '-i' )
puts 'No input file specified' unless file;
host = "10.0.0.10"
user = "user"
prod = "new-#{file}" # product filename (call it <file>-new)
rpath = "/home/#{user}/.rffmpeg" # remote computer operating directory
rfile = "#{rpath}/#{file}" # remote filename
rprod = "#{rpath}/#{prod}" # remote product
cmd = "ffmpeg -i #{rfile} #{rprod}"# remote command, constructed
pass = ask("Password: ") { |q| q.echo = false } # password from stdin
Net::SSH.start(host, user ) do |ssh|
ssh.sftp.connect do |sftp|
# upload local 'file' to remote 'rfile'
sftp.upload!(file, rfile)
# run remote command 'cmd' to produce 'rprod'
ssh.exec!(cmd)
# download remote 'rprod' to local 'prod'
sftp.download!(rprod, prod)
end
end
now my problem is at
ssh.exec!(cmd)
I want to display the cmd's output to the local user in real-time. But making it
puts ssh.exec!(cmd)
I only get the resulting output after the command has finished running. How would I have to change the code to make this work?
On the display side of your question, you can generate an updating progress bar in Ruby using the "\r" string char. This backs you up to the beginning of the current line allowing you to re-write it. For example:
1.upto(100) { |i| sleep 0.05; print "\rPercent Complete #{i}%"}
Or if you just want a progress bar across the screen you can simply do something similar to this:
1.upto(50) { sleep 0.05; print "|"}
Also, relating to stdout, in addition to flushing output per previous example (STDOUT.flush), you can ask Ruby to automatically sync writes to an IO buffer (in this case STDOUT) with associated device writes (basically turns off internal buffering):
STDOUT.sync = true
Also, I find that sometimes flush doesn't work for me, and I use "IO.fsync" instead. For me that's mostly been related to file system work, but it's worth knowing.
From ri Net::SSH::start:
-------------------------------------------------------- Net::SSH::start
Net::SSH::start(host, user, options={}, &block) {|connection| ...}
------------------------------------------------------------------------
The standard means of starting a new SSH connection. When used with
a block, the connection will be closed when the block terminates,
otherwise the connection will just be returned. The yielded (or
returned) value will be an instance of
Net::SSH::Connection::Session (q.v.). (See also
Net::SSH::Connection::Channel and Net::SSH::Service::Forward.)
Net::SSH.start("host", "user") do |ssh|
ssh.exec! "cp /some/file /another/location"
hostname = ssh.exec!("hostname")
ssh.open_channel do |ch|
ch.exec "sudo -p 'sudo password: ' ls" do |ch, success|
abort "could not execute sudo ls" unless success
ch.on_data do |ch, data|
print data
if data =~ /sudo password: /
ch.send_data("password\n")
end
end
end
end
ssh.loop
end
So it looks like you can get more interactive by using #open_channel
Here's some example code:
user#server% cat echo.rb
#! /usr/local/bin/ruby
def putsf s
puts s
STDOUT.flush
end
putsf "hello"
5.times do
putsf gets.chomp
end
putsf "goodbye"
And on your local machine:
user#local% cat client.rb
#! /usr/local/bin/ruby
require 'rubygems'
require 'net/ssh'
words = %w{ earn more sessions by sleaving }
index = 0;
Net::SSH.start('server', 'user') do |ssh|
ssh.open_channel do |ch|
ch.exec './echo.rb' do |ch, success|
abort "could not execute ./echo.rb" unless success
ch.on_data do |ch, data|
p [:data, data]
index %= words.size
ch.send_data( words[index] + "\n" )
index += 1
end
end
end
end
user#local% ./client.rb
[:data, "hello\n"]
[:data, "earn\n"]
[:data, "more\n"]
[:data, "sessions\n"]
[:data, "by\n"]
[:data, "sleaving\n"]
[:data, "goodbye\n"]
So you can interact with a running process this way.
It's important that the running process flush its output before requesting input - otherwise, the program might hang as the channel may not have received the unflushed output.