In ruby18 I sometimes did the following to get a subprocess with full control:
stdin, #stdin= IO.pipe
#stdout, stdout= IO.pipe
#stderr, stderr= IO.pipe
#pid= fork do
#stdin.close
STDIN.close
stdin.dup
#stdout.close
STDOUT.close
stdout.dup
#stderr.close
STDERR.close
stderr.dup
exec(...)
end
This does not work in ruby19. The close method for STDIN, STDOUT, STDERR does not close the underlying filedescriptor in ruby19. How do I do this in ruby19.
Check out Process.spawn, Open3, and the childprocess gem.
I can't tell exactly what you're trying to do there, but you can take control of a child process's IO in many ways.
Using Unix pipes:
readme, writeme = IO.pipe
pid = fork {
$stdout.reopen writeme
readme.close
exec(...)
}
Juggling the IOs with Process.spawn:
pid = spawn(command, :err=>:out)
Or wrapping the process in POpen3:
require 'open3'
include Open3
popen3(RUBY, '-r', THIS_FILE, '-e', 'hello("Open3", true)') do
|stdin, stdout, stderr|
stdin.write("hello from parent")
stdin.close_write
stdout.read.split("\n").each do |line|
puts "[parent] stdout: #{line}"
end
stderr.read.split("\n").each do |line|
puts "[parent] stderr: #{line}"
end
You might also consider Jesse Storimer's Working With Unix Processes. It has a lot of information and his writing style is very easy to read and understand. The book doubles as a reference guide that is somehow more useful than a lot of the actual documentation.
references:
http://pleac.sourceforge.net/pleac_ruby/processmanagementetc.html
http://rubydoc.info/stdlib/core/1.9.3/Process.spawn
http://devver.wordpress.com/2009/10/12/ruby-subprocesses-part_3/
This post shows one way to temporarily replace stdin in Ruby:
begin
save_stdin = $stdin # a dup by any other name
$stdin.reopen('/dev/null') # dup2, essentially
# do stuff
ensure
$stdin.reopen(save_stdin) # restore original $stdout
save_stdin.close # and dispose of the copy
end
Since this question is one of the top google hits for “ruby replace stdin,” I hope this will help others looking for how to do that.
Related
I'm trying to exec a shell process such that its standard output is prefixed with an identifier.
My approach is to write a custom IO object that re-implements write, passing it as the :out argument to exec (documented under Process::spawn).
require "delegate"
class PrefixedStdout < DelegateClass(IO)
def initialize(prefix, io)
#prefix = prefix
super(io)
end
def write(str)
super("#{#prefix}: #{str}")
end
end
pr_stdout = PrefixedStdout.new("my_prefix", $stdout)
pr_stdout.write("hello\n") # outputs "my_prefix: hello"
exec("echo hello", out: pr_stdout) # outputs "hello"
Somehow exec is bypassing PrefixedStdout#write and calling $stdout.write directly. How do I force exec to use my prefixed output stream as its stdout?
What gets preserved in the other process is the underlying file descriptor (or rather they are hooked up under the hood), so as I commented I don't think you'll ever get writes to that descriptor to be funnelled through your write method - exec replaces the running process with a new one.
A possible approach is to create a pipe, pass one end to your child process and then read from the other end, inserting prefixes as needed,
For example you might do
IO.pipe do |read_pipe, write_pipe|
fork do
exec("echo hello", out: write_pipe)
end
write_pipe.close
while line = read_pipe.gets
puts "prefix: #{line}"
end
end
You might also be interested in IO.popen which wraps some of this up.
Somehow exec is bypassing PrefixedStdout#write and calling
$stdout.write
Take a look at this example:
class MyIO < IO
def initialize(fd)
super
end
def write(str)
STDOUT.puts 'write called'
super
end
end
fd = IO.sysopen("data.txt", "w")
io = MyIO.new(fd)
io.write "goodbye\n"
puts '---now with exec()...'
exec("echo hello", :out => io)
--output:--
write called
---now with exec()...
Now, what do you think is in the file data.txt?
spoiler:
$cat data.txt
hello
So passing an IO object to exec() 'works', but not the way you expected: exec() never calls io.write() to write the output of the child process to io. Instead, I assume exec() obtains the file descriptor for io, then passes it to some C code, which does some system level redirection of the output from the child process to the file data.txt.
Do you have to use exec()? If not:
prefix = "prefix: "
cmd = 'echo hello'
output = `#{cmd}`
puts "#{prefix}#{output}"
--output:--
prefix: hello
Say I have a function like below, how do I capture the output of the Process.spawn call? I should also be able to kill the process if it takes longer than a specified timeout.
Note that the function must also be cross-platform (Windows/Linux).
def execute_with_timeout!(command)
begin
pid = Process.spawn(command) # How do I capture output of this process?
status = Timeout::timeout(5) {
Process.wait(pid)
}
rescue Timeout::Error
Process.kill('KILL', pid)
end
end
Thanks.
You can use IO.pipe and tell Process.spawn to use the redirected output without the need of external gem.
Of course, only starting with Ruby 1.9.2 (and I personally recommend 1.9.3)
The following is a simple implementation used by Spinach BDD internally to capture both out and err outputs:
# stdout, stderr pipes
rout, wout = IO.pipe
rerr, werr = IO.pipe
pid = Process.spawn(command, :out => wout, :err => werr)
_, status = Process.wait2(pid)
# close write ends so we could read them
wout.close
werr.close
#stdout = rout.readlines.join("\n")
#stderr = rerr.readlines.join("\n")
# dispose the read ends of the pipes
rout.close
rerr.close
#last_exit_status = status.exitstatus
The original source is in features/support/filesystem.rb
Is highly recommended you read Ruby's own Process.spawn documentation.
Hope this helps.
PS: I left the timeout implementation as homework for you ;-)
I followed Anselm's advice in his post on the Ruby forum here.
The function looks like this -
def execute_with_timeout!(command)
begin
pipe = IO.popen(command, 'r')
rescue Exception => e
raise "Execution of command #{command} unsuccessful"
end
output = ""
begin
status = Timeout::timeout(timeout) {
Process.waitpid2(pipe.pid)
output = pipe.gets(nil)
}
rescue Timeout::Error
Process.kill('KILL', pipe.pid)
end
pipe.close
output
end
This does the job, but I'd rather use a third-party gem that wraps this functionality. Anyone have any better ways of doing this? I have tried Terminator, it does exactly what I want but it does not seem to work on Windows.
I am trying to execute an interactive shell program on a remote host from another ruby program. For the sake of simplicity let's suppose that the program I want to execute is something like this:
puts "Give me a number:"
number = gets.chomp()
puts "You gave me #{number}"
The approach that most successful has been so far is using the one I got from here. It is this one:
require 'open3'
Open3.popen3("ssh -tt root#remote 'ruby numbers.rb'") do |stdin, stdout, stderr|
# stdin = input stream
# stdout = output stream
# stderr = stderr stream
threads = []
threads << Thread.new(stderr) do |terr|
while (line = terr.gets)
puts "stderr: #{line}"
end
end
threads << Thread.new(stdout) do |terr|
while (line = terr.gets)
puts "stdout: #{line}"
end
end
sleep(2)
puts "Give me an answer: "
answer = gets.chomp()
stdin.puts answer
threads.each{|t| t.join()} #in order to cleanup when you're done.
end
The problem is that this is not "interactive" enough to me, and the program that I would like to execute (not the simple numbers.rb) has a lot more of input / output. You can think of it as an apt-get install that will ask you for some input to solve some problems.
I have read about net::ssh and pty, but couldn't see if they were going to be the (easy/elegant) solution I am looking for.
The ideal solution will be to make it in such a way that the user does not realize that the IO is being done on a remote host: the stdin goes to the remote host stdin, the stdout from the remote host comes to me and I show it.
If you have any ideas I could try I will be happy to hear them. Thank you!
Try this:
require "readline"
require 'open3'
Open3.popen3("ssh -tt root#remote 'ruby numbers.rb'") do |i, o, e, th|
Thread.new {
while !i.closed? do
input =Readline.readline("", true).strip
i.puts input
end
}
t_err = Thread.new {
while !e.eof? do
putc e.readchar
end
}
t_out = Thread.new {
while !o.eof? do
putc o.readchar
end
}
Process::waitpid(th.pid) rescue nil
# "rescue nil" is there in case process already ended.
t_err.join
t_out.join
end
I got it working, but don't ask me why it works. It was mainly trial/error.
Alternatives:
Using Net::SSH, you need to use :on_process and a Thread: ruby net/ssh channel dies? Don't forget to add session.loop(0.1). More info at the link. The Thread/:on_process idea inspired me to write a gem for my own use: https://github.com/da99/Chee/blob/master/lib/Chee.rb
If the last call in your Ruby program is SSH, then you can exec ssh -tt root#remote 'ruby numbers.rb'. But, if you still want interactivity between User<->Ruby<->SSH, then the previous alternative is the best.
I'm creating a script to wrap jdb (java debugger). I essentially want to wrap this process and proxy the user interaction. So I want it to:
start jdb from my script
send the output of jdb to stdout
pause and wait for input when jdb does
when the user enters commands, pass it to jdb
At the moment I really want a pass thru to jdb. The reason for this is to initialize the process with specific parameters and potentially add more commands in the future.
Update:
Here's the shell of what ended up working for me using expect:
PTY.spawn("jdb -attach 1234") do |read,write,pid|
write.sync = true
while (true) do
read.expect(/\r\r\n> /) do |s|
s = s[0].split(/\r\r\n/)
s.pop # get rid of prompt
s.each { |line| puts line }
print '> '
STDOUT.flush
write.print(STDIN.gets)
end
end
end
Use Open3.popen3(). e.g.:
Open3.popen3("jdb args") { |stdin, stdout, stderr|
# stdin = jdb's input stream
# stdout = jdb's output stream
# stderr = jdb's stderr stream
threads = []
threads << Thread.new(stderr) do |terr|
while (line = terr.gets)
puts "stderr: #{line}"
end
end
threads << Thread.new(stdout) do |terr|
while (line = terr.gets)
puts "stdout: #{line}"
end
end
stdin.puts "blah"
threads.each{|t| t.join()} #in order to cleanup when you're done.
}
I've given you examples for threads, but you of course want to be responsive to what jdb is doing. The above is merely a skeleton for how you open the process and handle communication with it.
The Ruby standard library includes expect, which is designed for just this type of problem. See the documentation for more information.
Similar to Getting output of system() calls in Ruby , I am running a system command, but in this case I need to output the STDOUT from the command as it runs.
As in the linked question, the answer is again not to use system at all as system does not support this.
However this time the solution isn't to use backticks, but IO.popen, which returns an IO object that you can use to read the input as it is being generated.
In case someone might want to read stdout and stderr:
It is important to read them in parallel, not first one then the other. Because programs are allowed to output to stdout and stderr by turns and even in parallel. So, you need threads. This fact isn't even Ruby-specific.
Stolen from here.
require 'open3'
cmd = './packer_mock.sh'
data = {:out => [], :err => []}
# see: http://stackoverflow.com/a/1162850/83386
Open3.popen3(cmd) do |stdin, stdout, stderr, thread|
# read each stream from a new thread
{ :out => stdout, :err => stderr }.each do |key, stream|
Thread.new do
until (raw_line = stream.gets).nil? do
parsed_line = Hash[:timestamp => Time.now, :line => "#{raw_line}"]
# append new lines
data[key].push parsed_line
puts "#{key}: #{parsed_line}"
end
end
end
thread.join # don't exit until the external process is done
end
here is my solution
def io2stream(shell, &block)
Open3.popen3(shell) do |_, stdout, stderr|
while line = stdout.gets
block.call(line)
end
while line = stderr.gets
block.call(line)
end
end
end
io2stream("ls -la", &lambda { |str| puts str })
With following you can capture stdout of a system command:
output = capture(:stdout) do
system("pwd") # your system command goes here
end
puts output
shortened version:
output = capture(:stdout) { system("pwd") }
Similarly we can also capture standard errors too with :stderr
capture method is provided by active_support/core_ext/kernel/reporting.rb
Looking at that library's code comments, capture is going to be deprecated, so not sure what is the current supported method name is.