I wonder how can I interact with never-ending(eternal looping) child process.
source code of loop_puts.rb, child process :
loop do
str = gets
puts str.upcase
end
main.rb :
Process.spawn("ruby loop_puts.rb",{:out=>$stdout, :in=>$stdin})
I want to put some letter, not by my hand typing, and get result(not previous result) in variable.
how can I do this?
thanks
There are a number of ways to do this and it's hard to recommend one without more context.
Here's one way using a forked process and a pipe:
# When given '-' as the first param, IO#popen forks a new ruby interpreter.
# Both parent and child processes continue after the return to the #popen
# call which returns an IO object to the parent process and nil to the child.
pipe = IO.popen('-', 'w+')
if pipe
# in the parent process
%w(please upcase these words).each do |s|
STDERR.puts "sending: #{s}"
pipe.puts s # pipe communicates with the child process
STDERR.puts "received: #{pipe.gets}"
end
pipe.puts '!quit' # a custom signal to end the child process
else
# in the child process
until (str = gets.chomp) == '!quit'
# std in/out here are connected to the parent's pipe
puts str.upcase
end
end
Some documentation for IO#popen here. Note that this may not work on all platforms.
Other possible ways to approach this include Named Pipes, drb, and message queues.
Related
I have the following code snippet (a simplified representation of what I'm trying to do - training wheels). The sleep(2) would represent some network operation in my real code:
arr = []
5.times do |i|
rd, wr = IO.pipe
fork do
sleep(2) # I'm still waiting for the sleep to happen on each process ... not good, there is no parallelism here
rd.close
arr[i] = i
Marshal.dump(arr, wr)
end
wr.close
result = rd.read
arr = Marshal.load(result)
end
# Process.waitall
p arr
Q: is it possible to somehow create new processes in a loop, pass the results back but not waiting on each iteration. I'm pretty rusty and don't know / remember a great deal about IPC ... especially in Ruby.
Actual result is wait time of 2s*5 = 10s
Expected ~2s tootal (async processing of the sleep())
So a good comment clarifying things, explaining the theory would help a lot. Thanks.
In your loop you wait for each child process to write its results to the pipe before starting the next iteration.
The simplest fix would be to save the read ends of the pipes in an array and don’t read any of them until the loop is finished and you’ve started all the child processes:
arr = []
# array to store the IOs
pipes = []
5.times do |i|
rd, wr = IO.pipe
fork do
sleep(2)
rd.close
# Note only returning the value of i here, not the whole array
Marshal.dump(i, wr)
end
wr.close
#store the IO for later
pipes[i] = rd
end
# Now all child processes are started, we can read the results in turn
# Remember each child is returng i, not the whole array
pipes.each_with_index do |rd, i|
arr[i] = Marshal.load(rd.read)
end
A more complex solution if the wait/network times for different child processes variad might be to look at select, so you could read from whichever pipe was ready first.
Basically in my search for code which will loop, and terminate upon user input, i managed to find code here, and after some alteration, produced this:
#desired destination method, however loop persists!!
def desired_method
print "method entered"
end
Thread.new do
while line = STDIN.gets
break if line.chomp == "" # code detects user input
end
desired_method
end
# program will loop here until user presses enter
loop do
puts "foo"
sleep 1
end
This code is brilliant, and will enter the method 'desired_method' when i hit enter, however the loop persists!! printing 'foo' perpetually after "method entered"!!. I have done some research prior to posting this question on how to kill threads, which i believe may hold the answer. My attempts included naming the thread and using the 'thread.exit' function to kill it, however these techniques have remained unsuccessful.
Can anyone illustrate how i might enter the 'desired_method' method without the persisting "foo" print?
Thanks in advance, and greatly appreciated.
An easy solution here is to use semaphore, signalling between threads with a variable access to both places:
# This will be out stop flag, for signalling between threads.
#time_to_stop = false
def desired_method
print "method entered"
# Here we want the loop in the other thread to stop.
#time_to_stop = true
end
Thread.new do
while line = STDIN.gets
break if line.chomp == "" # code detects user input
end
desired_method
end
# program will loop here until user presses enter
loop do
puts "foo"
sleep 1
# only continue if the stop flag is not set.
break if #time_to_stop
end
Hope this helps.
I found out that you can't share variables in any way (not even $global ones)and that you must use pipes, so I will cut to the chase.
There are two processes: Process A gets input from somewhere (possibly pipe in the command line) and sends it to Process (fork) B which will print the last given value every second.
This is basically a simple, basic version of what I need. I tried to do it,but it doesn't show any output. I can't figure out why.
This is the source code:
#!/usr/bin/ruby
# create two pipes.
reader, writer = IO.pipe
fork do
loop do
# print the last received value every second (or so)
puts "Last name is: #{reader.read }"
sleep 1
end
end
fork do
ARGF.each_line do |e|
name = e.chomp
#send the data through the pipe
writer.write name
end
#make sure all forks are killed
Process.waitall
In order to get the desired input here is a sample script which you can pipe:
#!/usr/bin/ruby
arr = ['Sally', 'Mike', 'John', 'Steve', 'Iana', 'That guy']
loop do
puts arr[rand(5)]
sleep 1
STDOUT.flush
end
and simply pipe it:
./generate_input.rb | ./program.rb
IO#read with no length parameter reads until EOF. Since you're not sending EOF (that is, you're not closing the writer), it never returns. You might try something like reader.gets and writer.puts to write data one line at a time.
Something like this will work as expected:
reader, writer = IO.pipe
fork do
loop do
# print the last received value every second (or so)
puts "Last name is: #{reader.gets }"
sleep 1
end
end
fork do
ARGF.each_line do |e|
name = e.chomp
#send the data through the pipe
writer.puts name
end
end
#make sure all forks are killed
Process.waitall
I'm new to ruby and object oriented languages, and I'm having trouble figuring out a way to accomplish forking a process inside a method and passing the delayed output to be used outside the method while also returning the process id.
def method(arg)
proc_id = fork do
var = `command #{arg}`
end
return both = [proc_id, var]
end
This doesn't work as var will return nil since the process has not yet finished. How could I accomplish something like this?
UPDATE:
Using IO.pipe I was able to accomplish Inter-Process Communication. However, trying to use this solution inside a method will not allow me to return both proc_id and var without first waiting for the process to finish which forces me to create new arrays and iterations that would be otherwise unnecessary. The objective here is to have freedom to execute code outside the method while the fork process inside the method is still working.
arg_array = ["arg1", "arg2", "arg3", "arg4"]
input = []
output = []
proc_id = []
arg_array.each_index do |i|
input[i], output[i] = IO.pipe
proc_id[i] = fork do
input[i].close
output[i].write `command #{arg_array[i]}`
end
output[i].close
end
command2
command3
include Process
waitpid(proc_id[0])
command4
Process.waitall
arg_array.each_index do |x|
puts input[x].read
end
You need to use a little more time studying the concept of fork. The parent and child process after a fork cannot communicate (exchange variables) each other without using IPC (Inter-Processs Communication) which is somewhat complicated.
But for your purpose (getting the child process id, and its output), it's easier with Open3.popen2 or Open3.popen3.
http://www.ruby-doc.org/stdlib-1.9.3/libdoc/open3/rdoc/Open3.html#method-c-popen2
if you want to kick something off and save the child pid, that's fairly simple.
pid = fork
if pid
return pid
else
system("command #{arg}")
exit
end
a little bit clumsy, but basically, fork returns the child pid to the parent process, and nil to the child process. Make sure you exit the child, it won't do that automatically.
Thanks to jaeheung's suggestion, I've solved using Open3.popen2 (requires version 1.9.3).
arguments = ["arg1", "arg2", "arg3", "arg4"]
require 'open3'
include Open3
def method(arg)
input, output, thread = Open3.popen2("command #{arg}")
input.close
return [thread.pid, output]
end
thread_output = []
arguments.each do |i|
thread_output << method("#{i}")
end
command1
command2
include Process
waitpid(thread_output[0][0])
command3
Process.waitall
thread_output.each do |x|
puts x[1].read
end
I have the following code:
data_set = [1,2,3,4,5,6]
results = []
data_set.each do |ds|
puts "Before fork #{ds}"
r,w = IO.pipe
if pid = Process.fork
w.close
child_result = r.read
results << child_result
else
puts "Child worker for #{ds}"
sleep(ds * 5)
r.close
w.write(ds * 2)
exit
end
end
Process.waitall
puts "Ended everything #{results}"
Basically, I want each child to do some work, and then pass the result to the parent. My code doesn't run in parallel now, and I don't know where exactly my problem lies, probably it's because I'm doing a read in the parent, but I'm not sure. What would I need to do to get it to run async?
EDIT: I changed the code to this, and it seems to work ok. Is there any problem that I'm not seeing?
data_set = [1,2,3,4,5,6]
child_pipes = []
results = []
data_set.each do |ds|
puts "Before fork #{ds}"
r,w = IO.pipe
if pid = Process.fork
w.close
child_pipes << r
else
puts "Child worker for #{ds}"
sleep(ds * 5)
r.close
w.write(ds * 2)
exit
end
end
Process.waitall
puts child_pipes.map(&:read)
It's possible for a child to block writing to the pipe to the parent if its output is larger than the pipe capacity. Ideally the parent would perform a select loop on the child pipes or spawn threads reading from the child pipes so as to consume data as it becomes available to prevent children from stalling on a full pipe and failing. In practice, if the child output is small, just doing the waitall and read will work.
Others have solved these problems in reusable ways, you might try the the parallel gem to avoid writing a bunch of unnecessary code.