I am trying to do signal handling for my CLI app. I would like to avoid the self-pipe "trick", but also consider that using a per thread list of signals that is periodically polled is not the best idea. Hence, my idea was to use Queue to block until a new signal arrives. Here is my attempt:
class CLI
def initialize
#manager = Thread.new { sleep }
#signal_queue = Queue.new
setup_signal_handlers
end
def run
loop do
signal = #signal_queue.pop # This doesn't unblock!
handle_signal(signal)
end
rescue Interrupt
exit
end
private
def handle_signal(signal)
case signal
when 'INT'
raise Interrupt
end
end
def setup_signal_handlers
%w(INT).each do |signal|
trap signal do
#signal_queue.push signal # This works. #signal_queue.size is incremented
end
end
end
end
# Run with CLI.new.run and hit ctrl^C
where I've simplified #manager for clarity. The problem is that, even if the signal is trapped and pushed to #signal_queue, the blocking call #signal_queue.pop doesn't pick it up. What am I missing here?
It looks like you're experiencing Ruby bug #12405: Queue doesn't work inside of trap. Luckily this bug got fixed in Ruby versions 2.2.7, 2.3.4, and 2.4.1 (2.5.0 already includes that fix); therefore your code behaves as expected after upgrading to a more recent Ruby version.
Related
I'm trying to test a gem I'm creating with RSpec. The gem's purpose is to create queues (using 'bunny'). It will serve to communicate between processes on several servers.
But I cannot find documentation on how to safely create processes inside RSpec running environment without spawning several testing processes (all displaying example failures and successes).
Here is what I wanted the tests to do :
Spawn children processes, waiting on the queue
Push messages from the main RSpec process
Consumes the queue on the children processes
Wait for children to stop and get the number of messages received from each child.
For now I implemented a simple case where child is consuming only one message and then stops.
Here is my code currently :
module Queues
# Basic CR accepting only jobs of type cmd_line
class CR
attr_reader :nb_jobs
def initialize
# opening communication pipes
#rout, #wout = IO.pipe
#nb_jobs = nil # not yet available.
end
def main
#todo = JobPipe.instance
job = #todo.pop do |j|
# accept only CMD_LINE type of jobs.
j.type == Messages::Job::CMD_LINE
end
# run command
%x{#{job.cmd}}
#wout.puts "1" # saying that we did one job
end
def run
#pid = Process.fork
if #pid.nil? then
# we are in the child
self.main
#rout.close
#wout.close
exit
end
end
def wait
#nb_jobs = #rout.gets(nil).to_i
Process.wait(#pid)
#rout.close
#wout.close
#nb_jobs
end
end
#job = Messages::Job.new({:type => Messages::Job::CMD_LINE, :cmd => "sleep 1" })
RSpec.describe JobPipe do
context "one Orchestrator and one CR" do
before(:each) do
indalo_queue_pre_configure
end
it "can send a job with Orchestrator and be received by CR" do
cr = CR.new
cr.run # execute the C.R. process
todo = JobPipe.instance
todo.push(#job)
nb_jobs = cr.wait
expect(nb_jobs).to eql(1)
end
end
context "one Orchestrator and severals CR" do
it 'can send one job per CR and get all back' do
crs = Array.new(rand(2..10)) { CR.new }
crs.each do |cr|
cr.run
end
todo = JobPipe.instance
crs.each do |_|
todo.push(#job)
end
nb_jobs = 0
crs.each do |cr|
nb_jobs += cr.wait
end
expect(nb_jobs).to eql(crs.length)
end
end
end
end
Edit: The question is (sorry not putting it right away, this was a mistake):
Is there a way to use correctly RSpec on a multi-process environment ?
I'm not looking for a code review, just wanted to display a clear example of what I wanted to do. Here I used fork, but this duplicate all the process (including RSpec part) and got numerous RSpec outputs which is not what we would expect in a test suite.
I would expect that only the main program states the RSpec outputs/stats and the subprocesses just interact with it.
The only way I see to do that correctly is not fork, but call subprocesses through an other mean. Maybe I answer alone to this question...
But not knowing well RSpec, I was wondering if someone knew how to do it within RSpec without writing external code. It seems to me that having separate codes linked to a single test example is not a good idea.
What I found about multi-process testing is this plugin to RSpec. The only thing is I don't know about the mock concept, but maybe I have to learn about it...
Ok, I found an answer which is to use the &block argument of the Process.fork method. In this case, you don't really duplicate all the process, but just execute the block of code in an other process and then return 0 (like said in the Ruby doc).
This prevent the children to get all the RSpec environment and displaying plenty of times the states of your tests.
PS : Be careful not to forget to redirect STDOUT/STDERR of child process if you don't want them to pollute the STDOUT/STDERR of the test.
PS2: don't forget to close #wout on the parent side if you call #rout.gets(nil) in it, because having it opened on the parent prevent EOF from happening (a bug in the code I presented) even if you close it in the child.
PS3: Use two pipes instead of one to prevent child/parent to talk and listen in the same. Childhood error but I did it again.
PS4: Use exit statement (at the end of the &block) to prevent zombie state of the child and usure parent not waiting too long that the rest of the child process dies.
Sorry for that long post, but it's good it stays also for me ^^
I am trying to understand how signal handlers attach to a process and their scope for a process.
class Main
Signal.trap('USR1') do
Process2.kill
end
def main_process
#do something
p = Process2.new
output = p.give_output
#then again do something
end
end
class Process2
Signal.trap('USR1') do
Process2.kill
end
def self.kill
if ##static_object.blank?
#do nothing
else
#do something
end
end
def give_output
#do something
##static_object = true
#do something
end
end
Now if I issue a SIGUSR1 to the process while give_output is getting executed and ##static_object is not nil, the handler should behave as expected. However, even after give_output has finished execution, if a signal is sent, the handler inside Process2 will catch it. From what I understand, the handler gets attached to the process. Can we have two signal handlers for the same process for the same signal? For example - while give_output is executing and a signal is issued, the handler inside Process2 should get control otherwise another signal handler defined in Main should get control.
I was curious about this myself and came across the following resources: http://timuruski.net/blog/2014/signals-traps-and-rescues/, https://www.ruby-forum.com/topic/87221, and http://www.ibm.com/developerworks/library/l-reent/. That second link has some resources which seem very relevant to your question.
In short, you can only reliably trap one interrupt type per process. Signal interrupts are handled by the host OS. As its name suggests, interrupts halt the normal execution of your program until the interrupt is handled.
Summarizing the information given in the links posted by #Anthony E. Using signal trap it's not possible to trap a signal from multiple places in your code. If you want to handle signals differently at multiple places, use rescue blocks.
Signal Trap -
Signal.trap("USR1") do
#do something
end
Rescue Interrupt -
begin
#some functions
rescue Interrupt
#handle the exception
exit
end
Difference in the way signals are handled by rescue and trap -
Trap defines the handler at the top of the code, if a signal is triggered, it would be caught by the trap. On the other hand, rescue is defined at the bottom of the code, that is when there is no trap available, it will raise a Signal Exception which will be caught by the rescue block
The signal trap code gets associated with the process and cannot be replaced. If one wants to handle signals at different places of the program in a different way, the best way is to raise an exception and handle it
I am trying to use Ruby's daemon gem and loop the restart of a daemon that has its own loop. My code looks like this now:
require 'daemons'
while true
listener = Daemons.call(:force => true) do
users = accounts.get_updated_user_list
TweetStream::Client.new.follow(users) do |status|
puts "#{status.text}"
end
end
sleep(60)
listener.restart
end
Running this gives me the following error (after 60 seconds):
undefined method `restart' for #<Daemons::Application:0x007fc5b29f5658> (NoMethodError)
So obviously Daemons.call doesn't return a controllable daemon like I think it does. What do I need to do to set this up correctly. Is a daemon the right tool here?
I think this is what you're after, although I haven't tested it.
class RestartingUserTracker
def initialize
#client = TweetStream::Client.new
end
def handle_status(status)
# do whatever it is you're going to do with the status
end
def fetch_users
accounts.get_updated_user_list
end
def restart
#client.stop_stream
users = fetch_users
#client.follow(users) do |status|
handle_status(status)
end
end
end
EM.run do
client = RestartingUserTracker.new
client.restart
EM::PeriodicTimer.new(60) do
client.restart
end
end
Here's how it works:
TweetStream uses EventMachine internally, as a way of polling the API forever and handling the responses. I can see why you might have felt stuck, because the normal TweetStream API blocks forever and doesn't give you a way to intervene at any point. However, TweetStream does allow you to set up other things in the same event loop. In your case, a timer. I found the documentation on how to do that here: https://github.com/intridea/tweetstream#removal-of-on_interval-callback
By starting up our own EventMachine reactor, we're able to inject our own code into the reactor as well as use TweetStream. In this case, we're using a simple timer that just restarts the client every 60 seconds.
EventMachine is an implementation of something called the Reactor Pattern. If you want to fully understand and maintain this code, it would serve you well to find some resources about it and gain a full understanding. The reactor pattern is very powerful, but can be difficult to grasp at first.
However, this code should get you started. Also, I'd consider renaming the RestartingUserTracker to something more appropriate.
I think I'm running up against a fundamental misunderstanding on my part of how threading works in ruby and I'm hoping to get some insight.
I'd like to have a simple producer and consumer. First, a producer thread that pulls lines from a file and sticks them into a SizedQueue; when those run out, stick some tokens on the end to let the consumer(s) know things are done.
require 'thread'
numthreads = 2
filename = 'edition-2009-09-11.txt'
bq = SizedQueue.new(4)
producerthread = Thread.new(bq) do |queue|
File.open(filename) do |f|
f.each do |r|
queue << r
end
end
numthreads.times do
queue << :end_of_producer
end
end
Now a few consumers. For simplicity, let's have them do nothing.
consumerthreads = []
numthreads.times do
consumerthreads << Thread.new(bq) do |queue|
until (line = queue.pop) === :end_of_producer
# do stuff in here
end
end
end
producerthread.join
consumerthreads.each {|t| t.join}
puts "All done"
My understanding is that (a) the producer thread will block once the SizedQueue is full and eventually get back to filling it up, and (b) the consumer threads will pull from the SizedQueue, blocking when it empties, and eventually finish.
But under ruby1.9 (ruby 1.9.1p243 (2009-07-16 revision 24175) [i386-darwin9]) I get a deadlock error on the joins. What's going on here? I just don't see where there's any interaction between the threads except via the SizedQueue,which is supposed to be thread-safe.
Any insight would be much-appreciated.
Your understanding is correct and your code works on my machine, on a slightly newer version of Ruby (both ruby 1.9.2dev (2009-08-30 trunk 24705) [i386-darwin10.0.0] and ruby 1.9.2dev (2009-08-30 trunk 24705) [i386-darwin10.0.0])
I'm looking for a portable interface to POSIX alarm(2) (or similar) in Ruby. That's to say, I would like to be able to set a background timer to send a signal to the current process after n seconds.
I have found some good discussion from 2006 on the ruby-talk list that provides a solution using dl/import, but that's a bit of a hack (albeit a neat hack) and not very portable.
I've looked at the much-maligned Timeout module and that won't cut it under JRuby although it works fine with the traditional interpreter. My program is a small command-line shell that uses the Readline library:
TIMEOUT = 5 # seconds
loop do
input = nil
begin
Timeout.timeout(TIMEOUT) do
input = Readline::readline('> ', nil)
end
rescue Timeout::Error
puts "Timeout"
next
end
# do something with input
end
Under JRuby it seems the process blocks in the readline call and Timeout::Error is only thrown after (a) the timer expires and (b) the user enters a new line. And the exception doesn't get rescued. Hmm.
So I came up with this workaround:
require 'readline'
class TimeoutException < Exception ; end
TIMEOUT = 5 # seconds
loop do
input = nil
start_time = Time.now
thread = Thread.new { input = Readline::readline('> ', nil) }
begin
while thread.alive? do
sleep(1) # prevent CPU from melting
raise TimeoutException if(Time.now - start_time > TIMEOUT)
end
rescue TimeoutException
thread.exit
puts "Timeout"
end
# do something with input
end
This is... clunky (let's be polite). I just want alarm(2)! I don't really want to drag in non-core libraries (eg Terminator) for this. Is there a better way?
EDIT:
I can't get another alternative -- creating a thread that sleeps and then sends a signal to the process -- to work under JRuby either. Does JRuby eat signals? Example:
SIG = 'USR2'
Signal.trap(SIG) { raise }
Process.kill(SIG, Process.pid)
JRuby simply returns, Ruby returns the expected "unhandled exception" error.
I'm sorry that I don't have an answer to your larger problem of sending a signal after X seconds to a process, but it seems that all you want to do is timeout after X seconds of waiting for input, and if that's the case then I'd say you are looking for Kernel.select :D
I've personally never used this, but after doing a google for "non-blocking gets", and subsequently exploring links, I found these two to be invaluable discussions:
http://www.ruby-forum.com/topic/126795 (Discussion of multi-threaded gets)
http://www.ruby-forum.com/topic/121404 (Explanation of Kernel.select in 2nd post)
Here's a sample of how to use it. This will print out your prompt and wait for input... If there is no input after five seconds, then the program will end. If there is input, as soon as there is input it will spit it back out and end... Obviously you can modify this for your own purposes.
def prompt
STDOUT.write "> "
STDOUT.flush
end
def amusing_messages
[ "You must enter something!",
"Why did you even start me if you just wanted to stare at me?",
"Isn't there anything better you could be doing?",
"Just terminate me already... this is getting old",
"I'm waiting..."]
end
prompt
loop do
read_array, write_array, error_array = Kernel.select [STDIN], nil, nil, 5
if read_array.nil?
puts amusing_messages[rand(amusing_messages.length)]
else
puts "Result is: #{read_array[0].read_nonblock(30)}"
end
prompt
end
It's probably not as elegant as you might like, but it definitely gets the job done without mucking around with threads. Unfortunately, this won't help you should you want something more robust (timer/sending a signal to the process), and sadly, I have no clue if this works in JRuby. Would love to know if it does though :)