How to observe and rescue an error that happens in the thread without using .join? The code that I have right now:
Thread.abort_on_exception = true
begin
a = Thread.new { errory_code }
rescue Exception => detail
puts detail.message
end
sleep 1 #so that thread has enough time to execute
If I understand it correctly, Thread.abort_on_exception = true aborts thread execution (that is, raises an exception). But why rescue doesn't catch it?
You're expecting the rescue operation to catch exceptions from a bit of code that's run long after Ruby's exited that begin ... end block. That's not going to happen.
Remember when you're dealing with threads, things happen out of order.
The only exceptions you can catch there are those relating to the creation of the thread. What happens inside the thread is a whole other world.
Using join forces your code to wait until the thread is complete, so that gets the ordering correct. Thread exceptions can then be caught.
I figured out how to do it - I just need to wrap insides of my thread into a rescue block like that:
Thread.abort_on_exception = true
a = Thread.new do
begin #new
errory_code
rescue Exception => detail #new
puts detail.message #new
end #new
end
sleep 1 #so that thread has enough time to execute
Which is a horribly obvious solution, - but, since it took me so long to think of it, will hopefully help somebody.
Related
With the following script
threads = [
Thread.new { Thread.current.abort_on_exception = true; raise 'err' },
Thread.new { Thread.current.abort_on_exception = true; raise 'err' },
]
begin
threads.each(&:join)
rescue RuntimeError
puts "Got Error"
end
Half the time I get the expected "Got Error" with exit 0 and the other half I get test.rb:3:in block in <main>': err (RuntimeError).
Shouldn't rescue be able to handle this? If not what might be some alternative solutions for two threads raising an error simultaneously?
I've considered not using abort_on_exception = true but the problem is that if the first thread has, say sleep(10) before the raise, the second thread, which errors immediately, won't get caught until 10 seconds is up (due to the order of the threads array).
Ruby MRI version:
ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-darwin15]
Any ideas would be greatly appreciated. Thanks!
Update
jruby-9.1.6.0 doesn't seem to have this problem. Possibly because the inherent thread safeness of it. It always prints Got Error without any exceptions. Unfortunately, JRuby is not on option for us.
Here are a few pieces of the puzzle here.
Firstly, the program only waits for the main thread to finish:
Thread.new { Thread.current.abort_on_exception = true; raise 'Oh, no!' }
puts 'Ready or not, here I come'
The above may or may not raise the error.
Secondly, if you join on a thread, an exception raised by that threads is re-raised by the joined thread from the #join method:
gollum = Thread.new { raise 'My precious!!!' }
begin
gollum.join
rescue => e
# Prints 'My precious!!!'
puts e.message
end
Now at that point, the execution is returned to the thread that joined. It is no longer joined to the thread that caused the error or any other thread. The reason it's not joined to other threads is because you can only join one thread at the time. threads.each(&:join) actually joins you to the first, when it ends - to the second and so on:
frodo = Thread.new { raise 'Oh, no, Frodo!' }
sam = Thread.new { raise 'Oh, no, Sam!' }
begin
[frodo, sam].each(&:join)
rescue => e
puts e.message
end
puts 'This is the end, my only friend, the end.'
The above prints
Oh, no, Frodo!
This is the end, my only friend, the end.
Now lets put it together:
frodo = Thread.new { Thread.current.abort_on_exception = true; raise 'Oh, no, Frodo!' }
sam = Thread.new { Thread.current.abort_on_exception = true; raise 'Oh, no, Sam!' }
begin
[frodo, sam].each(&:join)
rescue => e
puts e.message
end
puts 'This is the end, my only friend, the end.'
Many things can happen here. What is of importance is, if we manage to join (we don't get an error before that), the rescue will catch the exception in the main thread from whichever thread manages to raise it first and then continue after the rescue. After that, the main thread (and thus the program) may or may not finish before the other thread raises its exception.
Let examine some possible outputs:
ex.rb:1:in `block in ': Oh, no, Frodo! (RuntimeError)
Frodo raised his exception before we joined.
Oh, no, Sam!
This is the end, my only friend, the end.
After we joined, Sam was the first to raise an error. After we printed the error message in the main thread, we also printed the end. Then the main thread finished, before Frodo could raise his error.
Oh, no, Frodo!ex.rb:2:in `block in ': Oh, no, Sam! (RuntimeError)
We managed to join. Frodo was the first to raise, we rescued and printed. Sam raised before we could print the end.
Oh, no, Sam!
This is the end, my only friend, the end.ex.rb:1:in `block in ': Oh, no, Frodo! (RuntimeError)
(Very rarely) We managed to get to the rescue. Sam raised an error first and we printed it from the main thread. We printed the end. Just after the print, but before the main thread is terminated, Frodo managed to jab his error as well.
As for a possible solution, you just need as many rescues as there are threads that might raise. Note that I also put the thread creation in the safeguarded block to ensure we catch potential errors before joins as well:
def execute_safely_concurrently(number_of_threads, &work)
return if number_of_threads.zero?
begin
Thread.new(&work).join
rescue => e
puts e
end
execute_safely_concurrently(number_of_threads.pred, &work)
end
execute_safely_concurrently(2) do
Thread.current.abort_on_exception = true
raise 'Handle me, bitte!'
end
After looking at #ndn's idea of wrapping each thread in it's own rescue. It looks like this solves the problem. Here is his modified example that doesn't block execution on join.
#threads = []
def execute_safely_concurrently(&work)
begin
#threads << Thread.new(&work)
rescue RuntimeError => e
puts "Child Thread Rescue: #{e}"
end
end
execute_safely_concurrently do
Thread.current.abort_on_exception = true
sleep(3)
raise 'Handle me, bitte 1!'
end
execute_safely_concurrently do
Thread.current.abort_on_exception = true
raise 'Handle me, bitte 2!'
end
begin
#threads.each(&:join)
rescue RuntimeError => e
puts "Main Thread Rescue: #{e}"
end
How to handle exceptions in concurrent-ruby thread pools (http://ruby-concurrency.github.io/concurrent-ruby/file.thread_pools.html)?
Example:
pool = Concurrent::FixedThreadPool.new(5)
pool.post do
raise 'something goes wrong'
end
# how to rescue this exception here
Update:
Here is simplified version of my code:
def process
pool = Concurrent::FixedThreadPool.new(5)
products.each do |product|
new_product = generate_new_product
pool.post do
store_in_db(new_product) # here exception is raised, e.g. connection to db failed
end
end
pool.shutdown
pool.wait_for_terminaton
end
So what I want to achive, is to stop processing (break loop) in case of any exception.
This exception is also rescued at higher level of application and there are executed some cleaning jobs (like setting state of model to failure and sending some notifications).
The following answer is from jdantonio from here https://github.com/ruby-concurrency/concurrent-ruby/issues/616
"
Most applications should not use thread pools directly. Thread pools are a low-level abstraction meant for internal use. All of the high-level abstractions in this library (Promise, Actor, etc.) all post jobs to the global thread pool and all provide exception handling. Simply pick the abstraction that best fits your use case and use it.
If you feel the need to configure your own thread pool rather than use the global thread pool, you can still use the high-level abstractions. They all support an :executor option which allows you to inject your custom thread pool. You can then use the exception handling provided by the high-level abstraction.
If you absolutely insist on posting jobs directly to a thread pool rather than using our high-level abstractions (which I strongly discourage) then just create a job wrapper. You can find examples of job wrappers in all our high-level abstractions, Rails ActiveJob, Sucker Punch, and other libraries which use our thread pools."
So how about an implementation with Promises ?
http://ruby-concurrency.github.io/concurrent-ruby/Concurrent/Promise.html
In your case it would look something like this:
promises = []
products.each do |product|
new_product = generate_new_prodcut
promises << Concurrent::Promise.execute do
store_in_db(new_product)
end
end
# .value will wait for the Thread to finish.
# The ! means, that all exceptions will be propagated to the main thread
# .zip will make one Promise which contains all other promises.
Concurrent::Promise.zip(*promises).value!
There may be a better way, but this does work. You will want to change the error handling within wait_for_pool_to_finish.
def process
pool = Concurrent::FixedThreadPool.new(10)
errors = Concurrent::Array.new
10_000.times do
pool.post do
begin
# do the work
rescue StandardError => e
errors << e
end
end
end
wait_for_pool_to_finish(pool, errors)
end
private
def wait_for_pool_to_finish(pool, errors)
pool.shutdown
until pool.shutdown?
if errors.any?
pool.kill
fail errors.first
end
sleep 1
end
pool.wait_for_termination
end
I've created an issue #634. Concurrent thread pool can support abortable worker without any problems.
require "concurrent"
Concurrent::RubyThreadPoolExecutor.class_eval do
# Inspired by "ns_kill_execution".
def ns_abort_execution aborted_worker
#pool.each do |worker|
next if worker == aborted_worker
worker.kill
end
#pool = [aborted_worker]
#ready.clear
stopped_event.set
nil
end
def abort_worker worker
synchronize do
ns_abort_execution worker
end
nil
end
def join
shutdown
# We should wait for stopped event.
# We couldn't use timeout.
stopped_event.wait nil
#pool.each do |aborted_worker|
# Rubinius could receive an error from aborted thread's "join" only.
# MRI Ruby doesn't care about "join".
# It will receive error anyway.
# We can "raise" error in aborted thread and than "join" it from this thread.
# We can "join" aborted thread from this thread and than "raise" error in aborted thread.
# The order of "raise" and "join" is not important. We will receive target error anyway.
aborted_worker.join
end
#pool.clear
nil
end
class AbortableWorker < self.const_get :Worker
def initialize pool
super
#thread.abort_on_exception = true
end
def run_task pool, task, args
begin
task.call *args
rescue StandardError => error
pool.abort_worker self
raise error
end
pool.worker_task_completed
nil
end
def join
#thread.join
nil
end
end
self.send :remove_const, :Worker
self.const_set :Worker, AbortableWorker
end
class MyError < StandardError; end
pool = Concurrent::FixedThreadPool.new 5
begin
pool.post do
sleep 1
puts "we shouldn't receive this message"
end
pool.post do
puts "raising my error"
raise MyError
end
pool.join
rescue MyError => error
puts "received my error, trace: \n#{error.backtrace.join("\n")}"
end
sleep 2
Output:
raising my error
received my error, trace:
...
This patch works fine for any version of MRI Ruby and Rubinius. JRuby is not working and I don't care. Please patch JRuby executor if you want to support it. It should be easy.
In RSpec, I have function that creates a new thread, and inside that thread performs some action–in my case, calls TCPSocket#readline. Here's the function as it is right now:
def read
Thread.new do
while line = #socket.readline
#TODO: stuff
end
end
end
Due to thread scheduling, my test will fail if written as such:
it "reads from socket" do
subject.socket.should_receive(:readline)
subject.read
end
Currently the only way I know to hack around this is to use sleep 0.1. Is there a way to properly delay the test until that thread is running?
If your goal is to assert the system state is changed by the execution of your second thread, you should join on the second thread in your main test thread:
it "reads from socket" do
subject.socket.should_receive(:readline)
socket_thread = subject.read
socket_thread.join
end
This is a bit of a hack, but here's a before block you can use in case you'd like the thread to yield but be able to call join at the end of the thread.
before do
allow(Thread).to receive(:new).and_yield.and_return(Class.new { def join; end }.new)
end
How can I capture an exception from another thread?
I want to do this for some particular thread, and not for other threads, so I cannot turn on abort_on_exception to true.
I want to do something in the main thread with the exception, so I cannot simply do puts, pp, etc. with the exception within the thread that raised in exception.
I found an answer in How to get error messages from ruby threads that suggests using catch, and I think that is the way I want to go, but I cannot not fully figure out how to do it. Or is there a better way? I also found an answer suggesting to use Queue class, but have concern that it may be overkill.
If you turn on abort_on_exception then you won't have a chance to catch it. You can, however, leave that off and simply catch it when you do the join operation on your thread.
thread = Thread.new do
raise "Uh oh"
end
begin
thread.join
rescue => e
puts "Caught exception: #{e}"
end
The alternative is to make the thread catch its own exception and save it somewhere you can fetch it from later. For instance:
exceptions = { }
Thread.new do
begin
raise "Uh oh"
rescue => e
exceptions[Thread.current] = e
end
end
sleep(1)
puts exceptions.inspect
# => {#<Thread:0x007f9832889920 dead>=>#<RuntimeError: Uh oh>}
I was passed a long running legacy ruby program, which has numerous occurrences of
begin
#dosomething
rescue Exception => e
#halt the exception's progress
end
throughout it.
Without tracking down every single possible exception these each could be handling (at least not immediately), I'd still like to be able to shut it down at times with CtrlC.
And I'd like to do so in a way which only adds to the code (so I don't affect the existing behavior, or miss an otherwise caught exception in the middle of a run.)
[CtrlC is SIGINT, or SystemExit, which appears to be equivalent to SignalException.new("INT") in Ruby's exception handling system. class SignalException < Exception, which is why this problem comes up.]
The code I would like to have written would be:
begin
#dosomething
rescue SignalException => e
raise e
rescue Exception => e
#halt the exception's progress
end
EDIT: This code works, as long as you get the class of the exception you want to trap correct. That's either SystemExit, Interrupt, or IRB::Abort as below.
The problem is that when a Ruby program ends, it does so by raising SystemExit. When a control-C comes in, it raises Interrupt. Since both SystemExit and Interrupt derive from Exception, your exception handling is stopping the exit or interrupt in its tracks. Here's the fix:
Wherever you can, change
rescue Exception => e
# ...
end
to
rescue StandardError => e
# ...
end
for those you can't change to StandardError, re-raise the exception:
rescue Exception => e
# ...
raise
end
or, at the very least, re-raise SystemExit and Interrupt
rescue SystemExit, Interrupt
raise
rescue Exception => e
#...
end
Any custom exceptions you have made should derive from StandardError, not Exception.
If you can wrap your whole program you can do something like the following:
trap("SIGINT") { throw :ctrl_c }
catch :ctrl_c do
begin
sleep(10)
rescue Exception
puts "Not printed"
end
end
This basically has CtrlC use catch/throw instead of exception handling, so unless the existing code already has a catch :ctrl_c in it, it should be fine.
Alternatively you can do a trap("SIGINT") { exit! }. exit! exits immediately, it does not raise an exception so the code can't accidentally catch it.
If you can't wrap your whole application in a begin ... rescue block (e.g., Thor) you can just trap SIGINT:
trap "SIGINT" do
puts "Exiting"
exit 130
end
130 is a standard exit code.
I am using ensure to great effect! This is for things you want to have happen when your stuff ends no matter why it ends.
Handling Ctrl-C cleanly in Ruby the ZeroMQ way:
#!/usr/bin/env ruby
# Shows how to handle Ctrl-C
require 'ffi-rzmq'
context = ZMQ::Context.new(1)
socket = context.socket(ZMQ::REP)
socket.bind("tcp://*:5558")
trap("INT") { puts "Shutting down."; socket.close; context.terminate; exit}
puts "Starting up"
while true do
message = socket.recv_string
puts "Message: #{message.inspect}"
socket.send_string("Message received")
end
Source
Perhaps the most simple solution?
Signal.trap('INT') { exit }
This is what I use, it works. Put it somewhere before a possible user interaction.
Here, a more verbose solution, to print something to STDERR and exit:
Signal.trap('INT') { abort 'Interrupted by user' }
See here for difference between exit and abort.