Ruby, catching library thread exceptions? - ruby

I'm using a gem that's throwing an exception in a background thread as below. I'd like to catch this exception but not sure about how to go about it. How would one go about handling exceptions in library threads?
#this class is in my code
class MQTT
def self.connect
#client = Client.connect(options)
end
ende
This class is in the library which is packaged as a gem, so I technically don't have access to it:
class Client
def self.connect(*args, &block)
client = Client.new(*args)
client.connect(&block)
return client
end
def connect(clientid=nil)
# Start packet reading thread
#read_thread = Thread.new(Thread.current) do |parent|
Thread.current[:parent] = parent
loop { receive_packet }
end
end
def receive_packet
begin
# Poll socket - is there data waiting?
result = IO.select([#socket], nil, nil, SELECT_TIMEOUT)
# Pass exceptions up to parent thread
rescue Exception => exp
unless #socket.nil?
#socket.close
#socket = nil
end
Thread.current[:parent].raise(exp)
end
end
end

I think you have 3 options.
You could return the exception to the calling thread:
def receive_packet
raise "Exception in #{Thread.current}"
rescue Exception => exp
return exp
end
t1 = Thread.new do
receive_packet
end
puts "t1: #{t1.value.inspect}"
You could catch the exception on joining the thread (note you could reraise here or use an ensure block to make sure your socket is closed):
def receive_packet
raise "Exception in #{Thread.current}"
rescue Exception => exp
# reraise the exception
raise exp
end
t = Thread.new do
receive_packet
end
begin
t.join
rescue => e
puts "Exception caught from joined thread #{e.message} "
end
or you set #abort_on_exception = true so that exceptions kill all threads:
Thread.abort_on_exception = true
begin
Thread.new do
receive_packet
end
sleep 1
rescue => e
puts "Exception raised immediately to main thread: #{e.message}"
end
Update Based on what you have above and your comment I guess you need to wait for the threads calling receive_packet to finish. So you would have to join them:
class Client
def self.connect(*args, &block)
client = Client.new(*args)
client.connect(&block)
return client
end
def initialize(args)
#count = 0
end
def connect(clientid=nil)
puts "Connecting. Thread.current is #{Thread.current}"
# Start packet reading thread
#read_thread = Thread.new(Thread.current) do |parent|
Thread.current[:parent] = parent
loop { receive_packet }
end
end
def receive_packet
begin
# Poll socket - is there data waiting?
# result = IO.select([#socket], nil, nil, SELECT_TIMEOUT)
sleep 0.1
#count += 1
puts "count is now #{#count}"
if #count == 3
raise "WOOT: #{#count}"
end
# Pass exceptions up to parent thread
rescue Exception => exp
unless #socket.nil?
#socket.close
#socket = nil
end
puts "Reraising error #{exp.inspect} from #{Thread.current} to #{Thread.current[:parent]}"
Thread.current[:parent].raise(exp)
end
end
end
class MQTT
def self.connect
#client = Client.connect(options = {})
end
end
begin
MQTT.connect
Thread.list.each do |t|
# Wait for the thread to finish if it isn't this thread (i.e. the main thread).
t.join if t != Thread.current
end
rescue => e
puts "Exception from child thread: #{e.inspect}"
end

Related

Ruby: how to handler thread exception?

my code here...
require 'thread'
$temp = Thread.new do
loop do
puts 'loop me'
begin
puts "try thread"
raise Exception.new('QwQ') if rand > 0.5
puts "skip try"
rescue
puts "QwQ"
end
sleep(0.5)
end
puts '...WTF'
end
loop do
puts "runner #{Thread.list.length} #{$temp.status}"
sleep(2)
end
how to keep runner and loop thread running? and how to fix it like this code?
I tried like Thread.abort_on_exception , but it will kill the process...
Catch the exception inside the thread, and set the error in a variable accessible by the main thread (for testing you could use a global variable like so: $thread_error).
If the error-variable exists, then raise it from the main thread.
You could also use a queue to communicate between the threads, but then it wouldn't be able to utilize multiple threads.
require 'thread'
$temp = Thread.new do
begin
loop do
puts 'loop me'
begin
puts "try thread"
raise Exception.new('QwQ') if rand > 0.5
puts "skip try"
rescue
puts "QwQ"
end
sleep(0.5)
end
puts '...WTF'
rescue Exception => e
$thread_error = e
raise e
end
end
loop do
puts "runner #{Thread.list.length} #{$temp.status}"
raise $thread_error if $thread_error
sleep(2)
end

Why do i never get to on_connect in ShadyHttpClient

So witness and observe the following code, my questions is why do i never make it to the on_connect after starting the cool.io loop in send_to_server, the l.run should fire off the request as per the documented example on the github, and how the code handles incoming connections in module Server #socket.attach(l)
l.run
which does work and accepts the incoming data and sends it to my parser, which does work and fires off all the way up until the aforementioned send_to_server. So what is going on here?
require 'cool.io'
require 'http/parser'
require 'uri'
class Hash
def downcase_key
keys.each do |k|
store(k.downcase, Array === (v = delete(k)) ? v.map(&:downcase_key) : v)
end
self
end
end
module ShadyProxy
extend self
module ClientParserCallbacks
extend self
def on_message_complete(conn)
lambda do
puts "on_message_complete"
PluginHooks.before_request_to_server(conn)
end
end
def on_headers_complete(conn)
lambda do |headers|
conn.headers = headers
end
end
def on_body(conn)
lambda do |chunk|
conn.body << chunk
end
end
end
module PluginHooks
extend self
def before_request_to_server(conn)
# modify request here
conn.parser.headers.delete "Proxy-Connection"
conn.parser.headers.downcase_key
send_to_server(conn)
end
def send_to_server(conn)
parser = conn.parser
uri = URI::parse(parser.request_url)
l = Coolio::Loop.default
puts uri.scheme + "://" + uri.host
c = ShadyHttpClient.connect(uri.scheme + "://" + uri.host,uri.port).attach(l)
c.connection_reference = conn
c.request(parser.http_method,uri.request_uri)
l.run
end
def before_reply_to_client(conn)
end
end
class ShadyHttpClient < Coolio::HttpClient
def connection_reference=(conn)
puts "haz conneciton ref"
#connection_reference = conn
end
def connection_reference
#connection_reference
end
def on_connect
super
#never gets here
#headers = nil
#body = ''
#buffer = ''
end
def on_connect_failed
super
# never gets here either
end
def on_response_header(header)
#headers = header
end
def on_body_data(data)
puts "on data?"
#body << data
STDOUT.write data
end
def on_request_complete
puts "Headers"
puts #headers
puts "Body"
puts #body
end
def on_error(reason)
STDERR.puts "Error: #{reason}"
end
end
class ShadyProxyConnection < Cool.io::TCPSocket
attr_accessor :headers, :body, :buffer, :parser
def on_connect
#headers = nil
#body = ''
#buffer = ''
#parser = Http::Parser.new
#parser.on_message_complete = ClientParserCallbacks.on_message_complete(self)
#parser.on_headers_complete = ClientParserCallbacks.on_headers_complete(self)
#parser.on_body = ClientParserCallbacks.on_body(self)
end
def on_close
puts "huh?"
end
def on_read(data)
#buffer << data
#parser << data
end
end
module Server
def run(opts)
begin
# Start our server to handle connections (will raise things on errors)
l = Coolio::Loop.new
#socket = Cool.io::TCPServer.new(opts[:host],opts[:port], ShadyProxy::ShadyProxyConnection)
#socket.attach(l)
l.run
# Handle every request in another thread
loop do
Thread.new s = #socket.accept
end
# CTRL-C
rescue Interrupt
puts 'Got Interrupt..'
# Ensure that we release the socket on errors
ensure
if #socket
#socket.close
puts 'Socked closed..'
end
puts 'Quitting.'
end
end
module_function :run
end
end
ShadyProxy::Server.run(:host => '0.0.0.0',:port => 1234)

How to resume from rescue clause in Ruby?

How can I write resuming into loops in Ruby? Here is a sample code.
#!/usr/bin/ruby
#
a = [1,2,3,4,5]
begin
a.each{|i|
puts i
if( i==4 ) then raise StandardError end # Dummy exception case
}
rescue =>e
# Do error handling here
next # Resume into the next item in 'begin' clause
end
However, when running, Ruby returns the error message
test1.rb:13: Invalid next
test1.rb: compile error (SyntaxError)
I'm using Ruby 1.9.3.
You should use retry instead of next; But this will cause infinite loop (retry restart from the beginning of the begin)
a = [1,2,3,4,5]
begin
a.each{|i|
puts i
if i == 4 then raise StandardError end
}
rescue =>e
retry # <----
end
If you want skip an item, and continue to next item, catch the exception inside the loop.
a = [1,2,3,4,5]
a.each{|i|
begin
puts i
if i == 4 then raise StandardError end
rescue => e
end
}
Move your exception catching into the each block e.g:
a = [1,2,3,4,5]
a.each do |i|
puts i
begin
# Dummy exception case
if( i==4 ) then raise StandardError end
rescue =>e
# Do error handling here
end
end

Handling exceptions raised in a Ruby thread

I am looking for a solution of classic problem of exception handling. Consider following piece of code:
def foo(n)
puts " for #{n}"
sleep n
raise "after #{n}"
end
begin
threads = []
[5, 15, 20, 3].each do |i|
threads << Thread.new do
foo(i)
end
end
threads.each(&:join)
rescue Exception => e
puts "EXCEPTION: #{e.inspect}"
puts "MESSAGE: #{e.message}"
end
This code catches the exception after 5 seconds.
But if I change the array as [15, 5, 20, 3], above code catch the exception after 15 seconds. In short, it always catch the exception raised in first thread.
Any idea, why so. Why doesn't it catch the exception after 3 seconds each time? How do I catch the first raised exception by any thread?
If you want any unhandled exception in any thread to cause the interpreter to exit, you need to set Thread::abort_on_exception= to true. Unhandled exception cause the thread to stop running. If you don't set this variable to true, exception will only be raised when you call Thread#join or Thread#value for the thread. If set to true it will be raised when it occurs and will propagate to the main thread.
Thread.abort_on_exception=true # add this
def foo(n)
puts " for #{n}"
sleep n
raise "after #{n}"
end
begin
threads = []
[15, 5, 20, 3].each do |i|
threads << Thread.new do
foo(i)
end
end
threads.each(&:join)
rescue Exception => e
puts "EXCEPTION: #{e.inspect}"
puts "MESSAGE: #{e.message}"
end
Output:
for 5
for 20
for 3
for 15
EXCEPTION: #<RuntimeError: after 3>
MESSAGE: after 3
Note: but if you want any particular thread instance to raise exception this way there are similar abort_on_exception= Thread instance method:
t = Thread.new {
# do something and raise exception
}
t.abort_on_exception = true
Thread.class_eval do
alias_method :initialize_without_exception_bubbling, :initialize
def initialize(*args, &block)
initialize_without_exception_bubbling(*args) {
begin
block.call
rescue Exception => e
Thread.main.raise e
end
}
end
end
Postponed exceptions processing (Inspired by #Jason Ling)
class SafeThread < Thread
def initialize(*args, &block)
super(*args) do
begin
block.call
rescue Exception => e
#exception = e
end
end
end
def join
raise_postponed_exception
super
raise_postponed_exception
end
def raise_postponed_exception
Thread.current.raise #exception if #exception
end
end
puts :start
begin
thread = SafeThread.new do
raise 'error from sub-thread'
end
puts 'do something heavy before joining other thread'
sleep 1
thread.join
rescue Exception => e
puts "Caught: #{e}"
end
puts 'proper end'
This will wait for the first thread to either raise or return (and re-raise):
require 'thwait'
def wait_for_first_block_to_complete(*blocks)
threads = blocks.map do |block|
Thread.new do
block.call
rescue StandardError
$!
end
end
waiter = ThreadsWait.new(*threads)
value = waiter.next_wait.value
threads.each(&:kill)
raise value if value.is_a?(StandardError)
value
end
Jason Ling's answer will miss out any arguments passed to Thread.new. This will break Puma and other gems. To avoid this problem, you can use:
Thread.class_eval do
alias_method :initialize_without_exception_bubbling, :initialize
def initialize(*args, &block)
initialize_without_exception_bubbling(*args) {
begin
block.call(*args)
rescue Exception => e
Thread.main.raise e
end
}
end
end

Deadlock in ThreadPool

I couldn't find a decent ThreadPool implementation for Ruby, so I wrote mine (based partly on code from here: http://web.archive.org/web/20081204101031/http://snippets.dzone.com:80/posts/show/3276 , but changed to wait/signal and other implementation for ThreadPool shutdown. However after some time of running (having 100 threads and handling about 1300 tasks), it dies with deadlock on line 25 - it waits for a new job there. Any ideas, why it might happen?
require 'thread'
begin
require 'fastthread'
rescue LoadError
$stderr.puts "Using the ruby-core thread implementation"
end
class ThreadPool
class Worker
def initialize(callback)
#mutex = Mutex.new
#cv = ConditionVariable.new
#callback = callback
#mutex.synchronize {#running = true}
#thread = Thread.new do
while #mutex.synchronize {#running}
block = get_block
if block
block.call
reset_block
# Signal the ThreadPool that this worker is ready for another job
#callback.signal
else
# Wait for a new job
#mutex.synchronize {#cv.wait(#mutex)} # <=== Is this line 25?
end
end
end
end
def name
#thread.inspect
end
def get_block
#mutex.synchronize {#block}
end
def set_block(block)
#mutex.synchronize do
raise RuntimeError, "Thread already busy." if #block
#block = block
# Signal the thread in this class, that there's a job to be done
#cv.signal
end
end
def reset_block
#mutex.synchronize {#block = nil}
end
def busy?
#mutex.synchronize {!#block.nil?}
end
def stop
#mutex.synchronize {#running = false}
# Signal the thread not to wait for a new job
#cv.signal
#thread.join
end
end
attr_accessor :max_size
def initialize(max_size = 10)
#max_size = max_size
#workers = []
#mutex = Mutex.new
#cv = ConditionVariable.new
end
def size
#mutex.synchronize {#workers.size}
end
def busy?
#mutex.synchronize {#workers.any? {|w| w.busy?}}
end
def shutdown
#mutex.synchronize {#workers.each {|w| w.stop}}
end
alias :join :shutdown
def process(block=nil,&blk)
block = blk if block_given?
while true
#mutex.synchronize do
worker = get_worker
if worker
return worker.set_block(block)
else
# Wait for a free worker
#cv.wait(#mutex)
end
end
end
end
# Used by workers to report ready status
def signal
#cv.signal
end
private
def get_worker
free_worker || create_worker
end
def free_worker
#workers.each {|w| return w unless w.busy?}; nil
end
def create_worker
return nil if #workers.size >= #max_size
worker = Worker.new(self)
#workers << worker
worker
end
end
Ok, so the main problem with the implementation is: how to make sure no signal is lost and avoid dead locks ?
In my experience, this is REALLY hard to achieve with condition variables and mutex, but easy with semaphores. It so happens that ruby implement an object called Queue (or SizedQueue) that should solve the problem. Here is my suggested implementation:
require 'thread'
begin
require 'fasttread'
rescue LoadError
$stderr.puts "Using the ruby-core thread implementation"
end
class ThreadPool
class Worker
def initialize(thread_queue)
#mutex = Mutex.new
#cv = ConditionVariable.new
#queue = thread_queue
#running = true
#thread = Thread.new do
#mutex.synchronize do
while #running
#cv.wait(#mutex)
block = get_block
if block
#mutex.unlock
block.call
#mutex.lock
reset_block
end
#queue << self
end
end
end
end
def name
#thread.inspect
end
def get_block
#block
end
def set_block(block)
#mutex.synchronize do
raise RuntimeError, "Thread already busy." if #block
#block = block
# Signal the thread in this class, that there's a job to be done
#cv.signal
end
end
def reset_block
#block = nil
end
def busy?
#mutex.synchronize { !#block.nil? }
end
def stop
#mutex.synchronize do
#running = false
#cv.signal
end
#thread.join
end
end
attr_accessor :max_size
def initialize(max_size = 10)
#max_size = max_size
#queue = Queue.new
#workers = []
end
def size
#workers.size
end
def busy?
#queue.size < #workers.size
end
def shutdown
#workers.each { |w| w.stop }
#workers = []
end
alias :join :shutdown
def process(block=nil,&blk)
block = blk if block_given?
worker = get_worker
worker.set_block(block)
end
private
def get_worker
if !#queue.empty? or #workers.size == #max_size
return #queue.pop
else
worker = Worker.new(#queue)
#workers << worker
worker
end
end
end
And here is a simple test code:
tp = ThreadPool.new 500
(1..1000).each { |i| tp.process { (2..10).inject(1) { |memo,val| sleep(0.1); memo*val }; print "Computation #{i} done. Nb of tasks: #{tp.size}\n" } }
tp.shutdown
You can try the work_queue gem, designed to coordinate work between a producer and a pool of worker threads.
I'm slightly biased here, but I would suggest modelling this in some process language and model check it. Freely available tools are, for example, the mCRL2 toolset (using a ACP-based language), the Mobility Workbench (pi-calculus) and Spin (PROMELA).
Otherwise I would suggest removing every bit of code that is not essential to the problem and finding a minimal case where the deadlock occurs. I doubt that it the 100 threads and 1300 tasks are essential to get a deadlock. With a smaller case you can probably just add some debug prints which provide enough information the solve the problem.
Ok, the problem seems to be in your ThreadPool#signal method. What may happen is:
1 - All your worker are busy and you try to process a new job
2 - line 90 gets a nil worker
3 - a worker get freed and signals it, but the signal is lost as the ThreadPool is not waiting for it
4 - you fall on line 95, waiting even though there is a free worker.
The error here is that you can signal a free worker even when nobody is listening. This ThreadPool#signal method should be:
def signal
#mutex.synchronize { #cv.signal }
end
And the problem is the same in the Worker object. What might happen is:
1 - The Worker just completed a job
2 - It checks (line 17) if there is a job waiting: there isn't
3 - The thread pool send a new job and signals it ... but the signal is lost
4 - The worker wait for a signal, even though it is marked as busy
You should put your initialize method as:
def initialize(callback)
#mutex = Mutex.new
#cv = ConditionVariable.new
#callback = callback
#mutex.synchronize {#running = true}
#thread = Thread.new do
#mutex.synchronize do
while #running
block = get_block
if block
#mutex.unlock
block.call
#mutex.lock
reset_block
# Signal the ThreadPool that this worker is ready for another job
#callback.signal
else
# Wait for a new job
#cv.wait(#mutex)
end
end
end
end
end
Next, the Worker#get_block and Worker#reset_block methods should not be synchronized anymore. That way, you cannot have a block assigned to a worker between the test for a block and the wait for a signal.
Top commenter's code has helped out so much over the years. Here it is updated for ruby 2.x and improved with thread identification. How is that an improvement? When each thread has an ID, you can compose ThreadPool with an array which stores arbitrary information. Some ideas:
No array: typical ThreadPool usage. Even with the GIL it makes threading dead easy to code and very useful for high-latency applications like high-volume web crawling,
ThreadPool and Array sized to number of CPUs: easy to fork processes to use all CPUs,
ThreadPool and Array sized to number of resources: e.g., each array element represents one processor across a pool of instances, so if you have 10 instances each with 4 CPUs, the TP can manage work across 40 subprocesses.
With these last two, rather than thinking about threads doing work think about the ThreadPool managing subprocesses that are doing the work. The management task is lightweight and when combined with subprocesses, who cares about the GIL.
With this class, you can code up a cluster based MapReduce in about a hundred lines of code! This code is beautifully short although it can be a bit of a mind-bend to fully grok. Hope it helps.
# Usage:
#
# Thread.abort_on_exception = true # help localize errors while debugging
# pool = ThreadPool.new(thread_pool_size)
# 50.times {|i|
# pool.process { ... }
# or
# pool.process {|id| ... } # worker identifies itself as id
# }
# pool.shutdown()
class ThreadPool
require 'thread'
class ThreadPoolWorker
attr_accessor :id
def initialize(thread_queue, id)
#id = id # worker id is exposed thru tp.process {|id| ... }
#mutex = Mutex.new
#cv = ConditionVariable.new
#idle_queue = thread_queue
#running = true
#block = nil
#thread = Thread.new {
#mutex.synchronize {
while #running
#cv.wait(#mutex) # block until there is work to do
if #block
#mutex.unlock
begin
#block.call(#id)
ensure
#mutex.lock
end
#block = nil
end
#idle_queue << self
end
}
}
end
def set_block(block)
#mutex.synchronize {
raise RuntimeError, "Thread is busy." if #block
#block = block
#cv.signal # notify thread in this class, there is work to be done
}
end
def busy?
#mutex.synchronize { ! #block.nil? }
end
def stop
#mutex.synchronize {
#running = false
#cv.signal
}
#thread.join
end
def name
#thread.inspect
end
end
attr_accessor :max_size, :queue
def initialize(max_size = 10)
#process_mutex = Mutex.new
#max_size = max_size
#queue = Queue.new # of idle workers
#workers = [] # array to hold workers
# construct workers
#max_size.times {|i| #workers << ThreadPoolWorker.new(#queue, i) }
# queue up workers (workers in queue are idle and available to
# work). queue blocks if no workers are available.
#max_size.times {|i| #queue << #workers[i] }
sleep 1 # important to give threads a chance to initialize
end
def size
#workers.size
end
def idle
#queue.size
end
# are any threads idle
def busy?
# #queue.size < #workers.size
#queue.size == 0 && #workers.size == #max_size
end
# block until all threads finish
def shutdown
#workers.each {|w| w.stop }
#workers = []
end
alias :join :shutdown
def process(block = nil, &blk)
#process_mutex.synchronize {
block = blk if block_given?
worker = #queue.pop # assign to next worker; block until one is ready
worker.set_block(block) # give code block to worker and tell it to start
}
end
end

Resources