I got a test example.
Here MyBoss should accomplish hard, but granulated work, despite some of its workes have to die.
require 'celluloid'
require 'celluloid/autostart'
class MyActor
include Celluloid
attr_reader :name, :peace_of_work
def initialize
#name = "actor_#{self.object_id}"
print "Created actor '#{#name}'\n"
end
def run(peace_of_work)
#peace_of_work = peace_of_work
sleep 0.1
raise "Actor '#{#name}' is dying" if rand(0..1.0) < 0.1
print "Actor '#{#name}' has #{peace_of_work}-th peace of work done\n"
end
end
class MyBoss
def initialize
end
def run work_peaces
#work_peaces = work_peaces
#actor_pool = MyActor.pool(size: 10, args: [])
work_peaces.each do |peace_of_work|
#actor_pool.async.run(peace_of_work)
end
end
end
if __FILE__ == $PROGRAM_NAME
boss = MyBoss.new
work_peaces = (0..999).to_a
boss.run(work_peaces)
sleep
end
Actors sporadically die. Obiously I need to redo screwed up work peaces. How can I trap actors deaths in case they are in a pool? Notice MyActor.pool(size: 10, args: [])
This is a known issue being resolved at the link below:
https://github.com/celluloid/celluloid-pool/issues/8
The gist is:
trap_exit already works for the Pool manager itself. When an actor dies, it needs to be taken out of the idle and busy sets of the Pool and a new actor needs to be spawned as idle ... otherwise the Pool becomes unstable.
A secondary method needs to be injected to run after Pool itself can correct its situation with a dead actor.
Related
I'm looking to extend a the simple circuter breaker written ruby to work across multiple thread...
And thus far I manage to accomplish something like this ..
## following is a simple cicruit breaker implementation with thread support.
## https://github.com/soundcloud/simple_circuit_breaker/blob/master/lib/simple_circuit_breaker.rb
class CircuitBreaker
class Error < StandardError
end
def initialize(retry_timeout=10, threshold=30)
#mutex = Mutex.new
#retry_timeout = retry_timeout
#threshold = threshold
reset!
end
def handle
if tripped?
raise CircuitBreaker::Error.new('circuit opened')
else
execute
end
end
def execute
result = yield
reset!
result
rescue Exception => exception
fail!
raise exception
end
def tripped?
opened? && !timeout_exceeded?
end
def fail!
#mutex.synchronize do
#failures += 1
if #failures >= #threshold
#open_time = Time.now
#circuit = :opened
end
end
end
def opened?
#circuit == :opened
end
def timeout_exceeded?
#open_time + #retry_timeout < Time.now
end
def reset!
#mutex.synchronize do
#circuit = :closed
#failures = 0
end
end
end
http_circuit_breaker = CircuitBreaker.new
http_circuit_breaker.handle { make_http_request }
but I'm not sure about few things ...
The multithreaded code has always puzzled me hence I'm not the entirely confident about the approach to say that the stuff seems correct.
Read operation are not under mutex:
While (I think, I have ensured that no data race condition every happens between two threads) mutex are applied for the write operation but the read operation is mutex free. Now, since there can be a scenario where a thread 1 has a held mutex while changing the #circuit or #failure variable but the other thread read the stale value.
So, I'm not able to think thorough does by achieving a full consistency(while applying the read lock) is worth a trade-off over here. Where consistency might be 100 % but the execution code as turn a bit slower because of the excessive lock.
it's unclear what you are asking, so i guess your post will be closed.
nevertheless, i think that the only thread-safe-way to implement a circuit-breaker would be to have the mutex around all data operartions which would result in a sequential flow, so it's basically useless.
otherwise you will have race-conditions like
thread-a starts (server does not respond immediately due to network issues)
thread-b starts (10 seconds later)
thread-b finishes all good
thread-a aborts due to a timeout -> opens circuit with stale data
a version that is mentioned in martin fowlers blog is a circuit-breaker in combination with a thread-pool: https://martinfowler.com/bliki/CircuitBreaker.html
Im need to retrieve data for account in parallel. I use celluloid for that
my code
class Daemon
def self.start
loop do
Account.all.each do |acc|
unless Celluloid::Actor[account.name]
supervise_as(account.name, account)
Celluloid::Actor[account.name].async(:perform)
end
sleep 10.minutes
end
end
def initialize(account)
#account = account
end
def perform
loop do
do_request
sleep 10
end
end
end
On start it always do parallel request for all members(bob, alice, etc).
But after some time it makes requests only for one user (alice for ex.)
PS in do_request used self-made redis queue
and in Celluloid::Actor.registed all accounts exists
I am implementing a simple program in Celluloid that ideally will run a few actors in parallel, each of which will compute something, and then send its result back to a main actor, whose job is simply to aggregate results.
Following this FAQ, I introduced a SupervisionGroup, like this:
module Shuffling
class AggregatorActor
include Celluloid
def initialize(shufflers)
#shufflerset = shufflers
#results = {}
end
def add_result(result)
#results.merge! result
#shufflerset = #shufflerset - result.keys
if #shufflerset.empty?
self.output
self.terminate
end
end
def output
puts #results
end
end
class EvalActor
include Celluloid
def initialize(shufflerClass)
#shuffler = shufflerClass.new
self.async.runEvaluation
end
def runEvaluation
# computation here, which yields result
Celluloid::Actor[:aggregator].async.add_result(result)
self.terminate
end
end
class ShufflerSupervisionGroup < Celluloid::SupervisionGroup
shufflers = [RubyShuffler, PileShuffle, VariablePileShuffle, VariablePileShuffleHuman].to_set
supervise AggregatorActor, as: :aggregator, args: [shufflers.map { |sh| sh.new.name }]
shufflers.each do |shuffler|
supervise EvalActor, as: shuffler.name.to_sym, args: [shuffler]
end
end
ShufflerSupervisionGroup.run
end
I terminate the EvalActors after they're done, and I also terminate the AggregatorActor when all of the workers are done.
However, the supervision thread stays alive and keeps the main thread alive. The program never terminates.
If I send .run! to the group, then the main thread terminates right after it, and nothing works.
What can I do to terminate the group (or, in group terminology, finalize, I suppose) after the AggregatorActor terminates?
What I did after all, is change the AggregatorActor to have a wait_for_results:
class AggregatorActor
include Celluloid
def initialize(shufflers)
#shufflerset = shufflers
#results = {}
end
def wait_for_results
sleep 5 while not #shufflerset.empty?
self.output
self.terminate
end
def add_result(result)
#results.merge! result
#shufflerset = #shufflerset - result.keys
puts "Results for #{result.keys.inspect} recorded, remaining: #{#shufflerset.inspect}"
end
def output
puts #results
end
end
And then I got rid of the SupervisionGroup (since I didn't need supervision, ie rerunning of actors that failed), and I used it like this:
shufflers = [RubyShuffler, PileShuffle, VariablePileShuffle, VariablePileShuffleHuman, RiffleShuffle].to_set
Celluloid::Actor[:aggregator] = AggregatorActor.new(shufflers.map { |sh| sh.new.name })
shufflers.each do |shuffler|
Celluloid::Actor[shuffler.name.to_sym] = EvalActor.new shuffler
end
Celluloid::Actor[:aggregator].wait_for_results
That doesn't feel very clean, it would be nice if there was a cleaner way, but at least this works.
The idea is that the input will be provided by the console, and will be identified with a unique 'id' as the first word of the input. A new thread is spawned when a new id is encountered, with the subsequent input being 'start'. The thread should close when an input with the same id will say 'close'. The ordering of the statements is random.
The trick here is to have a single thread (most easily the main thread) do all the reading. That thread will parse the command, start or stop threads, and dispatch commands to the threads. Each thread will have its own Queue from which it reads commands and into which the main thread puts the commands. Let's see how it might be done.
We'll start with a little module for the control commands, so that they're DRY:
module ControlCommands
START_COMMAND = 'start'
STOP_COMMAND = 'stop'
end
Now let's see the "main":
class Main
def initialize
#workers = Workers.new
#console = Console.new(#workers)
end
def run
#console.read_and_dispatch
#workers.join
end
end
Main.new.run
There's nothing much going on here. We make a console, and tell it to read commands and dispatch them to the workers. The console does not return from that until it has run out of input. The call to #workers.join make sure all the workers have finished working and properly shut down before the program exits.
Here's the Console class:
class Console
def initialize(workers)
#workers = workers
end
def read_and_dispatch
while input = gets
#workers.dispatch *parse_input(input)
end
end
private
def parse_input(input)
input.match(/^(\w+) *(.*)$/)[1..2]
end
end
read_and_dispatch is the main loop. All that it is responsible for is reading and parsing input. As long as there is input, it splits it into the worker name and command, and then tells the workers to handle the command.
Here's the Workers class:
class Workers
include ControlCommands
def initialize
#workers = {}
end
def dispatch(worker_name, command)
case command
when START_COMMAND
start_worker worker_name
when STOP_COMMAND
stop_worker worker_name
else
dispatch_worker worker_name, command
end
end
def join
#workers.each do |worker_name, worker|
worker.stop
worker.join
end
end
private
def start_worker(worker_name)
#workers[worker_name] = Worker.new(worker_name)
end
def stop_worker(worker_name)
#workers[worker_name].stop
end
def dispatch_worker(worker_name, command)
#workers[worker_name].dispatch command
end
end
This is where most of the meat is. This class creates workers (threads), stop them, and dispatches commands to them. Note that there is no error handling here: If you try to stop a thread that isn't started, start one that's already started, or dispatch a command to one that doesn't exist, the program will crash or misbehave. I'll leave the handling of those situations as an "exercise for the reader."
Here's the Worker class:
class Worker
include ControlCommands
def initialize(name)
#name = name
#commands = Queue.new
#thread = start_thread
end
def dispatch(command)
#commands.enq command
end
def stop
#commands.enq STOP_COMMAND
end
def join
#thread.join
end
private
def start_thread
Thread.new do
loop do
command = #commands.deq
break if command == STOP_COMMAND
process_command command
end
end
end
def process_command(command)
print "thread #{#name} received command #{command.inspect}\n"
end
end
This class contains the thread, and the queue used to communicate the between the main thread (the one reading the console) and the worker thread. That queue is also used to stop a thread, synchronously, by putting a STOP_COMMAND in the queue which the thread responds to by exiting. It is best, when you can afford to, to stop threads synchronously rather than asynchronously.
Here's a simple input file:
foo start
bar start
foo input
bar input
foo stop
bar stop
And the output when the program is presented with that input:
thread bar received command "input"
thread foo received command "input"
[EDIT NOTE: Noticed I had put the mutex creation in the constructor. Moved it and noticed no change.]
[EDIT NOTE 2: I changed the call to app.exec in a trial run to
while TRUE do
app.processEvents()
puts '."
end
I noticed that once the Soap4r service started running no process events ever got called again]
[EDIT NOTE 3: Created an associated question here: Thread lockup in ruby with Soap4r
I'm attempting to write a ruby program that receives SOAP commands to draw on a monitor (thus allowing remote monitor access). I've put together a simple test app to prototype the idea. The graphic toolkit is QT. I'm having what I assume is a problem with locking. I've added calls to test the methods in the server in the code shown. The server side that I'm testing right now is:
require 'rubygems'
require 'Qt4'
require 'thread'
require 'soap/rpc/standaloneserver'
class Box < Qt::Widget
def initialize(parent = nil)
super
setPalette(Qt::Palette.new(Qt::Color.new(250,0,0)))
setAutoFillBackground(true)
show
end
end
class SOAPServer < SOAP::RPC::StandaloneServer
##mutex = Mutex.new
def initialize(* args)
super
# Exposed methods
add_method(self, 'createWindow', 'x', 'y', 'width', 'length')
end
def createWindow(x, y, width, length)
puts 'received call'
windowID = 0
puts #boxList.length
puts #parent
##mutex.synchronize do
puts 'in lock'
box = Box.new(#parent)
box.setGeometry(x, y, width, length)
windowID = #boxList.push(box).length
print "This:", windowID, "--\n"
end
puts 'out lock'
return windowID
end
def postInitialize (parent)
#parent = parent
#boxList = Array.new
end
end
windowSizeX = 400
windowSizeY = 300
app = Qt::Application.new(ARGV)
mainwindow = Qt::MainWindow.new
mainwindow.resize(windowSizeX, windowSizeY)
mainwindow.show
puts 'Attempting server start'
myServer = SOAPServer.new('monitorservice', 'urn:ruby:MonitorService', 'localhost', 4004)
myServer.postInitialize(mainwindow)
Thread.new do
puts 'Starting?'
myServer.start
puts 'Started?'
end
Thread.new do
myServer.createWindow(10,0,10,10)
myServer.createWindow(10,30,10,10)
myServer.createWindow(10,60,10,10)
myServer.createWindow(10,90,10,10)
end
myServer.createWindow(10,10,10,10)
Thread.new do
app.exec
end
gets
Now when I run this I get the following output:
Attempting server start
Starting?
received call
0
#<Qt::MainWindow:0x60fea28>
in lock
received call
0
#<Qt::MainWindow:0x60fea28>
This:1--
in lock
This:2--
out lock
At that point I hang rather than recieving the total of five additions I expect. Qt does display the squares defined by "createWindow(10,0,10,10)" and "createWindow(10,10,10,10)". Given that "This:1--" and "This:2--" show within a nexted in/out lock pair I'm assuming I'm using mutex horribly wrong. This is my first time with threading in Ruby.