# Spawn workers to consume items from the iterator's enumerator based on the current concurrency level.
def spawn_workers
EM.next_tick(start_worker = proc{
if #workers < #concurrency and !#ended
# p [:spawning_worker, :workers=, #workers, :concurrency=, #concurrency, :ended=, #ended]
#workers += 1
#process_next.call
EM.next_tick(start_worker)
end
})
nil
end
I read this part of code from EM interator which is used by EM-sychrony#fiberd_interator.
I have some basic idea of Eventmachin, but I'm not very clear about this kind of recursive usage of next_tick, could any one give me a explaination about this plz?
In my opinion, it's just like a loop while it is handled by EM, not "while" or "for". Am I right? And why this?
It's not really a recursive call, think of it as "scheduling a proc to happen a moment later",
EventMachine is basically an endless loop that does stuff scheduled to happen in the next iteration of the loop (next tick),
Imagine next_tick method as a command queueing mechanism,
spawn_workers method schedules the start_worker proc to happen on the next iteration of the event loop.
In the next EM loop iteration start_worker proc will be ran and a #process_next.call will happen which I assume spawns the worker and thus it happens that the first worker is instantiated, the command
EM.next_tick(start_worker)
schedules the same block to happen in next iteration of the EM loop until all workers are spawned.
This means that, for example, if 8 workers need to be instantiated, one worker at a time will be spawned in next 8 ticks of the event loop
Related
Four threads loop for 10 million times each. On each loop they push a number if the list is empty else they pop a number from the list.
list = []
threads = []
4.times do |i|
threads << Thread.new do
1e7.to_i.times do |i|
if list.empty?
list << i
else
list.pop
end
end
end
end
threads.each(&:join)
p list
Since the loop executes an even number of times, I would expect the list to be empty after all the threads execute.
However, sometimes the list contains the number 9999999.
I thought that Array in MRI Ruby is thread safe because of the GIL.
How does the race condition happen in spite of the GIL?
Having only one thread executed at the time does not mean that a thread always stops at a useful line for example at the end of a block before the next thread gets its execution-time.
In your example, it is possible that one thread reads and evaluates list.empty? and then has to wait for another thread. The other thread reads and evaluates list.empty? too and gets the same result as the first thread. After that, both threads will execute the same branch of the if condition because they saw the same state.
While learning asyncio I was trying this code:
import asyncio
from asyncio.coroutines import coroutine
#coroutine
def coro():
counter: int = 0
while True:
print("Executed" + str(counter))
counter += 1
yield
loop = asyncio.get_event_loop()
loop.run_until_complete(coro())
loop.run_forever()
print("Finished!")
I was expecting the coroutine to be executed only once because it contains a yield and should have returned control to the caller. The output I was expecting was:
Executed 0
Finished!
I was expecting this behaviour because I thought the loop was going to run the coroutine forever once every "frame" returning to the caller after each execution (something like a background thread but in a cooperative way). But instead, it runs the coroutine forever without returning?. Output is the following:
Executed 0
Executed 1
Executed 2
Executed 3
...
Could anyone explain why this happens instead of my expectations?
Cheers.
You have a couple of problems. When you call run_until_complete, it waits for coro to finish before moving on to your run_forever call. As you've defined it, coro never finishes. It contains an infinite loop that does nothing to break out of the loop. You need a break or a return somewhere inside the loop if you want to move on to the next step in your application.
Once you've done that, though, your next call is to run_forever, which, just as its name suggests, will run forever. And in this case it won't have anything to do because you've scheduled nothing else with the event loop.
I was expecting the coroutine to be executed only once because it contains a yield and should have returned control to the caller.
Looking past the fact that your coroutine has no yield, awaiting (or yielding from depending on which syntax you choose to use) does not return control to the caller of run_until_complete or run_forever. It returns control to the event loop so that it can check for anything else that has been awaited and is ready to resume.
I am trying to wrap my head around Threads and Yielding in Ruby, and I have a question about how to pass a block to a yielding thread.
Specifically, I have a thread that is sleeping, and waiting to be told to do something, and I would like that Thread to execute a different block if told to (ie, it is sleeping, and if a user presses a button, do something besides sleep).
Say I have code like this:
window = Thread.new do
#thread1 = Thread.new do
# Do some cool stuff
# Decide it is time to sleep
until #told_to_wakeup
if block_given?
yield
end
sleep(1)
end
# At some point after #thread1 starts sleeping,
# a user might do something, so I want to execute
# some code in ##thread1 (unfortunately spawning a new thread
# won't work correctly in my case)
end
Is it possible to do that?
I tried using ##thread1.send(), but send was looking for a method name.
Thanks for taking the time to look at this!
Here's a simple worker thread:
queue = Queue.new
worker = Thread.new do
# Fetch an item from the work queue, or wait until one is available
while (work = queue.pop)
# ... Do something with work
end
end
queue.push(thing: 'to do')
The pop method will block until something is pushed into the queue.
When you're done you can push in a deliberately empty job:
queue.push(nil)
That will make the worker thread exit.
You can always expand on that functionality to do more things, or to handle more conditions.
I want to be able to stop and run specific thread in ruby in the following context:
thread_hash = Hash.new()
loop do
Thread.start(call.function) do |execute|
operation = execute.extract(some_value_from_incoming_message)
if thread_hash.has_key? operation
thread_hash[operation].run
elsif !thread_hash.has_key?
thread_hash[operation] = Thread.current
do_something_else_1
Thread.stop
do_something_else_2
Thread.stop
do_something_else_3
thread_hash.delete(operation)
else
exit
end
end
end
In human language script above acts as a server which receives a message, extracts some parameter from the incoming message. If that parameter is already in the thread_hash, suspended thread should be resumed.
If the parameter is not present in the thread_hash, parameter along with thread id is stored in the thread_hash, some function is executed and current thread is suspended until resumed in the new loop and again until do_something_else_3 function is executed and operation serviced in the current thread is removed from hash.
Can thread be resumed in Ruby based on thread id or should new thread be given name during start like
thr = Thread.start
and can be resumed only by this name like:
thr.run
Is the solution described above realistic? Could it cause some sort of leak/deadlock due to old thread resumption in the new thread or redundant threads are automatically taken care of by Ruby?
It sounds to me like you're trying to do everything in every thread: read input, run existing threads, store new threads, delete old threads. Why not break up the problem?
hash = {}
loop do
operation = get_value_from message
if hash[operation] and hash[operation].alive?
hash[operation].wakeup
else
hash[operation] = Thread.new do
do_something1
Thread.stop
do_something2
Thread.stop
do_something3
end
end
end
Instead of wrapping the whole contents of the loop in a thread, only thread the message processing code. That lets it run in the background while the loop goes back to waiting for a message. This solves any sort of race/deadlock problem since all of the thread management occurs in the main thread.
I have created a daemon in Ruby which has a counter incrementing inside of a loop. The loop does its business, then sleeps for 1 second, then continues. Simplified it's something like:
loop do
response = send_command
if response == 1
counter += 1
end
sleep(1)
end
Every 5 minutes I would like to call a method to database the counter value. I figure there are a few ways to do this. The initial way I considered was calling Time.now in the loop, examining it to match 5 minutes, 0 seconds, and if that matched, call the sql function. That seems terribly inefficient, however, and it could also miss a record if send_command took some time.
Another possibility may be to make available the counter variable, which could be called (and reset) via a socket. I briefly took a look at the Socket class, and that seems possible.
Is there an obvious/best way to do this that I'm missing?
If you just want to save every 5 minutes, you could just use a Thread. Something like:
Thread.new do
save_value_in_the_db(counter)
sleep 5*60
end
Note that the thread have access to counter if it is defined in the same block as the loop. you could also use an object and have the #counter declared insidd.
If you prefer to access remotely, you can do it with a socket or use a drb approach, that is probably easier. This drb tutorial seem to fit your requirements: http://ruby.about.com/od/advancedruby/a/drb.htm
I'd have the counter be updated every time through the loop, then periodically have something read that and update the database.
That makes a simpler main loop because it doesn't have to pay attention to how long it's needed to wait before exporting the value.
And, it's very common and normal to have a periodic task that samples a value and does something with it.
Creating a simple socket would work well. Ruby's Socket code RDoc has some samples for echo servers that might get you started.