If I put an object inside a thread, how do I call methods on that object. Example of what I want to do below & my current error.
undefined method `_method_name' for ["var", #Thread:0x00007f9b181edec0#b.rb:183
threads = {}
freq.each do |var|
threads[var] = Thread.new {object.new.method}
end
while true
threads.each do |thr|
thr.method_inside_the_object
end
end
When you write this:
Thread.new { ... }
The { ... } is a block that will be executed by the new thread. If you want the thread to do something interesting,
then you have to provide code (the ...) to do the interesting thing.
Typically, the original thread goes on to do something else concurrently with the new thread:
Thread.new { ... }
do_something_else()
Doing things concurrently (maybe even, in parallel) is the whole point of multi-threading after all.
Threads communicate by accessing shared objects... but it doesn't make any sense for one thread to look at a shared object until it knows the the the other thread has finished updating it. The simplest way is to join() the other thread.
t = Thread.new { ... }
do_something_else()
t.join() # this simply waits until the thread has finished.
Now about those shared objects. It's especially easy in Ruby.
shared_object = Hash.new()
t = Thread.new {
shared_object["a"] = ...
shared_object["b"] = ...
...
}
do_something_else()
t.join()
# after the join() call returns, it's safe to look in shared_object
# to see what the other thread left for us.
do_something_with(shared_object["a"])
...
There's a whole other issue that arises if you need the main thread to access shared_object concurrently with the new thread (i.e., before it calls t.join()). Google for "race condition", or "locking", or "mutual exclusion", or "mutex" for more information about why that's tricky, and how to do it safely.
Related
Running this
# in initialize
#queue = Queue.new
#threads = Array.new(NUM_THREADS) do
Thread.new do
until #queue.empty?
puts #queue.shift
end
end
end
# later in another method, calling
#threads.each { |t| puts t.alive? } # puts false
#queue.push('something else')
# new item is not processed by thread
How do I keep a Ruby thread alive so it can keep accepting stuff from a queue?
The issue is you initialize the threads before adding anything to the queue. The threads start and die before the #queue.push line is run.
If you want to keep alive the thread even if there is nothing in the queue, you can change the logic of the thread so it loops forever:
Thread.new do
loop do
if val = #queue.shift
puts val
end
end
end
You could decrease the CPU consumption by putting a sleep call inside the thread's loop, say it sleeps 0.1 seconds each iteration and thus it could process a max of 10 items per second. For example running the following in my Ruby REPL raises the process' CPU consumption from around 0 to 25% (which is undesirably high)
100.times { Thread.new { loop { } } }
But the following uses less than 1%:
100.times { Thread.new { loop { sleep 0.1 } } }
There are other ways to manage CPU consumption of background processes other than putting arbitrary sleep numbers in there: for example eventmachine, resque, or sidekiq.
You probably want to use Queue from Ruby core library.
Methods pop, deq and shift can be used to retrieve data from the queue.
With these methods, according to the documentation
If the queue is empty, the calling thread is suspended until data is
pushed onto the queue.
With Queue, your code snippet would look like so
#queue = Queue.new
#threads = Array.new(NUM_THREADS) do
Thread.new do
while (item = #queue.shift)
puts item
end
end
end
# later in another method
#threads.each { |t| puts t.alive? } # true
#queue.push('something else') # is printed out
Queue#shift keeps the thread waiting, until something is pushed into the queue. You still need the loop, so that after handling the item, the thread will stay alive, suspended, waiting for the next item.
I'm creating a lot of threads:
(1..255).each do |n|
Thread.new do
sleep(10) # does a lot of work
end
end
# at this point I need to make sure all the threads completed
I would've hoped I could add each thread to a ThreadGroup and call a function like wait_until_all_threads_complete on that ThreadGroup. But I don't see anything obvious in the Ruby docs.
Do I have to add each thread to an array and then iterate over each one calling thread.join? There must be an easier way for such an extremely common use case.
threads = (1..255).map do |n|
Thread.new do
sleep(10) # does a lot of work
end
end
threads.each do |thread|
thread.join
end
Use ThreadGroup#list
If you assign threads to a ThreadGroup with ThreadGroup#add, you can map Thread#join or other Thread methods onto each member of the group as returned by the ThreadGroup#list method. For example:
thread_group = ThreadGroup.new
255.times do
thread_group.add Thread.new { sleep 10 }
end
thread_group.list.map &:join
This will only join threads belonging to thread_group, rather than to ThreadGroup::Default.
In RSpec, I have function that creates a new thread, and inside that thread performs some action–in my case, calls TCPSocket#readline. Here's the function as it is right now:
def read
Thread.new do
while line = #socket.readline
#TODO: stuff
end
end
end
Due to thread scheduling, my test will fail if written as such:
it "reads from socket" do
subject.socket.should_receive(:readline)
subject.read
end
Currently the only way I know to hack around this is to use sleep 0.1. Is there a way to properly delay the test until that thread is running?
If your goal is to assert the system state is changed by the execution of your second thread, you should join on the second thread in your main test thread:
it "reads from socket" do
subject.socket.should_receive(:readline)
socket_thread = subject.read
socket_thread.join
end
This is a bit of a hack, but here's a before block you can use in case you'd like the thread to yield but be able to call join at the end of the thread.
before do
allow(Thread).to receive(:new).and_yield.and_return(Class.new { def join; end }.new)
end
Is it OK not to call Thread#join? In this case, I don't care if the thread blows up - I just want Unicorn to keep processing.
class MyMiddleware
def initialize(app)
#app = app
end
def call(env)
t = Thread.new { sleep 1 }
t.join # is it ok if I skip this?
#app.call env
end
end
Will I get "zombie threads" or something like that?
It's perfectly fine to not call join - in fact, join is often not needed at all with multithreaded code. You should only call join if you need to block until the new thread completes.
You won't get a "zombie" thread. The new thread will run until completion, then clean itself up for you.
In my main thread, I am trying to wait for two resources from two separate threads. The way I implemented is as below:
require 'thread'
def new_thread
Thread.current[:ready_to_go] = false
puts "in thread: new thread"
sleep(5)
puts "in thread: sleep finished"
Thread.current[:ready_to_go] = true
sleep(2)
puts "in thread: back to thread again!"
end
thread1 = Thread.new do
new_thread
end
thread2 = Thread.new do
new_thread
end
# the main thread wait for ready_to_go to start
while (!(thread1[:ready_to_go] && thread2[:ready_to_go]))
sleep(0.5)
end
puts "back to main!"
sleep(8)
puts "main sleep over!"
thread1.join
thread2.join
Is there any better way to implement this? I tried to use conditional variables: the two threads signal the conditional variables and the main thread waits for them. But the wait method requires a mutex in my main thread, which I am trying to avoid.
I'm not familiar with Ruby, but the first Google result for "Ruby wait for thread" says:
you can wait for a particular thread to finish by calling that thread's Thread#join method. The calling thread will block until the given thread is finished. By calling join on each of the requestor threads, you can make sure that all three requests have completed before you terminate the main program.
It's generally best to use synchronization methods to wait for something to complete, rather than looping until a particular state is reached.
One easy way would be to get a Queue (let's call it myqueue). It's threadsafe and located in the Thread module
Instead of
Thread.current[:ready_to_go] = true
... do
myqueue.push :ready_to_go
And then your main thread would be:
junk = myqueue.pop # wait for thread one to push
junk = myqueue.pop # wait for thread two to push
# go on with your work