I am trying to create many threads and return the result in a data structure and I read that Queue is thread-safe, but when I run the code it doesn't produce the expected result.
require 'thread'
class ThreadsTest
queue = Queue.new
threads = []
for i in 1..10
threads << Thread.new do
queue << i
end
end
threads.each { |t| t.join }
for i in 1..10
puts queue.pop()
end
end
The code prints: (always a little different)
4
4
4
4
10
10
10
10
10
10
I was expecting the numbers 1 through 10.
I have tried to synchronize it manually to no avail:
mutex = Mutex.new
for i in 1..10
threads << Thread.new do
mutex.synchronize do
queue << i
end
end
end
What am I missing?
Queue is thread-safe but your code is not. Just like variable queue, variable i is shared across your threads, so the threads refer to the same variable while it is being changed in the loop.
To fix it, you can pass the variable to Thread.new, which turns it into a thread-local variable:
threads << Thread.new(i) do |i|
queue << i
end
The i within the block shadows the outer i, because they have the same name. You can use another name (e.g. |thread_i|) if you need both.
Output:
3
2
10
4
5
6
7
8
9
1
Related
Why the result is not from 1 to 10, but 10s only?
require 'thread'
def run(i)
puts i
end
while true
for i in 0..10
Thread.new{ run(i)}
end
sleep(100)
end
Result:
10
10
10
10
10
10
10
10
10
10
10
Why loop? I am running while loop, because later I want to iterate through the DB table all the time and echo any records that are retrieved from the DB.
The block that is passed to Thread.new may actually begin at some point in the future, and by that time the value of i may have changed. In your case, they all have incremented up to 10 prior to when all the threads actually run.
To fix this, use the form of Thread.new that accepts a parameter, in addition to the block:
require 'thread'
def run(i)
puts i
end
while true
for i in 0..10
Thread.new(i) { |j| run(j) }
end
sleep(100)
end
This sets the block variable j to the value of i at the time new was called.
#DavidGrayson is right.
You can see here a side effect in for loop. In your case i variable scope is whole your file. While you are expecting only a block in your for loop as a scope. Actually this is wrong approach in idiomatic Ruby. Ruby gives you iterators for this job.
(1..10).each do |i|
Thread.new{ run(i)}
end
In this case scope of variable i will be isolated in block scope what means for each iteration you will get new local (for this block) variable i.
The problem is that you have created 11 threads that are all trying to access the same variable i which was defined by the main thread of your program. One trick to avoid that is to call Thread.new inside a method; then the variable i that the thread has access to is just the particular i that was passed to the method, and it is not shared with other threads. This takes advantage of a closure.
require 'thread'
def run(i)
puts i
end
def start_thread(i)
Thread.new { run i }
end
for i in 0..10
start_thread i
sleep 0.1
end
Result:
0
1
2
3
4
5
6
7
8
9
10
(I added the sleep just to guarantee that the threads run in numerical order so we can have tidy output, but you could take it out and still have a valid program where each thread gets the correct argument.)
I am calling the same Ruby function with a number of threads (for example 10 threads). Each thread passes different argument to function.
Example:
def test thread_no
puts "In thread no." + thread_no.to_s
end
num_threads = 6
threads=[]
for thread_no in 1..num_threads
puts "Creating thread no. "+thread_no.to_s
threads << Thread.new{test(thread_no)}
end
threads.each { |thr| thr.join }
Output:
Creating thread no. 1
Creating thread no. 2
Creating thread no. 3
Creating thread no. 4
In thread no.4
Creating thread no. 5
Creating thread no. 6
In thread no.6
In thread no.6
In thread no.6
In thread no.6
In thread no.6
Of course I want to get output: In thread no. 1 (2,3,4,5,6) Can I somehow achieve that this would work?
The problem is the for-loop. In Ruby, it reuses a single variable.
So all blocks of the thread bodies access the same variable. An this variable is 6 at the end of the loop. The thread itself may start only after the loop has ended.
You can solve this by using the each-loops. They are more cleanly implemented, each loop variable exists on its own.
(1..num_threads).each do | thread_no |
puts "Creating thread no. "+thread_no.to_s
threads << Thread.new{test(thread_no)}
end
Unfortunately, for loops in ruby are a source of surprises. So it is best to always use each loops.
Addition:
You an also give Thread.new one or several parameters, and these parameters get passed into the thread body block. This way you can make sure that the block uses no vars outside it's own scope, so it also works with for-loops.
threads << Thread.new(thread_no){|n| test(n) }
#Meier already mentioned the reason why for-end spits out different result than expected.
for loop is language syntax construction, it reuses the same local variable thread_no and thread_no yields 6 because your for loop ends before the last few threads start executing.
In order to get rid of such issue, you can keep a copy of the exact thread_no in an another scope - such as -
def test thread_no
puts "In thread no." + thread_no.to_s
end
num_threads = 6
threads = []
for thread_no in 1..num_threads
threads << -> (thread_no) { Thread.new { test(thread_no) } }. (thread_no)
end
threads.each { |thr| thr.join }
Say I have a method that includes a counter that outputs it's count to the screen on every tick.
Elsewhere in the program, a new version of this method is called, so they both/all run at once, have different counters, and update together with the tick. Is it possible to do this with Ruby? Normally creating another instance of an object is what I would do, I am still new to Ruby though and getting the hang of it.
I will edit with sample code of what I am trying to achieve later. I'm currently on a mobile without access to a computer.
Here I'm creating two instances of a Counter, both counters are initially set to 0. Then I launch them 3 seconds apart - each in its own thread. They start to print out numbers.
class Counter
def initialize
#counter = 0 # initial counter to 0
end
def run
loop do
# wait one second, print the counter and increase it
sleep 1
puts #counter
#counter += 1
end
end
end
threads = []
2.times do
# put each counter in a separate thread
threads << Thread.new do
counter = Counter.new
counter.run
end
sleep 3 # make a pause between launching counters
end
threads.each(&:join)
Output I get:
0 # first
1 # first
2 # first
0 # second
3 # first
1 # second
4 # first
2 # second
5 # first
The only trick here is to use Thread class, otherwise second counter will never start to work since the first counter will block the whole process.
You could use a queue and an external loop, something like:
class Counter
def initialize(start)
#count = start
end
def tick
#count += 1
puts #count
end
end
queue = []
queue << Counter.new(0)
queue << Counter.new(100)
5.times do |i|
puts "--- tick #{i} ---"
queue.each(&:tick)
sleep 1
end
Output:
--- tick 0 ---
1
101
--- tick 1 ---
2
102
--- tick 2 ---
3
103
--- tick 3 ---
4
104
--- tick 4 ---
5
105
Within the 5.times loop, tick is sent to each item in the queue. Note that the methods are called in the order the counters were added to the queue, i.e. they are not called simultaneously.
For your purpose you could use either Event loop, or Processes, or Threads. Because in common case Ruby will be blocked while method is executing (till it will return control with return).
class ThreadCounter
def run
#thread ||= Thread.new do
i = 0
while !#stop do
puts i+=1
sleep(1)
end
#stop = nil
end
end
def stop
#stop = true
#thread && #thread.join
end
end
counter1 = ThreadCounter.new
counter2 = ThreadCounter.new
counter1.run
counter2.run
# wait some time
counter1.stop
counter2.stop
Why the result is not from 1 to 10, but 10s only?
require 'thread'
def run(i)
puts i
end
while true
for i in 0..10
Thread.new{ run(i)}
end
sleep(100)
end
Result:
10
10
10
10
10
10
10
10
10
10
10
Why loop? I am running while loop, because later I want to iterate through the DB table all the time and echo any records that are retrieved from the DB.
The block that is passed to Thread.new may actually begin at some point in the future, and by that time the value of i may have changed. In your case, they all have incremented up to 10 prior to when all the threads actually run.
To fix this, use the form of Thread.new that accepts a parameter, in addition to the block:
require 'thread'
def run(i)
puts i
end
while true
for i in 0..10
Thread.new(i) { |j| run(j) }
end
sleep(100)
end
This sets the block variable j to the value of i at the time new was called.
#DavidGrayson is right.
You can see here a side effect in for loop. In your case i variable scope is whole your file. While you are expecting only a block in your for loop as a scope. Actually this is wrong approach in idiomatic Ruby. Ruby gives you iterators for this job.
(1..10).each do |i|
Thread.new{ run(i)}
end
In this case scope of variable i will be isolated in block scope what means for each iteration you will get new local (for this block) variable i.
The problem is that you have created 11 threads that are all trying to access the same variable i which was defined by the main thread of your program. One trick to avoid that is to call Thread.new inside a method; then the variable i that the thread has access to is just the particular i that was passed to the method, and it is not shared with other threads. This takes advantage of a closure.
require 'thread'
def run(i)
puts i
end
def start_thread(i)
Thread.new { run i }
end
for i in 0..10
start_thread i
sleep 0.1
end
Result:
0
1
2
3
4
5
6
7
8
9
10
(I added the sleep just to guarantee that the threads run in numerical order so we can have tidy output, but you could take it out and still have a valid program where each thread gets the correct argument.)
I want to create a script to calculate numbers in multiple threads. Each thread will calculate the powers of 2 but the first thread must start calculating from 2, the second from 4, and the third from 8, printing some text in-between.
Example:
Im a thread and these are my results
2
4
8
Im a thread and these are my results
4
8
16
Im a thread and these are my results
8
16
32
My fail code:
def loopa(s)
3.times do
puts s
s=s**2
end
end
threads=[]
num=2
until num == 8 do
threads << Thread.new{ loopa(num) }
num=num**2
end
threads.each { |x| puts "Im a thread and these are my results" ; x.join }
My fail results:
Im a thread and these are my results
8
64
4096
8
64
4096
8
64
4096
Im a thread and these are my results
Im a thread and these are my results
I suggest you read the "Threads and Processes" chapter Pragmatic Programmer's ruby book. Here's an old version online. The section called "Creating Ruby Threads" is especially relevant to your question.
To fix the problem, you need to change your Thread.new line to this:
threads << Thread.new(num){|n| loopa(n) }
Your version doesn't work because num is shared between threads, and may be changed by another thread. By passing the variable via a block, the block variable is no longer shared.
More Info
Also, there's an error in your math.
Output values will be:
Thread 1: 2 4 16
Thread 2: 4 16 256
Thread 3: 6 36 1296
"8" is never reached because the until condition quits as soon as it sees "8".
If you want clearer output, use this as the body of loopa:
3.times do
print "#{Thread.current}: #{s}\n"
s=s**2
end
This lets you distinguish the 3 threads. Note that it's better to use a print command with a newline-terminated string versus using puts without a newline, because the latter prints the newline as a separate instruction, which may be interrupted by another thread.
It's normal. Read what you write. Firstly you run 3 threads that are async so output will be in various of combinations of threads output. Then you write 'Im a thread and these are my results' and join each thread. Also remember that Ruby has only references. So if you pass num to thread and then change it it will change in all threads. To avoid it write:
threads = (1..3).map do |i|
puts "I'm starting thread no #{i}"
Thread.new { loopa(2**i) }
end
I feel the need to post a mathematically correct version:
def loopa(s)
3.times do
print "#{Thread.current}: #{s}\n"
s *= 2
end
end
threads=[]
num=2
while num <= 8 do
threads << Thread.new(num){|n| loopa(n) }
num *= 2
end
threads.each { |x| print "Im a thread and these are my results\n" ; x.join }
Bonus 1: threadless solution (naive)
power = 1
workers = 3
iterations = 3
(power ... power + workers).each do |pow|
worker_pow = 2 ** pow
puts "I'm a worker and these are my results"
iterations.times do |inum|
puts worker_pow
worker_pow *= 2
end
end
Bonus 2: threadless solution (cached)
power = 1
workers = 3
iterations = 3
cache_size = workers + iterations - 1
# generate all the values upfront
cache = []
(power ... power+cache_size).each do |i|
cache << 2**i
end
workers.times do |wnum|
puts "I'm a worker and these are my results"
# use a sliding-window to grab the part of the cache we want
puts cache[wnum,3]
end