I wrote a test program. In short, it does the following:
executes some random useless calcuations (but long enough) in single thread and prints execution
time
executes same program in 6 threads and prints execution time
Now, in both cases execution time is the same. What am I doing wrong?
Here are the sources:
def time
start = Time.now
yield
p Time.now - start
end
range_limit = 9999
i_exponent = 9999
time do
array = (1 .. range_limit).map { |i | i**i_exponent }
p (array.inject(:+)/2)[0]
end
time do
first_thread = Thread.new do
arr = (1..range_limit/6).map { |i| i**i_exponent }
arr.inject(:+)
end
second_thread = Thread.new do
arr = (range_limit/6..range_limit*2/6).map { |i| i**i_exponent }
arr.inject(:+)
end
third_thread = Thread.new do
arr = (range_limit*2/6..range_limit*3/6).map { |i| i**i_exponent }
arr.inject(:+)
end
fourth_thread = Thread.new do
arr = (range_limit*3/6..range_limit*4/6).map { |i| i**i_exponent }
arr.inject(:+)
end
fifth_thread = Thread.new do
arr = (range_limit*4/6..range_limit*5/6).map { |i| i**i_exponent }
arr.inject(:+)
end
sixth_thread = Thread.new do
arr = (range_limit*5/6..range_limit).map { |i| i**i_exponent }
arr.inject(:+)
end
first_thread.join
second_thread.join
third_thread.join
fourth_thread.join
fifth_thread.join
sixth_thread.join
result = first_thread.value + second_thread.value + third_thread.value + fifth_thread.value + sixth_thread.value
p (result/2)[0]
end
Interpreters like Ruby and Python are not truly parallel while executing multiple threads - to protect the state of the interpreter, they have a global VM lock that doesn't allow simultaneous execution.
To get the benefit of threads, you need to find a way to execute non-Ruby code that runs without the global lock, use multiple processes instead of threads. Another option is to switch to a ruby implementation that doesn't have the global lock, such as JRuby or Rubinius.
Related
According to this post, i += 1 is thread safe in MRI Ruby because the preemption only happens at the end of function call, not somewhere between i += 1.
A repeatable test below shows that this is true:
But why while true do i += 1 end is not thread safe, as shown by the second test below where thread1 is preempted by thread2 when thread1 is still executing while true do i += 1 end ?
Please help.
Below are the code reference:
test one:
100.times do
i = 0
1000.times.map do
Thread.new {1000.times {i += 1}}
end.each(&:join)
puts i
end
test two:
t1 = Thread.new do
puts "#{Time.new} t1 running"
i = 0
while true do i += 1 end
end
sleep 4
t2 = Thread.new do
puts "#{Time.new} t2 running"
end
t1.join
t2.join
According to this post, i += 1 is thread safe in MRI
Not quite. The blog post states that method invocations are effectively thread-safe in MRI.
The abbreviated assignment i += 1 is syntactic sugar for:
i = i + 1
So we have an assignment i = ... and a method call i + 1. According to the blog post, the latter is thread-safe. But it also says that a thread-switch can occur right before returning the method's result, i.e. before the result is re-assigned to i:
i = i + 1
# ^
# here
Unfortunately this isn't easy do demonstrate from within Ruby.
We can however hook into Integer#+ and randomly ask the thread scheduler to pass control to another thread:
module Mayhem
def +(other)
Thread.pass if rand < 0.5
super
end
end
If MRI ensures thread-safety for the whole i += 1 statement, the above shouldn't have any effect. But it does:
Integer.prepend(Mayhem)
10.times do
i = 0
Array.new(10) { Thread.new { i += 1 } }.each(&:join)
puts i
end
Output:
5
7
6
4
4
8
4
5
6
7
If you want thread-safe code, don't rely on implementation details (those can change). In the above example, you could wrap the sensitive part in a Mutex#synchronize call:
Integer.prepend(Mayhem)
m = Mutex.new
10.times do
i = 0
Array.new(10) { Thread.new { m.synchronize { i += 1 } } }.each(&:join)
puts i
end
Output:
10
10
10
10
10
10
10
10
10
10
I work on task about schedule and have class and .yml file where i want to retrieve data as "start" point, "end" point, "price" etc. And in my class I have method, where i want to determine if stations equal to user choice:
class Train
require 'yaml'
def initialize(time)
#time = YAML.load_file(time)
end
def calc(begin:, end:)
ary = #time['time']
start = begin
stop = end
res = []
loop do
tmp = ary.find { |h| h['begin'] == start }
break unless tmp
res << tmp
start = tmp['end']
break if start == stop
end
end
end
But it always fails on condition
break unless tmp
For example if write instance variable
a = Train.new("my_path_to yml")
a.calc(begin: ':washington', end: ':tokyo')
It executes nothing. Even if I refactor loop block and write "for" iterator it throw "else" condition:
for i in ary
if i['begin'] == 'washington'
puts "good"
else
puts "no way"
end
end
Here is my .yml file
time:
-
begin: :washington
end: :briston
time: 6
price: 3
-
begin: :briston
end: :dallas
time: 4
price: 2
-
begin: :dallas
end: :tokyo
time: 3.5
price: 3
-
begin: :tokyo
end: :chicago
time: 3.5
price: 3
-
begin: :chicago
end: :dellawer
time: 3.5
price: 3
Thanks in advance!
Try this change, check comments in code:
def calc(begin_:, end_:) # <-- don't call variables begin or end, they are reserved words
ary = #time['time']
start = begin_
stop = end_
res = []
loop do
tmp = ary.find { |h| h['begin'] == start }
break unless tmp
res << tmp
start = tmp['end']
break if start == stop
end
res # <-- return something
end
Call as:
train.calc(begin_: :washington, end_: :tokyo)
#=> [{"begin"=>:washington, "end"=>:briston, "time"=>6, "price"=>3}, {"begin"=>:briston, "end"=>:dallas, "time"=>4, "price"=>2}, {"begin"=>:dallas, "end"=>:tokyo, "time"=>3.5, "price"=>3}]
Take care not messing up strings with symbols.
ary.each do |i| # for i in ary <-- pythonic! :)
if i['begin'] == :washington # <-- should be a symbol to pass, not a string
puts "good"
else
puts "no way"
end
end
I am running multiple threads, and when one of the threads sets the global function '$trade_executed' to true I want it to kill all other threads and remove them from the global '$threads' array.
Then I restart the thread creation process.
Below is a simplified version of my codebase.
3 Threads are created and it looks like 2 threads are deleted but a third thread stays. (for reasons unknown)
Ideally this script would never print '2' or '3' because it would always trigger at '1' minute and kill all threads and reset.
*
thr.exit is preferred. I don't want any code pushed from other threads with a thr.join after $trade_executed is set
require 'thread'
class Finnean
def initialize
#lock = Mutex.new
end
def digger(minute)
sleep(minute * 60)
coco(minute)
end
def coco(minute)
#lock.synchronize {
puts "coco #{minute}"
$threads.each do |thr|
next if thr == Thread.current
thr.exit
end
$trade_executed = true
Thread.current.exit
}
end
end
minutes = [1, 2, 3]
$threads = Array.new
$trade_executed = false
abc = Finnean.new
def start_threads(minutes, abc)
minutes.each do |minute|
$threads << Thread.new {abc.digger(minute)}
puts minute
end
end
start_threads(minutes, abc)
while true
if $trade_executed != false then
count = 0
$threads.map! do |thr|
count += 1
puts "#{thr} & #{thr.status}"
thr.exit
$threads.delete(thr)
puts "Iteration #{count}"
end
count = 0
$threads.each do |thr|
count += 1
puts "#{thr}" ##{thr.status}
puts "Threads Still Left: #{count}"
end
$trade_executed = false
abc = Finnean.new
start_threads(minutes, abc)
end
end
Why not make a thread killer that you keep locked up until the first one finishes:
# Create two variables that can be passed in to the Thread.new block closure
threads = [ ]
killer = nil
# Create 10 threads, each of which waits a random amount of time before waking up the thread killer
10.times do |n|
threads << Thread.new do
sleep(rand(2..25))
puts "Thread #{n} finished!"
killer.wakeup
end
end
# Define a thread killer that will call `kill` on all threads, then `join`
killer = Thread.new(threads) do
Thread.stop
threads.each do |thread|
puts "Killing #{thread}"
thread.kill
thread.join
end
end
# The killer will run last, so wait for that to finish
killer.join
You can't force a thread to exit, but you can kill it. That generates an exception you could rescue and deal with as necessary.
How can I make these loops parallel with multithreading capability of ruby?
1.
from = 'a' * 1
to = 'z' * 3
("#{from}".."#{to}").each do |combination|
# ...
end
2.
##alphabetSet_tr.length.times do |i|
##alphabetSet_tr.length.times do |j|
##alphabetSet_tr.length.times do |k|
combination = ##alphabetSet_tr[i] + ##alphabetSet_tr[j] + ##alphabetSet_tr[k]
end
end
end
Note: ##alphabetSet_tr is an array which has 29 items
If you want to utilize your cores, you can use a Queue to divide the workload between a constant number of threads:
require 'thread'
queue = Queue.new
number_of_cores = 32
threads = (0..number_of_cores).map do
Thread.new do
combination = queue.pop
while combination
# do stuff with combination...
# take the next combination from the queue
combination = queue.pop
end
end
end
# fill the queue:
("#{from}".."#{to}").each do |combination|
queue << combination
end
# drain the threads
number_of_cores.times { queue << nil }
threads.each { |t| t.join }
If you fear that the size of the queue itself would be an issue - you can use SizedQueue which will block push operations if it gets larger than a certain size - capping its memory usage -
queue = SizedQueue.new(10000)
from = 'a' * 1
to = 'z' * 3
threads = ("#{from}".."#{to}").map do |combination|
Thread.new do
# ...
end
end
# In case you want the main thread waits all the child threads.
threads.each(&:join)
I am creating threads in a for loop, and I want to use the for loop's i as the name for each particular thread. When I run this, instead of getting 1,2 or 2,1, I am getting 2,2. Is there a better/safer way to pass variables into a thread?
ts = []
for i in 1..2 do
ts.push( Thread.new(i) do
x = i
puts x
end)
end
ts.each do |t|
t.join()
end
Your problem is that the i you are referring to is not a block variable passed to the thread but is the i defined outside of the thread. You need to add |i| to it, and you will get either 1, 2 or 2, 1.
ts = []
for i in 1..2 do
ts.push( Thread.new(i) do |i|
x = i
puts x
end)
end
ts.each do |t|
t.join()
end
By the way, a more rubyish way to write is:
ts = (1..2).map do |i|
Thread.new(i) do |i|
puts i
end
end.each(&:join)
If you are looking for a unique name for each thread, I suggest using the object id of the thread.
ts = (1..2).map do
Thread.new do
puts Thread.current.object_id
end
end.each(&:join)
You can pass variable through the block
ts = []
for i in 1..2 do
ts.push( Thread.new(i) do |i|
x = i
puts x
end)
end
ts.each do |t|
t.join()
end
# => 1
# => 2