I am having a very common refactor situation here with me, and after going through a few blogs I still didn't get any satisfactory comment on the same; so asking a question here.
h = {
a: 'a',
b: 'b'
}
new_hash = {}
new_hash[:a] = h[:a].upcase if h[:a].present?
According to my friend, this code can be refactored in following way to improve performance.
a = h[:a]
new_hash[:a] = a.upcase if a.present?
At first glance it looks a little optimized. But is it something that'll make a lot of difference or its an over-optimization? And which style should be preferred?
Looking for an expert advice :)
UPDATE with Benchmark n = 1000
user system total real
hash lookup 0.000000 0.000000 0.000000 ( 0.000014)
new var 0.000000 0.000000 0.000000 ( 0.000005)
AND op 0.000000 0.000000 0.000000 ( 0.000018)
try 0.000000 0.000000 0.000000 ( 0.000046)
UPDATE with Memory Benchmark using gem benchmark-memory
Calculating -------------------------------------
hash lookup 40.000 memsize ( 40.000 retained)
1.000 objects ( 1.000 retained)
1.000 strings ( 1.000 retained)
new var 0.000 memsize ( 0.000 retained)
0.000 objects ( 0.000 retained)
0.000 strings ( 0.000 retained)
AND op 40.000 memsize ( 40.000 retained)
1.000 objects ( 1.000 retained)
1.000 strings ( 1.000 retained)
try 200.000 memsize ( 40.000 retained)
5.000 objects ( 1.000 retained)
1.000 strings ( 1.000 retained)
Depending on your circumstances rails methods like present? can be dirty and definitely impact performance. If you are only concerned about a nil check and not things like empty Array or blank String then using pure ruby methods will be "much" faster (the quotes are to emphasize the fact that performance is completely inconsequential in this basic example)
Since we are benchmarking things.
Setup
h = {
a: 'a',
b: 'b'
}
class Object
def present?
!blank?
end
def blank?
respond_to?(:empty?) ? !!empty? : !self
end
end
def hash_lookup(h)
new_hash = {}
new_hash[:a] = h[:a].upcase if h[:a].present?
new_hash
end
def new_var(h)
new_hash = {}
a = h[:a]
new_hash[:a] = a.upcase if a.present?
new_hash
end
def hash_lookup_w_safe_nav(h)
new_hash = {}
new_hash[:a] = h[:a]&.upcase
new_hash
end
def hash_lookup_wo_rails(h)
new_hash = {}
new_hash[:a] = h[:a].upcase if h[:a]
new_hash
end
def new_var_wo_rails(h)
new_hash = {}
a = h[:a]
new_hash[:a] = a.upcase if a
new_hash
end
Benchmarks
N = [1_000,10_000,100_000]
require 'benchmark'
N.each do |n|
puts "OVER #{n} ITERATIONS"
Benchmark.bm do |x|
x.report(:new_var) { n.times {new_var(h)}}
x.report(:hash_lookup) { n.times {hash_lookup(h)}}
x.report(:hash_lookup_w_safe_nav) { n.times {hash_lookup_w_safe_nav(h)}}
x.report(:hash_lookup_wo_rails) { n.times {hash_lookup_wo_rails(h)}}
x.report(:new_var_wo_rails) { n.times {new_var_wo_rails(h)}}
end
end
Output
OVER 1000 ITERATIONS
user system total real
new_var 0.001075 0.000159 0.001234 ( 0.001231)
hash_lookup 0.002441 0.000000 0.002441 ( 0.002505)
hash_lookup_w_safe_nav 0.001077 0.000000 0.001077 ( 0.001077)
hash_lookup_wo_rails 0.001100 0.000000 0.001100 ( 0.001145)
new_var_wo_rails 0.001015 0.000000 0.001015 ( 0.001016)
OVER 10000 ITERATIONS
user system total real
new_var 0.010321 0.000000 0.010321 ( 0.010329)
hash_lookup 0.010104 0.000015 0.010119 ( 0.010123)
hash_lookup_w_safe_nav 0.007211 0.000000 0.007211 ( 0.007213)
hash_lookup_wo_rails 0.007508 0.000000 0.007508 ( 0.017302)
new_var_wo_rails 0.008186 0.000026 0.008212 ( 0.016679)
OVER 100000 ITERATIONS
user system total real
new_var 0.099400 0.000249 0.099649 ( 0.192481)
hash_lookup 0.101419 0.000009 0.101428 ( 0.199788)
hash_lookup_w_safe_nav 0.078156 0.000010 0.078166 ( 0.140796)
hash_lookup_wo_rails 0.078743 0.000000 0.078743 ( 0.166815)
new_var_wo_rails 0.073271 0.000000 0.073271 ( 0.125869)
Optimization wears different shoes, there's memory optimization, performance optimization, and there's the readability and how the code is structured.
Performance: There's nearly no effect whatsoever on the speed and performance because the hash is being accessed in O(1). Try using benchmark to see yourself that there's nearly no difference
You can check this article about hash lookup and why it's so fast
Memory: Your friend's Code is less optimized than yours because he initialized another object a while yours doesn't.
Readability and Style: At the first glance your friend's code looks like fewer lines and more descriptive. But keep in mind that you may need to do this for every key/value in the hash, so you may need to have a, b, and it goes on as your hash goes on (When it goes like that it's better to iterate over the hash of course). Not too much to look on here tbh
Related
I did some benchmarks:
require 'benchmark'
words = File.open('/usr/share/dict/words', 'r') do |file|
file.each_line.take(1_000_000).map(&:chomp)
end
Benchmark.bmbm(20) do |x|
GC.start
x.report(:map) do
words.map do |word|
word.size if word.size > 5
end.compact
end
GC.start
x.report(:each_with_object) do
words.each_with_object([]) do |word, long_sizes|
long_sizes << word.size if word.size > 5
end
end
end
Output (ruby 2.3.0):
Rehearsal --------------------------------------------------------
map 0.020000 0.000000 0.020000 ( 0.016906)
each_with_object 0.020000 0.000000 0.020000 ( 0.024695)
----------------------------------------------- total: 0.040000sec
user system total real
map 0.010000 0.000000 0.010000 ( 0.015004)
each_with_object 0.020000 0.000000 0.020000 ( 0.024183)
I cannot understand it because I thought that each_with_object should be faster: it needs only 1 loop and 1 new object to create a new array instead of 2 loops and 2 new objects in case when we combine map and compact.
Any ideas?
Array#<< needs to reallocate memory if the original memory space doesn't have enough room to hold the new item. See the implementation, especially this line
VALUE target_ary = ary_ensure_room_for_push(ary, 1);
While Array#map doesn't have to reallocate memory from time to time because it already knows the size of the result array. See the implementation, especially
collect = rb_ary_new2(RARRAY_LEN(ary));
which allocates the same size of memory as the original array.
I ran a benchmark to see whether memoizing attributes was faster than reading from a configuration hash. The code below is an example. Can anybody explain them?
The Test
require 'benchmark'
class MyClassWithStuff
DEFAULT_VALS = { one: '1', two: 2, three: 3, four: 4 }
def memoized_fetch
#value ||= DEFAULT_VALS[:one]
end
def straight_fetch
DEFAULT_VALS[:one]
end
end
TIMES = 10000
CALL_TIMES = 1000
Benchmark.bmbm do |test|
test.report("Memoized") do
TIMES.times do
instance = MyClassWithStuff.new
CALL_TIMES.times { |i| instance.memoized_fetch }
end
end
test.report("Fetched") do
TIMES.times do
instance = MyClassWithStuff.new
CALL_TIMES.times { |i| instance.straight_fetch }
end
end
end
Results
Rehearsal --------------------------------------------
Memoized 1.500000 0.010000 1.510000 ( 1.510230)
Fetched 1.330000 0.000000 1.330000 ( 1.342800)
----------------------------------- total: 2.840000sec
user system total real
Memoized 1.440000 0.000000 1.440000 ( 1.456937)
Fetched 1.260000 0.000000 1.260000 ( 1.269904)
I recently installed Ruby 2.0.0 and found that it now has a lazy method for the Enumerable mixin. From previous experience in functional languages, I know that this makes for more efficient code.
I did a benchmark (not sure if it is moot) of lazy versus eager and found that lazy was continually faster. Why is this? What makes lazy evaluation better for large input?
Benchmark code:
#!/usr/bin/env ruby
require 'benchmark'
num = 1000
arr = (1..50000).to_a
Benchmark.bm do |rep|
rep.report('lazy') { num.times do ; arr.lazy.map { |x| x * 2 }; end }
rep.report('eager') { num.times do ; arr.map { |x| x * 2}; end }
end
Benchmark report sample:
user system total real
lazy 0.000000 0.000000 0.000000 ( 0.009502)
eager 5.550000 0.480000 6.030000 ( 6.231269)
It's so lazy it's not even doing the work - probably because you're not actually using the results of the operation. Put a sleep() in there to confirm:
> Benchmark.bm do |rep|
rep.report('lazy') { num.times do ; arr.lazy.map { |x| sleep(5) }; end }
rep.report('notlazy') { 1.times do ; [0,1].map { |x| sleep(5) } ; end }
end
user system total real
lazy 0.010000 0.000000 0.010000 ( 0.007130)
notlazy 0.000000 0.000000 0.000000 ( 10.001788)
I have learned two array sorting methods in Ruby:
array = ["one", "two", "three"]
array.sort.reverse!
or:
array = ["one", "two", "three"]
array.sort { |x,y| y<=>x }
And I am not able to differentiate between the two. Which method is better and how exactly are they different in execution?
Both lines do the same (create a new array, which is reverse sorted). The main argument is about readability and performance. array.sort.reverse! is more readable than array.sort{|x,y| y<=>x} - I think we can agree here.
For the performance part, I created a quick benchmark script, which gives the following on my system (ruby 1.9.3p392 [x86_64-linux]):
user system total real
array.sort.reverse 1.330000 0.000000 1.330000 ( 1.334667)
array.sort.reverse! 1.200000 0.000000 1.200000 ( 1.198232)
array.sort!.reverse! 1.200000 0.000000 1.200000 ( 1.199296)
array.sort{|x,y| y<=>x} 5.220000 0.000000 5.220000 ( 5.239487)
Run times are pretty constant for multiple executions of the benchmark script.
array.sort.reverse (with or without !) is way faster than array.sort{|x,y| y<=>x}. Thus, I recommend that.
Here is the script as a Reference:
#!/usr/bin/env ruby
require 'benchmark'
Benchmark.bm do|b|
master = (1..1_000_000).map(&:to_s).shuffle
a = master.dup
b.report("array.sort.reverse ") do
a.sort.reverse
end
a = master.dup
b.report("array.sort.reverse! ") do
a.sort.reverse!
end
a = master.dup
b.report("array.sort!.reverse! ") do
a.sort!.reverse!
end
a = master.dup
b.report("array.sort{|x,y| y<=>x} ") do
a.sort{|x,y| y<=>x}
end
end
There really is no difference here. Both methods return a new array.
For the purposes of this example, simpler is better. I would recommend array.sort.reverse because it is much more readable than the alternative. Passing blocks to methods like sort should be saved for arrays of more complex data structures and user-defined classes.
Edit: While destructive methods (anything ending in a !) are good for performance games, it was pointed out that they aren't required to return an updated array, or anything at all for that matter. It is important to keep this in mind because array.sort.reverse! could very likely return nil. If you wish to use a destructive method on a newly generated array, you should prefer calling .reverse! on a separate line instead of having a one-liner.
Example:
array = array.sort
array.reverse!
should be preferred to
array = array.sort.reverse!
Reverse! is Faster
There's often no substitute for benchmarking. While it probably makes no difference in shorter scripts, the #reverse! method is significantly faster than sorting using the "spaceship" operator. For example, on MRI Ruby 2.0, and given the following benchmark code:
require 'benchmark'
array = ["one", "two", "three"]
loops = 1_000_000
Benchmark.bmbm do |bm|
bm.report('reverse!') { loops.times {array.sort.reverse!} }
bm.report('spaceship') { loops.times {array.sort {|x,y| y<=>x} }}
end
the system reports that #reverse! is almost twice as fast as using the combined comparison operator.
user system total real
reverse! 0.340000 0.000000 0.340000 ( 0.344198)
spaceship 0.590000 0.010000 0.600000 ( 0.595747)
My advice: use whichever is more semantically meaningful in a given context, unless you're running in a tight loop.
With comparison as simple as your example, there is not much difference, but as the formula for comparison gets complicated, it is better to avoid using <=> with a block because the block you pass will be evaluated for each element of the array, causing redundancy. Consider this:
array.sort{|x, y| some_expensive_method(x) <=> some_expensive_method(y)}
In this case, some_expensive_method will be evaluated for each possible pair of element of array.
In your particular case, use of a block with <=> can be avoided with reverse.
array.sort_by{|x| some_expensive_method(x)}.reverse
This is called Schwartzian transform.
In playing with tessi's benchmarks on my machine, I've gotten some interesting results. I'm running ruby 2.0.0p195 [x86_64-darwin12.3.0], i.e., latest release of Ruby 2 on an OS X system. I used bmbm rather than bm from the Benchmark module. My timings are:
Rehearsal -------------------------------------------------------------
array.sort.reverse: 1.010000 0.000000 1.010000 ( 1.020397)
array.sort.reverse!: 0.810000 0.000000 0.810000 ( 0.808368)
array.sort!.reverse!: 0.800000 0.010000 0.810000 ( 0.809666)
array.sort{|x,y| y<=>x}: 0.300000 0.000000 0.300000 ( 0.291002)
array.sort!{|x,y| y<=>x}: 0.100000 0.000000 0.100000 ( 0.105345)
---------------------------------------------------- total: 3.030000sec
user system total real
array.sort.reverse: 0.210000 0.000000 0.210000 ( 0.208378)
array.sort.reverse!: 0.030000 0.000000 0.030000 ( 0.027746)
array.sort!.reverse!: 0.020000 0.000000 0.020000 ( 0.020082)
array.sort{|x,y| y<=>x}: 0.110000 0.000000 0.110000 ( 0.107065)
array.sort!{|x,y| y<=>x}: 0.110000 0.000000 0.110000 ( 0.105359)
First, note that in the Rehearsal phase that sort! using a comparison block comes in as the clear winner. Matz must have tuned the heck out of it in Ruby 2!
The other thing that I found exceedingly weird was how much improvement array.sort.reverse! and array.sort!.reverse! exhibited in the production pass. It was so extreme it made me wonder whether I had somehow screwed up and passed these already sorted data, so I added explicit checks for sorted or reverse-sorted data prior to performing each benchmark.
My variant of tessi's script follows:
#!/usr/bin/env ruby
require 'benchmark'
class Array
def sorted?
(1...length).each {|i| return false if self[i] < self[i-1] }
true
end
def reversed?
(1...length).each {|i| return false if self[i] > self[i-1] }
true
end
end
master = (1..1_000_000).map(&:to_s).shuffle
Benchmark.bmbm(25) do|b|
a = master.dup
puts "uh-oh!" if a.sorted?
puts "oh-uh!" if a.reversed?
b.report("array.sort.reverse:") { a.sort.reverse }
a = master.dup
puts "uh-oh!" if a.sorted?
puts "oh-uh!" if a.reversed?
b.report("array.sort.reverse!:") { a.sort.reverse! }
a = master.dup
puts "uh-oh!" if a.sorted?
puts "oh-uh!" if a.reversed?
b.report("array.sort!.reverse!:") { a.sort!.reverse! }
a = master.dup
puts "uh-oh!" if a.sorted?
puts "oh-uh!" if a.reversed?
b.report("array.sort{|x,y| y<=>x}:") { a.sort{|x,y| y<=>x} }
a = master.dup
puts "uh-oh!" if a.sorted?
puts "oh-uh!" if a.reversed?
b.report("array.sort!{|x,y| y<=>x}:") { a.sort!{|x,y| y<=>x} }
end
I read in the documentation for the String class that eql? is a strict equality operator, without type conversion, and == is a equality operator which tries to convert second its argument to a String, and, the C source code for this methods confirms that:
The eql? source code:
static VALUE
rb_str_eql(VALUE str1, VALUE str2)
{
if (str1 == str2) return Qtrue;
if (TYPE(str2) != T_STRING) return Qfalse;
return str_eql(str1, str2);
}
The == source code:
VALUE
rb_str_equal(VALUE str1, VALUE str2)
{
if (str1 == str2) return Qtrue;
if (TYPE(str2) != T_STRING) {
if (!rb_respond_to(str2, rb_intern("to_str"))) {
return Qfalse;
}
return rb_equal(str2, str1);
}
return str_eql(str1, str2);
}
But when I tried to benchmark these methods, I was suprised that == is faster than eql? by up to 20%!
My benchmark code is:
require "benchmark"
RUN_COUNT = 100000000
first_string = "Woooooha"
second_string = "Woooooha"
time = Benchmark.measure do
RUN_COUNT.times do |i|
first_string.eql?(second_string)
end
end
puts time
time = Benchmark.measure do
RUN_COUNT.times do |i|
first_string == second_string
end
end
puts time
And results:
Ruby 1.9.3-p125:
26.420000 0.250000 26.670000 ( 26.820762)
21.520000 0.200000 21.720000 ( 21.843723)
Ruby 1.9.2-p290:
25.930000 0.280000 26.210000 ( 26.318998)
19.800000 0.130000 19.930000 ( 19.991929)
So, can anyone explain why the more simple eql? method is slower than == method in the case when I run it for two similar strings?
The reason you are seeing a difference is not related to the implementation of == vs eql? but is due to the fact that Ruby optimizes operators (like ==) to avoid going through the normal method lookup when possible.
We can verify this in two ways:
Create an alias for == and call that instead. You'll get similar results to eql? and thus slower results than ==.
Compare using send :== and send :eql? instead and you'll get similar timings; the speed difference disappears because Ruby will only use the optimization for direct calls to the operators, not with using send or __send__.
Here's code that shows both:
require 'fruity'
first = "Woooooha"
second = "Woooooha"
class String
alias same_value? ==
end
compare do
with_operator { first == second }
with_same_value { first.same_value? second }
with_eql { first.eql? second }
end
compare do
with_send_op { first.send :==, second }
with_send_eql { first.send :eql?, second }
end
Results:
with_operator is faster than with_same_value by 2x ± 0.1
with_same_value is similar to with_eql
with_send_eql is similar to with_send_op
If you're the curious, the optimizations for operators are in insns.def.
Note: this answer applies only to Ruby MRI, I would be surprised if there was a speed difference in JRuby / rubinius, for instance.
When doing benchmarks, don't use times, because that creates a closure RUN_COUNT times. The extra time taken as a result affects all benchmarks equally in absolute terms, but that makes it harder to notice a relative difference:
require "benchmark"
RUN_COUNT = 10_000_000
FIRST_STRING = "Woooooha"
SECOND_STRING = "Woooooha"
def times_eq_question_mark
RUN_COUNT.times do |i|
FIRST_STRING.eql?(SECOND_STRING)
end
end
def times_double_equal_sign
RUN_COUNT.times do |i|
FIRST_STRING == SECOND_STRING
end
end
def loop_eq_question_mark
i = 0
while i < RUN_COUNT
FIRST_STRING.eql?(SECOND_STRING)
i += 1
end
end
def loop_double_equal_sign
i = 0
while i < RUN_COUNT
FIRST_STRING == SECOND_STRING
i += 1
end
end
1.upto(10) do |i|
method_names = [:times_eq_question_mark, :times_double_equal_sign, :loop_eq_question_mark, :loop_double_equal_sign]
method_times = method_names.map {|method_name| Benchmark.measure { send(method_name) } }
puts "Run #{i}"
method_names.zip(method_times).each do |method_name, method_time|
puts [method_name, method_time].join("\t")
end
puts
end
gives
Run 1
times_eq_question_mark 3.500000 0.000000 3.500000 ( 3.578011)
times_double_equal_sign 2.390000 0.000000 2.390000 ( 2.453046)
loop_eq_question_mark 3.110000 0.000000 3.110000 ( 3.140525)
loop_double_equal_sign 2.109000 0.000000 2.109000 ( 2.124932)
Run 2
times_eq_question_mark 3.531000 0.000000 3.531000 ( 3.562386)
times_double_equal_sign 2.469000 0.000000 2.469000 ( 2.484295)
loop_eq_question_mark 3.063000 0.000000 3.063000 ( 3.109276)
loop_double_equal_sign 2.109000 0.000000 2.109000 ( 2.140556)
Run 3
times_eq_question_mark 3.547000 0.000000 3.547000 ( 3.593635)
times_double_equal_sign 2.437000 0.000000 2.437000 ( 2.453047)
loop_eq_question_mark 3.063000 0.000000 3.063000 ( 3.109275)
loop_double_equal_sign 2.140000 0.000000 2.140000 ( 2.140557)
Run 4
times_eq_question_mark 3.547000 0.000000 3.547000 ( 3.578011)
times_double_equal_sign 2.422000 0.000000 2.422000 ( 2.437422)
loop_eq_question_mark 3.094000 0.000000 3.094000 ( 3.140524)
loop_double_equal_sign 2.140000 0.000000 2.140000 ( 2.140557)
Run 5
times_eq_question_mark 3.578000 0.000000 3.578000 ( 3.671758)
times_double_equal_sign 2.406000 0.000000 2.406000 ( 2.468671)
loop_eq_question_mark 3.110000 0.000000 3.110000 ( 3.156149)
loop_double_equal_sign 2.109000 0.000000 2.109000 ( 2.156181)
Run 6
times_eq_question_mark 3.562000 0.000000 3.562000 ( 3.562386)
times_double_equal_sign 2.407000 0.000000 2.407000 ( 2.468671)
loop_eq_question_mark 3.109000 0.000000 3.109000 ( 3.124900)
loop_double_equal_sign 2.125000 0.000000 2.125000 ( 2.234303)
Run 7
times_eq_question_mark 3.500000 0.000000 3.500000 ( 3.546762)
times_double_equal_sign 2.453000 0.000000 2.453000 ( 2.468671)
loop_eq_question_mark 3.031000 0.000000 3.031000 ( 3.171773)
loop_double_equal_sign 2.157000 0.000000 2.157000 ( 2.156181)
Run 8
times_eq_question_mark 3.468000 0.000000 3.468000 ( 3.656133)
times_double_equal_sign 2.454000 0.000000 2.454000 ( 2.484296)
loop_eq_question_mark 3.093000 0.000000 3.093000 ( 3.249896)
loop_double_equal_sign 2.125000 0.000000 2.125000 ( 2.140556)
Run 9
times_eq_question_mark 3.563000 0.000000 3.563000 ( 3.593635)
times_double_equal_sign 2.453000 0.000000 2.453000 ( 2.453047)
loop_eq_question_mark 3.125000 0.000000 3.125000 ( 3.124900)
loop_double_equal_sign 2.141000 0.000000 2.141000 ( 2.156181)
Run 10
times_eq_question_mark 3.515000 0.000000 3.515000 ( 3.562386)
times_double_equal_sign 2.453000 0.000000 2.453000 ( 2.453046)
loop_eq_question_mark 3.094000 0.000000 3.094000 ( 3.140525)
loop_double_equal_sign 2.109000 0.000000 2.109000 ( 2.156181)
equal? is reference equality
== is value equality
eql? is value and type equality
The third method, eql? is normally used to test if two objects have the same value as well as the same type. For example:
puts "integer == to float: #{25 == 25.0}"
puts "integer eql? to float: #{25.eql? 25.0}"
gives:
Does integer == to float: true
Does integer eql? to float: false
So I thought since eql? does more checking it would be slower, and for strings it is, at least on my Ruby 1.93. So I figured it must be type dependent and did some tests.
When integer and floats are compared eql? is a bit faster. When integers are compared == is much faster, until x2. Wrong theory, back to start.
The next theory: comparing two values of the same type will be faster with one of both proved to be true, in the case they are of the same type == is always faster, eql? is faster when types are different, again until x2.
Don't have the time to compare all types but I'm sure you'll get varying results, although the same kind of comparison always gives similar results. Can somebody prove me wrong?
Here are my results from the test of the OP:
16.863000 0.000000 16.863000 ( 16.903000) 2 strings with eql?
14.212000 0.000000 14.212000 ( 14.334600) 2 strings with ==
13.213000 0.000000 13.213000 ( 13.245600) integer and floating with eql?
14.103000 0.000000 14.103000 ( 14.200400) integer and floating with ==
13.229000 0.000000 13.229000 ( 13.410800) 2 same integers with eql?
9.406000 0.000000 9.406000 ( 9.410000) 2 same integers with ==
19.625000 0.000000 19.625000 ( 19.720800) 2 different integers with eql?
9.407000 0.000000 9.407000 ( 9.405800) 2 different integers with ==
21.825000 0.000000 21.825000 ( 21.910200) integer with string with eql?
43.836000 0.031000 43.867000 ( 44.074200) integer with string with ==