Maybe this is a question for code review, but I think it's best suited here. Let me know if it needs to be moved.
I'm using Redis as what amounts to a long-term cache for some data in a Sinatra application. I'm iterating over all of the keys in Redis, pulling their data, and parsing it with JSON. It's taking quite a bit of time. here's the benchmark:
[13] pry(main)> Benchmark.measure do
[13] pry(main)* dkeys = redis.keys.delete_if {|e| e == "last_run"}
[13] pry(main)* #delayed = dkeys.map {|k| {k => JSON.parse(redis.get(k))}}
[13] pry(main)* end
=> 0.520000 0.160000 0.680000 (132.410716)
[14] pry(main)> #delayed.count
=> 1358
[15] pry(main)>
The delay is clearly in the map and I think the latency is the cost of calling redis 1300+ times.
Is there a way that I can pull all of the data from redis into an object so I won't have to call it on each step of the iteration?
You should take a look at the answer #peterpan refers to. In particular, mget should do the trick:
dkeys = redis.keys.delete_if {|e| e == "last_run"}
#delayed = redis.mget(dkeys).map {|s| JSON.parse(s)}
Related
This question already has answers here:
Ruby 'pass by value' clarification [duplicate]
(3 answers)
Closed 4 years ago.
Given the following two methods:
[53] pry(main)> def my_method
[53] pry(main)* leti = 'leti'
[53] pry(main)* edit(leti)
[53] pry(main)* leti
[53] pry(main)* end
=> :my_method
[54] pry(main)> def edit(a_leti)
[54] pry(main)* a_leti.gsub!('e', '3')
[54] pry(main)* a_leti
[54] pry(main)* end
=> :edit
[55] pry(main)> my_method
=> "l3ti"
Can someone explain why I am getting the value edited inside the edit method and not the original value ('leti'). I though Ruby was passed by value. In fact, if instead of using the function gsub I use a simple assignment, I get the original value. Does the gsub! make it by reference?
Thank you!
In Ruby: Objects like strings are passed by reference. Variables with objects like strings are in fact references to those strings. Parameters are passed by value. However, for strings, these are references to those strings.
So here is the classic example:
irb(main):004:0* a = "abcd"
=> "abcd"
irb(main):005:0> b = a
=> "abcd"
irb(main):006:0> b << "def"
=> "abcddef"
irb(main):007:0> a
=> "abcddef"
irb(main):008:0> b
=> "abcddef"
If you do not wish to modify the original string, you need to make a copy of it:
Three ways (of many) to do this are:
b = a.dup
b = a.clone
b = String.new a
Using dup
irb(main):009:0> a = "abcd"
=> "abcd"
irb(main):010:0> b = a.dup
=> "abcd"
irb(main):011:0> b << "def"
=> "abcddef"
irb(main):012:0> a
=> "abcd"
irb(main):013:0> b
=> "abcddef"
BTW: For myself, this effect is the number one cause of defects in my own code.
I'm having two similar hashes. The first one looks like:
{'1'=>
{'ab'=>{'a'=>1, 'b'=>4},
{'bc'=>{'b'=>2, 'c'=>1},
...
}
The second one is pretty similar:
{'1'=>
{'ab'=>{'v'=>1},
{'bc'=>{'v'=>2},
...
}
I'd like to merge these like:
{'1'=>
{'ab'=>{'a'=>1, 'b'=>4, 'v'=>1},
{'bc'=>{'b'=>2, 'c'=>1, 'v'=>2},
...
}
What is the best way to do that?
For these simple case:
h1.merge(h2) do |_, v1, v2|
v1.merge(v2) { |_, v1, v2| v1.merge(v2) }
end
#⇒ {"1"=>{"ab"=>{"a"=>1, "b"=>4, "v"=>1},
# "bc"=>{"b"=>2, "c"=>1, "v"=>2}}}
Also note that what you have posted as an input are not valid ruby objects.
Try Hash#deep_merge from ActiveSupport. https://apidock.com/rails/Hash/deep_merge
If you don't want to depend on active_support gem or not willing to monkey-patch core classes, you can just copy the algorithm from the AS and adapt to your needs.
# File activesupport/lib/active_support/core_ext/hash/deep_merge.rb, line 21
def deep_merge!(other_hash, &block)
other_hash.each_pair do |current_key, other_value|
this_value = self[current_key]
self[current_key] = if this_value.is_a?(Hash) && other_value.is_a?(Hash)
this_value.deep_merge(other_value, &block)
else
if block_given? && key?(current_key)
block.call(current_key, this_value, other_value)
else
other_value
end
end
end
self
end
UPDATE:
I'm not sure why the answer was downvoted. Here is deep_merge in action:
[10] pry(main)> a = {'1'=>
[10] pry(main)* {'ab'=>{'a'=>1, 'b'=>4},
[10] pry(main)* 'bc'=>{'b'=>2, 'c'=>1}}
[10] pry(main)* };
[11] pry(main)>
[12] pry(main)> b = {'1'=>
[12] pry(main)* {'ab'=>{'v'=>1},
[12] pry(main)* 'bc'=>{'v'=>2}}
[12] pry(main)* };
[13] pry(main)>
[14] pry(main)> a.deep_merge(b)
=> {"1"=>{"ab"=>{"a"=>1, "b"=>4, "v"=>1}, "bc"=>{"b"=>2, "c"=>1, "v"=>2}}}
[15] pry(main)>
Exactly what OP needs.
I'm seeing something peculiar in Ruby that I don't understand, wondering if someone could show me what I'm missing. Here's some pry output:
[77] pry(main)> c.fields['iparties'].type
=> Array
[78] pry(main)> c.fields['iparties'].type == Array
=> true
[79] pry(main)> case c.fields['iparties'].type
[79] pry(main)* when Array
[79] pry(main)* puts "Array!"
[79] pry(main)* else
[79] pry(main)* puts "Something else?"
[79] pry(main)* end
Something else?
=> nil
c is a Mongoid object, and obviously I'm taking a look at the type of one of the model's fields. The field is an Array, as shown in line 77 and confirmed in the comparison on line 78. However, when I run it through a "when" statement, it does not match on the Array comparator. If I change the case definition to be
case c.fields['iparties'].type.to_s
and I change the Array match to
when 'Array'
it works as expected and the output is "Array!".
I don't understand why it doesn't work as it's shown here though, so if someone could explain it to me, I'd really appreciate it.
It's because case uses ===:
Array == Array #=> true
Array === Array #=> false
As
Array.class #=> Class
Class.ancestors #=> [Class, Module, Object, Kernel, BasicObject]
so Module#=== applies:
mod === obj → true or false. Returns true if obj is an instance of mod or one of mod’s descendants.
Take the .type out of your case and it will work. The magic of the when is that when Array means if Array === val, and the === method on a class returns true if the argument is an instance of that class!
While working through http://docs.basho.com/riak/latest/dev/taste-of-riak/querying-ruby
I noticed what I can't store Time object in riak for some reason.
[3] pry(main)> client = Riak::Client.new(:protocol => "pbc", :pb_port => 8087, :host => "192.168.145.34")
=> #<Riak::Client [#<Node 192.168.145.34:8098:8087>]>
[4] pry(main)>
[5] pry(main)>
[6] pry(main)> tt = client.bucket('test')
=> #<Riak::Bucket {test}>
[7] pry(main)> v = tt.new("ttt")
=> #<Riak::RObject {test,ttt} [#<Riak::RContent [application/json]:nil>]>
[8] pry(main)> v.data = 1
=> 1
[9] pry(main)> v.store
=> #<Riak::RObject {test,ttt} [#<Riak::RContent [application/json]:1>]>
[10] pry(main)> v.data = Time.now
=> 2014-06-22 11:34:01 +0400
[11] pry(main)> v.store
SystemStackError: stack level too deep
from /home/maus/.gem/ruby/1.9.1/gems/pry-0.10.0/lib/pry/pry_instance.rb:353
After that I discovered what it's the case for user defined classes too:
[16] pry(main)> class Tst
[16] pry(main)* def initialize(x)
[16] pry(main)* #x = x
[16] pry(main)* end
[16] pry(main)* end
=> nil
[17] pry(main)> t111 = Tst.new(111)
=> #<Tst:0x9b13f4c #x=111>
[18] pry(main)> v.data = t111
=> #<Tst:0x9b13f4c #x=111>
[19] pry(main)> v.store
SystemStackError: stack level too deep
from /home/maus/.gem/ruby/1.9.1/gems/pry-0.10.0/lib/pry/pry_instance.rb:353
[20] pry(main)>
[21] pry(main)> v.data = [1,2,3]
=> [1, 2, 3]
[22] pry(main)> v.store
=> #<Riak::RObject {test,test_key} [#<Riak::RContent [application/json]:[1, 2, 3]>]>
Looks like something is terribly wrong with my install. But how to investigate it? I'm using riak-1.3.2 and ruby 1.9.3p194 with the following versions of gems:
i18n-0.6.9
builder-3.2.2
beefcake-0.3.7
multi_json-1.10.1
innertube-1.0.2
riak-client-1.4.4.1
The only place in the store method code path were I see the value being use is in BeefcakeProtobuffsBackend.ObjectMethods::dump_object which instantiates an RpbContent with :value => maybe_encode(robject.raw_data), so this would appear to be an issue with serializing the custom data.
RContent.raw_data calls Serializers::serialize
def serialize(content_type, content)
serializer_for(content_type).dump(content)
end
Serialize only understands a very few content types:
Serializers['text/plain'] = TextPlain
Serializers['application/json'] = ApplicationJSON
Serializers['application/x-ruby-marshal'] = ::Marshal
The default for content_type is application/json, whose dump method is:
def dump(object)
object.to_json(Riak.json_options)
end
Check that v.raw_data returns the value you expect, and if not, try setting v.content_type='text/plain'
For your custom class, make sure it has a to_json or to_s method that returns a string, and select the appropriate content_type.
I know that pry allows you to "cd" into other scopes.
pry(main)> cd Object
pry(Object)> ls
constants:
ARGF Encoding GC NIL
ArgumentError EncodingError Gem NilClass
ARGV Enumerable Hash Noexec
Array Enumerator IndexError NoMemoryError
...
But is there any way to move the interactive session into a block thats passed to a method?
pry(main)> cd some_instance.some_method do |some_argument|
pry(block)> ls
some_argument some_other_vars_available_within_block
pry(block)> end
result_of_block
pry(main)>
If possible, this would be exceedingly helpful. I've tried getting binding.pry to work myself in this fashion, without some luck, but I wanted to make sure I'm going down the right path before I continue.
Not sure i understand you fully, but you can try this, for example:
(master ✘)✹✭ ᐅ pry
[1] pry(main)> def hello(&block)
| Object.new.instance_exec(&block)
| end
=> nil
[2] pry(main)> block_binding = nil
=> nil
[3] pry(main)> hello { |some_argument = "testing"| u = 20; block_binding = binding }
=> #<Binding:0x007fdd33b1f680>
[4] pry(main)> cd block_binding
[5] pry(#<Object>):1> ls
locals: _ __ _dir_ _ex_ _file_ _in_ _out_ _pry_ _super block_binding some_argument u
[6] pry(#<Object>):1> some_argument
=> "testing"
[7] pry(#<Object>):1> u
=> 20
[8] pry(#<Object>):1> self
=> #<Object:0x007fdd33b1f888>
[9] pry(#<Object>):1>
Actually it turns out that nesting pry works beautifully. Kudos to the pry team.
Once I got my little binding.pry issue out of the way (my fault entirely).
Nesting pry: (note the 2 exit statements needed)
Hershwild:~ jstillwell$ pry
[1] pry(main)> binding.pry
[1] pry(main)> exit
=> nil
[2] pry(main)> exit
Hershwild:~ jstillwell$
Bringing forth pry from within a block:
Hershwild:~ jstillwell$ pry
[1] pry(main)> def test
[1] pry(main)* yield true
[1] pry(main)* end
=> nil
[2] pry(main)> test do |inside|
[2] pry(main)* binding.pry
[2] pry(main)* end
# after that end statement, pry resumes inside the block
[1] pry(main)> inside
=> true
[2] pry(main)> exit
=> nil
[3] pry(main)> exit
Hershwild:~ jstillwell$