Ruby leaked objects are referenced by RubyVm::Env - ruby

I am tracing a memory leak problem in our application (ruby 2.1). I am using both techniques: ObjectSpace.dump_all for dumping all objects to JSON stream then do an offline analysis. The second technique I used is live analysis with ObjectSpace.reachable_objects_from. In both ways, I found that my leaked objects are referenced by an object RubyVM::Env. Anyone could explain to me what is RubyVM::Env. How to remove those references?

RubyVM::Env is an internal ruby class that holds variable references. Here is my test:
require 'objspace'
a = Object.new
a_id = a.object_id # we use #object_id to avoid creating more reference to `a`
ObjectSpace.each_object.select{ |o| ObjectSpace.reachable_objects_from(o).map(&:object_id).include?(a_id) }.count
# => 1
env = ObjectSpace.each_object.select{ |o| ObjectSpace.reachable_objects_from(o).map(&:object_id).include?(a_id) }.first
# => #<RubyVM::Env:0x007ff39ac09a78>
ObjectSpace.reachable_objects_from(env).count
# => 5
a = nil # remove reference
ObjectSpace.reachable_objects_from(env).count
# => 4

Related

Object.respond_to? fails when retrieving method name symbol from array

I'm implementing page objects, and writing tests to verify them. I want to simplify the tests by storing the element names in an array of symbols and looping through it, but it's failing.
def setup
#browser = Watir::Browser.new :phantomjs
#export_page = ExportPage.new #browser
#assets = %i{:section :brand}
end
--
#PASSES
def test_static
$stdout.puts :section.object_id
raise PageElementSelectorNotFoundException, :section unless #export_page.respond_to? :section
end
> # 2123548
This passes because the target class does implement this method, however:
#FAILS
def test_iterator
#assets.each do |selector|
$stdout.puts selector.class
$stdout.puts selector.object_id
$stdout.puts :section.object_id
raise PageElementSelectorNotFoundException, selector unless #export_page.respond_to? selector
end
end
> # Testing started at 11:19 ...
> # Symbol
> # 2387188
> # 2123548
PageElementSelectorNotFoundException: :section missing from page
~/src/stories/test/pages/export_page_test.rb:20:in `block in test_iterator'
As you can see, I've checked the object ids of the symbols and they do seem to be different. Could this be why it's failing? Is there a solution to this?
When using short notation to declare an array of atoms, one should not put colons there:
- %i{:section :brand} # incorrect
+ %i{section brand} # correct
What you have actually defined by #assets = %i{:section :brand}, is the following array:
[:':section', :':brand']
Don't use the %i{} notation, because it automatically makes a symbol of the literal(s) that you specify.
This translates to:
#assets = [:":section", :":brand"]
which is technically an array of symbols, just not the symbols that you intended. This is why the object IDs don't match in your tests.
The %i{} syntax was added in Ruby 2.0. When used in code that may support for older Ruby versions, use a traditional array of symbols:
#assets = [:section, :brand]

What are the equivalent to Lodash's get and set in Ruby?

I would like to use something similar to Lodash's get and set, but in Ruby instead of JavaScript. I tried few searches but I can't find anything similar.
Lodash's documentation will probably explain it in a better way, but it's getting and setting a property from a string path ('x[0].y.z' for example). If the full path doesn't exist when setting a property, it is automatically created.
Lodash Set
Lodash Get
I eventually ported Lodash _.set and _.get from JavaScript to Ruby and made a Gem.
Ruby 2.3 introduces the new safe navigator operator for getting nested/chained values:
x[0]&.y&.z #=> result or nil
Otherwise, Rails monkey patches all objects with try(…), allowing you to:
x[0].try(:y).try(:z) #=> result or nil
Setting is a bit harder, and I'd recommend ensuring you have the final object before attempting to set a property, e.g.:
if obj = x[0]&.y&.z
z.name = "Dr Robot"
end
You can use the Rudash Gem that comes with most of the Lodash utilities, and not only the _.get and _.set.
Sometimes I have had the need to programmatically get the value for a property deep into an object, but the thing is that sometimes the property is really a method, and sometimes it needs parameters!
So I came up with this solution, hope it helps devising one for your problem:
(Needs Rails' #try)
def reduce_attributes_for( object, options )
options.reduce( {} ) do |hash, ( attribute, methods )|
hash[attribute] = methods.reduce( object ) { |a, e| a.try!(:send, *e) }
hash
end
end
# Usage example
o = Object.new
attribute_map = {
# same as o.object_id
id: [:object_id],
# same as o.object_id.to_s
id_as_string: [:object_id, :to_s],
# same as o.object_id.to_s.length
id_as_string_length: [:object_id, :to_s, :length],
# I know, this one is a contrived example, but its purpose is
# to illustrate how you would call methods with parameters
# same as o.object_id.to_s.scan(/\d/)[1].to_i
second_number_from_id: [:object_id, :to_s, [:scan, /\d/], [:[],1], :to_i]
}
reduce_attributes_for( o, attribute_map )
# {:id=>47295942175460,
# :id_as_string=>"47295942175460",
# :id_as_string_length=>14,
# :second_number_from_id=>7}

Ruby access propteries with dot-notation

I'm trying to build a class that will basically be used as a data structure for storing values/nested values. I want there to be two methods, get and set, that accept a dot-notated path to recursively set or get variables.
For example:
bag = ParamBag.new
bag.get('foo.bar') # => nil
bag.set('foo.bar', 'baz')
bag.get('foo.bar') # => 'baz'
The get method could also take a default return value if the value doesn't exist:
bag.get('foo.baz', false) # => false
I could also initialize a new ParamBag with a Hash.
How would I manage this in Ruby? I've done this in other languages, but in order to set a recursive path, I would take the value by reference, but I'm not sure how I'd do it in Ruby.
This was a fun exercise but still falls under the "you probably should not do this" category.
To accomplish what you want, OpenStruct can be used with some slight modifications.
class ParamBag < OpenStruct
def method_missing(name, *args, &block)
if super.nil?
modifiable[new_ostruct_member(name)] = ParamBag.new
end
end
end
This class will let you chain however many method calls together you would like and set any number of parameters.
Tested with Ruby 2.2.1
2.2.1 :023 > p = ParamBag.new
=> #<ParamBag>
2.2.1 :024 > p.foo
=> #<ParamBag>
2.2.1 :025 > p.foo.bar
=> #<ParamBag>
2.2.1 :026 > p.foo.bar = {}
=> {}
2.2.1 :027 > p.foo.bar
=> {}
2.2.1 :028 > p.foo.bar = 'abc'
=> "abc"
Basically, take your get and set methods away and call methods like you would normally.
I do not advise you actually do this, I would instead suggest you use OpenStruct by itself to acheive some flexibility without going too crazy. If you find yourself needing to chain a ton of methods and have them never fail, maybe take a step backwards and ask "is this really the right way to approach this problem?". If the answer to that question is a resounding yes, then ParamBag might just be perfect.

Weird behavior of #upcase! in Ruby

Consider the following code:
#person = { :email => 'hello#example.com' }
temp = #person.clone
temp[:email].upcase!
p temp[:email] # => HELLO#EXAMPLE.COM
p #person[:email] # => HELLO#EXAMPLE.COM, why?!
# But
temp[:email] = 'blah#example.com'
p #person[:email] # => HELLO#EXAMPLE.COM
Ruby version is: "ruby 2.1.0p0 (2013-12-25 revision 44422) [i686-linux]".
I have no idea why is it happening. Can anyone help, please?
In the clone documentation you can read:
Produces a shallow copy of obj—the instance variables of obj are
copied, but not the objects they reference. clone copies the frozen
and tainted state of obj.
Also pay attention to this:
This method may have class-specific behavior. If so, that behavior
will be documented under the #initialize_copy method of the class.
Meaning that in some classes this behaviour can be overrided.
So any object references will be kept, instead of creating new ones. So what you want is a deep copy you can use Marshal:
temp = Marshal.load(Marshal.dump(#person))

chef 11: any way to turn attributes into a ruby hash?

I'm generating a config for my service in chef attributes. However, at some point, I need to turn the attribute mash into a simple ruby hash. This used to work fine in Chef 10:
node.myapp.config.to_hash
However, starting with Chef 11, this does not work. Only the top-level of the attribute is converted to a hash, with then nested values remaining immutable mash objects. Modifying them leads to errors like this:
Chef::Exceptions::ImmutableAttributeModification
------------------------------------------------ Node attributes are read-only when you do not specify which precedence level to set. To
set an attribute use code like `node.default["key"] = "value"'
I've tried a bunch of ways to get around this issue which do not work:
node.myapp.config.dup.to_hash
JSON.parse(node.myapp.config.to_json)
The json parsing hack, which seems like it should work great, results in:
JSON::ParserError
unexpected token at '"#<Chef::Node::Attribute:0x000000020eee88>"'
Is there any actual reliable way, short of including a nested parsing function in each cookbook, to convert attributes to a simple, ordinary, good old ruby hash?
after a resounding lack of answers both here and on the opscode chef mailing list, i ended up using the following hack:
class Chef
class Node
class ImmutableMash
def to_hash
h = {}
self.each do |k,v|
if v.respond_to?('to_hash')
h[k] = v.to_hash
else
h[k] = v
end
end
return h
end
end
end
end
i put this into the libraries dir in my cookbook; now i can use attribute.to_hash in both chef 10 (which already worked properly and which is unaffected by this monkey-patch) and chef 11. i've also reported this as a bug to opscode:
if you don't want to have to monkey-patch your chef, speak up on this issue:
http://tickets.opscode.com/browse/CHEF-3857
Update: monkey-patch ticket was marked closed by these PRs
I hope I am not too late to the party but merging the node object with an empty hash did it for me:
chef (12.6.0)> {}.merge(node).class
=> Hash
I had the same problem and after much hacking around came up with this:
json_string = node[:attr_tree].inspect.gsub(/\=\>/,':')
my_hash = JSON.parse(json_string, {:symbolize_names => true})
inspect does the deep parsing that is missing from the other methods proposed and I end up with a hash that I can modify and pass around as needed.
This has been fixed for a long time now:
[1] pry(main)> require 'chef/node'
=> true
[2] pry(main)> node = Chef::Node.new
[....]
[3] pry(main)> node.default["fizz"]["buzz"] = { "foo" => [ { "bar" => "baz" } ] }
=> {"foo"=>[{"bar"=>"baz"}]}
[4] pry(main)> buzz = node["fizz"]["buzz"].to_hash
=> {"foo"=>[{"bar"=>"baz"}]}
[5] pry(main)> buzz.class
=> Hash
[6] pry(main)> buzz["foo"].class
=> Array
[7] pry(main)> buzz["foo"][0].class
=> Hash
[8] pry(main)>
Probably fixed sometime in or around Chef 12.x or Chef 13.x, it is certainly no longer an issue in Chef 15.x/16.x/17.x
The above answer is a little unnecessary. You can just do this:
json = node[:whatever][:whatever].to_hash.to_json
JSON.parse(json)

Resources