Hash in Ruby just uses its hash value (for strings and numbers). Internally, it uses the Murmur hash function. I wonder how it can can be done given that the probability of having the same hash value for two different keys is not zero.
Can you share with us how you came to the conclusion that Ruby uses only the hash value to determine equality?
The text below is to explain to others your excellent point that the probability of computing the same hash value for two different keys is not zero, so how can the Hash class rely on just the hash value to determine equality?
For the purpose of this discussion I will refer to Ruby hashes as maps, so as not to confuse the 2 uses of the term hash in the Ruby language (1, a computed value on an object, and 2, a map/dictionary of pairs of values and unique keys).
As I understand it, hash values in maps, sets, etc. are used as a quick first pass at determining possible equality. That is, if the hashes of 2 objects are equal, then it is possible that the 2 objects are equal; but it's also possible that the 2 objects are not equal, but coincidentally produce the same hash value.
In other words, the only sure thing you can tell about equality from the hash values of the objects being compared is that if hash1 != hash2 then the objects are definitely not equal.
If the 2 hashes are equal, then the 2 objects must be compared by their content (in Ruby, by calling the == method, I believe).
So comparing hashes is not a substitute for comparing the objects themselves, it's just a quick first pass used to optimize performance.
Remember that a "hash table" or dictionary is perfectly okay with collisions. In fact, it's expected and accommodated in any reasonable implementation.
Ideally you strive to have a hash with as few collisions as possible, and there are entire doctoral level discussions on what makes a good hashing function, but they are inevitable. When a collision does occur then two values share the same index in the container.
Regardless of how a value is hashed, any potential match based on hash must be evaluated. A direct comparison is performed to ensure that the value you're accessing is the one requested, not one that coincidentally maps to the same spot.
Normal hash tables can be thought of as an array of arrays even though this is all completely hidden from you in general purpose use.
You can implement your own hash table in Ruby if you want to explore how this behaves:
class ExampleHash
include Enumerable
def initialize
#size = 9
#slots = Array.new(#size) { [ ] }
end
def [](key)
#slots[key.hash % #size].each do |entry|
if (entry[0] == key)
return entry[1]
end
end
nil
end
def []=(key, value)
entries = #slots[key.hash % #size]
entries.each do |entry|
if (entry[0] == key)
entry[1] = value
return
end
end
entries << [ key, value ]
end
end
This is made easy since every object in Ruby has a built-in hash method that produces a large numerical value that's based on the object's content.
Related
In this presentation the speaker has created a value class.
In implementing it, he overrides #eql? and says that in Java development, the idiom is that whenever you override #eql? you must override #hash.
class Weight
# ...
def hash
pounds.hash
end
def eql?(other)
self.class == other.class &&
self.pounds == other.pounds
end
alias :== eql?
end
Firstly, what is the #hash method? I can see it returns an integer.
> 1.hash
=> -3708808305943022538
> 2.hash
=> 1196896681607723080
> 1.hash
=> -3708808305943022538
Using pry I can see that an integer responds to #hash but I cannot see where it inherits the method from. It's not defined on Numeric or Object. If I knew what this method did, I would probably understand why it needs to be overridden at the same time as #eql?.
So, why does #hash need to be overridden whenever eql? is overridden?
Firstly, what is the #hash method? I can see it returns an integer.
The #hash method is supposed to return a hash of the receiver. (The name of the method is a bit of a giveaway).
Using pry I can see that an integer responds to #hash but I cannot see where it inherits the method from.
There are dozens of questions of the type "Where does this method come from" on [so], and the answer is always the same: the best way to know where a method comes from, is to simply ask it:
hash_method = 1.method(:hash)
hash_method.owner #=> Kernel
So, #hash is inherited from Kernel. Note however, that there is a bit of a peculiar relationship between Object and Kernel, in that some methods that are implemented in Kernel are documented in Object or vice versa. This probably has historic reasons, and is now an unfortunate fact of life in the Ruby community.
Unfortunately, for reasons I don't understand, the documentation for Object#hash was deleted in 2017 in a commit ironically titled "Add documents". It is, however, still available in Ruby 2.4 (bold emphasis mine):
hash → integer
Generates an Integer hash value for this object. This function must have the property that a.eql?(b) implies a.hash == b.hash.
The hash value is used along with eql? by the Hash class to determine if two objects reference the same hash key. […]
So, as you can see, there is a deep and important relationship between #eql? and #hash, and in fact the correct behavior of methods that use #eql? and #hash depends on the fact that this relationship is maintained.
So, we know that the method is called #hash and thus likely computes a hash. We know it is used together with eql?, and we know that it is used in particular by the Hash class.
What does it do, exactly? Well, we all know what a hash function is: it is a function that maps a larger, potentially infinite, input space into a smaller, finite, output space. In particular, in this case, the input space is the space of all Ruby objects, and the output space is the "fast integers" (i.e. the ones that used to be called Fixnum).
And we know how a hash table works: values are placed in buckets based on the hash of their keys, if I want to find a value, then I only need to compute the hash of the key (which is fast) and know which bucket I find the value in (in constant time), as opposed to e.g. an array of key-value-pairs, where I need to compare the key against every key in the array (linear search) to find the value.
However, there is a problem: Since the output space of a hash is smaller than the input space, there are different objects which have the same hash value and thus end up in the same bucket. Thus, when two objects have different hash values, I know for a fact that they are different, but if they have the same hash value, then they could still be different, and I need to compare them for equality to be sure – and that's where the relationship between hash and equality comes from. Also note that when many keys and up in the same bucket, I will again have to compare the search key against every key in the bucket (linear search) to find the value.
From all this we can conclude the following properties of the #hash method:
It must return an Integer.
Not only that, it must return a "fast integer" (equivalent to the old Fixnums).
It must return the same integer for two objects that are considered equal.
It may return the same integer for two objects that are considered unequal.
However, it only should do so with low probability. (Otherwise, a Hash may degenerate into a linked list with highly degraded performance.)
It also should be hard to construct objects that are unequal but have the same hash value deliberately. (Otherwise, an attacker can force a Hash to degenerate into a linked list as a form of Degradation-of-Service attack.)
The #hash method returns a numeric hash value for the receiving object:
:symbol.hash # => 2507
Ruby Hashes are an implementation of the hash map data structure, and they use the value returned by #hash to determine if the same key is being referenced.
Hashes leverage the #eql? method in conjunction with #hash values to determine equality.
Given that these two methods work together to provide Hashes with information about equality, if you override #eql?, you need to also override #hash to keep your object's behavior consistent with other Ruby objects.
If you do NOT override it, this happens:
class Weight
attr_accessor :pounds
def eql?(other)
self.class == other.class && self.pounds == other.pounds
end
alias :== eql?
end
w1 = Weight.new
w2 = Weight.new
w1.pounds = 10
w2.pounds = 10
w1 == w2 # => true, these two objects should now be considered equal
weights_map = Hash.new
weights_map[w1] = '10 pounds'
weights_map[w2] = '10 pounds'
weights_map # => {#<Weight:0x007f942d0462f8 #pounds=10>=>"10 pounds", #<Weight:0x007f942d03c3c0 #pounds=10>=>"10 pounds"}
If w1 and w2 are considered equal, there should only be one key value pair in the hash. However, the Hash class is calling #hash which we did NOT override.
To fix this and truly make w1 and w2 equals, we override #hash to:
class Weight
def hash
pounds.hash
end
end
weights_map = Hash.new
weights_map[w1] = '10 pounds'
weights_map[w2] = '10 pounds'
weights_map # => {#<Weight:0x007f942d0462f8 #pounds=10>=>"10 pounds"}
Now hash knows these objects are equal and therefore stores only one key-value pair
I'm a ruby beginner and reading that a Hash does not have order. I tried to play with that concept but found I could still order things like so:
Travel_Plans = Hash.new
Travel_Plans[4] = "Colorado Springs"
Travel_Plans[1] = "Santa Fe"
Travel_Plans[2] = "Raton"
Travel_Plans[5] = "Denver"
Travel_Plans[3] = "Pueblo"
puts Travel_Plans.sort
Could someone explain what is meant by "Hash does not have order"?
If you could provide a simple example that would be great.
Ruby's Hash class represents a "hash map" or "key-value dictionary" in conventional terms. These are intended to be structures that allow quick random-access to individual elements, but the elements themselves have no intrinsic ordering.
Internally Ruby's Hash organizes elements into their various locations in memory using the hash method each object must provide to be used as a key. Ruby's Hash is unusually, if not ludicrously flexible, in that an object, any object, can be used as a key, and it's preserved exactly as-is. Contrast with JavaScript where keys must be strings and strings only.
That means you can do this:
{ 1 => 'Number One', '1' => 'String One', :one => 'Symbol One', 1.0 => 'Float One }
Where that has four completely different keys.
This is in contrast to Array where the ordering is an important part of how an array works. You don't want to have a queue where things go in one order and come out in another.
Now Ruby's Hash class used to have no intrinsic order, but due to popular demand now it stores order in terms of insertion. That is, the first items inserted are "first". Normally you don't depend on this behaviour explicitly, but it does show up if you're paying attention:
a = { x: '1', y: '2' }
# => {:x=>"1, :y=>"2"}
b = { }
b[:y] = '2'
b[:x] = '1'
b
# => {:y=>"2", :x=>"1"}
Note that the order of the keys in b are reversed due to inserting them in reverse order. They're still equivalent though:
a == b
# => true
When you call sort on a Hash you actually end up converting it to an array of key/value pairs, then sorting each of those:
b.sort
# => [[:x, "1"], [:y, "2"]]
Which you could convert back into a Hash if you want:
b.sort.to_h
# => {:x=>"1", :y=>"2"}
So now it's "ordered" properly. In practice this rarely matters, though, as you'll be accessing the keys individually as necessary. b[:x] doesn't care where the :x key is, it always returns the right value regardless.
Some things to note about Ruby:
Don't use Hash.new, instead just use { } to represent an empty Hash structure.
Don't use capital letters for variables, they have significant meaning in Ruby. Travel_Plans is a constant, not a variable, because it starts with a capital letter. Those are reserved for ClassName and CONSTANT_NAME type usage. This should be travel_plans.
First, the statement "[h]ash does not have order" is wrong as of today. It used to be true for really old and outdated versions of Ruby. You seem to have picked outdated information, which would be unreliable as of today.
Second, the code you provided:
puts Travel_Plans.sort
is irrelevant to showing your point that the hash Travel_Plans has, i.e. preserves, the order. What you should have done to check whether the order is preserved, is to simply do:
p Travel_Plans
which would always result in showing the keys in the order 4, 1, 2, 5, 3, which matches the order in which you assigned the key-values to the hash, thus indeed shows that hash preserves the order.
According to the specification, strings that are used as a key to a hash are duplicated and frozen. Other mutable objects do not seem to have such special consideration. For example, with an array key, the following is possible.
a = [0]
h = {a => :a}
h.keys.first[0] = 1
h # => {[1] => :a}
h[[1]] # => nil
h.rehash
h[[1]] # => :a
On the other hand, a similar thing cannot be done with a string key.
s = "a"
h = {s => :s}
h.keys.first.upcase! # => RuntimeError: can't modify frozen String
Why is string designed to be different from other mutable objects when it comes to a hash key? Is there any use case where this specification becomes useful? What other consequences does this specification have?
I actually have a use case where absence of such special specification about strings may be useful. That is, I read with the yaml gem a manually written YAML file that describes a hash. the keys may be strings, and I would like to allow case insensitivity in the original YAML file. When I read a file, I might get a hash like this:
h = {"foo" => :foo, "Bar" => :bar, "BAZ" => :baz}
And I want to normalize the keys to lower case to get this:
h = {"foo" => :foo, "bar" => :bar, "baz" => :baz}
by doing something like this:
h.keys.each(&:downcase!)
but that returns an error for the reason explained above.
In short it's just Ruby trying to be nice.
When a key is entered in a Hash, a special number is calculated, using the hash method of the key. The Hash object uses this number to retrieve the key. For instance, if you ask what the value of h['a'] is, the Hash calls the hash method of string 'a' and checks if it has a value stored for that number. The problem arises when someone (you) mutates the string object, so the string 'a' is now something else, let's say 'aa'. The Hash would not find a hash number for 'aa'.
The most common types of keys for hashes are strings, symbols and integers. Symbols and integers are immutable, but strings are not. Ruby tries to protect you from the confusing behaviour described above by dupping and freezing string keys. I guess it's not done for other types because there could be nasty performance side effects (think of large arrays).
Immutable keys make sense in general because their hash codes will be stable.
This is why strings are specially-converted, in this part of MRI code:
if (RHASH(hash)->ntbl->type == &identhash || rb_obj_class(key) != rb_cString) {
st_insert(RHASH(hash)->ntbl, key, val);
}
else {
st_insert2(RHASH(hash)->ntbl, key, val, copy_str_key);
}
In a nutshell, in the string-key case, st_insert2 is passed a pointer to a function that will trigger the dup and freeze.
So if we theoretically wanted to support immutable lists and immutable hashes as hash keys, then we could modify that code to something like this:
VALUE key_klass;
key_klass = rb_obj_class(key);
if (key_klass == rb_cArray || key_klass == rb_cHash) {
st_insert2(RHASH(hash)->ntbl, key, val, freeze_obj);
}
else if (key_klass == rb_cString) {
st_insert2(RHASH(hash)->ntbl, key, val, copy_str_key);
}
else {
st_insert(RHASH(hash)->ntbl, key, val);
}
Where freeze_obj would be defined as:
static st_data_t
freeze_obj(st_data_t obj)
{
return (st_data_t)rb_obj_freeze((VALUE) obj);
}
So that would solve the specific inconsistency that you observed, where the array-key was mutable. However to be really consistent, more types of objects would need to be made immutable as well.
Not all types, however. For example, there'd be no point to freezing immediate objects like Fixnum because there is effectively only one instance of Fixnum corresponding to each integer value. This is why only String needs to be special-cased this way, not Fixnum and Symbol.
Strings are a special exception simply as a matter of convenience for Ruby programmers, because strings are very often used as hash keys.
Conversely, the reason that other object types are not frozen like this, which admittedly leads to inconsistent behavior, is mostly a matter of convenience for Matz & Company to not support edge cases. In practice, comparatively few people will use a container object like an array or a hash as a hash key. So if you do so, it's up to you to freeze before insertion.
Note that this is not strictly about performance, because the act of freezing a non-immediate object simply involves flipping the FL_FREEZE bit on the basic.flags bitfield that's present on every object. That's of course a cheap operation.
Also speaking of performance, note that if you are going to use string keys, and you are in a performance-critical section of code, you might want to freeze your strings before doing the insertion. If you don't, then a dup is triggered, which is a more-expensive operation.
Update #sawa pointed out that leaving your array-key simply frozen means the original array might be unexpectedly immutable outside of the key-use context, which could also be an unpleasant surprise (although otoh it would serve you right for using an array as a hash-key, really). If you therefore surmise that dup + freeze is the way out of that, then you would in fact incur possible noticeable performance cost. On the third hand, leave it unfrozen altogether, and you get the OP's original weirdness. Weirdness all around. Another reason for Matz et al to defer these edge cases to the programmer.
See this thread on the ruby-core mailing list for an explanation (freakily, it happened to be the first mail I stumbled across when I opened up the mailing list in my mail app!).
I've no idea about the first part of your question, but hHere is a practical answer for the 2nd part:
new_hash = {}
h.each_pair do |k,v|
new_hash.merge!({k.downcase => v})
end
h.replace new_hash
There's lots of permutations of this kind of code,
Hash[ h.map{|k,v| [k.downcase, v] } ]
being another (and you're probably aware of these, but sometimes it's best to take the practical route:)
You are askin 2 different questions: theoretical and practical. Lain was the first to answer, but I would like to provide what I consider a proper, lazier solution to your practical question:
Hash.new { |hsh, key| # this block get's called only if a key is absent
downcased = key.to_s.downcase
unless downcased == key # if downcasing makes a difference
hsh[key] = hsh[downcased] if hsh.has_key? downcased # define a new hash pair
end # (otherways just return nil)
}
The block used with Hash.new constructor is only invoked for those missing keys, that are actually requested. The above solution also accepts symbols.
A very old question - but if anyone else is trying to answer the "how can I get around the hash keys are freezing strings" part of the question...
A simple trick you could do to solve the String special case is:
class MutableString < String
end
s = MutableString.new("a")
h = {s => :s}
h.keys.first.upcase! # => RuntimeError: can't modify frozen String
puts h.inspect
Doesn't work unless you are creating the keys, and unless you are then careful that it doesn't cause any problems with anything that strictly requires that the class is exactly "String"
Can somebody explain to me what the below code is doing. before and after are hashes.
def differences(before, after)
before.diff(after).keys.sort.inject([]) do |diffs, k|
diff = { :attribute => k, :before => before[k], :after => after[k] }
diffs << diff; diffs
end
end
It is from the papertrail differ gem.
It's dense code, no question. So, as you say before and after are hash(-like?) objects that are handed into the method as parameters. Calling before.diff(after) returns another hash, which then immediately has .keys called on it. That returns all the keys in the hash that diff returned. The keys are returned as an array, which is then immediately sorted.
Then we get to the most complex/dense bit. Using inject on that sorted array of keys, the method builds up an array (called diffs inside the inject block) which will be the return value of the inject method.
That array is made up of records of differences. Each record is a hash - built up by taking one key from the sorted array of keys from the before.diff(after) return value. These hashes store the attribute that's being diffed, what it looked like in the before hash and what it looks like in the after hash.
So, in a nutshell, the method gets a bunch of differences between two hashes and collects them up in an array of hashes. That array of hashes is the final return value of the method.
Note: inject can be, and often is, much, much simpler than this. Usually it's used to simply reduce a collection of values to one result, by applying one operation over and over again and storing the results in an accumlator. You may know inject as reduce from other languages; reduce is an alias for inject in Ruby. Here's a much simpler example of inject:
[1,2,3,4].inject(0) do |sum, number|
sum + number
end
# => 10
0 is the accumulator - the initial value. In the pair |sum, number|, sum will be the accumulator and number will be each number in the array, one after the other. What inject does is add 1 to 0, store the result in sum for the next round, add 2 to sum, store the result in sum again and so on. The single final value of the accumulator sum will be the return value. Here 10. The added complexity in your example is that the accumulator is different in kind from the values inside the block. That's less common, but not bad or unidiomatic. (Edit: Andrew Marshall makes the good point that maybe it is bad. See his comment on the original question. And #tokland points out that the inject here is just a very over-complex alternative for map. It is bad.) See the article I linked to in the comments to your question for more examples of inject.
Edit: As #tokland points out in a few comments, the code seems to need just a straightforward map. It would read much easier then.
def differences(before, after)
before.diff(after).keys.sort.map do |k|
{ :attribute => k, :before => before[k], :after => after[k] }
end
end
I was too focused on explaining what the code was doing. I didn't even think of how to simplify it.
It finds the entries in before and after that differ according to the underlying objects, then builds up a list of those differences in a more convenient format.
before.diff(after) finds the entries that differ.
keys.sort gives you the keys (of the map of differences) in sorted order
inject([]) is like map, but starts with diffs initialized to an empty array.
The block creates a diff line (a hash) for each of these differences, and then appends it to diffs.
Say I have a class called Person, and it contains things such as last name, first name, address, etc.
I also have a hash table of Person objects that needs to be sorted by last and first name.
I understand that a sort_by will not change the hash permanently, which is fine, I only need to print in that order. Currently, I am trying to sort/print in place using:
#hash.sort_by {|a,b| a <=> b}.each { |person| puts person.last}
I have overloaded the <=> operator to sort by last/first, but nothing appears to actually sort. The puts there simply outputs in the hash's original order. I have spent a good 4 days trying to figure this out (it is a school assignment, and my first Ruby program). Any ideas? I am sure this is easy, but I am having the hardest time bringing my brain out of the C++ way of thinking.
You appear to be confusing sort and sort_by
sort yields two objects from the collection to the block and expects you to return a <=> like value: -1,0 or 1 depending on whether the arguments are equal, ascending or descending, for example
%w(one two three four five).sort {|a,b| a.length <=> b.length}
Sorts the strings by length. This is the form to use if you want to use your <=> operator
sort_by yields one object from the collection at a time and expects you to return what you want to sort by - you shouldn't be doing any comparison here. Ruby then uses <=> on these objecfs to sort your collection. The previous example can e rewritten as
%w(one two three four five).sort_by {|s| s.length}
This is also known as a schwartzian transform
In your case the collection is a hash so things are slightly more complicated: the values that are passed into the block are arrays that contain key/value pairs, so you'll need to extract the person object from that pair. You could also just work on #hash.keys or #hash.values (depending on whether the person objects are keys or values)
If you've overidden the <=> operator to sort Person objects appropriately then you can simply do:
#hash.sort_by{ |key, person| person }
because sort_by will yield both the hash key, and the object (in your case a person) to each iteration of the block. So the code above will sort your hash based on Person objects - for which you've already specified an <=> operator.
When you #.sort_by with a Hash, the parameters passed to the block are 'key','value', not 'element a' and 'element b'. Try:
#hash.sort_by{|key,value| value.last}.each{|key,value| puts value.last}
Also, see Frederick Cheung's excellent explanation of #sort vs #sort_by.