Merge hashes based on particular key/value pair in ruby - ruby

I am trying to merge an array of hashes based on a particular key/value pair.
array = [ {:id => '1', :value => '2'}, {:id => '1', :value => '5'} ]
I would want the output to be
{:id => '1', :value => '7'}
As patru stated, in sql terms this would be equivalent to:
SELECT SUM(value) FROM Hashes GROUP BY id
In other words, I have an array of hashes that contains records. I would like to obtain the sum of a particular field, but the sum would grouped by key/value pairs. In other words, if my selection criteria is :id as in the example above, then it would seperate the hashes into groups where the id was the same and the sum the other keys.
I apologize for any confusion due to the typo earlier.

Edit: The question has been clarified since I first posted my answer. As a result, I have revised my answer substantially.
Here are two "standard" ways of addressing this problem. Both use Enumerable#select to first extract the elements from the array (hashes) that contain the given key/value pair.
#1
The first method uses Hash#merge! to sequentially merge each array element (hashes) into a hash that is initially empty.
Code
def doit(arr, target_key, target_value)
qualified = arr.select {|h|h.key?(target_key) && h[target_key]==target_value}
return nil if qualified.empty?
qualified.each_with_object({}) {|h,g|
g.merge!(h) {|k,gv,hv| k == target_key ? gv : (gv.to_i + hv.to_i).to_s}}
end
Example
arr = [{:id => '1', :value => '2'}, {:id => '2', :value => '3'},
{:id => '1', :chips => '4'}, {:zd => '1', :value => '8'},
{:cat => '2', :value => '3'}, {:id => '1', :value => '5'}]
doit(arr, :id, '1')
#=> {:id=>"1", :value=>"7", :chips=>"4"}
Explanation
The key here is to use the version of Hash#merge! that uses a block to determine the value for each key/value pair whose key appears in both of the hashes being merged. The two values for that key are represented above by the block variables hv and gv. We simply want to add them together. Note that g is the (initially empty) hash object created by each_with_object, and returned by doit.
target_key = :id
target_value = '1'
qualified = arr.select {|h|h.key?(target_key) && h[target_key]==target_value}
#=> [{:id=>"1", :value=>"2"},{:id=>"1", :chips=>"4"},{:id=>"1", :value=>"5"}]
qualified.empty?
#=> false
qualified.each_with_object({}) {|h,g|
g.merge!(h) {|k,gv,hv| k == target_key ? gv : (gv.to_i + hv.to_i).to_s}}
#=> {:id=>"1", :value=>"7", :chips=>"4"}
#2
The other common way to do this kind of calculation is to use Enumerable#flat_map, followed by Enumerable#group_by.
Code
def doit(arr, target_key, target_value)
qualified = arr.select {|h|h.key?(target_key) && h[target_key]==target_value}
return nil if qualified.empty?
qualified.flat_map(&:to_a)
.group_by(&:first)
.values.map { |a| a.first.first == target_key ? a.first :
[a.first.first, a.reduce(0) {|tot,s| tot + s.last}]}.to_h
end
Explanation
This may look complex, but it's not so bad if you break it down into steps. Here's what's happening. (The calculation of qualified is the same as in #1.)
target_key = :id
target_value = '1'
c = qualified.flat_map(&:to_a)
#=> [[:id,"1"],[:value,"2"],[:id,"1"],[:chips,"4"],[:id,"1"],[:value,"5"]]
d = c.group_by(&:first)
#=> {:id=>[[:id, "1"], [:id, "1"], [:id, "1"]],
# :value=>[[:value, "2"], [:value, "5"]],
# :chips=>[[:chips, "4"]]}
e = d.values
#=> [[[:id, "1"], [:id, "1"], [:id, "1"]],
# [[:value, "2"], [:value, "5"]],
# [[:chips, "4"]]]
f = e.map { |a| a.first.first == target_key ? a.first :
[a.first.first, a.reduce(0) {|tot,s| tot + s.last}] }
#=> [[:id, "1"], [:value, "7"], [:chips, "4"]]
f.to_h => {:id=>"1", :value=>"7", :chips=>"4"}
#=> {:id=>"1", :value=>"7", :chips=>"4"}
Comment
You may wish to consider makin the values in the hashes integers and exclude the target_key/target_value pairs from qualified:
arr = [{:id => 1, :value => 2}, {:id => 2, :value => 3},
{:id => 1, :chips => 4}, {:zd => 1, :value => 8},
{:cat => 2, :value => 3}, {:id => 1, :value => 5}]
target_key = :id
target_value = 1
qualified = arr.select { |h| h.key?(target_key) && h[target_key]==target_value}
.each { |h| h.delete(target_key) }
#=> [{:value=>2}, {:chips=>4}, {:value=>5}]
return nil if qualified.empty?
Then either
qualified.each_with_object({}) {|h,g| g.merge!(h) { |k,gv,hv| gv + hv } }
#=> {:value=>7, :chips=>4}
or
qualified.flat_map(&:to_a)
.group_by(&:first)
.values
.map { |a| [a.first.first, a.reduce(0) {|tot,s| tot + s.last}] }.to_h
#=> {:value=>7, :chips=>4}

Related

Most performant way to group/summarise two hashes?

I have two hashes with some data that I need to aggregate. The first one is a mapping of which ids (id_1, id_2, id_3, id_4) belong under what category (a, b, c):
hash_1 = {'a' => ['id_1','id_2'], 'b' => ['id_3'], 'c' => ['id_4']}
The second hash holds values of how many events happened per id for a given date (date_1, date_2, date_3):
hash_2 = {
'id_1' => {'date_1' => 5, 'date_2' => 6, 'date_3' => 8},
'id_2' => {'date_1' => 0, 'date_3' => 6},
'id_3' => {'date_1' => 0, 'date_2' => nil, 'date_3' => 1},
'id_4' => {'date_1' => 10, 'date_2' => 1}
}
What I want is to get the total event per category (a,b,c). For the above example, the result would look something like:
hash_3 = {'a' => (5+6+8+0+6), 'b' => (0+0+1), 'c' => (10+1)}
My problem is, that there are about 5000 categories, each pointing to typically 1 to 3 ids, and each ID having event counts for 30 dates or more. So this takes quite a bit of computation. What will be the most performant (time effective) way to do this grouping in Ruby?
update
This is what I tried so far (took like 6-8 seconds!, horribly slow):
def total_clicks_per_category
{}.tap do |res|
hash_1.each do |cat, ids|
res[cat] = total_event_per_ids(ids)
end
end
end
def total_event_per_ids(ids)
ids.reduce(0) do |memo, id|
events = hash_2.fetch(id, {})
memo + (events.values.reduce(:+) || 0)
end
end
P.S. I’m using Ruby 2.3.
I'm writing this on a phone so I cannot test right now, but it looks OK.
g = hash_2.each_with_object({}) { |(k,v),g| g[k] = v.values.compact.sum }
hash_3 = hash_1.each_with_object({}) { |(k,v),h| h[k] = g.values_at(*v).sum }
First, create an intermediate hash that holds the sum of hash_2:
hash_4 = hash_2.map{|k, v| [k, v.values.inject(:+)]}.to_h
# => {"id_1"=>19, "id_2"=>6, "id_3"=>1, "id_4"=>11}
Then do the final summation:
hash_3 = hash_1.map{|k, v| [k, v.map{|k| hash_4[k]}.inject(:+)]}.to_h
# => {"a"=>25, "b"=>1, "c"=>11}
Theory
5000*3*30 isn't that many. Ruby probably will need a second at most for this kind of job.
Hash lookup is fast by default, you won't be able to optimize much.
You could pre-calculate hash_2_sum, though :
hash_2_sum = {
'id_1' => 5+6+8,
'id_2' => 0+6,
'id_3' => 0+0+1,
'id_4' => 10+1
}
A loop on hash1 with hash_2_sum lookup, and you're done.
Code
Your example has been updated with some nil values. You need to remove them with compact, and make sure the sum is 0 when no element is found with inject(0, :+):
hash_1 = {'a' => ['id_1','id_2'], 'b' => ['id_3'], 'c' => ['id_4']}
hash_2 = {
'id_1' => { 'date_1' => 5, 'date_2' => 6, 'date_3' => 8 },
'id_2' => { 'date_1' => 0, 'date_3' => 6 },
'id_3' => { 'date_1' => 0, 'date_2' => nil, 'date_3' => 1 },
'id_4' => { 'date_1' => 10, 'date_2' => 1 }
}
hash_2_sum = hash_2.each_with_object({}) do |(key, dates), sum|
sum[key] = dates.values.compact.inject(0, :+)
end
hash_3 = hash_1.each_with_object({}) do |(key, ids), sum|
sum[key] = hash_2_sum.values_at(*ids).inject(0, :+)
end
# {"a"=>25, "b"=>1, "c"=>11}
Note
{}.tap do |res|
hash_1.each do |cat, ids|
res[cat] = total_event_per_ids(ids)
end
end
isn't very readable IMHO.
You can either use each_with_object or Array#to_h :
result = [1, 2, 3].each_with_object({}) do |i, hash|
hash[i] = i * i
end
#=> {1=>1, 2=>4, 3=>9}
result = [1, 2, 3].map { |i| [i, i * i] }.to_h
#=> {1=>1, 2=>4, 3=>9}

Averaging values across multiple hashes

EDIT I am accepting #CarySwoveland's answer because he got the closest on the first try, accounting for the most scenarios, and outputting the data into a hash so that you don't need to rely on order. Many honerable mentions though! Be sure to check out #ArupRakshit's answer as well if you want your output in an array!
I have an array of hashes like:
#my_hashes = [{"key1" => "10", "key2" => "5"...},{"key1" => "", "key2" => "9"...},{"key1" => "6", "key2" => "4"...}]
and I want an average for each key across the array. ie. 8.0,6.0...
Note that the hashes all have the exact same keys, in order, even if the value for the key is blank. Right now this works:
<%= #my_hashes[0].keys.each do |key| %>
<% sum = 0 %>
<% count = 0 %>
<% #my_hashes.each do |hash| %>
<% sum += hash[key].to_f %>
<% count += if hash[key].blank? then 0 else 1 end %>
<% end %>
<%= (sum/count) %>
<% end %>
but I feel like there may be a better way... any thoughts?
Do as below
#my_hashes = [{"key1" => "10", "key2" => "5"},{"key1" => "", "key2" => "9"},{"key1" => "6", "key2" => "4"}]
ar = #my_hashes[0].keys.map do |k|
a = #my_hashes.map { |h| h[k].to_f unless h[k].blank? }.compact
a.inject(:+)/a.size unless a.empty? #Accounting for "key1" => nil or "key1" => ""
end
ar # => [8, 6]
Another way:
#my_hashes = [ {"key1"=>"10", "key2"=>"5"},
{"key1"=> "", "key2"=>"9"},
{"key1"=> "6", "key2"=>"4"} ]
def avg(arr) arr.any? ? arr.reduce(:+)/arr.size.to_f : 0.0 end
(#my_hashes.each_with_object ( Hash.new { |h,k| h[k]=[] } ) {
|mh,h| mh.keys.each { |k| h[k] << mh[k].to_f unless mh[k].empty? } })
.each_with_object({}) { |(k,v),h| h[k] = avg(v) }
# => {"key1"=>8.0, "key2"=>6.0}
The object created by the first each_with_object is a hash whose default value is an empty array. That hash is represented by the block variable h. This means that if h[k] << mh[k].to_f is to be executed when h.key?(k) => false, h[k] = [] is executed first.
One could alternatively drop the avg method and create a temporary variable before computing the averages:
h = #my_hashes.each_with_object ( Hash.new { |h,k| h[k]=[] } ) { |mh,h|
mh.keys.each { |k| h[k] << mh[k].to_f unless mh[k].empty? } }
h.each_with_object({}) { |(k,v),h|
h[k] = ( avg(v) arr.any? ? arr.reduce(:+)/arr.size.to_f : 0.0 }
I think I found a quite elegant solution.
Here is a sample array:
a = [
{:a => 2, :b => 10},
{:a => 4, :b => 20},
{:a => 2, :b => 10},
{:a => 8, :b => 40},
]
And the solution:
class Array
def average
self.reduce(&:+) / self.size
end
end
r = a[0].keys.map do |key|
[key, a.map { |hash| hash[key] }.average]
end
puts Hash[*r.flatten]
Try this
#my_hashes = [{"key1" => "10", "key2" => "5"},{"key1" => "", "key2" => "9"},{"key1" => "6", "key2" => "4"}]
average_values = #my_hashes.map(&:values).transpose.map { |arr|
arr.map(&:to_f).inject(:+) / arr.size
}
with_keys = Hash[#my_hashes.first.keys.zip(average_values)]
average_values # => [5.333333333333333, 6.0]
with_keys # => {"key1"=>5.333333333333333, "key2"=>6.0}
if you want to exclude empty values from the average, could change average_values to reject empty values
average_values = #my_hashes.map(&:values).transpose.map { |arr|
arr.reject!(&:empty?)
arr.map(&:to_f).inject(:+) / arr.size
}
average_values # => [8.0, 6.0]
No super clean solution, but I would write:
a = [
{:a => 2, :b => 10},
{:a => 4, :b => 20},
{:a => 2, :b => 10},
{:a => 8, :b => 40},
]
grouped = a.flat_map(&:to_a).group_by{|x,|x}
grouped.keys.each do |key|
len = grouped[key].size
grouped[key] = 1.0 * grouped[key].map(&:last).inject(:+) / len
end

Ruby: Link two arrays of objects by attribute value

I'm pretty new in Ruby programming. In Ruby there are plenty ways to write elegant code. Is there any elegant way to link two arrays with objects of the same type by attribute value?
It's hard to explain. Let's look at the next example:
a = [ { :id => 1, :value => 1 }, { :id => 2, :value => 2 }, { :id => 3, :value => 3 } ]
b = [ { :id => 1, :value => 2 }, { :id => 3, :value => 4 } ]
c = link a, b
# Result structure after linkage.
c = {
"1" => {
:a => { :id => 1, :value => 1 },
:b => { :id => 1, :value => 1 }
},
"3" => {
:a => { :id => 3, :value => 3 },
:b => { :id => 3, :value => 4 }
}
}
So the basic idea is to get pairs of objects from different arrays by their common ID and construct a hash, which will give this pair by ID.
Thanks in advance.
If you want to take an adventure through Enumerable, you could say this:
(a.map { |h| [:a, h] } + b.map { |h| [:b, h] })
.group_by { |_, h| h[:id] }
.select { |_, a| a.length == 2 }
.inject({}) { |h, (n, v)| h.update(n => Hash[v]) }
And if you really want the keys to be strings, say n.to_s => Hash[v] instead of n => Hash[v].
The logic works like this:
We need to know where everything comes from we decorate the little hashes with :a and :b symbols to track their origins.
Then add the decorated arrays together into one list so that...
group_by can group things into almost-the-final-format.
Then find the groups of size two since those groups contain the entries that appeared in both a and b. Groups of size one only appeared in one of a or b so we throw those away.
Then a little injection to rearrange things into their final format. Note that the arrays we built in (1) just somehow happen to be in the format that Hash[] is looking for.
If you wanted to do this in a link method then you'd need to say things like:
link :a => a, :b => b
so that the method will know what to call a and b. This hypothetical link method also easily generalizes to more arrays:
def link(input)
input.map { |k, v| v.map { |h| [k, h] } }
.inject(:+)
.group_by { |_, h| h[:id] }
.select { |_, a| a.length == input.length }
.inject({}) { |h, (n, v)| h.update(n => Hash[v]) }
end
link :a => [...], :b => [...], :c => [...]
I assume that, for any two elements h1 and h2 of a (or of b), h1[:id] != h2[:id].
I would do this:
def convert(arr) Hash[arr.map {|h| [h[:id], h]}] end
ah, bh = convert(a), convert(b)
c = ah.keys.each_with_object({}) {|k,h|h[k]={a: ah[k], b: bh[k]} if bh.key?(k)}
# => {1=>{:a=>{:id=>1, :value=>1}, :b=>{:id=>1, :value=>2}},
# 3=>{:a=>{:id=>3, :value=>3}, :b=>{:id=>3, :value=>4}}}
Note that:
ah = convert(a)
# => {1=>{:id=>1, :value=>1}, 2=>{:id=>2, :value=>2}, 3=>{:id=>3, :value=>3}}
bh = convert(b)
# => {1=>{:id=>1, :value=>2}, 3=>{:id=>3, :value=>4}}
Here's a second approach. I don't like it as well, but it represents a different way of looking at the problem.
def sort_by_id(a) a.sort_by {|h| h[:id]} end
c = Hash[*sort_by_id(a.select {|ha| b.find {|hb| hb[:id] == ha[:id]}})
.zip(sort_by_id(b))
.map {|ha,hb| [ha[:id], {a: ha, b: hb}]}
.flatten]
Here's what's happening. The first step is to select only the elements ha of a for which there is an element hb of b for which ha[:id] = hb[id]. Then we sort both (what's left of) a and b on h[:id], zip them together and then make the hash c.
r1 = a.select {|ha| b.find {|hb| hb[:id] == ha[:id]}}
# => [{:id=>1, :value=>1}, {:id=>3, :value=>3}]
r2 = sort_by_id(r1)
# => [{:id=>1, :value=>1}, {:id=>3, :value=>3}]
r3 = sort_by_id(b)
# => [{:id=>1, :value=>2}, {:id=>3, :value=>4}]
r4 = r2.zip(r3)
# => [[{:id=>1, :value=>1}, {:id=>1, :value=>2}],
# [{:id=>3, :value=>3}, {:id=>3, :value=>4}]]
r5 = r4.map {|ha,hb| [ha[:id], {a: ha, b: hb}]}
# => [[1, {:a=>{:id=>1, :value=>1}, :b=>{:id=>1, :value=>2}}],
# [3, {:a=>{:id=>3, :value=>3}, :b=>{:id=>3, :value=>4}}]]
r6 = r5.flatten
# => [1, {:a=>{:id=>1, :value=>1}, :b=>{:id=>1, :value=>2}},
# 3, {:a=>{:id=>3, :value=>3}, :b=>{:id=>3, :value=>4}}]
c = Hash[*r6]
# => {1=>{:a=>{:id=>1, :value=>1}, :b=>{:id=>1, :value=>2}},
# 3=>{:a=>{:id=>3, :value=>3}, :b=>{:id=>3, :value=>4}}}
Ok, I've found the answer by myself. Here is a quite short line of code, which should do the trick:
Hash[a.product(b)
.select { |pair| pair[0][:id] == pair[1][:id] }
.map { |pair| [pair[0][:id], { :a => pair[0], :b => pair[1] }] }]
The product method gives us all possible pairs, then we filter them by equal IDs of pair elements. And then we map pairs to the special form, which will produce a Hash we are looking for.
So Hash[["key1", "value1"], ["key2", "value2"]] returns { "key1" => "value1", "key2" => "value2" }. And I use this to get the answer on my question.
Thanks.
P.S.: you can use pair.first instead of pair[0] and pair.last instead of pair[1] for better readability.
UPDATE
As Cary pointed out, it is better to replace |pair| with |ha, hb| to avoid these ugly indices:
Hash[a.product(b)
.select { |ha, hb| ha[:id] == hb[:id] }
.map { |ha, hb| [ha[:id], { :a => ha, :b => hb }] }]

Create Nested Hashes from a List of Hashes in Ruby

I have a set of categories and their values stored as a list of hashes:
r = [{:A => :X}, {:A => :Y}, {:B => :X}, {:A => :X}, {:A => :Z}, {:A => :X},
{:A => :X}, {:B => :Z}, {:C => :X}, {:C => :Y}, {:B => :X}, {:C => :Y},
{:C => :Y}]
I'd like to get a count of each value coupled with its category as a hash like this:
{:A => {:X => 4, :Y => 1, :Z => 1},
:B => {:X => 2, :Z => 1},
:C => {:X => 1, :Y => 3}}
How can I do this efficiently?
Here's what I have so far (it returns inconsistent values):
r.reduce(Hash.new(Hash.new(0))) do |memo, x|
memo[x.keys.first][x.values.first] += 1
memo
end
Should I first compute the counts of all instances of specific {:cat => :val}s and then create the hash? Should I give a different base-case to reduce and change the body to check for nil cases (and assign zero when nil) instead of always adding 1?
EDIT:
I ended up changing my code and using the below method to have a cleaner way of achieving a nested hash:
r.map do |x|
[x.keys.first, x.values.last]
end.reduce({}) do |memo, x|
memo[x.first] = Hash.new(0) if memo[x.first].nil?
memo[x.first][x.last] += 1
memo
end
The problem of your code is: memo did not hold the value.
Use a variable outside the loop to hold the value would be ok:
memo = Hash.new {|h,k| h[k] = Hash.new {|hh, kk| hh[kk] = 0 } }
r.each do |x|
memo[x.keys.first][x.values.first] += 1
end
p memo
And what's more, it won't work to init a hash nested inside a hash directly like this:
# NOT RIGHT
memo = Hash.new(Hash.new(0))
memo = Hash.new({})
Here is a link for more about the set default value issue:
http://www.themomorohoax.com/2008/12/31/why-setting-the-default-value-of-a-hash-to-be-a-hash-is-wrong
Not sure what "inconsistent values" means, but your problem is the hash you're injecting into is not remembering its results
r.each_with_object(Hash.new { |h, k| h[k] = Hash.new 0 }) do |individual, consolidated|
individual.each do |key, value|
consolidated[key][value] += 1
end
end
But honestly, it would probably be better to just go to wherever you're making this array and change it to aggregate values like this.
Functional approach using some handy abstractions -no need to reinvent the wheel- from facets:
require 'facets'
r.map_by { |h| h.to_a }.mash { |k, vs| [k, vs.frequency] }
#=> {:A=>{:X=>4, :Y=>1, :Z=>1}, :B=>{:X=>2, :Z=>1}, :C=>{:X=>1, :Y=>3}}

ruby array includes an id

I currently want to iterate over an array of Objects (2 properties: id & name) and check if the array contains a specific Id
How would I do this?
Enumerable#detect is ok, but I think that Enumerable#any? (which returns a boolean), is strictly what you asked for:
xs = [{:id => 1, :name => 'a'}, {:id => 2, :name => 'b'}]
puts xs.any? {|x| x[:id] == 1} # true
puts xs.any? {|x| x[:id] == 5} # false
Try detect
a = [{:id => 1, :name => 'a'}, {:id => 2, :name => 'b'}]
puts a.detect {|x| x[:id] == 1}

Resources