Related
I have two hashes with some data that I need to aggregate. The first one is a mapping of which ids (id_1, id_2, id_3, id_4) belong under what category (a, b, c):
hash_1 = {'a' => ['id_1','id_2'], 'b' => ['id_3'], 'c' => ['id_4']}
The second hash holds values of how many events happened per id for a given date (date_1, date_2, date_3):
hash_2 = {
'id_1' => {'date_1' => 5, 'date_2' => 6, 'date_3' => 8},
'id_2' => {'date_1' => 0, 'date_3' => 6},
'id_3' => {'date_1' => 0, 'date_2' => nil, 'date_3' => 1},
'id_4' => {'date_1' => 10, 'date_2' => 1}
}
What I want is to get the total event per category (a,b,c). For the above example, the result would look something like:
hash_3 = {'a' => (5+6+8+0+6), 'b' => (0+0+1), 'c' => (10+1)}
My problem is, that there are about 5000 categories, each pointing to typically 1 to 3 ids, and each ID having event counts for 30 dates or more. So this takes quite a bit of computation. What will be the most performant (time effective) way to do this grouping in Ruby?
update
This is what I tried so far (took like 6-8 seconds!, horribly slow):
def total_clicks_per_category
{}.tap do |res|
hash_1.each do |cat, ids|
res[cat] = total_event_per_ids(ids)
end
end
end
def total_event_per_ids(ids)
ids.reduce(0) do |memo, id|
events = hash_2.fetch(id, {})
memo + (events.values.reduce(:+) || 0)
end
end
P.S. I’m using Ruby 2.3.
I'm writing this on a phone so I cannot test right now, but it looks OK.
g = hash_2.each_with_object({}) { |(k,v),g| g[k] = v.values.compact.sum }
hash_3 = hash_1.each_with_object({}) { |(k,v),h| h[k] = g.values_at(*v).sum }
First, create an intermediate hash that holds the sum of hash_2:
hash_4 = hash_2.map{|k, v| [k, v.values.inject(:+)]}.to_h
# => {"id_1"=>19, "id_2"=>6, "id_3"=>1, "id_4"=>11}
Then do the final summation:
hash_3 = hash_1.map{|k, v| [k, v.map{|k| hash_4[k]}.inject(:+)]}.to_h
# => {"a"=>25, "b"=>1, "c"=>11}
Theory
5000*3*30 isn't that many. Ruby probably will need a second at most for this kind of job.
Hash lookup is fast by default, you won't be able to optimize much.
You could pre-calculate hash_2_sum, though :
hash_2_sum = {
'id_1' => 5+6+8,
'id_2' => 0+6,
'id_3' => 0+0+1,
'id_4' => 10+1
}
A loop on hash1 with hash_2_sum lookup, and you're done.
Code
Your example has been updated with some nil values. You need to remove them with compact, and make sure the sum is 0 when no element is found with inject(0, :+):
hash_1 = {'a' => ['id_1','id_2'], 'b' => ['id_3'], 'c' => ['id_4']}
hash_2 = {
'id_1' => { 'date_1' => 5, 'date_2' => 6, 'date_3' => 8 },
'id_2' => { 'date_1' => 0, 'date_3' => 6 },
'id_3' => { 'date_1' => 0, 'date_2' => nil, 'date_3' => 1 },
'id_4' => { 'date_1' => 10, 'date_2' => 1 }
}
hash_2_sum = hash_2.each_with_object({}) do |(key, dates), sum|
sum[key] = dates.values.compact.inject(0, :+)
end
hash_3 = hash_1.each_with_object({}) do |(key, ids), sum|
sum[key] = hash_2_sum.values_at(*ids).inject(0, :+)
end
# {"a"=>25, "b"=>1, "c"=>11}
Note
{}.tap do |res|
hash_1.each do |cat, ids|
res[cat] = total_event_per_ids(ids)
end
end
isn't very readable IMHO.
You can either use each_with_object or Array#to_h :
result = [1, 2, 3].each_with_object({}) do |i, hash|
hash[i] = i * i
end
#=> {1=>1, 2=>4, 3=>9}
result = [1, 2, 3].map { |i| [i, i * i] }.to_h
#=> {1=>1, 2=>4, 3=>9}
I have two arrays of hashes.
burgers = [
{:id => 1, :name => "cheese burger"},
{:id => 2, :name => "royale"},
{:id => 3, :name => "big mac"},
{:id => 4, :name => "angus beef"}
]
eaten = [
{:burger_id => 1},
{:burger_id => 2}
]
I would like to return an array or uneaten burgers, where burgers[:id] does not equal eaten[:burger_id]. In burgers_not_eaten_method, I have the expected return value.
def burgers_not_eaten
#Not sure how to compare burger[:id] with eaten[:burger_id]
burgers.reject { |burger| burger[:id] == #eaten burger_id }
# Expected: [{:id => 3, :name => "big mac"},{:id => 4, :name => "angus beef"}]
end
You're close, to make it easy I'd snag all the "eaten" ids into an array, and check for inclusion in that array, like so:
BURGERS = [
{:id => 1, :name => "cheese burger"},
{:id => 2, :name => "royale"},
{:id => 3, :name => "big mac"},
{:id => 4, :name => "angus beef"}
]
EATEN = [
{:burger_id => 1},
{:burger_id => 2}
]
def burgers_not_eaten
eaten_ids = EATEN.map { |e| e[:burger_id] }
BURGERS.reject { |burger| eaten_ids.include?(burger[:id]) }
end
burgers_not_eaten
# => [{:id=>3, :name=>"big mac"}, {:id=>4, :name=>"angus beef"}]
I am trying to merge an array of hashes based on a particular key/value pair.
array = [ {:id => '1', :value => '2'}, {:id => '1', :value => '5'} ]
I would want the output to be
{:id => '1', :value => '7'}
As patru stated, in sql terms this would be equivalent to:
SELECT SUM(value) FROM Hashes GROUP BY id
In other words, I have an array of hashes that contains records. I would like to obtain the sum of a particular field, but the sum would grouped by key/value pairs. In other words, if my selection criteria is :id as in the example above, then it would seperate the hashes into groups where the id was the same and the sum the other keys.
I apologize for any confusion due to the typo earlier.
Edit: The question has been clarified since I first posted my answer. As a result, I have revised my answer substantially.
Here are two "standard" ways of addressing this problem. Both use Enumerable#select to first extract the elements from the array (hashes) that contain the given key/value pair.
#1
The first method uses Hash#merge! to sequentially merge each array element (hashes) into a hash that is initially empty.
Code
def doit(arr, target_key, target_value)
qualified = arr.select {|h|h.key?(target_key) && h[target_key]==target_value}
return nil if qualified.empty?
qualified.each_with_object({}) {|h,g|
g.merge!(h) {|k,gv,hv| k == target_key ? gv : (gv.to_i + hv.to_i).to_s}}
end
Example
arr = [{:id => '1', :value => '2'}, {:id => '2', :value => '3'},
{:id => '1', :chips => '4'}, {:zd => '1', :value => '8'},
{:cat => '2', :value => '3'}, {:id => '1', :value => '5'}]
doit(arr, :id, '1')
#=> {:id=>"1", :value=>"7", :chips=>"4"}
Explanation
The key here is to use the version of Hash#merge! that uses a block to determine the value for each key/value pair whose key appears in both of the hashes being merged. The two values for that key are represented above by the block variables hv and gv. We simply want to add them together. Note that g is the (initially empty) hash object created by each_with_object, and returned by doit.
target_key = :id
target_value = '1'
qualified = arr.select {|h|h.key?(target_key) && h[target_key]==target_value}
#=> [{:id=>"1", :value=>"2"},{:id=>"1", :chips=>"4"},{:id=>"1", :value=>"5"}]
qualified.empty?
#=> false
qualified.each_with_object({}) {|h,g|
g.merge!(h) {|k,gv,hv| k == target_key ? gv : (gv.to_i + hv.to_i).to_s}}
#=> {:id=>"1", :value=>"7", :chips=>"4"}
#2
The other common way to do this kind of calculation is to use Enumerable#flat_map, followed by Enumerable#group_by.
Code
def doit(arr, target_key, target_value)
qualified = arr.select {|h|h.key?(target_key) && h[target_key]==target_value}
return nil if qualified.empty?
qualified.flat_map(&:to_a)
.group_by(&:first)
.values.map { |a| a.first.first == target_key ? a.first :
[a.first.first, a.reduce(0) {|tot,s| tot + s.last}]}.to_h
end
Explanation
This may look complex, but it's not so bad if you break it down into steps. Here's what's happening. (The calculation of qualified is the same as in #1.)
target_key = :id
target_value = '1'
c = qualified.flat_map(&:to_a)
#=> [[:id,"1"],[:value,"2"],[:id,"1"],[:chips,"4"],[:id,"1"],[:value,"5"]]
d = c.group_by(&:first)
#=> {:id=>[[:id, "1"], [:id, "1"], [:id, "1"]],
# :value=>[[:value, "2"], [:value, "5"]],
# :chips=>[[:chips, "4"]]}
e = d.values
#=> [[[:id, "1"], [:id, "1"], [:id, "1"]],
# [[:value, "2"], [:value, "5"]],
# [[:chips, "4"]]]
f = e.map { |a| a.first.first == target_key ? a.first :
[a.first.first, a.reduce(0) {|tot,s| tot + s.last}] }
#=> [[:id, "1"], [:value, "7"], [:chips, "4"]]
f.to_h => {:id=>"1", :value=>"7", :chips=>"4"}
#=> {:id=>"1", :value=>"7", :chips=>"4"}
Comment
You may wish to consider makin the values in the hashes integers and exclude the target_key/target_value pairs from qualified:
arr = [{:id => 1, :value => 2}, {:id => 2, :value => 3},
{:id => 1, :chips => 4}, {:zd => 1, :value => 8},
{:cat => 2, :value => 3}, {:id => 1, :value => 5}]
target_key = :id
target_value = 1
qualified = arr.select { |h| h.key?(target_key) && h[target_key]==target_value}
.each { |h| h.delete(target_key) }
#=> [{:value=>2}, {:chips=>4}, {:value=>5}]
return nil if qualified.empty?
Then either
qualified.each_with_object({}) {|h,g| g.merge!(h) { |k,gv,hv| gv + hv } }
#=> {:value=>7, :chips=>4}
or
qualified.flat_map(&:to_a)
.group_by(&:first)
.values
.map { |a| [a.first.first, a.reduce(0) {|tot,s| tot + s.last}] }.to_h
#=> {:value=>7, :chips=>4}
I have an array of hashes, id being one of the keys in the hashes. I want to sort the array elements according to a given order of ID values.
Suppose my array(size=5) is:
[{"id"=>1. ...}, {"id"=>4. ...}, {"id"=>9. ...}, {"id"=>2. ...}, {"id"=>7. ...}]
I want to sort the array elements such that their ids are in the following order:
[1,3,5,7,9,2,4,6,8,10]
So the expected result is:
[{'id' => 1},{'id' => 7},{'id' => 9},{'id' => 2},{'id' => 4}]
Here is a solution for any custom index:
def my_index x
# Custom code can be added here to handle items not in the index.
# Currently an error will be raised if item is not part of the index.
[1,3,5,7,9,2,4,6,8,10].index(x)
end
my_collection = [{"id"=>1}, {"id"=>4}, {"id"=>9}, {"id"=>2}, {"id"=>7}]
p my_collection.sort_by{|x| my_index x['id'] } #=> [{"id"=>1}, {"id"=>7}, {"id"=>9}, {"id"=>2}, {"id"=>4}]
Then you can format it in any way you want, maybe this is prettier:
my_index = [1,3,5,7,9,2,4,6,8,10]
my_collection.sort_by{|x| my_index.index x['id'] }
I would map the hash based on the values like so:
a = [{"id"=>1}, {"id"=>4}, {"id"=>9}, {"id"=>2}, {"id"=>7}]
[1,3,5,7,9,2,4,6,8,10].map{|x| a[a.index({"id" => x})] }.compact
#=> [{"id"=>1}, {"id"=>7}, {"id"=>9}, {"id"=>2}, {"id"=>4}]
General note on sorting. Use #sort_by method of the ruby's array class:
[{'id' => 1},{'id'=>3},{'id'=>2}].sort_by {|x|x['id'] }
# => [{"id"=>1}, {"id"=>2}, {"id"=>3}]
Or with usage #values method as a callback:
[{'id' => 1},{'id'=>3},{'id'=>2}].sort_by(&:values)
# => [{"id"=>1}, {"id"=>2}, {"id"=>3}]
or you can use more obvious version with #sort method:
[{'id' => 1},{'id'=>3},{'id'=>2}].sort {|x,y| x['id'] <=> y['id'] }
# => [{"id"=>1}, {"id"=>2}, {"id"=>3}]
For your case, to sort with extended condition use #% to split even and odd indexes:
[{'id' => 1},{'id'=>4},{'id'=>9},{'id'=>2},{'id'=>7}].sort do |x,y|
u = y['id'] % 2 <=> x['id'] % 2
u == 0 && y['id'] <=> x['id'] || u
end
# => [{"id"=>1}, {"id"=>7}, {"id"=>9}, {"id"=>2}, {"id"=>4}]
For your case, to sort with extended condition use #% to split according the index, even id value is absent in the index array:
index = [1,3,5,7,4,2,6,8,10] # swapped 2 and 4, 9 is absent
[{'id' => 1},{'id'=>4},{'id'=>9},{'id'=>2},{'id'=>7}].sort do |x,y|
!index.rindex( x[ 'id' ] ) && 1 || index.rindex( x[ 'id' ] ) <=> index.rindex( y[ 'id' ] ) || -1
end
# => [{"id"=>1}, {"id"=>7}, {"id"=>4}, {"id"=>2}, {"id"=>9}]
Why not just sort?
def doit(arr, order)
arr.sort { |h1,h2| order.index(h1['id']) <=> order.index(h2['id']) }
end
order = [1,3,5,7,9,2,4,6,8,10]
arr = [{'id' => 1}, {'id' => 4}, {'id' => 9}, {'id' => 2}, {'id' => 7}]
doit(arr, order)
#=> [{'id' => 1}, {'id' => 7}, {'id' => 9}, {'id' => 2}, {'id' => 4}]
a= [{"id"=>1}, {"id"=>4}, {"id"=>9}, {"id"=>2}, {"id"=>7}]
b=[1,3,5,7,9,2,4,6,8,10]
a.sort_by{|x| b.index (x['id'])}
I have a set of categories and their values stored as a list of hashes:
r = [{:A => :X}, {:A => :Y}, {:B => :X}, {:A => :X}, {:A => :Z}, {:A => :X},
{:A => :X}, {:B => :Z}, {:C => :X}, {:C => :Y}, {:B => :X}, {:C => :Y},
{:C => :Y}]
I'd like to get a count of each value coupled with its category as a hash like this:
{:A => {:X => 4, :Y => 1, :Z => 1},
:B => {:X => 2, :Z => 1},
:C => {:X => 1, :Y => 3}}
How can I do this efficiently?
Here's what I have so far (it returns inconsistent values):
r.reduce(Hash.new(Hash.new(0))) do |memo, x|
memo[x.keys.first][x.values.first] += 1
memo
end
Should I first compute the counts of all instances of specific {:cat => :val}s and then create the hash? Should I give a different base-case to reduce and change the body to check for nil cases (and assign zero when nil) instead of always adding 1?
EDIT:
I ended up changing my code and using the below method to have a cleaner way of achieving a nested hash:
r.map do |x|
[x.keys.first, x.values.last]
end.reduce({}) do |memo, x|
memo[x.first] = Hash.new(0) if memo[x.first].nil?
memo[x.first][x.last] += 1
memo
end
The problem of your code is: memo did not hold the value.
Use a variable outside the loop to hold the value would be ok:
memo = Hash.new {|h,k| h[k] = Hash.new {|hh, kk| hh[kk] = 0 } }
r.each do |x|
memo[x.keys.first][x.values.first] += 1
end
p memo
And what's more, it won't work to init a hash nested inside a hash directly like this:
# NOT RIGHT
memo = Hash.new(Hash.new(0))
memo = Hash.new({})
Here is a link for more about the set default value issue:
http://www.themomorohoax.com/2008/12/31/why-setting-the-default-value-of-a-hash-to-be-a-hash-is-wrong
Not sure what "inconsistent values" means, but your problem is the hash you're injecting into is not remembering its results
r.each_with_object(Hash.new { |h, k| h[k] = Hash.new 0 }) do |individual, consolidated|
individual.each do |key, value|
consolidated[key][value] += 1
end
end
But honestly, it would probably be better to just go to wherever you're making this array and change it to aggregate values like this.
Functional approach using some handy abstractions -no need to reinvent the wheel- from facets:
require 'facets'
r.map_by { |h| h.to_a }.mash { |k, vs| [k, vs.frequency] }
#=> {:A=>{:X=>4, :Y=>1, :Z=>1}, :B=>{:X=>2, :Z=>1}, :C=>{:X=>1, :Y=>3}}