I have records in my database like:
id | item_name | 2013-06-05T17:55:13+03:00
I want to group them by 'items per Day', 'items per Hour', 'items per 20 minutes'.
What is the best way to implement it?
The simple way:
by_day = array.group_by{|a| a.datetime.to_date}
by_hour = array.group_by{|a| [a.datetime.to_date, a.datetime.hour]}
by_20_minutes = array.group_by{|a| [a.datetime.to_date, a.datetime.hour, a.datetime.minute/20]}
require 'time'
def group_by_period(items)
groups = { :day => {}, :hour => {}, :t20min => {} }
items.reduce(groups) do |memo, item|
# Compute the correct buckets for the item's timestamp.
timestamp = Time.parse(item[2]).utc
item_day = timestamp.to_date.to_s
item_hour = timestamp.iso8601[0..12]
item_20min = timestamp.iso8601[0..15]
item_20min[14..18] = (item_20min[14..15].to_i / 20) * 20
# Place the item in each bucket.
[[:day,item_day], [:hour,item_hour], [:t20min,item_20min]].each do |k,v|
memo[k][v] = [] unless memo[k][v]
memo[k][v] << item
end
memo
end
end
sample_db_output = [
[1, 'foo', '2010-01-01T12:34:56Z'],
[2, 'bar', '2010-01-02T12:34:56Z'],
[3, 'gah', '2010-01-02T13:34:56Z'],
[4, 'zip', '2010-01-02T13:54:56Z']
]
group_by_period(sample_db_output)
# {:day=>
# {"2010-01-01"=>[[1, "foo", "2010-01-01T12:34:56Z"]],
# "2010-01-02"=>
# [[2, "bar", "2010-01-02T12:34:56Z"],
# [3, "gah", "2010-01-02T13:34:56Z"],
# [4, "zip", "2010-01-02T13:54:56Z"]]},
# :hour=>
# {"2010-01-01T12"=>[[1, "foo", "2010-01-01T12:34:56Z"]],
# "2010-01-02T12"=>[[2, "bar", "2010-01-02T12:34:56Z"]],
# "2010-01-02T13"=>
# [[3, "gah", "2010-01-02T13:34:56Z"], [4, "zip", "2010-01-02T13:54:56Z"]]},
# :t20min=>
# {"2010-01-01T12:20:00"=>[[1, "foo", "2010-01-01T12:34:56Z"]],
# "2010-01-02T12:20:00"=>[[2, "bar", "2010-01-02T12:34:56Z"]],
# "2010-01-02T13:20:00"=>[[3, "gah", "2010-01-02T13:34:56Z"]],
# "2010-01-02T13:40:00"=>[[4, "zip", "2010-01-02T13:54:56Z"]]}}
Related
So, I have a hash with arrays, like this one:
{"name": ["John","Jane","Chris","Mary"], "surname": ["Doe","Doe","Smith","Martins"]}
I want to merge them into an array of hashes, combining the corresponding elements.
The results should be like that:
[{"name"=>"John", "surname"=>"Doe"}, {"name"=>"Jane", "surname"=>"Doe"}, {"name"=>"Chris", "surname"=>"Smith"}, {"name"=>"Mary", "surname"=>"Martins"}]
Any idea how to do that efficiently?
Please, note that the real-world use scenario could contain a variable number of hash keys.
Try this
h[:name].zip(h[:surname]).map do |name, surname|
{ 'name' => name, 'surname' => surname }
end
I suggest writing the code to permit arbitrary numbers of attributes. It's no more difficult than assuming there are two (:name and :surname), yet it provides greater flexibility, accommodating, for example, future changes to the number or naming of attributes:
def squish(h)
keys = h.keys.map(&:to_s)
h.values.transpose.map { |a| keys.zip(a).to_h }
end
h = { name: ["John", "Jane", "Chris"],
surname: ["Doe", "Doe", "Smith"],
age: [22, 34, 96]
}
squish(h)
#=> [{"name"=>"John", "surname"=>"Doe", "age"=>22},
# {"name"=>"Jane", "surname"=>"Doe", "age"=>34},
# {"name"=>"Chris", "surname"=>"Smith", "age"=>96}]
The steps for the example above are as follows:
b = h.keys
#=> [:name, :surname, :age]
keys = b.map(&:to_s)
#=> ["name", "surname", "age"]
c = h.values
#=> [["John", "Jane", "Chris"], ["Doe", "Doe", "Smith"], [22, 34, 96]]
d = c.transpose
#=> [["John", "Doe", 22], ["Jane", "Doe", 34], ["Chris", "Smith", 96]]
d.map { |a| keys.zip(a).to_h }
#=> [{"name"=>"John", "surname"=>"Doe", "age"=>22},
# {"name"=>"Jane", "surname"=>"Doe", "age"=>34},
# {"name"=>"Chris", "surname"=>"Smith", "age"=>96}]
In the last step the first value of b is passed to map's block and the block variable is assigned its value.
a = d.first
#=> ["John", "Doe", 22]
e = keys.zip(a)
#=> [["name", "John"], ["surname", "Doe"], ["age", 22]]
e.to_h
#=> {"name"=>"John", "surname"=>"Doe", "age"=>22}
The remaining calculations are similar.
If your dataset is really big, you can consider using Enumerator::Lazy.
This way Ruby will not create intermediate arrays during calculations.
This is how #Ursus answer can be improved:
h[:name]
.lazy
.zip(h[:surname])
.map { |name, surname| { 'name' => name, 'surname' => surname } }
.to_a
Other option for the case where:
[..] the real-world use scenario could contain a variable number of hash keys
h = {
'name': ['John','Jane','Chris','Mary'],
'surname': ['Doe','Doe','Smith','Martins'],
'whathever': [1, 2, 3, 4, 5]
}
You could use Object#then with a splat operator in a one liner:
h.values.then { |a, *b| a.zip *b }.map { |e| (h.keys.zip e).to_h }
#=> [{:name=>"John", :surname=>"Doe", :whathever=>1}, {:name=>"Jane", :surname=>"Doe", :whathever=>2}, {:name=>"Chris", :surname=>"Smith", :whathever=>3}, {:name=>"Mary", :surname=>"Martins", :whathever=>4}]
The first part, works this way:
h.values.then { |a, *b| a.zip *b }
#=> [["John", "Doe", 1], ["Jane", "Doe", 2], ["Chris", "Smith", 3], ["Mary", "Martins", 4]]
The last part just maps the elements zipping each with the original keys then calling Array#to_h to convert to hash.
Here I removed the call .to_h to show the intermediate result:
h.values.then { |a, *b| a.zip *b }.map { |e| h.keys.zip e }
#=> [[[:name, "John"], [:surname, "Doe"], [:whathever, 1]], [[:name, "Jane"], [:surname, "Doe"], [:whathever, 2]], [[:name, "Chris"], [:surname, "Smith"], [:whathever, 3]], [[:name, "Mary"], [:surname, "Martins"], [:whathever, 4]]]
[h[:name], h[:surname]].transpose.map do |name, surname|
{ 'name' => name, 'surname' => surname }
end
In my array, I'm trying to retrieve the key with the largest value of "value_2", so in this case, "B":
myArray = [
"A" => {
"value_1" => 30,
"value_2" => 240
},
"B" => {
"value_1" => 40,
"value_2" => 250
},
"C" => {
"value_1" => 18,
"value_2" => 60
}
]
myArray.each do |array_hash|
array_hash.each do |key, value|
if value["value_2"] == array_hash.values.max
puts key
end
end
end
I get the error:
"comparison of Hash with Hash failed (ArgumentError)".
What am I missing?
Though equivalent, the array given in the question is generally written:
arr = [{ "A" => { "value_1" => 30, "value_2" => 240 } },
{ "B" => { "value_1" => 40, "value_2" => 250 } },
{ "C" => { "value_1" => 18, "value_2" => 60 } }]
We can find the desired key as follows:
arr.max_by { |h| h.values.first["value_2"] }.keys.first
#=> "B"
See Enumerable#max_by. The steps are:
g = arr.max_by { |h| h.values.first["value_2"] }
#=> {"B"=>{"value_1"=>40, "value_2"=>250}}
a = g.keys
#=> ["B"]
a.first
#=> "B"
In calculating g, for
h = arr[0]
#=> {"A"=>{"value_1"=>30, "value_2"=>240}}
the block calculation is
a = h.values
#=> [{"value_1"=>30, "value_2"=>240}]
b = a.first
#=> {"value_1"=>30, "value_2"=>240}
b["value_2"]
#=> 240
Suppose now arr is as follows:
arr << { "D" => { "value_1" => 23, "value_2" => 250 } }
#=> [{"A"=>{"value_1"=>30, "value_2"=>240}},
# {"B"=>{"value_1"=>40, "value_2"=>250}},
# {"C"=>{"value_1"=>18, "value_2"=>60}},
# {"D"=>{"value_1"=>23, "value_2"=>250}}]
and we wish to return an array of all keys for which the value of "value_2" is maximum (["B", "D"]). We can obtain that as follows.
max_val = arr.map { |h| h.values.first["value_2"] }.max
#=> 250
arr.select { |h| h.values.first["value_2"] == max_val }.flat_map(&:keys)
#=> ["B", "D"]
flat_map(&:keys) is shorthand for:
flat_map { |h| h.keys }
which returns the same array as:
map { |h| h.keys.first }
See Enumerable#flat_map.
Code
p myArray.pop.max_by{|k,v|v["value_2"]}.first
Output
"B"
I'd use:
my_array = [
"A" => {
"value_1" => 30,
"value_2" => 240
},
"B" => {
"value_1" => 40,
"value_2" => 250
},
"C" => {
"value_1" => 18,
"value_2" => 60
}
]
h = Hash[*my_array]
# => {"A"=>{"value_1"=>30, "value_2"=>240},
# "B"=>{"value_1"=>40, "value_2"=>250},
# "C"=>{"value_1"=>18, "value_2"=>60}}
k = h.max_by { |k, v| v['value_2'] }.first # => "B"
Hash[*my_array] takes the array of hashes and turns it into a single hash. Then max_by will iterate each key/value pair, returning an array containing the key value "B" and the sub-hash, making it easy to grab the key using first:
k = h.max_by { |k, v| v['value_2'] } # => ["B", {"value_1"=>40, "value_2"=>250}]
I guess the idea of your solution is looping through each hash element and compare the found minimum value with hash["value_2"].
But you are getting an error at
if value["value_2"] == array_hash.values.max
Because the array_hash.values is still a hash
{"A"=>{"value_1"=>30, "value_2"=>240}}.values.max
#=> {"value_1"=>30, "value_2"=>240}
It should be like this:
max = nil
max_key = ""
myArray.each do |array_hash|
array_hash.each do |key, value|
if max.nil? || value.values.max > max
max = value.values.max
max_key = key
end
end
end
# max_key #=> "B"
Another solution:
myArray.map{ |h| h.transform_values{ |v| v["value_2"] } }.max_by{ |k| k.values }.keys.first
You asked "What am I missing?".
I think you are missing a proper understanding of the data structures that you are using. I suggest that you try printing the data structures and take a careful look at the results.
The simplest way is p myArray which gives:
[{"A"=>{"value_1"=>30, "value_2"=>240}, "B"=>{"value_1"=>40, "value_2"=>250}, "C"=>{"value_1"=>18, "value_2"=>60}}]
You can get prettier results using pp:
require 'pp'
pp myArray
yields:
[{"A"=>{"value_1"=>30, "value_2"=>240},
"B"=>{"value_1"=>40, "value_2"=>250},
"C"=>{"value_1"=>18, "value_2"=>60}}]
This helps you to see that myArray has only one element, a Hash.
You could also look at the expression array_hash.values.max inside the loop:
myArray.each do |array_hash|
p array_hash.values
end
gives:
[{"value_1"=>30, "value_2"=>240}, {"value_1"=>40, "value_2"=>250}, {"value_1"=>18, "value_2"=>60}]
Not what you expected? :-)
Given this, what would you expect to be returned by array_hash.values.max in the above loop?
Use p and/or pp liberally in your ruby code to help understand what's going on.
I'm new to ruby, I am solving a problem that involves hashes and key. The problem asks me to Implement a method, #pet_types, that accepts a hash as an argument. The hash uses people's # names as keys, and the values are arrays of pet types that the person owns. My question is about using Hash#each method to iterate through each num inside the array. I was wondering if there's any difference between solving the problem using hash#each or hash.sort.each?
I spent several hours coming up different solution and still to figure out what are the different approaches between the 2 ways of solving the problem below.
I include my code in repl.it: https://repl.it/H0xp/6 or you can see below:
# Pet Types
# ------------------------------------------------------------------------------
# Implement a method, #pet_types, that accepts a hash as an argument. The hash uses people's
# names as keys, and the values are arrays of pet types that the person owns.
# Example input:
# {
# "yi" => ["dog", "cat"],
# "cai" => ["dog", "cat", "mouse"],
# "venus" => ["mouse", "pterodactyl", "chinchilla", "cat"]
# }
def pet_types(owners_hash)
results = Hash.new {|h, k| h[k] = [ ] }
owners_hash.sort.each { |k, v| v.each { |pet| results[pet] << k } }
results
end
puts "-------Pet Types-------"
owners_1 = {
"yi" => ["cat"]
}
output_1 = {
"cat" => ["yi"]
}
owners_2 = {
"yi" => ["cat", "dog"]
}
output_2 = {
"cat" => ["yi"],
"dog" => ["yi"]
}
owners_3 = {
"yi" => ["dog", "cat"],
"cai" => ["dog", "cat", "mouse"],
"venus" => ["mouse", "pterodactyl", "chinchilla", "cat"]
}
output_3 = {
"dog" => ["cai", "yi"],
"cat" => ["cai", "venus", "yi"],
"mouse" => ["cai", "venus"],
"pterodactyl" => ["venus"],
"chinchilla" => ["venus"]
}
# method 2
# The 2nd and 3rd method should return a hash that uses the pet types as keys and the values should
# be a list of the people that own that pet type. The names in the output hash should
# be sorted alphabetically
# switched_hash = Hash.new()
# owners_hash.each do |owner, pets_array|
# pets_array.each do |pet|
# select_owners = owners_hash.select { |owner, pets_array|
owners_hash[owner].include?(pet) }
# switched_hash[pet] = select_owners.keys.sort
# end
# end
# method 3
#switched_hash
# pets = Hash.new {|h, k| h[k] = [ ] } # WORKS SAME AS: pets = Hash.new( Array.new )
# owners = owners_hash.keys.sort
# owners.each do |owner|
# owners_hash[owner].each do |pet|
# pets[pet] << owner
# end
# end
# pets
# Example output:
# output_3 = {
# "dog" => ["cai", "yi"],
# "cat" => ["cai", "venus", "yi"], ---> (sorted alphabetically!)
# "mouse" => ["cai", "venus"],
# "pterodactyl" => ["venus"],
# "chinchilla" => ["venus"]
# }
I used a hash data structure in my program to first solve this problem. Then I tried to rewrite it using the pet_hash. And my final codes is the following:
def pet_types(owners_hash)
pets_hash = Hash.new { |k, v| v = [] }
owners_hash.each do |owner, pets|
pets.each do |pet|
pets_hash[pet] += [owner]
end
end
pets_hash.values.each(&:sort!)
pets_hash
end
puts "-------Pet Types-------"
owners_1 = {
"yi" => ["cat"]
}
output_1 = {
"cat" => ["yi"]
}
owners_2 = {
"yi" => ["cat", "dog"]
}
output_2 = {
"cat" => ["yi"],
"dog" => ["yi"]
}
owners_3 = {
"yi" => ["dog", "cat"],
"cai" => ["dog", "cat", "mouse"],
"venus" => ["mouse", "pterodactyl", "chinchilla", "cat"]
}
output_3 = {
"dog" => ["cai", "yi"],
"cat" => ["cai", "venus", "yi"],
"mouse" => ["cai", "venus"],
"pterodactyl" => ["venus"],
"chinchilla" => ["venus"]
}
puts pet_types(owners_1) == output_1
puts pet_types(owners_2) == output_2
puts pet_types(owners_3) == output_3
Hash#sort has the same effect (at least for my basic test) as Hash#to_a followed by Array#sort.
hash = {b: 2, a: 1}
hash.to_a.sort # => [[:a, 1, [:b, 2]]
hash.sort # => the same
Now let's look at #each, both on Hash and Array.
When you provide two arguments to the block, that can handle both cases. For the hash, the first argument will be the key and the second will be the value. For the nested array, the values essentially get splatted out to the args:
[[:a, 1, 2], [:b, 3, 4]].each { |x, y, z| puts "#{x}-#{y}-#{z}" }
# => a-1-2
# => b-3-4
So basically, you should think of Hash#sort to be a shortcut to Hash#to_a followed by Array#sort, and recognize that #each will work the same on a hash as a hash converted to array (a nested array). In this case, it doesn't matter which approach you take. Clearly if you need to sort iteration by the keys then you should use sort.
I have two hashes with some data that I need to aggregate. The first one is a mapping of which ids (id_1, id_2, id_3, id_4) belong under what category (a, b, c):
hash_1 = {'a' => ['id_1','id_2'], 'b' => ['id_3'], 'c' => ['id_4']}
The second hash holds values of how many events happened per id for a given date (date_1, date_2, date_3):
hash_2 = {
'id_1' => {'date_1' => 5, 'date_2' => 6, 'date_3' => 8},
'id_2' => {'date_1' => 0, 'date_3' => 6},
'id_3' => {'date_1' => 0, 'date_2' => nil, 'date_3' => 1},
'id_4' => {'date_1' => 10, 'date_2' => 1}
}
What I want is to get the total event per category (a,b,c). For the above example, the result would look something like:
hash_3 = {'a' => (5+6+8+0+6), 'b' => (0+0+1), 'c' => (10+1)}
My problem is, that there are about 5000 categories, each pointing to typically 1 to 3 ids, and each ID having event counts for 30 dates or more. So this takes quite a bit of computation. What will be the most performant (time effective) way to do this grouping in Ruby?
update
This is what I tried so far (took like 6-8 seconds!, horribly slow):
def total_clicks_per_category
{}.tap do |res|
hash_1.each do |cat, ids|
res[cat] = total_event_per_ids(ids)
end
end
end
def total_event_per_ids(ids)
ids.reduce(0) do |memo, id|
events = hash_2.fetch(id, {})
memo + (events.values.reduce(:+) || 0)
end
end
P.S. I’m using Ruby 2.3.
I'm writing this on a phone so I cannot test right now, but it looks OK.
g = hash_2.each_with_object({}) { |(k,v),g| g[k] = v.values.compact.sum }
hash_3 = hash_1.each_with_object({}) { |(k,v),h| h[k] = g.values_at(*v).sum }
First, create an intermediate hash that holds the sum of hash_2:
hash_4 = hash_2.map{|k, v| [k, v.values.inject(:+)]}.to_h
# => {"id_1"=>19, "id_2"=>6, "id_3"=>1, "id_4"=>11}
Then do the final summation:
hash_3 = hash_1.map{|k, v| [k, v.map{|k| hash_4[k]}.inject(:+)]}.to_h
# => {"a"=>25, "b"=>1, "c"=>11}
Theory
5000*3*30 isn't that many. Ruby probably will need a second at most for this kind of job.
Hash lookup is fast by default, you won't be able to optimize much.
You could pre-calculate hash_2_sum, though :
hash_2_sum = {
'id_1' => 5+6+8,
'id_2' => 0+6,
'id_3' => 0+0+1,
'id_4' => 10+1
}
A loop on hash1 with hash_2_sum lookup, and you're done.
Code
Your example has been updated with some nil values. You need to remove them with compact, and make sure the sum is 0 when no element is found with inject(0, :+):
hash_1 = {'a' => ['id_1','id_2'], 'b' => ['id_3'], 'c' => ['id_4']}
hash_2 = {
'id_1' => { 'date_1' => 5, 'date_2' => 6, 'date_3' => 8 },
'id_2' => { 'date_1' => 0, 'date_3' => 6 },
'id_3' => { 'date_1' => 0, 'date_2' => nil, 'date_3' => 1 },
'id_4' => { 'date_1' => 10, 'date_2' => 1 }
}
hash_2_sum = hash_2.each_with_object({}) do |(key, dates), sum|
sum[key] = dates.values.compact.inject(0, :+)
end
hash_3 = hash_1.each_with_object({}) do |(key, ids), sum|
sum[key] = hash_2_sum.values_at(*ids).inject(0, :+)
end
# {"a"=>25, "b"=>1, "c"=>11}
Note
{}.tap do |res|
hash_1.each do |cat, ids|
res[cat] = total_event_per_ids(ids)
end
end
isn't very readable IMHO.
You can either use each_with_object or Array#to_h :
result = [1, 2, 3].each_with_object({}) do |i, hash|
hash[i] = i * i
end
#=> {1=>1, 2=>4, 3=>9}
result = [1, 2, 3].map { |i| [i, i * i] }.to_h
#=> {1=>1, 2=>4, 3=>9}
I have an array of hashes:
array = [
{
id: 1,
name: "A",
points: 20,
victories: 4,
goals: 5,
},
{
id: 1,
name: "B",
points: 20,
victories: 4,
goals: 8,
},
{
id: 1,
name: "C",
points: 21,
victories: 5,
goals: 8,
}
]
To sort them using two keys I do:
array = array.group_by do |key|
[key[:points], key[:goals]]
end.sort_by(&:first).map(&:last)
But in my program, the sort criterias are stored in a database and I can get them and store in a array for example: ["goals","victories"] or ["name","goals"].
How can I sort the array using dinamic keys?
I tried many ways with no success like this:
criterias_block = []
criterias.each do |criteria|
criterias_block << "key[:#{criteria}]"
end
array = array.group_by do |key|
criterias_block
end.sort_by(&:first).map(&:last)
Array#sort can do this
criteria = [:points, :goals]
array.sort_by { |entry|
criteria.map { |c| entry[c] }
}
#=> [{:id=>1, :name=>"A", :points=>20, :victories=>4, :goals=>5},
# {:id=>1, :name=>"B", :points=>20, :victories=>4, :goals=>8},
# {:id=>1, :name=>"C", :points=>21, :victories=>5, :goals=>8}]
This works because if you sort an array [[1,2], [1,1], [2,3]], it sorts by the first elements, using any next elements to break ties
You can use values_at:
criteria = ["goals", "victories"]
criteria = criteria.map(&:to_sym)
array = array.group_by do |key|
key.values_at(*criteria)
end.sort_by(&:first).map(&:last)
# => [[{:id=>1, :name=>"A", :points=>20, :victories=>4, :goals=>5},
# {:id=>1, :name=>"B", :points=>20, :victories=>4, :goals=>8},
# {:id=>1, :name=>"C", :points=>21, :victories=>5, :goals=>8}]]
values_at returns an array of all the keys requested:
array[0].values_at(*criteria)
# => [4, 5]
I suggest doing it like this.
Code
def sort_it(array,*keys)
array.map { |h| [h.values_at(*keys), h] }.sort_by(&:first).map(&:last)
end
Examples
For array as given by you:
sort_it(array, :goals, :victories)
#=> [{:id=>1, :name=>"A", :points=>20, :victories=>4, :goals=>5},
# {:id=>1, :name=>"B", :points=>20, :victories=>4, :goals=>8},
# {:id=>1, :name=>"C", :points=>21, :victories=>5, :goals=>8}]
sort_it(array, :name, :goals)
#=> [{:id=>1, :name=>"A", :points=>20, :victories=>4, :goals=>5},
# {:id=>1, :name=>"B", :points=>20, :victories=>4, :goals=>8},
# {:id=>1, :name=>"C", :points=>21, :victories=>5, :goals=>8}]
For the first of these examples, you could of course write:
sort_it(array, *["goals", "victories"].map(&:to_sym))