How do I return an array of days and hours from a range? So far I have tried:
(48.hours.ago..Time.now.utc).map { |time| { :hour => time.hour } }.uniq
Returns:
[{:hour=>1}, {:hour=>2}, {:hour=>3}, {:hour=>4}, {:hour=>5}, {:hour=>6}, {:hour=>7}, {:hour=>8}, {:hour=>9}, {:hour=>10}, {:hour=>11}, {:hour=>12}, {:hour=>13}, {:hour=>14}, {:hour=>15}, {:hour=>16}, {:hour=>17}, {:hour=>18}, {:hour=>19}, {:hour=>20}, {:hour=>21}, {:hour=>22}, {:hour=>23}, {:hour=>0}]
Not ideal, since its iterates over every second. Which takes a long time. And I get several warning messages that say:
/Users/Chris/.rvm/gems/ruby-1.9.2-p290/gems/activesupport-3.2.2/lib/active_support/time_with_zone.rb:328: warning: Time#succ is obsolete; use time + 1
I am trying to return something like:
[{:day => 25, :hour=>1}, {:day => 25, :hour=>2}, {:day => 25, :hour=>3}, {:day => 25, :hour=>4} ... {:day => 26, :hour=>1}, {:day => 26, :hour=>2}, {:day => 26, :hour=>3}, {:day => 26, :hour=>4}]
Use Range#step, but as a precaution, convert the dates to integers first (apparently ranges using integers have step() optimized—YMMV). As a matter of style, I also truncate to the hour first.
Here's some quick 'n' dirty code:
#!/usr/bin/env ruby
require 'active_support/all'
s=48.hours.ago
n=Time.now
st=Time.local(s.year, s.month, s.day, s.hour).to_i
en=Time.local(n.year, n.month, n.day, n.hour).to_i
result = (st..en).step(1.hour).map do |i|
t = Time.at(i)
{ day: t.day, hour: t.hour }
end
puts result.inspect
Yields:
[{:day=>25, :hour=>11}, {:day=>25, :hour=>12}, {:day=>25, :hour=>13},
{:day=>25, :hour=>14}, {:day=>25, :hour=>15}, {:day=>25, :hour=>16},
...
stime = 48.hours.ago
etime=Time.now.utc
h = []
while stime <= etime
h.push({ :day => stime.day, :hour => stime.hour })
stime += 1.hour
end
Related
I have two hashes with some data that I need to aggregate. The first one is a mapping of which ids (id_1, id_2, id_3, id_4) belong under what category (a, b, c):
hash_1 = {'a' => ['id_1','id_2'], 'b' => ['id_3'], 'c' => ['id_4']}
The second hash holds values of how many events happened per id for a given date (date_1, date_2, date_3):
hash_2 = {
'id_1' => {'date_1' => 5, 'date_2' => 6, 'date_3' => 8},
'id_2' => {'date_1' => 0, 'date_3' => 6},
'id_3' => {'date_1' => 0, 'date_2' => nil, 'date_3' => 1},
'id_4' => {'date_1' => 10, 'date_2' => 1}
}
What I want is to get the total event per category (a,b,c). For the above example, the result would look something like:
hash_3 = {'a' => (5+6+8+0+6), 'b' => (0+0+1), 'c' => (10+1)}
My problem is, that there are about 5000 categories, each pointing to typically 1 to 3 ids, and each ID having event counts for 30 dates or more. So this takes quite a bit of computation. What will be the most performant (time effective) way to do this grouping in Ruby?
update
This is what I tried so far (took like 6-8 seconds!, horribly slow):
def total_clicks_per_category
{}.tap do |res|
hash_1.each do |cat, ids|
res[cat] = total_event_per_ids(ids)
end
end
end
def total_event_per_ids(ids)
ids.reduce(0) do |memo, id|
events = hash_2.fetch(id, {})
memo + (events.values.reduce(:+) || 0)
end
end
P.S. I’m using Ruby 2.3.
I'm writing this on a phone so I cannot test right now, but it looks OK.
g = hash_2.each_with_object({}) { |(k,v),g| g[k] = v.values.compact.sum }
hash_3 = hash_1.each_with_object({}) { |(k,v),h| h[k] = g.values_at(*v).sum }
First, create an intermediate hash that holds the sum of hash_2:
hash_4 = hash_2.map{|k, v| [k, v.values.inject(:+)]}.to_h
# => {"id_1"=>19, "id_2"=>6, "id_3"=>1, "id_4"=>11}
Then do the final summation:
hash_3 = hash_1.map{|k, v| [k, v.map{|k| hash_4[k]}.inject(:+)]}.to_h
# => {"a"=>25, "b"=>1, "c"=>11}
Theory
5000*3*30 isn't that many. Ruby probably will need a second at most for this kind of job.
Hash lookup is fast by default, you won't be able to optimize much.
You could pre-calculate hash_2_sum, though :
hash_2_sum = {
'id_1' => 5+6+8,
'id_2' => 0+6,
'id_3' => 0+0+1,
'id_4' => 10+1
}
A loop on hash1 with hash_2_sum lookup, and you're done.
Code
Your example has been updated with some nil values. You need to remove them with compact, and make sure the sum is 0 when no element is found with inject(0, :+):
hash_1 = {'a' => ['id_1','id_2'], 'b' => ['id_3'], 'c' => ['id_4']}
hash_2 = {
'id_1' => { 'date_1' => 5, 'date_2' => 6, 'date_3' => 8 },
'id_2' => { 'date_1' => 0, 'date_3' => 6 },
'id_3' => { 'date_1' => 0, 'date_2' => nil, 'date_3' => 1 },
'id_4' => { 'date_1' => 10, 'date_2' => 1 }
}
hash_2_sum = hash_2.each_with_object({}) do |(key, dates), sum|
sum[key] = dates.values.compact.inject(0, :+)
end
hash_3 = hash_1.each_with_object({}) do |(key, ids), sum|
sum[key] = hash_2_sum.values_at(*ids).inject(0, :+)
end
# {"a"=>25, "b"=>1, "c"=>11}
Note
{}.tap do |res|
hash_1.each do |cat, ids|
res[cat] = total_event_per_ids(ids)
end
end
isn't very readable IMHO.
You can either use each_with_object or Array#to_h :
result = [1, 2, 3].each_with_object({}) do |i, hash|
hash[i] = i * i
end
#=> {1=>1, 2=>4, 3=>9}
result = [1, 2, 3].map { |i| [i, i * i] }.to_h
#=> {1=>1, 2=>4, 3=>9}
I have a hash like this:
{'yes' => 23,
'b' => 'travel',
'yesterday' => 34,
5 => '234',
:yesss => :fg,
try: 30,
key: 'some value',
'yesterday1' => 34,
'yesteryear' => 2014}
How can I count all keys which includes yes?
I suppose you meant:
your_hash.count { |k, _| k.to_s.include?('yes') }
#=> 5
I have a helper module to generate an array hash data, which is something like:
[{:date => d, :total_amount => 31, :first_category => 1, :second_category => 2,...},
{:date => d+1, :total_amount => 31, :first_category => 1, :second_category => 2,...}]
So I make the method like:
def records_chart_data(category = nil, start = 3.weeks.ago)
total_by_day = Record.total_grouped_by_day(start)
category_sum_by_day = Record.sum_of_category_by_day(start)
(start.to_date..Time.zone.today).map do |date|
{
:date => date,
:total_amount => total_by_day[date].try(:first).try(:total_amount) || 0,
Category.find(1).title => category_sum_by_day[0][date].try(:first).try(:total_amount) || 0,
Category.find(2).title => category_sum_by_day[1][date].try(:first).try(:total_amount) || 0,
Category.find(3).title => category_sum_by_day[2][date].try(:first).try(:total_amount) || 0,
}
end
end
Since the Category will always change, I try to use loop in this method like:
def records_chart_data(category = nil, start = 3.weeks.ago)
total_by_day = Record.total_grouped_by_day(start)
category_sum_by_day = Record.sum_of_category_by_day(start)
(start.to_date..Time.zone.today).map do |date|
{
:date => date,
Category.all.each_with_index do |category, index|
category.title => category_sum_by_day[index][date].try(:first).try(:total_amount) || 0,
end
:total_amount => total_by_day[date].try(:first).try(:total_amount) || 0
}
end
end
But ruby alerts me with an error:
/Users/tsu/Code/CashNotes/app/helpers/records_helper.rb:10: syntax error, unexpected tASSOC, expecting keyword_end
category.title => category_sum_by_day[index][d...
Why does it say expecting keyword_end, and how should I fix it?
The method category_sum_by_day it calls looks like:
def self.sum_of_category_by_day(start)
records = where(date: start.beginning_of_day..Time.zone.today)
records = records.group('category_id, date(date)')
records = records.select('category_id, date, sum(amount) as total_amount')
records = records.group_by{ |r| r.category_id }
records.map do |category_id, value|
value.group_by {|r| r.date.to_date}
end
end
Or should I alter this method to generate a more friendly method for the helper above?
Category.all.each_with_index do |category, index|
category.title => category_sum_by_day # ...snip!
end
Unfortunately, this piece of code does not adhere to Ruby's grammar. The problem is the body of the block. x => y is not an expression and the syntax requires bodies of blocks to be expressions.
If you want to generate a hash by one key-value pair at a time try the following combination of Hash::[], Array#flatten and the splat operator (i.e. unary *):
Hash[*5.times.map { |i| [i * 3, - i * i] }.flatten]
As a result I'd rewrite the last expresion of records_chart_data more or less as follows
(start.to_date..Time.zone.today).map do |date|
categories = Hash[*Category.all.each_with_index do |category, index|
[ category.title, category_sum_by_day[...] ]
end .flatten]
{ :date => date,
:total_amount => total_by_day[date].try(:first).try(:total_amount) || 0
}.merge categories
end
If you consider it unreadable you can do it in a less sophisticated way, i.e.:
(start.to_date..Time.zone.today).map do |date|
hash = {
:date => date,
:total_amount => total_by_day[date].try(:first).try(:total_amount) || 0
}
Category.all.each_with_index do |category, index|
hash[category.title] = category_sum_by_day[...]
end
hash
end
Another idea is to use Array#reduce and adopt a more functional approach.
(start.to_date..Time.zone.today).map do |date|
Category.all.each_with_index.reduce({
:date => date,
:total_amount => total_by_day[date].try(:first).try(:total_amount) || 0
}) do |hash, (category, index)|
hash.merge category.title => category_sum_by_day[...]
end
hash
end
I have two arrays of hashes. The keys for the hashes are different:
player_scores1 = [{:first_name=>"Bruce", :score => 43, :time => 50},
{:first_name=>"Clark", :score => 45, :minutes => 20}]
player_scores2 = [{:last_name=>"Wayne", :points => 13, :time => 40},
{:last_name=>"Kent", :points => 3, :minutes => 20}]
I'd like to create a new array of hashes which adds up :score and :points together and assign it to a key called :score. I'd also like to combine the :first_name and :last_name and assign it to a key called :full_name. I want to discard any other keys.
This would result in this array:
all_players = [{:full_name => "Bruce Wayne", :score => 56},
{:full_name => "Clark Kent", :score => 48}]
Is there an elegant way to do this?
Something like this:
player_scores1.zip(player_scores2).map { |a,b|
{
:full_name => a[:first_name]+' '+b[:last_name],
:score => a[:score]+b[:points]
}
}
The code you're looking for is:
final = []
player_scores1.each_index do |index|
entry_1 = player_scores1.values(index)
entry_2 = player_scores2.values(index)[:first_name]
score = entry_1[:score] + entry_2[:points]
final << {:full_name => "#{entry_1[:first_name]} #{entry_2[:last_name]}", :score => score }
end
Any suggestions on tightening this up would be much appreciated!
This works. I don't if that's elegant enough though.
player_scores1 = [{:first_name=>"Bruce", :score => 43, :time => 50},
{:first_name=>"Clark", :score => 45, :minutes => 20}]
player_scores2 = [{:last_name=>"Wayne", :points => 13, :time => 40},
{:last_name=>"Kent", :points => 3, :minutes => 20}]
p (0...[player_scores1.length, player_scores2.length].min).map {|i| {
:full_name => player_scores1[i][:first_name] + " " + player_scores2[i][:last_name],
:score => player_scores1[i][:score] + player_scores2[i][:points]
}}
This example on Codepad.
This uses zip with a block to loop over the hashes, joining the names and summarizing:
all_players = []
player_scores1.zip(player_scores2) { |a, b|
all_players << {
:full_name => a[:first_name] + ' ' + b[:last_name],
:score => a[:score] + b[:points]
}
}
all_players # => [{:full_name=>"Bruce Wayne", :score=>56}, {:full_name=>"Clark Kent", :score=>48}]
We get our data from a sensor which records and stores data like hashes.
At any time it measures a few stuff like that:
{:temperature => 30, :pression => 100, :recorded_at => 14:34:23}
{:temperature => 30, :pression => 101, :recorded_at => 14:34:53}
{:temperature => 31, :pression => 102, :recorded_at => 14:34:24}
{:temperature => 30, :pression => 101, :recorded_at => 14:34:55}
{:temperature => 30, :pression => 102, :recorded_at => 14:34:25}
{:temperature => 31, :pression => 101, :recorded_at => 14:34:56}
We need to be able to export that data on a JSON format, but we have way too much data (the sensor records about every 30 seconds) and we need to remove some of the data. Ideally we'd want to export 1 measure per hour in the last 24 hours so we have something like
{0 => {:temperature => 30, :pression => 100}, 1 => {:temperature => 30, :pression => 100}, 2 => {:temperature => 30, :pression => 100}, 3 => {:temperature => 30, :pression => 100}, 4 => {:temperature => 30, :pression => 100}}
For each hour, the temperature is the average of all temperatures measured within that hour.
Also, if for any reason some data is missing for 1hour, I'd like to to extrapolate it by being the mean between the previous and next hour. Anybody can help?
More functional version (with simple interpolation of missing values)
probs = [{:temperature => .. }] # array of measurings
def average(list, key)
list.reduce(0){|acc,el| acc+el[key]} / list.length unless list.empty
end
prob_groups = probs.group_by{|prob| prob[:recorded_at][0,2].to_i}
average_groups = prob_groups.map do |hour,prob_group|
{ hour => {
:temperature => average(prob_group, :temperature),
:pression => average(prob_group, :pression)
}}
end.reduce{|acc,el| acc.merge(el)}
def interpolate(p, n, key)
(p[key] + n[key])/2 unless p.nil? || n.nil? || p[key].nil? || n[key].nil?
end
resuls = (1..24).map do |hour|
if average_groups[hour]
{ hour => average_groups[hour] }
else
{ hour => {
:temperature => interpolate(average_groups[hour-1], average_groups[hour+1], :temperature),
:pression => interpolate(average_groups[hour-1], average_groups[hour+1], :pression)
}}
end
end.reduce{|acc,el| acc.merge(el)}
Hope it works
something like this
t = [....] - array of measurings
result = {}
(1..24).each do|hour|
# measurings of given hour
measurings = t.select{|measuring| measuring[:recorded_at][0, 2].to_i == hour}
# average temperature of hour
sum = measurings.inject(0){|sum, measuring| sum + measuring[:temperature].to_i}
average_temperature = (measurings.length == 0)? nil: sum/measurings.length.to_f
result[hour] = average_temperature
end
If you are not interested on the history but only on an approximation of actual value(s), consider to use a "moving metric" (http://en.wikipedia.org/wiki/Moving_average).