Let's suppose I have the following data on ElasticSearch:
#timestamp; userId; currentPoints
August 7th 2017, 00:30:37.319; myUserName; 4
August 7th 2017, 00:43:22.121; myUserName; 10
August 7th 2017, 00:54:29.177; myUserName; 7
August 7th 2017, 01:10:29.352; myUserName; 4
August 7th 2017, 00:32:37.319; myOtherUserName; 12
August 7th 2017, 00:44:22.121; myOtherUserName; 17
August 7th 2017, 00:56:29.177; myOtherUserName; 8
August 7th 2017, 01:18:29.352; myOtherUserName; 11
I'm looking to draw a date histogram that will show me the sum of all max:currentPoints per username per hour, which whould generate the following data to plot:
August 7th 2017, 00; SumOfMaxCurrentPoints -> 27 (max from hour 00h from both users 10 + 17)
August 7th 2017, 00; SumOfMaxCurrentPoints -> 15 (max from hour 01h from both users 4 + 11)
This would usually be done with a subquery, extracting the max(currentPoints) for each hour, user and then sum the results and aggregate per hour.
Is this possible with Kibana Timelion for instance? I can't find a way to achieve this using the documentation.
Thanks
Alex
While working on another project, I've stumpled upon the answer to do this in Kibana/Elasticsearch without using Timelion.
The feature is called Sibling Pipeline Aggregation, and in this case you use the Sum Bucket. You can use it with any recent Kibana/Elastic visualization (I'm using version 5.5).
For a dataset such as:
#timestamp; userId; currentPoints
August 7th 2017, 00:30:37.319; myUserName; 4
August 7th 2017, 00:43:22.121; myUserName; 10
August 7th 2017, 00:54:29.177; myUserName; 7
August 7th 2017, 01:10:29.352; myUserName; 4
August 7th 2017, 00:32:37.319; myOtherUserName; 12
August 7th 2017, 00:44:22.121; myOtherUserName; 17
August 7th 2017, 00:56:29.177; myOtherUserName; 8
August 7th 2017, 01:18:29.352; myOtherUserName; 11
Where you want an hourly SUM of(currentPoints) all MAXs(currentPoints) per userId, resulting in:
August 7th 2017, 00; SumOfMaxCurrentPoints -> 27 (max from hour 00h from both users 10 + 17)
August 7th 2017, 00; SumOfMaxCurrentPoints -> 15 (max from hour 01h from both users 4 + 11)
You can do:
Metric
Aggregation: Sibling Pipeline Aggregation (Sum Bucket)
Bucket Aggregation Type: Terms
Bucket Field: userId
Bucket Size: Comfortable value above the # of users if you want total precision
Metric Aggregation: Max
Metric Field: currentPoints
Bucket
Buckets type: Split Rows
Bucket Aggregation: Date Histogram
Histogram Field: #timestamp
Histogram Interval: Hourly
Related
d3.timeParse can parse the time format, but no matter how the format is set, the result contains the year, month, day, hour, minute, and second.
Like this: Sat Jan 01 2020 00:00:00
It can't only correspond to the month and day, if i want to use d3.scaleTime () to scale to compare data of different years, how can I implement it?
For example:
===axis=======
x-axis: 12 month 366 day
y-axis: [max(value), mim(value)]
===data=======
data 1: 2016 = [{ date:2016/01/01,value=18},{ date:2016/01/02,value=199},...,{ date:2016/12/31,value=27}] (366 day)
data 2: 2017 = [{ date:2017/01/01,value=58},{ date:2017/01/02,value=9},...,{ date:2017/12/31,value=217}] (365 day)
data 3: 2018 (365 day)
.
.
data n: 2020 (366 day)
I'm a little bit confused about the correct usage of Calendar of Apple Foundations framework.
let calendar = Calendar(identifier: .iso8601)
let dayComponent = DateComponents(year: 2019, weekday: 1, weekOfYear: 6)
let date = calendar.date(from: dayComponent)
I need to get the first day of a given week of year. When using the code above the following dates are given depending on weekday:
//weekday:
//0 -> 08 FEB
//1 -> 09 FEB
//2 -> 03 FEB
//3 -> 04 FEB
//4 -> 05 FEB
Why does weekday starts with 0 at the current week (6) while switching to week 5 when increased?
Thanks for any help.
A couple of observations:
When iterate through weekdays, you want to go from 1 through 7, because, “Day, week, weekday, month, and year numbers are generally 1-based...” Date and Time Programming Guide: Date Components and Calendar Units. You can use range(of:in:for:), maximumRange(of:), etc., to find the range of possible values.
The weekday values from 1 through 7 do not mean “first day of the week”, “second day of the week”, etc. They refer to specific days of the week, e.g. for .iso8601, “Sun” is 1, “Mon” is 2, etc.
Make sure when you use weekOfYear, you use yearForWeekOfYear:
let calendar = Calendar(identifier: .iso8601)
let firstOfWeek = DateComponents(calendar: calendar, weekOfYear: 6, yearForWeekOfYear: 2019).date!
Your code is iterating through the weekdays. Consider this code that enumerates all of the days of the week for the sixth week of 2019 (i.e. the week starting Monday, February 4th and ending Sunday, February 10th):
let weekdays = calendar.range(of: .weekday, in: .weekOfYear, for: firstOfWeek)!
let daysOfTheWeek = Dictionary(uniqueKeysWithValues: zip(weekdays, calendar.weekdaySymbols))
for weekday in weekdays {
let date = DateComponents(calendar: calendar, weekday: weekday, weekOfYear: 6, yearForWeekOfYear: 2019).date!
print("The", daysOfTheWeek[weekday]!, "in the 6th week of 2019 is", formatter.string(from: date))
}
That results in:
The Sun in the 6th week of 2019 is Sunday, February 10, 2019
The Mon in the 6th week of 2019 is Monday, February 4, 2019
The Tue in the 6th week of 2019 is Tuesday, February 5, 2019
The Wed in the 6th week of 2019 is Wednesday, February 6, 2019
The Thu in the 6th week of 2019 is Thursday, February 7, 2019
The Fri in the 6th week of 2019 is Friday, February 8, 2019
The Sat in the 6th week of 2019 is Saturday, February 9, 2019
This is effectively what your code does and why you’re not seeing what you expected.
If you want iterate through the seven days of the week in order, just get the start of the week and then offset it from there:
let calendar = Calendar(identifier: .iso8601)
let startOfWeek = DateComponents(calendar: calendar, weekOfYear: 6, yearForWeekOfYear: 2019).date!
for offset in 0 ..< 7 {
let date = calendar.date(byAdding: .day, value: offset, to: startOfWeek)!
print(offset, "->", formatter.string(from: date))
}
That results in:
0 -> Monday, February 4, 2019
1 -> Tuesday, February 5, 2019
2 -> Wednesday, February 6, 2019
3 -> Thursday, February 7, 2019
4 -> Friday, February 8, 2019
5 -> Saturday, February 9, 2019
6 -> Sunday, February 10, 2019
You asked:
I need to get the first day of a given week of year.
Probably needless to say at this point, but just omit the weekday:
let startOfWeek = DateComponents(calendar: calendar, weekOfYear: 6, yearForWeekOfYear: 2019).date!
Also see Date and Time Programming Guide: Week-Based Calendars.
My customer has an event each second Monday of each month.
I need to mark them with red in calendar.
How do i "cleanly" find out the date of that Mondays?
Here's my version.
If the eighth of the month is a Monday, then it is the second Monday. If it is not a Monday, then how many days until the next Monday?
oct_2012 = Date.new 2012, 10, 8
oct_2012.wday # => 1, We're done!
nov_2012 = Date.new 2012, 11, 8
nov_2012.wday # => 4
nov_2012 + (8 - nov_2012.wday) # => 2012-11-12
Does that help?
Edit
Easier version: Just add and be done. This algorithm works even if the month starts on a Monday.
oct_2012 = Date.new 2012, 10, 1
oct_2012 + (8 - oct_2012.wday) # => 2012-10-08
nov_2012 = Date.new 2012, 11, 1
nov_2012 + (8 - nov_2012.wday) # => 2012-11-12
One rule and done!
You second Monday will always fall within the 8th and 14th of each month.
I have an object Report that contains two integer fields: Month and Year.
I need to sort it by "date"
Report.desc(:year).desc(:month).each do |a|
puts a.year.to_s + " " + a.month.to_s
end
results:
2011 12
2011 11
2012 7
2012 6
2012 5
2012 4
2012 3
2012 2
2012 1
While I would think to get
2012 7
2012 6
2012 5
2012 4
2012 3
2012 2
2012 1
2011 12
2011 11
What am I doing wrong?
The Mongoid Criteria looks like:
irb(main):043:0> Report.desc(:year).desc(:month)
=> #<Mongoid::Criteria
selector: {},
options: {:sort=>{"year"=>-1, "month"=>-1}},
class: Report,
embedded: true>
The result you get is only sorted by month not by the whole date(thus the result you get). Maybe use the sort_by method with a body that takes into account both the year and the month?
Something like:
Report.sort_by{|t| [-t.year, -t.month]}
EDIT: I am using the negatives of both the year and the month to achieve decreasing order.
I can only find algorithm for getting ISO 8601 week (week starts on a Monday).
However, the iCal spec says
A week is defined as a seven day period, starting on the day of the
week defined to be the week start (see WKST). Week number one of the
calendar year is the first week that contains at least four (4) days
in that calendar year.
Therefore, it is more complex than ISO 8601 since the start of week can be any day of the week.
Is there an algorithm to determine what is the week number of a date, with a custom start day of week?
or... is there a function in iCal4j that does this? Determine a weekno from a date?
Thanks!
p.s. Limitation: I'm using a JVM language that cannot extend a Java class, but I can invoke Java methods or instantiate Java classes.
if (input_date < firstDateOfTheYear(WKST, year))
{
return ((isLeapYear(year-1))?53:52);
}
else
{
return ((dayOfYear(input_date) - firstDateOfTheYear(WKST, year).day)/7 + 1);
}
firstDateOfTheYear returns the first calendar date given a start of week(WKST) and the year, e.g. if WKST = Thursday, year = 2012, then it returns Jan 5th.
dayOfYear returns sequencial numerical day of the year, e.g. Feb 1st = 32
Example #1: Jan 18th, 2012, start of week is Monday
dayOfYear(Jan 18th, 2012) = 18
firstDateOfTheYear(Monday, 2012) = Jan 2nd, 2012
(18 - 2)/7 + 1 = 3
Answer Week no. 3
Example #2: Jan 18th, 2012, start of week is Thursday
dayOfYear(Jan 18th, 2012) = 18
firstDateOfTheYear(Thursday, 2012) = Jan 5th, 2012
(18 - 5)/7 + 1 = 2
Answer Week no. 2
Example #3: Jan 1st, 2012, start of week is Monday
firstDateOfTheYear(Monday, 2012) = Jan 2nd, 2012
IsLeapYear(2012-1) = false
Jan 1st, 2012 < Jan 2nd, 2012
Answer Week no. 52
Let daysInFirstWeek be the number of days on the first week of the year that are in January. Week starts on a WKST day. (e.g. if Jan 1st is a WKST day, return 7)
Set dayOfYear to the n-th days of the input date's year (e.g. Feb 1st = 32)
If dayOfYear is less than or equal to daysInFirstWeek
3.1. if daysInFirstWeek is greater than or equal to 4, weekNo is 1, skip to step 5.
3.2. Let daysInFirstWeekOfLastYear be the number of days on the first week of the previous year that are in January. Week starts on a WKST day.
3.3. if daysInFirstWeekOfLastYear is 4 or last year is Leap year and daysInFirstWeekOfLastYear is 5, weekNo is 53, otherwise weekNo is 52, skip to step 5.
Set weekNo to ceiling((dayOfYear - daysInFirstWeek) / 7)
4.1. if daysInFirstWeek greater than or equal to 4, increment weekNo by 1
4.2. if daysInFirstWeek equal 53 and count of days on the first week (starting from WKST) of January in the year of inputDate's year + 1 is greater than or equal to 4, set weekNo to 1
return weekNo