NEST (2.x) Date Histogram Aggregation with fractional interval values - elasticsearch

I am using NEST (2.3.3) object initializer syntax for creating Date Histogram Aggregation. How can I specify the Fractional values for the Interval?
DateHistogramAggregation dateHistogram =
new DateHistogramAggregation("dateHistogram")
{
Field = "TimestampFieldName",
Interval = DateInterval.Hour
}
In the above data histogram aggregation I want to specify for example 1.5 hours. Is there a way I can do that?

Interval is a Union<DateInterval, Time> which means that it can take either a DateInterval enum value or a Time instance. Additionally, a string has an implicit conversion to an instance of Time. Putting these together, to set an interval of 1.5 hours would be
DateHistogramAggregation dateHistogram =
new DateHistogramAggregation("dateHistogram")
{
Field = "TimestampFieldName",
Interval = new Time("1.5h")
};
In this case, we can't take advantage of the implicit conversion from string to Time (and then Time to Union<DateInterval,Time>) because there is no implicit conversion from string to Union<DateInterval, Time>. In this case, we can just use the Time constructor and pass it a string value for 1.5 hours, and assign this instance of Time to the interval.

Related

Time duration in Google Data Studio

I have some data I collect regarding length of time that's stored in HH:MM format. The data is in relation to sleep patterns (i.e. sleep duration, time fell asleep, etc...).
I am trying to import the data in Google Data Studio (DS) as a numeric variable, but it appears as text. I can see in DS there is a duration (seconds) numeric format, how can I convert a text variable into a numeric one?
It would be easier to convert the fields in a Google Sheet, but I need them as HH:MM for other calculations.
Try this:
0) Create a new Calculated Field
1) Seconds
Use a formula to convert the Time values to a single value in Seconds, where HH:MM:SS represents the field name:
( CAST(REGEXP_EXTRACT(HH:MM:SS, "^(\\d{2})") AS NUMBER ) * 60 * 60 ) + ( CAST(REGEXP_EXTRACT(HH:MM:SS, ":(\\d{2}):") AS NUMBER ) * 60 ) + CAST(REGEXP_EXTRACT(HH:MM:SS, "(\\d{2})$") AS NUMBER )
2) Change Field Type
- Numeric > Duration (sec.)
Credit to Google support community
You can use the TODATE or MINUTE and SECOND function into a calculated field to extract minutes and second from a date. However don't expect to display minutes and second datapoint on a line chart in Data Studio, timeserie charts only support hour-level data at a minimum.

How to get only first part of the date ('DD') from an entire date?

I've converted a date into string. Now, I want to parse a string to get the DD part from DD-MM-YYYY.
For e.g.
If the date is 03-05-2017 (DD-MM-YYYY) then the goal is to get only first part of the string i.e. 03 (DD).
You've tagged this question as a ServiceNow question, so I assume you're using the ServiceNow GlideDateTime Class to derive the date as a string. If that is correct, did you know that you can actually derive the day of the month directly from the GlideDateTime object? You can use the getDayOfMonth(), getDayOfMonthLocalTime(), or getDayOfMonthUTC().
You could of course, also use String.prototype.indxOf() to get the first hyphen's location, and then return everything up to that location using String.prototype.slice().
Or, if you're certain that the day of the month in the string will contain an initial zero, you can simply .slice() out a new string from index 0 through index 2.
var date = '03-05-2017';
var newDate = date.slice(0, 2);
console.log(newDate); //==>Prints "03".
var alternateNewDate = date.slice(0, date.indexOf('-'));
console.log(alternateNewDate); //==>Prints "03".

How to insert previous month of data into database Nifi?

I have data in which i need to compare month of data if it is previous month then it should be insert otherwise not.
Example:
23.12.2016 12:02:23,Koji,24
22.01.2016 01:21:22,Mahi,24
Now i need to get first column of data (23.12.2016 12:02:23) and then get month (12) on it.
Compared that with before of current month like.,
If current month is 'JAN_2017',then get before of 'JAN_2017' it should be 'Dec_2016'
For First row,
compare this 'Dec_2016'[month before] with month of data 'Dec_2016' [23.12.2016].
It matched then insert into database.
EDIT 1:
i have already tried with your suggestions.
"UpdateAttribute to add a new attribute with the previous month value, and then RouteOnAttribute to determine if the flowfile should be inserted "
i have used below expression language in RouteOnAttribute,
${literal('Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec'):getDelimitedField(${csv.1:toDate('dd.MM.yyyy hh:mm:ss'):format('MM')}):equals(${literal('Dec,Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov'):getDelimitedField(${now():toDate(' Z MM dd HH:mm:ss.SSS yyyy'):format('MM'):toNumber()})})}
it could be failed in below data.,
23.12.2015,Andy,21
23.12.2017,Present,32
My data may contains some past years and future years
It matches with my expression it also inserted.
I need to check month with year in data.
How can i check it?
The easiest answer is to use the ExecuteScript processor with simple date logic (this will allow you to use the Groovy/Java date framework to correctly handle things like leap years, time zones, etc.).
If you really don't want to do that, you could probably use a regex and Expression Language in UpdateAttribute to add a new attribute with the previous month value, and then RouteOnAttribute to determine if the flowfile should be inserted into the database.
Here's a simple Groovy test demonstrating the logic. You'll need to add the code to process the session, flowfile, etc.
#Test
public void textScriptShouldFindPreviousMonth() throws Exception {
// Arrange
def input = ["23.12.2016 12:02:23,Koji,24", "22.01.2016 01:21:22,Mahi,24"]
def EXPECTED = ["NOV_2016", "DEC_2015"]
// Act
input.eachWithIndex { String data, int i ->
Calendar calendar = Date.parse("dd.MM.yyyy", data.tokenize(" ")[0]).toCalendar()
calendar.add(Calendar.MONTH, -1)
String result = calendar.format("MMM_yyyy").toUpperCase()
// Assert
assert result == EXPECTED[i]
}
}

How to use oracle TIMESTAMP with fractional seconds in NHibernate?

I have this mapping in my NHibernate.
LASTUPDATE is a TIMESTAMP row, which stores fractional seconds
However, when I have query like
q => q.Where(p => p.LastUpdate > _lastupdate);
where _lastupdate is a DateTime (with fractional seconds, of course) , NHibernate translate _lastupdate as '2012-03-12 8:48:25', losing the fractional seconds.
How can I fix this situation?
You have to explicitly map your property as type="Timestamp"
See 5.2.2. Basic value types

RavenDB time based query

I have a document structure JobData that stores time based data starting from time 0 to time t in Ticks. And usually the data is a document per second.
public class JobData
{
long Ticks {get;set;}
double JobValue {get;set;}
}
For simplicity I am showing only one parameter JobValue, but in reality it is a complex graph of data. My question is if given a given an input time in Ticks, what kind of query would be the best for finding the last JobData based on a given tick?
So if the database has a document at 1000 ticks and then the next one at 2000 ticks, and the user wants to find the state at 1500 ticks, he/she should get the JobData at 1000 ticks as the answer.
The query I am using now is:
var jobData = documentSession.Query<JobData>().Where(t => t.Ticks <= 1500).OrderByDescinding(t => t.Ticks).FirstOrDefault();
Is this the right and most efficient query? I have thousands of these JobData nodes and want to just get to the one that is the closest.
Thanks!
Ahmad,
Yes, that is the way to go about it. And it would be very fast.

Resources