i know that Processor can record and clear state.If i wanna set a specificed state,for example,reseting the state to point to the tenth row of the column.IDK whether can do that?
Any help is appreciated!
You can't do this since it doesn't save previous data.
Think of it as a map with phy_asset#!#created_time as key and 2020-03-23 13:29:05.0 as value. It doesn't know the previous values that it saved for this key.
It also cannot be changed by anyone but the processor.
What you can do, in your case at least, is to set Additional WHERE clause to created_time >= '<wanted-minimum_creation_time>' which would only query those with greater creation time than what you've chosen and will achieve the same result as you want.
Don't forget to reset the state though!
Related
I have time series data in Elasticsearch and I want to aggregate it to create histogram. What I want to achieve is to fill the null buckets with the value of the previous data point. I know that I can use min_doc_count: 0 but it will put the value as 0 and I couldn't find any out of the box way to do this via Elastic. May be there is some trick that I am not aware of?
Appreciate your feedback.
I think the Date Histogram Aggregation does not provide a native way to perform what you would like.
The closest thing I can think of is using missing value. However, this will set a static value to all the dates where no values are found, which is not exactly what you want.
I also thought of using Painless with the following logic:
Get the first value in the Histogram and store it in a variable current.
If the next value is different to 0, store this value to current.
If the value is 0, set the current value to the histogram date. Don't change current.
Repeat step 2 until you finish the Histogram.
Using painless, in my experience is really painful but you can consider it as an alternative.
Additionally, I would recommend you to limit ES to perform searches and aggregations. If you require additional logic to the output, consider performing it outside ES. You can use the Python ES Client for instance.
I can think of the following script with a similar logic as the Painless scenario:
current = 0
results = es.search(...)
for i in res["aggregations"]["my_histogram_name"]["buckets"]:
if not i["doc_count"]: #this is the same as "if i["doc_count"]==0"
i["doc_count"] = current
current = i["doc_count"] #changed or not, we always use the last value to current
After that, the histogram should look as you want and ready to be displayed.
Hope this is helpful! :)
I have a dataset like below.
Dataset:
Now the business logic is to find out the last paid date for each of the loans. I tried using a tmap component, it calls a java routine that has a static variable last_paid_dt which would store the transaction date when the daily deposit > 0. However, when the daily deposit is less than 0 the static var would not get changed. This works fine when the amount paid is 0.
Issue - See the red highlighted values in the table below
When the amount paid is reversed a day or after, the last paid should be from previous non-reversed positive amount. I was not able to get that done.
Also when a new loan id starts processing I need the static variable to get reset which is not currently happening.
If my current methodology is wrong, please help me doing in a better and efficient way. Thanks
Expected output:
First of all you need to use a Map component, with the key being the loanId.
You don't want to overwrite the value. I.e. If the key exists in your map, then do not overwrite it with a new value.
You can use the globalMap if you want, in that case I'd do:
globalMap.get("loan_"+loanId) != null ?
globalMap.put("loand_"+loanId,loanDate) : loanDate
then later:
globalMap.get("loan_"+loanId)
Not elegant, but works. a More elegant would be to define your own map that you put into globalMap and after the process null it out, so you free up the memory. But this all depends on the complexity of your job.
First, I don't have unique value to directly point out a value I want to retrieve, so there for, I cannot use the WHERE Statement. The reason for that, because I have a database where I constantly updated or deleted, so if I use WHERE statement I have to edit the code again.
Then do apply a unique value - add a field of data type AutoNumber or you will never get out of this mess.
Please help me with the following SAS problem. I need to transform my data set from "original" to "new" as shown in the picture. Because the "priority" variable can not be sorted, it seems that first. and last. variables would not work here, no? The goal is to have each sequence of priorities represent one entry in the "new" dataset.
Thank you!
p.s. I did not know how to create a table in this post so I just took a snapshot of the screen.
Seems fairly straightforward to me. Just create a batch ID.
data have_pret;
set have;
by subject;
if first.subject then batchID=0;
if priority=1 then batchID+1;
run;
Then you can transpose by subject/batchID. This assumes priority always starts at 1 - if it starts at > 1 sometimes, you may want to adjust your logic and keep track of prior value of priority.
I have several workflows where the users want to set the number of days before a date defined in an entity (e.g. contract end date) to receive a notice. Within a Wait Condition, is there a way to set the number of days before to an int attribute in the entity, rather than hard code it? Any workarounds if that is not possible?
No, unfortunately there is no way to do this out of th ebox. The best way for you to do this, assuming you're using a dropdown for the day values, would be to code each branch of wait logic for each selected value in the dropdown.