I know one can add milliseconds to a date for adding days or weeks:
https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#now
But since months' lengths are different, that will not work. How can I add 6 months to the now() function of NiFi?
Unfortunately doing this reliably (as you've noted) is not really ideal using only Expression Language.
The most reliable method to do this would be:
UpdateAttribute to set a new attribute with ${now()}
ExecuteGroovyScript with a simple piece of Groovy to
Get the attribute
Use Groovy date/time functions to safely add to the date
Set the attribute value
Cookbook Part 1 covers how to work with FF Attributes.
E.g.
flowFile = session.get()
if(!flowFile) return
my_date = flowFile.getAttribute('my_date')
// Modify your date here
flowFile = session.putAttribute(flowFile, 'my_date', my_date)
this code subtract 60 days from a attribute named 'date_original' and add new attribute with the new date
flowFile = session.get()
if(!flowFile) return
formatoAccettato = "yyyy-MM-dd"
dataOriginale = flowFile.getAttribute('data_originale')
dataOriginaleDate = Date.parse(formatoAccettato, dataOriginale)
sottrazione = dataOriginaleDate - 60
dataSottratta = sottrazione.format(formatoAccettato)
flowFile = session.putAttribute(flowFile, 'data_modificata', dataSottratta)
session.transfer(flowFile, REL_SUCCESS)
After needing this time and again, I just decided to do it the workaround way without using external scripts which potentially break with upgrades and adds complexity. The below deducts one month.
use below for testing dates:
${literal('2022-01'):toDate('yyyy-MM')} #202112
${literal('2022-09'):toDate('yyyy-MM')} #202208
${literal('2022-11'):toDate('yyyy-MM')} #202210
${this_month:equals("1"):ifElse("12",${this_month:minus(1)})}
${this_month:equals("1"):ifElse(${this_year:minus(1)},${this_year})}
${new_year:append(${new_month:padLeft(2,"0")})}
Hope this helps someone somewhere out there.
Related
I have a search query for Elasticsearch which uses a groovy inline script that I need to convert to Painless language
mdt= doc.eventstartdate.value;
dtevent = new DateTime(mdt).withTime(0,0,0,0).getMillis();
d = (dtevent<dtnow?dtnow:dtevent);
As you can see, this grabs the 'eventstartdate', strips the time off (sets to 00:00:00)
Then it compares with current date (dtnow) and if less than current date, changes it to the current date (effectively there are no past dates, just today onwards)
I'm not a Java programmer and I believe the date processing has changed in Painless so looking to convert the above?
Thanks
Fixed this with:
Instant startDate = Instant.ofEpochMilli(doc.eventstartdate.date.millis);
long startDateMS = startDate.truncatedTo(ChronoUnit.DAYS).getEpochSecond() * 1000;
def d = (startDateMS<params.dtnow?params.dtnow:startDateMS);
I have data in which i need to compare month of data if it is previous month then it should be insert otherwise not.
Example:
23.12.2016 12:02:23,Koji,24
22.01.2016 01:21:22,Mahi,24
Now i need to get first column of data (23.12.2016 12:02:23) and then get month (12) on it.
Compared that with before of current month like.,
If current month is 'JAN_2017',then get before of 'JAN_2017' it should be 'Dec_2016'
For First row,
compare this 'Dec_2016'[month before] with month of data 'Dec_2016' [23.12.2016].
It matched then insert into database.
EDIT 1:
i have already tried with your suggestions.
"UpdateAttribute to add a new attribute with the previous month value, and then RouteOnAttribute to determine if the flowfile should be inserted "
i have used below expression language in RouteOnAttribute,
${literal('Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov,Dec'):getDelimitedField(${csv.1:toDate('dd.MM.yyyy hh:mm:ss'):format('MM')}):equals(${literal('Dec,Jan,Feb,Mar,Apr,May,Jun,Jul,Aug,Sep,Oct,Nov'):getDelimitedField(${now():toDate(' Z MM dd HH:mm:ss.SSS yyyy'):format('MM'):toNumber()})})}
it could be failed in below data.,
23.12.2015,Andy,21
23.12.2017,Present,32
My data may contains some past years and future years
It matches with my expression it also inserted.
I need to check month with year in data.
How can i check it?
The easiest answer is to use the ExecuteScript processor with simple date logic (this will allow you to use the Groovy/Java date framework to correctly handle things like leap years, time zones, etc.).
If you really don't want to do that, you could probably use a regex and Expression Language in UpdateAttribute to add a new attribute with the previous month value, and then RouteOnAttribute to determine if the flowfile should be inserted into the database.
Here's a simple Groovy test demonstrating the logic. You'll need to add the code to process the session, flowfile, etc.
#Test
public void textScriptShouldFindPreviousMonth() throws Exception {
// Arrange
def input = ["23.12.2016 12:02:23,Koji,24", "22.01.2016 01:21:22,Mahi,24"]
def EXPECTED = ["NOV_2016", "DEC_2015"]
// Act
input.eachWithIndex { String data, int i ->
Calendar calendar = Date.parse("dd.MM.yyyy", data.tokenize(" ")[0]).toCalendar()
calendar.add(Calendar.MONTH, -1)
String result = calendar.format("MMM_yyyy").toUpperCase()
// Assert
assert result == EXPECTED[i]
}
}
I've created a Google sheet to keep a list of work tasks with a column to track the date on which items are created, and built a script to automatically populate the cells in that column with the day's date when a new line is inserted.
The cell (e.g. G9) that is target of the script uses the following validation formula to make sure that when users change the date, they use a date that is neither a weekend nor in the future:
=and(isdate(G9), weekday(G9,2)<6, G9<=today())
IT ONLY WORKS BUT ONLY IF THE SCRIPT IS RUN ANYTIME AFTER 8:00am ! If I try using it any earlier the cell validation will reject the input!
The script looks like this (curRow is the number of the row that's been added):
// Adds today's date without using =today()
var myrangename = "G"+curRow;
var dateCell = sheet.getRange(myrangename);
var d = new Date();
var dateArr = [];
dateArr[0]=d.getFullYear();
dateArr[1]=d.getMonth() + 1; //Months are zero based
dateArr[2]=d.getDate();
dateCell.setValue(dateArr.join('/'));
(n.b.: I cannot use the script to simply put =today() in the cell because all the entries would change every day. )
WHY DOES IT ONLY WORK AFTER 8:00AM? Is Google somehow running on a different time zone than my computer?? I'm based in the UK, so using BST, but that shouldn't be a problem, shouldn't it...?
Try
var d = new Date();
var d = Utilities.formatDate(d, "GMT+1", "yyyy-MM-dd HH:mm:ss");
I am not sure if google would recognise BST as a time zone, but you could also try
var d = Utilities.formatDate(d, "BST", "yyyy-MM-dd HH:mm:ss");
Thank you for your suggestion, Aprillion. Turns out that a Google Sheets file has its own internal time-zone setting! which in my case was set to American Pacific time (so 8hrs behind)
(You'd think it would pick up the date and time info automatically from Windows, like other applications do!)
To set the sheet's time-zone to the correct one, you need to go to the main menu, click 'File', then 'Spreadsheet settings...', and adjust as necessary.
The script and validation now all work fine.
Thank you all for your help.
I have a table in Mongo. One of the fields is a DateTime. I would like to be able to get all of the records that are only for a single day (i.e. 9/3/2011).
If I do something like this:
var list = (from c in col
where c.PublishDate == DateTime.Now
select c).ToList();
Then it doesn't work because it is using the time in the comparison. Normally I would just compare the ToShortDateString() but NoRM does not allow me to use this.
Thoughts?
David
The best way to handle this is normally to calculate the start datetime and end datetime for the date in question and then query for values in that range.
var start = DateTime.Now.Date;
var end = start.AddDays(1);
...
But you'd also be well advised to switch to the official C# driver now. You should also use UTC datetimes in your database (but that gets more complicated).
I have a table with many anniversaries : Date + Name.
I want to display the next anniversary and the one after with Linq.
How can i build the query ?
I use EF
Thanks
John
Just order by date and then use the .Take(n) functionality
Example with a list of some objects assuming you want to order by Date then Name:
List<Anniversaries> annivDates = GetAnnivDates();
List<Anniversaries> recentAnniv = annivDates.OrderBy(d => d.Date).ThenBy(d => d.Name).Take(2).ToList();
If the anniversaries are stored in regular DateTime structs, they may have the 'wrong' year set (i.e. wedding or birth year). I suggest writing a function which calculates the next date for an anniversary (based on the current day) like:
static DateTime CalcNext(DateTime anniversary) {
DateTime newDate = new DateTime(DateTime.Now.Year, anniversary.Month, anniversary.Day);
if (newDate < DateTime.Now.Date)
newDate = newDate.AddYear(1);
return newDate;
}
Then you proceed with sorting the dates and taking the first two values like described in the other postings:
(from e in anniversaries orderby CalcNext(e.Date) select e).Take(2)