Unusual datetime string parsing in Hive - hadoop

Hi I am trying to parse the following string in hive
"2016-09-30T21:59:58.093Z"
I would like to extract the year, month, day and hour from it.
I can use the year(), day() and to_date() function to extract up to the day, but I can't get the hour from it. If I do
hour("2016-09-30T21:59:58.093Z")
or
unix_timestamp("2016-09-30T21:59:58.093Z")
they will return NULL. Can someone suggest something?
thank you

hive> select from_unixtime(unix_timestamp("2016-09-30T21:59:58.093Z","yyyy-MM-dd'T'HH"),"yyyy-MM-dd HH");
OK
2016-09-30 21

You can use translate("2016-09-30T21:59:58.093Z", "T"," ") for the output

Related

How to calculate total time in hour using DAX?

I have to table, start and end with time as the data, I want to calculate the total hour from start to end.
I tried this but it does not return hour, it returns date and the date also same in the column:
Column = SUM('Data'[Finish Date])-SUM('Data'[Start Date])*24-12
Anyone can give me idea please. Thank you so much
Read the docs: https://learn.microsoft.com/en-us/dax/datediff-function-dax
Column = DATEDIFF('Data'[Start Date], 'Data'[Finish Date], HOUR)

How do I sort such a format of Date Timestamp on Google Spreadsheet?

The image is the format of my Date Timestamp on gSpreadsheet. I have tried to sort it from earliest to latest date timestamp but it is not sorting well. Is there any ways to help me in this? Thank you, appreciate it.
try:
=SORT(A2:A; INDEX(SPLIT(A2:A; ",");;1); 0; INDEX(SPLIT(A2:A; " ");;2), 0)
where you can change 0 (desc) to 1 (asc)
Note!
You are using an invalid date format, time date should be a number and formated as date like so
To sort as is:
Use this formula
=QUERY(SORT({ A2:A, IF(A2:A="",,ArrayFormula(INDEX(ArrayFormula(SPLIT(SUBSTITUTE(A2:A,"GMT",""), ",")),,1)+INDEX(ArrayFormula(SPLIT(SUBSTITUTE(A2:A,"GMT",""), ",")),,2)))}, 2,1), " Select Col1 ")

Azure Data Factory todate Day -1

I have some Expression to get date UTC+7 (Jakarta time) like this
toDate(toString((currentUTC()+ hours(7))),'yyyy-MM-dd HH:mm:ss')
and for example output is:
2021-12-28 20:15:23
I want from this expression is add day -1, and the output is:
2021-12-27 20:15:23
many big thanks for answer the question
You could try the below :
addDays(toDate(toString((currentUTC()+ hours(7))),'yyyy-MM-dd HH:mm:ss'),-1)

Retrieve database records between two weekdays

I have several records in my database, the table has a column named "weekday" where I store a weekday like "mon" or "fri". Now from the frontend when a user does search the parameters posted to the server are startday and endDay.
Now I would like to retrieve all records between startDay and endDay. We can assume startDay is "mon" and endDay is "sun". I do not currently know how to do this.
Create another table with the names of the days and their corresponding number. Then you'd just need to join up your current table with the days table by name, and then use the numbers in that table to do your queries.
Not exactly practical, but it is possible to convert sun,mon,tue to numbers using MySQL.
Setup a static year and week number like 201610 for the 10th week of this year, then use a combination of DATE_FORMAT with STR_TO_DATE:
DATE_FORMAT(STR_TO_DATE('201610 mon', '%X%V %a'), '%w')
DATE_FORMAT(STR_TO_DATE('201610 sun', '%X%V %a'), '%w')
DATE_FORMAT(STR_TO_DATE('201610 tue', '%X%V %a'), '%w')
These 3 statements will evaluate to 0,1,2 respectively.
The main thing this is doing is converting the %a format (Sun-Sat) to the %w format (0-6)
well i don't know the architecture of your application as i think storing and querying a week day string is not appropriate, but i can tell you a work around this.
make a helper function which return you an array of weekdays in the range i-e
function getWeekDaysArray($startWeekDay, $endWeekDay) {
returns $daysArray['mon','tue','wed'];
}
$daysRangeArray = getWeekDaysArray('mon', 'wed');
now with this array you can query in table
DB::table('TableName')->whereIn('week_day', $daysRangeArray)->get();
Hope this help

How can I group on the created_at field and return only the Date and hour

I run the following in Rails:
coe = trackers.where(:trackable_type => "Mailing").count(:group => 'DATE(created_at)')
which returns a hash with the items nicely grouped.
However I want to group not by date only by date but by the hour, but not the minutes or seconds.
How can I run this query so all the heavy lifting is done by Postgresql and not convert after I get the data set back?
My mistake I want to group by Date and Hour. Sorry for the confusion.
It sounds like you want the date_trunc() function:
http://www.postgresql.org/docs/current/interactive/functions-datetime.html#FUNCTIONS-DATETIME-TABLE
I want to group not by date but by the hour, but not the minutes or
seconds.
Bold emphasis mine.
So date_trunc()is probably not what you want, but EXTRACT or date_part():
coe = trackers.where(:trackable_type => "Mailing").count(:group => 'EXTRACT (hour FROM created_at)')

Resources