Is there any way to get the current date -1 in Hive means yesterdays date always?
And in this format- 20120805?
I can run my query like this to get the data for yesterday's date as today is Aug 6th-
select * from table1 where dt = '20120805';
But when I tried doing this way with date_sub function to get the yesterday's date as the below table is partitioned on date(dt) column.
select * from table1 where dt = date_sub(TO_DATE(FROM_UNIXTIME(UNIX_TIMESTAMP(),
'yyyyMMdd')) , 1) limit 10;
It is looking for the data in all the partitions? Why? Something wrong I am doing in my query?
How I can make the evaluation happen in a subquery to avoid the whole table scanned?
Try something like:
select * from table1
where dt >= from_unixtime(unix_timestamp()-1*60*60*24, 'yyyyMMdd');
This works if you don't mind that hive scans the entire table. from_unixtime is not deterministic, so the query planner in Hive won't optimize for you. For many cases (for example log files), not specifying a deterministic partition key can cause a very large hadoop job to start since it will scan the whole table, not just the rows with the given partition key.
If this matters to you, you can launch hive with an additional option
$ hive -hiveconf date_yesterday=20150331
And in the script or hive terminal use
select * from table1
where dt >= ${hiveconf:date_yesterday};
The name of the variable doesn't matter, nor does the value, you can set them in this case to get the prior date using unix commands. In the specific case of the OP
$ hive -hiveconf date_yesterday=$(date --date yesterday "+%Y%m%d")
In mysql:
select DATE_FORMAT(curdate()-1,'%Y%m%d');
In sqlserver :
SELECT convert(varchar,getDate()-1,112)
Use this query:
SELECT FROM_UNIXTIME(UNIX_TIMESTAMP()-1*24*60*60,'%Y%m%d');
It looks like DATE_SUB assumes date in format yyyy-MM-dd. So you might have to do some more format manipulation to get to your format. Try this:
select * from table1
where dt = FROM_UNIXTIME(
UNIX_TIMESTAMP(
DATE_SUB(
FROM_UNIXTIME(UNIX_TIMESTAMP(),'yyyy-MM-dd')
, 1)
)
, 'yyyyMMdd') limit 10;
Use this:
select * from table1 where dt = date_format(concat(year(date_sub(current_timestamp,1)),'-', month(date_sub(current_timestamp,1)), '-', day(date_sub(current_timestamp,1))), 'yyyyMMdd') limit 10;
This will give a deterministic result (a string) of your partition.
I know it's super verbose.
Related
I am trying to separate the time and date in one column to be independent off each other. I am new at writing scripts
this is my query:
select
*
from
[tablename]
where
to_date([column_name]) in ( '15-Jun-2021', '16-Jun-2021' )
and
to_char([column_name],'dd-Mon-yyyy HH:MM:ss') < '15-Jun-2021 19:54:30'
The way you put it, it would be
select *
from your_table
where date_column >= date '2021-06-15'
and date_column < to_date('15.06.2021 19:54:30', 'dd.mm.yyyy hh24:mi:ss')
because
date_column should be of date datatype. If it isn't, you'll have problems of many kinds in the future. Therefore,
don't to_date it, it is already a date
don't to_char it either, because you'd be comparing strings and get unexpected result. Use that function when you want to nicely display the result
the second condition you wrote makes the first one questionable. If date_column is less than value you wrote, then you can omit date '2021-06-16' from the first condition because you won't get any rows for that date anyway
date literal (date '2021-06-15') sets time to midnight, so condition I wrote should return rows you want
SQL> select date '2021-06-15' first,
2 to_date('15.06.2021 19:54:30', 'dd.mm.yyyy hh24:mi:ss') second
3 from dual;
FIRST SECOND
------------------- -------------------
15.06.2021 00:00:00 15.06.2021 19:54:30
SQL>
I need to restrict a query with a
SELECT ... FROM ...
WHERE my_date=(RESULT FROM A SELECT)
... ;
in order to achieve that I am using as result of the select a timestamp (if I instead use a datetime I get nothing from my select probably because the format I am using trims the datetime at the second).
Sadly this is not working because these kindo of queries:
select DISTINCT TO_DATE(TO_TIMESTAMP(TO_DATE('25-10-2017 00:00', 'dd-MM-yyyy HH24:MI'))) from DUAL;
return an
ORA-01830: date format picture ends before converting entire input string
how to deal with timestamp to date conversion?
If you want to just compare and check only he dates use trunc on both LHS and RHS.
SELECT ... FROM ...
WHERE trunc(my_date)=(select trunc(RESULT) FROM A)
... ;
This will just compare the dates by truncating the timestamp values
You can use the combination of "TRUNC" and "IN" keywords in your query to achieve what you are expecting. Please check the below query sample as a reference.
SELECT * FROM customer WHERE TRUNC(last_update_dt) IN (select DISTINCT (TRUNC(last_update_dt)) from ... )
Cheers !!
I am using Oracle XE on my machine.
Defined a table as :
Name Type
ENAME VARCHAR2(20)
DOJ DATE
Firing a simple select:
select * from test1.tasty1;
ENAME DOJ
sat 08-DEC-16
So ok - I am aware that DATE field has time component in it.
The following query using TRUNC works fine:
select * from test1.tasty1 where trunc(DOJ) = '08-DEC-16';
Now I wanted to test the 'to_date' function - but none of the below queries worked - wonder why ?
select * from test1.tasty1 where DOJ =
to_date('08-12-2016','DD-MM-YYYY');
select * from test1.tasty1 where
DOJ = to_date('08-DEC-2016','DD-MON-YYYY');
select * from
test1.tasty1 where DOJ = to_date('08-DEC-16','DD-MON-YY');
select
* from test1.tasty1 where DOJ = to_date('08-DEC-16','dd-mon-RR');
Had taken a look at the following on SO:
Oracle TO_DATE not working
so not sure what is wrong here ?
From your question and comments, it appears that this is the sequence of events which happened.
You did the following INSERT into your table:
INSERT INTO test1.tasty1 VALUES ('sat', SYSDATE)
Keep in mind that dates in Oracle have both a date and a time component. So even though you did insert the date '2016-12-08' you also inserted a time component. As a result, the following query is not returning any records:
SELECT * FROM test1.tasty1 WHERE DOJ = '2016-08-12'
This is probably because you never specified the time component, and therefore the record you inserted earlier is not matching. If you want to compare only the date portion, you can use TRUNC as follows:
SELECT * FROM test1.tasty1 WHERE TRUNC(DOJ) = '2016-08-12'
The solution to your problem moving forward would be to wrap SYSDATE with TRUNC during the insert, if you really only want to deal with the date components.
By the way, the format '08-DEC-16' used as a literal will not be recognized by Oracle as a valid date. Instead, use '2016-12-08'.
Have you tried like this as comparison of date with date is correct:
select * from test1.tasty1 where to_date(DOJ,'DD-MM-YYYY') = to_date('08-12-2016','DD-MM-YYYY');
Compare apples with apples and not with mangoes.
What it worked from is instead of make a date to compare, change the date column to char with to_char(datecol, 'DD-MM-YYYY') = '01-01-2022'
What is the equivalent query in hive for
select to_char(trunc(sysdate,'iw')-1)
You could go about this at least two ways:
Either implement your own function in hive using the UDF capability
OR
Use a manipulation of a case statement and modulus of the date:
In rough code this would be something like:
select pmod(datediff(date_column,'2012-01-02'),7)+1 as day_of_week, case when day_of_week = 1 then date_column else when day_of_week = 2 then date_add(date_column,-1)
etc.
first... sorry for my english.
I have a query like this:
Select *
From tableA
Where (
TO_NUMBER(TO_CHAR(dateA(+),'SYYYY')) = 2013
AND TO_NUMBER(TO_CHAR(dateA(+),'MM')) = 02
AND to_number(to_char(dateA(+),'dd')) <= 25
)
and retrieve me the data from each date until last number that I give as parameter, in this case the day 25. This working but delay very much because the form of "Where" statement... anybody know another way that retrieve the data so fast and with the same functionality?
It sounds like you want
SELECT *
FROM tableA
WHERE dateA BETWEEN trunc( date '2013-02-26', 'MM' ) AND date '2013-02-26'
This will return all the rows where dateA is between the first of the month and the specified date. If there is an index on dateA, Oracle would be able to use it for this sort of query (though whether it actually would is a separate issue).