I have a query like this:
SELECT
to_date(to_char((from_tz(to_timestamp(to_char(my_column, 'YYYY-MM-DD HH:MI:SS PM'), 'YYYY-MM-DD HH:MI:SS PM') ,'America/Edmonton')at time zone 'America/Vancouver'),'YYYY-MM-DD'),'YYYY-MM-DD') as Date_Column
FROM
my_table
It converts the time between MST to PST. This works most of the time except when daylight savings time occurs. Then a situation happens when in the clock moves forward an hour, and then since an hour is lost (for example 1:59 to the 3:00 am) then we are trying to convert a fictional time.
So I am aware of the issue, and I know Oracle will throw this error for that reason:
ORA-01878: specified field not found in datetime or interval
I've looked all over the Internet for a possible work around but can't really find anything that works. I just want to modify my query so that it can tell what time of year it is and either do the conversation when the time exists, or handle the time somehow when it does not exists. Perhaps add the extra hour when it's missing, and remove it when it's the other time of year.
Does anyone have a solution for this? Is this an impossible problem?
Instead of all the conversions you are using, just use AT TIME ZONE. You may need to take extra care depending on the data type of your column. The AT TIME ZONE functionality will always take into account daylights savings time.
If your column is a TIMESTAMP WITH TIME ZONE, that would be ideal
because you could just say AT TIME ZONE 'PST' and you would get the
time in PST.
If the column is the datatype TIMESTAMP, when you use AT TIME ZONE it
will convert from the time zone of the current session to whatever
you specify.
If your column is a DATE, you will need to convert it to a TIMESTAMP first, then you can use the AT TIME ZONE function. Converting the DATE to TIMESTAMP will make the time zone 0:00 offset so be sure to account for that.
Another handy trick is to use AT LOCAL. This will convert the timestamp to the time zone of the current session. This may be handy if you have users of your application that are in multiple time zones.
Example
I am in the time zone -04:00 (East coast US) which is why that appears as my time zone in the examples below.
WITH
my_table
AS
(SELECT TIMESTAMP '2021-01-01 8:00:00 -7:00' AS ts_with_tz,
TIMESTAMP '2021-01-01 8:00:00' AS ts,
DATE '2021-01-01' + (8 / 24) AS dt
FROM DUAL)
SELECT 1 AS example_num,
t.ts_with_tz,
t.ts,
t.dt
FROM my_table t
UNION ALL
SELECT 2 AS example_num,
t.ts_with_tz AT TIME ZONE 'PST',
t.ts AT TIME ZONE 'PST',
TO_TIMESTAMP (t.dt) AT TIME ZONE 'PST'
FROM my_table t
ORDER BY 1;
RESULT
EXAMPLE_NUM TS_WITH_TZ TS DT
______________ ______________________________________________________ ______________________________________________________ ______________________________________________________
1 01-JAN-21 08.00.00.000000000 AM -07:00 01-JAN-21 08.00.00.000000000 AM AMERICA/NEW_YORK 01-JAN-21 08.00.00.000000000 AM AMERICA/NEW_YORK
2 01-JAN-21 07.00.00.000000000 AM AMERICA/LOS_ANGELES 01-JAN-21 05.00.00.000000000 AM AMERICA/LOS_ANGELES 31-DEC-20 09.00.00.000000000 PM AMERICA/LOS_ANGELES
My company recently upgraded to Oracle 11g. I have a field - Timestamp with Time Zone that has data inserted in as "UTC". We query the data, make some changes and run a generic update statement that updates all the fields in the table. Since the 11g upgrade, the update statement converts the date/time assuming it's US/Central (my time zome). Example:
7/4/2015 7:40:04.000000 AM +00:00 turns into 7/4/2015 12:40:04.000000 AM +00:00 with an update statement like this:
Update Tablename
set FIELD_TIMESTAMP = to_timestamp('07/04/2015 7:40:04 AM', 'mm/dd/yyyy hh:mi:ss AM') at time zone 'UTC'
where ID = 201924;
I have tried just inserting a basic date, I have tried using different variations of the time zone. Nothing works, it always converts the timestamp.
How can I indicate to Oracle that the timestamp I'm updating is UTC and not US/Central time zone so it doesn't convert it?
Try using to_timestamp_tz. For example:
SQL> create table tz_test
(
id number,
col1 timestamp with time zone
)
Table created.
SQL> insert into tz_test values (1, systimestamp)
1 row created.
SQL> insert into tz_test values (2, to_timestamp('20150716 7:40:04', 'YYYYMMDD HH24:MI:SS'))
1 row created.
SQL> insert into tz_test values (3, to_timestamp_tz('20150716 7:40:04 -0:00', 'YYYYMMDD HH24:MI:SS TZH:TZM'))
1 row created.
SQL> commit
Commit complete.
SQL> select * from tz_test
ID COL1
---------- -----------------------------------
1 7/16/2015 1:29:47.031650 PM -04:00
2 7/16/2015 7:40:04.000000 AM -04:00
3 7/16/2015 7:40:04.000000 AM +00:00
3 rows selected.
My timezone is eastern (-4 from UTC), but the 3rd insert I specify the timezone to be used.
Reason behind this issue is that the TZ variable is not set properly in your.bash_profile.
As of now I am almost certain you database will be returning the systimestamp in central
SELECT SYSTIMESTAMP FROM DUAL;
In order to permanently fix (like you had prior to upgrade to Oracle 11g) you need to add or edit TZ variable to UTC. For instance if I want to change my timezone to Asia/Calcutta then I have to add/edit the TZ variable like mentioned below
TZ='Asia/Calcutta'; EXPORT TZ
Once you edit the .bash_profile you need to restart the listener service and database.
Or as an alternative you use can to_timestamp() and to_timestamp_tz() function as suggested by tbone to resolve this .
Regards
Andy
I have created a table in Oracle in which I have KPI_START_DATE column which is a Date datatype, and KPI_START_TIME which is a TIMESTAMP datatype.
Now I want to modify this date dataype for
KPI_START_DATE to dd/mm/yyyy
and
KPI_START_TIME to HH:MI:SS.
So that user should always enter the date and time in this column in this proper format.
I tried below query but its was giving error:
Alter table KPI_DEFINITION MODIFY(to_char(KPI_START_DATE,'dd/mm/yyyy') )
DATE and TIMESTAMP columns do not have any inherent readable format. The values are stored in Oracle's own internal representation, which has no resemblance to a human-readable date or time. At the point to retrieve or display a value you can convert it to whatever format you want, with to_char().
Both DATE and TIMESTAMP have date and time components (to second precision with DATE, and with fractional seconds with TIMESTAMP; plus time zone information with the extended data types), and you should not try to store them separately as two columns. Have a single column and extract the information you need at any time; to get the information out of a single column but split into two fields you could do:
select to_char(KPI_START, 'dd/mm/yyyy') as KPI_START_DATE,
to_char(KPI_START, 'hh24:mi:ss') as KPI_START_TIME
but you'd generally want both together anyway:
select to_char(KPI_START, 'dd/mm/yyyy hh24:mi:ss')
Also notice the 'hh24' format model to get the 24-hour clock time; otherwise you wouldn't see any difference between 3 a.m. and 3 p.m.
You can store a value in either type of column with the time set to midnight, but it does still have a time component - it is just midnight. You can't store a value in either type of column with just a time component - it has to have a date too. You could make that a nominal date and just ignore it, but I've never seen a valid reason to do that - you're wasting storage in two columns, and making searching for and comparing values much harder. Oracle even provides a default date if you don't specify one (first day of current month). But the value always has both a date and a time part:
create table KPI_DEFINITION (KPI_START date);
insert into KPI_DEFINITION (KPI_START)
values (to_date('27/01/2015', 'DD/MM/YYYY'));
insert into KPI_DEFINITION (KPI_START)
values (to_date('12:41:57', 'HH24:MI:SS'));
select to_char(KPI_START, 'YYYY-MM-DD HH24:MI:SS') from KPI_DEFINITION;
TO_CHAR(KPI_START,'YYYY-MM-DDHH24:MI:SS')
-----------------------------------------
2015-01-27 00:00:00
2015-01-01 12:41:57
Your users should be inserting a single value with both date and time as one:
insert into KPI_DEFINITION (KPI_START)
values (to_date('27/01/2015 12:41:57', 'DD/MM/YYYY HH24:MI:SS'));
select to_char(KPI_START, 'YYYY-MM-DD HH24:MI:SS') from KPI_DEFINITION;
TO_CHAR(KPI_START,'YYYY-MM-DDHH24:MI:SS')
-----------------------------------------
2015-01-27 12:41:57
You can also use date or timestamp literals, and if using to_date() you should always specify the full format - don't rely on NLS settings as they may be different for other users.
You should understand difference between datatype and format. DATE is a datatype. TIMESTAMP is a datatype. None of them have formats, they're just numbers.
When converting character datatype to or from date datatype, format should be applied. It's an attribute of an actual conversion, nothing else.
Look at this:
SQL> create table tmp$date(d date);
Table created
SQL> insert into tmp$date values (DATE '2010-11-01');
1 row inserted
SQL> insert into tmp$date values (DATE '2014-12-28');
1 row inserted
SQL> select d, dump(d) from tmp$date;
D DUMP(D)
----------- ---------------------------------
01.11.2010 Typ=12 Len=7: 120,110,11,1,1,1,1
28.12.2014 Typ=12 Len=7: 120,114,12,28,1,1,1
There is no any 'format' here.
DISPLAYING and STORING are NOT the same when it comes to DATE.
When people say Oracle isn’t storing the date in the format they wanted, what is really happening is Oracle is not presenting the date in the character string format they expected or wanted.
When a data element of type DATE is selected, it must be converted from its internal, binary format, to a string of characters for human consumption. The conversion of data from one type to another is known as known a “conversion”, “type casting” or “coercion”. In Oracle the conversion between dates and character strings is controlled by the NLS_DATE_FORMAT model. The NLS_DATE_FORMAT can be set in any of several different locations, each with its own scope of influence.
I could go on with my leacture over DATE data type, but I am glad that someone has already got a good writeup over this. Please read this https://edstevensdba.wordpress.com/2011/04/07/nls_date_format/
We've been debugging an issue with a SQL query executed from an app server running Java via Hibernate. The error:
[3/10/14 10:52:07:143 EDT] 0000a984 JDBCException W org.hibernate.util.JDBCExceptionReporter logExceptions SQL Error: 1878, SQLState: 22008
[3/10/14 10:52:07:144 EDT] 0000a984 JDBCException E org.hibernate.util.JDBCExceptionReporter logExceptions ORA-01878: specified field not found in datetime or interval
We've been able to narrow this down to the simple SQL below.
select *
from MY_TABLE T
where T.MY_TIMESTAMP >= (CURRENT_TIMESTAMP - interval '1' hour );
When we run this in the same database, we get the error:
ORA-01878: specified field not found in datetime or interval
01878. 00000 - "specified field not found in datetime or interval"
*Cause: The specified field was not found in the datetime or interval.
*Action: Make sure that the specified field is in the datetime or interval.
The MY_TIMESTAMP column is defined as TIMESTAMP(6).
FWIW, if we change the comparison in the SQL above from >= to <=, the query works.
We assume this has something to do with the time change (we're in America/New_York) but we're having problems trying to figure out where to go from here with our debugging.
Also, we've seen this problem with a similar query that's running through MyBatis and the error looks like:
### Error querying database. Cause: java.sql.SQLException: ORA-01878: specified field not found in datetime or interval
### The error may involve defaultParameterMap
### The error occurred while setting parameters
### Cause: java.sql.SQLException: ORA-01878: specified field not found in datetime or interval
UPDATE: A teammate on Windows changed her Windows Date and Time settings by un-checking "Automatically adjust clock for Daylight Saving Time" and then opened a new SQLDeveloper instance. The second instance is able to run the query without any issue but the first (with the old DST setting) still fails.
To avoid this error, consider using an explicit cast of the expression in the where clause to a timestamp type (timestamp without timezone), in this way:
select *
from MY_TABLE T
where T.MY_TIMESTAMP >= cast(CURRENT_TIMESTAMP - interval '1' hour As timestamp );
Alternatively you can explicitely set the session time zone to, for example '-05:00' - for New York standard (winter) time, using ALTER SESSION time_zone = '-05:00', or by setting ORA_SDTZ environment variable in all client's environments, see this link for details: http://docs.oracle.com/cd/E11882_01/server.112/e10729/ch4datetime.htm#NLSPG263
But it also depends on what really is stored in the timestamp column in the table, for example what a timestamp 2014-07-01 15:00:00 represents in fact, is it a "winter time" or a "summer time" ?
CURRENT_TIMESTAMP function returns a value of datatype TIMESTAMP WITH TIME ZONE
see this link: http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions037.htm
While comparing timestamps and dates, Oracle implicitely converts the data to the more precise data type using the session time zone !
See this link --> http://docs.oracle.com/cd/E11882_01/server.112/e10729/ch4datetime.htm#NLSPG251
In our particular case, Oracle casts timestamp column to the timestamp with time zone type.
Oracle determines a session timezone from the client environment.
You can determine current session timezone using this query:
select sessiontimezone from dual;
For example on my PC (Win 7), when the option ""Automatically adjust clock for Daylight Saving Time" is checked, this query returns (under SQLDeveloper):
SESSIONTIMEZONE
---------------
Europe/Belgrade
When i uncheck this option in Windows and then restart SQLDeveloper, it gives:
SESSIONTIMEZONE
---------------
+01:00
The former session timezone is a timezone with a region name, for which Oracle uses the Daylight Saving Time rules for this region in date calculations:
alter session set time_zone = 'Europe/Belgrade';
select cast( timestamp '2014-01-29 01:30:00' as timestamp with time zone ) As x,
cast( timestamp '2014-05-29 01:30:00' as timestamp with time zone ) As y
from dual;
session SET altered.
X Y
---------------------------- ----------------------------
2014-01-29 01:30:00 EUROPE/B 2014-05-29 01:30:00 EUROPE/B
ELGRADE ELGRADE
The latter timezone uses a fixed offset "+01:00" (always the "Winter time"), and Oracle does not apply any DST rules for it, it simply adds the fixed offset.
alter session set time_zone = '+01:00';
select cast( timestamp '2014-01-29 01:30:00' as timestamp with time zone ) As x,
cast( timestamp '2014-05-29 01:30:00' as timestamp with time zone ) As y
from dual;
session SET altered.
X Y
---------------------------- ----------------------------
2014-01-29 01:30:00 +01:00 2014-05-29 01:30:00 +01:00
Please note, for curiosity's sake, that Y results in the above represent two different times !!!
014-05-29 01:30:00 EUROPE/BELGRADE is not the same as: 2014-05-29 01:30:00 +01:00
but actually this:
014-05-29 01:30:00 EUROPE/BELGRADE is equal to: 2014-05-29 01:30:00 +02:00
The above is only to make you aware of how simple "box un-checking" could affect your queries, and where to dig for a reason when users complain "this query worked fine in January, but gave wrong results in July".
And still on the topic of ORA-01878 - let say my session is EUROPE/Warsaw and my table containts this timestamp (without time zone)
'TIMESTAMP'2014-03-30 2:30:00'
Note that in my region the DST change, in 2014 year, occurs on 30 of march at 2:00 a.m.
It simply means that on march 30, at 2:00 at night, I must wake up and shift my watch forward from 2:00 to 3:00 ;)
alter session set time_zone = 'Europe/Warsaw';
select cast( TIMESTAMP'2014-03-30 2:30:00' as timestamp with time zone ) As x
from dual;
SQL Error: ORA-01878: podane pole nie zostało znalezione w dacie-godzinie ani w interwale
01878. 00000 - "specified field not found in datetime or interval"
*Cause: The specified field was not found in the datetime or interval.
*Action: Make sure that the specified field is in the datetime or interval.
Oracle knows, that this timestamp is not valid in my region according to DST rules, because there is no time 2:30 on 30 of march - at 2:00 the clock is moved to 3:00, and there is no time 2:30. Therefore Oracle throws the error ORA-01878.
However this query works perfectly fine:
alter session set time_zone = '+01:00';
select cast( TIMESTAMP'2014-03-30 2:30:00' as timestamp with time zone ) As x
from dual;
session SET altered.
X
----------------------------
2014-03-30 02:30:00 +01:00
And this is a reason of this error - your table contains timestamps like 2014-03-09 2:30 or so (for New York, where DST shifts occur on 9 of March and 2 of November), and Oracle doesn't know how to convert them from timestamp (without TZ) to timestamp with TZ.
The last question - why the query with >= doesn't work, but the query with <= works fine ?
They work/don'n work, because SQLDeveloper returns only first 50 rows (maybe 100 ? It depends on settings). The query doesn't read the whole table, it stops when first 50(100) rows are fetched.
Change the "working" query to, for example:
select sum( EXTRACT(HOUR from MY_TIMESTAMP) ) from MY_TABLE
where MY_TIMESTAMP <= (CURRENT_TIMESTAMP - interval '1' hour );
This force the query to read all rows in the table, and the error will appear, I'am 100% sure.
Thanks to kordirko for the extremely detailed write up. I think in the future, we will be looking at different ways to compare dates that aren't as prone to error. In the meantime, we were able to figure out the problem and both a temporary and long-term solution.
First, more details on the issue. It turns out that the values being stored in the TIMESTAMP field in the database were incorrect. We saw this by using the dump function and examining the bytes. If you look at the 5th byte in the output below, you'll see the hour (this is actually the hour + 1 so 5 is actually 4AM). For the values between 3AM and 4AM, you'll notice that the 5th byte is 3 which represents 2AM. 2 AM March 9, 2014 in EST doesn't exist - this is an incorrect time according to DST rules and Oracle's rules.
09-MAR-14 03.06.21.522000000 AM Typ=180 Len=11: 120,114,3,9,3,7,22,31,29,22,128
09-MAR-14 03.32.37.869000000 AM Typ=180 Len=11: 120,114,3,9,3,33,38,51,203,227,64
09-MAR-14 03.36.49.804000000 AM Typ=180 Len=11: 120,114,3,9,3,37,50,47,236,17,0
09-MAR-14 03.43.47.328000000 AM Typ=180 Len=11: 120,114,3,9,3,44,48,19,140,226,0
09-MAR-14 03.47.55.255000000 AM Typ=180 Len=11: 120,114,3,9,3,48,56,15,50,253,192
09-MAR-14 03.55.45.129000000 AM Typ=180 Len=11: 120,114,3,9,3,56,46,7,176,98,64
09-MAR-14 04.05.03.325000000 AM Typ=180 Len=11: 120,114,3,9,5,6,4,19,95,27,64
09-MAR-14 04.28.41.267000000 AM Typ=180 Len=11: 120,114,3,9,5,29,42,15,234,24,192
09-MAR-14 04.35.16.072000000 AM Typ=180 Len=11: 120,114,3,9,5,36,17,4,74,162,0
09-MAR-14 04.41.07.260000000 AM Typ=180 Len=11: 120,114,3,9,5,42,8,15,127,73,0
09-MAR-14 04.46.31.047000000 AM Typ=180 Len=11: 120,114,3,9,5,47,32,2,205,41,192
09-MAR-14 04.53.33.471000000 AM Typ=180 Len=11: 120,114,3,9,5,54,34,28,18,227,192
After much research and discussion, we zeroed in on the fact that our version of the Oracle JDBC driver (11.2.0.2) might've been inserting the bad values. Oracle's information page on 11.2.0.3 references a bug that looks like it's our issue: "9785135 DST conversion not correct using jdbc 11g timestamtz". We wrote a quick test class that inserts values from March 9, 2014 1:50 AM to 4:00 AM using both the 11.2.0.2 and 11.2.0.3 driver. Here's a snippet of what was inserted into the db:
DRIVER_V JAVA_DATE_AS_STRING ORACLE_TIMESTAMP DUMP(ORACLE_TIMESTAMP)
11.2.0.2.0/11/2 Sun Mar 09 01:50:00 EST 2014 09-MAR-14 01.50.00.000000000 AM Typ=180 Len=7: 120,114,3,9,2,51,1
11.2.0.2.0/11/2 Sun Mar 09 03:00:00 EDT 2014 09-MAR-14 03.00.00.000000000 AM Typ=180 Len=7: 120,114,3,9,3,1,1 --Invalid timestamp
11.2.0.3.0/11/2 Sun Mar 09 01:50:00 EST 2014 09-MAR-14 01.50.00.000000000 AM Typ=180 Len=7: 120,114,3,9,2,51,1
11.2.0.3.0/11/2 Sun Mar 09 03:00:00 EDT 2014 09-MAR-14 03.00.00.000000000 AM Typ=180 Len=7: 120,114,3,9,4,1,1 --Correct timestamp
You'll notice that the 5th byte of the timestamp on the second row for 3:00 AM is incorrect (3). This was inserted using the 11.2.0.2 version. The same value inserted with the 11.2.0.3 version can be found on the fourth line with the correct 5th byte (4).
The long term fix here is to update our JDBC driver. The short term fix here was to find the rows that have the bad values and run an update statement on them from SQL Plus to set the time again (using the same value but SQL Plus will convert them correctly).