I have an issue importing date from a Tririga database into a SQL database. Mainly I cant convert the date properly and it looks like is not the commont format I have seen around.
Eg date value incomming 775724400000
Running something like select to_date('765187200000', 'DDMMYYYYHH24MISS') my_date FROM dual;
give me an error
ORA-01847: day of month must be between 1 and last day of month
01847. 00000 - "day of month must be between 1 and last day of month"
I found the following info from this link seems to be also from tririga
link_help
And the size of the number are about 10 digits meanwhile this one is 12 and I know for fact this dates should be from the past 10 years (most of them)
I can't seem to find anything that gives me an answer how to convert this into proper dates.
Any suggestions?
The input number appears to be "epoch", a count of milliseconds elapsed since 1 January 1970 GMT (UTC).
To convert to date is not difficult:
select date '1970-01-01' + (775724400000 / 86400000) as dt from dual;
DT
--------------------
1994-Aug-01 07:00:00
Note the hard-coded literals: date '1970-01-01' (epoch is by definition measured from midnight on this date) and 86400000. By one of the definitions (in the previous version of the International System of Units and Weights), a second is 1/86400 of a median day. In Oracle, date arithmetic is based on the number 1 representing one day, so to convert your milliseconds to days you must divide your input by 86400 * 1000.
The most delicate question has to do with time zones (and possibly daylight saving time, also related to time zone). In most cases, epoch is measured from midnight on 1 January 1970 GMT, not from midnight on 1 January 1970 in local time. Do you need to adjust for that? Only you and your business users can answer that question.
(As an aside, the number you provided does NOT represent a date in the past 10 years - not even close.)
I have a query like this:
SELECT
to_date(to_char((from_tz(to_timestamp(to_char(my_column, 'YYYY-MM-DD HH:MI:SS PM'), 'YYYY-MM-DD HH:MI:SS PM') ,'America/Edmonton')at time zone 'America/Vancouver'),'YYYY-MM-DD'),'YYYY-MM-DD') as Date_Column
FROM
my_table
It converts the time between MST to PST. This works most of the time except when daylight savings time occurs. Then a situation happens when in the clock moves forward an hour, and then since an hour is lost (for example 1:59 to the 3:00 am) then we are trying to convert a fictional time.
So I am aware of the issue, and I know Oracle will throw this error for that reason:
ORA-01878: specified field not found in datetime or interval
I've looked all over the Internet for a possible work around but can't really find anything that works. I just want to modify my query so that it can tell what time of year it is and either do the conversation when the time exists, or handle the time somehow when it does not exists. Perhaps add the extra hour when it's missing, and remove it when it's the other time of year.
Does anyone have a solution for this? Is this an impossible problem?
Instead of all the conversions you are using, just use AT TIME ZONE. You may need to take extra care depending on the data type of your column. The AT TIME ZONE functionality will always take into account daylights savings time.
If your column is a TIMESTAMP WITH TIME ZONE, that would be ideal
because you could just say AT TIME ZONE 'PST' and you would get the
time in PST.
If the column is the datatype TIMESTAMP, when you use AT TIME ZONE it
will convert from the time zone of the current session to whatever
you specify.
If your column is a DATE, you will need to convert it to a TIMESTAMP first, then you can use the AT TIME ZONE function. Converting the DATE to TIMESTAMP will make the time zone 0:00 offset so be sure to account for that.
Another handy trick is to use AT LOCAL. This will convert the timestamp to the time zone of the current session. This may be handy if you have users of your application that are in multiple time zones.
Example
I am in the time zone -04:00 (East coast US) which is why that appears as my time zone in the examples below.
WITH
my_table
AS
(SELECT TIMESTAMP '2021-01-01 8:00:00 -7:00' AS ts_with_tz,
TIMESTAMP '2021-01-01 8:00:00' AS ts,
DATE '2021-01-01' + (8 / 24) AS dt
FROM DUAL)
SELECT 1 AS example_num,
t.ts_with_tz,
t.ts,
t.dt
FROM my_table t
UNION ALL
SELECT 2 AS example_num,
t.ts_with_tz AT TIME ZONE 'PST',
t.ts AT TIME ZONE 'PST',
TO_TIMESTAMP (t.dt) AT TIME ZONE 'PST'
FROM my_table t
ORDER BY 1;
RESULT
EXAMPLE_NUM TS_WITH_TZ TS DT
______________ ______________________________________________________ ______________________________________________________ ______________________________________________________
1 01-JAN-21 08.00.00.000000000 AM -07:00 01-JAN-21 08.00.00.000000000 AM AMERICA/NEW_YORK 01-JAN-21 08.00.00.000000000 AM AMERICA/NEW_YORK
2 01-JAN-21 07.00.00.000000000 AM AMERICA/LOS_ANGELES 01-JAN-21 05.00.00.000000000 AM AMERICA/LOS_ANGELES 31-DEC-20 09.00.00.000000000 PM AMERICA/LOS_ANGELES
We've been debugging an issue with a SQL query executed from an app server running Java via Hibernate. The error:
[3/10/14 10:52:07:143 EDT] 0000a984 JDBCException W org.hibernate.util.JDBCExceptionReporter logExceptions SQL Error: 1878, SQLState: 22008
[3/10/14 10:52:07:144 EDT] 0000a984 JDBCException E org.hibernate.util.JDBCExceptionReporter logExceptions ORA-01878: specified field not found in datetime or interval
We've been able to narrow this down to the simple SQL below.
select *
from MY_TABLE T
where T.MY_TIMESTAMP >= (CURRENT_TIMESTAMP - interval '1' hour );
When we run this in the same database, we get the error:
ORA-01878: specified field not found in datetime or interval
01878. 00000 - "specified field not found in datetime or interval"
*Cause: The specified field was not found in the datetime or interval.
*Action: Make sure that the specified field is in the datetime or interval.
The MY_TIMESTAMP column is defined as TIMESTAMP(6).
FWIW, if we change the comparison in the SQL above from >= to <=, the query works.
We assume this has something to do with the time change (we're in America/New_York) but we're having problems trying to figure out where to go from here with our debugging.
Also, we've seen this problem with a similar query that's running through MyBatis and the error looks like:
### Error querying database. Cause: java.sql.SQLException: ORA-01878: specified field not found in datetime or interval
### The error may involve defaultParameterMap
### The error occurred while setting parameters
### Cause: java.sql.SQLException: ORA-01878: specified field not found in datetime or interval
UPDATE: A teammate on Windows changed her Windows Date and Time settings by un-checking "Automatically adjust clock for Daylight Saving Time" and then opened a new SQLDeveloper instance. The second instance is able to run the query without any issue but the first (with the old DST setting) still fails.
To avoid this error, consider using an explicit cast of the expression in the where clause to a timestamp type (timestamp without timezone), in this way:
select *
from MY_TABLE T
where T.MY_TIMESTAMP >= cast(CURRENT_TIMESTAMP - interval '1' hour As timestamp );
Alternatively you can explicitely set the session time zone to, for example '-05:00' - for New York standard (winter) time, using ALTER SESSION time_zone = '-05:00', or by setting ORA_SDTZ environment variable in all client's environments, see this link for details: http://docs.oracle.com/cd/E11882_01/server.112/e10729/ch4datetime.htm#NLSPG263
But it also depends on what really is stored in the timestamp column in the table, for example what a timestamp 2014-07-01 15:00:00 represents in fact, is it a "winter time" or a "summer time" ?
CURRENT_TIMESTAMP function returns a value of datatype TIMESTAMP WITH TIME ZONE
see this link: http://docs.oracle.com/cd/B19306_01/server.102/b14200/functions037.htm
While comparing timestamps and dates, Oracle implicitely converts the data to the more precise data type using the session time zone !
See this link --> http://docs.oracle.com/cd/E11882_01/server.112/e10729/ch4datetime.htm#NLSPG251
In our particular case, Oracle casts timestamp column to the timestamp with time zone type.
Oracle determines a session timezone from the client environment.
You can determine current session timezone using this query:
select sessiontimezone from dual;
For example on my PC (Win 7), when the option ""Automatically adjust clock for Daylight Saving Time" is checked, this query returns (under SQLDeveloper):
SESSIONTIMEZONE
---------------
Europe/Belgrade
When i uncheck this option in Windows and then restart SQLDeveloper, it gives:
SESSIONTIMEZONE
---------------
+01:00
The former session timezone is a timezone with a region name, for which Oracle uses the Daylight Saving Time rules for this region in date calculations:
alter session set time_zone = 'Europe/Belgrade';
select cast( timestamp '2014-01-29 01:30:00' as timestamp with time zone ) As x,
cast( timestamp '2014-05-29 01:30:00' as timestamp with time zone ) As y
from dual;
session SET altered.
X Y
---------------------------- ----------------------------
2014-01-29 01:30:00 EUROPE/B 2014-05-29 01:30:00 EUROPE/B
ELGRADE ELGRADE
The latter timezone uses a fixed offset "+01:00" (always the "Winter time"), and Oracle does not apply any DST rules for it, it simply adds the fixed offset.
alter session set time_zone = '+01:00';
select cast( timestamp '2014-01-29 01:30:00' as timestamp with time zone ) As x,
cast( timestamp '2014-05-29 01:30:00' as timestamp with time zone ) As y
from dual;
session SET altered.
X Y
---------------------------- ----------------------------
2014-01-29 01:30:00 +01:00 2014-05-29 01:30:00 +01:00
Please note, for curiosity's sake, that Y results in the above represent two different times !!!
014-05-29 01:30:00 EUROPE/BELGRADE is not the same as: 2014-05-29 01:30:00 +01:00
but actually this:
014-05-29 01:30:00 EUROPE/BELGRADE is equal to: 2014-05-29 01:30:00 +02:00
The above is only to make you aware of how simple "box un-checking" could affect your queries, and where to dig for a reason when users complain "this query worked fine in January, but gave wrong results in July".
And still on the topic of ORA-01878 - let say my session is EUROPE/Warsaw and my table containts this timestamp (without time zone)
'TIMESTAMP'2014-03-30 2:30:00'
Note that in my region the DST change, in 2014 year, occurs on 30 of march at 2:00 a.m.
It simply means that on march 30, at 2:00 at night, I must wake up and shift my watch forward from 2:00 to 3:00 ;)
alter session set time_zone = 'Europe/Warsaw';
select cast( TIMESTAMP'2014-03-30 2:30:00' as timestamp with time zone ) As x
from dual;
SQL Error: ORA-01878: podane pole nie zostało znalezione w dacie-godzinie ani w interwale
01878. 00000 - "specified field not found in datetime or interval"
*Cause: The specified field was not found in the datetime or interval.
*Action: Make sure that the specified field is in the datetime or interval.
Oracle knows, that this timestamp is not valid in my region according to DST rules, because there is no time 2:30 on 30 of march - at 2:00 the clock is moved to 3:00, and there is no time 2:30. Therefore Oracle throws the error ORA-01878.
However this query works perfectly fine:
alter session set time_zone = '+01:00';
select cast( TIMESTAMP'2014-03-30 2:30:00' as timestamp with time zone ) As x
from dual;
session SET altered.
X
----------------------------
2014-03-30 02:30:00 +01:00
And this is a reason of this error - your table contains timestamps like 2014-03-09 2:30 or so (for New York, where DST shifts occur on 9 of March and 2 of November), and Oracle doesn't know how to convert them from timestamp (without TZ) to timestamp with TZ.
The last question - why the query with >= doesn't work, but the query with <= works fine ?
They work/don'n work, because SQLDeveloper returns only first 50 rows (maybe 100 ? It depends on settings). The query doesn't read the whole table, it stops when first 50(100) rows are fetched.
Change the "working" query to, for example:
select sum( EXTRACT(HOUR from MY_TIMESTAMP) ) from MY_TABLE
where MY_TIMESTAMP <= (CURRENT_TIMESTAMP - interval '1' hour );
This force the query to read all rows in the table, and the error will appear, I'am 100% sure.
Thanks to kordirko for the extremely detailed write up. I think in the future, we will be looking at different ways to compare dates that aren't as prone to error. In the meantime, we were able to figure out the problem and both a temporary and long-term solution.
First, more details on the issue. It turns out that the values being stored in the TIMESTAMP field in the database were incorrect. We saw this by using the dump function and examining the bytes. If you look at the 5th byte in the output below, you'll see the hour (this is actually the hour + 1 so 5 is actually 4AM). For the values between 3AM and 4AM, you'll notice that the 5th byte is 3 which represents 2AM. 2 AM March 9, 2014 in EST doesn't exist - this is an incorrect time according to DST rules and Oracle's rules.
09-MAR-14 03.06.21.522000000 AM Typ=180 Len=11: 120,114,3,9,3,7,22,31,29,22,128
09-MAR-14 03.32.37.869000000 AM Typ=180 Len=11: 120,114,3,9,3,33,38,51,203,227,64
09-MAR-14 03.36.49.804000000 AM Typ=180 Len=11: 120,114,3,9,3,37,50,47,236,17,0
09-MAR-14 03.43.47.328000000 AM Typ=180 Len=11: 120,114,3,9,3,44,48,19,140,226,0
09-MAR-14 03.47.55.255000000 AM Typ=180 Len=11: 120,114,3,9,3,48,56,15,50,253,192
09-MAR-14 03.55.45.129000000 AM Typ=180 Len=11: 120,114,3,9,3,56,46,7,176,98,64
09-MAR-14 04.05.03.325000000 AM Typ=180 Len=11: 120,114,3,9,5,6,4,19,95,27,64
09-MAR-14 04.28.41.267000000 AM Typ=180 Len=11: 120,114,3,9,5,29,42,15,234,24,192
09-MAR-14 04.35.16.072000000 AM Typ=180 Len=11: 120,114,3,9,5,36,17,4,74,162,0
09-MAR-14 04.41.07.260000000 AM Typ=180 Len=11: 120,114,3,9,5,42,8,15,127,73,0
09-MAR-14 04.46.31.047000000 AM Typ=180 Len=11: 120,114,3,9,5,47,32,2,205,41,192
09-MAR-14 04.53.33.471000000 AM Typ=180 Len=11: 120,114,3,9,5,54,34,28,18,227,192
After much research and discussion, we zeroed in on the fact that our version of the Oracle JDBC driver (11.2.0.2) might've been inserting the bad values. Oracle's information page on 11.2.0.3 references a bug that looks like it's our issue: "9785135 DST conversion not correct using jdbc 11g timestamtz". We wrote a quick test class that inserts values from March 9, 2014 1:50 AM to 4:00 AM using both the 11.2.0.2 and 11.2.0.3 driver. Here's a snippet of what was inserted into the db:
DRIVER_V JAVA_DATE_AS_STRING ORACLE_TIMESTAMP DUMP(ORACLE_TIMESTAMP)
11.2.0.2.0/11/2 Sun Mar 09 01:50:00 EST 2014 09-MAR-14 01.50.00.000000000 AM Typ=180 Len=7: 120,114,3,9,2,51,1
11.2.0.2.0/11/2 Sun Mar 09 03:00:00 EDT 2014 09-MAR-14 03.00.00.000000000 AM Typ=180 Len=7: 120,114,3,9,3,1,1 --Invalid timestamp
11.2.0.3.0/11/2 Sun Mar 09 01:50:00 EST 2014 09-MAR-14 01.50.00.000000000 AM Typ=180 Len=7: 120,114,3,9,2,51,1
11.2.0.3.0/11/2 Sun Mar 09 03:00:00 EDT 2014 09-MAR-14 03.00.00.000000000 AM Typ=180 Len=7: 120,114,3,9,4,1,1 --Correct timestamp
You'll notice that the 5th byte of the timestamp on the second row for 3:00 AM is incorrect (3). This was inserted using the 11.2.0.2 version. The same value inserted with the 11.2.0.3 version can be found on the fourth line with the correct 5th byte (4).
The long term fix here is to update our JDBC driver. The short term fix here was to find the rows that have the bad values and run an update statement on them from SQL Plus to set the time again (using the same value but SQL Plus will convert them correctly).
This recent post got me busy investigating Julian date conversions in Oracle, and I've come across what I believe is a bug in Oracle 11.1. Test cases are:
Case 1.
SELECT TO_CHAR(TO_TIMESTAMP('0', 'J'), 'DD MON SYYYY') FROM DUAL
This should return "01 JAN -4713" as defined here, but instead raises the error
ORA-01854: julian date must be between 1 and 5373484
Case 2.
SELECT TO_CHAR(TO_TIMESTAMP('1', 'J'), 'DD MON SYYYY') FROM DUAL
This should return "02 JAN -4713" as an extension of the above (one day later than the Julian zero date), but instead returns "01 JAN -4712" (off by a day less than a year).
Case 3.
SELECT TO_CHAR(TO_TIMESTAMP('1721424', 'J'), 'DD MON SYYYY') FROM DUAL
returns "01 JAN 0001". That's fine (as far as it goes). If we then subtract 1 from the date value above we would expect it to return the previous day, i.e. 31 DEC -0001 (year zero does not exist); however, when we execute the following
SELECT TO_CHAR(TO_TIMESTAMP('1721423', 'J'), 'DD MON SYYYY') FROM DUAL
the following error is thrown:
ORA-01841: (full) year must be between -4713 and +9999, and not be 0
indicating that Oracle has attempted to generate a year of zero.
(Note that although TO_TIMESTAMP is used in the test cases above, the exact same problems occur when TO_DATE is used).
Does anyone know if
These problems have been documented by Oracle?
These problems still exist in 11.2?
Share and enjoy.
Per Phil's answer below, these issues still occur in 11.2.
Cthulhu fhtagn.
Same bugs in 10.2.0.4
When trying to see what Oracle is supposed to do, look at Oracle's documentation, "A Julian day number is the number of days since January 1, 4712 BC."
That wording does seam to imply that Julian 1 would be one days since January 1, 4712 BC, in other words January 2. However the current implementation of Julian date calculation has been in place for a long time, with existing code depending on the behavior. (I know we'd be screwed if the definition of Julian implemented in Oracle were to change.) At this point it would be at most a documentation bug to be days since December 31, 4713 BC.
EDIT Found a reference for Julian 1 being January 1, in Call Interface Programmer's Guide. Not someplace normal database programmers would ever look at.
The following explains the year difference between wikipedia and Oracle:
Oracle Database uses the astronomical system of calculating Julian
days, in which the year 4713 BC is specified as -4712. The historical
system of calculating Julian days, in contrast, specifies 4713 BC as
-4713. If you are comparing Oracle Julian days with values calculated using the historical system, then take care to allow for the 365-day
difference in BC dates. For more information, see
http://www.usno.navy.mil/USNO/astronomical-applications/astronomical-information-center/millennium.
Case 3 is news to me. Thank you for bringing it up. I do not know of any reference covering that behavior. Related:
SQL> select to_date('0001-01-01', 'YYYY-MM-DD')
- to_date ('-0001-12-31', 'SYYYY-MM-DD') from dual;
TO_DATE('0001-01-01','YYYY-MM-DD')-TO_DATE('-0001-12-31','SYYYY-MM-DD')
-----------------------------------------------------------------------
367
and
SQL> select months_between(to_date('0001-01-01', 'YYYY-MM-DD')
2 , to_date ('-0001-12-31', 'SYYYY-MM-DD')) from dual;
MONTHS_BETWEEN(TO_DATE('0001-01-01','YYYY-MM-DD'),TO_DATE('-0001-12-31','SYYYY-MM-DD'))
---------------------------------------------------------------------------------------
12.0322581
Apparently the non existent year 0 is a leap year.
Here's the same queries performed on 11.2.0.1.0:
PHIL#PHILL11G2 > select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for Linux: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production
Elapsed: 00:00:00.04
PHIL#PHILL11G2 > SELECT TO_CHAR(TO_TIMESTAMP('0', 'J'), 'DD MON SYYYY') FROM DUAL;
SELECT TO_CHAR(TO_TIMESTAMP('0', 'J'), 'DD MON SYYYY') FROM DUAL
*
ERROR at line 1:
ORA-01854: julian date must be between 1 and 5373484
Elapsed: 00:00:00.00
PHIL#PHILL11G2 > SELECT TO_CHAR(TO_TIMESTAMP('1', 'J'), 'DD MON SYYYY') FROM DUAL;
TO_CHAR(TO_T
------------
01 JAN -4712
Elapsed: 00:00:00.00
PHIL#PHILL11G2 > SELECT TO_CHAR(TO_TIMESTAMP('1721424', 'J'), 'DD MON SYYYY') FROM DUAL;
TO_CHAR(TO_T
------------
01 JAN 0001
Elapsed: 00:00:00.00
PHIL#PHILL11G2 > SELECT TO_CHAR(TO_TIMESTAMP('1721423', 'J'), 'DD MON SYYYY') FROM DUAL;
SELECT TO_CHAR(TO_TIMESTAMP('1721423', 'J'), 'DD MON SYYYY') FROM DUAL
*
ERROR at line 1:
ORA-01841: (full) year must be between -4713 and +9999, and not be 0
Elapsed: 00:00:00.04
PHIL#PHILL11G2 >
see U.S. NAVAL OBSERVATORY MODIFIED JULIAN DATE