I'm using Pentaho to datamask some of the information on the oracle DB
I have several transformations of the form:
SELECT -> data mask -> UPDATE rows based on primary key
I have tables where a timestamp is part of the primary key in the update step. Even though I am not masking or updating this field in any way, I get the error ORA-01843: not a valid month when performing the update.
I believe this is because when Pentaho takes in the timestamp from Step 1 it doesn’t actually keep it as a timestamp until I try the update and hence the primary key check. Outputting to excel, I see pentaho giving timestamps in the format
2014-07-30 15:44:31.869033 Europe/London (Pentaho)
But in DB the format is
30-JAN-15 09.21.38.109145000 AM (Oracle - TIMESTAMP(6) WITH LOCAL TIME ZONE)
I have tried to convert the pentaho field to a Timestamp (format: yyyy-MM-dd HH:mm:ss.SSSSSS) before the update step but receive errors if I try and use milliseconds.
2017/03/14 13:19:25 - Select values.0 - AUDIT_CREATE_TS Timestamp : couldn't convert string [2015-01-30 09:21:38.109145 Europe/London] to a timestamp, expecting format [yyyy-mm-dd hh:mm:ss.ffffff]
2017/03/14 13:19:25 - Select values.0 - Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]
If I replace my formatting to the one suggested by Pentaho I get "Illegal character 'f'" and then I am stuck in a loop.
Ignoring milliseconds seems to succeed but won’t give me any matches because it isn’t precise enough and returns no results from db..
Any help would be appreciated!
Not sure about Pentaho, but if you're looking for a conversion from this string:
'2015-01-30 09:21:38.109145 Europe/London'
to a timestamp with timezone in Oracle, it would be:
select to_timestamp_tz('2015-01-30 09:21:38.109145 Europe/London', 'YYYY-MM-DD HH24:MI:SS.FF6 TZR') from dual;
See Oracle Datetime Format Models document for more.
Related
I try to insert the created field in my_table. The created field has a datetime type. In my_table the field my_created has a date format. So I try to TRUNC the created field. However I'm getting the error ORA-01830: date format picture ends before converting entire input stringwhile inserting the truncated value. It seems, that the time is still there but is reset to 00:00. how can I get only the date without time? It happens only in perl. I'm getting only date in toad.
Very simplified code looks like:
my $SQL="SELECT
TRUNC(CREATED),
FROM
DBA_OBJECTS";
my $sth = $db->prepare($SQL);
$sth->execute();
my $date = $sth->fetchrow();
$SQL = "INSERT INTO MY_TABLE
(MY_CREATED)
VALUES (?)";
my $stmt = $dbh_master->prepare($SQL);
$stmt->execute($date);
EDIT:
I found an ugly workaround and I'm executing it like this:
$stmt->execute(substr($date, 0, 10));
However maybe someone has a nicer solution.
How can I get only the date without time?
In Oracle, a DATE is a binary data type that is composed of 7 bytes representing: century, year-of-century, month, day, hour, minute and second. It ALWAYS has those binary components so if you want an Oracle DATE data type then you cannot get it without a time.
The Oracle DATE data type was released with Oracle version 2 in 1979 and predates the ANSI/ISO standard (of 1986, where a DATE does not have a time component) and Oracle has maintained backwards compatibility with their previous data types rather than adopting the ANSI standard.
If you use the TRUNC(date_value, format_model) function then it will set the binary components of the DATE, up to the specified format model, to the minimum (0 for hours, minutes and seconds, 1 for days and months) but it will NOT give you a data type that does not have a time component.
It happens only in perl. I'm getting only date in toad.
No, you are getting the entire 7 byte binary value in Toad; however, the user interface is only choosing to show you the date component. There should be a setting in the preferences that can set the date format in Toad which will let you see the entire date-time components.
Oracle SQL/Plus and SQL Developer use the NLS_DATE_FORMAT session parameter and Toad may also be able to use that.
If you want to get the value as a DATE then it will always have a time component (even if you set that time component to zeros using TRUNC).
If you want to get the date so that it is formatted in a way without a time component then you need to convert it to another data type and can use TO_CHAR to format it as a string:
SELECT TO_CHAR(CREATED, 'YYYY-MM-DD')
FROM DBA_OBJECTS
But then you will be returning a (formatted) string and not a DATE data type.
I have a external table in hive, which has report_dt as column and datatype is string and has value like 2018-09-02 2018-09-03
And i want to create managed table from the query written on this external table with report_dt_1 as column with datatype as date
i have gone through some threads and i was able to query like
create table manag_newtable as select to_date(from_unixtime(unix_timestamp(report_dt,'yyyy-MM-dd'),'yyyy-MM-dd')) as report_dt_1 from exter_table;
so the above query creates a new table, but when i desc manag_newtable ,still that report_dt_1 shows as string datatype
Prior to Hive 2.1.0, TO_DATE returns a string.
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions
Perhaps you want to try a CAST.
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-date
Otherwise, I suggest leaving the column as epoch time because 1) with your current format you lose timezone information, although you never had it so you are assuming UTC? 2) Hive requires less deserialization time for numbers, making queries faster. If you have to format the epoch, do that at the reporting layer, not the storage layer
I am using SQL*Loader to import data from CSV to an Oracle table. My data has multiple date fields and i need to replace null values with '0001-01-01-00-00-00' and other values need to follow date format.
I used decode option like
decode(:QUOTE_CREATE_DT,NULL,'0001-01-01-00.00.00',
TO_DATE(:QUOTE_CREATE_DT,'MM/DD/YYYY HH24:MI:SS'))
which is not working when null values occur.
SQL*Loader error:
Record 2: Rejected - Error on table EX_QUOTE_MO_SAMPLE, column QUOTE_CREATE_DT.
ORA-01847: day of month must be between 1 and last day of month
Your version is relying on implicit conversion using the SQL*Loader session's NLS settings.
You could use decode or coalesce or nvl to provide a date string in the format you're converting, but inside the to_date() call:
QUOTE_CREATE_DT "TO_DATE(NVL(:QUOTE_CREATE_DT, '01/01/0001'), 'MM/DD/YYYY')"
I am trying to get a value of time-stamp column using jdbc driver. The following query works like a charm :
to_char(update_date,'YYYY-MM-DD HH24:MI:SS.FF9 TZR') update_date
However, instead of to_char, I need timestamp values and if I use the following query in my select statemnt, it gives
OTA-01821-Date format not recognized
error. please can someone help?
to_timestamp(update_date,'YYYY-MM-DD HH24:MI:SS.FF9 TZR') update_date
The immediate cause of the error is the TZR part of the format mask, which isn't valid for a plain timestamp. You could either convert to a timestamp with time zone with a different function:
to_timestamp_tz(update_date,'YYYY-MM-DD HH24:MI:SS.FF9 TZR')
or omit the time zone:
to_timestamp(update_date,'YYYY-MM-DD HH24:MI:SS.FF9')
But as uodate_date is already a timestamp (with [local] time zone), you are implicitly converting it to a string using your session's NLS_TIMESTAMP_TZ_FORMAT setting, before explicitly converting that string back to a timestamp (with or without time zone).
At best this is pointless, but if your NLS setting doesn't match the explicit format you use then it will give a different error or incorrect results.
If you want a timestamp value from Java then don't do any conversion - just select the original update_date column, and use the JDBC getTimestamp() to retrieve it.
I'm a Business Intelligence intern and am trying to write a simple ETL batch job to bring one table into our warehouse using SAP Data Services Designer. The source table has a timestamp column, which halts the job's execution, saying:
You cannot select directly from timestamp column . Using
a timestamp column in SQL transform may cause this error. See
documentation or notify Customer Support.
From the technical manual, this limitation is confirmed in the timestamp section, which reads:
You cannot use timestamp columns in the SQL transform or in an Oracle
stored procedure. To use a timestamp column in the SQL transform,
convert the timestamp column in the select list of the SQL transform
to a character format using the to_char function and convert it back
to timestamp using the to_date function."
I've tried remedying the problem by changing the output schema's column to a datetime type, and converting the timestamp in the SQL transform with
TO_DATE(TO_CHAR(SQL.DATETIME_STAMP, 'YYYY-MON-DD HH24:MI:SS'), 'YYYY-MON-DD HH24:MI:SS')
I'm missing a key concept as it still fails with error 54003 no matter what I try. Thoughts, anyone?