I have created table with single column having data type -dateTimeOffset value and inserted some values.
create table dto (dto datetimeoffset(7))
insert into dto values (GETDATE()) -- inserts date and time with 0 offset
insert into dto values (SYSDATETIMEOFFSET()) -- current date time and offset
insert into dto values ('20131114 08:54:00 +10:00') -- manual way
In Nifi,i have specified "Select * from dto" query in Execute SQL .
It shows below error..,
java.lang.IllegalArgumentException: createSchema: Unknown SQL type -155 cannot be converted to Avro type
If i change that column into dateTime then ExecuteSQL runs correctly but it doesn't worked in DateTimeOffset column.
Any help appreciated.
Many thanks
datetimeoffset is a MSSQL-specific JDBC type and is not supported by ExecuteSQL (which supports the standard JDBC types). You could try to cast the datetimeoffset field into some other standard type such as datetime, as described here.
I've created a Custom Processor and adapted the JdbcCommon.java class to include SQL Server's DATETIMEOFFSET. It's just one line of code. I'll try to see if I can ask them to merge this on the official repo.
This is a piece of my JdbcCommon.java:
case TIMESTAMP:
case TIMESTAMP_WITH_TIMEZONE:
case -101: // Oracle's TIMESTAMP WITH TIME ZONE
case -102: // Oracle's TIMESTAMP WITH LOCAL TIME ZONE
case -155: // SQL Server's DATETIMEOFFSET <---- added this line
addNullableField(builder, columnName,
u -> options.useLogicalTypes
? u.type(LogicalTypes.timestampMillis().addToSchema(SchemaBuilder.builder().longType()))
: u.stringType());
break;
Related
In my oracle database contains table Text_Details, field F_TEXT is now an NCLOB field so anywhere this is used in WHERE-clause comparisons may require casting it to other datatypes (NVARCHAR2(4000)probably) for those comparisons.
Text_Details
---------------
Name Data type
Id Number
F_Text NCLOB
any casting is required in where clause comparison with other data types(NVARCHAR2(4000))or any other data types?.Please help.
You can't put a LOBs in the WHERE clause. From the documentation:
however you can use to_char(F_Text) in your where clause like this:
SELECT * FROM WHERE to_char(F_Text) = 'something';
Using Datastage 11.5.0.2, Jobs failed when it try to load the data as "0" into DATE FIELD (DB2)..
In source DB, the column is VARCHAR whereas in target it is DATE field.. the only value in source which failed to load is 0. how to resolve.. any idea pl
Create one stage variable: if (input column value ) = 0 then use StringToDate("0001-01-01", "%dd:%mm:%yyyy")
Else StringToDate(inputcolumn, "%dd:%mm:%yyyy")
You can use the in the transformer stage,
StringToTimestamp(Column,"%yyyy-%mm-%dd")
also you can convert data in Source qualifier by using below in select query
to_date(column,"%yyyy-%mm-%dd")
Can use the to_date function
to_date(column,'YYYY-MM-DD')
TO_DATE(column, 'YYYY-MM-DD HH24:MI:SS')
I have stored some Oracle tables in Hadoop using AVRO file format and Hive external tables to access the data.
I have stored Date and Timestamp values as a formatted String, using the TO_CHAR function from Oracle on the import.
Now I want to export this exact data back with Spark to an Oracle table having a Date column. I use the command:
// Create a data frame from the Hive table
val data = sqlContext.sql("select * from avro_table")
// export df to existing oracle table
data.write.mode("overwrite").jdbc(jdbcString, "tableName", prop)
But then i get the error:
ORA-00902: invalid data type
This is because it tries to insert a string into a date column. Is there a safe way to insert a date / timestamp string from a Spark dataframe to an Oracle date / timestamp column? With safe i mean do not lose any timezone information.
You should use to_date, to_timestamp and/or date_format functions to do the transformation from stringified date/timestamp values to their corresponding type-aware ones.
date_format(dateExpr: Column, format: String): Column Converts a date/timestamp/string to a value of string in the format specified by the date format given by the second argument.
to_date(e: Column, fmt: String): Column Converts the column into a DateType with a specified format (see http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html) return null if fail.
to_timestamp(s: Column, fmt: String): Column Convert time string to a Unix timestamp (in seconds) with a specified format (see http://docs.oracle.com/javase/tutorial/i18n/format/simpleDateFormat.html) to Unix timestamp (in seconds), return null if fail.
Use select or withColumn operators.
A sample code could be as follows:
data.withColumn("real_date", date_format(...))
.write
.mode("overwrite")
.jdbc(jdbcString, "tableName", prop)
I have data in the date column as below.
reportDate
21-Jan-17
02-FEB-17
I want to write a query to fetch data for 01/21/2017?
Below query not working in Oracle.
SELECT * FROM tablename where reportDate=to_date('01/21/2017','mm/dd/yyyy')
What is the data type of reportDate? It may be DATE or VARCHAR2 and there is no way to know by just looking at it.
Run describe table_name (where table_name is the name of the table that contains this column) and see what it says.
If it's a VARCHAR2 then you need to convert it to a date as well. Use the proper format model: 'dd-Mon-rr'.
If it's DATE, it is possible it has time-of-day component; you could apply trunc() to it, but it is better to avoid calling functions on your columns if you can avoid it, for speed. In this case (if it's really DATE data type) the where condition should be
where report_date >= to_date('01/21/2017','mm/dd/yyyy')
and report_date < to_date('01/21/2017','mm/dd/yyyy') + 1
Note that the date on the right-hand side can also be written, better, as
date '2017-01-21'
(this is the ANSI standard date literal, which requires the key word date and exactly the format shown, since it doesn't use a format model; use - as separator and the format yyyy-mm-dd.)
The query should be something like this
SELECT *
FROM table_name
WHERE TRUNC(column_name) = TO_DATE('21-JAN-17', 'DD-MON-RR');
The TRUNC function returns a date value specific to that column.
The o/p which I got when I executed in sqldeveloper
https://i.stack.imgur.com/blDCw.png
i have tryed many things but not able to insert data in my timestamp column.
from toad its possible using this
UPDATE SUPPORTSTAFF
SET SUPPSTAFFJOINDATE=to_timestamp('27/02/2002 15:51.12.539880', 'dd/mm/yyyy hh24:mi.ss.ff')
where JDBCUSERID='5700';
its working
but how can i insert data from java class using create statment and execute query its giving me invalid month error
Use a PreparedStatement with a parameter for the timestamp, e.g.
UPDATE SUPPORTSTAFF SET SUPPSTAFFJOINDATE = ? where JDBCUSERID = ?
and then set the parameters:
statement.setTimestamp(1, new Timestamp(System.currentTimeMillis()));
statement.setString(2, "your ID");
(Then execute the statement, obviously.)