I am getting this error when attempting to stream an RDS MySQL table into Redshift: Error converting data, invalid type for parameter
The problem field is a DATETIME in MySQL and timestamp without time zone in Redshift (same happens for timestamp with time zone). Note: pipeline was working fine until I populated the date field.
We are using Debezium as the Kafka Connect source for getting data from RDS into Kafka. And the JDBC sink connector with Redshift JDBC driver for the sink.
Also... I am able to get the data flowing if I make the Redshift field a varchar or a bigint. When I do this, I see that the data is coming across as a unix epoch integer in ms. But we'd really like a timestamp!
Error message in context:
2018-10-18 22:48:32,972 DEBUG || INSERT sql: INSERT INTO "funschema"."test_table"("user_id","subscription_code","source","receipt","starts_on") VALUES(?,?,?,?,?) [io.confluent.connect.jdbc.sink.BufferedRecords]
2018-10-18 22:48:32,987 WARN || Write of 28 records failed, remainingRetries=7 [io.confluent.connect.jdbc.sink.JdbcSinkTask]
java.sql.BatchUpdateException: [Amazon][JDBC](10120) Error converting data, invalid type for parameter: 5.
at com.amazon.jdbc.common.SStatement.createBatchUpdateException(Unknown Source)
at com.amazon.jdbc.common.SStatement.access$100(Unknown Source)
at com.amazon.jdbc.common.SStatement$BatchExecutionContext.createBatchUpdateException(Unknown Source)
at com.amazon.jdbc.common.SStatement$BatchExecutionContext.createResults(Unknown Source)
at com.amazon.jdbc.common.SStatement$BatchExecutionContext.doProcess(Unknown Source)
at com.amazon.jdbc.common.SStatement$BatchExecutionContext.processInt(Unknown Source)
at com.amazon.jdbc.common.SStatement.processBatchResults(Unknown Source)
at com.amazon.jdbc.common.SPreparedStatement.executeBatch(Unknown Source)
at io.confluent.connect.jdbc.sink.BufferedRecords.flush(BufferedRecords.java:138)
at io.confluent.connect.jdbc.sink.JdbcDbWriter.write(JdbcDbWriter.java:66)
at io.confluent.connect.jdbc.sink.JdbcSinkTask.put(JdbcSinkTask.java:75)
Thanks,
Tom
Related
I am trying to insert data from Kafka to Teradata. The payload has some null values, and the JDBC sink is throwing the following error.
[Teradata JDBC Driver] [TeraJDBC 16.20.00.10] [Error 1063] [SQLState HY000] null is not supported as a data value with this variant of the setObject method; use the setNull method or the setObject method with a targetSqlType parameter
My connector config:
name=teradata-sink-K_C_OSUSR_DGL_DFORM_I1-V2
connector.class=io.confluent.connect.jdbc.JdbcSinkConnector
tasks.max=1
connection.url=connectionString
topics=POPS-P-OSUSR_DGL_DFORM_I1-J-V2-CAL-OUT
topic.prefix=
table.name.format=K_C_OSUSR_DGL_DFORM_I1_V2
batch.size=50000
errors.tolerance=all
errors.deadletterqueue.topic.name=POPS-P-OSUSR_DGL_DFORM_I1-V2-CAL-DEAD
errors.deadletterqueue.topic.replication.factor=1
Is there a way to achieve this? I do not know if I have to change some code into the sink or just change the connector config.
You are getting the error from some line that surely look like this:
ps.setObject(1, val);
This one will throw an exception if the val you try to insert has a null value.
The error is telling that you must specify the data type of the null values incoming. You could do this:
ps.setObject(1, val, Types.VARCHAR);
This way you are casting NULL to a VARCHAR, one of the supported targetSqlTypes.
Another option for the same purpose:
ps.setNull(1, Types.VARCHAR) ;
The problem we have is we are using the standard Kafka Connect to create a sink (we are not coding any custom connector).
We have configured both .properties files for worker and connector to create a link between the topic and the teradata table, and we run it using
.../confluent/bin/connect-standalone <worker.cfg> <connector.cfg>
When we create a message with "null" values and send it into the topic, the Sink Connector is unable to insert the record into the TD table.
I am trying to read data from Oracle server which is running in 'CST' timezone. My google DataFlow is running in 'us-central1' region. I am using Apache Beam-2.3.0 JDBCIO.read() method to read data from Oracle server. I am able to connect and read data from server using 'DirectRunner' but getting the following error in 'DataflowRunner' using ojdbc8 driver jar
(901b8e8f2f8a547a): java.lang.RuntimeException: org.apache.beam.sdk.util.UserCodeException: java.sql.SQLException: Cannot create PoolableConnectionFactory (ORA-00604: error occurred at recursive SQL level 1
ORA-01882: timezone region not found
)
at com.google.cloud.dataflow.worker.MapTaskExecutorFactory$3.typedApply(MapTaskExecutorFactory.java:338)
at com.google.cloud.dataflow.worker.MapTaskExecutorFactory$3.typedApply(MapTaskExecutorFactory.java:308)
at com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63)
at com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50)
at com.google.cloud.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87)
at com.google.cloud.dataflow.worker.MapTaskExecutorFactory.create(MapTaskExecutorFactory.java:154)
at com.google.cloud.dataflow.worker.DataflowWorker.doWork(DataflowWorker.java:308)
at com.google.cloud.dataflow.worker.DataflowWorker.getAndPerformWork(DataflowWorker.java:264)
at com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:133)
at com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:113)
at com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:100)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.beam.sdk.util.UserCodeException: java.sql.SQLException: Cannot create PoolableConnectionFactory (ORA-00604: error occurred at recursive SQL level 1
ORA-01882: timezone region not found
)
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:36)
at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn$DoFnInvoker.invokeSetup(Unknown Source)
at com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:63)
at com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:45)
at com.google.cloud.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:94)
at com.google.cloud.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:74)
at com.google.cloud.dataflow.worker.MapTaskExecutorFactory.createParDoOperation(MapTaskExecutorFactory.java:415)
at com.google.cloud.dataflow.worker.MapTaskExecutorFactory$3.typedApply(MapTaskExecutorFactory.java:326)
... 14 more
Caused by: java.sql.SQLException: Cannot create PoolableConnectionFactory (ORA-00604: error occurred at recursive SQL level 1
ORA-01882: timezone region not found
)
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2294)
at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2039)
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533)
at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.setup(JdbcIO.java:503)
Caused by: java.sql.SQLException: ORA-00604: error occurred at recursive SQL level 1
ORA-01882: timezone region not found
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:494)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:441)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:436)
at oracle.jdbc.driver.T4CTTIfun.processError(T4CTTIfun.java:1061)
at oracle.jdbc.driver.T4CTTIoauthenticate.processError(T4CTTIoauthenticate.java:550)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:623)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:499)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:1279)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:663)
at oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:688)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:39)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:691)
at org.apache.commons.dbcp2.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:39)
at org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:256)
at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2304)
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2290)
at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2039)
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533)
at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn.setup(JdbcIO.java:503)
at org.apache.beam.sdk.io.jdbc.JdbcIO$ReadFn$DoFnInvoker.invokeSetup(Unknown Source)
at com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.deserializeCopy(DoFnInstanceManagers.java:63)
at com.google.cloud.dataflow.worker.DoFnInstanceManagers$ConcurrentQueueInstanceManager.peek(DoFnInstanceManagers.java:45)
at com.google.cloud.dataflow.worker.UserParDoFnFactory.create(UserParDoFnFactory.java:94)
at com.google.cloud.dataflow.worker.DefaultParDoFnFactory.create(DefaultParDoFnFactory.java:74)
at com.google.cloud.dataflow.worker.MapTaskExecutorFactory.createParDoOperation(MapTaskExecutorFactory.java:415)
at com.google.cloud.dataflow.worker.MapTaskExecutorFactory$3.typedApply(MapTaskExecutorFactory.java:326)
at com.google.cloud.dataflow.worker.MapTaskExecutorFactory$3.typedApply(MapTaskExecutorFactory.java:308)
at com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:63)
at com.google.cloud.dataflow.worker.graph.Networks$TypeSafeNodeFunction.apply(Networks.java:50)
at com.google.cloud.dataflow.worker.graph.Networks.replaceDirectedNetworkNodes(Networks.java:87)
at com.google.cloud.dataflow.worker.MapTaskExecutorFactory.create(MapTaskExecutorFactory.java:154)
at com.google.cloud.dataflow.worker.DataflowWorker.doWork(DataflowWorker.java:308)
at com.google.cloud.dataflow.worker.DataflowWorker.getAndPerformWork(DataflowWorker.java:264)
at com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:133)
at com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:113)
at com.google.cloud.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:100)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Note: Connected and read data from server successfully using ojdbc6 driver in both DirectRunner and DataflowRunner. But I want to get it work using ojdbc8 driver jar.
The following is my datasourceConfiguration for JDBCIO
DataSourceConfiguration dataSourceConfiguration = JdbcIO.DataSourceConfiguration
.create(options.getDriverName(), options.getJdbcUrl())
.withUsername(options.getUsername())
.withPassword(options.getPassword())
.withConnectionProperties("timezone=CST");
Any inputs to make this connection successfull using ojdbc8 driver jar in 'DataflowRunner'?
It looks like the error (ORA-00604: error occurred at recursive SQL level 1
ORA-01882: timezone region not found is known and quite common with the Oracle JDBC drivers.
It happens when the the JDBC driver is not able, for some reason, to send the right Timezone ID to the server. After some investigation and searches across different sources (including other Stack Overflow cases such as this one or this other one), I have found different possible solutions, so let me summarize them here:
Make sure that you are using the latest available version of the ojdbc8 driver, because maybe there was something broken in the specific version you were using. Try to change to a different version and see if that works.
Try setting the default Timezone to your timezone before establishing the connection, as in Solution 2 below.
Add the configuration line in Solution 3 to the file oracle/jdbc/defaultConnectionProperties.properties.
Summarized solutions:
// Solution 2
TimeZone timeZone = TimeZone.getTimeZone("yourTimeZone");
TimeZone.setDefault(timeZone);
// Solution 3
oracle.jdbc.timezoneAsRegion=false
I was doing the Hadoop(2.6.0) twitter example by Flume(1.5.2) and Hive(0.14.0). I got data from twitter successfully via Flume and stored them to the my own hdfs.
But when I wanted to use hive to deal with these data to do some analyzing (only select one field from a table), the "Failed with exception java.io.IOException:org.apache.avro.AvroRuntimeException: java.io.EOFException" exception happened and little useful information I could find related to this exception.
Actuall I can fetch most records of a file successfully (like the information below, I fetched 5100 rows successfully) but it would fail in the end. As a result I cannot deal with all the tweets files together.
Time taken: 1.512 seconds, Fetched: 5100 row(s)
Failed with exception java.io.IOException:org.apache.avro.AvroRuntimeException: java.io.EOFException
15/04/15 19:59:18 [main]: ERROR CliDriver: Failed with exception java.io.IOException:org.apache.avro.AvroRuntimeException: java.io.EOFException
java.io.IOException: org.apache.avro.AvroRuntimeException: java.io.EOFException
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:663)
at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:561)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1621)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:267)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:783)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.avro.AvroRuntimeException: java.io.EOFException
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:222)
at org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader.next(AvroGenericRecordReader.java:153)
at org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader.next(AvroGenericRecordReader.java:52)
at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:629)
... 15 more
Caused by: java.io.EOFException
at org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473)
at org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128)
at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:259)
at org.apache.avro.io.ValidatingDecoder.readString(ValidatingDecoder.java:107)
at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:348)
at org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:341)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:154)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:177)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:148)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:139)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:233)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:220)
... 18 more
I use the hql below to create a table
CREATE TABLE tweets
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
TBLPROPERTIES ('avro.schema.url'='file:///home/hduser/hive-0.14.0-bin/tweetsdoc_new.avsc');
then load tweets file from hdfs
LOAD DATA INPATH '/user/flume/tweets/FlumeData.1429098355304' OVERWRITE INTO TABLE tweets;
Could anyone tell me the possible reason, or an effective way to find more details of the exception?
I had this annoying problem as well.
I looked at the produced binary file and debugged Avro deserialization of bits.
The reason for this EOFException was that Flume inserts new line character byte after every event (you can notice 0x0A after every record).
Avro deserializer thinks the file hasn't finished and interprets that character as some number of blocks to read, but then can't read out that number of blocks without hitting EOF.
I am facing an issue with persisting records for an Entity(YieldCurveArchive) with a field (reason) having a length equivalent to 2048 characters. Following are the scenarios and their results:
Upload new Entity with smaller reason fields: Works fine.
Upload Entity with reason fields
changed with larger (2048 characters) data set: Works fine. Again as
above. This is an update of the existing records.
Upload new Entity
with larger (2048 characters) reason fields: Fails.
I have also tried flushing the hibernate query buffer using entityManager.flush() but the above test results do not change.
I suspect there could be an issue with the buffering that Hibernate performs before actually doing a final insert into database:
Selection of the available records in database to compare with
existing dataset.
Insert being fired but only kept in the buffer of
hibernate to fire a bulk update.
Another select being fired for a
different client. Upon seeing another select, hibernate decides to
fire the insert into the database and then fails.
An excerpt of the trace from the failed logs:
2013-07-02 12:46:00,792 WARN [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - SQL Error: 1400, SQLState: 23000
2013-07-02 12:46:00,792 ERROR [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - ORA-01400: cannot insert NULL into ("CORE_TOTEM"."YC_MONTHLY_ARCHIVE"."PKEY")
2013-07-02 12:46:00,792 WARN [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - SQL Error: 1400, SQLState: 23000
2013-07-02 12:46:00,792 ERROR [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - ORA-01400: cannot insert NULL into ("CORE_TOTEM"."YC_MONTHLY_ARCHIVE"."PKEY")
2013-07-02 12:46:00,985 ERROR [pool-1-thread-4]: org.hibernate.event.def.AbstractFlushingEventListener - Could not synchronize database state with session
org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:94)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:275)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:266)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:64)
at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:996)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1141)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:65)
at markit.totem.dao.DaoHelper.list(DaoHelper.java:19)
at com.markit.totem.rates.yieldcurve.dao.YieldCurveArchiveDao.getArchivesMonthly(YieldCurveArchiveDao.java:172)
at com.markit.totem.rates.yieldcurve.dao.YieldCurveArchiveDao$$FastClassByCGLIB$$55ef1569.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:628)
at com.markit.totem.rates.yieldcurve.dao.YieldCurveMonthlyArchiveDao$$EnhancerByCGLIB$$6aa04fe7.getArchivesMonthly(<generated>)
at com.markit.totem.rates.yieldcurve.results.upload.monthly.YieldCurveMonthlyArchivePersister.getExisting(YieldCurveMonthlyArchivePersister.java:38)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveArchivePersister.persist(YieldCurveArchivePersister.java:68)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveArchivePersister$$FastClassByCGLIB$$ac0db3fb.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:628)
at com.markit.totem.rates.yieldcurve.results.upload.monthly.YieldCurveMonthlyArchivePersister$$EnhancerByCGLIB$$4b00cfa0.persist(<generated>)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveResultsUploader.upload(YieldCurveResultsUploader.java:158)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveResultsUploadTask.run(YieldCurveResultsUploadTask.java:53)
at com.markit.totem.workflow.WorkflowExecutor.executeWorkflowTask(WorkflowExecutor.java:258)
at com.markit.totem.workflow.WorkflowExecutor.executeSubWorkflow(WorkflowExecutor.java:227)
at com.markit.totem.workflow.WorkflowExecutor.access$000(WorkflowExecutor.java:17)
at com.markit.totem.workflow.WorkflowExecutor$1.run(WorkflowExecutor.java:72)
at markit.totem.dao.Transactionator.execute(Transactionator.java:19)
at markit.totem.dao.Transactionator$$FastClassByCGLIB$$c9204755.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:700)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:635)
at markit.totem.dao.Transactionator$$EnhancerByCGLIB$$4eec78e2.execute(<generated>)
at com.markit.totem.workflow.WorkflowExecutor.execute(WorkflowExecutor.java:83)
at com.markit.totem.workflow.WorkflowExecutor.execute(WorkflowExecutor.java:213)
at com.markit.totem.workflow.WorkflowManager$OldWorkflow.execute(WorkflowManager.java:218)
at com.markit.totem.workflow.WorkflowManager$1.run(WorkflowManager.java:119)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.sql.BatchUpdateException: ORA-01400: cannot insert NULL into ("CORE_TOTEM"."YC_MONTHLY_ARCHIVE"."PKEY")
at oracle.jdbc.driver.DatabaseError.throwBatchUpdateException(DatabaseError.java:367)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9055)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1723)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
My initial thoughts were that it fails due to the encoding difference in database and default encoding from Hibernate.
Like underlying database uses Oracle and the NLS_CHARACTERSET is defined as:
NLS_CHARACTERSET AL32UTF8
Hence it stores, 1 character as 4 bytes and any special character as 6 bytes. My insert into the Oracle fails for 1001 but passes till i insert 1000 characters. However, if I do an update with more characters thats not the case, which makes it more confusing.
Any pointers would be of great help?
I'm trying to get some data from a remote Oracle Database,
so I configured a new connection to the database and whn I press Test, it says that the connection was established succesfully, but when i tried a simple select query, Report Designer gives me an error:
org.pentaho.reporting.engine.classic.core.ReportDataFactoryException: Failed at query: select * from fact_table;
at org.pentaho.reporting.engine.classic.core.modules.misc.datafactory.sql.SimpleSQLReportDataFactory.queryData(SimpleSQLReportDataFactory.java:258)
at org.pentaho.reporting.engine.classic.core.modules.misc.datafactory.sql.SQLReportDataFactory.queryData(SQLReportDataFactory.java:171)
at org.pentaho.reporting.ui.datasources.jdbc.ui.JdbcPreviewWorker.run(JdbcPreviewWorker.java:103)
at java.lang.Thread.run(Unknown Source)
ParentException:
java.sql.SQLException: ORA-00911: invalid character
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:125)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:305)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:272)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:623)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:112)
at oracle.jdbc.driver.T4CStatement.execute_for_describe(T4CStatement.java:351)
at oracle.jdbc.driver.OracleStatement.execute_maybe_describe(OracleStatement.java:896)
at oracle.jdbc.driver.T4CStatement.execute_maybe_describe(T4CStatement.java:383)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:986)
at oracle.jdbc.driver.OracleStatement.doScrollExecuteCommon(OracleStatement.java:3763)
at oracle.jdbc.driver.OracleStatement.doScrollStmtExecuteQuery(OracleStatement.java:3887)
at oracle.jdbc.driver.OracleStatement.executeQuery(OracleStatement.java:1131)
at org.pentaho.reporting.engine.classic.core.modules.misc.datafactory.sql.SimpleSQLReportDataFactory.parametrizeAndQuery(SimpleSQLReportDataFactory.java:422)
at org.pentaho.reporting.engine.classic.core.modules.misc.datafactory.sql.SimpleSQLReportDataFactory.queryData(SimpleSQLReportDataFactory.java:254)
at org.pentaho.reporting.engine.classic.core.modules.misc.datafactory.sql.SQLReportDataFactory.queryData(SQLReportDataFactory.java:171)
at org.pentaho.reporting.ui.datasources.jdbc.ui.JdbcPreviewWorker.run(JdbcPreviewWorker.java:103)
at java.lang.Thread.run(Unknown Source)
So how can I get this done?
select * from fact_table; seems to be a valid query. Try to remove the semicolon at the end.