I am using Kafka source JDBC connector to pull DB events and I am running the kafka connect in standalone mode. When I run this file, I get the shown error. Please help me.
Code:
name=sailpointdb01107
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.password = xxxxx
connection.url = jdbc:oracle:thin:#xxxxx:1521/xxxxx
connection.user =xxxxx
query= SELECT * FROM (SELECT NAME, TO_TIMESTAMP('19700101', 'YYYYMMDD')+ NUMTODSINTERVAL(COMPLETED/1000,'SECOND') AS TASKFAILEDON FROM task WHERE COMPLETION_STATUS='Error')
mode= timestamp
timestamp.column.name=TASKFAILEDON
topic.prefix=testing
validate.non.null=false
transforms=TimestampConverter
transforms.TimestampConverter.type=org.apache.kafka.connect.transforms.TimestampConverter$Value
transforms.TimestampConverter.format=yyyy-MM-dd
transforms.TimestampConverter.target.type=Timestamp
transforms.TimestampConverter.target.field=TASKFAILEDON
Error:
Error[2019-10-01 15:17:45,058] ERROR WorkerSourceTask{id=sailpointdb01107-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask)
org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
Caused by: org.apache.kafka.connect.errors.ConnectException: Schema Schema{STRUCT} does not correspond to a known timestamp type format
at org.apache.kafka.connect.transforms.TimestampConverter.timestampTypeFromSchema(TimestampConverter.java:406)
at org.apache.kafka.connect.transforms.TimestampConverter.applyWithSchema(TimestampConverter.java:334)
at org.apache.kafka.connect.transforms.TimestampConverter.apply(TimestampConverter.java:275)
at org.apache.kafka.connect.runtime.TransformationChain.lambda$apply$0(TransformationChain.java:50)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)
... 11 more
[2019-10-01 15:17:45,059] ERROR WorkerSourceTask{id=sailpointdb01107-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
This line configured at connector could avoid that issue.
time.precision.mode: "connect"
Related
I have kafka 2.13-3.2.0 running on Window machine. I am trying stream Join operation and getting following error. I can see same issue fixed in 1.0.1 version as per : https://issues.apache.org/jira/browse/KAFKA-6162
But I am still getting this error with kafka 2.13-3.2.0.
Error Logs:
Caused by: org.rocksdb.RocksDBException: Failed to create dir: D:\tmp\kafka-streams\join_driver_application\1_0\KSTREAM-JOINTHIS-0000000014-store\KSTREAM-JOINTHIS-0000000014-store:1661385600000: Invalid argument
at org.rocksdb.RocksDB.open(Native Method)
at org.rocksdb.RocksDB.open(RocksDB.java:231)
at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:197)
... 23 more
I am using Hadoop-2.6.0 secured with kerberos. Installed hive server2 1.1.0 version with derby database as connectionurl, enabled security and enabled Authorization. When enabling transaction configuration, I am getting the below exception and cannot execute any queries;
Exception
Error: Error while compiling statement: FAILED: LockException [Error 10280]: Error communicating with the metastore (state=42000,code=10280)
Logs
[Error 10280]: Error communicating with the metastore
org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastore
at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:300)
at org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:927)
Caused by: MetaException(message:Unable to select from transaction database, java.sql.SQLSyntaxErrorException: Table/View 'TXNS' does not exist.
So i have created a below property in hive-site.xml file as mentioned in a blog here
Configuration
<property>
<name>hive.in.test</name>
<value>true</value>
</property>
If i set the above property then getting the below exception where i am struck and unable to solve it. I cannot run any query even use mydb;
Exception
Error: Error while compiling statement: FAILED: NullPointerException null (state=42000,code=40000)
Logs
Error executing statement:
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: NullPointerException null
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:103)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkQFileTestHack(TxnHandler.java:1146)
at org.apache.hadoop.hive.metastore.txn.TxnHandler.(TxnHandler.java:117)
I need a solution to work ACID transactions in Hive Server2. I found two related questions but not solved my issue.
hive 0.14 update and delete queries configuration error
Hive Transactions are crashing
Upgrade your hive mysql metastore db with hive-txn-schema-0.14.0.mysql.sql as follows..
mysql> SOURCE /usr/local/hadoop/hive/scripts/metastore/upgrade/mysql/hive-txn-schema-0.14.0.mysql.sql;
I am new to JasperReports Server and have searched for a resolve to this issue. I have found nothing. I inherited this server and the reports are not running as scheduled.
Job: 3rd (ID: 61)
Report unit: /reports/Scheduled/00_Schedule_Primer
Quartz Job: ReportJobs.job_61
Quartz Trigger: ReportJobs.trigger_62_1
Exceptions:
An error occurred while executing the report.
com.jaspersoft.jasperserver.api.JSException: jsexception.error.creating.connection
at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.JdbcDataSourceService.createConnection(JdbcDataSourceService.java:62)
at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.BaseJdbcDataSource.setReportParameterValues(BaseJdbcDataSource.java:52)
at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.JdbcDataSourceService.setReportParameterValues(JdbcDataSourceService.java:67)
at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.EngineServiceImpl.fillReport(EngineServiceImpl.java:743)
at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.EngineServiceImpl.fillReport(EngineServiceImpl.java:367)
at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.EngineServiceImpl.executeReport(EngineServiceImpl.java:876)
at com.jaspersoft.jasperserver.api.engine.jasperreports.domain.impl.ReportUnitRequest.execute(ReportUnitRequest.java:60)
at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.EngineServiceImpl.execute(EngineServiceImpl.java:301)
at com.jaspersoft.jasperserver.api.engine.scheduling.quartz.ReportExecutionJob.executeReport(ReportExecutionJob.java:444)
at com.jaspersoft.jasperserver.api.engine.scheduling.quartz.ReportExecutionJob.executeAndSendReport(ReportExecutionJob.java:372)
at com.jaspersoft.jasperserver.api.engine.scheduling.quartz.ReportExecutionJob.execute(ReportExecutionJob.java:188)
at org.quartz.core.JobRunShell.run(JobRunShell.java:195)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:520)
Caused by: java.sql.SQLException: Middleware connect fail:No RPC Connection active.
at com.ibm.u2.jdbc.UniJDBCMsgFactory.createException(UniJDBCMsgFactory.java:113)
at com.ibm.u2.jdbc.UniJDBCExceptionSupport.addAndThrowException(UniJDBCExceptionSupport.java:62)
at com.ibm.u2.jdbc.UniJDBCProtocolU2Impl.connect(UniJDBCProtocolU2Impl.java:746)
at com.ibm.u2.jdbc.UniJDBCProtocolU2Impl.executeOpenDatabase(UniJDBCProtocolU2Impl.java:243)
at com.ibm.u2.jdbc.UniJDBCConnectionImpl.<init>(UniJDBCConnectionImpl.java:116)
at com.ibm.u2.jdbc.UniJDBCDriver.connect(UniJDBCDriver.java:111)
at java.sql.DriverManager.getConnection(DriverManager.java:582)
at java.sql.DriverManager.getConnection(DriverManager.java:185)
at org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:48)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:290)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:771)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
at com.jaspersoft.jasperserver.api.engine.jasperreports.service.impl.JdbcDataSourceService.createConnection(JdbcDataSourceService.java:58)
... 12 more
I don't even have a clue where to start troubleshooting this. ANY help would be awesome!
It seems your connection url, username and password values are not proper.If these three things are proper then the connection will be created.Check in your Jasper server configuration.
The title is a bit weird as I'm having difficulties narrowing down the problem. I used my solution on Hadoop 2.0.0-cdh4.4.0 and hive 0.10 without issues.
I can't create a table using this SerDe: https://github.com/rcongiu/Hive-JSON-Serde
first try:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector.<init>(Lorg/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils$PrimitiveTypeEntry;)V
second try:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Could not initialize class org.openx.data.jsonserde.objectinspector.JsonObjectInspectorFactory
I can create a table with this SerDe: https://github.com/cloudera/cdh-twitter-example
I create an external table with tweets from flume. I can't do "SELECT * FROM tweets;"
FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: Failed with exception java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDejava.lang.RuntimeException: java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDe
I can do SELECT id, text FROM tweets;
I can do a SELECT COUNT(*) FROM tweets;
I can't self join this table:
Execution log at: /tmp/jochen.debie/jochen.debie_20140311121313_164611a9-b0d8-4e53-9bda-f9f7ac342aaf.log
2014-03-11 12:13:30 Starting to launch local task to process map join; maximum memory = 257294336
Execution failed with exit status: 2
Obtaining error information
Task failed!
Task ID:
Stage-5
mentioned execution log:
2014-03-11 12:13:30,331 ERROR mr.MapredLocalTask (MapredLocalTask.java:executeFromChildJVM(324)) - Hive Runtime Error: Map local work failed
org.apache.hadoop.hive.ql.metadata.HiveException: Failed with exception java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDejava.lang.RuntimeException: java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDe
Does anyone know how to fix this or at least show me where the problem is?
EDIT: Can it be a problem that I built the serde on a Hadoop 2.0.0-cdh4.4.0 and hive 0.10?
From what I've seen, Hive-.11+ has a bug in join with custom SerDe.
https://github.com/Esri/gis-tools-for-hadoop/issues/9
You might try the workaround of copying the JAR file containing the SerDe class, to $HIVE_HOME/lib .
(I see in your question you got ClassNotFoundException both in join and in other cases; so far the times I have encountered such were all with join.)
[Edit] Another workaround is to use HADOOP_CLASSPATH:
env HADOOP_CLASSPATH=some.jar:other.jar hive ...
[Edit] The work around applies to Hive versions 0.11 and 0.12; then 0.13 and above contain the fix for HIVE-6670.
Can any one suggest me why the following error is occurring and how to resolve it??
Not only the below command, running any command related to Hive is returning the same..
hive> show databases;
FAILED: Error in metadata: MetaException(message:Got exception: org.apache.thrif
t.transport.TTransportException java.net.SocketException: Connection reset by pe
er: socket write error)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTa
sk
Check out your hive-site.xml. It is possible your javax.jdo.option.ConnectionURL, the URL for the Hive metastore, isn't right.