Snowflake Kafka Connect: How to avoid the "Error 5016. SinkRecord.value and SinkRecord.valueSchema cannot be null"? - apache-kafka-connect

We are just starting to utilize Kafka Connect to pipe our data into Snowflake.
We are hitting this error:
"
com.snowflake.kafka.connector.internal.SnowflakeKafkaConnectorException:
[SF_KAFKA_CONNECTOR] Exception: Invalid SinkRecord received
[SF_KAFKA_CONNECTOR] Error Code: 5016
[SF_KAFKA_CONNECTOR] Detail: SinkRecord.value and SinkRecord.valueSchema cannot be null
"
What can we do / look at to avoid this error?

Related

org.h2.jdbc.JdbcSQLNonTransientException: IO Exception: null [90028-200]

My h2db version is h2-1.4.200.
the database started few times ago,reboot the database but cannot connected!
The problem is:
IO Exception: null [90028-200] 90028/90028 (Help)
org.h2.jdbc.JdbcSQLNonTransientException: IO Exception: null [90028-200]
Thanks a lot for your help!!!!!

Illegal 'sec-websocket-protocol' header: Invalid input 'EOI', expected sec-websocket-protocol

I'm trying to connect on Kubernetes websocket API but I get this error message:
java.lang.IllegalStateException: Message failed with status code 400 Bad Request; Error info: ErrorInfo(Illegal 'sec-websocket-protocol' header: Invalid input 'EOI', expected sec-websocket-protocol (line 1, column 1),
^)
at akka.http.impl.engine.ws.WebSocketClientBlueprint$UpgradeStage$1$$anon$1.onPush(WebSocketClientBlueprint.scala:107)
at akka.stream.impl.fusing.GraphInterpreter.processPush(GraphInterpreter.scala:747)
at akka.stream.impl.fusing.GraphInterpreter.execute(GraphInterpreter.scala:649)
at akka.stream.impl.fusing.GraphInterpreterShell.runBatch(ActorGraphInterpreter.scala:471)
at akka.stream.impl.fusing.GraphInterpreterShell.receive(ActorGraphInterpreter.scala:410)
at akka.stream.impl.fusing.ActorGraphInterpreter.akka$stream$impl$fusing$ActorGraphInterpreter$$processEvent(ActorGraphInterpreter.scala:603)
at akka.stream.impl.fusing.ActorGraphInterpreter$$anonfun$receive$1.applyOrElse(ActorGraphInterpreter.scala:618)
at akka.actor.Actor.aroundReceive(Actor.scala:502)
at akka.actor.Actor.aroundReceive$(Actor.scala:500)
at akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive(ActorGraphInterpreter.scala:529)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
at akka.actor.ActorCell.invoke(ActorCell.scala:495)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
I must be missing some configuration, do you knwo where this comes from?
Cheers
Got over it by defining subprotocol to channel.k8s.io:
WebSocketRequest(
uri,
extraHeaders = List(token),
subprotocol = Option("channel.k8s.io")
)

SQL error 17002 : Physical DB connection is invalid

I am trying to connect to oracle db(installed in remote machine) from my java application. but, i am getting sql error code 17022. pls let me know how to solve this issue.
Full stack trace
MSG: Connecting to jdbc:oracle:thin:#(description=(address_list=(address=(protocol=tcp)(host=dlocdb04)(port=50000))(address=(protocol=tcp)(host=dlocdb04)(port=50000)))(connect_data=(service_name=os02rtdb01svc.world)(server=dedicated))) with Oracle JDBC driver version: 9.2.0.1.0 [cumulative retries=0]
USER_ID: UNKNOWN
level: WARN
date: 2015-09-21 10:51:45,563
category: com.retek.merch.utils.ConnectionPool
MSG: [SQL Error code: 17002, State: null] Physical DB connection is invalid. Restarting pool due to the following error.
USER_ID: UNKNOWN
java.sql.SQLException: Io exception: Connection refused(DESCRIPTION=(ERR=1153)(VSNNUM=0)(ERROR_STACK=(ERROR=(CODE=1153)(EMFI=4)(ARGS='(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=dlocdb0401)(PORT=27320))(CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))null))'))(ERROR=(CODE=303)(EMFI=1))))
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:134)
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:179)
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:333)
at oracle.jdbc.driver.OracleConnection.<init>(OracleConnection.java:404)
at oracle.jdbc.driver.OracleDriver.getConnectionInstance(OracleDriver.java:468)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:314)
at java.sql.DriverManager.getConnection(DriverManager.java:512)
at java.sql.DriverManager.getConnection(DriverManager.java:140)
at oracle.jdbc.pool.OracleDataSource.getConnection(OracleDataSource.java:169)
at oracle.jdbc.pool.OracleConnectionPoolDataSource.getPhysicalConnection(OracleConnectionPoolDataSource.java:149)
at oracle.jdbc.pool.OracleConnectionPoolDataSource.getPooledConnection(OracleConnectionPoolDataSource.java:95)
at oracle.jdbc.pool.OracleConnectionPoolDataSource.getPooledConnection(OracleConnectionPoolDataSource.java:63)
at oracle.jdbc.pool.OracleConnectionCacheImpl.getNewPoolOrXAConnection(OracleConnectionCacheImpl.java:547)
at oracle.jdbc.pool.OracleConnectionCacheImpl.getPooledConnection(OracleConnectionCacheImpl.java:404)
at oracle.jdbc.pool.OracleConnectionCacheImpl.getConnection(OracleConnectionCacheImpl.java:298)
at oracle.jdbc.pool.OracleConnectionCacheImpl.getConnection(OracleConnectionCacheImpl.java:268)
at com.retek.merch.utils.ConnectionPool.get(ConnectionPool.java:346)
at com.retek.merch.utils.TransactionManager.start(TransactionManager.java:59)
at com.retek.reim.merch.utils.ReIMTransactionManager.start(ReIMTransactionManager.java:49)
at com.retek.reim.manager.LoginManager.login(LoginManager.java:72)
at com.retek.reim.ui.login.LoginAction.perform(LoginAction.java:47)
at org.apache.struts.action.ActionServlet.processActionPerform(ActionServlet.java:1786)
at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1585)
at org.apache.struts.action.ActionServlet.doPost(ActionServlet.java:509)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:638)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:720)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:199)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:145)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:955)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:139)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:955)
at org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2460)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:133)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:119)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:116)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:955)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:127)
at org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:955)
at org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:157)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:873)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:534)

Why Impala not working on hbase table?

I create an external table B of hbase table A using hive. I can successfully access the data of B.Then I followed the official guide to type in Imapla Shell:
invalidate metadata B;
And then I query this external table B in Impala Shell:
select * from B limit 4;
but it outputs:
ERROR: RuntimeException: couldn't retrieve HBase table (mv_p2pusers) info:
Enable/Disable failed
Here are some logs related:
11:13:58.937 AM INFO jni-util.cc:177
java.lang.RuntimeException: couldn't retrieve HBase table (mv_p2pusers) info:
Enable/Disable failed
at com.cloudera.impala.planner.HBaseScanNode.computeScanRangeLocations(HBaseScanNode.java:300)
at com.cloudera.impala.planner.HBaseScanNode.init(HBaseScanNode.java:125)
at com.cloudera.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:891)
at com.cloudera.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:1082)
at com.cloudera.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:526)
at com.cloudera.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:151)
at com.cloudera.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:117)
at com.cloudera.impala.planner.Planner.createPlan(Planner.java:47)
at com.cloudera.impala.service.Frontend.createExecRequest(Frontend.java:842)
at com.cloudera.impala.service.JniFrontend.createExecRequest(JniFrontend.java:146)
11:13:58.939 AM INFO status.cc:114
RuntimeException: couldn't retrieve HBase table (mv_p2pusers) info:
Enable/Disable failed
# 0x78b793 (unknown)
# 0xa68275 (unknown)
# 0x9802c6 (unknown)
# 0x99db78 (unknown)
# 0x99e6e4 (unknown)
# 0x9d50cb (unknown)
# 0xb33687 (unknown)
# 0xb29054 (unknown)
# 0x9ac52b (unknown)
# 0x1571c39 (unknown)
# 0x155d9cf (unknown)
# 0x155f914 (unknown)
# 0x92d363 (unknown)
# 0x92daca (unknown)
# 0xaa4faa (unknown)
# 0xaa7130 (unknown)
# 0xca79b3 (unknown)
# 0x386be079d1 (unknown)
# 0x386bae8b6d (unknown)
11:13:58.940 AM INFO impala-server.cc:824
UnregisterQuery(): query_id=d4269ff898eb4e7:1866144af0d14a7
11:13:58.940 AM INFO impala-server.cc:893
Cancel(): query_id=d4269ff898eb4e7:1866144af0d14a7
11:13:59.935 AM INFO ClientCnxn.java:975
Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
11:13:59.935 AM WARN ClientCnxn.java:1102
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
Java exception follows:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
11:14:01.036 AM INFO ClientCnxn.java:975
Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
11:14:01.037 AM WARN ClientCnxn.java:1102
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
Java exception follows:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
11:14:02.138 AM INFO ClientCnxn.java:975
Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
11:14:02.138 AM WARN ClientCnxn.java:1102
Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
Java exception follows:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
11:14:02.199 AM INFO impala-hs2-server.cc:795
GetSchemas(): request=TGetSchemasReq {
01: sessionHandle (struct) = TSessionHandle {
01: sessionId (struct) = THandleIdentifier {
01: guid (string) = "\xf8\xb9n\xe4\xb4\xf6N\xef\xad)9W.\x92#Y",
02: secret (string) = "\xc0?\xc7\xd9\x930C\x9b\xb5\xf6K\x8em\xcb\xf8\xe4",
},
},
}
11:14:02.203 AM INFO MetadataOp.java:414
Returning 19 schemas
It seems the hbase table B is either enabled nor disabled,very strange.I googled around,Is this related to the hbase security issues or the impala version problem?
Did anybody encountered the same problems?How to solve this?Thanks in advance.
Enable HBase service from Impala Configuration.
You can do it from Cloudera manager, Imapala->configuration
search from "Hbase" and enable the service.
PFA.

Intermittent JDBC error two-task conversion routine

I am intermittently getting the following jdbc error with my oracle db (in remote linux machine)
java.sql.SQLException: ORA-03120: two-task conversion routine: integer overflow
The query that mostly breaks(not the only one!) looks like this, note that I am using Groovy Sql
String insertScript =
"""
INSERT INTO data_table (invoke_id , trans_type , trans_from ,
trans_to , serv_type ,msg_type , man_id , recip_id ,
trx_id , part_id , msg_orig_time , create_time ,
action_code , num_ranges , use_trx_id, due_date,owner,route,lsa,
cl_id,hl_id,requ_type,lnp_type,use_invoke_id)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
"""
def params = [invokeId,transType,transFrom,transTo,servType,msgType,man,recip,portId,owner,
now,now,"W",1,trxId,dueDate,owner," ", "LSA","CCH"," ","Basic","lspp",invokeId]
sql.execute insertScript, params
What I do not understand is why my queries work some time and not other times. Generally it will happen during the first run and only rarely appear in subsequent runs.
[UPDATE] I am getting this error sometimes on application startup also, when spring tries to load its context. I am using spring and dbcp for the connection pooling.
Caused by: org.apache.commons.dbcp.SQLNestedException: Error preloading the connection pool
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1238)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:113)
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79)
... 46 more
Caused by: java.sql.SQLException: ORA-03120: two-task conversion routine: integer overflow
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:70)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:131)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:204)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:455)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:406)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:399)
at oracle.jdbc.driver.T4CTTIoauthenticate.receiveOauth(T4CTTIoauthenticate.java:799)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:368)
at oracle.jdbc.driver.PhysicalConnection.<init>(PhysicalConnection.java:508)
at oracle.jdbc.driver.T4CConnection.<init>(T4CConnection.java:203)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:33)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:510)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:294)
at org.apache.commons.pool.impl.GenericObjectPool.addObject(GenericObjectPool.java:1059)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1235)
... 49 more

Resources