Can any one suggest me why the following error is occurring and how to resolve it??
Not only the below command, running any command related to Hive is returning the same..
hive> show databases;
FAILED: Error in metadata: MetaException(message:Got exception: org.apache.thrif
t.transport.TTransportException java.net.SocketException: Connection reset by pe
er: socket write error)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTa
sk
Check out your hive-site.xml. It is possible your javax.jdo.option.ConnectionURL, the URL for the Hive metastore, isn't right.
Related
I am ingesting data from one cluster to another and I am using beeline to run MSCK file from 1st cluster.
MSCK was working until feb 27 and after that I have started getting below error messages.
NFO : Executing command: MSCK REPAIR TABLE cubcus_display
INFO : Starting task [Stage-0:DDL] in serial mode
ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask (state=\
08S01,code=1)
I have tried : https://www.ibm.com/support/pages/running-command-hive-table-results-running-msck-error-error-while-processing-statement-failed-error
But i did not work. How can we solve this?
Beeline version: 1.2.1000.2.6.5.0-292
Hive Version: 3.1.0.3.1.5.0-152
I am using Kafka source JDBC connector to pull DB events and I am running the kafka connect in standalone mode. When I run this file, I get the shown error. Please help me.
Code:
name=sailpointdb01107
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
connection.password = xxxxx
connection.url = jdbc:oracle:thin:#xxxxx:1521/xxxxx
connection.user =xxxxx
query= SELECT * FROM (SELECT NAME, TO_TIMESTAMP('19700101', 'YYYYMMDD')+ NUMTODSINTERVAL(COMPLETED/1000,'SECOND') AS TASKFAILEDON FROM task WHERE COMPLETION_STATUS='Error')
mode= timestamp
timestamp.column.name=TASKFAILEDON
topic.prefix=testing
validate.non.null=false
transforms=TimestampConverter
transforms.TimestampConverter.type=org.apache.kafka.connect.transforms.TimestampConverter$Value
transforms.TimestampConverter.format=yyyy-MM-dd
transforms.TimestampConverter.target.type=Timestamp
transforms.TimestampConverter.target.field=TASKFAILEDON
Error:
Error[2019-10-01 15:17:45,058] ERROR WorkerSourceTask{id=sailpointdb01107-0} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask)
org.apache.kafka.connect.errors.ConnectException: Tolerance exceeded in error handler
Caused by: org.apache.kafka.connect.errors.ConnectException: Schema Schema{STRUCT} does not correspond to a known timestamp type format
at org.apache.kafka.connect.transforms.TimestampConverter.timestampTypeFromSchema(TimestampConverter.java:406)
at org.apache.kafka.connect.transforms.TimestampConverter.applyWithSchema(TimestampConverter.java:334)
at org.apache.kafka.connect.transforms.TimestampConverter.apply(TimestampConverter.java:275)
at org.apache.kafka.connect.runtime.TransformationChain.lambda$apply$0(TransformationChain.java:50)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndRetry(RetryWithToleranceOperator.java:128)
at org.apache.kafka.connect.runtime.errors.RetryWithToleranceOperator.execAndHandleError(RetryWithToleranceOperator.java:162)
... 11 more
[2019-10-01 15:17:45,059] ERROR WorkerSourceTask{id=sailpointdb01107-0} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask)
This line configured at connector could avoid that issue.
time.precision.mode: "connect"
I get an error when trying to drop a table in hive:
> drop table my_table;
Error:
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
org.apache.thrift.transport.TTransportException:
java.net.SocketTimeoutException: Read timed out
I also don't have the related data on the HDFS, what could be the reason for that?
You can try to increase the socket timeout:
set hive.metastore.client.socket.timeout=5000
I am using Hadoop-2.6.0 secured with kerberos. Installed hive server2 1.1.0 version with derby database as connectionurl, enabled security and enabled Authorization. When enabling transaction configuration, I am getting the below exception and cannot execute any queries;
Exception
Error: Error while compiling statement: FAILED: LockException [Error 10280]: Error communicating with the metastore (state=42000,code=10280)
Logs
[Error 10280]: Error communicating with the metastore
org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastore
at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getValidTxns(DbTxnManager.java:300)
at org.apache.hadoop.hive.ql.Driver.recordValidTxns(Driver.java:927)
Caused by: MetaException(message:Unable to select from transaction database, java.sql.SQLSyntaxErrorException: Table/View 'TXNS' does not exist.
So i have created a below property in hive-site.xml file as mentioned in a blog here
Configuration
<property>
<name>hive.in.test</name>
<value>true</value>
</property>
If i set the above property then getting the below exception where i am struck and unable to solve it. I cannot run any query even use mydb;
Exception
Error: Error while compiling statement: FAILED: NullPointerException null (state=42000,code=40000)
Logs
Error executing statement:
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: NullPointerException null
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:315)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:103)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkQFileTestHack(TxnHandler.java:1146)
at org.apache.hadoop.hive.metastore.txn.TxnHandler.(TxnHandler.java:117)
I need a solution to work ACID transactions in Hive Server2. I found two related questions but not solved my issue.
hive 0.14 update and delete queries configuration error
Hive Transactions are crashing
Upgrade your hive mysql metastore db with hive-txn-schema-0.14.0.mysql.sql as follows..
mysql> SOURCE /usr/local/hadoop/hive/scripts/metastore/upgrade/mysql/hive-txn-schema-0.14.0.mysql.sql;
The title is a bit weird as I'm having difficulties narrowing down the problem. I used my solution on Hadoop 2.0.0-cdh4.4.0 and hive 0.10 without issues.
I can't create a table using this SerDe: https://github.com/rcongiu/Hive-JSON-Serde
first try:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. org.apache.hadoop.hive.serde2.objectinspector.primitive.AbstractPrimitiveJavaObjectInspector.<init>(Lorg/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils$PrimitiveTypeEntry;)V
second try:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Could not initialize class org.openx.data.jsonserde.objectinspector.JsonObjectInspectorFactory
I can create a table with this SerDe: https://github.com/cloudera/cdh-twitter-example
I create an external table with tweets from flume. I can't do "SELECT * FROM tweets;"
FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: Failed with exception java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDejava.lang.RuntimeException: java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDe
I can do SELECT id, text FROM tweets;
I can do a SELECT COUNT(*) FROM tweets;
I can't self join this table:
Execution log at: /tmp/jochen.debie/jochen.debie_20140311121313_164611a9-b0d8-4e53-9bda-f9f7ac342aaf.log
2014-03-11 12:13:30 Starting to launch local task to process map join; maximum memory = 257294336
Execution failed with exit status: 2
Obtaining error information
Task failed!
Task ID:
Stage-5
mentioned execution log:
2014-03-11 12:13:30,331 ERROR mr.MapredLocalTask (MapredLocalTask.java:executeFromChildJVM(324)) - Hive Runtime Error: Map local work failed
org.apache.hadoop.hive.ql.metadata.HiveException: Failed with exception java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDejava.lang.RuntimeException: java.lang.ClassNotFoundException: com.cloudera.hive.serde.JSONSerDe
Does anyone know how to fix this or at least show me where the problem is?
EDIT: Can it be a problem that I built the serde on a Hadoop 2.0.0-cdh4.4.0 and hive 0.10?
From what I've seen, Hive-.11+ has a bug in join with custom SerDe.
https://github.com/Esri/gis-tools-for-hadoop/issues/9
You might try the workaround of copying the JAR file containing the SerDe class, to $HIVE_HOME/lib .
(I see in your question you got ClassNotFoundException both in join and in other cases; so far the times I have encountered such were all with join.)
[Edit] Another workaround is to use HADOOP_CLASSPATH:
env HADOOP_CLASSPATH=some.jar:other.jar hive ...
[Edit] The work around applies to Hive versions 0.11 and 0.12; then 0.13 and above contain the fix for HIVE-6670.