I am debugging a Big Data code in Production environment of my company. Hive return the following error:
Exception: org.apache.hadoop.hive.ql.lockmgr.LockException: No record of lock could be found, may have timed out
Killing DAG...
Execution has failed.
Exception in thread "main" java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask.
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:282)
at org.apache.hive.jdbc.HiveStatement.executeUpdate(HiveStatement.java:392)
at HiveExec.main(HiveExec.java:159)
After investigation, I have found that this error could be caused by BoneCP in connectionPoolingType property, but the cluster support team told me that they fixed this bug by upgrading BoneCP.
My question is: can we INSERT INTO an external table in Hive, because I have doubt about the insertion script ?
Yes, you can insert into external table.
Related
The following query is giving me the error:
Execute error: Error while processing statement: FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
Does anyone know why or how to resolve this issue?
proc sql;
connect to hadoop(server='xxx' port=10000 schema=xxx SUBPROTOCOL=hive2 sql_functions=all);
execute(
create table a as
select
*,
lag(claim_flg,1) over (order by ptnt_id,month) as lag1
from b
) by hadoop;
disconnect from hadoop;
quit;
It appears to be a limitation issue in HIVE database:
Hive Limit of 127 Expressions per Table
Due to a limitation in the Hive database, tables can contain a maximum of 127 expressions. When the 128th expression is read, the directive fails and the SAS log receives a message similar to the following:
ERROR: java.sql.SQLException: Error while processing statement: FAILED:
Execution Error, return
code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
ERROR: Unable to execute Hadoop query.
ERROR: Execute error.
SQL_IP_TRACE: None of the SQL was directly passed to the DBMS.
The Hive limitation applies anytime a table is read as part of a directive. For SAS Data Loader, the error can occur in aggregations, profiles, when viewing results, and when viewing sample data.
Source: http://support.sas.com/documentation/cdl/en/dmddug/67908/HTML/default/viewer.htm#p1fl149uastoudn1v7r2u5ff8aft.htm
I have a hiveserver2 running and wrote a java program to query from hive.
I tried this query
SELECT * FROM table1
where, 'table1' is the table name in hive, and its works fine and gave me the results.
But when i tried to run
SELECT COUNT(*) FROM table1
it threw an exception
Exception in thread "main" java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
I check the logs and this was recorded
Job init failed : org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.io.FileNotFoundException: File does not exist: hdfs://vseccoetv04:9000/tmp/hadoop-yarn/staging/anonymous/.staging/job_1453359797695_0017/job.splitmetainfo
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.createSplits(JobImpl.java:1568)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1432)
at org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl$InitTransition.transition(JobImpl.java:1390)
....
I checked in a number of places, and other people too got 'FileNotFoundException' but not dude to this reason.
Is there any way to solve this problem?
Okay,
I figured out the problem myself :)
I had added some properties in the hive-site.xml file earlier to check support for transactions. I think i might have added some wrong values there. Now, I have removed the properties which i added, and have restarted hive service. Everything works fine :D
I have stored procedure, which has parallel queries and hints. Frequently we are getting error details as below.
An error occurred executing the stored procedure.
TM_6159 [4294965496] [
ORA-12801: error signaled in parallel query server P011, instance dbssd0011:proddb1 (1)
ORA-08103: object no longer exists
ORA-06512: at "PROD1.PKG_PTS", line 5068
ORA-06512: at line 2
Database driver error...
Function Name : ExecuteSP
Oracle Fatal Error
Database driver error...
Function Name : ExecuteSP
Oracle Fatal Error]**
After rerun it got succeeded, we are getting this error frequently every alternate day. We are in Oracle 10g.
Please share your inputs on the same.
Regards,
Shamsheer
I am trying to execute the query below:
INSERT OVERWRITE TABLE nasdaq_daily
PARTITION(stock_char_group)
select exchage, stock_symbol, date, stock_price_open,
stock_price_high, stock_price_low, stock_price_close,
stock_volue, stock_price_adj_close,
SUBSTRING(stock_symbol,1,1) as stock_char_group
FROM nasdaq_daily_stg;
I have already set hive.exec.dynamic.partition=true and hive.exec.dynamic.partiion.mode=nonstrict;.
Table nasdaq_daily_stg table contains proper information in the form of a number of CSV files. When I execute this query, I get this error message:
Caused by: java.lang.SecurityException: sealing violation: package org.apache.derby.impl.jdbc.authentication is sealed.
FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.MapRedTask
The mapreduce job didnt start at all. So there are no logs present in the jobtracker web-UI for this error. I am using derby to store meta-store information.
Can someone help me fix this?
Please try this. This may be the issue. You may be having Derby classes twice on your classpath.
"SecurityException: sealing violation" when starting Derby connection
DataStage -- 8.1
Database -- Oracle 10g
OS -- Unix
I have a DataStage job (FCT) which is doing a lookup based on two keys columns to the DIM table. This job ABORTED with the following error message.
"main_program: (aptoci.C:483). Message: ORA-04020: deadlock detected while trying to lock object DIM_TABLE_NAME "
Partition type for lookup stage -- Auto
Config file Nodes -- 2
Note:- Most of the times this job runs without any issues but sometimes fail with the above error message.
Don't understand what can cause deadlock here and how to resolve this issue.
Thanks in advance.