I'm getting the following error while executing queries against a database in impala. With other databases its working fine.
Error trace is as follows.
[Simba][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: select * from test_table limit 1, SQL state: {1}, Query: {2}.[]
java.sql.SQLException: [Simba][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: [Simba][JSQLEngine](12010) The table "test_table" could not be found., SQL state: HY000, Query: select count(*) from test_table.
at com.cloudera.impala.hivecommon.dataengine.HiveJDBCDataEngine.prepare(Unknown Source)
at com.cloudera.impala.jdbc.common.SStatement.executeNoParams(Unknown Source)
at com.cloudera.impala.jdbc.common.SStatement.executeQuery(Unknown Source)
Caused by: com.cloudera.impala.support.exceptions.GeneralException: [Simba][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: [Simba][JSQLEngine](12010) The table "test_table" could not be found., SQL state: HY000, Query: select count(*) from test_table.
... 3 more
if I execute show tables its listing the table name.
If I execute it from hue its not displaying anything in the result.
I tried by invalidating the metadata.
I tried by changing to latest driver jdbc41 same problem.
Where might be the problem?
In my case, this error was caused by not having a /user/scott directory on hdfs with write permissions for the Hiveserver (running as cloudera-scm user)(my jdbc connection uses scott as the user id). Once I created the dir and chmod'ed it, I could run all queries. Earlier only select * worked but select count(*) did not.
The problem was in the .avro file format. My teamlead has fixed it, Not sure what he has done he just said it was the problem of file format.
Related
I've removed my HDFS path /user/abc, and some Hive tables were stored in /user/abc/data/abc.db , with a rm -R command.
Despite having my regular tables correctly deleted with Hive SQL, my external tables didn't drop, with the following error:
[Code: 1, SQL State: 08S01] Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Failed to load storage handler: Error in loading storage handler.org.apache.phoenix.hive.PhoenixStorageHandler)
How can I safely delete the tables?
I tried using:
delete from TBL_COL_PRIVS where TBL_ID=[myexternaltableID];
delete from TBL_PRIVS where TBL_ID=[myexternaltableID];
delete from TBLS where TBL_ID=[myexternaltableID];
But it didn't work with the following error message:
[Code: 10297, SQL State: 42000] Error while compiling statement: FAILED: SemanticException [Error 10297]: Attempt to do update or delete on table sys.TBLS that is not transactional
Thank you,
NB: I know a schema is supposed to be deleted more safely with HiveQL but on this particular case, this was not done this way.
Solution is to delete the tables from Hive Metastore (PostgreSQL) with
delete from "TABLE_PARAMS" where "TBL_ID"='[myexternaltableID]';
delete from "TBL_COL_PRIVS" where "TBL_ID"='[myexternaltableID]';
delete from "TBL_PRIVS" where "TBL_ID"='[myexternaltableID]';
delete from "TBLS" where "TBL_ID"='[myexternaltableID]';
NB: Order is important.
I seem to be getting the below exception when I try to create a table using Hive client.
create table if not exists test (id int, name string) comment 'test table';
11:15:32.016 [HiveServer2-Background-Pool: Thread-34] ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler - Retrying HMSHandler after 2000 ms (attempt 1 of 10) with error: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MTable#784fafc2" using statement "INSERT INTO TBLS (TBL_ID,CREATE_TIME,DB_ID,LAST_ACCESS_TIME,OWNER,RETENTION,SD_ID,TBL_NAME,TBL_TYPE,VIEW_EXPANDED_TEXT,VIEW_ORIGINAL_TEXT) VALUES (?,?,?,?,?,?,?,?,?,?,?)" failed : Column 'IS_REWRITE_ENABLED' cannot accept a NULL value.
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:720)
Caused by: ERROR 23502: Column 'IS_REWRITE_ENABLED' cannot accept a NULL value.
at org.apache.derby.client.am.ClientStatement.completeExecute(Unknown Source)
at org.apache.derby.client.net.NetStatementReply.parseEXCSQLSTTreply(Unknown Source)
I did search but couldn't find a satisfactory resolution.
Here is my set up:
Hive 2.1.0
OS: Windows
Hadoop: 2.9.2
Derby: 10.14.2.0
What am i missing?
Thanks.
Seems like a compatibility issue with derby. I moved back to a earlier version of derby 10.2.1.1 and the issue went away.
I am running a small cluster in Amazone EMR in order to play with Apache Hive 2.3.5. It is my understanding that Apache Hive can import data from a remote database and have the cluster to run queries. I was following an example that is provided in Apache Hive web documentation (https://cwiki.apache.org/confluence/display/Hive/JdbcStorageHandler) and created the following code:
CREATE EXTERNAL TABLE hive_table
(
col1 int,
col2 string,
col3 date
)
STORED BY 'org.apache.hive.storage.jdbc.JdbcStorageHandler'
TBLPROPERTIES (
'hive.sql.database.type'='POSTGRES',
'hive.sql.jdbc.driver'='org.postgresql.Driver',
'hive.sql.jdbc.url'='jdbc:postgresql://<url>/<dbname>',
'hive.sql.dbcp.username'='<username>',
'hive.sql.dbcp.password'='<password>',
'hive.sql.table'='<dbtable>',
'hive.sql.dbcp.maxActive'='1'
);
But I get the following error:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException java.lang.IllegalArgumentException: Property hive.sql.query is required.)
According to the documentation, I need to specify either “hive.sql.table” or “hive.sql.query” to tell how to get data from jdbc database. But if I replace hive.sql.table with hive.sql.query I get the following error:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException java.lang.IllegalArgumentException: No enum constant org.apache.hive.storage.jdbc.conf.DatabaseType.POSTGRES)
I tried looking in the web for a solution and it doesn't look like anyone experience the same issues that I am having. Do I need to modify a config file or am I missing something critical in my code?
I think you are using a version of the jar which doesn't support POSTGRES.
Download the latest jar from this link:
http://repo1.maven.org/maven2/org/apache/hive/hive-jdbc-handler/3.1.2/hive-jdbc-handler-3.1.2.jar
Put this downloaded jar into a hdfs location.
Run hive normally.
Run command: add jar ${HDFS_PATH_TO_DOWNLOADED_JAR}
Run your create table command
I am trying to run this function to get the Current Date in Hive but I am getting the following error:
Error while compiling statement: FAILED: SemanticException No valid privileges Required privileges for this query: Server=server1->Db=_dummy_database->Table=_dummy_table->action=select;
I have searched online and being suggested following functions to get Current Date in Hive but all are giving same error:
SELECT from_unixtime(unix_timestamp()); --/Selecting Current Time stamp/
SELECT CURRENT_DATE; --/Selecting Current Date/
SELECT CURRENT_TIMESTAMP; --/Selecting Current Time stamp/
But all are showing error if I run them as they are given.
Right Answers:
1. SELECT from_unixtime(unix_timestamp()); - Only works for Impala
SELECT from_unixtime(unix_timestamp()) from any_table_name; - Works in HIVE
NOTE: Must use FROM clause with any_table_name present in database for HIVE
select unix_timestamp(current_timestamp) from table_name;
I have a table sdh in Hive which has 100000 rows.
When I execute the command
select * from sdh on CLI, all rows are displayed
But the same command just hangs when I run it via HiveServer2 on beeline
All other tables which have 1000 odd rows work fine via CLI or beeline.
Anyone else face similar issue?
I got error from logs
org.apache.thrift.TApplicationException: Internal error processing FetchResults
at org.apache.thrift.TApplicationException.read(TApplicationException.java:108)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:71)
at org.apache.hive.service.cli.thrift.TCLIService$Client.recv_FetchResults(TCLIService.java:489)
at org.apache.hive.service.cli.thrift.TCLIService$Client.FetchResults(TCLIService.java:476)
at org.apache.hive.jdbc.HiveQueryResultSet.next(HiveQueryResultSet.java:285)
at org.apache.hive.beeline.BufferedRows.<init>(BufferedRows.java:42)
at org.apache.hive.beeline.BeeLine.print(BeeLine.java:1541)
at org.apache.hive.beeline.Commands.execute(Commands.java:741)
at org.apache.hive.beeline.Commands.sql(Commands.java:657)
at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:763)
at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:630)
at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:363)
at org.apache.hive.beeline.BeeLine.main(BeeLine.java:346)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
Error: Error retrieving next row (state=,code=0)
Use the option "--incremental=false" with beeline. Otherwise, it tries to read the entire results into memory at same time, and runs out of memory if the results are too large for available memory.
More discussion on making this option the default is here - https://issues.apache.org/jira/browse/HIVE-7224
Please use the option --incremental=true and place it before the -e option in beeline. For me it worked that way.