Querying select * from table_name limit 10; in Hive returns correct data. But querying same table through presto connection returns below error :
Presto query has failed. cannot find field from
[0:error_error_error_error_error_error_error,
1:cannot_determine_schema, 2:check, 3:schema, 4:url, 5:and, 6:literal]
Querying select count(*) from table_name; in Hive returns correct data. But querying the same table through presto connection returns below error :
Presto query has failed. HIVE_CURSOR_ERROR
I have gone through this link1 and link2 already but it did not help me.
I have solved this issue by correcting avro schema.
Related
I have configured Hive Version 2.3.0 in azure sql database on DBR 10.X. I can see entries all delta tables in dbo.TBLS however [dbo].[COLUMNS_V2] shows only one entry per table which is like below..
CD_ID
COMMENT
COLUMN_NAME
TYPE_NAME
INTEGER_IDX
346
from deserializer
col
array
0
what I am missing here? why don't I see all columns for Table ID-346?
I came accross this same problem and found the cause for my case:
When I wrote data to the hive table in my databricks notebook I had:
myDf.write.mode('Delta').saveAsTable('myHiveDb.myTable')
This caused the columns to not show up in COLUMNS_V2. Instead you need to change the mode to 'Hive' instead of 'Delta':
myDf.write.mode('Hive').saveAsTable('myHiveDb.myTable')
On HDP cluster, I am trying create and integrate Hive tables with existing Hbase tables. It creates the hive table. But when I am try to query the hive table it throws the following exception especially when the number of columns exceeds 200.
I checked the # of columns in both hbase & hive is same. Not getting proper solution to debug it.
hive> select * from hbase_hive.lead;
FAILED: RuntimeException MetaException(message:org.apache.hadoop.hive.serde2.SerDeException
org.apache.hadoop.hive.hbase.HBaseSerDe:
columns has 273 elements while hbase.columns.mapping has 204 elements (counting the key if implicit))
Is there any column limitation in this case?
Please suggest me solution on this
This has fixed the issue.
https://hortonworks.my.salesforce.com/kA2E0000000LZQ5?srPos=0&srKp=ka2⟨=en_US
ROOT CAUSE:
A character limitation for PARAM_VALUE field in SERDE_PARAMS table in hive metastore for 4000 character is the root cause of this issue. This limitation prevents Hive from creating a table with high column numbers, eventually causing desc or select * from to fail with error above.
WORKAROUND: This issue can be worked around by doing the following in hive metastore
-- log into Hive Metastore DB -- >alter table SERDE_PARAMS MODIFY PARAM_VALUE VARCHAR(400000000);
I came across a bizarre Impala behaviour. I've create a table in HUE from a .csv file I've copied into the Hadoop cluster. I can correctly navigate the table in HUE via the Metastore Manager but I can't run the following query in Impala, as it throws an IllegalStateException: null exception:
select *
from my_db.my_table
limit 100;
The strange thing is that the following command retrieve the correct number of rows:
select
count(*)
from my_db.my_table;
The error is caused by invalid types. Not all hive data types are supported in impala. Impala has a timestamp and no date type. When your table has date type it will show as invalid_type in impala when described and impala cannot select this data type. For solution try changing the column to timestamp
Describe <table name>;
| invalid_type | |
| invalid_type | |
I'm getting the exact same issue. I changed the query to select each column from the table individually (i.e. select col1, col2, col3...etc.) and found that Impala didn't like a date datatype column. Changing it to timestamp fixed the issue and I can now do a select * from the table.
Assume that a table is created like following in Hive:
create table test1
(
field_name int
)comment='TestTableComment';
Now I'd like to get the comment for the table (TestTableComment) through JDBC in Java, how can I get it?
Have you tried DatabaseMetaData.getTables()?
Reference
Here are the environment details:
Hadoop: 2.4.0
Hive: 0.11.0
HBase: 0.94.18
I created a HBase table and imported 10,000 rows:
hbase(main):008:0> create 'genotype_tbl', 'cf'
Load data to the table.
hbase(main):008:0> count 'hbase_tbl'
10000 row(s) in 176.9310 seconds
I created a Hive table as described in this article (using instructions on this page: https://cwiki.apache.org/confluence/display/Hive/HBaseIntegration#HBaseIntegration-HiveHBaseIntegration)
CREATE EXTERNAL TABLE hive_tbl(key int, value string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:info")
TBLPROPERTIES("hbase.table.name" = "hbase_tbl");
However, when I do a count(*) on hive_tbl, it returns 0. There are no errors of any sort. Any help is appreciated.
This issue is resolved. The problem is with the hbase ImportTsv command. columns list was incorrect. Once, that was resolved, I could execute queries from Hive.