I am using hive 1.1 from cloudera CDH5.5
When I try to create a hive internal table as below I get the below error .
hive> create table acct_IK(acct_id int,acct_name String,trans_dt date)
> stored as parquet;
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.UnsupportedOperationException: Parquet does not support date. See HIVE-6384
Do I need to upgrade hive to next version?
How do I create a table with date datatype in that?
As was noted in HIVE-6384, Date is supported only from hive 1.2.0
Related
I created an Hive external table to query the existing data of es like below
CREATE EXTERNAL TABLE ods_es_data_inc
(`agent_id` STRING,
`dt_server_time` TIMESTAMP
) COMMENT 'bb_i_app'
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES(
'es.resource'='data*',
'es.nodes'='ip',
'es.port'='port',
'es.net.http.auth.user'='user',
'es.net.http.auth.pass'='pass'
)
when I query date field in Hive external table,I am getting below error
Error:Java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassCastException:org.apache.hadoop.hive.serde2 .io.Timestampwritable cannot be cast to org.apache.hadoop.hive.serde2.io.TimestampwritableV2 (state=,code=0)
My situation is very similar to this problem.But I have used the timestamp field when I create external table.
My component version:
Hive:3.1.0
ES-Hadoop:6.8.7
Elasticsearch:6.7.1
I switched Hive's execution engine from mr to spark.The error has not changed. After eliminating the component problem, I don't know whether it is the version mismatch or the table creation problem.
I have source data in orc format on HDFS.
I created external hive table on the top HDFS data with below command. I am using hive 1.2.1 version.
CREATE EXTERNAL TABLE IF NOT EXISTS test( table_columns ... ) ROW
FORMAT FIELDS TERMINATED BY '\u0001' STORED AS orc LOCATION 'path'
TBL PROPERTIES("orc.compress"="SNAPPY");
But while selecting data from table I am getting this exception.
"protobuf.InvalidProtocolBufferException: Protocol message was too large"
Please help me to resolve this issue.
Thanks.
I have a table in hive which is created using ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'.
Trying to update a record but I receive the following error message:
FAILED: SemanticException [Error 10294]: Attempt to do update or delete using transaction manager that does not support these operations. (state=42000,code=10294)
It seems that AvroSerDe does not support ACID transaction but I can't find any info about that.
Hive transactions doesn't support Avro file formats as of latest Hive release. And it's currently integrated to work with only ORC file formats.
According to Hive documentation "Only ORC file format is supported in this first release. The feature has been built such that transactions can be used by any storage format that can determine how updates or deletes apply to base records (basically, that has an explicit or implicit row id), but so far the integration work has only been done for ORC."
You can find more information about Hive transactions here
Hive date format not supporting in impala.
I created partition on date column in hive table but when i can access the same table from hive_metadata in impala its showing
CAUSED BY: TableLoadingException: Failed to load metadata for table
'employee_part' because of unsupported partition-column type 'DATE' in
partition column 'hiredate'.
Please let me know which date format does hive and impala commonly support.
I used date format in hive as yyyy-mm-dd
Impala doesnt support the hive date format.
You have to use a timestamp (which means that you will always carry time but it will be 00:00:00.0000). Then depending on the tool you use after, you have to make a convertion again unfortunately.
We are very new to Hadoop and Hive. We created normal Hive table and loaded data as well. But We are facing issue when we are creating table in Hive with JSON format. I have added serde jar also. We get the following error:
create table airline_tables(Airline string,Airlineid string,Sourceairport string,Sourceairportid string,Destinationairport string,`Destinationairportid string,Codeshare string,Stop string,Equipment String)` ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde'`location '/home/hduser/part-000';`
FAILED: Error in metadata: java.lang.NullPointerException
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Location is HDFS.
I am using Hive-0.9.0 and hadoop 1.0.1.
As i can see. You are using the native table of hive. So in that case you need to load the data in the table. If you dont want to load the data then you just put the path of that particular location in the table creation script. So, i think you missed the keyword "EXTERNAL" again create the table like this. create external table airline_tables(blah blah.....)