How do I INSERT OVERWRITE with a struct in HIVE? - insert

Question

I have a Hive table tweets stored as text that I am trying to write to another table tweetsORC that is ORC. Both have the same structure:
col_name data_type comment
racist boolean from deserializer
contributors string from deserializer
coordinates string from deserializer
created_at string from deserializer
entities struct<hashtags:array<string>,symbols:array<string>,urls:array<struct<display_url:string,expanded_url:string,indices:array<tinyint>,url:string>>,user_mentions:array<string>> from deserializer
favorite_count tinyint from deserializer
favorited boolean from deserializer
filter_level string from deserializer
geo string from deserializer
id bigint from deserializer
id_str string from deserializer
in_reply_to_screen_name string from deserializer
in_reply_to_status_id string from deserializer
in_reply_to_status_id_str string from deserializer
in_reply_to_user_id string from deserializer
in_reply_to_user_id_str string from deserializer
is_quote_status boolean from deserializer
lang string from deserializer
place string from deserializer
possibly_sensitive boolean from deserializer
retweet_count tinyint from deserializer
retweeted boolean from deserializer
source string from deserializer
text string from deserializer
timestamp_ms string from deserializer
truncated boolean from deserializer
user struct<contributors_enabled:boolean,created_at:string,default_profile:boolean,default_profile_image:boolean,description:string,favourites_count:tinyint,follow_request_sent:string,followers_count:tinyint,following:string,friends_count:tinyint,geo_enabled:boolean,id:bigint,id_str:string,is_translator:boolean,lang:string,listed_count:tinyint,location:string,name:string,notifications:string,profile_background_color:string,profile_background_image_url:string,profile_background_image_url_https:string,profile_background_tile:boolean,profile_image_url:string,profile_image_url_https:string,profile_link_color:string,profile_sidebar_border_color:string,profile_sidebar_fill_color:string,profile_text_color:string,profile_use_background_image:boolean,protected:boolean,screen_name:string,statuses_count:smallint,time_zone:string,url:string,utc_offset:string,verified:boolean> from deserializer
When I try to insert from tweets to tweetsORC I get:
INSERT OVERWRITE TABLE tweetsORC SELECT * FROM tweets;
FAILED: NoMatchingMethodException No matching method for class org.apache.hadoop.hive.ql.udf.UDFToString with (struct<hashtags:array<string>,symbols:array<string>,urls:array<struct<display_url:string,expanded_url:string,indices:array<tinyint>,url:string>>,user_mentions:array<string>>). Possible choices: _FUNC_(bigint) _FUNC_(binary) _FUNC_(boolean) _FUNC_(date) _FUNC_(decimal(38,18)) _FUNC_(double) _FUNC_(float) _FUNC_(int) _FUNC_(smallint) _FUNC_(string) _FUNC_(timestamp) _FUNC_(tinyint) _FUNC_(void)
The only help I have found on this kind of problem says to make a UDF use primitive types, but I am not using a UDF! Any help is much appreciated!
FYI: Hive version:
Hive 1.2.1000.2.4.2.0-258
Subversion git://u12-slave-5708dfcd-10/grid/0/jenkins/workspace/HDP-build-ubuntu12/bigtop/output/hive/hive-1.2.1000.2.4.2.0 -r 240760457150036e13035cbb82bcda0c65362f3a
EDIT: Create tables and sample data:
create table tweets (
contributors string,
coordinates string,
created_at string,
entities struct <
hashtags: array <string>,
symbols: array <string>,
urls: array <struct <
display_url: string,
expanded_url: string,
indices: array <tinyint>,
url: string>>,
user_mentions: array <string>>,
favorite_count tinyint,
favorited boolean,
filter_level string,
geo string,
id bigint,
id_str string,
in_reply_to_screen_name string,
in_reply_to_status_id string,
in_reply_to_status_id_str string,
in_reply_to_user_id string,
in_reply_to_user_id_str string,
is_quote_status boolean,
lang string,
place string,
possibly_sensitive boolean,
retweet_count tinyint,
retweeted boolean,
source string,
text string,
timestamp_ms string,
truncated boolean,
`user` struct <
contributors_enabled: boolean,
created_at: string,
default_profile: boolean,
default_profile_image: boolean,
description: string,
favourites_count: tinyint,
follow_request_sent: string,
followers_count: tinyint,
`following`: string,
friends_count: tinyint,
geo_enabled: boolean,
id: bigint,
id_str: string,
is_translator: boolean,
lang: string,
listed_count: tinyint,
location: string,
name: string,
notifications: string,
profile_background_color: string,
profile_background_image_url: string,
profile_background_image_url_https: string,
profile_background_tile: boolean,
profile_image_url: string,
profile_image_url_https: string,
profile_link_color: string,
profile_sidebar_border_color: string,
profile_sidebar_fill_color: string,
profile_text_color: string,
profile_use_background_image: boolean,
protected: boolean,
screen_name: string,
statuses_count: smallint,
time_zone: string,
url: string,
utc_offset: string,
verified: boolean>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED AS TEXTFILE;
LOAD DATA LOCAL INPATH '/home/ed/Downloads/hive-json-master/1abbo.txt' OVERWRITE INTO TABLE tweets;
create table tweetsORC (
racist boolean,
contributors string,
coordinates string,
created_at string,
entities struct <
hashtags: array <string>,
symbols: array <string>,
urls: array <struct <
display_url: string,
expanded_url: string,
indices: array <tinyint>,
url: string>>,
user_mentions: array <string>>,
favorite_count tinyint,
favorited boolean,
filter_level string,
geo string,
id bigint,
id_str string,
in_reply_to_screen_name string,
in_reply_to_status_id string,
in_reply_to_status_id_str string,
in_reply_to_user_id string,
in_reply_to_user_id_str string,
is_quote_status boolean,
lang string,
place string,
possibly_sensitive boolean,
retweet_count tinyint,
retweeted boolean,
source string,
text string,
timestamp_ms string,
truncated boolean,
`user` struct <
contributors_enabled: boolean,
created_at: string,
default_profile: boolean,
default_profile_image: boolean,
description: string,
favourites_count: tinyint,
follow_request_sent: string,
followers_count: tinyint,
`following`: string,
friends_count: tinyint,
geo_enabled: boolean,
id: bigint,
id_str: string,
is_translator: boolean,
lang: string,
listed_count: tinyint,
location: string,
name: string,
notifications: string,
profile_background_color: string,
profile_background_image_url: string,
profile_background_image_url_https: string,
profile_background_tile: boolean,
profile_image_url: string,
profile_image_url_https: string,
profile_link_color: string,
profile_sidebar_border_color: string,
profile_sidebar_fill_color: string,
profile_text_color: string,
profile_use_background_image: boolean,
protected: boolean,
screen_name: string,
statuses_count: smallint,
time_zone: string,
url: string,
utc_offset: string,
verified: boolean>
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
STORED AS ORC tblproperties ("orc.compress"="ZLIB");
data here.

Answer 1

Instead of using Select * I list the fields by name and the error goes.

Answer 2

Data type mismatch: The data type you want to insert is inconsistent with the field type in the corresponding data table. For example, if the field type declared when you create the table is string, but the field type you inserted is indeed the list type, this error will be thrown.