Trying to add a column to an external table in HIVE but get the error below. This table currently has a thousand partitions registered and I want' to avoid re-creating the table and then running MSCK REPAIR which would take a very long time to complete. Also, the table uses OpenCSVSerde format. How can I add a column
hive> ALTER TABLE schema.Table123 ADD COLUMNS (Column1000 STRING);
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Unable to alter table. java.lang.IllegalArgumentException: Error: type expected at the position 0 of '<derived from deserializer>' but '<' is found.
Related
Sqlldr throwing error while loading just one specific file which has truncate into table specified in its ctl file.
Is a append into table statement also required in the ctl file after the truncate statement to load data into the database table?
Rest other ctl files having just the append into table statement, are getting loaded fine into db tables.
Just this one throwing error. The associated table doesn't have any referential integrity constraint as mentioned in the oracle docs that in which case sqlldr throws error with truncate into table clause of ctl file.
im trying to create a simple external table in hive cli and I got an error while creating a partition. I have searched on google, but I was not able to get a proper result. Can you please help
hive (sampledb)> create external table externalhive(id int,name varchar(100),age tinyint,city varchar(100),state varchar(100)) partitioned by (year string)
> row format delimited fields terminated by '/t' stored as textfile location '/user/ah12x/external';
OK
Time taken: 0.169 seconds
hive (sampledb)> show tables;
OK
externalhive
hive (sampledb)> alter table externalhive add partition (year ='2014')
> location ('/2012');
FAILED: ParseException line 2:9 extraneous input '(' expecting StringLiteral near '<EOF>'
There is no need for braces after location, https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterTable/PartitionLocation
I created a hive table with a storage handler and now I want to add a column to that table but it gives me below error:
[Code: 10134, SQL State: 42000] Error while compiling statement: FAILED:
SemanticException [Error 10134]: ALTER TABLE can only be used for [ADDPROPS,
DROPPROPS] to a non-native table
As per the hive documentation any hive table you create with storage handler is non native table.
Here's a link https://cwiki.apache.org/confluence/display/Hive/StorageHandlers
There is a JIRA case for enhancement is open with Apache for the same.
https://issues.apache.org/jira/browse/HIVE-1240
For ex, I am using Druid Storage Handler in my case.
I created a hive table using:
CREATE TABLE druid_table_1
(`__time` TIMESTAMP, `dimension1` STRING, `metric1` int)
STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler';
and then I am trying to add a column:
ALTER TABLE druid_table_1 ADD COLUMNS (`dimension2` STRING);
With above approach I am getting an error.
Is there any other way to add a column to non native tables in hive without recreating it?
Patch is available in HDP 2.5+ from Hortonworks. Support for ADD columns has been added in ALTER statement.
Column can be added into druid table using ALTER table DDL in hive.
ALTER TABLE ADD COLUMNS (col_name data_type)
There is no need to specify partition spec as these are druid backed hive tables and partition/storage is maintained by druid.
I'm currently trying to get the error messages we get from our Oracle external table loading process to match the level of detail we get when loading via sqlldr.
Currently if I load a file in sqlldr and a record fails I get this error message which is pretty useful - I get the record number, the actual column name that failed and the record in a bad file.
Record 4: Rejected - Error on table ERROR_TEST, column COL1.
ORA-01722: invalid number
I've got an external table along with an INSERT statement to a target table that logs errors to a DBMS_ERRLOG table but this is the equivalent error message from that process.
ORA-01722: invalid number
Whilst this process has the benefit of recording the actual record in a table so you can see column name mappings it doesn't actually list which column has the issue. This is a big issue when looking at tables with many columns...
It looks as though I can have the REJECT LIMIT on the external table itself which will give the same above error but that will then lose the ERR table logging. I'm guessing it's a case of one or the other?
I'm trying to insert data into my partition table using insert statement but its getting error saying unable to Alter partition
The query that I used :
hive> INSERT INTO TABLE sms_partition table PARTITION(userc,coding) SELECT headerguid,messageid,userid,tonumber,fromnumber,smssplits,udh,operator,circle,vf,msgp,smstext,userc,coding from chotadata16june;
I got the error:
ERROR:Failed with exception org.apache.hadoop.hive.ql.metadata.hiveexception: unable to alter partition .For direct metastore db connections ,we dont support retries to the client level.