Facing a weird issue. Alter table command to drop partition works well for > or < or >= or <= signs but not for = check.
Working command:
ALTER TABLE XYZ DROP PARTITION(bizdate>'20231230');
Command that's not working and throwing an error stating that partition does not exist:
ALTER TABLE XYZ DROP PARTITION(bizdate='20231230');
When I do show partitions, I can see '20231230' partition.
Note: bizdate is a varchar(10)
Check the list of partition in the table:
SHOW PARTITIONS <table>;
Perhaps it tries dropping the partition. It seems that the data was removed at some point on HDFS but the hive tables metadata still thinks those partitions exist
ALTER TABLE *tableName* drop if exists PARTITION(bizdate="20231230");
& Re-pair the broken table
msck repair table *table_name*;
Related
I have 2 schemas TBCAM and AR_TBCAM. There is a table called BKP_COST_EVENT in TBCAM which I have partitioned and I have moved the partition data into another simple table in AR_TBCAM schema called BKP_COST_EVENT_P2016. Now the data has moved to AR_TBCAM schema via this query
ALTER TABLE BKP_COST_EVENT EXCHANGE PARTITION P2016 WITH TABLE AR_TBCAM.BKP_COST_EVENT_P2016 INCLUDING INDEXES WITHOUT VALIDATION;
Now I want to bring the partition data back into the original table BKP_COST_EVENT.
But when I run this query standing on AR_TBCAM
ALTER TABLE BKP_COST_EVENT_P2016 EXCHANGE PARTITION P2016 WITH TABLE TBCAM.BKP_COST_EVENT INCLUDING INDEXES WITHOUT VALIDATION;
It is giving this error:
Error starting at line : 1 in command -
ALTER TABLE BKP_COST_EVENT_P2016 EXCHANGE PARTITION P2016 WITH TABLE TBCAM.BKP_COST_EVENT INCLUDING INDEXES WITHOUT VALIDATION
Error report -
ORA-14501: object is not partitioned
14501. 00000 - "object is not partitioned"
*Cause: Table or index is not partitioned. Invalid syntax.
*Action: Retry the command with correct syntax.
Can anyone suggest what am I doing wrong? Or how to bring/restore the data back to my TBCAM schema table BKP_COST_EVENT.
I have not dropped the partition p2016 in the original BKP_COST_EVENT
In exchange partition syntax first table should be the one which is partitioned, second should be unpartitioned one.
So, your first command was correct but 2nd command is wrong.
If you are bringing back data to same table's partition run the same command again.
ALTER TABLE BKP_COST_EVENT EXCHANGE PARTITION P2016 WITH TABLE AR_TBCAM.BKP_COST_EVENT_P2016 INCLUDING INDEXES WITHOUT VALIDATION;
Also, if there are no indexes to be moved, its better not to use including indexes clause.
I am trying to drop a table and recreate it in Hive. After dropping the table if I run select query on the table it shows old rows which were in the table before dropping. How is this possible when the table is already dropped ? Why does it retain rows even after table is dropped and recreated ?
hive> select * from abc;
A 30
B 40
hive> drop table abc;
hive> create external table abc ( name string, qty int);
hive> select * from abc;
A 30
B 40
The problem is you are dropping the external table so whenever we dropped this table at that time source file of that table is still exist on that path so whenever we are going to create a new external table with same table name then data can directly extract from source path, for resolving this issue First get path of the table using following command :
hive> describe formatted database_name.table_name;
Then copy entire location which appear in description, for example :
/user/hive/warehouse/database_name.db/table_name
After this use following command to truncate all the data from given table :
hive> dfs -rmr /user/hive/warehouse/database_name.db/table_name;
OR
hive> dfs -rm -r /user/hive/warehouse/database_name.db/table_name;
Then you can wipe it completely using DROP TABLE command.
I don't know Hive, but if it is anything like Oracle (which I, kind of, know), then external table points to a file stored on your disk.
Therefore, once you dropped it you couldn't use it (of course). But then you created another EXTERNAL TABLE (see? 5th line in your example) and of course that you were able to select from it once again.
Because, you didn't delete the FILE that is a data source for that external table.
When I am trying to rename all partition columns in an existing table for date range of one year which are partitioned - this is what I am getting.
hive> ALTER TABLE test.usage PARTITION ('date') RENAME TO PARTITION (partition_date);
FAILED: ValidationFailureSemanticException Partition spec {partition_date=null} contains non-partition columns.
I got that syntax from here: 1
Given that I am not 100% sure if what you want is to rename the value of the partition or actually change the column which the table is partitioned.
Let's suppose you want to rename the partition's value.
There is an issue in Hive 0.13 and in Hive 0.14 is working fine. Anyways, this should work:
set fs.hdfs.impl.disable.cache=false;
set fs.file.impl.disable.cache=false;
Now run the query by setting this property.
> hive> set fs.hdfs.impl.disable.cache=false;
> hive> set fs.file.impl.disable.cache=false;
> hive> ALTER TABLE test.usage PARTITION (date='oldValue') RENAME TO PARTITION (date='newValue');
Let's suppose you want to change the partition column
In this case what you will need to do is to recreate the table changing the column which the table will be partitioned.
Note: Remember that if you already had data inside the partitions, you will need to reinsert the data.
Please see also this answer
i have created a non partitioned table and load data into the table,now i want to add a PARTITION on the basis of department into that table,can I do this?
If I do:
ALTER TABLE Student ADD PARTITION (dept='CSE') location '/test';
It gives me error:
FAILED: SemanticException table is not partitioned but partition spec exists: {dept=CSE}
please help. Thanks
First create a table in such a way so that you don't have partition column in the table.
create external table Student(col1 string, col2 string) partitioned by (dept string) location 'ANY_RANDOM_LOCATION';
Once you are done with the creation of the table then alter the table to add the
partition department wise like this :
alter table Student add partition(dept ='cse') location '/test';
I hope this will help.
You can't alter table partition if you didn't define partition while creation of table.
If, when altering a un-partitioned table to add partition, you get this error: "Semantic Exception table is not partitioned but partition spec exists: {dept=CSE}," it means you are trying to include the partitioned in the table itself.
You are not getting syntax error because the syntax of the command is correct and used to alter the partition column.
Learn more about Hive Tables:
https://www.dezyre.com//hadoop-tutorial/apache-hive-tutorial-tables
You can also check the possible alternation in table:
https://sites.google.com/site/hadoopandhive/home/how-to-create-table-partition-in-hive
Hope this helps.
I have dropped the all the partitions in the hive table by using the alter command
alter table emp drop partition (hiredate>'0');
After droping partitions still I can see the partitions metadata.How to delete this partition metadata? Can I use the same table for new partitions?
Partitioning is defined when the table is created. By running ALTER TABLE ... DROP PARTITION ... you are only deleting the data and metadata for the matching partitions, not the partitioning of the table itself.
Your best bet at this point will be to recreate the table without the partitioning. If there is some data you are trying to save, rename the current table, create the new table (without the partitioning), then run an INSERT from the old table to the new table.