I am facing issue with exchange partitioning and sub partitioning in Oracle.
Please explain with example.
Please find following example
CREATE TABLE DEPARTMENT
( DEPT_ID NUMBER(30,0) NOT NULL ENABLE,
IS_ACTIVE VARCHAR2(1 BYTE) NOT NULL ,
BUSINESS_DATE DATE NOT NULL ENABLE,
COMPANY_CODE VARCHAR2(4) DEFAULT 'C1' NOT NULL ,
CONSTRAINT PK_DEPARTMENT PRIMARY KEY (BUSINESS_DATE, COMPANY_CODE, DEPT_ID) ENABLE
)
PARTITION BY RANGE
(
BUSINESS_DATE
)
INTERVAL (
NUMTODSINTERVAL(1,'DAY')
)
SUBPARTITION BY LIST(COMPANY_CODE)
SUBPARTITION TEMPLATE
(
SUBPARTITION CMN_01 VALUES ('C1'),
SUBPARTITION CMN_02 VALUES ('C2'),
SUBPARTITION CMN_03 VALUES ('C3')
)
(
PARTITION DEFAULT_PART VALUES LESS THAN ('01-JAN-2012')
)
;
CREATE INDEX IDX1_DEPARTMENT ON DEPARTMENT (DEPT_ID, IS_ACTIVE);
CREATE TABLE TEMP_DEPARTMENT AS SELECT * FROM DEPARTMENT WHERE 1=2;
CREATE TABLE EMPLOYEE
( ID NUMBER(30,0) NOT NULL ENABLE,
DEPT_ID NUMBER(30,0) NOT NULL ENABLE,
BUSINESS_DATE DATE NOT NULL ENABLE,
COMPANY_CODE VARCHAR2(4) DEFAULT 'C1' NOT NULL,
CONSTRAINT PK_EMPLOYEE PRIMARY KEY (BUSINESS_DATE,COMPANY_CODE, ID) ENABLE,
CONSTRAINT FK1_EMPLOYEE_DEPT FOREIGN KEY (BUSINESS_DATE, COMPANY_CODE, DEPT_ID)
REFERENCES DEPARTMENT (BUSINESS_DATE, COMPANY_CODE, DEPT_ID) ENABLE NOVALIDATE
)
PARTITION BY RANGE
(
BUSINESS_DATE
)
INTERVAL (
NUMTODSINTERVAL(1,'DAY')
)
SUBPARTITION BY LIST(COMPANY_CODE)
SUBPARTITION TEMPLATE
(
SUBPARTITION CMN_01 VALUES ('C1'),
SUBPARTITION CMN_02 VALUES ('C2'),
SUBPARTITION CMN_03 VALUES ('C3')
)
(
PARTITION DEFAULT_PART VALUES LESS THAN ('01-JAN-2012')
);
Insert few thousands records into DEPARTMENT and EMPLOYEE tables for few partitions and their all sub partitions
select * from DEPARTMENT where BUSINESS_DATE='19-JAN-15' and COMPANY_CODE='C1';
select* from EMPLOYEE;
select * from all_tab_partitions where TABLE_OWNER='TEST' and
table_name='DEPARTMENT';
select * from ALL_TAB_SUBPARTITIONS where TABLE_OWNER='TEST' and
table_name='DEPARTMENT';
select * from all_tab_partitions where TABLE_OWNER='TEST' and
table_name='EMPLOYEE';
select * from ALL_TAB_SUBPARTITIONS where TABLE_OWNER='TEST' and
table_name='EMPLOYEE';
select * from TEMP_DEPARTMENT;
Exchange and Truncate sub partitions
1)Exchange sub partitioningALTER TABLE DEPARTMENT EXCHANGE SUBPARTITION SYS_SUBP7356 WITH TABLE TEMP_DEPARTMENT WITHOUT VALIDATION UPDATE GLOBAL INDEXES;
2) Truncate temp exchange table truncate table TEMP_DEPARTMENT;
3) Truncate sub partitioning (sub partition name from ALL_TAB_SUBPARTITIONS table) with global index update `ALTER TABLE DEPARTMENT TRUNCATE SUBPARTITION SYS_SUBP7356 DROP STORAGE UPDATE GLOBAL INDEXES;
Here the above 1 to 3 steps are executed for three sub-partitions
4) Get this partition name from all_tab_partitions table with global index update ALTER TABLE DEPARTMENT DROP PARTITION SYS_P7359 UPDATE GLOBAL INDEXES;
5) Truncate sub partitioning (sub partition name from ALL_TAB_SUBPARTITIONS table) with global index update ALTER TABLE EMPLOYEE TRUNCATE SUBPARTITION SYS_SUBP7360 DROP STORAGE UPDATE GLOBAL INDEXES;
Here the above 5 steps are executed for three sub-partitions
6) Get this partition name from all_tab_partitions table with global index update ALTER TABLE EMPLOYEE DROP PARTITION SYS_P7363 UPDATE GLOBAL INDEXES;
7) alter index PK_DEPARTMENT rebuild;
8) alter index IDX1_DEPARTMENT rebuild;
9) alter index PK_EMPLOYEE rebuild;
Steps 7 to 9 is require only when UPDATE GLOBAL INDEXES not used at the time of Altering Table for exchange, truncate and drop partition or sub partition
10)
begin
dbms_stats.gather_table_stats(ownname=>'TEST', tabname=>'DEPARTMENT', granularity=>'ALL' ,no_invalidate=>FALSE);
dbms_stats.gather_table_stats(ownname=>'TEST', tabname=>'EMPLOYEE', granularity=>'ALL' ,no_invalidate=>FALSE);
end;
UPDATE GLOBAL INDEXES is used at the time of Altering Table for exchange, truncate and drop partition or sub partition where global indexes are applied on that table then we can do DML operation on same table in parallel here.
UPDATE GLOBAL INDEXES is used to rebuild the indexes at same time with alter operation.
If UPDATE GLOBAL INDEXES is not used then DML operation is not allowed on same table in parallel. It will throw below ORA Exception for global indexes.
ERROR Message when perform insert operation after drop/exchange/truncate partition/sub partition from table:
SQL Error: ORA-01502: index 'TEST.PK_DEPARTMENT' or partition of such
index is in unusable state
01502. 00000 - "index '%s.%s' or partition of such index is in unusable state"
Cause: An attempt has been made to access an index or index partition that has been marked unusable by a direct load or by a DDL operation
Action: DROP the specified index, or REBUILD the specified index, or REBUILD the unusable index partition
If you don’t want to use UPDATE GLOBAL INDEXES and want parallel DML operation with purging then make indexes Local (must not contain primary key or unique constraint because it is default created global unique index). It’s on partition and sub partition and refers default table space.
With Local indexes you observed slow insertion on that table.
Related
Summary of the question: To Create table with partitions which are range partitioned. However records which do not know the range value should reside in a different (default) partition and be moved to the correct partition when the value is filled. The default partition would never be dropped while the other partitions would be dropped after a defined retention period via an script.
The whole story:
I have a table where the records have to be placed in a partition based on a date field. This is a growing table and after some time the data from these partitions can be purged. I used to create table with something like the snippet below.
This works fine because we knew the value of the date column based on which we partition (RDATE). However in our new project we do not know this when a record is inserted. The value would eventually be filled in during the course of the application processing.
My initial thought was to create MAXPARTITION (MAXVALUE) which would be a catch-all partition for records which do not have the date filled and enable ROW MOVEMENTS so that when the date is filled it moves into an appropriate partition. However I think it is not possible to have both MAXVALUE partition and interval partitioning together. Is that right?
Also Is there a better way to do this?
PARTITION BY RANGE ("RDATE") INTERVAL (NUMTODSINTERVAL (1,'DAY'))
SUBPARTITION BY HASH ("RKEY")
SUBPARTITION TEMPLATE (
SUBPARTITION "SP01",
SUBPARTITION "SP02",
SUBPARTITION "SP03",
SUBPARTITION "SP04",
SUBPARTITION "SP05",
SUBPARTITION "SP06",
SUBPARTITION "SP07",
SUBPARTITION "SP08",
SUBPARTITION "SP09",
SUBPARTITION "SP10",
SUBPARTITION "SP11",
SUBPARTITION "SP12",
SUBPARTITION "SP13",
SUBPARTITION "SP14",
SUBPARTITION "SP15",
SUBPARTITION "SP16" )
(PARTITION "INITIALPARTITION" VALUES LESS THAN (TO_DATE(' 2016-01-01 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN'))
I expect a table with default and range partitions and records to move to the range partitions from the default when a column is filled.
The column you use as partition key cannot be NULL but you can use a workaround like this:
CREATE TABLE ... (
...
RDATE DATE,
PARTITION_KEY DATE GENERATED ALWAYS AS (COALESCE(RDATE, DATE '1969-12-31'))
)
PARTITION BY RANGE (PARTITION_KEY) INTERVAL (NUMTODSINTERVAL (1,'DAY'))
...
(PARTITION INITIAL_PARTITION VALUES LESS THAN (DATE '1970-01-01'))
ENABLE ROW MOVEMENT;
If you insert a record with RDATE = NULL then it will be inserted into partition INITIAL_PARTITION. For the initial data (e.g. 1970-01-01) you must select a values whicc will never fall into the "real" date values. You could also use a date in far future, e.g.
CREATE TABLE ... (
...
RDATE DATE,
PARTITION_KEY DATE GENERATED ALWAYS AS (COALESCE(RDATE, DATE '2999-12-31'))
)
PARTITION BY RANGE (PARTITION_KEY) INTERVAL (NUMTODSINTERVAL (1,'DAY'))
...
(PARTITION INITIAL_PARTITION VALUES LESS THAN (DATE '2019-04-01'))
ENABLE ROW MOVEMENT;
-- Create DEFAULT_PARTITION
INSERT INTO ... (RDATE) VALUES (NULL);
ROLLBACK;
ALTER TABLE ... RENAME PARTITION FOR (TIMESTAMP '2999-12-31 00:00:00') TO DEFAULT_PARTITION;
Below the simplified structure of a table:
create table customer(
incident_id number,
customer_id number,
customer_name varchar2(400),
sla_id number
failure_start_date date,
failure_end_date date,
churn_flag number, -- 0 or 1
active number, -- 0 or 1
constraint pk_incident_id primary key (incident_id))
PARTITION BY LIST (active)
SUBPARTITION BY LIST (churn_flag)
SUBPARTITION TEMPLATE
( SUBPARTITION sp_churn_flag_1 VALUES (1)
, SUBPARTITION sp_churn_flag_0 VALUES (0)
)
(PARTITION sp_active_1 values (1)
, PARTITION sp_active_0 VALUES (0)
)
,
ENABLE ROW MOVEMENT COMPRESS FOR QUERY LOW;
Now I need to add additonally to the existing Composite-List-Partition an Interval-Range-Partitioning, in order to partitionate the data by month (failure_starte_date - YYYYMM). The table contains data from 200701 up to now (201511). Failure_start_date < 2013 should be partitionied into one partition for older data. All newer months should have an dedicated partition, whereas partitions for upcoming months shall be created automatically.
How can this be integrating into the already existing partitoning?
You cannot do it exactly the way you want. Partitioning strategies are limited in two relevant ways: first, composite strategies can only have two levels (you need 3) and second, interval partitioning, when used in a composite strategy must be at the top level.
Here is the closest legal thing to what you want:
CREATE TABLE matt_customer
(
incident_id NUMBER,
customer_id NUMBER,
customer_name VARCHAR2 (400),
sla_id NUMBER,
failure_start_date DATE,
failure_end_date DATE,
churn_flag VARCHAR2 (1), -- 0 or 1
active VARCHAR2 (1), -- 0 or 1
active_churn_flags VARCHAR2 (2) GENERATED ALWAYS AS (active || churn_flag) VIRTUAL,
CONSTRAINT pk_incident_id PRIMARY KEY (incident_id)
)
PARTITION BY RANGE
(failure_start_date)
INTERVAL ( NUMTOYMINTERVAL (1, 'MONTH') )
SUBPARTITION BY LIST
(active_churn_flags)
SUBPARTITION TEMPLATE (
SUBPARTITION sp_ac_00 VALUES ('00'),
SUBPARTITION sp_ac_01 VALUES ('01'),
SUBPARTITION sp_ac_10 VALUES ('10'),
SUBPARTITION sp_ac_11 VALUES ('11'))
(PARTITION customer_old VALUES LESS THAN (TO_DATE ('01-JAN-2013', 'DD-MON-YYYY')))
ENABLE ROW MOVEMENT
--COMPRESS FOR QUERY LOW;
;
This uses interval-list partitioning, and uses a virtual column to combine your active and churn_flag columns into one (I turned those columns into VARCHAR2(1) for simplicity.
To make use of partition pruning, your queries would need to be modified to select active_churn_flags = '01' for example, instead of specifying values for active and churn_flag independently.
Assuming following table:
create table INVOICE(
INVOICE_ID NUMBER
,INVOICE_SK NUMBER
,INVOICE_AMOUNT NUMBER
,INVOICE_TEXT VARCHAR2(4000 Char)
,B2B_FLAG NUMBER -- 0 or 1
,ACTIVE NUMBER(1) -- 0 or 1
)
PARTITION BY LIST (ACTIVE)
SUBPARTITION BY LIST (B2B_FLAG)
( PARTITION p_active_1 values (1)
( SUBPARTITION sp_b2b_flag_11 VALUES (1)
, SUBPARTITION sp_b2b_flag_10 VALUES (0)
)
,
PARTITION p_active_0 values (0)
( SUBPARTITION sp_b2b_flag_01 VALUES (1)
, SUBPARTITION sp_b2b_flag_00 VALUES (0)
)
)
For perfomance reasons the table should get a "Composite List-List" partitioning, see http://docs.oracle.com/cd/E18283_01/server.112/e16541/part_admin001.htm#i1006565.
The problematic point is, that the ACTIVE-Flag will change requently for a huge amount of records and sometimes also the B2B_FLAG. Will Oracle automatically recognize the records, for which the partitioning value has changed and move them to the appropriate partion or do I have to call some kind of maintenance function, in order to reorganize the partitions?
You need to enable row movement on the table or the update statement will fail with ORA-14402: updating partition key column would cause a partition change.
See the following testcase:
create table T_TESTPART
(
pk number(10),
part_key number(10)
)
partition by list (part_key) (
partition p01 values (1),
partition p02 values (2),
partition pdef values (default)
);
alter table T_TESTPART
add constraint pk_pk primary key (PK);
Now insert a row and try to update the partitioning value:
insert into t_testpart values (1,1);
update t_testpart set part_key = 2 where pk = 1;
You will now get the Error mentioned above.
If you enable row movement, the same statement will work and oracle will move the row to the other partition:
alter table t_testpart enable row movement;
update t_testpart set part_key = 2 where pk = 1;
I did not do any performance tests, but Oracle will probably delete the row from the first partition and insert it to the second partition. Consider this when using it in large scale.
In my own databases, I usually only use partitioning on columns that do not change.
Further reading:
http://www.dba-oracle.com/t_callan_oracle_row_movement.htm
I created a partitioned table with the below command:
create table A
(
ID number(10),
SUBSCRIBER_TYPE varchar2(20)
,
SSN varchar2(20)
)
ENABLE ROW MOVEMENT
INITRANS 4 STORAGE(FREELISTS 16)
PARTITION BY LIST (SUBSCRIBER_TYPE)
(
PARTITION P000 VALUES(0),
PARTITION P001 VALUES(1),
PARTITION P002 VALUES(2)
)
TABLESPACE TS_SUBSCRIBER01 ;
Now I need to change the column name SSN to SOCIAL_SECURITY_NUMBER and its datatype to number.
Can I use a simple ALTER command to modify the column of the table so that I it reflects the changes in all the partitions?
Yes you can use a simple alter command for the partitioned table.
Lets say i have a table with the following definition
create table dummy (col1 number(9) not null)
All the values in this dummy.col1 are 7 digit long. Now i want to reduce the length of this column from 9 - 7 using alter command. Oracle gives me error that column to be modified must be empty to decrease precision or scale. Makes sense.
I want to ask is there any work around to reduce the column size?
I can't delete the values in the column.
I can't copy values from this column to another since it has trillions of data.
The column size has no relationship to how the data is physically stored (they are variable length)
e.g. '23' in a number(2) will take exactly the same space if stored in a number(38)
It is purely a constraint on the maximum number that can be stored in the column therefore you could just add a constraint on the column:
ALTER TABLE dummy ADD
CONSTRAINT c1
CHECK (col1 < 9999999)
ENABLE
VALIDATE;
if you want it to go a little quicker change VALIDATE to NOVALIDATE obviously this will not check the validity of the existing data.
Kevin's answer is excellent.
The only other way to do it is to
rename the existing column,
create a new column with the old name and the new size,
issue an update statement to populate the new field (which you said you cannot do)
and then drop the renamed column.
Are you sure you cannot find some downtime one weekend to perform this task ?
Solution #1
My solution below keeps the original column order.
I found that to be important, especially if there are canned SQL statements out
there (middle tier, client tier) that point back to your database that do implicit
SELECTs.
i.e.
SELECT *
FROM tableName
WHERE ...;
INSERT INTO copyTableName(column1,column2,column3,...)
SELECT *
FROM tableName
WHERE ...;
Here goes:
Generate the DDLs for
1. The table containing the column you intend to resize
2. All the relationship constraints, indexes, check constraints, triggers that reference that table.
3. All the foreign keys of other tables that reference the primary key of this table.
Make sure each table-referencing-object DDL is stand-alone, separate from the
CREATE TABLE DDL.
You'll have something like
/* 1. The table containing the column you intend to resize */
CREATE TABLE tableName
(
column1 TYPE(size) [DEFAULT value] [NOT] NULL,
column2 TYPE(size) [DEFAULT value] [NOT] NULL,
column3 TYPE(size) [DEFAULT value] [NOT] NULL,
...
)
TABLESPACE tsName
[OPTIONS];
/* 2. All the relationship constraints, indexes, check constraints, triggers that reference that table. */
CREATE INDEX indexName ON tableName
(column1)
NOLOGGING
TABLESPACE INDX
NOPARALLEL;
CREATE INDEX compositeIndexName ON tableName
(column1,column2,...)
NOLOGGING
TABLESPACE INDX
NOPARALLEL;
CREATE UNIQUE INDEX pkName ON tableName
(column2)
NOLOGGING
TABLESPACE INDX
NOPARALLEL;
ALTER TABLE tableName ADD (
CHECK (column4 IS NOT NULL));
ALTER TABLE tableName ADD (
CONSTRAINT pkName
PRIMARY KEY
(column2)
USING INDEX
TABLESPACE INDX);
ALTER TABLE tableName ADD (
CONSTRAINT fkName
FOREIGN KEY (column2)
REFERENCES otherTable (column2));
/* 3. All the foreign keys of other tables that reference the primary key of this table. */
ALTER TABLE otherTableName ADD (
CONSTRAINT otherTableFkName
FOREIGN KEY (otherTableColumn2)
REFERENCES tableName (column1));
Copy out just the CREATE TABLE statement, change the table name and
reduce the size of the column you wish to modify:
CREATE TABLE tableName_YYYYMMDD
(
column1 TYPE(size) [DEFAULT value] [NOT] NULL,
column2 TYPE(reducedSize) [DEFAULT value] [NOT] NULL,
column3 TYPE(size) [DEFAULT value] [NOT] NULL,
...
)
TABLESPACE tsName
[OPTIONS];
Insert the data from tableName into tableName_YYYYMMDD:
INSERT /* APPEND */ INTO tableName_YYYYMMDD(
column1 ,
column2 ,
column3 ,
... )
SELECT
column1 ,
column2 ,
column3 ,
...
FROM tableName;
COMMIT;
Drop all objects referencing the original table.
Also, drop all foreign keys that reference the tableName primary key pkName.
Don't worry, you've saved the DDL so you'll be able to recreate them.
Notice that I drop indexes after copying the data out of tableName.
I do this because perhaps one of the indexes will be used in the
above SELECT so that operation will complete faster.
DROP INDEX indexName ;
DROP INDEX compositeIndexName ;
DROP UNIQUE INDEX pkName ;
ALTER TABLE tableName DROP CONSTRAINT pkName ;
ALTER TABLE tableName DROP CONSTRAINT fkName ;
ALTER TABLE otherTableName DROP CONSTRAINT otherTableFkName ;
Drop the original table.
DROP TABLE tableName;
Rename the new table.
ALTER TABLE tableName_YYYYMMDD RENAME TO tableName;
Recreate all referencing objects from the DDL statements you saved before.
/* 2. All the relationship constraints, indexes, check constraints, triggers that reference that table. */
CREATE INDEX indexName ON tableName
(column1)
NOLOGGING
TABLESPACE INDX
NOPARALLEL;
CREATE INDEX compositeIndexName ON tableName
(column1,column2,...)
NOLOGGING
TABLESPACE INDX
NOPARALLEL;
CREATE UNIQUE INDEX pkName ON tableName
(column2)
NOLOGGING
TABLESPACE INDX
NOPARALLEL;
ALTER TABLE tableName ADD (
CHECK (column4 IS NOT NULL));
ALTER TABLE tableName ADD (
CONSTRAINT pkName
PRIMARY KEY
(column2)
USING INDEX
TABLESPACE INDX);
ALTER TABLE tableName ADD (
CONSTRAINT fkName
FOREIGN KEY (column2)
REFERENCES otherTable (column2));
/* 3. All the foreign keys of other tables that reference the primary key of this table. */
ALTER TABLE otherTableName ADD (
CONSTRAINT otherTableFkName
FOREIGN KEY (otherTableColumn2)
REFERENCES tableName (column1));
Solution #2
Keep the column order but do not rebuild non-unique-used-by-PK indexes that might contain column2.
ALTER TABLE tableName ADD (column2Copy TYPE(reducedSize));
UPDATE tableName SET column2Copy = column2;
ALTER TABLE tableName MODIFY (column2 TYPE(size) NULL);
ALTER TABLE tableName DROP CONSTRAINT pkName;
DROP INDEX pkName;
UPDATE tableName SET column2 = null;
ALTER TABLE tableName MODIFY (column2 TYPE(reducedSize));
UPDATE tableName SET column2 = column2Copy;
ALTER TABLE tableName DROP COLUMN column2Copy;
CREATE UNIQUE INDEX pkName ON tableName
(column2)
NOLOGGING
TABLESPACE INDX
NOPARALLEL;
ALTER TABLE tableName ADD (
CONSTRAINT pkName
PRIMARY KEY
(column2)
USING INDEX
TABLESPACE INDX);
COMMIT;