Oracle DB daily partitioning - oracle

I have the following table
CREATE TABLE "METRIC_VALUE_RAW"
(
"SUBELEMENT_ID" INTEGER NOT NULL ,
"METRIC_METADATA_ID" INTEGER NOT NULL ,
"METRIC_VALUE_INT" INTEGER,
"METRIC_VALUE_FLOAT" FLOAT(126),
"TIME_STAMP" TIMESTAMP NOT NULL
) ;
Every hour data will be loaded into the table using sql loader.
I want to create partitions so that data for every day go into a partition.
In table I want to store data for 30 days. So when it crosses 30 days, the oldest partition should get deleted.
Can you share your ideas on how I can design the partitions.

here is an example how to do it on Oracle 11g and it works very well. I haven't tried it on Oracle 10g, you can try it.
This is the way, how to create a table with daily partitions:
CREATE TABLE XXX (
partition_date DATE,
...,
...,
)
PARTITION BY RANGE (partition_date)
INTERVAL (NUMTODSINTERVAL(1, 'day'))
(
PARTITION part_01 values LESS THAN (TO_DATE('2000-01-01','YYYY-MM-DD'))
)
TABLESPACE MY_TABLESPACE
NOLOGGING;
As you see above, Oracle will automaticaly create separate partitions for each distinct partition_day after 1st January 2000. The records, whose partition_date is older than this date, will be stored in partition called 'part_01'.
You can monitore your table partitions using this statement:
SELECT * FROM user_tab_partitions WHERE table_name = 'XXX';
Afterwards, when you would like to delete some partitions, use following command:
ALTER TABLE XXX DROP PARTITION AAAAAA UPDATE GLOBAL INDEXES
where 'AAAAAA' is parition name.
I hope it will help you!

As i said , There are big differences in partition automation between 10g and 11g.
In 10G you will have to manually manage the partitions during your ETL process (I'm sure every 10g DBA has a utility package he wrote to manage partitions ... ).
For steps 1 & 2 , you have several options
load data directly into the daily partition.
load data into a new partition and merge it into the daily one.
load data into a new partition every hour, and during a maintenance
window merge all hourly partitions into a daily partition.
The right way for you depends on your needs. Is the newly added data is queried immediately ? In what manner ? Would you query for data across several hours (or loads...) ? Are you showing aggregations ? are you performing DML operations on the data (DDL operations on partitions cause massive locking).
about 3, again - manually. drop old partitions.
In 11G, you have the new interval partition feature with automates some of the tasks mentioned above.

Following is a sample create table sql to parititon data:
CREATE TABLE quarterly_report_status (
report_id INT NOT NULL,
report_status VARCHAR(20) NOT NULL,
report_updated TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
)
PARTITION BY RANGE ( UNIX_TIMESTAMP(report_updated) ) (
PARTITION p0 VALUES LESS THAN ( UNIX_TIMESTAMP('2008-01-01 00:00:00') ),
PARTITION p1 VALUES LESS THAN ( UNIX_TIMESTAMP('2008-01-02 00:00:00') ),
PARTITION p2 VALUES LESS THAN ( UNIX_TIMESTAMP('2008-01-03 00:00:00') ),
PARTITION p3 VALUES LESS THAN ( UNIX_TIMESTAMP('2008-01-04 00:00:00') ),
PARTITION p4 VALUES LESS THAN ( UNIX_TIMESTAMP('2008-01-05 00:00:00') ),
PARTITION p5 VALUES LESS THAN ( UNIX_TIMESTAMP('2008-01-06 00:00:00') ),
PARTITION p6 VALUES LESS THAN ( UNIX_TIMESTAMP('2008-01-07 00:00:00') ),
PARTITION p7 VALUES LESS THAN ( UNIX_TIMESTAMP('2008-01-08 00:00:00') ),
PARTITION p8 VALUES LESS THAN ( UNIX_TIMESTAMP('2008-01-09 00:00:00') ),
PARTITION p9 VALUES LESS THAN (MAXVALUE)
);
Paritions will be created by DBa and rest will be taken care of by oracle.
If you want to delete partition then you will have to write separate jobs for it.

Related

partition in oracle

CREATE TABLE temp_stud as select * from STUD_MAST
PARTITION BY RANGE(ADM_DT)
(
PARTITION temp_stud1 VALUES LESS THAN(TO_DATE('02/01/2000','MM/DD/YYYY')),
PARTITION temp_stud2 VALUES LESS THAN(TO_DATE('03/01/2000','MM/DD/YYYY')),
PARTITION temp_stud3 VALUES LESS THAN(TO_DATE('04/01/2000','MM/DD/YYYY')),
PARTITION temp_stud4 VALUES LESS THAN(TO_DATE('05/01/2000','MM/DD/YYYY'))
);
I am getting a missing left parenthesis error for above table creation can anyone tell me what is the issue in above creation
Note: ADM_DT is a date column with data type char(8) and storing format YYMMDD
Please use below SQL. The Creation of Partition has be part of Create table.
CREATE TABLE temp_stud
PARTITION BY RANGE(ADM_DT)
(
PARTITION temp_stud1 VALUES LESS THAN(TO_DATE('02/01/2000','MM/DD/YYYY')),
PARTITION temp_stud2 VALUES LESS THAN(TO_DATE('03/01/2000','MM/DD/YYYY')),
PARTITION temp_stud3 VALUES LESS THAN(TO_DATE('04/01/2000','MM/DD/YYYY')),
PARTITION temp_stud4 VALUES LESS THAN(TO_DATE('05/01/2000','MM/DD/YYYY'))
)
as select * from STUD_MAST;

Repartitioning a table on a different column

We need to change the structure of our DB by re-partitioning a parent table and adding partitions to the child tables. The partitions will be based on range of a date-time field (which is the time of insertion).
The goal is to implement the changes with as little downtime as possible. So we have though on doing it on 2 phases: do the main load while the application is running and load the delta after the application has been shutdown.
I would like to know if the following steps are correct for the plan I mentioned, focusing on the part of re-partitioning the parent table. I have based on this example from Oracle: https://docs.oracle.com/database/121/SUTIL/GUID-56D8448A-0106-4B8C-85E0-11CFED8C71B1.htm
1. create a directory for the export/import of dumps
-- user: SYS
CREATE OR REPLACE DIRECTORY dumps AS '/data/tmp';
GRANT READ, WRITE ON DIRECTORY dumps TO APP_USR;
2. export the table while the application is running
# nohup expdp APP_USR/pwd parfile=export.par &
directory=dumps
dumpfile=parent_table.dmp
parallel=8
logfile=export.log
tables=parent_table
query=parent_table:"WHERE VERSION_TSP <= sysdate"
3. create a duplicate of the table with the new partitioning
CREATE TABLE "PARENT_TABLE_REPARTITIONED"
(
-- same fields as original
CONSTRAINT PK_TABLE_ PRIMARY KEY (foo, boo) -- with different name than original
)
PARALLEL 8
PARTITION BY RANGE (VERSION_TSP)
INTERVAL (NUMTOYMINTERVAL(1, 'MONTH'))
(
PARTITION PARENT_TABLE_PARTITION_ -- with different name than original (should I?)
VALUES LESS THAN (TO_DATE('2007-07-01', 'YYYY-MM-DD')) COMPRESS FOR OLTP
)
ENABLE ROW MOVEMENT
;
-- with new local indexes (named differently)
CREATE INDEX IDX_VERSION_TSP_ ON PARENT_TABLE_REPARTITIONED (VERSION_TSP) LOCAL;
4. populate the re-partitioned table while the application is running
# nohup impdp APP_USR/pwd parfile=import.par &
directory=dumps
dumpfile=parent_table.dmp
parallel=8
logfile=import.log
remap_table=PARENT_TABLE:PARENT_TABLE_REPARTITIONED
table_exists_action=APPEND
5. Application shutdown (and backup the DB)
6. populate the delta
-- execute with nohup
declare
startDate DATE;
begin
select max(version_tsp) into startDate from PARENT_TABLE_REPARTITIONED;
insert into PARENT_TABLE_REPARTITIONED select /*+ parallel(8) */ s.* from PARENT_TABLE s where s.VERSION_TSP > startDate;
commit;
end;
7. switch tables
drop table parent_table cascade constraints;
rename parent_table_repartitioned to parent_table;
8. deploy the new version of the application
Is ok to export ALL from the table or should I export DATA_ONLY?
But most importantly, are those steps a valid approach?
Are you on 12.2 or higher? It could be as easy as:
SQL> create table t
2 partition by range ( created )
3 (
4 partition p1 values less than ( date '2020-01-01' ),
5 partition p2 values less than ( date '2021-01-01' ),
6 partition p3 values less than ( date '2022-01-01' ),
7 partition p4 values less than ( date '2023-01-01' )
8 )
9 as select * from dba_objects
10 where created is not null
11 and last_ddl_time is not null;
Table created.
SQL>
SQL> create index t_ix on t ( owner ) local;
Index created.
SQL>
SQL>
SQL> alter table t modify
2 partition by range ( last_ddl_time )
3 (
4 partition p1 values less than ( date '2020-01-01' ),
5 partition p2 values less than ( date '2021-01-01' ),
6 partition p3 values less than ( date '2022-01-01' ),
7 partition p4 values less than ( date '2023-01-01' )
8 ) online ;
Table altered.

Automatic Oracle partitioning by month in the tablespace corresponding to the month

I need to partition my table by month automatically and that each partition is in the tablespace corresponding to the month. I don't need the round robin way. I want each partition to be in its tablespace. Thanks
As I already stated, you have to use round robin, because the tablespaces have to exist at desing time. Otherwise Oracle cannot create partitions automatically. Would be this:
CREATE TABLE MY_TABLE
(
START_TIME TIMESTAMP(0) NOT NULL,
more columns
)
PARTITION BY RANGE (START_TIME) INTERVAL (INTERVAL '1' MONTH)
STORE IN (
TS_JAN, TS_FEB, TS_MAR,
TS_APR, TS_MAY, TS_JUN,
TS_JUL, TS_AUG, TS_SEP,
TS_OKT, TS_NOV, TS_DEC
)
(
PARTITION P_INITIAL VALUES LESS THAN (TIMESTAMP '2021-01-01 00:00:00') TABLESPACE TS_JAN
);
Just ensure timestamp of the inital partition (e.g. 1st of January) matches the correct tablespace.
Otherwise you have to create all partitions and tablespaces manually in advance.

SQL Error: ORA-14006: invalid partition name

I am trying to partition an existing table in Oracle 12C R1 using below SQL statement.
ALTER TABLE TABLE_NAME MODIFY
PARTITION BY RANGE (DATE_COLUMN_NAME)
INTERVAL (NUMTOYMINTERVAL(1,'MONTH'))
(
PARTITION part_01 VALUES LESS THAN (TO_DATE('01-SEP-2017', 'DD-MON-RRRR'))
) ONLINE;
Getting error:
Error report -
SQL Error: ORA-14006: invalid partition name
14006. 00000 - "invalid partition name"
*Cause: a partition name of the form <identifier> is
expected but not present.
*Action: enter an appropriate partition name.
Partition needs to be done on the basis of data datatype column with the interval of one month.
Min value of Date time column in the Table is 01-SEP-2017.
You can't partition an existing table like that. That statement is modifying the partition that hasn't been created yet. I don't know the automatic way to do this operation and I am not sure that you can do it.
Although I have done this thing many times but with manual steps. Do the following if you can't find an automated solution:
Create a partitioned table named table_name_part with your clauses and all your preferences.
Insert into this partitioned table all rows from original table. Pay attention to compression. If you have some compression on table (Basic or HCC) you have to use + APPEND hint.
Create on partitioned table your constrains and indexes from the original table.
Rename the tables and drop the original table. Do not drop it until you make some counts on them.
I saw that your table has the option to auto-create partition if it does not exists. (NUMTOYMINTERVAL(1,'MONTH')) So you have to create your table with first partition only. I assume that you have here a lot of read-only data, so you won't have any problem with consistency instead of last month. Probably there is some read-write data so there you have to be more careful with the moment when you want to insert data in new table and switch tables.
Hope to help you. As far as I know there might be a package named DBMS_REDEFINITION that can help you with an automated version of my steps. If you need more details or need some help on my method, please don't hesitate.
UPDATE:
From Oracle 12c R2 you can convert a table from an unpartitioned to a partitioned one with your method. Find a link below. Now this is a challenge for me and I am trying to convert, but I think there is no way to make this conversion online in 12c R1.
In previous releases you could partition a non-partitioned table using
EXCHANGE PARTITION or DBMS_REDEFINITION in an "almost online" manner,
but both methods require multiple steps. Oracle Database 12c Release 2
makes it easier than ever to convert a non-partitioned table to a
partitioned table, requiring only a single command and no downtime.
https://oracle-base.com/articles/12c/online-conversion-of-a-non-partitioned-table-to-a-partitioned-table-12cr2
Solution
I found a solution for you. Here you will have all of my steps that I run to convert table online. :)
1. Create regular table and populate it.
CREATE TABLE SCOTT.tab_unpartitioned
(
id NUMBER,
description VARCHAR2 ( 50 ),
created_date DATE
);
INSERT INTO tab_unpartitioned
SELECT LEVEL,
'Description for ' || LEVEL,
ADD_MONTHS ( TO_DATE ( '01-JAN-2017', 'DD-MON-YYYY' ),
-TRUNC ( DBMS_RANDOM.VALUE ( 1, 4 ) - 1 ) * 12 )
FROM DUAL
CONNECT BY LEVEL <= 10000;
COMMIT;
2. Create partitioned table with same structure.
--If you are on 11g create table with CREATE TABLE command but with different name. ex: tab_partitioned
CREATE TABLE SCOTT.tab_partitioned
(
id NUMBER,
description VARCHAR2 ( 50 ),
created_date DATE
)
PARTITION BY RANGE (created_date)
INTERVAL( NUMTOYMINTERVAL(1,'YEAR'))
(PARTITION part_2015 VALUES LESS THAN (TO_DATE ( '01-JAN-2016', 'DD-MON-YYYY' )),
PARTITION part_2016 VALUES LESS THAN (TO_DATE ( '01-JAN-2017', 'DD-MON-YYYY' )),
PARTITION part_2017 VALUES LESS THAN (TO_DATE ( '01-JAN-2018', 'DD-MON-YYYY' )));
--this is an alter command that works only in 12c.
ALTER TABLE tab_partitioned
MODIFY
PARTITION BY RANGE (created_date)
(PARTITION part_2015 VALUES LESS THAN (TO_DATE ( '01-JAN-2016', 'DD-MON-YYYY' )),
PARTITION part_2016 VALUES LESS THAN (TO_DATE ( '01-JAN-2017', 'DD-MON-YYYY' )),
PARTITION part_2017 VALUES LESS THAN (TO_DATE ( '01-JAN-2018', 'DD-MON-YYYY' )));
3. Check if the table can be converted. This procedure should run without any error.
Prerequisites: table should have an UNIQUE INDEX and a Primary Key constraint.
EXEC DBMS_REDEFINITION.CAN_REDEF_TABLE('SCOTT','TAB_UNPARTITIONED');
4. Run the following steps like I have done.
EXEC DBMS_REDEFINITION.START_REDEF_TABLE('SCOTT','TAB_UNPARTITIONED','TAB_PARTITIONED');
var num_errors varchar2(2000);
EXEC DBMS_REDEFINITION.COPY_TABLE_DEPENDENTS('SCOTT','TAB_UNPARTITIONED','TAB_PARTITIONED', 1,TRUE,TRUE,TRUE,FALSE,:NUM_ERRORS,FALSE);
SQL> PRINT NUM_ERRORS -- Should return 0
EXEC DBMS_REDEFINITION.SYNC_INTERIM_TABLE('SCOTT','TAB_UNPARTITIONED','TAB_PARTITIONED');
EXEC DBMS_REDEFINITION.FINISH_REDEF_TABLE('SCOTT','TAB_UNPARTITIONED','TAB_PARTITIONED');
At the end of the script you will see that the original table is partitioned.
Try Oracle Live SQL I used to use Oracle 11g EE and got the same error message. So I tried Oracle live SQL and it perfectly worked. It has very simple and easy to understand interface,
For example, I'm creating a sales table and inserting some dummy data and partition it using range partitioning method,
CREATE TABLE sales
(product VARCHAR(300),
country VARCHAR(100),
sales_year DATE);
INSERT INTO sales (product, country, sales_year )
VALUES ('Computer','Kazakhstan',TO_DATE('01/02/2018','DD/MM/YYYY'));
INSERT INTO sales (product, country, sales_year )
VALUES ('Mobile Phone','China',TO_DATE('23/12/2019','DD/MM/YYYY'));
INSERT INTO sales (product, country, sales_year )
VALUES ('Camara','USA',TO_DATE('20/11/2020','DD/MM/YYYY'));
INSERT INTO sales (product, country, sales_year )
VALUES ('Watch','Bangladesh',TO_DATE('19/03/2020','DD/MM/YYYY'));
INSERT INTO sales (product, country, sales_year )
VALUES ('Cake','Sri Lanka',TO_DATE('13/04/2021','DD/MM/YYYY'));
ALTER TABLE sales MODIFY
PARTITION BY RANGE(sales_year)
INTERVAL(INTERVAL '1' YEAR)
(
PARTITION sales_2018 VALUES LESS THAN(TO_DATE('01/01/2019','DD/MM/YYYY')),
PARTITION sales_2019 VALUES LESS THAN(TO_DATE('01/01/2020','DD/MM/YYYY')),
PARTITION sales_2020 VALUES LESS THAN(TO_DATE('01/01/2021','DD/MM/YYYY')),
PARTITION sales_2021 VALUES LESS THAN(TO_DATE('01/01/2022','DD/MM/YYYY'))
)ONLINE;
Finally, I can write SELECT query for partitions to confirm that the partitions are created successfully.
SELECT *
FROM sales PARTITION (sales_2020);
And it gives the expected output,

Oracle 12c - Table with size larger than 5 terabytes

In our database(Oracle 12c, Exadata) we plan to store sales data. Input text files containing sales data comes daily basis(~1000 files every day each containing ~20000 rows). Text files are read and transfered to db asap. Acccording to our calculations it will grow up to 5 terabytes in one year.
Data format:
[transaction date][category][sales_number][buyer_id][other columns]
sales data comes in 10 different categories with same fields. Data logically can be stored in just one single table or can be divided into 10 tables (with respect to categories).
What is the best practice for storing such kind of big data in oracle? What kind of partitioning and indexing strategy should be applied?
Constraints: Data should be available for analysis in 2-3 days to marketing department. Queries based on [sales_number] or [category],[buyer_id] or [buyer_id] columns.
If the number of categories is known and fix then you can use a subpartition for each category.
One approach could be this one:
CREATE TABLE SALES_DATA
(
TRANSACTION_DATE TIMESTAMP(0) NOT NULL,
CATEGORY NUMBER NOT NULL,
SALES_NUMBER NUMBER,
BUYER_ID NUMBER,
[OTHER COLUMNS]
)
PARTITION BY RANGE (TRANSACTION_DATE) INTERVAL (INTERVAL '1' DAY)
SUBPARTITION BY LIST (CATEGORY)
SUBPARTITION TEMPLATE
(
SUBPARTITION CAT_1 VALUES (1),
SUBPARTITION CAT_2 VALUES (2),
SUBPARTITION CAT_3_AND_4 VALUES (3,4),
SUBPARTITION CAT_5 VALUES (5),
...
SUBPARTITION CAT_10 VALUES (10),
SUBPARTITION CAT_OTHERS VALUES (DEFAULT)
)
(
PARTITION P_INITIAL VALUES LESS THAN (TIMESTAMP '2018-01-01 00:00:00')
);
Local indexes would be needed on sales_number and buyer_id. You can put every (sub)partition into a separated tablespace if required.

Resources