I have a table which holds data for 12 hours. Every 5 minutes, it keeps deleting data which is more than 12 hours old and adds new data. It has almost 15-20 million rows. I want to create partition by hour and also index the table on column(time_stamp), to make the row fetching faster.
I will obviously do interval or range partitioning, but found that interval partitioning doesn't work on indexed table. So please help me with the syntax so that oracle creates 12 partitions and automatically adds new one when new time_stamp data is added which is after first 12 hours. I have already got a procedure to delete oldest partition which i will use so that there is always 12 hours of data.
I am stating the columns below.
CustomerId,ApplicationId,Time_Stamp,Service
I have tried to come up with this, but don't know how it will create new partitions
CREATE TABLE local_table
(customerid VARCHAR2(30),
applicationid VARCHAR2(30),
time_stamp TIMESTAMP,
service VARCHAR2(30))
PARTITION BY RANGE(time_stamp)
(
PARTITION t1 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 00:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t2 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 01:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t3 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 02:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t4 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 03:00:00.0','YYYY-MM- DD HH24:MI:SS.ff')),
PARTITION t5 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 04:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t6 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 05:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t7 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 06:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t8 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 07:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t9 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 08:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t10 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 09:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t11 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 10:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t12 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 11:00:00.0','YYYY-MM-DD HH24:MI:SS.ff'))
);
create index index_time_stamp on local_table(TIME_STAMP);
I am using- Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit
Create table with autopartitiong and LOCAL (partitioning) index.
The local_partitioned_index clauses let you specify that the index is partitioned on the same columns, with the same number of partitions and the same partition bounds as table. Oracle Database automatically maintains local index partitioning as the underlying table is repartitioned.
CREATE TABLE local_table
(customerid VARCHAR2(30),
applicationid VARCHAR2(30),
time_stamp TIMESTAMP,
service VARCHAR2(30))
PARTITION BY RANGE(time_stamp)
INTERVAL(NUMTODSINTERVAL(1, 'HOUR'))
(PARTITION t1 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 00:00:00.0','YYYY-MM-DD HH24:MI:SS.ff'))
);
CREATE INDEX index_time_stamp on local_table(TIME_STAMP) LOCAL;
SELECT *
FROM user_tab_partitions;
INSERT INTO local_table VALUES('1', 'a', sysdate, 'b');
SELECT *
FROM user_tab_partitions;
INSERT INTO local_table VALUES('2', 'c', sysdate + 1/1440, 'd');
SELECT *
FROM user_tab_partitions;
INSERT INTO local_table VALUES('3', 'e', sysdate + 1/24, 'f');
SELECT *
FROM user_tab_partitions;
The INTERVAL clause of the CREATE TABLE statement establishes interval
partitioning for the table. You must specify at least one range
partition using the PARTITION clause. The range partitioning key value
determines the high value of the range partitions, which is called the
transition point, and the database automatically creates interval
partitions for data beyond that transition point. The lower boundary
of every interval partition is the non-inclusive upper boundary of the
previous range or interval partition.
For example, if you create an interval partitioned table with monthly
intervals and the transition point at January 1, 2010, then the lower
boundary for the January 2010 interval is January 1, 2010. The lower
boundary for the July 2010 interval is July 1, 2010, regardless of
whether the June 2010 partition was previously created. Note, however,
that using a date where the high or low bound of the partition would
be out of the range set for storage causes an error. For example,
TO_DATE('9999-12-01', 'YYYY-MM-DD') causes the high bound to be
10000-01-01, which would not be storable if 10000 is out of the legal
range.
Some quick DRAFT for your second question about DROP PARTITION. Examine and debug before uncomment ALTER TABLE. You can create scheduler job for run this block of code every hour.
DECLARE
l_pt_cnt NUMBER;
l_pt_name VARCHAR2(100);
l_minrowid ROWID;
l_mindate TIMESTAMP;
BEGIN
-- get partition count
SELECT count(*)
INTO l_pt_cnt
FROM user_tab_partitions
WHERE table_name = 'LOCAL_TABLE';
IF l_pt_cnt > 12 THEN
SELECT min(time_stamp)
INTO l_mindate
FROM LOCAL_TABLE;
-- get ROWID with min date
SELECT min(rowid)
INTO l_minrowid
FROM LOCAL_TABLE
WHERE time_stamp = l_mindate;
-- get name of partition with row with min date
SELECT subobject_name
INTO l_pt_name
FROM LOCAL_TABLE
JOIN user_objects
ON dbms_rowid.rowid_object(LOCAL_TABLE.rowid) = user_objects.object_id
WHERE LOCAL_TABLE.rowid = l_minrowid;
DBMS_OUTPUT.put_line('ALTER TABLE LOCAL_TABLE DROP PARTITION ' || l_pt_name );
--EXECUTE IMMEDIATE 'ALTER TABLE LOCAL_TABLE DROP PARTITION ' || l_pt_name;
END IF;
END;
Related
I create table:
create table big_table(
bt_id number primary key,
bt_date date,
bt_value varchar2(20)
)
Then I wnat partition this table (code abbreviated):
alter table big_table modify
partition by range (bt_date)
interval(numtoyminterval(1, 'MONTH'))
subpartition by hash (bt_id)
(
partition nn_st_p1 values less than (to_date(' 2019-05-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
subpartitions 4
store in (ipr_tbl),
)online
Error message:
17:20:39 line 1: ORA-14006: invalid partition name
I can't understand what is wrong with my partition name?
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
Try something like this
CREATE TABLE big_table
(bt_id NUMBER PRIMARY KEY
, bt_date DATE
, bt_value VARCHAR2(20)
)
PARTITION BY RANGE (bt_date) INTERVAL (NUMTOYMINTERVAL(1,'MONTH'))
SUBPARTITION BY HASH (bt_id) SUBPARTITIONS 4
(PARTITION nn_st_p1 VALUES LESS THAN (TO_DATE('01-MAY-2019','dd-MON-yyyy'))
)
PARALLEL;
I need the list of unique values for group_name across tables T1 and T2. Both tables are partitioned and contains hundreds of millions of records. The column that I'm interested in has a local bitmap index with up-to-date statistics. There are roughly 500 unique values for this column. (See test case).
Given this information, the most efficient way to find the unique values would appear to be: Find the 500 or so unique values from T1, then find the 500 or so unique values from T2, and then deduplicate the list. So that translates into this query:
select distinct group_name from t1 union
select distinct group_name from t2;
However, the actual execution generated by Oracle is something like this:
SELECT
SORT UNIQUE <-- 500 records
UNION-ALL <-- 1,430,000,000 records
BITMAP INDEX FFS T1 <-- 1,300,000,000 records
BITMAP INDEX FFS T2 <-- 130,000,000 records
So the optimizer seems to have re-written the query to something like the below query, effectively skipping the unique operation from the intermediate steps:
select distinct group_name
from (select group_name from t1 union all -- No Unique here
select group_name from t2)
Here is my actual question:
Without re-writing the query, can I force the following execution plan using hints only? i.e the way my original query was actually written?
SELECT
SORT UNIQUE <-- 500
UNION-ALL <-- 1000
HASH UNIQUE <-- 500 <-- Reduce early
BITMAP INDEX FFS T1 <-- 1,300,000,000
HASH UNIQUE <-- 500 <-- Reduce early
BITMAP INDEX FFS T2 <-- 130,000,000
Here is a test case that creates the two tables above.
create table t1(id number, group_name number, payload varchar2(100)) nologging partition by hash(id) partitions 4;
create table t2(id number, group_name number, payload varchar2(100)) nologging partition by hash(id) partitions 4;
insert /*+ append */ all
into t1
into t2
select rownum as id
,mod(rownum, 500) as group_name
,lpad('x', 100, 'x') as payload
from dual connect by level <= 1e6;
create bitmap index t1_bx on t1(group_name) nologging local;
create bitmap index t2_bx on t2(group_name) nologging local;
I'm using Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
Edit:
Incidentally, while preparing the test case I found two ways around the problem, but I would still like to know if there is a way to /*+ hint */ my way out of the problem.
select group_name from t1 group by group_name union
select group_name from t2 group by group_name;
with v1 as(select /*+ materialize */distinct group_name from t1)
,v2 as(select /*+ materialize */distinct group_name from t2)
select group_name from v1 union
select group_name from v2;
I need to create a range-partitioned table:
i.e.
create table table1(item_id number(22), sys_entry_date timestamp default sysdate)
partition by range(sys_entry_date) interval(NUMTOYMINTERVAL(1,'YEAR'))
(partition p01 values less than (to_date('31-DEC-2016','DD-MON-YYYY')));
A few insert for demo purpose:
---Should lie in the main partition's main subpartition (crnt_part) since the it's part of the latest records received.
insert into table1 values(1, sysdatetime);
---Should lie in the main partition's subpartition of default section (prev_part) since the it's 2 days older
insert into table1 values(1, sysdatetime-3);
insert into table1 values(1, sysdatetime-4);
---Would help us identify the yearly partitions (suggestive)
insert into table1 values(2, sysdatetime-1500);
insert into table1 values(3, sysdatetime-1200);
insert into table1 values(4, sysdatetime-800);
insert into table1 values(1, sysdatetime-400);
I want to achieve following through it:
An Yearly partition;
Within the yearly, a sub-partition based on sys_entry_date which
2.a. latest 2 days held in crnt_part
2.b. remaining held in default partition, maybe in prev_part
Appreciate if someone could help in this particular context.
As we know that Interval partitioning is an extension of Range partitioning and it takes much of its syntax from
range partitioning.
From various sources on the net, I assume that interval partitioning creation query do have mandatory following clause:
VALUES LESS THAN (XXX)
But when we go for interval partitioning, is there any simpler way where we do not provide any VALUE LESS THAN CLAUSE.
I was searching for something similar like EQUAL TO (012019) where 012019 corresponds to the January month Interval
of 2019 year.
I have gone through following links for the help/understanding but couldn't find useful for my concern.
http://www.dba-oracle.com/t_interval_partitioning.htm
https://docs.oracle.com/database/121/VLDBG/GUID-C121EA1B-2725-4464-B2C9-EEDE0C3C95AB.htm
The code used by me is like as follows:
create table
pos_data (
start_date DATE,
store_id NUMBER,
inventory_id NUMBER(6),
qty_sold NUMBER(3)
)
PARTITION BY RANGE (start_date)
INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
(
PARTITION pos_data_p2 VALUES LESS THAN (TO_DATE('1-7-2007', 'DD-MM-YYYY')),
PARTITION pos_data_p3 VALUES LESS THAN (TO_DATE('1-8-2007', 'DD-MM-YYYY'))
);
From my search It looks like there is no other way apart from the one VALUE LESS THAN.
Please share if anyone have some understanding about someother approach for creating interval based partitioning.
Remainder: my concern is in BOLD above
I think what you are looking for is the partition extended name
PARTITION FOR(DATE'2019-01-01')
Actually the LESS THAN definition plays in interval partitioning close to zero role.
You use it only once while creating the table to define some lower bound of the data.
Here is an example to define a table containing data starting from the year 2019
create table pos_data (
start_date DATE,
store_id NUMBER
)
SEGMENT CREATION DEFERRED
PARTITION BY RANGE (start_date)
INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
(
PARTITION pos_data_init VALUES LESS THAN (TO_DATE('1-1-2019', 'DD-MM-YYYY'))
);
Note that the first partition is not an interval partition (INTERVAL = NO) and doesn't physically exists due to SEGMENT CREATION DEFERRED. It will also never contain any data, as you start with 2019 content.
select PARTITION_POSITION,PARTITION_NAME,INTERVAL,MIN_EXTENT, HIGH_VALUE
from user_tab_partitions where table_name = 'POS_DATA'
order by PARTITION_POSITION;
PARTITION_POSITION PARTITION_NAME INTERVAL MIN_EXTENT HIGH_VALUE
------------------ -------------- -------- ---------- -------------------------------
1 POS_DATA_INIT NO TO_DATE(' 2019-01-01 00:00:00',
New partitions are created on the fly e.g. while inserting new data, you don't need to specify LESS THAN
insert into pos_data(start_date,store_id) values(DATE'2019-01-01',1);
PARTITION_POSITION PARTITION_NAME INTERVAL MIN_EXTENT HIGH_VALUE
------------------ -------------- -------- ---------- -------------------------------
1 POS_DATA_INIT NO TO_DATE(' 2019-01-01 00:00:00',
2 SYS_P16713 YES 1 TO_DATE(' 2019-02-01 00:00:00',
While accessing the table you use the partition_extended_name, you may choose any date within the month to reference the partition.
select * from pos_data
partition for (date'2019-01-15');
Same syntax may be used for partition maintainance
alter table pos_data move partition for (date'2019-01-30') compress;
I am using oracle 12c Interval partitioning. I have created the range partition with 1 month interval and list sub-partition using a unique identifier (lets say LOGIN_INTFID).
In table DDL I have added the list of sub-partitions that were known to me at the time of table creation. Here is extract of table DDL:
CREATE TABLE TEST
(
UNIQUE_ID NUMBER(9) NOT NULL,
LOGIN_INTFID VARCHAR2(20) NOT NULL,
LOGIN_SEQNO NUMBER(15) NOT NULL,
LOGIN_DATE DATE DEFAULT SYSDATE NOT NULL
)
PARTITION BY RANGE (LOGIN_DATE)
INTERVAL (NUMTOYMINTERVAL(1,'MONTH'))
SUBPARTITION BY LIST (LOGIN_INTFID) SUBPARTITION TEMPLATE (
SUBPARTITION SP1 VALUES ('ABC'),
SUBPARTITION SP2 VALUES ('DEF'),
)
(PARTITION TEST_Y2018M7D1 VALUES LESS
THAN (TO_DATE('2018-07-01 23:59:59', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')))
;
With this, new partition and sub partitions are created successfully. Later on, I had added one more sub-partition using the following alter command:
ALTER TABLE TEST modify partition SYS_P7068 add subpartition SP3 values ('XYZ');
I have also altered table TEST with the intention that next time when a new partition gets created, this new sub-partition is included in the table automatically:
ALTER TABLE TEST SET SUBPARTITION TEMPLATE (SUBPARTITION SP3 VALUES('XYZ'));
However, this last part is not working as expected. New partitions are not getting created when I am trying to insert data containing XYZ sub partition value. New partitions are getting created only when sub-partition values ABC/DEF are inserted.
What am I doing wrong?
You can't add a new subpartition to a template.
As the documetation states:
You can modify a subpartition template of a composite partitioned table by replacing it with a new subpartition template.
You have to define a new template consisting of the old and new subpartitions.
This will be valid for the partitions not yet created, for existing partition you have to add the subpartitions manually.
Example - after creation of your table you get one partition with two subpartitions
select PARTITION_NAME, SUBPARTITION_NAME,HIGH_VALUE
from user_tab_subpartitions where table_name = 'TEST';
PARTITION_NAME SUBPARTITION_NAME HIGH_VALUE
-------------- ----------------- ----------
TEST_Y2018 TEST_Y2018M7D1_ 'ABC'
TEST_Y2018 TEST_Y2018M7D1_ 'DEF'
Inserting a row adds an other partition with the same two subpartitions:
insert into test (UNIQUE_ID,LOGIN_INTFID,LOGIN_SEQNO,LOGIN_DATE)
values(1,'ABC',1,DATE'2018-08-02');
PARTITION_NAME SUBPARTITION_NAME HIGH_VALUE
-------------- -------------------- ----------
TEST_Y2018 TEST_Y2018M7D1_SP1 'ABC'
TEST_Y2018 TEST_Y2018M7D1_SP2 'DEF'
SYS_P14654 SYS_SUBP14652 'ABC'
SYS_P14654 SYS_SUBP14653 'DEF'
Now you change the subpartition template - by defining all the new subspartitions
ALTER TABLE TEST SET SUBPARTITION TEMPLATE (
SUBPARTITION SP1 VALUES ('ABC'),
SUBPARTITION SP2 VALUES ('DEF'),
SUBPARTITION SP3 VALUES('XYZ'));
and adds an other row
insert into test (UNIQUE_ID,LOGIN_INTFID,LOGIN_SEQNO,LOGIN_DATE)
values(1,'ABC',1,DATE'2018-09-02');
The new partition has now as expected three subpartitions
PARTITION_NAME SUBPARTITION_NAME HIGH_VALUE
-------------- -------------------- ----------
TEST_Y2018 TEST_Y2018M7D1_SP1 'ABC'
TEST_Y2018 TEST_Y2018M7D1_SP2 'DEF'
SYS_P14654 SYS_SUBP14652 'ABC'
SYS_P14654 SYS_SUBP14653 'DEF'
SYS_P14658 SYS_SUBP14655 'ABC'
SYS_P14658 SYS_SUBP14656 'DEF'
SYS_P14658 SYS_SUBP14657 'XYZ'