I'm in the process of building a database storage for my app consisting on a single table with a huge data volume (hundreds of millions of records). I'm planning on having an index on the date field, since I'll be doing a batch recovery of all the records in a given period of time every now and then (for example, retrieving all records for the following day, at midnight).
Since the number of records is huge and performance is an important concern in this system, I would like to know if there is a way I can dynamically partition my table so that I can retrieve the records faster, creating and truncating partitions as they are no longer needed. For example, how would I go about creating a partition for the following day and populating it with the rest of the data after I'm done processing today's records?
In 11g we can define INTERVAL partitions, and Oracle will automatically create new partitions when it gets new records whose keys don't fit in any of the existing ranges. This is a very cool feature. Find out more.
One thing to bear in mind is that Partitioning is a chargeable extra on top of the Enterprise Edition license. So it is not cheap to use.
you can automate the process of creating or truncating partitions through the use of dynamic SQL. You would write procedures with either EXECUTE IMMEDIATE or DBMS_SQL and you would schedule them with DBMS_JOB or DBMS_SCHEDULER (DBMS_SCHEDULER is a 10g feature and is more versatile than DBMS_JOB).
You probably want to build the partition statements manually first and automate the process later when you are confident with the DDL. You will find all the synthax in the documentation for the ALTER TABLE statement.
Here is a sample of something I came up with for creating partitions using SYSDATE and an offset. I had to create replacement parameters with a string concatenated to the SYSDATE:
COLUMN temp_var new_value partition_name_01;
SELECT 'TABLE_NAME_' || TO_CHAR(SYSDATE - 3, 'YYYYMMDD') AS temp_var FROM dual;
COLUMN temp_var new_value partition_date_01;
SELECT TO_CHAR(SYSDATE - 3, 'SYYYY-MM-DD') || ' 00:00:00' AS temp_var FROM dual;
COLUMN temp_var new_value partition_name_02;
SELECT 'TABLE_NAME_' || TO_CHAR(SYSDATE - 2, 'YYYYMMDD') AS temp_var FROM dual;
COLUMN temp_var new_value partition_date_02;
SELECT TO_CHAR(SYSDATE - 2, 'SYYYY-MM-DD') || ' 00:00:00' AS temp_var FROM dual;
COLUMN temp_var new_value partition_name_03;
SELECT 'TABLE_NAME_' || TO_CHAR(SYSDATE - 1, 'YYYYMMDD') AS temp_var FROM dual;
COLUMN temp_var new_value partition_date_03;
SELECT TO_CHAR(SYSDATE - 1, 'SYYYY-MM-DD') || ' 00:00:00' AS temp_var FROM dual;
COLUMN temp_var new_value partition_name_04;
SELECT 'TABLE_NAME_' || TO_CHAR(SYSDATE, 'YYYYMMDD') AS temp_var FROM dual;
COLUMN temp_var new_value partition_date_04;
SELECT TO_CHAR(SYSDATE, 'SYYYY-MM-DD') || ' 00:00:00' AS temp_var FROM dual;
CREATE TABLE TABLE_NAME
(
SEQ_NO NUMBER NOT NULL,
INSERT_DATE DATE NOT NULL,
FIRST_NAME VARCHAR2 (256 BYTE),
LAST_NAME VARCHAR2 (256 BYTE),
ID_NUM NUMBER,
ID_STATUS NUMBER
)
PARTITION BY RANGE
(INSERT_DATE)
SUBPARTITION BY LIST
(ID_STATUS)
SUBPARTITION TEMPLATE (
SUBPARTITION SP1 VALUES (0) TABLESPACE &tblspce,
SUBPARTITION SP2 VALUES (1) TABLESPACE &tblspce,
SUBPARTITION SP3 VALUES (2) TABLESPACE &tblspce)
(
PARTITION &partition_name_01
VALUES LESS THAN
(TO_DATE ('&partition_date_01',
'SYYYY-MM-DD HH24:MI:SS',
'NLS_CALENDAR=GREGORIAN')),
PARTITION &partition_name_02
VALUES LESS THAN
(TO_DATE ('&partition_date_02',
'SYYYY-MM-DD HH24:MI:SS',
'NLS_CALENDAR=GREGORIAN')),
PARTITION &partition_name_03
VALUES LESS THAN
(TO_DATE ('&partition_date_03',
'SYYYY-MM-DD HH24:MI:SS',
'NLS_CALENDAR=GREGORIAN')),
sysdate
PARTITION &partition_name_04
VALUES LESS THAN
(TO_DATE ('&partition_date_04',
'SYYYY-MM-DD HH24:MI:SS',
'NLS_CALENDAR=GREGORIAN')))
ENABLE ROW MOVEMENT;
There's a product that takes care of it automatically. PartitionManager for Oracle provides automatic partition management, based on the organization retention, including purging and archiving old data, statistics copy etc. You can try it out at http://www.xyrosoft.com
Related
I create table:
create table big_table(
bt_id number primary key,
bt_date date,
bt_value varchar2(20)
)
Then I wnat partition this table (code abbreviated):
alter table big_table modify
partition by range (bt_date)
interval(numtoyminterval(1, 'MONTH'))
subpartition by hash (bt_id)
(
partition nn_st_p1 values less than (to_date(' 2019-05-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
subpartitions 4
store in (ipr_tbl),
)online
Error message:
17:20:39 line 1: ORA-14006: invalid partition name
I can't understand what is wrong with my partition name?
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
Try something like this
CREATE TABLE big_table
(bt_id NUMBER PRIMARY KEY
, bt_date DATE
, bt_value VARCHAR2(20)
)
PARTITION BY RANGE (bt_date) INTERVAL (NUMTOYMINTERVAL(1,'MONTH'))
SUBPARTITION BY HASH (bt_id) SUBPARTITIONS 4
(PARTITION nn_st_p1 VALUES LESS THAN (TO_DATE('01-MAY-2019','dd-MON-yyyy'))
)
PARALLEL;
As we know that Interval partitioning is an extension of Range partitioning and it takes much of its syntax from
range partitioning.
From various sources on the net, I assume that interval partitioning creation query do have mandatory following clause:
VALUES LESS THAN (XXX)
But when we go for interval partitioning, is there any simpler way where we do not provide any VALUE LESS THAN CLAUSE.
I was searching for something similar like EQUAL TO (012019) where 012019 corresponds to the January month Interval
of 2019 year.
I have gone through following links for the help/understanding but couldn't find useful for my concern.
http://www.dba-oracle.com/t_interval_partitioning.htm
https://docs.oracle.com/database/121/VLDBG/GUID-C121EA1B-2725-4464-B2C9-EEDE0C3C95AB.htm
The code used by me is like as follows:
create table
pos_data (
start_date DATE,
store_id NUMBER,
inventory_id NUMBER(6),
qty_sold NUMBER(3)
)
PARTITION BY RANGE (start_date)
INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
(
PARTITION pos_data_p2 VALUES LESS THAN (TO_DATE('1-7-2007', 'DD-MM-YYYY')),
PARTITION pos_data_p3 VALUES LESS THAN (TO_DATE('1-8-2007', 'DD-MM-YYYY'))
);
From my search It looks like there is no other way apart from the one VALUE LESS THAN.
Please share if anyone have some understanding about someother approach for creating interval based partitioning.
Remainder: my concern is in BOLD above
I think what you are looking for is the partition extended name
PARTITION FOR(DATE'2019-01-01')
Actually the LESS THAN definition plays in interval partitioning close to zero role.
You use it only once while creating the table to define some lower bound of the data.
Here is an example to define a table containing data starting from the year 2019
create table pos_data (
start_date DATE,
store_id NUMBER
)
SEGMENT CREATION DEFERRED
PARTITION BY RANGE (start_date)
INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
(
PARTITION pos_data_init VALUES LESS THAN (TO_DATE('1-1-2019', 'DD-MM-YYYY'))
);
Note that the first partition is not an interval partition (INTERVAL = NO) and doesn't physically exists due to SEGMENT CREATION DEFERRED. It will also never contain any data, as you start with 2019 content.
select PARTITION_POSITION,PARTITION_NAME,INTERVAL,MIN_EXTENT, HIGH_VALUE
from user_tab_partitions where table_name = 'POS_DATA'
order by PARTITION_POSITION;
PARTITION_POSITION PARTITION_NAME INTERVAL MIN_EXTENT HIGH_VALUE
------------------ -------------- -------- ---------- -------------------------------
1 POS_DATA_INIT NO TO_DATE(' 2019-01-01 00:00:00',
New partitions are created on the fly e.g. while inserting new data, you don't need to specify LESS THAN
insert into pos_data(start_date,store_id) values(DATE'2019-01-01',1);
PARTITION_POSITION PARTITION_NAME INTERVAL MIN_EXTENT HIGH_VALUE
------------------ -------------- -------- ---------- -------------------------------
1 POS_DATA_INIT NO TO_DATE(' 2019-01-01 00:00:00',
2 SYS_P16713 YES 1 TO_DATE(' 2019-02-01 00:00:00',
While accessing the table you use the partition_extended_name, you may choose any date within the month to reference the partition.
select * from pos_data
partition for (date'2019-01-15');
Same syntax may be used for partition maintainance
alter table pos_data move partition for (date'2019-01-30') compress;
I have a table which holds data for 12 hours. Every 5 minutes, it keeps deleting data which is more than 12 hours old and adds new data. It has almost 15-20 million rows. I want to create partition by hour and also index the table on column(time_stamp), to make the row fetching faster.
I will obviously do interval or range partitioning, but found that interval partitioning doesn't work on indexed table. So please help me with the syntax so that oracle creates 12 partitions and automatically adds new one when new time_stamp data is added which is after first 12 hours. I have already got a procedure to delete oldest partition which i will use so that there is always 12 hours of data.
I am stating the columns below.
CustomerId,ApplicationId,Time_Stamp,Service
I have tried to come up with this, but don't know how it will create new partitions
CREATE TABLE local_table
(customerid VARCHAR2(30),
applicationid VARCHAR2(30),
time_stamp TIMESTAMP,
service VARCHAR2(30))
PARTITION BY RANGE(time_stamp)
(
PARTITION t1 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 00:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t2 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 01:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t3 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 02:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t4 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 03:00:00.0','YYYY-MM- DD HH24:MI:SS.ff')),
PARTITION t5 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 04:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t6 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 05:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t7 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 06:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t8 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 07:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t9 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 08:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t10 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 09:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t11 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 10:00:00.0','YYYY-MM-DD HH24:MI:SS.ff')),
PARTITION t12 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 11:00:00.0','YYYY-MM-DD HH24:MI:SS.ff'))
);
create index index_time_stamp on local_table(TIME_STAMP);
I am using- Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit
Create table with autopartitiong and LOCAL (partitioning) index.
The local_partitioned_index clauses let you specify that the index is partitioned on the same columns, with the same number of partitions and the same partition bounds as table. Oracle Database automatically maintains local index partitioning as the underlying table is repartitioned.
CREATE TABLE local_table
(customerid VARCHAR2(30),
applicationid VARCHAR2(30),
time_stamp TIMESTAMP,
service VARCHAR2(30))
PARTITION BY RANGE(time_stamp)
INTERVAL(NUMTODSINTERVAL(1, 'HOUR'))
(PARTITION t1 VALUES LESS THAN(TO_TIMESTAMP('2015-02-25 00:00:00.0','YYYY-MM-DD HH24:MI:SS.ff'))
);
CREATE INDEX index_time_stamp on local_table(TIME_STAMP) LOCAL;
SELECT *
FROM user_tab_partitions;
INSERT INTO local_table VALUES('1', 'a', sysdate, 'b');
SELECT *
FROM user_tab_partitions;
INSERT INTO local_table VALUES('2', 'c', sysdate + 1/1440, 'd');
SELECT *
FROM user_tab_partitions;
INSERT INTO local_table VALUES('3', 'e', sysdate + 1/24, 'f');
SELECT *
FROM user_tab_partitions;
The INTERVAL clause of the CREATE TABLE statement establishes interval
partitioning for the table. You must specify at least one range
partition using the PARTITION clause. The range partitioning key value
determines the high value of the range partitions, which is called the
transition point, and the database automatically creates interval
partitions for data beyond that transition point. The lower boundary
of every interval partition is the non-inclusive upper boundary of the
previous range or interval partition.
For example, if you create an interval partitioned table with monthly
intervals and the transition point at January 1, 2010, then the lower
boundary for the January 2010 interval is January 1, 2010. The lower
boundary for the July 2010 interval is July 1, 2010, regardless of
whether the June 2010 partition was previously created. Note, however,
that using a date where the high or low bound of the partition would
be out of the range set for storage causes an error. For example,
TO_DATE('9999-12-01', 'YYYY-MM-DD') causes the high bound to be
10000-01-01, which would not be storable if 10000 is out of the legal
range.
Some quick DRAFT for your second question about DROP PARTITION. Examine and debug before uncomment ALTER TABLE. You can create scheduler job for run this block of code every hour.
DECLARE
l_pt_cnt NUMBER;
l_pt_name VARCHAR2(100);
l_minrowid ROWID;
l_mindate TIMESTAMP;
BEGIN
-- get partition count
SELECT count(*)
INTO l_pt_cnt
FROM user_tab_partitions
WHERE table_name = 'LOCAL_TABLE';
IF l_pt_cnt > 12 THEN
SELECT min(time_stamp)
INTO l_mindate
FROM LOCAL_TABLE;
-- get ROWID with min date
SELECT min(rowid)
INTO l_minrowid
FROM LOCAL_TABLE
WHERE time_stamp = l_mindate;
-- get name of partition with row with min date
SELECT subobject_name
INTO l_pt_name
FROM LOCAL_TABLE
JOIN user_objects
ON dbms_rowid.rowid_object(LOCAL_TABLE.rowid) = user_objects.object_id
WHERE LOCAL_TABLE.rowid = l_minrowid;
DBMS_OUTPUT.put_line('ALTER TABLE LOCAL_TABLE DROP PARTITION ' || l_pt_name );
--EXECUTE IMMEDIATE 'ALTER TABLE LOCAL_TABLE DROP PARTITION ' || l_pt_name;
END IF;
END;
I want to create an Oracle table which contains 3-4 columns in which I want to insert just time in one of the columns.
First, I set the datatype of that column to varchar2 but then I am unable to do any operations on that time.
create table attendance(
ID varchar2(10),
EmpIntime varchar2(10) ,
EmpoutTime varchar2(10));
I want to insert time in 9:00AM format and calculate total time.
Here is DEMO
i want output like that
"emp_in - emp_out = total working hours"
Just make it with to_date like below
select to_date(emp_out,'HH:MIAM')-to_date(emp_in,'HH:MIAM') from attendance
made an sqlfiddle http://sqlfiddle.com/#!4/96975/8 with the hours difference between time
Try this one,this will give you time in hours,minutes and seconds :-
create table attend(ID int,emp_in varchar2(10) ,emp_out varchar2(10));
insert all
into attend values(1,'09:00AM','05:00PM')
into attend values(2,'09:30AM','05:00PM')
into attend values(3,'10:00AM','08:00PM')
select * from dual;
commit;
select outtime-intime time_difference
from(
select to_timestamp(EMP_IN, 'hh:mi am') intime ,
to_timestamp(EMP_OUT, 'hh:mi am') outtime
FROM attend);
how to insert (text) time: 09:44:02 and date:11/09/2007 to date filed in Oracle ?
i have field Tdate (date) and Ttime (date) in oracle
thanks in advance
Since a date is a point in time you should really only use one field:
insert into your_table (dt)
values (to_date ('11/09/2007 09:44:02', 'dd/mm/yyyy hh24:mi:ss'));
Since your model contains two columns, you could use:
insert into your_table (Tdate, Ttime)
values (to_date('11/09/2007', 'dd/mm/yyyy'),
to_date ('09:44:02', 'hh24:mi:ss'));
Note however that by default your Ttime column will contain date information which may be false/irrelevant since you can not store only the time component:
SQL> select to_char(tdate), to_char(ttime) from your_table;
TO_CHAR(TDATE) TO_CHAR(TTIME)
------------------- -------------------
11/09/2007 00:00:00 01/08/2011 09:44:02
INSERT INTO <table>
(date_column)
VALUES
(TO_DATE(tdate||' '||ttime, 'DD/MM/YYYY HH24:MI:SS'))
/
If it is in PL/SQL
DECLARE
v_date_field DATE;
BEGIN
v_date_field := TO_DATE(tdate||' '||ttime, 'DD/MM/YYYY HH24:MI:SS');
--
INSERT INTO <table>
(date_column)
VALUES
(v_date_field);
END;
/
Don't forget to commit afterwards :-)
insert into tablename(tdate)
values(to_date('11/09/2007 09:44:02','dd/mm/yyyy hh24:mi:ss'))