I have a log table with a lot of information.
I would like to partition it into two: first part is the logs from the past month, since they are commonly viewed. Second part is the logs from the rest of the year (Compressed).
My problem is that all the examples of partitions where "up until 1/1/2013", "more recent than 1/1/2013" - That is with fixed dates...
What I am looking for/expecting is a way to define a partition on the last month, so that when the day changes, the logs from 30 days ago, are "automatically" transferred to the compressed partition.
I guess I can create another table which is completley compressed and move info using JOBS, but I was hoping for a built-in solution.
Thank you.
I think you want interval partitions based on a date. This will automatically generate the partitions for you. For example, monthly partitions would be:
create table test_data (
created_date DATE default sysdate not null,
store_id NUMBER,
inventory_id NUMBER,
qty_sold NUMBER
)
PARTITION BY RANGE (created_date)
INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
(
PARTITION part_01 values LESS THAN (TO_DATE('20130101','YYYYMMDD'))
)
As data is inserted, Oracle will put into the proper partition or create one if needed. The partition names will be a bit cryptic (SYS_xxxx), but you can use the "partition for" clause to grab only the month you want. For example:
select * from test_data partition for (to_date('20130101', 'YYYYMMDD'))
It is not possible to automatically transfer data to a compressed partition. You can, however, schedule a simple job to compress last month's partition at the beginning of every month with this statement:
ALTER TABLE some_table
MOVE PARTITION FOR (add_months(trunc(SYSDATE), -1)
COMPRESS;
If you wanted to stay with only two partitions: current month and archive for all past transactions you could also merge partitions with ALTER TABLE MERGE PARTITIONS, but as far as I'm concerned it would rebuild the whole archive partition, so I would discourage doing so and stay with storing each month in its separate partition.
Related
I want to create partitions dynamicaly that depends on sysdate, like:
create table partition_interval_demo
( id number(6)
, val varchar2(50)
, date_create date)
partition by range (sale_date) interval (interval '1' day)
( partition p1 values less than ((sysdate-90)));
but I have error like
"ORA-14019: partition bound element must be one of: string, datetime or interval literal, number, or MAXVALUE"
so what can I do here?
You can't do that. Following the documentation
Creating a Range-Partitioned Table
The PARTITION BY RANGE clause of the CREATE TABLE statement specifies
that the table or index is to be range-partitioned. The PARTITION
clauses identify the individual partition ranges, and the optional
subclauses of a PARTITION clause can specify physical and other
attributes specific to a partition segment. If not overridden at the
partition level, partitions inherit the attributes of their underlying
table.
The VALUES LESS THAN clause determines the partition bound: rows with
partitioning key values that compare less than the ordered list of
values specified by the clause are stored in the partition. Each
partition is given a name (sales_q1, sales_q2, ...), and each
partition is contained in a separate tablespace (tsa, tsb, ...).
That means that the range is determined for the key value. sysdate is a function, therefore it is not a value until it is executed. It can't be part of a range partition key value.
Keep in mind the purpose or partitioning. The idea is to split a big segment ( table ) into smaller ones ( partitions ) using a key column. In your case, you want to store in one partition the rows where the key column is less than sysdate-90, which means that today the row will go to one partition, but in one day if the condition sysdate-90 is met, the row should be moved to the other partition.
If you want to do this anyway, although I don't recommend it, you will have to develop your own maintenance process:
The table has keys for each date ( 3 months of partitions for each date)
The Partition others will get everything else, so anything that has no match with any of the existing partition keys.
Every day you will have to retrieve the records that are meeting the condition sysdate-90 and move them to the others partition.
For doing that you need to enable row movement in your table: alter table xx enable row movement
Hope it clarifies
You can use
partition p1 values less than (DATE '2021-01-01')
If your application adds data from let's say 2021-08-01, then Oracle will create single new partition for 2021-08-01, it can deal with gaps or missing days.
If you use SEGMENT CREATION DEFERRED clause (maybe meanwhile the default) then the initial partition does not even consume any space on your disk.
I'm trying to figure the best possible way to determine the partition strategy in Oracle 12c (12.2.0.1.0)
This post
is almost identical to my requirements. However, I want to know the best possible way to implement in Oracle 12c (12.2.0.1.0) version.
Here is my question:
We have four (4) distinct programs for which the bills are submitted in our system.
The approx volume of bills submitted per year is as follows:
Program_1 ~ 3M per year
Program_2 ~ 1M per year
Program_3 ~ 500K per year
Program_4 ~ 100K per year
My initial thought process is to create PARTITION BY LIST (PROGRAM) AND SUBPARTITION BY RANGE (BILL_SUBMISSION_DATE).
I would like to use oracle interval feature for SUBPARTITION, would like to know if there are any limitations with this approach.
Your approach of partitioning by PROGRAM and sub-partitioning by BILL_SUBMISSTION_DATE sounds good.
I have not tested performance differences (I imagine they would be negligible), but for coding the INTERVAL option makes querying and maintenance easier in my opinion.
For the following example the table partition clause I used was:
partition by range (INVOICE_MONTH) interval (numtoyminterval(1, 'MONTH'))
Example query, using old style partition names, query a partition for an invoice for April 2012, assuming I created a partition named INV201204 for those invoices for that month:
select * from MARK_INV_HDR
partition ('INV201204');
And the same query, using INTERVAL automatically generated partitions:
select * from MARK_INV_HDR
where invoice_month = to_date('2012-04', 'yyyy-mm');
The advantage of the later query is I don't have to know the naming convention for the partitions.
To drop the oldest partition, one query and one DDL:
select to_char(min(invoice_month), 'dd-Mon-yyyy') as min_inv_dt from MARK_INV_HDR;
MIN_INV_DT
-----------
01-Apr-2012
alter table mark_inv_hdr
drop partition for (TO_DATE('01-Apr-2012', 'dd-Mon-yyyy'))
update global indexes;
EDIT:
Update: I forgot that you cannot use the INTERVAL clause on a sub-partition; thanks to user12283435 for the reminder. In looking more closely at the question, it appears that there is probably no need to partition on PROGRAM, so just a single partition by range on BILL_SUBMISSION_DATE with the INTERVAL clause should work fine.
When you have a small set of values like you do for PROGRAM, no obvious reason to partition on it. The typical example of partitioning by list given in Oracle documentation is list of regions, for a global call center, so that you can do batch reports and maintenance on certain regions after business hours, etc. You can have a global bit-mapped index on PROGRAM, if you don't do many updates, if you query criteria frequently includes just one PROGRAM. (Updating a column with a bit-mapped index will briefly lock the table.)
We have a table with partitions. It also has an overflow partition (max partition) which sorts of acts as a catch-all for records which do not match the partition criteria. The idea was to create the partitions ahead of time so the records never end up in the max_partition. However for one table, this was missed out, so all the records ended up in that single partition.
Now most of these records are not used anymore so they can be deleted. However our approach is to drop the partitions when its too old. This cannot be done in this case. Is there an easy way to handle the purge?
Maybe its an idea to create the partitions now and move the records to them and then drop the partition now, but however it seems like its going to be very poor in performance. The other option was to create a temp table where a subset of records are moved and deleted from there, but again moving the records individually seems time consuming. This table has around 5 million records.
Which would be the best way forward, performance wise. We could manage a little downtime but not much.
We use Oracle 11g.
The table creation script looks something like this:
CREATE TABLE "TRANSACTIONS"
("year" number(4,0) NOT NULL ENABLE)
PARTITION BY RANGE ("year")
(PARTITION "P_OLD" VALUES LESS THAN (2010),
PARTITION "P_2011" VALUES LESS THAN (2011),
...
PARTITION "P_MAX" VALUES LESS THAN (MAXVALUE));
There is no need to drop the partition, you can purge it.
alter table TRANSACTIONS TRUNCATE PARTITION P_MAX UPDATE INDEXES;
or if you prefer, you can also delete the rows:
delete from TRANSACTIONS PARTITION (P_MAX);
You may use INTERVAL partition to make it simpler (actually I don't understand your question):
CREATE TABLE TRANSACTIONS (
...
TRANSACTION_DATE TIMESTAMP(0) NOT NULL
)
PARTITION BY RANGE (TRANSACTION_DATE) INTERVAL (INTERVAL '12' MONTH)
(PARTITION P_OLD VALUES LESS THAN (TIMESTAMP '2000-01-01 00:00:00' ) )
ENABLE ROW MOVEMENT;
I have daily transactions with up to 5-10 GB of data per day. In my view it makes more sense to partition by month..
Here is an example:
My Table has the following columns:
TRANSACTION_DATE TIMESTAMP -- transaction date
TRANSACTION_AMOUNT INTEGER - transaction amount
DWH_PARTITION STRING -- technical field that goes into PARTITIONED BY section
Now I want to query for the amount of transactions between January 15st 2015 and November 15th 2015.
My query would be
select sum(TRANSACTION_AMOUNT) from TEST where TRANSACTION_DATE >= CAST('2015-01-15' as timestamp) AND TRANSACTION_DATE < CAST('2015-11-15' as timestamp)
This query returns correct data but it does full table scan while I would like it to just use partitions 2015-01, 2015-02, .... 2015-11.
To do so I need to specify manually which partitions should I use so the query would be as follows:
select sum(TRANSACTION_AMOUNT) from TEST where TRANSACTION_DATE >= CAST('2015-01-15' as timestamp) AND TRANSACTION_DATE < CAST('2015-11-15' as timestamp) and DWH_PARTITION in ('2015-01',.........'2015-11');
Because we cannot partition by timestamp business analyst would have to know the exact partitioning pattern (whether given table is partitioned by month, day, etc.).
Please also note that information about dates need to be specified two times: one for transaction date and then for partitions.
Do you know some partitioning methods that can help to avoid having to specify the same information twice and release the user from having to know partitioning patters of all the tables they need to query?
It can be only achieved by range partitioning and currently it is not supported. Probably UDF might help, but 100% not sure.
We have solved that problem by providing simple web interface where user can choose the table, filter columns and under the covers application is intelligent enough to generate the query leveraging partition pruning.
I have a table with a huge volume of data. I need partitioning to be done on a daily basis automatically. I need the name of the partition to be the date of sysdate. How can I go about doing this?
There is currently (11gR2) no way to specify a name for the auto-generated partitions in an interval-partitioned table. See Common Questions On Interval Partitioning [ID 1479115.1] (Oracle support account required):
What will be the names of the automatically created interval partitions?
[...] Currently it is not possible to specify a mask or template for partition names, but the system generated name can be renamed [...]
You're also restricted to a partition key column which must be of type DATE or NUMBER, and a few other things (see that note).
You can follow the example in the Creating Partitions documentation for the syntax:
create table foo (date_created date, ...)
partition by range(date_created)
interval(numtodsinterval(1, 'DAY'))
(partition one values less than (to_date('01012013', 'DDMMYYYY')));
With the above, a new partition will be created whenever you insert a row with a date value this year or later. New partitions will not be created for dates before 2013.
To work around the partition name issue (if necessary at all), you could rename the partitions based on HIGH_VALUE in USER_TAB_PARTITIONS, although that doesn't sound very nice.
Another option is to not rename them at all and use this syntax when you want to query a specific partition:
select *
from foo
partition for (<the day you're interested in>);
See for example: Oracle Interval Partitioning Tips.