We have a table with partitions. It also has an overflow partition (max partition) which sorts of acts as a catch-all for records which do not match the partition criteria. The idea was to create the partitions ahead of time so the records never end up in the max_partition. However for one table, this was missed out, so all the records ended up in that single partition.
Now most of these records are not used anymore so they can be deleted. However our approach is to drop the partitions when its too old. This cannot be done in this case. Is there an easy way to handle the purge?
Maybe its an idea to create the partitions now and move the records to them and then drop the partition now, but however it seems like its going to be very poor in performance. The other option was to create a temp table where a subset of records are moved and deleted from there, but again moving the records individually seems time consuming. This table has around 5 million records.
Which would be the best way forward, performance wise. We could manage a little downtime but not much.
We use Oracle 11g.
The table creation script looks something like this:
CREATE TABLE "TRANSACTIONS"
("year" number(4,0) NOT NULL ENABLE)
PARTITION BY RANGE ("year")
(PARTITION "P_OLD" VALUES LESS THAN (2010),
PARTITION "P_2011" VALUES LESS THAN (2011),
...
PARTITION "P_MAX" VALUES LESS THAN (MAXVALUE));
There is no need to drop the partition, you can purge it.
alter table TRANSACTIONS TRUNCATE PARTITION P_MAX UPDATE INDEXES;
or if you prefer, you can also delete the rows:
delete from TRANSACTIONS PARTITION (P_MAX);
You may use INTERVAL partition to make it simpler (actually I don't understand your question):
CREATE TABLE TRANSACTIONS (
...
TRANSACTION_DATE TIMESTAMP(0) NOT NULL
)
PARTITION BY RANGE (TRANSACTION_DATE) INTERVAL (INTERVAL '12' MONTH)
(PARTITION P_OLD VALUES LESS THAN (TIMESTAMP '2000-01-01 00:00:00' ) )
ENABLE ROW MOVEMENT;
Related
I have table partitioned on a column(rcrd_expry_ts) of date type. We are updating this rcrd_expry_ts weekly by another job. We noticed the update query is taking quite longer time (1 to 1.5 min) even for few rows and I think longer time is taken for actually moving data internally to different partitioned. There can be a million of rows eligible to update rcrd_expry_ts by our weekly job.
CREATE TABLE tbl_parent
(
"parentId" NUMBER NOT NULL ENABLE,
"RCRD_DLT_TSTP" timestamp default timestamp '9999-01-01 00:00:00' NOT NULL
)
PARTITION BY RANGE ("RCRD_DLT_TSTP") INTERVAL (NUMTOYMINTERVAL('1','MONTH')) (PARTITION "P1" VALUES LESS THAN (TO_DATE('2010-01-01 00:00:00', 'YYYY-MM-DD HH24:MI:SS')));
CREATE TABLE tbl_child
(
"foreign_id" NUMBER NOT NULL ENABLE,
"id" NUMBER NOT NULL ENABLE,
constraint fk_id foreign key("foreign_id") references
tbl_parent("parentId")
)partition by reference (fk_id);
I am updating RCRD_DLT_TSTP in parent table from some another job (using simple update query) but I noticed that it took around 1 to 1.5 min to execute, probably due to creating partition and move data into corresponding partition. Is there any better way to achieve this in Oracle
The table has a referenced partitioned child. So any rows moving partition in the parent will have to be cascaded to the child table too.
This means you could be moving substantially more rows that the "few rows" that change in the parent.
It's also worth checking if the update can identify the rows it needs to change faster too.
You can do this by getting the plan for the update statement like this:
update /*+ gather_plan_statistics */ <your update statement>;
select *
from table(dbms_xplan.display_cursor( format => 'ALLSTATS LAST' ));
This will give you the plan for the update with its run time stats. This will help in identifying if there are any indexes you can create to improve performance.
Is there any better way to achieve this in Oracle
This is a question that needs to be answered in the larger context. You may well be able to make this process faster by unpartitioning the table and using indexes to identify the rows to change.
But this affects all the other statements that access this table. To what extent do they benefit from partitioning? If the answer is substantially, is it worth making this process faster at the expense of these others? What trade-offs are you willing to make here?
I want to create partitions dynamicaly that depends on sysdate, like:
create table partition_interval_demo
( id number(6)
, val varchar2(50)
, date_create date)
partition by range (sale_date) interval (interval '1' day)
( partition p1 values less than ((sysdate-90)));
but I have error like
"ORA-14019: partition bound element must be one of: string, datetime or interval literal, number, or MAXVALUE"
so what can I do here?
You can't do that. Following the documentation
Creating a Range-Partitioned Table
The PARTITION BY RANGE clause of the CREATE TABLE statement specifies
that the table or index is to be range-partitioned. The PARTITION
clauses identify the individual partition ranges, and the optional
subclauses of a PARTITION clause can specify physical and other
attributes specific to a partition segment. If not overridden at the
partition level, partitions inherit the attributes of their underlying
table.
The VALUES LESS THAN clause determines the partition bound: rows with
partitioning key values that compare less than the ordered list of
values specified by the clause are stored in the partition. Each
partition is given a name (sales_q1, sales_q2, ...), and each
partition is contained in a separate tablespace (tsa, tsb, ...).
That means that the range is determined for the key value. sysdate is a function, therefore it is not a value until it is executed. It can't be part of a range partition key value.
Keep in mind the purpose or partitioning. The idea is to split a big segment ( table ) into smaller ones ( partitions ) using a key column. In your case, you want to store in one partition the rows where the key column is less than sysdate-90, which means that today the row will go to one partition, but in one day if the condition sysdate-90 is met, the row should be moved to the other partition.
If you want to do this anyway, although I don't recommend it, you will have to develop your own maintenance process:
The table has keys for each date ( 3 months of partitions for each date)
The Partition others will get everything else, so anything that has no match with any of the existing partition keys.
Every day you will have to retrieve the records that are meeting the condition sysdate-90 and move them to the others partition.
For doing that you need to enable row movement in your table: alter table xx enable row movement
Hope it clarifies
You can use
partition p1 values less than (DATE '2021-01-01')
If your application adds data from let's say 2021-08-01, then Oracle will create single new partition for 2021-08-01, it can deal with gaps or missing days.
If you use SEGMENT CREATION DEFERRED clause (maybe meanwhile the default) then the initial partition does not even consume any space on your disk.
Question 1. Is the data not insert to the specified partition successfully?
As I need to delete the specified partition, so I need make sure the insert into specified partition must successfully although the num_rows is 0 in user_tab_partitions.
Question 2. I have googled some ideas, some say I need to analyze the table.
why need to analyze?
Question 3.
if I analyze the table, will the performance be different compared to not analyzing it?
PARTITION BY RANGE ( CYCLE_MTH ) INTERVAL ( numtoyminterval(1,'MONTH') )
(
PARTITION per_limra_p004
VALUES LESS THAN ( TO_DATE('01-11-2015','DD-MM-YYYY') ),
PARTITION per_limra_p003
VALUES LESS THAN ( TO_DATE('01-12-2015','DD-MM-YYYY') ),
PARTITION per_limra_p002
VALUES LESS THAN ( TO_DATE('01-01-2016','DD-MM-YYYY') ),
Oracle doesn't constantly update its statistics about tables and partitions, as this would be too heavy. Once you analyze the table, the statistics (e.g., those in user_tab_partitions) will be updated. Since Oracle now has different statistics, the optimizer may very well create different execution plans for your queries, and thus affect performance.
NUM_ROWS shows the number of rows based on last analyze, so you should not rely on it. I suggest to run a count on it, e.g.
SELECT COUNT(*)
FROM your_table PARTITION FOR (DATE '2016-04-01')
or
SELECT COUNT(ROWNUM)
FROM your_table PARTITION FOR (DATE '2016-04-01')
WHERE ROWNUM <= 1
It should be much faster than DBMS_STATS.GATHER_TABLE_STATS (provided you have any index on this table)
Based on the partition information you've added to your question, the answer is clear. The reason that
SELECT COUNT() FROM per_limra PARTITION FOR (DATE '2015-06-01')
and
SELECT COUNT() FROM per_limra PARTITION FOR (DATE '2015-05-01')
both return 1 is that they're both looking at the same partition. Your three partitions are
PARTITION per_limra_p004
VALUES LESS THAN ( TO_DATE('01-11-2015','DD-MM-YYYY'), i.e. less than 01-Nov-2015
PARTITION per_limra_p003
VALUES LESS THAN ( TO_DATE('01-12-2015','DD-MM-YYYY'), i.e. less than 01-Dec-2015
PARTITION per_limra_p002
VALUES LESS THAN ( TO_DATE('01-01-2016','DD-MM-YYYY'), i.e. less than 01-Jan-2016
The dates in your queries are DATE '2015-06-01', or 01-Jun-2015, and DATE '2015-05-01', or 01-May-2015, both of which fall into the range for partition per_limra_p004, which contains data for all dates prior to 01-Nov-2015. Thus, both queries return data from the same partition, which is why they return the same value.
Best of luck.
I'm working with an application that has a large amount of outdated data clogging up a table in my databank. Ideally, I'd want to delete all entries in the table whose reference date is too old:
delete outdatedTable where referenceDate < :deletionCutoffDate
If this statement were to be run, it would take ages to complete, so I'd rather break it up into chunks with the following:
delete outdatedTable where referenceData < :deletionCutoffDate and rownum <= 10000
In testing, this works suprisingly slowly. The following query, however, runs dramatically faster:
delete outdatedTable where rownum <= 10000
I've been reading through multiple blogs and similar questions on StackOverflow, but I haven't yet found a straightforward description of how/whether using rownum affects the Oracle optimizer when there are other Where clauses in the query. In my case, it seems to me as if Oracle checks
referenceData < :deletionCutoffDate
on every single row, executes a massive Select on all matching rows, and only then filters out the top 10000 rows to return. Is this in fact the case? If so, is there any clever way to make Oracle stop checking the Where clause as soon as it's found enough matching rows?
How about a different approach without so much DML on the table. As a permanent solution for future you could go for table partitioning.
Create a new table with required partition(s).
Move ONLY the required rows from your existing table to the new partitioned table.
Once the new table is populated, add the required constraints and indexes.
Drop the old table.
In future, you would just need to DROP the old partitions.
CTAS(create table as select) is another way, however, if you want to have a new table with partition, you would have to go for exchange partition concept.
First of all, you should read about SQL statement's execution plan and learn how to explain in. It will help you to find answers on such questions.
Generally, one single delete is more effective than several chunked. It's main disadvantage is extremal using of undo tablespace.
If you wish to delete most rows of table, much faster way usially a trick:
create table new_table as select * from old_table where date >= :date_limit;
drop table old_table;
rename table new_table to old_table;
... recreate indexes and other stuff ...
If you wish to do it more than once, partitioning is a much better way. If table partitioned by date, you can select actual date quickly and you can drop partion with outdated data in milliseconds.
At last, paritioning if a way to dismiss 'deleting outdated records' at all. Sometimes we need old data, and it's sad if we delete it by own hands. With paritioning you can archive outdated partitions outside of the database, but connects them when you need to access old data.
This is an old request, but I'd like to show another approach (also using partitions).
Depending on what you consider old, you could create corresponding partitions (optimally exactly two; one current, one old; but you could just as well make more), e.g.:
PARTITION BY LIST ( mod(referenceDate,2) )
(
PARTITION year_odd VALUES (1),
PARTITION year_even VALUES (0)
);
This could as well be months (Jan, Feb, ... Dec), decades (XX0X, XX1X, ... XX9X), half years (first_half, second_half), etc. Anything circular.
Then whenever you want to get rid of old data, truncate:
ALTER TABLE mytable TRUNCATE PARTITION year_even;
delete from your_table
where PK not in
(select PK from your_table where rounum<=...) -- these records you want to leave
I have a log table with a lot of information.
I would like to partition it into two: first part is the logs from the past month, since they are commonly viewed. Second part is the logs from the rest of the year (Compressed).
My problem is that all the examples of partitions where "up until 1/1/2013", "more recent than 1/1/2013" - That is with fixed dates...
What I am looking for/expecting is a way to define a partition on the last month, so that when the day changes, the logs from 30 days ago, are "automatically" transferred to the compressed partition.
I guess I can create another table which is completley compressed and move info using JOBS, but I was hoping for a built-in solution.
Thank you.
I think you want interval partitions based on a date. This will automatically generate the partitions for you. For example, monthly partitions would be:
create table test_data (
created_date DATE default sysdate not null,
store_id NUMBER,
inventory_id NUMBER,
qty_sold NUMBER
)
PARTITION BY RANGE (created_date)
INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
(
PARTITION part_01 values LESS THAN (TO_DATE('20130101','YYYYMMDD'))
)
As data is inserted, Oracle will put into the proper partition or create one if needed. The partition names will be a bit cryptic (SYS_xxxx), but you can use the "partition for" clause to grab only the month you want. For example:
select * from test_data partition for (to_date('20130101', 'YYYYMMDD'))
It is not possible to automatically transfer data to a compressed partition. You can, however, schedule a simple job to compress last month's partition at the beginning of every month with this statement:
ALTER TABLE some_table
MOVE PARTITION FOR (add_months(trunc(SYSDATE), -1)
COMPRESS;
If you wanted to stay with only two partitions: current month and archive for all past transactions you could also merge partitions with ALTER TABLE MERGE PARTITIONS, but as far as I'm concerned it would rebuild the whole archive partition, so I would discourage doing so and stay with storing each month in its separate partition.