Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I have created an interval partitioned table with interval ( NUMTODSINTERVAL(1,'day') ).
As transaction data gets logged to this table on a daily basis, oracle automatically creates daily partitions.
There is a requirement to truncate the previous day's partition on a daily basis without affecting current day partition.
Since delete is producing fragmentation, I've been asked to go with truncate or drop.
Welcome to SO!
Please see below a basic example of truncating/dropping preceding partitions from an interval partitioned table, based on a date that you pass.
Note: You may need to fine tune this further to accommodate the various scenarios and edge cases based on specific data loading behaviors and requirements.
For illustration, I'm creating a sample table with interval partitioning as follows ...
create table testpart
( value_date date )
partition by range ( value_date )
interval ( NUMTODSINTERVAL(1,'day') )
( partition p1 values less than ( date '2020-01-01' ));
Checking default partition on the newly created table ..
select table_name, partition_name, interval from dba_tab_partitions where table_name = 'TESTPART';
|TABLE_NAME |PARTITION_NAME |INTERVAL |
|TESTPART |P1 |NO |
Inserting data current date (e.g. 16th April) and previous date (15th April) ..
insert into testpart select date '2020-04-15' from dual;
insert into testpart select date '2020-04-16' from dual;
commit;
Checking inserted data ..
select * from testpart;
| VALUE_DATE |
| 4/15/2020 |
| 4/16/2020 |
Checking the automatically created interval partitions along with the high_value converted to date format (explanations mentioned alongside in comments) ...
Note: I've adapted the logic from this post on SO
select
table_name,
partition_name,
interval,
to_date ( -- convert to date datatype
trim ( -- trim whitespaces
'''' from regexp_substr (
extractvalue ( -- return scalar value of the single text node viz., high_value
dbms_xmlgen.getxmltype ( -- converts the results of a SQL query into XML format
'select high_value from all_tab_partitions where table_name='''
|| table_name
|| ''' and partition_name = '''
|| partition_name
|| ''''),
'//text()'),
'''.*?''')), -- regex pattern matching to fetch value between the first set of quotes
'syyyy-mm-dd hh24:mi:ss')
high_value_in_date_format
FROM all_tab_partitions
WHERE table_name = 'TESTPART'
;
|TABLE_NAME |PARTITION_NAME |INTERVAL |HIGH_VALUE_IN_DATE_FORMAT |
|TESTPART |SYS_P9064429 |YES |4/17/2020 |
|TESTPART |SYS_P9064428 |YES |4/16/2020 |
|TESTPART |P1 |NO |1/1/2020 |
Creating a procedure to accept date and match that to high_value to identify immediately preceding partition in the specified table and truncating/dropping that partition ...
create or replace procedure int_part_housekeeping
( p_date date,
p_table_name varchar2
)
as
l_part_name varchar2(30);
begin
-- identifying partition based on high value in all_tab_partitions
select partition_name
into l_part_name
from all_tab_partitions
where table_name = p_table_name
and
to_date ( -- convert to date datatype
trim ( -- trim whitespaces
'''' from regexp_substr (
extractvalue ( -- return scalar value of the single text node
dbms_xmlgen.getxmltype ( -- converts the results of a SQL query into XML format
'select high_value from all_tab_partitions where table_name='''
|| table_name
|| ''' and partition_name = '''
|| partition_name
|| ''''),
'//text()'),
'''.*?''')), -- regex pattern matching to fetch value between the first set of quotes
'syyyy-mm-dd hh24:mi:ss') = p_date
;
-- truncating preceding partition
dbms_output.put_line('Trucating partition for preceding interval partition :' || l_part_name );
execute immediate 'ALTER TABLE ' || p_table_name || ' TRUNCATE PARTITION (' || l_part_name || ')';
-- dropping preceding partition (note: interval needs to be reset before and after the drop operation)
dbms_output.put_line('Dropping partition for preceding interval partition :' || l_part_name);
execute immediate 'ALTER TABLE ' || p_table_name || ' SET INTERVAL ()';
execute immediate 'ALTER TABLE ' || p_table_name || ' DROP PARTITION (' || l_part_name || ')';
execute immediate 'ALTER TABLE ' || p_table_name || ' SET INTERVAL ( NUMTODSINTERVAL(1,''day'') )';
exception
when others then
dbms_output.put_line(sqlerrm);
raise;
end;
Execute the procedure passing current date and table name ..
set serveroutput on;
begin
int_part_housekeeping(date'2020-04-16','TESTPART');
end;
Output:
Trucating partition for preceding interval partition :SYS_P9064428
Dropping partition for preceding interval partition :SYS_P9064428
PL/SQL procedure successfully completed.
Checking if the required partitions were dropped ...
|TABLE_NAME |PARTITION_NAME |INTERVAL |HIGH_VALUE_IN_DATE_FORMAT |
|TESTPART |SYS_P9064429 |NO |4/17/2020 |
|TESTPART |P1 |NO |1/1/2020 |
You may then execute this procedure on-demand or use dbms scheduler to run at specific times.
If this suits your requirement, feel free to accept and vote
Related
The column HIGH_VALUE has data similar to below, of type LONG, and always has the same length:
TIMESTAMP' 2019-01-30 00:00:00'
How can I convert it to a DATE type without using a function?
My overall goal is to create a result set which can then be used as an inner query for other aggregations. For example, I would like to be able to sum the number of rows over a year according to the date produced by converting the HIGH_VALUE column to a date.
I have only read permissions on this database and therefore cannot create functions. I've seen other solutions on StackOverflow and other sites, but they all require creating a function.
ALL_TAB_PARTITIONS is a standard built-in Oracle table and therefore I'm not including the table structure. In case that's an issue, please let me know and I will create an example table.
An example query and the data one row that results from this query follows. Note that I cannot create tables on this database so I will also need an a method that works without creating a temporary table.
Insert into EXPORT_TABLE (TABLE_OWNER,TABLE_NAME,PARTITION_NAME,HIGH_VALUE,NUM_ROWS)
VALUES ('TO','TN','SYS_P201709','TIMESTAMP'' 2019-01-30 00:00:00''',5053133);
SELECT TABLE_OWNER, TABLE_NAME, PARTITION_NAME, HIGH_VALUE, NUM_ROWS
from ALL_TAB_PARTITIONS;
If you are using Oracle 12c you could still use function but defined inline:
WITH FUNCTION with_function(p_id IN NUMBER) RETURN NUMBER IS
BEGIN
-- logic here
RETURN p_id;
END;
SELECT with_function(id)
FROM your_table
Related: WITH Clause Enhancements in Oracle Database 12c Release 1 (12.1)
For the conversion of a LONG type ( HIGH_VALUE ) to TIMESTAMP,
One option is to use dynamic sql and perform your insert through an anonymous block. No Procedure or function is required.
DECLARE
tstamp TIMESTAMP;
BEGIN
FOR rec IN ( SELECT table_owner,table_name,partition_name,high_value,num_rows
FROM all_tab_partitions
WHERE ROWNUM < 5
) LOOP
EXECUTE IMMEDIATE 'BEGIN :dt := '
|| rec.high_value
|| '; END;'
USING OUT tstamp; --assign the long to an external timestamp variable
INSERT INTO export_table (
table_owner,table_name,partition_name,high_value,num_rows
) VALUES (rec.table_owner,
rec.table_name, rec.partition_name, tstamp, rec.num_rows
);
END LOOP;
END;
/
AS #APC commented, There's also a solution using Pure SQL, which
uses a slightly complex Xml expression.
Combining the pure SQL solution from APC's comment with the enhancements in Oracle 12 to allow functions to be declared in WITH clauses and Kaushik Nayak's method of using EXECUTE IMMEDIATE to convert the string value to a date then you can get this:
Oracle Setup - Test Table & Data:
CREATE TABLE EXPORT_TABLE (
TABLE_OWNER VARCHAR2(30),
TABLE_NAME VARCHAR2(30),
PARTITION_NAME VARCHAR2(30),
HIGH_VALUE LONG,
NUM_ROWS INTEGER
);
INSERT INTO EXPORT_TABLE VALUES ( 'TO', 'TN', 'PN', 'TIMESTAMP ''2019-06-26 12:34:56''', 12345 );
Query:
WITH FUNCTION clobToDate( value IN CLOB ) RETURN DATE
IS
ts DATE;
BEGIN
EXECUTE IMMEDIATE 'BEGIN :ts := ' || value || '; END;' USING OUT ts;
RETURN ts;
END;
SELECT TABLE_OWNER,
TABLE_NAME,
PARTITION_NAME,
clobToDate(
EXTRACTVALUE(
dbms_xmlgen.getxmltype(
'SELECT high_value'
|| ' FROM EXPORT_TABLE'
|| ' WHERE TABLE_OWNER = ''' || t.table_owner || ''''
|| ' AND TABLE_NAME = ''' || t.table_name || ''''
|| ' AND PARTITION_NAME = ''' || t.partition_name || ''''
),
'//text()'
)
) AS HIGH_VALUE,
NUM_ROWS
FROM EXPORT_TABLE t;
Output:
TABLE_OWNER | TABLE_NAME | PARTITION_NAME | HIGH_VALUE | NUM_ROWS
:---------- | :--------- | :------------- | :------------------ | -------:
TO | TN | PN | 2019-06-26 12:34:56 | 12345
db<>fiddle here
Update: If you want to aggregate some columns then:
WITH FUNCTION clobToDate( value IN CLOB ) RETURN DATE
IS
ts DATE;
BEGIN
EXECUTE IMMEDIATE 'BEGIN :ts := ' || value || '; END;' USING OUT ts;
RETURN ts;
END;
SELECT table_owner,
table_name,
MAX( high_value ) AS max_high_value,
SUM( num_rows ) AS total_rows
FROM (
SELECT TABLE_OWNER,
TABLE_NAME,
PARTITION_NAME,
clobToDate(
EXTRACTVALUE(
dbms_xmlgen.getxmltype(
'SELECT high_value'
|| ' FROM EXPORT_TABLE'
|| ' WHERE TABLE_OWNER = ''' || t.table_owner || ''''
|| ' AND TABLE_NAME = ''' || t.table_name || ''''
|| ' AND PARTITION_NAME = ''' || t.partition_name || ''''
),
'//text()'
)
) AS HIGH_VALUE,
NUM_ROWS
FROM EXPORT_TABLE t
)
GROUP BY table_owner, table_name;
db<>fiddle here
PROCEDURE purge_partitions
(
p_owner IN VARCHAR2
,p_name IN VARCHAR2
,p_retention_period IN NUMBER
) IS
BEGIN
FOR partition_rec IN (SELECT partition_name
,high_value
FROM dba_tab_partitions
WHERE table_owner = p_owner
AND table_name = p_name)
LOOP
IF SYSDATE >= add_months(to_date(substr(partition_rec.high_value
,12
,19)
,'YYYY-MM-DD HH24:MI:SS')
,p_retention_period)
THEN
execute_immediate('ALTER TABLE ' || p_owner || '.' ||
p_name || ' DROP PARTITION ' ||
partition_rec.partition_name)
END IF;
END LOOP;
END purge_partitions;
Purge_Partitions procedure deals with dropping partitions based on
specific retention priods mentioned in a seperate Config Table. I am
now trying to enhance this functionality which will take care of
rebuilding global indexes of those partitioned tables. Not sure how to go about this, any help is highly appreciated.
Consider the update_index_clauses, which keeps the indexes valid.
See the documentation and consideration for global indexes here
in your case it will be:
alter table ttttt drop partition pppppp update global indexes;
Alternatively let the indices be invalidated in the DROP PARTITION and rebuild them with alter index xxx rebuild. You can get the list of indexes to rebuild from this query
select OWNER, INDEX_NAME
from all_indexes
where owner = 'ooo' and table_name = 'tttttt' and status = 'UNUSABLE';
I have this table as my main table (100M rows):
create table prova_log(
id_dispositive number,
type number,
date_verification date,
status number
)
partition by range (date_verification) interval (NUMTODSINTERVAL(3,'DAY'))
subpartition by list (type)
subpartition TEMPLATE (
SUBPARTITION type1 VALUES (1),
SUBPARTITION type2 VALUES (2),
SUBPARTITION type3 VALUES (3),
SUBPARTITION type4 VALUES (4)
)
(
partition p0816 values less than (to_date('01/09/2016','dd/mm/yyyy'))
);
And I want to make some sort of backup with older values, so I created this (0 rows):
create table prova_log_old (
id_dispositive number,
type number,
date_verification date,
status number
)
partition by range (date_verification) interval (NUMTODSINTERVAL(3,'DAY'))
subpartition by list (type)
subpartition TEMPLATE (
SUBPARTITION type1 VALUES (1),
SUBPARTITION type2 VALUES (2),
SUBPARTITION type3 VALUES (3),
SUBPARTITION type4 VALUES (4)
)
(
partition p_old values less than (to_date('01/09/2016','dd/mm/yyyy'))
);
So I want to move/copy/exchange (whatever term) old partitions (15 days+) to prova_log_old.
To do so I've created this job:
PROCEDURE move_data_from_huge_table
IS
-- This will move all data after 'vcountdaystokeepdata' days
vcountdaystokeepdata NUMBER := 15;
vcountdatainsidepartition NUMBER := 0;
BEGIN
FOR item IN
(SELECT *
FROM (SELECT partition_name,
TO_DATE
(TRIM
('''' FROM REGEXP_SUBSTR
(EXTRACTVALUE
(DBMS_XMLGEN.getxmltype
( 'select high_value from all_tab_partitions where table_name='''
|| table_name
|| ''' and table_owner = '''
|| table_owner
|| ''' and partition_name = '''
|| partition_name
|| ''''
),
'//text()'
),
'''.*?'''
)
),
'syyyy-mm-dd hh24:mi:ss'
) high_value
FROM all_tab_partitions
WHERE table_name = 'PROVA_LOG')
WHERE high_value < SYSDATE - vcountdaystokeepdata)
LOOP
EXECUTE IMMEDIATE 'alter table PROVA_LOG EXCHANGE PARTITION '
|| item.partition_name
|| ' with table PROVA_LOG_OLD';
EXECUTE IMMEDIATE 'select count(*) from PROVA_LOG partition ('
|| item.partition_name
|| ')'
INTO vcountdatainsidepartition;
IF vcountdatainsidepartition = 0
THEN
EXECUTE IMMEDIATE 'ALTER TABLE PROVA_LOG DROP PARTITION '
|| item.partition_name
|| '';
END IF;
END LOOP;
END;
But when I run the procedure I got
ORA-14292 partitioning type of table must match subpartitioning type of composite partition
I assume that I must have a partition in my backup table with the same name as my main partitioned table right?
How can I make this work?
I tried to add a partition to my backup table but without success..It's important to mention that all partition's name are random (oracle generates it).
I still don't understand why you want to move the partitions, anyway I have a solution.
First of all, you can address a partition either like
SELECT COUNT(*) FROM PROVA_LOG PARTITION (SYS_P7138);
or you can do it as
SELECT COUNT(*) FROM PROVA_LOG PARTITION FOR (TO_DATE('2016-10-01', 'YYYY-MM-DD'));
or if you prefer DATE literals
SELECT COUNT(*) FROM PROVA_LOG PARTITION FOR (DATE '2016-10-01');
An automatic solution for you problem could be this one:
DECLARE
CURSOR TabPartitions IS
SELECT TABLE_NAME, PARTITION_NAME, HIGH_VALUE
FROM USER_TAB_PARTITIONS
WHERE TABLE_NAME = 'PROVA_LOG'
ORDER BY 1,2;
ts DATE;
BEGIN
FOR aPart IN TabPartitions LOOP
EXECUTE IMMEDIATE 'BEGIN :ret := '||aPart.HIGH_VALUE||'; END;' USING OUT ts;
IF ts <> DATE '2016-09-10' AND ts < SYSDATE - 15 THEN
--EXECUTE IMMEDIATE 'INSERT INTO PROVA_LOG_OLD SELECT * FROM PROVA_LOG PARTITION FOR (DATE '''||TO_CHAR(ts, 'yyyy-mm-dd')||''')';
--EXECUTE IMMEDIATE 'ALTER TABLE PROVA_LOG DROP PARTITION FOR (DATE '''||TO_CHAR(ts, 'yyyy-mm-dd')||''') UPDATE GLOBAL INDEXES';
EXECUTE IMMEDIATE 'ALTER TABLE PROVA_LOG EXCHANGE PARTITION FOR (DATE '''||TO_CHAR(ts, 'yyyy-mm-dd')||''') WITH TABLE PROVA_LOG_OLD INCLUDING INDEXES';
END IF;
END LOOP;
END;
Your backup table must be like this:
CREATE TABLE prova_log_old (
id_dispositive NUMBER,
TYPE NUMBER,
date_verification DATE,
status NUMBER
)
PARTITION BY LIST (TYPE)
(
PARTITION type1 VALUES (1),
PARTITION type2 VALUES (2),
PARTITION type3 VALUES (3),
PARTITION type4 VALUES (4)
);
or no partitioning at all
CREATE TABLE prova_log_old (
id_dispositive NUMBER,
TYPE NUMBER,
date_verification DATE,
status NUMBER
);
You are doing it wrong. You exchange partition with all partitioned table not with it partition, just look one more at your code
EXECUTE IMMEDIATE 'alter table PROVA_LOG EXCHANGE PARTITION '
|| item.partition_name
|| ' with table PROVA_LOG_OLD';
In case of exchange partition you should do as follows
Create empty table without partition with same structure like PROVA_LOG but not partitioned.
Exchange partition in production table with new_table
Exchange partition in hist table with new_table
A table TMP has 5 partitions, namely P_1, P_2,....P_5.
I need to drop some partitions of TMP; the partitions to drop are derived by another query.
Ex:
ALTER TABLE TMP DROP PARTITIONS (SELECT ... From ... //expression to get partition names )
Let's say the SELECT statement returns P_1 & P_5. The part query of the ALTER statement above doesn't work. Is there any way to drop partitions with input from a SELECT statement?
You can use dynamic sql in anonymous pl/sql block;
Begin
for i in (select part_name from ... //expression to get partition names) loop
execute immediate 'ALTER TABLE TMP DROP PARTITION ' || i.part_name;
end loop;
end;
For dropping multiple partitions on a go then;
declare
v_part varchar(1000);
Begin
select LISTAGG(partition_name, ', ') WITHIN GROUP (ORDER BY partition_name DESC)
into v_part
from ... //expression to get partition names;
execute immediate 'ALTER TABLE TMP DROP PARTITION ' || v_part;
end;
You may use the following sql to generate DML for dropping multiple table partitions.
select 'ALTER TABLE ' || TABLE_OWNER || '.' || TABLE_NAME || ' DROP PARTITION ' || '"' || PARTITION_NAME || '";' from DBA_TAB_PARTITIONS
where TABLE_NAME='%YOUR_PATTERN%'order by PARTITION_NAME;
You'll need to use dynamic SQL. Something this:
begin
for prec in (SELECT ... From ... //expression to get partition names )
loop
execute immediate 'ALTER TABLE TMP DROP PARTITION '
|| prec.partition_name;
end loop;
end;
/
Clearly you need to have complete faith that your query will return only the partitions you want to drop. Or equivalent faith in your Backup & Recover plans :)
Alternatively you can use a similar approach to generate a drop script which you can review before you run it.
You have to use pl/sql block for dropping partitions from a table with select query. Use listagg for making a comma separated list.
DECLARE
var1 varchar2(50);
BEGIN
SELECT listagg(Partition_names) into var1 from table_name//expression to get partition names ;
execute immediate 'alter table tmp drop PARTITION'||var1 ;
END;
Example on listagg
select LISTAGG(partition_name,',') within group(order by table_name) as comma_list
from ALL_TAB_PARTITIONS where TABLE_owner='OWNER' AND TABLE_NAME='TABLE_NAME'
Maybe it's could somebody help.
This script drop all partitions for all partition tables for specific schema. I use it with clear DB with METADATA, for changing started (referencial) partition.
ALTER TABLE SCHEMA_1.TABLE_1
SET INTERVAL ();
ALTER TABLE SCHEMA_1.TABLE_2
SET INTERVAL ();
ALTER TABLE SCHEMA_1.TABLE_3
SET INTERVAL ();
set lines 100
set heading off
spool runme.sql
select 'ALTER TABLE ' || TABLE_OWNER || '.' || TABLE_NAME || ' DROP PARTITION ' || '"' || PARTITION_NAME || '";' from DBA_TAB_PARTITIONS
where
TABLE_OWNER='SCHEMA_1'
-- and TABLE_NAME='TABLE_%'
and PARTITION_NAME LIKE 'SYS_P%'
;
#runme
ALTER TABLE SCHEMA_1.TABLE_1
SET INTERVAL (NUMTOYMINTERVAL(1,'MONTH'));
ALTER TABLE SCHEMA_1.TABLE_1
SET INTERVAL (NUMTOYMINTERVAL(1,'MONTH'));
ALTER TABLE SCHEMA_1.TABLE_3
SET INTERVAL (NUMTOYMINTERVAL(1,'MONTH'));
Yes, the script is semi-manual, but I think it's more safety.
ALTER TABLE ... INTERVAL it's need for droping last partition.
Interval must be same that it was before
Following statement allows me to retrieve information about a table's partitions:
select table_name, partition_name, high_value from user_tab_partitions where table_name = 'T1';
problem is that, for some unknown reason, high_value column's values are expressed as:
TO_DATE(' 2015-01-01 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')
is it possible to retrieve them as date? Or at least do something like "eval" on that expression?
That field is defined as a long type, so depending on the partitioning scheme, it can hold various values (a string like 'ABC', a number 123, a string representation of a date, whatever).
Anyway, sounds like you want to get data from a particular partition, and you're trying to figure out the partition name from the metadata. If thats the case, you can simply use the partition for clause:
select * from my_partitioned_table partition for (to_date('20150801', 'YYYYMMDD'));
Which would select data from the partition housing that particular date (assuming your partitioning by date). This is particularly useful in interval partitioning, where Oracle assigns partition names like SYS_xxxx which seem arbitrary at best.
If you'd like to drop partitions older than a given date, its a bit more tricky. The above syntax is for selecting data, not DDL (alter table). To do that, you could do something like this (loosely tested):
Create a function to identify which partitions hold data with dates less than a given reference date:
create or replace function fn_partition_is_earlier(i_part_tab_name varchar2, i_partition_position number, i_ref_date in date)
return number
is
l_date_str varchar2(2000);
l_date date;
begin
execute immediate 'select high_value from all_tab_partitions where table_name = :tab and partition_position = :pos'
into l_date_str
using i_part_tab_name, i_partition_position;
execute immediate 'select ' || l_date_str || ' from dual' into l_date;
if (l_date < i_ref_date) then
return 1;
end if;
return 0;
end;
Use the function as follows:
with part_name as (
select partition_name
from (
select fn_partition_is_earlier(p.table_name, p.partition_position, to_date('20130501', 'YYYYMMDD')) should_drop_flag, p.*
from all_tab_partitions p
where table_name = 'MY_TAB'
)
where should_drop_flag = 1
)
select 'alter table MY_TAB drop partition ' || part_name.partition_name || ' update global indexes;'
from part_name;
The output would give you the script you'd have the DBAs run off hours.
Hope that helps.