I need to truncate a table that has around 40 million records in it. i guess it is ok to REUSE STORAGE for the table since new data will be inserted after the truncate.
I need to know
approximately how much time the command
TRUNCATE TABLE <tablename> REUSE STORAGE; would take.
Can this be done offline, so that the operations/ commands on the DB
don't get affected?
Based on "so that the operations/ commands don't get affected", it sounds like you'd like to replace the contents of a table with "good" values, without anyone seeing an empty table. If users/processes need to be able to continue making changes to the table while you switch out the contents, you'll need to research DBMS_REDEFINITION. If this is a read-only table (in other words, you can do this operation while nobody is inserting/updating/deleting its contents), there's a nice DIY approach that takes advantage of partition exchanging:
1) Create a table which has identical partitioning and indexing as original, and has the desired "good" rows.
2)
IF YOUR TABLE IS PARTITIONED:
CREATE TABLE <tablename>_TMP AS SELECT * FROM <tablename> WHERE 1=0;
-- Add any local indexes on <tablename> as standard indexes on <tablename_tmp>
BEGIN
FOR R IN (SELECT * FROM USER_TAB_PARTITIONS WHERE TABLE_NAME = <tablename>)
LOOP
EXECUTE IMMEDIATE
'INSERT INTO <tablename>_TMP'
||' SELECT * FROM <good_data_tablename> PARTITION ('||R.PARTITION_NAME||')';
EXECUTE IMMEDIATE
'ALTER TABLE <tablename>_TMP'
||' EXCHANGE PARTITION WITH TABLE <tablename> PARTITION ' || R.PARTITION_NAME
||' INCLUDING INDEXES WITHOUT VALIDATION UPDATE GLOBAL INDEXES';
EXECUTE IMMEDIATE
'TRUNCATE TABLE <tablename>_TMP';
END LOOP
END;
/
DROP TABLE <tablename>_TMP ;
IF YOUR TABLE IS NOT PARTITIONED:
CREATE TABLE <tablename>_TMP
PARTITION BY RANGE (<non-null-column>)
(PARTITION ALL_DATA values less than (maxvalue))
AS SELECT * FROM <good_data_tablename>;
-- Add any <tablename> indexes as local indexes on <tablename>_TMP
ALTER TABLE <tablename> EXCHANGE PARTITION WITH TABLE <tablename>_TMP INCLUDING INDEXES WITHOUT VALIDATION;
DROP TABLE <tablename>_TMP ;
Related
I have written a plsql block to delete some records for a bunch of tables.
so to identify the records to be deleted I have created a cursor on top that query.
declare
type t_guid_invoice is ref cursor;
c_invoice t_guid_invoice;
begin
open c_invoice for
select * from a,b where a.col=b.col ;--(quite a complex join,renders 200k records)
loop fetch c_invoice into col1,col2,col3;
exit when c_invoice%NOTFOUND;
begin
DELETE
FROM tab2
WHERE cola= col1;
if SQL%rowcount > 0 then
dbms_output.put_line ( 'INFO: tab2 for ' || col1|| '/' || col2|| ' removed.');
else
dbms_output.put_line ( 'WARN: No tab2 for ' || col1|| '/' || col2|| ' found!');
end if;
eXception
when others then
dbms_output.put_line ( 'ERR: Problems while deleting tab2 for ' || col1|| '/' || col2 );
dbms_output.put_line ( SQLERRM );
end;
....
end loop;
This continues to loop through about 26 tables, there are some tables which are as big as 60 million records.
deletion is based on primary key in each table. All triggers are disabled before deletion process.
if I try to delete 10k records, it loops through 10k times, deleting multiple rows in each table but its taking as long as 30 minutes. There is no commit after each block, since I have to cater to simulation mode too.
Any suggestions to speed up the process? Thanks!
For sure, if you loop 10k times, all those DBMS_OUTPUT.PUT_LINE calls will slow things down (even if you aren't doing anything "smart") (and I wonder whether buffer is large enough). If you want to log what's going on, create a log table and an autonomous transaction procedure which will insert that info (and commit).
Apart from that, are tables properly indexed? E.g. that would be cola column in tab2 table (in code you posted). Did you collect statistics on tables and indexes? Probably wouldn't harm if you do that for the whole schema.
Did you check explain plan?
Do you know what takes the most time? Is it a ref cursor query (so it has to be optimized), or is it deleting itself?
Can't you avoid loop entirely? Row-by-row processing is slow. For example, instead of using ref cursor, create a table out of it, index it, and use it as
create table c_invoice as
select * from a join b on a.col = b.col;
create index i1inv_col1 on c_invoice (col1);
delete from tab2 t
where exists (select null
from c_invoice c
where c.col1 = t.cola
);
You typically never want to delete a large number of rows from a table in a loop.
You want to use one DELETE statement with an appropriate WHERE condition.
Additionaly while processing a large number of rows you typically do not want to use an index.
So your first step would be to check the rows that would not be deleted (your warnings)
You get the keys with the following query and you may log them
select a.col from a,b where a.col=b.col
minus
select cola from tab2;
In the second step you delete all rows with one statement.
delete
from tab2
where cola in (select a.col from a,b where a.col=b.col);
In problems check the execution plan, you expect TABLE ACCESS FULL (INDEX FAST FULL SCAN is fine as well) on all sources combined with a HASH JOIN.
how to move indexes to another tablespace
select owner, index_name, TABLE_NAME, a.TABLESPACE_NAME
from all_indexes a where a.TABLE_NAME = 'REFUND';
ALTER INDEX CBS.PK_REFUND_ID REBUILD TABLESPACE T_IDX;
ORA-14086: A partitioned index may not be rebuilt as a whole.
I can't execute this statement because I don't know in which partition the index is
ALTER INDEX PK_REFUND_ID REBUILD PARTITION xxxxxx TABLESPACE T_IDX;
Because you also need partition name info which can be reached from user[all/dba]_tab_partitions dictionary view. If you got the related partition name, then
ALTER INDEX PK_REFUND_ID REBUILD PARTITION PR_REFUND_OLD [ONLINE];
might be used just to rebuild the index.
or
ALTER INDEX PK_REFUND_ID REBUILD PARTITION PR_REFUND_OLD [ONLINE] TABLESPACE T_IDX;
might be used to move the index of the partition to a different tablespace. Using ONLINE option would suit well for the tables currently having DML activity.
Use the following code in order to run as a batch job
DECLARE
v_ts_name VARCHAR2(35) := 'T_IDX';
BEGIN
FOR c IN
(
SELECT 'ALTER INDEX '||i.index_name||' REBUILD PARTITION '||p.partition_name||
' ONLINE TABLESPACE '||v_ts_name AS command
FROM user_part_indexes i
JOIN user_ind_partitions p
ON p.index_name = i.index_name
AND i.table_name = 'REFUND'
AND p.tablespace_name != v_ts_name
)
LOOP
EXECUTE IMMEDIATE c.command;
END LOOP;
END;
/
Oracle - Alter all table column names with trim of white space in between names
For suppose column names before alter :
Home number
Mobile number
Local number
After alter column names shall be :
Homenumber
Mobilenumber
Localnumber
I've tried this way: but unable to crack:
UPDATE SA_VW_PHONENUMBER TN SET TN.Column_Name = TRIM (TN.Column_Name);
Fully automatic way
Use this cursor based DDL hacking - statement concat.
BEGIN
FOR alters IN
(
SELECT
'ALTER TABLE "'||table_name||'" RENAME COLUMN "'||column_name||
'" TO "'||replace(cols.column_name,' ','')||'"' sql_stmt
FROM all_tab_cols cols
WHERE REGEXP_LIKE(column_name,'[[:space:]]')
AND owner = user --Add real schema name here
ORDER BY 1
) LOOP
DBMS_OUTPUT.PUT_LINE ( alters.sql_stmt ||';') ;
EXECUTE IMMEDIATE alters.sql_stmt;
END LOOP;
END;
/
If you want to use the safe way
As I know you cannot perform a DDL as a dynamic SQL, so you cannot pass variables to the ALTER TABLE command, but here is what you can do instead of that.
Selecting the occurences:
SELECT table_name,column_name,replace(cols.column_name,' ','') as replace_name
FROM all_tab_cols
WHERE REGEXP_LIKE(column_name,'[[:space:]]');
Use the ALTER TABLE DDL command:
alter table T_TABLE rename column "COLUMN SPACE" TO "COLUMNNOSPACE";
Try the REPLACE function
UPDATE SA_VW_PHONENUMBER TN SET TN.Column_Name = REPLACE(TN.Column_Name,' ','')
I have to create a procedure which searches any recently added records and if there are then move them to ARCHIVE table.
This is my statement which filters recently added records
SELECT
CL_ID,
CL_NAME,
CL_SURNAME,
CL_PHONE,
VEH_ID,
VEH_REG_NO,
VEH_MODEL,
VEH_MAKE_YEAR,
WD_ID,
WORK_DESC,
INV_ID,
INV_SERIES,
INV_NUM,
INV_DATE,
INV_PRICE
FROM
CLIENT,
INVOICE,
VEHICLE,
WORKS,
WORKS_DONE
WHERE
Client.CL_ID=Invoice.INV_CL_ID and
Invoice.INV_CL_ID = Client.CL_ID and
Client.CL_ID = Vehicle.VEH_CL_ID and
Vehicle.VEH_ID = Works_Done.WD_VEH_ID and
Works_done.WD_INV_ID = Invoice.INV_ID and
WORKS_DONE.WD_WORK_ID = Works.WORK_ID and
Works_done. Timestamp >= sysdate -1;
You may need something like this (pseudo-code):
create or replace procedure moveRecords is
vLimitDate timestamp := systimestamp -1;
begin
insert into table2
select *
from table1
where your_date >= vLimitDate;
--
delete table1
where your_date >= vLimitDate;
end;
Here are the steps I've used for this sort of task in the past.
Create a global temporary table (GTT) to hold a set of ROWIDs
Perform a multitable direct path insert, which selects the rows to be archived from the source table and inserts their ROWIDs into the GTT and the rest of the data into the archive table.
Perform a delete from the source table, where the source table ROWID is in the GTT of rowids
Issue a commit.
The business with the GTT and the ROWIDs ensures that you have 100% guaranteed stability in the set of rows that you are selecting and then deleting from the source table, regardless of any changes that might occur between the start of your select and the start of your delete (other than someone causing a partitioned table row migration or shrinking the table).
You could alternatively achieve that through changing the transaction isolation level.
O.K. may be something like this...
The downside is - it can be slow for large tables.
The upside is that there is no dependence on date and time - so you can run it anytime and synchronize your archives with live data...
create or replace procedure archive is
begin
insert into archive_table
(
select * from main_table
minus
select * from archive_table
);
end;
I have a question regarding a unified insert query against tables with different data
structures (Oracle). Let me elaborate with an example:
tb_customers (
id NUMBER(3), name VARCHAR2(40), archive_id NUMBER(3)
)
tb_suppliers (
id NUMBER(3), name VARCHAR2(40), contact VARCHAR2(40), xxx, xxx,
archive_id NUMBER(3)
)
The only column that is present in all tables is [archive_id]. The plan is to create a new archive of the dataset by copying (duplicating) all records to a different database partition and incrementing the archive_id for those records accordingly. [archive_id] is always part of the primary key.
My problem is with select statements to do the actual duplication of the data. Because the columns are variable, I am struggling to come up with a unified select statement that will copy the data and update the archive_id.
One solution (that works), is to iterate over all the tables in a stored procedure and do a:
CREATE TABLE temp as (SELECT * from ORIGINAL_TABLE);
UPDATE temp SET archive_id=something;
INSERT INTO ORIGINAL_TABLE (select * from temp);
DROP TABLE temp;
I do not like this solution very much as the DDL commands muck up all restore points.
Does anyone else have any solution?
How about creating a global temporary table for each base table?
create global temporary table tb_customers$ as select * from tb_customers;
create global temporary table tb_suppliers$ as select * from tb_suppliers;
You don't need to create and drop these each time, just leave them as-is.
You're archive process is then a single transaction...
insert into tb_customers$ as select * from tb_customers;
update tb_customers$ set archive_id = :v_new_archive_id;
insert into tb_customers select * from tb_customers$;
insert into tb_suppliers$ as select * from tb_suppliers;
update tb_suppliers$ set archive_id = :v_new_archive_id;
insert into tb_suppliers select * from tb_suppliers$;
commit; -- this will clear the global temporary tables
Hope this helps.
I would suggest not having a single sql statement for all tables and just use and insert.
insert into tb_customers_2
select id, name, 'new_archive_id' from tb_customers;
insert into tb_suppliers_2
select id, name, contact, xxx, xxx, 'new_archive_id' from tb_suppliers;
Or if you really need a single sql statement for all of them at least precreate all the temp tables (as temp tables) and leave them in place for next time. Then just use dynamic sql to refer to the temp table.
insert into ORIGINAL_TABLE_TEMP (SELECT * from ORIGINAL_TABLE);
UPDATE ORIGINAL_TABLE_TEMP SET archive_id=something;
INSERT INTO NEW_TABLE (select * from ORIGINAL_TABLE_TEMP);