Situation:
Table TBL has ~10k entries for deletion,
Table TBL has 14 child tables with delete rule "no action",
I want to delete 10k entries and referenced entries in child tables.
Procedure:
Delete records in child tables,
Disable constraints (if constraints are not disabled deletion in next step takes forever),
Delete records in TBL table,
Enable constraints.
Is there more elegant way for doing this?
The problem is that third step takes too long because it is checking 14 big tables for existence of non existent records. My procedure has good execution time but I think that there is more elegant way.
It sounds like you need to index your foreign keys on your child tables. Every time you delete a parent record without an index on the child table it must do a full table scan of the child to check if the foreign key constraint was broken.
With the index, it is at worst case an index range scan.
Edit:
More info, and a script to determine if this is your problem can be found here.
http://asktom.oracle.com/tkyte/unindex/index.html
Related
I have a datamart with 5 dimension table and a fact table.
I'm trying to clean a dimension table that has few rows (4000 rows). But, the fact table have millions rows (25GB)(Indexes and partitions).
When I try to delete a row in the table dimension, the process becomes very slow. It's just as slow despite the absence of relationship with a row in the fact table (cascade delete).
Is there any way to optimize this?. Thanks in advance.
Presumably, there is a cascading delete of some sort between the dimension table and the fact table.
Adding an index on the key column in the fact table may be sufficient. Then Oracle can immediately tell if/where any given value is.
If that doesn't work, drop the foreign key constraint altogether. Delete the unused values and add the constraint back in.
You could try these strategies as well :
create another copy of the fact table but, without the dim foreign key column of the table to be cleaned.
create fact_table_new as
select dim1_k, dim2_k, dim3_k, dim4_k, dim5_k (not this column), fact_1, fact_2, ...
from fact_table ;
or
update fact_table
set dim5_fk_col = null
where dim5_fk_col in (select k_col from dim5_table) ;
I need to update the primary key of a large Index Organized Table (20 million rows) on Oracle 11g.
Is it possible to do this using multiple UPDATE queries? i.e. Many smaller UPDATEs of say 100,000 rows at a time. The problem is that one of these UPDATE batches could temporarily produce a duplicate primary key value (there would be no duplicates after all the UPDATEs have completed.)
So, I guess I'm asking is it somehow possible to temporarily disable the primary key constraint (but which is required for an IOT!) or alter the table temporarily some other way. I can have exclusive and offline access to this table.
The only solution I can see is to create a new table and when complete, drop the original table and rename the new table to the original table name.
Am I missing another possibility?
You can't disable / drop the primary key constraint from an IOT, since it is a unique index by definition.
When I need to change an IOT like this, I either do a CTAS (create table as) for a new plain heap table, do my maintenance, and then CTAS a new IOT.
Something like:
create table t_temp as select * from t_iot;
-- do maintenance
create table t_new_iot as select * from t_temp;
If, however, you need to simply add or join a new field to the existing key, you can do this in one step by creating the new IOT structure, then populating directly from the old IOT with a query.
Unfortunately, this is one of the downsides to IOTs.
I would recommend following method:
Create new IOT table partitioned by system with single partition
with exactly same structure as current one.
Lock current IOT table to prevent any DML.
insert into new table as select from current table changing PK values in select. This step
could be repeated several times if needed. In this case it's better
to do it in another session to keep lock on original table.
Exchange partition of new table with original table.
Found this here:
In general, consider creating an index on a column in any of the following situations:
A referential integrity constraint exists on the indexed column or
columns. The index is a means to avoid a full table lock that would
otherwise be required if you update the parent table primary key,
merge into the parent table, or delete from the parent table.
I don't understand why a full table lock would occurr in such situation. I would've thought that if I tried to delete/update the primary key in the parent table that a full table scan would be performed on the child table.
Where does the lock come from?
Have a look at this Tom Kyte blog entry. In it, he refers to the Oracle documentation, where this explanation is offered:
Prevents a full table lock on the child table. Instead, the database acquires a row lock on the index.
Removes the need for a full table scan of the child table. As an illustration, assume that a user removes the record for department 10 from the departments table. If employees.department_id is not indexed, then the database must scan employees to see if any employees exist in department 10.
In the first scenario, if the column is not indexed, the entire table must be locked because Oracle does not know which rows must be updated in the child table. With an index, Oracle can identify the rows in question and just lock them. Without the full table lock, it would be possible to modify the parent and have another session modify the child to something that violated the constraint.
In our Oracle 11g database we have a table, that has a primary key I_Node (int) and also a column called I_Parent_Node (int) that references back to another record in the same table. The root node has I_Parent_Node = null. In this way we form a tree structure of nodes, leaves, branches, whatever you want to call them.
Frequently we need to delete an entire branch of nodes at once, meaning a node and all of its children. At times this is many, many records, say 50,000 or more. Since a cascade delete is not allowed on a self-referencing table, we are forced to delete one by one starting with the leaves and working our way back up the tree. We have experienced hours-long delete times.
We are considering doing a "marking for deletion" technique, where a separate program would clean out the nodes marked for deletion during off-peak hours, but I am interested in whether a database design change or some other Oracle construct could help out here. I am not trained in Oracle aside from what I've learned on the job, and the people who created the database did not have such large quantities in mind. I am open to database design changes since it is not yet a fixed design.
You may want to consider separating the hierarchy structure from the main table. So you main table would just have primary ids (lets call it "ID"), and your hierarchy table would have "ID, ParentID, TreeID". ParentID is that ID's parent node, and TreeID is the highest parent in the tree (level 1).
So, a level 1 node would look like:
ID, ParentID, TreeID
1, [null], 1
A level 2 node would look like:
ID, ParentID, TreeID
2, 1, 1
A level 3 node would look like:
ID, ParentID, TreeID
3, 2, 1
And so on.
You would use Oracle hierarchy queries (Connect by queries) to query or traverse the trees. This table will be very thin (not many columns, these 3 + some modified dates maybe), so updating these relationships should be much faster and scale better than messing with the main table.
You should be able to do this with deferrable constraints and a hierarchical query.
If your foreign key constraint (on I_Parent_Node) is not already deferrable, drop it and recreate it with the keyword "DEFERRABLE".
Here's an example using the EMPLOYEES table from Oracle's examples (I modified the DEPARTMENTS table too so that this would execute, that's really not needed for an example though):
Drop & Recreate your foreign key if it's not currently deferrable:
alter table employees drop constraint emp_manager_fk;
alter table employees add constraint emp_manager_fk foreign key (manager_id) references employees(employee_id) deferrable;
In your transaction, defer your contraints, and delete using a hierarchical query:
set constraints all deferred;
delete
from employees e
where employee_id in (select employee_id
from employees
start with employee_id = 108
connect by prior employee_id = manager_id);
The "108" is the ID of my "parent" record.
I assume you've already done standard tuning - i.e. are the node and parent node ID columns suitable indexed?
(1) One approach to the problem is to use PL/SQL. Bulk collect the IDs to be deleted, using a hierarchical query that returns the leaf rows first, into an array; then do a bulk delete (FORALL) using the array.
(2) Another approach is a soft-delete - mark the rows as "deleted", but never actually delete them. You would need to modify your application (or use Oracle VPD to automatically omit the "deleted" rows from queries). This might work reasonably well if deleting a node is relatively rare; but if you're routinely deleting lots of nodes then this would clutter the table with a lot of old data.
So we've been told that one source of TM Enq contention can be unindexed FK's. My question is which one.
I have an INSERT INTO Table_B that is recording TM Enq Wait.
It contains a PK that is the parent to other tables and it has columns that are FK constrained to other PKs.
So which FKs need indexed: that table's columns or its children?
NB: I know that this isn't the only cause of TM Contention. Can you explain why it couldn't possibly be this if that's the case.
Not sure about Oracle TM Contention, but I'd say normally both sides of a foreign key relation are indexed. Otherwise, the database will have to do table scans.
The index on the parent record is used whenever you insert a new child record, to verify that the parent exists. Often this is a primary key as well, so of course has an index.
The index on the child record is used whenever you change or delete a parent record, to perform cascades (including refusing the update/delete).
The indices on both sides also give the database a good chance of doing fast (indexed) joins, no matter which side its optimizer prefers to come from.
EDIT: Having Googled TM contention, it sounds like you're probably missing the keys on the child records. But make sure to have them on both sides, really.
EDIT 2: Answering the comment,
If you have a OLTP table that has 13 FKs to lookup tables, I'm not
keen on 13 index updates in addition to the table, pk and any other
indexes. An index is important but for specific reasons. If you never
update the parent PK nor delete from the parent, the child index is
not so useful. Or is it?
Depends on the joins and queries you're running, then. E.g., if you run
a query like:
SELECT o.something
FROM oltp_tab o JOIN lookup l ON (o.lookup_no = l.lookup_no)
WHERE l.lookup_name = ?
then the query optimizer would probably like the index on the child
records.
Also, according to http://ashmasters.com/waits/enq-tm-contention/ you
pretty much need to have the indices if you change the parent tables at
all. Apparently you get them from having concurrent changes to the
parent and child tables, unless you have the index. So this is probably
what you're seeing (assuming you're not doing the obvious things, like
updating the referred to columns or deleting rows)
The parent (referenced) column of an enabled foreign key relationship has to be indexed because it has to have an enabled unique or primary key constraint on it.
What mode of TM Enqueue are you seeing?