Pattern to substitute for MERGE INTO Oracle syntax when not allowed - oracle

I have an application that uses the Oracle MERGE INTO... DML statement to update table A to correspond with some of the changes in another table B (table A is a summary of selected parts of table B along with some other info). In a typical merge operation, 5-6 rows (out of 10's of thousands) might be inserted in table B and 2-3 rows updated.
It turns out that the application is to be deployed in an environment that has a security policy on the target tables. The MERGE INTO... statement can't be used with these tables (ORA-28132: Merge into syntax does not support security policies)
So we have to change the MERGE INTO... logic to use regular inserts and updates instead. Is this a problem anyone else has run into? Is there a best-practice pattern for converting the WHEN MATCHED/WHEN NOT MATCHED logic in the merge statement into INSERT and UPDATE statements? The merge is within a stored procedure, so it's fine for the solution to use PL/SQL in addition to the DML if that is required.

Another way to do this (other than Merge) would be using two sql statements one for insert and one for update. The "WHEN MATCHED" and "WHEN NOT MATCHED" can be handled using joins or "in" Clause.
If you decide to take the below approach, it is better to run the update first (sine it only runs for the matching records) and then insert the non-Matching records. The Data sets would be the same either way, it just updates less number of records with the order below.
Also, Similar to the Merge, this update statement updates the Name Column even if the names in Source and Target match. If you dont want that, add that condition to the where as well.
create table src_table(
id number primary key,
name varchar2(20) not null
);
create table tgt_table(
id number primary key,
name varchar2(20) not null
);
insert into src_table values (1, 'abc');
insert into src_table values (2, 'def');
insert into src_table values (3, 'ghi');
insert into tgt_table values (1, 'abc');
insert into tgt_table values (2,'xyz');
SQL> select * from Src_Table;
ID NAME
---------- --------------------
1 abc
2 def
3 ghi
SQL> select * from Tgt_Table;
ID NAME
---------- --------------------
2 xyz
1 abc
Update tgt_Table tgt
set Tgt.Name =
(select Src.Name
from Src_Table Src
where Src.id = Tgt.id
);
2 rows updated. --Notice that ID 1 is updated even though value did not change
select * from Tgt_Table;
ID NAME
----- --------------------
2 def
1 abc
insert into tgt_Table
select src.*
from Src_Table src,
tgt_Table tgt
where src.id = tgt.id(+)
and tgt.id is null;
1 row created.
SQL> select * from tgt_Table;
ID NAME
---------- --------------------
2 def
1 abc
3 ghi
commit;
There could be better ways to do this, but this seems simple and SQL-oriented. If the Data set is Large, then a PL/SQL solution won't be as performant.

There are at least two options I can think of aside from digging into the security policy, which I don't know much about.
Process the records to merge row by row. Attempt to do the update, if it fails to update then insert, or vise versa, depending on whether you expect most records to need updating or inserting (ie optimize for the most common case that will reduce the number of SQL statements fired), eg:
begin
for row in (select ... from source_table) loop
update table_to_be_merged
if sql%rowcount = 0 then -- no row matched, so need to insert
insert ...
end if;
end loop;
end;
Another option may be to bulk collect the records you want to merge into an array, and then attempted to bulk insert them, catching all the primary key exceptions (I cannot recall the syntax for this right now, but you can get a bulk insert to place all the rows that fail to insert into another array and then process them).
Logically a merge statement has to check for the presence of each records behind the scenes anyway, and I think it is processed quite similarly to the code I posted above. However, merge will always be more efficient than coding it in PLSQL as it will be only 1 SQL call instead of many.

Related

Best practice for updating column specific triggers

Welcome Oracle pro's
In an Oracle 12 database (upgrade is already scheduled ;-)) we have a setup of different tables updating a common base table via "after update" triggers like following:
Search_Flat
ID
Field_A
Field_B
Field_C
Now table1 contains n columns where let's say 2 out of n are relevant for the Search_Flat table. As the update of table1 may only affect columns not relevant for Seach_Flat we want to add checks to the trigger. So our first approach is like following:
CREATE OR REPLACE TRIGGER tr_tbl_1_au_search
AFTER UPDATE OF
field_a,
field_b
ON schemauser.search_flat
FOR EACH ROW
BEGIN
IF :new.field_a <> :old.field_a THEN
UPDATE schemauser.search_flat SET field_a = :new.field_a WHERE id = :new.ID;
END IF;
IF :new.field_b <> :old.field_b THEN
UPDATE schemauser.search_flat SET field_b = :new.field_b WHERE id = :new.ID;
END IF;
END;
Alternatively we could also setup the trigger like following:
CREATE OR REPLACE TRIGGER tr_tbl_1_au_search
AFTER UPDATE OF
field_a,
field_b
ON schemauser.search_flat
FOR EACH ROW
BEGIN
IF :new.field_a <> :old.field_a OR :new.field_b <> :old.field_b THEN
UPDATE schemauser.search_flat
SET field_a = :new.field_a,
field_b = :new.field_b
WHERE id = :new.ID;
END IF;
END;
The question now is about the setup of the triggers themselves. Which approach is the better with respect to:
locking time of search_flat rows
overall performance of affected components (i.e., table_1, trigger and search_flat)
In production we are talking about 4 tables with 10 fields each considered in the triggers. And we have independent app servers accessing the shared database updating the 4 tables simultaneously. From time to time we detect the following error which is the reason we wan't to optimize the triggers:
ORA-02049: timeout: distributed transaction waiting for lock
Sidenote: This setup has been chosen instead of a view or materialized view due to performance reasons as the base table is used in gui with the requirement to be instantly updated and the number of records of the 4 feeding tables are too high for updating materialized view on update.
I'm looking forward to the discussion and your thoughts.
As I understand your post, you have 4 live tables (called "table1", "table2", etc.) that you want to search on, but querying from them is too slow, so you want to maintain a single, flattened table to search on instead and have triggers to keep that flattened table always up-to-date.
You want to know which of two trigger approaches is better.
I think the answer is "neither", since both are prone to deadlocks. Imagine this scenario
User 1 -
UPDATE table1
SET field_a = 500
WHERE <condition effecting 200 distinct IDs>
User 2 at about the same time -
UPDATE table1
SET field_b = 700
WHERE <condition effecting 200 distinct IDs>
Triggers start processing. You cannot control the order in which the rows are updated. Maybe it goes like this:
User 1's trigger, time index 100 ->
UPDATE search_flat SET field_a = 500 WHERE id = 90;
User 2's trigger, time index 101 ->
UPDATE search_flat SET field_b = 700 WHERE id = 91;
User 1's trigger, time index 102 ->
UPDATE search_flat SET field_a = 500 WHERE id = 91; (waits on user 2's session)
User 2's trigger, time index 103 ->
UPDATE search_flat SET field_b = 700 WHERE id = 90; (deadlock error)
User 2's original update fails and rolls back.
You have multiple concurrent processes all updating the same set of rows in search_flat with no control over the processing order. That is a recipe for deadlocks.
If you wanted to do this safely, you should consider neither of the FOR EACH ROW trigger approaches you outlines. Rather, make a compound trigger to do this.
Here's some sample code to illustrate the idea. Be sure to read the comments.
-- Aside: consider setting this at the system level if on 12.2 or later
-- alter system set temp_undo_enabled=false;
CREATE GLOBAL TEMPORARY TABLE table1_updates_gtt (
id NUMBER,
field_a VARCHAR2(80),
field_b VARCHAR2(80)
) ON COMMIT DELETE ROWS;
CREATE GLOBAL TEMPORARY TABLE table2_updates_gtt (
id NUMBER,
field_a VARCHAR2(80)
) ON COMMIT DELETE ROWS;
-- .. so on for table3 and 4.
CREATE OR REPLACE TRIGGER table1_search_maint_trg
FOR INSERT OR UPDATE OR DELETE ON table1 -- with similar compound triggers for table2, 3, 4.
COMPOUND TRIGGER
AFTER EACH ROW IS
BEGIN
-- Update the table-1 specific GTT with the changes.
CASE WHEN INSERTING OR UPDATING THEN
-- Assumes ID is immutable primary key
INSERT INTO table1_updates_gtt (id, field_a) VALUES (:new.id, :new.field_a);
WHEN DELETING THEN
INSERT INTO table1_updates_gtt (id, field_a) VALUES (:old.id, null); -- or figure out what you want to do about deletes.
END CASE;
END AFTER EACH ROW;
AFTER STATEMENT IS
BEGIN
-- Write the data from the GTT to the search_flat table.
-- NOTE: The ORDER BY in the next line is what saves us from deadlocks.
FOR r IN ( SELECT id, field_a, field_b FROM table1_updates_gtt ORDER BY id ) LOOP
-- TODO: replace with BULK processing for better performance, if DMLs can affect a lot of rows
UPDATE search_flat sf
SET sf.field_a = r.field_a,
sf.field_b = r.field_b
WHERE sf.id = r.id
AND ( sf.field_a <> r.field_a
OR (sf.field_a IS NULL AND r.field_a IS NOT NULL)
OR (sf.field_a IS NOT NULL AND r.field_a IS NULL)
OR sf.field_b <> r.field_b
OR (sf.field_b IS NULL AND r.field_b IS NOT NULL)
OR (sf.field_b IS NOT NULL AND r.field_b IS NULL)
);
END LOOP;
END AFTER STATEMENT;
END table1_search_maint_trg;
Also, as numerous commenters have pointed out, it's probably better to use a materialized view for this. If you are on 12.2 or later, real-time materialized views (aka "ENABLE ON QUERY COMPUTATION") offer a lot of promise for this sort of thing. No COMMIT overhead to your application and real-time search results. It's just that search time degrades slightly if there are a lot of recent updates to the underlying tables.

Inserting Row Number based on existing value in the table

I have a requirement that I need to insert row number in a table based on value already present in the table. For example, the max row_nbr record in the current table is something like this:
+----------+----------+------------+---------+
| FST_NAME | LST_NAME | STATE_CODE | ROW_NBR |
+----------+----------+------------+---------+
| John | Doe | 13 | 123 |
+----------+----------+------------+---------+
Now, I need to insert more records, with given FST_NAME and LST_NAME values. ROW_NBR needs to be generated while inserting the data into table with values auto-incrementing from 123.
I can't use a sequence, as my loading process is not the only process that inserts data into this table. And I can't use a cursor as well, as due to high volume of data the TEMP space gets filled up quickly. And I'm inserting data as given below:
insert into final_table
( fst_name,lst_name,state_code)
(select * from staging_table
where state_code=13);
Any ideas how to implement this?
It sounds like other processes are finding the current maximum row_nbr value and incrementing it as they do single-row inserts in a cursor loop.
You could do something functionally similar, either finding the maximum in advance and incrementing it (if you're already running this in a PL/SQL block):
insert into final_table (fst_name, lst_name, state_code, row_nbr)
select st.*, variable_holding_maximum + rownum
from staging_table st
where st.state_code=13;
or by querying the table as part of the query, which doesn't need PL/SQL:
insert into final_table (fst_name, lst_name, state_code, row_nbr)
select st.*, (select max(row_nbr) from final_table) + rownum
from staging_table st
where st.state_code=13;
db<>fiddle
But this isn't a good solution because it doesn't prevent clashes from different processes and sessions trying to insert at the same time; but neither would the cursor loop approach, unless it is catching unique constraint errors and re-attempting with a new value, perhaps.
It would be better to use a sequence, which would be an auto-increment column but you said you can't change the table structure; and you need to let the other processes continue to work without modification. You can still do that with a sequence and trigger approach, having the trigger always set the row_nbr value form the sequence, regardless of whether the insert statement supplied a value.
If you create a sequence that starts from the current maximum, with something like:
create sequence final_seq start with <current max + 1>
or without manually finding it:
declare
start_with pls_integer;
begin
select nvl(max(row_nbr), 0) + 1 into start_with from final_table;
execute immediate 'create sequence final_seq start with ' || start_with;
end;
/
then your trigger could just be:
create trigger final_trig
before insert on final_table
for each row
begin
:new.row_nbr := final_seq.nextval;
end;
/
Then your insert ... select statement doesn't need to supply or even think about the row_nbr value, so you can leave it as you have it now (except I'd avoid select * even in that construct, and list the staging table columns explicitly); and any existing inserts that do supply the row_nbr don't need to be modified and the value they supply will just be overwritten from the sequence.
db<>fiddle showing inserts with and withouth row_nbr specified.

Adding Data in 1 tables through trigger on multiple tables

How do I add data in a table which contains attributes of two different tables which are not linked together?
Table 1 has col1,col2,col3 (assume all are numbers)
Table 2 has col4,col5,col6 (assume all are numbers)
Table 3 has A,B,C,D,E,F (assume all are numbers)
Question is that if any insertion occurs on table 1 and table 2, their data should be loaded in table 3.
I used the normal approach
of
create or replace trigger trig_name
before insert on table1,table2 --> not allowed
for each row
begin
if inserting then
insert into table3 values (:new.col1,:new.col2,:new.col3,:new.col4,:new.col5,:new.col6)
end if;
end;
/
So if I have to make two different triggers for table 1 and table 2, wouldn't that create two rows of data leaving some columns null?
Although creating a view as suggested by #EdStevens is (IMO) a superior answer, what you want can be done. And yes it requires 2 triggers. In Oracle, and all other DBMS that I'm aware of, a trigger can only fire on 1 table.
However, they can be reduced somewhat. There is no need for the "if inserting" test. The trigger is declared as "on insert" therefore the test will always be true. The actual issue it you cannot refer columns from table2 in the table1 trigger and likewise the other way around. What you need is to name the columns you are inserting, actually always a better approach. So:
create or replace trigger table1_bir
before insert on table1
for each row
begin
insert into table3(col1, col2, col3)
values (:new.col1,:new.col2,:new.col3);
end;
/
create or replace trigger table2_bir
before insert on table2
for each row
begin
insert into table3(col4, col5, col6)
values (:new.col4,:new.col5,:new.col6);
end;
/

Inserting Record Type

I have a source table, and a history table. They are identical, except the history table has a notes column as the last column.
The table is large, 55 columns, which I do not want to list all the columns. We do not want to use a trigger on this, but just create the history entry in the code itself.
If I simply do an INSERT INTO <history> SELECT * FROM <source> WHERE...... I will get "not enough values".
I'm hoping to do something of this nature: (Note this is just an anonymous block)
DECLARE
v_old_rec company_table%ROWTYPE;
BEGIN
SELECT * INTO v_old_rec
FROM company_table
WHERE company_id = 32789;
INSERT INTO company_table_hist
VALUES v_old_rec || ',MONTHLY UPDATE';
END;
Anything like this possible, so I do not have to list 55 columns?
Thank you.
It's quite simple actually:
INSERT INTO company_table_hist
(SELECT t.*, MONTHLY_UPD
FROM company_table t
WHERE company_id = 32789);
Of course monthly_upd must be bound somehow

Trigger with subquery

I'm trying to create a trigger that controls cycles in a self-referencing table.
Unfortunately I've got an error. Could anyone tell me what I'm doing wrong?
CREATE OR REPLACE TRIGGER CHECK_CYCLE
BEFORE INSERT OR UPDATE ON DEPARTMENTS
FOR EACH ROW
BEGIN
IF :NEW.PARENT_ID =
(SELECT ID, PARENT_ID, NAME, LEVEL,
CONNECT_BY_ISLEAF AS ISLEAF,
PRIOR NAME AS PARENT_NAME,
CONNECT_BY_ROOT NAME AS ROOT
FROM DEPARTMENTS
START WITH PARENT_ID IS NULL
CONNECT BY PRIOR ID = PARENT_ID)
THEN
RAISE_APPLICATION_ERROR(20000, 'Sorry.');
END IF;
END;
Error(9,25): PLS-00103: Encountered the symbol "AS" when expecting one of the following:
, from
This seems to be the parser getting confused. If you remove the AS ROOT completely then it reverts to the expected PLS-00405: subquery not allowed in this context error. No idea why it is confused - you can run that query standalone - but since you can't do what you're attempting it's probably not worth worrying about too much.
Your subquery also has three columns and will return multiple rows, so comparing with the current row's scalar PARENT_ID isn't going to work. You could run your subquery separately and select the value you're actually interested in into a local variable, and check against that.
But you have a before-insert trigger and you're querying the table the trigger is against, so you'll get a mutating table exception anyway when you try to insert (or at least, if you try to insert multiple rows at once).
If I understand what you're trying to achieve, you can use an after-insert trigger instead:
create or replace trigger check_cycle
after insert or update on departments
declare
l_hascycle pls_integer;
begin
select max(connect_by_iscycle)
into l_hascycle
from departments
start with parent_id is null
connect by nocycle prior id = parent_id;
if l_hascycle = 1 then
raise_application_error(-20000, 'Sorry.');
end if;
end;
/
This uses the CONNECT_BY_ISCYCLE pseudocolumn, which will be zero for all rows if there is no cycling, and one for any column that cycles. Selecting and checking the max() of that means you'll have a single value of 0 or 1 in the local l_hascycle variable, and you can use that to decide whether to throw an exception.
Inserting a couple of test rows on top of existing non-cycling data such as:
ID PARENT_ID NAME
---------- ---------- ------------
1 Test
2 1 Test
... where the first new row is OK and the second would cause a cycle:
insert into departments (id, parent_id, name) values (3, 2, 'OK');
1 row inserted.
insert into departments (id, parent_id, name) values (2, 3, 'Cycles');
ORA-20000: Sorry.
ORA-06512: at "SCHEMA.CHECK_CYCLE", line 11
ORA-04088: error during execution of trigger 'SCHEMA.CHECK_CYCLE'
The first insert is still in effect (but not yet committed), the second was implicitly rolled back:
select * from departments:
ID PARENT_ID NAME
---------- ---------- ------------
1 Test
2 1 Test
3 2 OK
This won't catch cycles in data that is not connected to an existing tree culminating in a null parent, because of the start with clause; so for exampel with data you could insert 4,5 and 5,4 without raising an exception, as there is no route from either 4 or 5 to a row with a null parent. But that's a different issue and something you might want to test for separately.
It also can't see data from uncommitted changes in other sessions, so it's possible for two sessions to simultaneously insert rows that are valid independently but which still would form a cycle once both are committed.
I think I found the error in your query, it's in this line -
CONNECT_BY_ROOT NAME AS ROOT
which is not built correctly.
Try changing it to something like
CONNECT_BY_ROOT AS ROOT

Resources