I am using a view named V_ENT_MSG in my oracle procedure "TEST PROC" as shown below. The same view is used in multiple places inside the procedure and the procedure execution time is very high. The view returns more than 5 Million records every time. How can I improve procedure execution time. [ Note that I cannot change the view V_ENT_MSG].
create or replace PROCEDURE "TESTPROC"
AS
BEGIN
INSERT
INTO OUTBOUND1
(
DELIVERY_EVENT_ID,
MESSAGE_ID
)
SELECT
V2.DELIVERY_EVENT_ID,PS.MESSAGE_ID
FROM V_ENT_MSG V2,
R_SOURCE PS
WHERE V2.primary_source = PS.SOURCE;
COMMIT;
INSERT
INTO OUTBOUND2
(
DELIVERY_EVENT_ID,
MESSAGE_BODY
)
SELECT
V2.DELIVERY_EVENT_ID,PS2.MESSAGE_BODY
FROM V_ENT_MSG V2,
R_SOURCE2 PS2
WHERE V2.primary_source = PS2.SOURCE;
COMMIT;
INSERT
INTO OUTBOUND3
(
DELIVERY_EVENT_ID,
SUBJECT
)
SELECT
V2.DELIVERY_EVENT_ID,PS3.SUBJECT
FROM V_ENT_MSG V2,
R_SOURCE3 PS3
WHERE V2.primary_source = PS3.SOURCE;
COMMIT;
END TESTPROC;
You may be able to rewrite this as a multitable insert. To do this, you need to:
(Outer) join the view to each table
insert all the result of this query
Use the when clause to control which rows go in which tables
Which looks something like:
insert all
when ps.source is not null then
into outbound1 ( delivery_event_id, message_id )
values ( delivery_event_id, message_id )
when ps2.source is not null then
into outbound2 ( delivery_event_id, message_body )
values ( delivery_event_id, message_body )
when ps3.source is not null then
into outbound3 ( delivery_event_id, subject )
values ( delivery_event_id, subject )
select v.delivery_event_id,ps3.subject
from v_ent_msg v
left join r_source ps
on v.primary_source = ps.source
left join r_source2 ps2
on v.primary_source = ps2.source
left join r_source3 ps3
on v.primary_source = ps3.source;
This assumes that each primary_source joins to at most one row in r_sourceX. If it's 1:M and each value of primary_source can join to a different number of rows in each of the other tables it gets trickier.
You'll need to assign a row_number for each source value in each r_sourceX table and only insert those rows where this is one.
Additionally whether you have opted for GTT as suggested by #Littlefoot or not ,you can still use this approach.
How about leveraging INSERT ALL with WITH clause to achieve the same ,
INSERT ALL
/* inserting to OUTBOUND1 based on table_name = R_SOURCE defined during union */
WHEN table_name = 'R_SOURCE' THEN
INTO outbound1(delivery_event_id, message_id)
VALUES(delivery_event_id, message_type)
/* inserting to OUTBOUND2 based on table_name = R_SOURCE2 defined during union */
WHEN table_name = 'R_SOURCE2' THEN
INTO outbound2(delivery_event_id, message_body)
VALUES (delivery_event_id, message_type)
/* inserting to OUTBOUND3 based on table_name = R_SOURC3 defined during union */
WHEN table_name = 'R_SOURCE3' THEN
INTO outbound3(delivery_event_id, subject)
VALUES (delivery_event_id, message_type)
WITH ent_msg
AS
(SELECT v2.delivery_event_id,v2.primary_source
FROM v_ent_msg v2),
src_data
AS
( SELECT v2.delivery_event_id delivery_event_id,ps.message_id message_type,'R_SOURCE' table_name
FROM r_source ps
JOIN ent_msg v2
ON v2.primary_source = ps.source
UNION ALL
SELECT v2.delivery_event_id,ps2.message_body,'R_SOURCE2' table_name
FROM r_source2 ps2
JOIN ent_msg v2
ON v2.primary_source = ps2.source
UNION ALL
SELECT v2.delivery_event_id,ps3.subject,'R_SOURCE3' table_name
FROM r_source3 ps3
JOIN ent_msg v2
ON v2.primary_source = ps3.source
)
SELECT delivery_event_id,message_type,table_name
FROM src_data
I couldn't validate SQL because of tables are missing.
There are other ways too in case of performance issue where we can go for BULK COLLECT with LIMIT. I am not aware of the data set you are dealing with.
Hope this guides you proceeding with your question.
Related
I have a stored procedure which should return several results - but it returns only one row. I think it's the last row in result set.
I am not sure, but I think the problem is in this line of code:
select chi.id bulk collect into v_numbers from dual;
and that this line somehow overrides all previous results (there is several of them for each loop). How to insert into v_numbers without overriding previous results? I know that it's also wrong to insert only one row, but I haven't found solution to insert several rows from chi.
PROCEDURE GET_ATTRIBUTES(
P_AUTH_USE_ID IN NUMBER,
P_CATEGORY_ID IN NUMBER,
P_VERSION_ID IN NUMBER,
P_RESULT OUT TYPES.CURSOR_TYPE
) IS
v_numbers sys.odcinumberlist := null;
BEGIN
FOR item IN
(SELECT ID FROM INV_SRV WHERE SRV_CATEGORY_ID IN
(
SELECT id
FROM inv_srv_category
START WITH parent_category_id = P_CATEGORY_ID
CONNECT BY PRIOR id = parent_category_id
) OR SRV_CATEGORY_ID = P_CATEGORY_ID)
LOOP
for chi in (select s.id
from inv_srv s
start with s.parent_srv_id = item.id
connect by prior s.id = s.parent_srv_id
)
loop
select chi.id bulk collect into v_numbers from dual; --> here I should insert all rows from that loop, but I don't know how
end loop;
END LOOP;
OPEN P_RESULT FOR SELECT t.column_value from table(v_numbers) t; --> only one row is returned
END;
Use BULK COLLECT and FORALL for bulk inserts and better performance. The FORALL statement will allow the DML to be run for each row in the collection without requiring a context switch each time, thus improving the overall performance.
CREATE OR REPLACE PROCEDURE get_attributes (
p_auth_use_id IN NUMBER,
p_category_id IN NUMBER,
p_version_id IN NUMBER,
p_result OUT types.cursor_type
) IS
v_numbers sys.odcinumberlist := NULL;
BEGIN
SELECT s.id
BULK COLLECT --> Bulk collect all values
INTO v_numbers
FROM inv_srv s
start with s.parent_srv_id in (
SELECT ID FROM INV_SRV
WHERE SRV_CATEGORY_ID IN
(
SELECT id
FROM inv_srv_category
START WITH parent_category_id = P_CATEGORY_ID
CONNECT BY PRIOR id = parent_category_id
)
OR SRV_CATEGORY_ID = P_CATEGORY_ID)
connect by prior s.id = s.parent_srv_id;
FORALL i IN 1..v_numbers.COUNT
INSERT INTO your_table VALUES v_numbers ( i ); --> Bulk insert
END;
Every time the loop executes v_numbers will be re populated again and again so, either 1) use v_numbers.extend; v_numbers(v_numbers.last) = "Your Value" or write everything in a single bulk collect.
select s.id
bulk collect into v_numbers
from inv_srv s
start with s.parent_srv_id in (SELECT ID FROM INV_SRV
WHERE SRV_CATEGORY_ID IN
(
SELECT id
FROM inv_srv_category
START WITH parent_category_id = P_CATEGORY_ID
CONNECT BY PRIOR id = parent_category_id
)
OR SRV_CATEGORY_ID = P_CATEGORY_ID)
connect by prior s.id = s.parent_srv_id
This may be considered as a improper use of the PL/SQL loops (often connected with a catastrophic performance) in a situation where a SQL solution exists.
Why don't you simple defines the cursor as follows:
OPEN P_RESULT FOR
select s.id
from inv_srv s
start with s.parent_srv_id in
(SELECT ID FROM INV_SRV WHERE SRV_CATEGORY_ID IN
(SELECT id
FROM inv_srv_category
START WITH parent_category_id = 1
CONNECT BY PRIOR id = parent_category_id
) OR SRV_CATEGORY_ID = 1)
connect by prior s.id = s.parent_srv_id
;
The query is constructed from your outer and inner loop so that it returns the same result.
The transformation may not be trivial in generall case and must be carefully tested, but the performance profit may be high.
I have a function, which will get greatest of three dates from the table.
create or replace FUNCTION fn_max_date_val(
pi_user_id IN number)
RETURN DATE
IS
l_modified_dt DATE;
l_mod1_dt DATE;
l_mod2_dt DATE;
ret_user_id DATE;
BEGIN
SELECT MAX(last_modified_dt)
INTO l_modified_dt
FROM table1
WHERE id = pi_user_id;
-- this table contains a million records
SELECT nvl(MAX(last_modified_ts),sysdate-90)
INTO l_mod1_dt
FROM table2
WHERE table2_id=pi_user_id;
-- this table contains clob data, 800 000 records, the table 3 does not have user_id and has to fetched from table 2, as shown below
SELECT nvl(MAX(last_modified_dt),sysdate-90)
INTO l_mod2_dt
FROM table3
WHERE table2_id IN
(SELECT id FROM table2 WHERE table2_id=pi_user_id
);
execute immediate 'select greatest('''||l_modified_dt||''','''||l_mod1_dt||''','''||l_mod2_dt||''') from dual' into ret_user_id;
RETURN ret_user_id;
EXCEPTION
WHEN OTHERS THEN
return SYSDATE;
END;
this function works perfectly fine and executes within a second.
-- random user_id , just to test the functionality
SELECT fn_max_date_val(100) as max_date FROM DUAL
MAX_DATE
--------
27-02-14
For reference purpose i have used the table name as table1,table2 and table3 but my business case is similar to what i stated below.
I need to get the details of the table1 along with the highest modified date among the three tables.
I did something like this.
SELECT a.id,a.name,a.value,fn_max_date_val(id) as max_date
FROM table1 a where status_id ='Active';
The above query execute perfectly fine and got result in millisecods. But the problem came when i tried to use order by.
SELECT a.id,a.name,a.value,a.status_id,last_modified_dt,fn_max_date_val(id) as max_date
FROM table1 where status_id ='Active' a
order by status_id desc,last_modified_dt desc ;
-- It took almost 300 seconds to complete
I tried using index also all the values of the status_id and last_modified, but no luck. Can this be done in a right way?
How about if your query is like this?
select a.*, fn_max_date_val(id) as max_date
from
(SELECT a.id,a.name,a.value,a.status_id,last_modified_dt
FROM table1 where status_id ='Active' a
order by status_id desc,last_modified_dt desc) a;
What if you don't use the function and do something like this:
SELECT a.id,a.name,a.value,a.status_id,last_modified_dt x.max_date
FROM table1 a
(
select max(max_date) as max_date
from (
SELECT MAX(last_modified_dt) as max_date
FROM table1 t1
WHERE t1.id = a.id
union
SELECT nvl(MAX(last_modified_ts),sysdate-90) as max_date
FROM table2 t2
WHERE t2.table2_id=a.id
...
) y
) x
where a.status_id ='Active'
order by status_id desc,last_modified_dt desc;
Syntax might contain errors, but something like that + the third table in the derived table too.
I have two tables. if the data in table1 is more than a predefined limit (say 2), i need to copy the remaining contents of table1 to table2 and delete those same contents from table1.
I used the below query to insert the excess data from table1 to table2.
insert into table2
SELECT * FROM table1 WHERE ROWNUM < ((select count(*) from table1)-2);
Now i need the delete query to delete the above contents from table1.
Thanks in advance.
A straightforward approach would be an interim storage in a temporary table. Its content can be used to determine the data to be deleted from table1 as well as the source to feed table 2.
Assume (slightly abusing notation) to be the PK column (or that of any candidate key) of table1 - usually there'll be some key that comprises only 1 column.
create global temporary table t_interim as
( SELECT <pk> pkc FROM table1 WHERE ROWNUM < ((select count(*) from table1)-2 )
;
insert into table2
select * from table1 where <pk> IN (
select pkc from t_interim
);
delete from table1 where <pk> IN (
select pkc from t_interim
);
Alternative
If any key of table1 spans more than 1 column, use an EXISTS clause instead as follows ( denoting the i-th component of a candidate key in table1):
create global temporary table t_interim as
( SELECT <ck_1> ck1, <ck_2> ck2, ..., <ck_n> ckn FROM table1 WHERE ROWNUM < ((select count(*) from table1)-2 )
;
insert into table2
select * from table1 t
where exists (
select 1
from t_interim t_i
where t.ck_1 = t_i.ck1
and t.ck_2 = t_i.ck2
...
and t.ck_n = t_i.ckn
)
;
delete from table1 t where
where exists (
select 1
from t_interim t_i
where t.ck_1 = t_i.ck1
and t.ck_2 = t_i.ck2
...
and t.ck_n = t_i.ckn
)
;
(Technically you could try to adjust the first scheme by synthesizing a key from the components of any CK, eg. by concatenating. You run the risk of introducing ambiguities ( (a bc, ab c) -> (abc, abc) ) or run into implementation limits ( max. varchar length ) using the first method)
Note
In case the table doesn't have a PK, you can apply the technique using any candidate key of table1. There will always be one, in the extreme case it's the set of all columns.
This situation may be the right time to improve the db design and add a (synthetic) pk column to table1 ( and any other tables in the system that lack it).
I have a table ( table1 ) that represents item grouping and another table ( table2 ) that represents the items themselves.
table1.id is foreign key to table2 and in every record of table1 I also collect information like the total number of records in table2 associated with that particular record and the sum of various fields so that I can show the grouping and a summary of what's in it without having to query table2.
Usually items in table2 are added/removed one at a time, so I update table1 to reflect the changes in table2.
A new requirement arose, choosen items in a group must be moved to a new group. I thought of it as a 3 step operation:
create a new group in table1
update choosen records in table2 to point to the newly created rec in table1
the third step would be to subtract to the group the number of records / the sum of those other fields I need do show and add them to the new group, data that I can find simply querying table2 for items associated with the new group.
I came up with the following statement that works.
update table1 t1 set
countitems = (
case t1.id
when 1 then t1.countitems - ( select count( t2.id ) from table2 t2 where t2.id = 2 )
when 2 then ( select count( t2.id ) from table2 t2 where t2.id = 2 )
end
),
sumitems = (
case t1.id
when 1 then t1.sumitems - ( select sum( t2.num ) from table2 t2 where t2.id = 2 )
when 2 then ( select sum( t2.num ) from table2 t2 where t2.id = 2 )
end
)
where t1.id in( 1, 2 );
is there a way to rewrite the statement without having to repeat the subquery every time?
thanks
Piero
You can use a cursor and a bulk collect update statement on the rowid. That way you can simply write the join query with the desired result and update the table with those values. I always use this function and make slight adjustments each time.
declare
cursor cur_cur
IS
select ti.rowid row_id
, count(t2.id) countitems
, sum(t2.num) numitems
from table t1
join table t2 on t1.id = t2.t1_id
order by row_id
;
type type_rowid_array is table of rowid index by binary_integer;
type type_countitems_array is table of table1.countitems%type;
type type_numitems_array is table of table1.numitems%type;
arr_rowid type_rowid_array;
arr_countitems type_countitems_array;
arr_numitems type_numitems_array;
v_commit_size number := 10000;
begin
open cur_cur;
loop
fetch cur_cur bulk collect into arr_rowid, arr_countitems, arr_numitems limit v_commit_size;
forall i in arr_rowid.first .. arr_rowid.last
update table1 tab
SET tab.countitems = arr_countitems(i)
, tab.numitems = arr_numitems(i)
where tab.rowid = arr_rowid(i)
;
commit;
exit when cur_cur%notfound;
end loop;
close cur_cur;
commit;
exception
when others
then rollback;
raise_application_error(-20000, 'ERROR updating table1(countitems,numitems) - '||sqlerrm);
end;
What is the easiest way to INSERT a row if it doesn't exist, in PL/SQL (oracle)?
I want something like:
IF NOT EXISTS (SELECT * FROM table WHERE name = 'jonny') THEN
INSERT INTO table VALUES ("jonny", null);
END IF;
But it's not working.
Note: this table has 2 fields, say, name and age. But only name is PK.
INSERT INTO table
SELECT 'jonny', NULL
FROM dual -- Not Oracle? No need for dual, drop that line
WHERE NOT EXISTS (SELECT NULL -- canonical way, but you can select
-- anything as EXISTS only checks existence
FROM table
WHERE name = 'jonny'
)
Assuming you are on 10g, you can also use the MERGE statement. This allows you to insert the row if it doesn't exist and ignore the row if it does exist. People tend to think of MERGE when they want to do an "upsert" (INSERT if the row doesn't exist and UPDATE if the row does exist) but the UPDATE part is optional now so it can also be used here.
SQL> create table foo (
2 name varchar2(10) primary key,
3 age number
4 );
Table created.
SQL> ed
Wrote file afiedt.buf
1 merge into foo a
2 using (select 'johnny' name, null age from dual) b
3 on (a.name = b.name)
4 when not matched then
5 insert( name, age)
6* values( b.name, b.age)
SQL> /
1 row merged.
SQL> /
0 rows merged.
SQL> select * from foo;
NAME AGE
---------- ----------
johnny
If name is a PK, then just insert and catch the error. The reason to do this rather than any check is that it will work even with multiple clients inserting at the same time. If you check and then insert, you have to hold a lock during that time, or expect the error anyway.
The code for this would be something like
BEGIN
INSERT INTO table( name, age )
VALUES( 'johnny', null );
EXCEPTION
WHEN dup_val_on_index
THEN
NULL; -- Intentionally ignore duplicates
END;
I found the examples a bit tricky to follow for the situation where you want to ensure a row exists in the destination table (especially when you have two columns as the primary key), but the primary key might not exist there at all so there's nothing to select.
This is what worked for me:
MERGE INTO table1 D
USING (
-- These are the row(s) you want to insert.
SELECT
'val1' AS FIELD_A,
'val2' AS FIELD_B
FROM DUAL
) S ON (
-- This is the criteria to find the above row(s) in the
-- destination table. S refers to the rows in the SELECT
-- statement above, D refers to the destination table.
D.FIELD_A = S.FIELD_A
AND D.FIELD_B = S.FIELD_B
)
-- This is the INSERT statement to run for each row that
-- doesn't exist in the destination table.
WHEN NOT MATCHED THEN INSERT (
FIELD_A,
FIELD_B,
FIELD_C
) VALUES (
S.FIELD_A,
S.FIELD_B,
'val3'
)
The key points are:
The SELECT statement inside the USING block must always return rows. If there are no rows returned from this query, no rows will be inserted or updated. Here I select from DUAL so there will always be exactly one row.
The ON condition is what sets the criteria for matching rows. If ON does not have a match then the INSERT statement is run.
You can also add a WHEN MATCHED THEN UPDATE clause if you want more control over the updates too.
Using parts of #benoit answer, I will use this:
DECLARE
varTmp NUMBER:=0;
BEGIN
-- checks
SELECT nvl((SELECT 1 FROM table WHERE name = 'john'), 0) INTO varTmp FROM dual;
-- insert
IF (varTmp = 1) THEN
INSERT INTO table (john, null)
END IF;
END;
Sorry for I don't use any full given answer, but I need IF check because my code is much more complex than this example table with name and age fields. I need a very clear code. Well thanks, I learned a lot! I'll accept #benoit answer.
In addition to the perfect and valid answers given so far, there is also the ignore_row_on_dupkey_index hint you might want to use:
create table tq84_a (
name varchar2 (20) primary key,
age number
);
insert /*+ ignore_row_on_dupkey_index(tq84_a(name)) */ into tq84_a values ('Johnny', 77);
insert /*+ ignore_row_on_dupkey_index(tq84_a(name)) */ into tq84_a values ('Pete' , 28);
insert /*+ ignore_row_on_dupkey_index(tq84_a(name)) */ into tq84_a values ('Sue' , 35);
insert /*+ ignore_row_on_dupkey_index(tq84_a(name)) */ into tq84_a values ('Johnny', null);
select * from tq84_a;
The hint is described on Tahiti.
you can use this syntax:
INSERT INTO table_name ( name, age )
select 'jonny', 18 from dual
where not exists(select 1 from table_name where name = 'jonny');
if its open an pop for asking as "enter substitution variable" then use this before the above queries:
set define off;
INSERT INTO table_name ( name, age )
select 'jonny', 18 from dual
where not exists(select 1 from table_name where name = 'jonny');
You should use Merge:
For example:
MERGE INTO employees e
USING (SELECT * FROM hr_records WHERE start_date > ADD_MONTHS(SYSDATE, -1)) h
ON (e.id = h.emp_id)
WHEN MATCHED THEN
UPDATE SET e.address = h.address
WHEN NOT MATCHED THEN
INSERT (id, address)
VALUES (h.emp_id, h.address);
or
MERGE INTO employees e
USING hr_records h
ON (e.id = h.emp_id)
WHEN MATCHED THEN
UPDATE SET e.address = h.address
WHEN NOT MATCHED THEN
INSERT (id, address)
VALUES (h.emp_id, h.address);
https://oracle-base.com/articles/9i/merge-statement
CTE and only CTE :-)
just throw out extra stuff. Here is almost complete and verbose form for all cases of life. And you can use any concise form.
INSERT INTO reports r
(r.id, r.name, r.key, r.param)
--
-- Invoke this script from "WITH" to the end (";")
-- to debug and see prepared values.
WITH
-- Some new data to add.
newData AS(
SELECT 'Name 1' name, 'key_new_1' key FROM DUAL
UNION SELECT 'Name 2' NAME, 'key_new_2' key FROM DUAL
UNION SELECT 'Name 3' NAME, 'key_new_3' key FROM DUAL
),
-- Any single row for copying with each new row from "newData",
-- if you will of course.
copyData AS(
SELECT r.*
FROM reports r
WHERE r.key = 'key_existing'
-- ! Prevent more than one row to return.
AND FALSE -- do something here for than!
),
-- Last used ID from the "reports" table (it depends on your case).
-- (not going to work with concurrent transactions)
maxId AS (SELECT MAX(id) AS id FROM reports),
--
-- Some construction of all data for insertion.
SELECT maxId.id + ROWNUM, newData.name, newData.key, copyData.param
FROM copyData
-- matrix multiplication :)
-- (or a recursion if you're imperative coder)
CROSS JOIN newData
CROSS JOIN maxId
--
-- Let's prevent re-insertion.
WHERE NOT EXISTS (
SELECT 1 FROM reports rs
WHERE rs.name IN(
SELECT name FROM newData
));
I call it "IF NOT EXISTS" on steroids. So, this helps me and I mostly do so.