This question already has answers here:
Can We use threading in PL/SQL?
(9 answers)
Closed 6 years ago.
Basically I have a package in oracle 11g processing a file and validating and inserting information in multiple tables,
to achiev this I create a stored procedure that reads the file and spread the information, then I call multiple stored procedures to
validate and insert the data in each table (one procedure per table), for errors each SP insert a record in a common error table, at the end,
I call one last stored procedure that identifies if there is errors in the common error table and generate a file with those errors.
Now... I'm trying to improve the code in order to minimize times of execution, then I realice that each SP that validates and insert info
into table does not depend from other SP information, so I'm asking if there is a way to call all this SP in parallel.
TODAY
STORED PROCEDURE charge_file
STORED PROCEDURE insert_table1
STORED PROCEDURE insert_table2
STORED PROCEDURE insert_table3 ...
STORED PROCEDURE return_file
What I am trying to do
STORED PROCEDURE charge_file
STORED PROCEDURE insert_table1 - STORED PROCEDURE insert_table2 - STORED PROCEDURE insert_table3 ...
STORED PROCEDURE return_file
As example using parallel_execute:
create table proc_map (proc_id number, proc_name varchar2(64), is_active varchar2(1));
insert into proc_map (proc_id, proc_name, is_active) values (1, 'insert_table1', 'Y');
insert into proc_map (proc_id, proc_name, is_active) values (2, 'insert_table2', 'Y');
insert into proc_map (proc_id, proc_name, is_active) values (3, 'insert_table3', 'Y');
create or replace procedure p_run_proc (ip_start in number, ip_end in number) is
v_proc_name proc_map.proc_name%type;
begin
begin
select t.proc_name into v_proc_name
from proc_map t
where t.proc_id = ip_start;
exception
when no_data_found then null;
when too_many_rows then null;
end;
if v_proc_name is not null
then
execute immediate 'begin ' || v_proc_name || '; end;';
end if;
end;
declare
v_task_name varchar2(4000) := dbms_parallel_execute.generate_task_name;
v_sql varchar2(4000);
v_run varchar2(4000);
v_thread_count number;
v_task_status number;
begin
dbms_parallel_execute.create_task (task_name => v_task_name);
v_sql := 'select t.proc_id as num_col
,t.proc_id as num_col
from proc_map t
where t.is_active = ''Y''
order by t.proc_id';
dbms_parallel_execute.create_chunks_by_SQL (task_name => v_task_name, sql_stmt => v_sql, by_rowid => false);
v_run := 'begin p_run_proc (ip_start => :start_id, ip_end => :end_id); end;';
select count(*) into v_thread_count
from proc_map t
where t.is_active = 'Y';
dbms_parallel_execute.run_task (task_name => v_task_name
,sql_stmt => v_run
,language_flag => dbms_sql.native
,parallel_level => v_thread_count);
v_task_status := dbms_parallel_execute.task_status (task_name => v_task_name);
if v_task_status = dbms_parallel_execute.FINISHED
then
dbms_parallel_execute.drop_task (task_name => v_task_name);
else
raise_application_error (-20001, 'ORA in task ' || v_task_name);
end if;
end;
There is no built-in fork/background submit in PL/SQL, other than DBMS_SCHEDULER or the older DBMS_JOB. (DBMS_SCHEDULER is more fully featured and has far better tracking for failures.) I wrote a job control object a few years ago as a project but to be honest I've never used it. www.williamrobertson.net/documents/job-control-object.html
You can use run_job or submit.
Refer below link for example :-
http://www.dba-oracle.com/r_execute_pl_sql_in_parallel.htm
Related
I have written a package :
CREATE OR REPLACE
PACKAGE BODY NEW_HIRE_PKG
AS
PROCEDURE load_emp(
errbuf OUT VARCHAR2,
retcode OUT VARCHAR2 )
AS
CURSOR cur_person_info
IS
SELECT * FROM table_abc;
CURSOR cur_person_adr
IS
SELECT * FROM table_adr;
l_person_id NUMBER;
l_emp_num NUMBER;
lv_add_type VARCHAR2(100);
lv_address_line1 VARCHAR2(100);
BEGIN
FOR person_info_rec IN cur_address_info
LOOP
hr_employee_api.create_employee ( p_validate => FALSE,
--INPUT Parameter
P_HIRE_DATE =>person_info_rec.DATE_START,
-- output
p_employee_number => lc_employee_number, p_person_id => ln_person_id );
END LOOP ;
END;
PROCEDURE load_add(
errbuf OUT VARCHAR2,
retcode OUT VARCHAR2 )
as
ln_person_id number;
BEGIN
FOR address_info_rec IN cur_address_info
LOOP
BEGIN
hr_person_address_api.create_person_address
(p_validate => FALSE,
p_effective_date => TRUNC(SYSDATE),
p_person_id=> ln_person_id,
--output
p_address_type => lv_add_type,
p_address_line1 => lv_address_line1);
end;
end loop;
end;
end;
Now in the procedure the load_add there is a variable ln_person_id which should be the person ids generated in the procedure load_emp. I want to pass it in this procedure one by one. Can i do it by making ln_person_id an object ?
Judging by your code you have two concurrent programs - one that calls load_emp() and one that calls load_add(). If that's really the structure you want then the two calls will run in separate sessions and there's nothing you can do to pass a variable from one to the other. The best you could do would be to hold all the person_id values from load_emp in a custom table. The data could then later be consumed by load_add().
However I would restructure your package. Why not call load_add() from within the LOOP in load_emp() ?
I have a master stored procedure:
MASTER();
In this stored procedure, I call 3 other stored procedures:
SP1();
SP2();
SP3();
Right now, it's running serially, i.e. one after the other. I want to run it in parallel and once all the 3 stored procedures are completely executed, run next part of MASTER() stored procedure.
I am using Oracle Standard One Edition 11.2. How can I achieve this?
One way to archive that is to use DBMS_JOB or DBMS_SCHEDULER to launch the procedures in parallel and DBMS_ALERT to notify the master procedure when they are finished.
There is other way to achieve this.
Set up envirement:
create table t_procedure(id number, procedures varchar2(200));
create type l_char is table of varchar2(100);
create procedure goSleeep(p_sec number)is
begin
dbms_lock.sleep( p_sec );
end;
Package:
create or replace package goParallel is
TYPE t_referential_cursor IS REF CURSOR RETURN t_procedure%ROWTYPE;
function runParallel(p_cursor t_referential_cursor)
return l_char pipelined
parallel_enable(partition p_cursor BY HASH(id));
end;
create or replace package body goParallel is
function runParallel(p_cursor t_referential_cursor)
return l_char pipelined
parallel_enable(partition p_cursor BY HASH(id))
is
v_start date := sysdate;
v_end date;
vid number;
p_proc varchar2(200);
begin
loop
fetch p_cursor into vid, p_proc;
exit when p_cursor%notfound;
execute immediate p_proc;
v_end := sysdate;
pipe row( vid||' --- '||to_char(v_start,'HH24:MI:SS')||' - '|| to_char(v_end,'HH24:MI:SS'));
end loop;
return;
end;
end;
Insert some procedures to run.
insert into t_procedure values (1, 'begin goSleeep(5); end;');
insert into t_procedure values (2, 'begin goSleeep(8); end;');
insert into t_procedure values (3, 'begin goSleeep(9); end;');
commit;
And run it with strange way.
select * from table(goParallel.runParallel(cursor(select /*+ PARALLEL(a 8) */ * from t_procedure a)));
-- result: id - start - end
1 --- 12:15:54 - 12:15:59
2 --- 12:15:54 - 12:16:02
3 --- 12:15:54 - 12:16:03
I have a package which declares a collection of type table of some database table's %rowtype. It also declares a function to populate the package-level variable with some data. I can now print the data with dbms_output, seems fine.
But when I use the package-level variable in some sql I get the following error:
ORA-21700: object does not exist or is marked for delete
ORA-06512: at "TESTDB.SESSIONGLOBALS", line 17
ORA-06512: at line 5
Here is my code:
create some dummy data:
drop table "TESTDATA";
/
CREATE TABLE "TESTDATA"
( "ID" NUMBER NOT NULL ENABLE,
"NAME" VARCHAR2(20 BYTE),
"STATUS" VARCHAR2(20 BYTE)
);
/
insert into "TESTDATA" (id, name, status) values (1, 'Hans Wurst', 'J');
insert into "TESTDATA" (id, name, status) values (2, 'Hans-Werner', 'N');
insert into "TESTDATA" (id, name, status) values (3, 'Hildegard v. Bingen', 'J');
/
now create the package:
CREATE OR REPLACE
PACKAGE SESSIONGLOBALS AS
type t_testdata is table of testdata%rowtype;
v_data t_testdata := t_testdata();
function load_testdata return t_testdata;
END SESSIONGLOBALS;
and the package body:
CREATE OR REPLACE
PACKAGE BODY SESSIONGLOBALS AS
function load_testdata return t_testdata AS
v_sql varchar2(500);
BEGIN
if SESSIONGLOBALS.v_data.count = 0
then
v_sql := 'select * from testdata';
execute immediate v_sql
bulk collect into SESSIONGLOBALS.v_data;
dbms_output.put_line('data count:');
dbms_output.put_line(SESSIONGLOBALS.v_data.count);
end if; -- SESSIONGLOBALS.v_data.count = 0
-- ******************************
-- this line throws the error
insert into testdata select * from table(SESSIONGLOBALS.v_data);
-- ******************************
return SESSIONGLOBALS.v_data;
END load_testdata;
END SESSIONGLOBALS;
execute the sample:
DECLARE
v_Return SESSIONGLOBALS.T_TESTDATA;
BEGIN
v_Return := SESSIONGLOBALS.LOAD_TESTDATA();
dbms_output.put_line('data count (direct access):');
dbms_output.put_line(SESSIONGLOBALS.v_data.count);
dbms_output.put_line('data count (return value of function):');
dbms_output.put_line(v_Return.count);
END;
If the line marked above is commented out i get the expected result.
So can anyone tell me why the exception stated above occurs?
BTW: it is absolutely nessecary for me to execute the statement which populates the collection with data as dynamic sql because the tablename is not known at compiletime. (v_sql := 'select * from testdata';)
the solution is to use pipelined functions in the package
see: http://docs.oracle.com/cd/B19306_01/appdev.102/b14289/dcitblfns.htm#CHDJEGHC ( => section Pipelining Between PL/SQL Table Functions does the trick).
my package looks like this now (please take the table script from my question):
create or replace
PACKAGE SESSIONGLOBALS AS
v_force_refresh boolean;
function set_force_refresh return boolean;
type t_testdata is table of testdata%rowtype;
v_data t_testdata;
function load_testdata return t_testdata;
function get_testdata return t_testdata pipelined;
END SESSIONGLOBALS;
/
create or replace
PACKAGE BODY SESSIONGLOBALS AS
function set_force_refresh return boolean as
begin
SESSIONGLOBALS.v_force_refresh := true;
return true;
end set_force_refresh;
function load_testdata return t_testdata AS
v_sql varchar2(500);
v_i number(10);
BEGIN
if SESSIONGLOBALS.v_data is null then
SESSIONGLOBALS.v_data := SESSIONGLOBALS.t_testdata();
end if;
if SESSIONGLOBALS.v_force_refresh = true then
SESSIONGLOBALS.v_data.delete;
end if;
if SESSIONGLOBALS.v_data.count = 0
then
v_sql := 'select * from testdata';
execute immediate v_sql
bulk collect into SESSIONGLOBALS.v_data;
end if; -- SESSIONGLOBALS.v_data.count = 0
return SESSIONGLOBALS.v_data;
END load_testdata;
function get_testdata return t_testdata pipelined AS
v_local_data SESSIONGLOBALS.t_testdata := SESSIONGLOBALS.load_testdata();
begin
if v_local_data.count > 0 then
for i in v_local_data.first .. v_local_data.last
loop
pipe row(v_local_data(i));
end loop;
end if;
end get_testdata;
END SESSIONGLOBALS;
/
now i can do a select in sql like this:
select * from table(SESSIONGLOBALS.get_testdata());
and my data collection is only populated once.
nevertheless it is quite not comparable with a simple
select * from testdata;
from a performace point of view but i'll try out this concept for some more complicated use cases. the goal is to avoid doing some really huge select statements involving lots of tables distributed among several schemas (english plural for schema...?).
The syntax you use does not work:
insert into testdata select * from table(SESSIONGLOBALS.v_data); -- does not work
You have to use something like that:
forall i in 1..v_data.count
INSERT INTO testdata VALUES (SESSIONGLOBALS.v_data(i).id,
SESSIONGLOBALS.v_data(i).name,
SESSIONGLOBALS.v_data(i).status);
(which actually duplicates the rows in the table)
Package-level types cannot be used in SQL. Even if your SQL is called from within a package, it still can't see that package's types.
I'm not sure how you got that error message, when I compiled the package I got this error, which gives a good hint at the problem:
PLS-00642: local collection types not allowed in SQL statements
To fix this problem, create a type and a nested table of that type:
create or replace type t_testdata_rec is object
(
"ID" NUMBER,
"NAME" VARCHAR2(20 BYTE),
"STATUS" VARCHAR2(20 BYTE)
);
create or replace type t_testdata as table of t_testdata_rec;
/
The dynamic SQL to populate the package variable gets more complicated:
execute immediate
'select cast(collect(t_testdata_rec(id, name, status)) as t_testdata)
from testdata ' into SESSIONGLOBALS.v_data;
But now the insert will work as-is.
I want to know whether I can use the OLD and NEW objects for dynamic operations inside trigger.
What I am looking for is something like this :-
ABC is a table for which I need to write Trigger.
TracK_Table maintains list of columns for table which need to be tracked (logged).
f_log is a function that inserts changes in data into a tracking(log) table.
CREATE OR REPLACE TRIGGER trg_TRACK
AFTER INSERT OR UPDATE OR DELETE ON ABC
FOR EACH ROW
declare
v_old_val varchar2(1000);
v_new_val varchar2(1000);
n_ret int;
n_id varchar(50);
cursor cur_col is
SELECT COLUMN_NAME,
TABLE_name
FROM track_TABLE
WHERE upper(TABLE_NAME) = upper('ABC')
AND exists (select cname
from col
where UPPER(tname) =upper('ABC')
and upper(cname)=upper(COLUMN_NAME))
AND upper(allow) = 'Y';
begin
n_id:= :old.id;
for i_get_col in c_get_col
loop
execute immediate
'begin
:v_old_val:= select '||i_get_col.column_name ||'
from '||:old ||'
where id = '||n_id ||';
end;' using out v_old_val;
execute immediate
'begin
:v_new_val:= select '||i_get_col.column_name ||'
from '||:new ||'
where id = '||n_id ||';
end;' using out v_new_val;
n_ret := f_log(n_id,i_get_col.column_name,v_old_val,v_new_val);
end loop;
end;
/
One Option: Push the logic to check if a column is being tracke into the f_log procedure and then pass across all of the columns.
For example, if your track_Table holds (table_name, column_name, allow) values for each column that you want to trackm then something like this
CREATE OF REPLACE PROCEDURE f_log( p_id varchar2
,p_table_name varchar2
,p_column_name varchar2
,p_old_val varchar2
,p_new_val varchar2)
as
l_exists number;
cursor chk_column_track IS
SELECT 1
FROM track_TABLE
WHERE upper(TABLE_NAME) = upper(p_table_name)
AND UPPER(column_name) = upper(p_column_name)
AND upper(allow) = 'Y';
begin
open chk_column_track;
fetch chk_column_track into l_exists;
if chk_column_track%found then
--do the insert here
end if;
close chk_column_track;
end;
/
CREATE OR REPLACE TRIGGER trg_TRACK
AFTER INSERT OR UPDATE OR DELETE ON ABC
FOR EACH ROW
DECLARE
n_id varchar(50);
BEGIN
n_id := NVL(:old.id, :new.id);
-- send all of the values to f_log and have it decide whether to save them
f_log(:old.id,'COL1',:old.col1,:new.col1);
f_log(:old.id,'COL2',:old.col2,:new.col2);
f_log(:old.id,'COL3',:old.col3,:new.col3);
...
END;
And for goodness sake, upper-case the values in your track_table on insert so that you don't have to UPPER() the stored values thus making any index on those values useless!
Now, this will chew up some resources checking each column name on each operation, but if you are not running high-volumes then it might be manageable.
Otherwise you will need a more elegant solution. Like leveraging the power of collections and the TABLE() clause to do the track_table lookup in a bulk operation. Bear in mind that I am away from my database at the moment, so I have not test-compiled this code.
CREATE OR REPLACE TYPE t_audit_row AS OBJECT (
p_table_name varchar2(30)
,p_column_name varchar2(30)
,p_id varchar2(50)
,p_old_val varchar2(2000)
,p_new_val varchar2(2000)
);
CREATE OR REPLACE TYPE t_audit_row_table AS TABLE OF t_audit_row;
CREATE OR REPLACE PROCEDURE f_log (p_audit_row_table t_audit_Row_table)
AS
begin
-- see how we can match the contents of the collection to the values
-- in the table all in one query. the insert is just my way of showing
-- how this can be done in one bulk operation. Alternately you could make
-- the select a cursor and loop through the rows to process them individually.
insert into my_audit_log (table_name, column_name, id, old_val, new_val)
select p_table_name
,p_column_name
,p_id
,p_old_val
,p_new_val
FROM track_TABLE TT
,table(p_audit_row_table) art
WHERE tt.TABLE_NAME = art.p_table_name
AND tt.column_name = art.p_column_name
AND tt.allow = 'Y';
end;
/
CREATE OR REPLACE TRIGGER trg_TRACK
AFTER INSERT OR UPDATE OR DELETE ON ABC
FOR EACH ROW
DECLARE
l_id varchar(50);
l_audit_table t_audit_row_table;
BEGIN
l_id := NVL(:old.id, :new.id);
-- send all of the values to f_log and have it decide whether to save them
l_audit_table := t_audit_row_table (
t_audit_row ('ABC','COL1',l_id, :old.col1, :new.col1)
,t_audit_row ('ABC','COL2',l_id, :old.col2, :new.col2)
,t_audit_row ('ABC','COL3',l_id, :old.col3, :new.col3)
,...
,t_audit_row ('ABC','COLn',l_id, :old.coln, :new.coln)
);
f_log(l_audit_table);
end;
/
No, you cannot access the OLD and NEW pseudo-variables dynamically. What you can do is use your track_table data in a script or procedure to generate static triggers that look like:
CREATE OR REPLACE TRIGGER trg_TRACK
AFTER INSERT OR UPDATE OR DELETE ON ABC
FOR EACH ROW
DECLARE
n_id varchar(50);
BEGIN
n_id := NVL(:old.id, :new.id);
f_log(:old.id,'COL1',:old.col1,:new.col1);
f_log(:old.id,'COL3',:old.col3,:new.col3);
...
END;
So if the data in the TRACK_CHANGES table changes you just have to re-generate the triggers.
I have to write an Oracle procedure which should invoke an Oracle function returning REF_CURSOR. The function is declared like that
FUNCTION "IMPACTNET"."TF_CONVERTPARA" (PARASTRING IN NVARCHAR2) RETURN SYS_REFCURSOR
AS
c SYS_REFCURSOR;
BEGIN
OPEN c FOR
SELECT SUBSTR(element, 1, INSTR(element, '|') - 1) as key,
SUBSTR(element, INSTR(element, '|') + 1, 99999) as val
FROM (
SELECT REGEXP_SUBSTR(PARASTRING, '[^;]+', 1, LEVEL) element
FROM dual
CONNECT BY LEVEL < LENGTH(REGEXP_REPLACE(PARASTRING, '[^;]+')) + 1
);
RETURN c;
END;
Can you tell me what I need to write in order to invoke the function from within my procedure? I'd like to insert all the returned values (shaped a table with two columns) into a rational table.
Thank you in advance!
Something along the lines of this should work (obviously, I'm guessing about table names and column names and the exact logic that you're trying to implement)
CREATE PROCEDURE some_procedure_name
AS
l_rc SYS_REFCURSOR := impactnet.tf_convertpara( <<some string>> );
l_key VARCHAR2(100);
l_val VARCHAR2(100);
BEGIN
LOOP
FETCH l_rc
INTO l_key, l_val;
EXIT WHEN l_rc%notfound;
INSERT INTO some_table( key_column, val_column )
VALUES( l_key, l_val );
END LOOP;
END;
As Ollie points out, it would be more efficient to do a BULK COLLECT and a FORALL. If you're just dealing with a few thousand rows (since your function is just parsing the data in a delimited string, I'm assuming you expect relatively few rows to be returned), the performance difference is probably minimal. But if you're processing more data, the difference can be quite noticeable. Depending on the Oracle version and your specific requirements, you may be able to simplify the INSERT statement in the FORALL to insert a record rather than listing each column from the record individually.
CREATE PROCEDURE some_procedure_name
AS
TYPE key_val_rec
IS RECORD(
key VARCHAR2(100),
val VARCHAR2(100)
);
TYPE key_val_coll
IS TABLE OF key_val_rec;
l_rc SYS_REFCURSOR := impactnet.tf_convertpara( <<some string>> );
l_coll key_val_coll;
BEGIN
LOOP
FETCH l_rc
BULK COLLECT INTO l_coll
LIMIT 100;
EXIT WHEN l_coll.count = 0;
FORALL i IN l_coll.FIRST .. l_coll.LAST
INSERT INTO some_table( key_column, val_column )
VALUES( l_coll(i).key, l_coll(i).val );
END LOOP;
END;