I have a procedure as follows,
CREATE OR REPLACE PROCEDURE engineering_all ( idx IN NUMBER )
IS
tempstmt VARCHAR2(2000);
BEGIN
create_table_like( 'results_temp', 'results', 1);
tempstmt := 'ALTER TABLE results_temp CACHE';
EXECUTE IMMEDIATE tempstmt;
engineering('CONSERVATIVE', idx);
engineering('INTERMEDIATE', idx);
engineering('AGGRESSIVE', idx);
END;
/
Three calls to procedure engineering are independent of each other, so I want to parallelize this. I came across multiple ways like DBMS_PARALLEL_EXECUTE, DBMS_JOB, DBMS_SCHEDULER but not able to figure out which one is the most effective for my time-optimization objective.
Please help me out to figure out which one to choose and how can I implement that?
I suggest to use DBMS_PARALLEL_EXECUTE. The main session waits till child parallel sessions are finished. There are convenient views with the statistics and results - user_parallel_execute_chunks and user_parallel_execute_tasks. We use it in our project, find it quite convenient.
One point is that this package requires chunking that can be done only by rowid or by numbers. So first of all you have to create the procedure wich accepts numbers. Here's the one for your case:
create or replace procedure engineering_parallel(
iLaunchType number,
idx number
)
is
begin
if iLaunchType = 1 then
engineering('CONSERVATIVE',idx);
elsif iLaunchType = 2 then
engineering('INTERMEDIATE',idx);
elsif iLaunchType = 3 then
engineering('AGGRESSIVE',idx);
end if;
end;
And here you will find an example of launching your case in anonymous pl/sql block, you can easily convert it into engineering_all procedure:
declare
-- idx parameter
idx number := 0;
-- unique parallel task name
sTaskName varchar2(32767) := 'ENGINEERING-'||to_char(sysdate,'yyyy-mm-dd-hh24-mi-ss');
-- this is where you store the query to split into chunks
sChunkSQL varchar2(32767) := 'select level start_id, '||idx||' end_id'||chr(10)||
'from dual connect by level <= 3';
-- this is the procedure call
sParallelSQL varchar2(32767) := 'begin engineering_parallel(:start_id,:end_id); end;';
-- parallel degree
iParalleDegree number := 3;
begin
-- create a task
DBMS_PARALLEL_EXECUTE.create_task(task_name => sTaskName);
-- chunking
DBMS_PARALLEL_EXECUTE.create_chunks_by_sql(
task_name => sTaskName,
sql_stmt => sChunkSQL,
by_rowid => FALSE
);
-- launch. current session waits till all child parallel sessions are finished
DBMS_PARALLEL_EXECUTE.run_task(
task_name => sTaskName,
sql_stmt => sParallelSQL,
language_flag => DBMS_SQL.NATIVE,
parallel_level => iParalleDegree
);
dbms_output.put_line(
'Job is finished.'||
'Check user_parallel_execute_chunks, user_parallel_execute_tasks for the task '||
sTaskName
);
end;
Last point to consider - check if your version includes the fix of Bug 18966843:
DBMS_PARALLEL_EXECUTE PERFORMANCE DELAY AFTER UPGRADE TO 11.2.0.4
We faced it in 12.1, but there are patches to fix it.
If it's not fixed then you have a chance that the parallel degree at the end will be less than requested (down to 1).
I never used the first option you mentioned, but - choosing between DBMS_JOB and DBMS_SCHEDULER, DBMS_JOB is simpler so I'd choose that. In its simplest way, procedure might look like this:
CREATE OR REPLACE PROCEDURE engineering_all (idx IN NUMBER)
IS
l_job NUMBER;
tempstmt VARCHAR2 (2000);
BEGIN
create_table_like ('results_temp', 'results', 1);
tempstmt := 'ALTER TABLE results_temp CACHE';
EXECUTE IMMEDIATE tempstmt;
DBMS_JOB.submit (
job => l_job,
what => 'begin engineering(''CONSERVATIVE'', ' || idx || '); end;',
next_date => SYSDATE,
interval => NULL);
DBMS_JOB.submit (
job => l_job,
what => 'begin engineering(''INTERMEDIATE'', ' || idx || '); end;',
next_date => SYSDATE,
interval => NULL);
DBMS_JOB.submit (
job => l_job,
what => 'begin engineering(''AGGRESSIVE'', ' || idx || '); end;',
next_date => SYSDATE,
interval => NULL);
COMMIT;
END;
/
Related
What I want is to execute a procedure ( that will run on Oracle 11g) concurrently, and to commit the whole operation if an only if all the concurrent transactions succeeded.
Two ways of parallel execution that I thought of were DBMS_PARALLEL_EXECUTE and dbms_job.submit(), but as I understand it, in both cases the processes created are running in their separate sessions and each process commits the changes upon termination (or in case of an error can rollback its own changes).
What I would like, is to start parallel processes, wait until each one of them is finished, check if they all were successful and only then commit the changes (or rollback if at least one process failed).
Is the above scenario possible? (And how can it be implemented)
Thanks.
I am curious why this requirement came about; I would probably question whether it was really necessary if this came to me. But if you cannot question the requirement, this is how I would go about it.
-- We need to have a common persistent location for all of the jobs to read
CREATE TABLE test_tab
(
job_name VARCHAR2(30),
status VARCHAR2(30)
)
/
-- The procedure writing to our table must be autonomous so that updates occur
-- without committing the rest of the work
CREATE OR REPLACE PROCEDURE test_log
(
i_job_name IN VARCHAR2,
i_status IN VARCHAR2
) IS
PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
MERGE INTO test_tab tgt
USING dual
ON (tgt.job_name = i_job_name)
WHEN MATCHED THEN
UPDATE SET status = i_status
WHEN NOT MATCHED THEN
INSERT VALUES (i_job_name, i_status);
COMMIT;
END test_log;
/
CREATE OR REPLACE PROCEDURE test_proc(i_job_name IN VARCHAR2) IS
l_complete_cnt INTEGER;
l_error_cnt INTEGER;
l_waiting BOOLEAN := TRUE;
BEGIN
-- !!! Your code here !!!
/* -- Uncomment this block to prove the rollback scenario.
IF i_job_name LIKE '%8' THEN
raise_application_error(-20001, 'Throwing an error to prove rollback.');
END IF;*/
test_log(i_job_name, 'COMPLETE');
WHILE l_waiting LOOP
SELECT SUM(CASE WHEN status IN ('COMPLETE', 'COMMITTED') THEN 1 ELSE 0 END)
,SUM(CASE WHEN status = 'ERROR' THEN 1 ELSE 0 END)
INTO l_complete_cnt
,l_error_cnt
FROM test_tab
WHERE REGEXP_LIKE(job_name, 'TEST_JOB_\d');
IF l_complete_cnt = 8 THEN
COMMIT;
test_log(i_job_name, 'COMMITTED');
l_waiting := FALSE;
ELSIF l_error_cnt > 0 THEN
ROLLBACK;
test_log(i_job_name, 'ROLLBACK');
l_waiting := FALSE;
ELSE
dbms_lock.sleep(seconds => 5);
END IF;
END LOOP;
EXCEPTION
WHEN OTHERS THEN
test_log(i_job_name, 'ERROR');
RAISE;
END;
/
-- Begin test section
BEGIN
FOR i IN 1..8 LOOP
dbms_scheduler.create_job('TEST_JOB_'||i
,'PLSQL_BLOCK'
,'BEGIN test_proc(''TEST_JOB_'||i||'''); END;'
,start_date => SYSDATE
,enabled => TRUE);
END LOOP;
END;
/
SELECT * FROM test_tab;
TRUNCATE TABLE test_tab; --the table should be cleared between tests
Using Oracle 11.2
Hi,
Here is what I want to do: I'm scheduling jobs using dbms_scheduler. The number of jobs to schedule is not fixed and a max of 4 jobs should run at the same time. The procedure scheduling the jobs should wait until all jobs are completed. If one job fails, the "schedule" procedure should also fail and all remaining scheduled jobs should be deleted from the scheduler.
Currently I have had to sleeping and polling the table user_scheduler_jobs in a loop.
I'm new to PL/SQL and rather inexperienced so please don't be too harsh on me ;)
Here is my code so far.
First the snippet for scheduling the jobs:
BEGIN
FOR r IN (SELECT p_values FROM some_table WHERE flag = 0 )
LOOP
-- count running jobs
SELECT count(*) INTO v_cnt
FROM user_scheduler_jobs
WHERE job_name LIKE 'something%';
/*
If max number of parallel jobs is reached, then wait before starting a new one.
*/
WHILE v_cnt >= l_max_parallel_jobs
LOOP
dbms_lock.sleep(10);
SELECT count(*) INTO v_cnt
FROM user_scheduler_jobs
WHERE job_name LIKE 'something%' AND state = 'RUNNING';
SELECT count(*) INTO v_cnt_failures
FROM user_scheduler_jobs
WHERE job_name LIKE 'something%' AND state = 'FAILED' OR state = 'BROKEN';
IF v_cnt_failures > 0 THEN RAISE some_exception; END IF;
END LOOP;
-- Start a new Job
v_job_name := 'something_someting_' || p_values;
v_job_action := 'begin user.some_procedure(''' || r.p_values || '''); end;';
dbms_scheduler.create_job(job_name => v_job_name,
job_type => 'PLSQL_BLOCK',
job_action => v_job_action,
comments => 'Some comment ' || v_job_name,
enabled => FALSE,
auto_drop => FALSE);
dbms_scheduler.set_attribute(NAME => v_job_name,
ATTRIBUTE => 'max_failures',
VALUE => '1');
dbms_scheduler.set_attribute(NAME => v_job_name,
ATTRIBUTE => 'max_runs',
VALUE => '1');
dbms_scheduler.enable(v_job_name);
v_job_count := v_job_count + 1;
-- array for all jobs
v_jobs_aat(v_job_count) := v_job_name;
END LOOP;
-- ... Wait till all jobs have finisched.
check_queue_completion(v_jobs_aat); -- see procedure below
END;
Procedure for waiting till last four jobs have finisched:
PROCEDURE check_queue_completion(p_jobs_aat IN OUT t_jobs_aat) AS
v_state user_scheduler_jobs.state%TYPE;
v_index PLS_INTEGER;
v_done BOOLEAN := TRUE;
-- Exceptions
e_job_failure EXCEPTION;
BEGIN
WHILE v_done
LOOP
v_done := FALSE;
FOR i IN p_jobs_aat.first..p_jobs_aat.last
LOOP
SELECT state INTO v_state FROM user_scheduler_jobs WHERE job_name = p_jobs_aat(i);
--dbms_output.put_line('Status: ' || v_state);
CASE
WHEN v_state = 'SUCCEEDED' OR v_state = 'COMPLETED' THEN
dbms_output.put_line(p_jobs_aat(i) || ' SUCCEEDED');
dbms_scheduler.drop_job(job_name => p_jobs_aat(i), force => TRUE);
p_jobs_aat.delete(i);
WHEN v_state = 'FAILED' OR v_state = 'BROKEN' THEN
--Exception auslösen
dbms_output.put_line(p_jobs_aat(i) || ' FAILED');
RAISE e_job_failure;
WHEN v_state = 'RUNNING' OR v_state = 'RETRY SCHEDULED' THEN
NULL;
dbms_output.put_line(p_jobs_aat(i) || ' RUNNING or RETRY SCHEDULED');
v_done := TRUE;
/*DISABLED, SCHEDULED, REMOTE, CHAIN_STALLED*/
ELSE
dbms_output.put_line(p_jobs_aat(i) || ' ELSE');
dbms_scheduler.drop_job(job_name => p_jobs_aat(i), force => TRUE);
p_jobs_aat.delete(i);
END CASE;
END LOOP;
hifa.gen_sleep(30);
END LOOP;
IF p_jobs_aat.count > 0 THEN delete_jobs_in_queue(p_jobs_aat); END IF;
EXCEPTION WHEN e_job_failure THEN
delete_jobs_in_queue(p_jobs_aat);
RAISE_APPLICATION_ERROR(-20500, 'some error message');
END check_queue_completion;
It does the trick but it seems like some awful hack.
Isn't there a better way to:
Wait until all jobs have finished.
Just run four jobs at a time and start a new one as soon as one of the running jobs has finished.
Throw an exception if one job fails or is broken.
DECLARE
cnt NUMBER:=1;
BEGIN
WHILE cnt>=1
LOOP
SELECT count(1) INTO cnt FROM dba_scheduler_running_jobs srj
WHERE srj.job_name IN ('TEST_JOB1','TEST_JOB2');
IF cnt>0 THEN
dbms_lock.sleep (5);
END IF;
END LOOP;
dbms_output.put_line('ASASA');
END;
Use dbms_alert or dbms_pipe to send/receive information about job start/finish. Query the jobs table only if you do not receive the information in expected time.
Oracle Scheduler uses Oracle Rersource Manager heavily, Just submit your jobs, defined with an end notification and have a task waiting for your event Q that counts the jobs that are submitted and the jobs that are finished.
You use Oracle Resource manager to control the maximum number of jobs to run concurrently. This will also be based on the total database load, protecting other users agains a system flooded by jobs.
In a legacy system there is some PL/SQL procedure that calls the another procedure mutliple times with different parameters. The procedure contains a lot of PL/SQL logic (if, then, else).
As the execution of this procedure takes very long, we thought about using concurrency to speed things up without even touching the actual logic.
I understand that there are several ways of running (PL/)SQL in parallel on oracle (see bellow).
However, I wasn't able to find a way to pass different arguments/parameters to a PL/SQL procedure, execute them in parallel and wait until all procedures are finished executing (i.e. I'm looking for mechanism to join all threads or for a barrier mechanism in oracle).
Let's use the following simplified example on the SCOTT Schema:
DECLARE
PROCEDURE DELETE_BONUS(
in_job IN VARCHAR2)
IS
BEGIN
-- Imagine a lot of IF, ELSEIF, ELSE statements here
DELETE FROM BONUS WHERE JOB=in_job;
END;
BEGIN
INSERT into BONUS(ENAME, JOB) SELECT ROWNUM, 'A' FROM DUAL CONNECT BY LEVEL <= 1000000;
INSERT into BONUS(ENAME, JOB) SELECT ROWNUM, 'B' FROM DUAL CONNECT BY LEVEL <= 1000000;
INSERT into BONUS(ENAME, JOB) SELECT ROWNUM, 'C' FROM DUAL CONNECT BY LEVEL <= 1000000;
-- TODO execute those in parallel
DELETE_BONUS('A');
DELETE_BONUS('B');
DELETE_BONUS('C');
-- TODO wait for all procedures to finish
EXCEPTION
WHEN OTHERS THEN
RAISE;
END;
/
Here's what I found so far:
DBMS_JOB (deprecated)
DBMS_SCHEDULER (how to wait for jobs to finish? LOCKS?)
DBMS_SCHEDULER CHAINS (passing parameters/arguments is not really possible?!)
DBMS_PARALLEL_EXECUTE (can be used to run SQL queries in parallel but not PL/SQL procedures)
Can one of these approaches be used to fork and join the procedure calls? Or is there yet another approach that can?
I solved the problem using DBMS_SCHEDULER and PIPEs for synchronization/IPC that does not rely on polling and does not need additional tables. It still wakes once per finished job, though.
It's quite some effort, so if some can propose a simpler solution please share it!
Define a procedure that calls the actual procedure that can be run
from a program/job and handles IPC (write message to pipe when finished).
Define a program that calls this procedure and defines arguments to be passed to the procedure
Define a procedure that creates a job from the program, maps parameters to job arguments and runs the job
Define logic that waits for all jobs to finish: Wait until every job has sent a message on the pipe.
--
-- Define stored procedures to be executed by job
--
/** Actual method that should be run in parallel*/
CREATE OR REPLACE PROCEDURE PROC_DELETE_TEST_BONUS(
in_job IN VARCHAR2)
IS
BEGIN
-- Imagine a lot of IF, ELSEIF, ELSE statements here
DELETE FROM TEST_BONUS WHERE JOB=in_job;
END;
/
/** Stored procedure to be run from the job: Uses pipes for job synchronization, executes PROC_DELETE_TEST_BONUS. */
CREATE OR REPLACE PROCEDURE PROC_DELETE_TEST_BONUS_CONCUR(in_pipe_name IN VARCHAR2,
in_job IN VARCHAR2)
IS
flag INTEGER;
BEGIN
-- Execute actual procedure
PROC_DELETE_TEST_BONUS(in_job);
-- Signal completion
-- Use the procedure to put a message in the local buffer.
DBMS_PIPE.PACK_MESSAGE(SYSDATE ||': Success ' ||in_job);
-- Send message, success is a zero return value.
flag := DBMS_PIPE.SEND_MESSAGE(in_pipe_name);
EXCEPTION
WHEN OTHERS THEN
-- Signal completion
-- Use the procedure to put a message in the local buffer.
DBMS_PIPE.PACK_MESSAGE(SYSDATE ||':Failed ' || in_job);
-- Send message, success is a zero return value.
flag := DBMS_PIPE.SEND_MESSAGE(in_pipe_name);
RAISE;
END;
/
--
-- Run Jobs
--
DECLARE
timestart NUMBER;
duration_insert NUMBER;
jobs_amount NUMBER := 0;
retval INTEGER;
message VARCHAR2(4000);
rows_amount NUMBER;
/** Create and define a program that calls PROG_DELETE_TEST_BONUS_CONCUR to be run as job. */
PROCEDURE create_prog_delete_test_bonus
IS
BEGIN
-- define new in each run in order to ease development. TODO Once it works, no need to redefine for each run!
dbms_scheduler.drop_program(program_name => 'PROG_DELETE_TEST_BONUS_CONCUR', force=> TRUE);
dbms_scheduler.create_program ( program_name => 'PROG_DELETE_TEST_BONUS_CONCUR', program_action =>
'PROC_DELETE_TEST_BONUS_CONCUR', program_type => 'STORED_PROCEDURE', number_of_arguments => 2,
enabled => FALSE );
dbms_scheduler.DEFINE_PROGRAM_ARGUMENT( program_name => 'PROG_DELETE_TEST_BONUS_CONCUR',
argument_position => 1, argument_name => 'in_pipe_name', argument_type => 'VARCHAR2');
dbms_scheduler.DEFINE_PROGRAM_ARGUMENT( program_name=>'PROG_DELETE_TEST_BONUS_CONCUR',
argument_position => 2, argument_name => 'in_job', argument_type => 'VARCHAR2');
dbms_scheduler.enable('PROG_DELETE_TEST_BONUS_CONCUR');
END;
/** "Forks" a job that runs PROG_DELETE_TEST_BONUS_CONCUR */
PROCEDURE RUN_TEST_BONUS_JOB(
in_pipe_name IN VARCHAR2,
in_job IN VARCHAR2,
io_job_amount IN OUT NUMBER)
IS
jobname VARCHAR2(100);
BEGIN
jobname:=DBMS_SCHEDULER.GENERATE_JOB_NAME;
dbms_scheduler.create_job(job_name => jobname, program_name =>
'PROG_DELETE_TEST_BONUS_CONCUR');
dbms_scheduler.set_job_argument_value(job_name => jobname, argument_name =>
'in_pipe_name' , argument_value => in_pipe_name);
dbms_scheduler.set_job_argument_value(job_name => jobname, argument_name =>
'in_job' , argument_value => in_job);
dbms_output.put_line(SYSDATE || ': Running job: '|| jobname);
dbms_scheduler.RUN_JOB(jobname, false );
io_job_amount:= io_job_amount+1;
END;
-- Anonymous "Main" block
BEGIN
create_prog_delete_test_bonus;
-- Define private pipe
retval := DBMS_PIPE.CREATE_PIPE(DBMS_PIPE.UNIQUE_SESSION_NAME, 100, FALSE);
dbms_output.put_line(SYSDATE || ': Created pipe: ''' || DBMS_PIPE.UNIQUE_SESSION_NAME || ''' returned ' ||retval);
timestart := dbms_utility.get_time();
INSERT into TEST_BONUS(ENAME, JOB) SELECT ROWNUM, 'A' FROM DUAL CONNECT BY LEVEL <= 1000000;
INSERT into TEST_BONUS(ENAME, JOB) SELECT ROWNUM, 'B' FROM DUAL CONNECT BY LEVEL <= 1000000;
INSERT into TEST_BONUS(ENAME, JOB) SELECT ROWNUM, 'C' FROM DUAL CONNECT BY LEVEL <= 1000000;
COMMIT;
duration_insert := dbms_utility.get_time() - timestart;
dbms_output.put_line(SYSDATE || ': Duration (1/100s): INSERT=' || duration_insert);
SELECT COUNT(*) INTO rows_amount FROM TEST_BONUS;
dbms_output.put_line(SYSDATE || ': COUNT(*) FROM TEST_BONUS: ' || rows_amount);
timestart := dbms_utility.get_time();
-- -- Process sequentially
-- PROC_DELETE_TEST_BONUS('A');
-- PROC_DELETE_TEST_BONUS('B');
-- PROC_DELETE_TEST_BONUS('C');
-- start concurrent processing
RUN_TEST_BONUS_JOB(DBMS_PIPE.UNIQUE_SESSION_NAME, 'A', jobs_amount);
RUN_TEST_BONUS_JOB(DBMS_PIPE.UNIQUE_SESSION_NAME, 'B', jobs_amount);
RUN_TEST_BONUS_JOB(DBMS_PIPE.UNIQUE_SESSION_NAME, 'C', jobs_amount);
-- "Barrier": Wait for all jobs to finish
for i in 1 .. jobs_amount loop
-- Reset the local buffer.
DBMS_PIPE.RESET_BUFFER;
-- Wait and receive message. Timeout after an hour.
retval := SYS.DBMS_PIPE.RECEIVE_MESSAGE(SYS.DBMS_PIPE.UNIQUE_SESSION_NAME, 3600);
-- Handle errors: timeout, etc.
IF retval != 0 THEN
raise_application_error(-20000, 'Error: '||to_char(retval)||' receiving on pipe. See Job Log in table user_scheduler_job_run_details');
END IF;
-- Read message from local buffer.
DBMS_PIPE.UNPACK_MESSAGE(message);
dbms_output.put_line(SYSDATE || ': Received message on '''|| DBMS_PIPE.UNIQUE_SESSION_NAME ||''' (Status='|| retval ||'): ' || message);
end loop;
dbms_output.put(SYSDATE || ': Duration (1/100s): DELETE=');
dbms_output.put_line(dbms_utility.get_time() - timestart);
SELECT COUNT(*) INTO rows_amount FROM TEST_BONUS;
dbms_output.put_line(SYSDATE || ': COUNT(*) FROM TEST_BONUS: ' || rows_amount);
retval :=DBMS_PIPE.REMOVE_PIPE(DBMS_PIPE.UNIQUE_SESSION_NAME);
dbms_output.put_line(systimestamp || ': REMOVE_PIPE: ''' || DBMS_PIPE.UNIQUE_SESSION_NAME || ''' returned: ' ||retval);
EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line(SYSDATE || SUBSTR(SQLERRM, 1, 1000) || ' ' ||
SUBSTR(DBMS_UTILITY.FORMAT_ERROR_BACKTRACE, 1, 1000));
retval := DBMS_PIPE.REMOVE_PIPE(DBMS_PIPE.UNIQUE_SESSION_NAME);
dbms_output.put_line(SYSDATE || ': REMOVE_PIPE: ''' || DBMS_PIPE.UNIQUE_SESSION_NAME || ''' returned: ' ||retval);
-- Clean up in case of error
PROC_DELETE_TEST_BONUS('A');
PROC_DELETE_TEST_BONUS('B');
PROC_DELETE_TEST_BONUS('C');
RAISE;
END;
/
You should always keep in mind that the changes executed within the job are committed in a separate transaction.
Just to get a feeling for what this concurrency achieves, here a some averaged measured values: The sequential code in the question takes about 60s to complete, the parallel one about 40s.
It would be an interesting further investigation how this turns out when there are more than the three jobs running in parallel.
PS
A helpful query to find out about the status of the jobs is the following
SELECT job_name,
destination,
TO_CHAR(actual_start_date) AS actual_start_date,
run_duration,
TO_CHAR((ACTUAL_START_DATE+run_duration)) AS actual_end_date,
status,
error#,
ADDITIONAL_INFO
FROM user_scheduler_job_run_details
ORDER BY actual_start_date desc;
Just wanted to add a few notes about DBMS_PARALLEL_EXECUTE package from Oracle.
This can be used to do more than update a table, although many of the examples show this simple use case.
The trick is to use an anonymous block instead of a DML statement, and the rest of the examples are still relevant. So, instead of this:
l_sql_stmt := 'update EMPLOYEES e
SET e.salary = e.salary + 10
WHERE manager_id between :start_id and :end_id';
We might have this:
l_sql_stmt := 'BEGIN my_package.some_procedure(:start_id, :end_id); END;';
The rest of the example can be found in the "Chunk by User-Provided SQL" example section
You will still need to tell Oracle the start/end ids for each process(using CREATE_CHUNKS_BY_SQL), I typically store them in a separate lookup table (if pre-defined) or you can provide a SQL query that returns a set of start/end values. For the latter approach, try using NTILE. For example, using 8 chunks:
select min(id) as start_id, max(id) as end_id
from (
select id, ntile(8) over (order by 1) bucket
from some_table
where some_clause...
)
group by bucket
order by bucket;
Hope that helps
If yes, you can try this:
create a new table with tasks (and parameter of tasks)
create procedure that will read parameters from table, pass them to "legacy procedure" and then update task table after processing (using autonomous transactions) to show that processing is ended
outer procedure, that creates tasks, can scan task table to get information about progress. You can use DBMS_LOCK.SLEEP to wait.
I've trying to figure out oracle's DBMS_SCHEDULER (Oracle 11g) and need help setting up the following:
I have a procedure that calls a list of other procedures like this:
CREATE OR REPLACE
PROCEDURE RUN_JOBS AS
BEGIN
MYUSER.MYPROCEDURE1();
MYUSER.MYPROCEDURE2();
MYUSER.MYPROCEDURE3();
MYUSER.MYPROCEDURE4();
MYUSER.MYPROCEDURE5();
END;
/
I would like to use DBMS_SCHEDULER to run MYPROCEDURE3(), MYPROCEDURE4(), MYPROCEDURE5()
in parallel after the completion of MYPROCEDURE2().
Can someone show me an example on how to set this up?
You can refer to Chains under the DBMS_SCHEDULER package: http://docs.oracle.com/cd/B28359_01/server.111/b28310/scheduse009.htm
You can also achieve the same by going through Oracle Enterprise Manager, but I can't find any links to documentation right now.
You can do that using DBMS_SCHEDULER.
CREATE OR REPLACE PROCEDURE RUN_JOBS
AS
v_JobNum NUMBER := 1;
BEGIN
BEGIN
DBMS_JOB.SUBMIT(v_JobNum,'MYUSER.MYPROCEDURE1;',sysdate,'sysdate +1');
DBMS_JOB.SUBMIT(v_JobNum,'MYUSER.MYPROCEDURE2;',sysdate,'sysdate +1');
DBMS_JOB.SUBMIT(v_JobNum,'MYUSER.MYPROCEDURE3;',sysdate,'sysdate +1');
DBMS_JOB.SUBMIT(v_JobNum,'MYUSER.MYPROCEDURE4;',sysdate,'sysdate +1');
COMMIT;
END;
END RUN_JOBS;
/
This will submit the job and run them immediately.
create three different jobs for each procedure and schedule them at same time.
Here is my custom approach to parallellize work into N separate jobs retaining dbms_scheduler's logging and backpressure support. Date intervals are spread by mod N.
create table message_fixup_log (
source_date date not null,
started_at timestamp(6) not null,
finished_at timestamp(6),
fixed_message_count number(10)
);
alter table message_fixup_log add duration as (finished_at - started_at);
create unique index ix_message_fixup_log_date on message_fixup_log(source_date desc);
create or replace procedure message_fixup(jobNumber number, jobCount number, jobName varchar default null)
is
minSince date;
maxSince date;
since date;
msgUpdatedCount number;
begin
-- choose interval
select trunc(min(ts)) into minSince from message_part;
select trunc(max(ts))+1 into maxSince from message_part;
begin
select max(source_date) + jobCount into since from message_fixup_log
where finished_at is not null
and mod(source_date - minSince, jobCount) = jobNumber
and source_date >= minSince;
exception when no_data_found then null;
end;
if (since is null) then
since := minSince + jobNumber;
end if;
if (since >= maxSince) then
if (jobName is not null) then
dbms_scheduler.set_attribute(jobName, 'end_date', systimestamp + interval '1' second);
end if;
return;
end if;
insert into message_fixup_log(source_date, started_at) values(since, systimestamp);
-- perform some actual work for chosen interval
msgUpdatedCount := sql%rowcount;
update message_fixup_log
set fixed_message_count = msgUpdatedCount, finished_at = systimestamp
where source_date = since;
end;
-- manual test
--call message_fixup(0, 1);
declare
jobName varchar2(256);
jobCount number default 8;
begin
for jobNumber in 0..(jobCount-1) loop
jobName := 'message_fixup_job' || jobNumber;
begin
dbms_scheduler.drop_job(jobName, true);
exception
when others then null;
end;
dbms_scheduler.create_job(
job_name => jobName,
job_type => 'stored_procedure',
job_action => 'message_fixup',
enabled => false,
start_date => systimestamp,
repeat_interval => 'freq = minutely; interval = 1',
number_of_arguments => 3
);
dbms_scheduler.set_attribute(jobName, 'logging_level', dbms_scheduler.logging_full);
dbms_scheduler.set_job_anydata_value(jobName, 1, ANYDATA.ConvertNumber(jobNumber));
dbms_scheduler.set_job_anydata_value(jobName, 2, ANYDATA.ConvertNumber(jobCount));
dbms_scheduler.set_job_argument_value(jobName, 3, jobName);
dbms_scheduler.enable(jobName);
end loop;
end;
I need to sync an oracle text index. But job fails to create:
declare
v_job_id number(19,0);
begin
dbms_job.submit(
JOB => v_job_id,
WHAT => 'alter index NAME_IDX rebuild parameters (''sync'');',
NEXT_DATE => SYSDATE + (1/24),
INTERVAL => 'SYSDATE + (1/24) + 7'
);
end;
/
Or to run:
declare
v_job_id number(19,0);
begin
dbms_job.submit(
JOB => v_job_id,
WHAT => 'CTX_DDL(''NAME_IDX'');',
NEXT_DATE => SYSDATE + (1/24),
INTERVAL => 'SYSDATE + (1/24) + 7'
);
end;
/
But if I run any of those works:
alter index NAME_IDX rebuild parameters ('sync');
call CTX_DDL('NAME_IDX');
Any idea of the correct syntax?
Thank you.
PD: Ive been searching, but the only answer I found doesnt fit my requirements. I also apologize for my english.
You can run an anonymous block, CALL is not in PL/SQL, ALTER INDEX is DDL, and you need to specify which procedure in CTX_DDL you want to run:
WHAT => 'BEGIN EXECUTE IMMEDIATE ''alter index NAME_IDX rebuild parameters (''''sync'''')''; CTX_DDL.sync_index(''NAME_IDX''); END',
However, personally I prefer to encapsulate it in a procedure (or, even better, a package) and call the procedure from the job:
CREATE PROCEDURE rebuild_name_idx IS
BEGIN
EXECUTE IMMEDIATE 'alter index NAME_IDX rebuild parameters (''sync'')';
CTX_DDL.sync_index('NAME_IDX');
END;
/
declare
v_job_id number(19,0);
begin
dbms_job.submit(
JOB => v_job_id,
WHAT => 'rebuild_name_idx;',
NEXT_DATE => SYSDATE + (1/24),
INTERVAL => 'SYSDATE + (1/24) + 7'
);
end;
/
Also, I'm pretty sure you don't actually need to rebuild the index - you only need to call CTX_DDL.sync_index to refresh it from any DML on the table.