pl/sql using trigger to gather queued messages in queue - oracle

Question

I am trying to work on altering an already existing queuing system... Which just gives me a headache...
The logic behind the queues is all in Oracle DB as follows:
Database #1 sends notifications, which are queued in Database #2 in a specific queue, let's name it Q1.
There is a regular job, that executes a procedure in a package (DISPATCHER), which basically is a loop that dequeues all messages in Q1 until no messages are there. Its execution is scheduled every 5 minutes, yet with the amount of queue entries it runs longer than that.
Dispatcher procedure distributes the messages to additional queues, from which Java processes the messages and things go further.
There isn't much control over the dispatching JOB so I decided to look up ways of making the whole framework for dispatching jobs better. I must add it is an old implementation that worked for quite a while, but most don't find it easy to control or transparent.
My point is if there are any secure ways of this dispatching process?
I have created a trigger on the queue table that executes a single dequeue with all the processing of the message done in DISPATCHER and it works, but i don't know if it's best the idea.
Looked up creating subscribers, but can't add any as it's a single-consumer queue. Considering i would have to update the framework, recreating the queue would delete all the messages that already are in the queue, which is something i'd like to omit...
Tried a DBMS Scheduler job to execute the DISPATCHER for more control, but it just runs "when it has resources". I'm counting on a solution that works continuously and processes the messages as fast as it can.
Maybe there is some more elaborate way to solve this problem, but i'm still a bit fresh and can't seem to grasp the whole advanced queuing mechanism...
Thanks in advance.

Answer 1

Considering constantly running job it's possible to control procedure execution by sending message through DBMS_PIPE.
Control code sends stop message through dbms_pipe.
Dispatcher code runs constantly in DBMS_SCHEDULER job and checks if a stop message arrived in pipe.
Also it's possible to force an interrupt of DBMS_SHCEDULER job after some timeout.
Years ago I had implemented such a logic and this solution runs till now. Code below is an adapted sample which can help you to check if this approach acceptable and useful for your case.
Control package header:
create or replace package DispatcherControl is
-- Package to control job state
procedure StartDispatcher;
procedure StopDispatcher;
end;
Control package body:
create or replace package body DispatcherControl is
-- User who owns dispatcher job in DBMS_SCHEDULER
C_DISPATCH_PROGRAM_OWNER constant varchar2(100) := 'DISPATCH_USER';
-- Main procedure to handle dispatcher requests
C_DISPATCH_JOB_PROC constant varchar2(100) := 'DISPATCH_USER.DISPATCHER.MAINPROC';
-- Name for dispatch program
C_DISPATCH_PROGRAM_NAME constant varchar2(100) := 'DISPATCH_PROGRAM';
-- Description for dispatch program
C_DISPATCH_PROGRAM_COMMENT constant varchar2(100) := 'Q1 dispatch task';
-- Dispatcher job name.
C_DISPATCH_JOB_NAME constant varchar2(100) := 'DISPATCH_JOB';
-- Description for dispatcher job
C_DISPATCH_JOB_COMMENT constant varchar2(100) := 'Q1 dispatch process';
-- Pipe name for dispatch job control.
C_DISPATCH_PIPE_NAME constant varchar2(100) := 'DISPATCH_CONTROL';
-- Message text for pipe to stop dispatch job
C_PIPE_STOP_MSG constant varchar2(100) := 'stop_dispatch';
-- Check if DBMS_SCHEDULER program exists and create it if needed
procedure CheckDispatcherJobExists
is
begin
-- Return if program exists
for cDummy in (
select 1
from all_scheduler_programs
where
owner = C_DISPATCH_PROGRAM_OWNER and program_name = C_DISPATCH_PROGRAM_NAME
and
rownum = 1
) loop
return;
end loop;
-- Create disabled if not found
sys.dbms_scheduler.create_program(
program_name => C_DISPATCH_PROGRAM_OWNER || '.' || C_DISPATCH_PROGRAM_NAME,
program_type => 'STORED_PROCEDURE',
program_action => C_DISPATCH_JOB_PROC,
number_of_arguments => 0,
enabled => false,
comments => C_DISPATCH_PROGRAM_COMMENT
);
-- Enable program
sys.dbms_scheduler.enable(C_DISPATCH_PROGRAM_OWNER || '.' || C_DISPATCH_PROGRAM_NAME);
end;
-- Check status of dispatcher job and run it if not found
procedure CheckDispatcherJobState
is
begin
CheckDIspatcherJobExists;
-- Check if job is currently running according to scheduler info
for cDummy in (
select * from all_scheduler_jobs
where
owner = C_DISPATCH_PROGRAM_OWNER
and
job_name = C_DISPATCH_JOB_NAME
) loop
-- Job found, check if running
for cDummy2 in (
select * from all_scheduler_running_jobs
where
owner = C_CIS_PROGRAM_OWNER and job_name = C_DISPATCH_JOB_NAME
) loop
-- Check if job really running by checking sessions list
for cDummy3 in (
select 1
from
all_scheduler_running_jobs jobs,
sys.v_$session sessions
where
jobs.owner = C_DISPATCH_PROGRAM_OWNER and jobs.job_name = C_DISPATCH_JOB_NAME
and
sessions.sid = jobs.session_id and sessions.process = jobs.slave_os_process_id
and
sessions.action = jobs.job_name and sessions.username = jobs.owner
and
sessions.module = 'DBMS_SCHEDULER'
) loop
-- Ok, return
return;
end loop;
-- No process found for running job, stop and delete task before recreation
sys.dbms_scheduler.stop_job(C_DISPATCH_JOB_NAME, true);
end loop;
-- process found, but restart needed
sys.dbms_scheduler.drop_job(C_DISPATCH_JOB_NAME, true);
end loop;
-- Create one-time running job with manual start
sys.dbms_scheduler.create_job(
job_name => C_DISPATCH_JOB_NAME,
program_name => C_DISPATCH_PROGRAM_OWNER || '.' || C_DISPATCH_PROGRAM_NAME,
enabled => true,
auto_drop => true,
start_date => add_months(sysdate,1000),
repeat_interval => null,
end_date => null,
comments => C_DISPATCH_JOB_COMMENT
);
-- Run created task
sys.dbms_scheduler.run_job(
job_name => C_DISPATCH_JOB_NAME,
use_current_session => false
);
end;
-- Stop and drop dispatch job.
procedure DropDispatchJob
is
vSendRC integer;
begin
-- Send request through DBMS_PIPE and wait for timeout.
sys.dbms_pipe.reset_buffer;
sys.dbms_pipe.pack_message(C_PIPE_STOP_MSG);
vSendRC := sys.dbms_pipe.send_message(C_DISPATCH_PIPE_NAME,1);
if(vSendRC = 0) then
-- wait if sent Ok
sys.dbms_lock.sleep(0.25);
end if;
-- force job stop (not done in case of drop_job)
for cDummy in (
select 1 from all_scheduler_running_jobs
where
owner = C_DISPATCH_PROGRAM_OWNER
and
job_name = C_DISPATCH_JOB_NAME
) loop
begin
sys.dbms_scheduler.stop_job(C_DISPATCH_JOB_NAME, true);
exception
when others then begin
-- If sqlcode = -27366 then task finished, in other case it's unexpected
if(SQLCODE != -27366) then
raise;
end if;
end;
end;
end loop;
-- delete job if exists, allow stopping
for cDummy in (
select 1 from all_scheduler_jobs
where
owner = C_DISPATCH_PROGRAM_OWNER
and
job_name = C_DISPATCH_JOB_NAME
) loop
sys.dbms_scheduler.drop_job(C_DISPATCH_JOB_NAME, true);
end loop;
-- drop program if exists
for cDummy in (
select 1 from all_scheduler_programs
where
owner = C_DISPATCH_PROGRAM_OWNER
and
program_name = C_DISPATCH_PROGRAM_NAME
) loop
sys.dbms_scheduler.drop_program(C_DISPATCH_PROGRAM_NAME, true);
end loop;
-- Clear DBMS_PIPE messages after job stop (cover case of forced stop).
sys.dbms_pipe.purge(C_DISPATCH_PIPE_NAME);
end;
-- Start dispatcher process
procedure StartDispatcher
is
begin
-- Clear message queue before starting
sys.dbms_pipe.purge(C_DISPATCH_PIPE_NAME);
CheckDispatchJobState;
end;
-- Stop dispatcher process
procedure StopDispatcher
is
begin
DropDispatchJob;
end;
end;
Dispatcher package header:
create or replace package DISPATCH_USER.Dispatcher is
-- Main queue check procedure to run from job.
procedure MainProc;
end;
Dispatcher package body:
create or replace package body DISPATCH_USER.Dispatcher is
-- Normal wait time in seconds
NORMAL_CHECK_INTERVAL constant number := 0.01;
-- Checks if stop message received.
function CheckIsStopped return boolean
is
vPipeRC integer;
vMsg varchar2(1024);
begin
-- check if message exists
vPipeRC := sys.dbms_pipe.receive_message('DISPATCH_CONTROL',0);
if(vPipeRC = 0) then
-- check type of message content, must be varchar2 (look for constants in sys.dbms_pipe package).
if(sys.dbms_pipe.next_item_type = 9) then
-- check message content
sys.dbms_pipe.unpack_message(vMsg);
if(vMsg = 'stop') then
return true;
end if;
end if;
end if;
return false;
end;
-- Checks if error caused by external interrupt
function IsInterruptError(piSQLCode in number) return boolean
is
begin
if( piSQLCode in (
-1013, -- ORA-01013: User requested cancel of current operation
-28, -- ORA-00028: Your session has been killed
-13638, -- ORA-13638: The user interrupted the current operation
-13639, -- ORA-13639: The current operation was interrupted because it timed out.
-13668, -- ORA-13668: The current operation was aborted because it was blocking another session
-48223, -- ORA-48223: Interrupt Requested - Fetch Aborted - Return Code [string] [string]
-48495 -- ORA-48495: Interrupt requested
)
) then
return true;
end if;
return false;
end;
-- Main procedure for dispatcher job
procedure MainProc
is
vIsFound boolean;
vSQLCode number;
vErrMsg varchar2(2048);
vCheckInterval number;
vMESSAGE SOME_CUSTOM_MESSAGE_DATA_TYPE; -- Just for example
begin
vCheckInterval := NORMAL_CHECK_INTERVAL;
while(true) loop
vIsFound := false;
begin
vMESSAGE := GET_NEXT_MESSAGE_FROM_QUEUE; -- Just for example
if(vMESSAGE is not null) then
vIsFound := true;
-- Process received message her
DISPATCH_MESSAGE(vMESSAGE); -- Just for example
-- Commit changes to save a redo log from overflow.
commit;
end if;
exception
when others then begin
vSQLCode := SQLCODE;
vErrMsg := SQLERRM;
if( IsInterruptError(vSQLCode) ) then
-- Promote error if interrupted forcibly
raise;
end if;
-- In other cases just write error conditions to log and continue.
-- Log writing totally skipped from this example, so only comment here.
-- Also it's a place to perform extra error analysis.
-- E.g. if it is some temporary error caused by remote database shutdown
-- and so on, then increase vCheckInterval and don't stress server.
end;
end;
-- Check if interrupted programmatically from control procedure
if( CheckIsStopped ) then
-- normal exit, job finished
return;
end if;
-- If there are no new request then put process in sleep state
-- for a short time to release resources
if(not vIsFound) then
dbms_lock.sleep(vCheckInterval);
end if;
end loop;
-- This point never reached
null;
end;
end;
Please note, that code above is a simplified example (e.g. no logging of performed activity). Also, example code adopted to match question conditions and contain some assumptions like Oracle user names and generics like GET_NEXT_MESSAGE_FROM_QUEUE. Please feel free to ask if something not clear in example code.