How to dequeue messages from exception queue?
I searched in google but did not get any idea in brief.
First of all, you must start your exception queue:
EXECUTE DBMS_AQADM.START_QUEUE('OWNER.your_ex_q', true, true);
Then you can run the following instruction to dequeue, passing the MSG_ID:
DECLARE
dequeue_options dbms_aq.dequeue_options_t;
message_properties dbms_aq.message_properties_t;
message_handle RAW(16);
message aq.message_typ;
BEGIN
DBMS_AQ.DEQUEUE(queue_name => 'msg_queue',
dequeue_options => dequeue_options,
message_properties => message_properties,
payload => message,
msgid => message_handle);
COMMIT;
END;
If that's about Oracle Advanced Queuing (which I don't know), Google knows that Oracle says the following:
How do I dequeue from an exception queue?
The exception queue for a multiconsumer queue must also be a
multiconsumer queue.
Expired messages in multiconsumer queues cannot be dequeued by the
intended recipients of the message. However, they can be dequeued in
the REMOVE mode once (only once) using a NULL consumer name in dequeue
options. Messages can also be dequeued from an exception queue by
specifying the message ID.
Expired messages can be dequeued only by specifying message ID if the
multiconsumer exception queue was created in a queue table without the
compatible parameter or with the compatible parameter set to '8.0'
Is this what you're looking for?
Related
I have configured an AWS Lambda such that it will be triggered if a SQS queue receives a message. If any exception occurs while processing the message, the lambda will retry 1 time and if it fails again then the message will go to DLQ.
The configuration is:
on the queue: maxReceiveCount = 2 on the queue. redrive to the dlq. VisibilityTimeout = 30 seconds
on my lambda: ReservedConcurrentExecutions = 1, BatchSize = 1, timeout of 10 seconds
link to the sam template
If i make the lambda throw an error like that:
const receiveHandler = async (event) => {
throw new Error('Booooooom')
};
Then I send a batch of 3 messages.:
I see the failed invocations on CloudWatch
then I see 1 retry for each message on CloudWatch
then I receive the 3 messages in the dead-letter queue. Each message has a receiveCount of 3
Conclusion: everything works as expected
So I do another test. I change the code of my lambda, so it will timeout:
function delay(milliseconds) {
return new Promise(resolve => {
setTimeout(() => { resolve() }, milliseconds);
})
}
const receiveHandler = async (event) => {
// wait 20 seconds
await delay(20000);
};
exports.receiveHandler = receiveHandler;
I purge the dlq, I deploy the new stack, and I send a new batch of 3 messages.
What happens is:
I see in Cloudwatch my message 2 start being processed (1st attempt)
then i see my message 1 start being processed (1st attempt)
then i see my message 1 being processed again (retry)
then i see my message 3 start being processed (1st attempt)
All processings ended in timeout as expected
So 2 logs are missing:
retry for message 2
retry for message 3
However, when I poll the dlq, what I see is:
3 messages in the dlq
ReceiveCount = 3 for each message
So, if I believe the count in the DLQ, all my messages have been retried once. What could be the reason why CloudWatch is missing 2 messages ?
Edit: I did 2 successful tests:
if I triple the VisibilityTimeout and I redo the test, I see all my logs
if, instead, I triple the ReservedConcurrency, I also see all my logs
So, i think that with the configuration I set, some of my messages have no room to be retried, they are moved to the DLQ instead.
Thank you for your help guys !
Env: Oracle 12c
I currently have an Oracle Advanced Queue system setup as follows:
NAME QUEUE_TABLE QID QUEUE_TYPE MAX_RETRIES RETRY_DELAY RETENTION
--------------- --------------- ------- --------------- ----------- ----------- ---------
MY_WORK_Q MY_WORK_QT 2518333 NORMAL_QUEUE 100 0 0
MY_WORK_QT_E MY_WORK_QT 2518332 EXCEPTION_QUEUE 0 0 0
I also have registered a callback that calls a PL/SQL package procedure.
For some reason, I seem to have a situation where I am losing messages or messages are not being dequeued.
Based on this, I have the following questions:
Have I setup my actual queue - MY_WORK_Q with MAX_RETRIES and other info correctly?
Is there a way to check messages that have been enqueued?
Is there a way to check messages that have been de-queued?
Is there a means of checking the EXCEPTION_QUEUE/Exception table to see if lost messages have reached there?
I just can't see why I am losing messages within my queueing system and what I could check to see what might be causing the issue.
I also increased my MAX_RETRIES to 100 but still seem to be having issues.
Update
It seems like I am getting the error ORA-25263: no message in queue.
I am not sure if this is related to a timing issue on the dbms_aq.enqueue or dbms_aq.dequeue calls but on the dbms_aq.dequeue I have this set:
l_dequeue_options.wait := dbms_aq.no_wait;
Do I require say a 10 second wait instead of no_wait? I am unsure if this possible and whether the wait needs to be on the enqueue or dequeue steps.
Here is a simple example that works. Perhaps you can use it to identify your issue?
-- DROP TYPE some_type_t FORCE;
CREATE OR REPLACE TYPE some_type_t AS OBJECT (
a NUMBER,
b VARCHAR(100),
c DATE
);
CREATE OR REPLACE PROCEDURE some_type_callback(CONTEXT IN RAW,
reginfo IN sys.aq$_reg_info,
descr IN sys.aq$_descriptor,
payload IN RAW,
payloadl IN NUMBER) AS
-- Local variables
v_dequeue_options dbms_aq.dequeue_options_t;
v_message_properties dbms_aq.message_properties_t;
v_message_handle RAW(26);
v_some_type some_type_t;
BEGIN
-- Set the dequeue options from the descriptor
v_dequeue_options.consumer_name := descr.consumer_name;
v_dequeue_options.msgid := descr.msg_id;
-- Dequeue the message
dbms_aq.dequeue(queue_name => descr.queue_name,
dequeue_options => v_dequeue_options,
message_properties => v_message_properties,
payload => v_some_type,
msgid => v_message_handle);
END some_type_callback;
/
SELECT *
FROM user_errors e
WHERE e.name = 'SOME_TYPE_CALLBACK';
BEGIN
-- dbms_aqadm.drop_queue_table(queue_table => 'some_type_qt',
-- force => TRUE);
dbms_aqadm.create_queue_table(queue_table => 'some_type_qt',
queue_payload_type => 'some_type_t',
multiple_consumers => TRUE);
dbms_aqadm.create_queue(queue_name => 'some_type_q',
queue_table => 'some_type_qt',
retention_time => 86400); -- 1 day
dbms_aqadm.start_queue(queue_name => 'some_type_q');
dbms_aqadm.add_subscriber(queue_name => 'some_type_q',
subscriber => sys.aq$_agent(NAME => 'some_type_qs',
address => NULL,
protocol => NULL));
dbms_aq.register(sys.aq$_reg_info_list(sys.aq$_reg_info('some_type_q:some_type_qs',
dbms_aq.namespace_aq,
'plsql://some_type_callback',
hextoraw('FF'))),
1);
END;
/
SELECT *
FROM aq$some_type_qt;
-- nothing
DECLARE
v_some_type some_type_t;
eopt dbms_aq.enqueue_options_t;
mprop dbms_aq.message_properties_t;
enq_msgid RAW(16);
BEGIN
v_some_type := some_type_t(a => 42,
b => 'forty-two',
c => to_date('1/1/2942',
'mm/dd/yyyy'));
dbms_aq.enqueue(queue_name => 'some_type_q',
enqueue_options => eopt,
message_properties => mprop,
payload => v_some_type,
msgid => enq_msgid);
END;
/
SELECT *
FROM aq$some_type_qt;
-- msg_state = READY => PROCESSED
One thing that will definitely help you is setting retention_time on your queue table. You can then use the aq$ reference for the queue table to look at the messages. The msg_state column will show READY for a fresh message and PROCESSED for a message that was consumed. There are some other columns on there that can be helpful: retry_count for example.
If you are getting ORA-25263 it seems instead of handling the one message you are getting provoked for in your callback that you are trying to read a different message and clashing with another job that was started to consume the queue. This is resolved by the two lines I have in my callback before calling dequeue.
If you need to trigger when a message arrives and then preserve message order you need to add some locking and additional complexity to your callback. You could take a look at Metalink 225810.1 for examples on how to do this.
We have a process using Oracle AQ on Oracle 11.2. It was working just fine, buy yesterday, messages in the queue stopped getting dispatched to the subscriber. Here is the setup we have to create the queue and register the handler.
DBMS_AQADM.CREATE_QUEUE_TABLE (
queue_table => 'mdms_queue_table',
queue_payload_type => 'U$_MDM_QUEUE_OBJECT',
multiple_consumers => TRUE
);
DBMS_AQADM.CREATE_QUEUE (
queue_name => 'mdms_queue',
queue_table => 'mdms_queue_table'
);
DBMS_AQADM.START_QUEUE (
queue_name => 'mdms_queue'
);
DBMS_AQADM.ADD_SUBSCRIBER (
queue_name => 'mdms_queue',
subscriber => SYS.AQ$_AGENT(
'mdms_queue_subscriber',
NULL,
NULL )
);
DBMS_AQ.REGISTER (
SYS.AQ$_REG_INFO_LIST(
SYS.AQ$_REG_INFO(
'mdms_queue:mdms_queue_subscriber',
DBMS_AQ.NAMESPACE_AQ,
'plsql://P_MDMS_QUEUE_CB_PROCEDURE?PR=0',
HEXTORAW('FF')
)
),
1
);
DBMS_AQADM.GRANT_QUEUE_PRIVILEGE (
'ALL','UIMSMGR.MDMS_QUEUE', 'BAN_DEFAULT_M');
So now, when messages are posted to the queue, I run this query:
select DBA_QUEUES.NAME , aq.*
from gv$aq aq,
dba_queues
where aq.qid = dba_queues.qid
and
name like '%MDMS_QUEUE%'
And all the messages are sitting in the READY state. When I query v$process, I see that the program, "QMNC" is running as per this Oracle Troublshooting Doc. I can manually de-queue the messages by calling DBMS_AQ.dequeue(). Any suggestions on how to troubleshoot would be appreciated!
see this link for some aspects to be considered in oracle aq. Seems that some versions of the database have problems reported in metalink. Also try to recreate the objects if you have a working version before.
I've written a Continuous JMS Message reveiver :
Here, I'm using CLIENT_ACKNOWLEDGE because I don't want this thread to acknowledge the messages.
(...)
connection.start();
session = connection.createQueueSession(true, Session.CLIENT_ACKNOWLEDGE);
queue = session.createQueue(QueueId);
receiver = session.createReceiver(queue);
While (true) {
message = receiver.receive(1000);
if ( message != null ) {
// NB : I can only pass Strings to the other thread
sendMessageToOtherThread( message.getText() , message.getJMSMessageID() );
}
// TODO Implement criteria to exit the loop here
}
In another thread, I'll do something as follows (after successful processing) :
This is in a distinct JMS Connection executed simultaneously.
public void AcknowledgeMessage(String messageId) {
if (this.first) {
this.connection.start();
this.session = this.connection.createQueueSession( false, Session.AUTO_ACKNOWLEDGE );
this.queue = this.session.createQueue(this.QueueId);
}
QueueReceiver receiver = this.session.createReceiver(this.queue, "JMSMessageID='" + messageId + "'");
Message AckMessage = receiver.receive(2000);
receiver.close();
}
It appears that the message is not found (AckMessage is null after timeout) whereas it does exist in the Queue.
I suspect the message to be blocked by the continuous input thread.. indeed, when firing the AcknowledgeMessage() alone, it works fine.
Is there a cleaner way to retrieve 1 message ? based on its QueueId and messageId
Also, I feel like there could be a risk of memory leak in the continuous reader if it has to memorize the Messages or IDs during a long time.. justified ?
If I'm using a QueueBrowser to avoid impacting the Acknowledge Thread, it looks like I cannot have this continuous input feed.. right ?
More context : I'm using ActiveMQ and the 2 threads are 2 custom "Steps" of a Pentaho Kettle transformation.
NB : Code samples are simplified to focus on the issue.
Well, you can't read that message twice, since you have already read it in the first thread.
ActiveMQ will not delete the message as you have not acknowledge it, but it won't be visible until you drop the JMS connection (I'm not sure if there is a long timeout here as well in ActiveMQ).
So you will have to use the original message and do: message.acknowledge();.
Note, however, that sessions are not thread safe, so be careful if you do this in two different threads.
1st off I am new to Delphi so this may be a "mundane detail" that's being over looked. [sorry in advance]
I am getting an 'Invalid Transaction Object' error when I attempt to run a transaction through a datasnap server connected to an Oracle 11g DB.
Due to the system details and the companies business plan we have elected not to use ClientDataSets to handle our transactions. Instead we are attempting to make the Snap server very generic and only handle data access by receiving queries and returning native types.
With that being said here is some sample code that is giving me fits:
function TSnapMethods.TransUpdate: boolean;
var
dbx: TDBXTransaction;
params:TParams;
begin
SqlCon.Open;
dbx := SQLCon.DBXConnection.BeginTransaction(TDBXIsolations.ReadCommitted);
try
params:= TParams.Create(self);
with Params.AddParameter do
begin
name:= 'param';
DataType:= ftWideString;
ParamType:= ptInput;
asString:= 'Bugsville';
end;
with Params.AddParameter do
begin
name:= 'var';
DataType:= ftWideString;
ParamType:= ptInput;
asString:= 'ZZZTOP';
end;
sqlcon.Execute('Update Name set City=:param Where Abrv=:var',params);
SQLcon.CommitFreeAndNil(dbx);//Breaks here...
result:= true;
except
Sqlcon.RollbackFreeAndNil(dbx);//Breaks here also...
result:= false;
end;
end;
By calling SQLCon.DBXConnection.BeginTransaction(), you're bypassing the setting up of internal TTransactionItem which is checked when the transaction is committed when you call SQLcon.CommitFreeAndNil() on the SQLConnection object. Notice that you're starting the transaction on the DBXConnection object but not committing it likewise.
Replace
SQLCon.DBXConnection.BeginTransaction()
with
SQLCon.BeginTransaction()
From another source I got this helpful information:
http://codeverge.com/embarcadero.delphi.ide/record-not-found-or-changed-by-another/1061559
For start transaction:
transaction:=Datamodule.SqlConection.BeginTransaction(TDBXIsolations.ReadCommitted);
For commit:
DataModule1.SqlConection.CommitFreeAndNil(Transacao);
To rollback:
DataModule1.SqlConection.RollbackIncompleteFreeAndNil(Transacao)
And use
RollbackIncompleteFreeAndNil
instead
RollbackIncompleteFreeAndNil
like referenced by:
http://docwiki.embarcadero.com/Libraries/Tokyo/en/Data.SqlExpr.TSQLConnection.RollbackIncompleteFreeAndNil
Please try this and report the results.