I had a problem in implementation of dead letter queue in IBM MQ in .net6.
I am using IBMMQDotnetClient library from nuGet. The problem is that when we face to any message which is not successfully commited, the message from queue continues in while(true).
I want to get the message to dead letter queue and continue to next messages from the queue.
how can I use dead letter queue for this issue in .net6 project ?
When you use XMS you get the IBM MQ XMS library handling of backout threshold and backout.
See https://www.ibm.com/docs/en/ibm-mq/latest?topic=consumers-poison-messages-in-xms
You need to set BOTHRESH and BOQUEUE for your queue. This will be an MQ admin job. Then when read attempts for any message exceed the threshold the message is requeued to the backout queue. If the requeuing fails for any reason, the message is removed from the input queue and either requeued to the dead-letter queue, or discarded.
The only code change needed is to signal when a message can't be processed. You will need to transaction enable the session.
sessionWMQ = connectionWMQ.CreateSession(true, AcknowledgeMode.SessionTransacted);
commit good messages
sessionWMQ.Commit();
rollback (and signal a read failure) bad messages
sessionWMQ.Rollback();
To get you started there is a set of XMS samples that don't have transactions enabled at https://github.com/ibm-messaging/mq-dev-patterns/tree/master/dotnet
Related
I am using Nifi to get data from IBM MQ. It is working fine. My question is once the message is read from an MQ queue, does it get deleted from the queue? How to just read messages from the queue without deleting them from the queue?
My question is once the message is read from an MQ queue, does it get
deleted from the queue?
Yes, that is the default behavior.
How to just read messages from the queue without deleting them from
the queue?
You use the option: MQGMO_BROWSE_FIRST followed by MQGMO_BROWSE_NEXT on the MQGET API calls.
You can also open the queue for browse only. i.e. MQOO_BROWSE option for MQOPEN API call.
It sounds as if you would like to use a "publish/subscribe" model rather than a "point-to-point" model.
From ActiveMQ:
Topics In JMS a Topic implements publish and subscribe semantics. When
you publish a message it goes to all the subscribers who are
interested - so zero to many subscribers will receive a copy of the
message. Only subscribers who had an active subscription at the time
the broker receives the message will get a copy of the message.
Queues A JMS Queue implements load balancer semantics. A single
message will be received by exactly one consumer. If there are no
consumers available at the time the message is sent it will be kept
until a consumer is available that can process the message. If a
consumer receives a message and does not acknowledge it before closing
then the message will be redelivered to another consumer. A queue can
have many consumers with messages load balanced across the available
consumers.
If you have a queue, when a consumer consumes that message, it is removed from the queue so that the next consumer consumes the next message. With a topic, multiple consumers can be subscribed to that topic and retrieve the same message without being exclusive.
If neither of these work for you, I'm not sure what semantics you're looking for -- a "queue" which doesn't delete the message when it is consumed will never let a consumer access any but the first message.
I have an application listening to messages on an IBM Websphere MQ queue.
Once a message is consumed, the application performs some processing logic.
If the processing completed OK, I would like the application to acknowledge the message and have it removed from the queue.
If an error occurred while processing, I would like the message to remain in the queue.
How is this implemented? (I'm using the .NET API)
Thanks.
MQ supports a single-phase commit protocol. You specify syncpoint when you get the message, then issue COMMIT or ROLLBACK as required. The default action if the connection is lost is ROLLBACK and if the program deliberately ends without resolving the transaction a COMMIT is assumed. (This is platform dependent so the customary advice is to explicitly call COMMIT and not rely on the class destructors to do it for you.)
This works whether the message is persistent or not. However if the message has an expiry specified and expires after being rolled back there's a chance it won't be seen again.
Of course, if the program issues a ROLLBACK the message will normally be seen again since it goes back to the same spot int he queue and for a FIFO queue that's the top. If the problem with the message is not transient then this causes a poison message loop of read/rollback/repeat. To avoid that the app can check the backout count and if it exceeds some threshold requeue the message to an exception queue.
When using JMS or XMS this is done for you by the class libraries. If the input queue's BOQNAME and BOQTHRESH attributes are set the requeue is to the queue names in BOQNAME. Otherwise a requeue to the Dead Queue is attempted. IF that fails (as it should if the system is properly secured) the listener will stop receiving messages.
The usual advice is to always specify a backout queue and either let the classes use it or code the app to use it.
Please see Usage Notes for MQGET in the MQAPI Reference and the MQGetMessageOptions.NET page in the .Net class reference.
You may want to look at the MQ Reporting Options.
Expiry, Confirmation of Arrival and Confirmation of Delivery can be requested and sent via a response queue back to the sending application by the receiving Queue Manager.
Positive and Negative Acknowledgements can also be generated by the receiving application provided they use the related reporting attributes found in the Message Descriptor.
Exception can be requested and sent via a response queue back to the sending application by any Queue Manager in the transmission chain or generated by the receiving application.
1 Read the message using MQC.MQGMO_SYNCPOINT,
2 process it
3 call MQQueueManager.Commit()
If Commit() is not called explicitly, or implicitly (eg exception is thrown), all messages that have been de-queued will be re-enqueued.
I am not familiar with MQ so forgive me if I'm not explaining myself properly. We have received a message on a queue that is failing when being read with a 2110 MQRC_FORMAT_ERROR. It would seem that the problem is it doesn't have a 'Format' specified, e.g. we are expecting 'MQSTR'. The client code (C#) is catching the MQException and doing a MQQueueManager.Backout() however the message is just going back onto the queue and being read and rejected again by the client.
Looking at the queue I saw that it did not have a backout queue or backout threshold set. The queue manager also did not have a dead letter queue set. So what I've done is setup the DLQ and set the queue's backout queue to it with a backout threshold of 5. However the message is still stuck on the queue being continually read and put back. The backout count for the message is currently approaching 20 million.
I would like to get the backout and DLQ working but if there is another method to just manually delete or move this message that would help for now. It is just a test message so it doesn't matter if we lose it.
EDIT: I've had to clear the queue to get things moving so unfortunately might not be able to test any solutions until it happens again.
MQ native clients do not automatically move erroneous message to BackoutQ or DLQ. Application has to write additional code to move the messages to DLQ or Backout queue. However XMS .NET and JMS clients do this job. XMS .NET is a JMS specification implementation in C# language. XMS .NET comes bundled with MQ Clients package.
Coming to your case:
MQRC_FORMAT_ERROR is not actually an error, it's a warning telling the application that MQ client was not able to convert the incoming message and it delivered an unconverted message to the application. The application must handle this exception and should not rollback such messages. You will need to investigate the application that is sending such messages and fix any issues.
Please see the documentation here.
I created a AQ in oracle and wrote 2 JMS consumers in Java to listen to the queue. I have observed sometimes that if I produce some message in to queue; the count of dequeued messages from queue is greater than what enqueued. It means that some messages are consumed twice.
I have created queue with property:- multiple_consumers => FALSE
And JMS consumers are working in CLIENT_ACKNOWLEDGE mode
Please help me learn the possible reasons for such behavior and it's solution. So, that I can replicate the problem and solve above issue and ensure that the number of message enqueued is equal to number of message dequeued in case of multiple JMS consumers listening to same AQ .
Without having seen your code, CLIENT_ACKNOWLEDGE typically says you are sending acknowledgements manually. If you do not send an ack, the message won't get deleted and the broker will try to redeliver it at a later stage (like when you restart the connection or similar). This might be the cause of your concern.
We have set up BOTHRESH(5), BOQNAME(USER.ERR) in WebSphere MQ v7.0.1.9 queue manager.
When TX is rolled back by the MDB in a container managed TX in WebSphere application server v7, the messages are getting placed in DLQ instead of placing in original queue.
I checked the logs which stated that TX got rolled back successfully .
Can some one help me?
...the message are getting placed in DLQ instead of placing in original queue.
Setting BOQNAME and BOTHRESH results in the QMgr attempting to put the message in something other than the original queue. Once BOTHRESH is exceeded, the QMgr will first try to put the message into the queue named by BOQNAME, then into the DLQ if putting to BOQNAME is not possible. Only if both of these fail does the QMgr put the message back on the original queue or discard it if the message is non-persistent.
Some reasons that backing out to the queue named in BOQNAME or to the DLQ fail are:
The target queue does not exist.
The target queue is full.
The message size exceeds the target queue's MAXMSGL attribute.
The user is not authorized to put messages onto the target queue.
The target queue is of the wrong type (i.e. XMitQ or model queue).
When putting to the DLQ, the QMgr's DEADQ attribute is empty.
BOTHRESH is set but BOQNAME is not. In this case, the DLQ is the only queue tried.