We have set up BOTHRESH(5), BOQNAME(USER.ERR) in WebSphere MQ v7.0.1.9 queue manager.
When TX is rolled back by the MDB in a container managed TX in WebSphere application server v7, the messages are getting placed in DLQ instead of placing in original queue.
I checked the logs which stated that TX got rolled back successfully .
Can some one help me?
...the message are getting placed in DLQ instead of placing in original queue.
Setting BOQNAME and BOTHRESH results in the QMgr attempting to put the message in something other than the original queue. Once BOTHRESH is exceeded, the QMgr will first try to put the message into the queue named by BOQNAME, then into the DLQ if putting to BOQNAME is not possible. Only if both of these fail does the QMgr put the message back on the original queue or discard it if the message is non-persistent.
Some reasons that backing out to the queue named in BOQNAME or to the DLQ fail are:
The target queue does not exist.
The target queue is full.
The message size exceeds the target queue's MAXMSGL attribute.
The user is not authorized to put messages onto the target queue.
The target queue is of the wrong type (i.e. XMitQ or model queue).
When putting to the DLQ, the QMgr's DEADQ attribute is empty.
BOTHRESH is set but BOQNAME is not. In this case, the DLQ is the only queue tried.
Related
I had a problem in implementation of dead letter queue in IBM MQ in .net6.
I am using IBMMQDotnetClient library from nuGet. The problem is that when we face to any message which is not successfully commited, the message from queue continues in while(true).
I want to get the message to dead letter queue and continue to next messages from the queue.
how can I use dead letter queue for this issue in .net6 project ?
When you use XMS you get the IBM MQ XMS library handling of backout threshold and backout.
See https://www.ibm.com/docs/en/ibm-mq/latest?topic=consumers-poison-messages-in-xms
You need to set BOTHRESH and BOQUEUE for your queue. This will be an MQ admin job. Then when read attempts for any message exceed the threshold the message is requeued to the backout queue. If the requeuing fails for any reason, the message is removed from the input queue and either requeued to the dead-letter queue, or discarded.
The only code change needed is to signal when a message can't be processed. You will need to transaction enable the session.
sessionWMQ = connectionWMQ.CreateSession(true, AcknowledgeMode.SessionTransacted);
commit good messages
sessionWMQ.Commit();
rollback (and signal a read failure) bad messages
sessionWMQ.Rollback();
To get you started there is a set of XMS samples that don't have transactions enabled at https://github.com/ibm-messaging/mq-dev-patterns/tree/master/dotnet
I have an application listening to messages on an IBM Websphere MQ queue.
Once a message is consumed, the application performs some processing logic.
If the processing completed OK, I would like the application to acknowledge the message and have it removed from the queue.
If an error occurred while processing, I would like the message to remain in the queue.
How is this implemented? (I'm using the .NET API)
Thanks.
MQ supports a single-phase commit protocol. You specify syncpoint when you get the message, then issue COMMIT or ROLLBACK as required. The default action if the connection is lost is ROLLBACK and if the program deliberately ends without resolving the transaction a COMMIT is assumed. (This is platform dependent so the customary advice is to explicitly call COMMIT and not rely on the class destructors to do it for you.)
This works whether the message is persistent or not. However if the message has an expiry specified and expires after being rolled back there's a chance it won't be seen again.
Of course, if the program issues a ROLLBACK the message will normally be seen again since it goes back to the same spot int he queue and for a FIFO queue that's the top. If the problem with the message is not transient then this causes a poison message loop of read/rollback/repeat. To avoid that the app can check the backout count and if it exceeds some threshold requeue the message to an exception queue.
When using JMS or XMS this is done for you by the class libraries. If the input queue's BOQNAME and BOQTHRESH attributes are set the requeue is to the queue names in BOQNAME. Otherwise a requeue to the Dead Queue is attempted. IF that fails (as it should if the system is properly secured) the listener will stop receiving messages.
The usual advice is to always specify a backout queue and either let the classes use it or code the app to use it.
Please see Usage Notes for MQGET in the MQAPI Reference and the MQGetMessageOptions.NET page in the .Net class reference.
You may want to look at the MQ Reporting Options.
Expiry, Confirmation of Arrival and Confirmation of Delivery can be requested and sent via a response queue back to the sending application by the receiving Queue Manager.
Positive and Negative Acknowledgements can also be generated by the receiving application provided they use the related reporting attributes found in the Message Descriptor.
Exception can be requested and sent via a response queue back to the sending application by any Queue Manager in the transmission chain or generated by the receiving application.
1 Read the message using MQC.MQGMO_SYNCPOINT,
2 process it
3 call MQQueueManager.Commit()
If Commit() is not called explicitly, or implicitly (eg exception is thrown), all messages that have been de-queued will be re-enqueued.
I have a set up of an ActiveMQ broker and a single consumer. Consumer gets a message that he is not able to process because a service that it depends has a bug (once fixed it will be fine). So the message keeps being redelivered (consumer redelivery) - we use JMS sessions. With our current configuration it will keep redelivering it every 10 minutes for 1 day. That obviously causes a problem because other messages are not being consumed.
In order to solve this problem I have accessed the queue through JMX and tried to delete that message but it is not there. I guess it is cached on the consumer and not visible at the broker.
Is there any way to delete this message other than restarting the application?
Is it possible to configure the redelivery mechanism so that such message (that causes a live lock eventually) is put at the end of the queue so that other messages can be processed?
The 10 minutes for 1 day redelivery policy should stay as is.
I think you're right that the messages are stuck in the consumer's prefetch buffer, and I don't know of a way to delete them from there.
I'd change your redelivery policy to send to the DLQ after the second failure, with a much shorter interval between them, like 30 seconds, and I'd configure the DLQ strategy as an individualDeadLetterStrategy so you get a separate DLQ containing only messages from this particular queue. Then set up a consumer on this DLQ to move the messages to (the end of) the main queue whenever your reprocessing condition is met (whether that's after a certain delay, or based on reading some flag value from a database, or whatever). This consumer is where you'd implement "every 10 minutes for 1 day" logic, instead of in the redelivery policy where you currently have it.
That will keep the garbage ones out of the main queue so they don't delay other messages from being consumed, but still ensure that they will be reprocessed later. And it will put them on the broker instead of in the consumer's prefetch buffer, where you can view and delete them.
The only way to get it to the back of the queue is to reproduce it to the queue. Redelivery polices can only be configured down to the destination on the connection factory.
Given that you already have a connection, it shouldn't be to hard to create a producer that can either move the given message to a DLQ or produce it back to the queue when you run into that particular bug.
Setting jms.nonBlockingRedelivery=true on the connection factory resolved the problem. Now even if there is a message redelivered it does not block processing of other Messages.
We are reading messages from Websphere MQ using JMS and Spring Integration.
<task:executor id="demoMessageReceiverChannelTE" pool-size="1-3" queue-capacity="5" keep-alive="1" rejection-policy="CALLER_RUNS"/>
<int:channel id="demoMessageReceiverChannel">
<int:dispatcher task-executor="demoMessageReceiverChannelTE"/>
<int:interceptors>
<int:ref bean="messageReceiverInterceptor" />
</int:interceptors>
</int:channel>
<jms:message-driven-channel-adapter
id="demoMesssageReceiverAdapter" channel="demoMessageReceiverChannel"
connection-factory="srcJmsConnectionFactory" destination="srcReceiverQueue"
error-channel="errorChannel" message-converter="demoIncomingMessageConverter"
transaction-manager="srcJmsTransactionManager" send-timeout="65000"
header-mapper="demoJmsHeaders" />
There is another system, which put message on this Queue. (Out of scope for us)
We can have 2000 messages at a time on queue to be read.
Sometimes, Message Listener stops reading messages from Queue, when I tried to troubleshoot this issue, it gives reason that there are some un-commited message on queue, which listener trying to read and won't be able to process further and hangs there itself.
As when delete that message manually from queue, it processes other message perfectly.
So how to skip such uncommitted messages, so that system can continue with next message ?
Sounds like a classic unhandled poison message problem. When the app reads a message that it cannot process, the message is backed out to the queue. It is then read back in again since it is at the top of the queue. This results in the message appearing to be under syncpoint until finally the listener gives up and stops.
Sometimes the problem is in the app and it is the app explicitly calling ROLLBACK. Sometimes though the rollback happens befor the app even sees the message. For example, if the message cannot be converted to the local code page, or other low-level error.
If this is what is happening, the answer is to define a backout queue and then alter the input queue to point at it. For example, if the input queue is called SRC.RECEIVER.QUEUE you might do something like this on the QMgr using runmqsc:
DEFINE QL(SRC.RECEIVER.QUEUE.BKOUT)
ALTER QL(SRC.RECEIVER.QUEUE) BOQNAME(SRC.RECEIVER.QUEUE.BKOUT) BOTHRESH(15)
If the problem really is a poison message, the problem message will show up in the backout queue as an uncommitted message. As soon as the app issues the next COMMIT against the source queue, the backout message will become visible in the backout queue. If no other message arrives on the source queue, the poison message will either remain uncommitted in the backout queue or possibly revert to the source queue if ROLLBACK is called by the app or QMgr.
The setup of the backout queue is simple to do and should be standard practice so I'd recommend doing it whether or not it resolves the problem. Please see Handling poison messages in WebSphere MQ classes for JMS in the Infocenter.
I am not familiar with MQ so forgive me if I'm not explaining myself properly. We have received a message on a queue that is failing when being read with a 2110 MQRC_FORMAT_ERROR. It would seem that the problem is it doesn't have a 'Format' specified, e.g. we are expecting 'MQSTR'. The client code (C#) is catching the MQException and doing a MQQueueManager.Backout() however the message is just going back onto the queue and being read and rejected again by the client.
Looking at the queue I saw that it did not have a backout queue or backout threshold set. The queue manager also did not have a dead letter queue set. So what I've done is setup the DLQ and set the queue's backout queue to it with a backout threshold of 5. However the message is still stuck on the queue being continually read and put back. The backout count for the message is currently approaching 20 million.
I would like to get the backout and DLQ working but if there is another method to just manually delete or move this message that would help for now. It is just a test message so it doesn't matter if we lose it.
EDIT: I've had to clear the queue to get things moving so unfortunately might not be able to test any solutions until it happens again.
MQ native clients do not automatically move erroneous message to BackoutQ or DLQ. Application has to write additional code to move the messages to DLQ or Backout queue. However XMS .NET and JMS clients do this job. XMS .NET is a JMS specification implementation in C# language. XMS .NET comes bundled with MQ Clients package.
Coming to your case:
MQRC_FORMAT_ERROR is not actually an error, it's a warning telling the application that MQ client was not able to convert the incoming message and it delivered an unconverted message to the application. The application must handle this exception and should not rollback such messages. You will need to investigate the application that is sending such messages and fix any issues.
Please see the documentation here.