In our project, we consume a message from IBM MQ (ack-Manual) and after processing we publish the message to other IBM MQ ( either target queue/DL Q - if any errors)
Now, we need to reprocess the messages in DLQ and make sure these messages dont go in infinite loop.
Also, I need to filter the messages (while reading from DLQ) making sure these are not reprocessed messages..
I tried using selector 'attributes.headers.redelivery=false' - but I'm getting Selector Expression error.
Can anyone suggest me how to filter and process the messages in DLQ only once..
Related
Using Mule 4.4 on premise along with Apache ActiveMQ. I am trying to get a better understanding of how Mule handles messaging.
I tried searching the internet but am not finding any details about the same.
I have a jms:listener:
<jms:listener doc:name="Listener" config-ref="JMS_Config" destination="Consumer.mine2.VirtualTopic.mine.test">
<jms:consumer-type >
<jms:queue-consumer />
</jms:consumer-type>
</jms:listener>
and I have a jms:consume:
<jms:consume doc:name="Consume" config-ref="JMS_Config" destination="Consumer.mine1.VirtualTopic.mine.test">
<jms:consumer-type >
<jms:queue-consumer />
</jms:consumer-type>
</jms:consume>
To me both seem to be doing the same job i.e. consume messages from a queue / topic. So why do we have these two components in Mule?
jms listener is a source, so using it you can trigger a flow whenever there is a new message in queue.
jms consume is an operation, so you can use it anywhere within a flow's execution, i.e. like a http request component that you put in the middle of a flow.
Both of them will consume a message from the queue/topic. However, when you are using a listener you are basically saying that "There is a queue, I do not know when a new message will come in, but whenever it comes I need to perform these actions"
When you use a consume operation, you are saying "I am expecting some message soon and with that I will to these actions".
Now in both cases a message may not come at all, and both have there own way to deal with it. A listener, since it is a source, will just not simply trigger the flow and keep on waiting. A consume will block your execution until a message is there, or you can configure a time out to not be blocked fore ever.
A common use case can be reprocessing messages from a DLQ. Generally when you use a queue, you also have a DLQ so that the messages that failed during processing, from the "main" queue, can be sent to the DLQ and reprocessed later.
Now, in this architecture, you will typically use the jms listener only with the main queue for the processing of the messages. And you will have a separate flow that can have a http listener so that you can trigger an HTTP Endpoint whenever you are ready to reprocess the messages from the DLQ. This flow with http listener with consume all the messages (probably in a loop) from the DLQ and publish them back to the main queue
I'm newbie on IBM tool.
On my laptop, I have installed IIB v10 using a local MQ manager. I created 2 local queues on MQ: "Queue.In" and "Queue.Out".
I have a very simple scenario that I work on: I send a 'csv' file on the "Queue.In", get the message using 'MQInput node' transform the message into xml using 'mapping node', and then put the xml message to 'Queue.Out'.
Using the ‘flow exercicser’ I’m able to send my ‘csv’ file and I can see that the message get consumed and transformed to xml and send to “Queue.Out”
Problem: My xml message is successfully put in "Queue.Out", but disappear right after.
I want it to remain in the "Queue.Out".
I use “MQ explorer” to browse my queues.
Please advice.
I would always recommend using Rfhutil for Delivery and Subscription of messages onto/out of a queue. In Rfhutil you can also browse messages without taking them off the queue which can be useful.
I am unsure why the flow exerciser removes messages - this is a new feature in IIB10 so it might be that they wanted queues to be cleared after the flow execution has been completed to avoid clogging queues. I believe the point of the exerciser is to track the logic of the flows themselves not to get messages.
Hope this helps :)
I need to pause the receiver queue some time (for process the already received messages) in ibm websphere queue through the ibm java api.
If you want to stop Applications from being able to get messages from a Queue you can inhibit the Queue by setting the GET attribute on the Queue to DISABLED. This will still allow applications to put messages to the Queue but no applications will be able to get messages from the Queue. IF you want to stop messages from being able to be put to a queue you can do this by setting the PUT attribute to DISABLED.
When you're ready for the applications to be able to get messages from the Queue again you set the GET attribute to ENABLED and applications will be able to get messages from the Queue.
example MQSC command to set GET to DISABLED
ALTER QLOCAL('') GET(DISABLED)
Be warned: If you inhibit the put or get on a Queue, applications attempting to put or get messages on that Queue will receive an error return code of MQRC_PUT_INHIBITED or MQRC_GET_INHIBITED. If you don't anticipate this your applications could fail.
I am not familiar with MQ so forgive me if I'm not explaining myself properly. We have received a message on a queue that is failing when being read with a 2110 MQRC_FORMAT_ERROR. It would seem that the problem is it doesn't have a 'Format' specified, e.g. we are expecting 'MQSTR'. The client code (C#) is catching the MQException and doing a MQQueueManager.Backout() however the message is just going back onto the queue and being read and rejected again by the client.
Looking at the queue I saw that it did not have a backout queue or backout threshold set. The queue manager also did not have a dead letter queue set. So what I've done is setup the DLQ and set the queue's backout queue to it with a backout threshold of 5. However the message is still stuck on the queue being continually read and put back. The backout count for the message is currently approaching 20 million.
I would like to get the backout and DLQ working but if there is another method to just manually delete or move this message that would help for now. It is just a test message so it doesn't matter if we lose it.
EDIT: I've had to clear the queue to get things moving so unfortunately might not be able to test any solutions until it happens again.
MQ native clients do not automatically move erroneous message to BackoutQ or DLQ. Application has to write additional code to move the messages to DLQ or Backout queue. However XMS .NET and JMS clients do this job. XMS .NET is a JMS specification implementation in C# language. XMS .NET comes bundled with MQ Clients package.
Coming to your case:
MQRC_FORMAT_ERROR is not actually an error, it's a warning telling the application that MQ client was not able to convert the incoming message and it delivered an unconverted message to the application. The application must handle this exception and should not rollback such messages. You will need to investigate the application that is sending such messages and fix any issues.
Please see the documentation here.
I was wondering if there is a way for you to configure a queue to automatically clear messages? We are striving to partially implement a component of our architecture and want to be able to send to the queue, but have the queue automatically remove the messages that are being sent so that we don't have to run scripts, etc to perform the clean-up.
So far the only thing I have been able to find is to run CLEAR QLOCAL or set the messages to expire from the publishing application.
For you use case there are a few options in IBM MQ:
Create a QALIAS that points to a TOPIC object which has a topic string with no subscribers, messages put to the QA will just disappear.
Have the sending application set message expiry.
Use the IBM MQ CAPEXPRY feature to administratively force message expiry at the queue level.
Run a script to issue CLEAR QLOCAL against the queue. There can not be open handles on the queue for this to work.
Programmatically issue the equivalent PCF command to CLEAR QLOCAL against the queue. There cannot be open handles on the queue for this to work.
Run the IBM MQ dmpmqmsg utility against the queue to read and discard the messages.