I want to delete the sent deley message or modify the sent deley message expiration time.
You can reset the consume offset by RocketMQ DashBoard or command line sh mqadmin resetOffsetByTime,
sh mqadmin resetOffsetByTime -s timestamp -t topic -g group -n namesrv
No way to do that since RocketMQ does not support a message deletion in api.
You may consider achieving your goal when consuming.
For example, maybe you can define a consumer group called DelayMessageAdjuster, in this consumer you can check if the delay message should be aborted and adjust. If the answer is yes, then just send another new delay messages and something like this. I am not sure if this can achieve your goal.
Related
ActiveMQ 5.15.13
Context: I have a single queue with multiple Consumers. I want to stop some consumers from processing certain messages. This has to be dynamic, I don't want to create separate queues for this. This works without any problems. e.g. Consumer1 ignores Stocks -> Consumer1 can process all invoices and Consumer2 can process all Stocks
But if there is a large number of messages already in the Queue (of one type, e.g. stocks) and I send a message of another type (e.g. invoices), Consumer1 won't process the message of type invoices. It will instead be idle until Consumer2 has processed all Stocks messages. It does not happen every time, but quite often.
Is there any option to change the order of the new messages coming into the queue, such that an idle consumer with matching selector picks up the new message?
Things I've already tried:
using a PendingMessageLimitStrategy -> it seems like it does not work for queues
increasing the maxPageSize and maxBrowsePageSize in the hope that once all Messages are in RAM, the Consumers will search for their messages.
Exclusive Consumers aren't an option since I want to be able to use more than one Consumer per message type.
Im pretty sure that there is some configuration which allows this type of usage. I'm aware that there are better solutions for this issue, but sadly I can't use them easily due to other constraints.
Thanks a lot in advance!
EDIT: I noticed that when I'm refreshing on the localhost queue browser, the stuck messages get executed immediately. It seems like this action performs some sort of queue refresh where the messages get filtered based on their selector again. So I just need this action whenever a new message enters the queue...
This is a 'window' problem where the next set of 'stocks' data needs to be processed before the 'invoicing' data can be processed.
The gotcha with window problems like this is that you need to account for the fact that some messages may never come through, or a consumer may never come back online either. Also, eventually you will be asked 'how many invoices or stocks are left to be processed'-- aka observability.
ActiveMQ has you covered-- check out wild-card destinations and consumers.
Produce 'stocks' to:
queue://data.stocks.input
Produce 'invoices' to:
queue://data.invoices.input
You then setup consumes to connect:
queue://data.*.input
note: the wildard '*'.
ActiveMQ will match queues based on the wildcard pattern, and then process data accordingly. As a bonus, you can still use a selector.
I have a requirement, where I am publishing data to the IBM MQ and at around 10pm we run a scheduler that should consume only those messages that are published before 10pm. The messages that are published on or after 10pm should be picked next day.
Is there a way in IBM MQ where we can filter it based on the datetime and consume? Any suggestion?
Thanks in advance.
What you'd really like to be able to do is use a message selector something like this:-
Root.MQMD.PutDate = '20210924' AND Root.MQMD.PutTime < '22000000'
However, unfortunately, the only operators you are allowed to use with strings (and both of these fields are string fields) are = and <> (not-equals) (see IBM MQ Docs: Message selector syntax).
Alternatively, you could make use of QLOAD which can select messages based on time, and have it pre-process the queue by moving all the messages you should be processing to another queue and then allow your application to work on that entire queue. You could invoke QLOAD to do this as follows:-
qload -m QM1 -i INPUT.Q -Tt22:00 -o APPL.INPUT.Q
This command will read messages from a queue called INPUT.Q and will only move those that were put today before 22:00 (10pm) to the queue called APPL.INPUT.Q.
Of course, QLOAD is not doing anything here that you couldn't write into your own application. Just inspect the PutDate and PutTime fields in the MQMD and use those to decide whether to process the message or not.
You can try to use a selector expression in the listener to filter by time.
This is an example adapted from the documentation on how to use a selector expression:
<flow name="JMSConnectorPublish">
<jms:listener config-ref="JMS_Config" destination="in" selector="JMSPriority=9"/>
</flow>
It doesn't use the time. You'll need to be familiar with the headers and attributes from IBM MQ/JMS messages to find the right expression. Probably attributes.headers.JMSTimestamp is going to be of interest to you.
I want to read messages from JMS MQ or In-memory message store based on count.
Like I want to start reading the messages when the message count is 10, until that i want the message processor to be idle.
I want this to be done using WSO2 ESB.
Can someone please help me?
Thanks.
I'm not familiar with wso2, but from an MQ perspective, the way to do this would be to trigger the application to run once there are 10 messages on the queue. There are trigger settings for this, specifically TRIGTYPE(DEPTH).
To expand on Morag's answer, I doubt that WS02 has built-in triggers that would monitor the queue for depth before reading messages. I suspect it just listens on a queue and processes messages as they arrive. I also doubt that you can use MQ's triggering mechanism to directly execute the flow conveniently based on depth. So although triggering is a great answer, you need a bit of glue code to make that work.
Conveniently, there's a tutorial that provides almost all the information necessary to do this. Please see Mission:Messaging: Easing administration and debugging with circular queues for details. That article has the scripts necessary to make the Q program work with MQ triggering. You just need to make a couple changes:
Instead of sending a command to Q to delete messages, send a command to move them.
Ditch the math that calculates how many messages to delete and either move them in batches of 10, or else move all messages until the queue drains. In the latter case, make sure to tell Q to wait for any stragglers.
Here's what it looks like when completed: The incoming messages land on some queue other than the WS02 input queue. That queue is triggered based on depth so that the Q program (SupportPac MA01) copies the messages to the real WS02 input queue. After the messages are copied, the glue code resets the trigger. This continues until there are less than 10 messages on the queue, at which time the cycle idles.
I got it by pushing the message to db and get as per the count required as in this answer of me take a look at my answer
I am trying to build with amqmdnet a IBM MQ receiver with 2 functionalities:
ReceiveReply(byte correlationId) – reads REPLY messages for supplied correlation identifiers. Is implemented by a GET with a correlation MATCH.
event EventHandler NewMessage – notifies the subscriber of new REQUEST messages. Is implemented by a GET with unlimited wait in infinite while loop.
The problem is that I cannot limit the second GET to read only REQUEST messages and so the correlation GET is never successful. If I only do one GET I will have to implement correlation on my own.
Anyone tried to implement something similar?
Thanks,
Radu
The simplest option is to use two queues. There are ugly ways of achieving what you are after (imagine all request messages putting something in the correlid so you could get by correlid there too, for example) but in principle you have a request queue and a reply queue - dont share!
We have quartz process that polls a ActiveMQ JMS queue.
We know that we could get several messages a minute would like to only respond to the most current message at a configured polling rate of a minute or more.
We don't need to process any of the previous messages.
Is there a way to configure the queue to get this behavior?
Its seems like a topic has the ability to do this via the subscription recovery policy using a count of 1. We would like to do this using a queue to guarantee (more or less) a single delivery of the message.
Or is there a conceptual flaw in our assumptions...
Thanks
In my opinion there is no standard operation for this, so you will have to write some code....
One possible solution would be to use a QueueBrowser together with a QueueReceiver:
Through the QueueReceiver you would get an Enumeration of the messages in the queue. For each message you can now perform a receive with a MessageSelector on the JMSMessageID as long as hasMoreElements() returns true. The last message will be the one you want to have....
When using activemq, you can use "image caching" on topics. One of the settings there is to always keep the last mesage sent..
Take a look at the Subscription recovery Policy settings:
http://activemq.apache.org/subscription-recovery-policy.html