We have an application that publishes event notifications to a JMS Topic. The issue noticed here is that after a considerable amount of time, the message store in Weblogic reaches more that 10GB in size. Is there a way to implement a component that can remove messages from a JMS Topic which have crossed a certain age (say 30 days)?
Currently in place is a process, during downtime activity the message store is deleted. The process however, has a prerequisite check from the owners of the subscribing applications whether actions based on last message have been processed or not.
Thanks
Message accumulation for a JMS topic indicates there is at least one inactive durable subscription or perhaps slow subscription consumers. In general, you can prevent this by removing inactive durable subscriptions, speeding up slow subscription consumers, slowing down message production so that consumers can keep up, etc.
If you don't want to retain messages older than 30 days then you can try using the "Message Time-To-Live" functionality defined by the JMS specification. Section 4.8 of the JMS 1.1 spec states:
A client can specify a time-to-live value in milliseconds for each message it
sends. This value defines a message expiration time that is the sum of the
message’s time-to-live and the GMT it is sent (for transacted sends, this is the
time the client sends the message, not the time the transaction is committed).
A JMS provider should do its best to expire messages accurately; however, JMS
does not define the accuracy provided. It is not acceptable to simply ignore
time-to-live.
For more information on message expiration, see Section 3.4.9
"JMSExpiration."
The message's time-to-live can be set when the message is sent using either javax.jms.MessageProducer.setTimeToLive(long) or one of the overloaded send() methods. This will, of course, require changing the sending application's code.
Many brokers support setting the message's time-to-live or expiration time on the broker so that client modifications aren't strictly necessary. I'm not familiar enough with Weblogic to know if it supports this functionality or not, but it's worth investigating if you want to use this solution and don't want to modify your clients.
Came across the following code, which can help in clearing the messages by browsing thru the queue.
queue browser
Related
the documentation of ActiveMQ is giving me a hard time so you are my last hope :D
What i want:
An expiration date on messages in the outgoing queue. Messages in the incoming, DLQ and RTS Queue should stay for ever.
So expiration is always 0 except when a message is put in the outgoing queue. Then ActiveMQ has to set a new expiration day. Alternatively this could also be set by the Application
Or alternatively (maybe easier):
An expiration date on messages in the outgoing and incoming queue. Messages in the DLQ and RTS Queue should stay for ever.
So expiration is always 12345 (or so) except when a message is transfer to RTS or DLQ. Then ActiveMQ has to set the new expiration day to 0
What i tried:
Setting an expiry date for the outgoing messages explicitly in the
application message.setJMSExpiration(XYZ); however, the
expiration date was then always set to 0. Probably because of the
spring jms lib
Configure the deadLetterStrategy so all messages in the DLQ get a expiration of 0. But seems to have no effect. probably misconfigured
<deadLetterStrategy>
<individualDeadLetterStrategy
processNonPersistent="true"
queuePrefix="DLQ."
useQueueForQueueMessages="true"
processExpired="false"
expiration="0"/>
</deadLetterStrategy>
Maybe you can help me :)
Thank you in advance
Spring-jms:5.3.20
ActiveMQ:5.15.15
The JMS Message setJMSExpiration is explicitly documented as bit being meant for use by the user so that explains why calling it isn't working for you.
This method is for use by JMS providers only to set this field when a message is sent. This message cannot be used by clients to configure the expiration time of the message. This method is public to allow a JMS provider to set this field when sending a message whose implementation is not its own.
To define a time to live for the message you either need to use the JMS Producer method setTimeToLive or the send method that allows you to provide the TTL value on a per message basis.
Alternatively you can configure the ActiveMQ Timestamp Plugin to apply a TTL value to each incoming message.
If you want expired messages to go into the DLQ then I'd suggest not configuring the broker to not do that as you've done above by setting processExpired="false" as that is specifically telling the broker not to DLQ expired messages.
I'm confused when it comes to JMS Queue/Topic. What I want is messages should go to every subscriber and I want subscribers to receive messages from inactive time when they become active. However, I don't have control over whether or not subscribers have durable subscription. Is there a way to set up a persistent Queue, and set it up so that every subscriber will receive same message? And how to set this up using spring config
Thanks much.
This is mostly a question where the design of your system affects the outcome.
You could use UI tooling to create durable subscriptions for the clients that need to but that is cumbersome and error prone. You could use something like camel or other configuration on the target broker to fanout messages from an incoming Queue to outgoing Queues that map to the consumer subscriptions.
It all depends mostly on the requirements and your overall design so a real answer is beyond the scope of a SO answer without you doing some more legwork to narrow the scope a bit. JMS itself does not define any answer for this so it will come down a bit to the broker you've chosen and possibly other third party tooling that you might pick to do what you need.
I want the messages to get deleted from the JMS queue if they are not consumed within a certain time limit. What attribute or setting should I configure in while creating the queue?
I have tried expiry but even after the expiry time, I can see the message in the queue
Some JMS implementations will expire messages on queue, but some drop expired messages when consumed. So even though you can see an expired message in the queue, it should not be received by any messages receiver.
We have a JMS queue and multiple competing clients are reading from this queue.
Once the message is taken and successfully processed, we want to send the acknowledge to delete ( i.e. CLIENT ACKNOWLEDGE )
However, we want to make sure that if one client has picked the message another client should not take it from the queue.
Does activeMQ provide this feature out of the box using some configuration ?
Moreover:
If the message processing failed after picking the message, so it could not be acknowledged back, in this scenario we should like other client thread to pickup the message. Is it possible out of the box with configuration , may be specifying timeout values ?
Regards,
JE
You need to take some time to understand the difference between a Topic and a Qeueue in order to understand why the first question is not an issue.
For the second question it depends a bit on the ACK mode you are using and how you are processing messages sync or async. Normally for processing where you want to control redeliveries you would do the work inside of a transaction and if the processing fails the message would be redelivered when the TX is rolled back. ActiveMQ supports redelivery policies both client side and broker side that control how many time a message will be redelivered before sent to a DLQ.
I'm using ActiveMQ along with Mule (a kind of ESB based on Spring).
We got a fast producer and a slow consumer.
It's synchronous configuration with only one consumer.
Here the configuration of the consumer in spring style: http://pastebin.com/vweVd1pi
The biggest requirement is to keep the order of the messages.
However, after hours of running this code, suddenly, ActiveMQ skips 200 messages, and send the next ones.The 200 messages are still there in the activeMQ, they are not lost.
But our client (Mule), does have some custom code to check the order of the messages, using an unique identifier.
I had this issue already a few month ago. We change the consumer by using the parameter "jms.prefetchPolicy.queuePrefetch=1". It seemed to have worked well and to be the fix we needed unti now when the issue reappeared on another consumer.
Is it a bug, or a configuration issue ?
I can't talk about the requirement from a Mule perspective, but there are a couple of broker features that you should take a look at. There are two ways to guarantee message ordering in ActiveMQ:
Message groups are a way of ensuring that a set of related messages will be consumed by the same consumer in the order that they are placed on a queue. To use it you need to specify a JMSXGroupID header on related messages, and assign them an incrementing JMSXGroupSeq number. If a consumer dies, remaining messages from that group will be sent to another single consumer, while still preserving order.
Total message ordering applies to all messages on a topic. It is configured on the broker on a per-destination basis and requires no particular changes to client code. It comes with a synchronisation overhead.
Both features allow you to scale out to more than one consumer.