JMS Question:- Is there any way to change the messages order in JMSQUEUE based on custom rule/policy/strategy? Other then FIFO..
You can try setting JMSPriority property on the messages sent to the queue. JMSPriority can have value from 0 to 9, with 9 been the highest. If multiple messages are present in the queue then the message with the highest priority will be consumed first.
This link could be helpful for you,
http://www.techpaste.com/2012/04/prioritize-messages-enqueued-jms-queue/
Related
I'm working with Spring Cloud Stream and Rabbit, and I used the config defined here to set up a dead-letter queue (DLQ) and it works very nicely.
What I'd like to do is set a maximum amount of times a message goes to the DLQ before being discarded - is is possible to set this via config? If so, how? If not, what should I do to achieve this behaviour?
I'm looking for a code sample for the best answer, preferably in Kotlin (if relevant)
That depends whether you're using qurorum queues or not. I don't believe there's a default config for that without quorum queues. However, you should be able to store the redelivery count within a custom header or in the message itself.
Then if you set a MAX_REDELIVERY_COUNT constant in your application, you can check if the message exceeds the maximum number of redeliveries.
If you're not using quorum queues, I'd take a look at this answer:
How do I set a number of retry attempts in RabbitMQ?.
This answer has quite some good options.
However, when using quorum queues, you can set the delivery-limit option. More info on that can be found here: https://www.rabbitmq.com/quorum-queues.html#feature-matrix.
Edit 1: using custom headers
In order to publish a message with custom headers:
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("latitude", 51.5252949);
headers.put("longitude", -0.0905493);
channel.basicPublish(exchangeName, routingKey,
new AMQP.BasicProperties.Builder()
.headers(headers)
.build(),
messageBodyBytes);
As found on https://www.rabbitmq.com/api-guide.html#publishing.
The problem is that the headers can't be simply updated. However, you could do this with a workaround. Let's say you want a maximum of 5 retries per message. If the message can't be processed, send it to a DLX. If the message doesn't exceed the maximum retries, read the original headers of the message, update the custom retry count header and resend it to the original queue.
If the message gets in de DLX and does exceed the maximum retry count, send the message as is to the DLX with a different routing key, which is bound to a queue for the "definitive" dead messages.
That'd mean that you would get something like this in a simplified diagram:
This is just an idea, I don't know if it'll work for sure, but it's the best that I can think of in your situation.
Edit 2: using the autoBindDlq
It seems like the Spring Cloud Stream Binder for RabbitMQ has this option. In the docs as found on https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit, it says the following:
By using the optional autoBindDlq option, you can configure the binder to create and configure dead-letter queues (DLQs) (and a dead-letter exchange DLX, as well as routing infrastructure). By default, the dead letter queue has the name of the destination, appended with .dlq. If retry is enabled (maxAttempts > 1), failed messages are delivered to the DLQ after retries are exhausted. If retry is disabled (maxAttempts = 1), you should set requeueRejected to false (the default) so that failed messages are routed to the DLQ, instead of being re-queued. In addition, republishToDlq causes the binder to publish a failed message to the DLQ (instead of rejecting it). This feature lets additional information (such as the stack trace in the x-exception-stacktrace header) be added to the message in headers. See the frameMaxHeadroom property for information about truncated stack traces. This option does not need retry enabled. You can republish a failed message after just one attempt. Starting with version 1.2, you can configure the delivery mode of republished messages. See the republishDeliveryMode property.
I have 5 input Queues, 5 message flows for each. After some processing messages from all queues go to one transport queue.
Is there a possibility to set a priority across queues, for example messages from input queue 1 will always be processed and put in the transport queue first?
It sounds like you need to set the message priority.
You set the priority of a message (in the Priority field of the MQMD
structure) when you put the message on a queue
... you can create messages having priorities between 0 (the lowest) and 9
(the highest).
Hi use the Sequence Node to done this task.
https://www.ibm.com/support/knowledgecenter/en/SSMKHH_10.0.0/com.ibm.etools.mft.doc/bc28010_.htm
I need to modify some of the messages that are already present in the queues.
Can it be possible to modify the message and send new modified message on the same queue for processing.
If yes, what would be the best approach for this.
Thanks.
The short answer is no. In JMS messages are immutable once they have been sent. If you find you need to modify messages its recommended that you create a consumer with some selector which matches the messages you wish to update, consume them and send new modified messages, either to another queue or if you are careful, back to the original queue.
ref link Can I modify messages on a queue?
my opinion is if you are consume the message using selector like correlationId,then consume the particular message from queue and again post the new message with same correlationId of previous message.but this will depends up on your requirements.
We are using IBM WebSphere MQ as JMS provider with Spring MDP (Message Driven POJO).
Is there any way in JMS where we can configure time related properties in message so that message can be consumed at particular defined time only?
For example, if I am sending three messages into queue M1, M2 and M3. Where, I can configure M2 message property let say 3 AM. And consumer side, consumer can only pick this message # 3 AM only. If time is not defined, messages should be consumed in a way that JMS Receiver does.
JMS 2.0 specification has defines Delivery Delay. With this feature a message producer can specify that a message must not be delivered until after a specified time interval. The message will be available for delivery after the specified time. But this may not help you as you want to a message to be consumed at a specified time. Typically messaging applications are designed to consume messages as soon as they are made available by the messaging provider.
If you want to process messages at a specified time only, you could create another queue "queue_3am", and schedule a reader to run exactly at 3am.
A variation is to set the timestamp as a message property. So one queue could contain messages to be processed at different points in time. The reader could use message selectors to get relevant messages only.
But you should use a "message pickup timeframe" by adding two timestamps as message properties, for eaxmple set the window to 1 or 5 minutes.
The receiver can use a message selector: A selector is a condition using message properties.
Have a look at this
I have a design query regarding queues. My scenario is as follows:
I have to use a messaging system, with single producer and multiple consumers (asynchronous). The producer pushes different types of messages into the messaging system. Depending upon the message type, that particular consumer has to consume that message. (Each consumer is running on a different server). If one consumer is down and a message comes for that consumer, it will be in the messaging system only. If I use a message queue, the message in the queue will block the next messages that can be consumed by the other consumers. Are queues suitable for handling this kind of situation? Or do we need to go for a topic?
Whether you use a queue or a topic should depend on whether there an instance where multiple consumers must process the same message. If that is the case then a topic is required do generate that one-to-many pattern.
On the other hand, if any one message will only ever be consumed by one consumer, then you can use a queue or topic and the consumers specify the message type as a JMS selector. In this way, all consumers can listen on the same queue and each selects a different subset of messages. In the event one application is not there, it's messages do not "block the next messages that can be consumed by the other consumers" but rather they just stack up in the queue and other consumers still receive their messages based on selection criteria.
Please also realize that queues are lightweight constructions and you can easily have one queue per consumer. Typically, things providing a service listen on a well-known queue and each queue represents a different function of the service or a different service. Thus there may be many service input queues. Similarly, reply messages are generally uniquely addressed to the application instance that made the request and go to a unique, often dynamic, reply-to queue. Both of these implementations I have described lead to a separation of traffic across queues rather than pooling different message types into the same queue. Since JMS selectors always impart an additional processing cost, using more queues is generally more performant than selecting many types of message from the same queue.
I am responding to your question about selectors in the comment section here since I have more space and can put links in...
Section 3.8.1 of the JMS 1.1 spec states:
A JMS message selector allows a client to specify, by message header, the
messages it’s interested in. Only messages whose headers and properties
match the selector are delivered. The semantics of not delivered differ a bit
depending on the MessageConsumer being used. See Section 5.8,
“QueueReceiver,” and Section 6.11, “TopicSubscriber,” for more details.
Message selectors cannot reference message body values.
A message selector matches a message if the selector evaluates to true when
the message’s header field and property values are substituted for their
corresponding identifiers in the selector.
As noted above, selectors can be on fields that are implicit in the message such as MsgID or CorrelationID or thsey can be on fields specifically set by the message producer such as a message property. Either way, the client must specify the value of any selectors used by the message consumer.