When using an InOut paradigm to send request/response messages to a JMS endpoint, the Camel JMS documentation describes how to set a message expiration on the request message, but it doesn't describe whether the response message will have a timeout (JMS expiration) set when it's sent by the consumer of the request message. The documentation does describe the replyToDeliveryPersistent URI option that the consumer can set to specify that the response message it sends should be non-persistent, but I don't see anything that would let you specify whether the response message will have an expiration date set.
Is there some way to ensure that response messages will be expired (so they can be automatically removed from the broker) if they are unconsumed for a certain amount of time (e.g. because the producer of the request was killed before it read the response to its final request), without implementing a custom ActiveMQ consumer and losing the benefits of using Camel? I control both the producer and the consumer, so the changes can happen at whichever side they need to be made (and I'm aware that the consumer end is the place this would likely need to be done); the sole criterion here is that Camel must remain the method for processing the message and responding with a reply message, because having to implement that by hand would be worse than living with persistent reply messages.
The only way to set an expiration time (JMSExpiration header) on a message is from its production point (i.e. by the producer).
In your case, the consumer of your request should set the JMSExpiration date explicitly1, using the JMS component's URI option timeToLive, and there is no way to set this option from the requesting side.
1) JMS specification points out that instead of setting the JMSExpiration directly, JMS clients should specify the time-to-live. The header is then calculated as the sum of the time-to-live and the current GMT value.
Related
I'm working with Spring Cloud Stream and Rabbit, and I used the config defined here to set up a dead-letter queue (DLQ) and it works very nicely.
What I'd like to do is set a maximum amount of times a message goes to the DLQ before being discarded - is is possible to set this via config? If so, how? If not, what should I do to achieve this behaviour?
I'm looking for a code sample for the best answer, preferably in Kotlin (if relevant)
That depends whether you're using qurorum queues or not. I don't believe there's a default config for that without quorum queues. However, you should be able to store the redelivery count within a custom header or in the message itself.
Then if you set a MAX_REDELIVERY_COUNT constant in your application, you can check if the message exceeds the maximum number of redeliveries.
If you're not using quorum queues, I'd take a look at this answer:
How do I set a number of retry attempts in RabbitMQ?.
This answer has quite some good options.
However, when using quorum queues, you can set the delivery-limit option. More info on that can be found here: https://www.rabbitmq.com/quorum-queues.html#feature-matrix.
Edit 1: using custom headers
In order to publish a message with custom headers:
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("latitude", 51.5252949);
headers.put("longitude", -0.0905493);
channel.basicPublish(exchangeName, routingKey,
new AMQP.BasicProperties.Builder()
.headers(headers)
.build(),
messageBodyBytes);
As found on https://www.rabbitmq.com/api-guide.html#publishing.
The problem is that the headers can't be simply updated. However, you could do this with a workaround. Let's say you want a maximum of 5 retries per message. If the message can't be processed, send it to a DLX. If the message doesn't exceed the maximum retries, read the original headers of the message, update the custom retry count header and resend it to the original queue.
If the message gets in de DLX and does exceed the maximum retry count, send the message as is to the DLX with a different routing key, which is bound to a queue for the "definitive" dead messages.
That'd mean that you would get something like this in a simplified diagram:
This is just an idea, I don't know if it'll work for sure, but it's the best that I can think of in your situation.
Edit 2: using the autoBindDlq
It seems like the Spring Cloud Stream Binder for RabbitMQ has this option. In the docs as found on https://github.com/spring-cloud/spring-cloud-stream-binder-rabbit, it says the following:
By using the optional autoBindDlq option, you can configure the binder to create and configure dead-letter queues (DLQs) (and a dead-letter exchange DLX, as well as routing infrastructure). By default, the dead letter queue has the name of the destination, appended with .dlq. If retry is enabled (maxAttempts > 1), failed messages are delivered to the DLQ after retries are exhausted. If retry is disabled (maxAttempts = 1), you should set requeueRejected to false (the default) so that failed messages are routed to the DLQ, instead of being re-queued. In addition, republishToDlq causes the binder to publish a failed message to the DLQ (instead of rejecting it). This feature lets additional information (such as the stack trace in the x-exception-stacktrace header) be added to the message in headers. See the frameMaxHeadroom property for information about truncated stack traces. This option does not need retry enabled. You can republish a failed message after just one attempt. Starting with version 1.2, you can configure the delivery mode of republished messages. See the republishDeliveryMode property.
I have a webservice request that is handled by BPEL flow. This BPEL flow puts this message on a Queue using MQ JMS Import Binding. It contains a response queue information as well. I need to set custom correlationID or JMSMessageID value into CorrelationID so that BPEL flow can correlate the response to request. Please suggest if you have some idea.
JMSMessageID should not be set by an application. If it is set, during a message send, it will be ignored. When the send method returns, the JMSMessageID will be set to a JMS Provider assigned value. When a message is received, it will have a provider assigned unique value. Note JMSMessageID is unique.
Quoting from JMS 1.1 Specs:
A client can use the JMSCorrelationID header field to link one message with another. A typical use is to link a response message with its request message. Since each message sent by a JMS provider is assigned a message ID value, it is convenient to link messages via message ID. All message ID values must start with the ‘ID:’ prefix.
In some cases, an application (made up of several clients) needs to use an application-specific value for linking messages. For instance, an application may use JMSCorrelationID to hold a value referencing some external information. Application-specified values must not start with the ‘ID:’ prefix; this is reserved for provider-generated message ID values.
To set JMSCorrelationID, application need to call setJMSCorrelationID method of JMSMessage class with String parameter.
In CICS we have something called a ENQ command which is useful for single-threading based on desired values . Similarly do we have anything in Websphere?
ie., I want my MDB to read multiple messages from the input queue. However I want to process the messages strictly in the received order. Setting the value of 'Server session' in the activation spec property to 1 will achieve this but I want to do it programatically within my MDB
Below is a quote from JMS 1.1 specification. It seems that ordering is only guaranteed if you send messages from a single session; in that case the sessions input message stream will be ordered. Since no API exists for limiting session count, I guess you must depend on server facilities for this configuration.
4.4.10 Message Order JMS clients need to understand when they can depend on message order and when they cannot.
4.4.10.1 Order of Message Receipt Messages consumed by a session
define a serial order. This order is important because it defines the
effect of message acknowledgment. See Section 4.4.11 ”Message
Acknowledgment,” for more details. The messages for each of a
session’s consumers are interleaved in a session’s input message
stream. JMS defines that messages sent by a session to a destination
must be received in the order in which they were sent (see Section
4.4.10.2 ”Order of Message Sends,” for a few qualifications). This defines a partial ordering constraint on a session’s input message
stream. JMS does not define order of message receipt across
destinations or across a destination’s messages sent from multiple
sessions. This aspect of a session’s input message stream order is
timing-dependent. It is not under application control.
We are using IBM WebSphere MQ as JMS provider with Spring MDP (Message Driven POJO).
Is there any way in JMS where we can configure time related properties in message so that message can be consumed at particular defined time only?
For example, if I am sending three messages into queue M1, M2 and M3. Where, I can configure M2 message property let say 3 AM. And consumer side, consumer can only pick this message # 3 AM only. If time is not defined, messages should be consumed in a way that JMS Receiver does.
JMS 2.0 specification has defines Delivery Delay. With this feature a message producer can specify that a message must not be delivered until after a specified time interval. The message will be available for delivery after the specified time. But this may not help you as you want to a message to be consumed at a specified time. Typically messaging applications are designed to consume messages as soon as they are made available by the messaging provider.
If you want to process messages at a specified time only, you could create another queue "queue_3am", and schedule a reader to run exactly at 3am.
A variation is to set the timestamp as a message property. So one queue could contain messages to be processed at different points in time. The reader could use message selectors to get relevant messages only.
But you should use a "message pickup timeframe" by adding two timestamps as message properties, for eaxmple set the window to 1 or 5 minutes.
The receiver can use a message selector: A selector is a condition using message properties.
Have a look at this
I have many instances of my client application. These clients send requests to a server application via messaging and receive a reply. Normally the reply would be sent using a temporary queue.
Unfortunately I have to use the Stomp protocol which has no concept of temporary queues or topics. (Although the message broker has)
What's the best way to ensure only the original requestor receives the reply? Are there any best-practices for this unfortunate situation?
The customary solution when several requestors listen for replies on the same queue is to use correlation IDs to select messages. On the client side it looks like this:
Place a message on the request queue and commit.
Retrieve the JMSMessageID from the outbound message (the value is determined by the broker and updates the message object as a result of the send).
Receive a message from the reply queue specifying the JMSMessageID from the outbound message as the correlation ID in the selector.
Process and commit.
On the server side it looks like this:
Receive a message under syncpoint.
Process the request and prepare the response.
Set the JMSCorrelationID on the response to the value of JMSMessageID from the request.
Send the message.
Commit.
The consumer would set the selector something like this: activemq.selector:JMSCorrelationID=.
Since the broker creates a message ID that is supposed to be globally unique, the pattern of using it as the correlation ID prevents collisions that are possible when each requestor is allowed to specify it's own value.
The best way to implement this pattern with JMS (that I've found, anyway) is to create a pre-configured topic for the response messages, and use correlation selectors on the response message so that the client can get the correct one.
In more detail, this means setting a random ID on the request message (using setJMSCorrelationID()), and putting that message on the request Queue. The consumer of that request message processes it, creates the response message, sets the same correlation ID on the response message, and puts it on the response Topic. The client, meanwhile, is listening on the response topic with a selector expression which specifies the correlation ID that it's expecting.
The danger is that the response message is sent before the client can get around to listening for it, although that's probably unlikely. You can try using a pre-configured Queue for the responses rather than a topic, but I've found that topics tend to work more reliably (my JMS provider of choice is HornetQ - your mileage may vary).
All this is tell me that JMS is a very poor fit for the request/response model. The API just doesn't support it properly. This is hardly surprising, since that was never a use-case for JMS.
Something like a compute grid (Terracotta, Gigaspaces, Infinispan, etc) would likely yield better results, but that's not really an option for you.