I want to user Spring's retry on failed message processing when receiving messages from RabbitMQ.
Do I need to set requeue=true in order to for the message after several retries to end up in dead letter exchange?
Does retry means that the message is sent back to queue each time the processing is failed?
There are a number of properties related to retry; see the documentation.
You can add a RejectAndDontRequeueRecoverer bean to cause the message to be routed to the DLQ when retries are exhausted.
Related
I have Kafka listener writing data to a database, in case of database timeout (JdbcException), I'd like to retry, and if timeouts persist, stop consuming Kafka message.
As far as I understand, Spring Kafka 2.9 has 2 CommonErrorHandler implementations:
DefaultErrorHandler tries to redeliver messages several times and then send failed messages to logs or DLQ
CommonContainerStoppingErrorHandler stops listener container and message consumption
I would like to chain both of them: first try to redeliver messages several times, then stop container when delivery doesn't succeed.
How can I do that?
Use a DefaultErrorHandler with a custom recoverer that calls a CommonContainerStoppingErrorHandler after the retries are exhausted.
See this answer for an example
How do you exit spring boot application programmatically when retries are exhausted, to prevent kafka offset commit
(It uses the older SeekToCurrentErrorHandler, but the same concept applies.)
How can we implement redelivery policy from consumer side with stomp in ActiveMQ?
Is it even be possible?
Yes. Look to using the local transaction support in STOMP. This will enable you to commit and rollback messages to provide a redelivery pattern.
A recommended consumer error handling pattern:
If the message is invalid (ie.. bad JSON or XML) move to DLQ immediately. The message will never improve in quality and there is no reason to do repeated retries.
If the 'next step' in processing is down (ie. the database) reject delivery and the implement a delay between your next receive() call and track a retry limit to provide an escape hatch. This also has the benefit of allowing other consumers on the queue to attempt processing the message and eliminates the problem where one consumer has a dead path from holding up a messages.
In my project, I am using Spring Kafka listener to consume messages from Kafka. I have a doubt that if the consume method code gets blocked due to some reason and never returned back, in this case, will this listener be able to receive new messages and proceed further or it will be hanged? In my case, it looks like, Kafka listener also got blocked and not processing further messages, even, another consumer of same group is also not receiving messages.
No; you will not get more records while a thread is blocked, unless the concurrency is > 1 and there are at least that many partitions. Even then, you will receive no more messages for the partition(s) assigned to the blocked consumer.
I'm currently investigating the Smallrye Reactive Messaging integration in Quarkus.Sending and receiving messages is really simple and elegant at first glance.
But one thing which I didn't find out is: How to handle a re-delivery of messages?
Example: We receive a message and try to process it. Some exception (maybe a DB not available or an optimistic lock exception or something) happens.
In such a case I would throw an exception so that the message is not acknowledged. But currently I see no way how the message is redelivered.
I set-up a small dummy project to test this:
Quarkus
ActiveMQ Artemis
send a message (via Artemis console) into a queue
-- queue configured with max redelivery = 3
receive the message with Quarkus / Smallrye Reactive Messaging #Incoming annotation
throw exception in the #Incoming method
--> Message is removed from the Artemis queue
--> #Incoming method is only called once
If I shutdown the Quarkus App, the message can be seen again in the Artemis queue with redelivered flag set to true.
But I find no way how I can manage/configure a redelivery in the Smallrye Reactive Messaging so that this layer handles the redelivery of a message for n times and puts the message into a DLQ after the max retries.
Is there any way to do this?
I am developing an application in which I have jms message queue.
There is a producer which enqueue message to the queue and a consumer to dequeue the message.
There might be cases when consumer is not running. If a message is not consumed by the consumer within a certain amount of time I need to catch that from producer.
I want to use JMX to monitor message queue's message whether it is expired.
Any suggestion or sample code how to this.....
It depends on the JMX implementer... Some servers provide JMX implementations to monitor its resources. If its not provided, then you will need to write the JMX implementation that uses the API provided by the MQ implementer.
An easier way to solve this problem is to use the request-response pattern with expiry. The consumer needs to respond in a specified internal of time. If it can't then the message on the queue can expire. If the response is not received the producer can take further action. JMS selector with correlation ID can be used to relate the responses with the request.