In a kafka streams application if the process() method in your processor throws an uncaught exception the stream task will be killed. However if you have a punctuate() method (using context.schedule()) and it throws an uncaught exception the task will not be killed. Why is this? What is the proper way to kill the task if you wanted uncaught exceptions in punctuate() to stop the task?
Related
Using kafkastreams with Spring Boot, I have an exception that is caught in the punctuator like so:
try {
return record.toBuilder().apiResponse(userFeed.addActivity(activity).join()).success(Boolean.TRUE).build();
} catch (StreamException e) {
e.printStackTrace();
return record.toBuilder().success(Boolean.FALSE).build();
}
However, this exception is causing the streams app to shutdown:
org.apache.kafka.streams.errors.StreamsException: stream-thread [notification-service-StreamThread-1] task [0_2] Exception caught while punctuating processor 'KSTREAM-PROCESSOR-0000000001'
2023-01-20T15:39:33.269Z ERROR 1 --- [-StreamThread-1] org.apache.kafka.streams.KafkaStreams : stream-client [notification-service] Encountered the following exception during processing and the registered exception handler opted to SHUTDOWN_CLIENT. The streams client is going to shut down now.
How do I change the registered exception handler to not shut down the client? All the docs talk about uncaught exception handlers, but I am catching this exception. I don't even know what the registered handler is.
It turns out that the exception is not in fact caught correctly because of thread weirdness, so setting the uncaught exception handler fixed this.
I want to catch an exception that occurs within kafka listener container, log some attributes from the exception, then rethrow the exception. What is the best way to do this without modifying the existing error handling flow?
Add a MethodInterceptor to the container's adviceChain.
https://docs.spring.io/spring-kafka/docs/current/reference/html/#adviceChain
In my spring JMS listener onMessage method, I am using a reactive pipeline as as below where I receive a JMS message from a queue and transform that to a different format and publishing to another topic. Without the reactive flow, in the case of failure in output publishing, I can throw a runtimeexception for the same message to redeliver from the queue. But by using the below flow of reactive approach, I cannot find a way to throw an exception outside. Can you please help on what should be the approach here for throwing the exception outside to get the message redelivered
public void onMessage(Message message) {
Mono.just(message)
.doOnNext(LoggingUtil::logMessage)
.flatMapIterable(messageTransformer::transformToTriggerEvents)
.doOnNext(LoggingUtil::logTriggerEvent)
.doOnNext(triggerEventPublisher::publish)
.subscribe();
}
I'm using spring boot with spring-amqp and annotation based listener to consume message from a rabbitmq broker.
I've a spring component which contains a method like this:
#RabbitListener(queues = "tasks")
public void receiveMessage(#Payload Task task) {...}
I'm using the AUTO mode to acknowledge messages after successful execution of receiveMessage(...). If i detect a special error, i'm throwing AmqpRejectAndDontRequeueException to get this message into a configured dead letter queue. Now i need to nack a message only, so that the message gets requeued into the main queue of rabbitmq and another consumer has the possibility to work on that message again.
Which exception should i throw for that? I wouldn't like to use channel.basicNack(...) like described here (http://docs.spring.io/spring-integration/reference/html/amqp.html) if possible.
As long as defaultRequeueRejected is true (the default) in the container factory, throwing any exception other than AmqpRejectAndDontRequeueException will cause the message to be rejected and requeued.
The exception must not have a AmqpRejectAndDontRequeueException in its cause chain (the container traverses the causes to ensure there is no such exception).
As we known, if there is any exception is thrown at onMessage method of MessageListener, the JMS will try to consume that message again, which called "Redelivery".
But what I'm curious is how did it implement this feature? How did onMessage methods know if there is any exception is thrown? I know the answer may be a basic java knowledge. But unfortunately, I still have no idea on it.
So... is there anyone can clarify it?
This is specific to a particular JMS provider. But if your onMessage() method throws an exception, the code of the JMS provider that calls your onMessage() can just catch it, .e.g
try {
listener.onMessage(..);
} catch (Exception ex) {
//handle exception
}
And if an exception is caught, it can employ a strategy for redelivering the message. The client could send a message back to the JMS broker telling the broker that message delivery failed, and let the broker re-deliver that message.