I am using the spring-cloud-starter-aws-messaging in my Spring Boot application and #sqslistener to consume SQS messages. my consumer weirdly stops getting messages out of nowhere and the ApproximateNumberOfMessagesVisible gradually increases triggering a CloudWatch alarm. I could see no error logs that are generated before it stops getting any more messages. Am I missing something?
#SqsListener(value = "${sqs.queue.url.indexSavedSetQueue}",
deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
public void listenIndexSavedSetEvent(#NonNull String message) throws IOException {
log.info("Index saved set event received, message: {}", message);
IndexSavedSetPayloadDto indexSavedSetPayloadDto = objectMapper
.readValue(message, IndexSavedSetPayloadDto.class);
String setName = indexSavedSetPayloadDto.getSetName();
indexerService.indexSet(setName);
}
Related
I'm trying to handle exception with an implementation of RabbitListenerErrorHandler in my project which uses Spring AMQP.
Here is how I defined my consumer:
#RabbitListener(queues = "inqueue", autoStartup = "true", concurrency = "3", returnExceptions = "true", errorHandler = "customRabbitListenerErrorHandler")
Here is my customRabbitListenerErrorHandler:
#Override
public Object handleError(org.springframework.amqp.core.Message message, Message<?> message1, ListenerExecutionFailedException e) throws Exception {
ObjectMapper objectMapper = new ObjectMapper();
return MessageBuilder.withBody(objectMapper.writeValueAsString(myCustomData).getBytes()).andProperties(e.getFailedMessage().getMessageProperties()).build();
}
My problem is that I can't deliver this exception message to consumer side. I'm getting this exception:
Caused by: org.springframework.amqp.AmqpException: Cannot determine ReplyTo message property value: Request message does not contain reply-to property, and no default response Exchange was set.
From this message I understood that there is no replyTo property in my messageProperties but I don't know how can get/found it. How can I send this exception message to consumer side?
The error handler should not return a org.springframework.amqp.core.Message message, it should just return myCustomData.
However, that should not prevent finding the replyTo header; please provide an MCRE so I can see what's wrong.
I am using org.springframework.boot:spring-boot-starter-amqp:2.6.6 .
According to the documentation, I set up #RabbitListener - I use SimpleRabbitListenerContainerFactory and the configuration looks like this:
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ObjectMapper om) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setAcknowledgeMode(AcknowledgeMode.MANUAL);
factory.setConcurrentConsumers(rabbitProperties.getUpdater().getConcurrentConsumers());
factory.setMaxConcurrentConsumers(rabbitProperties.getUpdater().getMaxConcurrentConsumers());
factory.setMessageConverter(new Jackson2JsonMessageConverter(om));
factory.setAutoStartup(rabbitProperties.getUpdater().getAutoStartup());
factory.setDefaultRequeueRejected(false);
return factory;
}
The logic of the service is to receive messages from rabbitmq, contact an external service via the rest API (using rest template) and put some information into the database based on the results of the response (using spring data jpa). The service implemented it successfully, but during testing it ran into problems that if any exceptions occur during the work of those thrown up the stack, the message is not sent to the configured dlq, but simply hangs in the broker as unacked. Can you please tell me how you can tell spring amqp that if any error occurs, you need to redirect the message to dlq?
The listener itself looks something like this:
#RabbitListener(
queues = {"${rabbit.updater.consuming.queue.name}"},
containerFactory = "rabbitListenerContainerFactory"
)
#Override
public void listen(
#Valid #Payload MessageDTO message,
Channel channel,
#Header(AmqpHeaders.DELIVERY_TAG) Long deliveryTag
) {
log.debug(DebugMessagesConstants.RECEIVED_MESSAGE_FROM_QUEUE, message, deliveryTag);
messageUpdater.process(message);
channel.basicAck(deliveryTag, false);
log.debug(DebugMessagesConstants.PROCESSED_MESSAGE_FROM_QUEUE, message, deliveryTag);
}
In rabbit managment it look something like this:
enter image description here
and unacked will hang until the queue consuming application stops
See error handling documentation: https://docs.spring.io/spring-amqp/docs/current/reference/html/#annotation-error-handling.
So, you just don't do an AcknowledgeMode.MANUAL and rely on the Dead Letter Exchange configuration for those messages which are rejected in case of error.
Or try to use a this.channel.basicNack(deliveryTag, false, false) in case of messageUpdater.process(message); exception...
I have a FIFO queue in SQS, where I keep a JSON with some information to be read and processed by a service in Spring Boot.
I have verified that the queue is showing an error message [The title of the issue] and is not processing some messages.
Follow my code:
#JmsListener(destination = "document_analysis.fifo")
public void documentAnalysis(#payload String document) throws Exception {
ObjectMapper objectMapper = new ObjectMapper();
DocumentAnalysisDTO documentDTO = objectMapper.readValue(document, DocumentAnalysisDTO.class);
this.clientService.prepareCustomerDocument(documentDTO);
}
I am using Spring DefaultMessageListenerContainer and JMS Message Listener to consume messages from Solace Queue. Client Acknowledgement is set to true.
In case of exception messages remain in the queue since they were not acknowledged and these are not redelivered. New messages that were pumped post exception are processed.
Have read about using session.recover but how do we get handle to session . Also tried setting maxredelivery to 3 . But not working.
public void onMessage(Message message) {
String text = null;
ALNTLogger.trace(CLAZZ_NAME, "onMessage()", "Message Received:");
try {
TextMessage textMessage = (TextMessage) message;
text = textMessage.getText();
ALNTLogger.trace(CLAZZ_NAME, "onMessage()", "Message Received: " + text);
Document xmlDocument = parseXml(text);
Map < String, String > values = getValues(xmlDocument);
saveValues(values);
message.acknowledge();
} catch (Exception ex) {
ALNTLogger.error(CLAZZ_NAME, "onMessage()", "Failed to process message:" + text);
throw new RuntimeException(ex);
}
}
Any help will be appreciated
It is expected that the message is not redelivered when using CLIENT_ACKNOWLEDGE acknowledgement mode with a DefaultMessageListenerContainer.
The Spring documentation states the following:
The listener container offers the following message acknowledgment
options:
"sessionAcknowledgeMode" set to "AUTO_ACKNOWLEDGE" (default):
Automatic message acknowledgment before listener execution; no
redelivery in case of exception thrown.
"sessionAcknowledgeMode" set
to "CLIENT_ACKNOWLEDGE": Automatic message acknowledgment after
successful listener execution; no redelivery in case of exception
thrown.
"sessionAcknowledgeMode" set to "DUPS_OK_ACKNOWLEDGE": Lazy
message acknowledgment during or after listener execution; potential
redelivery in case of exception thrown.
"sessionTransacted" set to
"true": Transactional acknowledgment after successful listener
execution; guaranteed redelivery in case of exception thrown.
You can use the last option, transactional acknowledgements, in order to have the message redelivered when the onMessage() method does not return normally.
I am using Kafka Spring Integration for publishing and consuming messages using kafka. I see Payload is properly passed from producer to consumer, but the header information is getting overridden somewhere.
#ServiceActivator(inputChannel = "fromKafka")
public void processMessage(Message<?> message) throws InterruptedException,
ExecutionException {
try {
System.out.println("Headers :" + message.getHeaders().toString());
}
} catch (Exception e) {
e.printStackTrace();
}
}
I get following headers:
Headers :{timestamp=1440013920609, id=f8c645f7-677b-ec32-dad0-a7b79082ef81}
I am constructing the message at producer end like this:
Message<FeelDBMessage> message = MessageBuilder
.withPayload(samplePayloadObj)
.setHeader(KafkaHeaders.MESSAGE_KEY, "key")
.setHeader(KafkaHeaders.TOPIC, "sampleTopic").build();
// publish the message
publisher.publishMessage(message);
and below is the header info at producer:
headers={timestamp=1440013914085, id=c4159c1c-2c67-634b-ef8d-3fb026b1172e, kafka_messageKey=key, kafka_topic=sampleTopic}
Any idea why the Headers are overridden by a different value?
Just because by default Framework uses the immutable GenericMessage.
Any manipulation to the existing message (e.g. MessageBuilder.withPayload) will produce a new GenericMessage instance.
From other side Kafka doesn't support any headers abstraction like JMS or AMQP. That's why KafkaProducerMessageHandler just do this when it publishes a message to Kafka:
this.kafkaProducerContext.send(topic, partitionId, messageKey, message.getPayload());
As you see it doesn't send headers at all. So, other side (consumer) just deals with only message from the topic as a payload and some system options as headers like topic, partition, messageKey.
In two words: we don't transfer headers over Kafka because it doesn't support them.