Spring - Close RabbitMQ Consumer connection without closing spring application
We have a requirement we want to close the rabbitmq connection, wait for few minutes before cleaning up other resources and then closing the spring application. We have a buffer of 6 minutes to clean up the other resources. However, during this time, the rabbitmq connection is auto recreated.
We are creating the connections as follows...
#Configuration
public class RabbitMQSubscribersHolder extends BaseRabbitMQConfig {
private static final Logger LOGGER = LoggerFactory.getLogger(RabbitMQSubscribersHolder.class);
private ConcurrentMap<String, SimpleMessageListenerContainer> containers = new ConcurrentHashMap();
public RabbitMQSubscribersHolder() {
}
public void addSubscriber(String mqAlias, MessageListener receiver) {
ConnectionFactory connectionFactory = this.connectionFactory(mqAlias);
RabbitAdmin admin = null;
try {
admin = new RabbitAdmin(connectionFactory);
AbstractExchange exchange = this.exchange(mqAlias);
admin.declareExchange(exchange);
Queue queue = null;
if (!StringUtils.isEmpty(this.getMqQueueNameForQueueAlias(mqAlias))) {
queue = this.queue(mqAlias);
admin.declareQueue(queue);
} else {
queue = admin.declareQueue();
}
admin.declareBinding(this.binding(mqAlias, queue, exchange));
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(new String[]{queue.getName()});
container.setMessageListener(this.listenerAdapter(receiver));
container.setRabbitAdmin(admin);
container.start();
this.containers.put(mqAlias, container);
} catch (Exception var8) {
LOGGER.error(var8.getMessage(), var8);
}
}
MessageListenerAdapter listenerAdapter(MessageListener receiver) {
return new MessageListenerAdapter(receiver, "onMessage");
}
public SimpleMessageListenerContainer getContainer(String mqAlias) {
return (SimpleMessageListenerContainer)this.containers.get(mqAlias);
}
This is our destroy method
public void destroy() {
Iterator var1 = this.aliasToConnectionFactory.entrySet().iterator();
while(var1.hasNext()) {
Map.Entry<String, CachingConnectionFactory> entry = (Map.Entry)var1.next();
try {
((CachingConnectionFactory)entry.getValue()).destroy();
LOGGER.info("RabbitMQ caching connection factory closed for alias: {}", entry.getKey());
} catch (Exception var4) {
LOGGER.error("RabbitMQ caching connection destroy operation failed for alias: {} due to {}", entry.getKey(), StringUtils.getStackTrace(var4));
}
}
}
It seems the connection is re-created automatically as soon as it is destroyed (verified from Rabbit UI). Here are the logs:
2022-07-11 19:42:55.334 INFO 35761 --- [ Thread-11] c.g.f.f.r.GracefulShutdownRabbitMQ : RabbitMQ caching connection factory closed for alias: preMatchFreeze
2022-07-11 19:42:55.395 INFO 35761 --- [cTaskExecutor-1] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer: tags=[{amq.ctag-tSAgXuPdMnLd9q79ryBWGg=fury_pre_matchfreeze_queue}], channel=Cached Rabbit Channel: AMQChannel(amqp://gwportal#10.24.128.76:5672/,1), conn: Proxy#205b73d8 Shared Rabbit Connection: null, acknowledgeMode=AUTO local queue size=0
2022-07-11 19:42:55.405 INFO 35761 --- [ Thread-11] c.g.f.f.r.GracefulShutdownRabbitMQ : RabbitMQ caching connection factory closed for alias: parentMatchFreezeEds
2022-07-11 19:42:55.447 INFO 35761 --- [ Thread-11] c.g.f.f.r.GracefulShutdownRabbitMQ : RabbitMQ caching connection factory closed for alias: fantasy_matchFreeze
2022-07-11 19:42:55.489 INFO 35761 --- [ Thread-11] c.g.f.f.r.GracefulShutdownRabbitMQ : RabbitMQ caching connection factory closed for alias: parentMatchFreezeCore
2022-07-11 19:42:55.489 INFO 35761 --- [ Thread-11] c.g.f.fury.GracefulShutdownHandler : Waiting before starting graceful shutdown
2022-07-11 19:42:55.794 INFO 35761 --- [cTaskExecutor-1] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer: tags=[{amq.ctag-sK8_FY4PtHW52HXkQV3S_w=fury_match_freeze_queue}], channel=Cached Rabbit Channel: AMQChannel(amqp://gwportal#10.24.128.76:5672/,1), conn: Proxy#4bc9389 Shared Rabbit Connection: null, acknowledgeMode=AUTO local queue size=0
2022-07-11 19:42:56.046 INFO 35761 --- [cTaskExecutor-2] o.s.a.r.c.CachingConnectionFactory : Created new connection: SimpleConnection#3828e912 [delegate=amqp://gwportal#10.24.128.76:5672/, localPort= 58170]
2022-07-11 19:42:56.098 INFO 35761 --- [cTaskExecutor-1] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer: tags=[{amq.ctag-KyvnWaqZco9NRTessjupJQ=fantasy_parent_match_freeze_queue}], channel=Cached Rabbit Channel: AMQChannel(amqp://gwportal#10.24.128.76:5672/,1), conn: Proxy#17d45cfb Shared Rabbit Connection: null, acknowledgeMode=AUTO local queue size=0
2022-07-11 19:43:13.127 INFO 35761 --- [cTaskExecutor-2] o.s.a.r.c.CachingConnectionFactory : Created new connection: SimpleConnection#4c3cefad [delegate=amqp://gwportal#10.24.141.77:5672/, localPort= 58182]
2022-07-11 19:43:16.849 INFO 35761 --- [cTaskExecutor-2] o.s.a.r.c.CachingConnectionFactory : Created new connection: SimpleConnection#801eaf8 [delegate=amqp://gwportal#10.24.141.77:5672/, localPort= 58185]
You need to stop the message listener container(s) to prevent them from trying to reconnect.
I have two modules client and server. The client has controllers, which call methods in services on the server. When I try to call service methods I get a listener exception.
I have Spring configuration for RPC call, migrated from the old project.
Server-side:
I have a proxy factory:
#Component
public class AmqpProxyFactory {
private final ConnectionFactory connectionFactory;
private final String exchange;
private final Integer timeout;
private List<SimpleMessageListenerContainer> listenersRegistry;
private final MessageConverter clientConverter;
private final MessageConverter serverConverter;
#Autowired
public AmqpProxyFactory(ConnectionFactory connectionFactory,
#Value("${rabbit.marsFacade.exchange}") String exchange,
#Value("${rabbit.marsFacade.timeout}") Integer timeout,
#Qualifier("clientSecurityContextMessageConverter")MessageConverter clientConverter,
#Qualifier("serverSecurityContextMessageConverter")MessageConverter serverConverter) {
this.connectionFactory = connectionFactory;
this.exchange = exchange;
this.timeout = timeout;
this.clientConverter = clientConverter;
this.serverConverter = serverConverter;
listenersRegistry = new ArrayList<>();
}
private RabbitTemplate createTemplate(String route) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setRoutingKey(route);
rabbitTemplate.setExchange(exchange);
rabbitTemplate.setReplyTimeout(timeout);
rabbitTemplate.setMessageConverter(clientConverter);
// rabbitTemplate.setUseDirectReplyToContainer(false);
return rabbitTemplate;
}
public <T> T createClient(String route, Class<T> serviceInterface) throws Exception {
AmqpProxyFactoryBean proxy = new AmqpProxyFactoryBean();
proxy.setAmqpTemplate(createTemplate(route));
proxy.setServiceInterface(serviceInterface);
proxy.afterPropertiesSet();
return serviceInterface.cast(proxy.getObject());
}
private AmqpInvokerServiceExporter createExporter(Class<?> serviceInterface, Object service, String route) {
AmqpInvokerServiceExporter result = new AmqpInvokerServiceExporter();
result.setServiceInterface(serviceInterface);
result.setService(service);
result.setAmqpTemplate(createTemplate(route));
result.setMessageConverter(serverConverter);
return result;
}
public SimpleMessageListenerContainer createListenerContainer(Class<?> serviceInterface, Object service, String route, String queueName, Integer concurrency) {
SimpleMessageListenerContainer listenerContainer = new SimpleMessageListenerContainer();
listenerContainer.setQueues(new Queue(
queueName,
true,
false,
false,
ImmutableMap.<String, Object>builder()
.put("x-message-ttl", timeout)
.build()
));
listenerContainer.setMaxConcurrentConsumers(concurrency);
listenerContainer.setMessageListener(createExporter(serviceInterface, service, route));
listenerContainer.setConnectionFactory(connectionFactory);
listenersRegistry.add(listenerContainer);
return listenerContainer;
}
public List<SimpleMessageListenerContainer> getListenersRegistry() {
return listenersRegistry;
}
Here I create client
#Configuration
public class AmqpClientsConfig {
#Lazy
#Bean(name = "testRebbitService")
public TestRabbitService testRebbitService(AmqpProxyFactory amqpProxyFactory) throws Exception {
return amqpProxyFactory.createClient(TestRabbitServiceRabbitConfig.ROUTE_KEY, TestRabbitService.class);
}
Configuration for every using service:
#Configuration
public class TestRabbitServiceRabbitConfig {
public static final String QUEUE_NAME = "com.xxx.yyy.service.queue.testRabbitService";
public static final String ROUTE_KEY = "com.xxx.yyy.service.testRabbitService";
#Autowired
RabbitCommonConfig commonConfig;
#Bean
public Queue testRabbitServiceRabbitQueue() {
return commonConfig.buildServiceQueue(QUEUE_NAME);
}
#Bean
public Binding userServiceBinding() {
return BindingBuilder.bind(testRabbitServiceRabbitQueue()).to(commonConfig.directExchange()).with(ROUTE_KEY);
}
Rabbit common configuration class:
package com.xxx.yyy.rabbit.common;
import com.google.common.collect.ImmutableMap;
import org.springframework.amqp.core.AmqpAdmin;
import org.springframework.amqp.core.DirectExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.rabbit.connection.CachingConnectionFactory;
import org.springframework.amqp.rabbit.connection.ConnectionFactory;
import org.springframework.amqp.rabbit.core.RabbitAdmin;
import org.springframework.beans.factory.NoSuchBeanDefinitionException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.jmx.export.MBeanExporter;
#Configuration
#ComponentScan(basePackages = {
"com.xxx.yyy.rabbit",
"com.xxx.yyy.components",
"com.xxx.yyy.rabbit.common"
})
public class RabbitCommonConfig {
#Autowired
private ApplicationContext context;
#Value("${rabbit.host}")
private String rabbitHost;
#Value("${rabbit.password}")
private String rabbitPassword;
#Value("${rabbit.user}")
private String rabbitUserName;
#Value("${rabbit.port}")
private Integer rabbitPort = 5672;
#Value("${rabbit.marsFacade.exchange}")
private String exchangeName;
#Value("${rabbit.marsFacade.timeout}")
private Integer queueTimeout = 60000;
public Queue buildServiceQueue(String queueName) {
return new Queue(queueName, true, false, false, ImmutableMap.<String, Object>builder().put("x-message-ttl", queueTimeout).build());
}
#Bean
public ConnectionFactory rabbitConnectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(rabbitHost);
connectionFactory.setUsername(rabbitUserName);
connectionFactory.setPassword(rabbitPassword);
connectionFactory.setPort(rabbitPort);
try {
MBeanExporter mbeanExporter = context.getBean(MBeanExporter.class);
mbeanExporter.addExcludedBean("rabbitConnectionFactory");
} catch (NoSuchBeanDefinitionException ex) {
// No exporter. Exclusion is unnecessary
}
return connectionFactory;
}
#Bean
public AmqpAdmin amqpAdmin(ConnectionFactory rabbitConnectionFactory) {
return new RabbitAdmin(rabbitConnectionFactory);
}
#Bean
public DirectExchange directExchange() {
return new DirectExchange(exchangeName);
}
}
Service class:
#Service
public class TestRabbitServiceImpl implements TestRabbitService{
#Override
public void testRemoteMethod(String arg){
System.out.println(arg);
}
}
Configuration for listener:
#Configuration
#ComponentScan(basePackages = {"com.xxx.yyy.rabbit.common"})
#RequiredArgsConstructorpublic
class RabbitListenerConfiguration {
private final TestRabbitService testRebbitService;
#Value("${rabbit.marsFacade.concurrency}")
private Integer concurrency = 100;
#Bean
public SimpleMessageListenerContainer testRabbitServiceListenerContainer(AmqpProxyFactory amqpProxyFactory) {
return amqpProxyFactory.createListenerContainer(TestRabbitService.class, testRebbitService, TestRabbitServiceRabbitConfig.ROUTE_KEY, TestRabbitServiceRabbitConfig.QUEUE_NAME, concurrency);
}
}
Client side(module):
Rabbit common configuration class:
Configuration
#RequiredArgsConstructor
public class RabbitCommonConfig {
#Autowired
private ApplicationContext context;
#Value("${rabbit.host}")
private String rabbitHost;
#Value("${rabbit.password}")
private String rabbitPassword;
#Value("${rabbit.user}")
private String rabbitUserName;
#Value("${rabbit.port}")
private Integer rabbitPort = 5672;
#Value("${rabbit.marsFacade.exchange}")
private String exchangeName;
#Value("${rabbit.marsFacade.timeout}")
private Integer queueTimeout = 60000;
#Bean
public ConnectionFactory rabbitConnectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(rabbitHost);
connectionFactory.setUsername(rabbitUserName);
connectionFactory.setPassword(rabbitPassword);
connectionFactory.setPort(rabbitPort);
try {
MBeanExporter mbeanExporter = context.getBean(MBeanExporter.class);
mbeanExporter.addExcludedBean("rabbitConnectionFactory");
} catch (NoSuchBeanDefinitionException ex) {
// No exporter. Exclusion is unnecessary
}
return connectionFactory;
}
#Bean
public AmqpAdmin amqpAdmin(ConnectionFactory rabbitConnectionFactory) {
return new RabbitAdmin(rabbitConnectionFactory);
}
#Bean
public DirectExchange directExchange() {
return new DirectExchange(exchangeName);
}
#Bean(name = "clientSecurityContextMessageConverter")
public SecurityContextMessageConverter clientConverter() {
return new SecurityContextMessageConverter(true, false);
}
#Bean(name = "serverSecurityContextMessageConverter")
public SecurityContextMessageConverter serverConverter() {
return new SecurityContextMessageConverter(false, true);
}
}
The application property file has the same settings on the client and server sides:
rabbit:
marsFacade:
timeout: 60000
concurrency: 100
queueLength: 10000
exchange: com.xxx.yyy.remoting.marsFacade.exchange
host: localhost
port: 5672
user: guest
password: guest
When I try to call the remote method from the other module:
public class BaseController {
private final TestRabbitService testRabbitService;
#GetMapping
public String ping() {
testRabbitService.testRemoteMethod("hello");
return "Hello!";
}
Debug log trace on the server side when I start application:
DEBUG [core,,] 2802316 --- [ main] o.s.amqp.rabbit.core.RabbitAdmin : Binding destination [com.xxx.yyy.service.queue.testRabbitService (QUEUE)] to exchange [com.xxx.yyy.remoting.marsFacade.exchange] with routing key [com.xxx.yyy.service.testRabbitService]
...
DEBUG [core,,] 2802316 --- [enerContainer-1] o.s.a.r.listener.BlockingQueueConsumer : Started on queue 'com.xxx.yyy.service.queue.testRabbitService' with tag amq.ctag-nzefWR53naCaUNDBQZ1AWg: Consumer#71c6bf20: tags=[[amq.ctag-nzefWR53naCaUNDBQZ1AWg]], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,34), conn: Proxy#161722eb Shared Rabbit Connection: SimpleConnection#70ba6842 [delegate=amqp://guest#127.0.0.1:5672/, localPort= 37370], acknowledgeMode=AUTO local queue size=0
Debug log trace no the client side:
DEBUG [agent,,] 2804522 --- [on(4)-127.0.0.1] o.s.amqp.rabbit.core.RabbitAdmin : Binding destination [com.xxx.yyy.service.queue.testRabbitService (QUEUE)] to exchange [com.xxx.yyy.remoting.marsFacade.exchange] with routing key [com.xxx.yyy.service.testRabbitService]
When I try to call the method on the BaseBontroller, after the delay time i get "Listener threw exception" on the client side. And "No reply received from 'testRemoteMethod' with arguments '[hello]' - perhaps a timeout in the template?" on the server side.
Error stack on client:
2022-04-07 20:20:21.008 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.xxx.yyy.rest.front.controller.BaseController#ping()
2022-04-07T17:20:21 [http-nio-8080-exec-1] DEBUG o.s.w.s.m.m.a.RequestMappingHandlerMapping - Mapped to com.xxx.yyy.rest.front.controller.BaseController#ping()
2022-04-07 20:20:33.296 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] o.s.amqp.rabbit.core.RabbitTemplate : Executing callback RabbitTemplate$$Lambda$1623/0x00000008409c2040 on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,1), conn: Proxy#551ef755 Shared Rabbit Connection: SimpleConnection#6e552d5e [delegate=amqp://guest#127.0.0.1:5672/, localPort= 37372]
2022-04-07T17:20:33 [http-nio-8080-exec-1] DEBUG o.s.amqp.rabbit.core.RabbitTemplate - Executing callback RabbitTemplate$$Lambda$1623/0x00000008409c2040 on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,1), conn: Proxy#551ef755 Shared Rabbit Connection: SimpleConnection#6e552d5e [delegate=amqp://guest#127.0.0.1:5672/, localPort= 37372]
2022-04-07 20:20:33.334 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService
2022-04-07T17:20:33 [http-nio-8080-exec-1] DEBUG o.s.s.c.ThreadPoolTaskScheduler - Initializing ExecutorService
2022-04-07 20:20:33.334 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] .l.DirectReplyToMessageListenerContainer : Starting Rabbit listener container.
2022-04-07T17:20:33 [http-nio-8080-exec-1] DEBUG o.s.a.r.l.DirectReplyToMessageListenerContainer - Starting Rabbit listener container.
2022-04-07 20:20:33.346 INFO [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] .l.DirectReplyToMessageListenerContainer : Container initialized for queues: [amq.rabbitmq.reply-to]
2022-04-07T17:20:33 [http-nio-8080-exec-1] INFO o.s.a.r.l.DirectReplyToMessageListenerContainer - Container initialized for queues: [amq.rabbitmq.reply-to]
2022-04-07 20:20:33.429 INFO [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] .l.DirectReplyToMessageListenerContainer : SimpleConsumer [queue=amq.rabbitmq.reply-to, index=0, consumerTag=amq.ctag-eANKK31VIkaHjWZRT0NAVw identity=4372b241] started
2022-04-07T17:20:33 [http-nio-8080-exec-1] INFO o.s.a.r.l.DirectReplyToMessageListenerContainer - SimpleConsumer [queue=amq.rabbitmq.reply-to, index=0, consumerTag=amq.ctag-eANKK31VIkaHjWZRT0NAVw identity=4372b241] started
2022-04-07 20:20:33.431 DEBUG [agent,,] 2804522 --- [pool-1-thread-3] .l.DirectReplyToMessageListenerContainer : New SimpleConsumer [queue=amq.rabbitmq.reply-to, index=0, consumerTag=amq.ctag-eANKK31VIkaHjWZRT0NAVw identity=4372b241] consumeOk
2022-04-07T17:20:33 [pool-1-thread-3] DEBUG o.s.a.r.l.DirectReplyToMessageListenerContainer - New SimpleConsumer [queue=amq.rabbitmq.reply-to, index=0, consumerTag=amq.ctag-eANKK31VIkaHjWZRT0NAVw identity=4372b241] consumeOk
2022-04-07 20:20:33.442 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] o.s.amqp.rabbit.core.RabbitTemplate : Sending message with tag 1
2022-04-07T17:20:33 [http-nio-8080-exec-1] DEBUG o.s.amqp.rabbit.core.RabbitTemplate - Sending message with tag 1
2022-04-07 20:20:33.445 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] o.s.amqp.rabbit.core.RabbitTemplate : Publishing message [(Body:'[B#5d58fa73(byte[864])' MessageProperties [headers={}, correlationId=1, replyTo=amq.rabbitmq.reply-to, contentType=application/json, contentEncoding=UTF-8, contentLength=864, deliveryMode=PERSISTENT, priority=0, deliveryTag=0])] on exchange [com.xxx.yyy.remoting.marsFacade.exchange], routingKey = [com.xxx.yyy.service.testRabbitService]
2022-04-07T17:20:33 [http-nio-8080-exec-1] DEBUG o.s.amqp.rabbit.core.RabbitTemplate - Publishing message [(Body:'[B#5d58fa73(byte[864])' MessageProperties [headers={}, correlationId=1, replyTo=amq.rabbitmq.reply-to, contentType=application/json, contentEncoding=UTF-8, contentLength=864, deliveryMode=PERSISTENT, priority=0, deliveryTag=0])] on exchange [com.xxx.yyy.remoting.marsFacade.exchange], routingKey = [com.xxx.yyy.service.testRabbitService]
2022-04-07 20:21:33.452 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] o.s.amqp.rabbit.core.RabbitTemplate : Reply: null
2022-04-07T17:21:33 [http-nio-8080-exec-1] DEBUG o.s.amqp.rabbit.core.RabbitTemplate - Reply: null
2022-04-07 20:21:33.457 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] .m.m.a.ExceptionHandlerExceptionResolver : Using #ExceptionHandler com.xxx.yyy.rest.front.controller.RestExceptionHandler#handleServiceException(Exception, HttpServletResponse)
2022-04-07T17:21:33 [http-nio-8080-exec-1] DEBUG o.s.w.s.m.m.a.ExceptionHandlerExceptionResolver - Using #ExceptionHandler com.xxx.yyy.rest.front.controller.RestExceptionHandler#handleServiceException(Exception, HttpServletResponse)
2022-04-07 20:21:33.463 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] w.c.HttpSessionSecurityContextRepository : Did not store anonymous SecurityContext
2022-04-07T17:21:33 [http-nio-8080-exec-1] DEBUG o.s.s.w.c.HttpSessionSecurityContextRepository - Did not store anonymous SecurityContext
2022-04-07 20:21:33.464 DEBUG [agent,ed823cf4ed1fc76e,ed823cf4ed1fc76e] 2804522 --- [nio-8080-exec-1] .m.m.a.ExceptionHandlerExceptionResolver : Resolved [org.springframework.remoting.RemoteProxyFailureException: No reply received from 'testRemoteMethod' with arguments '[hello]' - perhaps a timeout in the template?]
2022-04-07T17:21:33 [http-nio-8080-exec-1] DEBUG o.s.w.s.m.m.a.ExceptionHandlerExceptionResolver - Resolved [org.springframework.remoting.RemoteProxyFailureException: No reply received from 'testRemoteMethod' with arguments '[hello]' - perhaps a timeout in the template?]
...
org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener threw exception
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.wrapToListenerExecutionFailedExceptionIfNeeded(AbstractMessageListenerContainer.java:1779)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1669)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1584)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1572)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1563)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1507)
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer$SimpleConsumer.callExecuteListener(DirectMessageListenerContainer.java:1107)
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer$SimpleConsumer.handleDelivery(DirectMessageListenerContainer.java:1067)
at com.rabbitmq.client.impl.ConsumerDispatcher$5.run(ConsumerDispatcher.java:149)
at com.rabbitmq.client.impl.ConsumerWorkService$WorkPoolRunnable.run(ConsumerWorkService.java:104)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: Reply received after timeout
at org.springframework.amqp.rabbit.core.RabbitTemplate.onMessage(RabbitTemplate.java:2659)
at org.springframework.amqp.rabbit.listener.DirectReplyToMessageListenerContainer.lambda$setMessageListener$0(DirectReplyToMessageListenerContainer.java:90)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1665)
... 11 common frames omitted
2022-04-07T17:21:33 [pool-1-thread-4] WARN o.s.a.r.l.ConditionalRejectingErrorHandler - Execution of Rabbit message listener failed.
org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener threw exception
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.wrapToListenerExecutionFailedExceptionIfNeeded(AbstractMessageListenerContainer.java:1779)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1669)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1584)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1572)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1563)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1507)
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer$SimpleConsumer.callExecuteListener(DirectMessageListenerContainer.java:1107)
at org.springframework.amqp.rabbit.listener.DirectMessageListenerContainer$SimpleConsumer.handleDelivery(DirectMessageListenerContainer.java:1067)
at com.rabbitmq.client.impl.ConsumerDispatcher$5.run(ConsumerDispatcher.java:149)
at com.rabbitmq.client.impl.ConsumerWorkService$WorkPoolRunnable.run(ConsumerWorkService.java:104)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: Reply received after timeout
at org.springframework.amqp.rabbit.core.RabbitTemplate.onMessage(RabbitTemplate.java:2659)
at org.springframework.amqp.rabbit.listener.DirectReplyToMessageListenerContainer.lambda$setMessageListener$0(DirectReplyToMessageListenerContainer.java:90)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1665)
... 11 common frames omitted
2022-04-07 20:21:33.631 ERROR [agent,,] 2804522 --- [pool-1-thread-4] .l.DirectReplyToMessageListenerContainer : Failed to invoke listener
Error stack on server:
022-04-07 20:21:33.617 DEBUG [core,,] 2802316 --- [enerContainer-1] o.s.amqp.rabbit.core.RabbitTemplate : Executing callback RabbitTemplate$$Lambda$1939/0x0000000840d30040 on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,36), conn: Proxy#161722eb Shared Rabbit Connection: SimpleConnection#70ba6842 [delegate=amqp://guest#127.0.0.1:5672/, localPort= 37370]
2022-04-07 20:21:33.618 DEBUG [core,,] 2802316 --- [enerContainer-1] o.s.amqp.rabbit.core.RabbitTemplate : Publishing message [(Body:'[B#6d9b27be(byte[6632])' MessageProperties [headers={}, correlationId=1, contentType=application/json, contentEncoding=UTF-8, contentLength=6632, deliveryMode=PERSISTENT, priority=0, deliveryTag=0])] on exchange [], routingKey = [amq.rabbitmq.reply-to.g1h2AA5yZXBseUAxNjEwNTI1MwAALWAAAAAEYkq0OA==.RB6LgnVMwPiBd8Hvvye0dg==]
2022-04-07 20:21:33.621 DEBUG [core,,] 2802316 --- [enerContainer-1] o.s.a.r.listener.BlockingQueueConsumer : Received message: (Body:'[B#144dbfd6(byte[833])' MessageProperties [headers={}, correlationId=1, replyTo=amq.rabbitmq.reply-to.g1h2AA5yZXBseUAxNjEwNTI1MwAAK8gAAAAEYkq0OA==.OJmY9VwG2ROnH9tN5z0Mvw==, contentType=application/json, contentEncoding=UTF-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=com.xxx.yyy.remoting.marsFacade.exchange, receivedRoutingKey=com.xxx.yyy.service.testRabbitService, deliveryTag=2, consumerTag=amq.ctag-nzefWR53naCaUNDBQZ1AWg, consumerQueue=com.xxx.yyy.service.queue.testRabbitService])
2022-04-07 20:21:33.625 DEBUG [core,,] 2802316 --- [enerContainer-1] o.s.amqp.rabbit.core.RabbitTemplate : Sending message with tag 2
2022-04-07 20:21:33.625 DEBUG [core,,] 2802316 --- [enerContainer-1] o.s.amqp.rabbit.core.RabbitTemplate : Publishing message [(Body:'[B#77a86b39(byte[833])' MessageProperties [headers={}, correlationId=2, replyTo=amq.rabbitmq.reply-to, contentType=application/json, contentEncoding=UTF-8, contentLength=833, deliveryMode=PERSISTENT, priority=0, deliveryTag=0])] on exchange [com.xxx.yyy.remoting.marsFacade.exchange], routingKey = [com.xxx.yyy.service.testRabbitService]
2022-04-07 20:21:33.627 DEBUG [core,,] 2802316 --- [pool-1-thread-8] o.s.a.r.listener.BlockingQueueConsumer : Storing delivery for consumerTag: 'amq.ctag-nzefWR53naCaUNDBQZ1AWg' with deliveryTag: '3' in Consumer#71c6bf20: tags=[[amq.ctag-nzefWR53naCaUNDBQZ1AWg]], channel=Cached Rabbit Channel: AMQChannel(amqp://guest#127.0.0.1:5672/,34), conn: Proxy#161722eb Shared Rabbit Connection: SimpleConnection#70ba6842 [delegate=amqp://guest#127.0.0.1:5672/, localPort= 37370], acknowledgeMode=AUTO local queue size=0
2022-04-07 20:22:33.626 DEBUG [core,,] 2802316 --- [enerContainer-1] o.s.amqp.rabbit.core.RabbitTemplate : Reply: null
2022-04-07 20:22:33.627 DEBUG [core,,] 2802316 --- [enerContainer-1] o.s.a.r.s.AmqpInvokerServiceExporter : Target method failed for RemoteInvocation: method name 'testRemoteMethod'; parameter types [java.lang.String]
org.springframework.remoting.RemoteProxyFailureException: No reply received from 'testRemoteMethod' with arguments '[hello]' - perhaps a timeout in the template?
at org.springframework.amqp.remoting.client.AmqpClientInterceptor.invoke(AmqpClientInterceptor.java:69) ~[spring-amqp-2.4.1.jar:2.4.1]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.14.jar:5.3.14]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.14.jar:5.3.14]
at com.sun.proxy.$Proxy260.testRemoteMethod(Unknown Source) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:na]
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:na]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Method.java:566) ~[na:na]
at org.springframework.remoting.support.RemoteInvocation.invoke(RemoteInvocation.java:215) ~[spring-context-5.3.14.jar:5.3.14]
at org.springframework.remoting.support.DefaultRemoteInvocationExecutor.invoke(DefaultRemoteInvocationExecutor.java:39) ~[spring-context-5.3.14.jar:5.3.14]
at org.springframework.remoting.support.RemoteInvocationBasedExporter.invoke(RemoteInvocationBasedExporter.java:78) ~[spring-context-5.3.14.jar:5.3.14]
at org.springframework.remoting.support.RemoteInvocationBasedExporter.invokeAndCreateResult(RemoteInvocationBasedExporter.java:114) ~[spring-context-5.3.14.jar:5.3.14]
at org.springframework.amqp.remoting.service.AmqpInvokerServiceExporter.onMessage(AmqpInvokerServiceExporter.java:81) ~[spring-amqp-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1721) ~[spring-rabbit-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1595) ~[spring-rabbit-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1572) ~[spring-rabbit-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1563) ~[spring-rabbit-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1507) ~[spring-rabbit-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:967) ~[spring-rabbit-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:914) ~[spring-rabbit-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:83) ~[spring-rabbit-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1291) ~[spring-rabbit-2.4.1.jar:2.4.1]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1197) ~[spring-rabbit-2.4.1.jar:2.4.1]
I have a Spring Cloud Stream application that receives messages from RabbitMQ using the Rabbit Binder, update my database and send one or many messages. My application can be summarized as this demo app:
The problem is that it doesn't seem that #Transactional works(or at least that's my impression) since if there's an exception the Database is rollbacked but messages are sent even the consumer/producer are configured by default as transacted.
Given that what I want to achieve is when an exception occurs I want the consumed messages go to DLQ after being retried the Database is rolled back and messages are not sent.
How can I achieve this?
This is the output of the demo application when I send a message my-input exchange
2021-01-19 14:31:20.804 ERROR 59593 --- [nput.my-group-1] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Exception thrown while invoking MyListener#process[1 args]; nested exception is java.lang.RuntimeException: MyError, failedMessage=GenericMessage [payload=byte[4], headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_receivedRoutingKey=#, amqp_receivedExchange=my-input, amqp_deliveryTag=2, deliveryAttempt=3, amqp_consumerQueue=my-input.my-group, amqp_redelivered=false, id=006f733f-5eab-9119-347a-625570383c47, amqp_consumerTag=amq.ctag-CnT_p-IXTJqIBNNG4sGPoQ, sourceData=(Body:'[B#177259f3(byte[4])' MessageProperties [headers={}, contentLength=0, receivedDeliveryMode=NON_PERSISTENT, redelivered=false, receivedExchange=my-input, receivedRoutingKey=#, deliveryTag=2, consumerTag=amq.ctag-CnT_p-IXTJqIBNNG4sGPoQ, consumerQueue=my-input.my-group]), contentType=application/json, timestamp=1611063077789}]
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:64)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:134)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:56)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:133)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:106)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:72)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:317)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:272)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:208)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter.access$1300(AmqpInboundChannelAdapter.java:66)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$Listener.lambda$onMessage$0(AmqpInboundChannelAdapter.java:308)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:225)
at org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$Listener.onMessage(AmqpInboundChannelAdapter.java:304)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1632)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1551)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1539)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1530)
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1474)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:967)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:913)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:83)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1288)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1194)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.RuntimeException: MyError
at com.example.demo.MyListener.process(DemoApplication.kt:46)
at com.example.demo.MyListener$$FastClassBySpringCGLIB$$4381219a.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:779)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:750)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:692)
at com.example.demo.MyListener$$EnhancerBySpringCGLIB$$f4ed3689.process(<generated>)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:55)
... 29 more
message should not be received here hello world
employee name still toto == toto
message should not be received here hello world
employee name still toto == toto
message should not be received here hello world
employee name still toto == toto
Since you are publishing the failed message to the DLQ, from a Rabbit perspective, the transaction was successful and the original message is acknowledged and removed from the queue, and the Rabbit transaction is committed.
You can't do what you want with republishToDlq.
It will work if you use the normal DLQ mechanism (republishToDlq=false, whereby the broker sends the original message to the DLQ) instead of republishing with the extra metadata.
If you want to republish with metadata, you could manually publish to the DLQ with a non-transactional RabbitTemplate (so the DLQ publish doesn't get rolled back with the other publishes).
EDIT
Here is an example of how to do what you need.
A few things to note:
We have to add an error handler to rethrow the exception.
We have to move retries to the listener container instead of the binder; otherwise, the retries will occur within the transaction and if retries are successful, multiple messages would be deposited on the output queue.
For stateful retry to work, we must be able to uniquely identify each message; the simplest solution is to have the sender set a unique message_id property (e.g. a UUID).
#SpringBootApplication
#EnableBinding(Processor.class)
public class So65792643Application {
public static void main(String[] args) {
SpringApplication.run(So65792643Application.class, args);
}
#Autowired
Processor processor;
#StreamListener(Processor.INPUT)
public void in(Message<String> in) {
System.out.println(in.getPayload());
processor.output().send(new GenericMessage<>(in.getPayload().toUpperCase()));
int attempt = RetrySynchronizationManager.getContext().getRetryCount();
if (in.getPayload().equals("okAfterRetry") && attempt == 1) {
System.out.println("success");
}
else {
throw new RuntimeException();
}
}
#Bean
RepublishMessageRecoverer repub(RabbitTemplate template) {
RepublishMessageRecoverer repub =
new RepublishMessageRecoverer(template, "DLX", "rk");
return repub;
}
#Bean
Queue dlq() {
return new Queue("my-output.dlq");
}
#Bean
DirectExchange dlx() {
return new DirectExchange("DLX");
}
#Bean
Binding dlqBinding() {
return BindingBuilder.bind(dlq()).to(dlx()).with("rk");
}
#ServiceActivator(inputChannel = "my-input.group1.errors")
void errorHandler(ErrorMessage message) {
MessagingException mex = (MessagingException) message.getPayload();
throw mex;
}
#RabbitListener(queues = "my-output.dlq")
void dlqListen(Message<String> in) {
System.out.println("DLQ:" + in);
}
#RabbitListener(queues = "my-output.group2")
void outListen(String in) {
if (in.equals("OKAFTERRETRY")) {
System.out.println(in);
}
else {
System.out.println("Should not see this:" + in);
}
}
/*
* We must move retries from the binder to stateful retries in the container so that
* each retry is rolled back, to avoid multiple publishes to output.
* See max-attempts: 1 in the yaml.
* In order for stateful retry to work, inbound messages must have a unique message_id
* property.
*/
#Bean
ListenerContainerCustomizer<AbstractMessageListenerContainer> customizer(RepublishMessageRecoverer repub) {
return (container, destinationName, group) -> {
if ("group1".equals(group)) {
container.setAdviceChain(RetryInterceptorBuilder.stateful()
.backOffOptions(1000, 2.0, 10000)
.maxAttempts(2)
.recoverer(recoverer(repub))
.keyGenerator(args -> {
// or generate a unique key some other way
return ((org.springframework.amqp.core.Message) args[1]).getMessageProperties()
.getMessageId();
})
.build());
}
};
}
private MethodInvocationRecoverer<?> recoverer(RepublishMessageRecoverer repub) {
return (args, cause) -> {
repub.recover(((ListenerExecutionFailedException) cause).getFailedMessage(), cause);
throw new AmqpRejectAndDontRequeueException(cause);
};
}
}
spring:
cloud:
stream:
rabbit:
default:
producer:
transacted: true
consumer:
transacted: true
requeue-rejected: true
bindings:
input:
destination: my-input
group: group1
consumer:
max-attempts: 1
output:
destination: my-output
producer:
required-groups: group2
okAfterRetry
2021-01-20 12:45:24.385 WARN 77477 --- [-input.group1-1] s.a.r.l.ConditionalRejectingErrorHandler : Execution of Rabbit message listener failed.
...
okAfterRetry
success
OKAFTERRETRY
notOkAfterRetry
2021-01-20 12:45:39.336 WARN 77477 --- [-input.group1-1] s.a.r.l.ConditionalRejectingErrorHandler : Execution of Rabbit message listener failed.
...
notOkAfterRetry
2021-01-20 12:45:39.339 WARN 77477 --- [-input.group1-1] s.a.r.l.ConditionalRejectingErrorHandler : Execution of Rabbit message listener failed.
...
DLQ:GenericMessage [payload=notOkAfterRetry, ..., x-exception-message...
I'm trying to connect a remote JBOSS, JBOSS EAP 7, JMS queue with spring boot.
Below my code, I have only a configuration class and consumer class.
I see through logs that I'm connecting to localhost, but... WHY???
#Configuration
#EnableJms
public class ActiveMqConnectionFactoryConfig {
String queueName= "JMSTestQueueName";
private static final String INITIAL_CONTEXT_FACTORY = "org.jboss.naming.remote.client.InitialContextFactory";
private static final String CONNECTION_FACTORY = "jms/RemoteConnectionFactory";
public ConnectionFactory connectionFactory() {
try {
System.out.println("Retrieving JMS queue with JNDI name: " + CONNECTION_FACTORY);
JndiObjectFactoryBean jndiObjectFactoryBean = new JndiObjectFactoryBean();
jndiObjectFactoryBean.setJndiName(CONNECTION_FACTORY);
jndiObjectFactoryBean.setJndiEnvironment(getEnvProperties());
jndiObjectFactoryBean.afterPropertiesSet();
return (QueueConnectionFactory) jndiObjectFactoryBean.getObject();
} catch (NamingException e) {
System.out.println("Error while retrieving JMS queue with JNDI name: [" + CONNECTION_FACTORY + "]");
} catch (Exception ex) {
System.out.println("Error error");
ex.getStackTrace();
}
return null;
}
#Bean
Properties getEnvProperties() throws NamingException {
Properties env = new Properties();
env.put(Context.INITIAL_CONTEXT_FACTORY, INITIAL_CONTEXT_FACTORY);
env.put(Context.PROVIDER_URL, "http-remoting://<REMOTE_ADDRESS>:8080?broker.persistent=false&broker.useJmx=false");
env.put(Context.SECURITY_PRINCIPAL, <USER>);
env.put(Context.SECURITY_CREDENTIALS, <PASSWORD>);
namingContext = new InitialContext(env);
return env;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory) throws NamingException {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setConcurrency("3-10");
JndiDestinationResolver jndiDestinationResolver = new JndiDestinationResolver();
jndiDestinationResolver.setJndiEnvironment(getEnvProperties());
factory.setDestinationResolver(jndiDestinationResolver);
return factory;
}
}
Listener Class:
#Configuration
public class jmsListener {
#JmsListener(destination = "queue/JMSTestQueueName", containerFactory = "jmsListenerContainerFactory")
public void receive(Message message) {
System.out.println("Received Message: " + message);
}
}
The log:
2020-05-27 00:21:58.571 INFO 10368 --- [ main] o.apache.activemq.broker.BrokerService : Apache ActiveMQ 5.15.7 (localhost, ID:ITPC020248-60530-1590531718443-0:1) is starting
2020-05-27 00:21:58.576 INFO 10368 --- [ main] o.apache.activemq.broker.BrokerService : Apache ActiveMQ 5.15.7 (localhost, ID:ITPC020248-60530-1590531718443-0:1) started
2020-05-27 00:21:58.577 INFO 10368 --- [ main] o.apache.activemq.broker.BrokerService : For help or more information please see: http://activemq.apache.org
2020-05-27 00:21:58.617 INFO 10368 --- [ main] o.a.activemq.broker.TransportConnector : Connector vm://localhost started
2020-05-27 00:21:58.659 INFO 10368 --- [ main] jboss.ConsumerClass : Started ConsumerClass in 1.922 seconds (JVM running for 3.504)
Without ?broker.persistent=false&broker.useJmx=false on the JNDI URL I obtain this:
2020-05-27 19:38:34.680 INFO 19752 --- [dpoint" task-13] org.jboss.ejb.client.remoting : EJBCLIENT000016: Channel Channel ID 8f759d73 (outbound) of Remoting connection 336369ee to /172.16.68.80:8080 can no longer process messages
2020-05-27 19:38:37.479 INFO 19752 --- [ main] jboss.ConsumerClass : Started ConsumerClass in 7.194 seconds (JVM running for 9.26)
2020-05-27 19:38:42.772 ERROR 19752 --- [enerContainer-2] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'queue/JMSTestQueueName' - retrying using FixedBackOff{interval=5000, currentAttempts=0, maxAttempts=unlimited}. Cause: AMQ119031: Unable to validate user
2020-05-27 19:38:48.068 ERROR 19752 --- [enerContainer-2] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'queue/JMSTestQueueName' - retrying using FixedBackOff{interval=5000, currentAttempts=1, maxAttempts=unlimited}. Cause: AMQ119031: Unable to validate user
2020-05-27 19:38:53.401 ERROR 19752 --- [enerContainer-2] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'queue/JMSTestQueueName' - retrying using FixedBackOff{interval=5000, currentAttempts=2, maxAttempts=unlimited}. Cause: AMQ119031: Unable to validate user
2020-05-27 19:38:58.699 ERROR 19752 --- [enerContainer-2] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'queue/JMSTestQueueName' - retrying using FixedBackOff{interval=5000, currentAttempts=3, maxAttempts=unlimited}. Cause: AMQ119031: Unable to validate user
2020-05-27 19:39:04.006 ERROR 19752 --- [enerContainer-2] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'queue/JMSTestQueueName' - retrying using FixedBackOff{interval=5000, currentAttempts=4, maxAttempts=unlimited}. Cause: AMQ119031: Unable to validate user
2020-05-27 19:39:09.320 ERROR 19752 --- [enerContainer-2] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'queue/JMSTestQueueName' - retrying using FixedBackOff{interval=5000, currentAttempts=5, maxAttempts=unlimited}. Cause: AMQ119031: Unable to validate user
Your connectionFactory() needs to be a #Bean in order to inject it as an argument into your container factory factory method.
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(ConnectionFactory connectionFactory) throws NamingException {
Since it's not a #Bean you are getting boot's default connection factory (presumably you have ActiveMQ on the class path).
I have a simple Spring Boot application(Spring Boot Version 1.5.3.RELEASE) for consuming JMS Messages off an ActiveMQ(version 5.14.5) Queue.
I want the messages to be consumed in a JMS transaction. If there is an exception during message consumption, I expect transaction to be rolled back and message not to be dequeued(taken off message queue). I can see transaction being rolled back in Spring logs, however the message is still dequeued from ActiveMQ queue(after six re delivery attempts).
Any pointers will be appreciated.
Here is the application code:
#SpringBootApplication
public class SpringJmsDemoApplication {
public static void main(String[] args) {
SpringApplication.run(SpringJmsDemoApplication.class, args);
}
#Bean
public JmsListenerContainerFactory<?> myFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory defaultJmsListenerContainerFactory = new DefaultJmsListenerContainerFactory();
defaultJmsListenerContainerFactory.setTransactionManager(jmsTransactionManager(connectionFactory));
defaultJmsListenerContainerFactory.setSessionTransacted(true);
defaultJmsListenerContainerFactory.setSessionAcknowledgeMode(Session.SESSION_TRANSACTED);
configurer.configure(defaultJmsListenerContainerFactory, connectionFactory);
return defaultJmsListenerContainerFactory;
}
#Bean
public PlatformTransactionManager jmsTransactionManager(ConnectionFactory connectionFactory) {
return new JmsTransactionManager(connectionFactory);
}
}
#Component
public class Receiver {
#JmsListener(destination = "mailbox", containerFactory = "myFactory")
#Transactional
public void receiveMessage(String email) {
System.out.println("Received <" + email + ">");
throw new RuntimeException("nooo");
}
}
Here is the log:
2017-05-24 09:51:59.865 DEBUG 8972 --- [DefaultMessageListenerContainer-1] o.s.j.connection.JmsTransactionManager : Created JMS transaction on Session [ActiveMQSession {id=ID:D6C0B8467A518-58248-1495590693980-1:32:1,started=false} java.lang.Object#65d647cd] from Connection [ActiveMQConnection
2017-05-24 09:51:59.867 DEBUG 8972 --- [DefaultMessageListenerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Received message of type [class org.apache.activemq.command.ActiveMQTextMessage] from consumer [ActiveMQMessageConsumer { value=ID:D6C0B8467A518-58248-1495590693980-1:32:1:1, started=true }] of transactional session [ActiveMQSession {id=ID:D6C0B8467A518-58248-1495590693980-1:32:1,started=true} java.lang.Object#65d647cd]
2017-05-24 09:51:59.867 DEBUG 8972 --- [DefaultMessageListenerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Rolling back transaction because of listener exception thrown: org.springframework.jms.listener.adapter.ListenerExecutionFailedException: Listener method 'public void com.anz.markets.springjmsdemo.Receiver.receiveMessage(java.lang.String)' threw exception; nested exception is java.lang.RuntimeException: nooo
2017-05-24 09:51:59.867 WARN 8972 --- [DefaultMessageListenerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Execution of JMS message listener failed, and no ErrorHandler has been set.
org.springframework.jms.listener.adapter.ListenerExecutionFailedException: Listener method 'public void com.anz.markets.springjmsdemo.Receiver.receiveMessage(java.lang.String)' threw exception; nested exception is java.lang.RuntimeException: nooo
at org.springframework.jms.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:112) ~[spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:69) ~[spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:721) ~[spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:681) ~[spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:651) ~[spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317) [spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:235) [spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166) [spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158) [spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055) [spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]
Caused by: java.lang.RuntimeException: nooo
at com.anz.markets.springjmsdemo.Receiver.receiveMessage(Receiver.java:12) ~[classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_72]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_72]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_72]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:180) ~[spring-messaging-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:112) ~[spring-messaging-4.3.8.RELEASE.jar:4.3.8.RELEASE]
at org.springframework.jms.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:104) ~[spring-jms-4.3.8.RELEASE.jar:4.3.8.RELEASE]
... 10 common frames omitted
2017-05-24 09:51:59.868 DEBUG 8972 --- [DefaultMessageListenerContainer-1] o.s.j.connection.JmsTransactionManager : Transactional code has requested rollback
2017-05-24 09:51:59.868 DEBUG 8972 --- [DefaultMessageListenerContainer-1] o.s.j.connection.JmsTransactionManager : Initiating transaction rollback
2017-05-24 09:51:59.868 DEBUG 8972 --- [DefaultMessageListenerContainer-1] o.s.j.connection.JmsTransactionManager : Rolling back JMS transaction on Session [ActiveMQSession {id=ID:D6C0B8467A518-58248-1495590693980-1:32:1,started=true} java.lang.Object#65d647cd]
According to ActiveMQ message re-delivery documentation, the messages, which failed to be delivered, will go to the dead letter queue (http://activemq.apache.org/message-redelivery-and-dlq-handling.html):
"The default Dead Letter Queue in ActiveMQ is called ActiveMQ.DLQ; all un-deliverable messages will get sent to this queue and this can be difficult to manage. So, you can set an individualDeadLetterStrategy in the destination policy map of the activemq.xml configuration file, which allows you to specify a specific dead letter queue prefix for a given queue or topic. You can apply this strategy using wild card if you like so that all queues get their own dead-letter queue, as is shown in the example below"
Please, extend you activemq.xml with individualDeadLetterStrategy to the queue.