When try to connect Rabbit MQ instance running on seperate host/port , getting this exception that is more related to restarting of consumer and getting timed out
2020-08-22 10:23:25.189 INFO [events-to-slack,,,] 1057 --- [an)#6c74acb-100] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer: tags=[{}], channel=null, acknowledgeMode=AUTO local queue size=0
2020-08-22 10:23:25.203 ERROR [events-to-slack,,,] 1057 --- [an)#6c74acb-110] o.s.a.r.l.SimpleMessageListenerContainer : Failed to check/redeclare auto-delete queue(s).
org.springframework.amqp.AmqpIOException: java.net.SocketTimeoutException: connect timed out
at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:306)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:547)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1389)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1370)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1346)
at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueProperties(RabbitAdmin.java:336)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.redeclareElementsIfNecessary(SimpleMessageListenerContainer.java:1128)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$800(SimpleMessageListenerContainer.java:99)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1304)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: connect timed out
I'm using spring boot 1.5.18. with Java 1.8
These are my configurations setup for rabbit-mq , here we can see that it's simply using topic exchange and tying to setup a listener with Queue bind with it
#RequiredArgsConstructor
#Configuration
public class RabbitMQConfiguration implements RabbitListenerConfigurer {
private final Config configuration;
private final RetryConfiguration retryConfig;
#Bean
public TopicExchange exchange() {
return new TopicExchange(configuration.getExchange());
}
#Bean
public TopicExchange delayExchange() {
TopicExchange topicExchange = new TopicExchange(configuration.getDelayExchange());
topicExchange.setDelayed(true);
return topicExchange;
}
#Bean
public Queue queue() {
return new Queue(configuration.getQueue());
}
#Bean
public List<Binding> bindings(#Qualifier("exchange") TopicExchange exchange,
#Qualifier("delayExchange") TopicExchange delayExchange,
#Qualifier("queue") Queue queue) {
List<Binding> binding = new ArrayList<>();
for (String routingKey : configuration.getRoutingKeys()) {
binding.add(BindingBuilder.bind(queue).to(exchange).with(routingKey));
binding.add(BindingBuilder.bind(queue)
.to(delayExchange)
.with(retryConfig.getPrefix() + routingKey));
}
return binding;
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
#Bean
public DefaultMessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(consumerJackson2MessageConverter());
return factory;
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar endpointRegistrar) {
endpointRegistrar.setMessageHandlerMethodFactory(messageHandlerMethodFactory());
}
Related
In my Spring Boot project I'm using RabbitMQ for messaging and have two exceptions CustomExceptionA and CustomExceptionB. I want my CustomExceptionA to be retried n times and CustomExceptionB not to retry but directly sent to DLQ.
Below are my configurations:-
yaml file
spring:
rabbitmq:
listener:
simple:
default-requeue-rejected: false
retry:
enabled: true
initial-interval: 2s
max-attempts: 3
max-interval: 2s
multiplier: 1
Configuration file
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory,
SimpleRabbitListenerContainerFactoryConfigurer configurer) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setErrorHandler(errorHandler());
return factory;
}
#Bean
public ErrorHandler errorHandler() {
return new ConditionalRejectingErrorHandler(customExceptionStrategy());
}
#Bean
FatalExceptionStrategy customExceptionStrategy() {
return new CustomFatalExceptionStrategy();
}
-----------------------------------------------------------------------------------------------------------------------------
#Component
public class CustomFatalExceptionStrategy extends ConditionalRejectingErrorHandler.DefaultExceptionStrategy {
#Override
public boolean isFatal(Throwable throwable) {
return (throwable.getCause() instanceof CustomExceptionB);
}
}
According to the blog : https://www.baeldung.com/spring-amqp-error-handling mechanism should work but for some reason it is not working for me.
Someone please look at the issue.
I have an application that should work with rabbitmq. I have RabbitMQConfig, which tries to connect with the rabbit, but if it fails to connect, the application does not start at all.
What I want to achieve is that after launching the application, it will try to connect to the rabbit and if it manages to connect, I will start functionality to create a queue and listen to it accordingly. Currently at startup if the rabbit is available and then disappears it starts throwing "java.net.ConnectException: Connection refused". Can I catch this error and how, both at startup and during the operation of the application.
This is config file:
public class RabbitMQConfig implements RabbitListenerConfigurer {
private ConnectionFactory connectionFactory;
#Autowired
public RabbitMQConfig(ConnectionFactory connectionFactory) {
this.connectionFactory = connectionFactory;
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(messageHandlerMethodFactory());
}
#Bean
MessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory messageHandlerMethodFactory = new DefaultMessageHandlerMethodFactory();
messageHandlerMethodFactory.setMessageConverter(consumerJackson2MessageConverter());
return messageHandlerMethodFactory;
}
#Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
#Bean
public RabbitTemplate rabbitTemplate() {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(new Jackson2JsonMessageConverter());
return rabbitTemplate;
}
#Bean
public AmqpAdmin amqpAdmin() {
return new RabbitAdmin(connectionFactory);
}
#Bean(name = "rabbitListenerContainerFactory")
public SimpleRabbitListenerContainerFactory simpleRabbitListenerContainerFactory(
SimpleRabbitListenerContainerFactoryConfigurer configurer,
ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
FixedBackOff recoveryBackOff = new FixedBackOff(10000,FixedBackOff.UNLIMITED_ATTEMPTS);
factory.setRecoveryBackOff(recoveryBackOff);
configurer.configure(factory, connectionFactory);
return factory;
}
Whit this methods i create and start listener:
#Service
public class RabbitMQService {
#RabbitListener(queues = "${queueName}", autoStartup = "false", id = "commandQueue")
public void receive(CommandDataDTO commandDataDTO) {
public void receive(Object rMessage) {
doSomething(rMessage);
}
public void createQueue() {
Queue queue = new Queue("queueName"), true, false, false);
Binding binding = new Binding("queueName"),
Binding.DestinationType.QUEUE, env.getProperty("spring.rabbitmq.exchange"),"rKey",
null);
admin.declareQueue(queue);
admin.declareBinding(binding);
startListener();
}
//When queue is active we start Listener
public void startListener() {
boolean isQueuqReady = false;
while (!isQueuqReady) {
Properties p = admin.getQueueProperties(env.getProperty("management.registry.info.device-type") + "_"
+ env.getProperty("management.registry.info.device-specific-type"));
if (p != null) {
log.info("Rabbit queue is up. Start listener.");
isQueuqReady = true;
registry.getListenerContainer("commandQueue").start();
}
}
}
Problem is that I want, regardless of whether there is a connection or not with the rabbit, the application to be able to work and, accordingly, to intercept when there is a connection and when not to do different actions.
I have a factory class that I use to create a mq consumer and a DefaultMessageListenerContainer bean from that consumer that will be used for consuming messages from a topic. It goes like this -
public class MQMessageFactory {
public static DemoMessageConsumer createMessageConumer(Map<String, String> queueDetails, Map<String, String> sslDetails) throws Exception {
MQConnectionFactory mqConnectionFactory = createMQConnectionFactory(queueDetails, sslDetails);
return new DemoMessageConsumer(queueDetails, mqConnectionFactory);
}
public static MQConnectionFactory createMQConnectionFactory(Map<String, String> queue, Map<String, String> sslDetails) throws Exception {
MQConnectionFactory cf = new MQConnectionFactory();
try {
cf.setHostName(queue.get("hostname"));
cf.setPort(queue.get("port"));
cf.setQueueManager(queue.get("queueManager"));
cf.setChannel(queue.get("channel"));
cf.setTransportType(WMQConstants.WMQ_CM_CLIENT);
cf.setStringProperty(WMQConstants.USERID, queue.get("username"));
cf.setStringProperty(WMQConstants.PASSWORD, queue.get("password"));
cf.setSSLCipherSuite(queue.get("sslCipherSuite"));
cf.setSSLSocketFactory(someMethodToCreateSSLContextFactory(sslDetails));
return cf;
} catch (JMSException e) {
throw new RuntimeException("Unable to establish connection with host: " + queue.get("hostname"), e);
}
}
}
public class DemoMessageConsumer implements SessionAwareMessageListener {
private static final Logger LOGGER = LogManager.getLogger(DemoMessageConsumer.class);
private SingleConnectionFactory singleCf;
private Map<String, String> properties;
private DefaultMessageListenerContainer container;
private Consumer<Message> messageProcessor;
public DemoMessageConsumer(Map<String, String> properties, MQConnectionFactory connectionFactory) {
this.singleCf = new SingleConnectionFactory(connectionFactory);
this.singleCf.setReconnectOnException(true);
this.singleCf.afterPropertiesSet();
this.properties = properties;
}
public DefaultMessageListenerContainer listen(String queueName, Executor taskExecutor, Consumer<Message> messageProcessor) {
this.messageProcessor = messageProcessor;
this.container = new DefaultMessageListenerContainer();
this.container.setConnectionFactory(singleCf);
this.container.setDestinationName(queueName);
// this.container.setAcceptMessagesWhileStopping(true);
this.container.setSessionTransacted(true);
this.container.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
this.container.setMessageListener(this);
this.container.setConcurrentConsumers(5);
this.container.setTaskExecutor(taskExecutor);
this.container.afterPropertiesSet();
this.container.start();
LOGGER.info("Consumer started");
return this.container;
}
#Override
public void onMessage(Message message, Session session) {
try {
LOGGER.info("Message received with MessageID: {}", message.getJMSMessageID());
this.messageProcessor.accept(message);
} catch (JMSException e) {
LOGGER.error("Error while processing the message", e);
}
}
public void triggerShutdown() {
LOGGER.info("Shutdown called");
this.container.stop();
while (this.container.isRunning()) ;
this.container.shutdown();
while (this.container.isActive()) ;
this.singleCf.destroy();
LOGGER.info("Listener is shutdown");
}
}
Further, I have a Spring boot project class where I actually create the beans and use them to listen to the queue -
#Getter
#Setter
#Configuration
#ConfigurationProperties(prefix = "ibmmq")
public class MqConsumerImpl implements ApplicationListener<ContextClosedEvent> {
private static final Logger LOGGER = LogManager.getLogger(MqConsumerImpl.class);
public Map<String, String> ssl;
public Map<String, String> queue;
#Lazy
#Autowired
#Qualifier("mqConsumer")
private DemoMessageConsumer consumer;
#Bean("mqConsumer")
public DemoMessageConsumer createConsumer() throws Exception {
return MQMessageFactory.createMessageConumer(queue, ssl);
}
#Bean("mqListener")
public DefaultMessageListenerContainer listen() {
return this.consumer.listen(queue.get("name"), Executors.newFixedThreadPool(3), message -> {
try {
LOGGER.info("{} Message reading started: {} ", Thread.currentThread().getName(), message.getBody(String.class));
// My business logic goes here
Thread.sleep(1000);
LOGGER.info("{} Message reading completed: {} ", Thread.currentThread().getName(), message.getBody(String.class));
} catch (Exception e) {
LOGGER.error(e);
}
});
}
#Override
public void onApplicationEvent(ContextClosedEvent e) {
this.consumer.triggerShutdown();
}
}
Now, I run the application and the messages are being consumer properly and everything goes well. I have a taskExecutor of three threads and all of them are being used for message consumption and task execution.
Then, I trigger a command to stop the application and one or all of the threads on which the business task were running MAY/MAY NOT throw this warning -
2020-01-14 16:29:15.110 WARN 68468 --- [pool-2-thread-3] o.s.j.l.DefaultMessageListenerContainer : Rejecting received message because of the listener container having been stopped in the meantime:
JMSMessage class: jms_text
JMSType: null
JMSDeliveryMode: 2
JMSDeliveryDelay: 0
JMSDeliveryTime: 0
JMSExpiration: 0
JMSPriority: 4
JMSMessageID: ID:414d5120484b49473033533120202020a2e4f05dce410b24
JMSTimestamp: 1578982170556
JMSCorrelationID: null
JMSDestination: queue:///QUEUE.NAME
JMSReplyTo: null
JMSRedelivered: false
JMSXAppID: demo.Application
JMSXDeliveryCount: 1
JMSXUserID: username
JMS_IBM_Character_Set: UTF-8
JMS_IBM_Encoding: 273
JMS_IBM_Format: MQSTR
JMS_IBM_MsgType: 8
JMS_IBM_PutApplType: 28
JMS_IBM_PutDate: 20200114
JMS_IBM_PutTime: 06093062
hello everyone new 1578982170555
2020-01-14 16:29:15.129 INFO 68468 --- [pool-2-thread-2] c.s.l.i.c.MqConsumerImpl : pool-2-thread-2 Message reading completed: hello everyone new 1578982170344
2020-01-14 16:29:16.180 INFO 68468 --- [ Thread-9] c.s.l.n.c.MqConsumerImpl : Listener for queue: XXXXXXXXX is shutdown
Process finished with exit code 1
Now, It is completely fine for me to see this. According to spring boot jms classes, this happens and the session rollback should be called in this case, so that when I restart my consumer, the message is redelivered. THIS IS NOT HAPPENING. I am not getting the message - hello everyone new 1578982170555 the next time I started the consumer and I got the next one. A message is hence lost without being processed. How can I safeguard it?
Note: As you can see in the logs, when this warning was raised the onMessage() method for this method was not called.
So I read all the examples, I have the com.rabbitmq.client.Channel and the #Header(AmqpHeaders.DELIVERY_TAG), but when I try to call Channel.basicNack(long deliveryTag, boolean multiple, boolean requeue) the result is "java.lang.IllegalStateException: Channel closed; cannot ack/nack". I can see that the CachingConnectionFactory does not support any of the acknowledge methods. So my question is what ConnectionFactory do I have to use and howto configure it, so that basicAck/basicNack works?
Spring Boot Version 2.1.0.RELEASE
application.yaml:
spring:
rabbitmq:
host: ${RABBITMQ_HOST:localhost}
port: ${RABBITMQ_PORT:5672}
username: ${RABBITMQ_USERNAME:guest}
password: ${RABBITMQ_PASSWORD:guest}
listener:
type: simple
simple:
acknowledge-mode: manual
Config class:
#EnableRabbit
#Configuration
public class RabbitMqConfig implements RabbitListenerConfigurer {
#Value("${app.rabbitmq.incoming-queue}")
private String incomingQueue;
private AmqpAdmin amqpAdmin;
#Autowired
public RabbitMqConfig(AmqpAdmin amqpAdmin) {
this.amqpAdmin = amqpAdmin;
}
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setChannelTransacted(true);
rabbitTemplate.setMessageConverter(jsonMessageConverter());
return rabbitTemplate;
}
#Bean
MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registar) {
registar.setMessageHandlerMethodFactory(createDefaultMessageHandlerMethodFactory());
}
#Bean
public DefaultMessageHandlerMethodFactory createDefaultMessageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(new MappingJackson2MessageConverter());
return factory;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(
ConnectionFactory connectionFactory, CommonAmqpErrorHandler commonAmqpErrorHandler) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setMessageConverter(jsonMessageConverter());
factory.setErrorHandler(commonAmqpErrorHandler);
return factory;
}
#PostConstruct
public void afterInit() {
amqpAdmin.declareQueue(new Queue(getDeadLetterQueueName(incomingQueue), true));
amqpAdmin.declareQueue(
QueueBuilder.durable(incomingQueue).withArgument("x-dead-letter-exchange", "")
.withArgument("x-dead-letter-routing-key",
getDeadLetterQueueName(incomingQueue)).build());
}
private String getDeadLetterQueueName(String queueName) {
return queueName + ".dead-letter.queue";
}
}
Listener code:
#Transactional(rollbackOn = Exception.class)
#RabbitListener(queues = "${app.rabbitmq.incoming-queue}", errorHandler = "notificationListenerErrorHandler")
public void onMessage(#Valid #Payload NotificationDto notification, Message message,
Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) long tag) throws IOException {
System.out.println("00000 > " + tag);
System.out.println("11111");
channel.basicNack(tag, false, true);
System.out.println("222222");
}
After starting it from scratch, it turns out that the
#Transactional(rollbackOn = Exception.class)
is causing the problem. If I remove it, it's working
In my project i am having 3 Classes App(Publisher), PojoListener(Receiver) , Config(bind both App and PojoListner). But i need to separate out the PojoListener from my project and deploy somewhere else so that my Listener class will continuous listen to the Rabbit Mq and ack the messages back to queue and then to App Class for any messages published by my App class.
As my config file is common for both Publisher and Receiver. Is there any way to seperate out them. I need to deploy my receiver on different server and Publisher on different. Both will listen to common queues.
**App.java** :
public class App {
public static void main(String[] args) throws UnsupportedEncodingException {
ApplicationContext context = new AnnotationConfigApplicationContext(RabbitMQConfig.class);
RabbitTemplate rabbitTemplate = context.getBean(RabbitTemplate.class);
String response = (String) rabbitTemplate.convertSendAndReceive("message from publisher");
System.out.println("message sent");
System.out.println("response from :" + response);
}
}
PojoListener.java :
public class PojoListener {
public String handleMessage(String msg) {
System.out.println("IN POJO RECEIVER!!!");
return msg.toUpperCase();
}
}
RabbitMQConfig.java :
#Configuration
public class RabbitMQConfig {
#Bean
public ConnectionFactory rabbitConnectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setHost("localhost");
connectionFactory.setUsername("guest");
connectionFactory.setPassword("guest");
return connectionFactory;
}
#Bean
public RabbitTemplate fixedReplyQRabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(rabbitConnectionFactory());
template.setExchange(ex().getName());
template.setRoutingKey("test");
template.setReplyQueue(replyQueue());
template.setReplyTimeout(60000);
return template;
}
#Bean
public SimpleMessageListenerContainer replyListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueues(replyQueue());
container.setMessageListener(fixedReplyQRabbitTemplate());
return container;
}
#Bean
public SimpleMessageListenerContainer serviceListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory());
container.setQueues(requestQueue());
container.setMessageListener(new MessageListenerAdapter(new PojoListener()));
return container;
}
#Bean
public DirectExchange ex() {
return new DirectExchange("rabbit-exchange", false, true);
}
#Bean
public Binding binding() {
return BindingBuilder.bind(requestQueue()).to(ex()).with("test");
}
#Bean
public Queue requestQueue() {
return new Queue("request-queue-spring");
}
#Bean
public Queue replyQueue() {
return new Queue("response-queue-spring");
}
#Bean
public RabbitAdmin admin() {
return new RabbitAdmin(rabbitConnectionFactory());
}
}