spring cloud rabbit configuration tries to connect to localhost 5672 - spring-boot

Question

There are tons of examples where java code is being used to configure pure rabbit, but spring boot related documentation is another story.
My requirement is pretty basic - Single queue, Single exchange, One dead-letter queue with predefined names. Some HA policy. Deployed to pivotal cloud-foundry.
The app should use cloud connection factory, so I don't have to specify connection info.
The app should create queue/exchange if they don't exist.
I need a controller that can use rabbitTemplate to send msgs.
I need a listener to process those messages, in multiple consumer threads.
I have tried to do properties-only approach. And I have tried tons of beans to do this. For some reason it doesn't want to connect to cloud service!
I am using spring boot app and running it in pivotal cloud foundry
Code
#Configuration
#EnableRabbit
#Profile("cloud")
#Slf4j
public class RabbitMQConfig extends AbstractCloudConfig {
final static String MAIN_QUEUE_NAME = "pricing";
private final static String QUEUE_EXCHANGE_NAME = "pricing-exchange";
private final static String DEAD_LETTER_QUEUE_NAME = "pricing-dl";
#Autowired
RabbitProperties rabbitProperties;
#Bean("pricingQueue")
#Primary
public Queue pricingQueue() {
return QueueBuilder.durable(MAIN_QUEUE_NAME)
.withArgument("x-dead-letter-exchange", "")
.withArgument("x-dead-letter-routing-key", DEAD_LETTER_QUEUE_NAME)
.withArgument("x-queue-master-locator", "min-masters")
.build();
}
#Bean
public Queue deadLetterQueue() {
return QueueBuilder.durable(DEAD_LETTER_QUEUE_NAME)
.withArgument("x-queue-master-locator", "min-masters")
.build();
}
#Bean
public DirectExchange exchange() {
return new DirectExchange(QUEUE_EXCHANGE_NAME);
}
#Bean
public Binding pricingBinding(#Qualifier("pricingQueue") Queue pricingQueue,
DirectExchange exchange) {
return BindingBuilder.bind(pricingQueue).to(exchange).with(MAIN_QUEUE_NAME);
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
I think this is happening because spring boot created the listener container and not me
Error
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'org.springframework.boot.actuate.autoconfigure.health.HealthIndicatorAutoConfiguration'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'management.health.status-org.springframework.boot.actuate.autoconfigure.health.HealthIndicatorProperties'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'org.springframework.boot.actuate.autoconfigure.health.HealthIndicatorAutoConfiguration' via constructor to bean named 'management.health.status-org.springframework.boot.actuate.autoconfigure.health.HealthIndicatorProperties'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'healthIndicatorRegistry'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'healthIndicatorRegistry' via factory method to bean named 'org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext#6483f5ae'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'rabbitHealthIndicator'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'org.springframework.boot.actuate.autoconfigure.amqp.RabbitHealthIndicatorAutoConfiguration'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'rabbitTemplate'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration$RabbitTemplateConfiguration'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'spring.rabbitmq-org.springframework.boot.autoconfigure.amqp.RabbitProperties'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration$RabbitTemplateConfiguration' via constructor to bean named 'spring.rabbitmq-org.springframework.boot.autoconfigure.amqp.RabbitProperties'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'rabbitConnectionFactory'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration$RabbitConnectionFactoryCreator'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'rabbitConnectionFactory' via factory method to bean named 'spring.rabbitmq-org.springframework.boot.autoconfigure.amqp.RabbitProperties'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'rabbitTemplate' via factory method to bean named 'rabbitConnectionFactory'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'jsonMessageConverter'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'rabbitMQConfig'
...
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'viewResolver' via factory method to bean named 'org.springframework.beans.factory.support.DefaultListableBeanFactory#1dde4cb2'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'amqpAdmin'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'amqpAdmin' via factory method to bean named 'rabbitConnectionFactory'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'org.springframework.boot.autoconfigure.amqp.RabbitAutoConfiguration$MessagingTemplateConfiguration'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'rabbitMessagingTemplate'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'rabbitMessagingTemplate' via factory method to bean named 'rabbitTemplate'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'org.springframework.boot.autoconfigure.amqp.RabbitAnnotationDrivenConfiguration'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'org.springframework.boot.autoconfigure.amqp.RabbitAnnotationDrivenConfiguration' via constructor to bean named 'spring.rabbitmq-org.springframework.boot.autoconfigure.amqp.RabbitProperties'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'simpleRabbitListenerContainerFactoryConfigurer'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'rabbitListenerContainerFactory'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'rabbitListenerContainerFactory' via factory method to bean named 'simpleRabbitListenerContainerFactoryConfigurer'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.createArgumentArray:777]- Autowiring by type from bean name 'rabbitListenerContainerFactory' via factory method to bean named 'rabbitConnectionFactory'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'directRabbitListenerContainerFactoryConfigurer'
...
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'org.springframework.boot.actuate.autoconfigure.web.server.ManagementContextAutoConfiguration'
[main] [DEBUG] [org.springframework.beans.factory.support.DefaultListableBeanFactory.getSingleton:213]- Creating shared instance of singleton bean 'management.server-org.springframework.boot.actuate.autoconfigure.web.server.ManagementServerProperties'
[main] [DEBUG] [org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.:264]- Inferred argument type for public void com.gm.gsmc.pricing.messenger.service.PricingMessageListener.handleMessage(com.gm.gsmc.pricing.domain.queue.QueueMessage) is class com.gm.gsmc.pricing.domain.queue.QueueMessage
[main] [DEBUG] [org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.setConcurrentConsumers:166]- Changing consumers from 1 to 5
[main] [DEBUG] [org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.start:1293]- Starting Rabbit listener container.
[main] [INFO ] [org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createBareConnection:482]- Attempting to connect to: [localhost:5672]
[main] [INFO ] [org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.checkMismatchedQueues:1713]- Broker not available; cannot force queue declarations during start: java.net.ConnectException: Connection refused (Connection refused)
[org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] [INFO ] [org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createBareConnection:482]- Attempting to connect to: [localhost:5672]
[org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] [ERROR] [org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.redeclareElementsIfNecessary:1764]- Failed to check/redeclare auto-delete queue(s). java.net.ConnectException: Connection refused (Connection refused)\n at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.8.0_212]\n at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[na:1.8.0_212]\n at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[na:1.8.0_212]\n at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[na:1.8.0_212]\n at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.8.0_212]\n at java.net.Socket.connect(Socket.java:589) ~[na:1.8.0_212]\n at com.rabbitmq.client.impl.SocketFrameHandlerFactory.create(SocketFrameHandlerFactory.java:60) ~[amqp-client-5.4.3.jar!/:5.4.3]\n at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1102) ~[amqp-client-5.4.3.jar!/:5.4.3]\n at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:1054) ~[amqp-client-5.4.3.jar!/:5.4.3]\n at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:484) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n ... 11 common frames omitted\nWrapped by: org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)\n at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:62) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:530) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:702) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.createConnection(ConnectionFactoryUtils.java:214) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:2076) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2050) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:2030) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.core.RabbitAdmin.getQueueProperties(RabbitAdmin.java:403) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.attemptDeclarations(AbstractMessageListenerContainer.java:1777) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.redeclareElementsIfNecessary(AbstractMessageListenerContainer.java:1758) ~[spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.initialize(SimpleMessageListenerContainer.java:1195) [spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1041) [spring-rabbit-2.1.8.RELEASE.jar!/:2.1.8.RELEASE]\n at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]\n

Answer 1

I am not sure why you have those amqpAdmin calls in your bean definitions (although they are commented out). You must NEVER talk to the broker in a bean definition - it's too early in the lifecycle.
The error in the first log leads me to believe that was not commented out. That log looks truncated too - there should be Caused by parts. You should never truncate a log when asking a question. A partial stack trace is useless.
Factory method 'msgQueueBinding' threw exception; nested exception is org.springframework.amqp.AmqpConnectException: java.net.ConnectException: Connection refused (Connection refused)
This clearly shows an attempt to connect from the msgQueueBinding method.
The following logs show the queue/exchange/binding being successfully declared when the listener starts.
I am guessing your admin calls are happening before the cloud code has replaced the connection factory bean definition.

Answer 2

Thanks for the pointers Gary
This is basically the config that "works"
If I take RabbitTemplate or Container Factory off, I start seeing errors
I took some code from RabbitAutoConfiguration and did my best to read the same yml and use the properties in the beans I create
#Configuration
#EnableRabbit
#Profile("cloud")
#Slf4j
public class RabbitMQConfig extends AbstractCloudConfig {
final static String MAIN_QUEUE_NAME = "pricing";
private final static String QUEUE_EXCHANGE_NAME = "pricing-exchange";
private final static String DEAD_LETTER_QUEUE_NAME = "pricing-dl";
#Autowired
RabbitProperties rabbitProperties;
private void debug(String step) {
CachingConnectionFactory connectionFactory = (CachingConnectionFactory) connectionFactory().rabbitConnectionFactory();
log.info("===> debug connection with {} {} ", step, connectionFactory.getClass().getName());
log.info("host {}", connectionFactory.getHost());
log.info("port {}", connectionFactory.getPort());
log.info("vh {}", connectionFactory.getVirtualHost());
log.info("user {}", connectionFactory.getUsername());
log.info("Auto recovery {}", connectionFactory.getRabbitConnectionFactory().isAutomaticRecoveryEnabled());
log.info("RabbitProperties {}", ToStringBuilder.reflectionToString(rabbitProperties));
}
#SuppressWarnings("DuplicatedCode")
#Bean
public RabbitTemplate rabbitTemplate(MessageConverter messageConverter) {
debug("rabbitTemplate");
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory().rabbitConnectionFactory());
rabbitTemplate.setMessageConverter(messageConverter);
if (rabbitProperties != null) {
RabbitProperties.Template properties = this.rabbitProperties.getTemplate();
PropertyMapper map = PropertyMapper.get();
if (properties.getRetry().isEnabled()) {
rabbitTemplate.setRetryTemplate(createRetryTemplate(properties.getRetry()));
}
map.from(properties::getReceiveTimeout).whenNonNull().as(Duration::toMillis)
.to(rabbitTemplate::setReceiveTimeout);
map.from(properties::getReplyTimeout).whenNonNull().as(Duration::toMillis).to(rabbitTemplate::setReplyTimeout);
map.from(properties::getExchange).to(rabbitTemplate::setExchange);
map.from(properties::getRoutingKey).to(rabbitTemplate::setRoutingKey);
map.from(properties::getDefaultReceiveQueue).whenNonNull().to(rabbitTemplate::setDefaultReceiveQueue);
}
return rabbitTemplate;
}
private RetryTemplate createRetryTemplate(RabbitProperties.Retry properties) {
PropertyMapper map = PropertyMapper.get();
RetryTemplate template = new RetryTemplate();
SimpleRetryPolicy policy = new SimpleRetryPolicy();
map.from(properties::getMaxAttempts).to(policy::setMaxAttempts);
template.setRetryPolicy(policy);
ExponentialBackOffPolicy backOffPolicy = new ExponentialBackOffPolicy();
map.from(properties::getInitialInterval).whenNonNull().as(Duration::toMillis)
.to(backOffPolicy::setInitialInterval);
map.from(properties::getMultiplier).to(backOffPolicy::setMultiplier);
map.from(properties::getMaxInterval).whenNonNull().as(Duration::toMillis).to(backOffPolicy::setMaxInterval);
template.setBackOffPolicy(backOffPolicy);
return template;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory() {
debug("rabbit listener");
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory().rabbitConnectionFactory());
factory.setMessageConverter(jsonMessageConverter());
factory.setAcknowledgeMode(AcknowledgeMode.AUTO);
factory.setConcurrentConsumers(rabbitProperties.getListener().getSimple().getConcurrency());
factory.setMaxConcurrentConsumers(rabbitProperties.getListener().getSimple().getMaxConcurrency());
factory.setPrefetchCount(rabbitProperties.getListener().getSimple().getPrefetch());
factory.setMissingQueuesFatal(false);
return factory;
}
#Bean
public AmqpAdmin amqpAdmin() {
ConnectionFactory connectionFactory = connectionFactory().rabbitConnectionFactory();
debug("amqpAdmin");
RabbitAdmin rabbitAdmin = new RabbitAdmin(connectionFactory);
rabbitAdmin.initialize();
//This is what puts HA policy on the queue it seems
Properties queueProperties = rabbitAdmin.getQueueProperties(MAIN_QUEUE_NAME);
log.debug("queueProperties {}", queueProperties);
return rabbitAdmin;
}
#PostConstruct
public void init() {
debug("postConstruct");
}
#Bean("pricingQueue")
#Primary
public Queue pricingQueue() {
return QueueBuilder.durable(MAIN_QUEUE_NAME)
.withArgument("x-dead-letter-exchange", "")
.withArgument("x-dead-letter-routing-key", DEAD_LETTER_QUEUE_NAME)
.withArgument("x-queue-master-locator", "min-masters")
.build();
}
#Bean
public Queue deadLetterQueue() {
return QueueBuilder.durable(DEAD_LETTER_QUEUE_NAME)
.withArgument("x-queue-master-locator", "min-masters")
.build();
}
#Bean
public DirectExchange exchange() {
return new DirectExchange(QUEUE_EXCHANGE_NAME);
}
#Bean
public Binding pricingBinding(#Qualifier("pricingQueue") Queue pricingQueue,
DirectExchange exchange) {
return BindingBuilder.bind(pricingQueue).to(exchange).with(MAIN_QUEUE_NAME);
}
#Bean
public DefaultMessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory factory = new DefaultMessageHandlerMethodFactory();
factory.setMessageConverter(consumerJackson2MessageConverter());
return factory;
}
#Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
#Bean
public MessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
}