I am using Spring Boot. I am trying to remove channels which are not in use using Spring AMQP (RabbitMQ) by Spring Boot. But not getting how to achieve it. Any help is appreciable.
ConnectionFactory Declaration:
public ConnectionFactory connectionFactory() {
final CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setUsername(userName);
connectionFactory.setPassword(password);
connectionFactory.setVirtualHost(centralHost);
connectionFactory.setHost(rabbitMqHost);
connectionFactory.setConnectionTimeout(connectionTimeout);
connectionFactory.setChannelCacheSize(4);
connectionFactory.setExecutor(Executors.newFixedThreadPool(rabbitmqThreads));
return connectionFactory;
}
ContainerFactory Declaration:
public DirectRabbitListenerContainerFactory rabbitDirectListenerContainerFactory(ConnectionFactory connectionFactory) {
DirectRabbitListenerContainerFactory factory = new DirectRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setAcknowledgeMode(AcknowledgeMode.NONE);
factory.setAfterReceivePostProcessors( m -> {
m.getMessageProperties().setContentType("text/plain");
return m;
});
return factory;
}
Why do you want to close it? That's the whole point of caching the channel(s); so we don't have to create a new one each time we publish a message.
The minimum cache size is 1.
You can call resetConnection() on the CachingConnectionFactory to close the connection and all cached channels.
Related
I have a Spring Boot application that uses RabbitMQ and has 3 queues (queue1, queue2 and queue3).
In this application i have one listener, that should only listen for messages on the queue named queue1 and ignore the other 2 queues, but it is getting messages from all queues.
This is my RabbitMQ config:
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(this.host);
connectionFactory.setPort(this.port);
connectionFactory.setUsername(this.user);
connectionFactory.setPassword(this.password);
return connectionFactory;
}
#Bean
SimpleMessageListenerContainer container(MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(this.connectionFactory());
container.setQueueNames(this.startQueueQueueName, this.printQueueQueueName, this.errorQueueQueueName);
container.setMessageListener(listenerAdapter);
return container;
}
#Bean
MessageListenerAdapter listenerAdapter(RabbitDocumentToPrintListener receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
and this is my listener
public void receiveMessage(String message) throws Exception {
this.logger.debug("Received message from Rabbit");
}
I've tried adding #RabbitListener(queues = "queue1", exclusive = true) to the listener, but it didn't work.
If someone could help me making this app to consume only queue1, I'd appreciate. Thanks!
I have an application which receives a message from one queue, processes it and sends it to another queue. When it's receiving a lot of messages (20 thousand or more), spring shows me this message when it tries to send the message to another queue:
connection error; protocol method: #method<connection.close>(reply-code=504 reply-text=CHANNEL_ERROR - second 'channel.open' seen class-id=20 method-id=10)
So I raised the channel cache size and created two CachingConnectionFactory one for consumer and another for the producer, this configurations I followed a note from spring doc:
When the application is configured with a single CachingConnectionFactory, as it is by default with Spring Boot auto-configuration, the application will stop working when the connection is blocked by the Broker. And when it is blocked by the Broker, any its clients stop to work. If we have producers and consumers in the same application, we may end up with a deadlock when producers are blocking the connection because there are no resources on the Broker anymore and consumers can’t free them because the connection is blocked. To mitigate the problem, there is just enough to have one more separate CachingConnectionFactory instance with the same options - one for producers and one for consumers. The separate CachingConnectionFactory isn’t recommended for transactional producers, since they should reuse a Channel associated with the consumer transactions.
Following this recommendations the error message disappeared, but now the application suddenly stops, it's not sending or receiving new messages and all queues are idle. It's kind strange because it has a low concurrency number on listener. What am I missing?
Configuration:
Spring Boot: 2.0.8.RELEASE
Spring AMQP: 2.0.11.RELEASE
RabbitMQ: 3.8.8
spring:
rabbitmq:
listener:
simple:
default-requeue-rejected: false
concurrency: 5
max-concurrency: 8
cache:
channel:
size: 1000
#Bean
public ConnectionFactory consumerConnectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setHost(properties.getHost());
connectionFactory.setPort(properties.getPort());
connectionFactory.setUsername(properties.getUsername());
connectionFactory.setPassword(properties.getPassword());
connectionFactory.setChannelCacheSize(properties.getCache().getChannel().getSize());
connectionFactory.setConnectionNameStrategy(cns());
return connectionFactory;
}
#Bean
public ConnectionFactory producerConnectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setHost(properties.getHost());
connectionFactory.setPort(properties.getPort());
connectionFactory.setUsername(properties.getUsername());
connectionFactory.setPassword(properties.getPassword());
connectionFactory.setChannelCacheSize(properties.getCache().getChannel().getSize());
connectionFactory.setConnectionNameStrategy(cns());
return connectionFactory;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(#Qualifier("consumerConnectionFactory") ConnectionFactory consumerConnectionFactory,
SimpleRabbitListenerContainerFactoryConfigurer configurer,
RabbitProperties properties) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setErrorHandler(errorHandler());
factory.setConcurrentConsumers(properties.getListener().getSimple().getConcurrency());
factory.setMaxConcurrentConsumers(properties.getListener().getSimple().getMaxConcurrency());
configurer.configure(factory, consumerConnectionFactory);
return factory;
}
#Bean
#Primary
public RabbitAdmin producerRabbitAdmin() {
return new RabbitAdmin(producerConnectionFactory());
}
#Bean
public RabbitAdmin consumerRabbitAdmin() {
return new RabbitAdmin(consumerConnectionFactory());
}
#Bean
#Primary
public RabbitTemplate producerRabbitTemplate() {
RabbitTemplate rabbitTemplate = new RabbitTemplate(producerConnectionFactory());
rabbitTemplate.setMessageConverter(jackson2JsonMessageConverter());
return rabbitTemplate;
}
#Bean
public RabbitTemplate consumerRabbitTemplate() {
RabbitTemplate rabbitTemplate = new RabbitTemplate(consumerConnectionFactory());
rabbitTemplate.setMessageConverter(jackson2JsonMessageConverter());
return rabbitTemplate;
}
After analize, the problem was due to Java Memory Heap limit. Besides, I updated my configuration, removed ConnectionFactory beans, and set a publisher factory to RabbitTemplate
So I ended with this:
#Bean
#Primary
public RabbitTemplate producerRabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(jackson2JsonMessageConverter());
rabbitTemplate.setUsePublisherConnection(true);
return rabbitTemplate;
}
#Bean
public RabbitTemplate consumerRabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(jackson2JsonMessageConverter());
return rabbitTemplate;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory,
SimpleRabbitListenerContainerFactoryConfigurer configurer,
RabbitProperties properties) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setErrorHandler(errorHandler());
factory.setConcurrentConsumers(properties.getListener().getSimple().getConcurrency());
factory.setMaxConcurrentConsumers(properties.getListener().getSimple().getMaxConcurrency());
configurer.configure(factory, connectionFactory);
return factory;
}
With this configuration memory consume was reduced and I was able to raise consumer concurrey numbers:
spring:
rabbitmq:
listener:
simple:
default-requeue-rejected: false
concurrency: 10
max-concurrency: 15
cache:
channel:
size: 1000
I'm looking now for the right cache channel size and to raise even more concurrency numbers.
I'm having difficulty getting a Spring 3 application to integrate with RabbitMQ, in order to receive messages from a queue (I do not need to send messages).
Part of the challenge is much of the documentation now relates to Spring Boot. The related Spring guide is helpful, but following the steps does not seem to work in my case. For instance, the guide includes the text:
The message listener container and receiver beans are all you need to listen for messages.
So I have setup the listener container and receiver beans with the following code.
Setting up message handler
#Component
public class CustomMessageHandler {
public void handleMessage(String text) {
System.out.println("Received: " + text);
}
}
Setting up configuration
#Configuration
public class RabbitConfig {
#Bean
public RabbitTemplate rabbitTemplate(final ConnectionFactory connectionFactory){
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setRoutingKey("queue-name");
return rabbitTemplate;
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setHost("...host...");
connectionFactory.setPort(5671);
connectionFactory.setVirtualHost("...virtual host..");
connectionFactory.setUsername("...username...");
connectionFactory.setPassword("...password...");
return connectionFactory;
}
#Bean
public MessageListenerAdapter messageListenerAdapter(CustomMessageHandler messageHandler) {
return new MessageListenerAdapter(messageHandler, "handleMessage");
}
#Bean
public SimpleMessageListenerContainer listenerContainer(ConnectionFactory connectionFactory,
MessageListenerAdapter messageListenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setQueueNames("queue-name");
container.setConnectionFactory(connectionFactory);
container.setMessageListener(messageListenerAdapter);
return container;
}
}
Unfortunately with this setup, the application will start up, but it never triggers the message handler. The queue it is trying to read from also has one message sitting in it, waiting to be consumed.
Any ideas on something that is missing, or appears misconfigured?
Thanks to some dependency management assistance from #GaryRussell, I was able to see that the version of spring-rabbit and spring-amqp were too recent. Using the older 1.3.9.RELEASE unfortunately proved to add additional challenges.
Some other assistance came in the form of using an actual RabbitMQ Java client. This option was much simpler to implement, and avoided the dependency problems. Ultimately I needed to include the following dependency:
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.7.3</version>
</dependency>
And then I simply followed their documentation on creating a connection, and consuming messages.
Voila, it works!
I am trying to connect to a remote HornetQ broker in a spring boot/spring jms application and setup a #JmsListener.
HornetQ ConnectionFactory is being fetched from JNDI registry that HornetQ instance hosts. Everything works fine as long as HornetQ security is turned off but when it is turned on I get this error
WARN o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'jms/MI/Notification/Queue' - trying to recover. Cause: User: null doesn't have permission='CONSUME' on address jms.queue.MI/Notification/Queue
I ran a debug session to figure out that ConnectionFactory instance being returned is HornetQXAConnectionFactory but user and password fields are not set, which I believe is why user is null. I verified that user principal and credentials are set in JNDI properties but somehow it is not being passed on to ConnectionFactory instance. Any help on how I can get this setup working would be greatly appreciated.
This is my jms related config
#Configuration
#EnableJms
public class JmsConfig {
#Bean
public JmsListenerContainerFactory<?> jmsListenerContainerFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setDestinationResolver(destinationResolver());
return factory;
}
#Bean // Serialize message content to json using TextMessage
public MessageConverter jacksonJmsMessageConverter() {
MappingJackson2MessageConverter converter = new MappingJackson2MessageConverter();
converter.setTargetType(MessageType.BYTES);
converter.setTypeIdPropertyName("_type");
return converter;
}
#Value("${jms.jndi.provider.url}")
private String jndiProviderURL;
#Value("${jms.jndi.principal}")
private String jndiPrincipal;
#Value("${jms.jndi.credentials}")
private String jndiCredential;
#Bean
public JndiTemplate jndiTemplate() {
Properties env = new Properties();
env.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory");
env.put("java.naming.provider.url", jndiProviderURL);
env.put("java.naming.security.principal", jndiPrincipal);
env.put("java.naming.security.credentials", jndiCredential);
return new JndiTemplate(env);
}
#Bean
public DestinationResolver destinationResolver() {
JndiDestinationResolver destinationResolver = new JndiDestinationResolver();
destinationResolver.setJndiTemplate(jndiTemplate());
return destinationResolver;
}
#Value("${jms.connectionfactory.jndiname}")
private String connectionFactoryJNDIName;
#Bean
public JndiObjectFactoryBean connectionFactoryFactory() {
JndiObjectFactoryBean jndiObjectFactoryBean = new JndiObjectFactoryBean();
jndiObjectFactoryBean.setJndiTemplate(jndiTemplate());
jndiObjectFactoryBean.setJndiName(connectionFactoryJNDIName);
jndiObjectFactoryBean.setResourceRef(true);
jndiObjectFactoryBean.setProxyInterface(ConnectionFactory.class);
return jndiObjectFactoryBean;
}
#Bean
public ConnectionFactory connectionFactory(JndiObjectFactoryBean connectionFactoryFactory) {
return (ConnectionFactory) connectionFactoryFactory.getObject();
}
}
JNDI and JMS are 100% independent as they are completely different specifications implemented in potentially completely different ways. Therefore the credentials you use for your JNDI lookup do not apply to your JMS resources. You need to explicitly set the username and password credentials on your JMS connection. This is easy using the JMS API directly (e.g. via javax.jms.ConnectionFactory#createConnection(String username, String password)). Since you're using Spring you could use something like this:
#Bean
public ConnectionFactory connectionFactory(JndiObjectFactoryBean connectionFactoryFactory) {
UserCredentialsConnectionFactoryAdapter cf = new UserCredentialsConnectionFactoryAdapter();
cf.setTargetConnectionFactory((ConnectionFactory) connectionFactoryFactory.getObject());
cf.setUsername("yourJmsUsername");
cf.setPassword("yourJmsPassword");
return cf;
}
Also, for what it's worth, the HornetQ code-base was donated to the Apache ActiveMQ project three and a half years ago now and it lives on as the Apache ActiveMQ Artemis broker. There's been 22 releases since then with numerous new features and bug fixes. I strongly recommend you migrate if at all possible.
Wrap the connection factory in a UserCredentialsConnectionFactoryAdapter.
/**
* An adapter for a target JMS {#link javax.jms.ConnectionFactory}, applying the
* given user credentials to every standard {#code createConnection()} call,
* that is, implicitly invoking {#code createConnection(username, password)}
* on the target. All other methods simply delegate to the corresponding methods
* of the target ConnectionFactory.
* ...
I have my JMS configuration like below (Spring boot 1.3.8);
#Configuration
#EnableJms
public class JmsConfig {
#Autowired
private AppProperties properties;
#Bean
TopicConnectionFactory topicConnectionFactory() throws JMSException {
return new TopicConnectionFactory(properties.getBrokerURL(), properties.getBrokerUserName(),
properties.getBrokerPassword());
}
#Bean
CachingConnectionFactory connectionFactory() throws JMSException {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(topicConnectionFactory());
connectionFactory.setSessionCacheSize(50);
return connectionFactory;
}
#Bean
JmsTemplate jmsTemplate() throws JMSException {
JmsTemplate jmsTemplate = new JmsTemplate(connectionFactory());
jmsTemplate.setPubSubDomain(Boolean.TRUE);
return jmsTemplate;
}
#Bean
DefaultJmsListenerContainerFactory defaultContainerFactory() throws JMSException {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setPubSubDomain(Boolean.TRUE);
factory.setRecoveryInterval(30 * 1000L);
return factory;
}
}
This should work fine. But i am worried about whats written on the doc of CachingConnectionFactory
Specially, these parts;
NOTE: This ConnectionFactory requires explicit closing of all Sessions obtained from its shared Connection
Note also that MessageConsumers obtained from a cached Session won't get closed until the Session will eventually be removed from the pool. This may lead to semantic side effects in some cases.
I thought the framework handled the closing session and connection part? If it does not; how should i close them properly?
or maybe i am missing something?
Any help is appreciated :)
F.Y.I : I Use SonicMQ as the broker
Yes, the JmsTemplate will close the session; the javadocs refer to direct use outside of the framework.