Spring RabbitMQ - add bindings during runtime - spring-boot

I've been following the guide https://spring.io/guides/gs/messaging-rabbitmq/#initial for setting up rabbitMQ communication via Spring. Queues, Exchanges, Bindings are declared and initialised on Application startup. The code, pretty much like the tutorial:
public static final String SPRINGQUEUE = "springqueue";
public static final String SPRINGEXCHANGE = "springexchange";
#Bean
Queue queue() {
return new Queue(SPRINGQUEUE, false);
}
#Bean
TopicExchange exchange() {
return new TopicExchange(SPRINGEXCHANGE);
}
#Bean
Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with("routingkey1");
}
#Bean
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory, MessageListener listener) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(SPRINGQUEUE);
container.setMessageListener(listener);
return container;
}
#Bean
MessageListener createListener(Queue queue, TopicExchange exchange) {
return new Listener(queue, exchange);
}
This listens to "routingkey1" just as expected, however, I would like to change the keys to listen for dynamically during runtime. I know this is possible using the RabbitMQ Java client, by simply binding a channel multiple times. Is there any way to do this through spring AMQP as well?

See RabbitAdmin API:
/**
* Declare a binding of a queue to an exchange.
* #param binding a description of the binding to declare.
*/
void declareBinding(Binding binding);
for example.
You don't listen to the routing key: you listen to the queue. The routing key is a part of publisher logic: you publish a message to the exchange with some routing key. The RabbitMQ broker already decides to what queue place this message to. Therefore see if your expectations with dynamic bindings and listening is what is possible with an AQMP protocol.

Related

How to consume only one RabbitMQ on an application that has 3 queues?

I have a Spring Boot application that uses RabbitMQ and has 3 queues (queue1, queue2 and queue3).
In this application i have one listener, that should only listen for messages on the queue named queue1 and ignore the other 2 queues, but it is getting messages from all queues.
This is my RabbitMQ config:
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory(this.host);
connectionFactory.setPort(this.port);
connectionFactory.setUsername(this.user);
connectionFactory.setPassword(this.password);
return connectionFactory;
}
#Bean
SimpleMessageListenerContainer container(MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(this.connectionFactory());
container.setQueueNames(this.startQueueQueueName, this.printQueueQueueName, this.errorQueueQueueName);
container.setMessageListener(listenerAdapter);
return container;
}
#Bean
MessageListenerAdapter listenerAdapter(RabbitDocumentToPrintListener receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
and this is my listener
public void receiveMessage(String message) throws Exception {
this.logger.debug("Received message from Rabbit");
}
I've tried adding #RabbitListener(queues = "queue1", exclusive = true) to the listener, but it didn't work.
If someone could help me making this app to consume only queue1, I'd appreciate. Thanks!

Configure dead letter queue. DLQ for a jms consumer

It is a rather simple question... I have a spring project where I consume queues (CONSUMER).
Now I want to configure individual dead letter queues for each queue I am consuming.
However, in my mind, the individual dead letter queues configuration must be done in the broker service (SERVER), not in the CONSUMER. Is it really so?
My code below WILL NOT work, correct?
#Bean
public DeadLetterStrategy deadLetterStrategy(){
IndividualDeadLetterStrategy dlq = new IndividualDeadLetterStrategy();
dlq.setQueueSuffix(".DLQ");
dlq.setUseQueueForQueueMessages(true);
return dlq;
}
#Bean
public ActiveMQConnectionFactory consumerActiveMQConnectionFactory() {
var activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(brokerUrl);
RedeliveryPolicy policy = activeMQConnectionFactory.getRedeliveryPolicy();
policy.setMaximumRedeliveries(maximumRedeliveries);
policy.setInitialRedeliveryDelay(0);
policy.setBackOffMultiplier(3);
policy.setUseExponentialBackOff(true);
return activeMQConnectionFactory;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
var factory = new DefaultJmsListenerContainerFactory();
factory.setSessionAcknowledgeMode(JmsProperties.AcknowledgeMode.CLIENT.getMode());
factory.setConcurrency(factoryConcurrency);
factory.setConnectionFactory(consumerActiveMQConnectionFactory());
return factory;
}
#Bean
public BrokerService broker() throws Exception {
final BrokerService broker = new BrokerService();
broker.addConnector(brokerUrl);
broker.setPersistent(false);
broker.setDestinationPolicy(policyMap());
return broker;
}
#Bean
public PolicyMap policyMap() {
PolicyMap destinationPolicy = new PolicyMap();
List<PolicyEntry> entries = new ArrayList<PolicyEntry>();
PolicyEntry queueEntry = new PolicyEntry();
queueEntry.setQueue(">"); // In activemq '>' does the same thing as '*' does in other languages
queueEntry.setDeadLetterStrategy(deadLetterStrategy());
entries.add(queueEntry);
destinationPolicy.setPolicyEntries(entries);
return destinationPolicy;
} }
#JmsListener(destination = "myqueue")
public void onMessage(Message message, Session session) throws JMSException {
try {
stuff()
message.acknowledge();
} catch (Exception ex) {
session.recover();
}
}
A JMS consumer in ActiveMQ 5.x cannot configure the broker side dead letter strategy, this must be done at the broker in the configuration XML or via programmatic broker configuration. You could configure it in spring as you've done if your broker is simply an in memory broker however that is of little use for most applications.
Refer to the broker documentation for more help on configuration of the broker.

Spring 3 and Rabbit MQ integration (not Spring Boot)

I'm having difficulty getting a Spring 3 application to integrate with RabbitMQ, in order to receive messages from a queue (I do not need to send messages).
Part of the challenge is much of the documentation now relates to Spring Boot. The related Spring guide is helpful, but following the steps does not seem to work in my case. For instance, the guide includes the text:
The message listener container and receiver beans are all you need to listen for messages.
So I have setup the listener container and receiver beans with the following code.
Setting up message handler
#Component
public class CustomMessageHandler {
public void handleMessage(String text) {
System.out.println("Received: " + text);
}
}
Setting up configuration
#Configuration
public class RabbitConfig {
#Bean
public RabbitTemplate rabbitTemplate(final ConnectionFactory connectionFactory){
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setRoutingKey("queue-name");
return rabbitTemplate;
}
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setHost("...host...");
connectionFactory.setPort(5671);
connectionFactory.setVirtualHost("...virtual host..");
connectionFactory.setUsername("...username...");
connectionFactory.setPassword("...password...");
return connectionFactory;
}
#Bean
public MessageListenerAdapter messageListenerAdapter(CustomMessageHandler messageHandler) {
return new MessageListenerAdapter(messageHandler, "handleMessage");
}
#Bean
public SimpleMessageListenerContainer listenerContainer(ConnectionFactory connectionFactory,
MessageListenerAdapter messageListenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setQueueNames("queue-name");
container.setConnectionFactory(connectionFactory);
container.setMessageListener(messageListenerAdapter);
return container;
}
}
Unfortunately with this setup, the application will start up, but it never triggers the message handler. The queue it is trying to read from also has one message sitting in it, waiting to be consumed.
Any ideas on something that is missing, or appears misconfigured?
Thanks to some dependency management assistance from #GaryRussell, I was able to see that the version of spring-rabbit and spring-amqp were too recent. Using the older 1.3.9.RELEASE unfortunately proved to add additional challenges.
Some other assistance came in the form of using an actual RabbitMQ Java client. This option was much simpler to implement, and avoided the dependency problems. Ultimately I needed to include the following dependency:
<dependency>
<groupId>com.rabbitmq</groupId>
<artifactId>amqp-client</artifactId>
<version>5.7.3</version>
</dependency>
And then I simply followed their documentation on creating a connection, and consuming messages.
Voila, it works!

Spring rabbitmq task queue concurrency

I have a task queue which is using a proxy to make http requests. The proxy is limited to 10 concurrent threads / connections. I don't have access to the logs of the proxy.
I am using the following code, and this is making requests on two threads named ntContainer#1-1 and container1. This is resulting in many requests that error due to using too many connections to the proxy.
Is the listener only using 1 default thread and the extra container thread or is there more going on behind the scenes with spring/rabbitmq?
Also how can I debug this further?
#Configuration
public class RabbitMQConfig {
public final static String EXCHANGE_NAME = "my-tx";
public final static String MY_PRODUCT_ROUTING_KEY = "my-product-routing-key";
public final static String MY_PRODUCT_QUEUE = "my-product";
#Bean
public TopicExchange topicExchange() {
return new TopicExchange(EXCHANGE_NAME);
}
#Bean
public Queue myProductQueue() {
return new Queue(MY_PRODUCT_QUEUE);
}
#Bean
Binding myProductBinding() {
return BindingBuilder.bind(myProductQueue()).to(topicExchange()).with(MY_PRODUCT_ROUTING_KEY);
}
#Bean
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory, MessageListenerAdapter messageListenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(MY_PRODUCT_QUEUE);
container.setMessageListener(messageListenerAdapter);
container.setPrefetchCount(1);
container.setConcurrentConsumers(1);
return container;
}
#Bean
MessageListenerAdapter messageListenerAdapter(MyListener myListener) {
return new MessageListenerAdapter(myListener, "process");
}
}
// listener
#RabbitListener(queues = RabbitMQConfig.MY_PRODUCT_QUEUE)
public void process(final Message message) {
// something like this
Jsoup.connect(message.getUrl()).proxy().execute()
}
Oops; I was looking at the question on my 'phone; I skipped past the container bean; I thought the container bean was a container factory not a container.
You have 2 listener containers -
#RabbitListener(queues = RabbitMQConfig.MY_PRODUCT_QUEUE)
public void process(final Message message) {
// something like this
Jsoup.connect(message.getUrl()).proxy().execute()
}
The framework will automatically create a container for that listener (it detects the annotation) and you have explicitly declared another container #Bean.
The proxy is limited to 10 concurrent threads / connections.
Even with 2 containers, you'll only get 2 threads, not 10.

Spring JMS HornetQ user is null

I am trying to connect to a remote HornetQ broker in a spring boot/spring jms application and setup a #JmsListener.
HornetQ ConnectionFactory is being fetched from JNDI registry that HornetQ instance hosts. Everything works fine as long as HornetQ security is turned off but when it is turned on I get this error
WARN o.s.j.l.DefaultMessageListenerContainer : Setup of JMS message listener invoker failed for destination 'jms/MI/Notification/Queue' - trying to recover. Cause: User: null doesn't have permission='CONSUME' on address jms.queue.MI/Notification/Queue
I ran a debug session to figure out that ConnectionFactory instance being returned is HornetQXAConnectionFactory but user and password fields are not set, which I believe is why user is null. I verified that user principal and credentials are set in JNDI properties but somehow it is not being passed on to ConnectionFactory instance. Any help on how I can get this setup working would be greatly appreciated.
This is my jms related config
#Configuration
#EnableJms
public class JmsConfig {
#Bean
public JmsListenerContainerFactory<?> jmsListenerContainerFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setDestinationResolver(destinationResolver());
return factory;
}
#Bean // Serialize message content to json using TextMessage
public MessageConverter jacksonJmsMessageConverter() {
MappingJackson2MessageConverter converter = new MappingJackson2MessageConverter();
converter.setTargetType(MessageType.BYTES);
converter.setTypeIdPropertyName("_type");
return converter;
}
#Value("${jms.jndi.provider.url}")
private String jndiProviderURL;
#Value("${jms.jndi.principal}")
private String jndiPrincipal;
#Value("${jms.jndi.credentials}")
private String jndiCredential;
#Bean
public JndiTemplate jndiTemplate() {
Properties env = new Properties();
env.put("java.naming.factory.initial", "org.jnp.interfaces.NamingContextFactory");
env.put("java.naming.provider.url", jndiProviderURL);
env.put("java.naming.security.principal", jndiPrincipal);
env.put("java.naming.security.credentials", jndiCredential);
return new JndiTemplate(env);
}
#Bean
public DestinationResolver destinationResolver() {
JndiDestinationResolver destinationResolver = new JndiDestinationResolver();
destinationResolver.setJndiTemplate(jndiTemplate());
return destinationResolver;
}
#Value("${jms.connectionfactory.jndiname}")
private String connectionFactoryJNDIName;
#Bean
public JndiObjectFactoryBean connectionFactoryFactory() {
JndiObjectFactoryBean jndiObjectFactoryBean = new JndiObjectFactoryBean();
jndiObjectFactoryBean.setJndiTemplate(jndiTemplate());
jndiObjectFactoryBean.setJndiName(connectionFactoryJNDIName);
jndiObjectFactoryBean.setResourceRef(true);
jndiObjectFactoryBean.setProxyInterface(ConnectionFactory.class);
return jndiObjectFactoryBean;
}
#Bean
public ConnectionFactory connectionFactory(JndiObjectFactoryBean connectionFactoryFactory) {
return (ConnectionFactory) connectionFactoryFactory.getObject();
}
}
JNDI and JMS are 100% independent as they are completely different specifications implemented in potentially completely different ways. Therefore the credentials you use for your JNDI lookup do not apply to your JMS resources. You need to explicitly set the username and password credentials on your JMS connection. This is easy using the JMS API directly (e.g. via javax.jms.ConnectionFactory#createConnection(String username, String password)). Since you're using Spring you could use something like this:
#Bean
public ConnectionFactory connectionFactory(JndiObjectFactoryBean connectionFactoryFactory) {
UserCredentialsConnectionFactoryAdapter cf = new UserCredentialsConnectionFactoryAdapter();
cf.setTargetConnectionFactory((ConnectionFactory) connectionFactoryFactory.getObject());
cf.setUsername("yourJmsUsername");
cf.setPassword("yourJmsPassword");
return cf;
}
Also, for what it's worth, the HornetQ code-base was donated to the Apache ActiveMQ project three and a half years ago now and it lives on as the Apache ActiveMQ Artemis broker. There's been 22 releases since then with numerous new features and bug fixes. I strongly recommend you migrate if at all possible.
Wrap the connection factory in a UserCredentialsConnectionFactoryAdapter.
/**
* An adapter for a target JMS {#link javax.jms.ConnectionFactory}, applying the
* given user credentials to every standard {#code createConnection()} call,
* that is, implicitly invoking {#code createConnection(username, password)}
* on the target. All other methods simply delegate to the corresponding methods
* of the target ConnectionFactory.
* ...

Resources