I would like to create rabbitmq queues everytime my Spring Boot application starts (if queues don't exists already).
In the current architecture, we manually create durabl queues using rabbitmq admin. But we want our application, to detect new queues in the configuration and create durable ones if they don't exist
That is built-in feature of Spring AMQP:
The AMQP specification describes how the protocol can be used to configure Queues, Exchanges and Bindings on the broker. These operations which are portable from the 0.8 specification and higher are present in the AmqpAdmin interface in the org.springframework.amqp.core package.
Since amqpAdmin bean is auto-configured by Spring Boot, you only need to declare particular #Beans for Queues, Exchanges and Bindings between them.
Related
While doing some load tests with the ActiveMQ Artemis broker and my Spring Boot application I am getting into performance issues.
What I am doing is, sending e.g. 12,000 messages per second to the broker with JMSeter and the application receives them and saves them to a DB. That works fine. But when I extend my application by a filter mechanism, which forwards events after saving to DB, back to the broker using jmsTemplate.send(destination, messageCreator) it goes very slow.
I first used ActiveMQ 5.x and there this mechanism works fine. There you could configure the ActiveMQConnectionFactory with setAsyncSend(true) to tune performance. For the ActiveMQ Artemis ConnectionFactory implementation there is no such a possibility. Is there another way to tune performance like in ActiveMQ 5.x?
I am using Apache ActiveMQ Artemis 2.16.0 (but also tried 2.15.0), artemis-jms-client 2.6.4, and Spring Boot 1.5.16.RELEASE.
The first thing to note is that you need to be very careful when using Spring's JmsTemplate to send messages as it employs a well-known anti-pattern that can really kill performance. It will actually create a new JMS connection, session, and producer for every message it sends. I recommend you use a connection pool like this one which is based on the ActiveMQ 5.x connection pool implementation but now supports JMS 2. For additional details about the danger of using JmsTemplate see the ActiveMQ documentation. This is also discussed in an article from Pivotal (i.e. the "owners" of Spring).
The second point here is that you can tune if persistent JMS messages are sent synchronously or not using the blockOnDurableSend URL property, e.g.:
tcp://localhost:61616?blockOnDurableSend=false
This will ensure that persistent JMS messages are sent asynchronously. This is discussed further in the ActiveMQ Artemis documentation.
We have configured our ActiveMQ message broker as a Spring Boot project and there's another Spring Boot application (let's call it service-A) that has a listener configured to listen to some topics using #JmsListener annotation. It's a Spring Cloud microservice appilcation.
The problem:
It is possible that service-A can have multiple instances running.
If we have 2 instances running, then any message coming on topic gets listened to twice.
How can we avoid every instance listening to the topic?
We want to make sure that the topic is listened to only once no matte the number of service-A instances.
Is it possible to run the microservice in a cluster mode or something similar? I also checked out ActiveMQ virtual destinations but not too sure if that's the solution to the problem.
We have also thought of an approach where we can decide who's the leader node from the multiple instances, but that's the last resort and we are looking for a cleaner approach.
Any useful pointers, references are welcome.
What you really want is a shared topic subscription which was added in JMS 2. Unfortunately ActiveMQ 5.x doesn't support JMS 2. However, ActiveMQ Artemis does.
ActiveMQ Artemis is the next generation broker from ActiveMQ. It supports most of the same features as ActiveMQ 5.x (including full support for OpenWire clients) as well as many other features that 5.x doesn't support (e.g. JMS 2, shared-nothing high-availability using replication, last-value queues, ring queues, metrics plugins for integration with tools like Prometheus, duplicate message detection, etc.). Furthermore, ActiveMQ Artemis is built on a high-performance, non-blocking core which means scalability is much better as well.
I am currently working on an Integration application that uses Camel with Spring Boot. There is a camel route in integration application that receive messages from source Artemis broker that is transformed and sent to another Artemis broker.
The camel route looks like this:
from(sourceQueue).process(transformProcessor).to(destinationQueue)
When the camel route starts, it recreates the queue names mentioned in the from and to and the previous messages are lost. We do not expect this to happen.
One way I found to do this is in the Artemis ActiveMQ broker.xml, disable the queue and topic auto creation and create the queue(s) using Artemis API.
My question is, can we configure camel JMS / AMQP component to create the queue only if it is not present and if present use the existing ones?
By default Camel will use DynamicDestinationResolver. You can create your own custom DestinationResolver and plug it in your endpoint (or into your component)
.to("jms:queue:myQueue?destinationResolver=MyCustomDestinationResolver");
You can also use JndiDestinationResolver, which by default does not fallback into creating a dynamic destination.
I don't know Artemis but it sounds weird for a broker to delete a queue with its messages. At least its "brother" ActiveMQ has by default the behavior you expect: queues are automatically created if they do not exist, but they just stay if they already exist.
Are you sure the queues are recreated on route start? Are these queues persistent? Could it be that a consumer just drains the queue? I also found a queue attribute of Artemis named auto-delete-queues that would delete the queue if it was drained by a consumer.
auto-delete-queues Whether or not to the broker should automatically delete auto-created JMS queues when they have both 0 consumers and 0 messages.
I am using Spring, Spring-Websocket, STOMP for my application, and RabbitMQ as broker. I need to log all messages going through RabbitMQ to Postgresql tables.
I know that I can write #MessageMapping in Spring and log there, but my problem is that some clients talk to RabbitMQ directly through MQTT protocol, and Spring does not support it yet (https://jira.spring.io/browse/SPR-12581). Moreover browser clients talk through Spring to RabbitMQ using STOMP protocol.
RabbitMQ allows to track all messages using Firehose tracer. How to properly listen to amq.rabbitmq.trace topic from Spring? Or do I need to write separate Java app as consumer?
The Spring AMQP is for you!
You bind some custom queue to to that amq.rabbitmq.trace with appropriate pattern (e.g. publish.#) and configure SimpleMessageListenerContainer to receive messages from that queue.
It can be done even with pretty simple config: #EnableRabbit and #RabbitListener on some POJO method. Anyway the Binding #Bean must be there to attache your queue to that exchange.
We currently use activemq for JMS within Camel. Now our requirements have mandated us to create a completely separate application (separate jvm instance) that also uses ActiveMQ Standalone to broker messages with external clients.
I want to know if we can use this separate standalone for the camel jms endpoints Or if it is possible to use the embedded activemq in camel for our new separate application.
You can use a single ActiveMQ instance. Just make sure you have no destination name collisions [unless they will share] and that your instance creates networked listeners using tcp:// or one of the other networked protocols, i.e. you would not want to use vm://.