While doing some load tests with the ActiveMQ Artemis broker and my Spring Boot application I am getting into performance issues.
What I am doing is, sending e.g. 12,000 messages per second to the broker with JMSeter and the application receives them and saves them to a DB. That works fine. But when I extend my application by a filter mechanism, which forwards events after saving to DB, back to the broker using jmsTemplate.send(destination, messageCreator) it goes very slow.
I first used ActiveMQ 5.x and there this mechanism works fine. There you could configure the ActiveMQConnectionFactory with setAsyncSend(true) to tune performance. For the ActiveMQ Artemis ConnectionFactory implementation there is no such a possibility. Is there another way to tune performance like in ActiveMQ 5.x?
I am using Apache ActiveMQ Artemis 2.16.0 (but also tried 2.15.0), artemis-jms-client 2.6.4, and Spring Boot 1.5.16.RELEASE.
The first thing to note is that you need to be very careful when using Spring's JmsTemplate to send messages as it employs a well-known anti-pattern that can really kill performance. It will actually create a new JMS connection, session, and producer for every message it sends. I recommend you use a connection pool like this one which is based on the ActiveMQ 5.x connection pool implementation but now supports JMS 2. For additional details about the danger of using JmsTemplate see the ActiveMQ documentation. This is also discussed in an article from Pivotal (i.e. the "owners" of Spring).
The second point here is that you can tune if persistent JMS messages are sent synchronously or not using the blockOnDurableSend URL property, e.g.:
tcp://localhost:61616?blockOnDurableSend=false
This will ensure that persistent JMS messages are sent asynchronously. This is discussed further in the ActiveMQ Artemis documentation.
Related
We have configured our ActiveMQ message broker as a Spring Boot project and there's another Spring Boot application (let's call it service-A) that has a listener configured to listen to some topics using #JmsListener annotation. It's a Spring Cloud microservice appilcation.
The problem:
It is possible that service-A can have multiple instances running.
If we have 2 instances running, then any message coming on topic gets listened to twice.
How can we avoid every instance listening to the topic?
We want to make sure that the topic is listened to only once no matte the number of service-A instances.
Is it possible to run the microservice in a cluster mode or something similar? I also checked out ActiveMQ virtual destinations but not too sure if that's the solution to the problem.
We have also thought of an approach where we can decide who's the leader node from the multiple instances, but that's the last resort and we are looking for a cleaner approach.
Any useful pointers, references are welcome.
What you really want is a shared topic subscription which was added in JMS 2. Unfortunately ActiveMQ 5.x doesn't support JMS 2. However, ActiveMQ Artemis does.
ActiveMQ Artemis is the next generation broker from ActiveMQ. It supports most of the same features as ActiveMQ 5.x (including full support for OpenWire clients) as well as many other features that 5.x doesn't support (e.g. JMS 2, shared-nothing high-availability using replication, last-value queues, ring queues, metrics plugins for integration with tools like Prometheus, duplicate message detection, etc.). Furthermore, ActiveMQ Artemis is built on a high-performance, non-blocking core which means scalability is much better as well.
We are using the Spring JmsTemplate implementation with a CachingConnectionFactory. We have configured the connection with a failover-url:
failover:(ssl://172.16.0.11:61616,ssl://172.16.0.12:61616)?maxReconnectDelay=2000
On the transport connector in ActiveMQ we have enabled the option "rebalanceClusterClients":
<transportConnector name="openwire" uri="ssl://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600" rebalanceClusterClients="true">
<publishedAddressPolicy>
<publishedAddressPolicy publishedHostStrategy="IPADDRESS" />
</publishedAddressPolicy>
</transportConnector>
However, all of the clients are connecting to the first broker in the list of brokers instead of some of them being rebalanced to the second broker.
Previously we did not use the Spring JMS implementation, but instead we used the ActiveMQ libraries directly. This implementation did allow rebalancing the connected clients.
Is something in Spring preventing the rebalancing? Perhaps the CachingConnectionFactory?
EDIT 2019-07-10
I've found these two (p1 and p2) posts on SO where it is stated that CachingConnectionFactory doesn't play nicely with the failover-protocol. However, I think this has been resolved since then as we do see the connections moving between brokers if a broker is turned off.
What we do not see is connections being balanced across brokers. We did see this behavior when we were still using our own custom JMS implementation. So might it be something in Spring or the JmsTemplate?
The actual problem was not ActiveMQ or Spring rather an external firewall prevented this from working.
I am using Spring, Spring-Websocket, STOMP for my application, and RabbitMQ as broker. I need to log all messages going through RabbitMQ to Postgresql tables.
I know that I can write #MessageMapping in Spring and log there, but my problem is that some clients talk to RabbitMQ directly through MQTT protocol, and Spring does not support it yet (https://jira.spring.io/browse/SPR-12581). Moreover browser clients talk through Spring to RabbitMQ using STOMP protocol.
RabbitMQ allows to track all messages using Firehose tracer. How to properly listen to amq.rabbitmq.trace topic from Spring? Or do I need to write separate Java app as consumer?
The Spring AMQP is for you!
You bind some custom queue to to that amq.rabbitmq.trace with appropriate pattern (e.g. publish.#) and configure SimpleMessageListenerContainer to receive messages from that queue.
It can be done even with pretty simple config: #EnableRabbit and #RabbitListener on some POJO method. Anyway the Binding #Bean must be there to attache your queue to that exchange.
An exclusive consumer in Activemq is one that is sent every message from a broker until the consumer dies or goes away, at which time the broker switches consumer.
What is it that defines when the switchover takes place? How do you configure this in Spring JMS/ActiveMQ?
It's not Spring JMS doing the checking; it's the JMS provider, ActiveMQ.
JMS is an API specification; an empty framework, essentially. ActiveMQ provides the implementation backing for managing connections, message brokering, load-balancing, fail-over, etc.
The ActiveMQ broker handles switching-over consumers based on queue properties (you don't need to do anything special in your code):
queue = new ActiveMQQueue("TEST.QUEUE?consumer.exclusive=true");
The switch-over takes place when either the consumer disconnects gracefully or the broker determines that the consumer has disappeared (via the wireFormat.maxInactivityDuration elapsing without any messages or keep-alives being received). You don't have to configure anything if you're happy with the default value of wireFormat.maxInactivityDuration (30 seconds), but you can tweak that if you want to change how long it takes before the broker gives up on a client.
Is there a way catch that time-out and send it back to user?
Not sure if Apache Camel provides something out of the box.
I want to test and alert the user which queues are up and running. So that it will be clear to user that which functionality of the application will work or not.
I am using both Spring and Apache Camel in my project.
Ah! I found out. Camel has an out of the box solution.
Camel 2.1: Specifies whether to test the connection on startup. This ensures that when Camel starts that all the JMS consumers have a valid connection to the JMS broker. If a connection cannot be granted then Camel throws an exception on startup. This ensures that Camel is not started with failed connections. From Camel 2.8 onwards also the JMS producers is tested as well.