We are currently writing a library that consumes rabbitmq events with spring-amqp.
This library needs to be used from some applications that themselves use rabbitmq with spring-amqp.
Is it possible to isolate the separate RabbitMQ Configurations from each other, so that the configurations form within the library dont interfere with the existing ones in the applications?
both would connect to the same rabbitmq cluster.
I looked through the documentation of spring-amqp but only found a way to split the rabbit configuration for consuming and producing events.
Since spring-amqp 2.3 there's a Multiple Broker (or Cluster) Support which could be used to create multiple connections to the same broker. You can find a sample config at this link.
Also, you can take a look at the spring-multirabbit library (https://github.com/freenowtech/spring-multirabbit) which is actually the ancestor to that feature in spring-amqp and can be used to add multiple RabbitMq connections support to a service that already has a Spring-configured connection in a non-intrusive way.
Related
We have configured our ActiveMQ message broker as a Spring Boot project and there's another Spring Boot application (let's call it service-A) that has a listener configured to listen to some topics using #JmsListener annotation. It's a Spring Cloud microservice appilcation.
The problem:
It is possible that service-A can have multiple instances running.
If we have 2 instances running, then any message coming on topic gets listened to twice.
How can we avoid every instance listening to the topic?
We want to make sure that the topic is listened to only once no matte the number of service-A instances.
Is it possible to run the microservice in a cluster mode or something similar? I also checked out ActiveMQ virtual destinations but not too sure if that's the solution to the problem.
We have also thought of an approach where we can decide who's the leader node from the multiple instances, but that's the last resort and we are looking for a cleaner approach.
Any useful pointers, references are welcome.
What you really want is a shared topic subscription which was added in JMS 2. Unfortunately ActiveMQ 5.x doesn't support JMS 2. However, ActiveMQ Artemis does.
ActiveMQ Artemis is the next generation broker from ActiveMQ. It supports most of the same features as ActiveMQ 5.x (including full support for OpenWire clients) as well as many other features that 5.x doesn't support (e.g. JMS 2, shared-nothing high-availability using replication, last-value queues, ring queues, metrics plugins for integration with tools like Prometheus, duplicate message detection, etc.). Furthermore, ActiveMQ Artemis is built on a high-performance, non-blocking core which means scalability is much better as well.
We are designing a solution that will consume messages from IBM MQ using JMS. The plan is to use WAS Liberty, so JMS is the technology of choice. We will create Message-Drive beans that will listen for messages in MQ queues.
We are considering both WAS Liberty and OpenLiberty as well.
The trick here is that we must implement it with fail-over, so that if one of our server fail, the other will keep consuming messages from MQ automatically. Like in a ative/passive mechanism.
I'm aware that the MQ adapter needs to be installed as it is not provided out-of-the-box.
I have the following questions:
Does WAS Liberty messaging implementation supports fail-over? Meaning that if the ative message consumer node fails, the stand-by node will automatically migrate and start consuming messages from MQ? What about OpenLiberty?
How can I configure the message system to work that way? Can you point out to the documentation?
Or is this feature only provided by WebSphere?
There is no such functionality in WebSphere Liberty or Open Liberty yet. You can create RFE here https://www.ibm.com/developerworks/rfe/?PROD_ID=544 .
There are ways to do it manually, check these links:
JMS Activation spec on Liberty: “WAS_EndpointInitialState” full profile equivalent property?
Controlling the state of endpoints at runtime
Solution that you could do:
create a script/application that will monitor your servers and call that API to enable/disable endpoint in specific server
or use Dynamic cluster/ auto scaling feature of Liberty and divide you app to two clusters - one with MDBs, one without. And then define policy that MDBs cluster has 1 instance always available. So once the server dies it is automatically restarted somewhere in the cluster
or use Kubernetes/ICP platform in the same way - so deploying 2 versions of app, and defining different replicasets parameters.
I am writing a service with Spring and I am using Spring AMQP in order to connect to Rabbitmq.
I have two rabbitmq clusters, one is only for publishing messages(the messages are sent to the other cluster via the federation plugin) and the other cluster is for declaring queues that end users will consume from.
The nodes sit behind aws lb, each cluster has a lb.
I am using CachingConnectionFactory and RabbitTemplate,RabbitAdmin in my code and I want to have connections to all the nodes so I can use them.
For the cluster that will contain the queues I added to the config the queue-master-locator=random so new queues will be declared in all the nodes in the cluster even if my service does not have a connection to them.
With the cluster that publishes messages I have more of a problem because I need a direct connection in my service to each of the nodes so I will be able to separate the load between the nodes.
So my problem is, how do I create connections in my service to all the nodes in the cluster so they will all be used for declaring queues and sending messages?
Now, after I will have some sort of solution to this issue, the next issue will be what happens when a new node is added to the cluster? How can I create a connection to it and start using it as well?
I am using Rabbitmq - 3.7.9, Spring - 2.0.5, Spring AMQP - 2.0.5
Thanks alot!
There is currently no mechanism to do anything like that.
By default, Spring AMQP opens only one connection (optionally two, one for publishing, one for consuming).
Even when using CacheMode.CONNECTION, you'll get a new connection for each consumer (and connections will be created and cached on demand for producers), you won't get any control as to which node it connects to; that's a function of the LB.
The framework does provide the LocalizedQueueConnectionFactory which will try to consume from the node that hosts a queue, but it won't work with a load balancer in place.
In general, however, such optimization is rarely needed.
Are you trying to solve an actual problem you are experiencing now, or something that you perceive that might be a problem?
It is generally best not to perform premature optimization.
I want to know the major difference between VM and JMS component of Mule ESB. Can someone help me to know it.
As per Mule documentation, VM transport is for intra-JVM communication between Mule flows. So, that means when you use a VM in your flow, you can communicate between different flows in the application.
A flow containing VM inbound cannot be called externally from external application as thus the flow is equivalent to a private flow used within the application. By default uses in-memory queues.
Please go through the documentation :- https://docs.mulesoft.com/mule-user-guide/v/3.8/vm-transport-reference
On the other hand as per Mule documentation, JMS is an external host, allows communication between different components of a distributed application and JMS transport lets you easily send and receive messages to queues and topics for any message service which implements the JMS specification.
A flow, which has JMS inbound can be called from externally unlike VM. Documentation is here :- https://docs.mulesoft.com/mule-user-guide/v/3.8/jms-transport-reference
Within the application, if you send the control from one flow to another flow we use VM.VM can be used as both inbound and outbound.
Outside the application, for example, A application want to send something to B application(external application) there we use JMS.
I've managed to get Spring Xd working for a scenario where I have data coming in from one JMS broker.
I potentially am facing a scenario where data ingestion could happen from different sources thereby needing me to connect to different brokers.
Based on my current understanding, I'm not quite sure how to do this as there exists a JMS config file which allows you to setup only one broker.
Is there a workaround to this?
At the moment, you would have to create a separate jms-[provider]-infrastructure-context.xml for each broker (in modules/common), say call the provider activemq2.
Then use --provider=activemq2 in the module definition.
(I recently used this technique to test sonicmq and hornetq providers).