To process a large number of messages coming to a queue i need guarantee of at least one jms connection to be there at any time. I am using spring and spring allows to have multiple sessions on a single connection only. In case one and only connection fails, application will come to standstill till spring reconnects to the JMS bridge.
So how can i create more than one connection to a queue in Spring, also how can i do connection pooling here.
The answer to this depends on whether you are using Spring inside a J2EE container(jboss etc.) or in a standalone application.
Standalone - you'll find pooling connections to be a problem. Springs SingleConnectionFactory can be setup to renew the connection on an exception garaunteeing that at some point a connection will come online and start processing the queue again, but you'll still have the problem of waiting for that single connection to renew, plus depending on what messaging implementation your dealing with and how it does load balancing you may find yourself stuck with a connection to a single node in a cluster.
If you are running in a container you can rely on the containers connection factory which will be much more robust. JBoss Messaging in the container for instance will failover seamlessly to other nodes and handles pooling under the covers, but if your working in the container its usually easier to bail on JMS template which kind of sucks and use whatever that container provides.
Related
I am writing a service with Spring and I am using Spring AMQP in order to connect to Rabbitmq.
I have two rabbitmq clusters, one is only for publishing messages(the messages are sent to the other cluster via the federation plugin) and the other cluster is for declaring queues that end users will consume from.
The nodes sit behind aws lb, each cluster has a lb.
I am using CachingConnectionFactory and RabbitTemplate,RabbitAdmin in my code and I want to have connections to all the nodes so I can use them.
For the cluster that will contain the queues I added to the config the queue-master-locator=random so new queues will be declared in all the nodes in the cluster even if my service does not have a connection to them.
With the cluster that publishes messages I have more of a problem because I need a direct connection in my service to each of the nodes so I will be able to separate the load between the nodes.
So my problem is, how do I create connections in my service to all the nodes in the cluster so they will all be used for declaring queues and sending messages?
Now, after I will have some sort of solution to this issue, the next issue will be what happens when a new node is added to the cluster? How can I create a connection to it and start using it as well?
I am using Rabbitmq - 3.7.9, Spring - 2.0.5, Spring AMQP - 2.0.5
Thanks alot!
There is currently no mechanism to do anything like that.
By default, Spring AMQP opens only one connection (optionally two, one for publishing, one for consuming).
Even when using CacheMode.CONNECTION, you'll get a new connection for each consumer (and connections will be created and cached on demand for producers), you won't get any control as to which node it connects to; that's a function of the LB.
The framework does provide the LocalizedQueueConnectionFactory which will try to consume from the node that hosts a queue, but it won't work with a load balancer in place.
In general, however, such optimization is rarely needed.
Are you trying to solve an actual problem you are experiencing now, or something that you perceive that might be a problem?
It is generally best not to perform premature optimization.
Our JMS Listener application connects to an ActiveMQ network of brokers through a load balancer, which we are told distributes connections amongst brokers in a round-robin fashion. Our spring boot application is creating a connection via the load balancer, which in turn feeds the connection to one of the brokers amongst the network of brokers. If a message is published to the brokers then it would be a lot quicker if the message was on the broker that the JMS listener connection lived on. However, the likelihood of that occurring is slim unless we can distribute the connections across the brokers.
I've tried increasing the concurrency in the DefaultJmsListenerContainerFactory, but that didn't do the trick. I was thinking about somehow extending the AbstractJmsListenerContainerFactory, and somehow create a Map of DefaultMessageListenerContainer instances but it looks like the createListenerContainer will only return an instance of whatever is parameterized in the AbstractJmsListenerContainerFactory and we cannot parameterize it with an instance of Map.
We are using Spring Boot 1.5.14.RELEASE.
== UPDATE ==
I've been playing around with the classes above, and it seems like it is inherent in Spring JMS that a Jms Listener be associated with a Single Message Listener Container, which in turn is associated with a single (potentially shared) connection.
For any folks that have JMS Application Listeners that are connecting to a load balanced network of brokers, are you creating a single connection that is connecting to a single broker, and if so, do you experience significant performance degradation as a result of the network of brokers having to move any inbound messages to a broker with consumers?
I'm trying to use Spring JMS messaging with #JmsListener in a scalable way, but I'm not seeing it happening. I have a ConnectionFactory bean that returns a factory that connects to an Oracle Advanced Queue through JMS and a database DataSource pool.
The problem starts as every #JmsListener receiver connects again to JMS (and hence to the database pool). My understand is that I can have many #JmsListener methods, one for each service, but in this way it's doing I'm very limited.
The shared connection is turned on, but since each #JmsListener creates a different DefaultMessageListenerContainer, each one have a database connection.
If I also want the services to handle messages concurrently and set container.setConcurrency("3-5"), then it opens 3 * numberOfListeners connections.
If I use container.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE) then from each second every listener container connects and disconnects from the JMS/database.
I want something that connects one time (or more, if there is concurrent jobs to process) to JMS/database, not to connect count-of-listener times nor to connect-disconnect at each second for every listener.
You can use a SingleConnectionFactory to wrap the vendor factory and all containers will use the same connection.
Is there an equivalent to the ActiveMQ 5 PooledConnectionFactory for Artemis? Why is it available in one and not the other?
Spring, for example, offers a generic CachingConnectionFactory. This is great, but it implements the SingleConnectionFactory and only "pools" one connection.
It would be key to have a similar mechanism in the Artemis client that actually pooled greater than one connections.
Another thought is that maybe it's not implemented because a single connection supports concurrent sessions! I haven't tested the performance of a using a new connection per session. Is the performance the same or similar?
The PooledConnectionFactory in the ActiveMQ 5.x code-base is generic and can actually be used with ActiveMQ Artemis so there was no reason to port it into the Artemis code-base. That said, the JMS connection pool implementation has been pulled out of the ActiveMQ 5.x code-base, cleaned up, modified to support JMS 2, and made available here.
I'm not clear on what you mean by "concurrent sessions." Do you mean that the connection supports concurrently creating session or that the sessions themselves support concurrent use? The former is supported, but the latter is not.
In terms of performance, you'd have to benchmark your specific use-case. There are too many variables to simply say one is better than the other.
My application stack consists of Spring MVC, Hibernate and MySQL hosted on Apache tomcat 7.
I have set up Spring to manage transactions and Hibernate session factory is utilizing the tomcat dbcp connection pool backed datasource for getting the connection.
I have a use case in my application in which I have a run a long running task which is initiated through the web UI (say a button click). This task runs for let’s say 10 minutes then my connection pool starts to throw connection closed exceptions. This is obviously because of connection pool setting in which if the connection is not returned to pool after a specific time, it is marked as abandoned and later removed. I could solve this by tinkering with the timeout settings and increasing it to a large enough value. But I may have several other use cases like this and may not currently have idea how long those will run.
So I am thinking of another approach here.
This use case will be initiated not very often, so I may use a separate datasource definition without using connection pool. Of course I can set two transaction managers in Spring with different names “abc” and “xyz” and use the #Transactional(name=”abc”) and #Transactional(name=”xyz)”. Both these transaction managers would use their respective datasources – one with connection pool to support common use cases and one without connection pool to support long running transaction. This way I won’t have to worry about changing the timeout configurations.
Will this be a generally accepted solution or should I take the timeout configuration approach?
Avoiding to use the connection pool will cause problems if you don't have another way to limit the number of connections that your application can initiate. For example (trivial example of cours) if your going to launch your batch process each time a user clicks a button, make sure you limit the times they can do this task.
Another way would be to define a new jdbc resource in your application server (jdbc/batchprocess) and configure in this resource a longer timeout. Then change from one to another using dynamic datasource routing.
You can open Hibernate Sessions, supplying your own Connection:
sessionFactory.withOptions().connection( yourConnection ).openSession();