I have a app server that runs multiple camel routes that are reading messages from a JMS Queue, each route runs with different selectors.
We have an app server for each group of clients. So at the end there are multiple connections created to the queue. And that seems to be affecting the read and write performance on the queue.
I tried to use a connection pool, (by the way i am using WMQ), the closest I got is to use spring's CachingConnectionFactory (as described in this post how to configure (spring) JMS connection Pool for WMQ).
I was hoping that the number of connections will reduce from each app server, but that did not happen. Even though i set the size to 5 on the CCFactory, i see 10 connections created to the queue. According to the documentation on CCFactory, explicit closing of the connections is required. Is it possible or does it even make sense to close the connection that is used by a camel route that is reading from a queue? Or is there a way to tell camel routes to share the connections? I thought the use of connection pool and factory will do that, but i don't see that from happening.
Can I achieve what I want to achieve?
Thanks in advance.
what you can do is have only one route reading from the queue instead of many, and then use some camel conditional logic to redirect the message to the correct processing route via direct endpoints, see also this example.
This way there would only one consumer for the queue, instead of multiple consumers each running it's own selector.
Related
I am trying to achieve a result using Spring. In my case; I need to connect to a queue on n (~6) different hosts/ports. Say,
jms://hostA:portA/Queue
jms://hostB:portB/Queue
jms://hostC:portC/Queue
...
I am completely lost on where to start now. Reading tons of documents and it broke my whole motivation.
p.s; All the examples and documentations are mostly focused on ActiveMQ which in my case is not valid.
I do not need any code sample but a light on my way to start from. Like a some class names to check etc..
The JMS provider is irrelevant.
You need a connection factory bean for each host and configure each component (JMS template, listener container, etc) to use the appropriate connection factory.
So using JMS and ActiveMQ, I can be sure that my message sent from my Spring Boot application using JmsTemplate will reach it's destination application even if that destination application is down at the time I send the message to ActiveMQ. As when the destination application starts up, it grabs the message from the queue. Great!
However.
What happens if my Spring Boot application tries to send a JMS message to a queue on the ActiveMQ server, but the ActiveMQ server is down at that point or the network is down and I get a connection refused exception?
What is the recommended way to make sure my application keeps trying to re-sends the message to ActiveMQ until it is successful? Is this something I have to develop into my application myself? Are there any nifty Spring tools or annotations which do this for me? Any advice on best practice or how I should be handling this scenario?
You can try Spring-Retry. Has lots of fine grain controls for it:
http://www.baeldung.com/spring-retry
https://github.com/spring-projects/spring-retry
If it is critical that you don't lose this message, you will want to save it to some alternative persistent store (e.g. filesystem, local mq server) along with whatever retry code you come up with. But for those occasional network glitches or a very temporary mq shutdown/restart, Spring-Retry alone should do the trick.
Couple of approaches I can think of
1. You can set up another ActiveMq as fallback. In your code you don't have to do anything, just change your broker url from
activemq.broker.url=tcp://amq01.blah.blah.com:61616
to
activemq.broker.url=failover:(tcp://amq01.blah.blah.com:61616,tcp://amq02.blah.blah.com:61616)?randomize=false
The rest is automatically taken care of. i.e. when one of them is down, the messages are sent to other.
Another approach is to send to a internal queue (like seda, direct) when activemq is down and read from there.
Adding failover to the url is one appropriate way.
And another reasonable way is to making sure activemq always online , as activemq has the master-slave mode(http://activemq.apache.org/masterslave.html) to get high availability.
I've managed to get Spring Xd working for a scenario where I have data coming in from one JMS broker.
I potentially am facing a scenario where data ingestion could happen from different sources thereby needing me to connect to different brokers.
Based on my current understanding, I'm not quite sure how to do this as there exists a JMS config file which allows you to setup only one broker.
Is there a workaround to this?
At the moment, you would have to create a separate jms-[provider]-infrastructure-context.xml for each broker (in modules/common), say call the provider activemq2.
Then use --provider=activemq2 in the module definition.
(I recently used this technique to test sonicmq and hornetq providers).
I am new to JMS and currently developing a simple Chat application explained in Oreilly 'Java Message service'. I've configured a TopicConnectionFactory in ActiveMQ that receives chat messages from TopicPublishers and dispatch that to TopicSubscribers.
My question is 'why do we need to create more than one TopicConnectionFactory' in any JMS application? Since Connectionfactory instances are not tied up with a Topic/Queue, why can't we use one instance of ConnectionFactory for creating connections to all the Topics (or Queues) configured in an application?
Technically speaking you are right. You may be able just to use one ConnectionFactory.
However it is a better design to use multiple ConnectionFactories depending on your requirements so the traffic would be spread out evenly and you do not run out of connections.
So if you know about a JMS Client application that may be problematic (the logic does not allow proper connection handling open/close), you may isolate it to use its own connection factory.
Also some connection factories allow a pool of 10 default active connections at the same time (it depends on the implementation/ settings) if you will need more you may use more than one connection factory.
I've configured a TopicConnectionFactory in ActiveMQ that receives chat messages from TopicPublishers and dispatch that to TopicSubscribers.
Very ambiguous statement. TopicConnectionFactory does not receive or send any messages. It is just one of the admin Objects used to create Connection which in turn creates Session which in turn creates your publishers and subscribers which publish and subscribe the messages.
why can't we use one instance of ConnectionFactory for creating connections to all the Topics (or Queues) configured in an application?
You definitely can. There is no one stopping you from doing that.
As per specs
A connection factory object encapsulates a set of connection configuration
parameters that has been defined by an administrator. A client uses it to
create a connection with a JMS provider.
So unless you have different configuration requirements you can use same ConnectionFactory to create multiple Connections. And yes as otc has mentioned above number of connections is one of the configuration parameter.
I'm new to ActiveMQ. I'm using it (and Apache Camel) for batch processing that ends up communicating with web services.
My question is how does ActiveMQ control how asynchronous it really is? In other words, if it can process 20 messages at the same time but the bottleneck is the web services on the other end, how can I control that? Can I slow ActiveMQ down?
Thanks!
if you're using apache camel 2.4+, you can use the throttler with camel to control message flow to endpoints - you can change the limit dynamically as of camel 2.8 - hope it helps.
The Camel throttler would be a good solution. It does however keep the messages in memory in the Camel application at the time. However it allows to react precisely.
Another alternative is the throttling in-flight route policy you can configure on the Camel ActiveMQ JMS consumer. That policy can be configured with a upper/lower water mark settings. Then the policy will automatic suspend/resume the AMQ consumer accordingly. You can extend this logic and use your custom metrics instead.
You can read about this policy here: http://camel.apache.org/routepolicy
This approach would then not keep any messages in memory, as it will control on the AMQ consumer side to suspend/resume it to "throttle" the flow.
You could also set a pre-fetch limit, preventing AMQ from dispatching messages on a specific transport to its consumers based on the amount of message_delivered responses it gets back from the client. Here is a reference