Enabling Priority with activemq using camel - spring

After trying to activate priority with activemq, I was told to try using camel (see here). But I can't get it working, and I'm not even sure how it should work.
I've added the following code, in my spring configuration:
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="jms:queue:myqueue" />
<resequence>
<batch-config batchSize="200" batchTimeout="3000" allowDuplicates="true" reverse="true"/>
<header>JMSPriority</header>
<to uri="mock:result"/>
</resequence>
</route>
</camelContext>
<bean id="jmsConfig" class="org.apache.camel.component.activemq.ActiveMQConfiguration">
<property name="connectionFactory" ref="pooledConnectionFactory"/>
<property name="transacted" value="false"/>
<property name="concurrentConsumers" value="10"/>
</bean>
<bean id="activemq" class="org.apache.camel.component.activemq.ActiveMQComponent">
<property name="configuration" ref="jmsConfig"/>
</bean>
But this configuration doesn't work (nothing is ordered), and I have a few questions:
What does mock:result mean? I couldn't find it in the documentation.
How is camel supposed to re-order the queue, is it done after the messages were created, or when a message is added?
Can it be independent from the spring activemq basic configuration? (I use here the camel ActiveMQComponent)

First, a mock endpoint is for unit testing. You can use it to validate that the expected messages were received:
MockEndpoint resultEndpoint = context.resolveEndpoint("mock:result", MockEndpoint.class);
resultEndpoint.expectedMessageCount(3);
resultEndpoint.expectedBodiesReceived("firstMessageBody", "secondMessageBody", "thirdMessageBody");
resultEndpoint.message(0).header("foo").isEqualTo("bar");
Next, the resequencer is designed to order messages based on some attribute (the header "JMSPriority" in your case) and will perform this over all messages that flow through it over a given timeout period or batch size (by default, the batch size is 100 and the timeout is 1000 ms).
Long story short, you can use the resequencer to order messages (in batches) before they are sent to the queue, in between queues or in between a queue and a consumer...

Related

JMS from Synchronous to Asynchronous

I was able to create a Synchronous service bus using the JMS, but I was not able to turn it to Asynchronous.
I'm trying to post a request to a service asynchronously, so if the service is down, I want the JMS queue keeps the request message and when the service starts up it delivers the message to the service and get back the response.
here is my code
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route >
<from id ="server" uri="jetty:http://0.0.0.0:9500/rsb/toService?matchOnUriPrefix=true&enableMultipartFilter=false&disableStreamCache=false"/>
<wireTap uri="log:test?level=INFO"><body><simple>Enter JMS test route and sending message to queue</simple></body></wireTap>
<!--<to uri="direct:queue"/>-->
<to uri="jms://testqueue?requestTimeout=360000&replyTo=bar&replyToDeliveryPersistent=true&exchangePattern=InOut&acknowledgementModeName=AUTO_ACKNOWLEDGE"/>
</route>
<route id="testqueuelistener" streamCache="true">
<from uri="jms://testqueue?replyToDeliveryPersistent=true&replyTo=bar" />
<wireTap uri="log:test?level=INFO"><body><simple>Message recieved from queue: ${body}</simple></body></wireTap>
<to uri="http://localhost:18402/Home/addUser?bridgeEndpoint=true&throwExceptionOnFailure=false"/>
<to uri="jms:service"/>
</route>
<route >
<from uri="jms:service"/>
<transform>
<simple>${body}</simple>
</transform>
</route>
</camelContext>
The issue is that you are accessing the JMS queue without using a transaction - so as soon as you get the message, it's gone from the queue. You need to use a transaction and only commit (or rollback) the message consumption after you've finished processing it.
The relevant Enterprise Integration Pattern is the transactional client. The JMS component documentation also provides some information about transaction. Finally, chapter 9 of Camel in Action (chapter 12 for the second edition) is dedicated to transactions (and I can't recommend it enough!).
You need to:
Obtain a JMS transaction manager (which transaction manager you use may depend on your specific use case)
Configure the Camel JMS component to use the transaction manager
Use a transaction policy to configure the transactional behavior of your route (or just mark the route as transacted, and use the default policy)
The configuration can look something like:
<!-- Import JMS connection factory -->
<osgi:reference id="jmsConnectionPool" interface="javax.jms.ConnectionFactory" />
<!-- We create a Spring JmsTransactionManager (our transaction manager could also be an
imported OSGi service, like we do for the connection factory; for example an XA transaction
manager) -->
<bean id="jmsTxManager" class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="jmsConnectionPool"/>
</bean>
<!-- We configure the JMS component to use the transaction manager-->
<bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="jmsConnectionPool" />
<property name="transacted" value="true"/>
<property name="transactionManager" ref="jmsTxManager"/>
</bean>
<!-- Here's an example of a transaction policy -->
<bean id="requiresNew" class="org.apache.camel.spring.spi.SpringTransactionPolicy">
<property name="transactionManager" ref="jtaTransactionManager"/>
<property name="propagationBehaviorName" value="PROPAGATION_REQUIRES_NEW"/>
</bean>
Here's a transacted route:
<route id="myRoute">
<from uri="jms://..." />
<transacted/>
...
</route>
And a route can use a specific transaction policy if we want:
<route id="myRoute">
<from uri="jms://..." />
<transacted ref="requiresNew" />
...
</route>

Solace JMS didn't run with parallel threads in Camel

In order to perform parallel processing of jms messages, I have configured the JmsComponent and connectionFactory as below.
After reading some posts and the official tutorial, seems that the below configuration should work for ActiveMQ. However, my testing shows that it doesn't work on Solace. Can someone give me a hint on this? Thanks.
// Route Definition - Camel Java DSL
from(INBOUND_ENDPOINT).setExchangePattern(ExchangePattern.InOnly).threads(5).bean(ThroughputMeasurer.class);
<!-- JMS Config -->
<bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="cachedConnectionFactory" />
<property name="acknowledgementModeName" value="AUTO_ACKNOWLEDGE" />
<property name="deliveryPersistent" value="false" />
<property name="asyncConsumer" value="true" />
<property name="concurrentConsumers" value="5" />
</bean>
<!-- jndiTemplate is omitted here -->
<bean id="connectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate" ref="jndiTemplate" />
<property name="jndiName" value="ceConnectionFactory" />
</bean>
<bean id="cachedConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="connectionFactory" />
<property name="sessionCacheSize" value="30" />
</bean>
I believe the problem here is that your consumer is bound to an exclusive queue, and only 1 consumer is allowed to process message. Binding to an non-exclusive queue should solve the problem.
Exclusive queues only allow the first consumer to consume from the queue.
All other consumers that are bound to the queue will not receive data. In the event that the first consumer disconnects, the next oldest consumer will begin receiving data. The other consumers can be thought of as being "standby" consumers that will take over the moment the first consumer disconnects.
Non-exclusive queues are used for load balancing. Messages that are spooled to the queue will be distributed to all consumers in a round-robin manner.
Check:
Are there any exception messages in the log?
Is the jndiName correct? Perhaps it should be jms/ceConnectionFactory?
Is the URI INBOUND_ENDPOINT correct?
...
Try to setup ActiveMQ first, and migrate the configuration to Solace.

How to create multiple listeners in ActiveMQ using spring JMS

I have a use case where I want to create multiple listeners(6) in an application. I want to subscribe to multiple destinations(6 topics). All the subscription are durable. I am using separate default message listener container(DMLC) for each listener and using different client id but I am confused about how connection factory should be used.
Should I use single ActiveMQ pooled connection factory with maxConnection specified to 6. OR should I use different pooled connection factory for each listener ?
Is there any harm is using pooledConnectionFactory with maxConnection for durable subscriber?
Source code:
Connection Factory:
<bean id="jmsFactory" class="org.apache.activemq.pool.PooledConnectionFactory"
destroy-method="stop">
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>${jms.broker.url}</value>
</property>
</bean>
</property>
<property name="maxConnections" value="6" />
and my listener is using it as: (I have 6 similar listeners similar to this using different destinations and client ID)
<bean id="listenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer"
<property name="connectionFactory" ref="jmsFactory" />
<property name="destination" ref="topic_pnlCompleteTopic" />
<property name="durableSubscriptionName" value="FAGCompletion" />
<property name="pubSubDomain" value="true" />
<property name="subscriptionDurable" value="${jms.fagsListener.durable}" />
<property name="clientId" value="${jms.fagsListener.clientId}" />
<property name="messageListener" ref="pnlMessageListener" />
<property name="messageSelector" value="JMSType = 'FAG Completion'" />
</bean>
That sounds all fine to me as long as you're not using this connection factory for something else. There is no reason to limit the number of connections to 6, you can put a higher number if you want and it will be only used if necessary. ConnectionFactory is typically the thing that you share for your whole app. See it as a kind of DataSource for JMS access
You listeners will typically only use one session each, since you are using topics. There is no reason to specify a limit on your pool, or to use multiple pools. You typically use one pooled connection factory for your application unless you see a real reason to limit or split it up into different pools.

Huge latency observed while committing transacted persistent AMQ messages

I have the following AMQ consumer configuration file which tries to consume 'persistent' messages from the queue. Also, the messages are 'transacted' (as I need to rollback if a message can't be processed in an expected way).
I see a problem with this configuration:
Whenever the consumer calls session.commit() after consuming the message, I see the commit call taking ~8 seconds to come out. I believe this is not expected.
Can someone point me if I have any issues with the below config?
<bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
</bean>
<bean id="simpleMessageListener" class="listener.StompSpringListener" scope="prototype" />
<bean id="destination" class="org.apache.activemq.command.ActiveMQQueue">
<property name="physicalName" value="JOBS.notifications"/>
</bean>
<bean id="poolMessageListener" class="org.springframework.aop.target.CommonsPoolTargetSource">
<property name="targetBeanName" value="simpleMessageListener"/>
<property name="maxSize" value="200"/>
</bean>
<bean id="messageListenerBean" class="org.springframework.aop.framework.ProxyFactoryBean">
<property name="targetSource" ref="poolMessageListener"/>
</bean>
<jms:listener-container
container-type="default"
connection-factory="amqConnectionFactory"
acknowledge="transacted"
concurrency="200-200"
cache="consumer"
prefetch="10">
<jms:listener destination="JOBS.notifications" ref="messageListenerBean" method="onMessage"/>
</jms:listener-container>
Take a look at the the ActiveMQ Spring support page. What I can see immediately is that you should be using a PoolingConnectionFactory instead of an ActiveMQConnectionFactory. This will ensure that only a single TCP connection is set up to the broker from your code, and will be shared between polling threads.

Losing messages from a topic with DefaultMessageListenerContainer

I'm using a DefaultMessageListenerContainer to consume messages from a topic (Broker is ActiveMQ). Because the consumers are created at runtime i'm doing the following:
1) I have a Conainer Template configured in spring
<bean id="topiccontainertemplate" class="org.springframework.jms.listener.DefaultMessageListenerContainer" scope="prototype" destroy-method="stop">
<property name="connectionFactory" ref="connectionfactory" />
<property name="pubSubDomain" value="true" />
<property name="cacheLevelName" value="CACHE_CONSUMER" />
<property name="destinationName" value="default" />
</bean>
2) When a new consumer is required i get a new one from the application conext and reconfigure the destinationName.
DefaultMessageListenerContainer container = context.getBean("topiccontainertemplate", DefaultMessageListenerContainer.class);
container.setDestinationName(localEntity.getId().getDestination());
container.setMessageListener(getListener());
container.start();
Unfortunately the container misses some messages on the topic.
Does anybody know what i'm doing wrong?
Your subscription does not look durable. If it was, messages would have been saved while your sub is offline/ starting up. Your subscriber will get msgs from the point it is started completly - msgs sent prior to that will be lost.
after some more tests we found a race condition during consumer creation. the messages weren't lost, they weren't distributed properly in our code.

Resources