Why is Spring JMS creating a JMS connection every second when connecting to an ActiveMQ Broker? - spring

I've created a Spring JMS application using version 4.1.2.RELEASE, which is connected to a broker that is running ActiveMQ 5.11.0. The problem that I'm seeing is as follows. In the logs, I notice that every second, I'm seeing a connection being created as such.
2017-06-21 13:10:21,046 | level=INFO | thread=ActiveMQ Task-1 | class=org.apache.activemq.transport.failover.FailoverTransport | Successfully connected to tcp://localhost:61616
I know that it is creating a new ActiveMQ connection each time, because it says successfully "connected" and not "reconnected" as shown in the code located here: http://grepcode.com/file/repo1.maven.org/maven2/com.ning/metrics.collector/1.3.3/org/apache/activemq/transport/failover/FailoverTransport.java#891
I don't have a caching connection factory set for my consumer, but I'm wondering if the following is the culprit when it comes to why I'm seeing constant connections being created.
factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE);
The following post states that consumers should not be cached, but I wonder if that applies to caching the connection + session. If the connection is cached, but the session is not, then I wonder if that creates a problem.
Why DefaultMessageListenerContainer should not use CachingConnectionFactory?
The following are the configurations that I'm using in my application. I am hoping that it is something that I've misconfigured, and would appreciate any insights that anyone has to offer.
Spring Configurations
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() throws Throwable {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE);
factory.setMaxMessagesPerTask(-1);
factory.setConcurrency(1);
factory.setSessionTransacted(true);
return factory;
}
#Bean
public CachingConnectionFactory cachingConnectionFactory(){
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory(connectionFactory());
cachingConnectionFactory.setCacheConsumers(false);
cachingConnectionFactory.setSessionCacheSize(1);
return cachingConnectionFactory;
}
#Bean
public ActiveMQConnectionFactory connectionFactory(){
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setInitialRedeliveryDelay(1000L);
redeliveryPolicy.setRedeliveryDelay(1000L);
redeliveryPolicy.setMaximumRedeliveries(6);
redeliveryPolicy.setUseExponentialBackOff(true);
redeliveryPolicy.setBackOffMultiplier(5);
ActiveMQConnectionFactory activeMQ = new ActiveMQConnectionFactory("admin", "admin", "tcp://localhost:61616");
activeMQ.setRedeliveryPolicy(redeliveryPolicy);
activeMQ.setPrefetchPolicy(prefetchPolicy());
return activeMQ;
}
#Bean
public JmsMessagingTemplate jmsMessagingTemplate(){
ActiveMQTopic activeMQ = new ActiveMQTopic("topic.out");
JmsMessagingTemplate jmsMessagingTemplate = new JmsMessagingTemplate(cachingConnectionFactory());
jmsMessagingTemplate.setDefaultDestination(activeMQ);
return jmsMessagingTemplate;
}
protected ActiveMQPrefetchPolicy prefetchPolicy(){
ActiveMQPrefetchPolicy prefetchPolicy = new ActiveMQPrefetchPolicy();
int prefetchValue = 1000;
prefetchPolicy.setQueuePrefetch(prefetchValue);
return prefetchPolicy;
}
Thanks,
Juan

The issue was indeed the following code.
factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE);
The moment that I removed it, the rapid connection creation stopped.

Related

When does IBM mq commits a message when we read messages through #JMSListener annotation

I am listening to the queue in my application like this
#JmsListener(containerFactory = "MessageListener", destination = "${mq.destination-name}")
My connection factory Bean looks like this
#Bean(name = "MessageListener")
public JmsListenerContainerFactory<?> mqMessageListenerContainer(
ConnectionFactory connectionFactory, QueueErrorHandler QueueErrorHandler)
{
SimpleJmsListenerContainerFactory simpleJmsListenerContainerFactory = new SimpleJmsListenerContainerFactory();
simpleJmsListenerContainerFactory.setConnectionFactory(connectionFactory);
simpleJmsListenerContainerFactory.setErrorHandler(QueueErrorHandler);
return simpleJmsListenerContainerFactory;
}
I want to understand when I don't have
simpleJmsListenerContainerFactory.setSessionTransacted(true);
simpleJmsListenerContainerFactory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
these set in factory configuration when is my application starts a transaction on reading a message and when does it commits ?

Two queue manager with different channel not allowed

I use spring boot 2.3.2 with ibm mq.
I use property file setup mq.
ibm.mq.queueManager=
ibm.mq.channel=
ibm.mq.connName
ibm.mq.user
ibm.mq.password
ibm.mq.useIBMCipherMappings=false
ibm.mq.userAuthenticationMQCSP=false
ibm.mq.sslCipherSuite=TLS_RSA_WITH_AES_128_CBC_SHA256
that works fine.
I need to create another factory to be able to connect to another channel. So I would like to create a similar one which is created by default but with different channel name.
So I created a config class
#EnableJms
#Configuration
public class JmsConfig {
#Bean
public MQQueueConnectionFactory jmsMQConnectionFactoryPayment() throws JMSException {
MQQueueConnectionFactory connectionFactory = new MQQueueConnectionFactory();
connectionFactory.setQueueManager("AD_TEST");
connectionFactory.setChannel("FROM.PAYMENTMNG");
connectionFactory.setConnectionNameList("wmqd1.int.test.com(1818),wmqd2.int.test.com(1818)");
connectionFactory.setBooleanProperty(WMQConstants.USER_AUTHENTICATION_MQCSP, false);
connectionFactory.setStringProperty(WMQConstants.WMQ_SSL_CIPHER_SUITE, "TLS_RSA_WITH_AES_128_CBC_SHA256");
connectionFactory.setIntProperty(CommonConstants.WMQ_CONNECTION_MODE, CommonConstants.WMQ_CM_CLIENT);
System.setProperty("com.ibm.mq.cfg.useIBMCipherMappings", String.valueOf(Boolean.FALSE));
connectionFactory.createConnection("123", "123");
return connectionFactory;
}
#Bean
JmsListenerContainerFactory<?> jmsContainerFactoryPayment() throws JMSException {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(jmsMQConnectionFactoryPayment());
return factory;
}
#Bean("payment")
JmsTemplate jmsTemplatePayment() throws JMSException {
JmsTemplate template = new JmsTemplate();
template.setConnectionFactory(jmsMQConnectionFactoryPayment());
return template;
}
}
In a class I have
#JmsListener(destination="xxx", containerFactory="jmsContainerFactoryPayment"){
....
}
When I start application, I get
com.ibm.msg.client.jms.DetailedIllegalStateException: JMSWMQ0018: E Failed to connect to queue manager 'AD_TEST' with connection mode 'Client' and host name 'wmqd1.int.test.com(1818),wmqd2.int.test.com(1818)'.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:489) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0 - p920-L200710.DE]
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:215) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0 - p920-L200710.DE]
at com.ibm.msg.client.wmq.internal.WMQConnection.<init>(WMQConnection.java:450) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0 - p920-L200710.DE]
at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:8475) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0
Caused by: com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') ; reason '2538' ('MQRC_HOST_NOT_AVAILABLE').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:203) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0 - p920-L200710.DE]
... 51 common frames omitted
Seem like we can't have two queue manager with same host but different channel
I have configured Spring Boot with IBM MQ to test this. Starting with the sample provided in this IBM Messaging GitHub Repo, I added two additional listeners as follows.
First I added some additional properties to the application.properties file
my.mq.queueManager=QM2
my.mq.channel=DEV.APP.SVRCONN
my.mq.connName=localhost(1415)
my.mq.user=<your_user_name>
my.mq.password=<your_password>
I left Application.java unchanged, copied Listener.java to create ListenerTwo.java and ListenerThree.java. I then added a new ListenerBeanConfig.java class to the sample.
ListenerTwo.java was changed to bind to a new ConnectionFactory configuration listenerTwoFactory which is created later.
#JmsListener(destination = Application.qName, containerFactory = "listenerTwoFactory")
ListenerThree.java was changed to bind to a new listenerThreeFactory configuration and Queue
#JmsListener(destination = "DEV.QUEUE.2", containerFactory = "listenerThreeFactory")
The ListenerBeanConfig.java class declaration was annotated so that I can access my properties by adding Strings for each property e.g., queueManager, channel, connName etc. and providing setter methods for each of them e.g.,
#Configuration
#EnableJms
#ConfigurationProperties(prefix="my.mq")
public class ListenerBeanConfig {
String connName;
public void setConnName(String value) {
System.out.println("connName is set to: "+value);
connName = value;
}
I registered the two new Listener beans
#Bean
public ListenerTwo myListenerTwo() {
return new ListenerTwo();
}
#Bean
public ListenerThree myListenerThree() {
return new ListenerThree();
}
I then created the new connection factory configurations listenerTwoFactory and listenerThreeFactory
For listenerTwoFactory I used the JMS classes provided by com.ibm.mq.jms in the Spring Boot config
JmsConnectionFactory cf;
#Bean
public DefaultJmsListenerContainerFactory listenerTwoFactory() {
DefaultJmsListenerContainerFactory containerFactory = new DefaultJmsListenerContainerFactory();
try {
JmsFactoryFactory ff = JmsFactoryFactory.getInstance(WMQConstants.WMQ_PROVIDER);
cf = ff.createConnectionFactory();
cf.setStringProperty(WMQConstants.WMQ_CONNECTION_NAME_LIST, connName);
cf.setStringProperty(WMQConstants.WMQ_CHANNEL, channel);
cf.setIntProperty(WMQConstants.WMQ_CONNECTION_MODE, WMQConstants.WMQ_CM_CLIENT);
cf.setStringProperty(WMQConstants.WMQ_QUEUE_MANAGER, queueManager);
cf.setStringProperty(WMQConstants.WMQ_APPLICATIONNAME, "Spring Boot ListenerTwo");
cf.setBooleanProperty(WMQConstants.USER_AUTHENTICATION_MQCSP, true);
cf.setStringProperty(WMQConstants.USERID, user);
cf.setStringProperty(WMQConstants.PASSWORD, password);
} catch (JMSException jmsex) {
System.out.println(jmsex);
}
containerFactory.setConnectionFactory(cf);
return containerFactory;
}
For the listenerThreeFactory I used the MQ JMS helper classes from com.ibm.mq.spring.boot.
#Bean
public DefaultJmsListenerContainerFactory listenerThreeFactory() {
MQConfigurationProperties myProps = new MQConfigurationProperties();
myProps.setUser(user);
myProps.setChannel(channel);
myProps.setConnName(connName);
myProps.setPassword(password);
myProps.setQueueManager(queueManager);
//No customizer
MQConnectionFactoryFactory mqcff = new MQConnectionFactoryFactory(myProps,null);
MQConnectionFactory mqcf = mqcff.createConnectionFactory(MQConnectionFactory.class);
DefaultJmsListenerContainerFactory containerFactory = new DefaultJmsListenerContainerFactory();
containerFactory.setConnectionFactory(mqcf);
return containerFactory;
}
Finally, I compiled and ran the new sample configuration. Using the IBM MQ Console for two IBM MQ queue manager docker instances, I put messages to QM1: DEV.QUEUE.1 and QM2: DEV.QUEUE.1, DEV.QUEUE.2. On the terminal see the following output.
========================================
Received message is: message 1
========================================
========================================
ListenerTwo received message is: message 2
========================================
========================================
ListenerThree received message is: message 3
========================================
Also tested with all three listeners connected to QM2 via a two different channels: DEV.APP.SVRCONN and DEV.APP.SVRCONN.TWO.
I am sure there are far more elegant ways to manage the additional properties.

How to put a message back in the queue using latest SpringBoot and ActiveMQ classic

I am designing a simple system where the flow is going to be like this:
Message Producer Microservice --> Active MQ --> Message Consumer Microservice --> Mongo DB
I need to design a queuing strategy in a way so that if MongoDB is down, I should not lose the message (because Message consumer will dequeue the message).
My consumer is written like this:
#JmsListener(destination = "Consumer.myconsumer.VirtualTopic.Tracking")
public void onReceiveFromQueueConsumer2(TrackingRequest trackingRequest) {
log.debug("Received tracking request from the queue by consumer 2");
log.debug(trackingRequest.toString());
}
How do you provide client acknowledgement?
You can use client acknowledge mode from your "Message Consumer Microservice." Since you're using a Spring JmsListener you can define the listener container using the containerFactory and then you can set the mode you want on your listener container using sessionAcknowledgeMode. See the Spring documentation for more details on what ack mode you might want to use here.
From the perspective of the ActiveMQ client you can configure redelivery semantics however you like in case of a failure. See the ActiveMQ documentation for more about that.
Alright, so I was able to solve this dilemma, here is what your config should be like (thanks to Justin for his valuable inputs):
#Bean
public ActiveMQConnectionFactory connectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(brokerUrl);
connectionFactory.setPassword(userName);
connectionFactory.setUserName(password);
connectionFactory.setTrustAllPackages(true);
connectionFactory.setRedeliveryPolicy(redeliveryPolicy());
return connectionFactory;
}
#Bean
public JmsTemplate jmsTemplate() {
JmsTemplate template = new JmsTemplate();
template.setConnectionFactory(connectionFactory());
return template;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory listenerCF = new DefaultJmsListenerContainerFactory();
listenerCF.setConnectionFactory(connectionFactory());
listenerCF.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
listenerCF.setSessionTransacted(true);
return listenerCF;
}
#Bean
public RedeliveryPolicy redeliveryPolicy() {
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setRedeliveryDelay(600000L); //keep trying every 10 minutes
redeliveryPolicy.setMaximumRedeliveries(-1); //Keep trying till its successfully inserted
return redeliveryPolicy;
}

Using a Connection Name List in a JMS Load Balanced Environment

Our JMS infrastructure is load balanced. As a result of this, I am attempting to use a connectionNameList when configuring the connection factory. The idea here is that any JMS message that arrives on either of the primary or secondary queue manager will get picked up and processed. However, it only appears that messages are being picked up by the primary.
Here is my listener annotation:
#JmsListener(destination = "${request-queue}", containerFactory = "DefaultJmsListenerContainerFactory")
public void onMessage(Message msg) {
System.out.println(msg.toString());
}
Here is the JMS listener container factory:
#Bean(name = "DefaultJmsListenerContainerFactory")
public DefaultJmsListenerContainerFactory createJmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(buildConnectionFactory());
factory.setConcurrency(numberOfListeners);
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
factory.setSessionTransacted(false);
factory.setErrorHandler(queueErrorHandler);
factory.setBackOff(getBackOffStrategy());
return factory;
}
And here is the connection factory:
#Bean(name = "MQConnectionFactory")
public ConnectionFactory buildConnectionFactory() {
try {
MQConnectionFactory mqcf = new MQConnectionFactory();
mqcf.setConnectionNameList(mq1.daluga.com(2171),mq2.daluga.com(2171));
mqcf.setChannel(channel);
mqcf.setTransportType(WMQConstants.WMQ_CM_CLIENT);
return mqcf;
} catch (Exception e) {
throw new RuntimeException(message, e);
}
}
I suspect something in my configuration is just not right. Is there anything obvious that folks see that might cause messages not to be picked up from the secondary queue manager?
Thanks!

Springboot JMS LIstener ActiveMQ is very slow

Im having a SpringBoot application which consume my custom serializable message from ActiveMQ Queue. So far it is worked, however, the consume rate is very poor, only 1 - 20 msg/sec.
#JmsListener(destination = "${channel.consumer.destination}", concurrency="${channel.consumer.maxConcurrency}")
public void receive(IMessage message) {
processor.process(message);
}
The above is my channel consumer class's snippet, it has a processor instance (injected, autowired and inside it i have #Async service, so i can assume the main thread will be released as soon as message entering #Async method) and also it uses springboot activemq default conn factory which i set from application properties
# ACTIVEMQ (ActiveMQProperties)
spring.activemq.broker-url= tcp://localhost:61616?keepAlive=true
spring.activemq.in-memory=true
spring.activemq.pool.enabled=true
spring.activemq.pool.expiry-timeout=1
spring.activemq.pool.idle-timeout=30000
spring.activemq.pool.max-connections=50
Few things worth to inform:
1. I run everything (Eclipse, ActiveMQ, MYSQL) in my local laptop
2. Before this, i also tried using custom connection factory (default AMQ, pooling, and caching) equipped with custom threadpool task executor, but still getting same result. Below is a snapshot performance capture which i took and updating every 1 sec
3. I also notive in JVM Monitor that the used heap keep incrementing
I want to know:
1. Is there something wrong/missing from my steps?I can't even touch hundreds in my message rate
2. Annotated #JmsListener method will execute process async or sync?
3. If possible and supported, how to use traditional sync receive() with SpringBoot properly and ellegantly?
Thank You
I'm just checking something similar. I have defined DefaultJmsListenerContainerFactory in my JMSConfiguration class (Spring configuration) like this:
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory(CachingConnectionFactory connectionFactory) {
// settings made based on https://bsnyderblog.blogspot.sk/2010/05/tuning-jms-message-consumption-in.html
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory(){
#Override
protected void initializeContainer(DefaultMessageListenerContainer container) {
super.initializeContainer(container);
container.setIdleConsumerLimit(5);
container.setIdleTaskExecutionLimit(10);
}
};
factory.setConnectionFactory(connectionFactory);
factory.setConcurrency("10-50");
factory.setCacheLevel(CACHE_CONSUMER);
factory.setReceiveTimeout(5000L);
factory.setDestinationResolver(new BeanFactoryDestinationResolver(beanFactory));
return factory;
}
As you can see, I took those values from https://bsnyderblog.blogspot.sk/2010/05/tuning-jms-message-consumption-in.html. It's from 2010 but I could not find anything newer / better so far.
I have also defined Spring's CachingConnectionFactory Bean as a ConnectionFactory:
#Bean
public CachingConnectionFactory buildCachingConnectionFactory(#Value("${activemq.url}") String brokerUrl) {
// settings based on https://bsnyderblog.blogspot.sk/2010/02/using-spring-jmstemplate-to-send-jms.html
ActiveMQConnectionFactory activeMQConnectionFactory = new ActiveMQConnectionFactory();
activeMQConnectionFactory.setBrokerURL(brokerUrl);
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory(activeMQConnectionFactory);
cachingConnectionFactory.setSessionCacheSize(10);
return cachingConnectionFactory;
}
This setting will help JmsTemplate with sending.
So my answer to you is set the values of your connection pool like described in the link. Also I guess you can delete spring.activemq.in-memory=true because (based on documentation) in case you specify custom broker URL, "in-memory" property is ignored.
Let me know if this helped.
G.

Resources