I am listening to the queue in my application like this
#JmsListener(containerFactory = "MessageListener", destination = "${mq.destination-name}")
My connection factory Bean looks like this
#Bean(name = "MessageListener")
public JmsListenerContainerFactory<?> mqMessageListenerContainer(
ConnectionFactory connectionFactory, QueueErrorHandler QueueErrorHandler)
{
SimpleJmsListenerContainerFactory simpleJmsListenerContainerFactory = new SimpleJmsListenerContainerFactory();
simpleJmsListenerContainerFactory.setConnectionFactory(connectionFactory);
simpleJmsListenerContainerFactory.setErrorHandler(QueueErrorHandler);
return simpleJmsListenerContainerFactory;
}
I want to understand when I don't have
simpleJmsListenerContainerFactory.setSessionTransacted(true);
simpleJmsListenerContainerFactory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
these set in factory configuration when is my application starts a transaction on reading a message and when does it commits ?
Related
I use spring boot 2.3.2 with ibm mq.
I use property file setup mq.
ibm.mq.queueManager=
ibm.mq.channel=
ibm.mq.connName
ibm.mq.user
ibm.mq.password
ibm.mq.useIBMCipherMappings=false
ibm.mq.userAuthenticationMQCSP=false
ibm.mq.sslCipherSuite=TLS_RSA_WITH_AES_128_CBC_SHA256
that works fine.
I need to create another factory to be able to connect to another channel. So I would like to create a similar one which is created by default but with different channel name.
So I created a config class
#EnableJms
#Configuration
public class JmsConfig {
#Bean
public MQQueueConnectionFactory jmsMQConnectionFactoryPayment() throws JMSException {
MQQueueConnectionFactory connectionFactory = new MQQueueConnectionFactory();
connectionFactory.setQueueManager("AD_TEST");
connectionFactory.setChannel("FROM.PAYMENTMNG");
connectionFactory.setConnectionNameList("wmqd1.int.test.com(1818),wmqd2.int.test.com(1818)");
connectionFactory.setBooleanProperty(WMQConstants.USER_AUTHENTICATION_MQCSP, false);
connectionFactory.setStringProperty(WMQConstants.WMQ_SSL_CIPHER_SUITE, "TLS_RSA_WITH_AES_128_CBC_SHA256");
connectionFactory.setIntProperty(CommonConstants.WMQ_CONNECTION_MODE, CommonConstants.WMQ_CM_CLIENT);
System.setProperty("com.ibm.mq.cfg.useIBMCipherMappings", String.valueOf(Boolean.FALSE));
connectionFactory.createConnection("123", "123");
return connectionFactory;
}
#Bean
JmsListenerContainerFactory<?> jmsContainerFactoryPayment() throws JMSException {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(jmsMQConnectionFactoryPayment());
return factory;
}
#Bean("payment")
JmsTemplate jmsTemplatePayment() throws JMSException {
JmsTemplate template = new JmsTemplate();
template.setConnectionFactory(jmsMQConnectionFactoryPayment());
return template;
}
}
In a class I have
#JmsListener(destination="xxx", containerFactory="jmsContainerFactoryPayment"){
....
}
When I start application, I get
com.ibm.msg.client.jms.DetailedIllegalStateException: JMSWMQ0018: E Failed to connect to queue manager 'AD_TEST' with connection mode 'Client' and host name 'wmqd1.int.test.com(1818),wmqd2.int.test.com(1818)'.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:489) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0 - p920-L200710.DE]
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:215) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0 - p920-L200710.DE]
at com.ibm.msg.client.wmq.internal.WMQConnection.<init>(WMQConnection.java:450) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0 - p920-L200710.DE]
at com.ibm.msg.client.wmq.factories.WMQConnectionFactory.createV7ProviderConnection(WMQConnectionFactory.java:8475) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0
Caused by: com.ibm.mq.MQException: JMSCMQ0001: IBM MQ call failed with compcode '2' ('MQCC_FAILED') ; reason '2538' ('MQRC_HOST_NOT_AVAILABLE').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:203) ~[com.ibm.mq.allclient-9.2.0.0.jar:9.2.0.0 - p920-L200710.DE]
... 51 common frames omitted
Seem like we can't have two queue manager with same host but different channel
I have configured Spring Boot with IBM MQ to test this. Starting with the sample provided in this IBM Messaging GitHub Repo, I added two additional listeners as follows.
First I added some additional properties to the application.properties file
my.mq.queueManager=QM2
my.mq.channel=DEV.APP.SVRCONN
my.mq.connName=localhost(1415)
my.mq.user=<your_user_name>
my.mq.password=<your_password>
I left Application.java unchanged, copied Listener.java to create ListenerTwo.java and ListenerThree.java. I then added a new ListenerBeanConfig.java class to the sample.
ListenerTwo.java was changed to bind to a new ConnectionFactory configuration listenerTwoFactory which is created later.
#JmsListener(destination = Application.qName, containerFactory = "listenerTwoFactory")
ListenerThree.java was changed to bind to a new listenerThreeFactory configuration and Queue
#JmsListener(destination = "DEV.QUEUE.2", containerFactory = "listenerThreeFactory")
The ListenerBeanConfig.java class declaration was annotated so that I can access my properties by adding Strings for each property e.g., queueManager, channel, connName etc. and providing setter methods for each of them e.g.,
#Configuration
#EnableJms
#ConfigurationProperties(prefix="my.mq")
public class ListenerBeanConfig {
String connName;
public void setConnName(String value) {
System.out.println("connName is set to: "+value);
connName = value;
}
I registered the two new Listener beans
#Bean
public ListenerTwo myListenerTwo() {
return new ListenerTwo();
}
#Bean
public ListenerThree myListenerThree() {
return new ListenerThree();
}
I then created the new connection factory configurations listenerTwoFactory and listenerThreeFactory
For listenerTwoFactory I used the JMS classes provided by com.ibm.mq.jms in the Spring Boot config
JmsConnectionFactory cf;
#Bean
public DefaultJmsListenerContainerFactory listenerTwoFactory() {
DefaultJmsListenerContainerFactory containerFactory = new DefaultJmsListenerContainerFactory();
try {
JmsFactoryFactory ff = JmsFactoryFactory.getInstance(WMQConstants.WMQ_PROVIDER);
cf = ff.createConnectionFactory();
cf.setStringProperty(WMQConstants.WMQ_CONNECTION_NAME_LIST, connName);
cf.setStringProperty(WMQConstants.WMQ_CHANNEL, channel);
cf.setIntProperty(WMQConstants.WMQ_CONNECTION_MODE, WMQConstants.WMQ_CM_CLIENT);
cf.setStringProperty(WMQConstants.WMQ_QUEUE_MANAGER, queueManager);
cf.setStringProperty(WMQConstants.WMQ_APPLICATIONNAME, "Spring Boot ListenerTwo");
cf.setBooleanProperty(WMQConstants.USER_AUTHENTICATION_MQCSP, true);
cf.setStringProperty(WMQConstants.USERID, user);
cf.setStringProperty(WMQConstants.PASSWORD, password);
} catch (JMSException jmsex) {
System.out.println(jmsex);
}
containerFactory.setConnectionFactory(cf);
return containerFactory;
}
For the listenerThreeFactory I used the MQ JMS helper classes from com.ibm.mq.spring.boot.
#Bean
public DefaultJmsListenerContainerFactory listenerThreeFactory() {
MQConfigurationProperties myProps = new MQConfigurationProperties();
myProps.setUser(user);
myProps.setChannel(channel);
myProps.setConnName(connName);
myProps.setPassword(password);
myProps.setQueueManager(queueManager);
//No customizer
MQConnectionFactoryFactory mqcff = new MQConnectionFactoryFactory(myProps,null);
MQConnectionFactory mqcf = mqcff.createConnectionFactory(MQConnectionFactory.class);
DefaultJmsListenerContainerFactory containerFactory = new DefaultJmsListenerContainerFactory();
containerFactory.setConnectionFactory(mqcf);
return containerFactory;
}
Finally, I compiled and ran the new sample configuration. Using the IBM MQ Console for two IBM MQ queue manager docker instances, I put messages to QM1: DEV.QUEUE.1 and QM2: DEV.QUEUE.1, DEV.QUEUE.2. On the terminal see the following output.
========================================
Received message is: message 1
========================================
========================================
ListenerTwo received message is: message 2
========================================
========================================
ListenerThree received message is: message 3
========================================
Also tested with all three listeners connected to QM2 via a two different channels: DEV.APP.SVRCONN and DEV.APP.SVRCONN.TWO.
I am sure there are far more elegant ways to manage the additional properties.
I am designing a simple system where the flow is going to be like this:
Message Producer Microservice --> Active MQ --> Message Consumer Microservice --> Mongo DB
I need to design a queuing strategy in a way so that if MongoDB is down, I should not lose the message (because Message consumer will dequeue the message).
My consumer is written like this:
#JmsListener(destination = "Consumer.myconsumer.VirtualTopic.Tracking")
public void onReceiveFromQueueConsumer2(TrackingRequest trackingRequest) {
log.debug("Received tracking request from the queue by consumer 2");
log.debug(trackingRequest.toString());
}
How do you provide client acknowledgement?
You can use client acknowledge mode from your "Message Consumer Microservice." Since you're using a Spring JmsListener you can define the listener container using the containerFactory and then you can set the mode you want on your listener container using sessionAcknowledgeMode. See the Spring documentation for more details on what ack mode you might want to use here.
From the perspective of the ActiveMQ client you can configure redelivery semantics however you like in case of a failure. See the ActiveMQ documentation for more about that.
Alright, so I was able to solve this dilemma, here is what your config should be like (thanks to Justin for his valuable inputs):
#Bean
public ActiveMQConnectionFactory connectionFactory() {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
connectionFactory.setBrokerURL(brokerUrl);
connectionFactory.setPassword(userName);
connectionFactory.setUserName(password);
connectionFactory.setTrustAllPackages(true);
connectionFactory.setRedeliveryPolicy(redeliveryPolicy());
return connectionFactory;
}
#Bean
public JmsTemplate jmsTemplate() {
JmsTemplate template = new JmsTemplate();
template.setConnectionFactory(connectionFactory());
return template;
}
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory listenerCF = new DefaultJmsListenerContainerFactory();
listenerCF.setConnectionFactory(connectionFactory());
listenerCF.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
listenerCF.setSessionTransacted(true);
return listenerCF;
}
#Bean
public RedeliveryPolicy redeliveryPolicy() {
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setRedeliveryDelay(600000L); //keep trying every 10 minutes
redeliveryPolicy.setMaximumRedeliveries(-1); //Keep trying till its successfully inserted
return redeliveryPolicy;
}
Our JMS infrastructure is load balanced. As a result of this, I am attempting to use a connectionNameList when configuring the connection factory. The idea here is that any JMS message that arrives on either of the primary or secondary queue manager will get picked up and processed. However, it only appears that messages are being picked up by the primary.
Here is my listener annotation:
#JmsListener(destination = "${request-queue}", containerFactory = "DefaultJmsListenerContainerFactory")
public void onMessage(Message msg) {
System.out.println(msg.toString());
}
Here is the JMS listener container factory:
#Bean(name = "DefaultJmsListenerContainerFactory")
public DefaultJmsListenerContainerFactory createJmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(buildConnectionFactory());
factory.setConcurrency(numberOfListeners);
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
factory.setSessionTransacted(false);
factory.setErrorHandler(queueErrorHandler);
factory.setBackOff(getBackOffStrategy());
return factory;
}
And here is the connection factory:
#Bean(name = "MQConnectionFactory")
public ConnectionFactory buildConnectionFactory() {
try {
MQConnectionFactory mqcf = new MQConnectionFactory();
mqcf.setConnectionNameList(mq1.daluga.com(2171),mq2.daluga.com(2171));
mqcf.setChannel(channel);
mqcf.setTransportType(WMQConstants.WMQ_CM_CLIENT);
return mqcf;
} catch (Exception e) {
throw new RuntimeException(message, e);
}
}
I suspect something in my configuration is just not right. Is there anything obvious that folks see that might cause messages not to be picked up from the secondary queue manager?
Thanks!
How to change the redelivery policy for the embedded ActiveMQ when using with Spring Boot? I tried specifying FixedBackOff on the DefaultJmsListenerContainerFactory but it didn't help. Below is code I am using to initialize the jms factory bean. I have a message consumer processing incoming messages on a queue. During processing because of unavailable resource, I throw a checked exception. I am hoping to have the message redelivered for processing after a fixed interval.
Spring Boot : 1.5.7.Release
Java : 1.7
#Bean
public JmsListenerContainerFactory<?> publishFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setBackOff(new FixedBackOff(5000, 5));
// This provides all boot's default to this factory, including the message converter
configurer.configure(factory, connectionFactory);
// You could still override some of Boot's default if necessary.
factory.setErrorHandler(new ErrorHandler() {
#Override
public void handleError(Throwable t) {
LOG.error("Error occured in JMS transaction.", t);
}
});
return factory;
}
Consumer Code:
#JmsListener(destination = "PublishQueue", containerFactory = "publishFactory")
#Transactional
public void receiveMessage(PublishData publishData) {
LOG.info("Processing incoming message on publish queue with transaction id: " + publishData.getTransactionId());
PublishUser user = new PublishUser();
user.setPriority(1);
user.setUserId(publishData.getUserId());
LOG.trace("Trying to enroll in the publish lock queue for user: " + user);
PublishLockQueue lockQueue = publishLockQueueService.createLock(user);
if (lockQueue == null)
throw new RuntimeException("Unable to create lock for publish");
LOG.trace("Publish lock queue obtained with lock queue id: " + lockQueue.getId());
try {
publishLockQueueService.acquireLock(lockQueue.getId());
LOG.trace("Acquired publish lock.");
}
catch (PublishLockQueueServiceException pex) {
throw new RuntimeException(pex);
}
try {
publishService.publish(publishData, lockQueue.getId());
LOG.trace("Completed publish of changes.");
sendPublishSuccessNotification(publishData);
}
finally {
LOG.trace("Trying to release lock to publish.");
publishLockQueueService.releaseLock(lockQueue.getId());
}
LOG.info("Publish has been completed for transaction id: " + publishData.getTransactionId());
}
#claus answerd: i tested it to work:
Its the consumer, you need to use transacted acknowledge mode to let the consumer rollback on exception and let ActiveMQ be able to re-deliver the message to the same consumer or another consumer if you have multiple consumers running. You can however configure redelivery options on the ActiveMQ such as backoff etc. The error handler above is just a noop listener which cannot do very much other than logging
I've created a Spring JMS application using version 4.1.2.RELEASE, which is connected to a broker that is running ActiveMQ 5.11.0. The problem that I'm seeing is as follows. In the logs, I notice that every second, I'm seeing a connection being created as such.
2017-06-21 13:10:21,046 | level=INFO | thread=ActiveMQ Task-1 | class=org.apache.activemq.transport.failover.FailoverTransport | Successfully connected to tcp://localhost:61616
I know that it is creating a new ActiveMQ connection each time, because it says successfully "connected" and not "reconnected" as shown in the code located here: http://grepcode.com/file/repo1.maven.org/maven2/com.ning/metrics.collector/1.3.3/org/apache/activemq/transport/failover/FailoverTransport.java#891
I don't have a caching connection factory set for my consumer, but I'm wondering if the following is the culprit when it comes to why I'm seeing constant connections being created.
factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE);
The following post states that consumers should not be cached, but I wonder if that applies to caching the connection + session. If the connection is cached, but the session is not, then I wonder if that creates a problem.
Why DefaultMessageListenerContainer should not use CachingConnectionFactory?
The following are the configurations that I'm using in my application. I am hoping that it is something that I've misconfigured, and would appreciate any insights that anyone has to offer.
Spring Configurations
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() throws Throwable {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE);
factory.setMaxMessagesPerTask(-1);
factory.setConcurrency(1);
factory.setSessionTransacted(true);
return factory;
}
#Bean
public CachingConnectionFactory cachingConnectionFactory(){
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory(connectionFactory());
cachingConnectionFactory.setCacheConsumers(false);
cachingConnectionFactory.setSessionCacheSize(1);
return cachingConnectionFactory;
}
#Bean
public ActiveMQConnectionFactory connectionFactory(){
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setInitialRedeliveryDelay(1000L);
redeliveryPolicy.setRedeliveryDelay(1000L);
redeliveryPolicy.setMaximumRedeliveries(6);
redeliveryPolicy.setUseExponentialBackOff(true);
redeliveryPolicy.setBackOffMultiplier(5);
ActiveMQConnectionFactory activeMQ = new ActiveMQConnectionFactory("admin", "admin", "tcp://localhost:61616");
activeMQ.setRedeliveryPolicy(redeliveryPolicy);
activeMQ.setPrefetchPolicy(prefetchPolicy());
return activeMQ;
}
#Bean
public JmsMessagingTemplate jmsMessagingTemplate(){
ActiveMQTopic activeMQ = new ActiveMQTopic("topic.out");
JmsMessagingTemplate jmsMessagingTemplate = new JmsMessagingTemplate(cachingConnectionFactory());
jmsMessagingTemplate.setDefaultDestination(activeMQ);
return jmsMessagingTemplate;
}
protected ActiveMQPrefetchPolicy prefetchPolicy(){
ActiveMQPrefetchPolicy prefetchPolicy = new ActiveMQPrefetchPolicy();
int prefetchValue = 1000;
prefetchPolicy.setQueuePrefetch(prefetchValue);
return prefetchPolicy;
}
Thanks,
Juan
The issue was indeed the following code.
factory.setCacheLevel(DefaultMessageListenerContainer.CACHE_NONE);
The moment that I removed it, the rapid connection creation stopped.