I am currently trying to support dynamic multiple jms provider scenario in my application. So far I did achieved to create DefaultMessageListenerContainer using post processor. Cool part is that the DefaultMessageContainerListener has destinationName property where you can easily set the queue to be listened/sent for messages.
However, the DefaultJmsListenerContainerFactory has no such method to set the queue name. I do reached at around the SimpleJmsListenerEndpoint that the DefaultJmsListenerContainerFactory using to initiate the container. But I am unable to find how to set it. Please see below what I did so far.
beanDefinitionRegistry.registerBeanDefinition("messageListenerContainer",
BeanDefinitionBuilder.rootBeanDefinition(DefaultJmsListenerContainerFactory.class)
.addPropertyReference("connectionFactory", "queueConnectionFactory")
.addPropertyReference("destinationResolver", "jndiDestinationResolver")
.addPropertyValue("concurrency", concurrency)
.addPropertyValue("sessionAcknowledgeMode", Session.AUTO_ACKNOWLEDGE)
.getBeanDefinition()
);
But as you can see I can not set the queue endpoint for listening. How can I do that from here?
Related
I'm using 2.1.0.RELEASE version of Spring boot with AMQP. Unfortunetly I need to connect to several different RabbitMQ servers. I had to exclude RabbitAutoConfiguration.class besause due to changes in above version of spring it's impossible to start without one of the ConnectionFactory beans as primary, but even if I set one of them as #Primary, obviously it doesn't work, because how would amqp/spring-boot know which queue to create on which server...
so, is it possible to automatically create queues on different servers with auto configuration disabled?
Yes, you need a RabbitAdmin for each connection factory.
By default all components will be declared on all brokers, but you can add conditions. See Conditional Declaration.
By default, all queues, exchanges, and bindings are declared by all RabbitAdmin instances (assuming they have auto-startup="true") in the application context.
#Bean
public Queue queue1() {
Queue queue = new Queue("foo");
queue.setAdminsThatShouldDeclare(admin1());
return queue;
}
I'm using #EnableJms and #JmsListener annotation to register a queue listener in my application, basing on this tutorial. I'm connecting to IBM MQ, getting connection factory using jndi. I have read about acknowledge modes etc. but still it's a new thing to me. And my problem is that the message is not being returned to a queue (the listener is never called again).
#Bean
public DefaultJmsListenerContainerFactory jmsListenerContainerFactory() {
DefaultJmsListenerContainerFactory factory
= new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setSessionTransacted(true);
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE); //I have tried also CLIENT_ACKNOWLEDGE
return factory;
}
#JmsListener(containerFactory = "jmsListenerContainerFactory", destination = "myQueue")
#SendTo("secondQueue")
public String testListener(String message){
if(true) throw new NullPointerException();
else return message;
}
Any help would be much appreciated.
I would have also a second question. From what I understand if I would like to implement any operation on a database, only way to rollback a commit (if something went wrong after this) would be to create a transaction manager? If not, I would need to detect a duplicated message.
First set the acknowledgement mode to Session.CLIENT_ACKNOWLEDGE
and when receiving the messages, if it's processed correctly then just call message.acknowledge() method or else don't call.
It will automatically stay in the queue and you don't need to resend it.
You need to use
import javax.jms.Message
I created simple Spring Boot app and Docker container of IBM MQ to test your case.
I found good instructions in this tutorial: https://developer.ibm.com/tutorials/mq-jms-application-development-with-spring-boot/
And in your case this environment behaves as expected: endless cycle of receive message -> NullPointerException -> return message -> ...
Than I found feature of IBM MQ called "Backout Queues & Thresholds", you'll found the explanation in this blog post: https://community.ibm.com/community/user/imwuc/browse/blogs/blogviewer?BlogKey=28814801-083d-4c80-be5f-90aaaf81cdfb
Briefly, it is possible to restrict number of times message returned to queue after exception, and after this limit send message to another queue.
May be in your case this feature used on your destination queue.
I am trying to connect to Solace Queues on a VPN different then default using Appache NIFI ConsumeJMS Processor. When I try to enable the JMSConnectionFactoryProvider I get the following error:
JMSConnectionFactoryProvider Failed to invoke #OnEnabled method due to
java.lang.IllegalStateException: java.lang.IllegalStateException:
Failed to load and/or instantiate class
'com.solacesystems.jms.SolConnectionFactory'
The NIFI JMSConnectionFactoryProvider provides a generic service to create vendor specific javax.jms.ConnectionFactory implementations. ConnectionFactory can be served once this service is configured successfully.
Why is NIFI Unable to find the class within the Solace JMS API Jar files?
----- Update --------
Solace JMS API 10.1.0 now contains an zero argument default constructor, and integration with NiFi is now possible.
Here is an example configuration:
Controller:
The MQ ConnectionFactory Implementation is set to com.solacesystems.jms.SolConnectionFactoryImpl.
ConsumeJMS Processor:
Username field can also take the form of "myUsername#myMessageVPN".
----- Original -------
The problem here is that Apache NiFi is not using a portable method of creating a ConnectionFactory. It is trying to create a ConnectionFactory by calling an zero argument default constructor, but there's no guarantee that one exists.
// From https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-jms-bundle/nifi-jms-cf-service/src/main/java/org/apache/nifi/jms/cf/JMSConnectionFactoryProvider.java
private void createConnectionFactoryInstance(ConfigurationContext context) {
String connectionFactoryImplName = context.getProperty(CONNECTION_FACTORY_IMPL).evaluateAttributeExpressions().getValue();
this.connectionFactory = Utils.newDefaultInstance(connectionFactoryImplName);
}
Note that there's an entry over at NiFi's JIRA https://issues.apache.org/jira/browse/NIFI-2701 to "Add JNDI Factory support for JMS ConnectionFactory service". (The initial description of that entry is a bit confusing, but the comments are clearer.)
At this point, Solace only supports the creation of the ConnectionFactory through the standard JNDI lookup - javax.naming.InitialContext.lookup() and through Solace's proprietary method - SolJmsUtility.createConnectionFactory().
Solace will investigate whether it is feasible to implement a zero argument default constructor for the ConnectionFactory.
A JNDI connection provider is developed and published at http://dev.solace.com/integration-guides/nifi/. For people that are interested in this provider, it is worthwhile finding out.
I'm unable to process large messages from IBM MQ and get the below error:
JMSCMQ0001: WebSphere MQ call failed with compcode '1' ('MQCC_WARNING') reason '2080' ('MQRC_TRUNCATED_MSG_FAILED')
I'm using the DefaultListenerContainer and not consuming via a MessageConsumer using IBM MQ Java API classes directly. I believe by using IBM MQ JMS API you can specific options before retrieving the message from the queue. But how do I do that with DefaultListenerContainer, is there a system property I can set for these?
If using IBM MQ JMS API(I'm not consuming message like this, pasted just for reference):
MQGetMessageOptions mqGetMessageOptions = new MQGetMessageOptions();
mqGetMessageOptions.waitInterval = ipreoProperties.getMqReceiveWaitTime();
mqGetMessageOptions.options = MQC.MQGMO_WAIT | MQC.MQPMO_SYNCPOINT | MQC.MQGMO_ACCEPT_TRUNCATED_MSG;
Below is my Java Config for the IBM MQ Connection:
#Bean
public CachingConnectionFactory ipreoMQCachingConnectionFactory() {
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory();
//Not defining MQQueueConnectionFactory as separate bean as Spring boot's auto-configuration finds two instances
//of ConnectionFactory and throws ambiguous implementation exception
//One implementation is CachingConnectionFactory and other one would be MQQueueConnectionFactory if defined separately
MQQueueConnectionFactory mqConnectionFactory = new MQQueueConnectionFactory();
try {
mqConnectionFactory.setHostName(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_MQ_HOSTNAME));
mqConnectionFactory.setQueueManager(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_MQ_QUEUE_MGR));
mqConnectionFactory.setPort(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_MQ_PORT, Integer.class));
mqConnectionFactory.setChannel(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_MQ_CHANNEL));
//mqConnectionFactory.setTransportType(WMQConstants.WMQ_CM_CLIENT);
//Setting connection mode as Client so it doesn't complain for native IBM MQ libraries
mqConnectionFactory.setIntProperty(CommonConstants.WMQ_CONNECTION_MODE, CommonConstants.WMQ_CM_CLIENT);
} catch (JMSException exception) {
exception.printStackTrace();
}
cachingConnectionFactory.setTargetConnectionFactory(mqConnectionFactory);
//Setting session caching size as 10, don't think we need more
cachingConnectionFactory.setSessionCacheSize(10);
cachingConnectionFactory.setReconnectOnException(true);
return cachingConnectionFactory;
}
public DefaultMessageListenerContainer ipreoDealActivityListenerContainer() {
DefaultMessageListenerContainer factory = new DefaultMessageListenerContainer();
factory.setConnectionFactory(ipreoMQCachingConnectionFactory());
factory.setDestinationName(env.getRequiredProperty(AppEnvPropertyConstants.JmsConstants.IPREO_DEAL_QUEUE_NAME));
factory.setMessageListener(ipreoDealActivityListener());
factory.setSessionAcknowledgeMode(Session.AUTO_ACKNOWLEDGE);
return factory;
}
#Bean
public MessageListener ipreoDealActivityListener() {
return new IpreoDealActivityListener();
}
Appreciate your help, thanks.
Adding a late response as it might be useful to someone.
In my case, when the java client had this exception, we noticed the actual message size was larger than the default 4 MB buffer size.
The Java API does not provide a hook to change buffer size. Hence, the buffer size has to be updated at the MQ server level.
First, we increased the message size in queue properties - It did not work.
Then, we increased the message size property at the MQ channel level as well, which finally resolved the issue.
To summarise, increase the buffer size at the MQ server for queue & the channel both.
On a client connection to a queue manager you can limit the size of messages on both the server and client side. I've seen this error before when the client side limit was smaller then the size of the message.
I don't know how you can set the message size limit directly in the JMS client, but you could use a Client Channel Definition Table. It's a file containing the details for connecting to queue managers, created on a queue manager and then copied to the client host. You need to reference the file by issuing setCCDTURL on the connection factory (setting the host, port and channel is not required when using a CCDT, the CCDT will specify those).
When the CCDT is created on the queue manager the appropriate message size limit needs to be set on the client channel.
The server side limit is set on the server connection channel.
Within the JMS client code handling of the receive buffer us handled automatically; the theory is that specific error should never be received by a JMS Application.
The first snippet of code is the Java Classes API and this could get that error.
How big actually are these messages? What level of the JMS client code are you using - make sure that it is the latest version. And certainly one of the 7.5 or 8 releases.
This answer also has some more information on this.
I am working on a brand new project in which I need to have listeners that will consume messages from several queues (no need to have producer for now).
Starting from scratch, I am using the last Spring JMS version (4.1.2).
Here is an extract of my configuration file:
<bean id="cachedConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory"
p:targetConnectionFactory-ref="jmsConnectionFactory"
p:sessionCacheSize="3" />
<bean id="jmsListenerContainerFactory"
class="org.springframework.jms.config.DefaultJmsListenerContainerFactory"
p:connectionFactory-ref="cachedConnectionFactory"
p:destinationResolver-ref="jndiDestinationResolver"
p:concurrency="3-5"
p:receiveTimeout="5000" />
But I think I may be wrong since DefaultJmsListenerContainerFactory will build regular DefaultMessageListenerContainerS. And, as stated in the doc, CachingConnectionFactory should not be used with a message listener container...
Even if I am using the new Spring 4.1 DefaultJmsListenerContainerFactory class the answer from post is still valid (cacheConsumers = true can be an issue + don't need to cache sessions for listener containers because the sessions are long lived) , right?
Instead of using the CachingConnectionFactory, I should use the SingleConnectionFactory (and not directly the broker implementation one)?
If the SingleConnectionFactory class should indeed be used, is the "reconnectOnException" property should be set to true (as it is done in the CachingConnectionFactory) or does the new "setBackOff" method (from DefaultJmsListenerContainerFactory) deals with the same kind of issues?
Thanks for any tips
Correct.
There's not really much benefit in using a SingleConnectionFactory unless you want to share a single connection across multiple containers; the DMLC will use a single connection from the vendor factory by default for all consumer threads (cacheLevel >= CACHE_CONNECTION), unless a TransactionManager is configured.
The container(s) will handle reconnection - even before the 'new' backOff property - backOff just adds more sophistication to the reconnection algorithm - it used to just retry every n seconds (5 by default).
As stated in the answer you cited, it's ok to use a CCF as long as you disable consumer caching.
Correction: Yes, when using the SingleConnectionFactory, you do need to set reconnectOnException to true in order for the container to properly recover its connection. Otherwise, it simply hands out the stale connection.