<bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="${mq.activemq.host}" />
<property name="userName" value="${mq.activemq.user}" />
<property name="password" value="${mq.activemq.pass}" />
<property name="maxThreadPoolSize" value="30" />
</bean>
<bean id="amqPooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" destroy-method="stop">
<property name="connectionFactory" ref="amqConnectionFactory" />
<property name="maxConnections" value="10" />
<property name="maximumActiveSessionPerConnection" value="300" />
<property name="idleTimeout" value="60000" />
</bean>
<bean id="queueListenerContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="amqPooledConnectionFactory" />
<property name="destination" ref="queueDestination" />
<property name="messageListener" ref="queueAwareMessageListener" />
<property name="taskExecutor" ref="queueListenerTaskExecutor" />
<property name="concurrency" value="5-30" />
</bean>
What is the relationship between maxThreadPoolSize, maxConnections, maximumActiveSessionPerConnection, and concurrency?
Why I set maxConnections = 10, but Listener only one Connector in the Connections, can't increase more?
The number of consumers is correct. It has 5 with initialization,and gradually increases with change.
The bottom line regarding the number of connections which will be used is determined by the org.springframework.jms.listener.DefaultMessageListenerContainer which you have configured (since that is the only component here actually creating connections). As far as I can tell it will only ever create a single connection so using a connection pool here appears pointless. The concurrency parameter simply controls the concurrent number of consumers on the connection.
By setting maxConnections = 10 on the org.apache.activemq.pool.PooledConnectionFactory you are just limiting the size of the connection pool. However, since the queueListenerContainer won't ever call createConnection() more than once it doesn't really matter.
You can read about the maxThreadPoolSize of the org.apache.activemq.ActiveMQConnectionFactory in the ActiveMQ documentation.
Related
I am using different connection factories for sending and receiving messages, having trouble with partial commit issues incase of delivey failures. jms:message-driven-channel-adapter uses the receiveConnectionFactory ro receive the messages from the queue. jms:outbound-channel-adapter uses the deliverConnectionFactory to send the messages multiple to downstream queues. We have only one JmsTransactionManager which uses the receiveConnectionFactory and the jms:outbound-channel-adapter configured with session-transacted="true".
<beans>
<bean id="transactionManager"
class="org.springframework.jms.connection.JmsTransactionManager">
<property name="connectionFactory" ref="receiveConnectionFactory" />
</bean>
<bean id="receiveConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory">
<bean class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="hostName" value="${mq.host}" />
<property name="channel" value="${mq.channel}" />
<property name="port" value="${mq.port}" />
</bean>
</property>
<property name="sessionCacheSize" value="${receive.factory.cachesize}" />
<property name="cacheProducers" value="${receive.cache.producers.enabled}" />
<property name="cacheConsumers" value="${receive.cache.consumers.enabled}" />
</bean>
<bean id="deliverConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory">
<bean class="com.ibm.mq.jms.MQQueueConnectionFactory">
<property name="hostName" value="${mq.host}" />
<property name="channel" value="${mq.channel}" />
<property name="port" value="${mq.port}" />
</bean>
</property>
<property name="sessionCacheSize" value="${send.factory.cachesize}" />
<property name="cacheProducers" value="${send.cache.producers.enabled}" />
<property name="cacheConsumers" value="${send.cache.consumers.enabled}" />
</bean>
<tx:advice id="txAdviceNew" transaction-manager="transactionManager">
<tx:attributes>
<tx:method name="send" propagation="REQUIRES_NEW" />
</tx:attributes>
</tx:advice>
<aop:config>
<aop:advisor advice-ref="txAdviceNew" pointcut="bean(inputChannel)" />
<aop:advisor advice-ref="txAdviceNew" pointcut="bean(errorChannel)" />
</aop:config>
<jms:message-driven-channel-adapter
id="mdchanneladapter" channel="inputChannel" task-executor="myTaskExecutor"
connection-factory="receiveConnectionFactory" destination="inputQueue"
error-channel="errorChannel" concurrent-consumers="${num.consumers}"
max-concurrent-consumers="${max.num.consumers}" max-messages-per-task="${max.messagesPerTask}"
transaction-manager="transactionManager" />
<jms:outbound-channel-adapter
connection-factory="deliverConnectionFactory" session-transacted="true"
destination-expression="headers.get('Deliver')" explicit-qos-enabled="true" />
</beans>
When there is MQ exception on any one destination, the partial commit occurs and then the failure queue commit happens. I am looking to see if I am missing some configuration to join the transactions so that the partial commit never happens.
I tried with only one connection factory for both send and receive (receiveConnectionFactory) and the parital commit is not happening, everything works as expected.
I tried with only one connection factory for both send and receive (receiveConnectionFactory) and the parital commit is not happening, everything works as expected.
That's the right way to go in your case.
I see that your two ConnectionFactories are only different by their objects. Everything rest looks like the same target MQ server.
If you definitely can't live with only one ConnectionFactory, you should consider to use JtaTransactionManager or configure org.springframework.data.transaction.ChainedTransactionManager for two JmsTransactionManagers - one per connection factory.
See Dave Syer's article on the matter: https://www.javaworld.com/article/2077963/open-source-tools/distributed-transactions-in-spring--with-and-without-xa.html
(I hope this below makes sense, I had a bit of a struggle with formatting).
We have an application (currently being refreshed from technology of circa 2005) whose owners require the use of Oracle AQ for message handling.
We are using the Spring JDBC extensions, and when the application begins to listen to the queue it gets a ClassCastException saying "X cannot be cast to oracle.jdbc.internal.OracleConnection".
Where X is either a WrappedConnectionJDK7 from jboss when we use a datasource defined in Wildfly e.g.
<bean id="dataSource-supsrep"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#localhost:1521:XE" />
<property name="username" value="...." />
<property name="password" value="...." />
</bean>
or X is a PoolGuardConnectionWrapper from commons.dbcp when we define the datasource like this:
<bean id="dataSource-supsrep"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#localhost:1521:XE" />
<property name="username" value="...." />
<property name="password" value="...." />
</bean>
Message Listener bean is defined like this :
<bean id="messageListener" class="xxx.report.dispatch.DispatchMDB" />
Message Listener container bean is like this :
<bean id="reportProcessorXslfoHigh"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="sessionTransacted" value="true" />
<property name="connectionFactory" ref="AQconnectionFactory" />
<property name="destinationName" value="Q1_XSLFO_HIGH" />
<property name="messageListener" ref="messageListener" />
<property name="transactionManager" ref="transactionManager" />
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="1" />
<property name="receiveTimeout" value="30000" />
<property name="idleTaskExecutionLimit" value="30" />
<property name="idleConsumerLimit" value="1" />
</bean>
The connection factory is like this :
<orcl:aq-jms-connection-factory id="AQconnectionFactory"
data-source="dataSource-supsrep" native-jdbc- extractor="oracleNativeJdbcExtractor" />
and it uses a Native JDBC Extractor :
<bean id="oracleNativeJdbcExtractor"
class="org.springframework.jdbc.support
.nativejdbc.OracleJdbc4NativeJdbcExtractor" />
So my question is - can this be overcome by Spring configuration details - or is there another way round it? Clearly the underlying connection is needed but after 3 days of research I cannot see how it can be done.
Any clues gratefully received !
Thanks,
John
I got one strange problem.
when I config a DataSourceTransactionManager with spring xml, the concurrent consumers of ActiveMQ were suppressed whatever I change "maxConcurrentConsumers" property value. I have 5 queues, the total concurrent consumers of all 5 queue always kept at 8.
if I remove DataSourceTransactionManager bean, each queue's concurrent consumers reached the max number 5 declared in "maxConcurrentConsumers" .
The DataSourceTransactionManager work for dataSource, i cannot understand why it affected to ActiveMQ.
version:
Spring 3.2.5.RELEASE
ActiveMq 5.9.0
application.xml
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driver}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.user}" />
<property name="password" value="${jdbc.password}" />
</bean>
<!-- once I add this, activemq total consumers always kept at 8 -->
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<!-- activemq consumer connection -->
<bean id="consumerConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory"
destroy-method="stop">
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>tcp://localhost:61616</value>
</property>
</bean>
</property>
<property name="maxConnections" value="5"></property>
</bean>
<!-- i have 5 queues -->
<bean id="test_1" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_1}" />
</bean>
<bean id="test_2" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_2}" />
</bean>
<bean id="test_3" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_3}" />
</bean>
<bean id="test_4" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_4}" />
</bean>
<bean id="test_5" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_5}" />
</bean>
<!-- consumer listener container -->
<bean id="testOneMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_1"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testTwoMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_2"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testThreeMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_3"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testFourMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_4"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testFiveMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_5"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
can someone help me!!!
after some test, i found a way to resolve this problem.
when i change the dataSource "maxActive" parameter to a number greater than sum of all mq listener maxConcurrentConsumers. it work fine.
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driver}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.user}" />
<property name="password" value="${jdbc.password}" />
<property name="maxActive" value="120" />
</bean>
it seems that the max number of activemq listener thread affected by Datasource maxActive parameter
I am trying to copy a file from FTP and want to place it in local machine.
For this I created an inbound channel configurations are:
<bean name="publishStockSessionFactory"
class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
<property name="host"
value="10.255.255.1" />
<property name="port" value="21" />
<property name="username"
value="test" />
<property name="password"
value="test" />
</bean>
<bean id="stockLocalDirectory" class="java.lang.String">
<constructor-arg
value="/opt/test" />
</bean>
<bean id="stockRemoteDirectory" class="java.lang.String">
<constructor-arg
value="stock" />
</bean>
<int-ftp:inbound-channel-adapter
local-directory="# {stockLocalDirectory}"
channel="stockFilesFromFTP"
session-factory="publishStockSessionFactory"
remote-directory="#{stockRemoteDirectory}"
delete-remote-files="true"
filename-regex="Stock*.csv" >
<int:poller fixed-rate="120000" max-messages-per-poll="100" />
</int-ftp:inbound-channel-adapter>
<int:publish-subscribe-channel id="stockFilesFromFTP" />
And the error coming while start up is
INFO | jvm 1 | main | 2013/01/15 22:20:02.715 | 2013-01-15 22:20:02,699
ERROR task-scheduler-4 ErrorHandler : failed to send message
to channel 'stockFilesFromFTP' within timeout: -1
Debugs we turned on are
log4j.logger.org.springframework.aop=DEBUG
org.springframework.integration.channel.DirectChannel=DEBUG
org.springframework.integration.channel.MessagePublishingErrorHandler=DEBUG
org.springframework.integration.config.xml.PointToPointChannelParser=DEBUG
Can you suggest me how I can debug this error?
Run with DEBUG level logging.
You must have at least one subscriber on channel stockFilesFromFTP.
I used following kind of configuation for FTPclientfactory bean. It has timeout param also, so in case you are facing issue due to connection time out , this will help you.
<bean id="testapp.standardFTPClientFactory" class="org.springframework.integration.ftp.DefaultFTPClientFactory"
abstract="true">
<property name="host" value="${ftp.host}"/>
<property name="username" value="${ftp.username}"/>
<property name="password" value="${ftp.password}"/>
<property name="port" value="${ftp.port}"/>
<property name="remoteWorkingDirectory" value="${ftp.remotedir}"/>
<property name="dataTimeout" value="3600000"/>
<property name="connectTimeout" value="${ftp.connectTimeout}"/>
<property name="clientMode" value="2"/>
</bean>
What is the best way to perform initialization on DefaultMessageListenerContainer initialization? Currently I am waiting for first message, and keep track of it using a boolean variable, which isn't so pretty. Is there a better way ? I want to read and load certain data into the cache once the Message Driven POJO is started up, so the message processing is faster.
(Edited)
Spring Config Fragement:
<bean id="itemListener" class="com.test.ItemMDPImpl" autowire="byName" />
<bean id="itemListenerAdapter" class="org.springframework.jms.listener.adapter.MessageListenerAdapter">
<property name="delegate" ref="itemListener" />
<property name="defaultListenerMethod" value="processItems" />
<property name="messageConverter" ref="itemMessageConverter" />
</bean>
<bean class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="itemMqConnectionFactory" />
<property name="destinationName" value="${item_queue_name}" />
<property name="messageListener" ref="itemListenerAdapter" />
<property name="transactionManager" ref="jtaTransactionManager" />
<property name="sessionTransacted" value="true" />
<property name="concurrentConsumers" value="1" />
<property name="receiveTimeout" value="3000" />
</bean>
I would like to have some initialization done before any message is received by the listener.
Can't you just use #PostConstruct to annotate a method on ItemMDPImpl to perform startup initialization, just like any other Spring bean?
4.9.6 #PostConstruct and #PreDestroy