I am trying to copy a file from FTP and want to place it in local machine.
For this I created an inbound channel configurations are:
<bean name="publishStockSessionFactory"
class="org.springframework.integration.ftp.session.DefaultFtpSessionFactory">
<property name="host"
value="10.255.255.1" />
<property name="port" value="21" />
<property name="username"
value="test" />
<property name="password"
value="test" />
</bean>
<bean id="stockLocalDirectory" class="java.lang.String">
<constructor-arg
value="/opt/test" />
</bean>
<bean id="stockRemoteDirectory" class="java.lang.String">
<constructor-arg
value="stock" />
</bean>
<int-ftp:inbound-channel-adapter
local-directory="# {stockLocalDirectory}"
channel="stockFilesFromFTP"
session-factory="publishStockSessionFactory"
remote-directory="#{stockRemoteDirectory}"
delete-remote-files="true"
filename-regex="Stock*.csv" >
<int:poller fixed-rate="120000" max-messages-per-poll="100" />
</int-ftp:inbound-channel-adapter>
<int:publish-subscribe-channel id="stockFilesFromFTP" />
And the error coming while start up is
INFO | jvm 1 | main | 2013/01/15 22:20:02.715 | 2013-01-15 22:20:02,699
ERROR task-scheduler-4 ErrorHandler : failed to send message
to channel 'stockFilesFromFTP' within timeout: -1
Debugs we turned on are
log4j.logger.org.springframework.aop=DEBUG
org.springframework.integration.channel.DirectChannel=DEBUG
org.springframework.integration.channel.MessagePublishingErrorHandler=DEBUG
org.springframework.integration.config.xml.PointToPointChannelParser=DEBUG
Can you suggest me how I can debug this error?
Run with DEBUG level logging.
You must have at least one subscriber on channel stockFilesFromFTP.
I used following kind of configuation for FTPclientfactory bean. It has timeout param also, so in case you are facing issue due to connection time out , this will help you.
<bean id="testapp.standardFTPClientFactory" class="org.springframework.integration.ftp.DefaultFTPClientFactory"
abstract="true">
<property name="host" value="${ftp.host}"/>
<property name="username" value="${ftp.username}"/>
<property name="password" value="${ftp.password}"/>
<property name="port" value="${ftp.port}"/>
<property name="remoteWorkingDirectory" value="${ftp.remotedir}"/>
<property name="dataTimeout" value="3600000"/>
<property name="connectTimeout" value="${ftp.connectTimeout}"/>
<property name="clientMode" value="2"/>
</bean>
Related
I have following configuration in the application that works perfectly fine with queues and redelivers the messages when a RuntimeException occurs with transaction rollback as expected.
But the same configuration fails to redeliver message with topics and emits following warning message:
WARN DefaultMessageListenerContainer - Setup of JMS message listener invoker failed for destination 'XXX_TOPIC' - trying to recover. Cause: JTA transaction unexpectedly rolled back (maybe due to a timeout); nested exception is javax.transaction.RollbackException: One or more resources refused to commit (possibly because of a timeout in the resource - see the log for details). This transaction has been rolled back instead.
Configuration:
<bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQXAConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
<!-- <property name="sendTimeout" value="1000"/> -->
<property name="redeliveryPolicy" ref="redeliveryPolicy"/>
</bean>
<!-- Atomikos Connection Factory Wrapper For Gobal Tx -->
<bean id="queueConnectionFactoryBean" class="com.atomikos.jms.AtomikosConnectionFactoryBean">
<property name="uniqueResourceName" value="amq1" />
<property name="xaConnectionFactory" ref="amqConnectionFactory" />
</bean>
<bean id="jmsConnectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory">
<ref bean="queueConnectionFactoryBean"/>
</property>
<property name="sessionCacheSize" value="20"/>
<property name="cacheConsumers" value="false"/>
</bean>
<bean id="jtaTransactionManager" class="org.springframework.transaction.jta.JtaTransactionManager">
<property name="transactionManager">
<bean class="com.atomikos.icatch.jta.UserTransactionManager"
init-method="init"
destroy-method="close">
<property name="forceShutdown" value="false" />
</bean>
</property>
<property name="userTransaction">
<bean class="com.atomikos.icatch.jta.UserTransactionImp">
<property name="transactionTimeout" value="300" />
</bean>
</property>
</bean>
<!-- Topic configuration -->
<bean id="messageListener1" class="com.x.y.impl.JmsMessageListener"></bean>
<bean id="container1" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsConnectionFactory"/>
<property name="destinationName" value="XXX_TOPIC"/>
<property name="messageListener" ref="messageListener1" />
<property name="pubSubDomain" value="true" />
<property name="transactionManager" ref="jtaTransactionManager" />
<property name="concurrency" value="1" />
<property name="receiveTimeout" value="3000" />
<!-- For local session and Atomikos-->
<property name="sessionTransacted" value="true"/>
</bean>
<!-- Bean configuration -->
<bean id="messageListener2" class="com.x.y.impl.JmsMessageListener"></bean>
<bean id="container2" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsConnectionFactory"/>
<property name="destinationName" value="XXX_QUEUE"/>
<property name="messageListener" ref="messageListener2" />
<property name="pubSubDomain" value="false" />
<property name="transactionManager" ref="jtaTransactionManager" />
<property name="concurrency" value="1" />
<!-- For local session and Atomikos-->
<property name="sessionTransacted" value="true"/>
</bean>
The Spring version used is 3.1, ActiveMQ 5.14, Atomikos 4.0.6.
Please let me know if i missed any configuration for topic DMLC.
<bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="${mq.activemq.host}" />
<property name="userName" value="${mq.activemq.user}" />
<property name="password" value="${mq.activemq.pass}" />
<property name="maxThreadPoolSize" value="30" />
</bean>
<bean id="amqPooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" destroy-method="stop">
<property name="connectionFactory" ref="amqConnectionFactory" />
<property name="maxConnections" value="10" />
<property name="maximumActiveSessionPerConnection" value="300" />
<property name="idleTimeout" value="60000" />
</bean>
<bean id="queueListenerContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="amqPooledConnectionFactory" />
<property name="destination" ref="queueDestination" />
<property name="messageListener" ref="queueAwareMessageListener" />
<property name="taskExecutor" ref="queueListenerTaskExecutor" />
<property name="concurrency" value="5-30" />
</bean>
What is the relationship between maxThreadPoolSize, maxConnections, maximumActiveSessionPerConnection, and concurrency?
Why I set maxConnections = 10, but Listener only one Connector in the Connections, can't increase more?
The number of consumers is correct. It has 5 with initialization,and gradually increases with change.
The bottom line regarding the number of connections which will be used is determined by the org.springframework.jms.listener.DefaultMessageListenerContainer which you have configured (since that is the only component here actually creating connections). As far as I can tell it will only ever create a single connection so using a connection pool here appears pointless. The concurrency parameter simply controls the concurrent number of consumers on the connection.
By setting maxConnections = 10 on the org.apache.activemq.pool.PooledConnectionFactory you are just limiting the size of the connection pool. However, since the queueListenerContainer won't ever call createConnection() more than once it doesn't really matter.
You can read about the maxThreadPoolSize of the org.apache.activemq.ActiveMQConnectionFactory in the ActiveMQ documentation.
(I hope this below makes sense, I had a bit of a struggle with formatting).
We have an application (currently being refreshed from technology of circa 2005) whose owners require the use of Oracle AQ for message handling.
We are using the Spring JDBC extensions, and when the application begins to listen to the queue it gets a ClassCastException saying "X cannot be cast to oracle.jdbc.internal.OracleConnection".
Where X is either a WrappedConnectionJDK7 from jboss when we use a datasource defined in Wildfly e.g.
<bean id="dataSource-supsrep"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#localhost:1521:XE" />
<property name="username" value="...." />
<property name="password" value="...." />
</bean>
or X is a PoolGuardConnectionWrapper from commons.dbcp when we define the datasource like this:
<bean id="dataSource-supsrep"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#localhost:1521:XE" />
<property name="username" value="...." />
<property name="password" value="...." />
</bean>
Message Listener bean is defined like this :
<bean id="messageListener" class="xxx.report.dispatch.DispatchMDB" />
Message Listener container bean is like this :
<bean id="reportProcessorXslfoHigh"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="sessionTransacted" value="true" />
<property name="connectionFactory" ref="AQconnectionFactory" />
<property name="destinationName" value="Q1_XSLFO_HIGH" />
<property name="messageListener" ref="messageListener" />
<property name="transactionManager" ref="transactionManager" />
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="1" />
<property name="receiveTimeout" value="30000" />
<property name="idleTaskExecutionLimit" value="30" />
<property name="idleConsumerLimit" value="1" />
</bean>
The connection factory is like this :
<orcl:aq-jms-connection-factory id="AQconnectionFactory"
data-source="dataSource-supsrep" native-jdbc- extractor="oracleNativeJdbcExtractor" />
and it uses a Native JDBC Extractor :
<bean id="oracleNativeJdbcExtractor"
class="org.springframework.jdbc.support
.nativejdbc.OracleJdbc4NativeJdbcExtractor" />
So my question is - can this be overcome by Spring configuration details - or is there another way round it? Clearly the underlying connection is needed but after 3 days of research I cannot see how it can be done.
Any clues gratefully received !
Thanks,
John
how to configure jms queue(using activemq inside karaf) inside blueprint.xml which present inside karaf deploy folder..
below is my code which shows config for jms connection inside blueprint.xml..
<bean id="activemqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616" />
<property name="userName" value="karaf" />
<property name="password" value="karaf" />
</bean>
<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="maxConnections" value="8" />
<property name="connectionFactory" ref="activemqConnectionFactory" />
</bean>
<bean id="resourceManager" class="org.apache.activemq.pool.ActiveMQResourceManager" init-method="recoverResource">
<property name="transactionManager" ref="transactionManager" />
<property name="connectionFactory" ref="activemqConnectionFactory" />
<property name="resourceName" value="activemq.localhost" />
</bean>
<reference id="transactionManager" interface="javax.transaction.TransactionManager" />
<service ref="activemqConnectionFactory" interface="javax.jms.ConnectionFactory">
<service-properties>
<entry key="name" value="connectionFactory" />
<entry key="osgi.jndi.service.name" value="jms/connectionFactory" />
</service-properties>
</service>
You don't configure a queue in OSGi, like you did in J2EE. You just use the connection factory in your plain Java or to configure the camel-jms component.
I got one strange problem.
when I config a DataSourceTransactionManager with spring xml, the concurrent consumers of ActiveMQ were suppressed whatever I change "maxConcurrentConsumers" property value. I have 5 queues, the total concurrent consumers of all 5 queue always kept at 8.
if I remove DataSourceTransactionManager bean, each queue's concurrent consumers reached the max number 5 declared in "maxConcurrentConsumers" .
The DataSourceTransactionManager work for dataSource, i cannot understand why it affected to ActiveMQ.
version:
Spring 3.2.5.RELEASE
ActiveMq 5.9.0
application.xml
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driver}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.user}" />
<property name="password" value="${jdbc.password}" />
</bean>
<!-- once I add this, activemq total consumers always kept at 8 -->
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<!-- activemq consumer connection -->
<bean id="consumerConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory"
destroy-method="stop">
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>tcp://localhost:61616</value>
</property>
</bean>
</property>
<property name="maxConnections" value="5"></property>
</bean>
<!-- i have 5 queues -->
<bean id="test_1" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_1}" />
</bean>
<bean id="test_2" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_2}" />
</bean>
<bean id="test_3" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_3}" />
</bean>
<bean id="test_4" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_4}" />
</bean>
<bean id="test_5" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_5}" />
</bean>
<!-- consumer listener container -->
<bean id="testOneMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_1"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testTwoMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_2"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testThreeMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_3"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testFourMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_4"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testFiveMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_5"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
can someone help me!!!
after some test, i found a way to resolve this problem.
when i change the dataSource "maxActive" parameter to a number greater than sum of all mq listener maxConcurrentConsumers. it work fine.
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driver}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.user}" />
<property name="password" value="${jdbc.password}" />
<property name="maxActive" value="120" />
</bean>
it seems that the max number of activemq listener thread affected by Datasource maxActive parameter