I'm using Apache camel 2.16.0 and CXF 3.1.2
How can i configure connection pooling (in cxf.xml) for all the http calls so that it can reuse the existing http connection instead of create a new connection every time ?
I see too may TIME_WAIT, CLOSE_WAIT and FIN_WAIT2 for each request, How can i close idle connections ?
<bean id="http" class="org.apache.camel.component.http.HttpComponent">
<property name="camelContext" ref="camel_context" />
<property name="httpConnectionManager" ref="myHttpConnectionManager" />
</bean>
<bean id="myHttpConnectionManager"
class="org.apache.commons.httpclient.MultiThreadedHttpConnectionManager">
<property name="params" ref="myHttpConnectionManagerParams" />
</bean>
<bean id="myHttpConnectionManagerParams"
class="org.apache.commons.httpclient.params.HttpConnectionManagerParams">
<property name="connectionTimeout" value="20000" />
<property name="soTimeout" value="20000" />
<property name="defaultMaxConnectionsPerHost" value="5" />
<property name="maxTotalConnections" value="10" />
</bean>
seems like maxTotalConnections isn't working !
maxTotalConnections is 10 but when i do a load test i can see more than 230 ESTABLISHED connections.. not only this
all the properties like connectionTimeout, soTimeout, defaultMaxConnectionsPerHost are all ignored !
any help is appreciated..
Related
I have two standalone broker consuming message from another system.
The client consumer uses Camel to consume messages, below is the part of the configuration where it used camel-activemq component.
I am trying to provide both the host information within the borkerURL but throwing some exception.
using jars of
spring-core 4.2.4,
spring-boot: 1.3.1,
activemq-camel: 5.12.1,
activemq-broker: 5.12.1, etc.
...
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="(tcp://host1:61616,tcp://host2:61616)?wireFormat.maxInactivityDuration=500000" />
</bean>
<bean id="jmsPooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="maxConnections" value="10" />
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="jmsPooledConnectionFactory" />
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="1" />
</bean>
<bean id="jms" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig" />
<property name="transacted" value="true" />
</bean>
...
The process throw exception, when i run the camel main with the context.
:PropertyAccessException 1: org.springframework.beans.MethodInvocationException: Property 'brokerURL' threw exception; nested exception is java.lang.IllegalArgumentException: Invalid broker URI: (tcp://host1:61616,tcp://host2:61616)?wireFormat.maxInactivityDuration=500000
Is there any way to pass in the borker URL of two standalone brokers?
If the brokers are part of a failover system, use the "failover" protocol of the ActiveMQ client.
If the brokers have different queues, create two different commponents with different configurations and broker URLs.
refer to jms1 or jms2 in your route.
We are using Spring Integration in our project and we have a requirement where If IBM MQ goes down then we will have to auto connect to IBM MQ when it is up. We have done this implementation using recoveryInterval option of org.springframework.jms.listener.DefaultMessageListenerContainer class. We have given recovery interval as 6 seconds so every 6 seconds system try to recover the MQ connection but now we have a requirement where we will have to do the autorecover twice only and after that if still MQ is down then stop the inbound adapter.
Is there any way in Spring Integration to mention the auto recovery retry count so that system will try to recover only for that retry count?
Below is my existing configuration.
<bean id="inQ" class="com.ibm.mq.jms.MQQueue">
<constructor-arg value="${mq.inbound.queue}" />
</bean>
<int:channel id="inbound" />
<int-jms:message-driven-channel-adapter
id="jmsIn" channel="inbound" container="messageListenerContainer"
acknowledge="transacted" auto-startup="false">
</int-jms:message-driven-channel-adapter>
<int:service-activator id="mainService"
input-channel="inbound" ref="messageListener" method="onMessage">
<int:request-handler-advice-chain>
<ref bean="retryWithBackoffAdviceSession" />
</int:request-handler-advice-chain>
</int:service-activator>
<bean id="messageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="mqConnectionFactory" />
<property name="destination" ref="inQ" />
<property name="sessionTransacted" value="true" />
<property name="maxConcurrentConsumers" value="${maxConcurrentConsumers}" />
<property name="concurrentConsumers" value="${concurrentConsumers}" />
<property name="receiveTimeout" value="${receiveTimeout}" />
<property name="recoveryInterval" value="60000" />
<property name="autoStartup" value="${autoStartup}" />
</bean>
Thanks
Sach
As an alternative to the recoveryInterval, you can now specify a Backoff instead (see the docs).
It doesn't provide a mechanism to stop the container but an appropriate backoff can effectively do what you want.
You would then need to programmatically stop/start to kick it off again.
My build in Travis CI failed after add configuration on Active MQ.
The log of Travis CI shows:
Caused by: javax.jms.JMSException: Could not connect to broker URL: tcp://localhost:61616. Reason: java.net.ConnectException: Connection refused
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:36)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:360)
at org.apache.activemq.ActiveMQConnectionFactory.createActiveMQConnection(ActiveMQConnectionFactory.java:305)
at org.apache.activemq.ActiveMQConnectionFactory.createConnection(ActiveMQConnectionFactory.java:245)
at org.springframework.jms.connection.SingleConnectionFactory.doCreateConnection(SingleConnectionFactory.java:365)
at org.springframework.jms.connection.SingleConnectionFactory.initConnection(SingleConnectionFactory.java:305)
at org.springframework.jms.connection.SingleConnectionFactory.getConnection(SingleConnectionFactory.java:283)
at org.springframework.jms.connection.SingleConnectionFactory.createConnection(SingleConnectionFactory.java:224)
at org.springframework.jms.support.JmsAccessor.createConnection(JmsAccessor.java:180)
at org.springframework.jms.listener.AbstractJmsListeningContainer.createSharedConnection(AbstractJmsListeningContainer.java:413)
at org.springframework.jms.listener.AbstractJmsListeningContainer.establishSharedConnection(AbstractJmsListeningContainer.java:381)
at org.springframework.jms.listener.AbstractJmsListeningContainer.doStart(AbstractJmsListeningContainer.java:285)
at org.springframework.jms.listener.SimpleMessageListenerContainer.doStart(SimpleMessageListenerContainer.java:209)
at org.springframework.jms.listener.AbstractJmsListeningContainer.start(AbstractJmsListeningContainer.java:270)
... 44 more
I'm using Spring MVC, here's the configuration for testing:
<beans "Some DTDs here">
<!-- Base Components -->
<context:annotation-config />
<context:component-scan base-package="com.trunkshell.voj" />
<!-- Some other configuration for JDBC and other service -->
<!-- Message Service -->
<bean id="amqConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<constructor-arg index="0" value="tcp://localhost:61616"/>
</bean>
<bean id="connectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<constructor-arg ref="amqConnectionFactory" />
</bean>
<bean id="defaultDestination" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="vojMessageQueue"/>
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="connectionFactory" />
<property name="defaultDestination" ref="defaultDestination" />
</bean>
<bean class="org.springframework.jms.listener.SimpleMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory" />
<property name="destinationName" value="vojMessageQueue" />
<property name="messageListener" ref="messageReceiver" />
</bean>
<bean id="messageSender" class="com.trunkshell.voj.util.MessageSender"></bean>
<bean id="messageReceiver" class="com.trunkshell.voj.util.MessageReceiver"></bean>
</beans>
I also noticed that Travis CI only support RabbitMQ and ZeroMQ.
Here's the reference: link
So what should I do to fix the error in Travis CI?
Or how to skip this problem in Travis CI.
I don't see that you actually start an ActiveMQ broker at port 61616. Either, do that, or preferably use the VM transport during the integration tests. I.e. change tcp://localhost:61616 to vm://localhost?broker.persistent=false
Of course, you have to be able to change that ActiveMQ URL when you deploy to a production environment. Maybe you can use spring property placeholders?
I have a spring cxf web service application deployed into a JBOSS server. The service is working fine and once in a while(within 5-6 days after the server start) , I get and error"Could not open JDBC Connection for transaction; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object" and that particular service goes down
I have around 17 services inside this application and only the service which had this DB connection issue goes down till I restart the server.
Other services are up.
Below is my JDBC template configuration.
<bean id="dataSource" destroy-method="close"
class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<property name="initialSize" value="10" />
<property name="maxIdle" value="10" />
<property name="maxActive" value="100" />
<property name="maxWait" value="1000" />
<property name="validationQuery" value="select 1 from sysibm.sysdummy1" />
<property name="testOnBorrow" value="true" />
<property name="testWhileIdle" value="true" />
<property name="timeBetweenEvictionRunsMillis" value="1200000" />
<property name="minEvictableIdleTimeMillis" value="1800000" />
<property name="numTestsPerEvictionRun" value="5" />
<property name="defaultAutoCommit" value="false" />
</bean>
Your help is greately appreciated.
I'd recommend that you switch to a JNDI data source managed by JBOSS.
It sounds like either a connection leak or that you have some really long-running processes that hold a database connection long-term, eventually exhausting your connection pool. It's also possible, though unlikely, that you have very high database latency, that a dead connection shows up in the pool, and that recognizing the dead connection, establishing a new one, and validating it takes longer than the 1 second you've given the pool.
You can try and change the datasource to the :
org.springframework.jdbc.datasource.DriverManagerDataSource
It's better to configure a data source with JBoss and do a Java EE-jndi lookup.
http://techdive.in/spring/spring-jndi-datasource-configuration-jboss
How can I setup connection pooling in Spring MVC? I am working on an intranet website powered by Spring MVC 2.5 and jQuery. This is my first attempt at web development.
I am not sure but, I am only using this in my spring configuration file and I saw this in the Spring MVC step By Step tutorial
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
</bean>
<bean id="propertyConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath:jdbc.properties</value>
</list>
</property>
</bean>
This looks good during development and connection speed is fast but I am not sure if this will still holds true if many users are concurrently connected.
How can I achieve this? I have read that this is not an optimal connection datasource.
You might want to look at c3p0, which has some powerful configuration and optimization available.
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
<property name="driverClass" value="..." />
<property name="jdbcUrl" value="..." />
<property name="user" value="..." />
<property name="password" value="..." />
</bean>
Your current setup is correct, all you need to do in order to use basic connection pooling is use a DataSource implementation provided by a connection pooling library, in your case Apache DBCP. See this post for a few links to other alternatives, C3P0 being one of them.
Note that when you actually use the DataSource bean you're injecting wrap it in a SimpleJdbcTemplate or use DataSourceUtils to obtain a Connection - see Spring JDBC Documentation
For connection Pooling
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
//Add this two more parameters
<property name="**initialSize**" value="20" />
<property name="**maxActive**" value="30" />
</bean>
connection pool will create 20 database connection as initialSize is 20 and goes up to 30 Database connection if required as maxActive is 30.