I am newbie to Java and c3p0 database pooling. We have build api on mulesoft and deployed to cloudhub (vpc + vpn config), the application connects to database on premise.
The application logs shows below warning about connection closed every 12 hours (exactly)
16:59:32.799 07/28/2021 Worker-0 C3P0PooledConnectionPoolManager[identityToken->|41d2c93]-HelperThread-#2 WARN
Statement close FAILED.
java.sql.SQLRecoverableException: Closed Connection
at oracle.jdbc.driver.PhysicalConnection.needLine(PhysicalConnection.java:3525)
at oracle.jdbc.driver.OracleStatement.closeOrCache(OracleStatement.java:1478)
at oracle.jdbc.driver.OracleStatement.close(OracleStatement.java:1461)
at oracle.jdbc.driver.OracleStatementWrapper.close(OracleStatementWrapper.java:122)
at com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:53)
at com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:325)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:510)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:464)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:436)
at com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:2211)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
If a request is received within last few minutes reaching 12 hr time then it fails with 500 error connection.ConnectionException: An attempt by a client to checkout a Connection has timed out
I have configured spring based datasource and here is my configuration
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-4.2.xsd
http://www.springframework.org/schema/jdbc
http://www.springframework.org/schema/jdbc/spring-jdbc-4.2.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-4.2.xsd
http://www.springframework.org/schema/security
http://www.springframework.org/schema/security/spring-security-4.2.xsd">
<bean id="oracle-jdbcdatasource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="oracle.jdbc.driver.OracleDriver"/>
<property name="jdbcUrl" value="jdbc:oracle:thin:#${oracle.host}:${oracle.port}/${oracle.database}"/>
<property name="user" value="${oracle.username}"/>
<property name="password" value="${oracle.password}"/>
<property name="minPoolSize" value="1"/>
<property name="maxPoolSize" value="5"/>
<property name="initialPoolSize" value="1"/>
<property name="acquireIncrement" value="1"/>
<property name="idleConnectionTestPeriod" value="300"/>
<property name="maxStatements" value="0"/>
<property name="checkoutTimeout" value="60000"/>
<property name="testConnectionOnCheckout" value="true" />
<property name="testConnectionOnCheckin" value="true" />
<property name="preferredTestQuery" value="select 1 from dual" />
<property name="maxConnectionAge" value="14400" />
<property name="maxIdleTime" value="7200" />
</bean>
</beans>
I have set maxConnectionAge to 14400 (4hrs) and maxIdleTime to 7200 (2hr) so I was not expecting to see connection closed error as it should have destroyed the connection before 12 hours and new connection should be in-place.
Version details
Mule Runtime - 4.3.0
Oracle jdbc8 - 19.3.0.0
c3p0 - 0.9.5.2
Can someone please advise if I am missing any configuration for c3p0 pooling ?
Thanks
I don't think there is anything wrong with the pool configuration. It might be a re-connection issue in the VPN, or even some limit in the database itself.
It sounds like you may be checking out Connections, but not promptly checking them back in. c3p0 won't expire old Connections if they remain checked out, only after they have been checked in.
The "An attempt by a client to checkout a Connection has timed out" message suggests that eventually you've exhausted the Connection pool, there are no Connections left to check out and maxPoolSize has been reached.
When your application checks out Connections, it should use the try-with-resources construct to ensure the Connection is promptly checked in after use.
Related
I'm using Apache camel 2.16.0 and CXF 3.1.2
How can i configure connection pooling (in cxf.xml) for all the http calls so that it can reuse the existing http connection instead of create a new connection every time ?
I see too may TIME_WAIT, CLOSE_WAIT and FIN_WAIT2 for each request, How can i close idle connections ?
<bean id="http" class="org.apache.camel.component.http.HttpComponent">
<property name="camelContext" ref="camel_context" />
<property name="httpConnectionManager" ref="myHttpConnectionManager" />
</bean>
<bean id="myHttpConnectionManager"
class="org.apache.commons.httpclient.MultiThreadedHttpConnectionManager">
<property name="params" ref="myHttpConnectionManagerParams" />
</bean>
<bean id="myHttpConnectionManagerParams"
class="org.apache.commons.httpclient.params.HttpConnectionManagerParams">
<property name="connectionTimeout" value="20000" />
<property name="soTimeout" value="20000" />
<property name="defaultMaxConnectionsPerHost" value="5" />
<property name="maxTotalConnections" value="10" />
</bean>
seems like maxTotalConnections isn't working !
maxTotalConnections is 10 but when i do a load test i can see more than 230 ESTABLISHED connections.. not only this
all the properties like connectionTimeout, soTimeout, defaultMaxConnectionsPerHost are all ignored !
any help is appreciated..
I've spent the last couple of days debugging some really strange errors.
I have an exposed WebService which calls a service-metod called "addNewOrder"
This service-method is annotated with
#Transactional(readOnly = false, propagation = Propagation.REQUIRES_NEW)
This service-method will need to read and write to the database.
I would think that the annotation above would ensure that every time this method is called, i'm getting a new transaction that can do what is needed.
But about 50% of the time, when "addNewOrder" is called, i get these strange errors:
org.hibernate.engine.jdbc.spi.SqlExceptionHelper:143 - SQL Error: 0, SQLState: S1009
org.hibernate.engine.jdbc.spi.SqlExceptionHelper:144 - Connection is read-only. Queries leading to data modification are not allowed
Does anyone have any clue as to why this might happen? And why is it random?
Im using Spring 3.2.4, Hibernate 4.1.7, Mysql-connector 5.1.26
This is my applicationContext snippet:
<?xml version="1.0" encoding="iso-8859-1"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-3.1.xsd">
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource"/>
<property name="packagesToScan" value="no.aida.model"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="${jpa.show.sql}"/>
<property name="generateDdl" value="${jpa.generate.ddl}"/>
<property name="databasePlatform" value="${jpa.dialect}"/>
</bean>
</property>
<property name="jpaDialect">
<bean id="jpaDialect" class="no.aida.dao.hibernate.IsolationSupportHibernateJpaDialect" />
</property>
</bean>
<!-- javax.sql.DataSource supplied by Jakarta Commons Connection Pooling -->
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${mysql.db.driver}"/>
<property name="url" value="${mysql.db.url}"/>
<property name="username" value="${mysql.db.username}"/>
<property name="password" value="${mysql.db.password}"/>
<property name="initialSize" value="3" />
<property name="maxActive" value="30" />
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory"/>
<property name="dataSource" ref="dataSource"/>
</bean>
<!-- Enable #Transactional support -->
<tx:annotation-driven transaction-manager="transactionManager"/>
</beans>
Well i'll be damned. Just a few mins after posting this i finally found the error. The reason it is random is because I have another service-method called "getStatus" which is annotated by read-only. And if this is called then every request using the same connection will also be read-only. But i still cant understand why the read-only is not re-set when the connection is used on a method annotated with readOnly = false...
And to add to the saga. The reason why the connections was not reset, was because we had implemented a custom JpaDialect that had forgotten to reset the connections... doooooh!
value="${mysql.db.url}"
Could you please specify the connect url? Do you use:
failOverReadOnly=false
or not? This should help.
BR, Ilya
I'm stumbled upon a problem while developing a Web Application based on Struts2 with Spring and Hibernate.
When I refresh the site a few times, for example 2-4 times, Hibernate is showing up an exception about Too many connections. I've tried to implement C3P0 Connection pool and have some problems with it
The hibernate.cfg.xml configuration:
<property name="hibernate.dialect">org.hibernate.dialect.MySQLDialect</property>
<property name="hibernate.connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="hibernate.connection.url">jdbc:mysql://localhost:3306/axelle</property>
<property name="hibernate.connection.username">axelle</property>
<property name="hibernate.connection.password">dbpassword</property>
<property name="hibernate.current_session_context_class">thread</property>
<property name="hibernate.c3p0.min_size">5</property>
<property name="hibernate.c3p0.max_size">20</property>
<property name="hibernate.c3p0.timeout">300</property>
<property name="hibernate.c3p0.max_statements">50</property>
<property name="hibernate.c3p0.idle_test_period">3000</property>
applicationContext.xml
<bean id="propertyConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"
p:location="classpath:jdbc.properties"/>
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource"
p:driverClassName="${jdbc.driverClassName}"
p:url="${jdbc.url}"
p:username="${jdbc.username}"
p:password="${jdbc.password}"
p:connectionProperties="${jdbc.connectionProperties}"/>
<!-- ADD PERSISTENCE SUPPORT HERE (jpa, hibernate, etc) -->
<bean id="sessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="configLocation">
<value>classpath:hibernate.cfg.xml</value>
</property>
</bean>
<!-- Transaction manager for a single Hibernate SessionFactory (alternative to JTA) -->
<bean id="transactionManager" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory">
<ref local="sessionFactory"/>
</property>
</bean>
The log output is:
org.hibernate.exception.JDBCConnectionException: Cannot open connection
and:
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: "Too many connections"
And this is how PROCESSLIST MySQL window looks: http://img844.imageshack.us/img844/3959/be69273cc2.png
I've set max_size of connections to 20, but it seems like it doesn't read the C3P0 configuration from file, cause from the screen we can see that number of connections is higher than 20, or maybe I'm doing something wrong, but where? I really need some help guys, I'll appreciate this, and thanks in advance.
Mention these property in your hibernate.cfg.xml file
<property name="hibernate.c3p0.acquire_increment">1</property>
<property name="hibernate.c3p0.idle_test_period">100</property>
<property name="hibernate.c3p0.max_size">10</property>
<property name="hibernate.c3p0.max_statements">10</property>
<property name="hibernate.c3p0.min_size">10</property>
<property name="hibernate.c3p0.timeout">100</property>
Refer this link for better understanding: Configuration Properties
In your applicationContext, instead of using <bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource" ... try using <bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" ...
The C3P0PooledDataSource will create a wrapped database connection using the specified DriverClassName, url, username and password
We are using Tomcat 6 with tomcat-jdbc.jar and tomcat pooling (we are currently using the latest version 7.0.28, but have tried with previous versions as well).
This is using PostgreSQL on the back-end.
About once every 2 - 5 days (it's rather "random" in happening), our server uses up all of the available db connections, in about a 10 minute period, and we see these in the logs (we've turned on FINE logging):
2012-10-01 18:40:29,998 ERROR [TP-Processor29] JDBCExceptionReporter.logExceptions(72) | [TP-Processor29] Timeout: Pool empty. Unable to fetch a connection in 0 seconds, none available[size:150; busy:41; idle:0; lastwait:0].
2012-10-01 18:40:30,000 ERROR [TP-Processor29] JDBCExceptionReporter.logExceptions(72) | [TP-Processor29] Timeout: Pool empty. Unable to fetch a connection in 0 seconds, none available[size:150; busy:41; idle:0; lastwait:0].
I'm trying to track down the root cause problem of this.
When looking at the db and this occurs, all of the connections are shown as "IDLE".
Our current configuration looks something like this:
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource" destroy-method="close">
<property name="driverClassName" value="org.postgresql.Driver"/>
<property name="url" value="jdbc:postgresql://db/dbname?useUnicode=true&characterEncoding=utf-8"/>
<property name="username" value="postgres"/>
<property name="password" value="postgres"/>
<property name="initialSize" value="8"/>
<property name="maxActive" value="150"/>
<property name="maxIdle" value="10"/>
<property name="maxWait" value="30"/>
<property name="defaultAutoCommit" value="true"/>
<property name="validationQuery" value="SELECT 1"/>
<property name="validationInterval" value="60000"/>
</bean>
And, our logging in logging.properties is:
org.apache.tomcat.jdbc.pool.level=FINE
Most of the time, we are using about 15 - 20 database connections, however, as I mentioned, the problem is that every few days, it rapidly uses up all of the connections, and requires a restart.
We're trying to track this down to being a Tomcat pool issue, a Hibernate issue, a Spring issue, or an app issue.
To start with, any ideas why:
A) Why does it show: [size:150; busy:41; idle:0; lastwait:0] ... where are the other 150-41 connections?
B) When logging, it shows when the connection opens, but not when it closes. How can we turn that on?
I have a spring cxf web service application deployed into a JBOSS server. The service is working fine and once in a while(within 5-6 days after the server start) , I get and error"Could not open JDBC Connection for transaction; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object" and that particular service goes down
I have around 17 services inside this application and only the service which had this DB connection issue goes down till I restart the server.
Other services are up.
Below is my JDBC template configuration.
<bean id="dataSource" destroy-method="close"
class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<property name="initialSize" value="10" />
<property name="maxIdle" value="10" />
<property name="maxActive" value="100" />
<property name="maxWait" value="1000" />
<property name="validationQuery" value="select 1 from sysibm.sysdummy1" />
<property name="testOnBorrow" value="true" />
<property name="testWhileIdle" value="true" />
<property name="timeBetweenEvictionRunsMillis" value="1200000" />
<property name="minEvictableIdleTimeMillis" value="1800000" />
<property name="numTestsPerEvictionRun" value="5" />
<property name="defaultAutoCommit" value="false" />
</bean>
Your help is greately appreciated.
I'd recommend that you switch to a JNDI data source managed by JBOSS.
It sounds like either a connection leak or that you have some really long-running processes that hold a database connection long-term, eventually exhausting your connection pool. It's also possible, though unlikely, that you have very high database latency, that a dead connection shows up in the pool, and that recognizing the dead connection, establishing a new one, and validating it takes longer than the 1 second you've given the pool.
You can try and change the datasource to the :
org.springframework.jdbc.datasource.DriverManagerDataSource
It's better to configure a data source with JBoss and do a Java EE-jndi lookup.
http://techdive.in/spring/spring-jndi-datasource-configuration-jboss