Tomcat Pool Empty - spring

We are using Tomcat 6 with tomcat-jdbc.jar and tomcat pooling (we are currently using the latest version 7.0.28, but have tried with previous versions as well).
This is using PostgreSQL on the back-end.
About once every 2 - 5 days (it's rather "random" in happening), our server uses up all of the available db connections, in about a 10 minute period, and we see these in the logs (we've turned on FINE logging):
2012-10-01 18:40:29,998 ERROR [TP-Processor29] JDBCExceptionReporter.logExceptions(72) | [TP-Processor29] Timeout: Pool empty. Unable to fetch a connection in 0 seconds, none available[size:150; busy:41; idle:0; lastwait:0].
2012-10-01 18:40:30,000 ERROR [TP-Processor29] JDBCExceptionReporter.logExceptions(72) | [TP-Processor29] Timeout: Pool empty. Unable to fetch a connection in 0 seconds, none available[size:150; busy:41; idle:0; lastwait:0].
I'm trying to track down the root cause problem of this.
When looking at the db and this occurs, all of the connections are shown as "IDLE".
Our current configuration looks something like this:
<bean id="dataSource" class="org.apache.tomcat.jdbc.pool.DataSource" destroy-method="close">
<property name="driverClassName" value="org.postgresql.Driver"/>
<property name="url" value="jdbc:postgresql://db/dbname?useUnicode=true&characterEncoding=utf-8"/>
<property name="username" value="postgres"/>
<property name="password" value="postgres"/>
<property name="initialSize" value="8"/>
<property name="maxActive" value="150"/>
<property name="maxIdle" value="10"/>
<property name="maxWait" value="30"/>
<property name="defaultAutoCommit" value="true"/>
<property name="validationQuery" value="SELECT 1"/>
<property name="validationInterval" value="60000"/>
</bean>
And, our logging in logging.properties is:
org.apache.tomcat.jdbc.pool.level=FINE
Most of the time, we are using about 15 - 20 database connections, however, as I mentioned, the problem is that every few days, it rapidly uses up all of the connections, and requires a restart.
We're trying to track this down to being a Tomcat pool issue, a Hibernate issue, a Spring issue, or an app issue.
To start with, any ideas why:
A) Why does it show: [size:150; busy:41; idle:0; lastwait:0] ... where are the other 150-41 connections?
B) When logging, it shows when the connection opens, but not when it closes. How can we turn that on?

Related

c3p0 database pooling Statement close FAILED. java.sql.SQLRecoverableException: Closed Connection

I am newbie to Java and c3p0 database pooling. We have build api on mulesoft and deployed to cloudhub (vpc + vpn config), the application connects to database on premise.
The application logs shows below warning about connection closed every 12 hours (exactly)
16:59:32.799 07/28/2021 Worker-0 C3P0PooledConnectionPoolManager[identityToken->|41d2c93]-HelperThread-#2 WARN
Statement close FAILED.
java.sql.SQLRecoverableException: Closed Connection
at oracle.jdbc.driver.PhysicalConnection.needLine(PhysicalConnection.java:3525)
at oracle.jdbc.driver.OracleStatement.closeOrCache(OracleStatement.java:1478)
at oracle.jdbc.driver.OracleStatement.close(OracleStatement.java:1461)
at oracle.jdbc.driver.OracleStatementWrapper.close(OracleStatementWrapper.java:122)
at com.mchange.v1.db.sql.StatementUtils.attemptClose(StatementUtils.java:53)
at com.mchange.v2.c3p0.impl.DefaultConnectionTester.activeCheckConnection(DefaultConnectionTester.java:325)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:510)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.testPooledConnection(C3P0PooledConnectionPool.java:464)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishIdleResource(C3P0PooledConnectionPool.java:436)
at com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask.run(BasicResourcePool.java:2211)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
If a request is received within last few minutes reaching 12 hr time then it fails with 500 error connection.ConnectionException: An attempt by a client to checkout a Connection has timed out
I have configured spring based datasource and here is my configuration
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-4.2.xsd
http://www.springframework.org/schema/jdbc
http://www.springframework.org/schema/jdbc/spring-jdbc-4.2.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-4.2.xsd
http://www.springframework.org/schema/security
http://www.springframework.org/schema/security/spring-security-4.2.xsd">
<bean id="oracle-jdbcdatasource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="oracle.jdbc.driver.OracleDriver"/>
<property name="jdbcUrl" value="jdbc:oracle:thin:#${oracle.host}:${oracle.port}/${oracle.database}"/>
<property name="user" value="${oracle.username}"/>
<property name="password" value="${oracle.password}"/>
<property name="minPoolSize" value="1"/>
<property name="maxPoolSize" value="5"/>
<property name="initialPoolSize" value="1"/>
<property name="acquireIncrement" value="1"/>
<property name="idleConnectionTestPeriod" value="300"/>
<property name="maxStatements" value="0"/>
<property name="checkoutTimeout" value="60000"/>
<property name="testConnectionOnCheckout" value="true" />
<property name="testConnectionOnCheckin" value="true" />
<property name="preferredTestQuery" value="select 1 from dual" />
<property name="maxConnectionAge" value="14400" />
<property name="maxIdleTime" value="7200" />
</bean>
</beans>
I have set maxConnectionAge to 14400 (4hrs) and maxIdleTime to 7200 (2hr) so I was not expecting to see connection closed error as it should have destroyed the connection before 12 hours and new connection should be in-place.
Version details
Mule Runtime - 4.3.0
Oracle jdbc8 - 19.3.0.0
c3p0 - 0.9.5.2
Can someone please advise if I am missing any configuration for c3p0 pooling ?
Thanks
I don't think there is anything wrong with the pool configuration. It might be a re-connection issue in the VPN, or even some limit in the database itself.
It sounds like you may be checking out Connections, but not promptly checking them back in. c3p0 won't expire old Connections if they remain checked out, only after they have been checked in.
The "An attempt by a client to checkout a Connection has timed out" message suggests that eventually you've exhausted the Connection pool, there are no Connections left to check out and maxPoolSize has been reached.
When your application checks out Connections, it should use the try-with-resources construct to ensure the Connection is promptly checked in after use.

oracle.jdbc.ReadTimeout working but weirdly behaving

I'm trying to configure read timeout for a query using dbcp. If query execution takes more than one minute, I want to time out the query. My datasource is.If i put 250 then it is timing out after 27 seconds.i think 250 is in milliseconds but it is timing out wrongly.can you please help me how to set the value for oracle.jdbc.ReadTimeout.
<bean id="appDataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="username" value="${app.jdbc.username}" />
<property name="password" val="${app.jdbc.password}" />
<prop name="connectionproperties" val="oracle.jdbc.ReadTimeout=2000"/>
</bean>
Can you confirm the JDBC driver version that you are using?

H2 Multiple connection

I have two applications : app1 and app2 , and i want that this two application use the same H2 file as a database .
I test this configuration in the two sides but it didn't work :
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="org.h2.Driver" />
<property name="url" value="jdbc:h2:~/dBTrunk;MODE=Oracle;AUTO_SERVER=TRUE" />
<property name="username" value="sa" />
<property name="password" value="" />
</bean>
I am open to all kind of solution my only goal is that app1 and app2 can work in parralel mode with the same H2 database .
Regards
This should work. The first app should open to the database file in embedded mode, but then start a database server. The second app will then try to connect to the first app's database server.
If it is not possible for the two apps to talk to each other using TCP/IP, then it wont work. Is there a firewall between them? Are there certain ports blocked?

Spring service going down after DB connection down

I have a spring cxf web service application deployed into a JBOSS server. The service is working fine and once in a while(within 5-6 days after the server start) , I get and error"Could not open JDBC Connection for transaction; nested exception is org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object" and that particular service goes down
I have around 17 services inside this application and only the service which had this DB connection issue goes down till I restart the server.
Other services are up.
Below is my JDBC template configuration.
<bean id="dataSource" destroy-method="close"
class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<property name="initialSize" value="10" />
<property name="maxIdle" value="10" />
<property name="maxActive" value="100" />
<property name="maxWait" value="1000" />
<property name="validationQuery" value="select 1 from sysibm.sysdummy1" />
<property name="testOnBorrow" value="true" />
<property name="testWhileIdle" value="true" />
<property name="timeBetweenEvictionRunsMillis" value="1200000" />
<property name="minEvictableIdleTimeMillis" value="1800000" />
<property name="numTestsPerEvictionRun" value="5" />
<property name="defaultAutoCommit" value="false" />
</bean>
Your help is greately appreciated.
I'd recommend that you switch to a JNDI data source managed by JBOSS.
It sounds like either a connection leak or that you have some really long-running processes that hold a database connection long-term, eventually exhausting your connection pool. It's also possible, though unlikely, that you have very high database latency, that a dead connection shows up in the pool, and that recognizing the dead connection, establishing a new one, and validating it takes longer than the 1 second you've given the pool.
You can try and change the datasource to the :
org.springframework.jdbc.datasource.DriverManagerDataSource
It's better to configure a data source with JBoss and do a Java EE-jndi lookup.
http://techdive.in/spring/spring-jndi-datasource-configuration-jboss

Setup Connection Pooling in Spring MVC

How can I setup connection pooling in Spring MVC? I am working on an intranet website powered by Spring MVC 2.5 and jQuery. This is my first attempt at web development.
I am not sure but, I am only using this in my spring configuration file and I saw this in the Spring MVC step By Step tutorial
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
</bean>
<bean id="propertyConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<value>classpath:jdbc.properties</value>
</list>
</property>
</bean>
This looks good during development and connection speed is fast but I am not sure if this will still holds true if many users are concurrently connected.
How can I achieve this? I have read that this is not an optimal connection datasource.
You might want to look at c3p0, which has some powerful configuration and optimization available.
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource">
<property name="driverClass" value="..." />
<property name="jdbcUrl" value="..." />
<property name="user" value="..." />
<property name="password" value="..." />
</bean>
Your current setup is correct, all you need to do in order to use basic connection pooling is use a DataSource implementation provided by a connection pooling library, in your case Apache DBCP. See this post for a few links to other alternatives, C3P0 being one of them.
Note that when you actually use the DataSource bean you're injecting wrap it in a SimpleJdbcTemplate or use DataSourceUtils to obtain a Connection - see Spring JDBC Documentation
For connection Pooling
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
//Add this two more parameters
<property name="**initialSize**" value="20" />
<property name="**maxActive**" value="30" />
</bean>
connection pool will create 20 database connection as initialSize is 20 and goes up to 30 Database connection if required as maxActive is 30.

Resources