When uploaded our application on prod server we faced strange behaviour - sometimes it stopped to get data from data base as though the logic and everything was right and local version was working perfect. After remote debugging we found that GenericObjectPool was blocking thread. And after some time spent on searching we found situation that fits our problem.
The pool just got exhausted when filled with 8 connections (default value) and by default it's behaviour is set to block thread when exhausted.
Here is my datamodel-context.xml config of BasicDataSource
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="url_to_db"/>
<property name="username" value="username"/>
<property name="password" value="12211"/>
<property name="defaultAutoCommit" value="false"/>
<property name="poolPreparedStatements" value="true"/>
<property name="initialSize" value="10"/>
<property name="maxIdle" value="5"/>
<property name="testOnBorrow" value="true"/>
<property name="logAbandoned" value="true"/>
</bean>
Found here two solutions - increase max active number or change pool's behaviour.
The first one seems to be simple - set property maxActive in BasicDataSource to some number (please correct me if I am mistaking). This way is unwanted because we can not know the exact number of possible simultanious connections.
It was decided to try the second way - to change behaviour to WHEN_EXHAUSTED_GROW.
So is there any way to configure spring default pool to change it's whenExhaustedAction behaviour? Or should I define my own connection pool for BasicDataSource? If so, please can you provide examples?
I appreciate any comments or advices. Thanks.
Related
Can a HikariCP Datasource be started with a Lazy configuration?
For that, i'm using Spring LazyConnectionDataSourceProxy.
<bean id="hikariConfig" class="com.zaxxer.hikari.HikariConfig" lazy-init="true">
<property name="poolName" value="TargetHikariCP" />
<property name="dataSourceClassName" value="oracle.jdbc.pool.OracleDataSource" />
<property name="connectionInitSql" value="SELECT 1 FROM DUAL"/>
<property name="leakDetectionThreshold" value="300000"/>
<property name="minimumIdle" value="1"/>
<property name="maximumPoolSize" value="10"/>
<property name="autoCommit" value="false"/>
<property name="dataSourceProperties"> <props> ... </props> </property>
</bean>
<bean id="dataSourceLazy" class="com.zaxxer.hikari.HikariDataSource" destroy-method="close" lazy-init="true">
<constructor-arg ref="hikariConfig" />
</bean>
<bean id="dataSource"
class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy">
<property name="targetDataSource" ref="dataSourceLazy" />
</bean>
<bean id="txManager"
class="org.springframework.jdbc.datasource.DataSourceTransactionManager" lazy-init="true">
<property name="dataSource" ref="dataSource" />
</bean>
Nevertheless, its not working, as the Datasource is started on project startup.
The same configuration, when using a org.springframework.jdbc.datasource.DriverManagerDataSource, works correctly.
In the version > 3 we can set setInitializationFailTimeout(-1);
According to docs:
Any value greater than zero will be treated as a timeout for pool initialization.The calling thread will be blocked from continuing until a successful connection
to the database, or until the timeout is reached. If the timeout is reached, then
a PoolInitializationException will be thrown.
A value of zero will not prevent the pool from starting in the
case that a connection cannot be obtained. However, upon start the pool will
attempt to obtain a connection and validate that the connectionTestQuery
and connectionInitSql are valid. If those validations fail, an exception
will be thrown. If a connection cannot be obtained, the validation is skipped
and the the pool will start and continue to try to obtain connections in the
background. This can mean that callers to DataSource#getConnection() may
encounter exceptions.
A value less than zero will bypass any connection attempt and validation during
startup, and therefore the pool will start immediately. The pool will continue to
try to obtain connections in the background. This can mean that callers to
DataSource#getConnection() may encounter exceptions.
HikariCP has a property, initializationFailFast, that controls whether the pool will "fail fast" if the pool cannot be seeded with initial connections successfully:
This property controls whether the pool will "fail fast" if the pool cannot be seeded with initial connections successfully. If you want your application to start even when the database is down/unavailable, set this property to false. Default: true
This property was documented in their site, but per version 2.6.2 its not, but it seems its still supported.
In my use case, the use of this property should be enough to solve my problem.
I have a Spring Hibernate application on Tomcat.
Connection pool is C3P0
I am rapidly encountering a thread pool maxed out warning from C3P0. Then all requests to the webapp hang.
I am still assuming that somewhere in the code I have missed an #Transaction annotation. I want to debug my code.
Question: Can I access the connection pool via code so I can debug when a connection is released and when it is not released?
UPDATE: Current c3p0 config:
<!-- Hibernate -->
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close">
<property name="driverClass" value="org.postgresql.Driver" />
<property name="jdbcUrl" value="jdbc:postgresql://localhost/ikoda01?useUnicode=true&characterEncoding=utf8" />
<property name="user" value="xxxuser" />
<property name="password" value="xxxxx" />
<property name="acquireIncrement" value="2" />
<property name="minPoolSize" value="3" />
<property name="maxPoolSize" value="50" />
<property name="maxIdleTime" value="600" />
</bean>
Can I access the connection pool via code so I can debug when a connection is released and when it is not released?
Yes
You don't even have to code, that's built in. If you configure c3p0 to debug Connection leaks, it will simply print stack traces of the codepaths that checked out the leaked Connections to your logs.
Update:
<property name="unreturnedConnectionTimeout" value="30" />
<property name="debugUnreturnedConnectionStackTraces" value="true" />
The value you should use for unreturnedConnectionTimeout depends on your application. It should be longer than the longest expected legit use of a Connection, but not the shorter it is, the more quickly you'll get log messages about the leak and the more smoothly your app will work around it. For most web-ish applications, the 30 secs shown above is conservative, clients aren't expected to ever wait around 30 secs for a response, so a timeout safely indicates a Connection leak.
I am developping a Spring MVC web application that use the dbcp database connection pool.
<bean id="datasourceAR_XXX" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" scope="singleton">
<property name="driverClassName"><value>oracle.jdbc.driver.OracleDriver</value></property>
<property name="url"><value>jdbc:oracle:thin:#XXX.XXX.com:1500:SERVICE</value></property>
<property name="maxActive"><value>100</value></property>
<property name="maxIdle"><value>10</value></property>
<property name="username"><value>XXX</value></property>
<property name="password"><value>XXX</value></property>
</bean>
I recently moved the scope of those beans to singleton because the amount of connection per session started to be a bit too much.
The problem is :
Our database is shutting down every sunday and the spring application seems to act strangely by keeping the socket open and does not refresh the connection as I thought it would do.
Is there a way to refresh the beans scoped as singleton in a way that will refresh the connection everyday and not be obliged to relaunch the application every monday?
What you want to do is to configure validation for your connections. When a connection is borrowed from the pool you want to make sure that that connection is valid. For this you can specify the validationQuery property on your datasource.
<bean id="datasourceAR_XXX" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" scope="singleton">
<property name="driverClassName"><value>oracle.jdbc.driver.OracleDriver</value></property>
<property name="url"><value>jdbc:oracle:thin:#XXX.XXX.com:1500:SERVICE</value></property>
<property name="maxActive"><value>100</value></property>
<property name="maxIdle"><value>10</value></property>
<property name="username"><value>XXX</value></property>
<property name="password"><value>XXX</value></property>
<property name="validationQuery" value="select 1 from dual" />
</bean>
See DBCP - validationQuery for different Databases for a list of possible validation queries for different databases.
There are some issues with Commons DBCP and it is pretty old (although there is a DBCP 2.x now). I would suggest moving to a different datasource like HikariCP this datasource is also a JDBC 4.x based datasource which allows for easier connection validation (it is part of the JDBC 4 spec).
<bean id="datasourceAR_XXX" class="com.zaxxer.hikari.HikariDataSource">
<property name="datasourceClassName" value="oracle.jdbc.pool.OracleDataSource"/>
<property name="maximumPoolSize" value="20" />
<property name="username" value="XXX" />
<property name="password" value="XXX" />
<property name="datasourceProperties">
<props>
<prop key="serverName">XXX.XXX.com</prop>
<prop key="port">1500</prop>
<prop key="databaseName">SERVICE</prop>
</props>
</property>
</bean>
If your oracle driver is new enough you don't need a validation query anymore as validation is provided by the driver instead of needing to be done with a query. Next to that you probably have better results with this pool.
Also you might have a bit of a large pool size, nice article/presentation about pool sizing can be found here.
In spring I have a datasource defined in this way:
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
<property name="URL" value="${jdbc.url}" />
<property name="user" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
<property name="connectionCachingEnabled" value="true"/>
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
This datasource is used by my REST service and everything works fine...anyway first REST call is very slow (about ~5 secs), after that EVERY other call is fast.
I think this is an initialization related problem, in the sense that initialization is made when first DB call is received.
Is there a way to tell spring to initialize this datasource on server startup?
I think this is an initialization related problem, in the sense that
initialization is made when first DB call is received.
With your current config I think that's what's happening.
Is there a way to tell spring to initialize this datasource on server
startup?
It's the behavior of the connection pool, not Spring. Spring is creating the bean when your app starts (you aren't using lazy-init="true" on the bean). However, the connection pool isn't creating connections to the database when Spring instantiates it. From the Oracle docs:
The initial pool size property specifies the number of available
connections that are created when the connection pool is initially
created or re-initialized. This property is typically used to reduce
the ramp-up time incurred by priming the pool to its optimal size.
A value of 0 indicates that no connections are pre-created. The
default value is 0.
Try setting a non-zero value for initialPoolSize.
Edit: Try setting ConnectionCacheProperties instead:
<property name="connectionCacheProperties">
<props merge="default">
<prop key="InitialLimit">5</prop>
</props>
</property>
We are using a j2ee application with spring framework 2.0. The server used is tomcat 5.5. The database used is mysql. We are using a VPS for hosting our application and we have noticed that the CPU usage increases with more users using our application. The CPU usage does not come down once the users stop using the application. Is it the connections that are not closed properly or is there any other issue?
Here is the servlet.xml configuration for the connections
<bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://localhost/myDB"/>
<property name="username" value="xxxx"/>
<property name="password" value="xxxx"/>
<property name="validationQuery" value="SELECT 1"/>
<property name="testOnBorrow" value="true"/>
</bean>
We have also tried using
<bean id="myDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://localhost/myDB"/>
<property name="username" value="xxxx"/>
<property name="password" value="xxxx"/>
</bean>
But both of them cause the same problem. Can anyone help us out quickly? Because we need to correct this issue at the earliest. Thanks in advance.
It is unlikely that high CPU usage be caused by some connection pool issues. It's probably a mistake within your application code. Did you monitored database connections — are they released and closed properly?
By the way, I'd suggest you switch to the native connection pool built in Tomcat. It can be obtained as a standard Java EE resource from the pseudo-JNDI implemented in Tomcat.