I'm using c3p0 ComboPooledDataSource to pool database connections (to an Oracle 10g database). I'm having problems with handling database connectivity outages.
If there is no connection to the database when the first connection is being acquired, the checkout timeout fires and it fails as expected.
However, if a connection outage occurs after one or more connections have been acquired and are already in the connection pool, calling getConnection() just hangs. No exception is thrown. I'm guessing this is because it's trying to use a pooled connection but that connection is no longer alive.
Is there a way to check whether the connection is valid before trying to use it? I tried setting testConnectionOnCheckout=true but it doesn't seem to have any effect.
This is the thread dump
C3P0PooledConnectionPoolManager[identityToken->2rvy8f8x1oujxrx1majv5s|be41d5]-
HelperThread-#2" daemon prio=6 tid=0x0307a800 nid=0x840 in Object.wait() [0x03d1f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method) - waiting on <0x28387f88> (a com.mchange.v2.async.ThreadPoolAsynchronousRunner)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:635)
- locked <0x28387f88> (a com.mchange.v2.async.ThreadPoolAsynchronousRunner)
Locked ownable synchronizers:
- None
I encountered the same issue. In my case, it was caused by the JDBC driver not being setup to timeout on socket failures. I made the following additions to my C3P0 ComboPooledDataSource configuration:
cpds = new ComboPooledDataSource();
...
//--------------------------------------------------------------------------------------
// NOTE: Once you decide to use cpds.setProperties() to set some connection properties,
// all properties must be set, including user/password, otherwise an exception
// will be thrown
Properties prop = new Properties();
prop.setProperty("oracle.net.CONNECT_TIMEOUT",
Integer.toString(JDBC_CONNECTION_TIMEOUT_IN_MILLISECONDS));
prop.setProperty("oracle.jdbc.ReadTimeout",
Integer.toString(JDBC_SOCKET_TIMEOUT_IN_MILLISECONDS));
prop.setProperty("user", username);
prop.setProperty("password", password);
cpds.setProperties(prop);
//--------------------------------------------------------------------------------------
...
The Oracle driver properties are applied when C3P0 creates the Connection object. These two properties in particular will cause an exception to be thrown if the socket connection is inactive for more than 30 seconds.
If you're not connecting to an Oracle database, there are similar properties for other JDBC drivers for other database vendors. Some of them are shown near the bottom of this page.
Related
I have a verticle, which consumes a message from the event bus and processes it. I have a question as to when the JDBC connection should be closed. There are 2 approaches
Closing the connection once the message is processed. But this will be very expensive because I will open/close connection every time.
Trust that vertx will close the connection when the verticle is stopped/undeployed (which is literally never) and that there wont be any memory leaks as long as the connection is open. I will open the connection in the start() method, so that whenever there is a message it available.
On the other hand, If I have an elastic search backend and I am using the elastic search SDK, which has a specific method to close the client, when should the connection be really closed?
Use a connection pool, that will take away most of the cost of closing/opening connections. When using a connection pool, closing the connection returns it to the connection pool for re-use.
The basic usage pattern is:
try (Connection connection = dataSource.getConnection()) {
// use connection
}
At the end of the block the connection is closed, which - if dataSource has a connection pool - will make it available for re-use.
You can always put your clean up code in Stop() method of Verticle interface. It will be called when the verticle starts it's un-deploy procedure.
See Vert.x Docs
We have a web application with the stack Spring, Hibernate, C3P0, Oracle DB Driver (habing an Oracle DB behind).
From time to time we experience blocking locks over a longer period of time which then get killed on the DB end. (we know this is caused by bad application design and we will fix it, but it's not the point of this quesion).
After the DB session was killed by DB it seems that the connection pool reuses the now broken connection which results in the error:
A PooledConnection that has already signalled a Connection error is still in use!
Another error has occurred [ java.sql.SQLRecoverableException: Closed Connection ] which will not be reported to listeners!
On the DataSource we configured
dataSource.setTestConnectionOnCheckin(true);
dataSource.setTestConnectionOnCheckout(true);
But it did not help. We expected that the connections fail these tests and then get renewed. But this does not happen.
Any hints for us how to recreate the broken connections?
This warning is given when a Connection that is already checked out experiences an Exception that causes c3p0 to treat it as invalid (so it will not be reincorporated back into the pool on close()), but the Connection continues to be used and experiences an Exception again. These are not broken Connections in the pool. They are broken Connections in-use by the application. So testing them on checkout (or checkin) doesn't do anything about them.
To get rid of this, you need to examine the Exception handling within your application code. Are there circumstances where an invalid Connection might have thrown an Exception, but that Exception was caught and the Connection reused?
The warning itself is harmless. It's just saying c3p0 already knows the Connection is bad, it won't emit an event to signal that again.
I am using C3P0NativeJdbcExtractor to extract the native JDBC connection as below.
public Connection getNativeConnection() throws SQLException{
C3P0NativeJdbcExtractor nativeJbdc;
nativeJbdc = new C3P0NativeJdbcExtractor();
return nativeJbdc.getNativeConnection(dataSource.getConnection());
}
Note that the data source here is obtained of a C3P0 Connection Pool. When I do a Connection.close() returned on this method, it is actually closing the connection instead of returning to the pool.
However if we close the unwrapped connection, then it is returned to the Pool.
Is there is a reason to why closing the wrapped connection here is failing to return the connection to the pool?
A connection pool like c3p0, holds a collection of physical ('native') connections created by a JDBC driver. When you ask it for a connection, it wraps that physical connection in a proxy, also known as the logical connection.
That proxy will intercept certain methods like Connection.close(). For close() instead of closing the connection, it invalidates the logical connection so it behaves as a closed connection, and it returns the physical connection to the connection pool.
Your code extracts the physical connection from the logical connection, and returns that instead, so if you call close() on that, you actually close the connection to the database instead of returning it to the pool.
You should almost never have a reason to extract the native connection like that. The only reason is when you need access to driver-specific features. You should try to use standard JDBC as much as possible, and only unwrap to access driver-specific features when you really need to.
When you call close(), make sure you call close() on the logical connection that you received from the connection pool, not on the unwrapped physical connection.
Does SQL Azure allow 3-rd party connection pool like HikariCP or BoneCP?
We configured HikariCP it works when we just run app but later db doesnt response on request. Is it HikariCP issue or it's common connection poool issue and no need spending more time on investigation?
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(50);
config.setDriverClassName(env.getProperty("jdbc.driverClassName"));
config.setJdbcUrl(env.getProperty("jdbc.url"));
config.setUsername(env.getProperty("jdbc.user"));
config.setPassword(env.getProperty("jdbc.pass"));
config.addDataSourceProperty("cachePrepStmts", env.getProperty("jdbc.cachePrepStmts"));
config.addDataSourceProperty("prepStmtCacheSize", env.getProperty("jdbc.prepStmtCacheSize"));
config.addDataSourceProperty("prepStmtCacheSqlLimit", env.getProperty("jdbc.prepStmtCacheSqlLimit"));
config.addDataSourceProperty("useServerPrepStmts", env.getProperty("jdbc.useServerPrepStmts"));
See this SQL Azure page re: Connection Constraints.
Maximum allowable durations are subject to change depending on the resource usage.
A logged-in session that has been idle for 30 minutes will be terminated
automatically. We strongly recommend that you use the connection pooling and
always close the connection when you are finished using it so that the unused
connection will be returned to the pool. For more information about connection
pooling, see Connection Pooling.
See if any of these errors match up to your logs. Search that page for "terminated" and "busy" to find error codes that might be relevant to your issue.
I would suggest setting the maxLifetime property in HikariCP to 15 minutes, and the idleTimeout to 2 minutes.
There is nothing on the SQL Azure side that would prohibit you from using a 3rd party connection pool. My guess is that the connection failed between the server and the client and the client didn't remove the connection from the pool.
Moving forward, I'd ensure that whichever 3-rd part connection pool you end up using tests that the connection exists before taking it out of the pool for use.
Hope that helps.
We are using ADO to access Oracle 10g release 2, Oledb provider for Oracle 10g. We are facing some issue with the connection pooling. The database reside on the remote machine and connection pooling is occuring as it should. But if the remote machine goes down for some reason, the connection is returned from the pool and query on that connection fails. When this connection is closed, it is returned back to the pool instead of being invalid. The subsequent connection opening requests are sucessfull but query fails. This is strange behaviour, according to OLEDB specifications, provider must support DBPROP_CONNECTIONSTATUS property, thus in case of invalid connection, it would not be returned back to the pool.
Things get weired when the remote machine comes up. The connections in the pool are still invalid and although the connection opening succeeds, query on the connection fails. Oracle OLEDB is unable to connect to the server anymore and we have to restart our application. Well this is undesired cause our application is a critical application.
Any ideas on how to get over this.
Thanks
Mubashir
If you are doing this programmatically, use a try block, so that if something does happen, it won't fail. With a try block, you can catch an exception and ignore it, so that the errors are shushed.
You could tell the pool to not accept invalid connections, by marking the connection invalid before it is returned to the pool.
Connections are recovered after 10 minutes by default. Time can be set by the registry key SPTimeout under the oledb provider's root key.
Actually the default connection pool timeout is 120 seconds at least for this Oracle 11 32-bit OLEDB installation. You can find the registry settings at:
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Oracle\KEY_orac
Where KEY_orac is the KEY_<oracle_home_name>
The key name is ORAMTS_CONN_POOL_TIMEOUT and the default value is 120.
It does not appear that you can set connection pool parameters at the connection string level.
In most connection pool implementation it is possible to check the connection before using it. For example: you define a check query like select * from dual and if you pick up a connection from the pool this query will be executed. If it fails, then the connection will be excluded from the pool and a new one will be opened.