Does SQL Azure allow 3-rd party connection pool like HikariCP or BoneCP?
We configured HikariCP it works when we just run app but later db doesnt response on request. Is it HikariCP issue or it's common connection poool issue and no need spending more time on investigation?
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(50);
config.setDriverClassName(env.getProperty("jdbc.driverClassName"));
config.setJdbcUrl(env.getProperty("jdbc.url"));
config.setUsername(env.getProperty("jdbc.user"));
config.setPassword(env.getProperty("jdbc.pass"));
config.addDataSourceProperty("cachePrepStmts", env.getProperty("jdbc.cachePrepStmts"));
config.addDataSourceProperty("prepStmtCacheSize", env.getProperty("jdbc.prepStmtCacheSize"));
config.addDataSourceProperty("prepStmtCacheSqlLimit", env.getProperty("jdbc.prepStmtCacheSqlLimit"));
config.addDataSourceProperty("useServerPrepStmts", env.getProperty("jdbc.useServerPrepStmts"));
See this SQL Azure page re: Connection Constraints.
Maximum allowable durations are subject to change depending on the resource usage.
A logged-in session that has been idle for 30 minutes will be terminated
automatically. We strongly recommend that you use the connection pooling and
always close the connection when you are finished using it so that the unused
connection will be returned to the pool. For more information about connection
pooling, see Connection Pooling.
See if any of these errors match up to your logs. Search that page for "terminated" and "busy" to find error codes that might be relevant to your issue.
I would suggest setting the maxLifetime property in HikariCP to 15 minutes, and the idleTimeout to 2 minutes.
There is nothing on the SQL Azure side that would prohibit you from using a 3rd party connection pool. My guess is that the connection failed between the server and the client and the client didn't remove the connection from the pool.
Moving forward, I'd ensure that whichever 3-rd part connection pool you end up using tests that the connection exists before taking it out of the pool for use.
Hope that helps.
Related
I am creating a VM in azure to upload a postgres instance in docker and connect to it with my local backend in Spring. What happens is that once connected to the DB after X time of inactivity when trying to make a request I get the following "HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection#f162126 (This connection has been closed.). Possibly consider using a shorter maxLifetime value." digging around I realized that it is as if my VM has some kind of behavior that when a connection becomes inactive it closes it causing the above error. The curious thing here is that the sessions are not closed as you can see in the following image even shutting down my backend the sessions are maintained and the only options to delete them is restarting the container in which the DB is hosted.
I have tried to reproduce this behavior on local but it never happens even if I leave the backend idle for an hour if I do the request to the DB it works as if nothing, it only happens with my VM in azure.
I want to clarify that the sessions that appear in the attached image no longer work, i.e. if I try to consume the DB from spring, the error I mentioned appears and automatically Hikari creates new sessions for its pool and I can reproduce this behavior until I reach 100 sessions that after a while would not work again and that Spring never closes when shutting down the backend.
HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection#f162126 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
This error is thrown by method isConnectionDead. While checking the connection, if it's still alive & can be used and it will issue an above error if it has already been closed.
You can adjust your maxLifetime setting, to resolve this problem. 30000ms (30 seconds) is the shortest value that is permitted (30 seconds), 1800000ms (30 minutes) is by default value.
A connection in the pool can only last for a certain amount of time which is controlled by the maxLifetime attribute. the value of this property it should be several seconds below any connection time restrictions set by the infrastructure or any databases.
Reference: Hikari Configuration Github.
Well, after much research and reviewing various sources it turns out that azure when creating a VM has certain security policies as Pedro Perez says in the following post in Stack Excahnge: Azure closing idle network connections
You're hitting a design feature of the software load balancer in front of your VMs. By default it will close any idle connections after 4 minutes, but you can configure the timeout to be anything between those 4 and 30 minutes
So in order to overwrite this policy that governs your VM you must do the process of creating a load balancer, do all the relevant configuration and create a Load balancing rule for port 5432 which is the default port of postgres and set the Idle timeout in a range of 4 to 30 min according to your needs.
finally configure your VM so its public ip points to the LB(Load Balancer) public ip and everything will work normally.
It is true that if you simply want to take advantage of Azure's default security policies on the VMs you create you should set the maxLifetime to a maximum of 4 minutes in your Spring application.properties or appliation.yml as #PratikLad says.
in my case I prefer to leave the default Hikari configuration (maxLifetime of 30 mins) so I need to create the LB but if you prefer to change the property by setting it to a maximum of 4 min you would not need to do all the above mentioned on the LB.
I have a verticle, which consumes a message from the event bus and processes it. I have a question as to when the JDBC connection should be closed. There are 2 approaches
Closing the connection once the message is processed. But this will be very expensive because I will open/close connection every time.
Trust that vertx will close the connection when the verticle is stopped/undeployed (which is literally never) and that there wont be any memory leaks as long as the connection is open. I will open the connection in the start() method, so that whenever there is a message it available.
On the other hand, If I have an elastic search backend and I am using the elastic search SDK, which has a specific method to close the client, when should the connection be really closed?
Use a connection pool, that will take away most of the cost of closing/opening connections. When using a connection pool, closing the connection returns it to the connection pool for re-use.
The basic usage pattern is:
try (Connection connection = dataSource.getConnection()) {
// use connection
}
At the end of the block the connection is closed, which - if dataSource has a connection pool - will make it available for re-use.
You can always put your clean up code in Stop() method of Verticle interface. It will be called when the verticle starts it's un-deploy procedure.
See Vert.x Docs
When our application tries to connect Oracle database, this exception is thrown:
java.sql.SQLRecoverableException: Connection has been administratively disabled by console/admin command. Try later. java.lang.Exception: It was disabled at Tue Oct 20 23:55:14 CEST 2015
But, from Weblogic console the connection test returns OK.
Weblogic version: 12.1.3.0.0
Any explanation is welcome. Thanks
The reason the test works is because that is creating a connection and running a test query. That is not what your code is doing when it is using the data source in an ejb. The code is going through the connection pool and that is what has been marked as bad. There is no solution provided by Oracle that I have found except to: restart the server and that will re-enable the connection pool.
I suspect you have "Test connections on reserve" set because that is when this usually arises. What Weblogic does is: before it returns a connection from the pool it will run a test query, if the test query fails it waits and runs it one more time. If the query fails again it marks the connection as unhealthy. If all the connections in the pool become unhealthy it will mark the pool as disabled and gives you the error message you see: 'Connection has been administratively disabled by console/admin command. Try later.'
In regards to the 'Try Later' part of the error message, as far as I can tell Oracle is wrong about trying again later. I have never seen it recover later.
I'd like to share this article that help me out to understand better my problem:
https://www.techpaste.com/2012/09/connection-administratively-destroyed-reconnect-oracle-weblogic-server/
The “java.sql.SQLException: Connection has been administratively destroyed.” is expected:
the DB was shutdown; even if it was restarted later, the JDBC connection are pointing to DB processes
that have been destroyed.
You need to restart your WebLogic Server to recreate new JDBC connections.
All the current transactions are lost, as the database was shutdown.
High availability of your RDBMS is required to minimize this issue.
Is there any other error before that one ?
Maybe in the log you can find that connection is been closed.
You can avoid this by selecting "Test on Connection Reserve" in the datasource.
We are using tomcat jdbc connection pooling with Oracle database.
Recently, we encountered problem that there are too many inactive session in Oracle database from JDBC THIN CLIENT.
Can anyone help us in this? Why is it causing inactive session in the database and what can be the solution to this.
Thank you.
Adjust settings of your pool, following the documentation. Setting maxIdle and minEvictableIdleTimeMillis to low values would ensure that idle connections are evicted fast, and that few idle connections stay open. Obviously, it will also make your pool less efficient, since connections will be closed and opened more frequently.
If you are getting SQLExceptions because connections have timed out, you should set a validationQuery (to something like SELECT 0 FROM DUAL) and your connections will be tested before they are checked-out of the pool. Any connection that has failed will be replaced by a working connection and then returned to your code.
We are using ADO to access Oracle 10g release 2, Oledb provider for Oracle 10g. We are facing some issue with the connection pooling. The database reside on the remote machine and connection pooling is occuring as it should. But if the remote machine goes down for some reason, the connection is returned from the pool and query on that connection fails. When this connection is closed, it is returned back to the pool instead of being invalid. The subsequent connection opening requests are sucessfull but query fails. This is strange behaviour, according to OLEDB specifications, provider must support DBPROP_CONNECTIONSTATUS property, thus in case of invalid connection, it would not be returned back to the pool.
Things get weired when the remote machine comes up. The connections in the pool are still invalid and although the connection opening succeeds, query on the connection fails. Oracle OLEDB is unable to connect to the server anymore and we have to restart our application. Well this is undesired cause our application is a critical application.
Any ideas on how to get over this.
Thanks
Mubashir
If you are doing this programmatically, use a try block, so that if something does happen, it won't fail. With a try block, you can catch an exception and ignore it, so that the errors are shushed.
You could tell the pool to not accept invalid connections, by marking the connection invalid before it is returned to the pool.
Connections are recovered after 10 minutes by default. Time can be set by the registry key SPTimeout under the oledb provider's root key.
Actually the default connection pool timeout is 120 seconds at least for this Oracle 11 32-bit OLEDB installation. You can find the registry settings at:
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Oracle\KEY_orac
Where KEY_orac is the KEY_<oracle_home_name>
The key name is ORAMTS_CONN_POOL_TIMEOUT and the default value is 120.
It does not appear that you can set connection pool parameters at the connection string level.
In most connection pool implementation it is possible to check the connection before using it. For example: you define a check query like select * from dual and if you pick up a connection from the pool this query will be executed. If it fails, then the connection will be excluded from the pool and a new one will be opened.