In one of spring boot microservice app, i was having problem with connection leakage. Some of the connection were not being returned to the Pool and i was using only default configuration for Hikari. Now I put a leakDetectionThreshold and it starts explicitly taking back connections and returning it to the Pool. My question is related to these broken connections, even though as per the specification of Hikari pool, if a connection retuned to pool in a broken state, Pool handlers would rollback the transactions so thay It didn't bleed out for the next consumer. But in my case, transactions were not rolled back. What is happening here?
Related
SpringBoot already is managing dataConnection then why is Hikari CP needed?
I have just started using SpringBoot so do not know much about SpringBoot and Hikari relation, although i read about Hikari but couldn't find any explicit explanation about its relationship with Springboot in presence of Spring data connection.
I read that Hikari is used when we need heavy db operations with lots of connections, if it is true then should we not use Hikari in follwoing scenario?
Scenario:
There is a small application, having maximum 8-10 REST calls once in a month or maximum fortnightly.That application needs to perform some probability and statistics related calculation.
Users login on that app at a time are of maximum 2-3 in numbers.
Do we still need to use Hikari?
There are two ways to communicate with the database from your application. You can either open a new DB connection any time you wish execute some query there, or you have a connection pool. Connection pool is a collection of reusable connections that application uses for DB communication. As establishing a new connection is relatively expensive operation, using connection pool gives you a significant performance improvement.
HikariCP is one of the connection pools libraries available in java and SpringBoot uses it as a default. As you don't need to do anything special to have it in your application, just enjoy your free lunch :)
HikariCP is used as the default connection pool in SpringBoot2, it was TomcatJDBC in SpringBoot 1. You must be using it as a default in your settings. You can overwrite it by setting another connection pool in your setting properties if you need. Please find more details about the connection pools and the default configurations of Spring Boot versions here.
Hikari is the default DataSource implementation with Spring Boot 2. This means we need not add explicit dependency in the pom.xml. The spring-boot-starter-JDBC and spring-boot-starter-data-JPA resolve it by default. To sum up, you require no other steps with Spring Boot 2.
Compared to other implementations, it promises to be lightweight and better performing.
Tuning Hikari Configuration Parameters:
spring.datasource.hikari.connection-timeout = 20000 #maximum number
of milliseconds that a client will wait for a connection
spring.datasource.hikari.minimum-idle= 10 #minimum number of idle
connections maintained by HikariCP in a connection pool
spring.datasource.hikari.maximum-pool-size= 10 #maximum pool size
spring.datasource.hikari.idle-timeout=10000 #maximum idle time for
connection
spring.datasource.hikari.max-lifetime= 1000 # maximum lifetime in
milliseconds of a connection in the pool after it is closed.
spring.datasource.hikari.auto-commit =true #default auto-commit
behavior.
HikariCP is a reliable, high-performance JDBC connection pool. It is much faster, lightweight, and has better performance as compared to other connection pool APIs. Because of all these compelling reasons, HikariCP is now the default pool implementation in Spring Boot 2. In this article, we will have a closer look to configure Hikari with Spring Boot.
I use spring boot 2 with Hikari connection pool, jpa and postgres.
Is there any reason to continue to use theses options
spring.datasource.testOnBorrow=true
spring.datasource.validationQuery=SELECT 1
spring.datasource.testWhileIdle
No, They are unknown properties to Hikari connection pool so no need ,
They exists only in Tomcat JDBC Connection Pool (used in old Spring boot) which you aren't using anymore.
Explanation of different between properties
DBCP testOnBorrow=false rollbackOnReturn=false
enableAutoCommitOnReturn=false
Issues:
testOnBorrow=false increases the likelihood of broken connections
given to your application rollbackOnReturn=false +
enableAutoCommitOnReturn=false, like the C3P0 "remediation" above can
bleed transactions between consumers or cause locks to be held for
extended periods
HikariCP Differentiators
Tests connections with isValid() before returning them from the pool,
with an optimization that bypasses the check if the connection was
utilized within the last 1000ms Tracks connection state (and
transaction state), and performs rollback() only the the case of a
non-autocommit connections with uncommitted changes
When a spring boot request starts, it obtains a connection from the pool. My question is - is this connection remains tied to the request thread (even if it is not executing any query) and only returned to the pool when request completes?
For example, if I'm doing something like:
Request starts
Execute a query (20ms)
Call external http service (1500ms)
Request completes
Would the connection obtained by this request thread remain occupied with the thread (and not available to other requests) for 20ms or 1520ms?
PS: I'm using Spring Boot 2.0 with HikariCP and I'm not using #Transactional.
Thanks.
If you have Spring MVC on your classpath, Spring will configure an instance of OpenEntityManagerInViewInterceptor automatically (in org.springframework.boot.autoconfigure.orm.jpa.JpaBaseConfiguration.JpaWebConfiguration), unless you configure spring.jpa.open-in-view to false. This interceptor will open an instance of EntityManager and bind it to the TransactionSynchronizationManager for the whole duration of the request.
What that means is that at the first use of a repository query, it will borrow a connection from the pool and keep it until the end of the request when the EntityManager is closed by the interceptor. It's the EntityManager's responsibility to return the connection to the pool. So, in that case, the connection will be borrowed for 1520ms in your example.
If you disable the spring.jpa.open-in-view option, a connection will be borrowed and returned at each use of a repository query given you don't use any explicit transaction. While this may look better, you have to keep in mind that managed instances of your entities will need to be reattached systematically on each persist operation so it may be costly. You will also loose the benefits of the EntityManager caching. To avoid this, use transactions to read/modify/persist your entities and avoid reattaching.
Finally, keep in mind that if you disable spring.jpa.open-in-view, since you don't have an EntityManager around for the duration of the request, you will need to make sure your lazy loaded relations are loaded inside a transaction otherwise you'll get the dreaded LazyInitializationException.
Basically it depends,
If you close conection it will be release back to pool and will be ready soon according to configuration (below) so it will be ~20ms (+ time to get back to pool)
If you don't close the connection it will wait until the configuration allow it, if it allow indefinite time, theoretically it can be causing a leak in your application and not return to pool until apllication is shutdown.
See answer about Hikari handling returning connections to pool:
The Hikari housekeeper runs every 30s, which closes any connections that are not in use and are older than maxLifetime. If there are more than minimumIdle number of connections the housekeeper will close connections that have been idle for longer than idleTimeout.
See more about max life time of connection:
by default Oracle does not enforce a max lifetime for connections (neither on JDBC driver side (1), nor on server side(2)). So in this respect, the "infrastructure-imposed connection time limit" is +infinity
I'm not completely understand how aries transaction manager works in FUSE ESB 7.1.0. I would be glad if someone can explain the following situation.
I use an jdbc XADataSource (org.postgresql.xa.PGXADataSource) in my bundle and here is what I see: it seems like the aries transaction manager doesn't close a jdbc connection when a message was processed in a transactional route and changes where commited to a DB. Connections leave in the idle state. But after some time they are finally closed.
This led me to big problems when I try to use connection pool: an application exhauses connection pool very fast and all opened connection leave in the busy state.
I have the next questions about the aries transaction manager:
When it does close jdbc connections?
How can I tell it to close a jdbc connection when a message is
processed and changes were committed?
I am using a connection pool and the Spring TransactionTemplate. If you want to shutdown the connection pool first all connection have to be returned to the pool, this means connection.close() has to be called. I have one thread using the TransactionTemplate for some queries and another thread that wants to call some shutdown method on the connection pool, but before doing this it first has to tell the TransactionTemplate to close all connection (actually only returning them to the pool).
How can this be done in Spring to immediately call close on the used connection?
If you are using Hibernate along with Spring please use:
hibernate.connection.release_mode=after_transaction
If you want to release connection just after transaction.
hibernate.connection.release_mode=after_statement
If you want to release connection after each statement
This two settings are the only ways I know that will make used connection being released faster than the default behavior. At least as far as Hibernate is concerned. If you are using other library please describe which.