I want a database connection that has gone idle to be dropped once its maxLifetime has been reached.
How can we do this in spring boot/hikari? Is this achieved with these 2 properties?
idleTimeout
maxLifetime
How can I check that idle connections are actually being dropped? Any logs I can activate?
What should be the ideal values?
read Hikari documentation
check DB activity via SQL (there you can see all transaction statistics)
add logging
try locally with Postgres in Docker
Related
The application i'm working on is in Spring Boot using Spring JDBCTemplate to connect to Teradata.
We face issues with Idle connections. we have about 6 different environments that create at some point 1672 sessions.
In order to limit the total pool size and the minimum idle connections i set it to:
hikari:
maximum-pool-size: 3
minimum-idle: 2
is there anything else recommend in otder to prevent the number of idle connections?
Thanks in advance
Assuming you're using Hikari for connection pool, you may use idleTimeout. Please refer Hikari configuration for all available properties.
In one of spring boot microservice app, i was having problem with connection leakage. Some of the connection were not being returned to the Pool and i was using only default configuration for Hikari. Now I put a leakDetectionThreshold and it starts explicitly taking back connections and returning it to the Pool. My question is related to these broken connections, even though as per the specification of Hikari pool, if a connection retuned to pool in a broken state, Pool handlers would rollback the transactions so thay It didn't bleed out for the next consumer. But in my case, transactions were not rolled back. What is happening here?
SpringBoot already is managing dataConnection then why is Hikari CP needed?
I have just started using SpringBoot so do not know much about SpringBoot and Hikari relation, although i read about Hikari but couldn't find any explicit explanation about its relationship with Springboot in presence of Spring data connection.
I read that Hikari is used when we need heavy db operations with lots of connections, if it is true then should we not use Hikari in follwoing scenario?
Scenario:
There is a small application, having maximum 8-10 REST calls once in a month or maximum fortnightly.That application needs to perform some probability and statistics related calculation.
Users login on that app at a time are of maximum 2-3 in numbers.
Do we still need to use Hikari?
There are two ways to communicate with the database from your application. You can either open a new DB connection any time you wish execute some query there, or you have a connection pool. Connection pool is a collection of reusable connections that application uses for DB communication. As establishing a new connection is relatively expensive operation, using connection pool gives you a significant performance improvement.
HikariCP is one of the connection pools libraries available in java and SpringBoot uses it as a default. As you don't need to do anything special to have it in your application, just enjoy your free lunch :)
HikariCP is used as the default connection pool in SpringBoot2, it was TomcatJDBC in SpringBoot 1. You must be using it as a default in your settings. You can overwrite it by setting another connection pool in your setting properties if you need. Please find more details about the connection pools and the default configurations of Spring Boot versions here.
Hikari is the default DataSource implementation with Spring Boot 2. This means we need not add explicit dependency in the pom.xml. The spring-boot-starter-JDBC and spring-boot-starter-data-JPA resolve it by default. To sum up, you require no other steps with Spring Boot 2.
Compared to other implementations, it promises to be lightweight and better performing.
Tuning Hikari Configuration Parameters:
spring.datasource.hikari.connection-timeout = 20000 #maximum number
of milliseconds that a client will wait for a connection
spring.datasource.hikari.minimum-idle= 10 #minimum number of idle
connections maintained by HikariCP in a connection pool
spring.datasource.hikari.maximum-pool-size= 10 #maximum pool size
spring.datasource.hikari.idle-timeout=10000 #maximum idle time for
connection
spring.datasource.hikari.max-lifetime= 1000 # maximum lifetime in
milliseconds of a connection in the pool after it is closed.
spring.datasource.hikari.auto-commit =true #default auto-commit
behavior.
HikariCP is a reliable, high-performance JDBC connection pool. It is much faster, lightweight, and has better performance as compared to other connection pool APIs. Because of all these compelling reasons, HikariCP is now the default pool implementation in Spring Boot 2. In this article, we will have a closer look to configure Hikari with Spring Boot.
I use spring boot 2 with Hikari connection pool, jpa and postgres.
Is there any reason to continue to use theses options
spring.datasource.testOnBorrow=true
spring.datasource.validationQuery=SELECT 1
spring.datasource.testWhileIdle
No, They are unknown properties to Hikari connection pool so no need ,
They exists only in Tomcat JDBC Connection Pool (used in old Spring boot) which you aren't using anymore.
Explanation of different between properties
DBCP testOnBorrow=false rollbackOnReturn=false
enableAutoCommitOnReturn=false
Issues:
testOnBorrow=false increases the likelihood of broken connections
given to your application rollbackOnReturn=false +
enableAutoCommitOnReturn=false, like the C3P0 "remediation" above can
bleed transactions between consumers or cause locks to be held for
extended periods
HikariCP Differentiators
Tests connections with isValid() before returning them from the pool,
with an optimization that bypasses the check if the connection was
utilized within the last 1000ms Tracks connection state (and
transaction state), and performs rollback() only the the case of a
non-autocommit connections with uncommitted changes
I am troubleshooting an issue with a Spring Boot app connecting to a PostgreSQL database. The app runs normally, but under fairly moderate load it will begin to log errors like this:
java.sql.SQLException: Timeout after 30000ms of waiting for a connection.
This is running on an Amazon EC2 instance connecting to a PostgreSQL RDS. The app is configured like the following:
spring.datasource.url=jdbc:postgresql://[rds_path]:5432/[db name]
spring.datasource.username=[username]
spring.datasource.password=[password]
spring.datasource.max-active=100
In the AWS console, I see 60 connections active to the database, but that is across several Spring Boot apps (not all this app). When I query the database for current activity using pg_stat_activity, I see all but one or 2 connections in an idle state. It would seem the Spring Boot app is not using all available connections? Or is somehow leaking connections? I'm trying to interpret how pg_stat_activity would show so many idle connections and the app still getting connection pool time outs.
Figured it out. Spring is using the Hikari database connection pooling (didn't realize that until after more closely inspecting the stack trace). Hikari configuration parameters have different names, to set the pool size you use maximum-pool-size. Updated that and problem solved.