Our SpringBoot tests use Hikari datasource obtained via JNDI.
So from test to test the embedded server is started/shutdown.
Since HikariPool start printed in the log, but none of shutdown, does it mean that our tests are causing connection leak ?
Found these:
INFO HikariDataSource - HikariPool - Starting...
INFO HikariDataSource - HikariPool - Start completed.
But none of these:
INFO HikariDataSource - HikariPool - Shutdown initiated...
INFO HikariDataSource - HikariPool - Shutdown completed.
Should I worry about it?
Any advice would be greatly appreciated.
Best regards,
SetNug
Related
I am deploying my Spring Boot REST API on AWS Fargate, which connects to AWS Aurora Postgresql Serverless V1.
I have configured the Aurora to scale the ACU to 0 when idle as in the following picture, so that I am not charge too much when I don't use the API.
Initially, my Spring Boot App maintains 10 idle connections by default, so I have tried to make it zero by adding the this to application.properties
spring.datasource.minimumIdle=0
And then I see from AWS console that the database connection has been reduced. But it remains 1 connection forever.
Please help suggest if you know how to make it zero.
The Spring Boot database configuration is basically like this
#Bean
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
Edit 1
I used the suggestion in the comment to check if the connection really comes from Spring Boot.
It turns out there is no active connection but /actuator/metrics/hikaricp.connections.idle always return the value of 1
{"name":"hikaricp.connections.idle","description":"Idle connections","baseUnit":null,"measurements":[{"statistic":"VALUE","value":1.0}],"availableTags":[{"tag":"pool","values":["HikariPool-1"]}]}
And it seems does not relate to health check because I have tried running it locally and the result of /actuator/metrics/hikaricp.connections.idle remains 1.
I set logging.level.root = trace to see what is happening.
There are only 2 things keep printing in the log periodically
The Hikari connection report, showing 1 idle connection
{"level":"DEBUG","ref":"|","marker":"INTERNAL","message":"HikariPool-1 - Before cleanup stats (total=1, active=0, idle=1, waiting=0)","logger":"com.zaxxer.hikari.pool.HikariPool","timestamp":"2022-06-14 16:15:16.799","thread":"HikariPool-1 housekeeper"}
{"level":"DEBUG","ref":"|","marker":"INTERNAL","message":"HikariPool-1 - After cleanup stats (total=1, active=0, idle=1, waiting=0)","logger":"com.zaxxer.hikari.pool.HikariPool","timestamp":"2022-06-14 16:15:16.800","thread":"HikariPool-1 housekeeper"}
{"level":"DEBUG","ref":"|","marker":"INTERNAL","message":"HikariPool-1 - Fill pool skipped, pool is at sufficient level.","logger":"com.zaxxer.hikari.pool.HikariPool","timestamp":"2022-06-14 16:15:16.800","thread":"HikariPool-1 housekeeper"}
Tomcat NioEndpoint, but I think it is not relevant
{"level":"DEBUG","ref":"|","marker":"INTERNAL","message":"timeout completed: keys processed=0; now=1655198117181; nextExpiration=1655198117180; keyCount=0; hasEvents=false; eval=false","logger":"org.apache.tomcat.util.net.NioEndpoint","timestamp":"2022-06-14 16:15:17.181","thread":"http-nio-8445-Poller"}
Thanks to the suggestion in the comment, this is because of the actuator health check, which can be solved by the following settings
management.health.db.enabled=false
I have a spring boot app which uses spring data and hikaricp for db connection pooling. I noticed the following behaviour that looks strange to me:
I have one method which is not transactional and in that method several db queries are executed using spring data repositories
public void testMethod(final Long firstRepositoryId, final Long secondRepositoryId) {
final DomainObject result = firstRepository.findById(firstRepositoryId);
// here there's some code that is processing the result without db queries
secondRepository.findById(secondRepositoryId);
// some more logic here without additional db queries
}
So as expected when there's no transaction on the method then the spring data methods opens a transaction for executing the query and complete it after the methods returns. I have enabled transaction logging so there's the following log output:
2021-06-03 15:34:30.961 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Getting transaction for [com.test.FirstRepository.findById]
2021-06-03 15:34:30.966 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Completing transaction for [com.test.FirstRepository.findById]
2021-06-03 15:34:30.967 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Getting transaction for [com.test.SecondRepository.findById]
2021-06-03 15:34:30.972 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Completing transaction for [com.test.SecondRepository.findById]
Everything seems to be exactly how I expects to be. The thing I can't understand is the hikari behaviour. This method is invoked within a http request. A connection is taken from hikari cp right after the execution of the firstRepository.findById but this connection is returned in the pool only after the http controller returns response. What I expect is that a connection is taken after a transaction is opened and returned back after the transaction is completed. Is there something that I miss or maybe I have some wrong configuration?
P.S. I'm monitoring the active hikari connections through the spring boot actuator prometheus data. And to be able to reproduce the behavior I explained above I'm suspending the connection thread with several debug breakpoints.
I found out what causes this behaviour- it's Spring functionality to maintain the hibernate session in view in order to be able to retrieve lazy loaded data in the view. In order to disable this you need the following property:
spring.jpa.open-in-view=false
Here's another SO post where is explained what it does:
What is this spring.jpa.open-in-view=true property in Spring Boot?
I have a spring boot application that uses hibernate with oracle and hibernate envers. The application takes a long time to starup. from the logs, I can see something like this
2020-06-05 09:51:40,727 [main] INFO org.hibernate.dialect.Dialect - HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
2020-06-05 09:52:20,219 [main] INFO org.hibernate.envers.boot.internal.EnversServiceImpl - Envers integration enabled? : true
2020-06-05 09:53:02,809 [main] INFO org.hibernate.engine.transaction.jta.platform.internal.JtaPlatformInitiator - HHH000490: Using JtaPlatform implementation: [org.hibernate.engine.transaction.jta.platform.internal.NoJtaPlatform]
if we see the timestamp between log statements the difference is pretty large.
can anyone suggest how I can improve the startup time?
We have an activemq listener MDB with EJB, listening to a broker over SSL using activemq-rar.rar connector that has the broker, SSL cert, credential configurations. The listener is running well in JBOSS EAP 6.4. However, the need is to convert this MDB into Spring Boot ActiveMQ listener. I could see many articles examples about Spring Boot ActiveMQ with simple tcp://localhost:61616. Is there a complete reference to working example of producing message to and listening message from external broker over SSL using Spring Boot.
I have tried to create produce and consume messages using tcp per the instructions at
https://www.devglan.com/spring-boot/spring-boot-jms-activemq-example
and
http://activemq.apache.org/how-do-i-use-ssl.html
spring.activemq.broker-url=ssl://localhost:61616
spring.activemq.user=admin
spring.activemq.password=admin
spring.activemq.keyStore=broker.ks
spring.activemq.keyStorePassword=password
javax.net.ssl.keyStore=client.ks
javax.net.ssl.keyStorePassword=password
javax.net.ssl.trustStore=client.ts
I am seeing below exception. Need help.
2019-01-14 00:54:33.209 INFO 13964 --- [ main] com.rime.springboot.amc.Application : Started Application in 5.783 seconds (JVM running for 7.081)
2019-01-14 00:54:38.203 INFO 13964 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : JMS message listener invoker needs to establish shared Connection
2019-01-14 00:54:38.244 ERROR 13964 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'inbound.queue' - retrying using FixedBackOff{interval=5000, currentAttempts=0, maxAttempts=unlimited}. Cause: Could not connect to broker URL: ssl://localhost:61616. Reason: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
2019-01-14 00:54:43.341 ERROR 13964 --- [enerContainer-1] o.s.j.l.DefaultMessageListenerContainer : Could not refresh JMS Connection for destination 'inbound.queue' - retrying using FixedBackOff{interval=5000, currentAttempts=1, maxAttempts=unlimited}. Cause: Could not connect to broker URL: ssl://localhost:61616. Reason: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
It appears that auto-configuration via properties is not currently supported. See: https://github.com/spring-projects/spring-boot/issues/17365 and https://github.com/spring-projects/spring-boot/issues/17589.
In the meantime, it is possible to override the default connection with your own ActiveMQSslConnectionFactory bean, providing the SSL configuration properties you need:
#Configuration
public class ActiveMQConfiguration {
#Bean
public ActiveMQSslConnectionFactory activeMQSslConnectionFactory(
#Value("${spring.activemq.broker-url}") String brokerUrl,
#Value("${spring.activemq.ssl.trustStorePath}") String trustStorePath,
#Value("${spring.activemq.ssl.trustStorePass}") String trustStorePass,
#Value("${spring.activemq.ssl.keyStorePath}") String keyStorePath,
#Value("${spring.activemq.ssl.keyStorePass}") String keyStorePass) throws Exception {
ActiveMQSslConnectionFactory factory = new ActiveMQSslConnectionFactory(brokerUrl);
factory.setTrustStore(trustStorePath);
factory.setTrustStorePassword(trustStorePass);
factory.setKeyStore(keyStorePath);
factory.setKeyStorePassword(keyStorePass);
return factory;
}
}
The spring.activemq.ssl.* properties aren't based on any existing or documented properties, so you don't need to use those specifically.
I am using Spring Boot (1.5.6), Hibernate, Postgres, Hikari (2.7.8) . My configuration is :
spring.datasource.hikari.minimumIdle=1
spring.datasource.hikari.maximumPoolSize=20
spring.datasource.hikari.idleTimeout=30000
spring.datasource.hikari.poolName=SpringBootJPAHikariCP
spring.datasource.hikari.maxLifetime=50000
spring.datasource.hikari.connectionTimeout=30000
What I expect is, Idle connections should be released after 30000 ms/ 30 sec of idleness.
The problem is with every request new connection being made leaving all the idle connection as it is. So after some times I end up with 20 idle connection and with a new request Hikari try to gain a new connection and get SpringBootJPAHikariCP - Connection is not available, request timed out after 30001ms.
So, What I am doing wrong.? Or Have the misunderstood the configuration?
Hikari initialization Log:
SpringBootJPAHikariCP - configuration:
allowPoolSuspension.............false
autoCommit......................true
catalog.........................none
connectionInitSql...............none
connectionTestQuery.............none
connectionTimeout...............30000
dataSource......................none
dataSourceClassName.............none
dataSourceJNDI..................none
dataSourceProperties............{password=<masked>}
driverClassName................."org.postgresql.Driver"
healthCheckProperties...........{}
healthCheckRegistry.............none
idleTimeout.....................30000
initializationFailFast..........true
initializationFailTimeout.......1
isolateInternalQueries..........false
jdbc4ConnectionTest.............false
jdbcUrl.........................jdbc:postgresql://localhost:5432/dbname
leakDetectionThreshold..........0
maxLifetime.....................50000
maximumPoolSize.................20
metricRegistry..................none
metricsTrackerFactory...........none
minimumIdle.....................1
password........................<masked>
poolName........................"SpringBootJPAHikariCP"
readOnly........................false
registerMbeans..................false
scheduledExecutor...............none
scheduledExecutorService........internal
schema..........................none
threadFactory...................internal
transactionIsolation............default
username........................"postgres"
validationTimeout...............5000
Update:
In last 24 hour I have tried several solution from different thread and none of them fixed my issue. So here is the observations that might be important.
SpringBootJPAHikariCP - Reset (autoCommit) on connection org.postgresql.jdbc.PgConnection#1344bbf1 found this log. studied
Reset (autoCommit) on connection in HikariCP this thread. tried to set auto commit same (true) at both side (hibernate and Hikari) and tried with false at both side also. still no luck.
Enables leakDetectionThreshold, got leak detection exception. So tried to understand if hibernate/spring transaction manager releasing connections. From bellow Logs it looks like hibernate working fine.
28 22:19:35- DEBUG - o.s.orm.jpa.JpaTransactionManager-371 :: Opened new EntityManager [org.hibernate.jpa.internal.EntityManagerImpl#4212be39] for JPA transaction
28 22:19:35- DEBUG - o.h.e.t.internal.TransactionImpl-51 :: begin
28 22:19:35- DEBUG - o.s.orm.jpa.JpaTransactionManager-403 :: Exposing JPA transaction as JDBC transaction [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#243e942]
2com.someentity.MyEntity#ac918eed-345f-4a6c-8539-fe14e7fc41e2
28 22:19:35- DEBUG - o.h.r.j.i.LogicalConnectionManagedImpl-137 :: Initiating JDBC connection release from afterTransaction
28 22:19:35- DEBUG - c.zaxxer.hikari.pool.ProxyConnection-242 :: SpringBootJPAHikariCP - Executed rollback on connection org.postgresql.jdbc.PgConnection#1344bbf1 due to dirty commit state on close().
28 22:19:35- DEBUG - o.h.e.i.AbstractFlushingEventListener-132 :: Processing flush-time cascades
28 22:19:35- DEBUG - o.h.e.i.AbstractFlushingEventListener-174 :: Dirty checking collections
28 22:19:35- DEBUG - org.hibernate.internal.SessionImpl-508 :: Disconnecting session
28 22:19:35- DEBUG - o.s.orm.jpa.JpaTransactionManager-759 :: Initiating transaction commit
28 22:19:35- DEBUG - o.s.orm.jpa.JpaTransactionManager-512 :: Committing JPA transaction on EntityManager [org.hibernate.jpa.internal.EntityManagerImpl#4212be39]
28 22:19:35- DEBUG - o.h.e.t.internal.TransactionImpl-62 :: committing
28 22:19:35- DEBUG - o.h.r.j.i.LogicalConnectionManagedImpl-137 :: Initiating JDBC connection release from afterTransaction
28 22:19:35- DEBUG - o.h.r.j.i.LogicalConnectionManagedImpl-137 :: Initiating JDBC connection release from afterTransaction
28 22:19:35- DEBUG - o.s.orm.jpa.JpaTransactionManager-600 :: Closing JPA EntityManager [org.hibernate.jpa.internal.EntityManagerImpl#4212be39] after transaction
28 22:19:35- DEBUG - o.s.o.jpa.EntityManagerFactoryUtils-435 :: Closing JPA EntityManager
All the idle connectiona are idle form postgres point of view and active form Hikari point of view. So when there are 5 idle connection from database, there are toatal = 5, active=4, idle = ,waiting=0 in Hikari log.
Note:
may be I am having this exact issue https://github.com/brettwooldridge/HikariCP/issues/109 in my case active connection increasing with every transaction.
HikariCP - connection is not available this is also a same issue. but nobody provide clear solution to this. btw I was using #Transactional from the begging as suggested by the accepted answer.
This was not any Hikari issue, there was a mistake in my end. Still posting the details of how this occurred, in case it helps someone.
I was using spring boot 1.5.6 (This was the latest version when I start working).
This version included spring-orm 4.3.1. This version of spring-orm included support for three versions of hibernate, Hibernate5, Hibernate4 and Hibernate3.
So I configured spring boot with bellow configuration for current_session_context_class.
spring.jpa.properties.hibernate.current_session_context_class=org.springframework.orm.hibernate4.SpringSessionContext
Everything was working fine until it comes to connection management of Hikari. What happened is spring-boot-starter-jpa for 1.5.6 included Hibernate5 (I mean hibernate core) .
So, After performing any DB operation spring loosing control of that connection (most possibility for this version mismatch). Hence the problem.
After changing
org.springframework.orm.hibernate4.SpringSessionContext
to
org.springframework.orm.hibernate5.SpringSessionContext
the problem was solved immediately.
My current configuration is
spring.jpa.properties.hibernate.current_session_context_class=org.springframework.orm.hibernate5.SpringSessionContext
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQL95Dialect
FYI, after solving the problem is switched to Spring Boot 2.