I have a spring boot application that is deployed to Azure. This application uses JPA to connect to a SQL Server database. Everything seems to work nicely, until after ~30 minutes of inactivity. My guess is the database is closing the connection and the spring boot (tomcat) container isn't properly handling this situation.
I'm injecting the datasouce properties via:
#Bean
#Primary
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource getDataSource() {
return DataSourceBuilder.create().build();
}
My application.properties are:
spring.datasource.url=jdbc:sqlserver://xxxxxx:1433;database=xxxxxx
spring.datasource.username=xxxxxxx
spring.datasource.password=xxxxxxxx
spring.datasource.driver-class-name=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.tomcat.test-on-borrow=true
spring.datasource.tomcat.validation-query=SELECT 1
spring.datasource.tomcat.validation-interval=0
spring.datasource.tomcat.removeAbandoned=true
spring.datasource.tomcat.removeAbandonedTimeout=120
spring.datasource.tomcat.timeBetweenEvictionRunsMillis=34000
spring.datasource.tomcat.minEvictableIdleTimeMillis=55000
spring.datasource.tomcat.initialSize=4
spring.datasource.tomcat.maxActive=16
However after a period of inactivity i'm getting SQLExceptions:
com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
I'm trying to determine the cause of this. I'm guesses connection validation isn't working correctly, but as I understand this, the connection should be validate by tomcat before the application is allowed to use it. Any help would be greatly appreciated.
Related
I have an Spring boot application that runs several services and uses oracle database. The database is maintained properly, indexes also added up, and when executing SQL statements directly on SQL Developer, it's getting executed in milliseconds.
In the spring boot, I use this to execute the statement:
Session session = sessionFactory.getCurrentSession();
session.createQuery("from Table where id = :id and status = 0").setParameter("id", id);
Here is the config properties for the database:
spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true
hibernate.hbm2ddl.auto=none
Here is the way of datasource initialization in the datasource config:
#Bean
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource(url, name, pw);
dataSource.setDriverClassName(...);
return dataSource;
}
Recently, it takes so much time to acquire the database connection, it can go up to 10 seconds jut for acquiring connection. I don't think there is any problem in the query. As from the database side, it's also ok. The resources of server which running this service and the database server are also fine, as well as the network. The servers also have auto-scale feature to create new instance when the memory getting low. I just can't figure out what should I do to improve the acquisition time. Could you please help?
In our Spring boot application, we have set the following properties in application.properties
server.port=xxxx
server.servlet.context-path=/
#datasource
spring.datasource.driverClassName=com.mysql.jdbc.Driver
spring.datasource.url=jdbc:mysql://<IP:Port>/<DB Name>
spring.datasource.username=<username>
spring.datasource.password=<pwd>
spring.jpa.hibernate.ddl-auto=update
spring.jpa.hibernate.connection.release_mode=after_statement
.....
//Interface using JpaRepository
#Repository
public interface UserlicensesRepo extends JpaRepository<Userlicenses, Integer> {
Optional<Userlicenses> findById(Integer us);
Optional<Userlicenses> findByUsernameAndUserToken(String user, String token);
}
.....
//Using the repo instance to make DB call
Optional<Userlicenses> ul=userlicensesRepo.findByUsernameAndUserToken(userName, userTokenEncrypted);
The application is functionally working well but over a period of 4-5 days it is creating a lot of orphan DB connections leading to memory leak. We tried setting the "release_mode", please check if we have missed anything or made a mistake in the above configuration. Appreciate your help.
I have this property file which I store in a GitHub repo:
spring.datasource.type=com.zaxxer.hikari.HikariDataSource
spring.datasource.jdbcUrl=*DB URL*
spring.datasource.username=*USERNAME*
spring.datasource.password=*ENCRYPTED PASSWORD*
spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
spring.datasource.hikari.connection-timeout=30000
spring.datasource.hikari.maximum-pool-size=10
My config server pulls up the details from this repo.
Now I have a client that requires these properties to configure the HikariDataSource.
but when I start the client I get the following Error:
***************************
APPLICATION FAILED TO START
***************************
Description:
Cannot determine embedded database url for database type NONE
Action:
If you want an embedded database please put a supported one on the classpath. If you have database settings to be loaded from a particular profile you may need to active it (no profiles are currently active).
Seems to me like Hikari is unable to get these properties at start-up.
I was configuring my Hikari DataSource like so in my config class:
#Bean
#ConfigurationProperties("spring.datasource")
public HikariDataSource dataSource() {
return (HikariDataSource) DataSourceBuilder.create().type(HikariDataSource.class).build();
}
After removing this bean I still get the same error.
Any idea about what I should be doing here?
Thanks.
I was able to fix this by adding #EnableAutoConfiguration(exclude={DataSourceAutoConfiguration.class}) to the main class.
I'm working on a Spring Boot application with Liquibase integration to setup the database. We use a different user for the database changes which we configured using the application.properties file
liquibase.user=abc
liquibase.password=xyz
liquibase.url=jdbc:postgresql://something.eu-west-1.rds.amazonaws.com:5432/app?ApplicationName=${appName}-liquibase
liquibase.enabled=true
liquibase.contexts=dev,postgres
We have at this moment 3 different microservices in deployment and we noticed that for every running instance, Liquibase opens 10 connections and it never closes these connections unless we stop the application. This basically means that in development we regularly hit the connection limit of our Amazon RDS instance.
Right now, in development, 40 of 74 active connections are occupied by Liquibase. If we ever want to go to production with this, having autoscaling enabled for all the microservices, that would mean we'll have to over-scale the database in order not to hit any connection limits.
Is there a way to
tell liquibase to not use a connection pool of 10 connections
tell liquibase to stop or close the connections
So far I found no documentation on how to do this.
Thanks to the response of Slava I managed to fix the problem with following datasource configuration class
#Configuration
public class LiquibaseDataSourceConfiguration {
private static final Logger LOG = LoggerFactory.getLogger(LiquibaseDataSourceConfiguration.class);
#Autowired
private LiquibaseDataSourceProperties liquibaseDataSourceProperties;
#LiquibaseDataSource
#Bean
public DataSource liquibaseDataSource() {
DataSource ds = DataSourceBuilder.create()
.username(liquibaseDataSourceProperties.getUser())
.password(liquibaseDataSourceProperties.getPassword())
.url(liquibaseDataSourceProperties.getUrl())
.driverClassName(liquibaseDataSourceProperties.getDriver())
.build();
if (ds instanceof org.apache.tomcat.jdbc.pool.DataSource) {
((org.apache.tomcat.jdbc.pool.DataSource) ds).setInitialSize(1);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMaxActive(2);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMaxAge(1000);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMinIdle(0);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMinEvictableIdleTimeMillis(60000);
} else {
// warnings or exceptions, whatever you prefer
}
LOG.info("Initialized a datasource for {}", liquibaseDataSourceProperties.getUrl());
return ds;
}
}
The documentation of the properties can be found on the site of Tomcat: https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html
initialSize: The initial number of connections that are created when the pool is started
maxActive: The maximum number of active connections that can be allocated from this pool at the same time
minIdle: The minimum number of established connections that should be kept in the pool at all times
maxAge: Time in milliseconds to keep this connection. When a connection is returned to the pool, the pool will check to see if the now - time-when-connected > maxAge has been reached, and if so, it closes the connection rather than returning it to the pool. The default value is 0, which implies that connections will be left open and no age check will be done upon returning the connection to the pool.
minEvictableIdleTimeMillis: The minimum amount of time an object may sit idle in the pool before it is eligible for eviction.
So it does not appear to be a connection leak, it's just the default configuration of the datasource which is not optimal for Liquibase if you use a dedicated datasource. I don't expect this to be a problem if the liquibase datasource is your primary datasource.
Update: This has been fixed in 2.5.0-M2 and Liquibase now uses a SimpleDriverDataSource without a connection pool.
Original answer: This change to connection pool management was introduced in Spring Boot version 2.0.6.RELEASE, and only takes effect if you use Spring Boot Actuator. There is an actuator endpoint (enabled by default) which allows you to get change sets applied by Liquibase. For this to work Liquibase keeps its database connections open. You can disable the endpoint with management.endpoint.liquibase.enabled = false, in which case the connection pool used by Liquibase will be shutdown after the initial run.
GitHub issue related to this change: https://github.com/spring-projects/spring-boot/issues/13832
Spring Boot Actuator (see 12. Liquibase: https://docs.spring.io/spring-boot/docs/2.0.6.RELEASE/actuator-api/html/
I don't know why liquibase doesn't close a connection, maybe it's a bug and you should create an issue for that.
To set connection pool for liquibase you have to create a custom data source and mark it with #LiquibaseDataSource annotation.
Related issues provide more details:
Possibility to specify custom dataSource configuration for liquibase only
Add LiquibaseDataSource annotation
I'm aware that H2 has a boolean property/setting called DATABASE_TO_UPPER, which you can set at least in the connection URL, as in: ;DATABASE_TO_UPPER=false
I’d like to set this to false, but in my Spring Boot app, I don’t explicitly have a H2 connection URL anywhere. Implicitly there sure is a connection URL though, as I can see in the logs:
o.s.j.d.e.EmbeddedDatabaseFactory: Shutting down embedded database:
url='jdbc:h2:mem:2fb4805b-f927-49b3-a786-2a2cac440f44;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false'
So the question is, what's the easiest way to tell H2 to disable DATABASE_TO_UPPER in this scenario? Can I do it in code when creating the H2 datasource with EmbeddedDatabaseBuilder (see below)? Or in application properties maybe?
This is how the H2 database is explicitly initialised in code:
#Configuration
#EnableTransactionManagement
public class DataSourceConfig {
#Bean
public DataSource devDataSource() {
return new EmbeddedDatabaseBuilder()
.generateUniqueName(true)
.setType(EmbeddedDatabaseType.H2)
.setScriptEncoding("UTF-8")
.ignoreFailedDrops(true)
.addScripts("db/init.sql", "db/schema.sql", "db/test_data.sql")
.build();
}
}
Also, I'm telling JPA/Hibernate not to auto-generate embedded database (without this there was an issue that two in-memory databases were launched):
spring.jpa.generate-ddl=false
spring.jpa.hibernate.ddl-auto=none
You can't w\ the generateUniqueName, but if you call setName("testdb;DATABASE_TO_UPPER=false") you can add parameters. I doubt this is officially supported, but it worked for me.
The spring code that generates the connection url is like this:
String.format("jdbc:h2:mem:%s;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false", databaseName)
You may want abandon using explicit creation via EmbeddedDatabaseBuilder. Spring Boot creates H2 instance automatically based on configuration. So I would try this in application.properties:
spring.datasource.url=jdbc:h2:file:~/testdb;DATABASE_TO_UPPER=false