I'm running a simple update query with MyBatis in my SpringBoot project, and using Hiraki connection pool.
In testing stage, only single tread of update query is execute in the program. When the program reach the line of update query, the Hikari connection start successfully but somehow the active connection will not be run and stuck forever.
//my hikari datasource setup
#Bean
public HikariDataSource dataSource() {
HikariDataSource db = new HikariDataSource();
db.setDriverClassName(driverClassName);
db.setJdbcUrl(url);
db.setUsername(username);
db.setPassword(pwd);
db.setReadOnly(false);
db.setMaximunPoolSize(80);
db.setConnectionTimeout(30000);
db.setIdleTimeout(30000);
db.setMaxLifetime(30000);
db.setMinimunIdle(5);
db.setValidationTimeout(500);
return db;
}
Following is the screenshot of stacktrace
It's weird, this situation will not occur if i use insert statement, it only happen when I call update statement
Related
I have an Spring boot application that runs several services and uses oracle database. The database is maintained properly, indexes also added up, and when executing SQL statements directly on SQL Developer, it's getting executed in milliseconds.
In the spring boot, I use this to execute the statement:
Session session = sessionFactory.getCurrentSession();
session.createQuery("from Table where id = :id and status = 0").setParameter("id", id);
Here is the config properties for the database:
spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true
hibernate.hbm2ddl.auto=none
Here is the way of datasource initialization in the datasource config:
#Bean
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource(url, name, pw);
dataSource.setDriverClassName(...);
return dataSource;
}
Recently, it takes so much time to acquire the database connection, it can go up to 10 seconds jut for acquiring connection. I don't think there is any problem in the query. As from the database side, it's also ok. The resources of server which running this service and the database server are also fine, as well as the network. The servers also have auto-scale feature to create new instance when the memory getting low. I just can't figure out what should I do to improve the acquisition time. Could you please help?
Hi I use spring jpa and as I understand its working mechanism truely it created at once in a singleton way.
Is it possible to change or recreate datasource on running environment.Scenario like this,Mypassword changed and if I wont stop application that time all my calls take exception.I have a mechanism to check password and change it dynamically and my others request get new password create new datasource and keep on working.
And another question is I have multiple datasource,at the application start if one of this datasource get exception that time my application cannot start.What I want if one of datasource not working ,application can continue to warmup and try to check creating datasource each related request.
I dont want to create persistencejpaconfig each request but I want to makes changes on datasource in every request if it is neccessary
#Configuration
#EnableTransactionManagement
public class PersistenceJPAConfig{
....
#Bean
public DataSource dataSource(){
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("********");
dataSource.setUrl("*********");
dataSource.setUsername( "**********" );
dataSource.setPassword( "********" );
return dataSource;
}
You can simply implement your own DataSource that delegates to the real DataSource and creates a new one when the password changes.
DelegatingDataSource might be of help, either as a basis class or, since you are going to change the DataSource as a template for an implementation.
I'm working on a Spring Boot application with Liquibase integration to setup the database. We use a different user for the database changes which we configured using the application.properties file
liquibase.user=abc
liquibase.password=xyz
liquibase.url=jdbc:postgresql://something.eu-west-1.rds.amazonaws.com:5432/app?ApplicationName=${appName}-liquibase
liquibase.enabled=true
liquibase.contexts=dev,postgres
We have at this moment 3 different microservices in deployment and we noticed that for every running instance, Liquibase opens 10 connections and it never closes these connections unless we stop the application. This basically means that in development we regularly hit the connection limit of our Amazon RDS instance.
Right now, in development, 40 of 74 active connections are occupied by Liquibase. If we ever want to go to production with this, having autoscaling enabled for all the microservices, that would mean we'll have to over-scale the database in order not to hit any connection limits.
Is there a way to
tell liquibase to not use a connection pool of 10 connections
tell liquibase to stop or close the connections
So far I found no documentation on how to do this.
Thanks to the response of Slava I managed to fix the problem with following datasource configuration class
#Configuration
public class LiquibaseDataSourceConfiguration {
private static final Logger LOG = LoggerFactory.getLogger(LiquibaseDataSourceConfiguration.class);
#Autowired
private LiquibaseDataSourceProperties liquibaseDataSourceProperties;
#LiquibaseDataSource
#Bean
public DataSource liquibaseDataSource() {
DataSource ds = DataSourceBuilder.create()
.username(liquibaseDataSourceProperties.getUser())
.password(liquibaseDataSourceProperties.getPassword())
.url(liquibaseDataSourceProperties.getUrl())
.driverClassName(liquibaseDataSourceProperties.getDriver())
.build();
if (ds instanceof org.apache.tomcat.jdbc.pool.DataSource) {
((org.apache.tomcat.jdbc.pool.DataSource) ds).setInitialSize(1);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMaxActive(2);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMaxAge(1000);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMinIdle(0);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMinEvictableIdleTimeMillis(60000);
} else {
// warnings or exceptions, whatever you prefer
}
LOG.info("Initialized a datasource for {}", liquibaseDataSourceProperties.getUrl());
return ds;
}
}
The documentation of the properties can be found on the site of Tomcat: https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html
initialSize: The initial number of connections that are created when the pool is started
maxActive: The maximum number of active connections that can be allocated from this pool at the same time
minIdle: The minimum number of established connections that should be kept in the pool at all times
maxAge: Time in milliseconds to keep this connection. When a connection is returned to the pool, the pool will check to see if the now - time-when-connected > maxAge has been reached, and if so, it closes the connection rather than returning it to the pool. The default value is 0, which implies that connections will be left open and no age check will be done upon returning the connection to the pool.
minEvictableIdleTimeMillis: The minimum amount of time an object may sit idle in the pool before it is eligible for eviction.
So it does not appear to be a connection leak, it's just the default configuration of the datasource which is not optimal for Liquibase if you use a dedicated datasource. I don't expect this to be a problem if the liquibase datasource is your primary datasource.
Update: This has been fixed in 2.5.0-M2 and Liquibase now uses a SimpleDriverDataSource without a connection pool.
Original answer: This change to connection pool management was introduced in Spring Boot version 2.0.6.RELEASE, and only takes effect if you use Spring Boot Actuator. There is an actuator endpoint (enabled by default) which allows you to get change sets applied by Liquibase. For this to work Liquibase keeps its database connections open. You can disable the endpoint with management.endpoint.liquibase.enabled = false, in which case the connection pool used by Liquibase will be shutdown after the initial run.
GitHub issue related to this change: https://github.com/spring-projects/spring-boot/issues/13832
Spring Boot Actuator (see 12. Liquibase: https://docs.spring.io/spring-boot/docs/2.0.6.RELEASE/actuator-api/html/
I don't know why liquibase doesn't close a connection, maybe it's a bug and you should create an issue for that.
To set connection pool for liquibase you have to create a custom data source and mark it with #LiquibaseDataSource annotation.
Related issues provide more details:
Possibility to specify custom dataSource configuration for liquibase only
Add LiquibaseDataSource annotation
I am fetching the connection in jdbctemplate in below fashion:-
getJdbcTemplate().getDataSource().getConnection()
Is it necessary to close the connection fetched in the above manner? The spring JDBCTemplate API states that connection closures will be handled automatically , so I am not sure if this is happening correctly.
http://docs.spring.io/spring/docs/3.0.x/spring-framework-reference/html/jdbc.html
When you are obtaining the DataSource from the JdbcTemplate and use that to obtain a Connection you are basically completely bypassing the JdbcTemplate. You now have a very complex way of obtaining a new Connection. Because this connection isn't managed by Spring but yourself you also need to close it and apply exception handling.
It is better to use the ConnectionCallback instead to get a Connection. The JdbcTemplate will then manage the Connection and do all resource handling.
getJdbcTemplate().execute(new ConnectionCallback<Void>() {
public Void doInConnection(Connection conn) {
// Your JDBC code here.
}
});
It would even better to use one of the other JdbcTemplate methods and write proper code which would save you from messing with plain JDBC code at all.
I have created glassfish connection pool with ResourceType as ConnectionPoolDataSource.So, glassfish will use the native connection pool implementation for connection pooling. I am not using XADatasource ResourceType as I don't want to perform any distributed transactions.
My application requires the use of TEMPERORY MYSQL table creation at run time. So I am using
the below code to get the connection from JNDI Datasource of glassfish.
#Resource(mappedName = "jdbc/xxxxx")
private DataSource dataSource;
public Connection getConnection() throws SQLException {
Connection con = dataSource.getConnection();
return con;
}
Now, My question is, Can I perform setAutoCommit(false), commit() and rollback(), close() on this Connection object????
In forum, I read that, we should not call these methods on Connection object, if we get the Connection from Container Managed Distributed Transaction (XADataSource) as its involved in distributed transactions.
But, I am getting this connection from non-distributed transactions.So, I can call those methods right???
Other question is, after performing db operations, If I call con.close(), will this connection go back to the connection pool again?