Writing a EJB 3 Service with container managed transaction. If i set autocommit false perform SQL updates and close the JDBC connection within the EJB transaction,will the transaction be still active and commit after the method completes without any exception with the connection currently closed
If you use CMT and connection from the JTA enabled datasource, then when you call connection.close() the connection is not actually closed but returned to the pool. You should also not change the autocommit with the connections taken from such datasource as they are by default set to false and shouldn't be changed. Yes, transaction will still be active, and will be commited/rollbacked, when the transaction scope ends (it doesn't have to be exactly when this particular method completes, since transaction could be started earlier and this method may only participate in it via REQUIRED, instead of running in separate as with REQIRES_NEW.
Related
Spring Data JPA or Hibernate have by default auto-commit set to false. This sounds reasonable as these frameworks work with entities and an update to an entity could involve updates to multiple tables through multiple SQL queries. So by setting auto-commit to false and explicitly controlling a transaction these frameworks ensure that a change to entity is atomic and consistent.
But now Hikari is the default connection pool provider for spring data jpa, and on looking at application logs I see hikari sets auto-commit to true for the connection pool.
2021-10-24 11:30:07.815 DEBUG [restartedMain] com.zaxxer.hikari.HikariConfig||HikariConfig.logConfiguration:1135: autoCommit................................true
Any explanation of why this is set so and does this effect transactions(I don't think it effects transactions as each transaction might again set auto-commit to false and thus take over when to commit the transaction.)
Edit - following #ken-chan answer and discussion.
For projects using spring data jpa with #Transactional (with 100% Hiberante), changing hikaris connection pool settings to auto-commit=false should give performance benefit. Please see answer and subsequent discussion for more details.
I think Hikari just follows the default auto-commit value (i.e. true) defined by JDBC in order to align with its default behaviour .(See this)
And your guess is correct , the framework will take care to configure the necessary auto-commit value of the JDBC Connection such that it can do the transactional stuff over multiple JDBC Statement.
For example in Spring #Transactionl using JDBC , the following codes show that if the auto-commit is enabled , it will disable it before executing any transactional codes. It also will re-enable it after completing the transaction.
// Switch to manual commit if necessary. This is very expensive in some JDBC drivers,
// so we don't want to do it unnecessarily (for example if we've explicitly
// configured the connection pool to set it already).
if (con.getAutoCommit()) {
txObject.setMustRestoreAutoCommit(true);
if (logger.isDebugEnabled()) {
logger.debug("Switching JDBC Connection [" + con + "] to manual commit");
}
con.setAutoCommit(false);
}
In spring boot-jpa if I am using hikaricp pool configuration in which I have set idle-timeout as 5 mins, max life time 2 mins and I have set jpa's ddl-auto property as create-drop, then if the connection which created the table sits idle for 7 mins, will drop the table in db?
Tables are dropped when entire SessionFactory is closed.
SessionFactory has a data source, which in your case maintains a pool of connections. Closing a connection doesn't imply that SessionFactory will get closed.
As the SessionFactory is not closed by a closed connection, the table is not dropped.
I'm working on a Spring Boot application with Liquibase integration to setup the database. We use a different user for the database changes which we configured using the application.properties file
liquibase.user=abc
liquibase.password=xyz
liquibase.url=jdbc:postgresql://something.eu-west-1.rds.amazonaws.com:5432/app?ApplicationName=${appName}-liquibase
liquibase.enabled=true
liquibase.contexts=dev,postgres
We have at this moment 3 different microservices in deployment and we noticed that for every running instance, Liquibase opens 10 connections and it never closes these connections unless we stop the application. This basically means that in development we regularly hit the connection limit of our Amazon RDS instance.
Right now, in development, 40 of 74 active connections are occupied by Liquibase. If we ever want to go to production with this, having autoscaling enabled for all the microservices, that would mean we'll have to over-scale the database in order not to hit any connection limits.
Is there a way to
tell liquibase to not use a connection pool of 10 connections
tell liquibase to stop or close the connections
So far I found no documentation on how to do this.
Thanks to the response of Slava I managed to fix the problem with following datasource configuration class
#Configuration
public class LiquibaseDataSourceConfiguration {
private static final Logger LOG = LoggerFactory.getLogger(LiquibaseDataSourceConfiguration.class);
#Autowired
private LiquibaseDataSourceProperties liquibaseDataSourceProperties;
#LiquibaseDataSource
#Bean
public DataSource liquibaseDataSource() {
DataSource ds = DataSourceBuilder.create()
.username(liquibaseDataSourceProperties.getUser())
.password(liquibaseDataSourceProperties.getPassword())
.url(liquibaseDataSourceProperties.getUrl())
.driverClassName(liquibaseDataSourceProperties.getDriver())
.build();
if (ds instanceof org.apache.tomcat.jdbc.pool.DataSource) {
((org.apache.tomcat.jdbc.pool.DataSource) ds).setInitialSize(1);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMaxActive(2);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMaxAge(1000);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMinIdle(0);
((org.apache.tomcat.jdbc.pool.DataSource) ds).setMinEvictableIdleTimeMillis(60000);
} else {
// warnings or exceptions, whatever you prefer
}
LOG.info("Initialized a datasource for {}", liquibaseDataSourceProperties.getUrl());
return ds;
}
}
The documentation of the properties can be found on the site of Tomcat: https://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html
initialSize: The initial number of connections that are created when the pool is started
maxActive: The maximum number of active connections that can be allocated from this pool at the same time
minIdle: The minimum number of established connections that should be kept in the pool at all times
maxAge: Time in milliseconds to keep this connection. When a connection is returned to the pool, the pool will check to see if the now - time-when-connected > maxAge has been reached, and if so, it closes the connection rather than returning it to the pool. The default value is 0, which implies that connections will be left open and no age check will be done upon returning the connection to the pool.
minEvictableIdleTimeMillis: The minimum amount of time an object may sit idle in the pool before it is eligible for eviction.
So it does not appear to be a connection leak, it's just the default configuration of the datasource which is not optimal for Liquibase if you use a dedicated datasource. I don't expect this to be a problem if the liquibase datasource is your primary datasource.
Update: This has been fixed in 2.5.0-M2 and Liquibase now uses a SimpleDriverDataSource without a connection pool.
Original answer: This change to connection pool management was introduced in Spring Boot version 2.0.6.RELEASE, and only takes effect if you use Spring Boot Actuator. There is an actuator endpoint (enabled by default) which allows you to get change sets applied by Liquibase. For this to work Liquibase keeps its database connections open. You can disable the endpoint with management.endpoint.liquibase.enabled = false, in which case the connection pool used by Liquibase will be shutdown after the initial run.
GitHub issue related to this change: https://github.com/spring-projects/spring-boot/issues/13832
Spring Boot Actuator (see 12. Liquibase: https://docs.spring.io/spring-boot/docs/2.0.6.RELEASE/actuator-api/html/
I don't know why liquibase doesn't close a connection, maybe it's a bug and you should create an issue for that.
To set connection pool for liquibase you have to create a custom data source and mark it with #LiquibaseDataSource annotation.
Related issues provide more details:
Possibility to specify custom dataSource configuration for liquibase only
Add LiquibaseDataSource annotation
I am getting the following error below
Unexpected error
org.springframework.transaction.CannotCreateTransactionException:
**CannotCreateTransactionException:** Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBC
Exception: Could not open connection
Here is the scenario in which it occurs :
I have recently moved some table from mysql to mongo. The code is written in such a way that either data would be taken from mongo/mysql.
The code is written in a method block which is annotated with #Transactional provided by spring framework.
There is hibernate layer which is using transaction provided by spring. c3p0 is the connection pool.
The parameter of connection pool is
hibernate.c3p0.min_size=5
hibernate.c3p0.timeout=1200
hibernate.c3p0.max_size=35
hibernate.c3p0.max_statements=50
The problem comes when we try to pull the data from mongo.Looks like the transaction is not getting closed because of mongo operation.The database connection is not getting released .It reaches the max size defined in the pool.
Tried the query in DB to find out the connection
show status like '%onn%';
Any suggestion to resolve this would really help.
Thanks
I'm getting the following stack trace:
org.springframework.orm.hibernate3.HibernateOptimisticLockingFailureException: Object of class [com.btfin.wrapcore.request.MFRequest] with identifier [2850448]: optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [com.btfin.wrapcore.request.MFRequest#2850448]
at org.springframework.orm.hibernate3.SessionFactoryUtils.convertHibernateAccessException(SessionFactoryUtils.java:672)
at org.springframework.orm.hibernate3.HibernateTransactionManager.convertHibernateAccessException(HibernateTransactionManager.java:793)
Which is due to an optimistic locking exception. I can address the root cause of this.
My question is - in this scenario - the exception handling sets the database connection to 'closed'. (Which causes issues with my connection pool).
What is the pattern for handling a database exception like HibernateOptimisticLockingFailureException that bubbles up through spring and hibernate and returns a closed connection?
Do you know the part in the Spring/Hibernate code that sets the connection to closed?
Hibernate Docs clearly state that if any exception occur while working with the Session, the Session can't be reused afterwards. Moreover, each Session can encompass several transactions and after each transaction commits - the same happens, the connection is closed.
But while using a connection pool, the connection is not literally closed, when close() method is invoked, the connection is returned to the pool without physical closing:
when an application closes its connection, the underlying physical
connection is recycled rather than being closed.
So if you have problems with the connection being physically closed, I would rather pay more attention to the pool, not to Hibernate or Spring - they can't do more than invoking close() which should work as I described earlier.