I am using spring data jpa and hikari connection pooling. In repository file, I am using methods to connect to database. I would like to know how and where to close the database connections.
Repository
public interface abc extends JPARepository<abc, int>{
List<abc> findById(int id);}
Any help on how to close the connections and where(service layer or repository) would be really appreciated
`
With Connection Pooling, using the framework, The task of creating a connection before each operation and closing the connection after the operation is now taken from the programmer and transferred to the Spring Context:-
The application requests a connection from the connection pool.
If an unused connection exists, it is returned by the pool; otherwise, the pool creates a new connection.
The application sends a query to the Hybrid Data Pipeline connectivity service.
The application obtains query results.
The application displays the result to the end-user.
The application closes the connection, which returns the connection to the pool.
Note: The application calls the close() method, which allows the connection to remain open. The pool receives the notification of the close request.
Related
When a spring boot request starts, it obtains a connection from the pool. My question is - is this connection remains tied to the request thread (even if it is not executing any query) and only returned to the pool when request completes?
For example, if I'm doing something like:
Request starts
Execute a query (20ms)
Call external http service (1500ms)
Request completes
Would the connection obtained by this request thread remain occupied with the thread (and not available to other requests) for 20ms or 1520ms?
PS: I'm using Spring Boot 2.0 with HikariCP and I'm not using #Transactional.
Thanks.
If you have Spring MVC on your classpath, Spring will configure an instance of OpenEntityManagerInViewInterceptor automatically (in org.springframework.boot.autoconfigure.orm.jpa.JpaBaseConfiguration.JpaWebConfiguration), unless you configure spring.jpa.open-in-view to false. This interceptor will open an instance of EntityManager and bind it to the TransactionSynchronizationManager for the whole duration of the request.
What that means is that at the first use of a repository query, it will borrow a connection from the pool and keep it until the end of the request when the EntityManager is closed by the interceptor. It's the EntityManager's responsibility to return the connection to the pool. So, in that case, the connection will be borrowed for 1520ms in your example.
If you disable the spring.jpa.open-in-view option, a connection will be borrowed and returned at each use of a repository query given you don't use any explicit transaction. While this may look better, you have to keep in mind that managed instances of your entities will need to be reattached systematically on each persist operation so it may be costly. You will also loose the benefits of the EntityManager caching. To avoid this, use transactions to read/modify/persist your entities and avoid reattaching.
Finally, keep in mind that if you disable spring.jpa.open-in-view, since you don't have an EntityManager around for the duration of the request, you will need to make sure your lazy loaded relations are loaded inside a transaction otherwise you'll get the dreaded LazyInitializationException.
Basically it depends,
If you close conection it will be release back to pool and will be ready soon according to configuration (below) so it will be ~20ms (+ time to get back to pool)
If you don't close the connection it will wait until the configuration allow it, if it allow indefinite time, theoretically it can be causing a leak in your application and not return to pool until apllication is shutdown.
See answer about Hikari handling returning connections to pool:
The Hikari housekeeper runs every 30s, which closes any connections that are not in use and are older than maxLifetime. If there are more than minimumIdle number of connections the housekeeper will close connections that have been idle for longer than idleTimeout.
See more about max life time of connection:
by default Oracle does not enforce a max lifetime for connections (neither on JDBC driver side (1), nor on server side(2)). So in this respect, the "infrastructure-imposed connection time limit" is +infinity
One other thing I'm finding, is that it appears that Javers is grabbing all of the available Connections out of my connection pool (created via Spring DataSourceBuilder). I'm not using Hibernate/JPA, just straight JDBC via JdbcTemplate and mostly MyBatis for my entity queries.
I've added a logging statement to my ConnectionProvider for Javers, and at the start of the application when it queries for the schema, it pulls 4 connections to check for each of the tables, and then never returns any of them even after the commit from the PlatformTransactionManager.
I understand from https://stackoverflow.com/a/35147884/570291 that it's supposed to participate in the same connection as the current Transaction. Since I'm not using Hibernate/JPA, does that mean I need to implement the connection tracking/etc from MyBatis to the Javers ConnectionProvider to return the same connection (if there is one), and then handle closing (returning to the pool) of that connection at the end of the transaction?
I found DataSourceUtils.getConnection(DataSource) which is a Spring utility class to get a connection from the given DataSource including if it's tied to a current transaction or not as appropriate. Using that in the ConnectionProvider looks like it's done the trick of keeping the connection for the existing transaction.
JaVers won't return connections to application's connection pool for the same reason as it won't call sql commit or rollback.
Managing connactions and transactions is the application's responsibility, not JaVers. We call it passive mode, from Javers doc:
- JaVers doesn’t create JDBC connections on its own and uses connections provided by an application (via ConnectionProvider.getConnection()).
- JaVers philosophy is to use application’s transactions and never to call SQL commit or rollback commands on its own.
Thanks to this approach, data managed by an application (domain objects) and data managed by JaVers (object snapshots) can be saved to SQL database in one transaction.
In JaVers project there is no mybatis support, so you need to implement ConnectionProvider for mybatis on your own.
Proper implementation of ConnectionProvider shouldn't create new sql connection for every getConnection() call. It should return the connection which underlies current application's transaction. Typically, it's implemented using ThreadLocal.
As you mentioned, ConnectionProvider should handle committing transactions and closing connections.
I have been researching Spring Data Rest especially for cassandra and one of the questions my coworkers and I had was when does Spring Data connect to the database. We don't always want a rest controller to connect to the database so when does spring establish a connection if say we had a class extend the CRUDRepository? Does it connect to the database during the start of application itself? Is that something we can control?
For example, I implemented this example on Spring's website:
https://spring.io/guides/gs/accessing-data-rest/
At what point in the code does spring connect to the database?
Spring will connect to the DB as soon as the Datasource get initialized. Basically, Spring contexts will become alive somehow (Web listeners, manually calling them) and start creating beans. As soon as it reaches the Datasource, connection will be made and the connection pool will be populated.
Of course the above is based on a normal out of the box configuration and everything can be setup up to your taste.
So unless, you decide to control the connections yourself, DB connections will be sitting there waiting to be used.
Disagree with the above answer.
As part of research i initiated the datasource using a bean configuration and then changed my database password(not in my spring application but the real db username password)
The connection stays for a while and then in some point of time (maybe idle time) it stops working and throws credential exception.
This is enough to say the JPA does not keep the connection sitting and waiting to be used but uses some mechanism to occupy/release the db connection as per the need.
Official Sybase jConnect Programmers Reference suggests following way to use pooled connections:
SybConnectionPoolDataSource connectionPoolDataSource = new SybConnectionPoolDataSource();
...
Connection ds = connectionPoolDataSource.getConnection();
...
ds.close();
However getDataSource always causes exception. I decompiled SybConnectionPoolDataSource and found that the method call explicitly generates an error:
public Connection getConnection() throws SQLException
{
ErrorMessage.raiseError("JZ0S3", "getConnection()");
return null;
}
Does anyone have an idea why the documentation contradicts to the implementation?
I can't comment specifically for Sybase because 1) I don't use it and 2) your link doesn't work, but I can try to give you a theory based on my own experience maintaining a JDBC driver (Jaybird/Firebird JDBC) and looking at what some of the other implementations do.
The ConnectionPoolDataSource is probably the least understood part of the JDBC API. Contrary to what the naming suggests and how it has been implemented in some JDBC implementations this interface SHOULD NOT provide connection pooling and should not implement DataSource (or at least: doing that can lead to confusion and bugs; my own experience).
The javadoc of the ConnectionPoolDataSource is not very helpful, the javax.sql package documentation provides a little bit more info, but you really need to look at the JDBC 4.1 specification, Chapter 11 Connection Pooling to get a good idea how it should work:
[...] the JDBC driver provides an implementation of ConnectionPoolDataSource that the application server uses to build and manage the connection pool.
In other words: ConnectionPoolDataSource isn't meant for direct use by a developer, but instead is used by an application server for its connection pool; it isn't a connection pool itself.
The application server provides its clients with an implementation of the DataSource interface that makes connection pooling transparent to the client.
So the connection pool is made available to the user by means of a normal DataSource implementation. The user uses this as would it be one that doesn't provide pooling, and uses the connections obtained as if it is a normal physical connection instead of one obtained from a connection pool:
When an application is finished using a connection, it closes the logical connection using the method Connection.close. This closes the logical connection but does not close the physical connection. Instead, the physical connection is returned to the pool so that it can be reused.
Connection pooling is completely transparent to the client: A client obtains a pooled connection and uses it just the same way it obtains and uses a non pooled connection.
This is further supported by the documentation of PooledConnection (the object created by a ConnectionPoolDataSource):
An application programmer does not use the PooledConnection interface directly; rather, it is used by a middle tier infrastructure that manages the pooling of connections.
When an application calls the method DataSource.getConnection, it gets back a Connection object. If connection pooling is being done, that Connection object is actually a handle to a PooledConnection object, which is a physical connection.
The connection pool manager, typically the application server, maintains a pool of PooledConnection objects. If there is a PooledConnection object available in the pool, the connection pool manager returns a Connection object that is a handle to that physical connection. If no PooledConnection object is available, the connection pool manager calls the ConnectionPoolDataSource method getPoolConnection to create a new physical connection. The JDBC driver implementing ConnectionPoolDataSource creates a new PooledConnection object and returns a handle to it.
Unfortunately, some of JDBC drivers have created data sources that provide connection pooling by implementing both DataSource and ConnectionPoolDataSource in a single class, instead of the intent of the JDBC spec of having a DataSource that uses a ConnectionPoolDataSource. This has resulted in implementations that would work if used as a normal DataSource, but would break if used as a ConnectionPoolDataSource (eg in the connection pool of an application server), or where the interface was misunderstood and the wrong methods where used to create connections (eg calling getPooledConnection().getConnection()).
I have seen implementations (including in Jaybird) where the getPooledConnection() would be used to access a connection pool internal to the implementation, or where only connections obtained from the getConnection() of the implementation would work correctly, leading to all kinds of oddities and incorrect behavior when that implementation was used to fill a connection pool in an application server using the getPooledConnection().
Maybe Sybase did something similar, and then decided that wasn't such a good idea so they changed the DataSource.getConnection() to throw an exception to make sure it wasn't used in this way, but at the same time maintaining the API compatibility by not removing the methods defined by DataSource. Or maybe they extended a normal DataSource to easily create the physical connection (instead of wrapping a normal one), but don't want users to use it as a DataSource.
I am using a connection pool and the Spring TransactionTemplate. If you want to shutdown the connection pool first all connection have to be returned to the pool, this means connection.close() has to be called. I have one thread using the TransactionTemplate for some queries and another thread that wants to call some shutdown method on the connection pool, but before doing this it first has to tell the TransactionTemplate to close all connection (actually only returning them to the pool).
How can this be done in Spring to immediately call close on the used connection?
If you are using Hibernate along with Spring please use:
hibernate.connection.release_mode=after_transaction
If you want to release connection just after transaction.
hibernate.connection.release_mode=after_statement
If you want to release connection after each statement
This two settings are the only ways I know that will make used connection being released faster than the default behavior. At least as far as Hibernate is concerned. If you are using other library please describe which.