I have a spring boot app which uses spring data and hikaricp for db connection pooling. I noticed the following behaviour that looks strange to me:
I have one method which is not transactional and in that method several db queries are executed using spring data repositories
public void testMethod(final Long firstRepositoryId, final Long secondRepositoryId) {
final DomainObject result = firstRepository.findById(firstRepositoryId);
// here there's some code that is processing the result without db queries
secondRepository.findById(secondRepositoryId);
// some more logic here without additional db queries
}
So as expected when there's no transaction on the method then the spring data methods opens a transaction for executing the query and complete it after the methods returns. I have enabled transaction logging so there's the following log output:
2021-06-03 15:34:30.961 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Getting transaction for [com.test.FirstRepository.findById]
2021-06-03 15:34:30.966 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Completing transaction for [com.test.FirstRepository.findById]
2021-06-03 15:34:30.967 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Getting transaction for [com.test.SecondRepository.findById]
2021-06-03 15:34:30.972 TRACE c681f76a-5d7e-41d5-9e50-fb6f96169681 --- [tp659271212-291] o.s.t.i.TransactionInterceptor : Completing transaction for [com.test.SecondRepository.findById]
Everything seems to be exactly how I expects to be. The thing I can't understand is the hikari behaviour. This method is invoked within a http request. A connection is taken from hikari cp right after the execution of the firstRepository.findById but this connection is returned in the pool only after the http controller returns response. What I expect is that a connection is taken after a transaction is opened and returned back after the transaction is completed. Is there something that I miss or maybe I have some wrong configuration?
P.S. I'm monitoring the active hikari connections through the spring boot actuator prometheus data. And to be able to reproduce the behavior I explained above I'm suspending the connection thread with several debug breakpoints.
I found out what causes this behaviour- it's Spring functionality to maintain the hibernate session in view in order to be able to retrieve lazy loaded data in the view. In order to disable this you need the following property:
spring.jpa.open-in-view=false
Here's another SO post where is explained what it does:
What is this spring.jpa.open-in-view=true property in Spring Boot?
Related
I am analysing a "classic" Hibernate error :
org.hibernate.LazyInitializationException : could not initialize proxy – no Session.
I am wondering how it could happen whereas the Spring Open In View mode is enabled?
If you have any documentation or knowledge on a possible reason, please share.
Thanks
Here are some feebacks on my context, and an example of what could cause LazyInitializationException event if Spring Open In View is enabled.
Context
My application is a REST API server developed with Spring Boot 2.5.3.
Spring Open In View is kept enabled by default.
A lot of Services are annotated with #Transactional methods.
Spring-data is used, and Entitymanager too, to create some native queries.
The error
One REST API request, that generates a lot requests to the database (both selections and insertions), fails with LazyInitializationException.
Debugging
By setting a breakpoint when LazyInitializationException is thrown, I discovered that the exception is thrown in org.hibernate.proxy.AbstractLazyInitializer#initialize, because the session is null. Then I discovered that the session is set to null, when the method EntityManager.clear is called. For any reason I don't know, this method was explicitely called in the code.
Fixing
So I just removed the call to EntityManager.clear, and the request works. I'm still wondering why previous developers wanted to clear the EntityManager, probably because they were confused with the transaction management.
Conclusion
Even in Spring Open In View is enabled, and as a result, event if a Hibernate Session is opened, calling EntityManager.clear unset the session to the entities loaded before. Then trying to access a Lazy Loaded field on those entities throws LazyInitializationException.
I hope this will help someone.
Here is the documentation from the latest hibernate version.
As you can see
Indicates an attempt to access not-yet-fetched data outside of a
session context. For example, when an uninitialized proxy or
collection is accessed after the session was closed.
Most probably somewhere you read some entity and then outside of a transaction you try to read some collection which was by default lazy loaded. So then another request needs to be done to database, but since you are out of transaction you receive this exception.
This usually occurs in case you load an entity without the annotation #Transactional or some other configuration to load it inside a transaction and then you give this to your controller to be returned to the user. Then the controller will use jackson or some other library to convert the entity object into a json and will try to read any field. At this point trying to read any lazy loaded collections from the entity, will result in this exception since the read will be outside of a transaction.
I am kind of frustrated right now as I thought this will be much easier and the issue would be documented way better but I just can not find a solution. Therefore I seek help here.
I am working on a Kotlin project which leverages spring boot version 2.5.3 and uses spring data jpa for database access and schema definition. Pretty common and straight-forward. Now assume we have some kind of UserService which contains a method updateUsername which gets a username as a parameter und updates the username after it verified its validity by an external service. For demo purposes of the issue I want to highlight, before we verify the username we set the username manually to "foo". This whole unit of work should happend in a transaction which is why the method is annotated with #Transactional. But because of the call to the external service the method will suspend when we wait for the http response (note the suspend keyword on both methods).
#Service
class UserService(private val userRepository: UserRepository) {
#Transactional
suspend fun setUsername(id: UUID, username: String): Person {
logger.info { "Updating username..." }
val user = userRepository.findByIdentityId(identityId = id)
?: throw IllegalArgumentException("User does not exist!")
// we update the username here but the change will be overridden after the verification to the actual username!
user.userName = "foo"
verifyUsername(username)
user.userName = username
return userRepository.save(user)
}
private suspend fun verifyUsername(username: String) {
// assume we are doing some kind of network call here which will suspend while waiting got a response
logger.info { "Verifying..." }
delay(1000)
logger.info { "Finished verifying!" }
}
}
This compiles successfully and I can also execute the method and it starts a new transaction, but the transaction will be commited as soon as we suspend the invocation of the verifyUsername method on calling delay(1000). Therefore our database will acually hold the value "foo" as the username until it is overwritten. But if the code after verifyUsername would fail and throw an exception, we could not rollback this change as the transaction was already commited and foo would stay in the database forever!!! This is definitely not the expected behavior as we only want to commit the transaction at the very end of our method, so we can rollback the transaction at any time if something went wrong. Here you can see the logs:
DEBUG o.s.orm.jpa.JpaTransactionManager - Creating new transaction with name [x.x.x.UserService.setUsername]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
DEBUG o.s.orm.jpa.JpaTransactionManager - Opened new EntityManager [SessionImpl(1406394125<open>)] for JPA transaction
DEBUG o.s.orm.jpa.JpaTransactionManager - Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#9ec4d42]
INFO x.x.x.UserService - Updating username...
DEBUG org.hibernate.SQL - select person0_.id as id1_6_, person0_.email as email2_6_, person0_.family_name as family_n3_6_, person0_.given_name as given_na4_6_, person0_.identity_id as identity5_6_, person0_.user_name as user_nam6_6_ from person person0_ where person0_.identity_id=?
INFO x.x.x.UserService - Verifying...
DEBUG o.s.orm.jpa.JpaTransactionManager - Initiating transaction commit
DEBUG o.s.orm.jpa.JpaTransactionManager - Committing JPA transaction on EntityManager [SessionImpl(1406394125<open>)]
DEBUG org.hibernate.SQL - update person set email=?, family_name=?, given_name=?, identity_id=?, user_name=? where id=?
DEBUG o.s.orm.jpa.JpaTransactionManager - Closing JPA EntityManager [SessionImpl(1406394125<open>)] after transaction
INFO x.x.x.UserService - Finished verifying
DEBUG o.s.orm.jpa.JpaTransactionManager - Creating new transaction with name [org.springframework.data.jpa.repository.support.SimpleJpaRepository.save]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT
DEBUG o.s.orm.jpa.JpaTransactionManager - Opened new EntityManager [SessionImpl(319912425<open>)] for JPA transaction
DEBUG o.s.orm.jpa.JpaTransactionManager - Exposing JPA transaction as JDBC [org.springframework.orm.jpa.vendor.HibernateJpaDialect$HibernateConnectionHandle#1dd261da]
DEBUG org.hibernate.SQL - select person0_.id as id1_6_0_, person0_.email as email2_6_0_, person0_.family_name as family_n3_6_0_, person0_.given_name as given_na4_6_0_, person0_.identity_id as identity5_6_0_, person0_.user_name as user_nam6_6_0_ from person person0_ where person0_.id=?
DEBUG o.s.orm.jpa.JpaTransactionManager - Initiating transaction commit
DEBUG o.s.orm.jpa.JpaTransactionManager - Committing JPA transaction on EntityManager [SessionImpl(319912425<open>)]
DEBUG org.hibernate.SQL - update person set email=?, family_name=?, given_name=?, identity_id=?, user_name=? where id=?
DEBUG o.s.orm.jpa.JpaTransactionManager - Closing JPA EntityManager [SessionImpl(319912425<open>)] after transaction
In this spring article it says that "Transactions on Coroutines are supported via the programmatic variant of the Reactive transaction management provided as of Spring Framework 5.2. For suspending functions, a TransactionalOperator.executeAndAwait extension is provided."
Does that mean that #Transactional just cannot be used on suspending methods and you should programatically handle transaction management? Update: This thread states that #Transactional should work for suspend functions.
I am also aware, that all my database operations are running synchronously, and I could use r2dbc to make them asynchronously (as long as my database provides the driver implementing the specification), but I think that my question does not relate to how I communicate with the database but more about how spring handles suspending calls withing #Transactional annotated methods.
What are your thoughts and recommendations here? I am definitely not the first dev who does suspending work in a transactional method in Kotlin, and still I am not able to find helpful resources on this issue.
Thanks guys!
JPA isn't supported with Coroutine Transactions as JPA is fully synchronous. Coroutine Transaction propagation works only with technologies that provide a reactive integration such as MongoDB, R2DBC, or Neo4j.
JPA assumes an imperative programming model hence its transaction manager stores transactional state in ThreadLocal storage. Reactive integrations and specifically Coroutines employ Coroutines context/Reactor's subscription context to keep track of the transaction status. There's no link between ThreadLocal and the context feature of Coroutines/Project Reactor.
Going forward, using blocking integrations such as JPA require special attention in the context of coroutines and their transactional scope needs to be constrained onto a single thread.
I am validating and uploading data in my webapp. In this process, I need to get user input, so the validating and uploading is split between 2 requests. In the 1st request I setup/fetch all the information and then save it in the session. This includes a Entity object that I later access in the 2nd request.
My question is how do I reattach the entity in the 2nd request so I don't get a LazyInitializationException? I have tried the following:
// we need to reattach the entities to the jpa session to avoid lazy-initialization exceptions
EntityManager manager = jpaContext.getEntityManagerByManagedType(myEntity.class);
manager.merge(myEntity);
I am getting the error: javax.persistence.TransactionRequiredException: No EntityManager with actual transaction available for current thread - cannot reliably process 'merge' call, even though I have the annotated the method with Spring's #Transactional.
These request method are Jersey REST service calls if that makes a difference. The jpa code lives in a commons jar file and this service is in the project that uses the commons jar.
We have JTA transactions(Atomikos) configured using Spring annotations across different places in our application. I need to get trace logs whenever a transaction started and completed.
For example, whenever the below method invoked within a new transaction,
#Transactional
void createAgent() { ... }
I need to log a message saying
Transaction started on AgentFactory::createAgent() ...
Transaction ended on AgentFactory::createAgent() ...
Can you please provide if there is any way to enable trace logging on transactions?
If you set loglevel to DEBUG or TRACE for org.springframework.transaction
you get the log entries you want. May be not in the exact format, but the information is provided.
I am getting the following error below
Unexpected error
org.springframework.transaction.CannotCreateTransactionException:
**CannotCreateTransactionException:** Could not open Hibernate Session for transaction; nested exception is org.hibernate.exception.GenericJDBC
Exception: Could not open connection
Here is the scenario in which it occurs :
I have recently moved some table from mysql to mongo. The code is written in such a way that either data would be taken from mongo/mysql.
The code is written in a method block which is annotated with #Transactional provided by spring framework.
There is hibernate layer which is using transaction provided by spring. c3p0 is the connection pool.
The parameter of connection pool is
hibernate.c3p0.min_size=5
hibernate.c3p0.timeout=1200
hibernate.c3p0.max_size=35
hibernate.c3p0.max_statements=50
The problem comes when we try to pull the data from mongo.Looks like the transaction is not getting closed because of mongo operation.The database connection is not getting released .It reaches the max size defined in the pool.
Tried the query in DB to find out the connection
show status like '%onn%';
Any suggestion to resolve this would really help.
Thanks