I'm working with SpringBoot 2.3.5 with Hibernate and Hikari Pool.
I've an entity, lets call it A, this entity has an "execution status counter" long field, incremented by an Async method, at the end of its executions, so I can split the execution on multiple threads, having a progress counter.
The increment is performed like this:
#Transactional(propagation = Propagation.REQUIRES_NEW)
#Lock(LockModeType.PESSIMISTIC_WRITE)
default void incrementStatusCount(String lotto, Long progressivo) {
A a = findById(...).get();
a.setStatoLdSN(a.getStatoLdSN() != null ? a.getStatoLdSN()+1:1L);
saveAndFlush(a);
}
And it works fine, so externally, using a DB Tool, I can see the counter update.
Now, at the end of my whole execution, I change the string status of my whole execution, so I load the entity, but the entity is already in the cache, so I call a refresh on the entity manager, I can see the query performad and also the retrieved values from the Hibernate log....and my counter 0.
Now, Oracle default isolation leve is READ_COMMITTED (and I verified it on the connection), and my incremented value is committed because I saw it using the DB client, no?
So why JPA, not even calling refresh is loading the right value?
Related
I'm using SpringBatch for my app. In one of the batch jobs, I need to process multiple data. Each data requires several database updates. And I need to make one transaction for one data. Meaning, if when processing one data an exception is thrown, database updates are rolled back for that data, then keep processing the next data.
I've put all database updates in one method in service layer. In my springbatch tasklet, I call that method for each data, like this;
for (RequestViewForBatch request : requestList) {
orderService.processEachRequest(request);
}
In the service class the method is like this;
Transactional(propagation = Propagation.NESTED, timeout = 100, rollbackFor = Exception.class)
public void processEachRequest(RequestViewForBatch request) {
//update database
}
When executing the task, it gives me this error message
org.springframework.transaction.NestedTransactionNotSupportedException: Transaction manager does not allow nested transactions by default - specify 'nestedTransactionAllowed' property with value 'true'
but i don't know how to solve this error.
Any suggestion would be appreciated. Thanks in advance.
The tasklet step will be executed in a transaction driven by Spring Batch. You need to remove the #Transactional on your processEachRequest method.
You would need a fault-tolerant chunk-oriented step configured with a skip policy. In this case, only faulty items will be skipped. Please refer to the Configuring Skip Logic section of the documentation. You can find an example here.
I have a very specific scenario that, during the execution of a query, specifically during the fetching rows from db to my resultset, I get an OutOfMemoryError.
The code is simple as it:
public interface MyRepository extends Repository<MyEntity, Long> {
#EntityGraph(value = "MyBigEntityGraphToFetchAllCollections", type = EntityGraphType.FETCH)
#QueryHints({#QueryHint(name = "org.hibernate.readOnly", value = "true")})
MyEntity getOneById(Long id);
}
public class MyService {
...
public void someMethodCalledInLoop(Long id) {
try{
return repository.getOneById(id);
} catch (OutOfMemoryError error) {
// Here the connection is closed. How to reset Hikaricp?
System.gc();
return null;
}
}
}
Seems weird a getOne consumes all the memory, but due to eager fetching about 80 collections and due to multiplication of rows, some cases are insupportable.
I know I have the option to lazely load the collections, but I don't want to. Hit database 1+N times on every load consumes more time and my application dont have it. Its a batch processing of milions of records and less than 0,001% has this impact in memory. So my strategy is just discard this few records and process the next ones.
Just after catch the OutOfMemoryError the memory is freed, the trouble entity turns garbage. But due to this Error, HikariCP closes (or is forced to) the connection.
In the next call of the method, hikaricp still gives me a closed connection. Seems due to memory lack hikaricp doesn't finished correctly the previous transaction and sticks in this state forever.
My intention, now, is to reset or recovery hikaricp. I don't need to care about other threads using the pool.
So, after all, my simple question is, how to programatically restart or recover hikarycp to its primary state, without reboot the application.
Thanks, a lot, for who read this.
Try adding this to your Hibernate configuration:
<property name="hibernate.hikari.connectionTestQuery">select 1</property>
This way HikariCP will test that the connection is still alive before giving it to Hibernate.
Nothing has worked so far.
I minimized the problem by adding a 'query hint' to the method:
#QueryHints({#QueryHint(name = "org.hibernate.timeout", value = "10")})
MyEntity getOneById(Long id);
99% of the resultsets are fetched in 1 or less second, but sometimes the resultset is so big that takes longer. This way the JDBC stops the result fetching before the memory gets compromised.
Is it possible to use a jpaRepository to perform CRUD operations within a scheduled Task (annotated with #Scheduled) without wrapping the scheduled method with #Transactional?
Use Case:
We have a scheduler to pop jobs and each job has certain operations to perform and fetches + saves data accordingly.
private void processJob(T job) throws Exception {
updateJobToTaken(job);
internalProcess(job); //calls a service to run the job operations
updateJobToDone(job);
}
1- If we don't wrap the method with #Transactional, all operations done within the internalProcess() are not saved to the DB. So we added #Transactional(propagation.REQUIRES_NEW)
2- This caused a regressions on our dryRun flag: We added a dryRun flag that is passed to the internalProcess method. If the flag is true, we skip calling repository.save(entities) and expected that none of the work was saved. So we simply logged the changes to the console.
if (!dryRun) {
setupsGroupRepository.save(setupsGroup);
}
But since the method is transactional, Spring committed the changes and everything was committed.
What we would like?
Controlling DB access explicitly. Meaning if I don't call repository.save(), i'm not expecting anything to be committed. And vice versa, when i call repository.save(), i'm expecting my updates to be committed fully to the DB.
When we call the internalProcess method directly without having a scheduledTask, everything runs perfectly. but now, it feels that spring transaction management gets in the way at every turn.
My problem is straightforward. I want to access some data from the database when the application loads on Tomcat. To do something at that point in time I use #PostConstruct (which does its job properly).
However, in that method I make 2 separate connections to the DB: one for bringing a list of entities and another for adding them into a common library. The second step implies some behind-the-scenes queries for resolving some lazy-loading associations. Here is the code snippet:
#Override
#PostConstruct
public void populateLibrary() {
// query for the Book Descriptors - 1st query works!!!
List<BookDescriptor> bookDescriptors= bookDescriptorService.list();
Session session = sessionFactory.openSession();
Transaction transaction = null;
try {
transaction = session.beginTransaction();
// resolving some lazy-loading associations - 2nd query fails!!!
for (BookDescriptor book: bookDescriptors) {
library.addEntry(book);
}
transaction.commit();
} catch (HibernateException e) {
transaction.rollback();
e.printStackTrace();
} finally {
session.close();
}
}
1st query works while the 2nd fails, as I wrote in the comments. The failure gives:
org.hibernate.LazyInitializationException: could not initialize proxy - no Session
at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:86)
at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:140)
at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:190)
at com.freightgate.domain.SecurityFiling_$$_javassist_7.getSfSubmissionType(SecurityFiling_$$_javassist_7.java)
at com.freightgate.dao.SecurityFilingTest.test(SecurityFilingTest.java:73)
Which is very odd since I explicitly opened and closed a transaction. However, if I inspect some details of how the 1st query works it seems like behind the scenes the session is bound to AbstractLazyInitializer class.
I resolved my problem by abstracting away the functionality from the for loop into a separate service class that is annotated with #Transactional(readOnly = true). Still I'm puzzled as to why the approch that I posted here fails.
If anyone has some hints, I'd be very happy to hear them.
You load entities in a first session, then close this session, then open a new session, and try to lazy-load collections of the entities. That can't work.
For lazy-loading to work, the entity must be attached to an open session. Just opening another session doesn't make any entity you have loaded before attached to this new session. In the meantime, some other transaction could have radically changed the database, the entity could not exist anymore...
The best solution is what you have done. Encapsulate evrything into a single transactional service. You could also have open the transaction before calling the first service, but why handle transactions programmatically, since Spring does it for you declaratively?
I'm using the TransactionScope class within a project based on Silverlight and RIA services. Each time I need to save some data, I create a TransactionScope object, save my data using Oracle ODP, then call the Complete method on my TransactionScope object and dispose the object itself:
public override bool Submit(ChangeSet changeSet)
{
TransactionOptions txopt = new TransactionOptions();
txopt.IsolationLevel = IsolationLevel.ReadCommitted;
using (TransactionScope tx = new TransactionScope(TransactionScopeOption.Required, txopt))
{
// Here I open an Oracle connection and fetch some data
GetSomeData();
// This is where I persist my data
result = base.Submit(changeSet);
tx.Complete();
}
return result;
}
My problem is, the first time I get the Submit method to be called, everything is fine, but if I call it a second time, the execution gets stuck for a couple of minutes after the call to Complete (so, when disposing tx), then I get the Oracle error "ORA-12154". Of course, I already checked that my persistence code completes without errors. Any ideas?
Edit: today I repeated the test and for some reason I'm getting a different error instead of the Oracle exception:
System.InvalidOperationException: Operation is not valid due to the current state of the object.
at System.Transactions.TransactionState.ChangeStatePromotedAborted(InternalTransaction tx)
at System.Transactions.InternalTransaction.DistributedTransactionOutcome(InternalTransaction tx, TransactionStatus status)
at System.Transactions.Oletx.RealOletxTransaction.FireOutcome(TransactionStatus statusArg)
at System.Transactions.Oletx.OutcomeEnlistment.InvokeOutcomeFunction(TransactionStatus status)
at System.Transactions.Oletx.OletxTransactionManager.ShimNotificationCallback(Object state, Boolean timeout)
at System.Threading._ThreadPoolWaitOrTimerCallback.PerformWaitOrTimerCallback(Object state, Boolean timedOut)
I somehow managed to solve this problem, although I still can't figure out the reason it showed up in the first place: I just moved the call to GetSomeData outside the scope of the distributed transaction. Since the call to Submit may open many connections and perform any kind of operations on the DB, I just can't tell why GetSomeData was causing this problem (it just opens a connection, calls a very simple stored function and returns a boolean). I can only guess that has something to do with the implementation of the Submit method and/or with the instantiation of multiple oracle connections within the same transaction scope.