I have a list of ~10 000 objects.
I am trying to call an mysql update query (procedure) and then to get the updated objects inside same transaction.
Can this be achieved ?
When I call a delete statement + flush(), hibernate retrieves me correct objects (deleted objects are missing)
But when I try update statement + flush(), hibernate retrieves me the initial unchanged objects.
#Transactional
void test() {
//...
em.createQuery("delete from StorefrontProduct sp where sp in (:storefrontProducts)")
.setParameter("storefrontProducts", storefrontProductsToDelete)
.executeUpdate();
// example
em.createQuery("update StorefrontProduct sp set sp.orderIndex=0 where sp.id=90")
.executeUpdate();
em.flush();
//Simple JPA query
List<StorefrontProduct> result = repository.findAllByPreviousOrderIndexIsNotNull();
//additional code....
}
After running the code from above and putting a breakpoint after findAll call, provided objects from 1-st query were deleted and flushed, but the update query was not flushed.
That is known counterintuitive behaviour of Hibernate.
First of all, em.flush() call might be superfluous if flush mode set to AUTO (in that case Hibernate automatically synchronises persistence context (session-level cache) with underlying database prior executing update/delete queries).
Delete and successive Select case:
you issues delete then select, since select does not see deleted records anymore you do not see deleted records in resultset, however if you call findById you may find deleted records.
Update and successive Select case:
you issues update then select, when processing resultset Hibernate sees both records stored in database and records stored in persistence context and it assumes that persistence context is a source of truth, that is the reason why you see "stale" data.
There are following options to mitigate that counterintuitive behaviour:
do not perform direct updates, use "slow" find/save API instead
either detach or refresh stale entities after direct update, em.clear() may also help, however it completely cleans up persistence context, which might be undesirable
For example, instead of
#Modifying(flushAutomatically = true)
Only
#Modifying
I do understand that the main point of it - to flush all cache to DB before querying,
But after we write #Query isn't JPQL query already processed by JPA? And JPA has default Flushmode = AUTO, so it should flush data to db anyway.
Am I thinking right? If yes, can I skip this annotation?
P.S. I haven't found anything on stackoverflow that is answering that question, so it is not a duplicate.
As it's discussed in this thread:
#Modifying annotation has clearAutomatically attribute which defines whether it should clear the underlying persistence context after executing the modifying query.
...
When executing modifying queries with this attribute activated, it drops all non-flushed changes still pending in the EntityManager.
...
Is it possible to add a new attribute to #Modifying annotation called flushAutomatically to automatically flush all non-persisted changes to the underlying context before executing the modifying query?
and further
JPA’s AUTO FlushModeType and Hibernate: if the current executed query is not going to hit the pending SQL INSERT/UPDATE/DELETE statements then the flush is not strictly required.
As stated in the reference documentation, the AUTO flush strategy may sometimes synchronize the current persistence context prior to a query execution.
The clearAutomatically property drops all the pending changes in the EntityManager that are not related to the current update query (cause they are not automatically flushed).
That's why I'm asking for a new property to force the EntityManager to flush changes.
So, this property have sense only when you use also clearAutomatically property: #Modifying(clearAutomatically = true, flushAutomatically = true).
i am trying to use manual locking inside a spring #Transactional method, using
EntityManager.find(class, id, LockModeType.PESSIMISTIC_WRITE);
However, when viewing the sql, hibernate does NOT generate any select ... for update statements (acquire an exclusive lock on the DB row) as i would expect, and the method does not synchronize correctly.
Strangely, calling em.lock() after fetching the entity using em.find() causes a select ... for update as I would expect for find(), as does adding an #Lock(PESSIMISTIC_WRITE) annotation to a fetching query.
However I would prefer using em.find() over the other two methods, but i cannot get that to work.
So, why is EntityManager.find silently ignoring lock options, while other locking methods work?
I am using Hibernate 5.4.4.FINAL, tried upgrading to 5.4.11.FINAL.
I also tried setting Isolation levels to SERIALIZABLE and READ_COMMITTED in the #Transactional.
Neither of these altered my results.
Hibernate, Hibernate Envers and QueryDSL are configured and working correctly in a Spring boot 1.4.1.RELEASE.
The problem is when using UpdateClause<JPAUpdateClause> updateQueryBuilder = queryFactory.update(collectionTransaction); to build update query and execute that update query, Hibernate Envers does not pick up and audit those changes.
Following is the Spring Data JPA repository that implements QueryDSL
public class CollectionTransactionRepositoryImpl extends QueryDslRepositorySupport implements CollectionTransactionRepositoryCustom {
#Autowired
private JPAQueryFactory queryFactory;
public CollectionTransactionRepositoryImpl() {
super(CollectionTransaction.class);
}
#Override
public Collection<CollectionTransaction> updateCollectionTransaction(UpdateCollectionTransaction updateCollectionTransaction) {
QCollectionTransaction collectionTransaction = QCollectionTransaction.collectionTransaction;
UpdateClause<JPAUpdateClause> updateQueryBuilder = queryFactory.update(collectionTransaction);
.....//Code omitted for brevity
long updated = updateQueryBuilder.execute();
//.....
return ...
}
}
Is it possible for Hibernate Envers to pick up changes in this situation ?
This is a known concern outlined in JIRA HHH-10318.
Envers works based on Hibernate's event subsystem where Hibernate effectively notifies various callbacks that state for an entity has been modified in some way, and provides both the prior and new entity state. This state is precisely what Envers uses to determine what changed and insert audit change rows.
Lets take a trivial example:
UPDATE MyEntity e SET e.status = :status
Hibernate will perform the following tasks:
Flush the persistence context any any modifications.
Invalidate any cached instances of MyEntity.
Execute the bulk update operation.
No where in any of these steps does Hibernate load any existing state. It simply guarantees that current changes are flushed prior to the bulk update and that any subsequent operations will fetch from the datastore rather than a cache due to the bulk update.
Therefore from Envers perspective, it gets no callbacks and thus isn't aware that any operation took place because Hibernate ORM cannot provide any entity state for such an operation, it simply does not exist.
The big question here is how (if possible) to model and handle a change unit for such an operation.
It's difficult because Envers would effectively need some type of PreBulkOpEvent so that it can cache what it needs that is about to change and a PostBulkOpEvent to require and merge the two results to generate change log entries. The concern with such a concept really centers around how to do this effectively to avoid
Running out of memory due to large result-set manipulation.
Long execution time to load state from datastore for large result-set manipulation.
Anyway, you're welcomed to read over the JIRA and provide any feedback or ideas. But presently, its just something that falls outside the scope of what we can capture at the present moment.
I have the following code:
println "######## RUNNING ProfessionaCustomer - ${pcCounter} under ${accountCustomer.customerNumber} Professional SQLid ${it.id}"
def professionalCustomerId = it.customerId
def professionalCustomer = ProfessionalCustomer.findByCustomerNumber(professionalCustomerId)
I have SQL logging on and I get:
######## RUNNING ProfessionaCustomer - 31 under 106450 Professional SQLid 100759
Hibernate: update base_domain set version=?, account_name=?, address_line1=?, address_line2=?, city=?, customer_number=?, date_created=?, disabled=?, last_updated=?, postal_code=?, primary_phone=?, state_or_province=? where id=? and version=?
Hibernate: update base_domain set version=?, address1=?, address2=?, city=?, customer_number=?, date_created=?, disabled=?, first_name=?, last_name=?, last_updated=?, middle_name=?, phone_number=?, postal_code=?, state=? where id=? and version=?
Hibernate: insert into account_customer_professionals (account_customer_id, professional_customer_id) values (?, ?)
Hibernate: select this_.id as id1_3_0_, this_.version as version2_3_0_, this_.address1 as address70_3_0_, this_.address2 as address71_3_0_, this_.city as city7_3_0_, this_.customer_number as customer8_3_0_, this_.date_created as date_cre9_3_0_, this_.disabled as disable10_3_0_, this_.first_name as first_n19_3_0_, this_.last_name as last_na20_3_0_, this_.last_updated as last_up11_3_0_, this_.middle_name as middle_72_3_0_, this_.phone_number as phone_n73_3_0_, this_.postal_code as postal_12_3_0_, this_.state as state74_3_0_ from base_domain this_ where this_.class='com.eveo.nplate.model.ProfessionalCustomer' and this_.customer_number=? limit ?
Which is updating the DB. This would explain why this is so slow, but I can't see any reason for this to happen.
Why would 'findBy' cause an update?
Hibernate doesn't immediately execute creates, updates, or deletes until it thinks it has to - it waits as long as possible (although it's rather pessimistic) and only flushes these changes when you tell it to, or when it thinks it needs to. In general the only time it will flush without an explicit call is when running queries. This is because any of the new instances, updated instances, and deleted instances that are in-memory (cached in the Hibernate Session, the 1st-level cache) could affect the query results, so they must be flushed to the database so you get the proper results for your query.
One exception to this is calling save() on a new instance. Grails flushes this because typically the id is assigned by the database, either via an auto-increment column or a sequence. To ensure that the in-memory state is the same as the database, it flushes the save() call so it can retrieve the id and set it in the instance. But if you retrieve a persistence instance (e.g. with a get() call, or with a criteria query, finder, etc.) and modify it, calling save() on that does not get automatically flushed. The same goes for delete() calls - not flushed.
Think of delete() and save() calls on persistent instances as messages to Hibernate that the action should be performed "eventually".
So when you execute a finder, or a criteria, "where", or HQL query, Hibernate will flush any un-flushed changes for you. If you don't want that to happen (e.g. in a custom domain class validator closure) you can run the query in a separate session, e.g. with the withNewSession method.
If you don't flush the session at all, either explicitly on the Session instance or by adding flush:true to a save or delete call, the session will be flushed, since Grails registers an OpenSessionInView interceptor that starts a session at the beginning of each request, and flushes and closes it at the end. This helps with lazy loading; since there's a session open and bound to a ThreadLocal in a known location, Hibernate and GORM (via Spring's HibernateTemplate) can use that open session to retrieve lazy-loaded collections and instances on-demand after the query runs.
Note also that you do not need to flush in a transaction. The transaction manager is a Spring HibernateTransactionManager that flushes before committing.
Probably there was some transaction in the session that was not persisted in the database.
When you ran the findBy hibernate took advantage of the connection to run the two queries. I believe this is what happened.