Spring Boot Data JPA - Modifying update query - Refresh persistence context - spring

I'm working with Spring Boot 1.3.0.M4 and a MySQL database.
I have a problem when using modifying queries, the EntityManager contains outdated entities after the query has executed.
Original JPA Repository:
public interface EmailRepository extends JpaRepository<Email, Long> {
#Transactional
#Modifying
#Query("update Email e set e.active = false where e.active = true and e.expire <= NOW()")
Integer deactivateByExpired();
}
Suppose we have Email [id=1, active=true, expire=2015/01/01] in DB.
After executing:
emailRepository.save(email);
emailRepository.deactivateByExpired();
System.out.println(emailRepository.findOne(1L).isActive()); // prints true!! it should print false
First approach to solve the problem: add clearAutomatically = true
public interface EmailRepository extends JpaRepository<Email, Long> {
#Transactional
#Modifying(clearAutomatically = true)
#Query("update Email e set e.active = false where e.active = true and e.expire <= NOW()")
Integer deactivateByExpired();
}
This approach clears the persistence context not to have outdated values, but it drops all non-flushed changes still pending in the EntityManager. As I use only save() methods and not saveAndFlush() some changes are lost for other entities :(
Second approach to solve the problem: custom implementation for repository
public interface EmailRepository extends JpaRepository<Email, Long>, EmailRepositoryCustom {
}
public interface EmailRepositoryCustom {
Integer deactivateByExpired();
}
public class EmailRepositoryImpl implements EmailRepositoryCustom {
#PersistenceContext
private EntityManager entityManager;
#Transactional
#Override
public Integer deactivateByExpired() {
String hsql = "update Email e set e.active = false where e.active = true and e.expire <= NOW()";
Query query = entityManager.createQuery(hsql);
entityManager.flush();
Integer result = query.executeUpdate();
entityManager.clear();
return result;
}
}
This approach works similar to #Modifying(clearAutomatically = true) but it first forces the EntityManager to flush all changes to DB before executing the update and then it clears the persistence context. This way there won't be outdated entities and all changes will be saved in DB.
I would like to know if there's a better way to execute update statements in JPA without having the issue of the outdated entities and without the manual flush to DB. Perhaps disabling the 2nd level cache? How can I do it in Spring Boot?
Update 2018
Spring Data JPA approved my PR, there's a flushAutomatically option in #Modifying() now.
#Modifying(flushAutomatically = true, clearAutomatically = true)

I know this is not a direct answer to your question, since you already have built a fix and started a pull request on Github. Thank you for that!
But I would like to explain the JPA way you can go. So you would like to change all entities which match a specific criteria and update a value on each. The normal approach is just to load all needed entities:
#Query("SELECT * FROM Email e where e.active = true and e.expire <= NOW()")
List<Email> findExpired();
Then iterate over them and update the values:
for (Email email : findExpired()) {
email.setActive(false);
}
Now hibernate knows all changes and will write them to the database if the transaction is done or you call EntityManager.flush() manually. I know this won't work well if you have a big amount of data entries, since you load all entities into memory. But this is the best way, to keep the hibernate entity cache, 2nd level caches and the database in sync.
Does this answer say "the `#Modifying´ annotation is useless"? No! If you ensure the modified entities are not in your local cache e.g. write-only application, this approach is just the way to go.
And just for the record: you don't need #Transactional on your repository methods.
Just for the record v2: the active column looks as it has a direct dependency to expire. So why not delete active completely and look just on expire in every query?

As klaus-groenbaek said, you can inject EntityManager and use its refresh method :
#Inject
EntityManager entityManager;
...
emailRepository.save(email);
emailRepository.deactivateByExpired();
Email email2 = emailRepository.findOne(1L);
entityManager.refresh(email2);
System.out.println(email2.isActive()); // prints false

Related

Spring JPA: How to check if Entity belongs to current JPA transaction?

I need the ability to check if Entity belongs to the current ongoing transaction. I have a JPA entity cache as a thread-local cache. Almost always when there is single #Transaction there is no issue when a same thread calls multiple times to save on service and NOT nested #Trasaction(Propagation.REQUIRES_NEW) (meaning when you have nested transactions) on same thread it does not work. Is there any way to check if JPA Entity ( MyExpesiveEntity) belongs to the current ongoing transaction?
ThreadLocal<Map<Long,MyExpesiveEntity>> cache = new ThreadLocal<Map<Long,MyExpesiveEntity>>()
#Entity('MyExpesiveEntity')
class MyExpesiveEntity{
}
#Transaction
save(DTO save){
MyExpesiveEntity entity = cache.get(dto.getID());
if(entity == null){
entity = myExpesiveRespository.findById(dto.getID());
cache.get().put(entity.getID(),entity);
}
entity.setXXX(dto.getXXX())
// THIS always works but Sometime when the caller is using
// Propagation.REQUIRES_NEW.
// It does not work when you have nested transactions.It throws duplicated id.
myExpesiveRespository.saveAndFlush(entity);
}
It's Hibernate specific, but yes, this is possible
public boolean containsEntity(EntityManager em, Class<?> entityClass, Object id) {
SessionImplementor session = em.unwrap(SessionImplementor.class);
EntityKey entityKey = session.generateEntityKey((Serializable) id, session.getFactory().getEntityPersister(entityClass.getName()));
PersistenceContext pc = session.getPersistenceContext();
return pc.getEntity(entityKey) != null || pc.getProxy(entityKey) != null;
}
Since the entity is always existing you could simply update based on the ID. See Modifying
#Modifying
#Query("update MyExpesiveEntity expEnt set expEnt.xxxx = ?1 where expEnt.id = ?2")
void setXxxById(String xxxx, Integer id);

Hibernate - Table Locked after update

I'm performing an update via a method using Hibernate and the EntityManager.
This update method is called multiple times (within a loop).
It seems like when I execute it the first time, it locks the table and does not free it.
When trying to update the table via SQL Developer after having closed the application, I see the table is still locked because the update is hanging.
What do you see as a solution to this problem? If you need more information, let me know.
Class
#Repository
#Transactional(propagation = REQUIRES_NEW)
public class YirInfoRepository {
#Autowired
EntityManager entityManager;
#Transactional(propagation = REQUIRES_NEW)
public void setSent(String id) {
String query = "UPDATE_QUERY";
Query nativeQuery = entityManager.createNativeQuery(String.format(query, id));
nativeQuery.executeUpdate();
}
}
UPDATE
After having waited more than one hour, I launched the application again and it worked fine once but now again, it hangs.
UPDATE 2 -- I'll give a maximum bounty to whoever helps me solve this
On another place I use an application managed entity manager and it still gives me the same type of errors.
public void fillYirInfo() {
File inputFile = new File("path");
try (InputStream inputStream = new FileInputStream(inputFile);
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream))) {
bufferedReader.lines().skip(1).limit(20).forEach(line -> {
String[] data = line.split(",");
String rnr = data[0];
String linked = data[1];
String email = data.length > 2 ? data[2] : "";
String insuredId = insuredPeopleRepository.getInsuredIdFromNationalId(rnr);
int modifiedCounter = 0;
if (!isNullOrEmpty(insuredId)) {
EntityManager entityManager = emf.createEntityManager();
EntityTransaction transaction = entityManager.getTransaction();
Query nativeQuery = entityManager.createNativeQuery(
"QUERY"
);
transaction.begin();
nativeQuery.executeUpdate();
entityManager.flush();
transaction.commit();
entityManager.close();
}
System.out.println(modifiedCounter + " rows modified");
});
} catch (IOException e) {
e.printStackTrace();
}
}
Try without an update-query:
#Repository
#Transactional(propagation = REQUIRES_NEW)
public class YirInfoRepository {
#Autowired
EntityManager entityManager;
#Transactional(propagation = REQUIRES_NEW)
public void setSent(String id) {
//guessing your class name and method..
final YirInfo yirInfo = entityManager.find(YirInfo.class, id);
yirInfo.setSent();
}
}
Might not be as fast as a single update query, but it's possible to get it reasonably fast, unless the amount of data is huge. This is the preferred way of using Hibernate/JPA, instead of thinking in terms of single values and SQL queries, you work with entities/objects and (sometimes) HQL/JPQL queries.
You are using #Transactional annotation. This means you are using Spring Transaction. Then in your UPDATE 2 you are using transaction by yourself and managed by spring (I guess it's another project or class not managed by Spring).
In any case what I would do is to try to update your records in single spring transaction and I'd not use #Transactional in DAO layer but in service layer. Something like this:
Service layer:
#Service
public class YirInfoService {
#Autowired
YirInfoRepository dao;
#Transactional(propagation = REQUIRES_NEW)
public void setSent(List < String > ids) {
dao.setSents(ids);
}
}
DAO layer:
#Repository
public class YirInfoRepository {
#Autowired
EntityManager entityManager;
//Here you can update by using and IN statement or by doing a cycle
//Let's suppose a bulk operation
public void setSents(List < String > ids) {
String query = "UPDATE_QUERY";
for (int i = 0; i < ids.size(); i++) {
String id = ids.get(i);
Query nativeQuery = entityManager.createNativeQuery(String.format(query, id));
nativeQuery.executeUpdate();
if (i % 20 == 0) {
entityManager.flush();
entityManager.clear();
}
}
}
}
The first thing you have to understand is that for the first example, you are using a native query to update rows in the DB. In this case you are completely skipping Hibernate to do anything for you.
In your second example, you have the same thing, you are updating via an update query. You don't need to flush the entity manager as it's only necessary for transferring the pending changes made to your entity objects within that entity manager.
Plus I don't know how your example works as you are autowiring the entity manager and not using the #PersistenceContext annotation. Make sure you use this one properly because you might have misconfigured the application. Also there is no need to manually create the entity manager when using Spring as it looks in the second example. Just use #PersistenceContext to get an entity manager in your app.
You are also mixing up transaction management. In the first example, it's enough if you put the #Transactional annotation to either of your method or to the class.
For the other example, you are doing manual transaction management which makes no sense in this case. If you are using Spring, you can simply rely on declarative transaction management.
The first thing I'd check here is to integrate datasource-proxy into your connection management and log out how your statements are executed. With this info, you can make sure that the query is sent to the DB side and the DB is executing it very slowly, or you are having a network issue between your app and db.
If you find out that the query is sent properly to the DB, you want to analyze your query, because most probably it's just executed very slowly and needs some optimizations. For this, you can use the Explain plan feature, to find out how your execution plan looks like and then make it faster.

Hibernate query.list() is slower than HibernateTemplate.find()

We upgraded our project from Spring 3x/Hibernate 3x to Spring 4.1.5/Hibernate 4.3.8.
Initially we were using Hibernate Tempate. During upgrade we removed the Hibernate Template of spring and used Spring's declarative transaction management.
Earlier our query with hibernateTemplate used to take very sort time to retrive 3500 records from DB. Now when are using query.list(), the running time is coming in minutes (4-5 mins approx).
Old Code
DAO Class:
---------
public List<LatestRecVO> getListRecs(String listId) {
HibernateTemplate ht = new HibernateTemplate(getSessionFactory());
List<LatestRecVO> listOfRecs = ht.find(" from LatestRecVO a where a.listId = ? order by a.listRecId asc", Long.valueOf(listId) );
return listOfRecs;
}
Service Class:
--------------
#Transactional
public List<LatestRecVO> getListRecs(String listId) {
List<LatestRecVO> listOfRecs = listDao.getListRecDetails(listId);
return listOfRecs;
}
New Code
DAO Class:
---------
public List<LatestRecVO> getListRecs(String listId) {
Query q = getSession().createQuery(" from LatestRecVO a where a.listId = (:listId) order by a.listRecId asc");
q.setParameter("listId", Long.valueOf(listId));
List<LatestRecVO> listOfRecs = q.list();
return listOfRecs;
}
Service Class:
--------------
#Transactional
public List<LatestRecVO> getListRecs(String listId) {
List<LatestRecVO> listOfRecs = listDao.getListRecDetails(listId);
return listOfRecs;
}
The entity class LatestRecVO do not have any associated entity with it.
I checked the Hibernate Template's find() method and saw its uses some caching.
Tried 2nd level cache along with query cache but it didnt helped. I may have configured 2nd level cache incorrectly but to try it again i want to be sure that its the way out else i would be wasting time.
I made show_sql as true and can see it just ran a singly query. On DB the same query takes some milliseconds to run. It seems like hibernate is taking time to build objects from the result.
On one of the post it was mentioned that its mandate to have a default constructor in our entities. I have not created any constructor in my entity class so i assume that i do have java's default constructor in place.
My table has 36 columns and in total there are 4k records to be fetched.
Any pointer in this will be really helpful.
Update
Sorry, I cannot post the complete code here, so just giving the details. I have the composite primary key for LatestRecVO. I have created a class LatestRecPK for primary key, it implements serializable and have #Embeddable annotation. In LatestRecVO i have given #IdClass(LatestRecPK.class) to include the primary key class. LatestRecVO has a CLOB property along with String, Long and #Temporal(TemporalType.DATE) properties and corresponding setters/getters.

How do I migrate my JPA DAO to Spring Data with second level cache?

I have bunch of JPA DAOs im looking to migrate to Spring Data JPA. Some of my DAOS have second-level / query caching set up.
I have a process where I only retrieve the ID in my queries, and then look up the entity using findByID(). This way, only the id's are multiplied in the different query caches, and the entire entities are in the second level cache.
Example:
#NamedQuery(name = "SystemUser.findByEmail",
query = "SELECT u.id FROM SystemUser u WHERE email=:email"),
…
public SystemUser findByEmail(String email) {
TypedQuery<Long> q = getEntityManager().createNamedQuery("SystemUser.findByEmail", Long.class);
q.setParameter("email", email);
q.setHint("org.hibernate.cacheable", true);
q.setHint("org.hibernate.cacheRegion", "query.systemUser");
List<Long> res = q.getResultList();
if (res != null && res.size() > 0) {
return findById(res.get(0));
}
return null;
}
I have several more findBy…-methods, all doing it like this. It feels like a good way to keep cache memory consumption down.
I'm kind of new to the Spring Data JPA business, but I can't see how I would go about realizing this here? The #Cacheable annotations seems only to deal with query caches, which to me would duplicate the entities in each query cache?
Is there any way to do this with Spring Data? Pointers would be much appreciated.
In Spring Data JPA just create a findByEmail method and either Spring Data JPA will found your named query or create one itself.
public class SystemUserRepository extends CrudRepository<SystemUser, Long> {
SystemUser findByEmail(String email);
}
Should be all you need to get the query executed and the desired result. Now with the #QueryHints you can add the hints you are setting now.
public class SystemUserRepository extends CrudRepository<SystemUser, Long> {
#QueryHints(
#QueryHint(name="org.hibernate.cacheable", value="true"),
#QueryHint(name="org.hibernate.cacheRegion", value="query.systemUser") )
SystemUser findByEmail(String email);
}
The result will be cached and still the user will come from the 2nd level cache (if available, else created). Assuming of course your entity is #Cacheable.
A nice read on how the 2 different caches work (together) can be found here. A small snippet on how the query cache works.
The query cache looks conceptually like an hash map where the key is composed by the query text and the parameter values, and the value is a list of entity Id's that match the query:
If you want more complex logic (and really implement the optimization you did) you can always implement your own repository.

JPA/Hiberante don't generate join sql for FetchType.EAGER while Spring #Transactional annotated

I have a very wired problem.
JPA/Hiberante don't generate join sql for FetchType.EAGER while Spring #Transactional annotated. But if I remove the #Transactional . Everything is fine.
Here is the code:
public class Item {
#ManyToOne
private Order order;
}
public class Order {
#OneToMany(mappedBy = "item", fetch = FetchType.EAGER, cascade = CascadeType.ALL)
private List<Item> items;
}
#Test
#Transactional
public void testFetch() throws Exception {
Item randomItem = new Item();
Order randomOrder = new Order();
//OrderService and itemService is implemented by Spring Roo standard.
orderService.saveOrder(randomOrder);
randomItem.setOrder(randomOrder);
itemService.saveItem(randomItem);
Order OrderResult = orderService.findOrder(randomOrder.getId());
final List<Item> itemSearchResult = OrderResult.getItems();
Assert.assertNotNull(itemSearchResult);
}
The assertNotNull will fail if #Transactional on. But will success if #Transactional commented.
I debug more information. Just to find out when #Transactional on Hibernate will not generate join sql for
orderService.findOrder(randomOrder.getId());
Alos I try to switch to elicpseLink as JPA provider. Things become worse, when #Transactional commented, orderService.findOrder(randomOrder.getId()) will return a empty list(not null, size 0).
Any advice? Many Thanks!
I can't comment on the joins in Hibernate except that you should specify fetch join in your query to be portable to other JPA providers. EclipseLink in particular does not join eager relationships without JPA settings or native query hints or #JoinFetch annotations.
As for the collection being empty on EclipseLink. This is because you only setting one side of the relationship. JPA requires you to set both sides of bidirectional relationships so that they remain consistent with what is in the database. When #Transactional is commented out, you are getting back the same randomOrder instance that had an empty items collection when persisted.
Try calling randomOrder.addItems(randomItem); and randomItem.setOrder(randomOrder); before the orderService.saveOrder(randomOrder); call.

Resources