We upgraded our project from Spring 3x/Hibernate 3x to Spring 4.1.5/Hibernate 4.3.8.
Initially we were using Hibernate Tempate. During upgrade we removed the Hibernate Template of spring and used Spring's declarative transaction management.
Earlier our query with hibernateTemplate used to take very sort time to retrive 3500 records from DB. Now when are using query.list(), the running time is coming in minutes (4-5 mins approx).
Old Code
DAO Class:
---------
public List<LatestRecVO> getListRecs(String listId) {
HibernateTemplate ht = new HibernateTemplate(getSessionFactory());
List<LatestRecVO> listOfRecs = ht.find(" from LatestRecVO a where a.listId = ? order by a.listRecId asc", Long.valueOf(listId) );
return listOfRecs;
}
Service Class:
--------------
#Transactional
public List<LatestRecVO> getListRecs(String listId) {
List<LatestRecVO> listOfRecs = listDao.getListRecDetails(listId);
return listOfRecs;
}
New Code
DAO Class:
---------
public List<LatestRecVO> getListRecs(String listId) {
Query q = getSession().createQuery(" from LatestRecVO a where a.listId = (:listId) order by a.listRecId asc");
q.setParameter("listId", Long.valueOf(listId));
List<LatestRecVO> listOfRecs = q.list();
return listOfRecs;
}
Service Class:
--------------
#Transactional
public List<LatestRecVO> getListRecs(String listId) {
List<LatestRecVO> listOfRecs = listDao.getListRecDetails(listId);
return listOfRecs;
}
The entity class LatestRecVO do not have any associated entity with it.
I checked the Hibernate Template's find() method and saw its uses some caching.
Tried 2nd level cache along with query cache but it didnt helped. I may have configured 2nd level cache incorrectly but to try it again i want to be sure that its the way out else i would be wasting time.
I made show_sql as true and can see it just ran a singly query. On DB the same query takes some milliseconds to run. It seems like hibernate is taking time to build objects from the result.
On one of the post it was mentioned that its mandate to have a default constructor in our entities. I have not created any constructor in my entity class so i assume that i do have java's default constructor in place.
My table has 36 columns and in total there are 4k records to be fetched.
Any pointer in this will be really helpful.
Update
Sorry, I cannot post the complete code here, so just giving the details. I have the composite primary key for LatestRecVO. I have created a class LatestRecPK for primary key, it implements serializable and have #Embeddable annotation. In LatestRecVO i have given #IdClass(LatestRecPK.class) to include the primary key class. LatestRecVO has a CLOB property along with String, Long and #Temporal(TemporalType.DATE) properties and corresponding setters/getters.
Related
I'm rewriting a big project with SpringBoot 2.2.6 and I'm run into a problem.
In the old project (pure ejb) when a complex entity is updated, the code build entity from DTO's as follows:
public Entity dtoToEntity(DTO dto) {
Entity entity = new Entity();
entity.setId(dto.getID());
// ecc...
// ecc...
entity.setSubEntity(dto.getSubEntity() != null ? new SubEntity(dto.getSubEntity().getId() : null);
// and so on
}
The important section is that related to subentity! After made a mapping like that the old project calls:
EntityManager.merge(entity);
I think that with merge call, if inside the database exists a SubEntity with specified id and other fields valorized, the other fields remain valid and are not set to null because they aren't declared in mapping.
But with SpringBoot I'm using JpaRepository and I don't think the same thing happens if I call:
jpaRepository.save(entity);
I think with this call the other fields of SubEntity with specified id will set to null!
Is it correct?
What can I solve this?
Thanks for your reply first of all!
You are right, I can't do something like that nor with EntityManager.merge() method! Than I try to explain better what I want to do:
Suppose I have a complex Entity, which has many nested Entities (which may have nested entities) as follows:
#Entity
public class Car {
private String name;
....
....
private Engine engine; // nested entity
private Chassis chassis; // nested entity
}
And:
#Entity
public class Engine {
private String company;
private Oil oil; // nested entity
....
}
Now suppose in the database I have a Car, with all relationship filled (Engine, Chassis, Oil ecc..) and suppose I want to update the Car name from Ferrari to Fiat, if I use pure SQL I can simply write:
update Car c set c.name = "Fiat" where c.id = [id];
Now if I use Spring JPA, to ensure that all nested entity (and their field) are not setting to null when I update my entity I have to do:
Car car = carRepository.findById([id]);
car.setName("Fiat"):
carRepository.save(car);
This way I will update Car name and I'm sure that all other entities will remain set because are loaded by findById() method.
My question and my goal is to know if is there a way to do something like that:
Car car = new Car();
car.setId(1); // id of Ferrari car
car.setName("Fiat");
someRepositoryOrEntityManager.saveOrUpdate(car);
And preserve all other field and relation without load all of these by the find method (maybe due to a performance reasons).
Did you give it a try or it is just guesswork?
First of all, you don't need to embrace spring data repositories. You can inject EntityManager if it helps in the migration process.
Secondly, look at the implementation of SimpleJpaRepository.save
#Transactional
public <S extends T> S save(S entity) {
if (entityInformation.isNew(entity)) {
em.persist(entity);
return entity;
} else {
return em.merge(entity);
}
}
This means that JpaRepository.save calls em.merge if it concludes that the entity is not new.
The check if the entity is new is in AbstractEntityInformation.isNew. It concludes that the entity is new only if its id is null (or 0 for primitive numerical types).
You assign the id from the dto. If it is not null (or non-zero for primitives), there is no reason to believe that the new code will behave in a different way than the old one.
Answer for updated question
If you want to modify an entity without fetching it, I would suggest JPQL or criteria query
Reference:
More about whether an entity is new or not, can be found here.
I have the following simple entity FileRegistry :
#Getter
#Setter
#Entity
#ToString
#Table(name = "file_store")
public class FileRegistry {
#Id
private String name;
/**
* Creation timestamp of the registry
* This value is automatically set by database, so setter method
* has been disabled
*/
#Setter(AccessLevel.NONE)
#Column(insertable = false, updatable = false)
private LocalDateTime creationDate;
}
The following FileRepository DAO:
#Repository
public interface FileRepository extends JpaRepository<FileRegistry, String> { }
and the following Spring Boot test :
#SpringBootTest(classes=PersistTestConfig.class, properties = { "spring.config.name=application,db"})
#ActiveProfiles("test")
#Transactional
public class FileRepositoryTest {
#Autowired
FileRepository fileRepository;
#Test
void insertFileTest() {
assertNotNull(fileRepository, "Error initializing File repository");
// Check registry before insertion
List<FileRegistry> allFiles = fileRepository.findAll();
assertNotNull(allFiles, "Error retrieving files from registry");
assertThat(allFiles.size(), is(0));
// Insert file
FileRegistry fileRegistry = new FileRegistry();
fileRegistry.setName("Test");
fileRepository.save(fileRegistry);
// Check that the insertion was successful
allFiles = fileRepository.findAll();
assertNotNull(allFiles, "Error retrieving files from registry");
assertThat(allFiles.size(), is(1));
assertEquals("File registry name mismatch", "Test", allFiles.get(0).getName());
System.out.println(allFiles.get(0));
}
}
Persistence configuration class defined as follows :
#Configuration
#EnableAutoConfiguration
#EnableTransactionManagement
#EnableJpaRepositories
public class PersistTestConfig {
}
The table file_store defined in H2 as :
CREATE TABLE file_store (name VARCHAR NOT NULL, creation_date TIMESTAMP(3) DEFAULT NOW() NOT NULL, CONSTRAINT file_store_pk PRIMARY KEY (name));
Everything works fine except that when I use #Transactional at test level (mainly to benefit from rollbacks i.e. db cleanup on each test) a null value is fetched for the creationDate field :
FileRegistry(name=Test, creationDate=null)
When I remove #Transactional from the test class, the fetched value contains the date as computed by H2 :
FileRegistry(name=Test, creationDate=2019-03-07T17:08:13.392)
I've tried to flush and merge manually the instance to no avail. To be honest, right now I'm a little bit lost on how #Transactional really works, in fact reading the docs and inspecting the code, the underlying JpaRepository implementation (SimpleJpaRepository) is annotated as #Transactional(readOnly = true).
A little help on this subject would be very appreciated.
Ok, figured it out.
Simply issuing a refresh entityManager.refresh(allFiles.get(0)); solves the issue.
I tested also using Hibernate's #Generated(INSERT) specific annotation in the entity creationDate field and it also worked fine.
By the way I've eventually decided to drop this thing in favor of using Spring Data's JpaAuditing features and annotating the field with #CreatedDate annotation to fill the value instead of relying on DB date (by the way, production-wise, you probably shouldn't rely on DB time). To me this is feels more, let's say, "correct" and springy way of doing things.
I have a one-to-many relationship defined as below
#Cacheable
#Entity
#NamedEntityGraph(
name = "Parent.Child",
attributeNodes = {
#NamedAttributeNode("children"),
}
)
public class Parent {
private Set<Child> children;
// getter - setter
}
Now in my DAL, i'm calling this method
#Override
public Parent getParentWithChildren(int id) {
EntityGraph<?> graph = entityManager.getEntityGraph("Parent.Child");
Map<String, Object> props = new HashMap<>();
props.put("javax.persistence.fetchgraph", graph);
return entityManager.find(Parent.class, id, props);
}
Since i have loaded Parent with Children, i should be able to use children collection outside of the transaction. But i'm getting Lazyinitialization Exception. This happens only when hibernate level 2 cache - ehcache is enabled. If i disable it from config, it works as expected. Also if i initialize collection explicitly after find, it works as expected. So is it a bug?. I'm using Hibernate 5.2.6.Final with JPA 2.1.
EDIT: One more thing i noticed was that entity loads fine for the first time, so that problem must be related with hibernate & cache provider.
In order for Hibernate to use an entity graph one must bypass the second-level cache (e.g. EhCache) by using the javax.persistence.cache.retrieveMode hint with the value CacheRetrieveMode.BYPASS:
final EntityGraph<?> graph = em.getEntityGraph("myGraph");
final Map<String, Object> hints = new HashMap<>();
hints.put("javax.persistence.cache.retrieveMode", CacheRetrieveMode.BYPASS);
hints.put("javax.persistence.fetchgraph", graph);
final SomeEntity entity = em.find(SomeEntity.class, 42, hints);
Note that the second-level cache will still be populated as usual.
No. It's not a bug. I guess Hibernate is delaying the loading from cache until really necessary. The implementation can decide to be lazy as see fit unless you ask for eager fetching.
So the general rule is to load everything you need before going out of the transaction (before closing the Hibernate session to be precise).
I am implementing queries in my web application with JPA repositories. The two main tables I am querying from are FmReportTb and SpecimenTb.
Here are the two entity classes (only important attributes are listed).
//FmReportTb.java
#Entity
#Table(name="FM_REPORT_TB")
public class FmReportTb implements Serializable {
#Column(name="ROW_ID")
private long rowId;
#Column(name="FR_BLOCK_ID")
private String frBlockId;
#Column(name="FR_FULL_NAME")
private String frFullName;
#OneToOne
#JoinColumn(name="SPECIMEN_ID")
private SpecimenTb specimenTb;
FmReportTb has OneToOne relationship with SpecimenTb.
#Entity
#Table(name="SPECIMEN_TB")
public class SpecimenTb implements Serializable {
private String mrn;
#OneToOne(mappedBy="specimenTb", cascade=CascadeType.ALL)
private FmReportTb fmReportTb;
The query I am working on is to find all records in FmReportTb and show a few attributes from FmReportTb plus mrn from SpecimenTb.
Here is my JPA repository for FmReportTb:
#Repository
public interface FmReportRepository extends JpaRepository<FmReportTb, Long> {
#Query("select f from FmReportTb f where f.deleteTs is not null")
public List<FmReportTb> findAllFmReports();
Since, I am only showing part of the attributes from FmReportTb and one attribute from SpecimenTb, I decided to create a Value Object for FmReportTb. The constructor of the VO class assigns attributes from FmReportTb and grabs mrn attribute from SpecimenTb based on the OneToOne relationship. Another reason for using VO is because table FmReportTb has a lot of OneToMany children entities. For this particular query, I don't need any of them.
public class FmReportVO {
private String frBlockId;
private Date frCollectionDate;
private String frCopiedPhysician;
private String frDiagnosis;
private String frFacilityName;
private String frFullName;
private String frReportId;
private String filepath;
private String mrn;
public FmReportVO(FmReportTb fmReport) {
this.frBlockId = fmReport.getFrBlockId();
this.frCollectionDate = fmReport.getFrCollectionDate();
this.frCopiedPhysician = fmReport.getFrCopiedPhysician();
this.frDiagnosis = fmReport.getFrDiagnosis();
this.frFacilityName = fmReport.getFrFacilityName();
this.frFullName = fmReport.getFrFullName();
this.frReportId = fmReport.getFrReportId();
this.mrn = fmReport.getSpecimenTb().getMrn();
}
I implemented findall method in servicebean class to return a list of FmReportTb VOs.
//FmReportServiceBean.java
#Override
public List<FmReportVO> findAllFmReports() {
List<FmReportTb> reports = fmReportRepository.findAllFmReports();
if (reports == null) {
return null;
}
List<FmReportVO> fmReports = new ArrayList<FmReportVO>();
for (FmReportTb report : reports) {
FmReportVO reportVo = new FmReportVO(report);
String filepath = fileLoadRepository.findUriByFileLoadId(report.getFileLoadId().longValue());
reportVo.setFilepath(filepath);
fmReports.add(reportVo);
}
return fmReports;
}
Lastly, my controller looks like this:
#RequestMapping(
value = "/ristore/foundation/",
method = RequestMethod.GET,
produces = "application/json")
public ResponseEntity<List<FmReportVO>> getAllFmReports() {
List<FmReportVO> reports = ristoreService.findAllFmReports();
if (reports == null) {
return new ResponseEntity<List<FmReportVO>>(HttpStatus.NOT_FOUND);
}
return new ResponseEntity<List<FmReportVO>>(reports, HttpStatus.OK);
}
There are about 200 records in the database. Surprisingly, it took almost 2 full seconds to retrieve all the records in JSON. Even though I did not index all the tables, this is way too slow. Similar query takes about probably a few ms on the database directly. Is it because I am using Value Objects or JPA query tends to be this slow?
EDIT 1
This may have to do with the fact that FmReportTb has almost 20 OneToMany entities. Although the fetchmode of these child entities are set to LAZY, JPA Data repository tends to ignore the fetchmode. So I ended up using NamedEntityGraph to specify the attributes EAGER. This next section is added to the head of my FmReportTb entity class.
#Entity
#NamedEntityGraph(
name = "FmReportGraph",
attributeNodes = {
#NamedAttributeNode("fileLoadId"),
#NamedAttributeNode("frBlockId"),
#NamedAttributeNode("frCollectionDate"),
#NamedAttributeNode("frDiagnosis"),
#NamedAttributeNode("frFullName"),
#NamedAttributeNode("frReportId"),
#NamedAttributeNode("specimenTb")})
#Table(name="FM_REPORT_TB")
And then #EntityGraph("FmReportGraph") was added before the JPA repository query to find all records. After doing that, the performance is improved a little bit. Now fetching 1500 records only takes about 10 seconds. However, it still seems too slow given each json object is fairly small.
Answering for the benefit of others with slow JPA queries...
As #Ken Bekov hints in the comments, foreign keys can help a lot with JPA.
I had a couple of tables with a many to one relationship - a query of 100,000 records was taking hours to perform. Without any code changes I reduced this to seconds just by adding a foreign key.
In phpMyAdmin you do this by creating a Relationship from the "many" table to the "one" table. For a detailed explanation see this question: Setting up foreign keys in phpMyAdmin?
and the answer by #Devsi Odedra
I'm working with Spring Boot 1.3.0.M4 and a MySQL database.
I have a problem when using modifying queries, the EntityManager contains outdated entities after the query has executed.
Original JPA Repository:
public interface EmailRepository extends JpaRepository<Email, Long> {
#Transactional
#Modifying
#Query("update Email e set e.active = false where e.active = true and e.expire <= NOW()")
Integer deactivateByExpired();
}
Suppose we have Email [id=1, active=true, expire=2015/01/01] in DB.
After executing:
emailRepository.save(email);
emailRepository.deactivateByExpired();
System.out.println(emailRepository.findOne(1L).isActive()); // prints true!! it should print false
First approach to solve the problem: add clearAutomatically = true
public interface EmailRepository extends JpaRepository<Email, Long> {
#Transactional
#Modifying(clearAutomatically = true)
#Query("update Email e set e.active = false where e.active = true and e.expire <= NOW()")
Integer deactivateByExpired();
}
This approach clears the persistence context not to have outdated values, but it drops all non-flushed changes still pending in the EntityManager. As I use only save() methods and not saveAndFlush() some changes are lost for other entities :(
Second approach to solve the problem: custom implementation for repository
public interface EmailRepository extends JpaRepository<Email, Long>, EmailRepositoryCustom {
}
public interface EmailRepositoryCustom {
Integer deactivateByExpired();
}
public class EmailRepositoryImpl implements EmailRepositoryCustom {
#PersistenceContext
private EntityManager entityManager;
#Transactional
#Override
public Integer deactivateByExpired() {
String hsql = "update Email e set e.active = false where e.active = true and e.expire <= NOW()";
Query query = entityManager.createQuery(hsql);
entityManager.flush();
Integer result = query.executeUpdate();
entityManager.clear();
return result;
}
}
This approach works similar to #Modifying(clearAutomatically = true) but it first forces the EntityManager to flush all changes to DB before executing the update and then it clears the persistence context. This way there won't be outdated entities and all changes will be saved in DB.
I would like to know if there's a better way to execute update statements in JPA without having the issue of the outdated entities and without the manual flush to DB. Perhaps disabling the 2nd level cache? How can I do it in Spring Boot?
Update 2018
Spring Data JPA approved my PR, there's a flushAutomatically option in #Modifying() now.
#Modifying(flushAutomatically = true, clearAutomatically = true)
I know this is not a direct answer to your question, since you already have built a fix and started a pull request on Github. Thank you for that!
But I would like to explain the JPA way you can go. So you would like to change all entities which match a specific criteria and update a value on each. The normal approach is just to load all needed entities:
#Query("SELECT * FROM Email e where e.active = true and e.expire <= NOW()")
List<Email> findExpired();
Then iterate over them and update the values:
for (Email email : findExpired()) {
email.setActive(false);
}
Now hibernate knows all changes and will write them to the database if the transaction is done or you call EntityManager.flush() manually. I know this won't work well if you have a big amount of data entries, since you load all entities into memory. But this is the best way, to keep the hibernate entity cache, 2nd level caches and the database in sync.
Does this answer say "the `#Modifying´ annotation is useless"? No! If you ensure the modified entities are not in your local cache e.g. write-only application, this approach is just the way to go.
And just for the record: you don't need #Transactional on your repository methods.
Just for the record v2: the active column looks as it has a direct dependency to expire. So why not delete active completely and look just on expire in every query?
As klaus-groenbaek said, you can inject EntityManager and use its refresh method :
#Inject
EntityManager entityManager;
...
emailRepository.save(email);
emailRepository.deactivateByExpired();
Email email2 = emailRepository.findOne(1L);
entityManager.refresh(email2);
System.out.println(email2.isActive()); // prints false