I have two entities linked by a #ManyToMany association. Let's say Text and Country. So in my Text entity, I have:
#ManyToMany(fetch = FetchType.LAZY)
private Set<Country> countries = new HashSet<>();
I want to perform a paginated query that returns texts, so in a JPA repository, I write:
Page<Text> findAll(Pageable pageable);
This works. The problem is that as countries are lazy loaded, hibernate executes an extra select query for each country to load.
So I can use #EntityGraph to fetch countries:
#EntityGraph("countries")
Page<Text> findAll(Pageable pageable);
It works, I have only one query, but it is wayyyyyyyyyy much longer (about 12x). I seems that it fetches the countries for all the texts (I have a lot of texts) before applying the pagination. This is confirmed by the Spring data team: https://github.com/spring-projects/spring-data-jpa/issues/1976
So is there a way to perform an optimized paginated query, without N+1 select and without loading the entire table ?
Thank you very much
Related
My goal is to do insert and update multiple documents in ElasticSearch using ElasticsearchRepository.
public interface EmployeeInfoRepository extends ElasticsearchRepository<EmployeeInfo, String> {
}
However, whenever I call saveAll(entities), the number of document is unchanged but it creates new indexes for those entities.
employeeInfoRepository.saveAll(employeeInfos);
If I insert 1000 elements, at first it will have 1000 docs and 1000 indexes, which is what I expected. Then I call saveAll two more times, it still has 1000 docs but now the number of indexes increases to 3000.
How can I update it properly?
It would be the best if it's just as easy as calling saveAll and the rest is handled by SpringBootData ElasticSearch.
Update 1:
There is no change with the data, however when I run saveAll, the storage_size keeps increasing. Not sure if it creates the indexes again and still keeps the old indexes.
If you define the id in your document class, elasticsearch updates/inserts the document with the same document id value.
Here an example:
#Document(indexName = "example_index")
public class Example {
#Id
#Field(type = FieldType.Long)
private long id;
}
Of course, you have to handle the logic for a unique id in your project.
I am a newbie to Java Persistence API and Hibernate and using Spring JPA repositories for querying in DB. Now I have two entities in Parent <-> Child relationship with Parent entity with #OneToMany and Child entity with #ManyToOne mapping.
Parent Entity:-
#Entity
#Table(name = "PERSONS")
public class Persons {
...
#OneToMany(mappedBy = "person", cascade = CascadeType.ALL, fetch = FetchType.LAZY, orphanRemoval = true)
public List<Cards> cards = new ArrayList<Cards>();
...
}
Child Entity:-
#Entity
#Table(name = "CARDS")
public class Cards {
...
#ToString.Exclude
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "PERSON_ID", nullable = false, insertable = false, updatable = false)
public Person person;
...
}
And I am using my PersonsRepository like below :-
#Repository
public interface PersonsRepository extends JpaRepository<Persons, String> {
....
}
Now the fetchType being used in the relationship is LAZY at both the ends. Now whenever I tried to loop over a List and tried to process the cards for each using person.getCards(), it gives me below error:-
org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: com.xxx.abc.Persons.cards, could not initialize proxy - no Session
at org.hibernate.collection.internal.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:606)
at org.hibernate.collection.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:218)
at org.hibernate.collection.internal.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:585)
at org.hibernate.collection.internal.AbstractPersistentCollection.read(AbstractPersistentCollection.java:149)
at org.hibernate.collection.internal.PersistentSet.iterator(PersistentSet.java:188)
at java.util.Spliterators$IteratorSpliterator.estimateSize(Spliterators.java:1821)
at java.util.Spliterator.getExactSizeIfKnown(Spliterator.java:408)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
Now I have found everyone saying that using LAZY is the best approach in Hibernate and it says lot more about the correct design of code as well. I agree the way I have used person.getCards() will not have any open session and that is the reason it is giving me LazyInitializationException but the intent behind this is to save a lot more DB calls.
Assuming I have 1000 persons list, that means I have to make 1000 separate calls to getCards() for each person. That's why if I use the FETCHTYPE.EAGER in Person #OneToMany, what is the performance impact since everything will be fetched eagerly.
Need suggestions about the best practices followed for such kind of problems. TIA.
Edit:-
I have a method in service class where I am using #transactional for that like below:-
#Transactional(readOnly = true)
public void fetchData(Integer param1, Timestamp param2, Timestamp param3, List<String> param4, NavigableMap<Long, List<Cards>> param5) {
List<Persons> validPersons = personRepo.getCardsPerPerson(param2, param3);
if(validPersons != null && !validPersons.isEmpty()) {
// store the cards on the basis of epoch timestamp
prepareTimestampVsCardsMap(validPersons, param4, param5);
}
}
private void prepareTimestampVsCardsMap(List<Persons> validPersons, List<String> uList, NavigableMap<Long, List<Cards>> timestampVsCardsList) {
for(Person person : validPersons) {
Long epoch = order.getOrderTime().getTime();
Set<Cards> cardsPerPerson = person.getCards();
}
}
Also, the query being used in repository for getting the cards associated to a person is using join fetch as below:-
#Query(value = "select p from Person p join fetch Cards c on p.id = c.id WHERE p.orderTime BETWEEN ?1 AND ?2 ORDER BY orderTime ASC")
public List<Person> getCardsPerPerson(Timestamp param1, Timestamp param2);
I am still getting the same above mentioned LazyInitializationException. Can anyone please help.
First of all, it's always better to use FetchType.LAZY instead of FetchType.EAGER. Why? Because you might not need all the data every time. If you want to return a list of Persons and display them somehow, somewhere, do you need to fetch all of their cards as well? If not, then FetchType.LAZY would be the better option, and you would then control how much data you need.
LazyInitializationException usually indicates that you didn't fetch all the data you need while your Session was opened. There are many ways to fetch associated data (none of which is keeping the Session opened while processing request):
1. using join fetch in your JPQL/HQL
#Query("select p from Person p join fetch p.cards where ...")
List<Person> getCardsPerPerson(Timestamp param1, Timestamp param2);
2. if you're using Spring Data, you could use #EntityGraph instead of join fetch
#EntityGraph(attributePaths = { "cards" })
List<Person> getPersons();
That way, every time you call getPersons, it will fetch cards as well. Of course, you couldn't use this one if you have to write #Query.
If you're using Spring Data's naming conventions for some simple queries, then #EntityGraph would be an option for fetching associations.
3. using Criteria API
Again, if you're using Spring Data, this is just a fallback solution in case you end up with MultipleBagFetchException. I will not go into details for this one, but in case you encounter this exception, you'll find solution in Vlad Mihalcea's blog post The best way to fix the Hibernate MultipleBagFetchException.
You are under the misconception that EAGER loading means Hibernate will fetch all data with one statement, this is false. With EAGER as a strategy, the framework will just do every query required to fetch all data for every entity.
Example: If one entity has 2 EAGER relationships, fetching one will result in 3 statements, one to load the entity, one for each of its relationships. If you have 3 entities, you will have 7 statements, the initial statement loading the 3 objects, plus 2 per object.
When your treatment requires everything, there is no real performance impact at the moment. But most applications are not made of one treatment. This means every treatment in your application will load everything which is EAGER, even if not needed. This will effectively slow everything down. You also risk loading all your database in memory if everything is in EAGER.
This is why LAZY is the recommended approach.
As for your LazyInitializationException, it seems in your stack trace that you are using the stream API. It's a wild guess due to missing details, but JPA/Hibernate doesn't handle sharing a session between threads, so if you are using parrallelStream it could cause the problem.
I am using rest services with spring data . when i get data from single table its return proper result in json format . but when i use many to many association between entities using hibernate i am getting an un acceptable result with following error in chrome's console .
Failed to load resource: net::ERR_INCOMPLETE_CHUNKED_ENCODING
My result looks like there just one row repeating itself , and its a particularly that field which is being used in new generated table by association.
[{"id":7,"name":"Milk pack","description":"haleeb","imageUrl":"milk.jpg","price":350.00,"category":null,"orderDetail":[]},{"id":8,"name":"oil","description":"olive oil ","imageUrl":"/resources/uploads/olive.png","price":670.00,"category":null,"orderDetail":[{"id":263,"productlist":[{"id":10,"name":"Mobile","description":"awesome design, slim design ","imageUrl":"/static/uploads","price":34569.00,"category":null,"orderDetail":[{"id":263,"productlist":[{"id":10,"name":"Mobile","description":"awesome design, slim design ","imageUrl":"/static/uploads","price":34569.00,"category":null,"orderDetail":[{"id":263,"productlist":[{"id":10,"name":"Mobile","description":"awesome design, slim design ","imageUrl":"/static/uploads","price":34569.00,"category":null,"orderDetail":[{"id":263,"productlist":[{"id":10,"name":"Mobile","description":"awesome design, slim design
.
.
.and so on
My entities are following
Product table
#ManyToMany(mappedBy = "productlist")
private List<OrderDetail> orderDetail =new ArrayList<OrderDetail>();
OrderDetail table
#ManyToMany
#JoinTable(
name="order_detail_productlist",
joinColumns=#JoinColumn(name="order_detail_id", referencedColumnName="id"),
inverseJoinColumns=#JoinColumn(name="productlist_id", referencedColumnName="id"))
private Set<Product> productlist = new HashSet<Product>();
I am using spring data jpa repository to get them
List<Product> findAll();
Note: which products those are not ordered yet that are working properly
You need the all log for the information, it maybe cause by loop when with jackson. So you need add #JsonIgnoreProperties.Please see http://stackoverflow.com/questions/3325387
My Problem is solved by using jackson 2.0 documentation
by Adding following annotation or OrderDetail table
#JsonBackReference
it breaks loop and show result properly
Is it possible to somehow get multiple objects from a one-to-many-collection by index/key, which is marked with extra lazy load?
I have a big collection where I can't fetch all entries but still want to get multiple objects from it.
For example:
class System
{
...
#OneToMany(mappedBy = "system")
#MapKey(name = "username")
#LazyCollection(LazyCollectionOption.EXTRA)
private Map<String, User> users = new HashMap<>();
public List<User> getUsers(List<String> usernames)
{
//what to do
}
}
It's just a simple example but it portraits my problem.
I know I could just use the Criteria API or (named) queries but I try to keep the logic where it belongs to.
Unfortunately it seems that Hibernate does not support loading multiple entries from a collection inside a entity.
Only ways I found:
use eager/lazy loading and get all objects (which won't work if there are many)
use extra lazy loading and get multiple objects by retrieving one by one (can hurt performance)
use Session.createFilter which can not be called inside an entity
Here we have a Manifest class that includes list of students and teachers, both could be null.
class Manifest{
#ElementCollection(fetch = FetchType.EAGER)
#CollectionTable(name = "MANIFEST_STUDENT")
List<String> students = new ArrayList<String>();
#ElementCollection(fetch = FetchType.EAGER)
#CollectionTable(name = "MANIFEST_TEACHER")
List<String> teachers = new ArrayList<String>();;
#ElementCollection(fetch = FetchType.EAGER)
#CollectionTable(name = "MANIFEST_OTHERS")
List<String> others = new ArrayList<String>();;
}
on the UI, there are two multiple select, one for student and one for teacher that let user choose for current manifest.
here is the problem:
When user deselect all students or teachers from the list(meaning remove all students or teachers from current manifest) and click save, unfortunately nothing can be saved, from UI and database it shows that the multiselect chosen looks the SAME as before.
from service layer, the code is simply like this.
manifest.merge();
It seems we must keep at least one student or teacher for the collection field to make the change valid. So what's going on here and what is the solution? BTW, we are on Openjpa.
Kind of resolve the issue, more like a work around:
Before calling merge(), place several condition checkers to make sure the collection fields are not null
public void save(Manifest entity) {
if(entity.getStudents()==null){
entity.setStudents(new ArrayList<String>());
}
if(entity.getTeachers()==null){
entity.setTeachers(new ArrayList<String>());
}
if(entity.getOthers()==null){
entity.setOthers(new ArrayList<String>());
}
entity.merge();
}
Simple as it, it seems the UI returns those collection fields as null even we initiate them as with empty String lists.
cheers.
Initializing a value in a JPA managed class, such as class Manifest, has no bearing on what, or how, JPA will create the class as JPA maps extracted rows to the class. In particular, the result of:
List<String> students = new ArrayList<String>();
is likely to be:
On creation (by JPA) of a new instance, assign an ArrayList<String>() to students.
JPA overwrites students with the data it extracts - the empty ArrayList is dereferenced/lost.
If your code is clearing a list, such as students, use obj.getStudents().clear(). More likely to run into problems if you call obj.setStudents(someEmptyList).
The issue here is how the JPA manager handles empty datasets: as null or as an empty list. The JPA spec (old, not sure about the just released update) doesn't take a position on this point. A relevant article here.
From your comments, it's apparent that OpenJPA may not be respecting a null value for a Collection/List, while it happily manages the necessary changes for when the value is set to an empty list instead. Someone knowing more about OpenJPA than I may be able to help at this stage - meanwhile you've got a workaround.