Hibernate N+1 issue for Multiple children - spring-boot

I have a entity class which has multiple children with oneToMany association:
public class A{
private Long id;
private String name;
#OneToMany(mappedBy = "A", fetch = FetchType.LAZY, cascade = CascadeType.ALL,
orphanRemoval = true)
private List<B>bList= new ArrayList<>();
#OneToMany(mappedBy = "A", fetch = FetchType.LAZY, cascade = CascadeType.ALL,
orphanRemoval = true)
private List<C>cList= new ArrayList<>();
#OneToMany(mappedBy = "A", fetch = FetchType.LAZY, cascade = CascadeType.ALL,
orphanRemoval = true)
private List<D>dList= new ArrayList<>();
//getters and setters
}
For B,C and D I have set ManyToOne. In a word, they are in a bi-directional relationship.
Now, If I fetch A by id, I see a lot of queries get fired which turns out to be N+1 problem. To solve this, I added #Fetch(FetchMode.SUBSELECT) to all of the oneToMany relationships above which cause less queries to be fired.
My question is:
is it okay using #Fetch(FetchMode.SUBSELECT) or I can optimize it further?
what if I want to fetch all "As" by calling findAll() method? What should be the syntax for multiple children? Like
"select a from A a join fetch a.b then ??"
List< A > findAll()

Now, If I fetch A by id, I see a lot of queries get fired which turns out to be N+1 problem. To solve this, I added #Fetch(FetchMode.SUBSELECT) to all of the oneToMany relationships above which cause less queries to be fired.
You are not saying how/when these queries are fired, so a possible reason for the problem is that you are returning entities from your HTTP endpoints which are then serialized. Using #Fetch(FetchMode.SUBSELECT) is one way to "improve" the performance but will only work nicely if the base query which you use to fetch A is simple. If it gets too complex (pagination, complex predicates, etc.) you should stick to the default which is SELECT and instead configure a proper batch size via the #BatchSize( size = 32 ) annotation. A good value for the batch size would be the amount of A instances that you expect to be returned such that only a single query is executed per collection. If you allow a max page size of e.g. 50, you setting the value to 50 would be perfect.
List< A > findAll()
Do not ever do this if you care about performance and usability. It rarely makes sense to allow returning all elements, as no user can handle more than ~20 elements at a time anyway. As mentioned before, always have some kind of upper limit on the page size to prevent misuses that can cause performance issues.

Related

#Batchsize annotation not working for OneToMany

I have following classes and on annotating #BatchSize annotation it is not working and I am getting n+1 select query.
Class Shipment{
#OneToMany(fetch = FetchType.LAZY, mappedBy = order.shipment, cascade = CascadeType.ALL,
orphanRemoval = true)
#BatchSize(size=20)
Set<Orders> orders = new Hashset(); <---- Batch size annotation not working
}
Order.class
class Order{
#ToString.Exclude
#ManyToOne
#JoinColumn(name = "item_fk")
Item item;
#ToString.Exclude
#ManyToOne
#JoinColumn(name = "shipment_fk")
Shipment shipment; }
Item.class
class Item{
String id;
String name;
}
What is mistake in implementation that i am getting n+1 queries?
Try to use List<Orders> instead of Set<Orders>.
Please note as it's mentioned in the documentation:
However, although #BatchSize is better than running into an N+1 query issue, most of the time, a DTO projection or a JOIN FETCH is a much better alternative since it allows you to fetch all the required data with a single query.
Your N + 1 query issue is due to the fact that you do eager fetching of Item in Order. Change to LAZY there and you should be good to go.

Avoid Lazy properties fetched in ResponseEntity

I'm new in Spring, Hibernate, JPA and it's API. I've created a #RestController and the related method is like,
#GetMapping("/user")
public ResponseEntity getListItemById(){
UserTypeEntity entity = userTypeRepository.findFirstByUserTypeId(1);
return ResponseEntity.ok().body(entity);
}
UserTypeEntity has 2 lazy getters,
#ManyToOne(fetch = FetchType.LAZY, optional = false)
#JoinColumn(name = "user_id", referencedColumnName = "user_id", nullable = false)
public UserEntity getUserByUserId() {
return userByUserId;
}
#ManyToOne(fetch = FetchType.LAZY, optional = false)
#JoinColumn(name = "user_type_list_item_id", referencedColumnName = "list_item_id", nullable = false)
public ListItemEntity getListItemByUserTypeListItemId() {
return listItemByUserTypeListItemId;
}
All the properties of ListItemEntity and UserEntity are null until and unless I use JOIN FETCH query. I've checked and verified that one.
(It might be familiar to any experienced)
Looks like following.
Here is the sample response I use to get,
(Sorry it couldn't even be formatted because of large data response, though I have single row in each table. It's Infinite recursion (StackOverflowError))
Everything is loaded eventually. I couldn't identify what the heck going wrong here. Why these lazy null properties are loaded and I got this weird and vague response? I've wasted whole day on this, plz help to get out of this.
Move your annotations from getter to field declaration and you will see consistent results.
After your edit:
#ShreeKrishna that was my point. Now I can provide clear explanation that this is expected behavior. Debugger shows you what fields are actually are and those are and WILL BE null as your lazy intitialization trigger is on GETTER method, not on field. So, don't pay attention to what debugger is showing you as long as you will access your properties via getters instead of direct access - you will be fine.

Checking for multiple child constraint violations in Hibernate/JPA

TL;DR: Is it possible to perform nested transactions in Hibernate that use SavePoints to rollback to specific states?
So I am attempting to persist a parent entity with a OneToMany mapping to child entities. This is working fine.
During this persistence, I would like to catch and log ALL constraint violations that occur. Currently, the FIRST entity (child or parent) to have a constraint violation throws a ConstraintViolationException and rolls back the transaction. I would like for the transaction to still be rolled back, but somehow collect ALL of the constraint violations that would occur.
Here is a brief outline of my entities:
ParentEntity.java
#Entity
#Table(name = "PARENT", schema = "SOMESCHEMA")
public class ParentEntity {
private static final ID_COLUMN = "ID_COLUMN";
#Id
#Column(name = ID_COLUMN)
private Long id;
#OneToMany(fetch = FetchType.LAZY, cascade = {CascadeType.ALL}, orphanRemoval = true)
#JoinColumn(name = ID_COLUMN, referencedColumnName = ID_COLUMN)
private List<childEntity> children;
}
ChildEntity.java
#Entity
#Table(name = "CHILD", schema = "SOMESCHEMA")
public class ChildEntity {
public ChildEntity(String input) {
this.validationString = input;
}
#Id
#Column(name = ParentEntity.ID_COLUMN)
private Long id;
#ManyToOne
#JoinColumn(name = ParentEntity.ID_COLUMN, insertable = false, updatable = false)
private ParentEntity parent;
// The field under validation (should be less than 25 char's long)
#Column(name = "VALIDATE_ME")
private String validationString;
}
Example run:
public void someMethod() {
ParentEntity parent = new ParentEntity();
parent.addChild(new Child("good input 1"));
parent.addChild(new Child("bad input 1 break here"));
parent.addChild(new Child("bad input 2 break here"));
parent.addChild(new Child("good input 2"));
dataAccessObject.persist(parent);
}
Results:
I see the transaction rolled back and the ConstraintViolationException only contains information for the first bad child.
Desired Results:
I see the transaction rolled back and the ConstraintViolationException show information for all the bad children regardless of how many children were bad. (Also, if the parent has a constraint violation, I would still like to check the child constraints)
Is this possible?
ConstraintViolationException is a HibernateException and all Hibernate exceptions are not recoverable. So if one exception is thrown, you cannot rely on the existing session state to continue processing reliably.
So this is not possible or recommended.
I found a way to accomplish my end result, although it was through means I did not expect.
In order to accomplish my goal, I would need to use nested transactions and SavePoints. At the end of the day, that implementation would still produce invalid data in my database, because during the time it takes to find an error in a child, some of the other children may have already been persisted and a consumer of my database would be unaware that the parent and all it's children were about to be deleted due to one or more bad entities (parents or children).
My solution:
I implemented a validator to validate all of the parents and children before going to persist them. The drawback to this method is that I have to annotate constraints on my entity fields, but double validation is never a bad thing.
The answer to my original question:
This is impossible to due with my version of Hibernate, unless I implemented custom SavePoint functionality to support nested transactions.

Spring Template Project jpa annotations

I have a one to many relationship: Order and Items. An Order may have lots of items
The code below (all code) is auto generated by Spring Template project:
On the Order Side:
#OneToMany(cascade=CascadeType.ALL)
#JoinColumn(name="ORDER_ID")
private Collection<Item> items = new LinkedHashSet<Item>();
On the Item Side:
#ManyToOne
private Order order;
Test code that works:
order.getItems().add(new Item());
session.save(order)
Order other = (Order) session.get(Order.class, order.getId());
assertEquals(1, other.getItems().size());
While searching through the internet and various sites with tutorials on 1 <---> Many relationship I usually find code such as example: hibernate one to many
The One Side (Stock)
#OneToMany(fetch = FetchType.LAZY, mappedBy = "stock")
public Set<StockDailyRecord> getStockDailyRecords() {
return this.stockDailyRecords;}
The many Side (Stock Record)
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "STOCK_ID", nullable = false)
public Stock getStock() {
return this.stock;}
Why is the JoinColumn Annotation placed differently?The owner of the relationship according to spring code is on the side of the One and not the many. why is that?
Althouth the relationship does not have the mappedBy atttibute at all so one could say that is one direction from order to items it seems bidirectional. i.e order.getItems().add(new Item) and order.getItems().iterator().next().getOrder() works as well!! On top of that the tables created in the actual database seem ok.
If we refactor the first code (see below the refactored) and place the annotations as the tutorials suggest for bidirectional mapping then 2/3 of the tests fail because no items are saved in the database (Assertion error expected 1 but was 0 when using order.getItems().add(new Item())) .Note that the spring tempate test code is left unchanged.Why?
Does it have to do with the annotations being placed on get methods vs annotations placed on private fields?
What is the proper way of saving Items and Order in database (java code)?
Thanks
The code refactored that fails the tests:
The One Side:
#OneToMany(cascade=CascadeType.ALL,mappedBy="order")
private Collection<Item> items = new LinkedHashSet<Item>();
The many side:
#ManyToOne
#JoinColumn(name="ORDER_ID")
private Order order;

#Transactional(readOnly = true) leads to LazyInitializationException

I have a many-to-many relation with an additional column in the link table. I've configured it in a way that the owning side fetches children eager (so I don't get LazyInitializationException) and in the opposite direction it is lazy. This works.
I now wanted to fine-tune the transactions (before there was just #Transactional on class level of DAO and Service classes. I set method getById to readOnly = true:
#Transactional(readOnly = true)
public Compound getById(Long id) {
return compoundDAO.getById(id);
}
After this change I get a LazyInitializationException in following snippet:
Compound compound = compoundService.getById(6L);
Structure structure = compound.getComposition().get(0).getStructure();
System.out.println("StructureId: "+ structure.getId()); // LazyInitializationException
If I remove (readOnly = true) this works! Can anyone explain this behavior? I use Spring + Hibernate. Kind of confusing as I don't see any reason why this should affect which data is loaded?
EDIT:
Snippets of relationship definitions. This is a many-to-many with a column in the link table.
Owning side (eg Compound contains Structures):
#OneToMany(fetch = FetchType.EAGER, mappedBy = "pk.compound",
cascade = CascadeType.ALL, orphanRemoval = true)
#OrderBy("pk.structure.id ASC")
private List<CompoundComposition> composition = new ArrayList<>();
Belongs to side:
#OneToMany(fetch = FetchType.LAZY, mappedBy = "pk.structure",
cascade = CascadeType.ALL)
#OrderBy("pk.compound.id ASC")
private List<CompoundComposition> occurence;
Many-To-One in #Embeddable ID class
#ManyToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL)
public Compound getCompound() {
return compound;
}
#ManyToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL)
public Structure getStructure() {
return structure;
}
EDIT 2:
Stack Trace
org.hibernate.LazyInitializationException: could not initialize proxy - no Session
at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:165) ~[hibernate-core-4.1.7.Final.jar:4.1.7.Final]
at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:272) ~[hibernate-core-4.1.7.Final.jar:4.1.7.Final]
at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:185) ~[hibernate-core-4.1.7.Final.jar:4.1.7.Final]
at org.bitbucket.myName.myApp.entity.Structure_$$_javassist_0.getId(Structure_$$_javassist_0.java) ~[classes/:na]
at org.bitbucket.myName.myApp.App.main(App.java:31) ~[classes/:na]
EDIT 3:
Also see my comment:
Log is very different with readOnly and it is missing the part were the relations are loaded, eg. some selects are missing in the log.
EDIT 4:
So I tired with a basic DriverManagerDataSource and no Connection pool. The issue is exactly the same. For me looks like an issue in Hibernate.
This is just wow. I'm starting to understand why some people hate ORMs...Just feels like I'm constantly having to spend hours to solve a weird issue and the solution is a very specific set of annotations + some code to work around the limitations of said annotations.
First to why this happens (why meaning with which annotations, but not in terms of making logical sense, which is the actual problem here as using common-sense is useless. Only trial and error helps). In the owning side, in #OneToMany I have orphanRemoval = true (which I have found out is required for consistency. one would think database constraints should handle that...just one of the many things that can drive you crazy.). It seems that if the transaction is not read-only, then this setting leads to some data being fetched even so its lazy, namely here:
#ManyToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL)
public Structure getStructure() {
return structure;
}
In a read-only transaction, this fetching does not happen. I would guess because if you can't change anything you will also not have to remove orphans and hence any data that the logic behind this setting requires is not needed in a read-only tx.
So the obvious solution would be in above relation to change to FetchType.EAGER. Wrong! If you do that you will not be able to update the owning side (Compound) using session.merge. This will lead to a StackOverFlowError.
The real solution was actually already mentioned. Just leave the config as is but explicitly load the desired relations in the Service layer:
#Transactional(readOnly = true)
#Override
public Compound getById(Long id) {
Compound compound = compoundDAO.getById(id);
for (CompoundComposition composition : compound.getComposition()){
Hibernate.initialize(composition.getStructure());
}
return compound;
}
I admit I'm tending to fall in the premature optimization trap. This doesn't look very efficient and also seems to break how SQL works in the first place. But then I'm in the lucky position that in most cases CompoundComposition will contain only 1 or 2 elements.
Perhaps you could put
value.getComposition().get(i).getStructure();
in the body of the getById() method, so that the lazy loading happens within the transaction. I realize in this case you'd have to loop over i which might be inconvenient.
Two things :-
Lazy fetch works on Collections Interface. Since ...
#ManyToOne(fetch = FetchType.LAZY, cascade = CascadeType.ALL)
public Structure getStructure() {
return structure;
}
... this is not a collection interface (like List<Structure> would have been), it will be fetched in Eager fetch mode.
Make service method as transactional. It seems that after fetching from the dao layer, your structure is detached with NEVER flush mode. This is the underlying ORM issue I guess.

Resources