I am trying to populate my database, and during this, I create a associations between employees and teams. I'm trying to access the team members on a team via a hibernate OneToMany reference and doing lazy loading. This works great in my app, except when I call it right after the entities are created.
I'm fairly certain this is a hibernate transaction or caching issue, as all of the rows exist in the database, but I'm struggling to understand what the issue is. I have tried converting all of the saves in the populateTeamsAndEmployees function to saveAndFlush, but that did not help.
How can I make team.getTeamMembers() work every time it is called?
Note: This only happens when I call the seedData() function via RequestMapping, but it doesn't happen when I call seedData() with a #Scheduled annotation
Function with Problem
void seedData() {
populateTeamsAndEmployees()
otherBusinessLogic()
}
public void otherBusinessLogic() {
List<Team> teams = teamRepository.findAll();
teams.foreach(team -> {
/*
works great every time, returns all team members
*/
List<Employee> teamMembers = employeeRepository.findEmployeeByTeamId(team.getId());
/*
this returns null when ran immediately after populateTeamsAndEmployees(),
but returns the same as the line above all other times
*/
List<Employee> thisDoesntWork = team.getTeamMembers();
});
}
public void populateTeamsAndEmployees() {
List<EmployeeFromOtherSystem> employeesToConvert = otherSystemRepo.findAll();
employeesToConvert.foreach(otherSysEmployee -> {
employeeRepository.save(otherSysEmployee.toEmployee());
});
}
Entities:
class Team {
#OneToMany(mappedBy = "team", fetch = FetchType.LAZY)
private List<Employee> teamMembers;
}
class Employee {
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "TEAM_ID")
private Team team;
}
So the problem was that JPA was caching the entity right after I created it.
I was able to fix it by calling em.clear()
See below for how to access EntityManager with a JpaRepository:
https://dzone.com/articles/accessing-the-entitymanager-from-spring-data-jpa
And see below for why to use .clear()
EntityManager refresh
Related
I'm currently working on a Spring Boot project for an online shop. It's my first project with Spring Boot (and my first post here), so my coding is not the best.
Context for the questions:
My shop (for now) has a lists of products and whishlists of different users (shopping lists), which have a bidirectional #ManyToMany relation (i left here the relevant details for my question(s)):
Product.java entity:
#Entity
public class Product extends RepresentationModel\<Product\>{
#Id
#GeneratedValue
#JsonView(ProductView.DescriptionExcluded.class)
private Integer id;
#ManyToMany()
#JoinTable(
name = "Shopping_Product",
joinColumns = { #JoinColumn(name = "id", referencedColumnName = "id") },
inverseJoinColumns = { #JoinColumn(name = "list_id", referencedColumnName = "list_id") })
#JsonIgnore
private Set<ShoppingList> shoppinglists = new HashSet<>();
// Constructor, getters, setters ....
ShoppingList.java entity:
#Entity
public class ShoppingList {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#JsonView(ShoppingListView.ProductsExcluded.class)
private Integer list_id;
#JsonView(ShoppingListView.ProductsIncluded.class)
#ManyToMany(mappedBy = "shoppinglists")
private Set<Product> products = new HashSet<>();
// Constructor, getters, setters ...
I chose Product as the owner because i wanted to delete (tho it would be more fit to show something like "offer expired", but I'll stick to delete for now) the product from all existing lists when the admin takes it down from the shop, which works as expected:
ProductResource.java (controller):
#DeleteMapping("/categs/*/sub/*/products/{id}")
public ResponseEntity<String> deleteProduct(#PathVariable int id) {
Optional<Product> optional = productRepository.findById(id);
if(!optional.isPresent()) throw new NotFoundException("Product id - " + id);
Product prod = optional.get();
productRepository.delete(prod);
return ResponseEntity.ok().body("Product deleted");
}
My problems now are related to the ShoppingList entity, which is not the owner.
Any call I make to the Product resource (controller) works as expected, but anything from the other side either fails or returns incomplete results, like the following:
1.
I call retrieve all products from a list and it returns only the first object (the list has at least 2):
ShoppingListResource.java (controller):
#RestController
public class ShoppingListResource {
#Autowired
private ProductRepository productRepository;
#Autowired
private UserRepository userRepository;
#Autowired
private ShoppingListRepository shoppinglistRepository;
#GetMapping("/user/lists/{id}")
public Set<Product> getShoppinglistProducts(#PathVariable int id) {
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
String currentPrincipalName = authentication.getName();
ShoppingList shoppingList = shoppinglistRepository.findById(id).get();
String name = shoppingList.getUser().getUsername();
if(!Objects.equals(currentPrincipalName, name)) throw new IllegalOperation("You can only check your list(s)!");
// All lists are shown for a product
// Product p = productRepository.findById(10111).get();
// Set<ShoppingList> set = p.getShoppinglists();
// set.stream().forEach(e -> log.info(e.toString()));
// Only first product is shown for a list
return shoppingList.getProducts();
This is what hibernate does on the last row (only returns 1/2 products)
Hibernate: select products0_.list_id as list_id2_3_0_,
products0_.id as id1_3_0_,
product1_.id as id1_1_1_,
product1_.description as descript2_1_1_,
product1_.name as name3_1_1_,
product1_.price as price4_1_1_,
product1_.subcat_id as subcat_i5_1_1_ from shopping_product products0_ inner join product product1_ on products0_.id=product1_.id where products0_.list_id=?
As i said above, I can delete a product and it gets removed automatically from all existing lists, but when i try the same from ShoppingList entity does nothing:
Same controller
#DeleteMapping("/user/lists/{id}")
public ResponseEntity<String> deleteShoppinglist(#PathVariable int id) {
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
String currentPrincipalName = authentication.getName();
ShoppingList shoppingList = shoppinglistRepository.findById(id).get();
String name = shoppingList.getUser().getUsername();
if(!Objects.equals(currentPrincipalName, name)) throw new IllegalOperation("You can only delete your list(s)!");
shoppinglistRepository.delete(shoppingList);
return ResponseEntity.ok().body("Shopping list deleted");
}
Also, when i try to add/delete product from an existing list, does nothing.
This is my repo with full code, if you'd like to test directly (dev branch is up to date):
https://github.com/dragostreltov/online-store/tree/dev
You can just use admin admin as authentication (on the H2 console too). More details on the readme.
All DB data at app start is inserted from a .sql file.
I checked other similar questions and tried different methods on my ShoppingList entity (on the delete issue), like:
#PreRemove
public void removeListsFromProducts() {
for(Product p : products) {
p.getShoppinglists().remove(this);
}
}
Spring/Hibernate: associating from the non-owner side
And still doesn't work.
UPDATE:
I found out what issues I was having, I'll post an answer with the solution.
For anyone who's got the same/similar problems as I did, this is how I resolved them:
For point 1
(Hibernate only retrieves the first product from a shoppingList (Set))
I made multiple tests on my retrieve method and found out my Set was only containing 1 object, despite calling .add(product) twice.
As you can see, I'm using HashSet for both entities:
In Product (owner):
private Set<ShoppingList> shoppinglists = new HashSet<>();
In ShoppingList (mappedBy):
private Set<Product> products = new HashSet<>();
Thanks to this answer: https://stackoverflow.com/a/16344031/18646899
I learnt:
HashSet (entirely reasonably) assumes reflexivity, and doesn't check for equality when it finds that the exact same object is already in the set, as an optimization. Therefore it will not even call your equals method - it considers that the object is already in the set, so doesn't add a second copy.
In particular, if x.equals(x) is false, then any containment check would also be useless.
Taking this into account, I overwrote the hashCode() and equals() methods in Product.class and now
shoppingList.getProducts()
works as expected.
For point 2
(not being able to delete associations of non-owner entity before deleting the row from it's table)
Added lazy fetch and cascade to Product #ManyToMany:
#ManyToMany(fetch = FetchType.LAZY, cascade = {CascadeType.PERSIST, CascadeType.MERGE, CascadeType.DETACH})
And added the following methods:
In Product class:
public void addShoppinglist(ShoppingList list) {
this.shoppinglists.add(list);
list.getProducts().add(this);
}
public void removeShoppinglist(ShoppingList list) {
this.shoppinglists.remove(list);
list.getProducts().remove(this);
}
In ShoppingList class:
public void addProduct(Product product) {
this.products.add(product);
product.getShoppinglists().add(this);
}
public void removeProduct(Product product) {
this.products.remove(product);
product.getShoppinglists().remove(this);
}
Added #Transactional and modified the method inside the controller (ShoppingListResource) for deleteShoppingList:
#RestController
public class ShoppingListResource {
...
#Transactional
#DeleteMapping("/user/lists/{id}")
public ResponseEntity<String> deleteShoppinglist(#PathVariable int id) {
...
shoppingList.getProducts().stream().forEach(e -> {
e.removeShoppinglist(shoppingList);
});
shoppinglistRepository.delete(shoppingList);
return ResponseEntity.ok().body("Shopping list deleted");
}
}
And now this is working as expected, the shoppingList's associations are deleted first then the shoppingList itself.
I'm observing a kind of strange behavior in my Spring application. Unfortunately I cannot share the complete code, but basically this is what it looks like:
// the repository
#Repository
public interface InboxRepo extends JpaRepository<Inbox, Long> {}
// the service
#Transactional
public void deleteInbox(long id) {
inboxRepo.deleteById(id);
}
When calling deleteInbox(), there is no exception or any kind of error but the Inbox item is not deleted from the database. Setting spring.jpa.show-sql=true shows that there isn't even a DELETE statement, i.e. for whatever reason, the code doesn't actually issue the deletion.
When defining a derived delete method in the repository, then the deletion works, but it doesn't yet make sense to me:
#Repository
public interface InboxRepo extends JpaRepository<Inbox, Long> {
// this seems to work
#Modifying
#Query("delete from Inbox i where i.id = ?1")
void delete(long id);
}
Dleting directly via an EntityManager also works. But what could be the reason that the "standard" JpaRepository methods don't work here?
I found the root cause. There was another entity having a reference to Inbox like this:
#OneToMany(mappedBy = "inbox", cascade = ALL, fetch = FetchType.EAGER)
private Set<Inbox> inbox = new HashSet<>();
The FetchType.EAGER in combination with the cascade caused the problem, i.e. as soon as the Inbox was deleted, this reference caused the Inbox to get "re-persisted". Setting FetchType.LAZY resolved the problem.
I have a set of simple models like this (getters and setters omitted for brevity):
#Entity
public class Customer {
#Id
private Integer id;
}
#Entity
public class Order {
#Id
private Integer id;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "customer_id")
private Customer customer;
}
I am trying to load an Order using a Spring JPA repository with a findById method, including the customer.
First I tried this:
#Transactional
Optional<Order> findById(Integer id) {
return repository.findById(id);
}
But when I tried to access Customer I got a LazyInitializationException: could not initialize proxy - no Session. So after referring to some other questions, I updated my method to be a bit uglier, but to explicitly call Hibernate.initialize:
#Transactional
Optional<Order> findById(Integer id) {
return repository.findById(id)
.map( order -> {
Hibernate.initialize(order.getCustomer());
return order;
);
}
But I still get org.hibernate.LazyInitializationException: could not initialize proxy - no Session. repository is a regular CrudRepository which provides the findById method out-of-the-box.
How can I initialize this lazily loaded child entity? My understanding is that the #Transactional indicates that I should still be within the transaction for the entirety of this method call. The only thing further downstream is the repository itself, which is just an interface, so I'm not sure how else to go about forcing the load of this child entity.
The Order entity and everything else in it is retrieved properly from the database; it's only when I try to get the lazy-loaded child entities that we start having issues.
The only way I managed to get this working was to write a custom hql method in the Repository using a left join fetch. While that works, it clutters up my repository with a method that is pretty much a duplicate of another and which I'm pretty sure I'm not actually supposed to need (so I would rather not do it this way.)
Spring-Boot 2.1.4.RELEASE, Spring 5.1.6.RELEASE, Hibernate 5.3.7.Final.
You have to define the method as public. See "Method visibility and #Transactional" in the spring docs.
This should work:
#Transactional
public Optional<Order> findById(Integer id) {
Optional<Order> order = repository.findById(id);
order.ifPresent(o -> Hibernate.initialize(o.getCustomer()));
return order;
}
I am currently in the middle of migrating my REST-Server to GraphQL (at least partly). Most of the work is done, but i stumbled upon this problem which i seem to be unable to solve: OneToMany relationships in a graphql query, with FetchType.LAZY.
I am using:
https://github.com/graphql-java/graphql-spring-boot
and
https://github.com/graphql-java/graphql-java-tools for the integration.
Here is an example:
Entities:
#Entity
class Show {
private Long id;
private String name;
#OneToMany(mappedBy = "show")
private List<Competition> competition;
}
#Entity
class Competition {
private Long id;
private String name;
#ManyToOne(fetch = FetchType.LAZY)
private Show show;
}
Schema:
type Show {
id: ID!
name: String!
competitions: [Competition]
}
type Competition {
id: ID!
name: String
}
extend type Query {
shows : [Show]
}
Resolver:
#Component
public class ShowResolver implements GraphQLQueryResolver {
#Autowired
private ShowRepository showRepository;
public List<Show> getShows() {
return ((List<Show>)showRepository.findAll());
}
}
If i now query the endpoint with this (shorthand) query:
{
shows {
id
name
competitions {
id
}
}
}
i get:
org.hibernate.LazyInitializationException: failed to lazily initialize
a collection of role: Show.competitions, could not initialize proxy -
no Session
Now i know why this error happens and what it means, but i don't really know were to apply a fix for this. I don't want to make my entites to eagerly fetch all relations, because that would negate some of the advantages of GraphQL. Any ideas where i might need to look for a solution?
Thanks!
My prefered solution is to have the transaction open until the Servlet sends its response. With this small code change your LazyLoad will work right:
import javax.servlet.Filter;
import org.springframework.orm.jpa.support.OpenEntityManagerInViewFilter;
#SpringBootApplication
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
/**
* Register the {#link OpenEntityManagerInViewFilter} so that the
* GraphQL-Servlet can handle lazy loads during execution.
*
* #return
*/
#Bean
public Filter OpenFilter() {
return new OpenEntityManagerInViewFilter();
}
}
I solved it and should have read the documentation of the graphql-java-tools library more carefully i suppose.
Beside the GraphQLQueryResolver which resolves the basic queries i also needed a GraphQLResolver<T> for my Showclass, which looks like this:
#Component
public class ShowResolver implements GraphQLResolver<Show> {
#Autowired
private CompetitionRepository competitionRepository;
public List<Competition> competitions(Show show) {
return ((List<Competition>)competitionRepository.findByShowId(show.getId()));
}
}
This tells the library how to resolve complex objects inside my Showclass and is only used if the initially query requests to include the Competitionobjects. Happy new Year!
EDIT 31.07.2019: I since stepped away from the solution below. Long running transactions are seldom a good idea and in this case it can cause problems once you scale your application. We started to implement DataLoaders to batch queries in an async matter. The long running transactions in combination with the async nature of the DataLoaders can lead to deadlocks: https://github.com/graphql-java-kickstart/graphql-java-tools/issues/58#issuecomment-398761715 (above and below for more information). I will not remove the solution below, because it might still be good starting point for smaller applications and/or applications which will not need any batched queries, but please keep this comment in mind when doing so.
EDIT: As requested here is another solution using a custom execution strategy. I am using graphql-spring-boot-starter and graphql-java-tools:
Create a Bean of type ExecutionStrategy that handles the transaction, like this:
#Service(GraphQLWebAutoConfiguration.QUERY_EXECUTION_STRATEGY)
public class AsyncTransactionalExecutionStrategy extends AsyncExecutionStrategy {
#Override
#Transactional
public CompletableFuture<ExecutionResult> execute(ExecutionContext executionContext, ExecutionStrategyParameters parameters) throws NonNullableFieldWasNullException {
return super.execute(executionContext, parameters);
}
}
This puts the whole execution of the query inside the same transaction. I don't know if this is the most optimal solution, and it also already has some drawbacks in regards to error handling, but you don't need to define a type resolver that way.
Notice that if this is the only ExecutionStrategy Bean present, this will also be used for mutations, contrary to what the Bean name might suggest. See https://github.com/graphql-java-kickstart/graphql-spring-boot/blob/v11.1.0/graphql-spring-boot-autoconfigure/src/main/java/graphql/kickstart/spring/web/boot/GraphQLWebAutoConfiguration.java#L161-L166 for reference. To avoid this define another ExecutionStrategy to be used for mutations:
#Bean(GraphQLWebAutoConfiguration.MUTATION_EXECUTION_STRATEGY)
public ExecutionStrategy queryExecutionStrategy() {
return new AsyncSerialExecutionStrategy();
}
For anyone confused about the accepted answer then you need to change the java entities to include a bidirectional relationship and ensure you use the helper methods to add a Competition otherwise its easy to forget to set the relationship up correctly.
#Entity
class Show {
private Long id;
private String name;
#OneToMany(cascade = CascadeType.ALL, mappedBy = "show")
private List<Competition> competition;
public void addCompetition(Competition c) {
c.setShow(this);
competition.add(c);
}
}
#Entity
class Competition {
private Long id;
private String name;
#ManyToOne(fetch = FetchType.LAZY)
private Show show;
}
The general intuition behind the accepted answer is:
The graphql resolver ShowResolver will open a transaction to get the list of shows but then it will close the transaction once its done doing that.
Then the nested graphql query for competitions will attempt to call getCompetition() on each Show instance retrieved from the previous query which will throw a LazyInitializationException because the transaction has been closed.
{
shows {
id
name
competitions {
id
}
}
}
The accepted answer is essentially
bypassing retrieving the list of competitions through the OneToMany relationship and instead creates a new query in a new transaction which eliminates the problem.
Not sure if this is a hack but #Transactional on resolvers doesn't work for me although the logic of doing that does make some sense but I am clearly not understanding the root cause.
For me using AsyncTransactionalExecutionStrategy worked incorrectly with exceptions. E.g. lazy init or app-level exception triggered transaction to rollback-only status. Spring transaction mechanism then threw on rollback-only transaction at the boundary of strategy execute, causing HttpRequestHandlerImpl to return 400 empty response. See https://github.com/graphql-java-kickstart/graphql-java-servlet/issues/250 and https://github.com/graphql-java/graphql-java/issues/1652 for more details.
What worked for me was using Instrumentation to wrap the whole operation in a transaction: https://spectrum.chat/graphql/general/transactional-queries-with-spring~47749680-3bb7-4508-8935-1d20d04d0c6a
I am assuming that whenever you fetch an object of Show, you want all the associated Competition of the Show object.
By default the fetch type for all collections type in an entity is LAZY. You can specify the EAGER type to make sure hibernate fetches the collection.
In your Show class you can change the fetchType to EAGER.
#OneToMany(cascade=CascadeType.ALL,fetch=FetchType.EAGER)
private List<Competition> competition;
You just need to annotate your resolver classes with #Transactional. Then, entities returned from repositories will be able to lazily fetch data.
I'm trying to save an Entity in DB using Spring Data/Crud Repository(.save) that has in it another entity that was loaded through a #Cache method. In other words, I am trying to save an Ad Entity that has Attributes entities in it, and those attributes were loaded using Spring #Cache.
Because of that, I'm having a Detached Entity Passed to Persist Exception.
My question is, is there a way to save the entity still using #Cache for the Attributes?
I looked that up but couldn't find any people doing the same, specially knowing that I am using CrudRepository that has only the method .save(), that as far as I know manages Persist, Update, Merge, etc.
Any help is very much appreciated.
Thanks in advance.
Ad.java
#Entity
#DynamicInsert
#DynamicUpdate
#Table(name = "ad")
public class Ad implements SearchableAdDefinition {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#ManyToOne(fetch = FetchType.LAZY, optional = false)
private User user;
#OneToMany(mappedBy = "ad", fetch = FetchType.LAZY, cascade = CascadeType.ALL)
private Set<AdAttribute> adAttributes;
(.....) }
AdAttribute.java
#Entity
#Table(name = "attrib_ad")
#IdClass(CompositeAdAttributePk.class)
public class AdAttribute {
#Id
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "ad_id")
private Ad ad;
#Id
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "attrib_id")
private Attribute attribute;
#Column(name = "value", length = 75)
private String value;
public Ad getAd() {
return ad;
}
public void setAd(Ad ad) {
this.ad = ad;
}
public Attribute getAttribute() {
return attribute;
}
public void setAttribute(Attribute attribute) {
this.attribute = attribute;
}
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
}
#Embeddable
class CompositeAdAttributePk implements Serializable {
private Ad ad;
private Attribute attribute;
public CompositeAdAttributePk() {
}
public CompositeAdAttributePk(Ad ad, Attribute attribute) {
this.ad = ad;
this.attribute = attribute;
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
CompositeAdAttributePk compositeAdAttributePk = (CompositeAdAttributePk) o;
return ad.getId().equals(compositeAdAttributePk.ad.getId()) && attribute.getId().equals(compositeAdAttributePk.attribute.getId());
}
#Override
public int hashCode() {
return Objects.hash(ad.getId(), attribute.getId());
}
}
Method using to load Attributes:
#Cacheable(value = "requiredAttributePerCategory", key = "#category.id")
public List<CategoryAttribute> findRequiredCategoryAttributesByCategory(Category category) {
return categoryAttributeRepository.findCategoryAttributesByCategoryAndAttribute_Required(category, 1);
}
Method used to create/persist the Ad:
#Transactional
public Ad create(String title, User user, Category category, AdStatus status, String description, String url, Double price, AdPriceType priceType, Integer photoCount, Double minimumBid, Integer options, Importer importer, Set<AdAttribute> adAtributes) {
//Assert.notNull(title, "Ad title must not be null");
Ad ad = adCreationService.createAd(title, user, category, status, description, url, price, priceType, photoCount, minimumBid, options, importer, adAtributes);
for (AdAttribute adAttribute : ad.getAdAttributes()) {
adAttribute.setAd(ad);
/* If I add this here, I don't face any exception, but then I don't take benefit from using cache:
Attribute attribute = attributeRepository.findById(adAttribute.getAttribute().getId()).get();
adAttribute.setAttribute(attribute);
*/
}
ad = adRepository.save(ad);
solrAdDocumentRepository.save(AdDocument.adDocumentBuilder(ad));
return ad;
}
I don't know if you still require this answer or not, since it's a long time, you asked this question. Yet i am going to leave my comments here, someone else might get help from it.
Lets assume, You called your findRequiredCategoryAttributesByCategory method, from other part of your application. Spring will first check at cache, and will find nothing. Then it will try to fetch it from Database. So it will create an hibernate session, open a transaction, fetch the data, close the transaction and session. Finally after returning from the function, it will store the result set in cache for future use.
You have to keep in mind, those values, currently in the cache, they are fetched using a hibernate session, which is now closed. So they are not related to any session, and now at detached state.
Now, you are trying to save and Ad entity. For this, spring created a new hibernate session, and Ad entity is attached to this particular session. But the attributes object, that you fetched from the Cache are detached. That's why, while you are trying to persist Ad entity, you are getting Detached Entity Exception
To resolve this issue, you need to re attach those objects to current hibernate session.I use merge() method to do so.
From hibernate documentation here https://docs.jboss.org/hibernate/orm/3.5/javadocs/org/hibernate/Session.html
Copy the state of the given object onto the persistent object with the same identifier. If there is no persistent instance currently associated with the session, it will be loaded. Return the persistent instance. If the given instance is unsaved, save a copy of and return it as a newly persistent instance. The given instance does not become associated with the session. This operation cascades to associated instances if the association is mapped with cascade="merge".
Simply put, this will attach your object to hibernate session.
What you should do, after calling your findRequiredCategoryAttributesByCategory method, write something like
List attributesFromCache = someService.findRequiredCategoryAttributesByCategory();
List attributesAttached = entityManager.merge( attributesFromCache );
Now set attributesAttached to your Ad object. This won't throw exception as attributes list is now part of current Hibernate session.