How to handle Spring Boot/ Spring Data projections with entity relationships (nested projection) - spring

I'm trying to get nested projections working in Spring Boot. I have 2 entities, Parent and Child, wheras Parent has a unidirectional #OneToMany relationship to Child.
Here are the classes: (using Lombok-Annotations)
#Entity
#Data #NoArgsConstructor
public class Parent {
#Id
#GeneratedValue
private long id;
private String basic;
private String detail;
#OneToMany(fetch = FetchType.EAGER)
private List<Child> children;
public Parent(String basic, String detail, List<Child> children) {
this.basic = basic;
this.detail = detail;
this.children = children;
}
}
#Entity
#Data #NoArgsConstructor
public class Child {
#Id
#GeneratedValue(strategy = GenerationType.TABLE)
private long id;
private String basic;
private String detail;
public Child(String basic, String detail) {
this.basic = basic;
this.detail = detail;
}
}
When im fetching the data without projecting i get the following:
[
{
"id": 1,
"basic": "parent-basic-1",
"detail": "parent-detail-1",
"children": [
{
"id": 1,
"basic": "child-basic-1",
"detail": "child-detail-1"
},
{
"id": 2,
"basic": "child-basic-2",
"detail": "child-detail-2"
}
]
},
{
"id": 2,
"basic": "parent-basic-2",
"detail": "parent-detail-2",
"children": [
{
"id": 3,
"basic": "child-basic-3",
"detail": "child-detail-3"
},
{
"id": 4,
"basic": "child-basic-4",
"detail": "child-detail-4"
}
]
}
and the goal would be the following:
{
"id": 1,
"basic": "parent-basic-1",
"children": [1,2]
},
{
"id": 2,
"basic": "parent-basic-2",
"children": [3,4]
}
However it seems completly impossible to achive this.
So far I've tried Constructor Projection:
#Value
public class ParentDto {
long id;
String basic;
// wanted to get it to work with just Child instead of ChildDto first, before getting ChildDto to work
Collection<Child> children;
public ParentDto(long id, String basic, Collection<Child> children) {
this.id = id;
this.basic = basic;
this.children = children;
}
}
// Constructor Projection in Repository
#Query("select new whz.springbootdemo.application.constructor_projection.ParentDto(p.id, p.basic, p.children) from Parent p")
List<ParentDto> findAllConstructorProjected();
but that leads to the following error:
could not prepare statement; SQL [select parent0_.id as col_0_0_, parent0_.basic as col_1_0_, . as col_2_0_ from parent parent0_ inner join parent_children children1_ on parent0_.id=children1_.parent_id inner join child child2_ on children1_.children_id=child2_.id]; nested exception is org.hibernate.exception.SQLGrammarException: could not prepare statement
Trying Dynamic Projection:
// Dynamic Projection in Repository
List<ParentDto> findAllDynamicProjectionBy();
leads to the following error:
org.hibernate.hql.internal.ast.QuerySyntaxException:
Unable to locate appropriate constructor on class [whz.springbootdemo.application.constructor_projection.ParentDto].
Expected arguments are: <b>long, java.lang.String, whz.springbootdemo.application.child.Child</b>
[select new whz.springbootdemo.application.constructor_projection.ParentDto(generatedAlias0.id, generatedAlias0.basic, children) from whz.springbootdemo.application.parent.Parent as generatedAlias0 left join generatedAlias0.children as children]; nested exception is java.lang.IllegalArgumentException: org.hibernate.hql.internal.ast.QuerySyntaxException: Unable to locate appropriate constructor on class [whz.springbootdemo.application.constructor_projection.ParentDto]. Expected arguments are: long, java.lang.String, whz.springbootdemo.application.child.Child [select new whz.springbootdemo.application.constructor_projection.ParentDto(generatedAlias0.id, generatedAlias0.basic, children) from whz.springbootdemo.application.parent.Parent as generatedAlias0 left join generatedAlias0.children as children]
which basically tells me that a join is executed, but the values arent grouped by the id of the parent, thus resulting in x rows, where x is the number of childs the parents has, each with the parents basic information and one of its childs information.
The only thing "working" is Interface Projection:
// Interface Projection in Repository
List<ParentDtoInterface> findAllInterfaceProjectedBy();
public interface ParentDtoInterface {
long getId();
String getBasic();
List<ChildDtoInterface> getChildren();
}
public interface ChildDtoInterface {
long getId();
}
It results in:
[
{
"id": 1,
"children": [
{
"id": 1
},
{
"id": 2
}
],
"basic": "parent-basic-1"
},
{
"id": 2,
"children": [
{
"id": 3
},
{
"id": 4
}
],
"basic": "parent-basic-2"
}
]
Now my problem with Interface-Projection is, that it will not just load the expected properties, but all properties, but jackson will only serialize those that the Interface provides, cause it uses the Class/Interface-Definition.
Parent loaded: (sql log; see line 4, detail information is loaded)
select
parent0_.id as id1_1_,
parent0_.basic as basic2_1_,
parent0_.detail as detail3_1_
from
parent parent0_
Also Interface Projection seems to be really slow (see this Stackoverflow question) and i still would have to unpack the children cause they are given as [{id:1},{id:2}] but i really need [1,2]. I know i can do this with #JsonIdentityReference(alwaysAsId = true) but thats just a workaround.
Also I'm abit confused why the data is loaded in n+1 queries - 1 for the parents, and another n (where n is the number of parents) for each parents childs:
select
parent0_.id as id1_1_,
parent0_.basic as basic2_1_,
parent0_.detail as detail3_1_
from
parent parent0_
select
children0_.parent_id as parent_i1_2_0_,
children0_.children_id as children2_2_0_,
child1_.id as id1_0_1_,
child1_.basic as basic2_0_1_,
child1_.detail as detail3_0_1_
from
parent_children children0_
inner join
child child1_
on children0_.children_id=child1_.id
where
children0_.parent_id=?
//... omitting further child queries
I have tried #OneToMany(fetch=FetchType.LAZY) and #Fetch(FetchType.JOINED) - both give the same result as above.
So the main question is: Is there any way to achive projection with Spring Boot for nested entities, so that only the needed data is loaded in as little as possible queries and in a best case scenario I can adjust it so that instead of having to load List children i can just load List childIds (maybe through a Jpa query that groups the joined rows by parentid and lets be extract needed data from the Child?).
Im using Hibernate and an In-Memory Database.
Thanks in regards for any answer or tip!
Edit: To clarify: I'm not trying to find a way to serialize the data in the wanted format - this i already can achive. The main focus is on only loading the neccessary information from the database.

this will always fetch the children but could give you the result you want.
public interface SimpleParentProjection {
String getBasic();
String getDetail();
#Value("#{T(SimpleParentProjection).toId(target.getChildren())}")
String[] getChildren();
static String[] toId(Set<Child> childSet) {
return childSet.stream().map(c -> String.valueOf(c.getId())).toArray(String[]::new);
}
}

Related

RestAPI response shows duplicate data

I'm trying to write a simple application with a One-to-Many association. When I fetch the Author, I repated data multiple times in the Postman response. Below are my entities and mapping.
#Entity
public class Books extends AbstractEntity implements Serializable{
// properties
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "auther_number", referencedColumnName="auther_id")
private Author author;
// get/set goes here.
}
#Entity
public class Author extends AbstractEntity implements Serializable{
// properties for autherID, name etc.
#OneToMany(mappedBy = "author",cascade = CascadeType.ALL,orphanRemoval = true, fetch = FetchType.LAZY)
private List<Books> bookList;
// to avoid synchronization issues. in 1-M Bi-direactional
public void addBooks(Books book) {
booklist.add(book);
book.setAuther(this);
}
// to avoid synchronization issues. in 1-M Bi-direactional
public void removeBooks(Books book) {
booklist.remove(book);
book.setAuthor(null);
}
// equals and hashcode methods
}
AutherserviceImpl.java
#Override
public List<Author> getAllAuthors() {
List<Author> authorList = (list<Author>) authorRepo.findAll();
return authorList ;
}
RestController
#GetMapping("/api/authors")
public ResponseEntity<Object> findAllAuthors(){
return new ResponseEntity<>(autherserviceImpl.getAllAuthors(),
HttpStatus.OK);
}
Below is the output in postman. Why is it duplicating? I have followed the samples given by this.
"authorNo": 4575600302,
"balance": 4458.0,
"books": [
{
"bookID": 3522,
"price": 458.0,
"ISBN": "1234",
"author": {
"authorNo": 4575600302,
"balance": 4458.0,
"books": [
{
"bookID": 3522,
"price": 458.0,
"ISBN": "1234",
"author": {
"authorNo": 4575600302,
"balance": 4458.0,
"books": [
{
"bookID": 3522,
"price": 458.0,
"ISBN": "1234",
"author": {
"authorNo": 4575600302,
"balance": 4458.0,
"books": [
{
Some questions in StackOverflow have suggested using Set instead of the list. But when I use Set, I get a casting error between Set and List. Not sure where is it throuwing exactly to fix the casting error. as I do not see any stack trace but only in the postman response I get that error.
How can I resolve this duplicate data showing issue? Note that in the Database has no duplicate records.
The problem is not JPA but the serialization of the author instance by Jackson. You have a bidirectional relationship between the authors and the books, i.e. Jackson serializes all books of an author and when it serializes a book, it will start the process of serializing the respective author again.
The simplest solution is to annotate the field author of Books with #JsonIgnore. Alternatively, you can annotate author with #JsonManagedReference and the field bookList of Author with #JsonBackReference. Then, the deserialization circle should be broken.
For a detailed guide, please have a look here: https://www.baeldung.com/jackson-bidirectional-relationships-and-infinite-recursion

How to show data to user with DTO

This is the data that I am currently showing to the user:
{
"id": 3,
"name": "AB:11",
"description": "AB:11 is an Imperial Black Barley Wine brewed with ginger, black raspberries and chipotle peppers. A 12.8% rollercoaster of ginger zestiness and chipotle smokiness, all bound together with dark berry tartness and the decadent residual body of a Black Barley Wine.",
"method": {
"mash_temp": [
{
"temp": {
"value": 68,
"unit": "celsius"
}
}
]
}
And I don't need this "method" field. I tried to show data with DTO which looks like this:
public class Beer {
private Integer id;
private String name;
private String description;
private Method method;
private List<MashTemp> mashTemp;
private Temp temp;
// getters & setters
My DTO is giving me back a response like a:
"id": 1,
"name": "Bitch Please (w/ 3 Floyds)",
"description": "This limited edition American Barley Wine was brewed in collaboration with 3 Floyds Brewery. This beer had all the warm, boozy and smoky aspects of an Islay Scotch whisky with the sweet malt and devastatingly bitter attributes of a barley wine. Peat smoke features prominently, backed up with a complex fruity hop profile.",
"method": {
"mash_temp": [
{
"temp": {
"value": 65
}
}
]
},
"mashTemp": null,
"temp": null
Any Idea how to fix this?
You need the following classes:
public class BeerDto {
private Integer id;
private String name;
private String description;
private List<MashTempDto> mashTemp;
}
public class MashTempDto {
private TempDto temp;
}
public class TempDto {
private Integer value;
}
Now you need to map your original Beer object to BeerDto (the same is true for MashTempDto and TempDto) which you will then make available in your API.
I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(Beer.class)
public interface BeerDto {
#IdMapping
Integer getId();
String getName();
String getDescription();
Set<MashTempDto> getMashTemp();
#EntityView(MashTemp.class)
interface MashTempDto {
TempDto getTemp();
}
#EntityView(Temp.class)
interface TempDto {
Integer getValue();
}
}
Querying is a matter of applying the entity view to a query, the simplest being just a query by id.
BeerDto a = entityViewManager.find(entityManager, BeerDto.class, id);
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
Page<BeerDto> findAll(Pageable pageable);
The best part is, it will only fetch the state that is actually necessary!

spring-data-redis, empty list attribute value becomes null

I'm in the process of porting some microservices from SpringBoot1.5 to 2.1.
We are using spring-data-redis. it seems the default internal moves from jedis to lettuce.
The thing is we now observe some weird behaviours, when we save an object and then retrieve it, there is a tiny difference:
empty list attributes are replaced with null.
Here is an example:
//repo
public interface TestRepository extends CrudRepository<Test, String> {}
...
//object
#RedisHash(timeToLive = 60)
public static class Test{
#Id private String id;
int age;
List<String> friends;
}
...
//saving then retreiving
Test test = new Test("1", 15, Collections.emptyList());
System.out.println(test);
testRepository.save(test);
Test testGet = testRepository.findById("1").get();
System.out.println(testGet);
and here is what happens:
//before
{
"id": "1",
"age": 15,
"friends": []
}
//after
{
"id": "1",
"age": 15
}
the friends empty list has disappeared. This new behaviour affects our code in many places leading to NullPointerExceptions etc.
Apparently, there are multiple serializers available but this doesn't seem to have any effect. Any idea?
https://docs.spring.io/spring-data/data-redis/docs/current/reference/html/#redis:serializer
for reference:
springBootVersion = '2.1.5.RELEASE'
springCloudVersion = 'Greenwich.SR1'
I met this problem too. I solved it like this:
#RedisHash(timeToLive = 60)
public class MyData implements Serializable {
#Id
private String id;
private List<Object> objects = new ArrayList<>();
}
If i will save MyData with empty list objects, when i pull it from Redis, objects in it will not be null and will be empty list. If i will save 'MyData' with not empty objects, objects not will be lost after deserialization.

How to properly design RestController for spring REST API with JPA?

I don't know how to properly design the architecture for a movie database web app backend.
I have a Movie Entity which has a list of Genre Entity and a Map of Actor Entitiys with their role in the movie:
// ...
#Data
#Entity
#NoArgsConstructor(access = AccessLevel.PRIVATE)
public class Movie {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String title;
private String director;
private Date releaseDate;
private Long posterId;
#ManyToMany
#JoinTable(
name = "MOVIE_GENRES",
joinColumns = #JoinColumn(name = "MOVIE_ID"),
inverseJoinColumns = #JoinColumn(name = "GENRE_ID"))
private Set<Genre> genres = new HashSet<>();
// TODO how to rename value column (CAST -> ACTOR_ID)
#OneToMany
#MapKeyColumn(name = "ACTOR_ROLE")
private Map<String, Actor> cast = new HashMap<>();
// ...
}
I also have a REST Controller for Movies:
#RestController
public class MovieController {
private MovieRepository repository;
public MovieController(MovieRepository repository) {
this.repository = repository;
}
#GetMapping("/api/movies")
public List<Movie> all() {
return repository.findAll();
}
#PostMapping("/api/movies")
Movie newMovie(#RequestBody Movie newMovie) {
return repository.save(newMovie);
}
#GetMapping("/api/movies/{id}")
Movie one(#PathVariable Long id) {
return repository.findById(id)
.orElseThrow(() -> new MovieNotFoundException(id));
}
#PutMapping("/api/movies/{id}")
Movie replaceMovie(#RequestBody Movie newMovie, #PathVariable Long id) {
return repository.findById(id)
.map(movie -> {
movie.setTitle(newMovie.getTitle());
movie.setDirector(newMovie.getDirector());
movie.setReleaseDate(newMovie.getReleaseDate());
movie.setGenres(newMovie.getGenres());
movie.setCast(newMovie.getCast());
return repository.save(movie);
})
.orElseGet(() -> {
newMovie.setId(id);
return repository.save(newMovie);
});
}
#DeleteMapping("/api/movies/{id}")
void deleteMovie(#PathVariable Long id) {
repository.deleteById(id);
}
}
This is how it looks when I call /api/movies. I receive all the genre and cast information too, for every movie. Is this okay? I don't even need all this information when getting a list of all movies.
If I follow REST principles, shouldn't I get the cast via /api/movies/{id}/cast? I know how to add another #RestMapping that returns only the cast but it doesn't change the fact that the cast will still be included in every /api/movies call.
{
"id": 1,
"title": "The Matrix",
"director": null,
"releaseDate": null,
"posterId": 1,
"genres": [
{
"id": 4,
"name": "Science Fiction"
},
{
"id": 1,
"name": "Action"
}
],
"cast": {
"Agent Smith": {
"id": 3,
"name": "Hugo Weaving",
"gender": "MALE",
"dateOfBirth": "1960-04-04"
},
"Morpheus": {
"id": 2,
"name": "Laurence Fishburne",
"gender": "MALE",
"dateOfBirth": "1961-07-30"
},
"Thomas A. Anderson / Neo": {
"id": 1,
"name": "Keanu Reeves",
"gender": "MALE",
"dateOfBirth": "1964-09-02"
}
}
}
Good question: You should do a little more of a "deep dive" on this to understand what's going on. It's not a matter of REST design in this case. I don't know if you were surprised to see the cast included with movie information but you should be.
Your code return repository.findById(id) … seems clearly intended to retrieve only movie information but you notice you are also getting the cast. Did you do print the sql statements from the Spring Data Jpa system to see if it was acting the way you expected? I suspect not, because if you had you would probably have noticed that several sql statements are being generated. The first for the movie and then subsequent statements for the cast.
Once you traced down why you are getting multiple sql statements you will find that the statements are coming from the Entity->JSON conversion. This means that when your service returns the Movie entity the spring framework needs to convert it to JSON to send it over the wire and that code is walking the object graph. The object graph are the instances of the entities you have created in the JVM when you queried the database. Since you have mapped the Movie/Cast relationship the movie object includes possible references to the cast and when the JSON transform code does a get on the cast property JPA detects that an issues another request since spring framework is still holding the database transaction in scope. If the transaction was out of scope you would have gotten a LazyInitialization exception. All this you should research a little more so that you understand it.
So, how to you go about making a better design? You are at least two possibilities that come to mind. First, you could remove the cast mapping. Why do you think you need to have the cast as a collection within the movie? If you want to get the cast for a movie you could simply call castRepository.findByMovie(movie) and get the list. Second, you could use a DTO, or Data Transfer Object, which is a separate POJO that defines what you want your REST interface to actually return. In this case it could be a MovieDto class that is the same as the Movie entity class but without the cast property. Then you would change your movieRepository to have an method defined as Optional<MovieDto> findById(Long id) and spring-data-jpa will use the Projections feature to translate your Movie entity into a MovieDto object automagically.
Using the Projections feature would be my recommended approach. DTO's are meant for "views" for the business layer of your application. Different consumers of your service might want different views into the world of movies. Casting agents might want a list of all movies an actor appeared in while movie critics might want a list of cast as well as the movie and movie buffs might just want the details of a movie. All different DTOs, or views, for the same database. I would also carefully consider whether you actually need a cast mapping. I notice you have a cast = new HashMap<>(); code segment that creates a HashMap for the movie side of the relationship but you shouldn't need that and in the typical read use case it will be thrown away stressing the garbage collector. Finally I notice that you defined cast as a Map but why did you do that? What is going in the key of the map, the movie name? That's bad design. Actors can appear in more than one movie and movies can have more than one actor so you should have a many-to-many relationship.
Finally, this is why "is this good design" questions are frowned upon in SO. The answers are complicated and usually opinionated and not what SO is meant for.

SpringBoot persisting entities with relationships

I have two entities like so
#Entity
public class Person {
#Id
private int id;
private String name;
#OneToMany(mappedBy="owner", cascase=CascadeType.ALL)
private List<Car> cars;
}
and
#Entity
public class Car {
#Id
private int car_id;
private String color;
#ManyToOne
#JoinColumn(name="person_id")
private Person owner;
}
So we have two entities with a relationship where a person can own many cars and each car can be owned by one person.
But this is attached to a restcontroller endpoint where it receives some json like:
{
"id": 1,
"name": "Joe",
"cars": [
{
"car_id": 1,
"color": "red",
},
{
"car_id": 2,
"color": "blue",
}
]
}
So, a person with 2 cars. The endpoint is like
#RequestMapping(value={"/people"},
method=RequestMethod.PUT,
produces=MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<?> updateApplication(#RequestBody Person personRequest) {
Person person = personRepo.save(personRequest);
return new ResponseEntity<>(person, HttpStatus.OK);
}
When this endpoint receives the above json, I would expect that it saves the request object to the db including handling inserts/updates/deletes as necessary. However what I get is an error from the DB that the Car's owner cant be null. I see that it's not specified in the json but because of the mappings between the entities, I would expect that it knows the relationship. What am I missing?
Well, it looks like a infinite recursion.
A Person have many Cars.
A Car have one owner which is a Person.
Just a suggestion:
When you retrieve your entity, the Spring expect to have a Person (owner) inside your Car JSON structure and a list of Car inside of your Person. That could be a problem if you want to retrieve the data, since it will create a infinite recursion.
You need to add a fetch type to LAZY in order to prevent this recursion to happen.
#OneToMany(mappedBy="owner", cascade=CascadeType.ALL, fetch = FetchType.LAZY)
Now, for your question, as I said, your Car need to have a Person, but in your request, you do not specify a Person (owner), so Spring don't know who the owner is. That's the main problem.
I would suggest you to update them separately:
#PutMapping("/people/{id}")
public ResponseEntity <?>handleUpdate(#PathVariable("id") Long personId, #RequestBody Person person) {
Optional <Person> personOptional = personRepository.findById(personId);
if (personOptional.isPresent()) {
Person owner = personOptional.get();
for (Car car: person.getCars()) {
car.setOwner(owner);
}
carRepository.saveAll(person.getCars());
person.setCars(null);
person.setId(personId);
personRepository.save(person);
}
return...;
}
In this example, I'm updating them separately, but in a single method.
Of course, you'll need to autowire both PersonRepository and CarRepository
UPDATE:
If you don't workaround someway, you'll need to have a JSON Request structure like this:
{
"id": 1,
"name": "Joe",
"cars": [
{
"car_id": 1,
"color": "red",
"owner" :
{
"id": "1"
}
},
{
"car_id": 2,
"color": "blue",
"owner" :
{
"id": "1"
}
}
]
}

Resources