In context of Spring Data REST I would like to expose search/filter functionality on the result of the single query.
If my query was findAll that would be out of the box (it would be enough to extend QueryDslPredicateExecutor). But how can I combine my custom query (findMyNotes) with additional filtering provided by the user (that are marked as done). I mean something like:
#Entity
class Note {
#Id
private UUID id;
private UUID personId;
private String status;
private String text;
}
#RepositoryRestResource(path = "notes", itemResourceRel = "notes", collectionResourceRel = "notes")
interface NoteRepository extends JpaRepository<Note, UUID>, QueryDslPredicateExecutor<Note> {
#RestResource(path="notes/my")
#Query("select n from Note n where n.personId=:creatorId")
Page<Note> findDone(#Param("creatorId") UUID creatorId, Predicate predicate, Pageable pageable);
}
I didn't find anything out of the box. Therefore I was thinking to use AOP and:
have multiple paths (multiple #RestResource ?) that map to org.springframework.data.querydsl.QueryDslPredicateExecutor#findAll(Predicate, org.springframework.data.domain.Pageable)
have my own annotation that would point me to the method that returns the original queryDSL Predicate (contents of #Query in my example)
by AOP calculate AND of those two predicates and pass it as a parameter to findAll method
The problem is that I don't know how to do the first point (apart from hacking it in servlet filters) - map multiple paths to the same findAll method.
Or alternatively is there maybe something working out of the box that I missed?
Related
Good day fellow hibernators!
I have a question on how the DistinctBy clause works in conjunction with Spring Data's projection
Assume I have 3 classes:
public class Task {
Long id;
#ManyToOne(fetch = LAZY)
#JoinColumn(name = "project_id")
private Project project;
#OneToOne
#JoinColumn(name = "contact_id")
private Contact assigned;
Boolean deleted;
// ...
}
public class Contact {
Long id;
// ...
}
public class Project {
Long id;
#OneToMany(fetch = LAZY, mappedBy = "project")
private Set<Task> tasks;
// ...
}
These would be my domain classes. Notice, Project does have a "One2Many" to Tasks, Contact does not. Now, I have 2 interfaces for my projections and the basic TaskRepo with 2 methods:
public interface JustProject {
Project getProject();
}
public interface JustAssignee {
Contact getContact();
}
public class TaskRepo extends CrudRepository<Task, Long>, JpaSpecificationExecutor<Task> {
List<JustAssignee> findDistinctByDeletedFalse();
List<JustProject> findDistinctByDeletedFalseAndDeletedFalse();
}
The way it works for me right now is that, findDistinctByDeletedFalse returns as many instances as there are distinct contacts for tasks (e.g. if there are 10 tasks but only 3 contacts, the method will return just 3 objects containing all the 3 distinct contacts). Same for findDistinctByDeletedFalseAndDeletedFalse but on project level.
Now I have a few questions here and would love to get some help in understanding how this works exactly.
is the distinct clause applied after the search is done?
my initial assumption was that this behavior would not work as it does now. I assumed that the distinct clause is applied before the result is fetched, meaning that it would be DISTINCT based on the underlying task model, not the returned JustContact or JustProject model.
is there any way I could somehow not abuse the ...AndDeletedFalse redundant appendix? I need both the two methods from the repo but I feel like I had to cheat just to obtain that result...
... am I doing something wrong? I wanted to get "all distinct contacts/projects assigned to all tasks" as elegant of a way as possible. I ended up thinking about this distinctby exactly because I was unsure on how it works and wanted to try mu luck out. I really didn't think it would work this way, but now that it does I would really want to understand why it does!
Many thanks <3
The DISTINCT keyword is applied to the query and therefore it's effect depends on the select list which in turn is controlled by the projection. Therefore if you have only project or only contact in your projection the DISTINCT will get applied to those values only. Note though, that this relies somewhat on the boundaries of the JPA specification and I wouldn't be surprised if you see different behaviour with different implementations. See https://github.com/eclipse-ee4j/jpa-api/issues/189 and https://github.com/eclipse-ee4j/jpa-api/issues/124 for somewhat related issues raised against the specification.
In oder to differentiate methods that otherwise only differ in the return value you might add any additional string between find and By in the method name. For example you might want to rename your methods to findDistinctContactsByDeletedFalse and findDistinctProjectsByDeletedFalse
I guess this is the best that you can get with Spring Data JPA. You might be able to use just a single method by using the dynamic projections approach, but I think this is a perfect use case for Blaze-Persistence Entity Views.
I created the library to allow easy mapping between JPA models and custom interface or abstract class defined models, something like Spring Data Projections on steroids. The idea is that you define your target structure(domain model) the way you like and map attributes(getters) via JPQL expressions to the entity model.
A DTO model for your use case could look like the following with Blaze-Persistence Entity-Views:
#EntityView(Task.class)
public interface TaskAggregateDto {
// A synthetic "id" to get a grouping context on object level
#IdMapping("1")
int getGroupKey();
Set<ProjectDto> getProjects();
Set<ContactDto> getContacts();
#EntityView(Project.class)
interface ProjectDto {
#IdMapping
Long getId();
String getName();
}
#EntityView(Contact.class)
interface ContactDto {
#IdMapping
Long getId();
String getName();
}
}
The Spring Data integration allows you to use it almost like Spring Data Projections: https://persistence.blazebit.com/documentation/entity-view/manual/en_US/index.html#spring-data-features
public interface TaskRepo extends CrudRepository<Task, Long>, JpaSpecificationExecutor<Task> {
TaskAggregateDto findOneByDeletedFalse();
}
I have two entities Questions and UserAnswers. I need to make an api in spring boot which returns all the columns from both the entities based on some conditions.
Conditions are:
I will be give a comparator eg: >, <, =, >=, <=
A column name eg: last_answered_at, last_seen_at
A value of the above column eg: 28-09-2020 06:00:18
I will need to return an inner join of the two entities and filter based on the above conditions.
Sample sql query based on above conditions will be like:
SELECT q,ua from questions q INNER JOIN
user_answers ua on q.id = ua.question_id
WHERE ua.last_answered_at > 28-09-2020 06:00:18
The problem I am facing is that the column name and the comparator for the query needs to be dynamic.
Is there an efficient way to do this using spring boot and JPA as I do not want to make jpa query methods for all possible combinations of columns and operators as it can be a very large number and there will be extensive use of if else?
I have developed a library called spring-dynamic-jpa to make it easier to implement dynamic queries with JPA.
You can use it to write the query templates. The query template will be built into different query strings before execution depending on your parameters when you invoke the method.
This sounds like a clear custom implementation of a repository method. Firstly, I will make some assumptions about the implementation of your entities. Afterwards, I will present an idea on how to solve your challenge.
I assume that the entities look basically like this (getters, setters, equals, hachCode... ignored).
#Entity
#Table(name = "questions")
public class Question {
#Id
#GeneratedValue
private Long id;
private LocalDateTime lastAnsweredAt;
private LocalDateTime lastSeenAt;
// other attributes you mentioned...
#OneToMany(mappedBy = "question", cascade = CascadeType.ALL, orphanRemoval = true)
private List<UserAnswer> userAnswers = new ArrayList();
// Add and remove methods added to keep bidirectional relationship synchronised
public void addUserAnswer(UserAnswer userAnswer) {
userAnswers.add(userAnswer);
userAnswer.setQuestion(this);
}
public void removeUserAnswer(UserAnswer userAnswer) {
userAnswers.remove(userAnswer);
userAnswer.setQuestion(null);
}
}
#Entity
#Table(name = "user_answers")
public class UserAnswer {
#Id
#GeneratedValue
private Long id;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "task_release_id")
private Question question;
}
I will write the code with the knowledge about the JPA of Hibernate. For other JPAs, it might work similarly or the same.
Hibernate often needs the name of attributes as a String. To circumvent the issue of undetected mistakes (especially when refactoring), I suggest the module hibernate-jpamodelgen (see the class names suffixed with an underscore). You can also use it to pass the names of the attributes as arguments to your repository method.
Repository methods try to communicate with the database. In JPA, there are different ways of implementing database requests: JPQL as a query language and the Criteria API (easier to refactor, less error prone). As I am a fan of the Criteria API, I will use the Criteria API together with the modelgen to tell the ORM Hibernate to talk to the database to retrieve the relevant objects.
public class QuestionRepositoryCustomImpl implements QuestionRepository {
#PersistenceContext
private EntityManager entityManager;
#Override
public List<Question> dynamicFind(String comparator, String attribute, String value) {
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<Question> cq = cb.createQuery(Question.class);
// Root gets constructed for first, main class in the request (see return of method)
Root<Question> root = cq.from(Question.class);
// Join happens based on respective attribute within root
root.join(Question_.USER_ANSWER);
// The following ifs are not the nicest solution.
// The ifs check what comparator String contains and adds respective where clause to query
// This .where() is like WHERE in SQL
if("==".equals(comparator)) {
cq.where(cb.equal(root.get(attribute), value));
}
if(">".equals(comparator)) {
cq.where(cb.gt(root.get(attribute), value));
}
if(">=".equals(comparator)) {
cq.where(cb.ge(root.get(attribute), value));
}
if("<".equals(comparator)) {
cq.where(cb.lt(root.get(attribute), value));
}
if("<=".equals(comparator)) {
cq.where(cb.le(root.get(attribute), value));
}
// Finally, query gets created and result collected and returned as List
// Hint for READ_ONLY is added as lists are often just for read and performance is better.
return entityManager.createQuery(cq).setHint(QueryHints.READ_ONLY, true).getResultList();
}
}
Using spring-data-jpa and working on getting data out of table where there are about a dozen columns which are used in queries to find particular rows, and then a payload column of clob type which contains the actual data that is marshalled into java objects to be returned.
Entity object very roughly would be something like
#Entity
#Table(name = "Person")
public class Person {
#Column(name="PERSON_ID", length=45) #Id private String personId;
#Column(name="NAME", length=45) private String name;
#Column(name="ADDRESS", length=45) private String address;
#Column(name="PAYLOAD") #Lob private String payload;
//Bunch of other stuff
}
(Whether this approach is sensible or not is a topic for a different discussion)
The clob column causes performance to suffer on large queries ...
In an attempt to improve things a bit, I've created a separate entity object ... sans payload ...
#Entity
#Table(name = "Person")
public class NotQuiteAWholePerson {
#Column(name="PERSON_ID", length=45) #Id private String personId;
#Column(name="NAME", length=45) private String name;
#Column(name="ADDRESS", length=45) private String address;
//Bunch of other stuff
}
This gets me a page of NotQuiteAPerson ... I then query for the page of full person objects via the personIds.
The hope is that in not using the payload in the original query, which could filtering data over a good bit of the backing table, I only concern myself with the payload when I'm retrieving the current page of objects to be viewed ... a much smaller chunk.
So I'm at the point where I want to map the contents of the original returned Page of NotQuiteAWholePerson to my List of Person, while keeping all the Paging info intact, the map method however only takes a Converter which will iterate over the NotQuiteAWholePerson objects ... which doesn't quite fit what I'm trying to do.
Is there a sensible way to achieve this ?
Additional clarification for #itsallas as to why existing map() will not suffice..
PageImpl::map has
#Override
public <S> Page<S> map(Converter<? super T, ? extends S> converter) {
return new PageImpl<S>(getConvertedContent(converter), pageable, total);
}
Chunk::getConvertedContent has
protected <S> List<S> getConvertedContent(Converter<? super T, ? extends S> converter) {
Assert.notNull(converter, "Converter must not be null!");
List<S> result = new ArrayList<S>(content.size());
for (T element : this) {
result.add(converter.convert(element));
}
return result;
}
So the original List of contents is iterated through ... and a supplied convert method applied, to build a new list of contents to be inserted into the existing Pageable.
However I cannot convert a NotQuiteAWholePerson to a Person individually, as I cannot simply construct the payload... well I could, if I called out to the DB for each Person by Id in the convert... but calling out individually is not ideal from a performance perspective ...
After getting my Page of NotQuiteAWholePerson I am querying for the entire List of Person ... by Id ... in one call ... and now I am looking for a way to substitute the entire content list ... not interively, as the existing map() does, but in a simple replacement.
This particular use case would also assist where the payload, which is json, is more appropriately persisted in a NoSql datastore like Mongo ... as opposed to the sql datastore clob ...
Hope that clarifies it a bit better.
You can avoid the problem entirely with Spring Data JPA features.
The most sensible way would be to use Spring Data JPA projections, which have good extensive documentation.
For example, you would first need to ensure lazy fetching for your attribute, which you can achieve with an annotation on the attribute itself.
i.e. :
#Basic(fetch = FetchType.LAZY) #Column(name="PAYLOAD") #Lob private String payload;
or through Fetch/Load Graphs, which are neatly supported at repository-level.
You need to define this one way or another, because, as taken verbatim from the docs :
The query execution engine creates proxy instances of that interface at runtime for each element returned and forwards calls to the exposed methods to the target object.
You can then define a projection like so :
interface NotQuiteAWholePerson {
String getPersonId();
String getName();
String getAddress();
//Bunch of other stuff
}
And add a query method to your repository :
interface PersonRepository extends Repository<Person, String> {
Page<NotQuiteAWholePerson> findAll(Pageable pageable);
// or its dynamic equivalent
<T> Page<T> findAll(Pageable pageable, Class<T>);
}
Given the same pageable, a page of projections would refer back to the same entities in the same session.
If you cannot use projections for whatever reason (namely if you're using JPA < 2.1 or a version of Spring Data JPA before projections), you could define an explicit JPQL query with the columns and relationships you want, or keep the 2-entity setup. You could then map Persons and NotQuiteAWholePersons to a PersonDTO class, either manually or (preferably) using your object mapping framework of choice.
NB. : There are a variety of ways to use and setup lazy/eager relations. This covers more in detail.
I am trying to use pagination with QueryDSL - using the com.mysema.querydsl package.
All my Querydsl query types look like this -
#Generated("com.mysema.query.codegen.EntitySerializer")
public class QCountry extends EntityPathBase<Country> {...}
Currently, my repository implementation class looks something like this -
#Override
public Page<Country> findPaginatedCountries(String country, Optional<String> status, Pageable pageable) {
QCountry qCountry= QCountry.someObject;
QActiveCountry qActiveCountry = QActiveCountry.activeCountry;
JPAQuery jpaQuery = new JPAQuery(entityManager);
QueryBase queryBase = jpaQuery.from(qCountry).innerJoin(qActiveCountry).fetch()
.where(qCountry.codeLeft.country.upper().eq(country.toUpperCase()))
.where(qCountry.codeRight.country.upper().eq(country.toUpperCase()));
if(status.isPresent()){
queryBase = queryBase.where(qActiveCountry.id(qCountry.active.id))
.where(qActiveCountry.status.upper().eq(status.get().toUpperCase()));
}
.......}
Now, I want this dynamic query to return a paginated response. I want to use Spring's pagination to do that and not manually set offset, size etc.
I know I can use QueryDslRepositorySupport class - as implemented here - https://github.com/keke77/spring-data-jpa-sample/blob/master/spring-data-jpa/src/main/java/com/gmind7/bakery/employee/EmployeeRepositoryImpl.java
Sample code from the above link -
#Override
public Page<Employees> QFindByOfficeCode(long officeCode, Pageable pageable) {
//JPAQuery query = new JPAQuery(em);
JPQLQuery query = from(QEmployees.employees).where(QEmployees.employees.officeCode.eq(officeCode));
query = super.getQuerydsl().applyPagination(pageable, query);
SearchResults<Employees> entitys = query.listResults(QEmployees.employees);
return new PageImpl<Employees>(entitys.getResults(), pageable, entitys.getTotal());
}
However, to do that -
I need to pass JPQLQuery object to the applyPagination method. How can I do that without changing my code (Ofcourse, the repository class will extend QueryDslRepositorySupport class). Currently, I am using JPAQuery as you can see.
OR
I probably need to change my QueryDSL types by having them extend EntityPath instead of EntityPathBase so that I can use JPQLQuery.from() to generate the query and then use the applyPagination method, which requires a JPQLQuery object. However, my Q classes are extending EntityPathBase class instead. Should I be use com.querydsl package instead of com.mysemsa.querydsl package to generate query types?
OR
Other option is to use the following - http://docs.spring.io/spring-data/commons/docs/current/api/org/springframework/data/querydsl/QueryDslPredicateExecutor.html#findAll-com.querydsl.core.types.Predicate-org.springframework.data.domain.Pageable-
Code snippet below -
Page<T> page = QueryDslPredicateExecutor.findAll(org.springframework.data.querydsl.Predicate predicate, Pageable pageable)
However, I am making joins between two tables and then filtering results with a where clause (as you can see above in my code). How can I pass a predicate object in the findAll method above? Not sure how to include a join in it.
Please let me know if the problem is not clear, I can add more details.
EDIT: There is a many to one relationship between Country and ActiveCountry.
Country class has an ActiveCountry reference. And we have to do a join between both ids. Is is possible that Country can have null ActiveCountry. Therefore, we want an inner join - only non null values for active country
#ManyToOne
#JoinColumn(name="id")
ActiveCountry active;
Step 1: Annotate the entity class with #QueryEntity
#Entity
#QueryEntity
public class Country {}
This seems to have been addressed already since the question shows Q classes.
Step 2: Have the repository interface extend QueryDslPredicateExecutor
public interface CountryRepository
extends PagingAndSortingRepository<Country, Long>
, QueryDslPredicateExecutor<Country> {
}
Step 3: Invoke the Page<T> findAll(Predicate query, Pageable page) method provided by QueryDslPredicateExecutor
public Page<Country> getCountries(String country, Optional<String> status, Pageable page) {
QCountry root = QCountry.country;
BooleanExpression query = root.codeLeft.country.equalsIgnoreCase(country);
query = query.and(root.codeRight.country.equalsIgnoreCase(country));
if (status.isPresent()) {
query = query.and(root.active.status.equalsIgnoreCase(status));
}
return countryRepository.findAll(query, page);
}
I'm trying to get a page of a partial entity (NetworkSimple) using the new feature of spring data, projections
I've checked the documentation and if I request only:
Collection<NetworkSimple> findAllProjectedBy();
It works, but if I'm using pageable:
Page<NetworkSimple> findAllProjectedBy(Pageable pageable);
It throws an error:
org.hibernate.jpa.criteria.expression.function.AggregationFunction$COUNT cannot be cast to org.hibernate.jpa.criteria.expression.CompoundSelectionImpl
Any one has already work with this ?
My NetworkSimple class is the following:
public interface NetworkSimple {
Long getId();
String getNetworkName();
Boolean getIsActive();
}
Note: This feature should work in the way described by the original poster but due to this bug it didn't. The bug has been fixed for the Hopper SR2 release, if you're stuck on an earlier version then the workaround below will work.
It is possible to use Pageable with the new query projection features introduced in Spring Data JPA 1.10 (Hopper). You will need to use the #Query annotation and manually write a query for the fields you require, they must also be aliased using AS to allow Spring Data to figure out how to project the results. There is a good example in spring-boot-samples part of the spring boot repository.
In your example it would be quite simple:
#Query("SELECT n.id AS id, n.name AS networkName, n.active AS isActive FROM Network n")
Page<NetworkSimple> findAllProjectedBy(Pageable pageable);
I have made the assumption that your entity looks something like this:
#Entity
public class Network
{
#Id
#GeneratedValue
private Long id;
#Column
private String name;
#Column
private boolean active;
...
}
Spring Data will derive a count query automatically for the paging information. It is also possible to make use of joins in the query to fetch associations and then summarise them in the projection.
I think you need create findAllProjectedBy() as specification.Then you can use findAll() method like this.
example :findAll(findAllProjectedBy(),pageable)
Following link may be help to find how to create specification in spring.
https://spring.io/blog/2011/04/26/advanced-spring-data-jpa-specifications-and-querydsl/
The issue may come from the method name. The by keyword means that you ae filterig data by a specific property: findByName for example. Its called query creation from method name:
http://docs.spring.io/spring-data/jpa/docs/1.10.1.RELEASE/reference/html/#repositories.query-methods.query-creation
So try with Page<NetworkSimple> findAll(Pageable pageable);
Even with spring-data-jpa 1.11.4, something like
public interface NetworkRepository extends JpaRepository<Network, String> {
Page<NetworkSimple> findAll(Pageable pageable);
}
would not compile; reporting
findAll(org.springframework.data.domain.Pageable) in NetworkRepository clashes with findAll(org.springframework.data.domain.Pageable) in org.springframework.data.repository.PagingAndSortingRepository
return type org.springframework.data.domain.Page<NetworkSimple> is not compatible with org.springframework.data.domain.Page<Network>
The workaround we found was to rename findAll to findAllBy, e.g.
public interface NetworkRepository extends JpaRepository<Network, String> {
Page<NetworkSimple> findAllBy(Pageable pageable);
}
You can use interface projection with Pageable like this :
Page<NetworkSimple> findPagedProjectedBy(Pageable pageable);
with some parameter :
Page<NetworkSimple> findPagedProjectedByName(String name, Pageable pageable);
Implementing interface projection with pagination
1. Our ResourceEntity.java class
#Getter
#Setter
#NoArgsConstructor
#Entity
public class ResourceEntity{
private Long id;
private String name;
}
2. Creating projection Interface name ProjectedResource.java, which maps data collected by the SQL query from repository layer method
public interface ProjectedResource {
Long getId();
String getName();
String getAnotherProperty();
}
3. Creating Repository layer method: getProjectedResources()
We are considering the database table name is resource.
We are only fetching id and name here.
#Query(name="select id, name, anotherProperty from resource", countQuery="select count(*) from resource", nativeQuery=true)
Page<ProjectedResource> getProjectedResources(Pageable page);
Hope the issue will be resolved!
You can use:
#Query("SELECT n FROM Network n")
Page<? extends NetworkSimple> findAllProjectedBy(Pageable pageable);