Spring pageable with specification add more LEFT JOIN on order by joined column - spring

Can anybody explain to me why Spring adds LEFT JOIN checking on pageable Order using its specification feature?
Here is the case:
I want to use Spring specification to generate dynamic query for filtering data and pageable to limit the data. The query is between UserEntity and UserGroupEntity, and I want to always fetch the UserGroupEntity on every query, therefore I add a checking block for COUNT and data query. If the query is a query COUNT then I use the Join object, otherwise I use the Fetch object.
public static Specification<UserEntity> filtered(final String searchTerm) {
return new Specification<UserEntity>() {
#SuppressWarnings("unchecked")
#Override
public Predicate toPredicate(Root<UserEntity> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
Join<UserEntity, UserGroupEntity> joinUserGroup = null;
if (QueryHelper.isQueryCount(query)) {
joinUserGroup = root.join("userGroup");
} else {
Fetch<UserEntity, UserGroupEntity> fetchUserGroup = root.fetch("userGroup");
joinUserGroup = (Join<UserEntity, UserGroupEntity>) fetchUserGroup;
}
if (searchTerm != null) {
List<Predicate> predicates = new ArrayList<Predicate>();
predicates.add(cb.like(root.get("lowerUsername").as(String.class), "%" + searchTerm.toLowerCase() + "%"));
predicates.add(cb.like(root.get("lowerEmail").as(String.class), "%" + searchTerm.toLowerCase() + "%"));
predicates.add(cb.like(joinUserGroup.get("lowerName").as(String.class), "%" + searchTerm.toLowerCase() + "%"));
cb.or(predicates.toArray(new Predicate[predicates.size()]));
}
return null;
}
};
}
This is how I call the specification on UserService:
Order order = new Order(Direction.ASC, "lowerEmail");
Page<UserEntity> pages = userRepository.findAll(filtered(searchTerm), new PageRequest(page, size, new Sort(order)));
The query is fine when I add an ordering using UserEntity column. Here is the generated HQL:
select generatedAlias0 from com.example.entity.UserEntity as generatedAlias0
inner join fetch generatedAlias0.userGroup as generatedAlias1
order by generatedAlias0.lowerEmail asc
But the query becomes strange (because I don't know why) when I use the the column from joined entity for ordering which is userGroup.lowerName. By using that column, Spring adds more LEFT JOIN in my query and makes it like this:
select generatedAlias0 from com.example.entity.UserEntity as generatedAlias0
left join generatedAlias0.userGroup as generatedAlias1
inner join fetch generatedAlias0.userGroup as generatedAlias2
order by generatedAlias1.lowerName asc
As I come across the spring-data-JPA code on Github, I found that the translation from specification to criteria is done in method getQuery(). This method calls other method toOrders() from class QueryUtils to apply the sort order. Method toOrders() finally calls method getOrCreateJoin() or isAlreadyFetched() which will check the the previous join attribute (the one I made in the specification) which is userGroup, but because the join type is not LEFT JOIN then Spring adds more join using LEFT JOIN in my query.
private static Join<?, ?> getOrCreateJoin(From<?, ?> from, String attribute) {
for (Join<?, ?> join : from.getJoins()) {
boolean sameName = join.getAttribute().getName().equals(attribute);
if (sameName && join.getJoinType().equals(JoinType.LEFT)) {
return join;
}
}
return from.join(attribute, JoinType.LEFT);
}
That's what I got from my search on the spring-data-jpa code, CMIIW. What is the purpose of the JoinType.LEFT actually I still don't understand. Your explanation will be very helpful for me (and us).
Now I think that I will use custom repository to generate dynamic query using JPQL until I understand the reason for that generated query with additional LEFT JOIN using specification and pageable.

I had the same issue, and found the solution and answer in https://stackoverflow.com/a/43965633/7280115. You may check if it also works for you

Related

Get Records on the basis of list of string Criteria Query Predicates

I created one class
class Employee { Integer id; String name; String departments; }
and in sql server database i have records
I stored departments as ";" separated. For Example Department = Computer;Civil
1,Chaitanya,Computer;Civil
2,Tom,Physics;Chemistry
3,Harry,Economics;Commerce
4,Henry,Computer;Civil;Mechanical
5,Ravi,null
Now i want to filter data with departments let's say there is one multiselect in frontend where i have list of departments and i select two departments for example-> Computer,Civil and in backend i got List<String> deparmentFilter as parameter say Computer;Civil
Now as per my requirement i have to return two data from Spring Boot Controller
1,Chaitanya,Computer;Civil
4,Henry,Computer;Civil;Mechanical
Right Now what i did is i executed the query to fetch all the records and then i right below logic
List<Employee> employeesToBeRemoved = new ArrayList<>();
if (!departmentNames.isEmpty()) {
allEmployees.forEach(employee -> {
if (employee.getDepartment() != null) {
Set<String> departmentNamesResult = new HashSet<>(Arrays.asList(employee.getDepartment().
split(";")));
Boolean isExist = Collections.disjoint(departmentNamesResult, departmentNames);
if (Boolean.TRUE.equals(isExist)) {
employeesToBeRemoved.add(employee);
}
} else {
employeesToBeRemoved.add(employee);
}
});
}
allEmployees.removeAll(employeesToBeRemoved);
I tried to move it to predicates but not able to do that, This solution is taking much time to execute,
Please suggest me some other better ways (optimized way) to improve performance.
Is there is any way to add this filter in predicates?
Another approach i am thinking (12/05/2022)
Let's say i have one table employee_department_mapping and in that table i have employeeId and departmentName so in this correct way to add predicate?
CriteriaQuery<Object> subQuery1 = criteriaBuilder.createQuery();
Root<EmployeeDepartmentMapping> subQueryEmpDptMp = subQuery1.from(EmployeeDepartmentMapping.class);
predicates1.add(subQueryEmpDptMp.get("departmentName").in(departmentNames));
You might achieve better performance by splitting your table and using join:
class Employee { Integer id; String name; Integer departmentsId; }
class EmployeeDepartments { Integer departmentsId; String department; }
You may use Element Collection to achieve this.
Now, instead of having a the following row:
1,Chaitanya,Computer;Civil
You will have the following:
table1:
1,Chaitanya,123
table2:
123,Compter
123,Civil
Execute a join to get all row from table2 with table1 to get your result

JPQL Join Fetch in jpql not working, running into org.springframework.dao.InvalidDataAccessApiUsageException

I am using JPQL queries for my rest application. spring boot 2.5 and mysql 5.7
I have one table that has 4 onetomany and I need to fetch all the relationships at one go for the findAll query.
In order to achieve that, I am using JPQL query with join fetch.
However I am not able to load data using join fetch for this.
#Repository
#Transactional
public interface CustomProductRepository extends JpaRepository<Product, Long>, JpaSpecificationExecutor<Product> {
default Page<Product> findByPagingCriteria(List<Long> productIds, Long categoryId, Long brandId, String partNumber, Pageable pageable) {
Page page = this.findAll((Specification<Product>) (root, query, criteriaBuilder) -> {
//setting fetch to load data at once instead of firing multiple queries
root.fetch("productModelMappings", JoinType.LEFT);
root.fetch("offers", JoinType.LEFT);
root.fetch("productAttributes", JoinType.LEFT);
root.fetch("productSellers", JoinType.LEFT);
List<Predicate> predicates = new ArrayList<>();
// query.where(root.get("id").in(Arrays.asList(1)));
if (categoryId != null) {
predicates.add(criteriaBuilder.and(criteriaBuilder.equal(root.get("categoryId"), categoryId)));
}
if (brandId != null) {
predicates.add(criteriaBuilder.and(criteriaBuilder.equal(root.get("brandId"), brandId)));
}
if (StringUtils.isNotEmpty(partNumber)) {
predicates.add(criteriaBuilder.and(criteriaBuilder.equal(root.get("partNumber"), partNumber)));
}
return criteriaBuilder.and(predicates.toArray(new Predicate[predicates.size()]));
}, pageable);
page.getTotalElements(); // get total elements
page.getTotalPages(); // get total pages
return page;
}
}
Getting the below error:
org.hibernate.QueryException: query specified join fetching, but the owner of the fetched association was not present in the select list [FromElement{explicit,not a collection join,fetch join,fetch non-lazy properties,classAlias=generatedAlias1,role=com.pitstop.catalogue.domain.Product.productModelMappings,tableName=product_model_mapping,tableAlias=productmod1_,origin=product product0_,columns={product0_.id,className=com.pitstop.catalogue.domain.ProductModelMapping}}] [select count(generatedAlias0) from com.pitstop.catalogue.domain.Product as generatedAlias0 left join fetch generatedAlias0.productModelMappings as generatedAlias1 left join fetch generatedAlias0.offers as generatedAlias2 left join fetch generatedAlias0.productAttributes as generatedAlias3 left join fetch generatedAlias0.productSellers as generatedAlias4 where 1=1]; nested exception is java.lang.IllegalArgumentException: org.hibernate.QueryException: query specified join fetching, but the owner of the fetched association was not present in the select list [FromElement{explicit,not a collection join,fetch join,fetch non-lazy properties,classAlias=generatedAlias1,role=com.pitstop.catalogue.domain.Product.productModelMappings,tableName=product_model_mapping,tableAlias=productmod1_,origin=product product0_,columns={product0_.id,className=com.pitstop.catalogue.domain.ProductModelMapping}}] [select count(generatedAlias0) from com.pitstop.catalogue.domain.Product as generatedAlias0 left join fetch generatedAlias0.productModelMappings as generatedAlias1 left join fetch generatedAlias0.offers as generatedAlias2 left join fetch generatedAlias0.productAttributes as generatedAlias3 left join fetch generatedAlias0.productSellers as generatedAlias4 where 1=1]

How to get Distinct record from JPA

I have implemented a method which gives me specification, but what I want is the query should be as below:
Select Distinct *
From (another select query)
I generate query dynamically.
How do I perform the same using specification in Spring Boot?
Try something like this
Specification<T> spec = getSpecification();
Specification<T> distinctSpec = (root, query, cb) -> {
query.distinct(true);
return spec.toPredicate(root, query, cb);
};
if you want to get distinct records, you have to write a query like this in the repository.
The below query gives the distinct author from the post table.
#Query("select distinct author from Post")
List<String> findUniqueAuthor();
Write this in the repository
#Query(value = "Select Distinct * From (another select query)", nativeQuery = true)
List<Object> findUniqueData();

JPA Criteria api - Total records for concrete query within pagination

I am programming function for pagination in my repository layer. Function receive as parameters spring's pageable object and some value like this:
public Page<Foo> filterFoo(Pageable pageable, String value) {
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<Foo> fooQuery = cb.createQuery(Foo.class);
Root<Foo> foo = fooQuery .from(Foo.class);
fooQuery .where(adding predicate for match value);
List<Foo> result = entityManager.createQuery(fooQuery )
.setFirstResult((pageable.getPageNumber() - 1) * pageable.getPageSize())
.setMaxResults(pageable.getPageSize())
.getResultList();
return new PageImpl<>(result, pageable, xxxx);
}
Function return spring's PageImpl object filled with my result. To PageImpl I also need set total count of objects which suit predicates. This count number have to be of course without maxResult and firstResult. Is possible create another database call with my fooQuery to get total database records for that query without limit? What is the best practise to use pageable and criteria api in JPA? Thank you in advice.
Because generated SQL uses aliases - you may need make separate query for get total count of rows.
For example:
CriteriaQuery<Long> countQuery = cb.createQuery(Long.class);
countQuery.select(cb.count(countQuery.from(Foo.class)));
if (Objects.nonNull(filters)) {
countQuery.where(filters);
}
return new PageImpl<>(result, pageable, em.createQuery(countQuery).getSingleResult());
where filters is equal to your adding predicate for match value expression.
Also, you may use a TupleQuery with custom SQL function for calculate count of rows in one select query.
Like this:
public class SqlFunctionsMetadataBuilderContributor implements MetadataBuilderContributor {
#Override
public void contribute(MetadataBuilder metadataBuilder) {
metadataBuilder.applySqlFunction(
"count_over",
new SQLFunctionTemplate(
StandardBasicTypes.LONG,
"(count(?1) over())"
)
);
}
}
and Criteria:
public Page<Foo> findAll(Specification<Foo> specification, Pageable pageable) {
CriteriaQuery<Tuple> cq = cb.createTupleQuery();
Root<Foo.class> fooRoot = cq.from(Foo.class);
cq.select(cb.tuple(fooRoot, cb.function("count_over", Long.class, fooRoot.get("id"))));
Predicate filters = specification.toPredicate(fooRoot, cq, cb);
if (Objects.nonNull(filters)) {
cq.where(filters);
}
TypedQuery<Tuple> query = em.createQuery(cq);
query.setFirstResult((int) pageable.getOffset());
query.setMaxResults(pageable.getPageSize());
List<Tuple> result = query.getResultList();
if (result.isEmpty()) {
return new PageImpl<>(List.of());
}
return new PageImpl<>(
result.stream().map(tuple -> (Foo) tuple.get(0)).collect(toUnmodifiableList()),
pageable,
(long) result.get(0).get(1)
);
}
See more about SQLFunction: https://vladmihalcea.com/hibernate-sql-function-jpql-criteria-api-query/ and Custom SQL for Order in JPA Criteria API

hibernate - Dynamic sort in a query

I have a Dao method that returns list of "posts" and looks like this:
public List<PostDTO> getPosts() {
Session session = sessionFactory.getCurrentSession();
return postList = session
.createQuery("select new com.poster.app.dto.PostDTO(p.id, p.date, p.title, p.text, p.imageUrl, p.author, p.category, count(c.post.id)) "
+ "from Post as p left join Comment as c ON p.id = c.post.id group by p.id",
PostDTO.class).getResultList();
}
So it basically just creates query and returns the dto's in that case. The thing is, i need to fetch exact same list BUT with different sorting. like i need to sort it dynamically by "newest", "most popular" and by "comments number" and i want to do this in one method instead of creating 3 methods for each ("newest", "most popular" and by "comments number"), how can i do that in hibernate?
You have choice between :
Use api criteria to build your query and add dynamically your order by clause.
Add the order by clause at the end of your query depending on your param
ex with 2nd option:
public List<PostDTO> getPostsOrderBy(String orderParam)
{
Session session = sessionFactory.getCurrentSession();
String query = "select new com.poster.app.dto.PostDTO(p.id, p.date, p.title, p.text, p.imageUrl, p.author, p.category, count(c.post.id)) "
+ "from Post as p left join Comment as c ON p.id = c.post.id group by p.id order by "+ orderParam;
return postList = session.createQuery(query,PostDTO.class).getResultList();
}

Resources