JPA findAll(Specification specification, Pageable pageable) count is very costly and slow - spring

public Page<MyObject> findByCriteria(MySearchFilters mySearchFilters, PageRequest pageRequest) {
Page<MyObject> all = myObjectRepository.findAll(new Specification<MyObject>() {
#Override
public Predicate toPredicate(Root<MyObject> root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder) {
List<Predicate> predicates = new ArrayList<>();
................ here is lots of different optional fields that might or not get added to query
return criteriaBuilder.and(predicates.toArray(new Predicate[predicates.size()]));
}
private void addLikeCriteria(String field, String fieldName, CriteriaBuilder criteriaBuilder, Root<MyObject> root, List<Predicate> predicates) {
predicates.add(criteriaBuilder.like(criteriaBuilder.lower(root.get(fieldName).as(String.class)), "%" + field.toLowerCase() + "%"));
}
}, pageRequest);
return all;
}
I am calling the method which in the background does the also count(field) query that is very costly! I analyzed it and if it would do count(*) instead it would be 10x faster as then it does not have to check the value.
Page<T> findAll(#Nullable Specification<T> spec, Pageable pageable);
Is it possible to make it count() instead or should I make a custom Specification, pageable repository method that would not do the count(field) and do the count() myself to attach to the result?

Related

Spring Jpa Specification with list of long where in query

I want to run a where in query with Spring JPA Specification and criteria builder. I am having issue where I will receive a List ids from request and run specification query but could't find any way to do so.
This is what I have done so far.
public class DistributorMasterDataSpecification implements Specification<DistributorMasterData> {
#Override
public Predicate toPredicate(Root<DistributorMasterData> root, CriteriaQuery<?> query, CriteriaBuilder criteriaBuilder) {
return
codeSpec()
.and(idSpec())
.toPredicate(root, query, criteriaBuilder);
}
private Specification<DistributorMasterData> idSpec() {
return ((root, query, criteriaBuilder) ->
Objects.isNull(filterDto.getDistributorIds()) ?
null : root.get(DistributorMasterData_.ID).in(filterDto.getDistributorIds())
);
}
}
I am unable to figure out how to prepare where in query with specification. With current implementation I facing the error below
org.springframework.dao.InvalidDataAccessApiUsageException: literal value cannot be null; nested exception is java.lang.IllegalArgumentException: literal value cannot be null
You can create a custom repository and after you do the implementation
public interface EditeurRepositoryCustom {
Page<User> search(UserSearch search, Pageable page);
}
#Repository
public class UserRepositoryCustomImpl extends SimpleJpaRepository<User,Long> implements EditeurRepositoryCustom {
#Autowired
private EntityManager entityManager;
public EditeurRepositoryCustomImpl(EntityManager em) {
super(User.class, em);
}
public Page<User> search(UserSearch search, Pageable page) {
Specification<User> hasExternalOrder = (Root<User> mainRoot, CriteriaQuery<?> mainCq, CriteriaBuilder mainCb) -> {
return mainRoot.get("externalOrder").isNotNull();
};
Specification<User> isPrincipal = (Root<User> mainRoot, CriteriaQuery<?> mainCq, CriteriaBuilder mainCb) -> {
if(!search.principal()){
return mainCb.and();
}
return mainCb.isTrue(mainRoot.get("main"));
};
};
return findAll(hasNameContain.and(hasExternalOrder).and(isPrincipal), page);
}
You can combine multiple spec

Wrong Pagination info using JpaSpecifications

I am using spring boot 2.2.6
I am implementing an API that has the option of filtering data and it must be returned paginated
so I used specifications (JpaSepecificationExecutor) and the specification has a join with another table
and this join is mandatory as a customerId is always given and added to predicate list
the problem is that the paginated info returned is wrong mainly (totalElements) and (totalPages)
this is the specifications class
`
#Data
#AllArgsConstructor
#NoArgsConstructor
public class CustomerSalesOrderSpecification implements Specification<SalesOrder> {
#NonNull
private Integer customerId;
private Integer orderId;
#DateTimeFormat(pattern = "yyyy-MM-dd")
private LocalDate purchasedDateFrom;
#DateTimeFormat(pattern = "yyyy-MM-dd")
private LocalDate purchasedDateTo;
private String customerFirstname;
private BigDecimal baseGrandTotalFrom;
private BigDecimal baseGrandTotalTo;
private String storeName;
#Override
public Predicate toPredicate(#NonNull Root<SalesOrder> root, #NonNull CriteriaQuery<?> query,
#NonNull CriteriaBuilder builder) {
List<Predicate> predicateList = new ArrayList<>();
predicateList.add(customerId(customerId).toPredicate(root, query, builder));
if (Objects.nonNull(orderId)) {
predicateList.add(orderId(orderId).toPredicate(root, query, builder));
}
if (Objects.nonNull(purchasedDateFrom)) {
predicateList.add(purchaseDateFrom(purchasedDateFrom).toPredicate(root, query, builder));
}
if (Objects.nonNull(purchasedDateTo)) {
predicateList.add(purchaseDateTo(purchasedDateTo).toPredicate(root, query, builder));
}
if (Objects.nonNull(customerFirstname)) {
predicateList.add(customerFirstname(customerFirstname).toPredicate(root, query, builder));
}
if (Objects.nonNull(baseGrandTotalFrom)) {
predicateList.add(baseGrandTotalFrom(baseGrandTotalFrom).toPredicate(root, query, builder));
}
if (Objects.nonNull(baseGrandTotalTo)) {
predicateList.add(baseGrandTotalTo(baseGrandTotalTo).toPredicate(root, query, builder));
}
if (Objects.nonNull(storeName)) {
predicateList.add(storeName(storeName).toPredicate(root, query, builder));
}
return builder.and(predicateList.toArray(new Predicate[]{}));
}
public Specification<SalesOrder> customerId(Integer customerId) {
return (root, query, builder) -> {
Join<SalesOrder, CustomerEntity> salesOrderCustomer = root.join("customerEntity");
return builder.equal(salesOrderCustomer.get("entityId"), customerId);
};
}
public Specification<SalesOrder> orderId(Integer orderId) {
return ((root, query, builder) -> builder.equal(root.get("entityId"), orderId));
}
public Specification<SalesOrder> purchaseDateFrom(LocalDate purchasedDateFrom) {
return (root,query,builder) -> builder.or(
builder.greaterThan(root.get("createdAt"),Timestamp.valueOf(purchasedDateFrom.atStartOfDay())),
builder.between(root.get("createdAt"),Timestamp.valueOf(purchasedDateFrom.atStartOfDay()),
Timestamp.valueOf(purchasedDateFrom.atTime(23,59))));
}
public Specification<SalesOrder> purchaseDateTo(LocalDate purchasedDateTo) {
return (root,query,builder) -> builder.or(
builder.lessThan(root.get("createdAt"),Timestamp.valueOf(purchasedDateTo.atStartOfDay())),
builder.between(root.get("createdAt"),Timestamp.valueOf(purchasedDateTo.atStartOfDay()),
Timestamp.valueOf(purchasedDateTo.atTime(23,59))));
}
public Specification<SalesOrder> customerFirstname(String customerFirstname) {
return ((root, query, builder) -> builder.like(root.get("customerFirstname"), "%" + customerFirstname + "%"));
}
public Specification<SalesOrder> baseGrandTotalFrom(BigDecimal baseGrandTotalFrom) {
return ((root, query, builder) -> builder.greaterThanOrEqualTo(root.get("baseGrandTotal"), baseGrandTotalFrom));
}
public Specification<SalesOrder> baseGrandTotalTo(BigDecimal baseGrandTotalTo) {
return ((root, query, builder) -> builder.lessThanOrEqualTo(root.get("baseGrandTotal"), baseGrandTotalTo));
}
public Specification<SalesOrder> storeName(String storeName) {
return ((root, query, builder) -> builder.like(root.get("storeName"), "%" + storeName + "%"));
}
}
`
for example if I filter based on id and name I get totalElements 20 when the real number is 377
and totalPages is 1 which is wrong of course
I tried writing the whole query in JPQL and it worked correctly so the issue is only I use specifications
with pagination and a join
I checked similar questions, but they were related to an error when using join fetch, I am using a normal join and there is no error just wrong pagination info in response
what might be the reason ?

No converter found capable of converting from type

org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [com.skywarelabs.runwayreporting.api.modules.asset.Asset] to type [com.skywarelabs.runwayreporting.api.modules.asset.AssetPageView]
I have two repository methods which do essentialy the same thing.
I'm expecting them to return an object of <Page>.
If I call findPageByAerodrome, this works correctly.
However, If I call findPageByFilter, I get the error below.
..the reason I want to use findPageByFilter is that I wish to extend this with additional filtering parameters.
'''
public interface AssetRepository extends PageCrudRepository<Asset, Long> {
#Query("SELECT a FROM Asset a WHERE " +
"(a.aerodrome = :aerodrome)")
<T> Page<T> findPageByFilter(#Param("aerodrome") Aerodrome aerodrome,
Pageable pageable, Class<T> type);
<T> Page<T> findPageByAerodrome(#Param("aerodrome") Aerodrome aerodrome,
Pageable pageable, Class<T> type);
,,,
Use this instead
<T> Page<T> findAll(Specification s, Pageable pageable, Class<T> type);
and pass a custom Specification where you create your equality filter like this:
<T> Page<T> findPageByFilter(#Param("aerodrome") Aerodrome aerodrome,
Pageable pageable, Class<T> type) {
return findAll(
new Specification() {
#Override
public Predicate toPredicate(Root<T> root, CriteriaQuery query, CriteriaBuilder cb) {
return cb.equal(root.get("aerodrome"), aerodrome);
}
},
pageable,
type
);
}
Figured it out. I need to add an entity specific constructor.
'''
public AssetPageView(Asset asset) {
this ( asset.getId(), asset.getAssetType(), asset.getAerodrome(), asset.getName(),
asset.getAreaOfOperationsLoc(), asset.getDisplayOrder() );
}
public AssetPageView(Long id, AssetType assetType, Aerodrome aerodrome, String name,
String areaOfOperationsLoc, Long displayOrder) {
this.id = id;
this.assetType = assetType.getName();
this.aerodrome = aerodrome.getCallSign();
this.name = name;
this.areaOfOperationsLoc = areaOfOperationsLoc;
this.displayOrder = displayOrder;
}
'''

Add WHERE IN clause to JPA Specification

I'm trying to implement search functionality limited by IN clause:
I want to implement search implementation with filter limitation:
#GetMapping("find")
public Page<MerchantUserDTO> getAllBySpecification(
#And({
#Spec(path = "name", spec = LikeIgnoreCase.class),
#Spec(path = "login", spec = LikeIgnoreCase.class),
#Spec(path = "email", spec = LikeIgnoreCase.class),
}) Specification<Users> specification,
#SortDefault(sort = "login", direction = Sort.Direction.DESC) Pageable pageable
) {
return merchantUserService.getAllBySpecification(specification, pageable)
.map(g -> MerchantUserDTO.builder()
.id(g.getId())
.login(g.getLogin())
.build()
);
}
#Override
public Page<Users> getAllBySpecification(Specification<Users> specification, Pageable pageable) {
return dao.findAllByTypeIn(specification, pageable, "MerchantUser");
}
Repository:
#Repository
public interface MerchantUserRepository extends JpaRepository<Users, Integer>, JpaSpecificationExecutor<Users> {
Page<Users> findAllByTypeIn(Pageable page, String... types);
Page<Users> findAllByTypeIn(Specification<Users> specification, Pageable pageable, String... types);
}
What is the proper way to extend the specification with IN clause?
specification.and(path.in(types)) path is a attribute but how to implement it properly?
Generally this can be achieved this way:
1) Create specification implementation
public class MerchantUserSpecification implements Specification<Users> {
private final List<String> types;
public MerchantUserSpecification(List<String> types) {
this.types = types;
}
#Override
public Predicate toPredicate(Root<Users> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
if (types != null && !types.isEmpty()) {
return root.get(Users_.type).in(types);
} else {
// always-true predicate, means that no filtering would be applied
return cb.and();
}
}
2) Use method Page findAll(#Nullable Specification spec, Pageable pageable); inherited from JpaSpecificationExecutor interface instead of using your custom findAllByTypeIn(Specification<Users> specification....)
#Override
public Page<Users> getAllBySpecification(Specification<Users> specification, Pageable pageable) {
// combine original specification (passed from outside) and filter-by-types specification
Specification<Users> finalSpec = specification
.and(new MerchantUserSpecification(Arrays.asList("MerchantUser")))
return dao.findAll(finalSpec, pageable)
}
P.S.
With Java 8+ and for simple cases (like yours) the code may be reduced even more. Instead of implementing Specification<T> in separate class you can just create a method
private Specification<Users> typeIn(List<String> types) {
return (root, query, cb) -> {
if (types != null && !types.isEmpty()) {
return root.get(Users_.type).in(types);
} else {
// always-true predicate, means that no filtering would be applied
return cb.and();
}
}
}
#Override
public Page<Users> getAllBySpecification(Specification<Users> specification, Pageable pageable) {
// combine original specification (passed from outside) and filter-by-types specification
Specification<Users> finalSpec = specification
.and(typeIn(Arrays.asList("MerchantUser")))
return dao.findAll(finalSpec, pageable)
}
UPDATE: Shortest way
#Override
public Page<Users> getAllBySpecification(Specification<Users> specification, Pageable pageable) {
// combine original specification (passed from outside) and filter-by-types specification
Specification<Users> finalSpec = specification
.and((root, query, cb) -> root.get(Users_.type).in(Arrays.asList("MerchantUser"))
return dao.findAll(finalSpec, pageable)
}

spring jpa query with pageable, sort and filter and return projection

I am using Spring Data Rest with org.springframework.boot 1.5.2 with hibernate 5.2.9. What i am trying to achieve is a way to use JPA to query with sort, filter, pageable that can return a subset of the entity or return a projection.
Below is the code that uses:
(1) Specification for filtering
(2) Projection and Excerpts to apply projection in collection
(3) The controller that tries to return Page,
but it only works if the return type is Page.
where Student is the entity, StudentLite is the projection
Question is:
(1) How to have a query+sort+filter that returns Page projection
(2) Possible to apply the Excerpts to just that query?
(3) Any way to use #JsonView in #RepositoryRestController to solve?
StudentRepository class
#RepositoryRestResource(excerptProjection = StudentLite.class)
public interface StudentRepository extends PagingAndSortingRepository<Student,Long>,
JpaSpecificationExecutor<Student> {}
and
StudentSpecification class
public class StudentSpecification {
public static Specification<Student> filteredStudentList(StudentSearch c) {
final StudentSearch criteria = c;
return new Specification<Student>() {
#Override
public Predicate toPredicate(Root<Student> root, CriteriaQuery<?> query, CriteriaBuilder cb) {
Join<Student, Contact> joinContact = root.join(Student_.contact);
Path<Contact> contact = root.get(Student_.contact);
Path<String> officialId = root.get(Student_.officialId);
Path<String> name = root.get(Student_.name);
Path<String> email = contact.get(Contact_.email);
Path<String> phoneMobile = contact.get(Contact_.phoneMobile);
final List<Predicate> predicates = new ArrayList<Predicate>();
if(criteria.getOfficialId()!=null) {
predicates.add(cb.like(officialId, "%" + criteria.getOfficialId() + "%"));
System.out.println("==not null...criteria.getOfficialId()="+criteria.getOfficialId()+" :officialId="+officialId.toString());
}
if(criteria.getName()!=null) {
predicates.add(cb.like(name, "%"+criteria.getName()+"%"));
}
if(criteria.getEmail()!=null) {
predicates.add(cb.like(email, "%"+criteria.getEmail()+"%"));
}
if(criteria.getPhoneMobile()!=null) {
predicates.add(cb.like(phoneMobile, "%"+criteria.getPhoneMobile()+"%"));
}
return cb.and(predicates.toArray(new Predicate[predicates.size()]));
}
};
}
}
and the controller where the class is annotated with #ExposesResourceFor(Student.class) and #RepositoryRestController :
#RequestMapping(method=RequestMethod.GET)
public #ResponseBody Page<StudentLite> getStudentList(Pageable pageable, #RequestParam Map<String,String> criteria) {
StudentSearch ss = new StudentSearch(criteria);
// Below statement fail, as findAll(...) is suppose to return Page<Student>
Page<StudentLite> pagedStudentLite = studentRep.findAll( StudentSpecification.filteredStudentList(ss), pageable);
return pagedStudentLite;
}

Resources