Multi-Column Search with Spring JPA Specifications - spring

I want to create a multi field search in a Spring-Boot back-end. How to do this with a Specification<T> ?
Environment
Springboot
Hibernate
Gradle
Intellij
The UI in the front end is a Jquery Datatable. Each column allows a single string search term to be applied. The search terms across more than one column is joined by a and.
I have the filters coming from the front end already getting populated into a Java object.
Step 1
Extend JPA Specification executor
public interface SomeRepository extends JpaRepository<Some, Long>, PagingAndSortingRepository<Some, Long>, JpaSpecificationExecutor {
Step2
Create a new class SomeSpec
This is where I am lost as to what the code looks like it and how it works.
Do I need a method for each column?
What is Root and what is Criteria Builder?
What else is required?
I am rather new at JPA so while I don't need anyone to write the code for me a detailed explanation would be good.
UPDATE
It appears QueryDSL is the easier and better way to approach this. I am using Gradle. Do I need to change my build.gradle from this ?

If you don't want to use QueryDSL, you'll have to write your own specifications. First of all, you need to extend your repository from JpaSpecificationExecutor like you did. Make sure to add the generic though (JpaSpecificationExecutor<Some>).
After that you'll have to create three specifications (one for each column), in the Spring docs they define these specifications as static methods in a class. Basically, creating a specification means that you'll have to subclass Specification<Some>, which has only one method to implement, toPredicate(Root<Some>, CriteriaQuery<?>, CriteriaBuilder).
If you're using Java 8, you can use lambdas to create an anonymous inner class, eg.:
public class SomeSpecs {
public static Specification<Some> withAddress(String address) {
return (root, query, builder) -> {
// ...
};
}
}
For the actual implementation, you can use Root to get to a specific node, eg. root.get("address"). The CriteriaBuilder on the other hand is to define the where clause, eg. builder.equal(..., ...).
In your case you want something like this:
public class SomeSpecs {
public static Specification<Some> withAddress(String address) {
return (root, query, builder) -> builder.equal(root.get("address"), address);
}
}
Or alternatively if you want to use a LIKE query, you could use:
public class SomeSpecs {
public static Specification<Some> withAddress(String address) {
return (root, query, builder) -> builder.like(root.get("address"), "%" + address + "%");
}
}
Now you have to repeat this for the other fields you want to filter on. After that you'll have to use all specifications together (using and(), or(), ...). Then you can use the repository.findAll(Specification) method to query based on that specification, for example:
public List<Some> getSome(String address, String name, Date date) {
return repository.findAll(where(withAddress(address))
.and(withName(name))
.and(withDate(date));
}
You can use static imports to import withAddress(), withName() and withDate() to make it easier to read. The where() method can also be statically imported (comes from Specification.where()).
Be aware though that the method above may have to be tweaked since you don't want to filter on the address field if it's null. You could do this by returning null, for example:
public List<Some> getSome(String address, String name, Date date) {
return repository.findAll(where(address == null ? null : withAddress(address))
.and(name == null ? null : withName(name))
.and(date == null ? null : withDate(date));
}

You could consider using Spring Data's support for QueryDSL as you would get quite a lot without having to write very much code i.e. you would not actually have to write the specifictions.
See here for an overview:
https://spring.io/blog/2011/04/26/advanced-spring-data-jpa-specifications-and-querydsl/
Although this approach is really convenient (you don’t even have to
write a single line of implementation code to get the queries
executed) it has two drawbacks: first, the number of query methods
might grow for larger applications because of - and that’s the second
point - the queries define a fixed set of criterias. To avoid these
two drawbacks, wouldn’t it be cool if you could come up with a set of
atomic predicates that you could combine dynamically to build your
query?
So essentially your repository becomes:
public interface SomeRepository extends JpaRepository<Some, Long>,
PagingAndSortingRepository<Some, Long>, QueryDslPredicateExecutor<Some>{
}
You can also get request parameters automatically bound to a predicate in your Controller:
See here:
https://spring.io/blog/2015/09/04/what-s-new-in-spring-data-release-gosling#querydsl-web-support
SO your Controller would look like:
#Controller
class SomeController {
private final SomeRepository repository;
#RequestMapping(value = "/", method = RequestMethod.GET)
String index(Model model,
#QuerydslPredicate(root = Some.class) Predicate predicate,
Pageable pageable) {
model.addAttribute("data", repository.findAll(predicate, pageable));
return "index";
}
}
So with the above in place it is simply a Case of enabling QueryDSL on your project and the UI should now be able to filter, sort and page data by various combinations of criteria.

Related

Spring Data - Build where clause at runtime

In Spring Data, how can I append more conditions to an existing query?
For example, I have the CrudRepository below:
#RepositoryRestResource
public interface MyRep extends CrudRepository<MyObject, Long> {
#Query("from MyObject mo where mo.attrib1 = :attrib1")
List<MyObj> findMyObjects(String attrib1, String conditions);
}
At runtime, I will need to call "findMyObjects" with two params. The first param is obviously the value of attrib1. the second param will be a where clause that would be determined at runtime, for example "attrib2 like '%xx%' and attrib3 between 'that' and 'this' and ...". I know this extra where condition will be valid, but I don't know what attributes and conditions will be in it. Is there anyway to append this where clause to the query defined in the #Query annotation?
Unfortunately, no. There is no straightforward way to achieve that.
You'll want to use custom reporistory methods where you'll be able to inject an EntityManager and interact with EntityManager.createQuery(...) directly.
Alternatively, you can build dynamic queries using Specifications or QueryDsl.
I ended up injecting an EntityManager that I obtained in the rest controller. Posting what I did here for criticism:
The repository code:
#RepositoryRestResource
public interface MyRepo extends CrudRepository<MyObject, Long> {
default List<MyObject> findByRuntimeConditions(EntityManager em, String runtimeConditions) {
String mySql = "<built my sql here. Watch for sql injection.>";
List<MyObject> list = em.createQuery(mySql).getResultList();
return list
}
}
The Rest controller code:
#RestController
public class DataController {
#Autowired
EntityManager em;
// of course watch for sql injection
#RequestMapping("myobjects/{runtimeConditions}")
public List<MyObject> getMyObjects(#PathVariable String runtimeConditions) {
List<MyObject> list = MyRepo.findByRuntimeConditions(em, runtimeConditions);
return list;
}
}

How can you use different operators with QueryDSL Web?

I am using QueryDSL in my Spring Boot project and planning to use Spring's web support for it (current query dsl web docs). The problem is, I can't find anything about using different operators. How can I define a not equals or matches regex operation? At first glance, all it does is translating your ?fieldname=value format GET request to a predefined operation you set in your repository. Can I extend it in a way to allow multiple operations for the same field?
Example.:
Currently I can get a QueryDsl Predicate by passing URL paramters, like ?user.company.id=1:
#Controller
class UserController {
#Autowired UserRepository repository;
#RequestMapping(value = "/", method = RequestMethod.GET)
Page<User> getUsers(#QuerydslPredicate(root = User.class) Predicate predicate,
Pageable pageable) {
return repository.findAll(predicate, pageable);
}
}
But as the documentation I linked states, I can only define a single operation for a certain field. What If I want the Users, where the user.lastName starts with something and still keep the possibility to query for exact match? (?lastName=Xyz,contains and ?lastName=Xyz,equals maybe)
The QuerydslBinderCustomizer defines operations per field basis, but you can only define how to handle that particular field, there is no possibility to add multiple operations.
Maybe I cannot do this with QueryDSL, but then generally in Spring boot how do you apply filters to a search query?
I'm doing something like that. Although I'm facing some limitations when I try to do more complicated actions. What I've done in some steps:
Create a new interface MyBinderCustomizer<T extends EntityPath<?>> that extends QuerydslBinderCustomizer<QUser> (note the Q of User, you want QueryDSL autogenerated class instead of your entity).
Implement customize method. For example:
#Override
public default void customize(QuerydslBindings bindings, T root) {
bindings.bind(String.class).all(MyBinderCustomizer::applyStringComparison);
}
static BooleanExpression applyStringComparison(Path<String> path, Collection<? extends String> strings) {
BooleanExpression result = null;
for (String s : strings) {
try {
final String[] parts = s.split(",");
final String operator = parts[0];
final String value = parts.length > 1 ? parts[1] : null;
final Method method = Arrays.stream(path.getClass().getMethods())
.filter(m -> operator.equals(m.getName()))
.filter(m -> BooleanExpression.class.equals(m.getReturnType()))
.filter(m -> m.getParameterTypes().length == (value == null ? 0 : 1))
.filter(m -> value == null || m.getParameterTypes()[0].equals(String.class) || m.getParameterTypes()[0].equals(Object.class))
.findFirst().get();
final BooleanExpression be;
if (value == null) {
be = (BooleanExpression) method.invoke(path);
} else {
be = (BooleanExpression) method.invoke(path, value);
}
result = result == null ? be : result.and(be);
} catch (Throwable t) {
throw new RuntimeException(t);
}
}
return result;
}
Note you should change value/operator order, so you can call no-value operators like isNull.
Your repository must extend MyBinderCustomizer<QUser> (note Q again).
This will let you use these operations:
public BooleanExpression StringExpression.like(java.lang.String)
public BooleanExpression StringExpression.notLike(java.lang.String)
public BooleanExpression StringExpression.notEqualsIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.containsIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.likeIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.startsWithIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.endsWithIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.equalsIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.startsWith(java.lang.String)
public BooleanExpression StringExpression.endsWith(java.lang.String)
public BooleanExpression StringExpression.matches(java.lang.String)
public BooleanExpression StringExpression.contains(java.lang.String)
public BooleanExpression StringExpression.isEmpty()
public BooleanExpression StringExpression.isNotEmpty()
public BooleanExpression SimpleExpression.isNull()
public BooleanExpression SimpleExpression.isNotNull()
public BooleanExpression SimpleExpression.ne(java.lang.Object)
public BooleanExpression SimpleExpression.eq(java.lang.Object)
The Spring Data QueryDSL Value Operators library extends Spring Data QueryDSL web support with operators for not only String fields, but also Number and Enum fields. It requires some special configuration to make it work for the non-String fields, as explained here:
Value operators work seemlessly on String based properties/fields. However these operators do not work well with non-string values like Number or Enum since by default QuerydslPredicateArgumentResolver that resolves annotation QuerydslPredicate, which is used to annotate search handling method on RESTful method (aka RestController methods), performs strong-typing as per the guiding design principle of Querydsl, i.e. it attempts to convert the value(s) received from HTTP request to exact type defined in corresponding Q-Classes. This works well without value operators and is inline with Querydsl promise of allowing type-safe queries however hinders the path for value-operators to do their trick.
The library provides two methods to make operators work for non-String fields:
a Filter that extracts operators from query parameters, so the query parameters can still be converted to their corresponding type (using strong-typing)
replacing the ConversionService in the QuerydslPredicateArgumentResolver so all query parameters are treated as String (loosing the strong-typing)
Both approaches are well documented, along with their use case and disadvantages.
I am currently evaluating approach 1, as this fits our use case, but I need to extend it to accommodate DateTime fields and some custom operators as well.
https://bitbucket.org/gt_tech/spring-data-querydsl-value-operators/src/master/
Documentation here says:
QuerydslPredicateArgumentResolver uses ConversionService for type-conversion. Since conversion of String to Enum or String to Integer is core to Spring's dependency injection, it isn't advisable to change those default built-in converters (never do it). The library provides an experimental combination of a BeanPostProcessor and a ServletFilter that can be explicitly configured in target application's context to disable the strong type-conversion attempted by QuerydslPredicateArgumentResolver.
So to achieve this you need to add this to the application context:
/**
* Note the use of delegate ConversionService which comes handy for types like
* java.util.Date for handling powerful searches natively with Spring data.
* #param factory QuerydslBindingsFactory instance
* #param conversionServiceDelegate delegate ConversionService
* #return
*/
#Bean
public QuerydslPredicateArgumentResolverBeanPostProcessor querydslPredicateArgumentResolverBeanPostProcessor(
QuerydslBindingsFactory factory, DefaultFormattingConversionService conversionServiceDelegate) {
return new QuerydslPredicateArgumentResolverBeanPostProcessor(factory, conversionServiceDelegate);
}
Let me know if someone has success implementing this experimental functionality.
You can try using an additional lightweight library that helps to query fields using different operators LIKE, IN, EQ, NE etc. All you have to do is to add the dependency:
<dependency>
<groupId>io.github.apulbere</groupId>
<artifactId>rsql-querydsl</artifactId>
<version>1.0</version>
</dependency>
Define a model that will represent your search criteria:
#Setter
#Getter
public class UserCriteria {
StringCriteria lastName = StringCriteria.empty();
}
Use it as query parameter in your controller and build a dynamic predicate based on it:
#GetMapping("/users")
List<User> search(UserCriteria criteria, Pageable page) {
var predicate = criteria.lastName.match(QUser.user.lastName);
return userRepository.findAll(predicate, page);
}
Finally, make requests:
LIKE example: /users?lastName.like=Xyz
Equals examples: /users?lastName=Xyz or /users?lastName.eq=Xyz
There are other operators too.

How does Spring Data JPA work internally

I was going through Spring Data JPA Tutorial.
I am confused on how does this framework work internally.
Let me state specific scenario
There was specific code
/**
* Custom finder
*/
public List<Location> getLocationByStateName(String name) {
#SuppressWarnings("unchecked")
List<Location> locs = entityManager
.createQuery("select l from Location l where l.state like :state")
.setParameter("state", name + "%").getResultList(); // note
return locs;
}
This was simply replaced by following interface
#Repository
public interface LocationJPARepository extends JpaRepository<Location, Long> {
List<Location> findByStateLike(String stateName);
}
And corresponding test case worked fine
#Test
public void testFindWithLike() throws Exception {
List<Location> locs = locationRepository.getLocationByStateName("New");
assertEquals(4, locs.size());
}
New test case
#Test
public void testFindWithLike() throws Exception {
List<Location> locs = locationJPARepository.findByStateLike("New");
assertEquals(4, locs.size());
}
My question
How does framework know if i am looking for exact match using = or partial match using SQL like operator (it cant be method name ?)
if it somehow decide I am looking for partial match then still there are sub options ... like name% or %name or %name% …
Also how it decides case is important in like ? ( i can have case-insensitive by using SQL like with toUpper() i.e. by comparing everything in upper case )
(added ques) is there a way i can check the EXACT SQL in log some where ??
Hope i was able to explain my question properly. Let me know if i need to add in more clarity.
I recommend to take a look at Query Creation section of the reference guide. It explains the rules pretty clearly.
For instance when you want to find User by first name and ignore case, you would use method name like findByFirstnameIgnoreCase which would translate into condition like UPPER(x.firstame) = UPPER(?1).
By default when you have findByProperty method, the match is exact, so if you want to have LIKE functionality you would use method name findByFirstnameLike which would in turn translate into condition where x.firstname like ?1.
You can combine these keywords, but it can get a little crazy. Personally I prefer using #Query annotation for more complicated queries to avoid super long repository method names.

Return custom-typed object from JpaRepository

I have the following repository:
public interface UserRepository extends BaseDAO<User> {
Collection<User> findByEmail(#Param("email") String email);
#Query("select new com.data.CustomUser(upper(substring(u.lastName, 1, 1)) as initial, count(*)) from User u join u.chats c where c.business=:business group by upper(substring(u.lastName, 1, 1)) order by initial")
List<CustomUser> getContactsIndex(#Param("email") String email);
}
which is exposed with Spring Data REST. The User object is a managed entity, while CustomUser not and as you can see, it's build on-fly by using custom query.
Once I want to call that function, it fails with Persistent entity must not be a null! exception. Is there any way to implement this behavior?
P.S. Expose CustomUser with separate repository is impossible because it is not a managed entity.
One challenge with using Spring Data Rest is when you hit an edge case and you don't know whether you've hit a bug or whether you're just outside the scope of what the library is intended for. In this case I think you are at the edge of what SDR will easily do for you, and it's time to implement your own controller.
Spring Data Rest is looking for an Entity - in your case a User - as the return type for ALL methods in the repository to expose under /entities/search, and breaks when it doesn't find that entity type. The User it wants to serialize isn't there, hence the "Persistent entity must not be null".
The way around this is to write a simple #Controller that has a #RequestMapping for the exact same url exposed by the repository method. This will override the SDR generated implementation for that url, and from that you can return whatever you want.
Your implementation might look something like this:
#Controller
public class CustomUserController {
private final UserRepository repository;
#Inject
public CustomUserController(UserRepository repo) {
repository = repo;
}
#RequestMapping(value = "/users/search/getContactsIndex", method = GET, produces = {MediaType.APPLICATION_JSON_VALUE})
public #ResponseBody List<CustomUser> getContactsIndex(#RequestParam String email) {
return repository.getContactsIndex(email);
}
}
Be aware that there is a "recommended" way to override functionality this way. There is an open issue to document the best way to do this.

Why is there a duplicate variable use in Spring Data JPA and QueryDSL?

There are a lot of examples online of using QueryDSL like this:
public class CustomerRepositoryImpl
extends QueryDslRepositorySupport
implements CustomerRepositoryCustom {
public Iterable<Customer> findAllLongtermCustomersWithBirthday() {
QCustomer customer = QCustomer.customer;
return from(customer)
.where(hasBirthday().and(isLongTermCustomer()))
.list(customer);
}
}
This code makes sense, but I am wondering why customer is "duplicated" in the method call to list().
Shouldn't the type be obvious from the reference in from(customer)?
from defines the source and list the projection. Querydsl query classes don't have any generic type argument for the return type, the projection (select part) is defined in the last part of the query construction chain.
Examples for cases where a different projection than the source is wanted
specific columns only: query.list(customer.firstName, customer.lastName)
constructor invocation : Projections.constructor(...)
Bean population: Projections.bean(...)
multiple from calls are used

Resources