How can you use different operators with QueryDSL Web? - spring

I am using QueryDSL in my Spring Boot project and planning to use Spring's web support for it (current query dsl web docs). The problem is, I can't find anything about using different operators. How can I define a not equals or matches regex operation? At first glance, all it does is translating your ?fieldname=value format GET request to a predefined operation you set in your repository. Can I extend it in a way to allow multiple operations for the same field?
Example.:
Currently I can get a QueryDsl Predicate by passing URL paramters, like ?user.company.id=1:
#Controller
class UserController {
#Autowired UserRepository repository;
#RequestMapping(value = "/", method = RequestMethod.GET)
Page<User> getUsers(#QuerydslPredicate(root = User.class) Predicate predicate,
Pageable pageable) {
return repository.findAll(predicate, pageable);
}
}
But as the documentation I linked states, I can only define a single operation for a certain field. What If I want the Users, where the user.lastName starts with something and still keep the possibility to query for exact match? (?lastName=Xyz,contains and ?lastName=Xyz,equals maybe)
The QuerydslBinderCustomizer defines operations per field basis, but you can only define how to handle that particular field, there is no possibility to add multiple operations.
Maybe I cannot do this with QueryDSL, but then generally in Spring boot how do you apply filters to a search query?

I'm doing something like that. Although I'm facing some limitations when I try to do more complicated actions. What I've done in some steps:
Create a new interface MyBinderCustomizer<T extends EntityPath<?>> that extends QuerydslBinderCustomizer<QUser> (note the Q of User, you want QueryDSL autogenerated class instead of your entity).
Implement customize method. For example:
#Override
public default void customize(QuerydslBindings bindings, T root) {
bindings.bind(String.class).all(MyBinderCustomizer::applyStringComparison);
}
static BooleanExpression applyStringComparison(Path<String> path, Collection<? extends String> strings) {
BooleanExpression result = null;
for (String s : strings) {
try {
final String[] parts = s.split(",");
final String operator = parts[0];
final String value = parts.length > 1 ? parts[1] : null;
final Method method = Arrays.stream(path.getClass().getMethods())
.filter(m -> operator.equals(m.getName()))
.filter(m -> BooleanExpression.class.equals(m.getReturnType()))
.filter(m -> m.getParameterTypes().length == (value == null ? 0 : 1))
.filter(m -> value == null || m.getParameterTypes()[0].equals(String.class) || m.getParameterTypes()[0].equals(Object.class))
.findFirst().get();
final BooleanExpression be;
if (value == null) {
be = (BooleanExpression) method.invoke(path);
} else {
be = (BooleanExpression) method.invoke(path, value);
}
result = result == null ? be : result.and(be);
} catch (Throwable t) {
throw new RuntimeException(t);
}
}
return result;
}
Note you should change value/operator order, so you can call no-value operators like isNull.
Your repository must extend MyBinderCustomizer<QUser> (note Q again).
This will let you use these operations:
public BooleanExpression StringExpression.like(java.lang.String)
public BooleanExpression StringExpression.notLike(java.lang.String)
public BooleanExpression StringExpression.notEqualsIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.containsIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.likeIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.startsWithIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.endsWithIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.equalsIgnoreCase(java.lang.String)
public BooleanExpression StringExpression.startsWith(java.lang.String)
public BooleanExpression StringExpression.endsWith(java.lang.String)
public BooleanExpression StringExpression.matches(java.lang.String)
public BooleanExpression StringExpression.contains(java.lang.String)
public BooleanExpression StringExpression.isEmpty()
public BooleanExpression StringExpression.isNotEmpty()
public BooleanExpression SimpleExpression.isNull()
public BooleanExpression SimpleExpression.isNotNull()
public BooleanExpression SimpleExpression.ne(java.lang.Object)
public BooleanExpression SimpleExpression.eq(java.lang.Object)

The Spring Data QueryDSL Value Operators library extends Spring Data QueryDSL web support with operators for not only String fields, but also Number and Enum fields. It requires some special configuration to make it work for the non-String fields, as explained here:
Value operators work seemlessly on String based properties/fields. However these operators do not work well with non-string values like Number or Enum since by default QuerydslPredicateArgumentResolver that resolves annotation QuerydslPredicate, which is used to annotate search handling method on RESTful method (aka RestController methods), performs strong-typing as per the guiding design principle of Querydsl, i.e. it attempts to convert the value(s) received from HTTP request to exact type defined in corresponding Q-Classes. This works well without value operators and is inline with Querydsl promise of allowing type-safe queries however hinders the path for value-operators to do their trick.
The library provides two methods to make operators work for non-String fields:
a Filter that extracts operators from query parameters, so the query parameters can still be converted to their corresponding type (using strong-typing)
replacing the ConversionService in the QuerydslPredicateArgumentResolver so all query parameters are treated as String (loosing the strong-typing)
Both approaches are well documented, along with their use case and disadvantages.
I am currently evaluating approach 1, as this fits our use case, but I need to extend it to accommodate DateTime fields and some custom operators as well.

https://bitbucket.org/gt_tech/spring-data-querydsl-value-operators/src/master/
Documentation here says:
QuerydslPredicateArgumentResolver uses ConversionService for type-conversion. Since conversion of String to Enum or String to Integer is core to Spring's dependency injection, it isn't advisable to change those default built-in converters (never do it). The library provides an experimental combination of a BeanPostProcessor and a ServletFilter that can be explicitly configured in target application's context to disable the strong type-conversion attempted by QuerydslPredicateArgumentResolver.
So to achieve this you need to add this to the application context:
/**
* Note the use of delegate ConversionService which comes handy for types like
* java.util.Date for handling powerful searches natively with Spring data.
* #param factory QuerydslBindingsFactory instance
* #param conversionServiceDelegate delegate ConversionService
* #return
*/
#Bean
public QuerydslPredicateArgumentResolverBeanPostProcessor querydslPredicateArgumentResolverBeanPostProcessor(
QuerydslBindingsFactory factory, DefaultFormattingConversionService conversionServiceDelegate) {
return new QuerydslPredicateArgumentResolverBeanPostProcessor(factory, conversionServiceDelegate);
}
Let me know if someone has success implementing this experimental functionality.

You can try using an additional lightweight library that helps to query fields using different operators LIKE, IN, EQ, NE etc. All you have to do is to add the dependency:
<dependency>
<groupId>io.github.apulbere</groupId>
<artifactId>rsql-querydsl</artifactId>
<version>1.0</version>
</dependency>
Define a model that will represent your search criteria:
#Setter
#Getter
public class UserCriteria {
StringCriteria lastName = StringCriteria.empty();
}
Use it as query parameter in your controller and build a dynamic predicate based on it:
#GetMapping("/users")
List<User> search(UserCriteria criteria, Pageable page) {
var predicate = criteria.lastName.match(QUser.user.lastName);
return userRepository.findAll(predicate, page);
}
Finally, make requests:
LIKE example: /users?lastName.like=Xyz
Equals examples: /users?lastName=Xyz or /users?lastName.eq=Xyz
There are other operators too.

Related

How to link a Vaadin Grid with the result of Spring Mono WebClient data

This seems to be a missing part in the documentation of Vaadin...
I call an API to get data in my UI like this:
#Override
public URI getUri(String url, PageRequest page) {
return UriComponentsBuilder.fromUriString(url)
.queryParam("page", page.getPageNumber())
.queryParam("size", page.getPageSize())
.queryParam("sort", (page.getSort().isSorted() ? page.getSort() : ""))
.build()
.toUri();
}
#Override
public Mono<Page<SomeDto>> getDataByPage(PageRequest pageRequest) {
return webClient.get()
.uri(getUri(URL_API + "/page", pageRequest))
.retrieve()
.bodyToMono(new ParameterizedTypeReference<>() {
});
}
In the Vaadin documentation (https://vaadin.com/docs/v10/flow/binding-data/tutorial-flow-data-provider), I found an example with DataProvider.fromCallbacks but this expects streams and that doesn't feel like the correct approach as I need to block on the requests to get the streams...
DataProvider<SomeDto, Void> lazyProvider = DataProvider.fromCallbacks(
q -> service.getData(PageRequest.of(q.getOffset(), q.getLimit())).block().stream(),
q -> service.getDataCount().block().intValue()
);
When trying this implementation, I get the following error:
org.springframework.core.codec.CodecException: Type definition error: [simple type, class org.springframework.data.domain.Page]; nested exception is com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `org.springframework.data.domain.Page` (no Creators, like default constructor, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
at [Source: (io.netty.buffer.ByteBufInputStream); line: 1, column: 1]
grid.setItems(lazyProvider);
I don't have experience with vaadin, so i'll talk about the deserialization problem.
Jackson needs a Creator when deserializing. That's either:
the default no-arg constructor
another constructor annotated with #JsonCreator
static factory method annotated with #JsonCreator
If we take a look at spring's implementations of Page - PageImpl and GeoPage, they have neither of those. So you have two options:
Write your custom deserializer and register it with the ObjectMapper instance
The deserializer:
public class PageDeserializer<T> extends StdDeserializer<Page<T>> {
public PageDeserializer() {
super(Page.class);
}
#Override
public Page<T> deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JacksonException {
//TODO implement for your case
return null;
}
}
And registration:
SimpleModule module = new SimpleModule();
module.addDeserializer(Page.class, new PageDeserializer<>());
objectMapper.registerModule(module);
Make your own classes extending PageImpl, PageRequest, etc. and annotate their constructors with #JsonCreator and arguments with #JsonProperty.
Your page:
public class MyPage<T> extends PageImpl<T> {
#JsonCreator
public MyPage(#JsonProperty("content_prop_from_json") List<T> content, #JsonProperty("pageable_obj_from_json") MyPageable pageable, #JsonProperty("total_from_json") long total) {
super(content, pageable, total);
}
}
Your pageable:
public class MyPageable extends PageRequest {
#JsonCreator
public MyPageable(#JsonProperty("page_from_json") int page, #JsonProperty("size_from_json") int size, #JsonProperty("sort_object_from_json") Sort sort) {
super(page, size, sort);
}
}
Depending on your needs for Sort object, you might need to create MySort as well, or you can remove it from constructor and supply unsorted sort, for example, to the super constructor. If you are deserializing from input manually you need to provide type parameters like this:
JavaType javaType = TypeFactory.defaultInstance().constructParametricType(MyPage.class, MyModel.class);
Page<MyModel> deserialized = objectMapper.readValue(pageString, javaType);
If the input is from request body, for example, just declaring the generic type in the variable is enough for object mapper to pick it up.
#PostMapping("/deserialize")
public ResponseEntity<String> deserialize(#RequestBody MyPage<MyModel> page) {
return ResponseEntity.ok("OK");
}
Personally i would go for the second option, even though you have to create more classes, it spares the tediousness of extracting properties and creating instances manually when writing deserializers.
There are two parts to this question.
The first one is about asynchronously loading data for a DataProvider in Vaadin. This isn't supported since Vaadin has prioritized the typical case with fetching data straight through JDBC. This means that you end up blocking a thread while the data is loading. Vaadin 23 will add support for doing that blocking on a separate thread instead of keeping the UI thread blocked, but it will still be blocking.
The other half of your problem doesn't seem to be directly related to Vaadin. The exception message says that the Jackson instance used by the REST client isn't configured to support creating instances of org.springframework.data.domain.Page. I don't have direct experience with this part of the problem, so I cannot give any advice on exactly how to fix it.

Spring Data - Build where clause at runtime

In Spring Data, how can I append more conditions to an existing query?
For example, I have the CrudRepository below:
#RepositoryRestResource
public interface MyRep extends CrudRepository<MyObject, Long> {
#Query("from MyObject mo where mo.attrib1 = :attrib1")
List<MyObj> findMyObjects(String attrib1, String conditions);
}
At runtime, I will need to call "findMyObjects" with two params. The first param is obviously the value of attrib1. the second param will be a where clause that would be determined at runtime, for example "attrib2 like '%xx%' and attrib3 between 'that' and 'this' and ...". I know this extra where condition will be valid, but I don't know what attributes and conditions will be in it. Is there anyway to append this where clause to the query defined in the #Query annotation?
Unfortunately, no. There is no straightforward way to achieve that.
You'll want to use custom reporistory methods where you'll be able to inject an EntityManager and interact with EntityManager.createQuery(...) directly.
Alternatively, you can build dynamic queries using Specifications or QueryDsl.
I ended up injecting an EntityManager that I obtained in the rest controller. Posting what I did here for criticism:
The repository code:
#RepositoryRestResource
public interface MyRepo extends CrudRepository<MyObject, Long> {
default List<MyObject> findByRuntimeConditions(EntityManager em, String runtimeConditions) {
String mySql = "<built my sql here. Watch for sql injection.>";
List<MyObject> list = em.createQuery(mySql).getResultList();
return list
}
}
The Rest controller code:
#RestController
public class DataController {
#Autowired
EntityManager em;
// of course watch for sql injection
#RequestMapping("myobjects/{runtimeConditions}")
public List<MyObject> getMyObjects(#PathVariable String runtimeConditions) {
List<MyObject> list = MyRepo.findByRuntimeConditions(em, runtimeConditions);
return list;
}
}

Using annotations in spring boot for putting data in correct format

I have a field in my entity that holds phone-number. According to the conventions of the project, I need to save it in E.164 format in the DB. At the moment I use #PrePersist and #PreUpdate annotations for changing the phone number to the specified format. This method is good for one or two entities but it becomes very error-prone when you have to repeat it over and over.
I was thinking that it would be awesome if I could put the code in annotation and the annotation reads the fields and changes its value just before the persistence something like what #LastModifiedDate and annotation do. I searched the web for the codes of this annotation but I didn't understand how they managed it.
How I can write an annotation that reads the value of a field and changes it before persistence, and how I can do it before some specific operations like delete (I want to set some params before deleting the object too)
Take a look at EntityListeners.
You can create a listener that checks your custom annotation and triggers the appropriate methods.
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.FIELD)
public #interface TheCustomAnnotation{
}
#Entity
#EntityListeners(TheListener.class)
public class TheEntity {
#TheCustomAnnotation
private String phoneNumber;
public class TheListener {
#PrePersist
public void prePersist(Object target) {
for(Field field : target.getClass().getDeclaredFields()){
Annotation[] annotations = field.getDeclaredAnnotations();
// Iterate annotations and check if yours is in it.
}
}
This is just an example.
#Pattern is a pretty powerful annotation that would be a good fit for validations if you are experienced with regular expressions.
For example,
#Pattern(regexp="^[0-9]{3}-[0-9]{3}-[0-9]{4}$")
private String phoneNumber;
The downside is that this only works for Strings though.
If you are interested more in conversions than validations, you may want to look into #JsonDeserialize if you are using Jackson.
For example:
#JsonDeserialize(using=PhoneNumberDeserializer.class)
private String phoneNumber;
Pattern phonePattern = Pattern.compile("^[0-9]{3}(.+)[0-9]{3}(.+)[0-9]{4}$");
public class PhoneNumberDeserializer extends JsonDeserializer<String> {
#Override
public String deserialize(JsonParser jsonParser,
DeserializationContext deserializationContext)
throws IOException, JsonProcessingException {
String phone = jsonParser.getText();
if (matcher.matches(phone)) {
Matcher matcher = phonePattern.matcher(phone);
for (int i = 1; i < matcher.groupCount(); i++) {
marcher.group(i).replaceAll(".*", "");
}
}
}
}
This will work for any type, not just strings.
Sorry it's a little convoluted, I was having fun reteaching myself.

Multi-Column Search with Spring JPA Specifications

I want to create a multi field search in a Spring-Boot back-end. How to do this with a Specification<T> ?
Environment
Springboot
Hibernate
Gradle
Intellij
The UI in the front end is a Jquery Datatable. Each column allows a single string search term to be applied. The search terms across more than one column is joined by a and.
I have the filters coming from the front end already getting populated into a Java object.
Step 1
Extend JPA Specification executor
public interface SomeRepository extends JpaRepository<Some, Long>, PagingAndSortingRepository<Some, Long>, JpaSpecificationExecutor {
Step2
Create a new class SomeSpec
This is where I am lost as to what the code looks like it and how it works.
Do I need a method for each column?
What is Root and what is Criteria Builder?
What else is required?
I am rather new at JPA so while I don't need anyone to write the code for me a detailed explanation would be good.
UPDATE
It appears QueryDSL is the easier and better way to approach this. I am using Gradle. Do I need to change my build.gradle from this ?
If you don't want to use QueryDSL, you'll have to write your own specifications. First of all, you need to extend your repository from JpaSpecificationExecutor like you did. Make sure to add the generic though (JpaSpecificationExecutor<Some>).
After that you'll have to create three specifications (one for each column), in the Spring docs they define these specifications as static methods in a class. Basically, creating a specification means that you'll have to subclass Specification<Some>, which has only one method to implement, toPredicate(Root<Some>, CriteriaQuery<?>, CriteriaBuilder).
If you're using Java 8, you can use lambdas to create an anonymous inner class, eg.:
public class SomeSpecs {
public static Specification<Some> withAddress(String address) {
return (root, query, builder) -> {
// ...
};
}
}
For the actual implementation, you can use Root to get to a specific node, eg. root.get("address"). The CriteriaBuilder on the other hand is to define the where clause, eg. builder.equal(..., ...).
In your case you want something like this:
public class SomeSpecs {
public static Specification<Some> withAddress(String address) {
return (root, query, builder) -> builder.equal(root.get("address"), address);
}
}
Or alternatively if you want to use a LIKE query, you could use:
public class SomeSpecs {
public static Specification<Some> withAddress(String address) {
return (root, query, builder) -> builder.like(root.get("address"), "%" + address + "%");
}
}
Now you have to repeat this for the other fields you want to filter on. After that you'll have to use all specifications together (using and(), or(), ...). Then you can use the repository.findAll(Specification) method to query based on that specification, for example:
public List<Some> getSome(String address, String name, Date date) {
return repository.findAll(where(withAddress(address))
.and(withName(name))
.and(withDate(date));
}
You can use static imports to import withAddress(), withName() and withDate() to make it easier to read. The where() method can also be statically imported (comes from Specification.where()).
Be aware though that the method above may have to be tweaked since you don't want to filter on the address field if it's null. You could do this by returning null, for example:
public List<Some> getSome(String address, String name, Date date) {
return repository.findAll(where(address == null ? null : withAddress(address))
.and(name == null ? null : withName(name))
.and(date == null ? null : withDate(date));
}
You could consider using Spring Data's support for QueryDSL as you would get quite a lot without having to write very much code i.e. you would not actually have to write the specifictions.
See here for an overview:
https://spring.io/blog/2011/04/26/advanced-spring-data-jpa-specifications-and-querydsl/
Although this approach is really convenient (you don’t even have to
write a single line of implementation code to get the queries
executed) it has two drawbacks: first, the number of query methods
might grow for larger applications because of - and that’s the second
point - the queries define a fixed set of criterias. To avoid these
two drawbacks, wouldn’t it be cool if you could come up with a set of
atomic predicates that you could combine dynamically to build your
query?
So essentially your repository becomes:
public interface SomeRepository extends JpaRepository<Some, Long>,
PagingAndSortingRepository<Some, Long>, QueryDslPredicateExecutor<Some>{
}
You can also get request parameters automatically bound to a predicate in your Controller:
See here:
https://spring.io/blog/2015/09/04/what-s-new-in-spring-data-release-gosling#querydsl-web-support
SO your Controller would look like:
#Controller
class SomeController {
private final SomeRepository repository;
#RequestMapping(value = "/", method = RequestMethod.GET)
String index(Model model,
#QuerydslPredicate(root = Some.class) Predicate predicate,
Pageable pageable) {
model.addAttribute("data", repository.findAll(predicate, pageable));
return "index";
}
}
So with the above in place it is simply a Case of enabling QueryDSL on your project and the UI should now be able to filter, sort and page data by various combinations of criteria.

override condition in Spring QuerydslPredicate

I'm using QuerydslPredicate (Spring 4.2.5, Spring Boot 1.3.3, querydsl-core 3.7.0) to create a search web service.
My Ticket entity has properties like name, description, etc.
I want a strict equality on the name field, but a "contains" comparison on the description.
The web service
public Page<Ticket> findAll(#QuerydslPredicate(root = Ticket.class) Predicate predicate, String description) {
BooleanBuilder builder = new BooleanBuilder(predicate);
if (isNotEmpty(description)) {
builder.and(QTicket.ticket.description.containsIgnoreCase(description));
}
return ticketService.findAll(builder, pageable);
}
Problem: when I query my web service like that: http...?description=foo, two comparisons are generated for the description (I started a debugger and looked at the generated BooleanBuilder). The pseudo-code looks like that: "description = foo AND description contains foo".
I'd like to keep the "contains" comparison only.
I found a workaround: I simply renamed web service's parameter description to descriptionFragment. This way, I can call http...?descriptionFragment=foo.
public Page<Ticket> findAll(#QuerydslPredicate(root = Ticket.class) Predicate predicate, String descriptionFragment) {
BooleanBuilder builder = new BooleanBuilder(predicate);
if (isNotEmpty(descriptionFragment)) {
builder.and(QTicket.ticket.description.containsIgnoreCase(descriptionFragment));
}
return ticketService.findAll(builder, pageable);
}
Question: I'd like to avoid this workaround. Is there a way to override default equality on the description field?
I found a solution: my TicketRepository should extends QuerydslBinderCustomizer
#Override
default void customize(QuerydslBindings bindings, QTicket qTicket) {
bindings.bind(qTicket.description).first(StringExpression::containsIgnoreCase);
}

Resources