Passing array of reg expressions to spring based mongo #Query - spring

I'm using Spring boot with mongodb. I've extended PagingAndSortingRepository repository and added the following function
#Query("{'title':{ $nin: [?0]}}")
List<Item> findItem(String[] exclude);
I want to be able to pass it an array of regular expressions such as /dog/,/cat/,/horse/ to exclude any item that may have one of these in it's title.
The above function does not work because the exclude is converted to a string. How can I pass an array of regular expressions to be able to do the above?

You can work it out by using a Querydsl predicate in one of your controller method.
Add something like this to your controller:
#RequestMapping(value="/search/findByNameRegexNotIn", method = RequestMethod.GET)
#ResponseBody
public List<Item> findByNameRegexNotIn(#RequestParam(name = "name") List<String> names) {
// build a query predicate
BooleanBuilder predicate = new BooleanBuilder(); // comes from the Querydsl library
for (String name : names) {
predicate.and(QItem.item.name.contains(name).not()); // the QItem class is generated by Querydsl
}
List<Item> items = (List<Item>)repository.findAll(predicate);
return items;
}
You can of course add a Pageable parameter and return a Page<Item> instead of a List.
Edit: another solution if you use Querydsl for this sole purpose is to override the default bindings of your query parameter.
public interface ItemRepository extends CrudRepository<Item, String>,
QueryDslPredicateExecutor<Item>, QuerydslBinderCustomizer<QItem> {
#Override
default public void customize(QuerydslBindings bindings, QItem item) {
bindings.bind(item.name).all(
(path, values) -> path.matches(StringUtils.collectionToDelimitedString(values, "|")).not());
// disable query on all parameters but the item name
bindings.including(item.name);
bindings.excludeUnlistedProperties(true);
}
}
The controller method:
#RequestMapping(value="/search/query", method = RequestMethod.GET)
#ResponseBody
public List<Item> queryItems(
#QuerydslPredicate(root = Item.class) Predicate predicate) {
List<Item> items = (List<Item>)repository.findAll(predicate);
return items;
}
Edit: if you don't wan't to override the default QuerydslBinderCustomizer#customize, you can also implement your own binder and specify it in the controller method.
public interface ItemRepository extends CrudRepository<Item, String>,
QueryDslPredicateExecutor<Item> {
...
}
The controller method:
#RequestMapping(value="/search/query", method = RequestMethod.GET)
#ResponseBody
public List<Item> queryItems(
#QuerydslPredicate(root = Item.class, bindings = ItemBinder.class) Predicate predicate) {
List<Item> items = (List<Item>)repository.findAll(predicate);
return items;
}
The binder class:
class ItemBinder implements QuerydslBinderCustomizer<QItem> {
#Override
public void customize(QuerydslBindings bindings, QItem item) {
bindings.bind(item.name).all(
(path, values) -> path.matches(StringUtils.collectionToDelimitedString(values, "|")).not()
);
bindings.including(item.name);
bindings.excludeUnlistedProperties(true);
}
}
Edit: for the sake of exhaustivity and those who don't want to hear about Querysl. Using the solution proposed in Spring Data Mongodb Reference.
Define a custom repository interface:
interface ItemRepositoryCustom {
public Page<Item> findByNameRegexIn(Collection<String> names, Pageable page);
}
Define an custom repository implementation (Impl postfix required!):
public class ItemRepositoryImpl implements ItemRepositoryCustom {
#Autowired
private MongoOperations operations;
#Override
public Page<Item> findByNameRegexNotIn(Collection<String> names, Pageable pageable) {
String pattern = StringUtils.collectionToDelimitedString(names, "|");
// this time we use org.springframework.data.mongodb.core.query.Query instead of Querydsl predicates
Query query = Query.query(where("name").regex(pattern).not()).with(pageable);
List<Item> items = operations.find(query, Item.class);
Page<Item> page = new PageImpl<>(items, pageable, items.size());
return page;
}
}
Now simply extend ItemRepositoryCustom:
public interface ItemRepository extends MongoRepository<Item, String>, ItemRepositoryCustom {
...
}
And you're done!

You can pass a java.util.regex.Pattern[] to the method. This will be converted to regex array under the hood:
#Query("{'title':{ $nin: ?0}}")
List<Item> findItem(Pattern[] exclude);

Related

Spring data elasticsearch embedded field mapping

I'm struggling with mappings some field. I looked for an answer but couldn't find anything solving my case. Let's cut to the chase.
I have my document class
#Doucment
public class DocumentClass {
#Field(type = FieldType.Nested)
private EmployeeId employeeId;
}
An EmployeeId is wrapper for my uuid identifier. This object has nothing but just getters and setters and jackson annotations. The thing is that object extends some base class so such objects like EmployeeId can inherit this object. This super class has field id and this causes the problem. When I post some data to elasticsearch then it looks like this:
{
"employeeId": {
"id": "someUUID"
}
}
But I want to map this to be like:
{
"employeeId": "someUUID"
}
I wonder if there is a way to flatten this object.
If I get it right, you want to convert your EmployeeId class to a String and back. You have 2 possibilities to do that:
Using a property converter
If you only want to convert an EmployeeId in this entity and might keep it as it is in another, you should use a property converter that is only registered for this property:
import org.springframework.data.elasticsearch.core.mapping.PropertyValueConverter;
public class EmployeeIdConverter implements PropertyValueConverter {
#Override
public Object write(Object value) {
return value instanceof EmployeeId employeeId ? employeeId.getId() : value.toString();
}
#Override
public Object read(Object value) {
return new EmployeeId(value.toString());
}
}
This converter must be registered on the property, notice that the field type is set to Keyword as it probably should not be analysed:
import org.springframework.data.elasticsearch.annotations.ValueConverter;
#Document
public class DocumentClass {
#Field(type = FieldType.Keyword)
#ValueConverter(EmployeeIdConverter.class)
private EmployeeId employeeId;
}
Using a global converter
If you are using this EmployeeId at several places you might want register globally 2 converters for the two conversion directions:
#WritingConverter
public class EmployeeIdToString implements Converter<EmployeeId, String>{
#Nullable
#Override
public String convert(EmployeeId employeeId) {
return employeeId.getId();
}
}
#ReadingConverter
public class StringToEmployeeId implements Converter<String, EmployeeId>{
#Nullable
#Override
public EmployeeId convert(String id) {
return new EmployeeId(id);
}
}
To register these, you need to provide a custom client configuration (see the documentation):
#Configuration
public class MyClientConfig extends ElasticsearchConfiguration {
#Override
public ClientConfiguration clientConfiguration() {
return ClientConfiguration.builder()
.connectedTo("localhost:9200")
.build();
}
#Override
public ElasticsearchCustomConversions elasticsearchCustomConversions() {
Collection<Converter<?, ?>> converters = new ArrayList<>();
converters.add(new EmployeeIdToString());
converters.add(new StringToEmployeeId());
return new ElasticsearchCustomConversions(converters);
}
}
In this case, only the field type needs to be adjusted
#Document
public class DocumentClass {
#Field(type = FieldType.Keyword)
private EmployeeId employeeId;
}
I am guessing you have an Id field which is a string.
You need to put a #JsonValue annotation on that field to make jackson serialize it the way you want.
Field annotated by JsonValue will be used to serialize your pojo into json.
If your Id field is private, then add the annotation on the getter of that field.
https://fasterxml.github.io/jackson-annotations/javadoc/2.8/com/fasterxml/jackson/annotation/JsonValue.html

Filter results from QueryDSL search

I'm using QueryDSL as part of Spring Data Rest to search entities from our API.
Is it possible to somehow filter the search API, so that by default it won't find for example Car entities that are "deactivated"?
Currently I've a flag on car entity that when it's set to true, it shouldn't be exposed through our search API, and the cars that have this property set should be left out from search.
https://docs.spring.io/spring-data/jpa/docs/1.9.0.RELEASE/reference/html/#core.web.type-safe
In case of using Spring Data REST and QueryDSL, to change standard behavior of queries we can use aspects.
For example: we need to show by default only those Models whose flag is set to true:
#Data
#NoArgsConstructor
#Entity
public class Model {
#Id #GeneratedValue private Integer id;
#NotBlank private String name;
private boolean flag;
}
In this case we implement the aspect like this:
#Aspect
#Component
public class ModelRepoAspect {
#Pointcut("execution(* com.example.ModelRepo.findAll(com.querydsl.core.types.Predicate, org.springframework.data.domain.Pageable))")
public void modelFindAllWithPredicateAndPageable() {
}
#Around("modelFindAllWithPredicateAndPageable()")
public Object filterModelsByFlag(final ProceedingJoinPoint pjp) throws Throwable {
Object[] args = pjp.getArgs();
Predicate predicate = (Predicate) args[0];
BooleanExpression flagIsTrue = QModel.model.flag.eq(true);
if (predicate == null) {
args[0] = flagIsTrue;
} else {
if (!predicate.toString().contains("model.flag")) {
args[0] = flagIsTrue.and(predicate);
}
}
return pjp.proceed(args);
}
}
This aspect intercepts all calls of method findAll(Predicate predicate, Pageable pageable) of our repo, and add the filter model.flag = true to the query if the request parameters was not set (predicate == null), or if they don't contain 'flag' parameter. Otherwise aspect does not modify original predicate.

Get all documents from an index using spring-data-elasticsearch

I am trying to connect to my external ElasticSearch server with Spring Boot.
If I do a curl from command line, I get expected results.
curl "http://ipAddr:9200/indexName/TYPE/_search?pretty=true"
But getting this error when I try to access it via Spring Boot.
<html><body><h1>Whitelabel Error Page</h1><p>This application has no explicit mapping for /error, so you are seeing this as a fallback.</p><div id='created'>Mon Sep 11 12:39:15 IST 2017</div><div>There was an unexpected error (type=Internal Server Error, status=500).</div><div>Could not write JSON: (was java.lang.NullPointerException); nested exception is com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: java.util.ArrayList[0]->org.springframework.data.elasticsearch.core.aggregation.impl.AggregatedPageImpl["facets"])</div></body></html>
Not sure why a NullPointerException and what is aggregartion.impl
Here is my Spring Application:
Controller:
#RestController
public class PojoController {
#Autowired
PojoService pojoService;
#RequestMapping(value = "/", method=RequestMethod.GET)
public #ResponseBody String index() {
return new String("Welcome:)");
}
#RequestMapping(value = "/all", method = RequestMethod.GET,
produces = { MediaType.APPLICATION_JSON_VALUE })
#ResponseBody List<POJO> findAll() {
try {
List<POJO> pojoObj = pojoService.findAll();
return pojoObj;
} catch (Exception exp) {
exp.printStackTrace();
return null;
}
}
}
Repository:
#Repository
public interface PojoRepository extends ElasticsearchRepository<POJO, Integer> {
List<POJO> findAll();
}
Service:
#Service
public class POJOServiceImpl implements POJOService{
private POJORepository pojoRepository;
private ElasticsearchTemplate elasticsearchTemplate;
#Autowired
public void setPojoRepository(PojoRepository pojoRepository) {
this.pojoRepository = pojoRepository;
}
public POJO findOne(String id) {
return pojoRepository.findOne(id);
}
public List<POJO> findAll() {
return (List<POJO>) pojoRepository.findAll();
}
}
POJO class:
#Document(indexName = "INDEX", type = "TYPE")
public class POJO {
#Id
private Integer id;
private String name;
public POJO(){
// empty
}
public POJO(Integerid, String name) {
super();
this.id = id;
this.name = name;
}
// getters and setters
}
I should be able to query all the documents in the index. Later on, I will try and use filters etc.
Any help is appreciated. Thanks :)
It looks like Jackson has a problem with handling your POJO (probably related to this issue: DATAES-274) - the problematic part is casting in repository from Iterable collection to List.
Update
In case of repositories, spring-data-elasticsearch behaves a bit different than you would expect. Taking your example:
#Repository
public interface PojoRepository extends ElasticsearchRepository<POJO, Integer> {
List<POJO> findAll();
}
and after calling in your rest controller:
List<POJO> pojoObj = pojoService.findAll();
in debugger you will see something like this:
You would expect that pojoObj list contains objects of POJO class.
And here comes the surprise - pojoObj ArrayList contains one object of AggregatedPageImpl type and its content field is the right list that contains your POJO objects.
This is the reason why you get:
Could not write JSON: ... java.util.ArrayList[0]->org.springframework.data.elasticsearch.core.aggregation.impl.AggregatedPageImpl[\"facets\"])
As I wrote before, Jackson cannot handle this while serializing POJO objects.
Solution 1
Let repositories return Iterable collection (by default).
#Repository
public interface PojoRepository extends ElasticsearchRepository<POJO, Integer> {
}
Move the conversion part to the service but use some utility method (here with Guava) in order to have it like this:
import com.google.common.collect.Lists;
public List<POJO> findAll() {
return Lists.newArrayList(pojoRepository.findAll());
}
Solution 2
Use Page in repository (here simplified version without parameters):
#Repository
public interface PojoRepository extends ElasticsearchRepository<POJO, Integer> {
Page<TestDto> findAll();
}
If you still want to operate on list - get content from page in service:
public List<POJO> findAll() {
return testDtoRepository.findAll().getContent();
}

Pass method argument in Aspect of custom annotation

I'm trying to use something similar to org.springframework.cache.annotation.Cacheable :
Custom annotation:
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface CheckEntity {
String message() default "Check entity msg";
String key() default "";
}
Aspect:
#Component
#Aspect
public class CheckEntityAspect {
#Before("execution(* *.*(..)) && #annotation(checkEntity)")
public void checkEntity(JoinPoint joinPoint, CheckEntitty checkEntity) {
System.out.println("running entity check: " + joinPoint.getSignature().getName());
}
}
Service:
#Service
#Transactional
public class EntityServiceImpl implements EntityService {
#CheckEntity(key = "#id")
public Entity getEntity(Long id) {
return new Entity(id);
}
}
My IDE (IntelliJ) doesn't see anything special with the key = "#id" usage in contrast to similar usages for Cacheable where it's shown with different color than plain text. I'm mentioning the IDE part just as a hint in case it helps, it looks like the IDE is aware in advance about these annotations or it just realizes some connection which doesn't exist in my example.
The value in the checkEntity.key is '#id' instead of an expected number.
I tried using ExpressionParser but possibly not in the right way.
The only way to get parameter value inside the checkEntity annotation is by accessing the arguments array which is not what I want because this annotation could be used also in methods with more than one argument.
Any idea?
Adding another simpler way of doing it using Spring Expression. Refer below:
Your Annotation:
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface CheckEntity {
String message() default "Check entity msg";
String keyPath() default "";
}
Your Service:
#Service
#Transactional
public class EntityServiceImpl implements EntityService {
#CheckEntity(keyPath = "[0]")
public Entity getEntity(Long id) {
return new Entity(id);
}
#CheckEntity(keyPath = "[1].otherId")
public Entity methodWithMoreThanOneArguments(String message, CustomClassForExample object) {
return new Entity(object.otherId);
}
}
class CustomClassForExample {
Long otherId;
}
Your Aspect:
#Component
#Aspect
public class CheckEntityAspect {
#Before("execution(* *.*(..)) && #annotation(checkEntity)")
public void checkEntity(JoinPoint joinPoint, CheckEntitty checkEntity) {
Object[] args = joinPoint.getArgs();
ExpressionParser elParser = new SpelExpressionParser();
Expression expression = elParser.parseExpression(checkEntity.keyPath());
Long id = (Long) expression.getValue(args);
// Do whatever you want to do with this id
// This works for both the service methods provided above and can be re-used for any number of similar methods
}
}
PS: I am adding this solution because I feel this is a simpler/clearner approach as compared to other answers and this might be helpful for someone.
Thanks to #StéphaneNicoll I managed to create a first version of a working solution:
The Aspect
#Component
#Aspect
public class CheckEntityAspect {
protected final Log logger = LogFactory.getLog(getClass());
private ExpressionEvaluator<Long> evaluator = new ExpressionEvaluator<>();
#Before("execution(* *.*(..)) && #annotation(checkEntity)")
public void checkEntity(JoinPoint joinPoint, CheckEntity checkEntity) {
Long result = getValue(joinPoint, checkEntity.key());
logger.info("result: " + result);
System.out.println("running entity check: " + joinPoint.getSignature().getName());
}
private Long getValue(JoinPoint joinPoint, String condition) {
return getValue(joinPoint.getTarget(), joinPoint.getArgs(),
joinPoint.getTarget().getClass(),
((MethodSignature) joinPoint.getSignature()).getMethod(), condition);
}
private Long getValue(Object object, Object[] args, Class clazz, Method method, String condition) {
if (args == null) {
return null;
}
EvaluationContext evaluationContext = evaluator.createEvaluationContext(object, clazz, method, args);
AnnotatedElementKey methodKey = new AnnotatedElementKey(method, clazz);
return evaluator.condition(condition, methodKey, evaluationContext, Long.class);
}
}
The Expression Evaluator
public class ExpressionEvaluator<T> extends CachedExpressionEvaluator {
// shared param discoverer since it caches data internally
private final ParameterNameDiscoverer paramNameDiscoverer = new DefaultParameterNameDiscoverer();
private final Map<ExpressionKey, Expression> conditionCache = new ConcurrentHashMap<>(64);
private final Map<AnnotatedElementKey, Method> targetMethodCache = new ConcurrentHashMap<>(64);
/**
* Create the suitable {#link EvaluationContext} for the specified event handling
* on the specified method.
*/
public EvaluationContext createEvaluationContext(Object object, Class<?> targetClass, Method method, Object[] args) {
Method targetMethod = getTargetMethod(targetClass, method);
ExpressionRootObject root = new ExpressionRootObject(object, args);
return new MethodBasedEvaluationContext(root, targetMethod, args, this.paramNameDiscoverer);
}
/**
* Specify if the condition defined by the specified expression matches.
*/
public T condition(String conditionExpression, AnnotatedElementKey elementKey, EvaluationContext evalContext, Class<T> clazz) {
return getExpression(this.conditionCache, elementKey, conditionExpression).getValue(evalContext, clazz);
}
private Method getTargetMethod(Class<?> targetClass, Method method) {
AnnotatedElementKey methodKey = new AnnotatedElementKey(method, targetClass);
Method targetMethod = this.targetMethodCache.get(methodKey);
if (targetMethod == null) {
targetMethod = AopUtils.getMostSpecificMethod(method, targetClass);
if (targetMethod == null) {
targetMethod = method;
}
this.targetMethodCache.put(methodKey, targetMethod);
}
return targetMethod;
}
}
The Root Object
public class ExpressionRootObject {
private final Object object;
private final Object[] args;
public ExpressionRootObject(Object object, Object[] args) {
this.object = object;
this.args = args;
}
public Object getObject() {
return object;
}
public Object[] getArgs() {
return args;
}
}
I think you probably misunderstand what the framework is supposed to do for you vs. what you have to do.
SpEL support has no way to be triggered automagically so that you can access the actual (resolved) value instead of the expression itself. Why? Because there is a context and as a developer you have to provide this context.
The support in Intellij is the same thing. Currently Jetbrains devs track the places where SpEL is used and mark them for SpEL support. We don't have any way to conduct the fact that the value is an actual SpEL expression (this is a raw java.lang.String on the annotation type after all).
As of 4.2, we have extracted some of the utilities that the cache abstraction uses internally. You may want to benefit from that stuff (typically CachedExpressionEvaluator and MethodBasedEvaluationContext).
The new #EventListener is using that stuff so you have more code you can look at as examples for the thing you're trying to do: EventExpressionEvaluator.
In summary, your custom interceptor needs to do something based on the #id value. This code snippet is an example of such processing and it does not depend on the cache abstraction at all.
Spring uses internally an ExpressionEvaluator to evaluate the Spring Expression Language in the key parameter (see CacheAspectSupport)
If you want to emulate the same behaviour, have a look at how CacheAspectSupport is doing it. Here is an snippet of the code:
private final ExpressionEvaluator evaluator = new ExpressionEvaluator();
/**
* Compute the key for the given caching operation.
* #return the generated key, or {#code null} if none can be generated
*/
protected Object generateKey(Object result) {
if (StringUtils.hasText(this.metadata.operation.getKey())) {
EvaluationContext evaluationContext = createEvaluationContext(result);
return evaluator.key(this.metadata.operation.getKey(), this.methodCacheKey, evaluationContext);
}
return this.metadata.keyGenerator.generate(this.target, this.metadata.method, this.args);
}
private EvaluationContext createEvaluationContext(Object result) {
return evaluator.createEvaluationContext(
this.caches, this.metadata.method, this.args, this.target, this.metadata.targetClass, result);
}
I don't know which IDE you are using, but it must deal with the #Cacheable annotation in a different way than with the others in order to highlight the params.
Your annotation can be used with methods with more than 1 parameter, but that doesn't mean you can't use the arguments array. Here's a sollution:
First we have to find the index of the "id" parameter. This you can do like so:
private Integer getParameterIdx(ProceedingJoinPoint joinPoint, String paramName) {
MethodSignature methodSignature = (MethodSignature) joinPoint.getSignature();
String[] parameterNames = methodSignature.getParameterNames();
for (int i = 0; i < parameterNames.length; i++) {
String parameterName = parameterNames[i];
if (paramName.equals(parameterName)) {
return i;
}
}
return -1;
}
where "paramName" = your "id" param
Next you can get the actual id value from the arguments like so:
Integer parameterIdx = getParameterIdx(joinPoint, "id");
Long id = joinPoint.getArgs()[parameterIdx];
Of course this assumes that you always name that parameter "id". One fix there could be to allow to specify the parameter name on the annotation, something like
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface CheckEntity {
String message() default "Check entity msg";
String key() default "";
String paramName() default "id";
}

EF Using generic and specific repositories

I have the following generic repository:
public class GenericRepository<TEntity> where TEntity : class {
private EFDbContext context;
private DbSet<TEntity> dbSet;
public GenericRepository(EFDbContext context) {}
public IEnumerable<TEntity> GetAll() {}
public IEnumerable<TEntity> Get(Expression<Func<TEntity, bool>> filter = null, Func<IQueryable<TEntity>, IOrderedQueryable<TEntity>> orderBy = null, string includeProperties = "") {}
public TEntity GetByID(int id) {}
public void Insert(TEntity entity) {}
public void Delete(int id) {}
public void Delete(TEntity entity) {}
public void Update(TEntity entity) {}
}
Now I understand how I can use this for my entities, but I don't understand what to do if an entity needs more than just these methods? Say I have an entity called 'Tournament' and I want to get all the groups in that tournament, where would the 'GetTournamentGroups' method go?
Should I, instead of using a generic repository, write a basic repository interface which all the entities' own interfaces inherit from and then just add the additional methods needed for each entity?
You could also simply use
var TournamentGroups = new List<Group>();
using( var GenericRepo = new GenericRepository<Group>())
{
TournamentGroups = GenericRepo.Get(group => group.TournamentId == tournamentId);
}
somewhere with your normal repository. It seems a little overkill to make a class which does this.
Method GetTournamentGroups() seems to belong to the layer above Repository layer:
class TournamentModel
{
private int tournamentId; // initialized in constructor
private GenericRepository<Group> repository; // initialized in constructor
IEnumerable<Group> GetTournamentGroups()
{
return repository.Get( group => group.TournamentId == tournamentId);
}
}
I'd recommend using IQueryable insteam of IEnumerable in your GenericRepository, though. It would be more flexible if you need to apply filters or sort expressions before you make an actual database query.

Resources