Spring Data Elasticsearch to fetch Documents between two dates generates wrong query - spring

I am trying to fetch documents which are greater or lesser than specified date.
I am using the below searchQuery for this purpose.
QueryBuilder queryBuilder = QueryBuilders.boolQuery()
.must(QueryBuilders.rangeQuery("date")
.gt("2015-06-25T00:00:00")
.lt("2015-06-25T00:00:00"));
The query generated from the above querybuilder is like this.
{
"bool" : {
"must" : [ {
"range" : {
"date" : {
"from" : "2015-06-25T00:00:00",
"to" : "2015-06-25T00:00:00",
"include_lower" : false,
"include_upper" : false
}
}
}
} ]
}
Even when i use functions gt and lt of rangequery the query is generated as from and to.
What is the solution so that a query can be generated like this.
{
"bool" : {
"must" : [ {
"range" : {
"date" : {
"gt" : "2015-06-25T00:00:00",
"lt" : "2015-06-25T00:00:00",
"include_lower" : false,
"include_upper" : false
}
}
}
} ]
}
This is the test class i have written.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = { ElasticSearchConfiguration.class }, loader = AnnotationConfigContextLoader.class)
public class ElasticSearchTest {
#Autowired
private ElasticsearchTemplate elasticsearchTemplate;
#Autowired
private Client client;
#Test
public void testAggregation(){
QueryBuilder querybuilder = QueryBuilders.boolQuery().must(QueryBuilders.rangeQuery("receiptdate").gte("2015-06-25T00:00:00").lte ("2015-07-25T00:00:00")));
final SearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(qb)
.build();
final List<Test> records = elasticsearchTemplate.queryForList(searchQuery, Test.class);
}
}
Any suggestions on how to achieve this in Spring Data Elasticsearch would be helpful.

Your query would not return any results, since you're looking for dates strictly greater and strictly lower than the same date. You need to use gte and lte instead:
QueryBuilder queryBuilder = QueryBuilders.boolQuery()
.must(QueryBuilders.rangeQuery("date")
.gte("2015-06-25T00:00:00")
.lte("2015-06-25T00:00:00"));
The official parameters of the range query are gt, gte, lt and lte.
The from, to, include_lower and include_upper parameters are old deprecated parameters, which the RangeQueryBuilder is still using but can (and will) be removed at anytime.
Just know that:
from + include_lower: false is equivalent to gt
from + include_lower: true is equivalent to gte
to + include_upper: false is equivalent to lt
to + include_upper: true is equivalent to lte

Related

Elasticsearch Jest client add condition to json query

I am using Elasticsearch 6.3 with Jest client 6.3 (Java API)
Search search = new Search.Builder(jsonQueryString)
.addIndex("SOME_INDEX")
.build();
SearchResult result = jestClient.execute(search);
And this is my sample JSON query
{
"query": {
"bool" : {
"filter": {
"match" :{
"someField" : "some value"
}
}
}
}
}
The JSON query string is accepted as a POST request body and then passed to the Jest client. Before I can execute the json query on the Jest client, I need to add conditions to the query for e.g.
{
"query": {
"bool" : {
"filter": {
"match" :{
"someField" : "some value"
}
}
},
"must": {
"match" :{
"systemField" : "pre-defined value"
}
}
}
}
}
Is there an API that allows to parse the JSON query and add conditions to it before it can be executed on Jest client? The JSON query can be any query supported by Query DSL and not necessarily contain bool condition. I need to add a pre-defined condition to the query. I appreciate any help on this. Thanks very much.
There is no out of the box Elasticsearch or Jest API to achieve the above, the workaround I implemented is using Jackson ObjectMapper
// convert the search request body into object node
ObjectNode searchRequestNode = objectMapper.readValue(queryString, ObjectNode.class);
// extract the query
String query = searchRequestNode.get("query").toString();
// wrap the original query and add conditions
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
boolQueryBuilder.must(QueryBuilders.wrapperQuery(query));
boolQueryBuilder.filter(QueryBuilders.termsQuery("fieldA", listOfValues));
boolQueryBuilder.filter(QueryBuilders.termQuery("fieldB", value));
// convert querybuilder to json query string
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(queryBuilder);
String queryWithFilters = searchSourceBuilder.toString();
// convert json string to object node
ObjectNode queryNode = objectMapper.readValue(queryWithFilters, ObjectNode.class);
// replace original query with the new query containing added conditions
searchRequestNode.set("query", queryNode.get("query"));
String finalSearchRequestWithOwnFilters = searchRequestNode.toString();

Multiple aggregation and Unwind - Spring data Mongodb

I have a mongo query which is used to unwind four arrays of objects and filter the data based on the matching criteria. How do i perform the same operation in Spring data mongodb
I have worked with single unwind but could not find any with multiple unwind and match operation.
db.generator.aggregate([
{ $unwind :'$eCList'},
{ $unwind :'$pcList'},
{ $unwind :'$cityList'},
{ $unwind :'$eATypeList'},
{ $match : {'eCList.eCCode': { $eq : 'enccode1'} }},
{ $match : {'pcList.pcCode': { $eq : 'pccode1'} }},
{ $match : {'cityList.cityCode': { $eq : 'citycode1'} }},
{ $match : {'eATypeList.eATypeCode': { $eq : 'eATypeCode1'} }},
{ $project : {generatorData : '$generatorData',pcList:'$pcList',
generatorCode: '$generatorCode', eCId : '$eCList.eCId',
eCCode : '$eCList.eCCode', eCKey : '$eCList.eCKey' } }
])
You can try below aggregation in 1.10.x spring mongo version.
Aggregation aggregation = Aggregation.newAggregation(
Aggregation.unwind("eCList"),
Aggregation.unwind("pcList"),
Aggregation.unwind("cityList"),
Aggregation.unwind("eATypeList"),
Aggregation.match(Criteria.where("eCList.eCCode").is("enccode1")),
Aggregation.match(Criteria.where("pcList.pcCode").is("pccode1")),
Aggregation.match(Criteria.where("cityList.cityCode").is("citycode1")),
Aggregation.match(Criteria.where("eATypeList.eATypeCode").is("eATypeCode1")),
Aggregation.project("generatorData", "pcList", "generatorCode").
andInclude(Aggregation.bind("eCId", "eCList.eCId")).
andInclude(Aggregation.bind("eCCode", "eCList.eCCode")).
andInclude(Aggregation.bind("eCKey", "eCList.eCKey"))
);
List<BasicDBObject> results = mongoTemplate.aggregate( aggregation, "generator", BasicDBObject.class).getMappedResults();
Generated Shell Query:
{
"aggregate":"__collection__",
"pipeline":[
{"$unwind":"$eCList"},
{"$unwind":"$pcList"},
{"$unwind":"$cityList"},
{"$unwind":"$eATypeList"},
{"$match":{"eCList.eCCode":"enccode1"}},
{"$match":{"pcList.pcCode":"pccode1"}},
{"$match":{"cityList.cityCode":"citycode1"}},
{"$match":{"eATypeList.eATypeCode":"eATypeCode1"}},
{"$project":{
"generatorData":1,
"pcList":1,
"generatorCode":1,
"eCId":"$eCList.eCId",
"eCCode":"$eCList.eCCode",
"eCKey":"$eCList.eCKey"}}
]
}
With static imports
import org.springframework.data.mongodb.core.query.Criteria;
import static org.springframework.data.mongodb.core.aggregation.Aggregation.*;
Aggregation aggregation = newAggregation(
unwind("eCList"),
unwind("pcList"),
unwind("cityList"),
unwind("eATypeList"),
match(where("eCList.eCCode").is("enccode1")),
match(where("pcList.pcCode").is("pccode1")),
match(where("cityList.cityCode").is("citycode1")),
match(where("eATypeList.eATypeCode").is("eATypeCode1")),
project("generatorData", "pcList", "generatorCode").
andInclude(bind("eCId", "eCList.eCId")).
andInclude(bind("eCCode", "eCList.eCCode")).
andInclude(bind("eCKey", "eCList.eCKey"))
);

Spring Data Elasticsearch wildcard query with sort

I wrote a search query for elasticsearch:
{
"query": {
"query_string": {
"fields": ["studentName", "countryName"],
"query" : "*o*"
}
},
"sort" : [{
"studentName" : { "order": "desc" }
}]
}
This is executed for localhost:9200/myindex/mytype/_search (POST).
I get correct results based on the query which are sorted on the basis of sort part. But when I convert this into Spring Data Query like:
#Query("{ \"query\": { \"query_string\" : { \"fields\" : [\"studentName\", \"countryName\"], \"query\":\"*?0*\"}}," +
" \"sort\" : [{ \"?1\" : { \"order\": \"?2\" }}]}")
Page<Student> freeTextSearchPortSort(String freeText, String sortBy, String sortOrder, Pageable pageable);
I always get the same result, sorted on the order of insertion. What do I need to do differently?
You need to create PageRequest which has a constructor that takes page,size ,direction,properties. PageRequest implements Pageable interface.
Your query would look like:
#Query("{ \"query\": { \"query_string\" : { \"fields\" : [\"studentName\", \"countryName\"], \"query\":\"*?0*\"}}")
Page<Student> freeTextSearchPortSort(String freeText, Pageable pageable);
The location from where this is being called will look like.
PageRequest pageRequest = new PageRequest(0, no_of_rec_to_be_fetched, Sort.Direction.fromString("desc"), "studentName")
freeTextSearchPortSort ("someText",pageRequest)
Hope this helps!!

Spring Data Rest and excerpt Projections

I am using the inlineAddress sample of the Spring Data Rest documentation.
/persons return the address inline as expected.
Now I add a projection to the AddressRepository
#RepositoryRestResource(excerptProjection = AddressProjection.class)
Which is as below
#Projection(name = "AddressesProjection", types = Address.class)
public interface AddressProjection {
public String getStreet();
}
This is causing the /persons call to have an address projection as _embedded
{
"_embedded" : {
"persons" : [ {
"firstName" : "dfdf",
"lastName" : "2",
"addresses" : [ {
"street" : "tx",
"state" : "tx",
"country" : "dfd"
} ],
"_embedded" : {
"addresses" : [ {
"street" : "tx",
"_links" : {
"self" : {
"href" : "/api/addresses/1{?projection}",
"templated" : true
}
}
} ]
},
"_links" : {
"self" : {
"href" : " api/persons/1{?projection}",
"templated" : true
},
"addresses" : {
"href" : " /api/persons/1/addresses"
}
}
} ]
}
}
I dont know if this is expected. This behaviour is causing repeated information when I have a oneToMany relation like order/Comments and have projection on both order and comments and when I access order/1/comments I see the order also embedded for each comments.
I have a similar issue with spring-data-rest 2.5.6. So I'd like to add this.
If :
an A entity has a #OneToMany relationship to a B entity
the B entity's repository #RepositoryRestResource contains an excerptProjection
Then spring-data-rest will embed the B entity's list in any A entity (in _embedded).
If there is no excerptProjection, the list won't be embedded.
I'd like to be able to choose what I want to be embedded, but at moment, I found no solution to do so.
For anybody that is looking for the answer to this issue, I actually did find a solution.
Based on the example in Spring-RestBucks you will need to have a custom RepresentationModelProcessor on A entity.
Also consider the official Spring HATEOAS Documentation on RepresentationModelProcessor.
Applying to the above example, you would do:
public class PersonRepresentationProcessor implements RepresentationModelProcessor<EntityModel<Person>> {
private final EntityLinks entityLinks;
#Override
public EntityModel<Person> process(EntityModel<Person> personModel) {
// create new EntityModel without the embedded collection
TypedEntityLinks<Person> typedPersonLink = entityLinks.forType(Person::getId);
EntityModel<Person> newPerson = new EntityModel<>(personModel.getContent(),
personModel.getLinks());
// add more links or other modifications
return newPerson;
}
}

Spring Data MongoDB Date between two Dates

i'm using Spring Data for MongoDB and got the following classes
class A {
List<B> b;
}
class B {
Date startDate;
Date endDate;
}
when i save an object of A it gets persisted like
{
"_id" : "DQDVDE000VFP8E39",
"b" : [
{
"startDate" : ISODate("2009-10-05T22:00:00Z"),
"endDate" : ISODate("2009-10-29T23:00:00Z")
},
{
"startDate" : ISODate("2009-11-01T23:00:00Z"),
"endDate" : ISODate("2009-12-30T23:00:00Z")
}
]
}
Now i want to query the db for documents matching entries in b where a given date is between startDate and endDate.
Query query = new Query(Criteria.where("b").elemMatch(
Criteria.where("startDate").gte(date)
.and("endDate").lte(date)
);
Which results in the following mongo query:
{
"b": {
"$elemMatch": {
"startDate" : { "$gte" : { "$date" : "2009-11-03T23:00:00.000Z"}},
"endDate" : { "$lte" : { "$date" : "2009-11-03T23:00:00.000Z"}}
}
}
}
but returns no resulting documents. Does anybody know what i'm doing wrong? I don't get it...
Thank you very much in advance!!
If you want to find docs where date is between the startDate and endDate of a b array element then you need to reverse your gte and lte calls:
Query query = new Query(Criteria.where("b").elemMatch(
Criteria.where("startDate").lte(date)
.and("endDate").gte(date)
);
{"created_at":{$gt:ISODate("2013-04-30T00:00:00Z"),$lt:ISODate("2013-04-30T23:59:59Z")}}

Resources