Filter Creation - elasticsearch

So after much experimentation I have found that the syntax I needed to filter products based on their specs is as follows
{
"post_filter": {
"bool": {
"filter": [{
"nested": {
"path": "productSpecification",
"query": {
"bool": {
"filter": [{
"term": {
"productSpecification.name": "Brand"
}
},
{
"terms": {
"productSpecification.value": [
"Brand1"
]
}
}
]
}
}
}
},
{
"nested": {
"path": "productSpecification",
"query": {
"bool": {
"filter": [{
"term": {
"productSpecification.name": "Guarantee"
}
},
{
"terms": {
"productSpecification.value": [
"3 years"
]
}
}
]
}
}
}
}
]
}
}
}
I'm now experimenting with creating a QueryContainer function to build this based on the selected name/values, If anyone can give me a point in the right direction on this that would be much appreciated.
Thanks

Ok I am now building a query from nest that works but is a bit messier than the DSL query above (not sure if someone can spot what Im doing below that makes it have some unnecessary bool filters
Here is my nest syntax
.PostFilter(pf => FilterTest(elasticParams,pf))
private static QueryContainer FilterTest(ElasticParams elasticParams, QueryContainerDescriptor<Product>q) => q
.Bool(b => b
.Filter(fi => FilterSelected(elasticParams, fi)));
private static QueryContainer FilterSelected2(ElasticParams elasticParams, QueryContainerDescriptor<Product> q) =>
elasticParams.Filters.Aggregate(new QueryContainer(), (c, s) => c && +q.Nested(n => n
.Path(p => p.ProductSpecification)
.Query(qq => qq
.Bool(bo => bo
.Filter(fi => fi
.Term(tt => tt
.Field(a => a.ProductSpecification.Suffix("name"))
.Value(s.name)) && fi
.Terms(ttt => ttt
.Field(fff => fff.ProductSpecification.Suffix("value"))
.Terms(s.Values))
)))));
As mentioned before this is filters as I would want however it DSL that it outputs is rather fugly as can be seen below
{
"post_filter": {
"bool": {
"filter": [{
"bool": {
"filter": [{
"nested": {
"query": {
"bool": {
"filter": [{
"bool": {
"must": [{
"term": {
"productSpecification.name": {
"value": "Brand"
}
}
}, {
"terms": {
"productSpecification.value": ["Brand1", "Brand2"]
}
}]
}
}]
}
},
"path": "productSpecification"
}
}, {
"nested": {
"query": {
"bool": {
"filter": [{
"bool": {
"must": [{
"term": {
"productSpecification.name": {
"value": "Guarantee"
}
}
}, {
"terms": {
"productSpecification.value": ["3 years"]
}
}]
}
}]
}
},
"path": "productSpecification"
}
}]
}
}]
}
}
}

Related

Complex nested Elastic NEST Query

How would I convert this to the equivalent Nest query?
{
"query": {
"nested": {
"path": "boundedContexts",
"query": {
"nested": {
"path": "boundedContexts.aggregateRoots",
"query": {
"nested": {
"path": "boundedContexts.aggregateRoots.modelMetaData",
"query": {
"bool": {
"must": [
{ "match": { "boundedContexts.aggregateRoots.modelMetaData.modelReferenceId": "4e7c5c0e-93a7-4bf6-9705-cf1327760e21" } },
{ "match": { "boundedContexts.aggregateRoots.modelMetaData.modelType.name": "AggregateRoot" } }
]
}
}
}
}
}
}
}
},
"size": 1,
"sort": [
{
"generatedDate": {
"order": "desc"
}
}
]
}
I am trying to generate an equivalent Nest query similar to this ..
ISearchResponse<ViewModels.DomainModel> response = null;
response = await _elasticClient.SearchAsync<ViewModels.DomainModel>(s => s.Index(_modelMetadataProvider.CurrentIndexName)
.Query(q => q.Nested(n1 => n1.Path("boundedContexts")
.Query(q2 => q2.Nested(n2 => n2.Path("boundedContexts.aggregateRoots")
.Query(q3 => q3.Nested(n3 => n3.Path("boundedContexts.aggregateRoots.modelMetaData").
Query(q4 =>q4.Bool(b => b.Must(bs =>
bs.Match(p => p.Field("boundedContexts.aggregateRoots.modelMetaData.modelReferenceId")))))))))))
.Size(1).Sort(s=>s.Descending("generatedDate")));
I am stuck on how to compare to a variable against at the p.Field("boundedContexts.aggregateRoots.modelMetaData.modelReferenceId") level of query.

ElasticSearch should with nested and bool must_not exists

With the following mapping:
"categories": {
"type": "nested",
"properties": {
"category": {
"type": "integer"
},
"score": {
"type": "float"
}
}
},
I want to use the categories field to return documents that either:
have a score above a threshold in a given category, or
do not have the categories field
This is my query:
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "categories",
"query": {
"bool": {
"must": [
{
"terms": {
"categories.category": [
<id>
]
}
},
{
"range": {
"categories.score": {
"gte": 0.5
}
}
}
]
}
}
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "categories"
}
}
]
}
}
],
"minimum_should_match": 1
}
}
}
It correctly returns documents both with and without the categories field, and orders the results so the ones I want are first, but it doesn't filter the results having score below the 0.5 threshold.
Great question.
That is because categories is not exactly a field from the elasticsearch point of view[a field on which inverted index is created and used for querying/searching] but categories.category and categories.score is.
As a result categories being not found in any document, which is actually true for all the documents, you observe the result what you see.
Modify the query to the below and you'd see your use-case working correctly.
POST <your_index_name>/_search
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "categories",
"query": {
"bool": {
"must": [
{
"terms": {
"categories.category": [
"100"
]
}
},
{
"range": {
"categories.score": {
"gte": 0.5
}
}
}
]
}
}
}
},
{
"bool": {
"must_not": [ <----- Note this
{
"nested": {
"path": "categories",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "categories.category"
}
},
{
"exists": {
"field": "categories.score"
}
}
]
}
}
}
}
]
}
}
],
"minimum_should_match": 1
}
}
}

how to write existsQuery in nest 1.7

I'm using nest 1.7 and i need to write this query:
GET _search
{
"from": 0,
"size": 3,
"query": {
"bool": {
"must": [
{
"constant_score": {
"filter": {
"bool": {
"must": [
{
"bool": {
"must": [
{
"exists": {
"field": "Collaborateurs"
}
},
{
"exists": {
"field": "Collaborateurs.Nom"
}
},
{
"exists": {
"field": "Collaborateurs.Fonction"
}
},
{
"exists": {
"field": "Collaborateurs.TagVisuel"
}
},
{
"exists": {
"field": "siAnnuaire"
}
},
{
"term": {
"siAnnuaire": {
"value": true
}
}
},
{
"exists": {
"field": "TagPhoto"
}
},
{
"range": {
"NbAnnonce": {
"gt": 0
}
}
},
{
"geo_distance": {
"distance": "10.0km",
"AgenceLocation": {
"lat": 48.8523513700019,
"lon": 2.35127712591128
}
}
}
]
}
}
]
}
}
}
},
{
"function_score": {
"functions": [
{
"random_score": {
"seed": 69937385
}
}
]
}
}
]
}
}
}
Using the fluent API, the method calls follow pretty much the structure of the query DSL json
var response = client.Search<Document>(x => x
.From(0)
.Size(3)
.Query(q => q
.Bool(b => b
.Must(m => m
.ConstantScore(cs => cs
.Filter(csf => csf
.Bool(cb => cb
.Must(
cm => cm.Exists(p => p.Collaborateurs),
cm => cm.Exists(p => p.Collaborateurs.Nom),
cm => cm.Exists(p => p.Collaborateurs.Fonction)
// etc, etc for the other queries
)
)
)
), m => m
.FunctionScore(fs => fs
.Functions(fu => fu
.RandomScore(69937385)
)
)
)
)
)
);
which yields
{
"from": 0,
"size": 3,
"query": {
"bool": {
"must": [
{
"constant_score": {
"filter": {
"bool": {
"must": [
{
"exists": {
"field": "collaborateurs"
}
},
{
"exists": {
"field": "collaborateurs.nom"
}
},
{
"exists": {
"field": "collaborateurs.fonction"
}
}
]
}
}
}
},
{
"function_score": {
"functions": [
{
"random_score": {
"seed": 69937385
}
}
]
}
}
]
}
}
}
field names have been camel-cased by default, but you can change this behaviour using .SetDefaultPropertyNameInferrer(Func<string, string>) on ConnectionSettings.

How to combine multiple bool queries in elasticsearch

I want to create the equivalent of the following query -
(city = 'New York' AND state = 'NY') AND ((businessName='Java' and businessName='Shop') OR (category='Java' and category = 'Shop'))
I tried different combinations of bool queries using must and should but nothing seems to be working. Can this be done?
How about something like this:
{
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"city": "New york"
}
},
{
"term": {
"state": "NY"
}
},
{
"bool": {
"should": [
{
"bool": {
"must": [
{
"term": {
"businessName": "Java"
}
},
{
"term": {
"businessName": "Shop"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"category": "Java"
}
},
{
"term": {
"category": "Shop"
}
}
]
}
}
]
}
}
]
}
}
}

ElasticSearch ignoring sort when filtered

ElasticSearch Version: 0.90.1, JVM: 1.6.0_51(20.51-b01-457)
I'm trying to do two things with my ElasticSearch query: 1) filter the results based on a boolean (searchable) and "open_date < tomorrow" and 2) two sort by the field "open_date" DESC
This produces the following query:
{
"query": {
"bool": {
"should": [
{
"prefix": {
"name": "foobar"
}
},
{
"query_string": {
"query": "foobar"
}
},
{
"match": {
"name": {
"query": "foobar"
}
}
}
],
"minimum_number_should_match": 1
},
"filtered": {
"filter": {
"and": [
{
"term": {
"searchable": true
}
},
{
"range": {
"open_date": {
"lt": "2013-07-16"
}
}
}
]
}
}
},
"sort": [
{
"open_date": "desc"
}
]
}
However, the results that come back are not being sorted by "open_date". If I remove the filter:
{
"query": {
"bool": {
"should": [
{
"prefix": {
"name": "foobar"
}
},
{
"query_string": {
"query": "foobar"
}
},
{
"match": {
"name": {
"query": "foobar"
}
}
}
],
"minimum_number_should_match": 1
}
},
"sort": [
{
"open_date": "desc"
}
]
}
... the results come back as expected.
Any ideas?
I'm not sure about the Tire code, but the JSON does not correctly construct a filtered query. My guess is that this overflows and causes the sort element to also not be correctly parsed.
A filtered query should be constructed like this (see http://www.elasticsearch.org/guide/reference/query-dsl/filtered-query/ ):
{
"query": {
"filtered": { // Note: this contains both query and filter
"query": {
"bool": {
"should": [
{
"prefix": {
"name": "foobar"
}
},
{
"query_string": {
"query": "foobar"
}
},
{
"match": {
"name": {
"query": "foobar"
}
}
}
],
"minimum_number_should_match": 1
}
},
"filter": {
"and": [
{
"term": {
"searchable": true
}
},
{
"range": {
"open_date": {
"lt": "2013-07-16"
}
}
}
]
}
}
},
"sort": [
{
"open_date": "desc"
}
]
}
Cheers,
Boaz

Resources