There is Location field exists with geo_point type
I want to implement with some conditions below.
If there is a location, the closer it is, boost it
If there is no location, boost by 5
Ultimately, i want to implement the following features: If the location exists, will show it in the order of distance, but we would like to boost documents without location so that they are not pushed out last.
below is my query. I reached to get nearest document by location. But i don't know how to boost which is no location.
{
"query": {
"bool": {
"must": {
"match_all": {}
},
"should": {
"distance_feature": {
"field": "location",
"pivot": "1000m",
"boost": 8,
"origin": {
"lat": 33.489009,
"lon": 133.022831
}
}
},
"filter": [
{
"terms" : {
"state": ["AVAILABLE"]
}
}
]
}
}
}
You could try to do it like this:
{
"query": {
"bool": {
"must": {
"match_all": {}
},
"minimum_should_match": 1,
"should": [
{
"distance_feature": {
"field": "locations.parcelLocation",
"pivot": "1000m",
"boost": 8,
"origin": {
"lat": 33.489009,
"lon": 133.022831
}
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "location",
"boost": 5
}
}
}
}
],
"filter": [
{
"terms": {
"state": [
"AVAILABLE"
]
}
}
]
}
}
}
I have been following the guidelines from this post. I can get the desired output but in the same DSL how can I limit the size of results for each index ?
Full text Search with Multiple index in Elastic Search using NEST C#
POST http://localhost:9200/componenttypeindex%2Cprojecttypeindex/Componenttype%2CProjecttype/_search?pretty=true&typed_keys=true
{
"query": {
"bool": {
"should": [
{
"bool": {
"filter": [
{
"term": {
"_index": {
"value": "componenttypeindex"
}
}
}
],
"must": [
{
"multi_match": {
"fields": [
"Componentname",
"Summary^1.1"
],
"operator": "or",
"query": "test"
}
}
]
}
},
{
"bool": {
"filter": [
{
"term": {
"_index": {
"value": "projecttypeindex"
}
}
}
],
"must": [
{
"multi_match": {
"fields": [
"Projectname",
"Summary^0.3"
],
"operator": "or",
"query": "test"
}
}
]
}
}
]
}
}
}
With your given query, you could use aggregations to group and limit number of hits per index (in this case, limiting to 5):
{
"size": 0,
"query": {
... Same query as above ...
},
"aggs": {
"index_agg": {
"terms": {
"field": "_index",
"size": 20
},
"aggs": {
"hits_per_index": {
"top_hits": {
"size": 5
}
}
}
}
}
}
Here is my current elastic query:
{
"from": 0,
"size": 10,
"query": {
"function_score": {
"query": {
"bool": {
"must": [{
"multi_match": {
"query": "ocean",
"fields": [],
"fuzziness": "AUTO"
}}],
"must_not": [{
"exists": {
"field": "parentId"
}
}]
}
},
"functions" : [
{
"gauss": {
"createTime": {
"origin": "2020-07-09T23:50:00",
"scale": "365d",
"decay": 0.3
}
}
}
]
}
}
}
How do I properly add filters to this? I think maybe the fact that I'm using function_score makes this different? I would like to add a hard filter, for example, only show me results with uploadUser: 'Mr. Bean' ... but still keep the scoring in place for the results that pass this filter.
I tried using filter in various places, also using must but I either get no results or all the results.
I'm using Elastic Search 7. Thanks for your help
You can try this below search query:
Refer this ES official documentation to know more about Function score query
{
"from": 0,
"size": 10,
"query": {
"function_score": {
"query": {
"bool": {
"filter": {
"term": {
"uploadUser": "Mr. Bean"
}
},
"must": [
{
"multi_match": {
"query": "ocean",
"fields": [
],
"fuzziness": "AUTO"
}
}
],
"must_not": [
{
"exists": {
"field": "parentId"
}
}
]
}
},
"functions": [
{
"gauss": {
"createTime": {
"origin": "2020-07-09T23:50:00",
"scale": "365d",
"decay": 0.3
}
}
}
]
}
}
}
I have an elastic query aggregation in which I need to filter aggregation on the basis on index name. Query section actually working on multiple indexes, but I want to filter aggregation for particular index. Please help me how we can pass index filter in aggregation -
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"query_string": {
"fields": [
"productDesc",
"productDescription"
],
"default_operator": "AND",
"query": "machine"
}
}
]
}
}
],
"must": [ ],
"must_not": [ ]
}
},
"size": 0,
"aggs": {
"RelatedKeywords": { //here I want to add filter of index
"sampler": {
"shard_size": 20
},
"aggregations": {
"keywords": {
"significant_text": {
"field": "productDesc",
"size": 100,
"filter_duplicate_text": true
}
}
}
}
}
}
You can do it like this:
{
"aggs": {
"index": {
"filter": {
"term": {
"_index": "index-name"
}
},
"aggs": {
"RelatedKeywords": {
"sampler": {
"shard_size": 20
},
"aggregations": {
"keywords": {
"significant_text": {
"field": "productDesc",
"size": 100,
"filter_duplicate_text": true
}
}
}
}
}
}
}
}
In a new project of ours, we were inspired by this article http://project-a.github.io/on-site-search-design-patterns-for-e-commerce/#generic-faceted-search for doing our “facet” structure. And while I have got it working to the extent the article describes, I have run into issues in getting it to work when selecting facets. I hope someone can give a hint as to something to try, so I don’t have to redo all our aggregations into separate aggregation calculations again.
The problem is basically that we are using a single aggregation to calculate all the “facets” at once, but when I add a filter (fx. checking a brand name), then it “removes” all the other brands when returning the aggregates. What I basically want is that it should use that brand as filter when calculating the other facets, but not when calculating the brand aggregations. This is necessary so the user can, for example, choose multiple brands.
Looking at https://www.contorion.de/search/Metabo_Fein/ou1-ou2?q=Winkelschleifer&c=bovy (which is the site described in the above article), I have selected the “Metabo” and “Fein” manufacturer (Hersteller), and unfolding the Hersteller menu it shows all manufacturers and not just the ones selected. So I know it’s possible somehow and I hope some one out there has a hint as to how to write the aggregations / filters, so I get the "correct e-commerce facet behavior".
On the products in ES I have the following structure: (the same as in the original article, though “C#’ified” in naming)
"attributeStrings": [
{
"facetName": "Property",
"facetValue": "Organic"
},
{
"facetName": "Property",
"facetValue": "Without parfume"
},
{
"facetName": "Brand",
"facetValue": "Adidas"
}
]
So the above product has 2 attributes/facet groups – Property with 2 values (Organic, Without parfume) and Brand with 1 value (Adidas).
Without any filters I calculate the aggregations from the following query:
"aggs": {
"agg_attr_strings_filter": {
"filter": {},
"aggs": {
"agg_attr_strings": {
"nested": {
"path": "attributeStrings"
},
"aggs": {
"attr_name": {
"terms": {
"field": "attributeStrings.facetName"
},
"aggs": {
"attr_value": {
"terms": {
"field": "attributeStrings.facetValue",
"size": 1000,
"order": [
{
"_term": "asc"
}
]
} } } } } } } }
Now if I select Property "Organic" and Brand "Adidas" I build the same aggregation, but with a filter to apply those two constraints (which is were it kind of goes wrong...):
"aggs": {
"agg_attr_strings_filter": {
"filter": {
"bool": {
"filter": [
{
"nested": {
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Property"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Organic"
]
}
}
]
}
},
"path": "attributeStrings"
}
},
{
"nested": {
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Brand"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Adidas"
]
}
}
]
}
},
"path": "attributeStrings"
}
}
]
}
},
"aggs": {
"agg_attr_strings": {
"nested": {
"path": "attributeStrings"
},
"aggs": {
"attr_name": {
"terms": {
"field": "attributeStrings.facetName",
},
"aggs": {
"attr_value": {
"terms": {
"field": "attributeStrings.facetValue",
"size": 1000,
"order": [
{
"_term": "asc"
}
]
} } } } } } } }
The only way I can see forward with this model, is to calculate the aggregation for each selected facet and somehow merge the result. But it seems very complex and kind of defeats the point of having the model as described in the article, so I hope there's a more clean solution and someone can give a hint at something to try.
The only way I can see forward with this model, is to calculate the aggregation for each selected facet and somehow merge the result.
This is exactly right. If one facet (e.g. brand) is selected than you can not use global brand filter if you also want to fetch other brands for multi-selection. What you can do is apply all other filters on selected facets, and all filters on non-selected facets. As a results you will have n+1 separate aggregations for n selected filters - first one is for all facets and the rest are for selected facets.
In your case query might look like:
{
"aggs": {
"agg_attr_strings_filter": {
"filter": {
"bool": {
"filter": [
{
"nested": {
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Property"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Organic"
]
}
}
]
}
},
"path": "attributeStrings"
}
},
{
"nested": {
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Brand"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Adidas"
]
}
}
]
}
},
"path": "attributeStrings"
}
}
]
}
},
"aggs": {
"agg_attr_strings": {
"nested": {
"path": "attributeStrings"
},
"aggs": {
"attr_name": {
"terms": {
"field": "attributeStrings.facetName"
},
"aggs": {
"attr_value": {
"terms": {
"field": "attributeStrings.facetValue",
"size": 1000,
"order": [
{
"_term": "asc"
}
]
}
}
}
}
}
}
}
},
"special_agg_property": {
"filter": {
"nested": {
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Brand"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Adidas"
]
}
}
]
}
},
"path": "attributeStrings"
}
},
"aggs": {
"special_agg_property": {
"nested": {
"path": "attributeStrings"
},
"aggs": {
"agg_filtered_special": {
"filter": {
"query": {
"match": {
"attributeStrings.facetName": "Property"
}
}
},
"aggs": {
"facet_value": {
"terms": {
"size": 1000,
"field": "attributeStrings.facetValue"
}
}
}
}
}
}
}
},
"special_agg_brand": {
"filter": {
"nested": {
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Property"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Organic"
]
}
}
]
}
},
"path": "attributeStrings"
}
},
"aggs": {
"special_agg_brand": {
"nested": {
"path": "attributeStrings"
},
"aggs": {
"agg_filtered_special": {
"filter": {
"query": {
"match": {
"attributeStrings.facetName": "Brand"
}
}
},
"aggs": {
"facet_value": {
"terms": {
"size": 1000,
"field": "attributeStrings.facetValue"
}
}
}
}
}
}
}
}
}
}
This query looks super big and scary but generating such query can be done with few dozen lines of code.
When parsing query results, you need to first parse general aggregation (one that uses all filters) and after special facet aggregations. From the upper example, first parse results from agg_attr_strings_filter but those results will also contain aggregation values for Brand and Property that should be overwritten by aggregation values from special_agg_property and special_agg_brand
Also, this query is efficient since Elasticsearch does good job in caching separate filter clauses so applying same filters in different parts of query should be cheap.
But it seems very complex and kind of defeats the point of having the model as described in the article, so I hope there's a more clean solution and someone can give a hint at something to try.
There is really no way around the fact that you need to apply different filters to different facets and at the same time have different query filters. If you need to support "correct e-commerce facet behavior" you will have complex query :)
Disclaimer: I'm coauthor of the mentioned article.
The issue comes from the fact that you are adding a filter on Property and Organic inside your aggregation, hence the more facets you pick, the more you will restrain the terms you will get. In that article, the filter they use is in fact a post_filter, both names were allowed until recently, but filter got removed because that was causing ambiguities.
What you need to do is to move that filter outside the aggregations into the post_filter section, so that the results get correctly filtered out by whatever facets have been picked, but all your facets still get computed correctly on the whole document set.
{
"post_filter": {
"bool": {
"filter": [
{
"nested": {
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Property"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Organic"
]
}
}
]
}
},
"path": "attributeStrings"
}
},
{
"nested": {
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Brand"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Adidas"
]
}
}
]
}
},
"path": "attributeStrings"
}
}
]
}
},
"aggs": {
"agg_attr_strings_full": {
"nested": {
"path": "attributeStrings"
},
"aggs": {
"attr_name": {
"terms": {
"field": "attributeStrings.facetName"
},
"aggs": {
"attr_value": {
"terms": {
"field": "attributeStrings.facetValue",
"size": 1000,
"order": [
{
"_term": "asc"
}
]
}
}
}
}
}
},
"agg_attr_strings_filtered": {
"filter": {
"bool": {
"filter": [
{
"nested": {
"path": "attributeStrings",
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Property"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Organic"
]
}
}
]
}
}
}
},
{
"nested": {
"path": "attributeStrings",
"query": {
"bool": {
"filter": [
{
"term": {
"attributeStrings.facetName": {
"value": "Brand"
}
}
},
{
"terms": {
"attributeStrings.facetValue": [
"Adidas"
]
}
}
]
}
}
}
}
]
}
},
"aggs": {
"nested": {
"path": "attributeStrings"
},
"aggs": {
"attr_name": {
"terms": {
"field": "attributeStrings.facetName"
},
"aggs": {
"attr_value": {
"terms": {
"field": "attributeStrings.facetValue",
"size": 1000,
"order": [
{
"_term": "asc"
}
]
}
}
}
}
}
}
}
}
}