I have a problem with my elasticsearch DSL, in that when using facet navigation, when I apply my facet filter, the next set of results don't include any further facets, even though I've asked for them.
When I do the initial search, I get the results I want back:
{
"sort": {
"_score": {},
"salesQuantity": {
"order": "asc"
}
},
"query": {
"filtered": {
"query": {
"match": {
"categoryTree": "D01"
}
},
"filter": {
"term": {
"publicwebEnabled": true,
"parentID": 0
}
}
}
},
"facets": {
"delivery_locations": {
"terms": {
"field": "delivery_locations",
"all_terms": true
}
},
"categories": {
"terms": {
"field": "categoryTree",
"all_terms": true
}
},
"collectable": {
"terms": {
"field": "collectable",
"all_terms": true
}
}
},
"from": 0,
"size": 12}
When I then apply a filter like so, the results I get back do not include the facets:
{
"sort": {
"_score": {},
"salesQuantity": {
"order": "asc"
}
},
"query": {
"filtered": {
"query": {
"match": {
"categoryTree": "D01"
}
},
"filter": {
"term": {
"publicwebEnabled": true,
"parentID": 0
},
"or": [
{
"range": {
"Retail_Price": {
"to": "49.99",
"from": "0"
}
}
}
]
}
}
},
"facets": {
"delivery_locations": {
"terms": {
"field": "delivery_locations",
"all_terms": true
}
},
"categories": {
"terms": {
"field": "categoryTree",
"all_terms": true
}
},
"collectable": {
"terms": {
"field": "collectable",
"all_terms": true
}
}
},
"from": 0,
"size": 12}
NOTE, I'm adding the OR filter above - because users may choose multiple price ranges to filter on.
Am I doing something wrong?
I want the new facets returned as altering the prices would obviously alter the facet counts of the other facets...
Add the original term-filter inside the or-filter, or add another boolean filter to wrap your whole filter inside a boolean expression. I dont think you can add the two filters just by comma-separating them like that.
Related
Hi I have the below elastic search query using this in dev tools. I keep getting errors for my bool query but it seems correct looking at #timestamp field and trying to only retrieve one day worth of data.
"input": {
"search": {
"request": {
"indices": [
"<iovation-*>"
],
"body": {
"size": 0,
"query": {
"bool": {
"must": {
"range": {
"#timestamp": {
"gte": "now-1d"
}
}
}
},
"aggs": {
"percentiles": {
"percentiles": {
"field": "logstash.load.duration",
"percents": 95,
"keyed": false
}
},
"dates": {
"date_histogram": {
"field": "#timestamp",
"calendar_interval": "5m",
"min_doc_count": 1
}
}
}
}
}
}
}
},
Any help is appreciated thanks!
There are few errors in your query
Whenever aggregation is used along with the query part, then the structure is
{
"query": {},
"aggs": {}
}
You are missing one } at the end of the query part
Calendar Intervals do not accept multiple quantities like 2d, 2m, etc.
If you have a fixed interval, then you can refer to the fixed_interval param
Modify your query as
{
"size": 0,
"query": {
"bool": {
"must": {
"range": {
"#timestamp": {
"gte": "now-1d"
}
}
}
} // note this
},
"aggs": {
"percentiles": {
"percentiles": {
"field": "logstash.load.duration",
"percents": 95,
"keyed": false
}
},
"dates": {
"date_histogram": {
"field": "timestamp",
"fixed_interval": "5m", // note this
"min_doc_count": 1
}
}
}
}
I've got an index with a lot of records with many fields, including "cacheName" & "cache_ip". Each unique value of "cacheName" has 1 or more records with 1 or more values of corresponding "cache_ip". Each record has a unique 'ts' (timestamp) field as well. For example:
{
"cacheName": "c001.abc001.xyz",
"cache_ip": "1.1.1.0",
},
{
"cacheName": "c001.abc001.xyz",
"cache_ip": "1.1.2.0",
},
{
"cacheName": "c002.efg001.mno",
"cache_ip": "1.1.9.1",
},
{
"cacheName": "c002.efg001.mno",
"cache_ip": "1.1.9.1",
},
I'm trying to craft a search that will return, at most, each unique 'cacheName' & 'cache_ip' record. For the above example, I would get back a total of 3 hits ("cacheName"="c002.efg001.mno" would only be returned once, since it only has one unique permutation).
This is the closest that I've come, but it always returns a Null value for "cache_ip" instead of the actual value (there are no null values in the actual data):
{
"size": 0, 'sort': [{'ts': {'order': 'desc'}}],
"query": {
"bool": {
"must": [
{"match_all": {}},
{"range": {'ts': {'gte': '20200818T010100Z', 'format': 'basic_date_time_no_millis'}}},
]
}
},
"aggs": {
"cacheName": {
"terms": {
"field": "cacheName",
"size": 10000, "order": {"_key": "desc"},
},
"aggs": {
"cache_ip": {"terms": {"field": "cache_ip"}},
},
},
},
}
I'd appreciate any insight, as I'm pulling my hair out trying to make this work.
thanks!
One way to achieve what you want to is use scripting to create all the permutations and you wouldn't need the second terms sub-aggregation:
{
"size": 0,
"sort": [
{
"ts": {
"order": "desc"
}
}
],
"query": {
"bool": {
"must": [
{
"range": {
"ts": {
"gte": "20200818T010100Z",
"format": "basic_date_time_no_millis"
}
}
}
]
}
},
"aggs": {
"cacheName": {
"terms": {
"script": {
"source": "[doc.cache_name.value ?: 'no.name', doc.cache_ip.value ?: 'no.ip'].join('-')"
},
"size": 10000,
"order": {
"_key": "desc"
}
}
}
}
}
I am trying to add a prefix condition for my ES query in a "must" clause.
My current query looks something like this:
body = {
"query": {
"bool": {
"must":
{ "term": { "article_lang": 0 }}
,
"filter": {
"range": {
"created_time": {
"gte": "now-3h"
}
}
}
}
},
"aggs": {
"articles": {
"terms": {
"field": "article_id.keyword",
"order": {
"score": "desc"
},
"size": 1000
},
"aggs": {
"score": {
"sum": {
"field": "score"
}
}
}
}
}
}
I need to add a mandatory condition to my query to filter articles whose id starts with "article-".
So, far I have tried this:
{
"query": {
"bool": {
"should": [
{ "term": { "article_lang": 0 }},
{ "prefix": { "article_id": {"value": "article-"} }}
],
"filter": {
"range": {
"created_time": {
"gte": "now-3h"
}
}
}
}
},
"aggs": {
"articles": {
"terms": {
"field": "article_id.keyword",
"order": {
"score": "desc"
},
"size": 1000
},
"aggs": {
"score": {
"sum": {
"field": "score"
}
}
}
}
}
}
I am fairly new to ES and from the documentations online, I know that "should" is to be used for "OR" conditions and "must" for "AND". This is returning me some data but as per the condition it will be consisting of either article_lang=0 or articles starting with article-. When I use "must", it doesn't return anything.
I am certain that there are articles with id starting with this prefix because currently, we are iterating through this result to filter out such articles. What am I missing here?
In your prefix query, you need to use the article_id.keyword field, not article_id. Also, you should prefer filter over must since you're simply doing yes/no matching (aka filters)
{
"query": {
"bool": {
"filter": [ <-- change this
{
"term": {
"article_lang": 0
}
},
{
"prefix": {
"article_id.keyword": { <-- and this
"value": "article-"
}
}
}
],
"filter": {
"range": {
"created_time": {
"gte": "now-3h"
}
}
}
}
},
"aggs": {
"articles": {
"terms": {
"field": "article_id.keyword",
"order": {
"score": "desc"
},
"size": 1000
},
"aggs": {
"score": {
"sum": {
"field": "score"
}
}
}
}
}
}
I am trying to run a filtered aggregation like below but getting error.
"Unknown key for a START_OBJECT in [associations]: [disabledDate]. Can anyone review the query and suggest any changes required.
STEPS in the query:
1. Query all documents with versionDate less than or equal to the given
date.
2. Aggregate on Id.
3. Run a subaggregation top hits query with missing disabledDate filter.
4. apply post filter for missing disabledDate.
{
"query": {
"bool": {
"must": [
{
"range": {
"versionDate": {
"from": null,
"to": "2016-05-25T20:53:22.742Z",
"include_lower": false,
"include_upper": true
}
}
},
{
"terms": {
"domainId": [
"yy"
]
}
},
{
"terms": {
"termId": [
"rr"
]
}
}
]
}
},
"aggregations": {
"associations": {
"terms": {
"field": "id",
"size": 0,
"execution_hint": "global_ordinals_low_cardinality",
"order": {
"_term": "asc"
},
"disabledDate": {
"filters": {
"missing": {
"field": "disbaledDate"
}
},
"aggregations": {
"top": {
"top_hits": {
"size": 1,
"_source": {
"includes": [],
"excludes": []
},
"sort": [
{
"versionDate": {
"order": "desc"
}
}
]
}
}
}
}
}
}
},
"post_filter": {
"missing": {
"field": "disabledDate"
}
}
}
I have an elastic search running with documents like this one:
{
id: 1,
price: 620000,
propertyType: "HO",
location: {
lat: 51.41999,
lon: -0.14426
},
active: true,
rentOrSale: "S",
}
I'm trying to use aggregates to get statistics about a certain area using aggregations and the query I'm using is the following:
{
"sort": [
{
"id": "desc"
}
],
"query": {
"bool": {
"must": [
{
"term": {
"rentOrSale": "s"
}
},
{
"term": {
"active": true
}
}
]
},
"filtered": {
"filter": {
"and": [
{
"geo_distance": {
"distance": "15.0mi",
"location": {
"lat": 51.50735,
"lon": -0.12776
}
}
}
]
}
}
},
"aggs": {
"propertytype_agg": {
"terms": {
"field": "propertyType"
},
"aggs": {
"avg_price": {
"avg": {
"field": "price"
}
}
}
},
"bed_agg": {
"terms": {
"field": "numberOfBedrooms"
},
"aggs": {
"avg_price": {
"avg": {
"field": "price"
}
}
}
}
}
}
But in the result I can't see the aggregations. As soon as I remove either the bool or filtered part of the query I can see the aggregations. I can't figure out why this is happening, nor how do I get the aggregations for these filters. I've tried using the answer to this question but I've not been able to solve it. Any ideas?
I think your query need to be slightly re-arranged - move the "filtered" further up and repeat the "query" command:
"query": {
"filtered": {
"query" : {
"bool": {
...
}
},
"filter": {
...
}
}
}