Which DSL is correct for performing a pre-filtered query? - elasticsearch

I've looked back at some queries I have saved, and it appears I've managed to achieve essentially the same query in three different ways. They all return the same data, but which one is 'correct'? I.e., which one contains no superfluous code and is most performant?
Option 1
{
"query":{
"bool":{
"must":[
{
"match":{
"event":"eventname"
}
},
{
"range":{
"#timestamp":{
"gt":"now-70s"
}
}
}
]
}
},
"aggs":{
"myterms":{
"terms":{
"field":"fieldname"
}
}
}
}
Option 2
{
"query":{
"filtered":{
"filter":{
"bool":{
"must":[
{
"match":{
"event":"eventname"
}
},
{
"range":{
"#timestamp":{
"gt":"now-70s"
}
}
}
]
}
}
}
},
"aggs":{
"myterms":{
"terms":{
"field":"fieldname"
}
}
}
}
Option 3
{
"query":{
"filtered":{
"query":{
"bool":{
"must":[
{
"match":{
"event":"eventname"
}
},
{
"range":{
"#timestamp":{
"gt":"now-70s"
}
}
}
]
}
}
}
},
"aggs":{
"myterms":{
"terms":{
"field":"fieldname"
}
}
}
}
If I were to guess, I'd go for Option 2, as the others appear that they might be running match as query. But the documentation is pretty confusing regarding the correct form that DSL queries should take.

Based on your comment, I'd go for option 2 but with a simple term filter for starters instead of match which isn't allowed in filters.
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"event": "eventname"
}
},
{
"range": {
"#timestamp": {
"gt": "now-70s"
}
}
}
]
}
}
}
},
"aggs": {
"myterms": {
"terms": {
"field": "event"
}
}
}
}

Related

How to use "OR" in Dev Tool Query

Hi Bellow Search provides me Log where it has both "value": "HB" and "value": "1234567" as, I am using Term, however, What I am looking for this if this match
("value": "HB" OR "value": "TR" ) AND "value": "1234567"
but not understanding how to do in below,
Can anyone please help me
GET _search
{ "query": { "bool": { "must": [ { "match": {"log.file.path":"mylog.log" } }
{
"term": {
"GPS-LOG.COMMAND": {
"value": "HB"
}
}
},
{
"term": {
"GPS-LOG.IMEI": {
"value": "1234567"
}
}
}
], "filter": {
"range": {
"#timestamp": {
"gte": "now-10m"
}
} }
} }
At first glace, it seems like this should have a simple solution. However, since you are using the term query, you can only search one value at a time. I don't know your mapping but if you are using a text field you shouldn't be using term query.
However, to solve this using the term query, you have to create the OR operator using the minimum_should_match combined with should.
See the following code:
GET _search
{
"query":{
"bool":{
"must":[
{
"match":{
"log.file.path":"mylog.log"
}
},
{
"term":{
"GPS-LOG.IMEI":{
"value":"1234567"
}
}
},
{
"bool":{
"should":[
{
"term":{
"GPS-LOG.COMMAND":{
"value":"HB"
}
}
},
{
"term":{
"GPS-LOG.COMMAND":{
"value":"TR"
}
}
}
],
"minimum_should_match":1
}
}
],
"filter":{
"range":{
"#timestamp":{
"gte":"now-10m"
}
}
}
}
}
}

Elasticsearch Aggregations: filtering a global aggregation with nested queries

I have 'nested' mapping like so:
"stringAttributes":{
"type":"nested",
"properties":{
"Name":{
"type":"keyword"
},
"Value":{
"type":"keyword"
}
}
},
and thus have docs that such as:
stringAttributes:[
{
Name:"supplier",
Value:"boohoo"
},
{
Name:"brand",
Value:"gucci"
},
{
Name:"primaryColour",
Value:"black"
},
{
Name:"secondaryColour",
Value:"green"
},
{
Name:"size",
Value:"12"
}
]
In building faceted search I believe I need a global aggregation. I.e. when a supplier is filtered by a user, the result set will not contains docs from other suppliers, so the regular aggregation will not contain any of the other supplier.
The query could include the following clauses:
"must": [
{
"nested": {
"path": "stringAttributes",
"query": {
"bool": {
"must": [
{
"term": {
"stringAttributes.Name": "supplier"
}
},
{
"terms": {
"stringAttributes.Value": [
"boohoo"
]
}
}
]
}
}
}
},
{
"nested": {
"path": "stringAttributes",
"query": {
"bool": {
"must": [
{
"term": {
"stringAttributes.Name": "brand"
}
},
{
"terms": {
"stringAttributes.Value": [
"warehouse"
]
}
}
]
}
}
}
}
]
So in this case I need a global aggregation that is then filtered by all OTHER filters applied (e.g. by brand) that will return the other suppliers that could be selected given these other filters.
This is what I have so far. It returns the 'global' unfiltered results however. At this point I am completely stumped.
{
"global":{},
"aggs":{
"inner":{
"filter":{
"nested":{
"query":{
"bool":{
"filter":[
{
"term":{
"stringAttributes.Name":{
"value":"brand"
}
}
},
{
"terms":{
"stringAttributes.Value":[
"warehouse"
]
}
}
]
}
},
"path":"stringAttributes"
}
}
},
"aggs":{
"nested":{
"path":"stringAttributes"
},
"aggs":{
"aggs":{
"filter":{
"match":{
"stringAttributes.Name":"supplier"
}
},
"aggs":{
"facet_value":{
"terms":{
"size":1000,
"field":"stringAttributes.Value"
}
}
}
}
}
}
}
}
Any suggestions for filtering a global aggregation with nested attributes? I have read through a lot of documentation of various other answers on SO but still struggling to understand why this particular agg is not being filtered.
My suggested answer after some more digging...
{
"global":{
},
"aggs":{
"inner":{
"filter":{
"nested":{
"query":{
"bool":{
"filter":[
{
"term":{
"stringAttributes.Name":{
"value":"brand"
}
}
},
{
"terms":{
"stringAttributes.Value":[
"warehouse"
]
}
}
]
}
},
"path":"stringAttributes"
}
},
"aggs":{
"nested":{
"path":"stringAttributes"
},
"aggs":{
"agg_filtered_special":{
"filter":{
"match":{
"stringAttributes.Name":"supplier"
}
},
"aggs":{
"facet_value":{
"terms":{
"size":1000,
"field":"stringAttributes.Value"
}
}
}
}
}
}
}
}
}

Filtering ElasticSearch query where date value is lte a given value or missing

I need to filter an ES query where the value of a date field is LTE a given value or the field is missing altogether. Here's my query at this point:
{
"from":0,
"size":50,
"query":{
"bool":{
"filter":[
{
"term":{
"corpusid.string.as_is":"42:6:4"
}
},
{
"nested":{
"path":"category.object",
"query":{
"bool":{
"must":[
{
"bool":{
"should":[
{
"range":{
"category.object.startdate":{
"lte":"2021-03-09T19:32:11.316Z"
}
}
},
{
"must_not":[
{
"exists":{
"field":"category.object.startdate"
}
}
]
}
]
}
}
]
}
}
}
}
]
}
}
}
When I submit that query, I get the error "[must_not] query malformed, no start_object after query name". We're running ElasticSearch version 5.3.1 in case that matters.
I refactored the query a bit. Removed a must, added a bool for the must_not.
{
"from":0,
"size":50,
"query":{
"bool":{
"filter":[
{
"term":{
"corpusid.string.as_is":"42:6:4"
}
},
{
"nested":{
"path":"category.object",
"query":{
"bool":{
"should": [
{
"range":{
"category.object.startdate":{
"lte":"2021-03-09T19:32:11.316Z"
}
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "category.object.startdate"
}
}
}
}
]
}
}
}
}
]
}
}
}

How to sort mutliple array value search in elasticsearch

I have an array of values that I want to search and display its contents. What is the best method to sort the array and show the result in the array's current format.
Example my array value
$myarray=array('84790','19162002','74739','86439','88820','19560020','19634461','19624154','19624091','19577228');
{
"query":{
"filtered":{
"filter":{
"bool":{
"should":[
{
"term":{
"podcast_id":"84790"
}
},
{
"term":{
"podcast_id":"19162002"
}
},
{
"term":{
"podcast_id":"74739"
}
},
{
"term":{
"podcast_id":"86439"
}
},
{
"term":{
"podcast_id":"88820"
}
},
{
"term":{
"podcast_id":"19560020"
}
},
{
"term":{
"podcast_id":"19634461"
}
},
{
"term":{
"podcast_id":"19624154"
}
},
{
"term":{
"podcast_id":"19624091"
}
},
{
"term":{
"podcast_id":"19577228"
}
}
]
}
}
}
}
}
I am using php, curl and post method.
Thanks
You can use Terms filter insted.
somthing like this :
{
"query": {
"filtered": {
"filter": {
"terms": {
"podcast_id": [
"84790",
"19162002",
.....
]
}
}
}
}
}

How to do mulitple text search in elastic search

I want to do multiple text search in same field
for example in sub_cat_seo_url field i want to get 'english-news' and 'business-news' filter by language and region
when i tried like below code it is not working
{
"query":{
"filtered":{
"query":{
"query_string":{
"query":[
"english-news",
"business-news"
],
"fields":[
"sub_cat_seo_url"
]
}
},
"filter":{
"bool":{
"must":[
{
"term":{
"lang":"en"
}
},
{
"term":{
"region":"1"
}
}
]
}
}
}
}
}
For single text search it is working fine
{
"query":{
"filtered":{
"query":{
"query_string":{
"query":"english-news",
"fields":[
"sub_cat_seo_url"
]
}
},
"filter":{
"bool":{
"must":[
{
"term":{
"lang":"en"
}
},
{
"term":{
"region":"1"
}
}
]
}
}
}
}
}
Please help what have to change in my code, to do multi text search in same field (or operation)
Thanks
Thanigaivelan
Try out this
{
"query": {
"filtered": {
"query": {
"query_string": {
"query": "\"english-news\" AND \"business-news\"",
"fields": [
"sub_cat_seo_url"
]
}
},
"filter": {
"bool": {
"must": [
{
"term": {
"lang": "en"
}
},
{
"term": {
"region": "1"
}
}
]
}
}
}
}
}

Resources