After upgrading to Elasticsearch 2.x I got an issue with the following query:
{ "query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"terms": {
"_type": [
"xxx",
"yyy"
]
}
},
{
"exists": {
"field": "aaa"
}
},
{
"exists": {
"field": "bbb"
}
},
{
"exists": {
"field": "ccc"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"_type": "eee"
}
},
{
"term": {
"f": 0
}
}
]
}
}
]
}
}
} } }
Basically, I do not know how to replace the 'must' inside the 'should' filter with the new query DSL rules in Elasticsearch 2.x.
Thanks in advance.
You can simply remove the filtered/filter part and modify your query like this:
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"terms": {
"_type": [
"xxx",
"yyy"
]
}
},
{
"exists": {
"field": "aaa"
}
},
{
"exists": {
"field": "bbb"
}
},
{
"exists": {
"field": "ccc"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"_type": "eee"
}
},
{
"term": {
"f": 0
}
}
]
}
}
]
}
}
}
Related
I would like to have an intersection of 2 queries
I got 3 documents in the index:
"_id": "68c220aa-ea51-4f84-b880-29af3302cae9",
"_id": "b6c1c3c5-e959-480f-a145-f5598fafea66",
"_id": "2d30de72-0a2b-465c-8770-970ad9760d47",
Query1:
{
"from": 0,
"query": {
"nested": {
"path": "attributes",
"query": {
"bool": {
"must": [
{
"bool": {
"must": [
{
"match_phrase": {
"attributes.asReference": {
"query": "8670ff39-6a0d-4ae8-e217-08d88efd4771"
}
}
},
{
"match_phrase": {
"attributes.attributeId": {
"query": "f51ca670-4223-4ea2-8007-d111dd38a14f"
}
}
}
]
}
}
]
}
}
}
},
"size": 10,
"sort": [
{
"modified": {
"order": "asc"
}
},
{
"created": {
"order": "asc"
}
}
]
}
returns all 3 documents as it should
"_id": "68c220aa-ea51-4f84-b880-29af3302cae9",
"_id": "b6c1c3c5-e959-480f-a145-f5598fafea66",
"_id": "2d30de72-0a2b-465c-8770-970ad9760d47",
Then I do query2:
{
"from": 0,
"query": {
"nested": {
"path": "attributes",
"query": {
"bool": {
"must": [
{
"bool": {
"must": [
{
"match_phrase": {
"attributes.asShortString": {
"query": "RA-005"
}
}
},
{
"match_phrase": {
"attributes.attributeId": {
"query": "7ff3dbc1-3586-4475-9162-5430bb06c6d0"
}
}
}
]
}
}
]
}
}
}
},
"size": 10,
"sort": [
{
"modified": {
"order": "asc"
}
},
{
"created": {
"order": "asc"
}
}
]
}
returns 1 document:
"_id": "b6c1c3c5-e959-480f-a145-f5598fafea66"
But when I combine the queries to:
{
"from": 0,
"query": {
"nested": {
"path": "attributes",
"query": {
"bool": {
"must": [
{
"bool": {
"must": [
{
"match_phrase": {
"attributes.asReference": {
"query": "8670ff39-6a0d-4ae8-e217-08d88efd4771"
}
}
},
{
"match_phrase": {
"attributes.attributeId": {
"query": "f51ca670-4223-4ea2-8007-d111dd38a14f"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"match_phrase": {
"attributes.asShortString": {
"query": "RA-005"
}
}
},
{
"match_phrase": {
"attributes.attributeId": {
"query": "7ff3dbc1-3586-4475-9162-5430bb06c6d0"
}
}
}
]
}
}
]
}
}
}
},
"size": 10,
"sort": [
{
"modified": {
"order": "asc"
}
},
{
"created": {
"order": "asc"
}
}
]
}
Here I do not get any documents
So the subqueries are working but combined it does not work (it produces 0 results)
What am I missing here?
Due to the way nested documents and queries work, you need to have two separate nested queries in your bool/must query, because each will/might match a different nested document of the same parent document:
{
"from": 0,
"query": {
"bool": {
"must": [
{
"nested": {
"path": "attributes",
"query": {
"bool": {
"must": [
{
"bool": {
"must": [
{
"match_phrase": {
"attributes.asReference": {
"query": "8670ff39-6a0d-4ae8-e217-08d88efd4771"
}
}
},
{
"match_phrase": {
"attributes.attributeId": {
"query": "f51ca670-4223-4ea2-8007-d111dd38a14f"
}
}
}
]
}
}
]
}
}
}
},
{
"nested": {
"path": "attributes",
"query": {
"bool": {
"must": [
{
"bool": {
"must": [
{
"match_phrase": {
"attributes.asShortString": {
"query": "RA-005"
}
}
},
{
"match_phrase": {
"attributes.attributeId": {
"query": "7ff3dbc1-3586-4475-9162-5430bb06c6d0"
}
}
}
]
}
}
]
}
}
}
}
]
}
},
"size": 10,
"sort": [
{
"modified": {
"order": "asc"
}
},
{
"created": {
"order": "asc"
}
}
]
}
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "categories",
"query": {
"terms": {
"categories.id": [
9
]
}
}
}
},
{
"terms": {
"deleted": [
false
]
}
},
{
"terms": {
"published": [
true
]
}
},
{
"nested": {
"path": "vendors",
"query": {
"terms": {
"vendors.isDeletedVendor": [
false
]
}
}
}
},
{
"nested": {
"path": "vendors",
"query": {
"terms": {
"vendors.isPublishedVendor": [
true
]
}
}
}
}
],
"should": [
{
"bool": {
"must": [
{
"bool": {
"should": [
{
"bool": {
"must_not": [
{
"nested": {
"path": "manufacturers",
"query": {
"exists": {
"field": "manufacturers"
}
}
}
}
]
}
}
]
}
},
{
"bool": {
"should": [
{
"nested": {
"path": "vendors",
"query": {
"terms": {
"vendors.id": [
87
]
}
}
}
}
]
}
}
]
}
},
{
"bool": {
"must": [
{
"bool": {
"should": [
{
"bool": {
"must_not": [
{
"nested": {
"path": "manufacturers",
"query": {
"exists": {
"field": "manufacturers"
}
}
}
}
]
}
}
]
}
},
{
"bool": {
"should": [
{
"nested": {
"path": "vendors",
"query": {
"terms": {
"vendors.id": [
56
]
}
}
}
}
]
}
}
]
}
}
]
}
},
"size": 1000,
"_source": {
"includes": [
"modelName",
"vendors.id",
"manufacturers.id",
"id"
]
}
}
So what I am doing here is something like this:
Select * from data where
(manufacturerIds is null and vendorIds in 87) or
(manufacturerIds is null and vendorIds in 56)
But it is returning results like this:
"hits" : [
{
"_index" : "onoff-live",
"_type" : "_doc",
"_id" : "130011",
"_score" : 5.0,
"_source" : {
"modelName" : "Galaxy A20",
"manufacturers" : [
{
"id" : 216
}
],
"id" : 130011,
"vendors" : [
{
"id" : 23
}
]
}
}]
I don't know what I am missing, please give me suggestions, if you want mapping I will post it in reply...
I am using Elasticsearch to search the Packetbeat indices to identify if two IP addresses communicate. If IP xx.xx.xx.xx talks to IP yy.yy.yy.yy OR if IP yy.yy.yy.yy talk to IP xx.xx.xx.xx, I want to know about it. Below is my DSL but all the returned results are not relevant at all. What am I doing wrong? Thanks!
GET /packetbeat-*/_search?size=100&pretty
{
"query": {
"bool": {
"must": [
{
"term": {
"_type": "flow"
}
}
],
"must_not": [
{
"term": {
"source.ip": "127.0.0.1"
}
},
{
"term": {
"dest.ip": "127.0.0.1"
}
}
],
"should": [
{
"bool": {
"must": [
{
"term": {
"_type": "flow"
}
},
{
"term": {
"source.ip": "xx.xx.xx.xx"
}
},
{
"term": {
"dest.ip": "yy.yy.yy.yy"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"_type": "flow"
}
},
{
"term": {
"source.ip": "yy.yy.yy.yy"
}
},
{
"term": {
"dest.ip": "xx.xx.xx.xx"
}
}
]
}
}
],
"filter": {
"range": {
"#timestamp": {
"gte": "now-30d/d",
"lte": "now-1d/d"
}
}
}
}
}
}
To simplify your query:
_type: flow
not localhost
source.ip != dest.ip
source.ip OR dest.ip equal to IP_X OR IP_Y
Have a look, according to this answer:
{
"query": {
"bool": {
"must": [
{
"term": {
"_type": "flow"
}
},
{
"script": {
"script": "doc['source.ip'].value != doc['dest.ip'].value"
}
},
{
"terms": {
"source.ip": [
"IP_X",
"IP_Y"
]
}
},
{
"terms": {
"dest.ip": [
"IP_X",
"IP_Y"
]
}
}
],
"must_not": [
{
"term": {
"source.ip": "127.0.0.1"
}
},
{
"term": {
"dest.ip": "127.0.0.1"
}
}
],
"filter": {
"range": {
"#timestamp": {
"gte": "now-30d/d",
"lte": "now-1d/d"
}
}
}
}
}
}
{
"sort": [
{
"is_active": "asc"
}
],
"fields": [
"is_job_seeking", "is_active"
],
"query": {
"bool": {
"must": [
{
"bool": {
"must": {
"term": {
"is_job_seeking": 1
}
}
}
}
]
}
}
}
this query return me all document which has is_job_seeking=1, and is_active=0 and is_active=1 and that's fine, now when I want to boost score for document which has is_active=1 I have add boosting like
{
"sort": [
{
"is_active": "asc"
}
],
"fields": [
"is_job_seeking", "is_active"
],
"query": {
"bool": {
"must": [
{
"bool": {
"must": {
"term": {
"is_job_seeking": 1
}
}
}
},
{
"boosting": {
"positive": {
"term": {
"is_active": 1
}
},
"negative": {
"term": {
"is_active": 0
}
},
"negative_boost": 0.3
}
}
]
}
}
}
but this give me results only with is_active=1
Try this:
{
"sort": [
{
"is_active": "asc"
}
],
"fields": [
"is_job_seeking",
"is_active"
],
"query": {
"bool": {
"must": [
{
"bool": {
"must": {
"term": {
"is_job_seeking": 1
}
}
}
}
],
"should": [
{
"boosting": {
"positive": {
"term": {
"is_active": 1
}
},
"negative": {
"term": {
"is_active": 0
}
},
"negative_boost": 0.3
}
}
]
}
}
}
How can I order the results by _score?
I can't figure out how to calculate the score for each result, also :)
I managed to write this:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"type_licitatie": "3"
}
},
{
"term": {
"tip_sursa": "5"
}
}
]
}
}
}
},
"sort": [
{
"_score": {
"order": "desc"
}
}
]
}
and this:
{
"query": {
"function_score": {
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"term": {
"country_id": "1"
}
},
{
"term": {
"industry_id": "3"
}
}
]
}
}
}
},
"script_score" : {
"script": "(doc['country_id'].values=1) + (doc['industry_id'].values=3)"
},
"boost_mode": "replace"
}
}
}