Elasticsearch query over nested object - elasticsearch

trying to query a particular index with nested field according to the following
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"terms": {
"search_field": [
"search_key"
]
}
}
]
}
},
"nested": {
"path": "defined_path", //defined path is mapped nested
"query": {
"bool": {
"must": [
{
"exists": {
"field": "defined_path.some_field"
}
}
]
}
}
}
]
}
}
}
but this tends to give out some error and the syntax(query_parsing_exception) shows to be wrong but doesn't seem the case.
what might be the issue?

You're missing to encapsulate the 2nd must condition inside {}
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"terms": {
"search_field": [
"search_key"
]
}
}
]
}
},
{
"nested": {
"path": "defined_path",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "defined_path.some_field"
}
}
]
}
}
}
}
]
}
}
}

Related

Elasticsearch filter by matching query in all nested documents

I have a problem with filtering elastic documents by nested documents.
In general document has a list of nested assets and each asset have a list of teamIds
Sample cut off document:
{
"assets":[
{
"id":100,
"teams":[
1
]
},
{
"id":101,
"teams":[
4,
3
]
}
]
}
Expected result is to get root document where all assets have at least one matching team
I've tried:
{
"from": 0,
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"bool": {
"must": [
{
"nested": {
"path": "assets",
"query": {
"terms": {
"assets.teams": [
1
]
}
}
}
}
]
}
},
{
"bool": {
"must_not": [
{
"nested": {
"path": "assets",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "assets"
}
}
]
}
}
}
}
]
}
}
]
}
}
]
}
},
"size": 999
}
Unfortunately this query return document. In this case I do expect it returns document if query contains ids like [1,3], [1,4] or [1,3,4]
Thanks in advance
To find a document where all nested documents contain any of given terms
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "assets",
"query": {
"bool": {
"must_not": [
{
"terms": {
"assets.teams": [
"1"
]
}
}
]
}
}
}
}
]
}
}
}
````
In the above, nested query returns documents where a nested document does not contain any of given term, then outer must_not excludes those documents.
In other words first find documents where a nested document doesnot contain given term and then exclude those documents.
If you want to include documents where teams field is not present use below
````
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "assets",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "assets.teams"
}
}
],
"must_not": [
{
"terms": {
"assets.teams": [
"1"
]
}
}
]
}
}
}
}
]
}
}
}
````

Filter on field value or not exists

I'm trying to filter a field for a specific value OR that it does not exist.
I have the part for the specific value
{
"query": {
"match": {
"app.serviceType": {
"query": "MY_VALUE",
"type": "phrase"
}
}
}
}
I'd also like to add to this any case where the field serviceType doesn't exist at all.
Essentially I'd like the equivalent of this:
serviceType == "MY_VALUE" || string.IsNullOrEmpty(serviceType)
This request must match with your use case :
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"exists": {
"field": "serviceType"
}
},
{
"match_phrase": {
"serviceType": "MY_VALUE"
}
}
]
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "serviceType"
}
}
]
}
}
]
}
}
}
Based on the previous answer (which didn't work but got me close) I was able to get it to work.
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"exists": {
"field": "app.serviceType"
}
},
{
"match_phrase": {
"app.serviceType": "MY_VALUE"
}
}
]
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "app.serviceType"
}
}
]
}
}
]
}
}
}

Filter by array item or empty array

I'm trying to filter by an array that must be either empty or contain the item 1.
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"terms": {
"memberOrganizationFamilyIds": [
1
]
}
}
],
"must_not": [
{
"exists": {
"field": "memberOrganizationFamilyIds"
}
}
]
}
}
}
}
}
According to the docs this is how it should be but it isn't working.
If we apply the first filter it works.
{
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"terms": {
"memberOrganizationFamilyIds": [
1
]
}
}
]
}
}
}
}
}
If we apply the second filter it also works.
{
"query": {
"bool": {
"filter": {
"bool": {
"must_not": [
{
"exists": {
"field": "memberOrganizationFamilyIds"
}
}
]
}
}
}
}
}
But not together.
I'm trying to filter by an array that must be either empty or contain the item 1
Try this you should add should (meaning) or
{
"query": {
"bool": {
"filter": {
"bool": {
"should": [
{
"terms": {
"memberOrganizationFamilyIds": [
1
]
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "memberOrganizationFamilyIds"
}
}
]
}
}
]
}
}
}
}
}

How to let ES not score on should cause

i am just using ES to filter out some data, not need score function at all. but there are some or logic i have to include. i know we can replace must with filter, so ES will not score it and may cache it. How to profile should cause?
it this two query have different?
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"term": {
"closeStatus": 0
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "closeStatus"
------------------------------------------------
{
"query": {
"bool": {
"should": [
{
"term": {
"closeStatus": 0
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "closeStatus"
If you don't care about the score, use constant_score query:
{
"query": {
"constant_score": {
"filter": {
"bool": {
"should": [
{
"term": {
"closeStatus": 0
}
},
{
"bool": {
"must_not": {
"exists": {"field": "closeStatus"}
}
}
}
]
}
}
}
}
}

Elastic Search - OR querying for non matches

I'm having trouble querying in elastic search. I'm searching over a specific set of data defined by the state_id, and then wanting to return all the states which do not have either one of the cities defined by the identifiers below.
The query below returns 18 results with just "city_id_1", and 0 results with "city_id_2". With both though, I return 0 results (since "city_id_2" is on every state record). What I want to do is still return the 18 results, but query over both cities.
I feel like my query should be working, and basically doing a NOT (A or B) style query, equivalent to NOT A and NOT B, but basically the 0 results seems to be overriding the 18.
Is there a way I can change my query to get the results I want, or is this something elasticsearch cannot do?
{
"query": {
"bool": {
"must": [
{ "terms": { "state_id": ["4ca16f80-da79-11e5-9874-64006a4f57cb"]}}
],
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"bool": {
"should": [
{"term": { "cities.identifier": "city_id_1"}},
{"term": { "cities.identifier": "city_id_2"}}
]
}
}
}
}
]
}
},
"size": 10
}
Try this on for size. Elasticsearch is silly. The filter needs to be in each of the nested queries.
{
"query": {
"bool": {
"should": [
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"term": { "cities.identifier": "city_id_1"}
}
}
}
],
"filter":[
{
"term":{
"state_id":"4ca16f80-da79-11e5-9874-64006a4f57cb"
}
}
]
}
}
},
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"term": { "cities.identifier": "city_id_2"}
}
}
}
],
"filter":[
{
"term":{
"state_id":"4ca16f80-da79-11e5-9874-64006a4f57cb"
}
}
]
}
}
}
]
}
},
"size": 10
}
If you want NOT A AND NOT B behaviour you need to make a little change
{
"query": {
"bool": {
"must": [
{ "terms": { "state_id": ["4ca16f80-da79-11e5-9874-64006a4f57cb"]}}
],
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"bool": {
"must": [ ====> Use must instead of should
{"term": { "cities.identifier": "city_id_1"}},
{"term": { "cities.identifier": "city_id_2"}}
]
}
}
}
}
]
}
},
"size": 10
}
This will exclude those record which will have both city_id_1 and city_id_2.
As per my understanding, you are looking our for NOT A or NOT B kind of a clause. Please check the query below and see if it fits your requirement
{
"query": {
"bool": {
"must": [
{ "terms": { "state_id": ["4ca16f80-da79-11e5-9874-64006a4f57cb"]}}
],
"should": [
{
"nested": {
"path": "cities",
"query": {
"bool": {
"must_not": [
{"term": { "cities.identifier": "city_id_1"}}
]
}
}
}
},
{
"nested": {
"path": "cities",
"query": {
"bool": {
"must_not": [
{"term": { "cities.identifier": "city_id_2"}}
]
}
}
}
}
],
"minimum_number_should_match": 1
}
},
"size": 10
}

Resources