I'm having trouble querying in elastic search. I'm searching over a specific set of data defined by the state_id, and then wanting to return all the states which do not have either one of the cities defined by the identifiers below.
The query below returns 18 results with just "city_id_1", and 0 results with "city_id_2". With both though, I return 0 results (since "city_id_2" is on every state record). What I want to do is still return the 18 results, but query over both cities.
I feel like my query should be working, and basically doing a NOT (A or B) style query, equivalent to NOT A and NOT B, but basically the 0 results seems to be overriding the 18.
Is there a way I can change my query to get the results I want, or is this something elasticsearch cannot do?
{
"query": {
"bool": {
"must": [
{ "terms": { "state_id": ["4ca16f80-da79-11e5-9874-64006a4f57cb"]}}
],
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"bool": {
"should": [
{"term": { "cities.identifier": "city_id_1"}},
{"term": { "cities.identifier": "city_id_2"}}
]
}
}
}
}
]
}
},
"size": 10
}
Try this on for size. Elasticsearch is silly. The filter needs to be in each of the nested queries.
{
"query": {
"bool": {
"should": [
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"term": { "cities.identifier": "city_id_1"}
}
}
}
],
"filter":[
{
"term":{
"state_id":"4ca16f80-da79-11e5-9874-64006a4f57cb"
}
}
]
}
}
},
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"term": { "cities.identifier": "city_id_2"}
}
}
}
],
"filter":[
{
"term":{
"state_id":"4ca16f80-da79-11e5-9874-64006a4f57cb"
}
}
]
}
}
}
]
}
},
"size": 10
}
If you want NOT A AND NOT B behaviour you need to make a little change
{
"query": {
"bool": {
"must": [
{ "terms": { "state_id": ["4ca16f80-da79-11e5-9874-64006a4f57cb"]}}
],
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"bool": {
"must": [ ====> Use must instead of should
{"term": { "cities.identifier": "city_id_1"}},
{"term": { "cities.identifier": "city_id_2"}}
]
}
}
}
}
]
}
},
"size": 10
}
This will exclude those record which will have both city_id_1 and city_id_2.
As per my understanding, you are looking our for NOT A or NOT B kind of a clause. Please check the query below and see if it fits your requirement
{
"query": {
"bool": {
"must": [
{ "terms": { "state_id": ["4ca16f80-da79-11e5-9874-64006a4f57cb"]}}
],
"should": [
{
"nested": {
"path": "cities",
"query": {
"bool": {
"must_not": [
{"term": { "cities.identifier": "city_id_1"}}
]
}
}
}
},
{
"nested": {
"path": "cities",
"query": {
"bool": {
"must_not": [
{"term": { "cities.identifier": "city_id_2"}}
]
}
}
}
}
],
"minimum_number_should_match": 1
}
},
"size": 10
}
Related
With the following mapping:
"categories": {
"type": "nested",
"properties": {
"category": {
"type": "integer"
},
"score": {
"type": "float"
}
}
},
I want to use the categories field to return documents that either:
have a score above a threshold in a given category, or
do not have the categories field
This is my query:
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "categories",
"query": {
"bool": {
"must": [
{
"terms": {
"categories.category": [
<id>
]
}
},
{
"range": {
"categories.score": {
"gte": 0.5
}
}
}
]
}
}
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "categories"
}
}
]
}
}
],
"minimum_should_match": 1
}
}
}
It correctly returns documents both with and without the categories field, and orders the results so the ones I want are first, but it doesn't filter the results having score below the 0.5 threshold.
Great question.
That is because categories is not exactly a field from the elasticsearch point of view[a field on which inverted index is created and used for querying/searching] but categories.category and categories.score is.
As a result categories being not found in any document, which is actually true for all the documents, you observe the result what you see.
Modify the query to the below and you'd see your use-case working correctly.
POST <your_index_name>/_search
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "categories",
"query": {
"bool": {
"must": [
{
"terms": {
"categories.category": [
"100"
]
}
},
{
"range": {
"categories.score": {
"gte": 0.5
}
}
}
]
}
}
}
},
{
"bool": {
"must_not": [ <----- Note this
{
"nested": {
"path": "categories",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "categories.category"
}
},
{
"exists": {
"field": "categories.score"
}
}
]
}
}
}
}
]
}
}
],
"minimum_should_match": 1
}
}
}
i am just using ES to filter out some data, not need score function at all. but there are some or logic i have to include. i know we can replace must with filter, so ES will not score it and may cache it. How to profile should cause?
it this two query have different?
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"term": {
"closeStatus": 0
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "closeStatus"
------------------------------------------------
{
"query": {
"bool": {
"should": [
{
"term": {
"closeStatus": 0
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "closeStatus"
If you don't care about the score, use constant_score query:
{
"query": {
"constant_score": {
"filter": {
"bool": {
"should": [
{
"term": {
"closeStatus": 0
}
},
{
"bool": {
"must_not": {
"exists": {"field": "closeStatus"}
}
}
}
]
}
}
}
}
}
`"query": {
"function_score": {
"query": {
"bool": {
"must": [],
"should": [],
"filter": [
{
"terms": {
"category": "type-1",
"product": "product-A"
},
"terms": {
"category": "type-2",
"product": "product-B"
}
}
]
}
},
"functions": []
}
},`
I want to pass multiple combination query like above is it possible, what should be the correct query format
in sql my query would be
select * from product where (category='type1' and product=product-A) or (category='type2' and product=product-B) or (category='type3' and product=product-C)
i want to replicate above query
If you want to make a OR statement in a bool query you should is a nested bool query with multiple should clause.
so try :
{
"query": {
"function_score": {
"query": {
"bool": {
"must": [],
"should": [],
"filter": [
{
"bool": {
"should": [
{
"bool": {
"must": [
{
"term": {
"category": "type-1"
}
},
{
"term": {
"product": "product-A"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"category": "type-2"
}
},
{
"term": {
"product": "product-B"
}
}
]
}
}
]
}
}
]
}
}
}
},
"functions": []
}
and if you have no must clause you can move your filters clauses into the main should as only document that matchs at least one of the clause will match.
trying to query a particular index with nested field according to the following
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"terms": {
"search_field": [
"search_key"
]
}
}
]
}
},
"nested": {
"path": "defined_path", //defined path is mapped nested
"query": {
"bool": {
"must": [
{
"exists": {
"field": "defined_path.some_field"
}
}
]
}
}
}
]
}
}
}
but this tends to give out some error and the syntax(query_parsing_exception) shows to be wrong but doesn't seem the case.
what might be the issue?
You're missing to encapsulate the 2nd must condition inside {}
{
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"terms": {
"search_field": [
"search_key"
]
}
}
]
}
},
{
"nested": {
"path": "defined_path",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "defined_path.some_field"
}
}
]
}
}
}
}
]
}
}
}
I'm running Elasticsearch 1.5.2 and trying the following query:
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"gender": "male"
}
}
]
}
},
"query": {
"bool": {
"must": [
{
"match_all": {}
}
],
"should": [
{
"term": {
"top_users": 1,
"boost": 2
}
}
]
}
}
}
}
}
Everything is fine until I add the "boost": 2 to the should -> term part. The complete query is much more complex, that's why I need to boost, but the remaining queries don't make any difference: ES returns an error 400 if a term query gets a boost argument:
QueryParsingException[[index_name] [_na] query malformed, must start with start_object]
Any suggestions?
It should be like this:
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"gender": "male"
}
}
]
}
},
"query": {
"bool": {
"must": [
{
"match_all": {}
}
],
"should": [
{
"term": {
"top_users": {
"value": "1",
"boost": 2
}
}
}
]
}
}
}
}
}