So, I am having issues with Nested+Filter queries. I am firing same queries (except how json positions the key, value pairs) and getting different results (one right response and another throws error). Sounds crazy right? Well I tried it several times, cross-checked it, its true.
Successful Query:
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": {
"nested": {
"filter": {
"bool": {
"must": {
"nested": {
"filter": {
"bool": {
"must": [{
"term": {
"stream.tagging.tag": "some_tag"
}
}]
}
},
"path": "stream.tagging"
}
}
}
},
"path": "stream"
}
}
},
"and": [{
"not": {
"term": {
"deleted": "true"
}
}
}, {
"term": {
"source": 11
}
}]
}
}
}
}
Error Query:
{
"query": {
"filtered": {
"filter": {
"and": [{
"not": {
"term": {
"deleted": "true"
}
}
}, {
"term": {
"source": 11
}
}],
"bool": {
"must": {
"nested": {
"filter": {
"bool": {
"must": {
"nested": {
"filter": {
"bool": {
"must": [{
"term": {
"stream.tagging.tag": "some_tag"
}
}]
}
},
"path": "stream.tagging"
}
}
}
},
"path": "stream"
}
}
}
}
}
}
}
Error Message:
nested: QueryParsingException[[stream_rules_test] No filter registered for [must]]; }]","status":400
Also note when I fire them individually, only nested and only filter queries, it works fine too. I hope I am not making some silly mistake, but I am almost sure I am not missing anything.
ES Version: 1.5
The issue appears to be with the version with new versions not supporting "and: queries. Below query works all the time. Still confused why it doesn't work when key, value pairs are changed.
{
"query": {
"filtered": {
"filter": {
"bool": {
"must_not": {
"term": {
"deleted": "true"
}
},
"must": [{
"term": {
"source": 11
}
},{
"nested": {
"filter": {
"bool": {
"must": {
"nested": {
"filter": {
"bool": {
"must": [{
"term": {
"stream.tagging.tag": "some_tag"
}
}]
}
},
"path": "stream.tagging"
}
}
}
},
"path": "stream"
}
}]
}
}
}
}
}
Related
With the following mapping:
"categories": {
"type": "nested",
"properties": {
"category": {
"type": "integer"
},
"score": {
"type": "float"
}
}
},
I want to use the categories field to return documents that either:
have a score above a threshold in a given category, or
do not have the categories field
This is my query:
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "categories",
"query": {
"bool": {
"must": [
{
"terms": {
"categories.category": [
<id>
]
}
},
{
"range": {
"categories.score": {
"gte": 0.5
}
}
}
]
}
}
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "categories"
}
}
]
}
}
],
"minimum_should_match": 1
}
}
}
It correctly returns documents both with and without the categories field, and orders the results so the ones I want are first, but it doesn't filter the results having score below the 0.5 threshold.
Great question.
That is because categories is not exactly a field from the elasticsearch point of view[a field on which inverted index is created and used for querying/searching] but categories.category and categories.score is.
As a result categories being not found in any document, which is actually true for all the documents, you observe the result what you see.
Modify the query to the below and you'd see your use-case working correctly.
POST <your_index_name>/_search
{
"query": {
"bool": {
"should": [
{
"nested": {
"path": "categories",
"query": {
"bool": {
"must": [
{
"terms": {
"categories.category": [
"100"
]
}
},
{
"range": {
"categories.score": {
"gte": 0.5
}
}
}
]
}
}
}
},
{
"bool": {
"must_not": [ <----- Note this
{
"nested": {
"path": "categories",
"query": {
"bool": {
"must": [
{
"exists": {
"field": "categories.category"
}
},
{
"exists": {
"field": "categories.score"
}
}
]
}
}
}
}
]
}
}
],
"minimum_should_match": 1
}
}
}
I am using a functionScoreQuery provided by the elastic-builder npm to query my elasticsearch, query is getting created but i am not able to get outer query params for my query as shown below.
i.e the outer query params are missing and that is why the query does not execute so i had to manually append query { } in my body. So if anyone can help me out and tell me what i am missing in my npm query to get those query params.
var not_body = elasticbuilder.functionScoreQuery()
.query(elasticbuilder.matchAllQuery())
.functions([
elasticbuilder.weightScoreFunction()
.filter(elasticbuilder.boolQuery().mustNot([
elasticbuilder.hasChildQuery(
elasticbuilder.boolQuery().must([
elasticbuilder.matchPhraseQuery("name", "raju" )
])
).type('student')
]))
.weight(2),
elasticbuilder.weightScoreFunction()
.filter(elasticbuilder.boolQuery().must([
elasticbuilder.hasChildQuery(
elasticbuilder.boolQuery().must([
elasticbuilder.matchPhraseQuery("class", "12")
])
).type('info')
]))
.weight(2)
]).minScore(4).scoreMode('sum');
Current Output body via this query:
{
"function_score": {
"functions": [
{
"filter": {
"bool": {
"must_not": {
"has_child": {
"query": {
"bool": {
"must": {
"match_phrase": {
"name" : "raju"
}
}
}
},
"type": "student"
}
}
}
},
"weight": 2
},
{
"filter": {
"bool": {
"must": {
"has_child": {
"query": {
"bool": {
"must": {
"match_phrase": {
"class" : "12"
}
}
}
},
"type": "info"
}
}
}
},
"weight": 2
}
],
"query": {
"match_all": {}
},
"min_score": 4,
"score_mode": "sum"
}
}
Expected Output body:
{
"query": {
"function_score": {
"functions": [
{
"filter": {
"bool": {
"must_not": {
"has_child": {
"query": {
"bool": {
"must": {
"match_phrase": {
"name" : "raju"
}
}
}
},
"type": "student"
}
}
}
},
"weight": 2
},
{
"filter": {
"bool": {
"must": {
"has_child": {
"query": {
"bool": {
"must": {
"match_phrase": {
"class" : "12"
}
}
}
},
"type": "info"
}
}
}
},
"weight": 2
}
],
"query": {
"match_all": {}
},
"min_score": 4,
"score_mode": "sum"
}
}
}
You should wrap this in a elasticbuilder.requestBodySearch()
In your case
elasticbuilder.requestBodySearch().query(not_body)
should do the job
i am just using ES to filter out some data, not need score function at all. but there are some or logic i have to include. i know we can replace must with filter, so ES will not score it and may cache it. How to profile should cause?
it this two query have different?
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"term": {
"closeStatus": 0
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "closeStatus"
------------------------------------------------
{
"query": {
"bool": {
"should": [
{
"term": {
"closeStatus": 0
}
},
{
"bool": {
"must_not": {
"exists": {
"field": "closeStatus"
If you don't care about the score, use constant_score query:
{
"query": {
"constant_score": {
"filter": {
"bool": {
"should": [
{
"term": {
"closeStatus": 0
}
},
{
"bool": {
"must_not": {
"exists": {"field": "closeStatus"}
}
}
}
]
}
}
}
}
}
I'm having trouble querying in elastic search. I'm searching over a specific set of data defined by the state_id, and then wanting to return all the states which do not have either one of the cities defined by the identifiers below.
The query below returns 18 results with just "city_id_1", and 0 results with "city_id_2". With both though, I return 0 results (since "city_id_2" is on every state record). What I want to do is still return the 18 results, but query over both cities.
I feel like my query should be working, and basically doing a NOT (A or B) style query, equivalent to NOT A and NOT B, but basically the 0 results seems to be overriding the 18.
Is there a way I can change my query to get the results I want, or is this something elasticsearch cannot do?
{
"query": {
"bool": {
"must": [
{ "terms": { "state_id": ["4ca16f80-da79-11e5-9874-64006a4f57cb"]}}
],
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"bool": {
"should": [
{"term": { "cities.identifier": "city_id_1"}},
{"term": { "cities.identifier": "city_id_2"}}
]
}
}
}
}
]
}
},
"size": 10
}
Try this on for size. Elasticsearch is silly. The filter needs to be in each of the nested queries.
{
"query": {
"bool": {
"should": [
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"term": { "cities.identifier": "city_id_1"}
}
}
}
],
"filter":[
{
"term":{
"state_id":"4ca16f80-da79-11e5-9874-64006a4f57cb"
}
}
]
}
}
},
{
"query": {
"bool": {
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"term": { "cities.identifier": "city_id_2"}
}
}
}
],
"filter":[
{
"term":{
"state_id":"4ca16f80-da79-11e5-9874-64006a4f57cb"
}
}
]
}
}
}
]
}
},
"size": 10
}
If you want NOT A AND NOT B behaviour you need to make a little change
{
"query": {
"bool": {
"must": [
{ "terms": { "state_id": ["4ca16f80-da79-11e5-9874-64006a4f57cb"]}}
],
"must_not": [
{
"nested": {
"path": "cities",
"query": {
"bool": {
"must": [ ====> Use must instead of should
{"term": { "cities.identifier": "city_id_1"}},
{"term": { "cities.identifier": "city_id_2"}}
]
}
}
}
}
]
}
},
"size": 10
}
This will exclude those record which will have both city_id_1 and city_id_2.
As per my understanding, you are looking our for NOT A or NOT B kind of a clause. Please check the query below and see if it fits your requirement
{
"query": {
"bool": {
"must": [
{ "terms": { "state_id": ["4ca16f80-da79-11e5-9874-64006a4f57cb"]}}
],
"should": [
{
"nested": {
"path": "cities",
"query": {
"bool": {
"must_not": [
{"term": { "cities.identifier": "city_id_1"}}
]
}
}
}
},
{
"nested": {
"path": "cities",
"query": {
"bool": {
"must_not": [
{"term": { "cities.identifier": "city_id_2"}}
]
}
}
}
}
],
"minimum_number_should_match": 1
}
},
"size": 10
}
I'm running Elasticsearch 1.5.2 and trying the following query:
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"gender": "male"
}
}
]
}
},
"query": {
"bool": {
"must": [
{
"match_all": {}
}
],
"should": [
{
"term": {
"top_users": 1,
"boost": 2
}
}
]
}
}
}
}
}
Everything is fine until I add the "boost": 2 to the should -> term part. The complete query is much more complex, that's why I need to boost, but the remaining queries don't make any difference: ES returns an error 400 if a term query gets a boost argument:
QueryParsingException[[index_name] [_na] query malformed, must start with start_object]
Any suggestions?
It should be like this:
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"gender": "male"
}
}
]
}
},
"query": {
"bool": {
"must": [
{
"match_all": {}
}
],
"should": [
{
"term": {
"top_users": {
"value": "1",
"boost": 2
}
}
}
]
}
}
}
}
}