What is wrong with the following elasticsearch query? - elasticsearch

I want to search fulltext on title and body and filter on answer count.
I read the elasticsearch documentation for combining filters and built this query.
"query": {
"bool": {
"minimum_should_match": "25%",
"should": [
{
"query_string": {
"query": "elasticsearch",
"analyze_wildcard": "True",
"fields": [
"body"
]
}
},
{
"query_string": {
"query": "test",
"analyze_wildcard": "True",
"fields": [
"title"
]
}
}
]
},
"constant_score": {
"filter": {
"query": {
"bool": {
"must": [
{
"range": {
"answer_count": {
"gte": 0,
"lte": 0
}
}
}
]
}
}
}
}
}
}
It keeps giving me this error.
RequestError: TransportError(400, u'parsing_exception', u'no [query] registered for [filter]')

You need to write it like this:
{
"query": {
"bool": {
"minimum_should_match": "25%",
"should": [
{
"query_string": {
"query": "elasticsearch",
"analyze_wildcard": "True",
"fields": [
"body"
]
}
},
{
"query_string": {
"query": "test",
"analyze_wildcard": "True",
"fields": [
"title"
]
}
}
],
"filter": [
{
"range": {
"answer_count": {
"gte": 0,
"lte": 0
}
}
}
]
}
}
}

Related

How to query elasticsearch for a specific kibana visualization data?

In our cluster's kibana dashboard, I see a visualization which gives me the total count of incoming traffic to an application. What I want is to get the same incoming traffic count using a curl call so that I can automate some reporting. To do so, first I inspect the visualization and click on request, below is what I got
{
"aggs": {},
"size": 0,
"_source": {
"excludes": []
},
"stored_fields": [
"*"
],
"script_fields": {},
"docvalue_fields": [
{
"field": "#timestamp",
"format": "date_time"
},
{
"field": "time",
"format": "date_time"
}
],
"query": {
"bool": {
"must": [],
"filter": [
{
"bool": {
"filter": [
{
"bool": {
"must_not": {
"bool": {
"should": [
{
"query_string": {
"fields": [
"remote_addr"
],
"query": "\\1\\0\\.\\0\\.*"
}
}
],
"minimum_should_match": 1
}
}
}
},
{
"bool": {
"filter": [
{
"bool": {
"must_not": {
"bool": {
"should": [
{
"query_string": {
"fields": [
"remote_addr"
],
"query": "\\1\\0\\0\\.\\0\\.*"
}
}
],
"minimum_should_match": 1
}
}
}
},
{
"bool": {
"filter": [
{
"bool": {
"must_not": {
"bool": {
"should": [
{
"match_phrase": {
"upstream_addr.keyword": “IP_ADDR:PORT”
}
}
],
"minimum_should_match": 1
}
}
}
},
{
"bool": {
"filter": [
{
"bool": {
"must_not": {
"bool": {
"should": [
{
"match_phrase": {
"upstream_addr.keyword": “IP_ADDR:PORT”
}
}
],
"minimum_should_match": 1
}
}
}
},
{
"bool": {
"filter": [
{
"bool": {
"must_not": {
"bool": {
"should": [
{
"match_phrase": {
"upstream_addr.keyword": “IP_ADDR:PORT”
}
}
],
"minimum_should_match": 1
}
}
}
},
{
"bool": {
"must_not": {
"bool": {
"should": [
{
"match_phrase": {
"upstream_addr.keyword": “IP_ADDR:PORT”
}
}
],
"minimum_should_match": 1
}
}
}
}
]
}
}
]
}
}
]
}
}
]
}
}
]
}
},
{
"match_all": {}
},
{
"match_phrase": {
"kubernetes.labels.app.keyword": {
"query": "kong"
}
}
},
{
"exists": {
"field": "status"
}
},
{
"range": {
"#timestamp": {
"format": "strict_date_optional_time",
"gte": "2021-01-05T09:32:46.946Z",
"lte": "2021-01-05T09:47:46.946Z"
}
}
}
],
"should": [],
"must_not": [
{
"bool": {
"should": [
{
"match_phrase": {
"http_user_agent": "CloudWatchSynthetics"
}
},
{
"match_phrase": {
"http_user_agent": "Amazon-Route53-Health-Check-Service"
}
}
],
"minimum_should_match": 1
}
}
]
}
}
}
Now, I took this request body, and made a curl call to elasticsearch like below
curl -u elastic:password -x GET "localhost:9200/_mget?pretty" -H 'Content-Type: application/json' -d'
<request_body_that_I_have_pasted_above>
'
But, this throws below error
{
"error" : {
"root_cause" : [
{
"type" : "parsing_exception",
"reason" : "unexpected token [START_OBJECT], expected [FIELD_NAME] or [START_ARRAY]",
"line" : 3,
"col" : 11
}
],
"type" : "parsing_exception",
"reason" : "unexpected token [START_OBJECT], expected [FIELD_NAME] or [START_ARRAY]",
"line" : 3,
"col" : 11
},
"status" : 400
}
Is my approach right? what am I doing wrong here?

Weighted search on one field and a normal search on other field

I am trying to perform a search by matching the search query to either the tag or the name of the doc, I also have a filter on the top, so I do have to use must.
Here is what I have been trying,
{
"query": {
"bool": {
"filter": {
"term": {
"type.primary": "audio"
}
},
"must": [
{
"nested": {
"path": "tags",
"score_mode": "sum",
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{
"match": {
"tags.tag": "big"
}
}
]
}
},
"field_value_factor": {
"field": "tags.weight"
},
"boost_mode": "multiply",
"boost": 10
}
}
}
},
{
"bool": {
"must": [
{
"multi_match": {
"query": "big",
"fields": [
"name"
],
"type": "phrase_prefix"
}
}
]
}
}
]
}
}
}
This just results in empty.
If I use should instead of must the query works fine, but it gives me all results with the filter of type.primary: audio.
I am pretty sure there is some other way to search for the name field. Thanks.
You're almost there! In your must, you declare that both tags and name has to hit. Try the following:
GET /_search
{
"query": {
"bool": {
"filter": {
"term": {
"type.primary": "audio"
}
},
"must": [
{
"bool": {
"should": [
{
"nested": {
"path": "tags",
"score_mode": "sum",
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{
"match": {
"tags.tag": "big"
}
}
]
}
},
"field_value_factor": {
"field": "tags.weight"
},
"boost_mode": "multiply",
"boost": 10
}
}
}
},
{
"multi_match": {
"query": "big",
"fields": [
"name"
],
"type": "phrase_prefix"
}
}
]
}
}
]
}
}
}

Elasticsearch - boosting specific documents in every search

I'm very new to Elasticsearch. I'm using it to filtering and also boosting some fields at query time. This is the code part for boosting and filtering:
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"multi_match": {
"type": "best_fields",
"query": "exampleKeyword",
"fields": [
"exampleField1^0",
"exampleField2^50",
"exampleField3^10",
"exampleField4^10",
"exampleField5^5"
],
"boost": 50
}
}]
}
}
],
"filter": [
{
"bool": {
"must": [
{
"bool": {
"must": [
{
"term": {
"bla": {
"value": ""
}
}
}
]
}
}, {
"term": {
"active": {
"value": "true"
}
}
},
{
"range": {
"closingDate": {
"gte": "201710310000",
"lte": "999912312359"
}
}
},
Now I want to boost some specific documents. I'll give an array of integers for example Field6 and if my search results contain the elements of the array, these documents should get boosted with, I dont know, 100 to my scale.
How can I do this? Finally I dont want to expand the result set. Just want to boost more the desired ids if results contain these ids.
Using function_score you can do something around these lines:
{
"query": {
"bool": {
"must": [
{
"function_score": {
"query": {
"bool": {
"should": [
{
"multi_match": {
"type": "best_fields",
"query": "bla",
"fields": [
"exampleField1^0",
"exampleField2^50",
"exampleField3^10",
"exampleField4^10",
"exampleField5^5"
],
"boost": 50
}
}
]
}
},
"functions": [
{
"filter": {
"ids": {
"values": [
1,
5
]
}
},
"weight": 10
}
],
"score_mode": "max",
"boost_mode": "multiply"
}
}
],
"filter": [
{
"bool": {
"must": [
{
"bool": {
"must": [
{
"term": {
"bla": {
"value": ""
}
}
}
]
}
},
{
"term": {
"active": {
"value": "true"
}
}
},
{
"range": {
"closingDate": {
"gte": "201710310000",
"lte": "999912312359"
}
}
}
]
}
}
]
}
}
}

Elasticsearch - Filtered query with weighted types

I have inherited an Elasticsearch query that I am trying to modify. The query I have at the moment is:
{
"fields": [
],
"from": 0,
"size": 51,
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [
"data.*"
],
"default_operator": "AND",
"query": "*Search term*"
}
},
"filter": [
{
"terms": {
"type": [
"typeOne",
"typeTwo",
"typeThree"
]
}
}
]
}
}
}
Now what I have been trying to do is boost one of these terms over the other 2 in the results but have not been able to get it to work. I have tried adding a "boost" value but this has oddly given me the opposite effect - it disables any type that is given a boost.
I tried the following as the "filter" object:
"filter": [
{
"bool": {
"should": [
{
"term": {
"type": "typeOne"
}
},
{
"term": {
"type": "typeTwo"
}
},
{
"term": {
"type": "typeThree",
"boost": 2
}
}
]
}
}
]
But as I said before, instead of boosting "typeThree" it removes all "typeThree" from the results.
Can anyone help me boost a specific term type?
There are multiple ways to structure the query to achieve the above , one approach would be using function_score .It would look something on these lines
Example:
"query": {
"function_score": {
"functions": [
{
"filter": {
"term": {
"type": "typeThree"
}
},
"weight": 2
}
],
"score_mode": "sum",
"boost_mode": "sum",
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [
"data.*"
],
"default_operator": "AND",
"query": "*search term*"
}
},
"filter": [
{
"terms": {
"type": [
"typeOne",
"typeTwo",
"typeThree"
]
}
}
]
}
}
}
}
You can enable explain to see how this affects the scoring
While keety's answer was 98% of the way there, it took a bit of extra googling to get it all together. The problem is that "weight" doesn't work here, instead you must use "boost_factor". The final query looks like this:
{
"fields": [
],
"from": 0,
"size": 51,
"query": {
"function_score": {
"functions": [
{
"filter": {
"term": {
"type": "typeOne"
}
},
"boost_factor": 1.2
},
{
"filter": {
"term": {
"type": "typeTwo"
}
},
"boost_factor": 1.1
},
{
"filter": {
"term": {
"type": "typeThree"
}
},
"boost_factor": 1
}
],
"score_mode": "sum",
"boost_mode": "sum",
"query": {
"filtered": {
"query": {
"query_string": {
"fields": [
"data.*"
],
"default_operator": "AND",
"query": "*search term*"
}
},
"filter": [
{
"terms": {
"type": [
"typeOne",
"typeTwo",
"typeThree"
]
}
}
]
}
}
}
}
}

random_score does not work properly with "should"

In the following code I always get "Alexander McQueen" products coming first, no matter what I set the seed to.
How can I change my search query to properly shuffle results?
{
"query": {
"function_score": {
"random_score": {
"seed": 99287
},
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"query_string": {
"query": "(adidas originals)",
"default_operator": "AND",
"fields": [
"name^4",
"description"
]
}
},
{
"terms": {
"category": [
"Fashion",
"Sports",
"Other",
""
]
}
},
{
"term": {
"currency": {
"term": "USD"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"query_string": {
"query": "(alexander mcqueen)",
"default_operator": "AND",
"fields": [
"name^4",
"description"
]
}
},
{
"terms": {
"category": [
"Fashion"
]
}
},
{
"term": {
"currency": {
"term": "USD"
}
}
}
]
}
}
]
}
}
}
},
"size": 40,
"from": 0
}
That's because the random score is being multiplied by the _score from the original query. If you want the results to be purely based on the random score, then set the boost_mode to replace (instead of the default multiply).
See the function_score documentation.

Resources