I have an existing elastic bool query. I've added a dense vector field to the index and would like to search it all in one query. The compound query part of the Elastic docs seems to imply you can do this, but I can't make it work (I get a runtime error) and haven't been able to find any examples. Here's a simplified version of what I'm trying.
localQuery = {
'bool':
'should': [
{
"match_phrase": {
"field1": {
"query": query,
"boost": 10
}
}
},
{
"match_phrase": {
"field2": {
"query": query,
"boost": 6
}
}
},
{
"script_score": {
"query": {
"match_all": {}
},
"script": {
"source": "cosineSimilarity(params.element_desc_vector,
'description_vec') + 1.0",
"params": {"element_desc_vector": queryList}
}
}
}
]
}
I'd appreciate any suggestions, pointers to examples or even a flat "no you can't do that".
Thanks
Howard
Trying to do the same, I eventually found you could access the score from within the script. So you could add the score returned in the "should" clause to that of the cosine similarity.
Also I put the bool clause inside the script_score and not vice-versa.
local_query = {
"script_score": {
"query": {
"bool": {
"should": [
{
"match_phrase": {
"field1": {
"query": query,
"boost": 10
}
}
},
{
"match_phrase": {
"field2": {
"query": query,
"boost": 6
}
}
}
]
}
},
"script": {
"source": "(cosineSimilarity(params.element_desc_vector, 'description_vec') + 1.0) + _score",
"params": {
"element_desc_vector": queryList
}
}
}
}
Related
Let's say I have the following Documents:
[
{
"name": "Berlin",
"name_english": "Berlin"
},
{
"name": "München",
"name_english": "Munich"
}
]
Now I do query 1:
{
"query": {
"bool": {
"should": [
{
"match": {
"name": {
"query": "Munich"
}
}
},
{
"match": {
"name_english": {
"query": "Munich"
}
}
}
]
}
}
}
Then I do query 2:
{
"query": {
"bool": {
"should": [
{
"match": {
"name": {
"query": "Berlin"
}
}
},
{
"match": {
"name_english": {
"query": "Berlin"
}
}
}
]
}
}
}
Query 1 will have a lower score than query 2, because query 2 has 2 hits. My goal now is to have only 1 hit maximum of the fields to contribute to the score. Is that possible somehow? Like "If there is a hit in the first Match Query, dont do the second one".
There is no out of the box solution, but maybe it's possible using the painless script, or you another way is you handle it from your application by sending queries in if..else conditions.
I have below elasticsearch query and I want to set the priority order in my query.
irrespetive of scoure.
eg:
like if I set priority of attack_id > name > description
in the match query, then the result should come in this sorted order
attack_id, name, description
and if I set name > attack_id > description
name, attack_id, description
boosting query
function query I have tried both of these but don't get success. so I will be very grateful if someone helps me with this.
GET tridant_testing/_search
{
"query": {
"bool": {
"should": [
{
"match": {
"attack_id": "T1592"
}
},
{
"match": {
"name": "T1592"
}
},
{
"match": {
"description": "T1592"
}
}
]
}
}
}
You can use boost param in match query to boost specific query clause like below:
{
"query": {
"bool": {
"should": [
{
"match": {
"attack_id": {
"query": "T1592",
"boost": 6
}
}
},
{
"match": {
"name": {
"query": "T1592",
"boost": 3
}
}
},
{
"match": {
"description": {
"query": "T1592"
}
}
}
]
}
}
}
Note: You may need to increase or decrease boost value as per your need.
Hello elastic experts!
I am new to elasticsearch. I am trying to build a search query with multiple or matching. I am boosting the query for different matching conditions. But here I need a bit more information. I need to know which conditions contributed to the search result. Is there any way to know which match conditions were hit by the query string?
{
"query": {
"bool": {
"should": [
{
"term": {
"title.keyword": {
"value": "Ski trip",
"boost": 1
}
}
},
{
"match_phrase_prefix": {
"title": {
"query": "Ski trip",
"boost": 0.8
}
}
},
{
"match": {
"title": {
"query": "Ski trip",
"operator": "and",
"boost": 0.6
}
}
},
{
"match": {
"description": {
"query": "Ski trip",
"boost": 0.3
}
}
}
]
}
}
}
I have a scenario in Elasticsearch where my indexed docs are like this :-
{"id":1,"name":"xyz", "address": "xyz123"}
{"id":1,"name":"xyz", "address": "xyz123"}
{"id":1,"name":"xyz", "address": "xyz123", "note": "imp"}
Here the requirement stress that we have to do a term match query and then provide relevance score to them which is a straight forward thing but the additional aspect here is if any doc found in search result has note field then it should be given higher relevance. How can we achieve it with DSL query? Using exists we can check which docs contain notes but how to integrate with match query in ES query. Have tried lot of ways but none worked.
With ES 5, you could boost your exists query to give a higher score to documents with a note field. For example,
{
"query": {
"bool": {
"must": {
"match": {
"name": {
"query": "your term"
}
}
},
"should": {
"exists": {
"field": "note",
"boost": 4
}
}
}
}
}
With ES 2, you could try a boosted filtered subset
{
"query": {
"function_score": {
"query": {
"match": { "name": "your term" }
},
"functions": [
{
"filter": { "exists" : { "field" : "note" }},
"weight": 4
}
],
"score_mode": "sum"
}
}
}
I believe that you are looking for boosting query feature
https://www.elastic.co/guide/en/elasticsearch/reference/5.1/query-dsl-boosting-query.html
{
"query": {
"boosting": {
"positive": {
<put yours original query here>
},
"negative": {
"filtered": {
"filter": {
"exists": {
"field": "note"
}
}
}
},
"negative_boost": 4
}
}
}
{
"query": {
"custom_filters_score": {
"query": {
"term": {
"name": "user1234"
}
},
"filters": [
{
"filter": {
"term": {
"subject": "math"
}
},
"script": "_score + doc['subject_score'].value"
}
]
}
}
}
If script is having like above it gives Error: unresolvable property or identifier: _score
If script is like "script": "doc['subject_score'].value" It multiplies the _score in similar way boost does. I want to replace the elasticsearch _score with custom score.
If I understood you correctly you would like to use elasticsearch scoring if subject is not math and you would like to use custom scoring with subject is math. If you are using Elasticsearch v0.90.4 or higher, it can be achieved using new function_score query:
{
"query": {
"function_score": {
"query": {
"term": {
"name": "user1234"
}
},
"functions": [{
"filter": {
"term": {
"subject": "math"
}
},
"script_score": {
"script": "doc[\"subject_score\"].value"
}
}, {
"boost_factor": 0
}],
"score_mode": "first",
"boost_mode": "sum"
}
}
}
Prior to v0.90.4 you would have to resort to using combination of custom_score and custom_filters_score:
{
"query": {
"custom_score": {
"query": {
"custom_filters_score": {
"query": {
"term": {
"name": "user1234"
}
},
"filters": [{
"filter": {
"term": {
"subject": "math"
}
},
"script": "-1.0"
}]
}
},
"script": "_score < 0.0 ? _score * -1.0 + doc[\"subject_score\"].value : _score"
}
}
}
or as #javanna suggested, use multiple custom_score queries combined together by bool query:
{
"query": {
"bool": {
"disable_coord": true,
"should": [{
"filtered": {
"query": {
"term": {
"name": "user1234"
}
},
"filter": {
"bool": {
"must_not": [{
"term": {
"subject": "math"
}
}]
}
}
}
}, {
"filtered": {
"query": {
"custom_score": {
"query": {
"term": {
"name": "user1234"
}
},
"script": "doc['subject_score'].value"
}
},
"filter": {
"term": {
"subject": "math"
}
}
}
}]
}
}
}
Firstly I'd like to say that there are many ways of customising the scoring in elasticsearch and it seems like you may have accidentally picked the wrong one. I will just summarize two and you will see what the problem is:
Custom Filters Score
If you read the docs (carefully) on custom_filters_score then you will see that it there for performance reasons, to be able to use for scoring the the faster filter machinery of elasticsearch. (Filters are faster as scoring is not calculated when computing the hit set, and they are cached between requests.)
At the end of the docs; it mentions custom_filters_score can take a "script" parameter to use instead of a "boost" parameter per filter. Best way to think of this is to calculate a number, which will be passed up to the parent query to be combined with the other sibling queries to calculate the total score for the document.
Custom Score Query
Reading the docs this is used when you want to customise the score from the query and change it how you wish. There is a _score variable available to you to use in your "script" which is the score of the query inside the custom_score query.
Try this:
"query": {
"filtered": {
"query": {
"custom_score": {
"query": {
"match_all": {}
},
"script": "doc['subject_score'].value" //*see note below
}
},
"filter": {
"and": [
{
"term": {
"subject": "math"
}
},
{
"term": {
"name": "user1234"
}
}
]
}
}
}
*NOTE: If you wanted to you could use _score here. Also, I moved both your "term" parts to filters as any match of a term would get the same score and filters are faster.
Good luck!