I am trying to rescore my query results using a nested query, but I get the following exception, which I believe means that rescore does not support nested queries:
nested: ElasticsearchIllegalArgumentException[rescore doesn't support [path]];
Is that so ?
The nested objects contain a key field that should be matched and a weight field that should be used as score. Here is the query:
POST myindex/_search
{
"query" : {
"match" : {
"field1" : {
"query" : "my_query_string",
"type" : "boolean"
}
}
},
"rescore" : {
"window_size" : 50,
"query": {
"nested": {
"path": "path.to.nested.object",
"score_mode" : "avg",
"query": {
"function_score": {
"query":{
"constant_score": {
"query": {
"match": {
"path.to.nested.object.key": "my_query_string"
}
}
}
},
"script_score": {
"script": "doc['path.to.nested.object.weight'].value"
}
}
}
}
}
}
}
There is a syntax error in your query With re-score you need to use rescore_query since query_rescorer is the only implementation supported at the moment.
The following should work :
POST myindex/_search
{
"query": {
"match": {
"field1": {
"query": "my_query_string",
"type": "boolean"
}
}
},
"rescore": {
"window_size": 50,
"query": {
"rescore_query": {
"nested": {
"path": "path.to.nested.object",
"score_mode": "avg",
"query": {
"function_score": {
"query": {
"constant_score": {
"query": {
"match": {
"path.to.nested.object.key": "my_query_string"
}
}
}
},
"script_score": {
"script": "doc['path.to.nested.object.weight'].value"
}
}
}
}
}
}
}
}
Related
I am currently working on a search engine and i've started to implement semantic search. I use open distro version of elastic and my mapping look like this for the moment :
{
"settings": {
"index": {
"knn": true,
"knn.space_type": "cosinesimil"
}
},
"mappings": {
"properties": {
"title": {
"type" : "text"
},
"data": {
"type" : "text"
},
"title_embeddings": {
"type": "knn_vector",
"dimension": 600
},
"data_embeddings": {
"type": "knn_vector",
"dimension": 600
}
}
}
}
for basic knn_vector search i use this :
{
"size": size,
"query": {
"script_score": {
"query": {
"match_all": { }
},
"script": {
"source": "cosineSimilarity(params.query_value, doc[params.field1]) + cosineSimilarity(params.query_value, doc[params.field2])",
"params": {
"field1": "title_embeddings",
"field2": "data_embeddings",
"query_value": query_vec
}
}
}
}
}
and i've managed to get a, kind of, hybrid search with this :
{
"size": size,
"query": {
"function_score": {
"query": {
"multi_match": {
"query": query,
"fields": ["data", "title"]
}
},
"script_score": {
"script": {
"source": "cosineSimilarity(params.query_value, doc[params.field1]) + cosineSimilarity(params.query_value, doc[params.field2])",
"params": {
"field1": "title_embeddings",
"field2": "data_embeddings",
"query_value": query_vec
}
}
}
}
}
}
The problem is that if i don't have the word in the document, then it is not returned. For example, with the first search query, when i search for trump (which is not in my dataset) i manage to get document about social network and politic. I don't have these results with the hybrid search.
I have tried this :
{
"size": size,
"query": {
"function_score": {
"query": {
"match_all": { }
},
"functions": [
{
"filter" : {
"multi_match": {
"query": query,
"fields": ["data", "title"]
}
},
"weight": 1
},
{
"script_score" : {
"script" : {
"source": "cosineSimilarity(params.query_value, doc[params.field1]) + cosineSimilarity(params.query_value, doc[params.field2])",
"params": {
"field1": "title_embeddings",
"field2": "data_embeddings",
"query_value": query_vec
}
}
},
"weight": 4
}
],
"score_mode": "sum",
"boost_mode": "sum"
}
}
}
but the multi match part give a constant score to all documents that match and i want to use the filter to rank my document like in normal full text query. Any idea to do it ? Or should i use another strategy? Thank you in advance.
After the help of Archit Saxena here is the solution of my problems :
{
"size": size,
"query": {
"function_score": {
"query": {
"bool": {
"should" : [
{
"multi_match" : {
"query": query,
"fields": ["data", "title"]
}
},
{
"match_all": { }
}
],
"minimum_should_match" : 0
}
},
"functions": [
{
"script_score" : {
"script" : {
"source": "cosineSimilarity(params.query_value, doc[params.field1]) + cosineSimilarity(params.query_value, doc[params.field2])",
"params": {
"field1": "title_embeddings",
"field2": "data_embeddings",
"query_value": query_vec
}
}
},
"weight": 20
}
],
"score_mode": "sum",
"boost_mode": "sum"
}
}
}
Our Elastic Mapping
> {"mappings": {
> "products" : {
> "properties":{
> "name " : {
> "type" : "keyword"
> },
> "resellers" : {
> "type" : "nested",
> "properties" : {
> "name" : { "type" : "text" },
> "price" : { "type" : "double" }
> }
> }
> }
> }
> }}
In this mapping each product stores the list of resellers which are selling it at specific price. We have requirment to find all distinct prices of specific reseller.
As per my understanding we need to have query DSL which should first have nested filter on given reseller and then apply count aggregation. We have formed following ES query for ES 5.6 version:-
{
"query": {
"nested": {
"path": "resellers",
"query": {
"bool": {
"filter": {
"match_phrase_prefix": {
"resellers.name": "flipkart"
}
}
}
}
}
},
"aggs": {
"narrow": {
"filter": {
"nested": {
"path": "resellers",
"query": {
"bool": {
"filter": {
"term": {
"resellers.name": "flipkart"
}
}
}
}
}
},
"aggs": {
"state": {
"nested": {
"path": "resellers"
},
"aggs": {
"count": {
"terms": {
"field": "resellers.price"
}
}
}
}
}
}
}
}
This query is generating incorrect output. Output also contain price of other resellers which are present in elastic documents ( like Amazon,Snapdeal etc). Can somebody help to correct query?
Try this query instead (i.e. filter inside the nested aggregation, not outside of it):
{
"query": {
"nested": {
"path": "resellers",
"query": {
"bool": {
"filter": {
"match_phrase_prefix": {
"resellers.name": "flipkart"
}
}
}
}
}
},
"aggs": {
"resellers": {
"nested": {
"path": "resellers"
},
"aggs": {
"narrow": {
"filter": {
"term": {
"resellers.name": "flipkart"
}
},
"aggs": {
"count": {
"terms": {
"field": "resellers.price"
}
}
}
}
}
}
}
}
Here is my mapping:
{"field_name": {
"dynamic": "strict",
"properties": {...},
"type": "nested"
}}
And I am trying to filter only documents which have at least one field_name.
I tried:
{"query": {
"bool": {
"filter": [ { "script" : {
"script" : {
"inline": "doc['field_name'].length >= 1",
"lang": "painless"
} } ]
}
} }
But elasticsearch is screaming at me about No field found for [field_name] in mapping with types [type_name].
I also tried to wrap the previous query into a nested but didn't work either:
{ "nested": {
"path": "field_name",
"query": {
"bool": {
"filter": [ {
"script": {
"script": {
"inline": "doc['field_name'].length >= 1",
"lang": "painless"
}
}
} ]
}
}
} }
This gave the same error as above.
Any ideas?
if all object has same field , you can use exist to check if object exist, then use sum to calc count,then use script score to choose the condition you want. like below code
{
"query": {
"function_score": {
"query": {
"nested": {
"path": "field_name",
"query": {
"exists": {
"field": "field_name.same_field"
}
},
"score_mode": "sum"
}
},
"functions": [
{
"script_score": {
"script": {
"source": "_score >= 1 ? 1 : 0"
}
}
}
],
"boost_mode": "replace"
}
},
"min_score": 1
}
What I ended up doing is adding a field my_array_length during construction time. Like that I can just filter by the value of this field.
Simple approach would be using exists term for each of the fields:
{
"query": {
"filtered": {
"filter": {
"bool": {
"should": [
{
"exists": {
"field": "field_name.dynamic"
}
},
{
"exists": {
"field": "field_name.properties"
}
},
{
"exists": {
"field": "field_name.type"
}
}
],
"minimum_should_match": 1
}
}
}
}
}
You define should clause with minimum_should_match and get only relevant documents.
See exists, bool-query
I have a query in Elastic search to get unique values for specific field.
How to get Unique values using where clause.
where field1:xyz, field2:yzx etc
{
"size": 20,
"aggs" : {
"pf" : {
"terms" : { "field" : "platform" }
}
}}
I think you are looking for aggregations with filters
{
"size":0, // <-- If you are using ES 2.0 or above, setting size=0 will only return aggregations and no results
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [{
"term": {
"field1": "xyz"
}
}, {
"term": {
"field2": "abc"
}
}]
}
}
}
},
"aggregations": {
"pf": {
"terms": {
"field": "platform"
}
}
}
}
I have two assembled queries that work as expected.
First one uses constant score, while matching range between two values:
GET /_search
{
"query" : {
"constant_score" : {
"filter" : {
"range" : {
"locationId" : {
"gte" : 100012138,
"lt" : 101000349
}
}
}
}
}
}
The second one searches for bool.
GET /_search
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [{
"match": {
"name": "Barcelona"
}
}]
}
}
}
}
}
Now I need to merge them and I am struggling how, because tried many combinations of putting in different scopes, but not successful.
So this query returns an error.
GET /_search
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [{
"match": {
"name": "sÃdlisko"
}
}]
}
}
},
"constant_score" : {
"filter" : {
"range" : {
"locationId" : {
"gte" : 100012138,
"lt" : 1000010349
}
}
}
}
}
}
Error:
... failed to parse search source. expected field name but got
[START_OBJECT]
You could just put constant score query inside bool must clause
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{
"match": {
"name": "sidlisko"
}
},
{
"constant_score": {
"filter": {
"range": {
"locationId": {
"gte": 100012138,
"lt": 1000010349
}
}
}
}
}
]
}
}
}
}
}
I've managed to establish this query and it appears to work.
This looks as the most optimised one.
GET /_search
{
"query": {
"filtered": {
"query": {
"bool": {
"must": [{
"match": {
"fullAddress": "sidlisko"
}
}]
}
},
"filter" : {
"range" : {
"locationId" : {
"gte": 100012138,
"lt": 1000010349
}
}
}
}
}
}