I am using ElasticSearch 7.2.0,
I have documents in an index with three fields - id, field1, field2
I want to query and return those documents whose field1 > 20 AND field1 > field2
Following is the data that the index has -
Following is the query that I'm trying -
GET /test/_search
{
"query": {
"function_score": {
"query": {
"constant_score": {
"filter": {
"bool": {
"must": [
{
"range" : {
"field1" : {
"gte" : 20
}
}
},
{
"script": {
"source": "doc['field1'].value > doc['field2'].value",
"params": {
}
}}
]
}
}
}
}
}
}
}
Following is the error -
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[script] query does not support [source]",
"line": 18,
"col": 29
}
],
"type": "parsing_exception",
"reason": "[script] query does not support [source]",
"line": 18,
"col": 29
},
"status": 400
}
Your query is correct! You just need to wrap the script inside another script. The first is to denote a script query, the second is to actually define the script:
{
"query": {
"function_score": {
"query": {
"constant_score": {
"filter": {
"bool": {
"must": [
{
"range": {
"field1": {
"gte": 20
}
}
},
{
"script": {
"script": {
"source": "doc['field1'].value > doc['field2'].value",
"params": {}
}
}
}
]
}
}
}
}
}
}
}
Related
I am running the following query and getting an error:
Query :
POST /sbl_nmon2019.12.02/_search?size=0
{"query":{
"bool":{
"must" : [{
"range":{"#timestamp":{"gte": "now-30m"}},
"aggs":{"max_cpu" : {"field":"cpu_consumed"}},
"match":{"Server" : "siebeldbnode01"}
}]
}
}}
Error:
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[range] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
"line": 5,
"col": 5
}
],
"type": "parsing_exception",
"reason": "[range] malformed query, expected [END_OBJECT] but found [FIELD_NAME]",
"line": 5,
"col": 5
},
"status": 400
}
The objective is to find max of a numberic field fron an index for last 30 minutes of a specific node.
SY
Your query is not properly formatted, it should look like this instead.
POST /sbl_nmon2019.12.02/_search
{
"size": 0,
"query": {
"bool": {
"filter": [
{
"range": {
"#timestamp": {
"gte": "now-30m"
}
}
},
{
"match": {
"Server": "siebeldbnode01"
}
}
]
}
},
"aggs": {
"max_cpu": {
"max": {
"field": "cpu_consumed"
}
}
}
}
MUST attribute values should be separate object.
Correct format:
POST /sbl_nmon2019.12.02/_search?size=0
{
"query": {
"bool": {
"must": [
{
"match": {
"Server": "siebeldbnode01"
}
},
{
"range": {
"#timestamp": {
"gte": "now-30m"
}
}
}
]
},
"aggs": {
"max_cpu": {
"field": "cpu_consumed"
}
}
}
}
Wrong Format:
"must" : [{
"range":{"#timestamp":{"gte": "now-30m"}},
"aggs":{"max_cpu" : {"field":"cpu_consumed"}},
"match":{"Server" : "siebeldbnode01"}
}]
I am using Elasticsearch version 6.7. I have the following mapping:
{
"customers": {
"mappings": {
"customer": {
"properties": {
"name": {
"type": "keyword"
},
"permissions": {
"type": "nested",
"properties": {
"entityId": {
"type": "keyword"
},
"entityType": {
"type": "keyword"
},
"permission": {
"type": "keyword"
},
"permissionLevel": {
"type": "keyword"
},
"userId": {
"type": "keyword"
}
}
}
}
}
}
}
}
I want to run a query to that shows all customers who have > 0 permissions. I have tried the following:
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"lang": "painless",
"source": "params._source != null && params._source.permissions != null && params._source.permissions.size() > 0"
}
}
}
}
}
}
But this returns no hits because params._source is null as Painless does not have access to the _source document according to this Stackoverflow post. How can I write a Painless script that gives me all customers who have > 0 permissions?
Solution 1: Using Script with must query
POST <your_index_name>/_search
{
"query": {
"bool": {
"must": [
{
"script": {
"script": {
"lang": "painless",
"inline": """
ArrayList st = params._source.permissions;
if(st!=null && st.size()>0)
return true;
"""
}
}
}
]
}
}
}
Solution 2: Using Exists Query on nested fields
You could simply make use of Exists query something like the below to get customers who have > 0 permissions.
Query:
POST <your_index_name>/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "permissions",
"query": {
"bool": {
"should": [
{
"exists":{
"field": "permissions.permission"
}
},
{
"exists":{
"field": "permissions.entityId"
}
},
{
"exists":{
"field": "permissions.entityType"
}
},
{
"exists":{
"field": "permissions.permissionLevel"
}
}
]
}
}
}
}]
}
}
}
Solution 3: Create definitive structure but add empty values to the fields
Another alternative would be to ensure all documents would have the fields.
Basically,
Ensure that all the documents would have the permissions nested document
However for those who would not have the permissions, just set the field permissions.permission to 0
Construct a query that could help you get such documents accordingly
Below would be a sample document for a user who doesn't have permissions:
POST mycustomers/customer/1
{
"name": "john doe",
"permissions": [
{
"entityId" : "null",
"entityType": "null",
"permissionLevel": 0,
"permission": 0
}
]
}
The query in that case would be as simple as this:
POST <your_index_name>/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "permissions",
"query": {
"range": {
"permissions.permission": {
"gte": 1
}
}
}
}
}
]
}
}
}
Hope this helps!
I'm having trouble adding a filter to my existing multimatch query which is embedded inside of a function_score.
Ideally, I'd like to filter by "term" : { "lang" : "en" }, only get back documents which are in the english language.
I've tried moving around the order, tried wrapping my query in bool, but just can't get the filter to work with the other functions I'm using.
My query code:
GET /my_index/_search/
{
"query": {
"function_score": {
"query": {
"bool": {
"filter": {
"term": {
"lang": "en"
}
},
"multi_match": {
"query": "Sample Query here",
"type": "most_fields",
"fields": [
"body",
"title",
"permalink",
"name"
]
}
}
},
"script_score": {
"script": {
"source": "_score + 10"
}
}
}
}
}
Error code:
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[bool] query does not support [multi_match]",
"line": 11,
"col": 19
}
],
"type": "parsing_exception",
"reason": "[bool] query does not support [multi_match]",
"line": 11,
"col": 19
},
"status": 400
}
I'm using the latest version of Elasticsearch (I believe 6.2)
Try wrapping your multi_match in a must clause like so
"must": {
"multi_match": ...
}
The error message is clear, bool query accepts only filter, must, should
Final Solution:
GET /my_index/_search/
{
"query": {
"function_score": {
"query": {
"bool" : {
"filter": {
"term": {
"lang": "en"
}
},
"must" : {
"multi_match" : {
"query": "Sample Query Here",
"type": "most_fields",
"fields": [ "body", "title", "permalink", "name"]
}
}
}
},
"script_score" : {
"script" : {
"source": "_score + 10"
}
}
}
}
}
I would want to do search with a filter that exclude result that not match a condition OR anothor condition:
I tried to do a should into a filter but it fails:
POST /my_index/_search
{
"query": {
"bool": {
"filter": [
{
"should": [
{
"match": {
"type1_title": "searched match"
}
},
{
"match": {
"type2_title": "searched match"
}
}
]
}
]
}
}
}
it raises that error:
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "[should] query malformed, no start_object after query name",
"line": 9,
"col": 21
}
],
"type": "parsing_exception",
"reason": "[should] query malformed, no start_object after query name",
"line": 9,
"col": 21
},
"status": 400
}
Do you know if we can do an or in a filter?
Why not simply bool/should, there's no need for filter here
POST /my_index/_search
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"match": {
"type1_title": "searched match"
}
},
{
"match": {
"type2_title": "searched match"
}
}
]
}
}
}
If you really want to keep the bool/filter/should construct then you need to do it like this:
POST /my_index/_search
{
"query": {
"bool": {
"filter": [
{
"bool": {
"should": [
{
"match": {
"type1_title": "searched match"
}
},
{
"match": {
"type2_title": "searched match"
}
}
]
}
}
]
}
}
}
I am very new to elastic search, We are migrating from Solr to elastic-search. As part of migration working converting existing Solr query to elastic-search DSL query.
Here is the DSL query I have partially completed using function score feature.
{
"query": {
"function_score": {
"query": {
"filtered": {
"match": {
"name": "barack obama"
},
"filter": {
"range": {
"relevance": {
"gte": 6
}
},
"bool": {
"must_not": [
{
"terms": {
"classIds": [
199,
220
],
"execution": "and"
}
}
],
"must": [
{
"term": {
"classIds": 10597
}
}
]
}
}
}
},
"boost_mode": "replace",
"functions": [
{
"script_score": {
"script": {
"lang": "groovy",
"file": "calculate-score",
"params": {
"relevance_boost": 1,
"class_penalize": 0.25
}
}
}
}
]
}
}
}
This query returning error while am running against elastic-search cluster. Please help me to figure out the issue.
Here calculate-score is groovy script and its working fine, I tested that with simple query.
Here is the error response:
{
"error": {
"root_cause": [
{
"type": "query_parsing_exception",
"reason": "[filtered] query does not support [match]",
"index": "nodes_5e27a7d3-b370-40bd-9e71-cf04a36297c0",
"line": 6,
"col": 11
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "nodes_5e27a7d3-b370-40bd-9e71-cf04a36297c0",
"node": "NOAwAtVwQS25egu7AIaHEg",
"reason": {
"type": "query_parsing_exception",
"reason": "[filtered] query does not support [match]",
"index": "nodes_5e27a7d3-b370-40bd-9e71-cf04a36297c0",
"line": 6,
"col": 11
}
}
]
},
"status": 400
}
Here is Solr query I am trying to convert to elastic-search:
SOLR QUERY (UNIQUE_NODE_CORE): q={!boost b="product(pow(field(relevance),1.0000),if(exists(query({!v='all_class_ids:226'})),0.25,1),if(exists(query({!v='all_class_ids:14106'})),0.25,1),if(exists(query({!v='all_class_ids:656'})),0.25,1))"}
raw_name:"barack obama"
&rows=1
&start=0
&sort=score desc,relevance desc
-&fq=class_id:"10597"
-fq=relevance:[6 TO *]
-&fq=-all_class_ids:"14127"
-&fq=-all_class_ids:"14106"
-&fq=-all_class_ids:"226"
&fl=ontology_id,url_friendly_name,name,score,raw_notable_for,property_207578
Just need help to run filtered query with function score.
Great job, you're almost there, you're just missing a query section inside your filtered query in order to wrap the match query. As well, the range filter can be inserted into the bool/must. Quite a mouthful, I know.
{
"query": {
"function_score": {
"query": {
"filtered": {
"query": {
"match": {
"name": "barack obama"
}
},
"filter": {
"bool": {
"must_not": [
{
"terms": {
"classIds": [
199,
220
],
"execution": "and"
}
}
],
"must": [
{
"range": {
"relevance": {
"gte": 6
}
}
},
{
"term": {
"classIds": 10597
}
}
]
}
}
}
},
"boost_mode": "replace",
"functions": [
{
"script_score": {
"script": {
"lang": "groovy",
"file": "calculate-score",
"params": {
"relevance_boost": 1,
"class_penalize": 0.25
}
}
}
}
]
}
}
}
Note that since ES 2.0 the filtered query is deprecated and you can rewrite it with a bool/must/filter query like this:
{
"query": {
"function_score": {
"query": {
"bool": {
"must": {
"match": {
"name": "barack obama"
}
},
"filter": [
{
"range": {
"relevance": {
"gte": 6
}
}
},
{
"term": {
"classIds": 10597
}
}
],
"must_not": [
{
"terms": {
"classIds": [
199,
220
],
"execution": "and"
}
}
]
}
},
"boost_mode": "replace",
"functions": [
{
"script_score": {
"script": {
"lang": "groovy",
"file": "calculate-score",
"params": {
"relevance_boost": 1,
"class_penalize": 0.25
}
}
}
}
]
}
}
}