Using ES 2.3.3
A section of my document mapping is as follows :
"rents": {
"type": "nested",
"properties": {.........e.g. "avg"}
},
"location": {
"type": "geo_point"
}
And in script, I try to find distance of this field
{
"nested": {
"path": "rents",
"query": {
"function_score": {
"query": {...some rents queries....},
"functions": [
{
"script_score": {
"params": {
"center_lat": 23.509518,
"center_long":-18.378842
},
"script": "return doc['location'].distanceInMiles(center_lat, center_long)*rents.avg;"
}
}
]
}
}
}
}
but this keeps throwing error:
"reason": {
"type": "script_exception",
"reason": "failed to run inline script [return doc['location'].distanceInMiles(center_lat, center_long)*rents.avg;] using lang [groovy]",
"caused_by": {
"type": "null_pointer_exception",
"reason": null
}
}
I am following the guide https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-scripting.html#_document_fields then what am I doing wrong? Please note all my documents have location field, none of them are null or empty. It seems doc is not available in my above script.
Please note even if I remove rents.avg from the script error remains there. So my suspicion is doc is not available inside path: rents ? If so then how to get it work?
Related
I have following mappings:
"properties": {
"created": {
"type": "date"
},
"id": {
"type": "keyword"
},
"identifier": {
"type": "keyword"
},
"values": {
"properties": {
"description_created-date": {
"properties": {
"<all_channels>": {
"properties": {
"<all_locales>": {
"type": "date"
}
}
}
}
},
"footwear_size-option": {
"properties": {
"<all_channels>": {
"properties": {
"<all_locales>": {
"type": "keyword"
}
}
}
}
}
}
}
}
Now I would like to create a query based on description_created-date field and use this value in painless script by comparing to some date.
GET index/pim_catalog_product/_search
{
"query": {
"constant_score": {
"filter": {
"bool": {
"filter": [
{
"script": {
"script": {
"source": "doc['values']['description_created-date']['<all_channels>']['<all_locales>'].value == '2019-12-19'",
"lang": "painless"
}
}
}
]
}
}
}
}
}
But I get following error:
{
"shard": 0,
"index": "index",
"node": "cmh1RMS1SHO92SA3jPAkJA",
"reason": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:81)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:39)",
"doc['values']['description_created-date']['<all_channels>']['<all_locales>'].value == '2019-12-19'",
" ^---- HERE"
],
"script": "doc['values']['description_created-date']['<all_channels>']['<all_locales>'].value == '2019-12-19'",
"lang": "painless",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "No field found for [values] in mapping with types [pim_catalog_product]"
}
}
}
(I know I can't compare dates like this, but this is another problem).
Searching by values.description_created-date field works:
GET index/pim_catalog_product/_search
{
"query": {
"match": {
"values.description_created-date.<all_channels>.<all_locales>": "2019-12-19"
}
}
}
And when I get specific document, value of this field is presented like this:
"values": {
"description_created-date": {
"<all_channels>": {
"<all_locales>": "2019-12-19"
}
}
}
How can I use this field in script filter? I need this to perform something like this:
(pseudocode)
"source": "doc['values']['stocks_created-date'].value > doc['created'].value + 2 days"
I'm using elasicsearch v6.5.0, here is a docker-compose file with elasticsearch and kibana:
version: '3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.0
environment:
- discovery.type=single-node
ports:
- 9200:9200
kibana:
image: docker.elastic.co/kibana/kibana:6.5.0
ports:
- 5601:5601
and gist with full mappings and sample data here
Thanks.
Thanks for the expanded mappings! When calling a field within a nested object, try referring to the inner field using the dot notation. Example:
"source": "doc['values.description_created.<all_channels>.<all_locales>'].value == 2019"
Also, you could reduce your compound queries to just your main constant_score compound query. Example:
GET index/_search
{
"query": {
"constant_score": {
"filter": {
"script": {
"script": {
"source": "doc['values.description_created.<all_channels>.<all_locales>'].value == 2019"
}
}
},
"boost": 1
}
}
}
NOTE: The "boost" value is optional, but it's the default if you don't provide a boost value.
I used script_score to customize the scoring:
GET /customer/_search
{
"query": {
"function_score": {
"query": {
"match": {
"name": "Mark"
}
},
"script_score": {
"script": {
"lang": "painless",
"file": "test"
}
}
}
}
}
I set "file": "test", and put test.groovy file in config/scripts directory, but I got these error:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "[script] unknown field [file], parser not found"
}
],
"type": "illegal_argument_exception",
"reason": "[script] unknown field [file], parser not found"
},
"status": 400
}
[script] unknown field [file], parser not found! Why? Should I need to install some plugins?
Elasticsearch version : 6.2.3
Plugins installed: None
JVM version : 1.8.0_181
OS version: Ubuntu Linux 4.4.0-124-generic
File scripts have been removed in ES 6.0, you should now use stored scripts instead.
You can easily migrate your Groovy script to Painless.
First, store your script:
POST _scripts/test
{
"script": {
"lang": "painless",
"source": "Math.log(_score * 2)"
}
}
Then use it in your query:
GET /customer/_search
{
"query": {
"function_score": {
"query": {
"match": {
"name": "Mark"
}
},
"script_score": {
"script": {
"id": "test"
}
}
}
}
}
below is my query, I want to change score calculation using function_score feature:
{
"size": 1,
"query":{
"function_score": {
"query": {
"bool": {
"must": [
{
"match": {
"messageText": "car"
}
}
]
}
},
"script_score" : {
"script" : "doc['time_views'].values[doc['time_views'].values.length-1]"
}
,
"boost_mode": "replace"
}
},
"from": 0
}
but I got this error response
{
"error": {
"root_cause": [
{
"type": "script_exception",
"reason": "failed to run inline script [doc['time_views'].values[doc['time_views'].values.length-1]] using lang [groovy]"
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "datacollection",
"node": "TWeZV3R6Rq-WYQ2YIHjILQ",
"reason": {
"type": "script_exception",
"reason": "failed to run inline script [doc['time_views'].values[doc['time_views'].values.length-1]] using lang [groovy]",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "No field found for [time_views] in mapping with types [message]"
}
}
}
]
},
"status": 500
}
some solutions says using quotation in "doc['time_views']" causes the problem when query has been send from command prompt tools. I don't know why!
I don't use any command prompt tools. I create the query in java code directly
EDIT
this is my index mapping:
"mappings": {
"message": {
"properties": {
"text": {
"type": "string"
},
"time_views": {
"type": "nested",
"properties": {
"backupTimestamp": {
"type": "long"
},
"views": {
"type": "integer"
}
}
}
}
}
}
}
I want to use "views" of last item of "time_views". so I try below scripts too, but each of them throw different error:
"doc['time_views.views'].values[doc['time_views.views'].values.length-1]"
error: java.util.ArrayList cannot be cast to java.lang.Number
"doc['time_views.views'].values[doc['time_views.views'].values.size()-1]"
error: failed to run inline script [doc['time_views.views'].values[doc['time_views.views'].values.size()-1]] using lang [groovy]
"doc['time_views'].values[doc['time_views'].values.size()-1].views"
error: failed to run inline script [doc['time_views'].values[doc['time_views'].values.size()-1].views] using lang [groovy]"
I'm really new in elasticsearch and groovy language. I didn't care about that "time_views" is nested Object, also I don't know syntax of groovy exactly, after some affort I found my mistakes and the solution:
{
"size": 1,
"query":{
"function_score": {
"query": {
"bool": {
"must": [
{
"match": {
"messageText": "car"
}
}
]
}
},
"script_score" : {
"script" : "doc['time_views.views'].values.get(doc['time_views.views'].values.size()-1)"
}
,
"boost_mode": "replace"
}
},
"from": 0
}
It's work as I expected
I have inserted a new field wiki_collection and set its value to true in my document by using:
POST /pibtest1/_update_by_query
{
"script": {
"inline": "ctx._source.wiki_collection=true"
},
"query": {
"match": {
"url":"http%3A%2F%2Fwww.searchtechnologies.com%2Fbundles%2Fmodernizr%3Fv%3D7-yR01kdRVQ7W1RQzMBVKYLDhCt0itEATWHixfzE8Os1"
}
}
}
But now I want to delete this field. I am trying to do this:
POST /pibtest1/_update
{
"script" : "ctx._source.remove(\"wiki_collection\")"
}
But I am getting an error which says:
{
"error": {
"root_cause": [
{
"type": "invalid_type_name_exception",
"reason": "Document mapping type name can't start with '_'"
}
],
"type": "invalid_type_name_exception",
"reason": "Document mapping type name can't start with '_'"
},
"status": 400
}
Is there any other way to delete a field in elasticsearch?
Edit: I have updated by query:
POST /pibtest1/_update_by_query
{
"script": {
"inline": "ctx._source.remove(\"wiki_collection\")"
},
"query": {
"match": {
"url": "http://www.searchtechnologies.com/bundles/modernizr?v=7-yR01kdRVQ7W1RQzMBVKYLDhCt0itEATWHixfzE8Os1"
}
}
}
But now I am getting a new filed called "remove='wiki_collection'" in my documents which looks like this: http://i.stack.imgur.com/QvxIa.png
I want to remove/delete this wiki_collection field from my documents.
Your update is wrong. It should either be (you specify the complete path - index/type/ID):
POST /pibtest1/test/234/_update
{
"script": {
"inline": "ctx._source.remove(\"wiki_collection\")"
}
}
Or you use the same _update_by_query:
POST /pibtest1/_update_by_query
{
"script": {
"inline": "ctx._source.remove(\"wiki_collection\")"
},
"query": {
"match": {
"url": "whatever"
}
}
}
I am facing an issue while trying to execute a script within an ES JSON request
The request:
POST _search
{
"query": {
"bool": {
"must": [
{
"match_all": {}
}
]
}
},
"aggs": {
"bucket_histogram": {
"histogram": {
"field": "dayTime",
"interval": 10
},
"aggs": {
"get_average": {
"avg": {
"field": "value"
}
},
"check-threshold": {
"bucket_script": {
"buckets_path": {
"averageValue": "get_average"
},
"script": "averageValue - doc[\"thresholdValue\"].value"
}
}
}
}
}
}
But I get this error instead of returning values
{
"error": {
"root_cause": [],
"type": "reduce_search_phase_exception",
"reason": "[reduce] ",
"phase": "fetch",
"grouped": true,
"failed_shards": [],
"caused_by": {
"type": "groovy_script_execution_exception",
"reason": "failed to run inline script [averageValue - doc[\"thresholdValue\"].value] using lang [groovy]",
"caused_by": {
"type": "missing_property_exception",
"reason": "No such property: doc for class: 7dcca7d142ac809a7192625d43d95bde9883c434"
}
}
},
"status": 503
}
Yet if I remove doc[\"thresholdValue\"] and enter a number everything works fine.
You are using a bucket_script, which is a part of the pipeline aggregations released with Elasticsearch 2.0. Pipeline aggregations work against other aggregations and not documents, which is why the doc context is not supplied to the aggregation.
If you want to process aggregations against specific documents, then perhaps you want the scripted metric aggregation instead.