I used script_score to customize the scoring:
GET /customer/_search
{
"query": {
"function_score": {
"query": {
"match": {
"name": "Mark"
}
},
"script_score": {
"script": {
"lang": "painless",
"file": "test"
}
}
}
}
}
I set "file": "test", and put test.groovy file in config/scripts directory, but I got these error:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "[script] unknown field [file], parser not found"
}
],
"type": "illegal_argument_exception",
"reason": "[script] unknown field [file], parser not found"
},
"status": 400
}
[script] unknown field [file], parser not found! Why? Should I need to install some plugins?
Elasticsearch version : 6.2.3
Plugins installed: None
JVM version : 1.8.0_181
OS version: Ubuntu Linux 4.4.0-124-generic
File scripts have been removed in ES 6.0, you should now use stored scripts instead.
You can easily migrate your Groovy script to Painless.
First, store your script:
POST _scripts/test
{
"script": {
"lang": "painless",
"source": "Math.log(_score * 2)"
}
}
Then use it in your query:
GET /customer/_search
{
"query": {
"function_score": {
"query": {
"match": {
"name": "Mark"
}
},
"script_score": {
"script": {
"id": "test"
}
}
}
}
}
Related
I'm trying to extract a term using a pattern from a text field using a painless script on elasticsearch 5.6.
GET _search
{
"size": 5,
"script_fields": {
"ipaddress": {
"script": {
"inline": "Pattern pattern=Pattern.compile(\\\"([A-Z])\\w+\\\"); Matcher matcher =pattern.matcher(doc['Message'].value); if(matcher.find()){return matcher.group();}return null;",
"lang": "painless"
}
}
},
"query": {
"bool": {
"should": [
{
"match_phrase": {
"Message": "Failed password"
}
},
{
"regexp": {
"Message": {
"value": "([A-Z])\\w+"
}
}
}
]
}
}
}
Instead I receive the error
{
"error": {
"root_cause": [
{
"type": "script_exception",
"reason": "compile error",
"script_stack": [
"... pattern=Pattern.compile(\\\"([A-Z])\\w+\\\"); Matcher ...",
" ^---- HERE"
],
"script": "Pattern pattern=Pattern.compile(\\\"([A-Z])\\w+\\\"); Matcher matcher =pattern.matcher(doc['Message'].value); if(matcher.find()){return matcher.group();}return null;",
"lang": "painless"
},
{
"type": "circuit_breaking_exception",
"reason": "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_per_minute] setting",
"bytes_wanted": 0,
"bytes_limit": 0
}
],
It seems quoting the regex pattern is the issue as shown in the error ^-------HERE. How can I quote the pattern?
I have following mappings:
"properties": {
"created": {
"type": "date"
},
"id": {
"type": "keyword"
},
"identifier": {
"type": "keyword"
},
"values": {
"properties": {
"description_created-date": {
"properties": {
"<all_channels>": {
"properties": {
"<all_locales>": {
"type": "date"
}
}
}
}
},
"footwear_size-option": {
"properties": {
"<all_channels>": {
"properties": {
"<all_locales>": {
"type": "keyword"
}
}
}
}
}
}
}
}
Now I would like to create a query based on description_created-date field and use this value in painless script by comparing to some date.
GET index/pim_catalog_product/_search
{
"query": {
"constant_score": {
"filter": {
"bool": {
"filter": [
{
"script": {
"script": {
"source": "doc['values']['description_created-date']['<all_channels>']['<all_locales>'].value == '2019-12-19'",
"lang": "painless"
}
}
}
]
}
}
}
}
}
But I get following error:
{
"shard": 0,
"index": "index",
"node": "cmh1RMS1SHO92SA3jPAkJA",
"reason": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:81)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:39)",
"doc['values']['description_created-date']['<all_channels>']['<all_locales>'].value == '2019-12-19'",
" ^---- HERE"
],
"script": "doc['values']['description_created-date']['<all_channels>']['<all_locales>'].value == '2019-12-19'",
"lang": "painless",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "No field found for [values] in mapping with types [pim_catalog_product]"
}
}
}
(I know I can't compare dates like this, but this is another problem).
Searching by values.description_created-date field works:
GET index/pim_catalog_product/_search
{
"query": {
"match": {
"values.description_created-date.<all_channels>.<all_locales>": "2019-12-19"
}
}
}
And when I get specific document, value of this field is presented like this:
"values": {
"description_created-date": {
"<all_channels>": {
"<all_locales>": "2019-12-19"
}
}
}
How can I use this field in script filter? I need this to perform something like this:
(pseudocode)
"source": "doc['values']['stocks_created-date'].value > doc['created'].value + 2 days"
I'm using elasicsearch v6.5.0, here is a docker-compose file with elasticsearch and kibana:
version: '3'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.0
environment:
- discovery.type=single-node
ports:
- 9200:9200
kibana:
image: docker.elastic.co/kibana/kibana:6.5.0
ports:
- 5601:5601
and gist with full mappings and sample data here
Thanks.
Thanks for the expanded mappings! When calling a field within a nested object, try referring to the inner field using the dot notation. Example:
"source": "doc['values.description_created.<all_channels>.<all_locales>'].value == 2019"
Also, you could reduce your compound queries to just your main constant_score compound query. Example:
GET index/_search
{
"query": {
"constant_score": {
"filter": {
"script": {
"script": {
"source": "doc['values.description_created.<all_channels>.<all_locales>'].value == 2019"
}
}
},
"boost": 1
}
}
}
NOTE: The "boost" value is optional, but it's the default if you don't provide a boost value.
I'm using the official Docker image for Elasticsearch OSS (docker.elastic.co/elasticsearch/elasticsearch-oss:6.2.4) and can't seem to get script_score working at all. It seems like scripting isn't enabled.
For example, this:
POST http://localhost:9200/address/address/_search
{
"query": {
"function_score": {
"query": {
"match": {
"fullAddress": {
"query": "13 fake",
"operator": "and"
}
}
},
"script_score": {
"lang": "expression",
"source": "doc['flatNumber'].length"
}
}
}
}
gives me this:
{
"error": {
"root_cause": [
{
"type": "parsing_exception",
"reason": "script_score query does not support [source]",
"line": 13,
"col": 15
}
],
"type": "parsing_exception",
"reason": "script_score query does not support [source]",
"line": 13,
"col": 15
},
"status": 400
}
I tried enabling it:
PUT http://localhost:9200/_cluster/settings
{
"persistent": {
"script.engine.groovy.inline.aggs": "on"
}
}
but to no avail:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "persistent setting [script.engine.groovy.inline.aggs], not recognized"
}
],
"type": "illegal_argument_exception",
"reason": "persistent setting [script.engine.groovy.inline.aggs], not recognized"
},
"status": 400
}
How do I get script_score working?
You're simply missing a script section in your script_score. Modify it like this and it will work:
"script_score": {
"script": {
"lang": "expression",
"source": "doc['flatNumber'].length"
}
}
Using ES 2.3.3
A section of my document mapping is as follows :
"rents": {
"type": "nested",
"properties": {.........e.g. "avg"}
},
"location": {
"type": "geo_point"
}
And in script, I try to find distance of this field
{
"nested": {
"path": "rents",
"query": {
"function_score": {
"query": {...some rents queries....},
"functions": [
{
"script_score": {
"params": {
"center_lat": 23.509518,
"center_long":-18.378842
},
"script": "return doc['location'].distanceInMiles(center_lat, center_long)*rents.avg;"
}
}
]
}
}
}
}
but this keeps throwing error:
"reason": {
"type": "script_exception",
"reason": "failed to run inline script [return doc['location'].distanceInMiles(center_lat, center_long)*rents.avg;] using lang [groovy]",
"caused_by": {
"type": "null_pointer_exception",
"reason": null
}
}
I am following the guide https://www.elastic.co/guide/en/elasticsearch/reference/2.3/modules-scripting.html#_document_fields then what am I doing wrong? Please note all my documents have location field, none of them are null or empty. It seems doc is not available in my above script.
Please note even if I remove rents.avg from the script error remains there. So my suspicion is doc is not available inside path: rents ? If so then how to get it work?
I have inserted a new field wiki_collection and set its value to true in my document by using:
POST /pibtest1/_update_by_query
{
"script": {
"inline": "ctx._source.wiki_collection=true"
},
"query": {
"match": {
"url":"http%3A%2F%2Fwww.searchtechnologies.com%2Fbundles%2Fmodernizr%3Fv%3D7-yR01kdRVQ7W1RQzMBVKYLDhCt0itEATWHixfzE8Os1"
}
}
}
But now I want to delete this field. I am trying to do this:
POST /pibtest1/_update
{
"script" : "ctx._source.remove(\"wiki_collection\")"
}
But I am getting an error which says:
{
"error": {
"root_cause": [
{
"type": "invalid_type_name_exception",
"reason": "Document mapping type name can't start with '_'"
}
],
"type": "invalid_type_name_exception",
"reason": "Document mapping type name can't start with '_'"
},
"status": 400
}
Is there any other way to delete a field in elasticsearch?
Edit: I have updated by query:
POST /pibtest1/_update_by_query
{
"script": {
"inline": "ctx._source.remove(\"wiki_collection\")"
},
"query": {
"match": {
"url": "http://www.searchtechnologies.com/bundles/modernizr?v=7-yR01kdRVQ7W1RQzMBVKYLDhCt0itEATWHixfzE8Os1"
}
}
}
But now I am getting a new filed called "remove='wiki_collection'" in my documents which looks like this: http://i.stack.imgur.com/QvxIa.png
I want to remove/delete this wiki_collection field from my documents.
Your update is wrong. It should either be (you specify the complete path - index/type/ID):
POST /pibtest1/test/234/_update
{
"script": {
"inline": "ctx._source.remove(\"wiki_collection\")"
}
}
Or you use the same _update_by_query:
POST /pibtest1/_update_by_query
{
"script": {
"inline": "ctx._source.remove(\"wiki_collection\")"
},
"query": {
"match": {
"url": "whatever"
}
}
}