Unable to do Elastic Search update by query script for a string field. This is my query
curl -X POST "http://localhost:9200/z5-purchase-orders/_update_by_query?pretty" -H 'Content-Type: application/json' -d'
{
"script": {
"source": "ctx._source.facility_name = VCU",
"lang": "painless"
},
"query": {
"bool" : {
"must" : [
{ "match" : {"facility_id" : 0} },
{ "match" : {"customer_id" : 1002} }
]
}
}
}
'
I did the same query for integer field. It worked fine, with string its causing error
{
"error" : {
"root_cause" : [
{
"type" : "script_exception",
"reason" : "compile error",
"script_stack" : [
"... ._source.facility_name = VCU",
" ^---- HERE"
],
"script" : "ctx._source.facility_name = VCU",
"lang" : "painless"
}
],
"type" : "script_exception",
"reason" : "compile error",
"script_stack" : [
"... ._source.facility_name = VCU",
" ^---- HERE"
],
"script" : "ctx._source.facility_name = VCU",
"lang" : "painless",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "Variable [VCU] is not defined."
}
},
"status" : 400
}
Please help with this string update by query. I use elastic search version 7.
To fix that error, you simply need to stringify the value VCU
"ctx._source.facility_name = 'VCU'"
If you still get an error after that, it's going to be a different one, feel free to share it.
After escaping the characters it worked fine.
"script": {
"source": "ctx._source.facility_name = \"VCU\"",
"lang": "painless"
},
Related
We have had this query for a long time
{
"size": 50,
"query": {
"match_all": {}
},
"version": false,
"seq_no_primary_term": false,
"sort": [
{
"_script": {
"script": {
"source": "(doc[params.f].size() != 0) ? (params.m['' + doc[params.f].value] ?: params.o): params.o",
"lang": "painless",
"params": {
"f": "scoreSerial",
"m": {
"0": "UNDEFINED",
"1": "FRUSTRATED",
"2": "TOLERATED",
"3": "SATISFIED"
},
"o": "ZZZZZ"
}
},
"type": "string",
"order": "asc"
}
}
]
}
This suddenly stopped working with the error
{
"error" : {
"root_cause" : [
{
"type" : "bootstrap_method_error",
"reason" : "bootstrap_method_error: CallSite bootstrap method initialization exception"
}
],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [
{
"shard" : 0,
"index" : "visit-global-standard-w2022.31",
"node" : "olbLlZh7RYuwaf5S-B9v8g",
"reason" : {
"type" : "script_exception",
"reason" : "runtime error",
"script_stack" : [
"(doc[params.f].size() != 0) ? (params.m['' + doc[params.f].value] ?: params.o): params.o",
" ^---- HERE"
],
"script" : "(doc[params.f].size() != 0) ? (params.m['' + doc[params.f].value] ?: params.o): params.o",
"lang" : "painless",
"position" : {
"offset" : 58,
"start" : 0,
"end" : 88
},
"caused_by" : {
"type" : "bootstrap_method_error",
"reason" : "bootstrap_method_error: CallSite bootstrap method initialization exception"
}
}
}
],
"caused_by" : {
"type" : "bootstrap_method_error",
"reason" : "bootstrap_method_error: CallSite bootstrap method initialization exception"
}
},
"status" : 400
}
The mapping of scoreSerial is
"scoreSerial" : {
"type" : "long"
}
It was working for more than 3 years. We are now on the 7.10 (for more than 6 months, when the above script was working) ES version. But suddenly it started to fail.
The strange thing is that it is not happening in all environments. Is it some java version related or something else? We can not also reproduce this locally.
It might have to do with type coercion, I've seen similar problems in the past (although not with painless but groovy), so simply replacing
'' + doc[params.f].value
by
doc[params.f].value.toString()
might do the trick
(i.e. explicitly getting the string version of the doc value instead of relying on type coercion). What you did should work according to the docs, though.
I am trying to add conditions if field exist, then sort according to it otherwise use another field. Since one of either will exist.
Here is my query:
GET /my_index/_search
{
"query": {
"match_all": {}
},
"sort": {
"_script": {
"type":"number",
"script": "if(doc['contextDates.event.date'].value != 0){ return doc['contextDates.event.date'].value} else { return doc['contextDates.start.date'].value}",
"order": "asc"
}
}
}
When I execute this query, I get following error:
"failed_shards" : [
{
"shard" : 0,
"index" : "my_inedx",
"node" : "UxKwS8SIR-uIbzo5_0IbcQ",
"reason" : {
"type" : "script_exception",
"reason" : "runtime error",
"script_stack" : [
"return doc['contextDates.event.date'].value} else { ",
" ^---- HERE"
],
"script" : "if(doc['contextDates.event.date'].value != 0){ return doc['contextDates.event.date'].value} else { return doc['contextDates.start.date'].value}",
"lang" : "painless",
"position" : {
"offset" : 84,
"start" : 47,
"end" : 99
},
"caused_by" : {
"type" : "wrong_method_type_exception",
"reason" : "cannot convert MethodHandle(Dates)JodaCompatibleZonedDateTime to (Object)double"
}
}
}
]
I have tried Double.parseDouble method as well but it doesn't work. This is what I have inside document for contextDates
"contextDates" : {
"event" : {
"date" : "2020-06-26T00:00:00.000Z",
"resolution" : "day",
"score" : 0,
"type" : "event"
}
}
The doc value you're getting is of type JodaCompatibleZonedDateTime which you're trying to compare to a double value, so you need to modify your script like this
if(doc['contextDates.event.date'].value.getMillis() != 0){ return doc['contextDates.event.date'].value.getMillis()} else { return doc['contextDates.start.date'].value.getMillis()}
Say I have a field, data.url. Some our logs contain this field, some do not. I want to return only results where data.url is more than, say, 50 characters long. Really I just need a list of URLs.
I'm trying:
GET _search
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"source": "doc['data.url'].value.length() > 50",
"lang": "painless"
}
}
}
}
}
}
But get mixed errors:
{
"error" : {
"root_cause" : [
{
"type" : "script_exception",
"reason" : "runtime error",
"script_stack" : [
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:90)",
"org.elasticsearch.search.lookup.LeafDocLookup.get(LeafDocLookup.java:41)",
"doc['data.url'].value.length() > 50",
" ^---- HERE"
],
"script" : "doc['data.url'].value.length() > 50",
"lang" : "painless",
"position" : {
"offset" : 4,
"start" : 0,
"end" : 35
}
},
or
"type" : "script_exception",
"reason" : "runtime error",
"script_stack" : [
"org.elasticsearch.index.fielddata.ScriptDocValues$Strings.get(ScriptDocValues.java:496)",
"org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:503)",
"doc['data.url'].value.length() > 50",
" ^---- HERE"
],
"script" : "doc['data.url'].value.length() > 50",
"lang" : "painless",
"position" : {
"offset" : 15,
"start" : 0,
"end" : 35
}
With
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "No field found for [data.url] in mapping with types []"
}
and sometimes
"caused_by" : {
"type" : "illegal_state_exception",
"reason" : "A document doesn't have a value for a field! Use doc[<field>].size()==0 to check if a document is missing a field!"
}
This field definitely exists; I can see it in the logs, search in the search field, and using term works:
GET _search
{
"query": {
"bool": {
"filter": {
"term": {
"data.url": "www.google.com"
}
}
}
}
}
What am I missing?
I'm using Elasticsearch 7.8.
Since you are using version 7.*, you need to use this below script query
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"source": "doc['data.url.keyword'].length > 50",
"lang": "painless"
}
}
}
}
}
}
If data.url field is of keyword type, then ignore the ".keyword" at the end of the field
We need to add two new fields to an existing ElasticSearch (7.9 oss) instance.
Field 1: Date Field
We want to add an optional date field. It shouldn't have a value upon creation.
How to do this with update_by_query?
Tried this:
POST orders/_update_by_query
{
"query": {
"match_all": {}
},
"script": {
"source": "ctx._source.new_d3_field",
"lang": "painless",
"type": "date",
"format": "yyyy/MM/dd HH:mm:ss"
}
}
Field 2: Boolean field with ? in name
We want to keep the ? so that it matches the other fields that we already have in ES.
Also worth noting that even removing the ? and doing the below the field doesn't appear to be a boolean.
Tried this:
POST orders/_update_by_query
{
"query": {
"match_all": {}
},
"script": {
"source": "ctx._source.new_b_field? = false",
"lang": "painless"
}
}
Which gave the error:
{
"error" : {
"root_cause" : [
{
"type" : "script_exception",
"reason" : "compile error",
"script_stack" : [
"ctx._source.new_b_field? = false",
" ^---- HERE"
],
"script" : "ctx._source.new_b_field? = false",
"lang" : "painless",
"position" : {
"offset" : 25,
"start" : 0,
"end" : 32
}
}
],
"type" : "script_exception",
"reason" : "compile error",
"script_stack" : [
"ctx._source.new_b_field? = false",
" ^---- HERE"
],
"script" : "ctx._source.new_b_field? = false",
"lang" : "painless",
"position" : {
"offset" : 25,
"start" : 0,
"end" : 32
},
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "invalid sequence of tokens near ['='].",
"caused_by" : {
"type" : "no_viable_alt_exception",
"reason" : null
}
}
},
"status" : 400
}
Also tried:
POST orders/_update_by_query?new_b_field%3F=false
Which gave:
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "request [/orders/_update_by_query] contains unrecognized parameter: [new_b_field?]"
}
],
"type" : "illegal_argument_exception",
"reason" : "request [/orders/_update_by_query] contains unrecognized parameter: [new_b_field?]"
},
"status" : 400
}
If you want to add two new fields to an existing ElasticSearch index that don't have value upon creation you should update its mapping using Put mapping API
PUT /orders/_mapping
{
"properties": {
"new_d3_field": {
"type": "date",
"format": "yyyy/MM/dd HH:mm:ss"
},
"new_b_field?": {
"type": "boolean"
}
}
}
If you still want to use _update_by_query you should set an initial value, then the field will be added.
POST orders/_update_by_query?wait_for_completion=false&conflicts=proceed
{
"query": {
"match_all": {}
},
"script": {
"source": "ctx._source.new_d3_field=params.date;ctx._source.new_b_field = params.val",
"lang": "painless",
"params": {
"date": "1980/01/01",
"val": false
}
}
}
Update By Query API is used to update documents so I guess you can't add a field to your schema without updating at list one doc. what you can do is to set a dummy doc and update only this certain doc. Something like that:
POST orders/_update_by_query
{
"query": {
"match": {
"my-field":"my-value"
}
},
"script": {
"source": "ctx._source.new_d3_field=params.date;ctx._source.new_b_field = params.val",
"lang": "painless",
"params": {
"date": "1980/01/01",
"val": false
}
}
}
I'm following an Elasticsearch tutorial, but ran into a problem when trying to use parameters in a script.
Step 1: Create a new document - OK (index = website; type = blog; id = 1)
curl -XPUT localhost:9200/website/blog/1?pretty -d '{
"title":"my first post",
"tags" : [ "tag1" ]
}'
Step 2: Use script to append an extra value to tags array - ERROR
curl -XPOST localhost:9200/website/blog/1/_update?pretty -d '{
"script" : "ctx._source.tags+=new_tag",
"params" : {
"new_tag" : "tag2"
}
}'
The error message is this, mentioning "reason" : "Variable [new_tag] is not defined." But I have defined the variable new_tag as described on the tutorial page. What am I doing wrong?
"error" : {
"root_cause" : [
{
"type" : "remote_transport_exception",
"reason" : "[mrukUvA][127.0.0.1:9300][indices:data/write/update[s]]"
}
],
"type" : "illegal_argument_exception",
"reason" : "failed to execute script",
"caused_by" : {
"type" : "script_exception",
"reason" : "compile error",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "Variable [new_tag] is not defined."
},
"script_stack" : [
"ctx._source.tags+=new_tag",
" ^---- HERE"
],
"script" : "ctx._source.tags+=new_tag",
"lang" : "painless"
}
},
"status" : 400
}
Step 2 (retry) Qualifying new_tag with params - ERROR
curl -XPOST localhost:9200/website/blog/1/_update?pretty -d '{
"script" : {
"inline": "ctx._source.tags+=params.new_tag",
"params" : {
"new_tag" : "tag2"
}
}
}'
Gives error
{
"error" : {
"root_cause" : [
{
"type" : "remote_transport_exception",
"reason" : "[mrukUvA][127.0.0.1:9300][indices:data/write/update[s]]"
}
],
"type" : "illegal_argument_exception",
"reason" : "failed to execute script",
"caused_by" : {
"type" : "script_exception",
"reason" : "runtime error",
"caused_by" : {
"type" : "class_cast_exception",
"reason" : "Cannot cast java.lang.String to java.util.ArrayList"
},
"script_stack" : [
"ctx._source.tags+=params.new_tag",
" ^---- HERE"
],
"script" : "ctx._source.tags+=params.new_tag",
"lang" : "painless"
}
},
"status" : 400
}
As a sanity check to make sure the document is valid
$ curl -XGET localhost:9200/website/blog/1?pretty
{
"_index" : "website",
"_type" : "blog",
"_id" : "1",
"_version" : 27,
"found" : true,
"_source" : {
"title" : "my first post",
"tags" : [
"tag1"
]
}
}
So the document does have the valid field tag which is an array.
Your syntax is slightly off, if you have parameters you need to inline the script. Try this:
curl -XPOST localhost:9200/website/blog/1/_update?pretty -d '{
"script" : {
"inline": "ctx._source.tags.add(params.new_tag)",
"params" : {
"new_tag" : "tag2"
}
}
}'
you can use "inline" although it is deprecated. now you alse can use "source" to replace "inline" for no warnings. for example:
"script" : {
"source": "ctx._source.tags.add(params.new_tag)",
"params": {
"new_tag":"tag1"
}
}