Add date field and boolean with ? in name to existing Elasticsearch documents - elasticsearch

Question

We need to add two new fields to an existing ElasticSearch (7.9 oss) instance.
Field 1: Date Field
We want to add an optional date field. It shouldn't have a value upon creation.
How to do this with update_by_query?
Tried this:
POST orders/_update_by_query
{
"query": {
"match_all": {}
},
"script": {
"source": "ctx._source.new_d3_field",
"lang": "painless",
"type": "date",
"format": "yyyy/MM/dd HH:mm:ss"
}
}
Field 2: Boolean field with ? in name
We want to keep the ? so that it matches the other fields that we already have in ES.
Also worth noting that even removing the ? and doing the below the field doesn't appear to be a boolean.
Tried this:
POST orders/_update_by_query
{
"query": {
"match_all": {}
},
"script": {
"source": "ctx._source.new_b_field? = false",
"lang": "painless"
}
}
Which gave the error:
{
"error" : {
"root_cause" : [
{
"type" : "script_exception",
"reason" : "compile error",
"script_stack" : [
"ctx._source.new_b_field? = false",
" ^---- HERE"
],
"script" : "ctx._source.new_b_field? = false",
"lang" : "painless",
"position" : {
"offset" : 25,
"start" : 0,
"end" : 32
}
}
],
"type" : "script_exception",
"reason" : "compile error",
"script_stack" : [
"ctx._source.new_b_field? = false",
" ^---- HERE"
],
"script" : "ctx._source.new_b_field? = false",
"lang" : "painless",
"position" : {
"offset" : 25,
"start" : 0,
"end" : 32
},
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "invalid sequence of tokens near ['='].",
"caused_by" : {
"type" : "no_viable_alt_exception",
"reason" : null
}
}
},
"status" : 400
}
Also tried:
POST orders/_update_by_query?new_b_field%3F=false
Which gave:
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "request [/orders/_update_by_query] contains unrecognized parameter: [new_b_field?]"
}
],
"type" : "illegal_argument_exception",
"reason" : "request [/orders/_update_by_query] contains unrecognized parameter: [new_b_field?]"
},
"status" : 400
}

Answer 1

If you want to add two new fields to an existing ElasticSearch index that don't have value upon creation you should update its mapping using Put mapping API
PUT /orders/_mapping
{
"properties": {
"new_d3_field": {
"type": "date",
"format": "yyyy/MM/dd HH:mm:ss"
},
"new_b_field?": {
"type": "boolean"
}
}
}
If you still want to use _update_by_query you should set an initial value, then the field will be added.
POST orders/_update_by_query?wait_for_completion=false&conflicts=proceed
{
"query": {
"match_all": {}
},
"script": {
"source": "ctx._source.new_d3_field=params.date;ctx._source.new_b_field = params.val",
"lang": "painless",
"params": {
"date": "1980/01/01",
"val": false
}
}
}
Update By Query API is used to update documents so I guess you can't add a field to your schema without updating at list one doc. what you can do is to set a dummy doc and update only this certain doc. Something like that:
POST orders/_update_by_query
{
"query": {
"match": {
"my-field":"my-value"
}
},
"script": {
"source": "ctx._source.new_d3_field=params.date;ctx._source.new_b_field = params.val",
"lang": "painless",
"params": {
"date": "1980/01/01",
"val": false
}
}
}