Elasticsearch: remove/update field inside nested object - elasticsearch

Question

{
"_index" : "test",
"_type" : "test",
"_id" : "1212",
"_version" : 5,
"found" : true,
"_source" : {
"count" : 42,
"list_data" : [ {
"list_id" : 11,
"timestamp" : 1469125397
}, {
"list_id" : 122,
"timestamp" : 1469050965
} ]
}
}
This is my document schema.list_data is nested object. I have requirement to update/delete particular filed inside list_data. I am able to update count field using groovy script.
$ curl -XPOST 'localhost:9200/index/type/1212/_update?pretty' -d '
{
"script" : "ctx._source.count = 41"
}'
But don't know how to update nested object.
For example i want to add this into list_data.
{
"list_id" : 121,
"timestamp" : 1469050965
}
and my document should change to:
{
"_index" : "test",
"_type" : "test",
"_id" : "1212",
"_version" : 6,
"found" : true,
"_source" : {
"count" : 41,
"list_data" : [ {
"list_id" : 11,
"timestamp" : 1469125397
}, {
"list_id" : 121,
"timestamp" : 1469050965
}, {
"list_id" : 122,
"timestamp" : 1469050965
} ]
}
}
and if i perform delete based on list_id = 122 my record should look like
{
"_index" : "test",
"_type" : "test",
"_id" : "1212",
"_version" : 7,
"found" : true,
"_source" : {
"count" : 41,
"list_data" : [ {
"list_id" : 11,
"timestamp" : 1469125397
}, {
"list_id" : 121,
"timestamp" : 1469050965
}]
}
}

Answer 1

To add a new element to your nested field you can proceed like this:
$ curl -XPOST 'localhost:9200/index/type/1212/_update?pretty' -d '
{
"script" : "ctx._source.list_data += newElement",
"params": {
"newElement": {
"list_id" : 121,
"timestamp" : 1469050965
}
}
}'
To remove an existing element from your nested field list, you can proceed like this:
$ curl -XPOST 'localhost:9200/index/type/1212/_update?pretty' -d '
{
"script" : "ctx._source.list_data.removeAll{it.list_id == remove_id}",
"params": {
"remove_id" : 122
}
}'

Answer 2

I was getting the error [UpdateRequest] unknown field [params] as I was using the latest version of ElasticSearch 7.9.0 (When wrote this answer 7.9.0 was the latest), seems like the syntax is changed a bit.
Following should work for newer versions of ElasticSearch:
$ curl -XPOST 'localhost:9200/<index-name>/_update/1212'
{
"script": {
"source": "ctx._source.list_data.removeIf(list_item -> list_item.list_id == params.remove_id);",
"params": {
"remove_id": 122
}
}
}

Answer 3

I don't know why, but I find that
ctx._source.list_data.removeAll{it.list_id == remove_id}
can't work. Instead I use removeIf like this:
ctx._source.list_data.removeIf{list_item -> list_item.list_id == remove_id}
where list_item could be arbitrary string.

Answer 4

What worked for me was the instructions in the following link. Perhaps it is the version of ES.