Is it possible to update nested field by query? - elasticsearch

I am using update by query plugin (https://github.com/yakaz/elasticsearch-action-updatebyquery/) to update documents by query.
In my case, there is nested field in document, the mapping is something like this:
"mappings": {
"mytype": {
"properties": {
"Myfield1": {
"type": "nested",
"properties": {
"field1": {
"type": "string"
},
"field2": {
"type": "long"
}
}
},
"Title": {
"type": "string"
}
}
}
}
Then I want to update the nested field Myfield1 by query with following request:
But unfortunately, it does not work.
{
"query": {
"match": {
"Title": "elasticsearch"
}
},
"script": "ctx._source.Myfield1 = [{'nestfield1':'foo blabla...','nestfield2':100},{'nestfield1':'abc...','nestfield2':200}]"
}
Does update by query support nested object?
BTW: any other ways to update document by query?
Is the update by query plugin the only choice?

This example uses _update_by_query
POST indexname/type/_update_by_query
{
"query": {
"match": {
"Title": "elasticsearch"
}
},
"script": {
"source": "ctx._source.Myfield1= params.mifieldAsParam",
"params": {
"mifieldAsParam": [
{
"nestfield1": "foo blabla...",
"nestfield2": 100
},
{
"nestfield1": "abc...",
"nestfield2": 200
}
]
},
"lang": "painless"
}
}

Nested elements need to be iterated in painless script to update values
POST /index/_update_by_query
{
"script": {
"source": "for(int i=0;i<=ctx._source['Myfield1'].size()-1;i++){ctx._source.Myfield1[i].field1='foo blabla...';ctx._source.Myfield1[i].field2=100}",
"lang": "painless"
},
"query": {
"match": {
"Title": "elasticsearch"
}
}
}
Nested elements value update if index is known
POST /index/_update_by_query
{
"script": {
"source": "ctx._source.Myfield1[0].field1='foo blabla...';ctx._source.Myfield1[0].field2=100;ctx._source.Myfield1[1].field1='abc...';ctx._source.Myfield1[1].field2=200;",
"lang": "painless"
},
"query": {
"match": {
"Title": "elasticsearch"
}
}
}

You can try with params, something like this:
"query" : {
"match_all" : {}
},
"script" : "ctx._source.Myfield1 = Myfield1;",
"params": {
"Myfield1": {
"nestfield1": "foo blabla..."
}
}
In my case I'm moving the data from not nested fields in nested fields. I need to add fake information to initialize the nested field. It looks like that:
"query" : {
"match_all" : {}
},
"script" : "ctx._source.Myfield1 = Myfield1; ctx._source.Myfield1.nestfield1 = ctx._source.Myfield1Nestfield1; ctx._source.Myfield1.nestfield2 = ctx._source.Myfield1Nestfield2;",
"params": {
"Myfield1": {
"nestfield1": "init_data"
}
}

Related

How to filter on nested document length by script in Elasticsearch

I am trying to filter documents that have at least a given amount of items in a nested field, but I keep getting the following exception:
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "No field found for [items] in mapping"
}
Here's an example code to reproduce:
PUT store
{
"mappings": {
"properties": {
"subject": {
"type": "keyword"
},
"items": {
"type": "nested",
"properties": {
"name": {
"type": "keyword"
},
"count": {
"type": "integer"
}
}
}
}
}
}
POST store/_bulk?refresh=true
{"create":{"_index":"store","_id":"1"}}
{"type":"appliance","items":[{"name":"Color TV"}]}
{"create":{"_index":"store","_id":"2"}}
{"type":"vehicle","items":[{"name":"Car"},{"name":"Bicycle"}]}
{"create":{"_index":"store","_id":"3"}}
{"type":"instrument","items":[{"name":"Guitar"},{"name":"Piano"},{"name":"Drums"}]}
GET store/_search
{
"query": {
"bool": {
"filter": [
{
"script": {
"script": {
"source": "doc['items'].size() > 1"
}
}
}
]
}
}
}
Please note that this is only a simplified filter script of what I really wanted to do, and if I can get over this, I will probable be able to solve my task as well.
Any help would be appreciated.
I ended up solving it with a custom score approach:
GET store/_search
{
"min_score": 0.1,
"query": {
"function_score": {
"query": {
"match_all": {}
},
"functions": [
{
"script_score": {
"script": {
"source": "params['_source']['items'].length > 1 ? 1 : 0"
}
}
}
]
}
}
}

Check if id exist in array field or not using script_fields

i like to get a script field if a specific value exist in a array .
Here is my mapping -
"mappings": {
"properties": {
"field1": {
"type": "text",
},
"field2": {
"type": "text",
},
}
}
Here is my example documents -
{
"field1": "hello 1",
"field2": ["id1","id3"]
}
{
"field1": "hello 2",
"field2": ["id2","id3"]
}
{
"field1": "hello 3",
}
{
"field1": "mobile 1",
"field2": ["id1","id4"]
}
Here is my search query -
{
"query": {
"match" : {
"title": "hello"
}
},
"script_fields":{
"testField2":{
"script": {
"source":"doc['field2'].value == params.id ? true : false",
"params": {
"id": "id1"
}
}
}
}
}
what i want to achieve is to get all those document which has "hello" in their title and add a custom field (testField2) in every doc . the value of that custom field would be True if "field2" contain a specific id ("id1") else False
If field2 is only mapped as text, you won't be able to use its field values (via doc['field2'].value). You can only use its _source:
POST your-index/_search
{
"query": {
"match": {
"field1": "hello"
}
},
"script_fields": {
"testField2": {
"script": {
"source": "params._source.containsKey('field2') ? params._source.field2.contains(params.id) : false",
"params": {
"id": "id1"
}
}
}
}
}
Note that accessing _source is not as efficient as using the doc values -- so in short, you were using the right strategy but with the wrong mapping.
In order to be able to access the doc values, either use the keyword data type or set the fielddata parameter on your existing text field to true. I'm going to go with the keyword here:
DELETE your-index
PUT your-index
{
"mappings": {
"properties": {
"field1": {
"type": "text"
},
"field2": {
"type": "keyword" <--
}
}
}
}
After reindexing (which is required after a "mapping breaking change") you can then call:
POST your-index/_search
{
"query": {
"match": {
"field1": "hello"
}
},
"script_fields": {
"testField2": {
"script": {
"source": "doc['field2'].size() != 0 ? doc['field2'].asList().contains(params.id) : false",
"params": {
"id": "id1"
}
}
}
}
}
Finally, notice the .asList() method. It's needed to convert the doc values which are an instance of the class ScriptDocValues.Strings to a consumable ArrayList on which you can then call the conventional .contains() method.

How do I get the size of a 'nested' type array through a Painless script in Elasticsearch version 6.7?

I am using Elasticsearch version 6.7. I have the following mapping:
{
"customers": {
"mappings": {
"customer": {
"properties": {
"name": {
"type": "keyword"
},
"permissions": {
"type": "nested",
"properties": {
"entityId": {
"type": "keyword"
},
"entityType": {
"type": "keyword"
},
"permission": {
"type": "keyword"
},
"permissionLevel": {
"type": "keyword"
},
"userId": {
"type": "keyword"
}
}
}
}
}
}
}
}
I want to run a query to that shows all customers who have > 0 permissions. I have tried the following:
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"lang": "painless",
"source": "params._source != null && params._source.permissions != null && params._source.permissions.size() > 0"
}
}
}
}
}
}
But this returns no hits because params._source is null as Painless does not have access to the _source document according to this Stackoverflow post. How can I write a Painless script that gives me all customers who have > 0 permissions?
Solution 1: Using Script with must query
POST <your_index_name>/_search
{
"query": {
"bool": {
"must": [
{
"script": {
"script": {
"lang": "painless",
"inline": """
ArrayList st = params._source.permissions;
if(st!=null && st.size()>0)
return true;
"""
}
}
}
]
}
}
}
Solution 2: Using Exists Query on nested fields
You could simply make use of Exists query something like the below to get customers who have > 0 permissions.
Query:
POST <your_index_name>/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "permissions",
"query": {
"bool": {
"should": [
{
"exists":{
"field": "permissions.permission"
}
},
{
"exists":{
"field": "permissions.entityId"
}
},
{
"exists":{
"field": "permissions.entityType"
}
},
{
"exists":{
"field": "permissions.permissionLevel"
}
}
]
}
}
}
}]
}
}
}
Solution 3: Create definitive structure but add empty values to the fields
Another alternative would be to ensure all documents would have the fields.
Basically,
Ensure that all the documents would have the permissions nested document
However for those who would not have the permissions, just set the field permissions.permission to 0
Construct a query that could help you get such documents accordingly
Below would be a sample document for a user who doesn't have permissions:
POST mycustomers/customer/1
{
"name": "john doe",
"permissions": [
{
"entityId" : "null",
"entityType": "null",
"permissionLevel": 0,
"permission": 0
}
]
}
The query in that case would be as simple as this:
POST <your_index_name>/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "permissions",
"query": {
"range": {
"permissions.permission": {
"gte": 1
}
}
}
}
}
]
}
}
}
Hope this helps!

Elasticsearch search template not working with terms query

I have an index with a long field and i am just trying to use search template to use terms query but it throws exception.
"pid": {
"type": "long"
}
Search Template:
PUT /_search/template/article_query_template
{
"template": {
"query": {
"terms": {
"pid": "{{articleId}}"
}
}
}
}
Search Query :
POST test2*/_search
{
"query": {
"template": {
"id": "article_query_template",
"params" : {
"articleId" : ["1"]
}
}
}
}
Exception : reason": "[terms] query does not support [pid]".
Its working without template. How to fix this issue.
Create your template like :
{
"source": {
"query": {
"term": {
"message": "{{query_string}}"
}
}
}
}
and pass params like :
{
"id": "<templateName>",
"params": {
"query_string": "search for these words"
}}
More details are available here.

How to delete a field in elasticsearch 2.3.0? I am using sense to do my queries

I have inserted a new field wiki_collection and set its value to true in my document by using:
POST /pibtest1/_update_by_query
{
"script": {
"inline": "ctx._source.wiki_collection=true"
},
"query": {
"match": {
"url":"http%3A%2F%2Fwww.searchtechnologies.com%2Fbundles%2Fmodernizr%3Fv%3D7-yR01kdRVQ7W1RQzMBVKYLDhCt0itEATWHixfzE8Os1"
}
}
}
But now I want to delete this field. I am trying to do this:
POST /pibtest1/_update
{
"script" : "ctx._source.remove(\"wiki_collection\")"
}
But I am getting an error which says:
{
"error": {
"root_cause": [
{
"type": "invalid_type_name_exception",
"reason": "Document mapping type name can't start with '_'"
}
],
"type": "invalid_type_name_exception",
"reason": "Document mapping type name can't start with '_'"
},
"status": 400
}
Is there any other way to delete a field in elasticsearch?
Edit: I have updated by query:
POST /pibtest1/_update_by_query
{
"script": {
"inline": "ctx._source.remove(\"wiki_collection\")"
},
"query": {
"match": {
"url": "http://www.searchtechnologies.com/bundles/modernizr?v=7-yR01kdRVQ7W1RQzMBVKYLDhCt0itEATWHixfzE8Os1"
}
}
}
But now I am getting a new filed called "remove='wiki_collection'" in my documents which looks like this: http://i.stack.imgur.com/QvxIa.png
I want to remove/delete this wiki_collection field from my documents.
Your update is wrong. It should either be (you specify the complete path - index/type/ID):
POST /pibtest1/test/234/_update
{
"script": {
"inline": "ctx._source.remove(\"wiki_collection\")"
}
}
Or you use the same _update_by_query:
POST /pibtest1/_update_by_query
{
"script": {
"inline": "ctx._source.remove(\"wiki_collection\")"
},
"query": {
"match": {
"url": "whatever"
}
}
}

Resources