I have some data in an elasticsearch index that looks like this:
{
"field1" : {
"en-US" : {
"value1" : "example",
"value2" : "example2"
}
},
"field2" : "x"
}
I need to update the value of the nested fields by query and my update script looks like this:
{
"query": {
"bool": {
"must": [
{
"terms": {
"field2": [
"x"
]
}
}
]
}
},
"script": {
"lang": "painless",
"params": {
"value": example
},
"source": "ctx._source.field1.en-US.value1 = params.value"
}
}
This gives an error as it takes the '-' b/w en-US as the subtraction operation; I can update the whole object by making this change to the script:
"script": {
"lang": "painless",
"params": {
"value": {
"value1" : "example3"
}
},
"source": "ctx._source.field1['en-US'] = params.value"
}
This works but I have to update the whole object but not one particular key inside the object. Is there a way to achieve this?
Related
I have a bunch of objects like this one in Elastic:
{
"id" : "b397ab9c-2bab-4cce-a419-ba6b23545772",
"name": "Alice",
// other fields omitted
"foo" : [
{
"id" : "1",
"bar" : 20
},
{
"id" : "2",
"bar" : 20
}
]
},
{
"id" : "826784cb-7dcd-4d2c-b444-a466032a7b06",
"name": "Bob",
// other fields omitted
"foo" : [
{
"id" : "1",
"bar" : 15
}
]
}
I am trying to make a score based on any elements in foo of which the id is 2. So in the above objects, Alice has a score of 20 and Bob has a score of 0 (as there's no element with id 2 in his foo array. I'm however a bit stuck in how to have my script_score look for the specific element. Here is my query:
"query": {
"function_score": {
"score_mode": "max",
"functions": [
{
"filter": {
"term": {
"foo.id": {
"value": "2"
}
}
},
"script_score": {
"script": {
"source": "doc['foo'].values /* what goes here? */",
"params": {
}
}
}
}
]
}
}
When foo is not of type nested, you'll have lost the connection between id and bar due to value flattenning.
After you've reconfigured your mapping to be nested, you can do
{
"query": {
"nested": {
"path": "foo",
"query": {
"function_score": {
"score_mode": "max",
"functions": [
{
"script_score": {
"script": {
"source": "if (doc['foo.id.keyword'].value == '2') { def bar = doc['foo.bar'].value; return bar } return 0",
"params": {}
}
}
}
]
}
}
}
}
}
but without the filter because that'd have filtered out doc#2 since it does not meet that condition and you probably want to do some sort of a scripted fallback.
Do note that once you're in the nested context, you don't have access to other object groups in your foo array. In other words, when you're in id: 2, you don't see the id: 1 group. More info here.
So I got this document with a
nested array that I want to filter with this query.
I want ES to return all documents where all items have changes = 0 and that only.
If document has even a single item in the list with a change = 1, that's discarded.
Is there any way I can achieve this starting from the query I have already wrote? Or should I use a script instead?
DOCUMENTS:
{
"id": "abc",
"_source" : {
"trips" : [
{
"type" : "home",
"changes" : 0
},
{
"type" : "home",
"changes" : 1
}
]
}
},
{
"id": "def",
"_source" : {
"trips" : [
{
"type" : "home",
"changes" : 0
},
{
"type" : "home",
"changes" : 0
}
]
}
}
QUERY:
GET trips_solutions/_search
{
"query": {
"bool": {
"must": [
{
"term": {
"id": {
"value": "abc"
}
}
},
{
"nested": {
"path": "trips",
"query": {
"range": {
"trips.changes": {
"gt": -1,
"lt": 1
}
}
}
}
}
]
}
}
}
EXPECTED RESULT:
{
"id": "def",
"_source" : {
"trips" : [
{
"type" : "home",
"changes" : 0
},
{
"type" : "home",
"changes" : 0
}
]
}
}
Elasticsearch version: 7.6.2
Already read this answers but they didn't help me:
https://discuss.elastic.co/t/how-to-match-all-item-in-nested-array/163873
ElasticSearch: How to query exact nested array
First off, if you filter by id: abc, you obviously won't be able to get id: def back.
Second, due to the nature of nested fields which are treated as separate subdocuments, you cannot query for all trips that have the changes equal to 0 -- the connection between the individual trips is lost and they "don't know about each other".
What you can do is return only the trips that matched your nested query using inner_hits:
GET trips_solutions/_search
{
"_source": "false",
"query": {
"bool": {
"must": [
{
"nested": {
"inner_hits": {},
"path": "trips",
"query": {
"term": {
"trips.changes": {
"value": 0
}
}
}
}
}
]
}
}
}
The easiest solution then is to dynamically save this nested info on a parent object like discussed here and using range/term query on the resulting array.
EDIT:
Here's how you do it using copy_to onto the doc's top level:
PUT trips_solutions
{
"mappings": {
"properties": {
"trips_changes": {
"type": "integer"
},
"trips": {
"type": "nested",
"properties": {
"changes": {
"type": "integer",
"copy_to": "trips_changes"
}
}
}
}
}
}
trips_changes will be an array of numbers -- I presume they're integers but more types are available.
Then syncing a few docs:
POST trips_solutions/_doc
{"trips":[{"type":"home","changes":0},{"type":"home","changes":1}]}
POST trips_solutions/_doc
{"trips":[{"type":"home","changes":0},{"type":"home","changes":0}]}
And finally querying:
GET trips_solutions/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "trips",
"query": {
"term": {
"trips.changes": {
"value": 0
}
}
}
}
},
{
"script": {
"script": {
"source": "doc.trips_changes.stream().filter(val -> val != 0).count() == 0"
}
}
}
]
}
}
}
Note that we first filter normally using the nested term query to narrow down our search context (scripts are slow so this is useful). We then check if there are any non-zero changes in the accumulated top-level changes and reject those that apply.
I'm trying to get the difference between two fields. I'm using Elasticsearch 5.5.
I have already tried
{
"query": {
"bool": {
"filter": {
"script": {
"script": "doc['students.total_fee'].value - doc['students.paid_fee'].value"
}
}
}
}
}
but it is returning empty "hits".
I have also tried
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "students",
"query": {
"script": {
"script": {
"inline": "doc['students.total_fee'].value - doc['students.paid_fee'].value",
"lang": "expression"
}
}
}
}
}
]
}
}
and it is returning "0".
also the "script_fields" did not worked.
{ "script_fields" :
{ "difference" :
{ "script" : "doc['students.total_fee'].value - doc['students.paid_fee'].value"
} } }
suppose I have data in following format.
"_source" : {
"students" : [
{
"name" : "A",
"total_fee" : 12345,
"paid_fee" : 12344.8
},
{
"name" : "B",
"total_fee" : 23456,
"paid_fee" : 23455.6
}
]
}
Now I want to get the difference between "total_fee" and "paid_fee" for each student.
I expect to get an array of differences for all students.
Thanks in advance :D
You need to use the below query and my es version is 6.2.2. It is giving a perfect result. But remember scripting is generally CPU intensive.
If you normal field then below query working fine.
{
"size": 10,
"script_fields": {
"fare_diff": {
"script": "doc[\"students.total_fee\"].value - doc[\"students.paid_fee\"].value"
}
}
}
If you are using a nested field parameter then the query would be like below.
{
"script_fields": {
"fare_diff": {
"script": {
"lang": "painless",
"source": "int total = 0; def l = new ArrayList(); for (int i = 0; i < params['_source']['students'].size(); ++i) { l.add(params['_source']['students'][i]['total_fee'] - params['_source']['students'][i]['paid_fee']);} return l.toArray();"
}
}
}
}
Reason: Because nested documents are indexed as separate documents, they can only be accessed within the scope of the nested query, the nested/reverse_nested aggregations, or nested inner hits.
Hi Below is my document
"catid": [
514500
],
"studentid": 5282439,
In this catid field , I want to add new element 543 , where studentid = 5282439
I am trying the below query but it's giving me an exception
POST /parts/_update_by_query
{
"query": {
"match": {
"studentid": 5282439
}
},
"script" : "ctx._source.catid+= [543 ]"
}
I am getting the below exception:
"root_cause": [
{
"type": "class_cast_exception",
"reason": "java.lang.String cannot be cast to java.util.Map"
}
]
--> If the above answer doesn't work try this one
POST /parts/_update_by_query
{
"query": {
"match": {
"studentid": 5282439
}
},
"script" : {
"lang":"painless",
"inline": "ctx._source.catid.add(params.newsupp)",
"params":{
"newsupp":5302
}
}
}
The script part is not correct, change it to this instead (i.e. move the script to the script.inline property):
POST /parts/_update_by_query
{
"query": {
"match": {
"studentid": 5282439
}
},
"script" : {
"inline": "ctx._source.catid += [543 ]"
}
}
I am using update by query plugin (https://github.com/yakaz/elasticsearch-action-updatebyquery/) to update documents by query.
In my case, there is nested field in document, the mapping is something like this:
"mappings": {
"mytype": {
"properties": {
"Myfield1": {
"type": "nested",
"properties": {
"field1": {
"type": "string"
},
"field2": {
"type": "long"
}
}
},
"Title": {
"type": "string"
}
}
}
}
Then I want to update the nested field Myfield1 by query with following request:
But unfortunately, it does not work.
{
"query": {
"match": {
"Title": "elasticsearch"
}
},
"script": "ctx._source.Myfield1 = [{'nestfield1':'foo blabla...','nestfield2':100},{'nestfield1':'abc...','nestfield2':200}]"
}
Does update by query support nested object?
BTW: any other ways to update document by query?
Is the update by query plugin the only choice?
This example uses _update_by_query
POST indexname/type/_update_by_query
{
"query": {
"match": {
"Title": "elasticsearch"
}
},
"script": {
"source": "ctx._source.Myfield1= params.mifieldAsParam",
"params": {
"mifieldAsParam": [
{
"nestfield1": "foo blabla...",
"nestfield2": 100
},
{
"nestfield1": "abc...",
"nestfield2": 200
}
]
},
"lang": "painless"
}
}
Nested elements need to be iterated in painless script to update values
POST /index/_update_by_query
{
"script": {
"source": "for(int i=0;i<=ctx._source['Myfield1'].size()-1;i++){ctx._source.Myfield1[i].field1='foo blabla...';ctx._source.Myfield1[i].field2=100}",
"lang": "painless"
},
"query": {
"match": {
"Title": "elasticsearch"
}
}
}
Nested elements value update if index is known
POST /index/_update_by_query
{
"script": {
"source": "ctx._source.Myfield1[0].field1='foo blabla...';ctx._source.Myfield1[0].field2=100;ctx._source.Myfield1[1].field1='abc...';ctx._source.Myfield1[1].field2=200;",
"lang": "painless"
},
"query": {
"match": {
"Title": "elasticsearch"
}
}
}
You can try with params, something like this:
"query" : {
"match_all" : {}
},
"script" : "ctx._source.Myfield1 = Myfield1;",
"params": {
"Myfield1": {
"nestfield1": "foo blabla..."
}
}
In my case I'm moving the data from not nested fields in nested fields. I need to add fake information to initialize the nested field. It looks like that:
"query" : {
"match_all" : {}
},
"script" : "ctx._source.Myfield1 = Myfield1; ctx._source.Myfield1.nestfield1 = ctx._source.Myfield1Nestfield1; ctx._source.Myfield1.nestfield2 = ctx._source.Myfield1Nestfield2;",
"params": {
"Myfield1": {
"nestfield1": "init_data"
}
}