i need help with finding and replacing a string from all the indexes and replace it with another string. I wanna change my attribut libelle when his value is 'Vol, Incendie' to 'Vol et incendie'.
here is my elasticJson example :
"devisdata": [
{
"nameWhoMadeIt": " ADMIN admin",
"Garanties": [
{
"libelle": "Vol, incendie",
"id": 2
},
{
"libelle": "bdg",
"id": 2
},
]
},
{
"nameWhoMadeIt": " ADMIN admin",
"Garanties": [
{
"libelle": "Vol, incendie",
"id": 2
},
{
"libelle": "bdg",
"id": 2
},
]
},
]
}
and here is my request :
{
"script": {
"source": "for (int i=0; i <= ctx._source.devisdata.length; i++) {
for (int j=0; j <= ctx._source.devisdata[i].Garanties.length; j++) {
if(ctx._source.devisdata[i].Garanties[j].libelle == 'Vol, Incendie') {
ctx._source.devisdata[i].Garanties[j].libelle = params.value; break
}
}
}",
"lang": "painless",
"params": {
"value": "Vol et incendie"
}
}
}
I find the answer, i just went out of the array. i only needed to do < not <=.
is there a better way to do this without looping ??
Related
I am able to categorize various error like this ---
But i want to send email to groups based on error message.
Something like ---
when error ie "key"= "Response status code does not indicate success Service Unavailable" ---send email to group 1 [user1#gmail.com,user2#gmail.com,user3#gmail.com]
when error ie "key"= "Response status code does not indicate success Gateway" ---send email to group 2 [user4#gmail.com,user5#gmail.com,user6#gmail.com]
I have done upto this much ---
"actions": {
"send_email": {
"throttle_period_in_millis": 300000,
"condition": {
"script": {
"source": " def status = false; for(int i=0; i<ctx.payload.failure_request.aggregations.categories.buckets.length;i++) {if(ctx.payload.failure_request.aggregations.categories.buckets[i].key.contains('Response status code does not indicate success')) {status = true}} return status ",
"lang": "painless"
}
},
"email": {
"profile": "standard",
"to": [
"avinash.singh1#spglobal.com"
],
"subject": "{{ctx.metadata.email_subject}}",
"body": {
"html": "Error Found: <ul> {{ctx.payload.aggregations.categories.buckets.length}}"
}
}
}
}
Even Email is going to the given email when condition is pass ie when key contains that message.
But I want to send email based on message match for specific group at one go.
can any one help me on this if we have something in painless language to write logic like case statement.
Appreciate your help in advance.
These is my advice, I hope that can help you.
solution one: match with a string
"actions": {
"email_group_one" : {
"condition": {
"script": {
"source": "def status = ctx.payload.failure_request.aggregations.categories.buckets; if (status.size() == 0) return false; return hosts.stream().anyMatch(p -> p.key == 'Response status code does not indicate success Service Unavailable');"
"lang": "painless"
}
},
"email" : {
"to" : ["user1#gmail.com","user2#gmail.com","user3#gmail.com"],
"subject" : "YOUR SUBJEC",
"body" : {
"html": "YOUR HTML CODE"
}
}
},
"email_group_two" : {
"condition": {
"script": {
"source": "def status = ctx.payload.failure_request.aggregations.categories.buckets; if (status.size() == 0) return false; return hosts.stream().anyMatch(p -> p.key == 'Response status code does not indicate success Gateway');"
"lang": "painless"
}
},
"email" : {
"to" : ["user4#gmail.com","user5#gmail.com","user5#gmail.com"],
"subject" : "YOUR SUBJECT",
"body" : {
"html": "YOUR HTML CODE"
}
}
}
}
solution two: match with multiple values like a,b,c and d
"actions": {
"email_group_one" : {
"condition": {
"script": {
"source": "def myArray= ['a', 'b', 'c', 'd'];def status = ctx.payload.failure_request.aggregations.categories.buckets; if (status.size() == 0) return false; return hosts.stream().anyMatch(p -> p.key in myArray);"
"lang": "painless"
}
},
"email" : {
"to" : ["user1#gmail.com","user2#gmail.com","user3#gmail.com"],
"subject" : "YOUR SUBJEC",
"body" : {
"html": "YOUR HTML CODE"
}
}
},
"email_group_two" : {
"condition": {
"script": {
"source": "def myArray= ['e', 'f', 'g', 'h'];def status = ctx.payload.failure_request.aggregations.categories.buckets; if (status.size() == 0) return false; return hosts.stream().anyMatch(p -> p.key in myArray);"
"lang": "painless"
}
},
"email" : {
"to" : ["user4#gmail.com","user5#gmail.com","user5#gmail.com"],
"subject" : "YOUR SUBJECT",
"body" : {
"html": "YOUR HTML CODE"
}
}
}
}
the code has not been tested, you may have syntax errors.
I am using painless script in ES and encountering a null pointer exception, here is my script and also I have added the response for better understanding, I use ES v7.3.
POST test_script/_update/1
{
"scripted_upsert": true,
"script": {
"lang": "painless",
"source": "int getBrowsersObjectIndex(def x, def y){for (int i = 0; i < x.length; i++) {if (x[i].deviceType == y.deviceType) {if (x[i].osName == y.osName) {if(x[i].browserName == y.browserName) {return i}}}}return -1} int getDevicesObjectIndex(def x, def y){for (int i = 0; i < x.length; i++) {if (x[i].deviceId == y.deviceId) {if (x[i].appName == y.appName) {if (x[i].appNameSpace == y.appNameSpace){return i}}}}return -1}if (!ctx._source.containsKey('browsers')) {ctx._source['browsers'] = []}if (!ctx._source.containsKey('devices')) {ctx._source['devices'] = []}for (int index = 0; index < params.iterate.length; index++) {if (params.iterate[index].browserObject) {int browserIndex = getBrowsersObjectIndex(ctx._source.browsers, params.iterate[index].browserObject);if (browserIndex >= 0) {ctx._source.browsers.remove(browserIndex);}ctx._source.browsers.add(params.iterate[index].browserObject);}if (params.iterate[index].deviceObject) {int deviceIndex = getDevicesObjectIndex(ctx._source.devices, params.iterate[index].deviceObject);if (deviceIndex >= 0) {ctx._source.devices.remove(deviceIndex);}ctx._source.devices.add(params.iterate[index].deviceObject);}}",
"params": {
"iterate": [
{
"deviceObject": {
"deviceId": "162c04e48832e338",
"appName": "test_app",
"appNameSpace": "com.test",
"appBuild": 55,
"appVersion": "4.6.3",
"deviceName": "OP4B80L1",
"manufacturer": "OPPO",
"model": "CPH1937",
"networkCarrier": "Jio",
"os": "Android",
"platform": "Android",
"timezone": "Asia/Kolkata",
"version": "10"
}
},
{
"browserObject": {
"deviceType": "mobile",
"osName": "Android",
"browserName": "Chrome",
"osVersion": 10,
"browserVersion": "80.0.3987.99"
}
}
]
}
},
"upsert": {}
}
Error is:
{
"error": {
"root_cause": [
{
"type": "remote_transport_exception",
"reason": "[connecto][192.168.36.235:9300][indices:data/write/update[s]]"
}
],
"type": "illegal_argument_exception",
"reason": "failed to execute script",
"caused_by": {
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"if (params.iterate[index].browserObject) {int ",
" ^---- HERE"
],
"script": "int getBrowsersObjectIndex(def x, def y){for (int i = 0; i < x.length; i++) {if (x[i].deviceType == y.deviceType) {if (x[i].osName == y.osName) {if(x[i].browserName == y.browserName) {return i}}}}return -1} int getDevicesObjectIndex(def x, def y){for (int i = 0; i < x.length; i++) {if (x[i].deviceId == y.deviceId) {if (x[i].appName == y.appName) {if (x[i].appNameSpace == y.appNameSpace){return i}}}}return -1}if (!ctx._source.containsKey('browsers')) {ctx._source['browsers'] = []}if (!ctx._source.containsKey('devices')) {ctx._source['devices'] = []}for (int index = 0; index < params.iterate.length; index++) {if (params.iterate[index].browserObject) {int browserIndex = getBrowsersObjectIndex(ctx._source.browsers, params.iterate[index].browserObject);if (browserIndex >= 0) {ctx._source.browsers.remove(browserIndex);}ctx._source.browsers.add(params.iterate[index].browserObject);}if (params.iterate[index].deviceObject) {int deviceIndex = getDevicesObjectIndex(ctx._source.devices, params.iterate[index].deviceObject);if (deviceIndex >= 0) {ctx._source.devices.remove(deviceIndex);}ctx._source.devices.add(params.iterate[index].deviceObject);}}",
"lang": "painless",
"caused_by": {
"type": "null_pointer_exception",
"reason": null
}
}
},
"status": 400
}
Seems like the error is in the if statement where I am checking if either of browserObject or deviceObject exists inside the iterate array of objects.
You need to perform an explicit null-check, i.e.
if (params.iterate[index].browserObject != null)
Besides, in Kibana Dev Tools, you can use triple quotes """ to properly format your code so it's more legible and easier to read/maintain:
POST test_script/_update/1
{
"scripted_upsert": true,
"script": {
"lang": "painless",
"source": """
int getBrowsersObjectIndex(def x, def y){
for (int i = 0; i < x.length; i++) {
if (x[i].deviceType == y.deviceType) {
if (x[i].osName == y.osName) {
if(x[i].browserName == y.browserName) {
return i
}
}
}
}
return -1
}
int getDevicesObjectIndex(def x, def y){
for (int i = 0; i < x.length; i++) {
if (x[i].deviceId == y.deviceId) {
if (x[i].appName == y.appName) {
if (x[i].appNameSpace == y.appNameSpace){
return i
}
}
}
}
return -1
}
if (!ctx._source.containsKey('browsers')) {
ctx._source['browsers'] = []
}
if (!ctx._source.containsKey('devices')) {
ctx._source['devices'] = []
}
for (int index = 0; index < params.iterate.length; index++) {
if (params.iterate[index].browserObject != null) {
int browserIndex = getBrowsersObjectIndex(ctx._source.browsers, params.iterate[index].browserObject);
if (browserIndex >= 0) {
ctx._source.browsers.remove(browserIndex);
}
ctx._source.browsers.add(params.iterate[index].browserObject);
}
if (params.iterate[index].deviceObject != null) {
int deviceIndex = getDevicesObjectIndex(ctx._source.devices, params.iterate[index].deviceObject);
if (deviceIndex >= 0) {
ctx._source.devices.remove(deviceIndex);
}
ctx._source.devices.add(params.iterate[index].deviceObject);
}
}
""",
"params": {
"iterate": [
{
"deviceObject": {
"deviceId": "162c04e48832e338",
"appName": "test_app",
"appNameSpace": "com.test",
"appBuild": 55,
"appVersion": "4.6.3",
"deviceName": "OP4B80L1",
"manufacturer": "OPPO",
"model": "CPH1937",
"networkCarrier": "Jio",
"os": "Android",
"platform": "Android",
"timezone": "Asia/Kolkata",
"version": "10"
}
},
{
"browserObject": {
"deviceType": "mobile",
"osName": "Android",
"browserName": "Chrome",
"osVersion": 10,
"browserVersion": "80.0.3987.99"
}
}
]
}
},
"upsert": {}
}
How query only those users whose Itemcount > 10 from the complex nested object(with dynamic key) from comosdb using sql api? UDF not preferred.
Something like,
Select c.username from c where c.Data[*].Order.ItemCount > 10;
{
{
"Username": "User1",
"Data": {
"RandomGUID123": {
"Order": {
"Item": "ItemName123",
"ItemCount" : "40"
},
"ShipmentNumber": "7657575"
},
"RandomGUID976": {
"Order": {
"Item": "ItemName7686"
"ItemCount" : "7"
},
"ShipmentNumber": "876876"
}
}
},
{
"Username": "User2",
"Data": {
"RandomGUID654": {
"Order": {
"Item": "ItemName654",
"ItemCount" : "9"
},
"ShipmentNumber": "7612575"
},
"RandomGUID908": {
"Order": {
"Item": "ItemName545"
"ItemCount" : "6"
},
"ShipmentNumber": "6454"
}
}
}
}
I'm not sure about how to handle unknown keys, but if you're willing to model the key as a value instead (simpler and cleaner I'd argue), you could have:
{
"Username": "User1",
"Data": [
{
"Id": "RandomGUID123",
"Order": {
"Item": "ItemName123",
"ItemCount": 40
},
"ShipmentNumber": "7657575"
},
{
"Id": "RandomGUID976",
"Order": {
"Item": "ItemName7686",
"ItemCount": 7
},
"ShipmentNumber": "876876"
}
]
}
With a query like:
SELECT DISTINCT VALUE(c.Username)
FROM c
JOIN (SELECT VALUE d from d IN c.Data where d["Order"].ItemCount > 10)
Result:
[
"User1"
]
"Order" is a reserved keyword and requires the bracket syntax to reference.
As Noah answers,model the key as a value is a way to achieve.
Additionally,there is another way to achieve without changing your schema of your document .Create UDF like this:
function getResult(data){
for(var key in data){
const itemCount = data[key].Order.ItemCount;
if (parseFloat(itemCount).toString() != "NaN" && parseFloat(itemCount) > 10 ) {
return true;
}
}
return false;
}
Then run this sql:
SELECT c.Username FROM c where udf.getResult(c.Data)
Result:
[
{
"Username": "User1"
}
]
GET _doc/1
"_source": {
"documents": [
{
"docid": "ID001",
"added_vals": [
{
"code": "123",
"label": "Abc"
},
{
"code": "113",
"label": "Xyz"
}
]
},
{
"docid": "ID002",
"added_vals": [
{
"code": "123",
"label": "Abc"
}
]
}
],
"id": "1"
}
POST /_bulk
{ "update": { "_id": "1"}}
{ "doc": { "documents": [ { "docid": "ID001", "status" : "cancelled" } ], "id": "1" }, "doc_as_upsert": true }
The problem above is when I run my bulk update script it replaces that document field, removing the added_vals list. Would I be able to achieve this using painless script? Thank you.
Using elasticsearch painless scripting
POST /_bulk
{ "update": { "_id": "1"} }
{ "scripted_upsert":true, "script" :{ "source": "if(ctx._version == null) { ctx._source = params; } else { def param = params; def src = ctx._source; for(s in src.documents) { boolean found = false; for(p in param.documents) { if (p.docid == s.docid) { found = true; if(s.added_vals != null) { p.added_vals = s.added_vals; } } } if(!found) param.documents.add(s); } ctx._source = param; }", "lang": "painless", "params" : { "documents": [ { "docid": "ID001", "status" : "cancelled" } ], "id": "1" } }, "upsert" : { } }
well, this one worked for me. I need to tweak a few more things that I require, but I will just leave it here for someone who may need it. Didnt know it was this simple. If there is any other answer that might be easier, please do submit so. Thanks.
"script" :
if(ctx._version == null)
{
ctx._source = params;
}
else
{
def param = params;
def src = ctx._source;
for(s in src.documents)
{
boolean found = false;
for(p in param.documents)
{
if (p.docid == s.docid)
{
found = true;
if(s.added_vals != null)
{
p.added_vals = s.added_vals;
}
}
}
if(!found) param.documents.add(s);
}
ctx._source = param;
}
I am not sure if I should modify the params directly so I used and pass the params to the param variable. I also used scripted_upsert: true with a ctx._version not null check.
How to update document with field mapping Object(multi=True),
when a document can have both single (dictionary) and multiple values (list of dictionaries).
Example of documents in the same index:
A single value in items:
{
"title": "Some title",
"items": {
"id": 123,
"key": "foo"
}
}
Multiple values in items:
{
"title": "Some title",
"items": [{
"id": 456,
"key": "foo"
}, {
"id": 789,
"key": "bar"
}]
}
You can try to use the following script.
I intentionally formatted inline attribute to show what's inside.
POST index_name/_update_by_query
{
"search": {
"term": {
"items.key": "foo"
}
},
"script": {
"inline": "
if (ctx._source.items instanceof List) {
for (item in ctx.source.items) {
if (item.key == params.old_value) {
item.key = params.new_value;
break;
}
}
} else {
ctx._source.items.key = params.new_value;
}
",
"params": {"old_value": "foo", "new_value": "bar"},
"lang": "painless'
}
}
And to make it work, replace inline attribute with a single line value.
"inline": "if (ctx._source.items instanceof List) {for (item in ctx.source.items) {if (item.key == params.old_value) {item.key = params.new_value;break;}}} else {ctx._source.items.key = params.new_value;}"