How to fetch field inside a nested ES document - elasticsearch

Question

I have a curl command to fetch data from nested ES documents by date.
Currently it is not working.
Refer to the following for the mapping:
{
"test" : {
"mappings" : {
"doc" : {
"properties" : {
"#timestamp" : {
"type" : "date"
},
"#version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"_APIName" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"_parameters" : {
"properties" : {
"event" : {
"properties" : {
"body_json" : {
"properties" : {
"apps" : {
"properties" : {
"bundle" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"model_name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"serial_number" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"version" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}
}
},
"_stackName" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"beat" : {
"type" : "object"
},
"category" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"log" : {
"properties" : {
"name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"log_name" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"message" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"offset" : {
"type" : "long"
},
"prospector" : {
"properties" : {
"type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"source" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"stack" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
}
}
}
}
The following is a sample document in ES.
{
"_index": "test",
"_type": "doc",
"_id": "odUvZFjNxoBJGtXhSoBA",
"_version": 1,
"_score": null,
"_source": {
"log.name": "information",
"offset": 8106321,
"prospector": {
"type": "log"
},
"#version": "1",
"beat": {},
"_stackName": "test",
"_APIName": "Information",
"category": "lambda",
"#timestamp": "2019-04-16T02:22:32.000Z",
"_parameters": {
"event": {
"body_json": {
"model_name": "model-01",
"serial_number": "1234567890",
"version": "1.2",
"apps": [
{
"name": "app1",
"version": "1.0.14"
},
{
"name": "app2",
"version": "1.0.15"
}
]
}
}
},
"stack": "test"
},
"fields": {
"#timestamp": [
"2019-04-16T02:22:32.000Z"
]
}
}
This is my curl command:
#!/bin/bash
curl -XGET "http://localhost:9200/test*/_search?pretty" -H 'Content-Type: application/json' -d' {
"query": {
"bool":{
"must":[
{
"range": {
"#timestamp": {
"gte": 1546837215000,
"lte": 1552712415000,
"format": "epoch_millis"
}
}
}
]
}
},
"aggs": {
"source_bucket": {
"nested": {
"path": "_source._parameters.event.body_json"
},
"aggs": {
"model_name": {
"terms": {
"script": {
"inline": "def model = doc['_source._parameters.event.body_json.model_name'].value;\n def serial = doc['_source._parameters.event.body_json.serial_number'].value;\nreturn \"model + serial\";",
"lang": "painless"
}
}
}
}
}
}
}'
As of now, returns this error:
{
"error" : {
"root_cause" : [
{
"type" : "script_exception",
"reason" : "compile error",
"script_stack" : [
"def model = doc[_parameters.event.body_js ...",
" ^---- HERE"
],
"script" : "def model = doc[_parameters.event.body_json.model_name.keyword].value;\n def serial = doc[_parameters.event.body_json.serial_number.keyword].value;\nreturn model + serial;",
"lang" : "painless"
}
],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [
{
"shard" : 0,
"index" : "test",
"node" : "-OHA7hfMTBGqlTNwjOOngg",
"reason" : {
"type" : "script_exception",
"reason" : "compile error",
"script_stack" : [
"def model = doc[_parameters.event.body_js ...",
" ^---- HERE"
],
"script" : "def model = doc[_parameters.event.body_json.model_name.keyword].value;\n def serial = doc[_parameters.event.body_json.serial_number.keyword].value;\nreturn model + serial;",
"lang" : "painless",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "Variable [_parameters] is not defined."
}
}
}
]
},
"status" : 500
}
How can I effectively get model_name and serial_number, concatenate them and return?

Answer 1

Ok, you don't have any nested fields in your mapping, so your query should look like this instead:
#!/bin/bash
curl -XGET "http://localhost:9200/test*/_search?pretty" -H 'Content-Type: application/json' -d'{
"query": {
"bool": {
"filter": [
{
"range": {
"#timestamp": {
"gte": 1546837215000,
"lte": 1552712415000,
"format": "epoch_millis"
}
}
},
{
"exists": {
"field": "_parameters.event.body_json"
}
}
]
}
},
"aggs": {
"model_name": {
"terms": {
"script": {
"source": "def model = doc['_parameters.event.body_json.model_name.keyword'].value;\n def serial = doc['_parameters.event.body_json.serial_number.keyword'].value;\nreturn model + serial;",
"lang": "painless"
}
}
}
}
}'