How to perfrom `sum` and `avg` aggs in even if the mappings of the field is on `text` and `keyword` types? - elasticsearch

Question

I'm trying to perform a sum and avg aggregations on my Elasticsearch query, everything works perfectly fine but I've encountered a problem -- I want to perform the aforementioned aggs to my nested fields that are on text / keyword types.
The reason that they're as such, is because we'll be using the keywords analyzer when we are performing the search API if these specific nested field and subfields are required.
Here's my mapping:
"eng" : {
"type" : "nested",
"properties" : {
"date_updated" : {
"type" : "long"
},
"soc_angry_count" : {
"type" : "float"
},
"soc_comment_count" : {
"type" : "float"
},
"soc_dislike_count" : {
"type" : "float"
},
"soc_eng_score" : {
"type" : "float"
},
"soc_er_score" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"soc_haha_count" : {
"type" : "float"
},
"soc_kf_score" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"soc_like_count" : {
"type" : "float"
},
"soc_love_count" : {
"type" : "float"
},
"soc_mm_score" : {
"type" : "float"
},
"soc_sad_count" : {
"type" : "float"
},
"soc_save_count" : {
"type" : "float"
},
"soc_share_count" : {
"type" : "float"
},
"soc_te_score" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
},
"soc_view_count" : {
"type" : "float"
},
"soc_wow_count" : {
"type" : "float"
}
}
}
Please focus on the soc_er_score, soc_kf_score and soc_te_score subfields of the eng nested field...
When I'm performing the following aggs, it's working fine:
'aggs' => [
'ENGAGEMENT' => [
'nested' => [
'path' => "eng"
],
'aggs' => [
'ARTICLES' => [
//Use Histogram because the pub_date is of
//long data type
//Use interval 86400 to represent 1 day
'histogram' => [
'field' => "eng.date_updated",
"interval" => "86400",
],
'aggs'= [
'SUM' => [
'sum' => [
"field" => "eng.soc_like_score"
]
]
]
]
]
]
]
Here's the output after doing the search API
BUT if the query is like this:
'aggs' => [
'ENGAGEMENT' => [
'nested' => [
'path' => "eng"
],
'aggs' => [
'ARTICLES' => [
//Use Histogram because the pub_date is of
//long data type
//Use interval 86400 to represent 1 day
'histogram' => [
'field' => "eng.date_updated",
"interval" => "86400",
],
'aggs'= [
'SUM' => [
'sum' => [
"field" => "eng.soc_te_score"
]
]
]
]
]
]
]
The output looks like this:
SOLUTIONS PERFORMED
SOLUTION 1 (for confirmation)
After reading some thorough forum discussions, I've learned that java-based parsing is available but it seems not working on my end
Here's my revised query:
'aggs'= [
'SUM' => [
'sum' => [
"field" => "Float.parseFloat(eng.soc_te_score).value"
]
]
]
But unfortunately, it's responding with null or 0 values
By the way, I'm using Laravel as my Web Framework, that's why this is how my debugger or error message window look like
Requesting for your help please, thank you in advance!

Answer 1

I would create another numeric subfield in addition to the keyword one. So you can use the keyword field for search and the numeric one for aggregations.
For example, modify your mapping like this:
"soc_er_score" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
},
"numeric" : {
"type" : "long",
"ignore_malformed": true
}
}
},
You can then use:
soc_er_score for full text search
soc_er_score.keyword for sorting, terms aggregations and exact matching
soc_er_score.numeric for sum and other metric aggregations.
If you already have data in your index, simply modify the mapping by adding the new sub-field, like this:
PUT my-index/_mapping/doc
{
"properties": {
"eng": {
"soc_er_score" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
},
"numeric" : {
"type" : "long",
"ignore_malformed": true
}
}
}
}
}
}
And then call the update by query endpoint in order to pick up the new mapping:
POST my-index/_update_by_query
When done, the eng.soc_er_score.numeric field will be indexed for all your existing documents.

Answer 2

I was able to solve my problem with the following simple script:
"aggs" = [
'SUM' => [
'sum' => [
"script" => "Float.parseFloat(doc['eng.soc_te_score.keyword'].value)"
]
]
];
With this, even if my nested field is of text and keyword type, I can still compute for their average and sum