Is there any way to aggregate only on query hits in elasticsearch? - elasticsearch

Question

I have an ES query where I want to perform some query to retrieve 100 elements matching the query criteria and then perform aggregation on those values. But what happens is the query returns 100 hits and aggregation returns 100 buckets if i provide size 100, but the hits do not match the value in the buckets.
I tried loading all values with "size": 0 but I have large records and this takes a lot of time.
I have also tried using 2 queries (using Terms agg which is quite heavy) but I want to accomplish this with one query if possible. Is there any way to achieve this?
{
"size": 10,
"query": {
"bool": {
"must": [
{
"range": {
"amount": {
"gte": 10000,
"lte": 20000
}
}
}
]
}
},
"_source": {
"include":["id","amount"]
},
"aggs": {
"ID": {
"terms": {
"field": "id"
},
"aggs": {
"SumAgg": {
"sum: {
"field": "paidAmount"
}
}
}
}
}
}
Edit:
Here is the response:
"hits": {
"total": 712,
"max_score": 1,
"hits": [
{
"_score": 1,
"_source": {
"amount": 15732,
"id": 18xxxxxxx108
}
},
{
"_score": 1,
"_source": {
"amount": 11485,
"id": 33xxxxxxx107
}
},
{
"_score": 1,
"_source": {
"amount": 16757,
"id": 34xxxxxxx286
}
},
{
"_score": 1,
"_source": {
"amount": 16134,
"id": 29xxxxxxx018
}
},
{
"_score": 1,
"_source": {
"amount": 11767,
"id": 11xxxxxxx017
}
},
{
"_score": 1,
"_source": {
"amount": 16744,
"id": 38xxxxxxx106
}
},
{
"_score": 1,
"_source": {
"amount": 10587,
"id": 34xxxxxxx113
}
},
{
"_score": 1,
"_source": {
"amount": 18704,
"id": 34xxxxxxx177
}
},
{
"_score": 1,
"_source": {
"amount": 10077,
"id": 13xxxxxxx306
}
},
{
"_score": 1,
"_source": {
"amount": 12812,
"id": 46xxxxxxx334
}
}
]
},
"aggregations": {
"ID": {
"doc_count_error_upper_bound": 7,
"sum_other_doc_count": 702,
"buckets": [
{
"key": 24,
"doc_count": 1,
"SumAgg": {
"value": 17176
}
},
{
"key": 27,
"doc_count": 1,
"SumAgg": {
"value": 19924
}
},
{
"key": 81,
"doc_count": 1,
"SumAgg": {
"value": 19784
}
},
{
"key": 93,
"doc_count": 1,
"SumAgg": {
"value": 10942
}
},
{
"key": 124,
"doc_count": 1,
"SumAgg": {
"value": 12337
}
},
{
"key": 148,
"doc_count": 1,
"SumAgg": {
"value": 18604
}
},
{
"key": 158,
"doc_count": 1,
"SumAgg": {
"value": 14680
}
},
{
"key": 217,
"doc_count": 1,
"SumAgg": {
"value": 17295
}
},
{
"key": 273,
"doc_count": 1,
"SumAgg": {
"value": 10989
}
},
{
"key": 321,
"doc_count": 1,
"SumAgg": {
"value": 13917
}
}
]
}
}
I want the ids to be the same in both context.

Answer 1

Just realized that elasticsearch has an aggregation called Sampler Aggregation which allows you to run an aggregation query on only the top samples.
This was an experimental feature till version 5.x but it appears that they have released this post 6.0 version onwards. My bad for missing out on this!! :(
Below is how your query would be formulated:
Query:
POST <your_index_name>/_search
{
"size":10,
"query":{
"bool":{
"must":[
{
"range":{
"amount":{
"gte":10000,
"lte":20000
}
}
}
]
}
},
"_source":{
"include":[
"id",
"amount"
]
},
"aggs":{
"mysampler":{
"sampler":{ <---- Note this
"shard_size":10
},
"aggs":{
"ID":{
"terms":{
"field":"id"
},
"aggs":{
"SumAgg":{
"sum":{
"field":"amount"
}
}
}
}
}
}
}
}
Hope this helps and if you think this helps, feel free to accept this as answer and/or upvote it!!