Is it possible to make elasticsearch aggregations faster when I only want the top 10 buckets? - performance

Question

I understand elasticsearch aggregation queries take a long time to execute by nature, especially on high cardinality fields. In our use case, we only need to bring back the first x buckets sorted alphabetically. Considering we only need to bring back 10 buckets, is there a way to make our query faster? Is there a way to get Elasticsearch to look at only the first 10 buckets in each shard and compare those only?
Here is my query...
{
"size": "600",
"timeout": "60s",
"query": {
"bool": {
"must": [
{
"match_all": {
"boost": 1
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
},
"aggregations": {
"firstname": {
"terms": {
"field": "firstname.keyword",
"size": 10,
"min_doc_count": 1,
"shard_min_doc_count": 0,
"show_term_doc_count_error": false,
"order": {
"_key": "asc"
},
"include": ".*.*",
"exclude": ""
}
}
}
}
I think I am on to something using a composite aggregation instead. Like this...
"home_address1": {
"composite": {
"size": 10,
"sources": [
{
"home_address1": {
"terms": {
"field": "home_address1.keyword",
"order": "asc"
}
}
}
]
}
}
Testing in Postman shows that this request takes way faster. Is this expected? If so, awesome. How can I add the include, exclude attributes to the composite query? For example, sometimes I only want to include buckets whose value matches "A.*"
If this query should not be any faster, then why does it appear so?

Answer 1

Composite terms aggs unfortunately don't support include, exclude and many other standard terms aggs' parameters so you've got 2 options:
Filter out the docs that don't match your prefix from within the query, like #Val pointed out.
or
use a script to do the filtering for you. This should be your last resort though -- scripts are pretty much guaranteed to run more slowly than standard query filters.
{
"size": 0,
"aggs": {
"home_address1": {
"composite": {
"size": 10,
"sources": [
{
"home_address1": {
"terms": {
"order": "asc",
"script": {
"source": """
def val = doc['home_address1.keyword'].value;
if (val == null) { return null; }
if (val.indexOf('A.') === 0) {
return val;
}
return null;
""",
"lang": "painless"
}
}
}
}
]
}
}
}
}
BTW your original terms agg already seems optimized so it surprises me that the composite is faster.