elasticsearch group every X values - elasticsearch

Question

Using elasticsearch, I query for a particular field. Is there then a way to aggregate every X values?
For instance, let's say I query 10 documents for field "myField", returning 10 values,
1, 4, 2, 4, 5, 3, 3, 2, 1, 4.
Is there a way to aggregate such that every 2 values are averaged, yielding
2.5, 3, 4, 2.5, 2.5 ?

Answer 1

You can do some interesting--and perhaps inadvisable--stuff with scripted metric aggregations. They let you define map-reduce scripts that run against your documents. You can get yourself in trouble with this, of course.
But just to see if I could do it, I set up a simple, single-shard index with the data you provided:
PUT /test_index
{"settings": {"number_of_shards": 1}}
POST /test_index/doc/_bulk
{"index":{"_id":1}}
{"num":1}
{"index":{"_id":2}}
{"num":4}
{"index":{"_id":3}}
{"num":2}
{"index":{"_id":4}}
{"num":4}
{"index":{"_id":5}}
{"num":5}
{"index":{"_id":6}}
{"num":3}
{"index":{"_id":7}}
{"num":3}
{"index":{"_id":8}}
{"num":2}
{"index":{"_id":9}}
{"num":1}
{"index":{"_id":10}}
{"num":4}
Then I can average every two documents like this:
POST /test_index/_search
{
"size": 0,
"aggs": {
"profit": {
"scripted_metric": {
"init_script" : "_agg['nums'] = []; _agg['avgs'] = [];",
"map_script" : "_agg.nums.add(doc['num'].value); if(_agg.nums.size() == 2){ _agg.avgs.add((_agg.nums[0] + _agg.nums[1])/2.0); _agg['nums'] = [];}",
"combine_script" : "return _agg.avgs",
"reduce_script" : "return _aggs"
}
}
}
}
...
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 10,
"max_score": 0,
"hits": []
},
"aggregations": {
"profit": {
"value": [
[
2.5,
3,
4,
2.5,
2.5
]
]
}
}
}
It doesn't seem to respect sorting order in the query, though the outcome is deterministic as far as I can tell.
What I did here only works with a single shard; you could probably generalize it somehow if you wanted to tinker with it long enough.
Also, big fat disclaimer: doing this in production might be a bad idea. You'd want to test this sort of thing out on small data sets first before you potentially crash your cluster with out-of-memory errors. Also only use scripting if your cluster isn't open to the big bad Internet.
Here is some code I used to play around with it:
http://sense.qbox.io/gist/c31f089e63200127fd9ca09992004db8bb11b890