Select TOP + GROUP BY + SHORT in Elasticsearch? - elasticsearch

Question

Assume the following stockInWarehouse schema:
{
product_db: {
mappings: {
stockInWarehouse: {
properties: {
sku: {
type: "string"
},
arrivalTime: {
type: "date",
format: "dateOptionalTime"
}
}
}
}
}
}
The data in stockInWarehouse look like:
{
"hits": {
"total": 5,
"hits": [
{
"_index": "product_db",
"_type": "stockInWarehouse",
"_id": "1",
"_source": {
"sku": "item 1",
"arrivalTime": "2015-11-11T19:00:10.231Z"
}
},
{
"_index": "product_db",
"_type": "stockInWarehouse",
"_id": "2",
"_source": {
"sku": "item 2",
"arrivalTime": "2015-11-12T19:00:10.231Z"
}
},
{
"_index": "product_db",
"_type": "stockInWarehouse",
"_id": "3",
"_source": {
"sku": "item 1",
"arrivalTime": "2015-11-12T19:35:10.231Z"
}
},
{
"_index": "product_db",
"_type": "stockInWarehouse",
"_id": "4",
"_source": {
"sku": "item 1",
"arrivalTime": "2015-11-13T19:56:10.231Z"
}
},
{
"_index": "product_db",
"_type": "stockInWarehouse",
"_id": "5",
"_source": {
"sku": "item 3",
"arrivalTime": "2015-11-15T19:56:10.231Z"
}
}
]
}
}
What i am trying to do is to fetch TOP documents by arrivalTime (aka most recent documents) however i want them to be sorted by another field (sku) and limit to available sku. The expected result would look like this:
{
"hits": {
"total": 3,
"hits": [
{
"_index": "product_db",
"_type": "stockInWarehouse",
"_id": "5",
"_source": {
"sku": "item 3",
"arrivalTime": "2015-11-15T19:56:10.231Z"
}
},
{
"_index": "product_db",
"_type": "stockInWarehouse",
"_id": "4",
"_source": {
"sku": "item 1",
"arrivalTime": "2015-11-13T19:56:10.231Z"
}
},
{
"_index": "product_db",
"_type": "stockInWarehouse",
"_id": "2",
"_source": {
"sku": "item 2",
"arrivalTime": "2015-11-12T19:00:10.231Z"
}
}
]
}
}
If I sort by arrivalTime, the result sku list will contains item 3, item 1, item 1, item 2, item 1 (duplicate). If I sort by sku, result list will not reflect correct arrivalTime order.
Is this type of query possible in Elasticsearch? How can I archive this?

Answer 1

How about this one?
{
"size": 0,
"aggs": {
"terms_agg": {
"terms": {
"field": "sku",
"size": 100,
"order": {
"max_date_agg": "desc"
}
},
"aggs": {
"max_date_agg": {
"max": {
"field": "arrivalTime"
}
}
}
}
}
}
I have made size : 100 assuming you have lot of products.
Note You need to add index : not_analyzed to your mapping of sku
This is the result of the query
"aggregations": {
"terms_agg": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "item 3",
"doc_count": 1,
"max_date_agg": {
"value": 1447617370231,
"value_as_string": "2015-11-15T19:56:10.231Z"
}
},
{
"key": "item 1",
"doc_count": 3,
"max_date_agg": {
"value": 1447444570231,
"value_as_string": "2015-11-13T19:56:10.231Z"
}
},
{
"key": "item 2",
"doc_count": 1,
"max_date_agg": {
"value": 1447354810231,
"value_as_string": "2015-11-12T19:00:10.231Z"
}
}
]
}
}
I hope it helps!!