Compare last two elasticsearch inputs and show smaller - elasticsearch

Question

I have nested query where I want to compare last two inputs and display smaller one.
For example:
"price_history":[
{"id":0,
"price":16.99,
"date":"2021-02-07"
},
"id":1,
"price":20.99,
"date":"2021-02-08"
},
{"id":2,
"price":16.99,
"date":"2021-02-09"
}
]
So I want only id 1 and 2 to be compared and only id 2 to be shown.
I am looking for help to build such a query and am open to any other data model suggestions.

Answer 1

I understand you want to compare the last 2 prices and display the lowest. That information can be calculated on index time so it should be done there instead of re-calculating on each query.
I will show you how to do it on index time assuming you have no control on the software is ingesting the data using a Pipeline that will process your data before putting it in elasticsearch.
We are going to create a new field called best_price that store this price so you can then make queries against this field instead of calculate it on each query.
Ingesting data
POST test_uzer/_doc
{
"price_history": [
{
"id": 0,
"price": 16.99,
"date": "2021-02-07"
},
{
"id": 1,
"price": 20.99,
"date": "2021-02-08"
},
{
"id": 2,
"price": 16.99,
"date": "2021-02-09"
}
]
}
POST test_uzer/_doc
{
"price_history": [
{
"id": 0,
"price": 1.99,
"date": "2021-02-07"
},
{
"id": 1,
"price": 15.99,
"date": "2021-02-08"
},
{
"id": 2,
"price": 16.99,
"date": "2021-02-09"
}
]
}
Creating the ingest pipeline
PUT _ingest/pipeline/best_price
{
"description": "return the best price between the 2 last",
"processors": [
{
"script": {
"lang": "painless",
"source": "def prices = ctx.price_history; def length = prices.length; ctx.best_price = prices[length - 1].price > prices[length - 2].price ? prices[length - 2].price : prices[length - 1].price"
}
}
]
}
Reindexing the data to have the new field
POST _reindex
{
"source": {
"index": "test_uzer"
},
"dest": {
"index": "test_uzer_new",
"pipeline": "best_price"
}
}
Add the ingest pipeline as default to apply to all the new documents
PUT test_uzer_new/_settings
{
"index": {
"default_pipeline": "best_price"
}
}
Ingest document to test
POST test_uzer_new/_doc
{
"price_history": [
{
"id": 0,
"price": 2,
"date": "2021-02-07"
},
{
"id": 1,
"price": 3,
"date": "2021-02-08"
},
{
"id": 2,
"price": 1,
"date": "2021-02-09"
}
]
}
best_price should be 1
POST test_uzer_new/_search
{
"took" : 0,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : 1.0,
"hits" : [
{
"_index" : "test_uzer_new",
"_type" : "_doc",
"_id" : "a8oz-ncBRP0FeAG5geN1",
"_score" : 1.0,
"_source" : {
"best_price" : 16.99,
"price_history" : [
{
"date" : "2021-02-07",
"price" : 16.99,
"id" : 0
},
{
"date" : "2021-02-08",
"price" : 20.99,
"id" : 1
},
{
"date" : "2021-02-09",
"price" : 16.99,
"id" : 2
}
]
}
},
{
"_index" : "test_uzer_new",
"_type" : "_doc",
"_id" : "bMpE-ncBRP0FeAG54ONR",
"_score" : 1.0,
"_source" : {
"best_price" : 15.99,
"price_history" : [
{
"date" : "2021-02-07",
"price" : 1.99,
"id" : 0
},
{
"date" : "2021-02-08",
"price" : 15.99,
"id" : 1
},
{
"date" : "2021-02-09",
"price" : 16.99,
"id" : 2
}
]
}
},
{
"_index" : "test_uzer_new",
"_type" : "_doc",
"_id" : "bspJ-ncBRP0FeAG59uOi",
"_score" : 1.0,
"_source" : {
"best_price" : 1,
"price_history" : [
{
"date" : "2021-02-07",
"price" : 2,
"id" : 0
},
{
"date" : "2021-02-08",
"price" : 3,
"id" : 1
},
{
"date" : "2021-02-09",
"price" : 1,
"id" : 2
}
]
}
}
]
}
}
Works!
Of course there are many ways to achieve what you want, but the point is index and calculate as much data as you can before querying. This will make your searches faster.