I can use aggregate to make some stats between two timestamps as following:
{
"size": 0,
"query": {
"bool": {
"filter": [
{
"term": {
"status": "ok"
}
},
{
"term": {
"deviceId": "123456789"
}
},
{
"range": {
"time": {
"gte": 1669852800,
"lt": 1671062400
}
}
}
]
}
},
"aggs": {
"results": {
"date_histogram": {
"field": "time",
"fixed_interval": "60",
}
}
}
}
Is it possible to query the results contain specific time range daily only? For example, 7am - 9am daily between Dec.1 to Dec.15. How to achieve it?
I found the solution on elasticsearch v7.15.2 as following:
{
"size": 0,
"query": {
"bool": {
"filter": [
{
"term": {
"status": "ok"
}
},
{
"term": {
"deviceId": "123456789"
}
},
{
"range": {
"time": {
"gte": 1669852800,
"lt": 1671062400
}
}
},
{
"script": {
"script": {
"source": "doc.time.value.getHourOfDay() >= params.min && doc.time.value.getHourOfDay() < params.max",
"params": {
"min": 8,
"max": 10
}
}
}
}
]
}
},
"aggs": {
"results": {
"date_histogram": {
"field": "time",
"fixed_interval": "60"
}
}
}
}
The syntax is slightly different from the comment above, but it works.
Related
I am not able to perform nested aggregation in a specific date range.
Ideally, I would want to get % values of two fields for the previous day and the current day. Here's the JSON (this is a part of the elastic watcher and not the whole config):
{
"metadata":{
"threshold":5,
"interval":"2m",
"window":"2d"
},
"trigger":{
"schedule":{
"interval":"2m"
}
},
"input":{
"search":{
"request":{
"indices":[
"filebeat-*"
],
"types":[
"doc"
],
"body": {
"aggs": {
"aggs1": {
"range": {
"date_range": {
"ranges": [
{
"from": "now-2d/d"
},
{
"to": "now-2d/d"
}
]
},
"aggs": {
"max": {
"script": {
"source": "(doc['upstream'].value\/100)"
}
}
}
}
},
"aggs2": {
"range": {
"date_range": {
"ranges": [
{
"from": "now-2d/d"
},
{
"to": "now-2d/d"
}
]
}
},
"aggs": {
"max": {
"script": {
"source": "(doc['downstream'].value\/100)"
}
}
}
},
"aggs3": {
"range": {
"date_range": {
"ranges": [
{
"from": "now-1d/d"
},
{
"to": "now/d"
}
]
}
},
"aggs": {
"max": {
"script": {
"source": "(doc['upstream'].value\/100)"
}
}
}
},
"aggs4": {
"range": {
"date_range": {
"ranges": [
{
"from": "now-1d/d"
},
{
"to": "now/d"
}
]
}
},
"aggs": {
"max": {
"script": {
"source": "(doc['downstream'].value\/100)"
}
}
}
}
},
"query": {
"bool": {
"filter": {
"range": {
"#timestamp": {
"lte": "now",
"gte": "now-{{ctx.metadata.window}}"
}
}
}
}
}
}
}
}
}
}
All I want is to have 4 values for two fields for current day and previous day so that I can get the difference between the values for further procesing.
Thanks.
I query an index with wildcard (interactive*) to get all documents for the two indices interactive-foo* & interactive-bar*.
For some of my aggregations all of the indices are relevant but for others only interactive-foo* OR interactive-bar*. So I just want to filter for these 'subindices' in the aggregation.
GET _search
{
"query":{
"bool": {
"must": [
{
"range": {
"timestamp": {
"gte": "2017-08-01 00:00:00",
"lte": "2017-08-31 23:59:59"
}
}
},
{
"match": {
"key": "SOME_KEY"
}
}
]
}
},
"size":0,
"aggs": {
// This one should be filtered and just count for interactive-bar*
"bar_count": {
"value_count": {
"field": "SOME_FIELD"
}
},
// This one should be filtered and just count for interactive-foo*
"foo_count": {
"value_count": {
"field": "SOME_FIELD"
}
}
}
}
You can use a filter aggregation like this:
{
"query": {
"bool": {
"must": [
{
"range": {
"timestamp": {
"gte": "2017-08-01 00:00:00",
"lte": "2017-08-31 23:59:59"
}
}
},
{
"match": {
"key": "SOME_KEY"
}
}
]
}
},
"size": 0,
"aggs": {
"bar_count": {
"filter": {
"indices": {
"indices": ["interactive-bar-*"]
}
},
"aggs": {
"bar_count": {
"value_count": {
"field": "SOME_FIELD"
}
}
}
},
"foo_count": {
"filter": {
"indices": {
"indices": ["interactive-foo-*"]
}
},
"aggs": {
"foo_count": {
"value_count": {
"field": "SOME_FIELD"
}
}
}
}
}
}
Note though that the indices query has been deprecated in ES 5.0. What you should do instead is to use a terms query on the _index field and list all the indices you want to include in your aggregation, like this:
"size": 0,
"aggs": {
"bar_count": {
"filter": {
"terms": {
"_index": ["interactive-foo-2017.08.14", "interactive-foo-2017.08.15"]
}
},
"aggs": {
"bar_count": {
"value_count": {
"field": "SOME_FIELD"
}
}
}
},
"foo_count": {
"filter": {
"terms": {
"_index": ["interactive-bar-2017.08.14", "interactive-bar-2017.08.15"]
}
},
"aggs": {
"foo_count": {
"value_count": {
"field": "SOME_FIELD"
}
}
}
}
}
}
I have this query which is hitting the below table. How can I add source field to it to return specific columns from the _source field of this index?
GET mktg_percolate_postmetrics_mib/_search
{
"query": {
"bool": {
"must": [
{
"range": {
"end_date": {
"gte": "now-30d",
"lte": "now",
"format": "epoch_millis"
}
}
}
]
}
},
"size": 0,
"aggs": {
"2": {
"terms": {
"field": "post_id",
"size": 25,
"order": {
"1": "desc"
}
},
"aggs": {
"1": {
"sum": {
"script": "doc['facebook_likes'].value + doc['facebook_shares'].value + doc['facebook_comments'].value + doc['facebook_post_consumptions_by_type_link_clicks'].value",
"lang": "expression"
}
}
}
}
}
}
Idea: Search Top events on specific range and order by start_time. Like:
{
"from": 0,
"size": 7,
"query": {
"filtered": {
"query": { "match_all": {} },
"filter": {
"and": [
{ "bool": { "must_not": { "term": { "status": "OK" } } } },
{ "bool": { "must": { "term": { "is_blocked": false } } } }, {
"range": {
"start_time": {
"gte": "2016-01-01",
"lte": "2016-03-01"
}
}
}, {
"bool": {
"must": {
"geo_distance": {
"distance": "150km",
"coordinates": "xx.xxx, zz.zz "
}
}
}
}
]
}
}
},
"sort": [{ "start_time": "asc" },
{ "attending": "desc" }
]
}
I quite new on this concept of aggregations so still with basic problems to understand
I wanna 7 results of top events for the next 2 months. So I have two attributes to look. The max of people attending(attendings) is the definition of Top, but also I wanna order this by time(start_time: asc)
What I start to wrote but is wrong:
{
"aggs": {
"aggs": {
"event_interval": {
"date_histogram": {
"field": "start_time",
"interval": "2M",
"format": "dateOptionalTime"
}
},
"max_attending": { "max": { "field": "attending" } },
"_source": {
"include": [
"name"
]
}
}
}
}
I'm not sure you need to be using an aggregation to get what you are looking for, I think that a simple query can yield the results you would like to see, try this:
{
"size": 7,
"sort": {
"attending": {
"order": "desc"
}
},
"query": {
"bool": {
"filter": [
{
"range": {
"start_time": {
"gte": "now-2M",
"lte": "now"
}
}
}
]
}
}
}
I have a filtered elasticsearch query that works, but I want to use minimum_should_match to instruct ES to return only results that have at least 3 should matches. But I can't seem to figure out where to put minimum_should_match. Where should I put it?
{
"size": 100,
"sort": {
"price_monthly": "asc"
},
"query": {
"filtered": {
"query": {
"match_all": []
},
"filter": {
"bool": {
"must": [],
"should": [
[
{
"range": {
"mb.untouched": {
"gte": "0",
"lt": "500"
}
}
},
{
"range": {
"mb.untouched": {
"gte": "500",
"lt": "1000"
}
}
}
],
[
{
"range": {
"minutes.untouched": {
"gte": "0",
"lt": "100"
}
}
},
{
"range": {
"minutes.untouched": {
"gte": "200",
"lt": "300"
}
}
}
],
[
{
"range": {
"sms.untouched": {
"gte": "750",
"lt": "1000"
}
}
}
]
],
"must_not": {
"missing": {
"field": "provider.untouched"
}
}
}
},
"strategy": "query_first"
}
},
"aggs": {
"provider.untouched": {
"terms": {
"field": "provider.untouched"
}
},
"prolong.untouched": {
"terms": {
"field": "prolong.untouched"
}
},
"duration.untouched": {
"terms": {
"field": "duration.untouched"
}
},
"mb.untouched": {
"histogram": {
"field": "mb.untouched",
"interval": 500,
"min_doc_count": 1
}
},
"sms.untouched": {
"histogram": {
"field": "sms.untouched",
"interval": 250,
"min_doc_count": 1
}
},
"minutes.untouched": {
"histogram": {
"field": "minutes.untouched",
"interval": 100,
"min_doc_count": 1
}
},
"price_monthly.untouched": {
"histogram": {
"field": "price_monthly.untouched",
"interval": 5,
"min_doc_count": 1
}
}
}
}
In order to use minimum_should_match, you need to rewrite your filtered query a little bit, i.e. you need to move your should clause to the query part of the filtered query and just keep must_not in the filter part (because missing is a filter). Then you can add minimum_should_match: 3 in the bool query part as shown below:
{
"size": 100,
"sort": {
"price_monthly": "asc"
},
"query": {
"filtered": {
"query": {
"bool": {
"minimum_should_match": 3,
"must": [],
"should": [
[
{
"range": {
"mb.untouched": {
"gte": "0",
"lt": "500"
}
}
},
{
"range": {
"mb.untouched": {
"gte": "500",
"lt": "1000"
}
}
}
],
[
{
"range": {
"minutes.untouched": {
"gte": "0",
"lt": "100"
}
}
},
{
"range": {
"minutes.untouched": {
"gte": "200",
"lt": "300"
}
}
}
],
[
{
"range": {
"sms.untouched": {
"gte": "750",
"lt": "1000"
}
}
}
]
]
}
},
"filter": {
"bool": {
"must_not": {
"missing": {
"field": "provider.untouched"
}
}
}
},
"strategy": "query_first"
}
},
"aggs": {
"provider.untouched": {
"terms": {
"field": "provider.untouched"
}
},
"prolong.untouched": {
"terms": {
"field": "prolong.untouched"
}
},
"duration.untouched": {
"terms": {
"field": "duration.untouched"
}
},
"mb.untouched": {
"histogram": {
"field": "mb.untouched",
"interval": 500,
"min_doc_count": 1
}
},
"sms.untouched": {
"histogram": {
"field": "sms.untouched",
"interval": 250,
"min_doc_count": 1
}
},
"minutes.untouched": {
"histogram": {
"field": "minutes.untouched",
"interval": 100,
"min_doc_count": 1
}
},
"price_monthly.untouched": {
"histogram": {
"field": "price_monthly.untouched",
"interval": 5,
"min_doc_count": 1
}
}
}
}