I need to search for max time (Recent entries), out of those max entries I want one with max value.
I've tried various types of nesting on aggregation, using filters,etc. but it doesn't seem to work out. Any help?
Example :-
Mapping -
{
"trytime": {
"mappings": {
"value": {
"properties": {
"time": {
"type": "long"
},
"value": {
"type": "long"
}
}
}
}
}
}
Values :-
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 4,
"max_score": 1,
"hits": [
{
"_index": "trytime",
"_type": "value",
"_id": "2",
"_score": 1,
"_source": {
"time": 9,
"value": 5
}
},
{
"_index": "trytime",
"_type": "value",
"_id": "4",
"_score": 1,
"_source": {
"time": 6,
"value": 10
}
},
{
"_index": "trytime",
"_type": "value",
"_id": "1",
"_score": 1,
"_source": {
"time": 9,
"value": 6
}
},
{
"_index": "trytime",
"_type": "value",
"_id": "3",
"_score": 1,
"_source": {
"time": 9,
"value": 9
}
}
]
}
}
I need the maximum time and maximum value corresponding to such time (There will be multiple values corresponding to a particular time).
Query :-
GET /trytime/_search
{
"size": 0,
"aggs":{
"max_Value": {
"max": {
"field": "value"
}
}
},
"query": {
"bool": {
"must": [
{
"range": {
"time": {
"gte": ___NEED-MAX-TIME-VALUE-HERE___
}
}
}
]
}
}
}
If I understood your question correctly, this nested aggregation should give you 3 latest times and for each time you get the max value. Set size to one if want only one result.
{
"size": 0,
"aggs": {
"times": {
"terms": {
"field": "time",
"size": 3,
"order": {
"_term": "desc"
}
},
"aggs": {
"max_vals": {
"max": {
"field": "value"
}
}
}
}
}
}
I'm not sure if this could be achieved by date histogram aggregation as well.
Related
Is there way to get max and min for all documents in index, not only max and min from category "game" without making another request to elastic?
{
"query": {
"bool": {
"must": [
{
"match": {
"category": "game"
}
}
]
}
},
"aggs": {
"maxPoints": {
"max": {
"field": "points"
}
},
"minPoints": {
"min": {
"field": "points"
}
}
}
Here is some data data i have, with query above I want to get this 2 docs from category game and min 0, max 100 instead of min 10, max 20.
[
{
"id": 1,
"category": "offer",
"points": 0
},
{
"id": 2,
"category": "game",
"points": 10
},
{
"id": 3,
"category": "game",
"points": 20
},
{
"id": 4,
"category": "offer",
"points": 100
}
]
Yeah, just remove the match clause, and add match_all query to include all the documents in your index. Use post_filter to get the expected results in a single ES call.
{
"query": {
"match_all": {}
},
"aggs": {
"maxPoints": {
"max": {
"field": "points"
}
},
"minPoints": {
"min": {
"field": "points"
}
}
},
"post_filter": { // Note this
"term": {
"category": "game"
}
}
}
Output
{
"took": 8,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "65406564",
"_type": "_doc",
"_id": "2",
"_score": 1.0,
"_source": {
"id": 2,
"category": "game",
"points": 10
}
},
{
"_index": "65406564",
"_type": "_doc",
"_id": "3",
"_score": 1.0,
"_source": {
"id": 3,
"category": "game",
"points": 20
}
}
]
},
"aggregations": {
"maxPoints": {
"value": 100.0
},
"minPoints": {
"value": 0.0
}
}
}
I use: Elasticsearch 7.7 , Kibana 7.7
For example, lets take two indexes:
User index with simple mapping:
PUT /user_index
{
"mappings": {
"properties": {
"user_id": { "type": "text" },
"user_phone": { "type": "text" },
"name": { "type": "text" }
}
}
}
Check with simple mapping:
PUT /check_index
{
"mappings": {
"properties": {
"user_id": { "type": "text" },
"price": { "type": "integer" },
"goods_count": {"type": "integer"}
}
}
}
I want to build table visualization like that:
________________________________________________________________________
user_id | user_phone | average_price | sum_goods_count |
___________|_______________|_____________________|______________________
1 | 123 | 512 | 64 |
___________|_______________|_____________________|______________________
2 | 456 | 256 | 16 |
___________|_______________|_____________________|______________________
So my questions are:
Is it real?
Do I understand correctly that I need to query these two indexes, get a list of users, and then in a loop create shopping carts with checks?
First thing first, you should try to de-normalize data in ES as much as possible to get the best performance and capability offered by it, And I went through the samples provided by you and comments in the question and it seems it can be easily achieved in your use-case and shown in below example, by combining user and check index into single index.
Index mapping
{
"mappings": {
"properties": {
"user_id": {
"type": "text",
"fielddata": "true"
},
"price": {
"type": "integer"
},
"goods_count": {
"type": "integer"
}
}
}
}
Index Data:
With the index mapping defined above, index these three documents, where one document is having "user_id":"1" and 2 documents have "user_id":"2"
{
"user_id":"1",
"price":500,
"goods_count":100
}
{
"user_id":"2",
"price":500,
"goods_count":100
}
{
"user_id":"2",
"price":100,
"goods_count":200
}
Search Query:
Refer to ES official documentation on Terms Aggregation, Top Hits aggregation, Sum aggregation and Avg aggregation to get detailed explanation.
{
"size": 0,
"aggs": {
"user": {
"terms": {
"field": "user_id"
},
"aggs": {
"top_user_hits": {
"top_hits": {
"_source": {
"includes": [
"user_id"
]
}
}
},
"avg_price": {
"avg": {
"field": "price"
}
},
"goods_count": {
"sum": {
"field": "goods_count"
}
}
}
}
}
}
Search Result:
{
"took": 10,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": null,
"hits": [
]
},
"aggregations": {
"user": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "2",
"doc_count": 2,
"top_user_hits": {
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "stof_63925596",
"_type": "_doc",
"_id": "2",
"_score": 1.0,
"_source": {
"user_id": "2"
}
},
{
"_index": "stof_63925596",
"_type": "_doc",
"_id": "3",
"_score": 1.0,
"_source": {
"user_id": "2"
}
}
]
}
},
"avg_price": {
"value": 300.0
},
"goods_count": {
"value": 300.0
}
},
{
"key": "1",
"doc_count": 1,
"top_user_hits": {
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "stof_63925596",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"user_id": "1"
}
}
]
}
},
"avg_price": {
"value": 500.0
},
"goods_count": {
"value": 100.0
}
}
]
}
}
}
As you can see in the search results above, for "user_id":"2" the average price is (500+100)/2 = 300 and sum of goods_count is 100+200 = 300.
Similarly for "user_id":"1" the average price is 500/1 = 500 and sum of goods_count is 100.
I have the following field in my index
field1:{key:value}
Is it possible to sort my query on sum of values in field1.
Thanks
Here's one way you could do this, assuming you know the fields ahead of time. It should be possible with some minor refinements if you need to wildcard the fields. This assumes the sibling fields on the nested type are numeric.
Example mapping:
"test": {
"mappings": {
"type1": {
"properties": {
"field1": {
"properties": {
"key1": {
"type": "integer"
},
"key2": {
"type": "integer"
}
}
}
}
}
}
}
Default results:
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"_index": "test",
"_type": "type1",
"_id": "AV8O7956gIcGI2d5A_5g",
"_score": 1,
"_source": {
"field1": {
"key1": 11,
"key2": 17
}
}
},
{
"_index": "test",
"_type": "type1",
"_id": "AV8O78FqgIcGI2d5A_5f",
"_score": 1,
"_source": {
"field1": {
"key1": 5,
"key2": 6
}
}
}
]
}
Query with script:
GET /test/_search
{
"query": {
"function_score": {
"query": {
"match_all": {}
},
"functions": [
{
"script_score": {
"script": "return (doc['field1.key1'].value + doc['field1.key2'].value) * -1"
}
}
]
}
}
}
Logic taking the lowest score as the best score (least negative in this case):
{
"took": 18,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": -11,
"hits": [
{
"_index": "test",
"_type": "type1",
"_id": "AV8O78FqgIcGI2d5A_5f",
"_score": -11,
"_source": {
"field1": {
"key1": 5,
"key2": 6
}
}
},
{
"_index": "test",
"_type": "type1",
"_id": "AV8O7956gIcGI2d5A_5g",
"_score": -28,
"_source": {
"field1": {
"key1": 11,
"key2": 17
}
}
}
]
}
}
Hopefully this gives you the gist of whatever specific scoring logic you need
I cannot seem to aggregate my query results when using my custom query parser. I get a result set by these are not aggregated. When using a standard query parser like match everything turns out well.
What works:
GET pages/_search
{
"query": {
"match": {
"text": "binomial"
}
},
"aggs": {
"docs": {
"terms": {
"field": "rooturl"
}
}
}
}
returns a nice aggregated result:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 10,
"max_score": 11.11176,
"hits": [
...
{
"_index": "pages",
"_type": "doc",
"_id": "AVcq6z6lzDazctHi91RE",
"_score": 3.3503218,
"_source": {
"rooturl": "document",
"type": "equation",
"url": "document:poly",
"text": "coefficient"
}
},
{
"_index": "pages",
"_type": "doc",
"_id": "AVcq6z6xzDazctHi91RF",
"_score": 3.3503218,
"_source": {
"rooturl": document",
"type": "equation",
"url": "document:poly",
"text": "dot"
}
}
...
]
},
"aggregations": {
"docs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "document",
"doc_count": 10
}
]
}
}
}
But when using my custom query parser, The result is not aggregated.
Query:
GET pages/_search
{
"query": {
"my_custom_query_parser": {
"query": "binomial"
}
},
"aggs": {
"docs": {
"terms": {
"field": "rooturl"
}
}
}
}
Can anyone point me into the right direction?
I have data, that has an attribute like this
apiUrl:/REST/endpoint/123
Now I would like to show all the urls and I am trying to use an aggregate function (apiUrl.raw is not_analyzed part of the multifield):
POST /index/type/_search
{
"aggregations": {
"application": {
"terms": {
"field": "apiUrl.raw"
}
}
}
}
When running this query, no results get returned. What am I doing wrong? I would expect something along the lines (and the count of occurence):
/REST/api1/123
/REST/otherApi/345
Thanks!
Your query does return non-empty results. Compare and let us know what was the difference:
PUT index
PUT index/type/_mapping
{
"properties" : {
"apiUrl": {
"type": "multi_field",
"fields": {
"apiUrl": {"type":"string", "index":"analyzed"},
"raw": {"type":"string", "index":"not_analyzed"}
}
}
}
}
GET index/type/_mapping
PUT index/type/1
{
"apiUrl":"/REST/api1/123"
}
PUT index/type/2
{
"apiUrl":"/REST/otherApi/345"
}
GET index/type/_search?fields=apiUrl.raw
GET index/type/_search
{
"aggregations": {
"application": {
"terms": {
"field": "apiUrl.raw"
}
}
}
}
Response:
{
"took": 76,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"_index": "index",
"_type": "type",
"_id": "1",
"_score": 1,
"_source": {
"apiUrl": "/REST/api1/123"
}
},
{
"_index": "index",
"_type": "type",
"_id": "2",
"_score": 1,
"_source": {
"apiUrl": "/REST/otherApi/345"
}
}
]
},
"aggregations": {
"application": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "/REST/api1/123",
"doc_count": 1
},
{
"key": "/REST/otherApi/345",
"doc_count": 1
}
]
}
}
}