I am quite new to elasticsearch, I am trying to sort on a subaggregations. that is my results should be sorted based on the sub aggregations first. I have tried lot of things to enable this sort but it isn't working. Can anyone help with this?
{
"aggs": {
"distinct_part": {
"terms": {
"field": "part",
"size": 1000
}
},
"aggs": {
"distinct_manufacturer": {
"terms": {
"field": "manufacturer",
"size": 1000
}
}
}
}
I am trying to sort on the manufacturer, my entire result should be sorted on that? Can someone point me on how I can achieve that?
I tried to do a test locally with your query. I did a small correction if I understood your issue well. I ingested the following data in the index "subsorting":
"part": "car",
"manufacturer": "brandA"
"part": "car",
"manufacturer": "brandB"
"part": "car",
"manufacturer": "brandC"
"part": "motor",
"manufacturer": "brandA"
"part": "motor",
"manufacturer": "brandB"
"part": "motor",
"manufacturer": "brandC"
Note: Both part and manufacturer are mapped as text.
GET subsorting/_search
{
"size": 0,
"aggs": {
"distinct_part": {
"terms": {
"field": "part.keyword",
"size": 1000
},
"aggs": {
"distinct_manufacturer": {
"terms": {
"field": "manufacturer.keyword",
"order": {
"_key": "asc"
},
"size": 1000
}
}
}
}
}
}
If both fields "part" and "manufacturer" are mapped as keywords, remove the ".keywords" from the query.
The response from the above query is as follows if sorted as ascending order:
"aggregations" : {
"distinct_part" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "motor",
"doc_count" : 4,
"distinct_manufacturer" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "brandA",
"doc_count" : 2
},
{
"key" : "brandB",
"doc_count" : 1
},
{
"key" : "brandC",
"doc_count" : 1
}
]
}
},
{
"key" : "car",
"doc_count" : 3,
"distinct_manufacturer" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "brandA",
"doc_count" : 1
},
{
"key" : "brandB",
"doc_count" : 1
},
{
"key" : "brandC",
"doc_count" : 1
}
]
}
}
]
}
}
If you need the result as descending order, here is the response where "_key": "desc":
"aggregations" : {
"distinct_part" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "motor",
"doc_count" : 4,
"distinct_manufacturer" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "brandC",
"doc_count" : 1
},
{
"key" : "brandB",
"doc_count" : 1
},
{
"key" : "brandA",
"doc_count" : 2
}
]
}
},
{
"key" : "car",
"doc_count" : 3,
"distinct_manufacturer" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "brandC",
"doc_count" : 1
},
{
"key" : "brandB",
"doc_count" : 1
},
{
"key" : "brandA",
"doc_count" : 1
}
]
}
}
]
}
}
Links:
https://www.elastic.co/guide/en/elasticsearch/reference/7.9/search-aggregations-bucket-terms-aggregation.html
Related
Please can someone help with the below Question.
https://discuss.elastic.co/t/nested-aggregation-with-and-always-return-0-match/315722?u=chattes
I have used following aggregations
1. Terms aggregation
2. Bucket selector
3. Nested aggregation
First I have grouped by user id using terms aggregation. Then further grouped by skill Id. Using bucket selector I have filtered users which have documents under two skills.
Query
GET index5/_search
{
"size": 0,
"aggs": {
"users": {
"terms": {
"field": "id",
"size": 10
},
"aggs": {
"skills": {
"nested": {
"path": "skills"
},
"aggs": {
"filter_skill": {
"terms": {
"field": "skills.id",
"size": 10,
"include": [
553,
426
]
}
}
}
},
"bucket_count": {
"bucket_selector": {
"buckets_path": {
"skill_count": "skills>filter_skill._bucket_count"
},
"script": "params.skill_count ==2"
}
}
}
}
}
}
Results
"aggregations" : {
"users" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : 1,
"doc_count" : 1,
"skills" : {
"doc_count" : 3,
"filter_skill" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "426",
"doc_count" : 1
},
{
"key" : "553",
"doc_count" : 1
}
]
}
}
},
{
"key" : 2,
"doc_count" : 1,
"skills" : {
"doc_count" : 2,
"filter_skill" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "426",
"doc_count" : 1
},
{
"key" : "553",
"doc_count" : 1
}
]
}
}
}
]
}
I am trying to get word count trends in docs as aggregate result . Although using the following approach I am able to get the doc count aggregation result but I am not able to find any resources using which I can get word count for the month of jan , feb & mar
PUT test/_doc/1
{
"description" : "one two three four",
"month" : "jan"
}
PUT test/_doc/2
{
"description" : "one one test test test",
"month" : "feb"
}
PUT test/_doc/3
{
"description" : "one one one test",
"month" : "mar"
}
GET test/_search
{
"size": 0,
"query": {
"match": {
"description": {
"query": "one"
}
}
},
"aggs": {
"monthly_count": {
"terms": {
"field": "month.keyword"
}
}
}
}
OUTPUT
{
"took" : 706,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"monthly_count" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "feb",
"doc_count" : 1
},
{
"key" : "jan",
"doc_count" : 1
},
{
"key" : "mar",
"doc_count" : 1
}
]
}
}
}
EXPECTED WORD COUNT OVER MONTH
"aggregations" : {
"monthly_count" : {
"buckets" : [
{
"key" : "feb",
"word_count" : 2
},
{
"key" : "jan",
"word_count" : 1
},
{
"key" : "mar",
"word_count" : 3
}
]
}
}
Maybe this query can help you:
GET test/_search
{
"size": 0,
"aggs": {
"monthly_count": {
"terms": {
"field": "month.keyword"
},
"aggs": {
"count_word_one": {
"terms": {
"script": {
"source": """
def str = doc['description.keyword'].value;
def array = str.splitOnToken(' ');
int i = 0;
for (item in array) {
if(item == 'one'){
i++
}
}
return i;
"""
},
"size": 10
}
}
}
}
}
}
Response:
"aggregations" : {
"monthly_count" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "feb",
"doc_count" : 1,
"count_word_one" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "2",
"doc_count" : 1
}
]
}
},
{
"key" : "jan",
"doc_count" : 1,
"count_word_one" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1",
"doc_count" : 1
}
]
}
},
{
"key" : "mar",
"doc_count" : 1,
"count_word_one" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "3",
"doc_count" : 1
}
]
}
}
]
}
}
I'm interested in aggregating my data into buckets, but I want to put two distinct values to the same bucket.
This is what I mean:
Say I have this query:
GET _search
{
"size": 0,
"aggs": {
"my-agg-name": {
"terms": {
"field": "ecs.version"
}
}
}
}
it returns this response:
"aggregations" : {
"my-agg-name" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1.12.0",
"doc_count" : 642826144
},
{
"key" : "8.0.0",
"doc_count" : 204064845
},
{
"key" : "1.1.0",
"doc_count" : 16508253
},
{
"key" : "1.0.0",
"doc_count" : 9162928
},
{
"key" : "1.6.0",
"doc_count" : 1111542
},
{
"key" : "1.5.0",
"doc_count" : 10445
}
]
}
}
every distinct value of the field ecs.version is in it's own bucket.
But say I wanted to define my buckets such that:
bucket1: [1.12.0, 8.0.0]
bucket2: [1.6.0, 8.4.0]
bucket3: [1.0.0, 8.8.0]
Is this possible in anyway?
I know I can just return all the buckets and do the sum programmatically, but this list can be very long, I don't think it would be efficient. Am I wrong?
You can use Runtime Mapping to generat runtime field and that field will be use for aggregation. I have done below exmaple on ES 7.16.
I have index some of the sample document and below is aggregation output without join on multipul values:
"aggregations" : {
"version" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1.12.0",
"doc_count" : 3
},
{
"key" : "1.6.0",
"doc_count" : 3
},
{
"key" : "8.4.0",
"doc_count" : 3
},
{
"key" : "8.0.0",
"doc_count" : 2
}
]
}
}
You can use below query with runtime mapping but you need to add multipul if condition for your version mappings:
{
"size": 0,
"runtime_mappings": {
"normalized_version": {
"type": "keyword",
"script": """
String version = doc['version.keyword'].value;
if (version.equals('1.12.0') || version.equals('8.0.0')) {
emit('1.12.0, 8.0.0');
} else if (version.equals('1.6.0') || version.equals('8.4.0')){
emit('1.6.0, 8.4.0');
}else {
emit(version);
}
"""
}
},
"aggs": {
"genres": {
"terms": {
"field": "normalized_version"
}
}
}
}
Below is output of above aggregation query:
"aggregations" : {
"genres" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "1.6.0, 8.4.0",
"doc_count" : 6
},
{
"key" : "1.12.0, 8.0.0",
"doc_count" : 5
}
]
}
}
I have created an index and indexed the document in elasticsearch it's working fine but here the challenge is i have to get an aggregate count of category field based on uniqueid i have given my sample documents below.
{
"UserID":"A1001",
"Category":"initiated",
"policyno":"5221"
},
{
"UserID":"A1001",
"Category":"pending",
"policyno":"5222"
},
{
"UserID":"A1001",
"Category":"pending",
"policyno":"5223"
},
{
"UserID":"A1002",
"Category":"completed",
"policyno":"5224"
}
**Sample output for UserID - "A1001"**
initiated-1
pending-2
**Sample output for UserID - "A1002"**
completed-1
How to get the aggregate count from above given Json documents like the sample output mentioned above
I suggest a terms aggregation as shown in the following:
{
"size": 0,
"aggs": {
"By_ID": {
"terms": {
"field": "UserID.keyword"
},
"aggs": {
"By_Category": {
"terms": {
"field": "Category.keyword"
}
}
}
}
}
}
Here is a snippet of the response:
"hits" : {
"total" : {
"value" : 4,
"relation" : "eq"
},
"max_score" : null,
"hits" : [ ]
},
"aggregations" : {
"By_ID" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "A1001",
"doc_count" : 3,
"By_Category" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "pending",
"doc_count" : 2
},
{
"key" : "initiated",
"doc_count" : 1
}
]
}
},
{
"key" : "A1002",
"doc_count" : 1,
"By_Category" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "completed",
"doc_count" : 1
}
]
}
}
]
}
}
I would like a query which it returns the number of times a field is repeated, according to the unique value of another field
I have this json:
"name" : james,
"city" : "chicago" <----------- same
},
{
"name" : james,
"city" : "san francisco"
},
{
"name" : james,
"city" : "chicago" <-----------same
},
{
"name" : Mike,
"city" : "chicago"
},
{
"name" : Mike,
"city" : "texas"<-----------same
},
{
"name" : Mike,
"city" : "texas"<-----------same
},
{
"name" : Peter,
"city" : "chicago"
},
I want to make a query where I count based on the unique value of two fields.
For example, james is equal to 2, because there are two equal fields (name: james, city, chicago) and a different field (name: james, city: san francisco)
The output would then be the following:
{
"key" : "james",
"doc_count" : 2
},
{
"key" : "Mike",
"doc_count" : 2
},
{
"key" : "Peter",
"doc_count" : 1
},
It is possible to do a single value count of two fields?
You can do a two level terms aggregation:
{
"size": 0,
"aggs": {
"names": {
"terms": {
"field": "name.keyword",
"size": 10
},
"aggs": {
"citys_by_name": {
"terms": {
"field": "city.keyword",
"size": 10,
"min_doc_count": 2
}
}
}
}
}
}
The response will looks like this:
"aggregations" : {
"names" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "james",
"doc_count" : 15,
"citys_by_name" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "chicago",
"doc_count" : 14
}
]
}
},
{
"key" : "Peter",
"doc_count" : 2,
"citys_by_name" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "chicago",
"doc_count" : 2
}
]
}
},
{
"key" : "mike",
"doc_count" : 2,
"citys_by_name" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [ ]
}
}
]
}
}
Or you can concatenate fields:
GET test/_search
{
"size": 0,
"aggs": {
"names": {
"terms": {
"script": {
"source": "return doc['name.keyword'].value + ' ' + doc['city.keyword'].value",
"lang": "painless"
},
"field": "name.keyword",
"size": 10,
"min_doc_count": 2
}
}
}
}
The response will looks lie this:
"aggregations" : {
"names" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "james chicago",
"doc_count" : 14
},
{
"key" : "Peter chicago",
"doc_count" : 2
}
]
}
}
If you want more stats on buckets, use the stats_buckets aggregation:
{
"size": 0,
"aggs": {
"names": {
"terms": {
"script": {
"source": "return doc['name.keyword'].value + ' ' + doc['city.keyword'].value",
"lang": "painless"
},
"field": "name.keyword",
"size": 10,
"min_doc_count": 2
}
},
"names_stats":{
"stats_bucket": {
"buckets_path":"names._count"
}
}
}
}
Will result:
"aggregations" : {
"names" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"key" : "james PARIS",
"doc_count" : 15
},
{
"key" : "james chicago",
"doc_count" : 13
},
{
"key" : "samuel PARIS",
"doc_count" : 11
},
{
"key" : "fred PARIS",
"doc_count" : 2
}
]
},
"names_stats" : {
"count" : 4,
"min" : 2.0,
"max" : 15.0,
"avg" : 10.25,
"sum" : 41.0
}
}
This was the solution that solved the problem for me
GET test/_search?filter_path=aggregations.count
{
"size": 0,
"aggs": {
"names": {
"terms": {
"script": {
"source": "return doc['name.keyword'].value + ' ' + doc['city.keyword'].value",
"lang": "painless"
},
"field": "name.keyword",
"size": 10,
"min_doc_count": 2
}
},
"count":{
"cardinality": {"script": "return doc['name.keyword'].value + ' ' + doc['city.keyword'].value"
}
}
}
}
Output:
{
"aggregations" : {
"count" : {
"value" : 2
}
}
}