ElasticSearch .NET Nest Sub Aggregation - elasticsearch

Consider the following aggregation and sub aggregation:
.Aggregations(a => a
.Terms("ByCustomerByImpressionCount", tad => tad
.Field(o => o.CustomerName)
.OrderDescending("sumImpressionCount")
.Aggregations(ad => ad
.Sum("sumImpressionCount", sad => sad
.Field(o => o.ImpressionCount)
)
)
))
The query works fine, and I get results.
I can access the top aggregation with:
var agg = results.Aggs.Terms("ByCustomerByImpressionCount");
According to the docs, I should be able to get the sub agg with:
var subAgg = agg.Aggs.Sum("sumImpressionCount");
But Aggs isn't even a property of agg - any ideas?
EDIT:
This is the only way I've been able to get to the sub aggregation:
PopulateList(results.Aggs.Terms("CustomersByImpressionCount").Items, CustomersByImpressionCount);
private void PopulateList(IList<KeyItem> items, List<DataContainer> listToPopulate)
{
items.ForEach(item =>
{
listToPopulate
.Add(new DataContainer() { Name = item.Key, Sum = ((ValueMetric)item.Aggregations.First().Value).Value });
});
}
...but this seems to be overly complicated. Here is the ES query:
"query": {
"bool": {
"must": [
{ "range": { "requestedShipDate": {
"gte": "2014-11-26",
"lte": "2015-02-03"
} }
}
],
"must_not": [
{"terms": {
"status": [
"Order Shipped",
"Canceled",
"Completed"
]
}}
]
}
},
"aggs": {
"ByCustomerByImpressionCount": {
"terms": {
"field": "customerName",
"order": { "sumImpressionCount": "desc" }
},
"aggs": {
"sumImpressionCount": { "sum": { "field": "impressionCount" }}
}
}}
And here is partial response:
"ByCustomerByImpressionCount": {
"doc_count_error_upper_bound": -1,
"sum_other_doc_count": 26785,
"buckets": [
{
"key": "<One of our customers>",
"doc_count": 1722,
"sumImpressionCount": {
"value": 9583176
}
}

The documentation is wrong. We'll fix that ASAP.
Here is the correct way to access your aggregations:
var terms = results.Aggs.Terms("ByCustomerByImpressionCount");
foreach(var term in terms.Items)
{
var sum = term.Sum("sumImpressionCount");
}

Related

How to group docs by latest and run aggregation on the group in elastic search?

I need to get all clients under a partnerId & since there would be many duplicate clients I need to group the clients by clientId,reportDate and run some aggregations on it.
Index data is like below -
[
{ partnerId: "PID1234", clientId: "c1234", reportDate: "2022-02-01" }, // dup
{ partnerId: "PID1234", clientId: "c1234", reportDate: "2030-02-01" }, // dup, agg should take this one only since this is the latest.
{ partnerId: "PID1111", clientId: "c1222", reportDate: "2010-02-01" },
{ partnerId: "PID2222", clientId: "c1444", reportDate: "2013-02-01" },
]
I need to do something like the below query, the problem is top hits don't accept sub aggregations -
{
"query": {
"bool": {
"must": [
{
"term": {
"partnerId": "PID1234"
}
}
]
}
},
"aggs": {
"groupp": {
"top_hits": {
"sort": [
{
"clientId": {
"order": "desc"
}
}
],
"size": 1,
"aggs": {
"total_engagement_count": {
"sum": { "field": "recommendations.totalEngagement" }
}
}
}
}
}
}
Maybe you need to change your approach.
Look at the answer below, aggs metrics don't allow sub aggs.
https://github.com/elastic/elasticsearch/issues/16537#issuecomment-181965367

Nest multiple terms in DSL syntax

I'm trying to convert following query into NEST DSL syntax:
{
"aggs": {
"color": {
"terms": {
"field": "Color"
}
},
"weight": {
"terms": {
"field": "Weight"
}
}
}
}
Each of the terms is coming from the list which holds name and fieldId. So far I've managed to do this:
var request2 = _client.Search<T>(s => s
.Aggregations(aggs =>
aggs.Terms("Weight", x => x.Field("Weight")).Terms("Color", x => x.Field("Weight"))));
Which works as expected however I need to be able to supply parameters Weight and Color dynamically as the method can be called with different set of parameters. Is there a way how to use something like:
aggs.Terms(x => x.field( myList.foreach(y.value))));
I guess this would work better with the Object initializer syntax, however I would rather get this working inside dsl.
Something like the following would work
var client = new ElasticClient();
var fields = new List<string>
{
"color",
"weight",
"foo",
"bar"
};
var response = client.Search<object>(s => s
.Aggregations(aggs =>
{
foreach (var field in fields)
{
aggs.Terms(field, t => t.Field(field));
}
return aggs;
})
);
which generates the following request
{
"aggs": {
"color": {
"terms": {
"field": "color"
}
},
"weight": {
"terms": {
"field": "weight"
}
},
"foo": {
"terms": {
"field": "foo"
}
},
"bar": {
"terms": {
"field": "bar"
}
}
}
}

Writing a NEST query to sort aggregation buckets by score

What I want is to create an aggregation bucket for each unitId (which is a field in my document). I want each bucket to be ordered by the max score in that bucket. I have written the following query which does what I want:
"aggs": {
"UnitAggregationBucket": {
"terms": {
"field": "unitId",
"size": 10,
"order": {
"max_score": "desc"
}
},
"aggs": {
"max_score": {
"max": {
"script": "_score"
}
}
}
}
I am using script to find the max score per bucket, in a sub-aggregation. I don't know how to write the above query using NEST?
Upadate:
This is the answer that I got from Elastic Community:
With 6.x, this would be something like:
var client = new ElasticClient();
var searchResponse = client.Search<object>(s => s
.Aggregations(a => a
.Terms("UnitAggregationBucket", t => t
.Field("unitId")
.Size(10)
.Order(o => o
.Descending("maximum_score")
)
.Aggregations(aa => aa
.Max("maximum_score", m => m
.Script("_score")
)
)
)
)
);
var termsAgg = searchResponse.Aggregations.Terms("UnitAggregationBucket");
foreach(var bucket in termsAgg.Buckets)
{
// do something with buckets
var maxScore = bucket.Max("maximum_score").Value;
}
Note that you can't use max_score for the name of the aggregation as
it's a reserved keyword name in the client, which the client uses in
its heuristics based aggregation response JSON deserialization method.
Original Answer
I managed to write the following NEST Query:
var unitAggregations = new TermsAggregation("UnitAggregationBucket")
{
Size 10,
Field = Field<MyDocument>(p => p.UnitId),
Order = new List<TermsOrder>
{
new TermsOrder()
{
Key = "max_score_in_bucket",
Order = SortOrder.Descending
}
},
Aggregations = new MaxAggregation("max_score_in_bucket", string.Empty)
{
Script = new InlineScript("_score")
}
};
Which produces, the following json:
"aggs": {
"UnitAggregationBucket": {
"aggs": {
"max_score_in_bucket": {
"max": {
"script": {
"source": "_score"
}
}
}
},
"terms": {
"field": "unitId",
"order": [
{
"max_score_in_bucket": "desc"
}
],
"size": 10
}
}
}
It's not the exact json that I wanted, but it does what I want.
Note: max_score is a reserved key-word in Elasticsearch, so I had to use a different name: max_score_in_bucket

Elasticsearch - How to get the sum of fields with terms?

I'm trying to get a sum of some fields, with terms (let's say, number of files of specific user with Id).
First I tried:
_mainManager.Client.Search<object>
(q => q
.Type("Mail")
.Filter(c => c.Term("SentMail_Sender_Id", userId))
.Aggregations(a => a.Terms("sum", g => g.Field("SentMail_Upload_Files_Count")))
.Size(1));
But no luck in Agg property, so I tried this:
_mainManager.Client.Search<object>
(q => q
.Type("Mail")
.Aggregations(a => a.Filter("fil", b => b.Filter(c => c.Term("SentMail_Sender_Id", userId))).Sum("sum", f => f.Field("SentMail_Upload_Files_Count"))));
But then again, no luck there.
Can anyone help?
The following code gives you the result that you need I thing:
PUT /mail/message/1
{
"SentMail_Sender_Id":1,
"SentMail_Upload_Files_Count":10
}
PUT /mail/message/2
{
"SentMail_Sender_Id":1,
"SentMail_Upload_Files_Count":2
}
PUT /mail/message/3
{
"SentMail_Sender_Id":2,
"SentMail_Upload_Files_Count":7
}
GET /mail/_search?search_type=count
{
"query": {
"filtered": {
"filter": {
"term": {
"SentMail_Sender_Id": 1
}
}
}
},
"aggs": {
"total": {
"stats": {
"field": "SentMail_Upload_Files_Count"
}
}
}
}
The response is:
"aggregations": {
"total": {
"count": 2,
"min": 2,
"max": 10,
"avg": 6,
"sum": 12
}
}

Elastic Search - Creating Query with NEST

Given the query:
GET places/_search
{
"query": {
"match_all": {}
},
"size": 0,
"aggs": {
"filtered_cells": {
"filter": {
"geo_bounding_box": {
"loc": {
"top_left": {
"lat": 53.480950,
"lon": -2.2374300
},
"bottom_right": {
"lat": 51.5085300,
"lon": -0.1257400
}
}
}
},
"aggs": {
"cells": {
"geohash_grid": {
"field": "loc",
"precision": "precision"
},
"aggs": {
"center_lat": {
"avg": {
"script": "doc['loc'].lat"
}
},
"center_lon": {
"avg": {
"script": "doc['loc'].lon"
}
}
}
}
}
}
}
}
I am trying to move this query into a piece of code I am developing, but I am having trouble structuring the query above using the NEST library.
Here is what I have so far:
var s = new SearchDescriptor<FacebookGraphResponse>()
.MatchAll()
.From(0)
.Size(10)
.Filter(filter => filter
.GeoBoundingBox(f => f.Loc, 53.480950, -2.2374300, 51.5085300, -0.1257400)
);
Is anyone able to provide any input on how I can translate this query to nest, as I am currently stumped.
Many Thanks
Solved it!
var s = new SearchDescriptor<FacebookGraphResponse>()
.MatchAll()
.Size(0)
.Aggregations(aggs => aggs
.Filter("filtered_cells", f => f
.Filter(f1 => f1
.GeoBoundingBox(bb => bb.Loc, -2.2374300, 53.480950, -0.1257400, 51.5085300))
.Aggregations(a => a
.GeoHash("cells", geo => geo
.GeoHashPrecision(GeoHashPrecision.Precision10)
.Field("loc")
.Aggregations(a1 => a1
.Average("center_lat", x => x.Script("doc['loc'].lat"))
.Average("center_lon", x => x.Script("doc['loc'].lon")))))));

Resources