NEST - How can i do multiple nested aggregation? - elasticsearch

How can I do multiple nested aggregation?
I have tried something like this:
Aggregations(x => x
.Nested("Facets", y => y.Path("categories")
.Aggregations(r => r.Terms("categories", w => w.Field(q => q.Categories.FirstOrDefault().Id))
)).Nested("Facets2", s => s.Path("brand")
.Aggregations(e => e.Terms("brand", w => w.Field(q => q.Brand.Id))
)));
But it returns Facets2 as a child of Facets
Can anyone help?

The aggregations that you have work as expected with NEST client version 1.7.1 as demonstrated with this example
void Main()
{
var settings = new ConnectionSettings();
var connection = new InMemoryConnection(settings);
var client = new ElasticClient(connection : connection);
var response = client.Search<Example>(s => s
.Aggregations(aggs => aggs
.Nested("Facets", nested => nested
.Path(p => p.Categories)
.Aggregations(r => r
.Terms("categories", w => w
.Field(q => q.Categories.FirstOrDefault().Id)
)
)
)
.Nested("Facets2", nested => nested
.Path(p => p.Brand)
.Aggregations(e => e
.Terms("brand", w => w
.Field(q => q.Brand.Id)
)
)
)
)
);
Console.WriteLine(Encoding.UTF8.GetString(response.RequestInformation.Request));
}
public class Example
{
public IList<Category> Categories { get; set; }
public Brand Brand { get; set; }
}
public class Brand
{
public int Id { get; set; }
}
public class Category
{
public int Id { get; set; }
}
This outputs the following request query
{
"aggs": {
"Facets": {
"nested": {
"path": "categories"
},
"aggs": {
"categories": {
"terms": {
"field": "categories.id"
}
}
}
},
"Facets2": {
"nested": {
"path": "brand"
},
"aggs": {
"brand": {
"terms": {
"field": "brand.id"
}
}
}
}
}
}

Related

Nest NamedFilters requires promise

I have simple elastic request:
var request4 =
_client.Search<T>(s => s
.Aggregations(aggs =>
aggs.Filters("FacetedSearch",
f => f
.NamedFilters(g =>
{
var namedFilters = new NamedFiltersContainer();
foreach (var facet in _facets)
{
namedFilters.Add(facet.Key,
Query<T>.Terms(p => new Nest.TermsQuery
{Field = facet.Key, Terms =
new[] {facet.Value}}));
}
return namedFilters;
})
However, compiler requires Nest.IPromise<Nest.INamedFiltersContainer> I haven't found any constructor which could construct a promise of a given type. Is there any way to cast the NamedFiltersContainer into a prmosie ?
You can use the parameters passed to the delegates to compose the named filters
var client = new ElasticClient();
var _facets = new Dictionary<string, string>
{
["foo"] = "bar",
["baz"] = "quux"
};
var request4 = client.Search<object>(s => s
.Aggregations(aggs =>
aggs.Filters("FacetedSearch", f => f
.NamedFilters(g =>
{
foreach (var facet in _facets)
{
g.Filter(facet.Key, Query<object>.Terms(p => p
.Field(facet.Key)
.Terms(new[] { facet.Value })
)
);
}
return g;
})
)
)
);
yields
{
"aggs": {
"FacetedSearch": {
"filters": {
"filters": {
"foo": {
"terms": {
"foo": ["bar"]
}
},
"baz": {
"terms": {
"baz": ["quux"]
}
}
}
}
}
}
}

Translate ElasticSearch query to Nest c#

I need some help in creating an AggregationDictionary from the following elasticsearch query
GET organisations/_search
{
"size": 0,
"aggs": {
"by_country": {
"nested": {
"path": "country"
},
"aggs": {
"by_country2": {
"filter": {
"bool": {
"must": [
{
"term": {
"country.isDisplayed": "true"
}
}
]
}
},
"aggs": {
"by_country3": {
"terms": {
"field": "country.displayName.keyword",
"size": 9999
}
}
}
}
}
}
}
}
I managed to write this horrible piece of code which I am pretty sure it is wrong, I am totally new to this.
AggregationDictionary aggs = new AggregationDictionary()
{
{
"countries_step1",
new NestedAggregation("countries_step1")
{
Path = "country",
Aggregations = new AggregationDictionary()
{
{
"countries_step2",
new FilterAggregation("countries_step2")
{
Filter = new BoolQuery
{
Must = new QueryContainer[] {
new NestedQuery
{
Query = new TermQuery
{
Field = "country.isDisplayed",
Value = true
}
}
}
},
Aggregations = new AggregationDictionary
{
{
"countries_step3",
new TermsAggregation("countries_step3")
{
Field = "country.displayName.keyword",
Size = 9999
}
}
}
}
}
}
}
}
};
Can someone tell me if I am in the correct direction? I am using Nest 6.6.0. Is there any tool that helps with these translations?
What you have so far is pretty solid, but when you try to execute this aggregation with the following call
var searchAsync = await client.SearchAsync<Document>(s => s.Size(0).Aggregations(aggs));
you will get this error
{
"error" : {
"root_cause" : [
{
"type" : "illegal_argument_exception",
"reason" : "query malformed, empty clause found at [14:22]"
}
],
"type" : "illegal_argument_exception",
"reason" : "query malformed, empty clause found at [14:22]"
},
"status" : 400
}
Checking request which was sent to elasticsearch give us the answer why it happened
{
"aggs": {
"countries_step1": {
"aggs": {
"countries_step2": {
"aggs": {
"countries_step3": {
"terms": {
"field": "country.displayName.keyword",
"size": 9999
}
}
},
"filter": {}
}
},
"nested": {
"path": "country"
}
}
},
"size": 0
}
filter clause is empty, this is because you tried to used nested query but you didn't pass path parameter. We don't need nested query here (as shown in your example query), we can simplify the whole query to
var aggs = new AggregationDictionary()
{
{
"countries_step1",
new NestedAggregation("countries_step1")
{
Path = "country",
Aggregations = new AggregationDictionary()
{
{
"countries_step2",
new FilterAggregation("countries_step2")
{
Filter = new BoolQuery
{
Must = new QueryContainer[]
{
new TermQuery
{
Field = "country.isDisplayed",
Value = true
}
}
},
Aggregations = new AggregationDictionary
{
{
"countries_step3",
new TermsAggregation("countries_step3")
{
Field = "country.displayName.keyword",
Size = 9999
}
}
}
}
}
}
}
}
};
Now we have a valid request sent to elasticsearch.
There are a couple of things we can improve here:
1. Remove unnecessary bool query
Filter = new BoolQuery
{
Must = new QueryContainer[]
{
new TermQuery
{
Field = "country.isDisplayed",
Value = true
}
}
},
to
Filter =
new TermQuery
{
Field = "country.isDisplayed",
Value = true
},
2. Replace string field names
Usually, when doing calls from .Net there is some kind of POCO type which is helping us with writing strongly-typed requests to elasticsearch which helps us managing clean code and refactoring. With this, we can change field definition from
"country.displayName.keyword"
to
Infer.Field<Document>(f => f.Country.FirstOrDefault().DisplayName.Suffix("keyword"))
my types definition
public class Document
{
public int Id { get; set; }
[Nested]
public List<Country> Country { get; set; }
}
public class Country
{
public bool IsDisplayed { get; set; }
public string DisplayName { get; set; }
}
3. Consider using a fluent syntax
With NEST you can write queries in two ways: using object initializer syntax (which you did) or with help of fluent syntax. Have a look. Trying to write above query with the fluent syntax you will get something like
var searchResponse = await client.SearchAsync<Document>(s => s
.Size(0)
.Aggregations(a => a.Nested("by_country", n => n
.Path(p => p.Country)
.Aggregations(aa => aa
.Filter("by_country2", f => f
.Filter(q => q
.Term(t => t
.Field(field => field.Country.FirstOrDefault().IsDisplayed)
.Value(true)))
.Aggregations(aaa => aaa
.Terms("by_country3", t => t
.Field(field => field.Country.FirstOrDefault().DisplayName.Suffix("keyword"))
.Size(9999)
)))))));
which I find a little bit easier to follow and write, maybe it will be better for you as well.
As a final note, have a look into docs and check how you can debug your queries.
Hope that helps.

Elasticsearch/Nest - Combining multiple range queries (OIC syntax)

ES newbie here (transitioning over from the SolrNet world, so I am looking for pointers with a similar syntax)
Can anyone help with formulating a query that might look something like the code snippet below without the lambdas. Unfortunately there are very few examples in the plain old OI syntax and I have been stuck....
How would I formulate a set of boolean range queries, in solrnet I would do something like this to obtain items that fall in a certain price band:
List<ISolrQuery> queryList = new List<ISolrQuery>();
double[] priceList = GetPrices(..) //double array
for (int i = 1; i <= 10; i++)
{
queryList.Add(new solr.Query(new SolrQueryByRange<double>("price", priceList[i] * 0.95, priceList[i] * 1.15));
}
var results = _solr.Query(new SolrMultipleCriteriaQuery(queryList), new QueryOptions
{
Rows = 50,
Fields = new[] { "Item", "Created", "Price" },
});
Could anyone help with equivalent Elasticsearch/Nest query?
Something like the following
void Main()
{
var client = new ElasticClient(connection: new InMemoryConnection());
// prices from your GetPrices method...
var prices = new[] { 1d, 2d };
var searchRequest = new SearchRequest<Document>("index", "document")
{
Fields = new List<Nest.PropertyPathMarker>{ "Item", "Created", "Price" },
Size = 50,
Query = new BoolQuery
{
Should = prices.Select(p => new RangeQuery
{
Field = "price",
GreaterThanOrEqualTo = (p * 0.95).ToString(),
LowerThanOrEqualTo = (p * 1.15).ToString()
}.ToContainer())
}.ToContainer()
};
var result = client.Search<Document>(searchRequest);
Console.WriteLine(Encoding.UTF8.GetString(result.RequestInformation.Request));
}
public class Document
{
public string Item { get; set; }
public DateTime Created { get; set;}
public double Price {get; set;}
}
This searches for"document" types in index "index" (strongly typed to your Document POCO), using a bool query and should clauses of ranges constructed from the prices - if a document's price field falls into any one of the ranges, it will be considered a match.
The resulting query DSL from the NEST query above is
{
"size": 50,
"fields": [
"Item",
"Created",
"Price"
],
"query": {
"bool": {
"should": [
{
"range": {
"price": {
"gte": "0.95",
"lte": "1.15"
}
}
},
{
"range": {
"price": {
"gte": "1.9",
"lte": "2.3"
}
}
}
]
}
}
}

NEST ElasticSearch Guid problems

I have got some problems to filter my queries on field Guid. Here a sample of my code. What did I miss?
public class myObject
{
public Guid Id {get;set}
public String Field1 { get; set; }
public String Field2 { get; set; }
public String Fieldn { get; set; }
public ReadingRightsEnum ReadingRights { get; set; }
public Guid UserId { get; set; }
}
// Index fct example
public void IndexMyObject(obj)
{
var result = await myClient.IndexAsync(obj, i => i
.Index("myIndexName")
.Type("myObject")
.Id(obj.Id.ToString())
.Refresh());
}
// Index fct example
public void SearchOnlyInMyUserObject(string userQuery, Guid userId)
{
var searchResult = await myClient.SearchAsync<myObject>(body =>
body.Query(q =>
q.QueryString(qs => qs.MinimumShouldMatchPercentage(100).Query(userQuery))
&& q.Term(i => i.UserId, userId))
.Fields(f => f.Id)
.Size(200));
}
// Index fct example with filter
public void SearchOnlyInMyUserObject(string userQuery, Guid userId)
{
var filters = new List<FilterContainer>
{
new FilterDescriptor<myObject>().Bool(b => b.Must(m => m.Term(i => i.UserId, userId)));
};
var searchResult = await myClient.SearchAsync<myObject>(body =>
body
.Filter(f => f.And(filters.ToArray()))
.Query(q =>
q.QueryString(qs => qs.MinimumShouldMatchPercentage(100).Query(userQuery)))
.Fields(f => f.Id)
.Size(200));
}
Both functions work fine if I filter on others parameters but return nothing when I filter on Guid. Should a convert my Guid to string when I index my object?
If i do http://xxxxxxx:9200/_search?q=userId:e4aec7b4-c400-4660-a09e-a0ce064f612e it's work fine.
Any ideas?
Thanks by advance
Edit 06/12 here a sample of myindex:
myIndexName":{
"mappings":{
"myObject":{
"properties":{
"readingrights":{
"type":"integer"
},
"id":{
"type":"string"
},
"field1":{
"type":"string"
},
"field2":{
"type":"string"
},
"userId":{
"type":"string"
}
}
}
}
}
GUID field is tricky in Elastic. If you use analyze function of elastic client it will show you how it breaks up a GUID.
AnalyzeRequest obj = new AnalyzeRequest(_index, item);
_client.Analyze(obj);
When you create an entity, You need to define guid as not be analyzed.
[String(Index = FieldIndexOption.NotAnalyzed)]
public Guid TextAssetId
I didn't fine for now why Guid got problems... but I fine a poor alternative way for now:
Instead of :
q.QueryString(qs => qs.MinimumShouldMatchPercentage(98).Query(userQuery))
&& q.Term(i => i.UserId, userId)
I do a double QueryString:
q.QueryString(qs => qs.MinimumShouldMatchPercentage(98).Query(userQuery))
&& q.QueryString(qs => qs.MinimumShouldMatchPercentage(100).Query(" \"" + userId+ "\""))

NEST Returns Nothing on Fuzzy Search

I use the following query in Sense and I get some results back:
POST myindex/mytype/_search
{
"query": {
"fuzzy_like_this_field" : {
"BookLabel" : {
"like_text" : "myBook",
"max_query_terms" : 12
}
}
}
}
But with the following code using Nest I get nothing:
var docs = client.Search<dynamic>(b => b
.Index("myindex")
.Type("mytype")
.Query(q => q
.Fuzzy(fz => fz
.OnField("BookLabel")
.Value("myBook")
)
)
).Documents.ToList();
I can't see the difference between them. What am I missing?
You NEST query you have above produces the following query DSL
{
"query": {
"fuzzy": {
"BookLabel": {
"value": "myBook"
}
}
}
}
To get the nearest equivalent to a fuzzy_like_this_field query (which is deprecated in Elasticsearch 1.6.0 and will be removed in 2.0), you can run a fuzzy_like_this query only on the field you're interested in
void Main()
{
var settings = new ConnectionSettings(new Uri("http://localhost:9200"));
var connection = new InMemoryConnection(settings);
var client = new ElasticClient(connection: connection);
var docs = client.Search<dynamic>(b => b
.Index("myindex")
.Type("mytype")
.Query(q => q
.FuzzyLikeThis(fz => fz
.LikeText("myBook")
.MaxQueryTerms(12)
.OnFields(new [] { "BookLabel" })
)
)
);
Console.WriteLine(Encoding.UTF8.GetString(docs.RequestInformation.Request));
}
This outputs the following query DSL
{
"query": {
"flt": {
"fields": [
"BookLabel"
],
"like_text": "myBook",
"max_query_terms": 12
}
}
}
which should yield the same results as you see in Sense.

Resources