QueryContainerDescriptor vs QueryContainer vs QueryBase - elasticsearch

Can anyone explain what is the difference between QueryContainerDescriptor, QueryContainer & QueryBase?
How can I assign a query (or QueryBase) to QueryContainer?
In the code below, I can assign the same TermQuery to QueryBase and QueryContainer objects:
QueryBase bq = new TermQuery
{
Field = Field<POCO>(p => p.Title),
Value = "my_title"
};
QueryContainer tq = new TermQuery
{
Field = Field<POCO>(p => p.Title),
Value = "my_title"
};
Also I am not sure if there is any difference between, creating a TermQuery using QueryContainerDescriptor and the above method?
QueryContainer qcd = new QueryContainerDescriptor<POCO>().
Term(r => r.Field(f => f.Title).Value("my_title"));

QueryBase is the base type for all concrete query implementations
QueryContainer is a container for a query. It is used in places where a query is expected.
QueryContainerDescriptor<T> is a type for building a QueryContainer using a builder / fluent interface pattern.
NEST supports both an Object Initializer syntax where requests can be composed through instantiating types and composing an object graph by assigning types to properties, and also a Fluent API syntax, where requests can be composed using Lambda expressions and a fluent interface pattern. All *Descriptor types within NEST are builders for the Fluent API syntax. Use whichever syntax you prefer, or mix and match as you see fit :)
You might be thinking, why do we need QueryContainer, why not just use QueryBase? Well, within the JSON representation, a query JSON object is keyed against the name of the query as a property of an outer containing JSON object i.e.
{
"query": { // <-- start of outer containing JSON object
"term": { // <-- start of JSON query object
"field": {
"value": "value"
}
}
}
}
Relating back to C# types, QueryBase will be serialized to the query JSON object and QueryContainer will be the outer containing JSON object. To make it easier to compose queries, there are implicit conversions from QueryBase to QueryContainer, so often you just need to instantiate a derived QueryBase implementation and assign it to a property of type QueryContainer
var client = new ElasticClient();
var termQuery = new TermQuery
{
Field = "field",
Value = "value"
};
var searchRequest = new SearchRequest<MyDocument>
{
Query = termQuery // <-- Query property is of type QueryContainer
};
var searchResponse = client.Search<MyDocument>(searchRequest);
With QueryContainerDescriptor<T>, you often don't need to instantiate an instance outside of the client call, as an instance will be instantiated within the call. Here's the same request with the Fluent API
client.Search<MyDocument>(s => s
.Query(q => q
.Term("field", "value")
)
);

Related

Dropdowns driven by ElasticSearch aggregations

Currently have dropdowns used in a search component that are driven by ES using terms aggregations. A dropdown when selected should filter the other dropdowns but not itself any further. I am currently essentially doing two searches to accomplish this and collecting aggregations from both. first is size 0 with the given query excluding the most recently applied filters:
(simplified for the post)
// for aggregation on last filter selected
var newestFilterResponse = await _client.SearchAsync<Index>(s => s
.Index(_index)
.From(0)
.Size(0)
.Query(QueryClosure(phrase, fields, filtersMinusLast))
.Aggregations(FilterOptionsAggregationClosure(lastFilter)));
// actual query for results and aggregation on other filters
var response = await _client.SearchAsync<Index>(s => s
.Index(_index)
.From(0)
.Size(500)
.Query(QueryClosure(phrase, fields, allFilters))
.Aggregations(FilterOptionsAggregationClosure(allFiltersExceptLast)));
use below to build terms aggregations for each dropdown
Func<AggregationContainerDescriptor<Index>, IAggregationContainer> FilterOptionsAggregationClosure(List<FilterButton> filterButtons)
{
return delegate (AggregationContainerDescriptor<Index> aggregationContainerDescriptor)
{
foreach (var filterButton in filterButtons)
{
aggregationContainerDescriptor = aggregationContainerDescriptor
.Terms(filterButton.AggregationName, t => t
.Field(filterButton.FieldToSearch)
.Size(10000)
);
}
return aggregationContainerDescriptor;
};
}
use below to build query (simplified to just filter for example, but there is multimatch, prefix and some more in my actual use case)
Func<QueryContainerDescriptor<Index>, QueryContainer> QueryClosure(string phrase, Fields fields, List<FilterApplied> filtersApplied)
{
return delegate (QueryContainerDescriptor<Index> queryContainerDescriptor)
{
// multimatch, prefix, etc. on phrase for fields supplied
QueryContainer queryContainer = GetKeywordQuery(phrase, fields);
QueryContainer filterQuery = new();
foreach (var filter in filtersApplied)
{
QueryContainer sameFilterQuery = new QueryContainerDescriptor<Index>()
.Bool(boolQuery => boolQuery
.Filter(f => f
.Terms(terms => terms
.Field(filter.FieldToSearch)
.Terms(filter.Values))));
filterQuery = filterQuery && sameFilterQuery;
}
return queryContainer && filterQuery;
};
}
i feel like this is not the way. Any suggestions to improve on this would be greatly appreciated. thanks!
Ended up caching dropdown options in session storage, we wanted to have url params be able to generate exact state of page when sharing links but ended up ditching that with regards to filter dropdown state so we could eliminate the need to do the isolated aggregation on the last filter type applied

How can i construct a NEST query with optional parameters?

I'm using the NEST .NET client (6.3.1), and trying to compose a search query that is based on a number of (optional) parameters.
Here's what i've got so far:
var searchResponse = await _client.SearchAsync<Listing>(s => s
.Query(qq =>
{
var filters = new List<QueryContainer>();
if (filter.CategoryType.HasValue)
{
filters.Add(qq.Term(p => p.CategoryType, filter.CategoryType.Value));
}
if (filter.StatusType.HasValue)
{
filters.Add(qq.Term(p => p.StatusType, filter.StatusType.Value));
}
if (!string.IsNullOrWhiteSpace(filter.Suburb))
{
filters.Add(qq.Term(p => p.Suburb, filter.Suburb));
}
return ?????; // what do i do her?
})
);
filter is an object with a bunch of nullable properties. So, whatever has a value i want to add as a match query.
So, to achieve that i'm trying to build up a list of QueryContainer's (not sure that's the right way), but struggling to figure out how to return that as a list of AND predicates.
Any ideas?
Thanks
Ended up doing it by using the object initialisez method, instead of the Fluent DSL"
var searchRequest = new SearchRequest<Listing>
{
Query = queries
}
queries is a List<QueryContainer>, which i just build up, like this:
queries.Add(new MatchQuery
{
Field = "CategoryType",
Query = filter.CategoryType
}
I feel like there's a better way, and i don't like how i have to hardcode the 'Field' to a string... but it works. Hopefully someone shows me a better way!

elasticsearch NEST : get TopHits result directly without using bucket.TopHits()

With nest I am doing a Terms aggregation .
I am also doing an inner TopHits aggregation .
My result give me all results infos in the response object except TopHits values which i can read thanks to TopHits() method.
I would like to have tophits values directly in result without using NEST TopHits() methode for reading into aggs. I would like to have all datas in info as we have in elastic search classic requests.
This is what i am actually doing :
My aggregation request:
var response = Client.Search<myclass>(s => s
.Type("type")
.Aggregations(a => a
.Terms("code_bucket", t => t
.Field("field_of_aggregation")
.Size(30)
.Order(TermsOrder.CountAscending)
.Aggregations(a2 => a2
.TopHits("code_bucket_top_hits", th => th.Size(20))
)
)));
I receive a result object in wich i can access all infos except TopHits.
if we examine results we can see TopHits values are stored in private field "_hits":
If I stringify result object , i can see the total number of TopHits, but I can't see the field _hits so i can see the documents:
JavaScriptSerializer js = new JavaScriptSerializer();
string json = js.Serialize(response);
json does not contains topHits result:
I can access to values but i need to use nest method TopHits():
var firstBucket= response.Aggs.Terms("code_bucket");
foreach (var bucket in firstBucket.Buckets)
{
var hits = bucket.TopHits("code_bucket_top_hits");
foreach (var hit in hits.Documents<myclass>())
{
var prop1= hit.prop1;
var prop2= hit.prop2;
}
}
}
But it would be really usefule if i could have all infos in one , like we have when we do elasticsearch request without nest
Do you know if there is a way?
NEST is a higher level abstraction over Elasticsearch that models each request and response with strong types, providing fluent and object initializer syntaxes to build requests, and methods to access portions of the response, without having to handle JSON serialization yourself.
Sometimes however, you might want to manage this yourself, which is what it sounds like you'd like to do. In these cases, Elasticsearch.Net can be used, which is a low level client for Elasticsearch and is unopinionated in how you model your requests and responses.
You can use the client in Elasticsearch.Net instead of NEST, however, the good news is NEST uses Elasticsearch.Net under the covers and also exposes the low level client through the .LowLevel property on IElasticClient. Why would you want to use the lowlevel client on NEST as opposed to just using Elasticsearch.Net directly? A major reason to do so is that you can take advantage of strong types for requests and responses when you need to and leverage NEST's usage of Json.NET for serialization, but bypass this and make calls with the low level client when you want/need to.
Here's an example
var client = new ElasticClient();
var searchRequest = new SearchRequest<Question>
{
Size = 0,
Aggregations = new TermsAggregation("top_tags")
{
Field = "tags",
Size = 30,
Order = new[] { TermsOrder.CountAscending },
Aggregations = new TopHitsAggregation("top_tag_hits")
{
Size = 20
}
}
};
var searchResponse = client.LowLevel.Search<JObject>("posts", "question", searchRequest);
// this will be of type JObject. Do something with it
searchResponse.Body
Here, I can use NEST's object initializer syntax to construct a request, but use the low level client to deserialize the response to a Json.NET JObject. You can deserialize to a T of your choosing by changing it in client.LowLevel.Search<T>(). You could for example use
var searchResponse = client.LowLevel.Search<string>("posts", "question", searchRequest);
to return a string, or
var searchResponse = client.LowLevel.Search<Stream>("posts", "question", searchRequest);
to return a stream, etc.

Bulk Update on ElasticSearch using NEST

I am trying to replacing the documents on ES using NEST. I am seeing the following options are available.
Option #1:
var documents = new List<dynamic>();
`var blkOperations = documents.Select(doc => new BulkIndexOperation<T>`(doc)).Cast<IBulkOperation>().ToList();
var blkRequest = new BulkRequest()
{
Refresh = true,
Index = indexName,
Type = typeName,
Consistency = Consistency.One,
Operations = blkOperations
};
var response1 = _client.Raw.BulkAsync<T>(blkRequest);
Option #2:
var descriptor = new BulkDescriptor();
foreach (var eachDoc in document)
{
var doc = eachDoc;
descriptor.Index<T>(i => i
.Index(indexName)
.Type(typeName)
.Document(doc));
}
var response = await _client.Raw.BulkAsync<T>(descriptor);
So can anyone tell me which one is better or any other option to do bulk updates or deletes using NEST?
You are passing the bulk request to the ElasticsearchClient i.e. ElasticClient.Raw, when you should be passing it to ElasticClient.BulkAsync() or ElasticClient.Bulk() which can accept a bulk request type.
Using BulkRequest or BulkDescriptor are two different approaches that are offered by NEST for writing queries; the former uses an Object Initializer Syntax for building up a request object while the latter is used within the Fluent API to build a request using lambda expressions.
In your example, BulkDescriptor is used outside of the context of the fluent API, but both BulkRequest and BulkDescriptor implement IBulkRequest so can be passed to ElasticClient.Bulk(IBulkRequest).
As for which to use, in this case it doesn't matter so whichever you prefer.

IQueryable to IQueryable<T>

is it possibile convert an IQueryable object to IQueryable where T is a mapped entity? (T will be a POCO class).
Thanks in advance.
Just Cast<T>() it. Assuming it is a queryable of the same type. Otherwise you could use the OfType<T>() filtering method to filter out items of a certain type.
IQueryable query = ...;
IQueryable<MyType> x = query.Cast<MyType>(); // assuming the queryable is of `MyType` objects
IQueryable<MyDerivedType> y = query.OfType<MyDerivedType>(); // filter out objects derived from `MyType` (`MyDerivedType`)
However in your case, you say that you are using Dynamic LINQ and doing a dynamic projection. Consider this completely made up query:
var query = dc.SomeTable
.Where("SomeProperty = \"foo\"")
.Select("new (SomeProperty, AnotherProperty)");
It results in a query of type IQueryable. You cannot cast this to a query of a specific type IQueryable<T> after all, what is T? What the Dynamic LINQ library does is creates a type that derives from DynamicCass. You could cast to IQueryable<DynamicClass> (query.Cast<DynamicClass>()) but you will not have access to the properties so it's moot.
Really the only nice option you have is to use dynamic to access these properties in this case.
foreach (dynamic x in query)
{
string someProperty = x.SomeProperty;
int anotherProperty = x.AnotherProperty;
// etc...
}
If you want to convert this to a query of your POCO objects, you'll have to do the conversion as a separate step but using LINQ to Objects.
IEnumerable<SomePoco> query =
dc.SomeTable
.Where("SomeProperty = \"foo\"")
.Select("new (SomeProperty, AnotherProperty)")
.Cast<DynamicObject>().AsEnumerable().Cast<dynamic>()
.Select(x => new SomePoco
{
SomeProperty = x.SomeProperty,
AnotherProperty = x.AnotherProperty,
});
If you must have an IQueryable<T>, then you should not use dynamic projections in the first place.
IQueryable<SomePoco> query =
dc.SomeTable
.Where("SomeProperty = \"foo\"")
.Select(x => new SomePoco
{
SomeProperty = x.SomeProperty,
AnotherProperty = x.AnotherProperty,
});
Seeing as how the cast is not working for LINQ to Entities, then I suppose the only option you have to get a strongly type collection of your POCO objects is to break this out into a loop.
var query = dc.SomeTable
.Where("SomeProperty = \"foo\"")
.Select("new (SomeProperty, AnotherProperty)");
var result = new List<SomePoco>();
foreach (dynamic x in query)
{
result.Add(new SomePoco
{
SomeProperty = x.SomeProperty,
AnotherProperty = x.AnotherProperty,
});
}

Resources