I'd like to append multiple bool filters with NEST, but I can't (practically) do it in a single statement as I want to build a set of Bool filters depending on different conditions.
Something like this pseudo code:
// Append a filter
searchDescriptor.Filter(f => f.Bool(b => b.Must(m => m.Term(i => i.SomeProperty, "SomeValue"))));
if (UserId.HasValue)
{
// Optionally append another filter (AND condition with the first filter)
searchDescriptor.Filter(f => f.Bool(b => b.Must(m => m.Term(i => i.AnotherProperty, "MyOtherValue"))));
}
var result = Client.Search(searchDescriptor);
Now it seems when the second optional filter is appended, it essentially replaces the first filter.
I'm sure I'm missing something syntactically, but I can't figure it out and the NEST documentation is a bit thin on the filter DSL. :)
The section on writing queries pretty much applies to filters as well:
https://www.elastic.co/guide/en/elasticsearch/client/net-api/current/writing-queries.html
The solution you ended up with is less than ideal since you are wrapping bool filters inside an and filter which have very different caching mechanisms (and filters don't use the cached bitsets so in most cases perform worse then regular bool filters).
Highly recommend reading http://www.elastic.co/blog/all-about-elasticsearch-filter-bitsets/ as it explains really well what the differences between and/or/not filters vs bool filters are.
You can actually rewrite this like so:
Client.Search(s=>s
.Filter(f=> {
BaseFilter ff = f.Term(i => i.MyProperty, "SomeValue");
if (UserId.HasValue)
ff &= f.Term(i => i.AnotherProperty, "AnotherValue");
return ff;
})
);
If the second term is searching using UserId you can take advantage of NEST's conditionless queries
Client.Search(s=>s
.Filter(f=>
f.Term(i => i.MyProperty, "SomeValue")
&& f.Term(i => i.AnotherProperty, UserId)
)
);
If UserId is null then the term query won't be generated as part of the query, nest in this case won't even wrap the single remaining term in a bool filter but just send it as a plain term filter instead.
Ah, something like this seems to work:
var filters = new List<BaseFilter>();
// Required filter
filters.Add(new FilterDescriptor<MyType>().Bool(b => b.Must(m => m.Term(i => i.MyProperty, "SomeValue"))));
if (UserId.HasValue)
{
filters.Add(new FilterDescriptor<MyType>().Bool(b => b.Must(m => m.Term(i => i.AnotherProperty, "AnotherValue"))));
}
// Filter with AND operator
searchDescriptor.Filter(f => f.And(filters.ToArray()));
var result = Client.Search(searchDescriptor);
Related
Below is the NEST query and aggregations:
Func<QueryContainerDescriptor<ConferenceWrapper>, QueryContainer> query =
q =>
q.Term(p => p.type, "conference")
// && q.Term(p => p.conference.isWaitingAreaCall, true)
&& q.Range(d => d.Field("conference.lengthSeconds").GreaterThanOrEquals(minSeconds))
&& q.DateRange(qd => qd.Field("conference.firstCallerStart").GreaterThanOrEquals(from.ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ss.fffZ")))
&& q.DateRange(qd => qd.Field("conference.firstCallerStart").LessThanOrEquals(to.ToUniversalTime().ToString("yyyy-MM-ddTHH:mm:ss.fffZ")));
Func<AggregationContainerDescriptor<ConferenceWrapper>, IAggregationContainer> waitingArea =
a => a
.Terms("both", t => t
.Field(p => p.conference.orgNetworkId) // seems ignore this field
.Field(p => p.conference.isWaitingAreaCall)
// .Field(new Field( p => p.conference.orgNetworkId + "-ggg-" + p.conference.networkId)
.Size(300)
.Aggregations(a2 => a2.Sum("sum-length", d2 => d2.Field("conference.lengthSeconds"))));
I have called .Field(p => p.conference.orgNetworkId) followed by .Field(p => p.conference.isWaitingAreaCall) But it seems the NEST client tries to ignore the first field expression.
Is is possible to have multiple fields to be the terms group by?
Elasticsearch doesn't support a terms aggregation on multiple fields directly; the calls to .Field(...) within NEST are assignative rather than additive, so the last call will overwrite any previously set values.
In order to aggregate on multiple fields, you can either
Create a composite field at index time that incorporates the values that you wish to aggregate on
or
Use a Script to generate the terms on which to aggregate at query time, by combining the two field values.
The performance of the first option will be better than the second.
I've got an Observable that listens to some user input from a text box. If the observed string's length is >=3 (filter), it executes some HTTP call (switchMap).
Now I'd like to detect somehow if the user input has been filtered. Reason:
If the HTTP call has been done, it should show the results.
If the user input got filtered (== is invalid), it should clear the results.
Here's the code I'd like to have (see: ifFiltered):
this.userInput.valueChanges
.filter(val => val && val.length >= 3)
.ifFiltered(() => this.results = [])
.switchMap(val => getDataViaHTTP())
.subscribe(val => this.results = val);
I know, I could place that logic within the filter function for this simple example. But what if I have 10 different filters?
Did I miss any method that satisfies my needs?
Thanks in advance!
Either use partition like here RxJS modeling if else control structures with Observables operators
Or instead of filter use map and pipe the object if the former filter condition is true or null otherwise. so you can catch the null where ever you want in your chain with a filter.
Last option call some function in the else part of the filter function
We've had a similar case and tried it with partition as mentioned above but found it much handier to use throw here. So for your code
this.userInput.valueChanges
.do(val => {
if (!val || val.length < 3) {
throw new ValueTooShortError();
}
})
.switchMap(val => getDataViaHTTP())
.do(val => this.results = val)
.catch(() => this.results = [])
.subscribe();
I suggest having a common event stream, creating two filtered streams, and merging the two before subscription:
var o = this.userInput.valueChanges;
var empty= o.filter(t=> t.length < 3)
.map(t=>[])
var nonempty = o.filter(t=> t.length >= 3)
.switchMap(t=> getDataViaHTTP());
empty.merge(nonempty).subscribe(val => this.results = val);
I found another nice solution for my use case using Validators:
(I know that this is no solution using Observables as the question stated. Instead it's using Angular2 features to workaround the problem nicely.)
this.userInput.validator = Validators.compose([
Validators.required,
Validators.minLength(3)
]);
this.userInput.valueChanges
.filter(val => this.userInput.valid)
.switchMap(val => getDataViaHTTP())
.subscribe(val => this.results = val);
Now I can use the userInput.valid property and/or the userInput.statusChanges Observable to keep track of the input value.
May be it's late, but wanted to post for the members still seeking a more cleaner approach to validate IF EMPTY inside .map:
of(fooBar).pipe(
map(
(val) =>
({
...val,
Foo: (val.Bar
? val.Foo.map((e) => ({
title: e.Title,
link: e.Link,
}))
: []) as fooModal[],
}));
This code returns a empty array if val.bar is missing, but it's just an example you can use any validation & expression instead.
i want to be able to dynamically add fields for highlighting in elasticsearch using nest. currently it looks like it's not a able to be iterated in any fashion.
i've tried iterating within the .OnFields function in order to produce a list of .OnField functions, but it says it's not iterable.
in this example, i want to dynamically add 'artist' and 'title' and add/remove others based on user input. is this possible?
s.Highlight(h => h
.OnFields(f => f
.OnField("artist")
.OnField("title")
.PreTags("<em>")
.PostTags("</em>")
));
Highlight takes an array of Action<HighlightFieldDescriptor<T>>. You are only passing a single Action<HighlightFieldDescriptor<T>> and calling OnField multiple times on it, which keeps replacing the last value.
It should be this instead:
s.Highlight(h => h
.OnFields(
f => f.OnField("artist").PreTags("<em>").PostTags("</em>"),
f => f.OnField("title").PreTags("<em>").PostTags("</em>")
));
From the code in your follow up post, here's a solution using LINQ:
s.Highlight(h => h
.OnFields(
SearchFields(searchDescriptor.SearchModifier).Select(x => new Action<HighlightFieldDescriptor>(f => f.OnField(x))).ToArray()
));
i realized i had confused a couple of types:
HighlightFieldDescriptor and HighlightDescriptor. sorry. here's my implementation (so i can mark as answered)
s.Highlight(h => h
.OnFields(f =>
GetFieldsHighligthDescriptor(searchDescriptor, f)
)
);
private void GetFieldsHighligthDescriptor(SearchQueryDescriptor searchDescriptor, HighlightFieldDescriptor<Product> f)
{
foreach (var b in SearchFields(searchDescriptor.SearchModifier))
{
f.OnField(b);
}
}
EDIT
actually, this isn't working because it's only return the last entry in my SearchFields array... back to the drawing board?
I am trying to create table headers that sort during a back end call in nhibernate. When clicking the header it sends a string indicating what to sort by (ie "Name", "NameDesc") and sending it to the db call.
The db can get quite large so I also have back end filters and pagination built into reduce the size of the retrieved data and therefore the orderby needs to happen before or at the same time as the filters and skip and take to avoid ordering the smaller data. Here is an example of the QueryOver call:
IList<Event> s =
session.QueryOver<Event>(() => #eventAlias)
.Fetch(#event => #event.FiscalYear).Eager
.JoinQueryOver(() => #eventAlias.FiscalYear, () => fyAlias, JoinType.InnerJoin, Restrictions.On(() => fyAlias.Id).IsIn(_years))
.Where(() => !#eventAlias.IsDeleted);
.OrderBy(() => fyAlias.RefCode).Asc
.ThenBy(() => #eventAlias.Name).Asc
.Skip(numberOfRecordsToSkip)
.Take(numberOfRecordsInPage)
.List();
How can I accomplish this?
One way how to achieve this (one of many, because you can also use some fully-typed filter object etc or some query builder) could be like this draft:
Part one and two:
// I. a reference to our query
var query = session.QueryOver<Event>(() => #eventAlias);
// II. join, filter... whatever needed
query
.Fetch(#event => #event.FiscalYear).Eager
var joinQuery = query
.JoinQueryOver(...)
.Where(() => !#eventAlias.IsDeleted)
...
Part three:
// III. Order BY
// Assume we have a list of strings (passed from a UI client)
// here represented by these two values
var sortBy = new List<string> {"Name", "CodeDesc"};
// first, have a reference for the OrderBuilder
IQueryOverOrderBuilder<Event, Event> order = null;
// iterate the list
foreach (var sortProperty in sortBy)
{
// use Desc or Asc?
var useDesc = sortProperty.EndsWith("Desc");
// Clean the property name
var name = useDesc
? sortProperty.Remove(sortProperty.Length - 4, 4)
: sortProperty;
// Build the ORDER
order = order == null
? query.OrderBy(Projections.Property(name))
: query.ThenBy(Projections.Property(name))
;
// use DESC or ASC
query = useDesc ? order.Desc : order.Asc;
}
Finally the results:
// IV. back to query... call the DB and get the result
IList<Event> s = query
.List<Event>();
This draft is ready to do sorting on top of the root query. You can also extend that to be able to add some order statements to joinQuery (e.g. if the string is "FiscalYear.MonthDesc"). The logic would be similar, but built around the joinQuery (see at the part one)
I have this query that does what I want which is to return true if any of the materials is comparable in the material groups list.
mgroup.MaterialGroups.Select(x => x.Materials
.Any(m => Convert.ToBoolean(m.Comparable)))
.Any(x => x.Equals(true))
What I would like to add to this query is to also include this one.
mgroup.Materials.Any(m => Convert.ToBoolean(m.Comparable));
How can I combine mgroup and it's materialgroups together in the query so I can select both of their Materials? Thanks.
EDIT - After fighting with LINQ for awhile I broke down and just combined as
mgroup.Materials.Any(m => Convert.ToBoolean(m.Comparable) ||
mgroup.MaterialGroups.Select(x => x.Materials
.Any(c => Convert.ToBoolean(c.Comparable)))
.Any(x => x.Equals(true)))
It works as expected but it's horribly long and it's embedded in an Asp.net MVC view to makes things even worse. If anyone can simplify this that would be amazing.
P.S.- If you are wondering why I added the extra .Any(x => x.Equals(true) at the end it's because without it the query returns an IEnumerable of bools instead of bool.
IEnumerable<Material> allMaterials =
mgroup.Materials.Concat(
mgroup.MaterialGroups.SelectMany(group => group.Materials));
bool result = allMaterials.Any(m => Convert.ToBoolean(m.Comparable));