I've been trying to update to ES6 and NEST 6 and running into issues with NEST serializing of search requests - specifically serializing Terms queries where the underlying C# type is an enum.
I've got a Status enum mapped in my index as a Keyword, and correctly being stored in its string representation by using NEST.JsonNetSerializer and setting the contract json converter as per Elasticsearch / NEST 6 - storing enums as string
The issue comes when trying to search based on this Status enum. When I try to use a Terms query to specify multiple values, these values are being serialized as integers in the request and causing the search to find no results due to the type mismatch.
Interestingly the enum is serialized correctly as a string in a Term query, so I'm theorizing that the StringEnumConverter is being ignored in a scenario where it's having to serialize a collection of enums rather than a single enum.
Lets show it a little more clearly in code. Here's the enum and the (simplified) model used to define the index:
public enum CampaignStatus
{
Active = 0,
Sold = 1,
Withdrawn = 2
}
public class SalesCampaignSearchModel
{
[Keyword]
public Guid Id { get; set; }
[Keyword(DocValues = true)]
public CampaignStatus CampaignStatus { get; set; }
}
Here's a snippet of constructing the settings for the ElasticClient:
var pool = new SingleNodeConnectionPool(new Uri(nodeUri));
var connectionSettings = new ConnectionSettings(pool, (builtin, serializerSettings) =>
new JsonNetSerializer(builtin,
serializerSettings,
contractJsonConverters: new JsonConverter[]{new StringEnumConverter()}
)
)
.EnableHttpCompression();
Here's the Term query that correctly returns results:
var singleTermFilterQuery = new SearchDescriptor<SalesCampaignSearchModel>()
.Query(x => x.Term(y => y.Field(z => z.CampaignStatus).Value(CampaignStatus.Active)));
Generating the request:
{
"query": {
"term": {
"campaignStatus": {
"value": "Active"
}
}
}
}
Here's the Terms query that does not return results:
var termsFilterQuery = new SearchDescriptor<SalesCampaignSearchModel>()
.Query(x => x.Terms(y => y.Field(z => z.CampaignStatus).Terms(CampaignStatus.Active, CampaignStatus.Sold)));
Generating the request:
{
"query": {
"terms": {
"campaignStatus": [
0,
1
]
}
}
}
So far I've had a pretty good poke around at the options being presented by the JsonNetSerializer, tried a bunch of the available attributes (NEST.StringEnumAttribute, [JsonConverter(typeof(StringEnumConverter))] rather than using the global one on the client, having an explicit filter object with ItemConverterType set on the collection of CampaignStatuses, etc.) and the only thing that has had any success was a very brute-force .ToString() every time I need to query on an enum.
These are toy examples from a reasonably extensive codebase that I'm trying to migrate across to NEST 6, so what I'm wanting is to be able to specify global configuration somewhere rather than multiple developer teams needing to be mindful of this kind of eccentricity.
So yeah... I've been looking at this for a couple of days now. Good chances there's something silly I've missed. Otherwise I'm wondering if I need to be providing some JsonConverter with a contract that would match to an arbitrary collection of enums, and whether NEST and their tweaked Json.NET serializer should just be doing that kind of recursive resolution out of the box already.
Any help would be greatly appreciated, as I'm going a bit crazy with this one.
Related
Using ElasticSearch.Net v6.0.2
Given the Indexed item
{
"PurchaseFrequency": 76,
"purchaseFrequency": 80
}
and the POCO Object
public class Product
{
public int PurchaseFrequency { get; set; }
}
and the setting
this.DefaultFieldNameInferrer(x => x);
Nest is returning a PurchaseFrequency = 80 even though this is the wrong field.
How can I get NEST to pull the correct cased field from ElasticSearch?
I don't think that this is going to be easily possible because this behaviour is defined in Json.NET, which NEST uses internally (not a direct dependency in 6.x, it's IL-merged into the assembly).
For example,
JsonConvert.DeserializeAnonymousType("{\"a\":1, \"A\":2}", new { a = 0 })
deserializes the anonymous type property a value to 2. But
JsonConvert.DeserializeAnonymousType("{\"A\":2, \"a\":1}", new { a = 0 })
deserializes the anonymous type property a value to 1 i.e. the order of properties as they appear in the returned JSON has a bearing on the final value assigned to a property on an instance of a type.
If you can, avoid JSON property names that differ only in case. If you can't, then you'd need to hook up the JsonNetSerializer in the NEST.JsonSerializer nuget package and write a custom JsonConverter for your type which only honours the exact casing expected.
I am using ElasticSearch and the NEST .Net library for implementing the Search functionality needed in our app. In my model, I have a type that contains Nested objects as per below.
[ElasticType(Name = "x")]
public class X
{
[ElasticProperty(IncludeInAll = false, Index = FieldIndexOption.NotAnalyzed)]
public string Id { get; set; }
[ElasticProperty(Type = FieldType.Nested)]
public List<Y> Ys { get; set; }
}
Any queries executed against X are actually executed against the List of Ys. I would like to highlight the hits in the nested objects and based on https://github.com/elasticsearch/elasticsearch/issues/5245 .
However, in order to use the proposed workaround, the include_in_parent option should be true for the nested object.
How can this option be enabled using the NEST library? Is there any ElasticProperty property (I haven’t found any obvious one) or some other way to do so?
Thank you
Apparently this can be done only by using fluent syntax. For the above case the code would be:
.AddMapping<X>(m => m
.Properties(p => p
.NestedObject<Y>(n => n
.Name("ys")
.IncludeInParent())
I typically do mobile app development, which doesn't always have .Select. However, I've seen this used a bit, but I don't really know what it does or how it's doing whatever it does. It is anything like
from a in list select a // a.Property // new Thing { a.Property}
I'm asking because when I've seen code using .Select(), I was a bit confused by what it was doing.
.Select() is from method syntax for LINQ, select in your code from a in list select a is for query syntax. Both are same, query syntax compiles into method syntax.
You may see: Query Syntax and Method Syntax in LINQ (C#)
Projection:
Projection Operations - MSDN
Projection refers to the operation of transforming an object into a
new form that often consists only of those properties that will be
subsequently used. By using projection, you can construct a new type
that is built from each object. You can project a property and perform
a mathematical function on it. You can also project the original
object without changing it.
You may also see:
LINQ Projection
The process of transforming the results of a query is called
projection. You can project the results of a query after any filters
have been applied to change the type of the collection that is
returned.
Example from MSDN
List<string> words = new List<string>() { "an", "apple", "a", "day" };
var query = from word in words
select word.Substring(0, 1);
In the above example only first character from each string instance is selected / projected.
You can also select some fields from your collection and create an anonymous type or an instance of existing class, that process is called projection.
from a in list select new { ID = a.Id}
In the above code field Id is projected into an anonymous type ignoring other fields. Consider that your list has an object of type MyClass defined like:
class MyClass
{
public int Id { get; set; }
public string Name { get; set; }
public string Address { get; set; }
}
Now you can project the Id and Name to an anonymous type like:
Query Syntax:
var result = from a in list
select new
{
ID = a.Id,
Name = a.Name,
};
Method Syntax
var result = list.Select(r => new { ID = r.Id, Name = r.Name });
You can also project result to a new class. Consider you have a class like:
class TemporaryHolderClass
{
public int Id { get; set; }
public string Name { get; set; }
}
Then you can do:
Query Syntax:
var result = from a in list
select new TemporaryHolderClass
{
Id = a.Id,
Name = a.Name,
};
Method Syntax:
var result = list.Select(r => new TemporaryHolderClass
{
Id = r.Id,
Name = r.Name
});
You can also project to the same class, provided you are not trying to project to classes generated/created for LINQ to SQL or Entity Framework.
My summary is it takes results (or a subset of results) and allows you to quickly restructure it for use in the local context.
The select clause produces the results of the query and specifies the
"shape" or type of each returned element. For example, you can specify
whether your results will consist of complete Customer objects, just
one member, a subset of members, or some completely different result
type based on a computation or new object creation.
Source: http://msdn.microsoft.com/en-us/library/bb397927.aspx
There are a lot of possible uses for this but one is taking a complex object which of many other contains a property that is a string -- say Name -- and allows you to return an enumeration with just the entries of Name. I believe you can also do the opposite -- use that property ( for example) and create / return new type of object while passing in a property or properties.
It means "mapping". Map each element of a sequence to a transformed sequence. I hadn't comprehended its meaning before I looked at the image.
Where does the meaning of the word come from?
Simply, math! https://mathworld.wolfram.com/Projection.html
Using Orchard CMS, I am dealing with a record and a part proxy, but cannot figure out how to save it into the DB. In fact, I confess I don't even know how to get the items I'm trying to save into this paradigm. I was originally using enum's for choices:
MyEmum.cs:
public enum Choices { Choice1, Choice2, Choice3, Choice4 }
MyRecord.cs:
public virtual string MyProperty { get; set; }
MyPart.cs:
public IEnumerable<string> MyProperty
{
get
{
if (String.IsNullOrWhiteSpace(Record.MyProperty)) return new string[] { };
return Record
.MyProperty
.Split(new[] { '.' }, StringSplitOptions.RemoveEmptyEntries)
.Select(r => r.Trim())
.Where(r => !String.IsNullOrEmpty(r));
}
set { Record.MyProperty = value == null ? null : String.Join(",", value); }
}
Now, in my service class, I tried something like:
public MyPart Create(MyPartRecord record)
{
MyPart part = Services.ContentManager.Create<MyPart>("My");
...
part.MyProperty = record.MyProperty; //getting error here
...
return part;
}
However, I am getting the following error: Cannot implicitly convert 'string' to System.Collections.Generic.IEnumerable<string>'
Essentially, I am trying to save choices from a checkboxlist (one or more selections) as a comma-separated list in the DB.
And this doesn't even get me over the problem of how do I use the enum. Any thoughts?
For some background:
I understand that the appropriate way to handle this relationship would be to create a separate table and use IList<MyEnum>. However, this is a simple list that I do not intend to manipulate with edits (in fact, no driver is used in this scenario as I handle this on the front-end with a controller and routes). I am just capturing data and redisplaying it in the Admin view for statistical/historical purposes. I may even consider getting rid of the Part (considering the following post: Bertrand's Blog Post.
It should be:
part.MyProperty = new[] {"foo", "bar"};
for example. The part's setter will store the value on the record's property as a comma-separated string, which will get persisted into the DB.
If you want to use enum values, you should use the Parse and ToString APIs that .NET provide on Enum.
I am developing an ASP.NET MVC3 application in C#.
I am trying to implement in my application a "narrow-down" functionality applied the result-set obtained from a search.
In short, after I perform a search and the results displayed in the center of the page, I would like to have on the left/right side of the page a CheckBoxList helper for each property of the search result. The CheckBox of each CheckBoxList represent the distinct values of the property.
For instance if I search Product and it has a Color property with values blue, red and yellow, I create a CheckBoxList with text Color and three CheckBox-es one for each color.
After a research on the Web I found this Dynamic LINQ library made available by Scott Guthrie. Since the most recent example/tutorial I found is from 2009, I was wondering whether this library is actually good (and maintained) or not.
In the latter case is jQuery the best way to implement such functionality?
You can solve it by building the needed predicate expressions dynamically, using purely .NET framework.
See code sample below. Depending on the criteria, this will filter on multiple properties. I've used IQuerable because this will enable both In-Memory as remote scenario's such as Entity Framework. If you're going with Entity Framework, you could also just build an EntitySQL string dynamically. I expect that will perform better.
There is a small portion of reflection involved (GetProperty). But this could be improved by performing caching inside the BuildPredicate method.
public class Item
{
public string Color { get; set; }
public int Value { get; set; }
public string Category { get; set; }
}
class Program
{
static void Main(string[] args)
{
var list = new List<Item>()
{
new Item (){ Category = "Big", Color = "Blue", Value = 5 },
new Item (){ Category = "Small", Color = "Red", Value = 5 },
new Item (){ Category = "Big", Color = "Green", Value = 6 },
};
var criteria = new Dictionary<string, object>();
criteria["Category"] = "Big";
criteria["Value"] = 5;
var query = DoDynamicWhere(list.AsQueryable(), criteria);
var result = query.ToList();
}
static IQueryable<T> DoDynamicWhere<T>(IQueryable<T> list, Dictionary<string, object> criteria)
{
var temp = list;
//create a predicate for each supplied criterium and filter on it.
foreach (var key in criteria.Keys)
{
temp = temp.Where(BuildPredicate<T>(key, criteria[key]));
}
return temp;
}
//Create i.<prop> == <value> dynamically
static Expression<Func<TType, bool>> BuildPredicate<TType>(string property, object value)
{
var itemParameter = Expression.Parameter(typeof(TType), "i");
var expression = Expression.Lambda<Func<TType, bool>>(
Expression.Equal(
Expression.MakeMemberAccess(
itemParameter,
typeof(TType).GetProperty(property)),
Expression.Constant(value)
),
itemParameter);
return expression;
}
}
I don't really get why would you need the Dynamic LINQ here? Are the item properties not known at compile-time? If you can access a given item properties by name, eg. var prop = myitem['Color'], you don't need Dynamic LINQ.
It depends on how you render the results. There is a lot of ways to achieve the desired behavior, in general:
Fully client-side. If you do everything client-side (fetching data, rendering, paging) - jQuery would be the best way to go.
Server-side + client-side. If you render results on the server, you may add HTML attributes (for each property) to each search result markup and filter those client-side. The only problem in this case can be paging (if you do paging server-side, you will be able to filter the current page only)
Fully server-side. Post the form with search parameters and narrow down the search results using LINQ - match the existing items' properties with form values.
EDIT
If I were you (and would need to filter results server-side), I'd do something like:
var filtered = myItems.Where(i => i.Properties.Match(formValues))
where Match is an extension method that checks if a given list of properties matches provided values. Simple as this - no Dynamic LINQ needed.
EDIT 2
Do you need to map the LINQ query to the database query (LINQ to SQL)? That would complicate things a bit, but is still doable by chaining multiple .Where(...) clauses. Just loop over the filter properties and add .Where(...) to the query from previous iteration.
you may have a look at PredicateBuilder from the author of C# 4.0 in a Nutshell
As already pointed out by #Piotr Szmyd probabbly you don't need dynamic Linq. Iterating over all properties of T doesn'require dynamic linq. Dynamic Linq is mainly usefull to build complete queries on the client side and send it in string format to the server.
However now, it become obsolete, since Mvc 4 supports client side queries through Api Controllers returning an IQueryable.
If you just need to iterate over all properties of T you can do it with reflection and by building the LambdaExpressions that will compose the filtering criterion. You can do it with the static methods of the Expression class.
By using such static methods you can build dynamically expressions like m => m.Name= "Nick" with a couple instructions...than you put in and them...done you get and expression you can apply to an exixting IQueryable
LINQ implementation still has not changed so there should be no problem using the dynamic LINQ library. It simply creates LINQ expressions from strings.
You can use AJAX to call action methods that run the LINQ query and return JSON data. JQuery would populate HTML from the returned data.