Using ElasticSearch.Net v6.0.2
Given the Indexed item
{
"PurchaseFrequency": 76,
"purchaseFrequency": 80
}
and the POCO Object
public class Product
{
public int PurchaseFrequency { get; set; }
}
and the setting
this.DefaultFieldNameInferrer(x => x);
Nest is returning a PurchaseFrequency = 80 even though this is the wrong field.
How can I get NEST to pull the correct cased field from ElasticSearch?
I don't think that this is going to be easily possible because this behaviour is defined in Json.NET, which NEST uses internally (not a direct dependency in 6.x, it's IL-merged into the assembly).
For example,
JsonConvert.DeserializeAnonymousType("{\"a\":1, \"A\":2}", new { a = 0 })
deserializes the anonymous type property a value to 2. But
JsonConvert.DeserializeAnonymousType("{\"A\":2, \"a\":1}", new { a = 0 })
deserializes the anonymous type property a value to 1 i.e. the order of properties as they appear in the returned JSON has a bearing on the final value assigned to a property on an instance of a type.
If you can, avoid JSON property names that differ only in case. If you can't, then you'd need to hook up the JsonNetSerializer in the NEST.JsonSerializer nuget package and write a custom JsonConverter for your type which only honours the exact casing expected.
Related
I've been trying to update to ES6 and NEST 6 and running into issues with NEST serializing of search requests - specifically serializing Terms queries where the underlying C# type is an enum.
I've got a Status enum mapped in my index as a Keyword, and correctly being stored in its string representation by using NEST.JsonNetSerializer and setting the contract json converter as per Elasticsearch / NEST 6 - storing enums as string
The issue comes when trying to search based on this Status enum. When I try to use a Terms query to specify multiple values, these values are being serialized as integers in the request and causing the search to find no results due to the type mismatch.
Interestingly the enum is serialized correctly as a string in a Term query, so I'm theorizing that the StringEnumConverter is being ignored in a scenario where it's having to serialize a collection of enums rather than a single enum.
Lets show it a little more clearly in code. Here's the enum and the (simplified) model used to define the index:
public enum CampaignStatus
{
Active = 0,
Sold = 1,
Withdrawn = 2
}
public class SalesCampaignSearchModel
{
[Keyword]
public Guid Id { get; set; }
[Keyword(DocValues = true)]
public CampaignStatus CampaignStatus { get; set; }
}
Here's a snippet of constructing the settings for the ElasticClient:
var pool = new SingleNodeConnectionPool(new Uri(nodeUri));
var connectionSettings = new ConnectionSettings(pool, (builtin, serializerSettings) =>
new JsonNetSerializer(builtin,
serializerSettings,
contractJsonConverters: new JsonConverter[]{new StringEnumConverter()}
)
)
.EnableHttpCompression();
Here's the Term query that correctly returns results:
var singleTermFilterQuery = new SearchDescriptor<SalesCampaignSearchModel>()
.Query(x => x.Term(y => y.Field(z => z.CampaignStatus).Value(CampaignStatus.Active)));
Generating the request:
{
"query": {
"term": {
"campaignStatus": {
"value": "Active"
}
}
}
}
Here's the Terms query that does not return results:
var termsFilterQuery = new SearchDescriptor<SalesCampaignSearchModel>()
.Query(x => x.Terms(y => y.Field(z => z.CampaignStatus).Terms(CampaignStatus.Active, CampaignStatus.Sold)));
Generating the request:
{
"query": {
"terms": {
"campaignStatus": [
0,
1
]
}
}
}
So far I've had a pretty good poke around at the options being presented by the JsonNetSerializer, tried a bunch of the available attributes (NEST.StringEnumAttribute, [JsonConverter(typeof(StringEnumConverter))] rather than using the global one on the client, having an explicit filter object with ItemConverterType set on the collection of CampaignStatuses, etc.) and the only thing that has had any success was a very brute-force .ToString() every time I need to query on an enum.
These are toy examples from a reasonably extensive codebase that I'm trying to migrate across to NEST 6, so what I'm wanting is to be able to specify global configuration somewhere rather than multiple developer teams needing to be mindful of this kind of eccentricity.
So yeah... I've been looking at this for a couple of days now. Good chances there's something silly I've missed. Otherwise I'm wondering if I need to be providing some JsonConverter with a contract that would match to an arbitrary collection of enums, and whether NEST and their tweaked Json.NET serializer should just be doing that kind of recursive resolution out of the box already.
Any help would be greatly appreciated, as I'm going a bit crazy with this one.
I have two objects.
[DataContract]
public class Record
{
[DataMember]
public string Id { get; set; }
}
And this class:
public class BatteryStatus : Record
{
[DataMember]
public DateTime RetrieveTime { get; set; }
}
I'm using Lite DB as a local NoSQL option to query and save the data. I'm needing to find and delete the values based after some time. Here's my code doing so:
var col = db.GetCollection<BatteryStatus>(CollectionName);
var test = col.FindAll()
.Where(x => x.Id == status.Id).ToList();
var result = col.Find(Query.EQ("Id", status.Id.ToString())).ToList();
Test returns with the with the object, but the result value doesn't. Lite DB only uses the Query or the BSONId as a way to delete an object. I don't have a BSON id attached to it (it's a referenced definition so I can't change it).
How can I use the "Query" function in order to get a nested value so I can delete it?
Classes has properties, BSON documents has fields. By default, LiteDB convert all property names to same name in BSON document except _id field which is document identifier.
If you want query using Linq, you will use properties expressions. If you are using Query object class, you must use field name.
var result = col.FindById(123);
// or
var result = col.FindOne(x => x.Id == 123);
// or
var result = col.FindOne(Query.EQ("_id", 123));
Find using _id always returns 1 (or zero) document.
I figured out the problem with LiteDB, since I was using the property name of "Id", the BSON interpreted that as the "_id" of the JSON object, and merging their two values. I solve the issue by renaming the "Id" property to something else.
Using Orchard CMS, I am dealing with a record and a part proxy, but cannot figure out how to save it into the DB. In fact, I confess I don't even know how to get the items I'm trying to save into this paradigm. I was originally using enum's for choices:
MyEmum.cs:
public enum Choices { Choice1, Choice2, Choice3, Choice4 }
MyRecord.cs:
public virtual string MyProperty { get; set; }
MyPart.cs:
public IEnumerable<string> MyProperty
{
get
{
if (String.IsNullOrWhiteSpace(Record.MyProperty)) return new string[] { };
return Record
.MyProperty
.Split(new[] { '.' }, StringSplitOptions.RemoveEmptyEntries)
.Select(r => r.Trim())
.Where(r => !String.IsNullOrEmpty(r));
}
set { Record.MyProperty = value == null ? null : String.Join(",", value); }
}
Now, in my service class, I tried something like:
public MyPart Create(MyPartRecord record)
{
MyPart part = Services.ContentManager.Create<MyPart>("My");
...
part.MyProperty = record.MyProperty; //getting error here
...
return part;
}
However, I am getting the following error: Cannot implicitly convert 'string' to System.Collections.Generic.IEnumerable<string>'
Essentially, I am trying to save choices from a checkboxlist (one or more selections) as a comma-separated list in the DB.
And this doesn't even get me over the problem of how do I use the enum. Any thoughts?
For some background:
I understand that the appropriate way to handle this relationship would be to create a separate table and use IList<MyEnum>. However, this is a simple list that I do not intend to manipulate with edits (in fact, no driver is used in this scenario as I handle this on the front-end with a controller and routes). I am just capturing data and redisplaying it in the Admin view for statistical/historical purposes. I may even consider getting rid of the Part (considering the following post: Bertrand's Blog Post.
It should be:
part.MyProperty = new[] {"foo", "bar"};
for example. The part's setter will store the value on the record's property as a comma-separated string, which will get persisted into the DB.
If you want to use enum values, you should use the Parse and ToString APIs that .NET provide on Enum.
I am getting an error
Error 826 Cannot implicitly convert type 'System.Collections.Generic.IEnumerable' to 'OVR.Models.ChildInfoModel'. An explicit conversion exists (are you missing a cast?) C:\dev.....\Controllers\NewRecordController.cs 129 34 OVR
public ActionResult ChildInfo(ChildInfoModel childInfoModel)
{
//INewRecord newRecord = (INewRecord)TempData["newRecord"];
if (TempData["CertNum"].ToString() == "")
{
return View();
}
else
{
string certNum = (string)TempData["CertNum"];
//call up Child Info Screen
var repo = new Get_Home_Repository();
childInfoModel = repo.Get_Screen_DataDS(certNum, "ChildInfo");
//childInfoModel = cModel;
return View(childInfoModel);
}
}
TempData["CertNum"] contains the value I need
The View is strongly typed to the model with #model OVR.Models.ChildInfoModel
I added in the parameter (ChildInfoModel childInfoModel) but if a link is not clicked a blank page is to be pulled up and allowed to be filled out, otherwise I pass in the parameters needed to pull data from a repository which it then populates the interface IChildInfo of which contains the data from a list as IEnumerable. I figured I could somehow explicitly convert .. or that the Interface IChildInfo needs to get updated in order to pass back the model to the view. How do I get the model populated from the repository data?
What is Get_Screen_DataDS returning? Whats the method signature return? It seems you are returning IEnumerable from Get_Screen_DS - what is this an IEnumerable of?
You are assigning to an instance of childInfoModel which I'm assuming is not an IEnumerable but I can't really tell from what you have posted.
Should you be returning an IEnumerable? If ChildInfoModel implements IEnumerable then just simply cast it
childInfoModel = (ChildInfoModel) repo.Get_Screen_DataDS(certNum, "ChildInfo");
However in this case you should probably just hat Get_Screen_DataDS return type ChildInfoModel.
Also, it will help you much in the long haul to stick to consistent and standard .net naming conventions.
var repo would be var repository.
GetScreenDataDS would be.... GetScreenData() or GetScreenDataByCert()
FxCop rules can help you with this : ) Just my extra .00000002
Here's my problem. A class which defines an order has a property called PaymentStatus, which is an enum defined like so:
public enum PaymentStatuses : int
{
OnDelivery = 1,
Paid = 2,
Processed = 3,
Cleared = 4
}
And later on, in the class itself, the property definition is very simple:
public PaymentStatuses? PaymentStatus { get; set; }
However, if I try to save an order to the Azure Table Storage, I get the following exception:
System.InvalidOperationException: The type Order+PaymentStatuses' has no settable properties.
At this point I thought using enum isn't possible, but a quick Google search returned this: http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/7eb1a2ca-6c1b-4440-b40e-012db98ccb0a
This page lists two answers, one of which seems to ignore the problems and suggests that using an enum in Azure Storage is fine.
Now, I don't NEED to store the enum in the Azure Table Storage as such, I could just as well store a corresponding int, however, I do need this property to be exposed in the WCF service.
I've tried making the property use get and set to return the enum from a stored integer, and remove this property from Azure by using the WritingEntity event on my DataContext, but I get that exception before the event for this entity is fired.
At this point, I'm at a loss, I don't know what else I can do to have this property in WCF as an enum, but have Azure store just the int.
Enum is not supported. Even though it is defined like an int, it is really not an integral type supported by Table Storage. Here is the list of types supported. An enum is just a string expression of an integral number with an object-oriented flavor.
You can store int in table storage and then convert it using Enum.Parse.
Here's a simple workaround:
public int MyEnumValue { get; set; } //for use by the Azure client libraries only
[IgnoreProperty] public MyEnum MyEnum
{
get { return (MyEnum) MyEnumValue; }
set { MyEnumValue = (int) value; }
}
It would have been nicer if a simple backing value could have been employed rather than an additional (public!) property - without the hassle of overriding ReadEntity/WriteEntity of course. I opened a user voice ticket that would facilitate that, so you might want to upvote it.
ya i was having this same problem
i changed my property which was earlier enum to int. now this int property parses the incoming int and saves it into a variale of the same enum type so now the code that was
public CompilerOutputTypes Type
{get; set;}
is chaged to
private CompilerOutputTypes type;
public int Type
{
get {return (int)type;}
set { type = (CompilerOutputTypes)value; }
}
Just suggestions...
I remember that in WCF you have to mark enums with special attributes: http://msdn.microsoft.com/en-us/library/aa347875.aspx
Also, when you declare PaymentStatuses? PaymentStatus, you are declaring Nullable<PaymentStatuses> PaymentStatus. The ? sintax is just syntactic sugar. Try to remove the ? and see what happen (you could add a PaymentStatuses.NoSet = 0 , because the default value for an Int32 is 0).
Good luck.
Parvs solution put me on the right track but I had some minor adjustments.
private string _EnumType;
private EnumType _Type;
//*********************************************
//*********************************************
public string EnumType
{
get { return _Type.ToString(); }
set
{
_EnumType = value;
try
{
_Type = (EnumType)Enum.Parse(typeof(EnumType), value);
}
catch (Exception)
{
_EnumType = "Undefined";
_Type = [mynamespace].EnumType.Undefined;
}
}
}
I have come across a similar problem and have implemented a generic object flattener/recomposer API that will flatten your complex entities into flat EntityProperty dictionaries and make them writeable to Table Storage, in the form of DynamicTableEntity.
Same API will then recompose the entire complex object back from the EntityProperty dictionary of the DynamicTableEntity.
This is relevant to your question because the ObjectFlattenerRecomposer API supports flattening property types that are normally not writeable to Azure Table Storage like Enum, TimeSpan, all Nullable types, ulong and uint by converting them into writeable EntityProperties.
The API also handles the conversion back to the original complex object from the flattened EntityProperty Dictionary. All that the client needs to do is to tell the API, I have this EntityProperty Dictionary that I just read from Azure Table (in the form of DynamicTableEntity.Properties), can you convert it to an object of this specific type. The API will recompose the full complex object with all of its properties including 'Enum' properties with their original correct values.
All of this flattening and recomposing of the original object is done transparently to the client (user of the API). Client does not need to provide any schema or any knowledge to the ObjectFlattenerRecomposer API about the complex object that it wants to write, it just passes the object to the API as 'object' to flatten it. When converting it back, the client only needs to provide the actual type of object it wants the flattened EntityProperty Dictionary to be converted to. The generic ConvertBack method of the API will simply recompose the original object of Type T and return it to the client.
See the usage example below. The objects do not need to implement any interface like 'ITableEntity' or inherit from a particular base class either. They do not need to provide a special set of constructors.
Blog: https://doguarslan.wordpress.com/2016/02/03/writing-complex-objects-to-azure-table-storage/
Nuget Package: https://www.nuget.org/packages/ObjectFlattenerRecomposer/
Usage:
//Flatten object (ie. of type Order) and convert it to EntityProperty Dictionary
Dictionary<string, EntityProperty> flattenedProperties = EntityPropertyConverter.Flatten(order);
// Create a DynamicTableEntity and set its PK and RK
DynamicTableEntity dynamicTableEntity = new DynamicTableEntity(partitionKey, rowKey);
dynamicTableEntity.Properties = flattenedProperties;
// Write the DynamicTableEntity to Azure Table Storage using client SDK
//Read the entity back from AzureTableStorage as DynamicTableEntity using the same PK and RK
DynamicTableEntity entity = [Read from Azure using the PK and RK];
//Convert the DynamicTableEntity back to original complex object.
Order order = EntityPropertyConverter.ConvertBack<Order>(entity.Properties);