I am working on Nest version 1.4 . Currently I am handling csv files with column names varying in every new insertion. In order to do that I have to chamge my class name every now and then, eg suppose
Suppose there are only three fields in csv file like severity,entity_type and operation then I define my class like this
[ElasticProperty(Name = "severity", Store = true, Index = FieldIndexOption.NotAnalyzed, Type = FieldType.String)]
public string severity { get; set; }
[ElasticProperty(Name = "entity_type", Store = true, Index = FieldIndexOption.NotAnalyzed, Type = FieldType.String)]
public string entity_type { get; set; }
[ElasticProperty(Name = "operation", Store = true, Index = FieldIndexOption.NotAnalyzed, Type = FieldType.String)]
public string operation { get; set; }
But in another csv file i have a new field called "number", then again i need to add this attribute to this class which becomes tedious. Is there something called dynamic mapping in nest version 1.4 that can automatically map the column names from the csv files? Also I tried with MULTI-FIELD MAPPING but it doesnt seem to work. Please suggest.
Thanks
Nilanjana
Related
I used to have String attribute on class properties of type enumeration in Elastic Search 2.0 so that enumeration value to be stored as string rather than integer:
public enum PaperType
{
A4 = 0,
A3 = 1
}
and class to be stored as document in ElasticSearch
[ElasticsearchType(Name = "Paper")]
public class Paper
{
[String(Store = false, Index = FieldIndexOption.Analyzed)]
public string Name { get; set; }
[String(Store = false, Index = FieldIndexOption.Analyzed)]
public PaperType Type { get; set; }
}
So in Type would be stored as A3 or A4 rather than 0 or 1.
in ElasticSearch 5, these attributes have changed,
How could I achieve the same behavior in ElasticSearch 5 or what attribute and how it should be?
Thanks
Take a look at attribute mapping in the documentation; you're likely looking to map them as Keyword types.
If you want to save all enum as string, you can add the Json.Net StringEnumConverter to the JsonSerializerSettings to use for any type that is an enum.
We have a collection contains documents in the server. Each document is like:
{ _id: "...", Prop1: "", Prop2: "", Prop3: "", LargeField: "", ... }
There're many other fields but they're not required by the client.
I want to load documents as MyDoc class whose definition is:
public class MyDoc {
public string Id { get; set; }
public string Prop1 { get; set; }
public string Prop2 { get; set; }
public string Prop3 { get; set; }
public string LargeField { get; set; }
}
I've tried:
var client = new MongoClient(uri);
var database = client.GetDatabase("MyDatabase");
var collection = database.GetCollection<MyDocs>("MyDocs");
var allDocs = collection.Find().ToList();
Then it will load all the fields for each document, so I have to put [BsonIgnoreExtraElements] on MyDoc. The issue here is that the document is large but I only needs a limit subset of fields. Is it possible to let the driver know I only need the fields defined in the class?
If not, is it possible to exclude some of the fields like the LargeField to make the result set smaller? I've tried:
var fieldsBuilder = Builders<MyDoc>.Projection;
var fields = fieldsBuilder.Exclude(d => d.LargeField);
var allDocs = collection.Find().Project(fields).ToList();
But now allDocs becomes BsonDocument list instead of the MyDoc list. How to query MyDoc with projection?
Can someone help? It's rather simple in legacy MongoDB driver but I don't know how to do it in the new driver. Thanks.
I had a similar issue, I believe you need to specify the generic type, for some reason Project automatically assumes BsonDocument. This should fix it from BsonDocument to your class.
Change:
var allDocs = collection.Find().Project(fields).ToList();
To:
var allDocs = collection.Find<MyDoc>().Project<MyDoc>(fields).ToList();
As for how to include only certain fields, this can be done just as you are doing it with builder (using Include) or with a string in json form like:
var allDocs = collection.Find<MyDoc>().Project<MyDoc>("{Prop1: 1, Prop2: 1}").ToList();
I would highly recommend checking out projection on this guy's post:
https://www.codementor.io/pmbanugo/working-with-mongodb-in-net-part-3-skip-sort-limit-and-projections-oqfwncyka
From this article:
This brings us to another difference: with a projection definition, it implicitly converts the document type from Student to BsonDocument, so what we get back is a fluent object that, in result, will be a BsonDocument (even though what we're working with is the Student type). If we want to work with Student, we have to indicate that we still want to keep the type to Student.
A newer way:
var fieldsBuilder = Builders<MyDoc>.Projection;
var fields = fieldsBuilder.Exclude(d => d.BigField1).Exclude(d => d.BigField2);
return Collection.Find(x => x.id.Equals(id)).Project<MyDoc>(fields).ToEnumerable();
Gina
We can use FindAsync as well. and to use projection in FindSync, we have to pass the FindOptions something like this: It will select countryName, ID and population.
FindAsync loads documents one by one from DB cursor in async manner, it's a good choice when you have a huge database.
Relevant Code:
cursor = await collection.FindAsync(y => y.CountryID > 205,
new FindOptions<MyDoc, MyDoc>()
{
BatchSize = 20,
NoCursorTimeout = true,
AllowPartialResults = true,
Projection = "{'_id':1,'Name':1,'Population':1}"
}).ConfigureAwait(false);
Once again, I am facing an issue, this time with LINQ Expression builder and this time I am even struggling to find the reason why it's not working. I have a Database-First EF project with quite a few tables. For this specific case, I have to use 2 of them - DocHead and Contragent. MyService.metadata.cs looks like this:
[MetadataTypeAttribute(typeof(DocHead.DocHeadMetadata))]
public partial class DocHead
{
// This class allows you to attach custom attributes to properties
// of the DocHead class.
//
// For example, the following marks the Xyz property as a
// required property and specifies the format for valid values:
// [Required]
// [RegularExpression("[A-Z][A-Za-z0-9]*")]
// [StringLength(32)]
// public string Xyz { get; set; }
internal sealed class DocHeadMetadata
{
// Metadata classes are not meant to be instantiated.
private DocHeadMetadata()
{
}
public string doc_Code { get; set; }
public string doc_Name { get; set; }
public string doc_ContrCode { get; set; }
//...
[Include]
public Contragent Contragent { get; set; }
}
}
[MetadataTypeAttribute(typeof(Contragent.ContragentMetadata))]
public partial class Contragent
{
// This class allows you to attach custom attributes to properties
// of the Contragent class.
//
// For example, the following marks the Xyz property as a
// required property and specifies the format for valid values:
// [Required]
// [RegularExpression("[A-Z][A-Za-z0-9]*")]
// [StringLength(32)]
// public string Xyz { get; set; }
internal sealed class ContragentMetadata
{
// Metadata classes are not meant to be instantiated.
private ContragentMetadata()
{
}
public string Code { get; set; }
public string Name { get; set; }
//...
I take some docHeads like this:
IQueryable<DocHead> docHeads = new MyEntities().DocHead;
Then I try to sort them like this:
docHeads = docHeads.OrderByDescending(x => x.Contragent.Name);
It is all working like I want it. I get those docHeads sorted by the name of the joined Contragent. My problem is that I will have to sort them by a field, given as a string parameter. I need to be able to write something like this:
string field = "Contragent.Name";
string linq = "docHeads = docHeads.OrderByDescending(x => x." + field + ")";
IQueryable<DocHead> result = TheBestLinqLibraryInTheWorld.PrepareLinqQueryable(linq);
Unfortunately, TheBestLinqLibraryInTheWorld does not exist (for now). So, I have set up a method as a workaround.
public static IQueryable<T> OrderByField<T>(this IQueryable<T> q, string SortField, bool Ascending)
{
var param = Expression.Parameter(typeof(T), "x");
var prop = Expression.Property(param, SortField); // normally returns x.sortField
var exp = Expression.Lambda(prop, param); // normally returns x => x.sortField
string method = Ascending ? "OrderBy" : "OrderByDescending";
Type[] types = new Type[] { q.ElementType, exp.Body.Type };
var mce = Expression.Call(typeof(Queryable), method, types, q.Expression, exp); // normally returns sth similar to q.OrderBy(x => x.sortField)
return q.Provider.CreateQuery<T>(mce);
}
Normally... yes, when it comes to own properties of the class DocHead - those prefixed with doc_. The disaster strikes when I call this method like this:
docHeads = docHeads.OrderByField<DocHead>("Contragent.Name", true); // true - let it be Ascending order
To be more specific, the exception in the title is thrown on line 2 of the method OrderByField():
var prop = Expression.Property(param, SortField);
In My.edmx (the model), the tables DocHead and Contragent have got a relation already set up for me, which is the following: 0..1 to *.
Once again, I have no problem writing "static" queries at all. I have no problem creating "dynamic" ones via the method OrderByField(), but only when it comes to properties of the class DocHead. When I try to order by a prop of the joined Contragent class - the disaster strikes. Any help will be greatly appretiated, thank you!
The problem is that Expression.Property method does not support nested properties. It does exactly what it says - creates expression that represents a property denoted by propertyName parameter of the object denoted by the expression parameter.
Luckily it can easily be extended. You can use the following simple Split / Aggregate trick anytime you need to create a nested property access expression:
var prop = SortField.Split('.').Aggregate((Expression)param, Expression.Property);
I am displaying search results using sitecore and solr. The scenario is if user search for something, search code first will check if that keyword is in "keywords" field of sitecore items, if it finds it will show the result and if it does not find, it will check for that keyword in title and description fields.
Now the problem is -
below condition is always false and never gives the result, in spite of values in the keywords field.
var Query = searchContext.GetQueryable<SearchResultItem>()
.Where(i => (i["keywords"].Contains(SearchQuery)));
where as the same query for title and description works fine
var Query2 = searchContext.GetQueryable<SearchResultItem>()
.Where(i => (i["title"].Contains(SearchQuery)
|| i["description"].Contains(SearchQuery)));
for each sitecore item, I have title, description and keywords fields.
Below is the detail code snippet.
public class SearchModel
{
public string SearchQuery { get; set; }
List<WebSiteSearchResult> lst = new List<WebSiteSearchResult>();
public IEnumerable<WebSiteSearchResult> SiteSearchResult()
{
var searchContext = ContentSearchManager.GetIndex("sitecore_web_index").CreateSearchContext();
var Query = searchContext.GetQueryable<SearchResultItem>().Where(i => (i["keywords"].Contains(SearchQuery)));
var result = Query.GetResults();
if (result.TotalSearchResults != 0)
{
//some operations
}
else
{
var Query2 = searchContext.GetQueryable<SearchResultItem>().Where(i => (i["title"].Contains(SearchQuery) || i["description"].Contains(SearchQuery)));
var result2 = Query2.GetResults();
if (result2.TotalSearchResults != 0)
{
//some operations
}
}
lst = lst.OrderBy(i => i.itemBoostingRatio).ToList();
return lst;
}
}
public class WebSiteSearchResult
{
public string itemKeywords { get; set; }
public int itemBoostingRatio { get; set; }
public string itemTitle { get; set; }
public string itemDescription { get; set; }
}
And, here my sitecore items: http://i.stack.imgur.com/liw59.png
Given your Sitecore item keywords are in a SLT field ("Loans, Car Loan, Bike Loan"), keep in mind that this field is a tokenized index. This means during storage processing, the query is split into tokens using whitespace and punctuation characters as delimiters, seen below:
Now if we search using Equals and Contains, the search log will give us the appropriate serialized query to test in solr:
Equals: ?q=keywordstest_t:("bike loan")
This is telling us to find any term in the list that matches this given value. If our term was "scooter loan", we would not get any results back.
Contains: ?q=keywordstest_t:(*bike loan*)
This is telling us to return any result item where any term in the list contains any of the words/sub-words. This isn't search performant. If our term was "scooter L", we would get a result back because of the L.
A couple things to check:
Is this value set to true in the Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config?
<indexAllFields>true</indexAllFields>
For best practices, create a new Search Result Item class that inherits from Sitecore's base SearchResultItemso that you can use it this instance to directly call the field property 'Keywords':
public class BasePageSearchResultItem : SearchResultItem
{
[IndexField("keywords")]
public string Keywords { get; set; }
}
The Equals query should be used to find one of these terms in the indexed field:
var Query = searchContext.GetQueryable<SearchResultItem>()
.Where(i => i.Keywords.Equals(SearchQuery));
The result set will depend on the requirements, dictated by the search query "scooter loan" and given a sample item's keyword field value "Loans, Car Loan, Bike Loan":
Search on scooter and loan separately, where each search query term is an exact match or is contained in an indexed term
Search on scooter loan as a whole
I have a List<Product> contains properties of Bikes (Name, ProductName, Color, List Price). I'm struggling to figure out how to write a search function using LINQ. I'd like to find a name of Bike. Any suggest will be help me some ways.
Imagine that your name is taken from a variable called nameToSearch.
This is if you want to get the Product.
string nameToSearch = "BikeName";
List<Product> list = bikes.Where(x => x.Name == nameToSearch).ToList();
I assume you have the following Product class:
public class Product
{
public String Name { get; set; }
public String ProductName { get; set; }
public String Color { get; set; }
public String List { get; set; }
public String Price { get; set; }
}
You also mentioned you have your data in a List<Product>. I will give a demo name for it:
List<Product> myProductList = GetProductList();
// Where GetProductList() will create a new List<Product> and populate it.
String bikeNameFilter = GetNameFilter();
// You can chnage this by the string you want for filtering.
You can use the following to get your data:
List<Product> myFilteredProductList = (from p in myProductList
where p.Name = bikeNameFilter
select p;
).ToList()
Obviously you can change the filter you want to use to another property of your product. Finally to get the actual name, you can loop through the list you just got:
foreach (var p in myFilteredProductList)
{
Console.WriteLine(p.ProductName);
// Use this value wherever you want.
}
Take a look at a nuget package I have created
http://www.nuget.org/packages/NinjaNye.SearchExtensions
This will enable the following (and more) which will return results where the search term appears in any of the properties specified
var result = products.Search("searchTerm", p => p.Name, p => p.ProductName);
Performing a search against all string properties can be done as follows:
var result = products.Search("searchTerm");
Alternatively, you can perform an AND search where the search term exists in a set of properties as follows:
string searchTerm = "searchTerm";
var result = products.Search(searchTerm, p => p.Name)
.Search(searchTerm, p => p.ProductName);
For more information take a look at the projects GitHub page or my blog posts
UPDATE: don't forget the using directive...
using NinjaNye.SearchExtensions