Is there a way in NEST to skip a nested collection or type from being indexed, but still include with the document?
I can use below to completely skip the property, not just from being indexed:
[ElasticProperty(OptOut=true)]
public List<MyClass> subtype { get; set; }
Using Index=FieldIndexOption.no appears to have no effect (the mapping looks the same):
[ElasticProperty(Index=FieldIndexOption.no)]
public List<MyClass> subtype { get; set; }
I want to avoid specify the FieldIndexOption.no on each property of the nest type. Is there a another way?
Edit 1:
Code for creating the index:
elasticClient.CreateIndex("MyParentClass", new IndexSettings());
elasticClient.MapFromAttributes<MyParentClass>();
We're currently on version 0.12 of NEST (upgrade pending).
Related
I want to put documents from different customers to different indexes in Elasticsearch.
Documents have some common part in structure and a part that vary from customer to customer named Data.
C#
class Entity
{
public Guid CustomerId { get; set; }
public IDictionary<string, object> Data { get; set; }
}
JSON
{
"customerId": "10000000-0000-0000-0000-000000000000",
"data": {......}
}
Say we have 10000 "customers" with a million of documents for each. "Customers" may be created and deleted dynamically.
Is it good idea to put documents from different customers to different indexes in Elasticsearch?
Is it possible to create a new index in Elasticsearch based on customerId field of inserted document dynamically? How to do it using .Net client?
I'm looking for something like:
var index = entity.CustomerId;
client.CreateDocument<Entity>(entity, index);
PS I'm using Elasticsearch v7.6
Well, I found an answer. Anyway would be nice if someone comment it how is it good strategically.
Here we go:
var indexName = entity.CustomerId.ToString();
var request = new IndexRequest<Entity>(entity, indexName);
client.Index<Entity>(entity);
I am trying to sort the result set based on a field name. But Sort doesn't works with string type.
Tried Code:-
public class Company
{
public long Number { get; set; }
public string Name{ get; set; }
}
My problem is : Sorting is not done when I use SortAscending API, like below
var resultSet = client.Search<Article>(s => s
.Type("Company")
.From(0)
.Size(200)
.QueryString("Stack OverFlow")
.SortAscending(f => f.Name));
Note: Documents are listed as Sorted if I set field name as Number(f => f.Number)
Please help
Your issue with sorting on the name field in your index is probably related to the fact that the field is being analyzed/tokenized. From the Elasticsearch Sort Guide:
For string based types, the field sorted on should not be analyzed / tokenized.
Therefore, you need to provide an additional field that is not analyzed/tokenized to perform your sort against. You can accomplish this by adding an additional field to your documents and setting the mapping for that type/field to not_analyzed or you can leverage multi_field (now just fields in version 1.x) on your existing name field. Please refer to the following for guidance on how to accomplish either of these options:
Multi-Fields (or Fields in v1.X)
Mapping
I have the following classes :
public class Parent
{
public string ParentProp { get; set; }
public IEnumerable<Child> ManyChildren { get; set; }
}
public class Child
{
public string ChildName { get; set; }
public int Value { get; set; }
}
Say I have an OData operation defined which returns IEnumberable<Parent>. Can I write an $orderby clause which performs the following operation ('parents' is an IEnumerable<Parent>) :
parents.OrderBy(x => x.ManyChildren.Single(y => y.ChildName == "Child1").Value);
I know I can write custom actions (http://msdn.microsoft.com/en-us/library/hh859851(v=vs.103).aspx) to do this ordering for me, but I'd rather use an $orderby clause.
(The only SO question which asked something similar is a little dated - How can I order objects according to some attribute of the child in OData?)
As I tried is possible with nesting $orderby in $expand so will be:
odata/User?&$select=Active,Description,Name,UserId&$expand=Company($select=Active,Name,CreatedBy,CompanyId;$orderby=Active asc)
And what you get is somthing like:
ORDER BY [Project2].[UserId] ASC, [Project2].[C19] ASC
will order a company collection for each user separately.
I think in version OData Client for .NET 6.7.0 is supported, in release notes is writhing:
In query options
$id, $select, $expand(including nested query options)....
I see in version 6.1 that values for nested options exist and is in:
DataQueryOptions->SelectExpand->SelectExpandClasue->SelectedItems->ExpandNavigationItem->OrderByOption
but is not working.
I tried and with System.Web.OData 5.6 and all releated dependencies but seams is not working.
My conclusion:
Seams that is everiting prepared like DataQueryOptions exist nested orderby but is not working.
Like I find out standard seams is going in that direction.
https://issues.oasis-open.org/browse/ODATA-32
It depends on your OData service implementation. Which kind of service are you using? WCFDS, WebAPI, or the service you implement yourself?
Url parser do can parse the URL such as root/People?$orderby=Company/Name. The translator is implemented by service.
And I agree with the answer in related question: "it's not possible to do this with a navigation property that has a cardinality of many". Since it's has a cardinality of many, service cannot know which one should be used to sorting.
I want to create a new model object named Movie_Type in my ASP.NET MVC web application. What will be the differences if I define the navigation proprty of this class to be either List, ICollection or IQueryable as following?
public partial class Movie_Type
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public List<Movie> Movies { get; set; }
}
OR
public partial class Movie_Type
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public IQueryable<Movie> Movies { get; set; }
}
OR
public partial class Movie_Type
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public ICollection<Movie> Movies { get; set; }
}
Edit:-
#Tomas Petricek.
thanks for your reply. in my case i am using the database first approach and then i use DbContext template to map my tables, which automatically created ICollection for all the navigation properties, So my questions are:-
1. Does this mean that it is not always the best choice to use Icollection. And i should change the automatically generated classes to best fit my case.
2. Secondly i can manage to choose between lazy or Eager loading by defining .include such as
var courses = db.Courses.Include(c => c.Department);
Regardless of what i am using to define the navigation properties. So i can not understand ur point.
3. i did not ever find any examples or tutorials that use IQuerable to define the navigation properties ,, so what might be the reason?
BR
You cannot use a navigation property of type IQueryable<T>. You must use ICollection<T> or some collection type which implements ICollection<T> - like List<T>. (IQueryable<T> does not implement ICollection<T>.)
The navigation property is simply an object or a collection of objects in memory or it is null or the collection is empty.
It is never loaded from the database when you load the parent object which contains the navigation property from the database.
You either have to explicitely say that you want to load the navigation property together with the parent which is eager loading:
var movieTypes = context.Movie_Types.Include(m => m.Movies).ToList();
// no option to filter or sort the movies collection here.
// It will always load the full collection into memory
Or it will be loaded by lazy loading (which is enabled by default if your navigation property is virtual):
var movieTypes = context.Movie_Types.ToList();
foreach (var mt in movieTypes)
{
// one new database query as soon as you access properties of mt.Movies
foreach (var m in mt.Movies)
{
Console.WriteLine(m.Title);
}
}
The last option is explicit loading which comes closest to your intention I guess:
var movieTypes = context.Movie_Types.ToList();
foreach (var mt in movieTypes)
{
IQueryable<Movie> mq = context.Entry(mt).Collection(m => m.Movies).Query();
// You can use this IQueryable now to apply more filters
// to the collection or sorting, for example:
mq.Where(m => m.Title.StartWith("A")) // filter by title
.OrderBy(m => m.PublishDate) // sort by date
.Take(10) // take only the first ten of result
.Load(); // populate now the nav. property
// again this was a database query
foreach (var m in mt.Movies) // contains only the filtered movies now
{
Console.WriteLine(m.Title);
}
}
There are two possible ways of looking at things:
Is the result stored in memory as part of the object instance?
If you choose ICollection, the result will be stored in memory - this may not be a good idea if the data set is very large or if you don't always need to get the data. On the other hand, when you store the data in memory, you will be able to modify the data set from your program.
Can you refine the query that gets sent to the SQL server?
This means that you would be able to use LINQ over the returned property and the additional LINQ operators would be translated to SQL - if you don't choose this option, additional LINQ processing will run in memory.
If you want to store data in memory, then you can use ICollection. If you want to be able to refine the query, then you need to use IQueryable. Here is a summary table:
| | Refine query | Don't change query |
|-----------------|--------------|--------------------|
| In-memory | N/A | ICollection |
| Lazy execution | IQueryable | IEnumerable |
More of a standard is IEnumerable as it is the least common denominator.
Iqueryable can be returned if you want extra querying functionality to the caller without having 10 repository methods to handle varying querying scenarios.
A downside is ienumerable could 'count()' slowly but if the object implements ICollection then this interface is checked for this value first without having to enumerate all items.
Also be aware if you return iqueryable to an untrusted caller they can do some casting and method calls on the iqueryable and get access to the context, connection, connection string, run queries, etc
Also note nhibernate for example has a query object you can pass to a repository to specify options. With entity framework you need to return IQueryable to enhance querying criteria
The collection that entity framework actually creates for you if you use virtual navigation properties implements ICollection, but not IQueryable, so you cannot use IQueryable for your navigation properties, as Slauma says.
You are free to define your properties as IEnumerable, as ICollection extends IEnumerable, but if you do this then you will lose your ability to add new child items to these navigation properties.
I am very new to OData (only started on it yesterday) so please excuse me if this question is too dumb :-)
I have built a test project as a Proof of Concept for migrating our current web services to OData. For this test project, I am using Reflection Providers to expose POCO classes via OData. These POCO classes come from in-memory cache. Below is the code so far:
public class DataSource
{
public IQueryable<Category> CategoryList
{
get
{
List<Category> categoryList = GetCategoryListFromCache();
return categoryList.AsQueryable();
}
}
// below method is only required to allow navigation
// from Category to Product via OData urls
// eg: OData.svc/CategoryList(1)/ProductList(2) and so on
public IQueryable<Category> ProductList
{
get
{
return null;
}
}
}
[DataServiceKeyAttribute("CategoryId")]
public class Category
{
public int CategoryId { get; set; }
public string CategoryName { get; set; }
public List<Product> ProductList { get; set; }
}
[DataServiceKeyAttribute("ProductId")]
public class Product
{
public int ProductId { get; set; }
public string ProductName { get; set; }
}
To the best of my knowledge, OData is going to use LINQ behind the scenes to query these in-memory objects, ie: List in this case if somebody navigates to OData.svc/CategoryList(1)/ProductList(2) and so on.
Here is the problem though: In the real world scenario, I am looking at over 18 million records inside the cache representing over 24 different entities.
The current production web services make very good use of .NET Dictionary and Hashtable collections to ensure very fast look ups and to avoid a lot of looping. So to get to a Product having ProductID 2 under Category having CategoryID 1, the current web services just do 2 look ups, ie: first one to locate the Category and the second one to locate the Product inside the Category. Something like a btree.
I wanted to know how could I follow a similar architecture with OData where I could tell OData and LINQ to use Dictionary or Hashtables for locating records rather than looping over a Generic List?
Is it possible using Reflection Providers or I am left with no other choice but to write my custom provider for OData?
Thanks in advance.
You will need to process expression trees, so you will need at least partial IQueryable implementation over the underlying LINQ to Objects. For this you don't need a full blown custom provider though, just return you IQueryable from the propties on the context class.
In that IQueryable you would have to recognize filters on the "key" properties (.Where(p => p.ProductID = 2)) and translate that into a dictionary/hashtable lookup. Then you can use LINQ to objects to process the rest of the query.
But if the client issues a query with filter which doesn't touch the key property, it will end up doing a full scan. Although, your custom IQueryable could detect that and fail such query if you choose so.