List vs IEnumerable vs IQueryable when defining Navigation property - linq

I want to create a new model object named Movie_Type in my ASP.NET MVC web application. What will be the differences if I define the navigation proprty of this class to be either List, ICollection or IQueryable as following?
public partial class Movie_Type
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public List<Movie> Movies { get; set; }
}
OR
public partial class Movie_Type
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public IQueryable<Movie> Movies { get; set; }
}
OR
public partial class Movie_Type
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public ICollection<Movie> Movies { get; set; }
}
Edit:-
#Tomas Petricek.
thanks for your reply. in my case i am using the database first approach and then i use DbContext template to map my tables, which automatically created ICollection for all the navigation properties, So my questions are:-
1. Does this mean that it is not always the best choice to use Icollection. And i should change the automatically generated classes to best fit my case.
2. Secondly i can manage to choose between lazy or Eager loading by defining .include such as
var courses = db.Courses.Include(c => c.Department);
Regardless of what i am using to define the navigation properties. So i can not understand ur point.
3. i did not ever find any examples or tutorials that use IQuerable to define the navigation properties ,, so what might be the reason?
BR

You cannot use a navigation property of type IQueryable<T>. You must use ICollection<T> or some collection type which implements ICollection<T> - like List<T>. (IQueryable<T> does not implement ICollection<T>.)
The navigation property is simply an object or a collection of objects in memory or it is null or the collection is empty.
It is never loaded from the database when you load the parent object which contains the navigation property from the database.
You either have to explicitely say that you want to load the navigation property together with the parent which is eager loading:
var movieTypes = context.Movie_Types.Include(m => m.Movies).ToList();
// no option to filter or sort the movies collection here.
// It will always load the full collection into memory
Or it will be loaded by lazy loading (which is enabled by default if your navigation property is virtual):
var movieTypes = context.Movie_Types.ToList();
foreach (var mt in movieTypes)
{
// one new database query as soon as you access properties of mt.Movies
foreach (var m in mt.Movies)
{
Console.WriteLine(m.Title);
}
}
The last option is explicit loading which comes closest to your intention I guess:
var movieTypes = context.Movie_Types.ToList();
foreach (var mt in movieTypes)
{
IQueryable<Movie> mq = context.Entry(mt).Collection(m => m.Movies).Query();
// You can use this IQueryable now to apply more filters
// to the collection or sorting, for example:
mq.Where(m => m.Title.StartWith("A")) // filter by title
.OrderBy(m => m.PublishDate) // sort by date
.Take(10) // take only the first ten of result
.Load(); // populate now the nav. property
// again this was a database query
foreach (var m in mt.Movies) // contains only the filtered movies now
{
Console.WriteLine(m.Title);
}
}

There are two possible ways of looking at things:
Is the result stored in memory as part of the object instance?
If you choose ICollection, the result will be stored in memory - this may not be a good idea if the data set is very large or if you don't always need to get the data. On the other hand, when you store the data in memory, you will be able to modify the data set from your program.
Can you refine the query that gets sent to the SQL server?
This means that you would be able to use LINQ over the returned property and the additional LINQ operators would be translated to SQL - if you don't choose this option, additional LINQ processing will run in memory.
If you want to store data in memory, then you can use ICollection. If you want to be able to refine the query, then you need to use IQueryable. Here is a summary table:
| | Refine query | Don't change query |
|-----------------|--------------|--------------------|
| In-memory | N/A | ICollection |
| Lazy execution | IQueryable | IEnumerable |

More of a standard is IEnumerable as it is the least common denominator.
Iqueryable can be returned if you want extra querying functionality to the caller without having 10 repository methods to handle varying querying scenarios.
A downside is ienumerable could 'count()' slowly but if the object implements ICollection then this interface is checked for this value first without having to enumerate all items.
Also be aware if you return iqueryable to an untrusted caller they can do some casting and method calls on the iqueryable and get access to the context, connection, connection string, run queries, etc
Also note nhibernate for example has a query object you can pass to a repository to specify options. With entity framework you need to return IQueryable to enhance querying criteria

The collection that entity framework actually creates for you if you use virtual navigation properties implements ICollection, but not IQueryable, so you cannot use IQueryable for your navigation properties, as Slauma says.
You are free to define your properties as IEnumerable, as ICollection extends IEnumerable, but if you do this then you will lose your ability to add new child items to these navigation properties.

Related

Net Core - .Include() results in loop causing Visual Studio debug to crash

My database has a one-to-many relation between "UsageRecord" and "Dimension"
This is modelled as follows (using a Database-First approach):
public partial class Dimension
{
...
public virtual ICollection<UsageRecord> UsageRecord { get; set; }
}
Usage Record class:
public partial class UsageRecord
{
public long Id { get; set; }
...
public long DimensionId { get; set; }
public virtual Dimension Dimension { get; set; }
}
So, if i query the list of UsageRecords (EagerLoading):
_context.Set<UsageRecord>.Where(x => x.ProductId == productId).ToList()
i get a list of UsageRecord objects I can navigate through during debug:
Please notice that the Dimension object is null, and this is correct since i haven't included it in the query.
Now, If i try to include it, the application crashes:
_context.Set<UsageRecord>.Where(x => x.ProductId == productId).Include(p => p.Dimension).ToList();
Postman exits with a 502 error, and the VS Debug first shows a list of question marks "?" before crashing.
I think this is due to the fact that by Including the Dimension object, this loops through the list of UsageRecords attached and then the Dimension again and again.
How can I avoid it?
In order to retrieve your result from LINQ query you can solve your issue in these ways:
Configure your serializer to ignore loops
Create a view model for your controller's action
Use anonymous type from Select result in your controller's action

NHibernate Many-To-Many Performance Issue

My application has the following entities (with a many-to-many relationship between Product and Model):
public class TopProduct {
public virtual int Id { get; set; }
public virtual Product Product { get; set; }
public virtual int Order { get; set; }
}
public class Product {
public virtual int Id { get; set; }
public virtual string Name { get; set; }
public virtual IList<Model> Models { get; set; }
}
public class Model {
public virtual string ModelNumber { get; set; }
public virtual IList<Product> Products { get; set; }
}
Note: A product could have 1000s of models.
I need to display a list of TopProducts and the first 5 models (ordered alphabetically) against each one.
For example say I have the following query:
var topProducts = session.Query<TopProduct>()
.Cacheable()
.Fetch(tp => tp.Product).ThenFetchMany(p => p.Models)
.OrderBy(tp => tp.Order)
.ToList();
If I now say:
foreach (var topProduct in topProducts) {
var models = topProduct.Product.Models.Take(5).ToList();
...
}
This executes extremely slowly as it retrieves an item from the second level cache for each model. Since there could be 1000s of models against a product, it would need to retrieve 1000s of items from the cache the second time it is executed.
I have been racking my brain trying to think of a better way of doing this but so far I am out of ideas. Unfortunately my model and database cannot be modified at this stage.
I'd appreciate the help. Thanks
The key to your problem is understanding how entity and query caching work.
Entity caching stores, essentially, the POID of an entity and its property values.
When you want to get/initialize an instance, NH will first check the cache to see if the values are there, in order to avoid a db query.
Query caching, on the other hand, stores a query as the key (to simplify, let's say it's the command text and the parameter values), and a list of entity ids as the value (this is assuming your result is a list of entities, and not a projection)
When NH executes a cacheable query, it will see if the results are cached. If they are, it will load the proxies from those ids. Then, as you use them, it will initialize them one by one, either from the entity cache or from the db.
Collection cache is similar.
Usually, getting many second-level cache hits for those entity loads is a good thing. Unless, of course, you are using a distributed cache located in a separate machine, in which case this is almost as bad as getting them from the db.
If that is the case, I suggest you skip caching the query.

Model binding in the controller when form is posted - navigation properties are not loaded automatically

I'm using the Entity Framework version 4.2. There are two classes in my small test app:
public class TestParent
{
public int TestParentID { get; set; }
public string Name { get; set; }
public string Comment { get; set; }
public virtual ICollection<TestChild> TestChildren { get; set; }
}
public class TestChild
{
public int TestChildID { get; set; }
public int TestParentID { get; set; }
public string Name { get; set; }
public string Comment { get; set; }
public virtual TestParent TestParent { get; set; }
}
Populating objects with data from the database works well. So I can use testParent.TestChildren.OrderBy(tc => tc.Name).First().Name etc. in my code.
Then I built a standard edit form for my testParents. The controller look like this:
public class TestController : Controller
{
private EFDbTestParentRepository testParentRepository = new EFDbTestParentRepository();
private EFDbTestChildRepository testChildRepository = new EFDbTestChildRepository();
public ActionResult ListParents()
{
return View(testParentRepository.TestParents);
}
public ViewResult EditParent(int testParentID)
{
return View(testParentRepository.TestParents.First(tp => tp.TestParentID == testParentID));
}
[HttpPost]
public ActionResult EditParent(TestParent testParent)
{
if (ModelState.IsValid)
{
testParentRepository.SaveTestParent(testParent);
TempData["message"] = string.Format("Changes to test parents have been saved: {0} (ID = {1})",
testParent.Name,
testParent.TestParentID);
return RedirectToAction("ListParents");
}
// something wrong with the data values
return View(testParent);
}
}
When the form is posted back to the server the model binding appears to be working well - i.e. testParent looks okay (id, name and comment set as expected). However the navigation property TestChildren remains at NULL.
This I guess is not sooo surprising since the model binding merely extracts the form values as they were sent from the browser and pushes them into an object of the TestParent class. Populating testParent.TestChildren however requires an immediate roundtrip to the database which is the responsibility of the Entity Framework. And EF probably doesn't get involved in the binding process.
I was however expecting the lazy loading to kick in when I call testParent.TestChildren.First(). Instead that leads to an ArgumentNullException.
Is it necessary to tag an object in a special way after model binding so that the Entity Framework will do lazy loading? How can I achieve this?
Obviously I could manually retrieve the children with the second repository testChildRepository. But that (a) doesn't feel right and (b) leads to problems with the way my repositories are set up (each using their own DBContext - which is an issue that I haven't managed to come to terms with yet).
In order to get lazy loading for your child collection two requirements must be fulfilled:
The parent entity must be attached to an EF context
Your parent entity must be a lazy loading proxy
Both requirements are met if you load the parent entity from the database through a context (and your navigation properties are virtual to allow proxy creation).
If you don't load the entity from the database but create it manually you can achieve the same by using the appropriate EF functions:
var parent = context.TestParents.Create();
parent.TestParentID = 1;
context.TestParents.Attach(parent);
Using Create and not new is important here because it creates the required lazy loading proxy. You can then access the child collection and the children of parent with ID = 1 will be loaded lazily:
var children = parent.TestChildren; // no NullReferenceException
Now, the default modelbinder has no clue about those specific EF functions and will simply instantiate the parent with new and also doesn't attach it to any context. Both requirements are not fulfilled and lazy loading cannot work.
You could write your own model binder to create the instance with Create() but this is probably the worst solution as it would make your view layer very EF dependent.
If you need the child collection after model binding I would in this case load it via explicit loading:
// parent comes as parameter from POST action method
context.TestParents.Attach(parent);
context.Entry(parent).Collection(p => p.TestChildren).Load();
If your context and EF is hidden behind a repository you will need a new repository method for this, like:
void LoadNavigationCollection<TElement>(T entity,
Expression<Func<T, ICollection<TElement>>> navigationProperty)
where TElement : class
{
_context.Set<T>().Attach(entity);
_context.Entry(entity).Collection(navigationProperty).Load();
}
...where _context is a member of the repository class.
But the better way, as Darin mentioned, is to bind ViewModels and then map them to your entities as needed. Then you would have the option to instantiate the entities with Create().
One possibility is to use hidden fields inside the form that will store the values of the child collection:
#model TestParent
#using (Html.BegniForm())
{
... some input fields of the parent
#Html.EditorFor(x => x.TestChildren)
<button type="submit">OK</button>
}
and then have an editor template for the children containing the hidden fields (~/Views/Shared/EditorTemplates/TestChild.cshtml):
#model TestChild
#Html.HiddenFor(x => x.TestChildID)
#Html.HiddenFor(x => x.Name)
...
But since you are not following good practices here and are directly passing your domain models to the view instead of using view models you will have a problem with the recursive relationship you have between the children and parents. You might need to manually populate the parent for each children.
But a better way would be to query your database in the POST action and fetch the children that are associated to the given parent since the user cannot edit the children inside the view anyway.

AutoMapper on complicated Views (multiple looping)

Earlier today, a helpful person (here on Stack Overflow) pointed me towards AutoMapper, I checked it out, and I liked it a lot! Now however I am a little stuck.
In my Code First MVC3 Application, on my [Home/Index] I need to display the following information from my Entities:
List of Posts [ int Id, string Body, int Likes, string p.User.FirstName, string p.User.LastName ]
List of Tags [int Id, string Name]
List of All Authors that exist on my Database [ string UrlFriendlyName ]
So far I have managed only point 1 in the list by doing the following for my Index ViewModel:
public class IndexVM
{
public int Id { get; set; }
public string Body { get; set; }
public int Likes { get; set; }
public string UserFirstName { get; set; }
public string UserLastName { get; set; }
}
And on the Home Controller, Index ActionMethod I have:
public ActionResult Index()
{
var Posts = postsRepository.Posts.ToList();
Mapper.CreateMap<Post, IndexVM>();
var IndexModel = Mapper.Map<List<Post>, List<IndexVM>>(Posts);
return View(IndexModel);
}
Finally on my View I have it strongly typed to:
#model IEnumerable<BlogWeb.ViewModels.IndexVM>
And I am passing each Item in the IndexVM IEnumberable to a Partial View via:
#foreach (var item in Model)
{
#Html.Partial("_PostDetails", item)
}
My question is, how can I also achieve point 2 and 3, whilst not breaking what I've achieved in point 1.
I tried putting the stuff I currently have for IndexVM into a SubClass, and having a List Property on the Parent class, but it didn't work.
From the ASP.NET MVC2 In Action Book:
Some screens are more complex than a single table. They may feature
multiple tables and additional fields of other data: images, headings,
subtotals, graphs, charts, and a million other things that complicate
a view. The presentation model solution scales to handle them all.
Developers can confidently maintain even the gnarliest screens as long
as the presentation model is designed well. If a screen does contain
multiple complex elements, a presentation model can be a wrapper,
composing them all and relieving the markup file of much complexity. A
good presentation model doesn’t hide this complexity—it represents it
accurately and as simply as possible, and it separates the data on a
screen from the display.
Make a ViewModel that represents your screen. Then build it up and pass it to the View. This book is great and talks about using a presentation model. With AutoMapper, think about how you would accomplish your mapping without it, then make use of it. AutoMapper isn't going to do anything magic, it eliminates keyboard slapping.
AutoMapper aside, take your list of requirments:
List of Posts [ int Id, string Body, int Likes, string p.User.FirstName, string p.User.LastName ]
List of Tags [int Id, string Name]
List of All Authors that exist on my Database [ string
UrlFriendlyName ]
and assuming you have these Model entites: Post, Tag, Author
Personally I don't like passing Model entities to my presentation in MVC or MVVM but that's me. Say we follow that here and create PostDisplay, TagDisplay, and AuthorDisplay.
Based on the View's requirements the ViewModel will look like this:
Public class IndexVM
{
Public List<PostDisplay> Posts {get; set;}
Public List<TagDisplay> Tags {get; set;}
Public List<AuthorDisplay> Authors {get; set;}
}
In this case the way the View is composed will require you to build it up:
public ActionResult Index()
{
var posts = postsRepository.Posts.ToList();
var tags = postsRepository.Tags.ToList();
var authors = postsRepository.Authors.ToList();
Mapper.CreateMap<Post, PostDisplay>();
Mapper.CreateMap<Tag, TagDisplay>();
Mapper.CreateMap<Author, AuthorDisplay>();
private var IndexVM = new IndexVM
{
Posts = Mapper.Map<List<Post>, List<PostDisplay>>(posts),
Tags = Mapper.Map<List<Tag>, List<TagDisplay>>(tags),
Authors = Mapper.Map<List<Author>, List<AuthorDisplay>>(authors)
};
return View(IndexVM);
}
So, what you end up with is a ViewModel to pass to your view that represents exactly what you want to display and isn't tightly coupled to your Domain Model. I can't think of a way to have AutoMapper map three separate result lists into one object.
To clarify, AutoMapper will map child collections so a structure like:
public class OrderItemDto{}
public class OrderDto
{
public List<OrderItemDto> OrderItems { get; set; }
}
will map to:
public class OrderItem{}
public class Order
{
public List<OrderItem> OrderItems { get; set; }
}
As long as you tell it how to map the types: OrderDto -> Order and OrderItemDto -> OrderItem.
As an alternative to including all of your lists of entities on a single viewmodel, you could use #Html.Action. Then, in your screen view:
#Html.Action("Index", "Posts")
#Html.Action("Index", "Tags")
#Html.Action("Index", "Authors")
This way, your Index / Screen view & model don't need to know about the other viewmodels. The partials are delivered by separate child action methods on separate controllers.
All of the automapper stuff still applies, but you would still map your entities to viewmodels individually. The difference is, instead of doing the mapping in HomeController.Index(), you would do it in PostsController.Index(), TagsController.Index(), and AuthorsController.Index().
Response to comment 1
public class IndexVM
{
// need not implement anything for Posts, Tags, or Authors
}
Then, implement 3 different methods on 3 different controllers. Here is one example for the PostsController. Follow the same pattern for TagsController and AuthorsController
// on PostsController
public PartialViewResult Index()
{
var posts = postsRepository.Posts.ToList();
// as mentioned, should do this in bootstrapper, not action method
Mapper.CreateMap<Post, PostModel>();
// automapper2 doesn't need source type in generic args
var postModels = Mapper.Map<List<PostModel>>(posts);
return PartialView(postModels);
}
You will have to create a corresponding partial view for this, strongly-typed as #model IEnumerable<BlogWeb.ViewModels.PostModel>. In that view, put the HTML that renders the Posts UI (move from your HomeController.Index view).
On your HomeController, just do this:
public ActionResult Index()
{
return View(new IndexVM);
}
Keep your view strongly-typed on the IndexVM
#model IEnumerable<BlogWeb.ViewModels.IndexVM>
... and then get the Posts, Tags, and Authors like so:
#Html.Action("Index", "Posts")
Response to comment 2
Bootstrapping... your Mapper.CreateMap configurations only have to happen once per app domain. This means you should do all of your CreateMap calls from Application_Start. Putting them in the controller code just creates unnecessary overhead. Sure, the maps need to be created - but not during each request.
This also helps with unit testing. If you put all of your Mapper.CreateMap calls into a single static method, you can call that method from a unit test method as well as from Global.asax Application_Start. Then in the unit test, one method can test that your CreateMap calls are set up correctly:
AutoMapperBootStrapper.CreateAllMaps();
Mapper.AssertConfigurationIsValid();

How to use a Dictionary or Hashtable for LINQ query performance underneath an OData service

I am very new to OData (only started on it yesterday) so please excuse me if this question is too dumb :-)
I have built a test project as a Proof of Concept for migrating our current web services to OData. For this test project, I am using Reflection Providers to expose POCO classes via OData. These POCO classes come from in-memory cache. Below is the code so far:
public class DataSource
{
public IQueryable<Category> CategoryList
{
get
{
List<Category> categoryList = GetCategoryListFromCache();
return categoryList.AsQueryable();
}
}
// below method is only required to allow navigation
// from Category to Product via OData urls
// eg: OData.svc/CategoryList(1)/ProductList(2) and so on
public IQueryable<Category> ProductList
{
get
{
return null;
}
}
}
[DataServiceKeyAttribute("CategoryId")]
public class Category
{
public int CategoryId { get; set; }
public string CategoryName { get; set; }
public List<Product> ProductList { get; set; }
}
[DataServiceKeyAttribute("ProductId")]
public class Product
{
public int ProductId { get; set; }
public string ProductName { get; set; }
}
To the best of my knowledge, OData is going to use LINQ behind the scenes to query these in-memory objects, ie: List in this case if somebody navigates to OData.svc/CategoryList(1)/ProductList(2) and so on.
Here is the problem though: In the real world scenario, I am looking at over 18 million records inside the cache representing over 24 different entities.
The current production web services make very good use of .NET Dictionary and Hashtable collections to ensure very fast look ups and to avoid a lot of looping. So to get to a Product having ProductID 2 under Category having CategoryID 1, the current web services just do 2 look ups, ie: first one to locate the Category and the second one to locate the Product inside the Category. Something like a btree.
I wanted to know how could I follow a similar architecture with OData where I could tell OData and LINQ to use Dictionary or Hashtables for locating records rather than looping over a Generic List?
Is it possible using Reflection Providers or I am left with no other choice but to write my custom provider for OData?
Thanks in advance.
You will need to process expression trees, so you will need at least partial IQueryable implementation over the underlying LINQ to Objects. For this you don't need a full blown custom provider though, just return you IQueryable from the propties on the context class.
In that IQueryable you would have to recognize filters on the "key" properties (.Where(p => p.ProductID = 2)) and translate that into a dictionary/hashtable lookup. Then you can use LINQ to objects to process the rest of the query.
But if the client issues a query with filter which doesn't touch the key property, it will end up doing a full scan. Although, your custom IQueryable could detect that and fail such query if you choose so.

Resources