My database has a one-to-many relation between "UsageRecord" and "Dimension"
This is modelled as follows (using a Database-First approach):
public partial class Dimension
{
...
public virtual ICollection<UsageRecord> UsageRecord { get; set; }
}
Usage Record class:
public partial class UsageRecord
{
public long Id { get; set; }
...
public long DimensionId { get; set; }
public virtual Dimension Dimension { get; set; }
}
So, if i query the list of UsageRecords (EagerLoading):
_context.Set<UsageRecord>.Where(x => x.ProductId == productId).ToList()
i get a list of UsageRecord objects I can navigate through during debug:
Please notice that the Dimension object is null, and this is correct since i haven't included it in the query.
Now, If i try to include it, the application crashes:
_context.Set<UsageRecord>.Where(x => x.ProductId == productId).Include(p => p.Dimension).ToList();
Postman exits with a 502 error, and the VS Debug first shows a list of question marks "?" before crashing.
I think this is due to the fact that by Including the Dimension object, this loops through the list of UsageRecords attached and then the Dimension again and again.
How can I avoid it?
In order to retrieve your result from LINQ query you can solve your issue in these ways:
Configure your serializer to ignore loops
Create a view model for your controller's action
Use anonymous type from Select result in your controller's action
Related
I am using EF6 code first to execute a query that pulls a large amount of data for parallelized GPU processing. The linq query returns an IEnumerable.
IEnumerable<DatabaseObject> results = ( from items in _myContext.DbSet
select items).Include("Table1").Include("Table2");
Now, I need to perform some statistical analysis on the complete set of data, and present the result to the user.
Unfortunately, because of the sheer size of the returned data, just doing a
results.ToList() is taking an extremely long time to complete... and I haven't even begun the parallelized processing of the data as yet!
I there anything that I can do to make this more efficient other than reducing the amount of data being pulled? This is not an option since it is the complete set of data that needs to be processed.
EDIT 1
My current code first is as follows:
public class Orders
{
[Key]
public virtual DateTime ServerTimeId
{
get;
set;
}
public string Seller
{
get;
set;
}
public decimal Price
{
get;
set;
}
public decimal Volume
{
get;
set;
}
public List<Table1> Tables1{ get; set; }
public List<Table2> Table22{ get; set; }
}
Although by not using .Include my query speeds up significantly, if I do not use .Include ("Tables1).Include("Tables2") these fields are null
in the final result for this query:
var result = ( from items in _context.DbOrders
select orderbook ).Include("Tables1").Include("Tables2")
In my DbContext, I have defined:
public DbSet<Orderok> DbOrders { get; set; }
If there is a way to force EF6 to populate these tables without the use of .Include, then I'd be very pleased if someone could instruct me.
You can load the main table, DbOrders and the child tables separately into the context:
_myContext.Configuration.ProxyCreationEnabled = false;
_myContext.DbOrders.Load();
_myContext.Table1.Load();
_myContext.Table2.Load();
Now the context is fully charged with the data you need. I hope you won't run into an out of memory exception (because then the whole approach collapses).
Entity Framework excecutes relationship fixup, which means that it populates the navigation properties DbOrders.Table1 and DbOrders.Table1.
Disabling proxy creation has two reasons:
The materialized objects will be as light-weight as possible
Lazy loading is disabled, otherwise it would be triggered when you access a navigation property.
Now you can continue working wit the data by accessing the Local collection:
from entity in _myContext.DbOrders.Local
...
You can further try to speed op the process by unmapping all database fields that you don't need. This makes the SQL result sets smaller and the materialized object will be even lighter. To achieve that, maybe you have to create a dedicated context.
My application has the following entities (with a many-to-many relationship between Product and Model):
public class TopProduct {
public virtual int Id { get; set; }
public virtual Product Product { get; set; }
public virtual int Order { get; set; }
}
public class Product {
public virtual int Id { get; set; }
public virtual string Name { get; set; }
public virtual IList<Model> Models { get; set; }
}
public class Model {
public virtual string ModelNumber { get; set; }
public virtual IList<Product> Products { get; set; }
}
Note: A product could have 1000s of models.
I need to display a list of TopProducts and the first 5 models (ordered alphabetically) against each one.
For example say I have the following query:
var topProducts = session.Query<TopProduct>()
.Cacheable()
.Fetch(tp => tp.Product).ThenFetchMany(p => p.Models)
.OrderBy(tp => tp.Order)
.ToList();
If I now say:
foreach (var topProduct in topProducts) {
var models = topProduct.Product.Models.Take(5).ToList();
...
}
This executes extremely slowly as it retrieves an item from the second level cache for each model. Since there could be 1000s of models against a product, it would need to retrieve 1000s of items from the cache the second time it is executed.
I have been racking my brain trying to think of a better way of doing this but so far I am out of ideas. Unfortunately my model and database cannot be modified at this stage.
I'd appreciate the help. Thanks
The key to your problem is understanding how entity and query caching work.
Entity caching stores, essentially, the POID of an entity and its property values.
When you want to get/initialize an instance, NH will first check the cache to see if the values are there, in order to avoid a db query.
Query caching, on the other hand, stores a query as the key (to simplify, let's say it's the command text and the parameter values), and a list of entity ids as the value (this is assuming your result is a list of entities, and not a projection)
When NH executes a cacheable query, it will see if the results are cached. If they are, it will load the proxies from those ids. Then, as you use them, it will initialize them one by one, either from the entity cache or from the db.
Collection cache is similar.
Usually, getting many second-level cache hits for those entity loads is a good thing. Unless, of course, you are using a distributed cache located in a separate machine, in which case this is almost as bad as getting them from the db.
If that is the case, I suggest you skip caching the query.
I want to create a new model object named Movie_Type in my ASP.NET MVC web application. What will be the differences if I define the navigation proprty of this class to be either List, ICollection or IQueryable as following?
public partial class Movie_Type
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public List<Movie> Movies { get; set; }
}
OR
public partial class Movie_Type
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public IQueryable<Movie> Movies { get; set; }
}
OR
public partial class Movie_Type
{
public int Id { get; set; }
public string Name { get; set; }
public string Description { get; set; }
public ICollection<Movie> Movies { get; set; }
}
Edit:-
#Tomas Petricek.
thanks for your reply. in my case i am using the database first approach and then i use DbContext template to map my tables, which automatically created ICollection for all the navigation properties, So my questions are:-
1. Does this mean that it is not always the best choice to use Icollection. And i should change the automatically generated classes to best fit my case.
2. Secondly i can manage to choose between lazy or Eager loading by defining .include such as
var courses = db.Courses.Include(c => c.Department);
Regardless of what i am using to define the navigation properties. So i can not understand ur point.
3. i did not ever find any examples or tutorials that use IQuerable to define the navigation properties ,, so what might be the reason?
BR
You cannot use a navigation property of type IQueryable<T>. You must use ICollection<T> or some collection type which implements ICollection<T> - like List<T>. (IQueryable<T> does not implement ICollection<T>.)
The navigation property is simply an object or a collection of objects in memory or it is null or the collection is empty.
It is never loaded from the database when you load the parent object which contains the navigation property from the database.
You either have to explicitely say that you want to load the navigation property together with the parent which is eager loading:
var movieTypes = context.Movie_Types.Include(m => m.Movies).ToList();
// no option to filter or sort the movies collection here.
// It will always load the full collection into memory
Or it will be loaded by lazy loading (which is enabled by default if your navigation property is virtual):
var movieTypes = context.Movie_Types.ToList();
foreach (var mt in movieTypes)
{
// one new database query as soon as you access properties of mt.Movies
foreach (var m in mt.Movies)
{
Console.WriteLine(m.Title);
}
}
The last option is explicit loading which comes closest to your intention I guess:
var movieTypes = context.Movie_Types.ToList();
foreach (var mt in movieTypes)
{
IQueryable<Movie> mq = context.Entry(mt).Collection(m => m.Movies).Query();
// You can use this IQueryable now to apply more filters
// to the collection or sorting, for example:
mq.Where(m => m.Title.StartWith("A")) // filter by title
.OrderBy(m => m.PublishDate) // sort by date
.Take(10) // take only the first ten of result
.Load(); // populate now the nav. property
// again this was a database query
foreach (var m in mt.Movies) // contains only the filtered movies now
{
Console.WriteLine(m.Title);
}
}
There are two possible ways of looking at things:
Is the result stored in memory as part of the object instance?
If you choose ICollection, the result will be stored in memory - this may not be a good idea if the data set is very large or if you don't always need to get the data. On the other hand, when you store the data in memory, you will be able to modify the data set from your program.
Can you refine the query that gets sent to the SQL server?
This means that you would be able to use LINQ over the returned property and the additional LINQ operators would be translated to SQL - if you don't choose this option, additional LINQ processing will run in memory.
If you want to store data in memory, then you can use ICollection. If you want to be able to refine the query, then you need to use IQueryable. Here is a summary table:
| | Refine query | Don't change query |
|-----------------|--------------|--------------------|
| In-memory | N/A | ICollection |
| Lazy execution | IQueryable | IEnumerable |
More of a standard is IEnumerable as it is the least common denominator.
Iqueryable can be returned if you want extra querying functionality to the caller without having 10 repository methods to handle varying querying scenarios.
A downside is ienumerable could 'count()' slowly but if the object implements ICollection then this interface is checked for this value first without having to enumerate all items.
Also be aware if you return iqueryable to an untrusted caller they can do some casting and method calls on the iqueryable and get access to the context, connection, connection string, run queries, etc
Also note nhibernate for example has a query object you can pass to a repository to specify options. With entity framework you need to return IQueryable to enhance querying criteria
The collection that entity framework actually creates for you if you use virtual navigation properties implements ICollection, but not IQueryable, so you cannot use IQueryable for your navigation properties, as Slauma says.
You are free to define your properties as IEnumerable, as ICollection extends IEnumerable, but if you do this then you will lose your ability to add new child items to these navigation properties.
I'm using the Entity Framework version 4.2. There are two classes in my small test app:
public class TestParent
{
public int TestParentID { get; set; }
public string Name { get; set; }
public string Comment { get; set; }
public virtual ICollection<TestChild> TestChildren { get; set; }
}
public class TestChild
{
public int TestChildID { get; set; }
public int TestParentID { get; set; }
public string Name { get; set; }
public string Comment { get; set; }
public virtual TestParent TestParent { get; set; }
}
Populating objects with data from the database works well. So I can use testParent.TestChildren.OrderBy(tc => tc.Name).First().Name etc. in my code.
Then I built a standard edit form for my testParents. The controller look like this:
public class TestController : Controller
{
private EFDbTestParentRepository testParentRepository = new EFDbTestParentRepository();
private EFDbTestChildRepository testChildRepository = new EFDbTestChildRepository();
public ActionResult ListParents()
{
return View(testParentRepository.TestParents);
}
public ViewResult EditParent(int testParentID)
{
return View(testParentRepository.TestParents.First(tp => tp.TestParentID == testParentID));
}
[HttpPost]
public ActionResult EditParent(TestParent testParent)
{
if (ModelState.IsValid)
{
testParentRepository.SaveTestParent(testParent);
TempData["message"] = string.Format("Changes to test parents have been saved: {0} (ID = {1})",
testParent.Name,
testParent.TestParentID);
return RedirectToAction("ListParents");
}
// something wrong with the data values
return View(testParent);
}
}
When the form is posted back to the server the model binding appears to be working well - i.e. testParent looks okay (id, name and comment set as expected). However the navigation property TestChildren remains at NULL.
This I guess is not sooo surprising since the model binding merely extracts the form values as they were sent from the browser and pushes them into an object of the TestParent class. Populating testParent.TestChildren however requires an immediate roundtrip to the database which is the responsibility of the Entity Framework. And EF probably doesn't get involved in the binding process.
I was however expecting the lazy loading to kick in when I call testParent.TestChildren.First(). Instead that leads to an ArgumentNullException.
Is it necessary to tag an object in a special way after model binding so that the Entity Framework will do lazy loading? How can I achieve this?
Obviously I could manually retrieve the children with the second repository testChildRepository. But that (a) doesn't feel right and (b) leads to problems with the way my repositories are set up (each using their own DBContext - which is an issue that I haven't managed to come to terms with yet).
In order to get lazy loading for your child collection two requirements must be fulfilled:
The parent entity must be attached to an EF context
Your parent entity must be a lazy loading proxy
Both requirements are met if you load the parent entity from the database through a context (and your navigation properties are virtual to allow proxy creation).
If you don't load the entity from the database but create it manually you can achieve the same by using the appropriate EF functions:
var parent = context.TestParents.Create();
parent.TestParentID = 1;
context.TestParents.Attach(parent);
Using Create and not new is important here because it creates the required lazy loading proxy. You can then access the child collection and the children of parent with ID = 1 will be loaded lazily:
var children = parent.TestChildren; // no NullReferenceException
Now, the default modelbinder has no clue about those specific EF functions and will simply instantiate the parent with new and also doesn't attach it to any context. Both requirements are not fulfilled and lazy loading cannot work.
You could write your own model binder to create the instance with Create() but this is probably the worst solution as it would make your view layer very EF dependent.
If you need the child collection after model binding I would in this case load it via explicit loading:
// parent comes as parameter from POST action method
context.TestParents.Attach(parent);
context.Entry(parent).Collection(p => p.TestChildren).Load();
If your context and EF is hidden behind a repository you will need a new repository method for this, like:
void LoadNavigationCollection<TElement>(T entity,
Expression<Func<T, ICollection<TElement>>> navigationProperty)
where TElement : class
{
_context.Set<T>().Attach(entity);
_context.Entry(entity).Collection(navigationProperty).Load();
}
...where _context is a member of the repository class.
But the better way, as Darin mentioned, is to bind ViewModels and then map them to your entities as needed. Then you would have the option to instantiate the entities with Create().
One possibility is to use hidden fields inside the form that will store the values of the child collection:
#model TestParent
#using (Html.BegniForm())
{
... some input fields of the parent
#Html.EditorFor(x => x.TestChildren)
<button type="submit">OK</button>
}
and then have an editor template for the children containing the hidden fields (~/Views/Shared/EditorTemplates/TestChild.cshtml):
#model TestChild
#Html.HiddenFor(x => x.TestChildID)
#Html.HiddenFor(x => x.Name)
...
But since you are not following good practices here and are directly passing your domain models to the view instead of using view models you will have a problem with the recursive relationship you have between the children and parents. You might need to manually populate the parent for each children.
But a better way would be to query your database in the POST action and fetch the children that are associated to the given parent since the user cannot edit the children inside the view anyway.
I have a big object that I resorted to serializing using #Html.Serialize():
[Serializable]
public class ModelB
{
public List<ModelA> ListOfModelA { get; set; }
// more stuff
}
This object contains a list of objects from a class that contains several properties. Some of them I include them in my view, while other I do not even bother to put them as hidden fields, as I have them in my serialized model.
[Serializable]
public class ModelA
{
public string StringA { get; set; }
public string StringB { get; set; }
// more stuff
public string HiddenStringA { get; set; }
public string HiddenStringB { get; set; }
// more stuff
}
Now, when I post back the form with my changes I reconstract my model and then I update it using the dictionary of values obtained from the form.
[HttpPost]
public ActionResult Edit([Deserialize] ModelTwo model,
FormCollection form)
{
TryUpdateModel(model, form.ToValueProvider());
// more stuff
}
I step in my code and just before I do the update, I see that my deserialized model contains a list ListOfModelA that in turn contains all the elements that should be there, and within them I can see all the HiddenStringA and HiddenStringB properties. Then I peek inside the form and I see a dictionary with keys like these:
ListOfModelA[0].StringA
ListOfModelA[0].StringB
ListOfModelA[1].StringB
ListOfModelA[1].StringB
while there are NO keys for the rest of the properties like this one:
ListOfModelA[0].HiddenStringA
Next, I move one step further and I let the code do the TryUpdateModel. Now, looking inside the ListOfModelA property, all the elements have been replaced with new ones that have all the hidden values null. It is as if the update reconstructed whole elements (with the limited information it had), rather than updating only the properties for which it had information.
Is this the expected behaviour? Is there a way to keep my model, and update only the properties that have keys in the dictionary?
Thanks,
Panos
My problem is more complex than the one described above, but the solultion that I found for the specific design is along the following lines. I serialize the part of the model that is a list, and bind the whole model through an argument of the action. This way I avoid using the TryUpdateModel, although this is not what causes me the trouble. Then, I inject the non-null values into the deserialized model from the model created from the form, and lastly I point the corresponding part of the latter to the former.
[HttpPost]
public ActionResult Edit(ModelTwo model,
[Deserialize] List<ModelA> serializedList)
{
serializedList.InjectFrom<NonNullLoopValueInjection>(model.ListOfModelA);
model.ListOfModelA = serializedList;
// more stuff
}
For the injection, I use Value Injecter where NonNullLoopValueInection is defined below
public class NonNullLoopValueInjection : LoopValueInjection
{
protected override bool AllowSetValue(object value)
{
return value != null;
}
}
Maybe not the best design, but it is working, and it might lead me to something better. Any feedback is more than welcome.