EF core 2 first query slow - linq

I'm using EF core 2 as ORM in my project.
I faced this problem while executing this query:
var query = (from droitsGeo in _entities.DroitsGeos
join building in _entities.Batiments
on droitsGeo.IdPerimetre equals building.IdBatiment
where droitsGeo.IdUtilisateur == idUser &&
droitsGeo.IdClient == idClient &&
building.Valide == true &&
droitsGeo.IdNiveauPerimetre == geographicalLevel
orderby sort ascending
select new GeographicalModel
{
Id = building.IdBatiment,
IdParent = building.IdEtablissement,
Label = building.LibBatiment,
});
First execution tooks about 5 second and second less than one second as show below :
First execution of query :
Time elapsed EF: 00:00:04.8562419
After first execution of query :
Time elapsed EF: 00:00:00.5496862
Time elapsed EF: 00:00:00.6658079
Time elapsed EF: 00:00:00.6176030
I have same result using Stored procedure.
When i execute sql query generated by EF in SQL Server, the result is returned in less than a second.
what is wrong with EF Core 2 or did i miss something in configuration?

The EF by default tracks all the entities you run queries against.
When you run it for the first time the track change mechanism kicks in... that's why it takes a little bit longer.
You can avoid this, especially when retrieving collections by using .AsNoTracking() when composing the query.
Take a look:
var items = DbContext.MyDbSet
.Include(SecondObject)
.AsNoTracking()
.ToList();

EF core needs to compile LINQ quires using reflection therefor first queries are always slow. There is already a GitHub issue here

I have a simple idea to resolve this issue with the help of stored procedures and thereafter AutoMapper.
Create an stored procedures that return all the columns that you want, no matter if they are from different tables. Once the data is received from the stored procedure and you have received the object in one of your Model classes, you can then use AutoMapper to map only the relevant attributes to other classes. Please note that I am not giving you a tutorial of how to use stored procedure. I am giving you an example that might explain better:
A stored procedure is created which returns results from three tables named A, B and C.
A model class named SP_Result.cs is created corresponding to created stored procedure to map the received object of stored procedure (this is required when working with stored procedures in EF Core)
'ViewModels` are created having same attributes as returning from each table A, B and C.
Thereafter, mapping configurations will be created for SP_Result with ViewModel of Class A, Class B and Class C. e.g. CreateMap<SP_Result, ViewModel_A>(); CreateMap<SP_Result, ViewModel_B>();. I suppose, you would have a request and response objects which can be used instead of ViewModels. Name the properties accordingly in the stored procedure using AS keyword. e.g. Select std_Name AS 'Name'
This mapping will map the individual properties to each class. AutoMapper ignore the properties which do not exists in either of the classes mentioned in Mapping Configuration.
If you are selecting a list of objects where each object does have its own list of objects, this scenario will generally create N + 1 queries in EF. In fact, if you try to achieve this using stored procedures, you will have to create multiple queries or run the stored procedure multiple times (in a loop may be), or you will end up receiving Cartesian product.

Related

How can Linq be so incredibly fast? C#

Let's say I have 100 000 objects of type Person which have a date property with their birthday in them.
I place all the objects in a List<Person> (or an array) and also in a dictionary where I have the date as the key and every value is a array/list with persons that share the same birthday.
Then I do this:
DateTime date = new DateTime(); // Just some date
var personsFromList = personList.Where(person => person.Birthday == date);
var personsFromDictionary = dictionary[date];
If I run that 1000 times the Linq .Where lookup will be significantly faster in the end than the dictionary. Why is that? It does not seem logical to me. Is the results being cached (and used again) behind the scenes?
From Introduction to LINQ Queries (C#) (The Query)
... the important point is that in LINQ, the query variable itself takes no action and returns no data. It just stores the information that is required to produce the results when the query is executed at some later point.
This is known as deferred execution. (later down the same page):
As stated previously, the query variable itself only stores the query commands. The actual execution of the query is deferred until you iterate over the query variable in a foreach statement. This concept is referred to as deferred execution...
Some linq methods must iterate the IEnumerable and therefor will execute immediately - methods like Count, Max, Average etc' - all the aggregation methods.
Another way to force immediate execution is to use ToArray or ToList, which will execute the query and store it's results in an array or list.

Lightswitch 2013 Linq queries to Get min value

I'm writing a timesheet application (Silverlight) and I'm completely stuck on getting linq queries working. I'm netw to linq and I just read, and did many examples from, a Linq book, including Linq to Objects, linq to SQl and linq to Entities.(I assume, but am not 100% sure that the latter is what Lightswitch uses). I plan to study a LOT more Linq, but just need to get this one query working.
So I have an entity called Items which lists every item in a job and it's serial no
So: Job.ID int, ID int, SerialNo long
I also have a Timesheets entity that contains shift dates, job no and start and end serial no produced
So Job.ID int, ShiftDate date, Shift int, StartNo long, EndNo long
When the user select a job from an autocomplete box, I want to look up the MAX(SerialNo) for that job in the timesheets entity. If that is null (i.e. none have been produced), I want to lookup the MIN(SerialNo) from the Items entity for that job (i.e. what's the first serial no they should produce)
I realize I need a first or default and need to specify the MIN(SerialNo) from Items as a default.
My Timesheet screen uses TimesheetProperty as it's datasource
I tried the following just to get the MAX(SerialNo) from Timesheets entity:
var maxSerialNo =
(from ts in this.DataWorkspace.SQLData.Timesheets
where ts.Job.ID == this.TimesheetProperty.Job.ID
select ts.StartNo).Min();
but I get the following errors:
Instance argument: cannot convert from 'Microsoft.LightSwitch.IDataServiceQueryable' to 'System.Collections.Generic.IEnumerable
'Microsoft.LightSwitch.IDataServiceQueryable' does not contain a definition for 'Min' and the best extension method overload 'System.Linq.Enumerable.Min(System.Collections.Generic.IEnumerable)' has some invalid arguments
I also don't get why I can't use this:
var maxSerialNo = this.DataWorkspace.SQLData.Timesheets.Min(ts => ts.StartNo);
Can anyone point me in the right direction?
Thanks
Mark
IDataServiceQueryable doesn't support full set of LINQ operator like IEnumerable has.
IDataServiceQueryable – This is a LightSwitch-specific type that allows a restricted set of “LINQ-like” operators that are remote-able to the middle-tier and ultimately issued to the database server. This interface is the core of the LightSwitch query programming model. IDataServiceQueryable has a member to execute the query, which returns results that are IEnumerable. [Reference]
Possible solution is, execute your query first to get collection of type IEnumerable by calling .ToList(), then you can call .Min() against the first query result. But that isn't good idea if you have large amount of data, because .ToList() will retrieve all data match the query and do further processing in client side, which is inefficient.
Another way is, change your query using only operators supported by IDataServiceQueryable to avoid retrieving unnecessary data to client. For example, to get minimum StartNo you can try to use orderby descending then get the first data instead of using .Min() operator :
var minStartNo =
(
from ts in this.DataWorkspace.SQLData.Timesheets
where ts.Job.ID == this.TimesheetProperty.Job.ID
orderby ts.StartNo descending select ts
).FirstOrDefault();

Data Entity Framework and LINQ - Get giant data set and execute a command one at a time on each object

We are pulling in a giant dataset of records (in the 100's of thousands) and then need to update a field on each one, one at a time in an atomic transation. They records are unrelated to each other and we don't want to do a blind update to all couple hundred thousand (there are views and indexes on this table that make that very prohibitive). The ONLY way that I could get this to work without doing a giant transation was as follows (container is a reference to a custom ObjectContext):
var expiredWorkflows = from iw in container.InitiatedWorkflows
where iw.InitiationStatusID != 1 && iw.ExpirationDate < DateTime.Now
select iw.ID;
foreach (int expiredWorkflow in expiredWorkflows)
container.ExecuteStoreCommand("UPDATE dbo.InitiatedWorkflow SET InitiationStatusID = 7 WHERE ID = #ID", new SqlParameter() { ParameterName = "#ID", Value = expiredWorkflow.ToString() } );
We tried looping through each one and just updating the field via the container and then calling SaveChanges(), but that runs everything as one transaction. We tried calling SaveChanges() in the foreach loop, but that threw transaction exceptions. Is there any way to what we are trying to do using the ObjectContext, so it would do something like (the above select would be changed to return the full object, not just the ID):
foreach (var expiredWorkflow in expiredWorkflows)
expiredWorkflow.InitiationStatusID = 7
container.SaveChanges(SaveOptions.OneAtATime);
Speaking generally, if the operation you need to carry out is as simple as the sort of UPDATE your code above suggests, this is the sort of operation that will run far better on the back end database--assuming, of course, there's some clear way to select only the rows that need to be changed. Entity Framework is intended more for manipulating small to medium sets of objects that can easily be loaded into memory and twiddled there, not large bulk-processing operations for which stored procedures are often best. EF can certainly perform those big operations, but it will take a lot longer to execute one SQL statement per row.

WCF Data Services - neither .Expand or .LoadProperty seems to do what I need

I am building a school management app where they track student tardiness and absences. I've got three entities to help me in this. A Students entity (first name, last name, ID, etc.); a SystemAbsenceTypes entity with SystemAbsenceTypeID values for Late, Absent-with-Reason, Absent-without-Reason; and a cross-reference table called StudentAbsences (matching the student IDs with the absence-type ID, plus a date, and a Notes field).
What I want to do is query my entities for a given student, and then add up the number of each kind of Absence, for a given date range. I prepare my currentStudent object without a problem, then I do this...
Me.Data.LoadProperty(currentStudent, "StudentAbsences") 'Loads the cross-ref data
lblDaysLate.Text = (From ab In currentStudent.StudentAbsences Where ab.SystemAbsenceTypes.SystemAbsenceTypeID = Common.enuStudentAbsenceTypes.Late).Count.ToString
...and this second line fails, complaining "Object reference not set to an instance of an object."
I presume the problem is that while it DOES see that there are (let's say) four absences for the currentStudent (ie, currentStudent.StudentAbsences.Count = 4) -- it can't yet "peer into" each one of the absences to look at its type. In fact, each of the four StudentAbsence objects has a property called SystemAbsenceType, which then finally has the SystemAbsenceTypeID.
How do I use .Expand or .LoadProperty to make this happen? Do I need to blindly loop through all these collections, firing off .LoadProperty on everything before I can do my query?
Is there some other technique?
When you load the student, try expanding the related properties.
var currentStudent = context.Students.Expand("StudentAbsences")
.Expand("StudentAbsences/SystemAbsenceTypes")
.Where(....).First();

Linq stored procedure with dynamic results

So I'm extremely new to Linq in .Net 3.5 and have a question. I use to use a custom class that would handle the following results from a store procedure:
Set 1: ID Name Age
Set 2: ID Address City
Set 3: ID Product Price
With my custom class, I would have received back from the database a single DataSet with 3 DataTables inside of it with columns based on what was returned from the DB.
My question is how to I achive this with LINQ? I'm going to need to hit the database 1 time and return multiple sets with different types of data in it.
Also, how would I use LINQ to return a dynamic amount of sets depending on the parameters (could get 1 set back, could get N amount back)?
I've looked at this article, but didn't find anything explaining multiple sets (just a single set that could be dynamic or a single scalar value and a single set).
Any articles/comments will help.
Thanks
I believe this is what you're looking for
Linq to SQL Stored Procedures with Multiple Results - IMultipleResults
I'm not very familiar with LINQ myself but here is MSDN's site on LINQ Samples that might be able to help you out.
EDIT: I apologize, I somehow missed the title where you mentioned you wanted help using LINQ with Stored Procedures, my below answer does not address that at all and unfortunately I haven't had the need to use sprocs with LINQ so I'm unsure if my below answer will help.
LINQ to SQL is able hydrate multiple sets of data into a object graph while hitting the database once. However, I don't think LINQ is going to achieve what you ultimately want -- which as far as I can tell is a completely dynamic set of data that is defined outside of the query itself. Perhaps I am misunderstanding the question, maybe it would help if you provide some sample code that your existing application is using?
Here is a quick example of how I could hydrate a anonymous type with a single database call, maybe it will help:
var query = from p in db.Products
select new
{
Product = p,
NumberOfOrders = p.Orders.Count(),
LastOrderDate = p.Orders.OrderByDescending().Take(1).Select(o => o.OrderDate),
Orders = p.Orders
};

Resources