Below is a snippit of simplified version of a problem I am having with Entity Framework v4 where the first load seems to take around 30 seconds on a table with 36 rows!
After that it is very quick to load until you change the search params, then it takes 30 seconds again but once that combination of search params has been done once it is quick.
This is repeated each time a different combination of params is used.
IQueryable<User> result= GetAllUsers();
if (!String.IsNullOrWhiteSpace(firstNameSearchParam))
{
result = result.Where(u => u.firstname.contains(firstNameSearchParam))
}
if (!String.IsNullOrWhiteSpace(lastNameSearchParam))
{
result = result.Where(u => u.lastname.contains(lastNameSearchParam))
}
Var ret = result.ToArray();
Any ideas would be really appreciated.
I'm not sure if pre-compiling the views will help. I tried but couldn't get it to work.
how long it takes when you execute the query from the sql side?
you can use this idea mentioned by scott here on this link Dynamic Linq
and i think this will work with you also with Entity Framework, and there is another idea that you can use the entity framework metadata
hope that this will help you
regards
Related
I'm using EF 5 with Oracle database.
I'm doing a select count in a table with a specific parameter. When I'm using EF, the query returns the value 31, as expected, But the result takes about 10 seconds to be returned.
using (var serv = new Aperam.SIP.PXP.Negocio.Modelos.SIP_PA())
{
var teste = (from ens in serv.PA_ENSAIOS_UM
where ens.COD_IDENT_UNMET == "FBLDY3840"
select ens).Count();
}
If I execute the simple query bellow the result is the same (31), but the result is showed in 500 milisecond.
SELECT
count(*)
FROM
PA_ENSAIOS_UM
WHERE
COD_IDENT_UNMET 'FBLDY3840'
There are a way to improve the performance when I'm using EF?
Note: There are 13.000.000 lines in this table.
Here are some things you can try:
Capture the query that is being generated and see if it is the same as the one you are using. Details can be found here, but essentially, you will instantiate your DbContext (let's call it "_context") and then set the Database.Log property to be the logging method. It's fine if this method doesn't actually do anything--you can just set a breakpoint in there and see what's going on.
So, as an example: define a logging function (I have a static class called "Logging" which uses nLog to write to files)
public static void LogQuery(string queryData)
{
if (string.IsNullOrWhiteSpace(queryData))
return;
var message = string.Format("{0}{1}",
queryData.Trim().Contains(Environment.NewLine) ?
Environment.NewLine : "", queryData);
_sqlLogger.Info(message);
_genLogger.Trace($"EntityFW query (len {message.Length} chars)");
}
Then when you create your context point to LogQuery:
_context.Database.Log = Logging.LogQuery;
When you do your tests, remember that often the first run is the slowest because the server has to actually do the work, but on the subsequent runs, it often uses cached data. Try running your tests 2-3 times back to back and see if they don't start to run in the same time.
I don't know if it generates the same query or not, but try this other form (which should be functionally equivalent, but may provide better time)
var teste = serv.PA_ENSAIOS_UM.Count(ens=>ens.COD_IDENT_UNMET == "FBLDY3840");
I'm wondering if the version you have pulls data from the DB and THEN counts it. If so, this other syntax may leave all the work to be done at the server, where it belongs. Not sure, though, esp. since I haven't ever used EF with Oracle and I don't know if it behaves the same as SQL or not.
Entity Framework is building a SQL query that takes less than 2 seconds. Even though the query is ugly, it's fast. In the EF SELECT statementI'm building a DTO based on my entites. The DTO has around 20 properties and it's plain. So, EF is running a query that returns these 20 properties I need. Everything seems fine but building the DTO takes almost 10 seconds. I have already told EF not to track changes (AsNoTracking() over the db sets) so I'm not sure what's happening between the query is executed and the DTO is built.
What's EF doing under the hood that makes it so slow? Again, the query is fine and returns really fast but building the DTO with those properties is really slow.
I replaced this logic by a stored procedure that returns the same 20 properties and it is really, really fast. The DTO is the same so there has to be something in the middle, between reading the results of the query and building the DTO that is different.
I hope someone can help me figure out what's going on.
EDIT:
I'm adding how I'm populating the DTO based on a comment.
.Select(p => new PMPanelOffersDTO()
{
OfferId = p.Id,
OfferDate = p.DateCreated,
UserId = p.UserId,
OfferAmountId = p.AmountId,
OfferAmountTypeId = p.Amount != null ? p.Amount.OfferType.Id : default(int?),
OfferAmountType = p.Amount != null ? p.Amount.OfferType.Name : null,
TowerCompanyName = p.Amount != null && p.Amount.Tower != null ? p.Amount.Tower.CompanyName : null,
... { the same for the other properties }
Do you apply paging? (I mean skip&take) And most importantly where exactly ToList() is used?
We have exactly the same problem, solved in a such a way that use ToList() after DTO construction.
Try using .ToList() on the resulting object. That's all it took in our case. I agree it made no sense. Apparently the IEnumerable<> was not a happy thing
Time to leave the shy mode behind and make my first post on stackoverflow.
After doing loads of research (plugins, performance, indexes, types of update, friends) and after trying several approaches I was unable to find a proper answer/solution.
So if possible I would like to get your feedback/help in a Microsoft Dynamics CRM 2013/2015 plugin performance issue (or coding technique)
Scenario:
Microsoft Dynamics CRM 2013/2015
2 Entities with Relationship 1:N
EntityA
EntityB
EntityB has the following columns:
Id | EntityAId | ColumnDemoX (decimal) | ColumnDemoY (currency)
Entity A has: 500 records
Entity B has: 150 records per each Entity A record. So 500*150 = 75000 records.
Objective:
Create a Post Entity A Plugin Update to "mimic" the following SQL command
Update EntityB
Set ColumnDemoX = (some quantity), ColumnDemoY = (some quantity) * (some value)
Where EntityAId = (some id)
One approach could be:
using (var serviceContext = new XrmServiceContext(service))
{
var query = from a in serviceContext.EntityASet
where a.EntityAId.Equals(someId)
select a;
foreach (EntityA entA in query)
{
entA.ColumnDemoX = (some quantity);
serviceContext.UpdateObject(entA);
}
serviceContext.SaveChanges();
}
Problem:
The foreach for 150 records in the post plugin update will take 20 secs or more.
While the
Update EntityB Set ColumnDemoX = (some quantity), ColumnDemoY = (some quantity) * (some value) Where EntityAId = (some id)
it will take 0.00001 secs
Any suggestion/solution?
Thank you all for reading.
H
You can use the ExecuteMultipleRequest, when you iterate the 150 entities, save the entities you need to update and after that call the request. If you do this, you only call the service once, that's very good for the perfomance.
If your process could be bigger and bigger, then you should think making it asynchronous as a plug-in or a custom activity workflow.
This is an example:
// Create an ExecuteMultipleRequest object.
requestWithResults = new ExecuteMultipleRequest()
{
// Assign settings that define execution behavior: continue on error, return responses.
Settings = new ExecuteMultipleSettings()
{
ContinueOnError = false,
ReturnResponses = true
},
// Create an empty organization request collection.
Requests = new OrganizationRequestCollection()
};
// Add a UpdateRequest for each entity to the request collection.
foreach (var entity in input.Entities)
{
UpdateRequest updateRequest = new UpdateRequest { Target = entity };
requestWithResults.Requests.Add(updateRequest);
}
// Execute all the requests in the request collection using a single web method call.
ExecuteMultipleResponse responseWithResults =
(ExecuteMultipleResponse)_serviceProxy.Execute(requestWithResults);
Few solutions comes to mind but I don't think they will please you...
Is this really a problem ? Yes it's slow and database update can be so much faster. However if you can have it as a background process (asynchronous), you'll have your numbers anyway. Is it really a "I need this numbers in the next second as soon as I click or business will go down" situation ?
It can be a reason to ditch 2013. In CRM 2015 you can use a calculated field. If you need this numbers only to show up in forms (eg. you don't use them in reporting), you could also do it in javascript.
Warning this is for the desesperate call. If you really need your update to be synchronous, immediate, you can't use calculated fields, you really know what your doing etc... Why not do it directly in the database? I know this is a very bad advice. There are a lot of reason not to do it this way (you can read a few here). It's unsupported and if you do something wrong it could go really bad. But if your real situation is as simple as your example (just a calculated field, no entity creation, no relation modification), you could do it this way. You'll have to consider many things: you won't have any audit on the fields, no security, caching issues, no modified by, etc. Actually I pretty much advise against this solution.
1 - Put it this logic to async workflow.
OR
2 - Don't use
serviceContext.UpdateObject(entA);
serviceContext.SaveChanges();.
Get all the records (150) from post stage update the fields and ExecuteMultipleRequest to update crm records in one time.
Don't send update request for each and every record
is there a way to chain queries in the "Play!" framework instead of manually writing a query ?
Something like:
Model m = Model.where("name","Bill").where("gender","m").first();
I wonder because I recently switched from Codeigniter + dmz to grails then now to "Play!" and have been a bit disappointed by the queries chaining compared to the framework mentioned above
PS: I'm obviously not talking about the fetch() and the from() methods.
There is a JpqlSelect object with wich you can do
public static Model findBill() {
JpqlSelect select = new JpqlSelect().from("Model").where("name=?").param("Bill").andWhere("gender=?").param("m");
return find(select.toString(), select.getParams().toArray()).first();
}
This helper is far from being complete but it may suits your needs
You can do it with java.persistence Criteria API like that:
CriteriaQuery q = JPA.em().getCriteriaBuilder().createQuery(Model.class)
q.where(<Expression>).where(<Expression>);
TypedQuery<Model> tq = JPA.em().createQuery(q);
List<Model> result = tq.getResultList();
Look here and here for start
An other option could be using http://www.querydsl.com/ with it's jpa integration.
I've been developing a webapp using Linq to NHibernate for the past few months, but haven't profiled the SQL it generates until now. Using NH Profiler, it now seems that the following chunk of code hits the DB more than 3,000 times when the Linq expression is executed.
var activeCaseList = from c in UserRepository.GetCasesByProjectManagerID(consultantId)
where c.CompletionDate == null
select new { c.PropertyID, c.Reference, c.Property.Address, DaysOld = DateTime.Now.Subtract(c.CreationDate).Days, JobValue = String.Format("£{0:0,0}", c.JobValue), c.CurrentStatus };
Where the Repository method looks like:
public IEnumerable<Case> GetCasesByProjectManagerID(int projectManagerId)
{
return from c in Session.Linq<Case>()
where c.ProjectManagerID == projectManagerId
select c;
}
It appears to run the initial Repository query first, then iterates through all of the results checking to see if the CompletionDate is null, but issuing a query to get c.Property.Address first.
So if the initial query returns 2,000 records, even if only five of them have no CompletionDate, it still fires off an SQL query to bring back the address details for the 2,000 records.
The way I had imagined this would work, is that it would evaluate all of the WHERE and SELECT clauses and simply amalgamate them, so the inital query would be like:
SELECT ... WHERE ProjectManager = #p1 AND CompleteDate IS NOT NULL
Which would yield 5 records, and then it could fire the further 5 queries to obtain the addresses. Am I expecting too much here, or am I simply doing something wrong?
Anthony
Change the declaration of GetCasesByProjectManagerID:
public IQueryable<Case> GetCasesByProjectManagerID(int projectManagerId)
You can't compose queries with IEnumerable<T> - they're just sequences. IQueryable<T> is specifically designed for composition like this.
Since I can't add a comment yet. Jon Skeet is right you'll want to use IQueryable, this is allows the Linq provider to Lazily construct the SQL. IEnumerable is the eager version.