I ran across an interesting Linq to SQL, uh, feature, the other day. Perhaps someone can give me a logical explanation for the reasoning behind the results. Take the code below as my example which utilizes the AdventureWorks database setup in a Linq to SQL DataContext. This is a clip from my unit test. The resulting customer returned from a call to both CustomerQuery_Test_01() and CustomerQuery_Test_02() is the same. However, the query executed on the SQLServer are different is a major way. The method CustomerQuery_Test_01 us causing the entire Customer table to be materialized, which the call to CustomerQuery_Test_02 is only causing the single customer to be materialized. The resulting SQL Queries are at the bottom of this post. Anyone have a good reason for this? To me, it was highly non-intuitive.
protected virtual Customer GetByPrimaryKey(Func<Customer, bool> keySelection)
{
AdventureWorksDataContext context = new AdventureWorksDataContext();
return (from r in context.Customers select r).SingleOrDefault(keySelection);
}
[TestMethod]
public void CustomerQuery_Test_01()
{
Customer customer = GetByPrimaryKey(c => c.CustomerID == 2);
}
[TestMethod]
public void CustomerQuery_Test_02()
{
AdventureWorksDataContext context = new AdventureWorksDataContext();
Customer customer = (from r in context.Customers select r).SingleOrDefault(c => c.CustomerID == 2);
}
Query for CustomerQuery_Test_01 (notice the lack of a where clause)
SELECT [t0].[CustomerID], [t0].[NameStyle], [t0].[Title], [t0].[FirstName], [t0].[MiddleName], [t0].[LastName], [t0].[Suffix], [t0].[CompanyName], [t0].[SalesPerson], [t0].[EmailAddress], [t0].[Phone], [t0].[PasswordHash], [t0].[PasswordSalt], [t0].[rowguid], [t0].[ModifiedDate]
FROM [SalesLT].[Customer] AS [t0]
Query for CustomerQuery_Test_02 (notice the where clause)
SELECT [t0].[CustomerID], [t0].[NameStyle], [t0].[Title], [t0].[FirstName], [t0].[MiddleName], [t0].[LastName], [t0].[Suffix], [t0].[CompanyName], [t0].[SalesPerson], [t0].[EmailAddress], [t0].[Phone], [t0].[PasswordHash], [t0].[PasswordSalt], [t0].[rowguid], [t0].[ModifiedDate]
FROM [SalesLT].[Customer] AS [t0]
WHERE [t0].[CustomerID] = #p0
Func<Customer, bool> keySelection
That's not an Expression<Func<Customer, bool>>... the compiler resolved Enumerable.Single instead of Queryable.Single
Related
How do I left outer join two tables on two fields in linq?
I have a sql:
select a.*, b.* from courselist as a
left outer join Summary as b
on a.subject = b.Subject and a.catalog =
b.Catalogno
where a.degree_id = 1
order by a.sequenceNo
Below is my linq query, but there is error underline "join", failed in the call to "Groupjoin". I don't know how to correct that.
var searchResults = (from a in db.courselist
join b in db.Summary on
new { a.subject,a.catalog } equals
new { b.Subject, b.Catalogno } into ab
where a.degree_id == 1
orderby a.degree_sequenceNo
from b in ab.DefaultIfEmpty()
select new
{
Courselist = a,
Summary = b
}
).ToList();
Thanks.
I've checked your code again,
I found it's fault
you just need to specify join parameters name like this:
new { suject = a.subject, catalog = a.catalog } equals
new { suject = b.subject, catalog = b.Catalogno } into ab
It seems you are missing the reference, the query doesn't have an error
try to use this:
using System.Linq;
The main issue when people start using LINQ is that they keep thinking in the SQL way, they design the SQL query first and then translate it to LINQ. You need to learn how to think in the LINQ way and your LINQ query will become neater and simpler. For instance, in your LINQ you don't need joins. You should use Associations/Navigation Properties instead. Check this post for more details.
There should be a relationship between courselist and Summary, in which case, you can access Summary through courselist like this:
var searchResults = (from a in db.courselist
where a.degree_id == 1
orderby a.degree_sequenceNo
select new {
Courselist = a,
Summary = a.Summary
}).ToList();
If there is no relationship between the two, then you should reconsider your design.
I have two functions as follow -
public IQueryable<RequestSummaryDTO> GetProgramOfficerUSA(Guid officerId)
{
List<string> officerCountries = UnityProvider.Instance.Get<IProgramOfficerService>().GetPOCountries(officerId).Select(c => c.CNTR_ID.ToUpper()).ToList();
Expression<Func<TaskRequest, bool>> countriesFilter = (a) => officerCountries.Contains(a.tblTaskDetail.FirstOrDefault().tblOrganization.ORG_Country.ToUpper());
Expression<Func<TaskRequest, bool>> USAAndNotDelegated = LinqUtils.And(this.USAFilter(), this.NotDelegatedFilter(officerId));
Expression<Func<TaskRequest, bool>> countriesOrOwned = LinqUtils.Or(countriesFilter, this.OwnedFilter(officerId));
Expression<Func<TaskRequest, bool>> filter = LinqUtils.And(USAAndNotDelegated, countriesOrOwned);
return this.Get(filter, TaskRequestState.USA);
}
private IQueryable<RequestSummaryDTO> Get(Expression<Func<TaskRequest, bool>> additionalFilter, TaskRequestState? TaskRequestState = null)
{
var Tasks = this.TaskRequestRepository.List(additionalFilter).Where(x => x.tblTaskDetail.FirstOrDefault().PD_TaskRequestID == x.Id &&
(x.tblRequestDetail.AD_Status == (int)RequestStatus.Paid || x.tblRequestDetail.AD_Status == (int)RequestStatus.NotConfirmed));
if (TaskRequestState == TaskRequestState.USA)
{
Tasks = Tasks.Where(w => (w.tblTaskDetail.FirstOrDefault().PD_PStatus == null || w.tblTaskDetail.FirstOrDefault().PD_PStatus == 125));
}
else
{
Tasks = Tasks.Where(w => w.State == null);
}
return Tasks.ToList().Select(TaskSummaryFactory.CreateDto).AsQueryable();
}
I used Expression> and IQueryable in the LINQ. It is supposed to use LINQ To SQL instead of LINQ To Objects. The performance should be good.
But I do not see that. I believe the LINQ pulls every table's data into memory to process using LINQ To Objects. I do not see a join sql query sent to SQL Server tracking by SQL profile but a bunch of individual table selection statement.
It takes over 10 minutes to get something return from
return Tasks.ToList().Select(TaskSummaryFactory.CreateDto).AsQueryable();
I know the problem is in
Expression<Func<TaskRequest, bool>> countriesFilter = (a) => officerCountries.Contains(a.tblTaskDetail.FirstOrDefault().tblOrganization.ORG_Country.ToUpper());
tblTaskDetail is a big table. If I switch to a smaller one, the performance is noticeably improved.
Anyone can help find out why is wrong there.
Thanks,
Update 1 -
The statement from Entity Framework logged in SQL Profile are all like this -
exec sp_executesql N'SELECT [Extent1].[ORG_ID] AS [ORG_ID], [Extent1].[ORG_CreatedBy] AS [ORG_CreatedBy], [Extent1].[ORG_CreatedOn] AS [ORG_CreatedOn] FROM [dbo].[tblOrganization] AS [Extent1] WHERE [Extent1].[ORG_ID] = #EntityKeyValue1',N'#EntityKeyValue1 uniqueidentifier',#EntityKeyValue1='E8C3F120-AA40-445E-A8A0-2937F330D347'
They are all just having individual table select statement, not joined sql statement.
Update 2 -
I was wrong in Update 1. I missed the join SQL statement. The problem is that the generated SQL is too poor. There are 6 nested select statements, 11 LEFT OUTER JOINs, and 10 OUTER APPLYs. The query is too long and can not post here. Executing the generated SQL takes 9 minutes.
I have upgraded the EF STE 5 to EF6. Now the performance is better. The page loading time is cut down to 1.5 minutes from 12 minutes.
i have create a request in SQL and put them in dataset. apparently it hang when the data very huge. so i use an Entity.
my original sql is like this:
SELECT NO_ORDRE,ORDRE.CODE_DEST as CODE_DEST,REF_EXPED,ORDRE.MODAL_MODE,RS_NOM,ADRESSE,TEL,VILLE,
ORDRE.NBR_COLIS,ORDRE.POID,DATE_CREE,DATE_CLOTUR,STATUT_ORDRE,ORDRE.TRANSPORTEUR,ORDRE.LIB_TOURNE,
ORDRE.DATE_CLOTUR_REEL,ORDRE.OBS,AUTRE_REF,
ORDRE.CODE_CLIENT+'_'+CAST(NOID as VARCHAR(50))+'_'+SUBSTRING(NO_ORDRE_CUMMUL, 0, CHARINDEX('_', NO_ORDRE_CUMMUL + '_')) as NOLV
FROM ORDRE
LEFT OUTER JOIN LETTRE_VOIT_FINAL
ON charindex('_'+cast(ORDRE.NO_ORDRE as varchar(255))+'_', '_'+LETTRE_VOIT_FINAL.NO_ORDRE_CUMMUL+'_') > 0
WHERE DATE_CREE BETWEEN #DATE_CREE_DEB AND #DATE_CREE_FIN
ORDER BY NO_ORDRE DESC
and i try my linq like this:
public IQueryable<ORDRE> Get_OrdreEntity(DateTime datedeb, DateTime datefin)
{
try
{
IQueryable<ORDRE> LesListe;
Soft8Exp_ClientEntities oEntite_T = new Soft8Exp_ClientEntities();
var query = from o in oEntite_T.ORDRE
where o.DATE_CREE >= datedeb && o.DATE_CREE <= datefin
select o;
LesListe = query;
return LesListe;
}
catch (Exception excThrown)
{
throw new Exception("Err_02", excThrown);
}
}
it works well but i don't know how to make a join from this sql:
LEFT OUTER JOIN LETTRE_VOIT_FINAL
ON charindex('_'+cast(ORDRE.NO_ORDRE as varchar(255))+'_', '_'+LETTRE_VOIT_FINAL.NO_ORDRE_CUMMUL+'_') > 0
and how can i translate it to linq from this sql:
ORDRE.CODE_CLIENT+'_'+CAST(NOID as VARCHAR(50))+'_'+SUBSTRING(NO_ORDRE_CUMMUL, 0, CHARINDEX('_', NO_ORDRE_CUMMUL + '_')) as NOLV
I can't see any reason to have exception handling in the Get_OrdreEntity function. It should be coded in way it just work. Debug it. In any way you do nothing in catch.
I you query and filter data in this function and want to get results it is a good idea to return collection instead of query in the result of this function to eliminate performance and side effect isssues. I.e. return IEnumerable, ICollection, IList wherether you want.
It is easy to find a ton of Linq join examples, just use Google. Here is all you need.
I am using this query:
return from oi in NHibernateSession.Current.Query<BlaInteraction>()
select new BlaViewModel
{
...
NoPublications = oi.Publications.Count(),
...
};
BlaInteraction contains an IList of publications (i.e. entities). To determine the number of publications one does not really need to do all the joins for a publication. Can I prevent nhibernate from using joins in the generated sql (e.g. using projection???) somehow?
Thanks.
Christian
PS:
This is what NH produces (slightly adapted):
select cast(count(*) as INT) from RelationshipStatementPublications publicatio21_, Publication publicatio22_ inner join Statements publicatio22_1_ on publicatio22_.StatementId=publicatio22_1_.DBId where publicatio21_.StatementId = 22762181 and publicatio21_.PublicationId=publicatio22_.StatementId
This is what would be sufficient:
select cast(count(*) as INT) from RelationshipStatementPublications publicatio21_ where publicatio21_.StatementId = 22762181
Why can't you just create another query ?
Session.QueryOver<Publication>().Where(x => x.BlaInteractionId == idSentAsParameter).Select(Projections.RowCount()).SingleOrDefault<int>();
I think that's will work
return from oi in NHibernateSession.Current.Query<BlaInteraction>()
select new BlaViewModel
{
...
NoPublications = Session.QueryOver<Publication>().Where(x => x.BlaInteractionId == oi.Id).Select(Projections.RowCount()).SingleOrDefault<int>();
...
};
Another edit, have you tried lazy="extra" ?
Ok the best solution I have found so far is to use a FNH Formula:
mapping.Map(x => x.NOPublications).Formula("(select count(distinct RelationshipStatementPublications.PublicationId) from RelationshipStatementPublications where RelationshipStatementPublications.StatementId = DBId)");
public virtual int NOPublications {get; private set;}
when I map from the domain to the view model I use:
NoPublications = oi.NOPublications,
Christian
After profiling my Entity Framework 4.0 based database layer I have found the major performance sinner to be a simple LINQ Any() I use to check if an entity is already existing in the database. The Any() check performs orders of magnitude slower than saving the entity.
There are relatively few rows in the database and the columns being checked are indexed.
I use the following LINQ to check for the existence of a setting group:
from sg in context.SettingGroups
where sg.Group.Equals(settingGroup) && sg.Category.Equals(settingCategory)
select sg).Any()
This generates the following SQL (additionally my SQL profiler claims the query is executed twice):
exec sp_executesql N'SELECT
CASE WHEN ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[SettingGroups] AS [Extent1]
WHERE ([Extent1].[Group] = #p__linq__0) AND ([Extent1].[Category] = #p__linq__1)
)) THEN cast(1 as bit) WHEN ( NOT EXISTS (SELECT
1 AS [C1]
FROM [dbo].[SettingGroups] AS [Extent2]
WHERE ([Extent2].[Group] = #p__linq__0) AND ([Extent2].[Category] = #p__linq__1)
)) THEN cast(0 as bit) END AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]',N'#p__linq__0 nvarchar(4000),#p__linq__1 nvarchar(4000)',#p__linq__0=N'Cleanup',#p__linq__1=N'Mediator'
Right now I can only think of creating stored procedures to solve this problem, but I would of course prefer to keep the code in LINQ.
Is there a way to make such an "Exist" check run faster with EF?
I should probably mention that I also use self-tracking-entities in an n-tier architecture. In some scenarios the ChangeTracker state for some entities is set to "Added" even though they already exist in the database. This is why I use a check to change the ChangeTracker state accordingly if updating the database caused an insert failure exception.
Try adding index to the database table "SettingGroups", by Group & Category.
BTW, does this produce similar sql?
var ok = context.SettingGroups.Any(sg => sg.Group==settingGroup && sg.Category==settingCategory);
The problem is Entity Framework (at least EF4) is generating stupid SQL. The following code seems to generate decent SQL with minimal pain.
public static class LinqExt
{
public static bool BetterAny<T>( this IQueryable<T> queryable, Expression<Func<T, bool>> predicate)
{
return queryable.Where(predicate).Select(x => (int?)1).FirstOrDefault().HasValue;
}
public static bool BetterAny<T>( this IQueryable<T> queryable)
{
return queryable.Select(x => (int?)1).FirstOrDefault().HasValue;
}
}
Then you can do:
(from sg in context.SettingGroups
where sg.Group.Equals(settingGroup) && sg.Category.Equals(settingCategory)
select sg).BetterAny()
or even:
context.SettingGroups.BetterAny(sg => sg.Group.Equals(settingGroup) && sg.Category.Equals(settingCategory));
I know it sounds a miserable solution, but what happens if you use Count instead of Any?
Have you profiled the time to execute the generated select statement against the time to execute the select what you would expect/like to be produced ? It is possible that it is not as bad as it looks.
The section
SELECT
1 AS [C1]
FROM [dbo].[SettingGroups] AS [Extent1]
WHERE ([Extent1].[Group] = #p__linq__0) AND ([Extent1].[Category] = #p__linq__1)
is probably close to what you would expect to be produced. It is quite possible that the query optimiser will realise the second query is the same as the first and hence it may add very little time to the overall query.