Linq-to-entities - Include() method not loading - linq

If I use a join, the Include() method is no longer working, eg:
from e in dc.Entities.Include("Properties")
join i in dc.Items on e.ID equals i.Member.ID
where (i.Collection.ID == collectionID)
select e
e.Properties is not loaded
Without the join, the Include() works
Lee

UPDATE: Actually I recently added another Tip that covers this, and provides an alternate probably better solution. The idea is to delay the use of Include() until the end of the query, see this for more information: Tip 22 - How to make include really include
There is known limitation in the Entity Framework when using Include().
Certain operations are just not supported with Include.
Looks like you may have run into one on those limitations, to work around this you should try something like this:
var results =
from e in dc.Entities //Notice no include
join i in dc.Items on e.ID equals i.Member.ID
where (i.Collection.ID == collectionID)
select new {Entity = e, Properties = e.Properties};
This will bring back the Properties, and if the relationship between entity and Properties is a one to many (but not a many to many) you will find that each resulting anonymous type has the same values in:
anonType.Entity.Properties
anonType.Properties
This is a side-effect of a feature in the Entity Framework called relationship fixup.
See this Tip 1 in my EF Tips series for more information.

Try this:
var query = (ObjectQuery<Entities>)(from e in dc.Entities
join i in dc.Items on e.ID equals i.Member.ID
where (i.Collection.ID == collectionID)
select e)
return query.Include("Properties")

So what is the name of the navigation property on "Entity" which relates to "Item.Member" (i.e., is the other end of the navigation). You should be using this instead of the join. For example, if "entity" add a property called Member with the cardinality of 1 and Member had a property called Items with a cardinality of many, you could do this:
from e in dc.Entities.Include("Properties")
where e.Member.Items.Any(i => i.Collection.ID == collectionID)
select e
I'm guessing at the properties of your model here, but this should give you the general idea. In most cases, using join in LINQ to Entities is wrong, because it suggests that either your navigational properties are not set up correctly, or you are not using them.

So, I realise I am late to the party here, however I thought I'd add my findings. This should really be a comment on Alex James's post, but as I don't have the reputation it'll have to go here.
So my answer is: it doesn't seem to work at all as you would intend. Alex James gives two interesting solutions, however if you try them and check the SQL, it's horrible.
The example I was working on is:
var theRelease = from release in context.Releases
where release.Name == "Hello World"
select release;
var allProductionVersions = from prodVer in context.ProductionVersions
where prodVer.Status == 1
select prodVer;
var combined = (from release in theRelease
join p in allProductionVersions on release.Id equals p.ReleaseID
select release).Include(release => release.ProductionVersions);
var allProductionsForChosenRelease = combined.ToList();
This follows the simpler of the two examples. Without the include it produces the perfectly respectable sql:
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name]
FROM [dbo].[Releases] AS [Extent1]
INNER JOIN [dbo].[ProductionVersions] AS [Extent2] ON [Extent1].[Id] = [Extent2].[ReleaseID]
WHERE ('Hello World' = [Extent1].[Name]) AND (1 = [Extent2].[Status])
But with, OMG:
SELECT
[Project1].[Id1] AS [Id],
[Project1].[Id] AS [Id1],
[Project1].[Name] AS [Name],
[Project1].[C1] AS [C1],
[Project1].[Id2] AS [Id2],
[Project1].[Status] AS [Status],
[Project1].[ReleaseID] AS [ReleaseID]
FROM ( SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Name] AS [Name],
[Extent2].[Id] AS [Id1],
[Extent3].[Id] AS [Id2],
[Extent3].[Status] AS [Status],
[Extent3].[ReleaseID] AS [ReleaseID],
CASE WHEN ([Extent3].[Id] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C1]
FROM [dbo].[Releases] AS [Extent1]
INNER JOIN [dbo].[ProductionVersions] AS [Extent2] ON [Extent1].[Id] = [Extent2].[ReleaseID]
LEFT OUTER JOIN [dbo].[ProductionVersions] AS [Extent3] ON [Extent1].[Id] = [Extent3].[ReleaseID]
WHERE ('Hello World' = [Extent1].[Name]) AND (1 = [Extent2].[Status])
) AS [Project1]
ORDER BY [Project1].[Id1] ASC, [Project1].[Id] ASC, [Project1].[C1] ASC
Total garbage. The key point to note here is the fact that it returns the outer joined version of the table which has not been limited by status=1.
This results in the WRONG data being returned:
Id Id1 Name C1 Id2 Status ReleaseID
2 1 Hello World 1 1 2 1
2 1 Hello World 1 2 1 1
Note that the status of 2 is being returned there, despite our restriction. It simply does not work.
If I have gone wrong somewhere, I would be delighted to find out, as this is making a mockery of Linq. I love the idea, but the execution doesn't seem to be usable at the moment.
Out of curiosity, I tried the LinqToSQL dbml rather than the LinqToEntities edmx that produced the mess above:
SELECT [t0].[Id], [t0].[Name], [t2].[Id] AS [Id2], [t2].[Status], [t2].[ReleaseID], (
SELECT COUNT(*)
FROM [dbo].[ProductionVersions] AS [t3]
WHERE [t3].[ReleaseID] = [t0].[Id]
) AS [value]
FROM [dbo].[Releases] AS [t0]
INNER JOIN [dbo].[ProductionVersions] AS [t1] ON [t0].[Id] = [t1].[ReleaseID]
LEFT OUTER JOIN [dbo].[ProductionVersions] AS [t2] ON [t2].[ReleaseID] = [t0].[Id]
WHERE ([t0].[Name] = #p0) AND ([t1].[Status] = #p1)
ORDER BY [t0].[Id], [t1].[Id], [t2].[Id]
Slightly more compact - weird count clause, but overall same total FAIL.
Has anybody actually ever used this stuff in a real business application? I'm really starting to wonder...
Please tell me I've missed something obvious, as I really want to like Linq!

Try the more verbose way to do more or less the same thing obtain the same results, but with more datacalls:
var mydata = from e in dc.Entities
join i in dc.Items
on e.ID equals i.Member.ID
where (i.Collection.ID == collectionID)
select e;
foreach (Entity ent in mydata) {
if(!ent.Properties.IsLoaded) { ent.Properties.Load(); }
}
Do you still get the same (unexpected) result?
EDIT: Changed the first sentence, as it was incorrect. Thanks for the pointer comment!

Related

Why does LINQ-to-EF render certain "Contains" as two separate JOINs? [duplicate]

I know that there are some questions about this already, most relate to either old issues which were resolved or multiple tables. This question is not covered in any of the other 'left outer join' issues I saw, I get an INNER JOIN and LEFT OUTER JOIN to the same table at the same query.
The table outline:
Users: id (PK)
Name (VARCHAR)
ProfileImageUri (VARCHAR)
Locations: id (PK)
LocationBPNTips: id (PK)
TipText (VARCHAR)
CreatedAt (Datetime)
UserId (int) (FK to User.id, navigation property is called User)
LocationId (int) (FK to Location.id)
(there is more, but it is not relevant :) )
In my scenario, I am performing a query to a referenced table via projection and I get an extra left outer join, this is what I run (the commented parts are irrelevant to the problem, commented out for cleaner SQL, EF does the sorting right (even better than I imagined :) ) ):
LocationBPNTips
.Where(t => t.LocationId == 33)
//.OrderByDescending(t => intList.Contains(t.UserId))
//.ThenByDescending(t => t.CreatedAt)
.Select(tip => new LocationTipOutput
{
CreatedAt = tip.CreatedAt,
Text = tip.TipText,
LocationId = tip.LocationId,
OwnerName = tip.User.Name,
OwnerPhoto = tip.User.ProfileImageUri
}).ToList();
And this is is the generated SQL
SELECT
[Extent1].[LocationId] AS [LocationId],
[Extent1].[CreatedAt] AS [CreatedAt],
[Extent1].[TipText] AS [TipText],
[Extent2].[Name] AS [Name],
[Extent3].[ProfileImageUri] AS [ProfileImageUri]
FROM [dbo].[LocationBPNTips] AS [Extent1]
INNER JOIN [dbo].[Users] AS [Extent2] ON [Extent1].[UserId] = [Extent2].[Id]
LEFT OUTER JOIN [dbo].[Users] AS [Extent3] ON [Extent1].[UserId] = [Extent3].[Id]
WHERE 33 = [Extent1].[LocationId]
As you can see, the LEFT OUTER JOIN is done on the same table of the INNER JOIN
I think the optimal code will be (note, I renamed Extent3 to Extent2 manually, and added the comment. this was not generated by EF!!) - with my current data, this runs about 22% faster (with the sorting, this % should be higher without the sort) as no need for an extra join..
SELECT
[Extent1].[LocationId] AS [LocationId],
[Extent1].[CreatedAt] AS [CreatedAt],
[Extent1].[TipText] AS [TipText],
[Extent2].[Name] AS [Name],
[Extent2].[ProfileImageUri] AS [ProfileImageUri]
FROM [dbo].[LocationBPNTips] AS [Extent1]
INNER JOIN [dbo].[Users] AS [Extent2] ON [Extent1].[UserId] = [Extent2].[Id]
--LEFT OUTER JOIN [dbo].[Users] AS [Extent3] ON [Extent1].[UserId] = [Extent3].[Id]
WHERE 33 = [Extent1].[LocationId]
The different queries I have tried (the projection is into an anonymous type in these):
LocationBPNTips
.Where(t => t.LocationId == 33)
//.OrderByDescending(t => intList.Contains(t.UserId))
//.ThenByDescending(t => t.CreatedAt)
.Select(tip => new
{
CreatedAt = tip.CreatedAt,
Text = tip.TipText,
LocationId = tip.LocationId,
OwnerName = tip.User,
OwnerPhoto = tip.User
}).ToList()
SQL output was messed up, it selected the entire user table twice in the same format as above, inner then left outer. I think that I can see in theory why this happens for this case, because I asked for the data twice - although it could have been done in memory and not by the SQL with an extra join - but in my case I did not ask for data twice, I asked for different columns only once. I did this test to see if the double join is consistent.
I also tried running:
LocationBPNTips
.Where(t => t.LocationId == 33)
.Select(tip => new
{
CreatedAt = tip.CreatedAt,
Text = tip.TipText,
LocationId = tip.LocationId,
OwnerName = tip.User.Name
}).ToList()
This one returned clean, single inner join as expected, but it is not what I am trying to do
So the question is: Is this a bug? am I using the EF incorrectly?
I've seen similar problem already. We can call it bug or feature. Simply EF query generation is far from ideal. ADO.NET team fixes some problems with every major release. I don't have June CTP 2011 currently installed to verify if it also happens in the first preview of the next version.
Edit:
According to this answer similar issue was fixed in June CTP 2011.

Oracle: column ambiguously defined

I know there are lots of questions like these, but my question is not how to get rid of this error but to know how this worked earlier in 9-th version of Oracle.
I've got an old sources written in Ruby and Oracle DB which recently was upgraded to version=11.
I cannot edit data in Oracle DB, only read. so there are two tables lets say: table A(id, name, type, customer) and table B(id,a_id,type,person)
so. there is a query in the source code:
select a.id,b.id from a join b on a.id = b.a_id where type = 'A'
so in Oracle 9 this worked perfectly but now i've got "column ambiguously defined" error.
What i'd like to know is:
where type = 'A'
is the same as
where a.type = 'A' AND b.type = 'A'
or
where a.type = 'A' OR b.type = 'A'
?
I think this was a bug with the ANSI style join. Use DBMS_XPLAN to find which table was being filtered in the old database.
Or better still, work out from the business logic what they query SHOULD have been.
No, and that's the problem: It could mean
where a.type = 'A'
or it could mean
where b.type = 'A'
with potentially different results; hence the error saying it is ambiguously defined.
I think you should test in Oracle 9 (where you say it works) and compare the output of the ambiguous query:
--- Base
select a.id,b.id from a join b on a.id = b.a_id where type = 'A'
with both the non-ambiguous ones:
--- QueryA
select a.id,b.id from a join b on a.id = b.a_id where a.type = 'A'
and:
--- QueryB
select a.id,b.id from a join b on a.id = b.a_id where b.type = 'A'
Something like this would do:
select a.id,b.id from a join b on a.id = b.a_id where type = 'A'
MINUS
select a.id,b.id from a join b on a.id = b.a_id where a.type = 'A'
(in short):
(Base)
MINUS
(QueryA)
and then:
(QueryA)
MINUS
(Base)
If both of the above MINUS queries return 0 rows, then BASE query is interpreted as QueryA.
Check similarly and compare Base with QueryB.
Another plausible reason for this error is that during (or about the same period with ) the migration, a type column was added in the 2nd table. Do you have old versions of the database tables' definitions to check that?
All - keep in mind there was a major change to the optimization engine for 11g. If you set your query optimizer level to 10.2.x on your 11g instance I bet the query would start working again.
That being said you should provide the alias for it so it's not ambiguous to the database server or the DBA / Developer coming behind you. :)

LINQ nested joins

Im trying to convert a SQL join to LINQ. I need some help in getting the nested join working in LINQ.
This is my SQL query, Ive cut it short just to show the nested join in SQL:
select distinct
txtTaskStatus as TaskStatusDescription,
txtempfirstname+ ' ' + txtemplastname as RaisedByEmployeeName,
txtTaskPriorityDescription as TaskPriorityDescription,
dtmtaskcreated as itemDateTime,
dbo.tblTask.lngtaskid as TaskID,
dbo.tblTask.dtmtaskcreated as CreatedDateTime,
convert(varchar(512), dbo.tblTask.txttaskdescription) as ProblemStatement,
dbo.tblTask.lngtaskmessageid,
dbo.tblMessage.lngmessageid as MessageID,
case when isnull(dbo.tblMessage.txtmessagesubject,'') <> '' then txtmessagesubject else left(txtmessagedescription,50) end as MessageSubject,
dbo.tblMessage.txtmessagedescription as MessageDescription,
case when dbo.tblMessage.dtmmessagecreated is not null then dbo.tblMessage.dtmmessagecreated else CAST(FLOOR(CAST(dtmtaskcreated AS DECIMAL(12, 5))) AS DATETIME) end as MessageCreatedDateTime
FROM
dbo.tblAction RIGHT OUTER JOIN dbo.tblTask ON dbo.tblAction.lngactiontaskid = dbo.tblTask.lngtaskid
LEFT OUTER JOIN dbo.tblMessage ON dbo.tblTask.lngtaskmessageid = dbo.tblMessage.lngmessageid
LEFT OUTER JOIN dbo.tblTaskCommentRecipient
RIGHT OUTER JOIN dbo.tblTaskComment ON dbo.tblTaskCommentRecipient.lngTaskCommentID = dbo.tblTaskComment.lngTaskCommentID
ON dbo.tblTask.lngtaskid = dbo.tblTaskComment.lngTaskCommentTaskId
A more seasoned SQL programmer wouldn't join that way. They'd use strictly left joins for clarity (as there is a strictly left joining solution available).
I've unraveled these joins to produce a hierarchy:
Task
Action
Message
TaskComment
TaskCommentRecipient
With associations created in the linq to sql designer, you can reach these levels of the hierarchy:
//note: these aren't outer joins
from t in db.Tasks
let actions = t.Actions
let message = t.Messages
let comments = t.TaskComments
from c in comments
let recipients = c.TaskCommentRecipients
DefaultIfEmpty produces a default element when the collection is empty. Since these are database rows, a default element is a null row. That is the behavior of left join.
query =
(
from t in db.Tasks
from a in t.Actions.DefaultIfEmpty()
from m in t.Messages.DefaultIfEmpty()
from c in t.Comments.DefaultIfEmpty()
from r in c.Recipients.DefaultIfEmpty()
select new Result()
{
TaskStatus = ???
...
}
).Distinct();
Aside: calling Distinct after a bunch of joins is a crutch. #1 See if you can do without it. #2 If not, see if you can eliminate any bad data that causes you to have to call it. #3 If not, call Distinct in a smaller scope than the whole query.
Hope this helps.
SELECT [t0].[OrderID], [t0].[CustomerID], [t0].[EmployeeID], [t0].[OrderDate], [t0].[RequiredDate], [t0].[ShippedDate], [t0].[ShipVia], [t0].[Freight], [t0].[ShipName], [t0].[ShipAddress], [t0].[ShipCity], [t0].[ShipRegion], [t0].[ShipPostalCode], [t0].[ShipCountry]
FROM [Orders] AS [t0]
LEFT OUTER JOIN ([Order Details] AS [t1]
INNER JOIN [Products] AS [t2] ON [t1].[ProductID] = [t2].[ProductID]) ON [t0].[OrderID] = [t1].[OrderID]
can be write as
from o in Orders
join od in (
from od in OrderDetails join p in Products on od.ProductID equals p.ProductID select od)
on o.OrderID equals od.OrderID into ood from od in ood.DefaultIfEmpty()
select o

Linq to entities Left Join

I want to achieve the following in Linq to Entities:
Get all Enquires that have no Application or the Application has a status != 4 (Completed)
select e.*
from Enquiry enq
left outer join Application app
on enq.enquiryid = app.enquiryid
where app.Status <> 4 or app.enquiryid is null
Has anyone done this before without using DefaultIfEmpty(), which is not supported by Linq to Entities?
I'm trying to add a filter to an IQueryable query like this:
IQueryable<Enquiry> query = Context.EnquirySet;
query = (from e in query
where e.Applications.DefaultIfEmpty()
.Where(app=>app.Status != 4).Count() >= 1
select e);
Thanks
Mark
In EF 4.0+, LEFT JOIN syntax is a little different and presents a crazy quirk:
var query = from c1 in db.Category
join c2 in db.Category on c1.CategoryID equals c2.ParentCategoryID
into ChildCategory
from cc in ChildCategory.DefaultIfEmpty()
select new CategoryObject
{
CategoryID = c1.CategoryID,
ChildName = cc.CategoryName
}
If you capture the execution of this query in SQL Server Profiler, you will see that it does indeed perform a LEFT OUTER JOIN. HOWEVER, if you have multiple LEFT JOIN ("Group Join") clauses in your Linq-to-Entity query, I have found that the self-join clause MAY actually execute as in INNER JOIN - EVEN IF THE ABOVE SYNTAX IS USED!
The resolution to that? As crazy and, according to MS, wrong as it sounds, I resolved this by changing the order of the join clauses. If the self-referencing LEFT JOIN clause was the 1st Linq Group Join, SQL Profiler reported an INNER JOIN. If the self-referencing LEFT JOIN clause was the LAST Linq Group Join, SQL Profiler reported an LEFT JOIN.
Do this:
IQueryable<Enquiry> query = Context.EnquirySet;
query = (from e in query
where (!e.Applications.Any())
|| e.Applications.Any(app => app.Status != 4)
select e);
I don't find LINQ's handling of the problem of what would be an "outer join" in SQL "goofy" at all. The key to understanding it is to think in terms of an object graph with nullable properties rather than a tabular result set.
Any() maps to EXISTS in SQL, so it's far more efficient than Count() in some cases.
Thanks guys for your help. I went for this option in the end but your solutions have helped broaden my knowledge.
IQueryable<Enquiry> query = Context.EnquirySet;
query = query.Except(from e in query
from a in e.Applications
where a.Status == 4
select e);
Because of Linq's goofy (read non-standard) way of handling outers, you have to use DefaultIfEmpty().
What you'll do is run your Linq-To-Entities query into two IEnumerables, then LEFT Join them using DefaultIfEmpty(). It may look something like:
IQueryable enq = Enquiry.Select();
IQueryable app = Application.Select();
var x = from e in enq
join a in app on e.enquiryid equals a.enquiryid
into ae
where e.Status != 4
from appEnq in ae.DefaultIfEmpty()
select e.*;
Just because you can't do it with Linq-To-Entities doesn't mean you can't do it with raw Linq.
(Note: before anyone downvotes me ... yes, I know there are more elegant ways to do this. I'm just trying to make it understandable. It's the concept that's important, right?)
Another thing to consider, if you directly reference any properties in your where clause from a left-joined group (using the into syntax) without checking for null, Entity Framework will still convert your LEFT JOIN into an INNER JOIN.
To avoid this, filter on the "from x in leftJoinedExtent" part of your query like so:
var y = from parent in thing
join child in subthing on parent.ID equals child.ParentID into childTemp
from childLJ in childTemp.Where(c => c.Visible == true).DefaultIfEmpty()
where parent.ID == 123
select new {
ParentID = parent.ID,
ChildID = childLJ.ID
};
ChildID in the anonymous type will be a nullable type and the query this generates will be a LEFT JOIN.

What is the difference between using Join in Linq and "Olde Style" pre ANSI join syntax?

This question follows on from a question I asked yesterday about why using the join query on my Entities produced horrendously complicated SQL. It seemed that performing a query like this:
var query = from ev in genesisContext.Events
join pe in genesisContext.People_Event_Link
on ev equals pe.Event
where pe.P_ID == key
select ev;
Produced the horrible SQL that took 18 seconds to run on the database, whereas joining the entities through a where clause (sort of like pre-ANSI SQL syntax) took less than a second to run and produced the same result
var query = from pe in genesisContext.People_Event_Link
from ev in genesisContext.Events
where pe.P_ID == key && pe.Event == ev
select ev;
I've googled all over but still don't understand why the second is produces different SQL to the first. Can someone please explain the difference to me? When should I use the join keyword
This is the SQL that was produced when I used Join in my query and took 18 seconds to run:
SELECT
1 AS [C1],
[Extent1].[E_ID] AS [E_ID],
[Extent1].[E_START_DATE] AS [E_START_DATE],
[Extent1].[E_END_DATE] AS [E_END_DATE],
[Extent1].[E_COMMENTS] AS [E_COMMENTS],
[Extent1].[E_DATE_ADDED] AS [E_DATE_ADDED],
[Extent1].[E_RECORDED_BY] AS [E_RECORDED_BY],
[Extent1].[E_DATE_UPDATED] AS [E_DATE_UPDATED],
[Extent1].[E_UPDATED_BY] AS [E_UPDATED_BY],
[Extent1].[ET_ID] AS [ET_ID],
[Extent1].[L_ID] AS [L_ID]
FROM [dbo].[Events] AS [Extent1]
INNER JOIN [dbo].[People_Event_Link] AS [Extent2] ON EXISTS (SELECT
1 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
LEFT OUTER JOIN (SELECT
[Extent3].[E_ID] AS [E_ID]
FROM [dbo].[Events] AS [Extent3]
WHERE [Extent2].[E_ID] = [Extent3].[E_ID] ) AS [Project1] ON 1 = 1
LEFT OUTER JOIN (SELECT
[Extent4].[E_ID] AS [E_ID]
FROM [dbo].[Events] AS [Extent4]
WHERE [Extent2].[E_ID] = [Extent4].[E_ID] ) AS [Project2] ON 1 = 1
WHERE ([Extent1].[E_ID] = [Project1].[E_ID]) OR (([Extent1].[E_ID] IS NULL) AND ([Project2].[E_ID] IS NULL))
)
WHERE [Extent2].[P_ID] = 291
This is the SQL that was produce using the ANSI Style syntax (and is fairly close to what I would write if I were writing the SQL myself):
SELECT * FROM Events AS E INNER JOIN People_Event_Link AS PE ON E.E_ID=PE.E_ID INNER JOIN PEOPLE AS P ON P.P_ID=PE.P_ID
WHERE P.P_ID = 291
Neither of the above queries are entirely "correct." In EF, it is generally correct to use the relationship properties in lieu of either of the above. For example, if you had a Person object with a one to many relationship to PhoneNumbers in a property called Person.PhoneNumbers, you could write:
var q = from p in Context.Person
from pn in p.PhoneNumbers
select pn;
The EF will build the join for you.
In terms of the question above, the reason the generated SQL is different is because the expression trees are different, even though they produce equivalent results. Expression trees are mapped to SQL, and you of course know that you can write different SQL which produces the same results but with different performance. The mapping is designed to produce decent SQL when you write a farily "conventional" EF query.
But the mapping is not so smart as to take a very unconventional query and optimize it. In your first query, you state that the objects must be equivalent. In the second, you state that the ID property must be equivalent. My sample query above says "just get the details for this one record." The EF is designed to work with the way I show, principally, but also handles scalar equivalence well.

Resources