I'm having issues with GROUP BY and eager loading. I try to explain what im doing.
I'm querying a datacontext ctx for events
The event class has the following properties
string Description
DateTime Date
bool IsDeleted
Guid SubjectId
string Title
DateTime Created
Guid ForProjectId
Person TriggeredBy
List<Guid> Targets
There are muttiple events with the same SubjectId and i would like to end up having events with unique SubjectIds and that are the newest in the group. I end up with the following query.
var events = from x in
(from e in ctx.Events
.Include("TriggeredBy")
.Include("Targets")
group e by e.SubjectId
into g
select new
{
GroupId = g.Key,
EventsWithSameSubjectId = g,
}
)
select x.EventsWithSameSubjectId
.OrderByDescending(y => y.Created).FirstOrDefault();
The query compile fine and returns the right resulting set. But the included properties are always null.
When i strip the query to see if eagor loading is working properly....
var events = (from e in ctx.Events.OfType<DataNotificationEvent>()
.Include("TriggeredBy")
.Include("Targets")
select e).ToList();
This return the events with all the included properties.
Is this a known issue / bug with Linq / EF or is there any way i can get rid of this error.
Regards
Vincent Ottens
You're projecting onto an anonymous type, so Include() isn't going to work like that. Because what you've done with the group and projecting into the anonymous type is to change the shape of the query. That tosses out the eager loading. Reading this article might help.
Thnx for the quick answer. You pointed me in the right direction. Here is the solution i came up with:
using MyFunc = Func<ExtendedCoreContext, Guid, IQueryable<DataNotificationEvent>>;
private static readonly MyFunc GetMentionsNewCompiledQuery =
CompiledQuery.Compile<ExtendedCoreContext, Guid, IQueryable<DataNotificationEvent>>(
(ctx, personId) => ((ObjectQuery<DataNotificationEvent>)(
from x in (
from e in ctx.Events.OfType<DataNotificationEvent>()
group e by e.SubjectId
into g
select g.OrderByDescending(p => p.Created).FirstOrDefault()
)
orderby x.Created descending
where x.Targets.Any(t => t.Id == personId)
select x
))
.Include(EntityProperties.Event.TriggeredBy)
.Include(EntityProperties.DataNotificationEvent.Targets)
);
Related
I want to change code below to be sql translateable because now i get exception.
Basicallly i want list of customers from certain localisation and there could be more than one customer with the same CustomerNumber so i want to take the one that was most recently added.
In other words - distinct list of customers from localisation where "distinct algorithm" works by taking the most recently added customer if there is conflict.
The code below works only if it is client side. I could move Group By and Select after ToListAsync but i want to avoid taking unnecessary data from database (there is include which includes list that is pretty big for every customer).
var someData = await DbContext.Set<Customer>()
.Where(o => o.Metadata.Localisation == localisation)
.Include(nameof(Customer.SomeLongList))
.GroupBy(x => x.CustomerNumber)
.Select(gr => gr.OrderByDescending(x => x.Metadata.DateAdded).FirstOrDefault())
.ToListAsync();
Short answer:
No way. GroupBy has limitation: after grouping only Key and Aggregation result can be selected. And you are trying to select SomeLongList and full entity Customer.
Best answer:
It can be done by the SQL and ROW_NUMBER Window function but without SomeLongList
Workaround:
It is because it is not effective
var groupingQuery =
from c in DbContext.Set<Customer>()
group c by new { c.CustomerNumber } into g
select new
{
g.Key.CustomerNumber,
DateAdded = g.Max(x => x.DateAdded)
};
var query =
from c in DbContext.Set<Customer>().Include(x => x.SomeLongList)
join g in groupingQuery on new { c.CustomerNumber, c.DateAdded } equals
new { g.CustomerNumber, g.DateAdded }
select c;
var result = await query.ToListAsync();
So we copy and paste the exact same query from LinqPad into our EF 4.3 application, pointed at the exact same database and get a different result. In LinqPad we get 2 records returned. In our application we reaise an error "Object reference not set to an instance of an object."
var Shippings = shippingRepository.All.ToArray();
var SalesOrderHeaders = salesOrderHeaderRepository.All.ToArray();
var Customers = customerRepository.All.ToArray();
var Stores = storeRepository.All.ToArray();
var Departments = departmentRepository.All.ToArray();
var toShip = from sh in Shippings
join h in SalesOrderHeaders on sh.OrderId equals h.SalesOrderHeaderId
join c in Customers on h.CustomerId equals c.CustomerId
join st in Stores on h.StoreId equals st.StoreId
join d in Departments on h.DepartmentId equals d.DepartmentId into outer
from o in outer.DefaultIfEmpty()
select new
{
OrderId = sh.OrderId,
CustomerName = c.Name,
StoreName = st.Name,
DepartmentName = (o.Name == null) ? o.Name : "None",
DeliveryDate = h.DeliveryDateTime
};
In the application code, when we remove the outer join (to add Departments) and it's associated field the query returns the same 2 records asn in LinqPad.
Does anyone have any insight into how to fix this feature?
Click on "Add a connection" in linqpad and select datacontext from assembly like
You can choose Entity Framework datacontext or Entity Framework BDContext with POCO depending upon your scenario. click next and provide path to the assembly along with connection string and you will be good to go.
In LINQPad are you actually querying against your entity model? Take a look at this link if you aren't. I had a similar problem when starting out and didn't realize I had set up a default LINQ to SQL connection earlier and was querying against that.
Can anyone help?
I have 1 class, basically it holds Members and within that class is a List.
The members i have in a List also... So basically it goes like this,
I have 2 members and each member has a number of sessions.
I wish to only return each member with 1 Session.
I have done a LINQ query, but of course it doesn't work...
I think i need to do a self join, any ideas?
Basically my error is m doesn't exist in my subquery self join.
var sessions =
from m in this.members
join s in
(
from se in m.Sessions
group se by se.Name into g
select new {Name = g.Key, SessioEndTime = g.Max(a=>a.SessioEndTime)}
)
on m.Name equals s.Name
select new { MemberName = m.Name, SessionTime = s.SessioEndTime}
I would appreciate any feedback anyone has.
Thanks in advance.
EDIT
Ok i managed to do it like the following, but is this the best way?
var sessions =
from m in this.members
let sn = m.Sessions.OrderByDescending(a => a.SessionEndTime).FirstOrDefault()
select new { MemberName = m.Name, SessionTime = sn.SessioEndTime}
This way sn contains 1 record, but i have access to all the properties...
But is this the best way to do using a LET?
Thanks.
Unless I am missing something you need this, no?
var sessions =
from m in members
select new {
MemberName = m.Name,
SessionTime = m.Sessions.Max(s => s.SessioEndTime)
};
You have to change the way you think about LINQ queries, think more from object point rather than from SQL implementation point. What is it that I need? I need all members, each with its latest session end time, then act on that.
EDIT:
The let option you used is ok, just keep something in mind FirstOrDefault will return null if member has an empty list of Sessions, and then sn.SessionEndTime hits null reference. If on the other hand you are certain that every member has at least one session use First instead or aggregate.
Also don't use FirstOrDefault() in the let, it kind of messes up the LINQ and prevents it from tying it to the master (causing a separate SQL query for each master to detect missing subsets), so usable queries with let are:
from m in Members
let sn = m.Sessions.Max(s => s.SessioEndTime)
select new { MemberName = m.Name, SessionTime = sn};
from m in Members
let sn = m.Sessions.OrderByDescending(a => a.SessioEndTime).First()
select new { MemberName = m.Name, SessionTime = sn.SessioEndTime};
As for ordering vs Max aggregation, both queries will generate a subquery:
-- MAX
SELECT [t0].[Name] AS [MemberName], (
SELECT MAX([t1].[SessioEndTime])
FROM [Session] AS [t1]
WHERE [t1].[memberId] = [t0].[id]
) AS [SessionTime]
FROM [Member] AS [t0]
GO
-- ordering
SELECT [t0].[Name] AS [MemberName], (
SELECT [t2].[SessioEndTime]
FROM (
SELECT TOP (1) [t1].[SessioEndTime]
FROM [Session] AS [t1]
WHERE [t1].[memberId] = [t0].[id]
ORDER BY [t1].[SessioEndTime] DESC
) AS [t2]
) AS [SessionTime]
FROM [Member] AS [t0]
With a descending index on SessioEndTime the ordering script is about twice slower (you can get execution plans for these to check for yourself), without the index its about 5times slower.
I have this sql that i want to have written in linq extension method returning an entity from my edm:
SELECT p.[Id],p.[Firstname],p.[Lastname],prt.[AddressId],prt.[Street],prt.[City]
FROM [Person] p
CROSS APPLY (
SELECT TOP(1) pa.[AddressId],a.[ValidFrom],a.[Street],a.[City]
FROM [Person_Addresses] pa
LEFT OUTER JOIN [Addresses] AS a
ON a.[Id] = pa.[AddressId]
WHERE p.[Id] = pa.[PersonId]
ORDER BY a.[ValidFrom] DESC ) prt
Also could this be re-written in linq extension method using 3 joins?
Assuming you have set the Person_Addresses table up as a pure relation table (i.e., with no data besides the foreign keys) this should do the trick:
var persons = model.People
.Select(p => new { p = p, a = p.Addresses.OrderByDescending(a=>a.ValidFrom).First() })
.Select(p => new { p.p.Id, p.p.Firstname, p.p.LastName, AddressId = p.a.Id, p.a.Street, p.a.City });
The first Select() orders the addresses and picks the latest one, and the second one returns an anonymous type with the properties specified in your query.
If you have more data in your relation table you're gonna have to use joins but this way you're free from them. In my opinion, this is more easy to read.
NOTE: You might get an exception if any entry in Persons have no addresses connected to them, although I haven't tried it out.
I am looking to optimize my LINQ query because although it works right, the SQL it generates is convoluted and inefficient...
Basically, I am looking to select customers (as CustomerDisplay objects) who ordered the required product (reqdProdId), and are registered with a credit card number (stored as a row in RegisteredCustomer table with a foreign key CustId)
var q = from cust in db.Customers
join regCust in db.RegisteredCustomers on cust.ID equals regCust.CustId
where cust.CustomerProducts.Any(co => co.ProductID == reqdProdId)
where regCust.CreditCardNumber != null && regCust.Authorized == true
select new CustomerDisplay
{
Id = cust.Id,
Name = cust.Person.DisplayName,
RegNumber = cust.RegNumber
};
As an overview, a Customer has a corresponding Person which has the Name; PersonID is a foreign key in Customer table.
If I look at the SQL generated, I see all columns being selected from the Person table. Fyi, DisplayName is an extension method which uses Customer.FirstName and LastName. Any ideas how I can limit the columns from Person?
Secondly, I want to get rid of the Any clause (and use a sub-query) to select all other CustomerIds who have the required ProductID, because it (understandably) generates an Exists clause.
As you may know, LINQ has a known issue with junction tables, so I cannot just do a cust.CustomerProducts.Products.
How can I select all Customers in the junction table with the required ProductID?
Any help/advice is appreciated.
The first step is to start your query from CustomerProducts (as Alex Said):
IQueryable<CustomerDisplay> myCustDisplay =
from custProd in db.CustomerProducts
join regCust in db.RegisteredCustomers
on custProd.Customer.ID equals regCust.CustId
where
custProd.ProductID == reqProdId
&& regCust.CreditCardNumber != null
&& regCust.Authorized == true
select new CustomerDisplay
{
Id = cust.Id,
Name = cust.Person.Name,
RegNumber = cust.RegNumber
};
This will simplify your syntax and hopefully result in a better execution plan.
Next, you should consider creating a foreign key relationship between Customers and RegisteredCustomers. This would result in a query that looked like this:
IQueryable<CustomerDisplay> myCustDisplay =
from custProd in db.CustomerProducts
where
custProd.ProductID == reqProdId
&& custProd.Customer.RegisteredCustomer.CreditCardNumber != null
&& custProd.Customer.RegisteredCustomer.Authorized == true
select new CustomerDisplay
{
Id = cust.Id,
Name = cust.Person.Name,
RegNumber = cust.RegNumber
};
Finally, for optimum speed, have LINQ compile your query at compile time, rather than run time by using a compiled query:
Func<MyDataContext, SearchParameters, IQueryable<CustomerDisplay>>
GetCustWithProd =
System.Data.Linq.CompiledQuery.Compile(
(MyDataContext db, SearchParameters myParams) =>
from custProd in db.CustomerProducts
where
custProd.ProductID == myParams.reqProdId
&& custProd.Customer.RegisteredCustomer.CreditCardNumber != null
&& custProd.Customer.RegisteredCustomer.Authorized == true
select new CustomerDisplay
{
Id = cust.Id,
Name = cust.Person.Name,
RegNumber = cust.RegNumber
};
);
You can call the compiled query like this:
IQueryable<CustomerDisplay> myCustDisplay = GetCustWithProd(db, myParams);
I'd suggest starting your query from the product in question, e.g. something like:
from cp in db.CustomerProducts
join .....
where cp.ProductID == reqdProdID
As you have found, using a property defined as an extension function or in a partial class will require that the entire object is hydrated first and then the select projection is done on the client side because the server has no knowledge of these additional properties. Be glad that your code ran at all. If you were to use the non-mapped value elsewhere in your query (other than in the projection), you would likely see a run-time exception. You can see this if you try to use the Customer.Person.DisplayName property in a Where clause. As you have found, the fix is to do the string concatenation in the projection clause directly.
Lame Duck, I think there is a bug in your code as the cust variable used in your select clause isn't declared elsewhere as a source local variable (in the from clauses).