For a LINQ query like:
var entities = from Account p in context.Accounts
where p.LastTimeServerSettingsChanged > p.LastTimeDeviceConnected
select p;
the query that is generated is:
SELECT
[Extent1].[Username] AS [Username],
[Extent1].[LastTimeDeviceConnected] AS [LastTimeDeviceConnected],
[Extent1].[LastTimeServerSettingsChanged] AS [LastTimeServerSettingsChanged]
FROM [dbo].[Account] AS [Extent1]
WHERE [Extent1].[LastTimeServerSettingsChanged] > [Extent1].[LastTimeDeviceConnected]
And this does not work (no results).
And the following also generates the same SQL (hence no results also)
var entities = context.Accounts.Where(k => k.LastTimeServerSettingsChanged > k.LastTimeDeviceConnected).Select(k => k);
My question is why, and how can this query be performed (using LINQ)?
The above code works fine. I was hitting the wrong database and hence was getting the wrong result. GIGO. QED.
Related
I have the following LINQ query (using EF Core 6 and MS SQL Server):
var resultSet = dbContext.Systems
.Include(system => system.Project)
.Include(system => system.Template.Type)
.Select(system => new
{
System = system,
TemplateText = system.Template.TemplateTexts.FirstOrDefault(templateText => templateText.Language == locale.LanguageIdentifier),
TypeText = system.Template.Type.TypeTexts.FirstOrDefault(typeText => typeText.Language == locale.LanguageIdentifier)
})
.FirstOrDefault(x => x.System.Id == request.Id);
The requirement is to retrieve the system matching the requested ID and load its project, template and template's type info. The template has multiple TemplateTexts (one for each translated language) but I only want to load the one matching the requested locale, same deal with the TypeTexts elements of the template's type.
The LINQ query above does that in one query and it gets converted to the following SQL query (I edited the SELECT statements to use * instead of the long list of columns generated):
SELECT [t1].*, [t2].*, [t5].*
FROM (
SELECT TOP(1) [p].*, [t].*, [t0].*
FROM [ParkerSystems] AS [p]
LEFT JOIN [Templates] AS [t] ON [p].[TemplateId] = [t].[Id]
LEFT JOIN [Types] AS [t0] ON [t].[TypeId] = [t0].[Id]
LEFT JOIN [Projects] AS [p0] ON [p].[Project_ProjectId] = [p0].[ProjectId]
WHERE [p].[SystemId] = #__request_Id_1
) AS [t1]
LEFT JOIN (
SELECT [t3].*
FROM (
SELECT [t4].*, ROW_NUMBER() OVER(PARTITION BY [t4].[ReferenceId] ORDER BY [t4].[Id]) AS [row]
FROM [TemplateTexts] AS [t4]
WHERE [t4].[Language] = #__locale_LanguageIdentifier_0
) AS [t3]
WHERE [t3].[row] <= 1
) AS [t2] ON [t1].[Id] = [t2].[ReferenceId]
LEFT JOIN (
SELECT [t6].*
FROM (
SELECT [t7].*, ROW_NUMBER() OVER(PARTITION BY [t7].[ReferenceId] ORDER BY [t7].[Id]) AS [row]
FROM [TypeTexts] AS [t7]
WHERE [t7].[Language] = #__locale_LanguageIdentifier_0
) AS [t6]
WHERE [t6].[row] <= 1
) AS [t5] ON [t1].[Id0] = [t5].[ReferenceId]
which is not bad, it's not a super complicated query, but I feel like my requirement can be solved with a much simpler SQL query:
SELECT *
FROM [Systems] AS [p]
JOIN [Templates] AS [t] ON [p].[TemplateId] = [t].[Id]
JOIN [TemplateTexts] AS [tt] ON [p].[TemplateId] = [tt].[ReferenceId]
JOIN [Types] AS [ty] ON [t].[TypeId] = [ty].[Id]
JOIN [TemplateTexts] AS [tyt] ON [ty].[Id] = [tyt].[ReferenceId]
WHERE [p].[SystemId] = #systemId and tt.[Language] = 2 and tyt.[Language] = 2
My question is: is there a different/simpler LINQ expression (either in Method syntax or Query syntax) that produces the same result (get all info in one go) because ideally I'd like to not have to have an anonymous object where the filtered sub-collections are aggregated. For even more brownie points, it'd be great if the generated SQL would be simpler/closer to what I think would be a simple query.
Is there a different/simpler LINQ expression (...) that produces the same result
Yes (maybe) and no.
No, because you're querying dbContext.Systems, therefore EF will return all systems that match your filter, also when they don't have TemplateTexts etc. That's why it has to generate outer joins. EF is not aware of your apparent intention to skip systems without these nested data or of any guarantee that these systems don't occur in the database. (Which you seem to assume, seeing the second query).
That accounts for the left joins to subqueries.
These subqueries are generated because of FirstOrDefault. In SQL it always requires some sort of subquery to get "first" records of one-to-many relationships. This ROW_NUMBER() OVER construction is actually quite efficient. Your second query doesn't have any notion of "first" records. It'll probably return different data.
Yes (maybe) because you also Include data. I'm not sure why. Some people seem to think Include is necessary to make subsequent projections (.Select) work, but it isn't. If that's your reason to use Includes then you can remove them and thus remove the first couple of joins.
OTOH you also Include system.Project which is not in the projection, so you seem to have added the Includes deliberately. And in this case they have effect, because the entire entity system is in the projection, otherwise EF would ignore them.
If you need the Includes then again, EF has to generate outer joins for the reason mentioned above.
EF decides to handle the Includes and projections separately, while hand-crafted SQL, aided by prior knowledge of the data could do that more efficiently. There's no way to affect that behavior though.
This LINQ query is close to your SQL, but I'm afraid of correctness of the result:
var resultSet =
(from system in dbContext.Systems
from templateText in system.Template.TemplateTexts
where templateText.Language == locale.LanguageIdentifier
from typeText in system.Template.Type.TypeTexts
where typeText.Language == locale.LanguageIdentifier
select new
{
System = system,
TemplateText = templateText
TypeText = typeText
})
.FirstOrDefault(x => x.System.Id == request.Id);
I have a table which contains ~600k records and 33 columns. In my project I am using EF Core (2.0.1) to retrieve data from database. I am having issues with below code:
var theCounter = (from f in _context.tblData.Take(100000)
group f by f.TypeId into data
select new DataDto { ID = data.Key, Count = data.Count() }).ToList();
This code is a part of REST API and when I am testing it from SOAP UI, I am gettin timeout error. When I tested the code for
Take(1000)
There are around 300 unique TypeIds.
it works fine. Any ideas how I can make it work?
-- EDIT 1:
Here is what I see when debugging the code:
Microsoft.EntityFrameworkCore.Query:Warning: Query: '(from TblData <generated>_1 in DbSet<TblData> select [<generated>_1]).Take(__p_0)' uses a row limiting operation (Skip/Take) without OrderBy which may lead to unpredictable results.
Microsoft.EntityFrameworkCore.Query:Warning: Query: '(from TblData <generated>_1 in DbSet<TblData> select [<generated>_1]).Take(__p_0)' uses a row limiting operation (Skip/Take) without OrderBy which may lead to unpredictable results.
Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'GroupBy([f].TypeId, [f])' could not be translated and will be evaluated locally.
Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'GroupBy([f].TypeId, [f])' could not be translated and will be evaluated locally.
Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'Count()' could not be translated and will be evaluated locally.
Microsoft.EntityFrameworkCore.Database.Command:Information: Executed DbCommand (131ms) [Parameters=[#__p_0='?'], CommandType='Text', CommandTimeout='30']
SELECT [t2].[Id], [t2].[at], [t2].[add], [t2].[AddDate], [t2].[aftc], [t2].[aftcd], [t2].[aid], [t2].[afl], [t2].[prdid], [t2].[cid], [t2].[TypeId], [t2].[env], [t2].[ext], [t2].[extddcode], [t2].[fn], [t2].[fn], [t2].[fic], [t2].[gid], [t2].[grp], [t2].[hnm], [t2].[IP], [t2].[icid], [t2].[ln], [t2].[lg], [t2].[pcid], [t2].[ret], [t2].[rts], [t2].[rnam], [t2].[sled], [t2].[seq], [t2].[sid], [t2].[styp]
FROM (
SELECT TOP(#__p_0) [t1].[Id], [t1].[at], [t1].[add], [t1].[AddDate], [t1].[aftc], [t1].[aftcd], [t1].[aid], [t1].[afl], [t1].[prdid], [t1].[cid], [t1].[TypeId], [t1].[env], [t1].[ext], [t1].[extddcode], [t1].[fn], [t1].[fn], [t1].[fic], [t1].[gid], [t1].[grp], [t1].[hnm], [t1].[IP], [t1].[icid], [t1].[ln], [t1].[lg], [t1].[pcid], [t1].[ret], [t1].[rts], [t1].[rnam], [t1].[sled], [t1].[seq], [t1].[sid], [t1].[styp]
FROM [TblData] AS [t1]
) AS [t2]
WHERE [t2].[TypeId] IS NOT NULL
ORDER BY [t2].[TypeId]
I think it is not translated properly. Any ideas why?
-- EDIT 2:
I have changed my queries to:
var query = _context.TblData
.Select(a => new {ID = a.Id, TypeId= a.TypeId})
.Distinct();
var q1 = query.GroupBy(p => p.TypeId)
.Select(g => new DataDto {TypeId= g.Key, Count = g.Count()});
return await q1.ToListAsync();
But it was translated to:
SELECT DISTINCT [a0].[Id], [a0].[TypeId] AS [TypeId]
FROM [tblData] AS [a0]
ORDER BY [a0].[TypeId]
When I checked directly in the database this query takes 14 seconds to execute. Any idea why it was not translated to something like:
SELECT DISTINCT [a0].[Id], COUNT([TypeId]) AS [TypeId]
FROM [tblData] AS [a0]
GROUP BY COUNT([a0].[Id])
ORDER BY [a0].[TypeId]
I had to upgrade EF Core version to 2.1 and LINQ is now translated properly into SQL.
I get the following error when trying to run a specific query against RavenDB:
Can't extract value from expression of type: ArrayIndex
Here is the query that is generating the error:
people = from p in RavenSession.Query<DBObjects.Person, People_ByNameAndTrashedSortByFirstNameAndLastName>()
orderby p.FirstName, p.LastName
select p;
...
//building LINQ query
...
people = from p in people
where ((p.FirstName.StartsWith(SearchWords[0])) && (p.LastName.StartsWith(SearchWords[1])))
select p;
...
//later
foreach(DBObject.Person person in people) //triggers error listed above
{
}
I'm wondering if this is a limitation of RavenDB. I noticed that if I switch && with ||, then I get no error. Of course, I don't get the results I want either. I've also tried rewriting the query as:
people = from p in people
where p.FirstName.StartsWith(SearchWords[0])
where p.LastName.StartsWith(SearchWords[1])
select p;
I get the same error.
I've also tried using a dynamic index instead of a static index. I get the same error.
I suppose maybe RavenDB LINQ driver cannot cope with extracting value from array inside the query. Try to put already extracted values in the place of SearchWords[0] and SearchWords[1] instead:
var pref1 = SearchWords[0];
var pref2 = SearchWords[1];
... where ( p.FirstName.StartsWith(pref1) && p.LastName.StartsWith(pref2) )
Given the following tables:
tool
*toolid
*n other fields
process
*processid
*n other fields
toolprocess
*toolprocessid
*toolid
*processid
*n other fields
When trying to select all tools for a specific process I get up to a few thousand selects on toolprocess where my Linq looks like this:
from tool in tools
where toolprocesses.Any(t=>t.Tool.Id==tool.Id)
select tool
where toolprocesses contains the list of toolprocesses with the same processid
In SQL I would just write
SELECT * FROM TOOL WHERE toolid IN
(SELECT TOOLID FROM TOOLPROCESS WHERE processid = 'someid');
It takes almost no time and works as expected
How can I get NHibernate to create this query (or something similar)?
I don't know if you can do it in Query, but you can do it in QueryOver / Criteria.
In QueryOver it would look like:
var subQuery = QueryOver.Of<Toolprocess>()
.Where(x => x.Process.Id == id)
.Select(x => x.Tool.Id);
var result = session.QueryOver<Tool>()
.WithSubquery.WhereProperty(x => x.Id).In(subQuery)
.List();
http://www.philliphaydon.com/2010/09/28/queryover-with-nhibernate-3-lovin-it/
Alternatively, if you want to do Exists rather than In, I've blogged about it here:
http://www.philliphaydon.com/2011/01/19/revisiting-exists-in-nhibernate-3-0-and-queryover/
Try
from t in Session.Query<Tool>()
join tp in Session.Query<Toolprocess>() on t equals tp.Tool
where tp.Process.Id == 'someid'
select t;
I assume that you're using NH 3.X. This should be even faster than the Select...Where...In query.
I have the following query to start with:
var query = from p in db.Products
from pc in p.NpProductCategories
where pc.CategoryId == categoryId
select p;
I'm applying some more filtering on it and in the end I want to sort the results:
if (orderBy == ProductSortingEnum.Name)
query = query.OrderBy(x => x.Name);
else
query = query.OrderBy(............);
My big problem (coming from not knowing linq too good) is the ELSE here. How can I sort results by a column that is not in the current result set? I would like to somehow link to another linq query in the orderby. The sorting I'm trying to achive is to link to NpProductVariants query using the ProductId to match between NpProductVariant and Products
and sort by the Price of the NpProductVariant
Assuming you have the relationship set up in the dbml...
For one to one (and many to one):
query = query.OrderBy(p => p.NpProductVariant.Price);
For one to many:
query = query.OrderBy(p => p.NpProductVariants.Select(v => v.Price).Max());
Also:
var query =
from p in db.Products
where p.NpProductCategories.Any(pc => pc.CategoryId == categoryId)
select p;
I think you can hook your Join to your query as long as it is returning the same thing. So maybe something like (I'm not 100 % sure since I haven't tried it):
query = from i1 in query
join i2 in query2 on i1.PropertyToJoin equals i2.PropertyToJoin
orderby i1.OrderProp1, i2.OrderProp2
select i1;
But I think it might be a good idea to check the generated sql so it is still effective.