I want to get the latest record of each group from a SQL Server table using Linq.
Table Example:
I want to get this result:
My Linq query returns one record for each company, but it doesn't return the most recent ones:
var query = from p in db.Payments
where p.Status == false
&& DateTime.Compare(DateTime.Now, p.NextPaymentDate.Value) == 1
group p by p.CompanyID into op
select op.OrderByDescending(nd => nd.NextPaymentDate.Value).FirstOrDefault();
What am i missing here? Why isn't the NextPaymentDate being ordered correctly?
!!UPDATE!!
My query is working as expected. After analysing #Gilang and #JonSkeet comments i ran further tests and found that i wasn't getting the intended results due to a column that wasn't being updated.
var query = from p in db.Payments
where p.Status == false
group p by p.CompanyID into op
select new {
CompanyID = op.Key,
NextPaymentDate = op.Max(x => x.NextPaymentDate),
Status = false
};
The reason your query is not being ordered correctly is that your query does not do proper grouping. You did correctly grouping by CompanyID, but then you have to retrieve the maximum NextPaymentDate by calling aggregate function.
Status can be assigned false because it is already filtered by Where clause in the early clauses.
Related
I want to do a Linq query that joins three tables, but only returns data from two of them (the third is only joined for ordering purposes). I'm trying to order by columns that aren't in the output of the produced query, but they seem to be ignored:
var records = from q in _pdxContext.Qualifier
join aql in _pdxContext.ApplicationQualifierLink on q.Id equals aql.QualifierId
join qt in _pdxContext.QualifierType on q.QualifierTypeId equals qt.Id
where SOME_LIST.Contains(aql.ApplicationId)
orderby aql.Sequence
select new Qualifier
{
Id = q.Id,
QualifierType = new QualifierType
{
Id = qt.Id, Value = qt.Value
}
};
return records.Distinct().ToList();
The output SQL from this does NOT have an ORDER BY clause.
If I change the orderby to read like so:
orderby q.Id
... then the output SQL has the order by clause.
Does Linq ignore orderby statements when the mentioned columns aren't used in the output (as appears to be the case here)? If so, how do I order by columns not in the output?
It seems this is an SQL limitation. The error from the SQL Server engine:
"ORDER BY items must appear in the select list if SELECT DISTINCT is specified."
So, as written, I can't do what I want to do.
I ended up using:
using (var cnn = new SqlConnection(_connectionString))
{
string sql = #"select
min(q.Id) Id, q.QualifierTypeId, q.QualifierTypeId, min(q.AcaId) AcaId,
q.QualifierTypeId Id, qt.Value
from
qdb.Qualifier q
inner join qdb.QualifierType qt on qt.Id = q.QualifierTypeId
inner join ApplicationQualifierLink l on l.QualifierId = q.id
where l.ApplicationId in (" + string.Join(",", applicationIds) + #")
group by q.Text, q.QualifierTypeId, qt.Value";
qualifiers = cnn.Query<Qualifier, QualifierType, Qualifier>(sql,
(qualifier, type) =>
{
qualifier.QualifierType = type; return qualifier;
}
).ToList();
}
Note: When you attempt to use order by and distinct as in my original clause, no error is given, entity framework silently discards the order by without any error.
I have a table which contains ~600k records and 33 columns. In my project I am using EF Core (2.0.1) to retrieve data from database. I am having issues with below code:
var theCounter = (from f in _context.tblData.Take(100000)
group f by f.TypeId into data
select new DataDto { ID = data.Key, Count = data.Count() }).ToList();
This code is a part of REST API and when I am testing it from SOAP UI, I am gettin timeout error. When I tested the code for
Take(1000)
There are around 300 unique TypeIds.
it works fine. Any ideas how I can make it work?
-- EDIT 1:
Here is what I see when debugging the code:
Microsoft.EntityFrameworkCore.Query:Warning: Query: '(from TblData <generated>_1 in DbSet<TblData> select [<generated>_1]).Take(__p_0)' uses a row limiting operation (Skip/Take) without OrderBy which may lead to unpredictable results.
Microsoft.EntityFrameworkCore.Query:Warning: Query: '(from TblData <generated>_1 in DbSet<TblData> select [<generated>_1]).Take(__p_0)' uses a row limiting operation (Skip/Take) without OrderBy which may lead to unpredictable results.
Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'GroupBy([f].TypeId, [f])' could not be translated and will be evaluated locally.
Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'GroupBy([f].TypeId, [f])' could not be translated and will be evaluated locally.
Microsoft.EntityFrameworkCore.Query:Warning: The LINQ expression 'Count()' could not be translated and will be evaluated locally.
Microsoft.EntityFrameworkCore.Database.Command:Information: Executed DbCommand (131ms) [Parameters=[#__p_0='?'], CommandType='Text', CommandTimeout='30']
SELECT [t2].[Id], [t2].[at], [t2].[add], [t2].[AddDate], [t2].[aftc], [t2].[aftcd], [t2].[aid], [t2].[afl], [t2].[prdid], [t2].[cid], [t2].[TypeId], [t2].[env], [t2].[ext], [t2].[extddcode], [t2].[fn], [t2].[fn], [t2].[fic], [t2].[gid], [t2].[grp], [t2].[hnm], [t2].[IP], [t2].[icid], [t2].[ln], [t2].[lg], [t2].[pcid], [t2].[ret], [t2].[rts], [t2].[rnam], [t2].[sled], [t2].[seq], [t2].[sid], [t2].[styp]
FROM (
SELECT TOP(#__p_0) [t1].[Id], [t1].[at], [t1].[add], [t1].[AddDate], [t1].[aftc], [t1].[aftcd], [t1].[aid], [t1].[afl], [t1].[prdid], [t1].[cid], [t1].[TypeId], [t1].[env], [t1].[ext], [t1].[extddcode], [t1].[fn], [t1].[fn], [t1].[fic], [t1].[gid], [t1].[grp], [t1].[hnm], [t1].[IP], [t1].[icid], [t1].[ln], [t1].[lg], [t1].[pcid], [t1].[ret], [t1].[rts], [t1].[rnam], [t1].[sled], [t1].[seq], [t1].[sid], [t1].[styp]
FROM [TblData] AS [t1]
) AS [t2]
WHERE [t2].[TypeId] IS NOT NULL
ORDER BY [t2].[TypeId]
I think it is not translated properly. Any ideas why?
-- EDIT 2:
I have changed my queries to:
var query = _context.TblData
.Select(a => new {ID = a.Id, TypeId= a.TypeId})
.Distinct();
var q1 = query.GroupBy(p => p.TypeId)
.Select(g => new DataDto {TypeId= g.Key, Count = g.Count()});
return await q1.ToListAsync();
But it was translated to:
SELECT DISTINCT [a0].[Id], [a0].[TypeId] AS [TypeId]
FROM [tblData] AS [a0]
ORDER BY [a0].[TypeId]
When I checked directly in the database this query takes 14 seconds to execute. Any idea why it was not translated to something like:
SELECT DISTINCT [a0].[Id], COUNT([TypeId]) AS [TypeId]
FROM [tblData] AS [a0]
GROUP BY COUNT([a0].[Id])
ORDER BY [a0].[TypeId]
I had to upgrade EF Core version to 2.1 and LINQ is now translated properly into SQL.
I have two similar queries, the first one:
var activatedSerialNumbers = (from activation in entities.Activations
where !canceledActivationsIds.Contains(activation.Id)
where activation.CustomerId == customerId
join licenseConfiguration in entities.LicenseConfigurations
on activation.Id equals licenseConfiguration.ActivationId
where licenseConfiguration.ProductId == productId
join activatedSerialNumber in entities.ActivatedSerialNumbers
on activation.Id equals activatedSerialNumber.ActivationId
where deactivatedSams.All(dsn => dsn.ToLower() !=
activatedSerialNumber.Name.ToLower())
select new SamWithLicense
{
Name = activatedSerialNumber.Name,
Features = licenseConfiguration.LicenseFeatures
}).ToList();
The second:
var activationsForSam = (from activation in entities.Activations
where !canceledActivationsIds.Contains(activation.Id)
where activation.CustomerId == customerId
let activatedSerialNumbers = activation.ActivatedSerialNumbers
.Select(sn => sn.Name.ToLower())
where activatedSerialNumbers.Contains(loweredSn)
join licenseConfiguration in entities.LicenseConfigurations
on activation.Id equals activatedProduct.ActivationId
select new SamWithLicense
{
Name = selectedSerialNumber,
Features = licenseConfiguration.LicenseFeatures
}).ToList();
In some situations I execute them one after another and in most cases it works fine, but somethimes - not. In the result of second query Counter takes from another row:
Visual Studio - Quick watch
SQL Management Studio
I guess it's a matter of a EF cache or smth, but don't know how to fix it properly.
In your first query you are joining the Activation Id (PK) to LicenseConfigurations ActivationId (FK)
join licenseConfiguration in entities.LicenseConfigurations
on activation.Id equals licenseConfiguration.ActivationId
in your second query, it looks like you are joining on a value defined outside of the query "activatedProduct"
join licenseConfiguration in entities.LicenseConfigurations
on activation.Id equals activatedProduct.ActivationId
I have the following query to start with:
var query = from p in db.Products
from pc in p.NpProductCategories
where pc.CategoryId == categoryId
select p;
I'm applying some more filtering on it and in the end I want to sort the results:
if (orderBy == ProductSortingEnum.Name)
query = query.OrderBy(x => x.Name);
else
query = query.OrderBy(............);
My big problem (coming from not knowing linq too good) is the ELSE here. How can I sort results by a column that is not in the current result set? I would like to somehow link to another linq query in the orderby. The sorting I'm trying to achive is to link to NpProductVariants query using the ProductId to match between NpProductVariant and Products
and sort by the Price of the NpProductVariant
Assuming you have the relationship set up in the dbml...
For one to one (and many to one):
query = query.OrderBy(p => p.NpProductVariant.Price);
For one to many:
query = query.OrderBy(p => p.NpProductVariants.Select(v => v.Price).Max());
Also:
var query =
from p in db.Products
where p.NpProductCategories.Any(pc => pc.CategoryId == categoryId)
select p;
I think you can hook your Join to your query as long as it is returning the same thing. So maybe something like (I'm not 100 % sure since I haven't tried it):
query = from i1 in query
join i2 in query2 on i1.PropertyToJoin equals i2.PropertyToJoin
orderby i1.OrderProp1, i2.OrderProp2
select i1;
But I think it might be a good idea to check the generated sql so it is still effective.
I have some Linq code and it's working fine. It's a query that has a subquery in the Where clause. This subquery is doing a groupby. Works great.
The problem is that I don't know how to grab one of the results from the subquery out of the subquery into the parent.
Frst, here's the code. After that, I'll expplain what piece of data i'm wanting to extract.
var results = (from a in db.tblProducts
where (from r in db.tblReviews
where r.IdUserModified == 1
group r by
new
{
r.tblAddress.IdProductCode_Alpha,
r.tblAddress.IdProductCode_Beta,
r.tblAddress.IdProductCode_Gamma
}
into productGroup
orderby productGroup.Count() descending
select
new
{
productGroup.Key.IdProductCode_Alpha,
productGroup.Key.IdProductCode_Beta,
productGroup.Key.IdProductCode_Gamma,
ReviewCount = productGroup.Count()
}).Take(3)
.Any(
r =>
r.IdProductCode_Alpha== a.IdProductCode_Alpha&&
r.IdProductCode_Beta== a.IdProductCode_Beta&&
r.IdProductCode_Gamma== a.IdProductCode_Gamma)
where a.ProductFirstName == ""
select new {a.IdProduct, a.FullName}).ToList();
Ok. I've changed some field and tables names to protect the innocent. :)
See this last line :-
select new {a.IdProduct, a.FullName}).ToList();
I wish to include in that the ReviewCount (from the subquery). I'm jus not sure how.
To help understand the problem, this is what the data looks like.
Sub Query
IdProductCode_Alpha = 1, IdProductCode_Beta = 2, IdProductCode_Gamma = 3, ReviewCount = 10
... row 2 ...
... row 3 ...
Parent Query
IdProduct = 69, FullName = 'Jon Skeet's Wonder Balm'
So the subquery grabs the actual data i need. The parent query determines the correct product, based on the subquery filters.
EDIT 1: Schema
tblProducts
IdProductCode
FullName
ProductFirstName
tblReviews (each product has zero to many reviews)
IdProduct
IdProductCode_Alpha (can be null)
IdProductCode_Beta (can be null)
IdProductCode_Gamma (can be null)
IdPerson
So i'm trying to find the top 3 products a person has done reviews on.
The linq works perfectly... except i just don't know how to include the COUNT in the parent query (ie. pull that result from the subquery).
Cheers :)
Got it myself. Take note of the double from at the start of the query, then the Any() being replaced by a Where() clause.
var results = (from a in db.tblProducts
from g in (
from r in db.tblReviews
where r.IdUserModified == 1
group r by
new
{
r.tblAddress.IdProductCode_Alpha,
r.tblAddress.IdProductCode_Beta,
r.tblAddress.IdProductCode_Gamma
}
into productGroup
orderby productGroup.Count() descending
select
new
{
productGroup.Key.IdProductCode_Alpha,
productGroup.Key.IdProductCode_Beta,
productGroup.Key.IdProductCode_Gamma,
ReviewCount = productGroup.Count()
})
.Take(3)
Where(g.IdProductCode_Alpha== a.IdProductCode_Alpha&&
g.IdProductCode_Beta== a.IdProductCode_Beta&&
g.IdProductCode_Gamma== a.IdProductCode_Gamma)
where a.ProductFirstName == ""
select new {a.IdProduct, a.FullName, g.ReviewCount}).ToList();
While I don't understand LINQ completely, but wouldn't the JOIN work?
I know my answer doesn't help but it looks like you need a JOIN with the inner table(?).
I agree with shahkalpesh, both about the schema and the join.
You should be able to refactor...
r => r.IdProductCode_Alpha == a.IdProductCode_Alpha &&
r.IdProductCode_Beta == a.IdProductCode_Beta &&
r.IdProductCode_Gamma == a.IdProductCode_Gamma
into an inner join with tblProducts.