SQL query with Group By and Having Clause in LINQ structure - linq

how do I write this query in LINQ (c# EF6)?
Can someone please help me here - I am new to Entity Framework Structure - So bit hard for me to use different clauses
SELECT
sum(remainingamount) TotalSiteCreditAmount,
max(expirationutcdatetime) MaxExpiryDate
FROM
WholesaleCredits
WHERE
ExpirationUTCDateTime > Getdate()
GROUP BY
registeredcustomerid,
siteid
HAVING
registeredcustomerid = :registeredCustomerId
AND siteid = :siteId
Tried below thing as of now :
var data = context.WholesaleCredit
.Where(x => x.ExpirationUTCDateTime > DateTime.Now)
.GroupBy (x => x.RegisteredCustomerId)
Entity Used in code:
public partial class WholesaleCredits
{
public virtual int Id { get; set; }
public virtual decimal CreditAmount { get; set; }
public virtual decimal RemainingAmount { get; set; }
public virtual Site Site { get; set; }
public virtual DateTime GeneratedUTCDateTime { get; set; }
public virtual DateTime ExpirationUTCDateTime { get; set; }
public virtual int RegisteredCustomerId { get; set; }
}

You do not need HAVING here and Grouping should be provided by constant, because you have filter on grouping keys:
var data = context.WholesaleCredit
.Where(x => x.ExpirationUTCDateTime > DateTime.Now && x.RegisteredCustomerId == registeredCustomerId && x.Site.Id == siteid)
.GroupBy(x => 1)
.Select(g => new
{
TotalSiteCreditAmount = g.Sum(x => x.RemainingAmount),
MaxExpiryDate = g.Max(x => x.ExpirationUTCDateTime)
})
.First();

Related

EntityFrameworkCore throws NullReferenceException when querying data

I'm using EF Core 2.2 to connect to an Oracle11g database (using these old versions is a must), when I try to query one of my tables it throws a NullReferenceException in the EF Core external code.
The Query:
string username = "Administrator";
var user = _context.Users.Where(x => x.UserName.ToUpper() == username.ToUpper())
.Include(x => x.UserProfile)
.Include(x => x.UserRoleOrganizations).ThenInclude(s => s.Role)
.Include(x => x.UserRoleOrganizations).ThenInclude(s => s.Organization);
return user.FirstOrDefault();
The exception:
System.NullReferenceException
HResult=0x80004003
Message=Object reference not set to an instance of an object.
Source=Microsoft.EntityFrameworkCore.Relational
StackTrace:
at Microsoft.EntityFrameworkCore.Storage.TypedRelationalValueBufferFactoryFactory.CreateGetValueExpression(Expression dataReaderExpression, Int32 index, TypeMaterializationInfo materializationInfo, Boolean detailedErrorsEnabled, Boolean box)
at Microsoft.EntityFrameworkCore.Storage.TypedRelationalValueBufferFactoryFactory.<>c__DisplayClass13_0.<CreateArrayInitializer>b__0(TypeMaterializationInfo mi, Int32 i)
at System.Linq.Enumerable.<SelectIterator>d__5`2.MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Dynamic.Utils.CollectionExtensions.ToReadOnly[T](IEnumerable`1 enumerable)
at System.Linq.Expressions.Expression.NewArrayInit(Type type, IEnumerable`1 initializers)
at Microsoft.EntityFrameworkCore.Storage.TypedRelationalValueBufferFactoryFactory.CreateArrayInitializer(CacheKey cacheKey, Boolean detailedErrorsEnabled)
at Microsoft.EntityFrameworkCore.Storage.TypedRelationalValueBufferFactoryFactory.<Create>b__11_0(CacheKey k)
at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
at Microsoft.EntityFrameworkCore.Internal.NonCapturingLazyInitializer.EnsureInitialized[TParam,TValue](TValue& target, TParam param, Func`2 valueFactory)
at Microsoft.EntityFrameworkCore.Query.Internal.ShaperCommandContext.NotifyReaderCreated(DbDataReader dataReader)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.BufferlessMoveNext(DbContext _, Boolean buffer)
at Oracle.EntityFrameworkCore.Storage.Internal.OracleExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable`1.Enumerator.MoveNext()
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ResultEnumerable`1.GetEnumerator()
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.<_TrackEntities>d__17`2.MoveNext()
at Microsoft.EntityFrameworkCore.Query.Internal.LinqOperatorProvider.ExceptionInterceptor`1.EnumeratorExceptionInterceptor.MoveNext()
at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)
at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass15_1`1.<CompileQueryCore>b__0(QueryContext qc)
at Base.Security.Business.UserManagement.GetUser(String username) in C:\Users\Afagh2\source\repos\HSE2_Oracle\Base.Security\Business\UserManagement.cs:line 88
The generated query:
[Parameters=[:ToUpper_0='ADMINISTRATOR' (Size = 256)], CommandType='Text', CommandTimeout='0']
Select
K0 "Id", K1 "AccessFailedCount", K2 "ConcurrencyStamp", K3 "Email", K4 "EmailConfirmed", K5 "FirstName", K6 "IsActive", K7 "IsExternal", K8 "LastName", K9 "LockoutEnabled", K10 "LockoutEnd", K11 "NationalCode", K12 "NormalizedEmail", K13 "NormalizedUserName", K14 "OrganizationName", K15 "OrganizationType", K16 "PasswordHash", K17 "PersonnelCode", K18 "PhoneNumber", K19 "PhoneNumberConfirmed", K20 "PositionName", K21 "SecurityStamp", K22 "TwoFactorEnabled", K23 "UserName", K24 "Id", K25 "FileContent", K26 "FileExtension", K27 "FileName" from(
SELECT "x"."Id" K0, "x"."AccessFailedCount" K1, "x"."ConcurrencyStamp" K2, "x"."Email" K3, "x"."EmailConfirmed" K4, "x"."FirstName" K5, "x"."IsActive" K6, "x"."IsExternal" K7, "x"."LastName" K8, "x"."LockoutEnabled" K9, "x"."LockoutEnd" K10, "x"."NationalCode" K11, "x"."NormalizedEmail" K12, "x"."NormalizedUserName" K13, "x"."OrganizationName" K14, "x"."OrganizationType" K15, "x"."PasswordHash" K16, "x"."PersonnelCode" K17, "x"."PhoneNumber" K18, "x"."PhoneNumberConfirmed" K19, "x"."PositionName" K20, "x"."SecurityStamp" K21, "x"."TwoFactorEnabled" K22, "x"."UserName" K23, "x.UserProfile"."Id" K24, "x.UserProfile"."FileContent" K25, "x.UserProfile"."FileExtension" K26, "x.UserProfile"."FileName" K27
FROM "AppUser" "x"
LEFT JOIN "AppUserProfilePicture" "x.UserProfile" ON ("x"."Id" = "x.UserProfile"."Id")
WHERE ("UPPER"("x"."UserName") = :ToUpper_0)
ORDER BY "x"."Id"
) "m1"
where rownum <= 1
The query works as expected in Oracle SQL Developer. It seems that EF Core has some trouble converting the results to the entity (which in this case, the result should be none because the table has no data.) but I can not debug its code. (Or can I?)
The entity is configured in OnModelCreating:
builder.Entity<ApplicationUser>(b =>
{
b.HasKey(u => u.Id);
b.HasIndex(u => u.NormalizedUserName).HasName("UserNameIndex").IsUnique();
b.HasIndex(u => u.NormalizedEmail).HasName("EmailIndex");
b.ToTable("AppUser");
b.Property(u => u.ConcurrencyStamp).IsConcurrencyToken();
b.Property(u => u.UserName).HasMaxLength(256);
b.Property(u => u.NormalizedUserName).HasMaxLength(256);
b.Property(u => u.Email).HasMaxLength(256);
b.Property(u => u.NormalizedEmail).HasMaxLength(256);
b.HasOne(x => x.UserProfile).WithOne(x => x.User).HasForeignKey<ApplicationUserProfilePicture>(x => x.Id);
b.HasMany<IdentityUserClaim<int>>().WithOne().HasForeignKey(uc => uc.UserId).IsRequired();
b.HasMany<IdentityUserLogin<int>>().WithOne().HasForeignKey(ul => ul.UserId).IsRequired();
b.HasMany<IdentityUserToken<int>>().WithOne().HasForeignKey(ut => ut.UserId).IsRequired();
b.HasMany<ApplicationUserRoleOrganization>().WithOne().HasForeignKey(ur => ur.UserId).IsRequired();
});
I can create and query other tables. Only AppUser table doesn't work in Oracle, it works fine using the SQL Server provider.
Tell me if more info about anything is needed.
Update
User class:
public class ApplicationUser : Microsoft.AspNetCore.Identity.IdentityUser<int>
{
public string FirstName { get; set; }
public string LastName { get; set; }
[StringLength(450)]
public string PositionName { get; set; }
[StringLength(450)]
public string NationalCode { get; set; }
[StringLength(450)]
public string PersonnelCode { get; set; }
public bool IsActive { get; set; } = true;
public bool IsExternal { get; set; } = false;
[Required]
public OrganizationTypeEnum OrganizationType { get; set; }
public string OrganizationName { get; set; }
public virtual ApplicationUserProfilePicture UserProfile { get; set; }
public virtual ICollection<ApplicationUserRoleOrganization> UserRoleOrganizations { get; set; }
}
UserRoleOrganization class:
public class ApplicationUserRoleOrganization : Microsoft.AspNetCore.Identity.IdentityUserRole<int>
{
public int Id { get; set; }
public override int RoleId { get => base.RoleId; set => base.RoleId = value; }
public override int UserId { get => base.UserId; set => base.UserId = value; }
public virtual int OrganizationId { get; set; }
public virtual ApplicationUser User { get; set; }
public virtual ApplicationRole Role { get; set; }
public virtual Organization Organization { get; set; }
}
The configuration of UserRoleOrganization:
builder.Entity<ApplicationUserRoleOrganization>(b =>
{
b.HasKey(r => r.Id);
b.HasIndex(t => t.OrganizationId).HasName("OrganizationIndex");
b.HasIndex(t => t.RoleId).HasName("RoleIndex");
b.HasIndex(t => t.UserId).HasName("UserIndex");
b.HasOne(userRole => userRole.Role)
.WithMany(role => role.UserRoleOrganizations)
.HasForeignKey(userRole => userRole.RoleId).HasConstraintName("FK_RoleOrg_Role_RoleId");
b.HasOne(userRole => userRole.Organization)
.WithMany(role => role.UserRoleOrganizations)
.HasForeignKey(userRole => userRole.OrganizationId).HasConstraintName("FK_RoleOrg_Org_OrgId");
b.HasOne(userRole => userRole.User)
.WithMany(user => user.UserRoleOrganizations)
.HasForeignKey(userRole => userRole.UserId).HasConstraintName("FK_RoleOrg_User_UserId");
b.ToTable("AppUserRoleOrganization");
});
I just noticed a text is written in my output just before the exception:
fail: Microsoft.EntityFrameworkCore.Query[10100]
An exception occurred while iterating over the results of a query for context type 'Base.Security.Data.SecurityDbContext'.
You have included UserRoleOrganizations twice . Try to do including the good old way:
string username = "Administrator";
return _context.Users.Where(x => x.UserName.ToUpper() == username.ToUpper())
.Include(x => x.UserProfile)
.Include("UserRoleOrganizations.Organization");
.Include("UserRoleOrganizations.Role")
.FirstOrDefault();
But I don't like UserRoleOrganizations.Organization . Could you please show your User and UserRoleOrganizations clases?
Update.
Since I can see the classes, I think you can try to reverse query, maybe just for testing:
var userRoleOrganizations=_context.UserRoleOrganizations
.Where(x => x.User.UserName.ToUpper() == username.ToUpper())
.Include(s => s.Role)
.Include(s => s.Organization)
.ToList();
var user = _context.Users
.Where(x => x.UserName.ToUpper() == username.ToUpper())
.Include(x => x.UserProfile)
.FirstOrDefault();
user.UserRoleOrganizations=userRoleOrganizations;
return user;
And try to change the UserRoleOrganizations class:
public class UserRoleOrganizations
{
....
public override int? RoleId { get => base.RoleId; set => base.RoleId = value; }
public override int? UserId { get => base.UserId; set => base.UserId = value; }
public virtual int? OrganizationId { get; set; }
....
}

How and where to use AddRange() method

I want to display related data from second table with each value in first table
i have tried this query
public ActionResult Index()
{
List<EmployeeAtt> empWithDate = new List<EmployeeAtt>();
var employeelist = _context.TblEmployee.ToList();
foreach (var employee in employeelist)
{
var employeeAtt = _context.AttendanceTable
.GroupBy(a => a.DateAndTime.Date)
.Select(g => new EmployeeAtt
{
Date = g.Key,
Emp_name = employee.EmployeeName,
InTime = g.Any(e => e.ScanType == "I") ? g.Where(e =>
e.ScanType == "I").Min(e =>
e.DateAndTime.ToShortTimeString())
.ToString() : "Absent",
OutTime = g.Any(e => e.ScanType == "O") ? g.Where(e =>
e.ScanType == "O").Max(e =>
e.DateAndTime.ToShortTimeString())
.ToString() : "Absent"
});
empWithDate.AddRange(employeeAtt);
}
return View(empWithDate);
}
Here is my attendance Table
AttendanceTable
Results
I want to display the shortest time with "I" Column value against each employee and last time with "O" Column value as out time. I think i am not using AddRange() at proper place. Where it should go then?
public partial class TblEmployee
{
public TblEmployee()
{
AttendanceTable = new HashSet<AttendanceTable>();
}
public int EmpId { get; set; }
public string EmployeeName { get; set; }
public virtual ICollection<AttendanceTable> AttendanceTable { get; set; }
}
public partial class AttendanceTable
{
public int Id { get; set; }
public int AttendanceId { get; set; }
public int EmployeeId { get; set; }
public string ScanType { get; set; }
public DateTime DateAndTime { get; set; }
public virtual TblEmployee Employee { get; set; }
}
The actual problem is not related to AddRange(), you need a where clause before GroupBy() to limit attendances (before grouping) to only records related to that specific employee, e.g.
_context.AttendanceTable
.Where(a => a.Employee == employee.EmployeeName)
.GroupBy(a => a.DateAndTime.Date)
...
Depended on your model, it is better to use some kind of ID instead of EmployeeName for comparison if possible.
Also you can use SelectMany() instead of for loop and AddRange() to combine the results into a single list. like this:
List<EmployeeAtt> empWithDate = _context.TblEmployee.ToList()
.SelectMany(employee =>
_context.AttendanceTable
.Where(a => a.Employee == employee.EmployeeName)
.GroupBy(a => a.DateAndTime.Date)
.Select(g => new EmployeeAtt
{
...
})
);
...

Selecting single element from each group that cointains maximum DateTime value

model :
public class ReferenceParameterHistory
{
[Key]
public int IDReferenceParameterHistory { get; set; }
public double Value { get; set; }
public string Value_S { get; set; }
public DateTimeOffset CreatedAt { get; set; }
[Required]
public bool IsStable { get; set; }
public int? IDReference { get; set; }
public Reference Reference { get; set; }
[Required]
public int IDParameterTemplate { get; set; }
public ParameterTemplate ParameterTemplate { get; set; }
}
My code in ASP.NET core controller :
[HttpGet]
public async Task<IActionResult> GetReferenceParameterHistory(int? IDparameterTemplate,
int? IDreference,
DateTimeOffset? startDate,
DateTimeOffset? endDate,
bool latestOnly)
{
try
{
IQueryable<ReferenceParameterHistory> query = _context.ReferenceParameterHistory.OrderByDescending(rph => rph.IDReferenceParameterHistory);
if (IDparameterTemplate != null && IDparameterTemplate > 0)
query = query.Where(rph => rph.IDParameterTemplate == IDparameterTemplate);
if (IDreference != null && IDreference > 0)
query = query.Where(rph => rph.IDReference == IDreference);
if (startDate != null)
query = query.Where(rph=> rph.CreatedAt >= startDate);
if (endDate != null)
query = query.Where(rph => rph.CreatedAt <= endDate);
if (latestOnly)
{
// I tried this but it doesnt compile and I don't have idea how to solve this ....
//query = (from rph in query
// group rph by rph.IDParameterTemplate
// into groups
// where groups.Max(rph => rph.CreatedAt)
// select groups.Key);
}
var referenceParameterHistory = await query.AsNoTracking().ToListAsync();
if (referenceParameterHistory.Any())
return new ObjectResult(referenceParameterHistory);
return new NotFoundResult();
}
catch (Exception ex)
{
_logger.LogError(ex.Message);
return new StatusCodeResult(500);
}
}
I have a database table based on that ReferenceParameterHistory model class. I want to group records exctracted from that table by IDParameterTemplate and from each group I need to extract records that have the highest value in CreatedAt column (latest records). So each group contains many recods but I need to get only these with max value in CreatedAt column. The result should be IEnumerable of ReferenceParameterHistory since I store that query in an IQueryable variable and then send it to SQL Server to process the query. Commented code in my example is just what I tried but I don't know how to do that.
How can I solve that problem ?
You reused the variable rph inside the lambda.
How to select only the records with the highest date in LINQ
query = (from rph in query
group rph by rph.IDParameterTemplate into g
select g.OrderByDescending(t=>t.CreatedAt).FirstOrDefault());

ASP.Net MVC converting Sql to Linq

I'm updating an old app, to use EF and Linq. I'm having trouble with one of the queries - in SQL it is:
SELECT id, type_id, rule_name, rule_start, rule_end, rule_min
FROM Rules
WHERE (rule_min > 0)
AND (rule_active = 1)
AND (rule_fri = 1)
AND ('2012-01-01' BETWEEN rule_start AND rule_end)
AND (id IN
(SELECT rule_id
FROM RulesApply
WHERE (type_id = 3059)))
ORDER BY pri
So far I have:
var rules = db.Rules.Include("RulesApply")
.Where(t => (t.rule_active == 1)
&& (t.rule_min > 0)
&& (dteFrom >= t.rule_start && dteFrom <= t.rule_end)
&& (this is where I'm stuck)
)
.OrderBy(r => r.pri);
It's the last subquery I'm stuck with adding into the LINQ above:
AND (id IN
(SELECT rule_id
FROM RulesApply
WHERE (type_id = 3059)))
Models are:
public class Rule
{
[Key]
public Int64 id { get; set; }
public Int64 hotel_id { get; set; }
public byte rule_active { get; set; }
public DateTime rule_start { get; set; }
public DateTime rule_end { get; set; }
public int rule_min { get; set; }
public int pri { get; set; }
public virtual ICollection<RuleApply> RulesApply { get; set; }
}
public class RuleApply
{
[Key, Column(Order = 0)]
public Int64 type_id { get; set; }
[Key, Column(Order = 1)]
public Int64 rule_id { get; set; }
[ForeignKey("rule_id")]
public virtual Rule Rule { get; set; }
}
Can anyone please help me complete this query?
Thank you,
Mark
Try doing this:
var rules = db.Rules.Include("RulesApply")
.Where(t => (t.rule_active == 1)
&& (t.rule_min > 0)
&& (dteFrom >= t.rule_start && dteFrom <= t.rule_end)
&& t.RulesApply.Any(a => a.type_id == 3059)
.OrderBy(r => r.pri);
If t.RulesApply is illegal (i.e. doesn't compile), then replace it with the correct reference to the navigation property found on your Rules object that points to the RulesApply object.
If you have set up navigational properties between the entities, you can navigate from one to the other:
//This gets the RulesApply object
var rulesapply = db.RulesApply.Single(x=> x.type_id == 3059);
//This gets all Rules connected to the rulesapply object through its navigational property
var rules = rulesapply.Rules;
//You can use LINQ to further refine what you want
rules = rules.Where( x=> /* and so on...*/ );
You can stack these statements together on a single line, I only split them up for readability purposes :)

Entity Framework, MVC 3, OrderBy in LINQ To Entities

I've got the following query:
model.Page = db.Pages
.Where(p => p.PageId == Id)
.Include(p => p.Series
.Select(c => c.Comics
.Select(col => col.Collection)))
.SingleOrDefault();
This works great, although I now need to order the Comics by a property called 'ReadingOrder'.
I've tried:
model.Page = db.Pages
.Where(p => p.PageId == Id)
.Include(p => p.Series.Select(c => c.Comics.OrderBy(o => o.ReadingOrder)
.Select(col => col.Collection)))
.SingleOrDefault();
But this results in the following error:
The Include path expression must refer to a navigation property
defined on the type. Use dotted paths for reference navigation
properties and the Select operator for collection navigation
properties. Parameter name: path
Any ideas what this error means?
Thanks in advance
EDIT:
My models:
public class Page
{
public int PageId { get; set; }
public string Title { get; set; }
public ICollection<Series> Series { get; set; }
}
public class Series
{
public int SeriesId { get; set; }
public int PageId { get; set; }
public string Title { get; set; }
public Page Page { get; set; }
public ICollection<Comic> Comics { get; set; }
}
public class Comic
{
public int ComicId { get; set; }
public string Title { get; set; }
public int ReadingOrder { get; set; }
public string Subtitle { get; set; }
public int CollectionId { get; set; }
public Collection Collection { get; set; }
}
public class Collection
{
public int CollectionId { get; set; }
public string Title { get; set; }
public ICollection<Comic> Comics { get; set; }
}
The exception "...Include path expression must refer to a navigation property..." basically complains that c.Comics.OrderBy is not a navigation property. (It's a legitimate complaint, I think.)
Actually it's not supported by EF to apply sorting (and also filtering) in eager loading statements (Include).
So, what can you do?
Option 1:
Sort in memory after you have loaded the entity:
model.Page = db.Pages
.Where(p => p.PageId == Id)
.Include(p => p.Series.Select(c => c.Comics
.Select(col => col.Collection)))
.SingleOrDefault();
if (model.Page != null)
{
foreach (var series in model.Page.Series)
series.Comics = series.Comics.OrderBy(c => c.ReadingOrder).ToList();
}
Ugly, but because you are loading apparently only a single Page object by id it's possibly faster (LINQ to Objects in memory) than the following options (if Series and Comics collections are not extraordinarily long).
Option 2:
Break down the query in parts which mix eager and explicite loading:
model.Page = db.Pages
.Where(p => p.PageId == Id)
.Include(p => p.Series) // only Series collection is included
.SingleOrDefault();
if (model.Page != null)
{
foreach (var series in model.Page.Series)
db.Entry(series).Collection(s => s.Comics).Query()
.Include(c => c.Collection)
.OrderBy(c => c.ReadingOrder)
.Load(); // one new DB query for each series in loop
}
Option 3:
Projection?
Here and here is by the way something about the dangers of complex Include chains of multiple navigation properties. It can load huge amounts of duplicated data. Include ensures that you only have one DB roundtrip but possibly at the cost of much more transfered data. Explicite loading has multiple roundtrips but with possibly less data in total.
(I know, I gave you this Include...Select...Select...Select... chain, but how could I know that you would take me serious :). Well, depending on the size of your nested collections it can still be the best option.)
Off the top of my head, untested:
model.Page = db.Pages
.Where(p => p.PageId == Id)
.Include(p => p.Series
.Select(c => c.Comics
.Select(col => col.Collection)
.OrderBy(o => o.ReadingOrder)))
.SingleOrDefault();

Resources