EFCore 2.2 GroupBy Sum and DateDiff - linq

I'm trying to translate the following SQL in to an EF Core query and I'm getting warnings that GroupBy and Sum will be evaluated locally. Is there anyway currently to write this that it will fully translate to SQL?
SELECT UserId, ST.StatusId, SUM(DATEDIFF(MINUTE, StartDate, ISNULL(EndDate,GETDATE()))) AS Time
FROM StatusTransaction ST
WHERE
TeamManagerId = 1
AND StartDate >= N'01-01-2019'
AND ISNULL(EndDate,GETDATE()) <= N'01-02-2019'
GROUP BY UserId, ST.StatusId
ORDER BY UserId
And these are the EF queries I've used:
var efFunction = await context
.Where(st => st.TeamManagerId == tmId && st.StartDate >= dateFrom && (st.EndDate ?? DateTime.Now) <= dateTo)
.GroupBy(st => new { st.UserId, st.StatusId })
.Select(g => new
{
g.Key.UserId,
g.Key.StatusId,
Time = g.Sum(st => Microsoft.EntityFrameworkCore.EF.Functions.DateDiffMinute(st.StartDate, st.EndDate)) // null check not done on end date - (st.EndDate ?? DateTime.Now) causes an error here
}).ToListAsync(cancellationToken).ConfigureAwait(false);
var simpleDateSubtraction = await context
.Where(st => st.TeamManagerId == tmId && st.StartDate >= dateFrom && (st.EndDate ?? DateTime.Now) <= dateTo)
.GroupBy(st => new { st.UserId, st.StatusId })
.Select(g => new
{
g.Key.UserId,
g.Key.StatusId,
Time = g.Sum(st => st.EndDate.Value.Subtract(st.StartDate).Minutes)// null check not done on end date - (st.EndDate ?? DateTime.Now) causes an error here
}).ToListAsync(cancellationToken).ConfigureAwait(false);
var groupBySimpleSum = await context
.Where(st => st.TeamManagerId == tmId)
.GroupBy(st => new { st.TeamManagerId, st.OperationsManagerId })
.Select(g => new
{
g.Key.OperationsManagerId,
g.Key.TeamManagerId,
Foo = g.Sum(st => st.UserId) // nonsense but a simple column to sum, this translates fully to SQL
}).ToListAsync(cancellationToken).ConfigureAwait(false);

First, EF Core still doesn't support translating TimeSpan operations, and DateTime difference produces TimeSpan, hence EF.Functions.DateDiff methods are the right way to go.
Second, it still can translate GroupBy aggregates only on simple member accessor expressions. So you have to either pre Select the GroupBy expressions:
var query = context
.Where(st => st.TeamManagerId == tmId
&& st.StartDate >= dateFrom
&& (st.EndDate ?? DateTime.Now) <= dateTo
)
.Select(st => new
{
st.UserId,
st.StatusId,
Time = EF.Functions.DateDiffMinute(st.StartDate, st.EndDate ?? DateTime.Now)
})
.GroupBy(st => new { st.UserId, st.StatusId })
.Select(g => new
{
g.Key.UserId,
g.Key.StatusId,
Time = g.Sum(st => st.Time)
});
or use the GroupBy overload which allows pre selecting the source for the aggregates:
var query = context
.Where(st => st.TeamManagerId == tmId
&& st.StartDate >= dateFrom
&& (st.EndDate ?? DateTime.Now) <= dateTo
)
.GroupBy(st => new { st.UserId, st.StatusId }, st => new
{
Time = EF.Functions.DateDiffMinute(st.StartDate, st.EndDate ?? DateTime.Now)
})
.Select(g => new
{
g.Key.UserId,
g.Key.StatusId,
Time = g.Sum(st => st.Time)
});

Related

LINQ: Cannot get more than two Where clauses to work

I cannot get the following to work properly. I want to search my database using more than 2 conditions where it will be possible to search only on one, two or all three, thus narrowing the search. At the moment, I can only get it to work with one condition at a time, and it only evaluates the first condition when having more than 3.
How can I Include more than 2 conditions? I have tried several proposed solutions from other sources, that uses if statements and where clauses, but so far none of them have worked.
This is my controller
public IActionResult Index(string searchString, int searchProject, string searchTitel)
{
var Projektdokumenter = from p in _context.ProjekterDokutypeDokuundertype
.Include(p => p.Dokumenttype)
.Include(p => p.Dokuundertype)
.Include(p => p.Forfatter)
.Include(p => p.P)
.Include(p => p.Sprog)
select p;
if (!String.IsNullOrEmpty(searchString) ^ searchProject != null ^ !String.IsNullOrEmpty(searchTitel))
{
if (!String.IsNullOrEmpty(searchString))
{
var c = Projektdokumenter.Where(p => p.Forfatter.Initialer.Contains(searchString)).ToList();
return View(c);
}
{
if (searchProject != 0)
{
var d = Projektdokumenter.Where(p => p.P.ProjektId.Equals(searchProject)).ToList();
return View(d);
}
{
if(!String.IsNullOrEmpty(searchTitel))
{
var f = Projektdokumenter.Where(p => p.Titel.Contains(searchTitel)).ToList();
return View(f);
}
}
}
}
var e = Projektdokumenter.Where(c.Contains(searchString) && p.P.ProjektId.Equals(searchProject) && p.Titel.Contains(searchTitel)).ToList();
return View(e);
}
IQueryable can be combined, so just write in native way:
{
var query = _context.ProjekterDokutypeDokuundertype
.Include(p => p.Dokumenttype)
.Include(p => p.Dokuundertype)
.Include(p => p.Forfatter)
.Include(p => p.P)
.Include(p => p.Sprog)
.AsQueryable();
if (!string.IsNullOrEmpty(searchString))
{
query = query.Where(p => p.Forfatter.Initialer.Contains(searchString));
}
if (searchProject != 0)
{
query = query.Where(p => p.P.ProjektId == searchProject);
}
if (!string.IsNullOrEmpty(searchTitel))
{
query = query.Where(p => p.Titel.Contains(searchTitel))
}
return View(query.ToList());
}

How do I exclude certain parts of a .Include on a condition?

We are trying to exclude certain data from our queries based on a condition, but we can't seem to get it right. Our current code looks like this:
var PullData = _context.RequestInformation.Include(x => x.RequestMinistry)
.Include(x => x.RequestApps)
.ThenInclude(x => x.RequestAppDataChoices)
.Where(x => x.RequestStatus != "Cancelled With Reason")
.Where(y => y.RequestApps.SelectMany(x => x.RequestAppDataChoices)
.Where(s => s.AppDataChoiceDl.Contains(groupName))
.Where(a => a.RequestApps.AppProvisioningCompleteTime != null)
.Where(aa => aa.RequestApps.AppTeamCompleteTime == null)
.Where(ab => !ab.RequestApps.AppStatus.Contains("Rejected")).Any());
This is fine as long as we are trying to pull all of the "RequestApps" with each set of "RequestInformation", however if we want to pull just specific applications based on conditions, it doesn't work. We want to only show the request apps based on the the conditions above (The appdatachoicedl contains "groupname", the completetime != null while the other completetime IS null, and that the status is not rejected), but instead, we are getting all the request information that matches those conditions.
I tried something like this, but it still doesn't quite give me the results I need. It gives me random blank results as well:
var PullDataV2 = from reqInfo in _context.RequestInformation
join RA in _context.RequestApps on reqInfo.ReqNum equals RA.ReqNum
join RADC in _context.RequestAppDataChoices on RA.PkId equals RADC.RequestAppsId
where RADC.AppDataChoiceDl == groupName
select new RequestInformation()
{
ProvisioningCompleteTime = reqInfo.ProvisioningCompleteTime,
RequestMinistry = reqInfo.RequestMinistry,
RequestApps = reqInfo.RequestApps,
UserFirstName = reqInfo.UserFirstName,
UserLastName = reqInfo.UserLastName,
SubmitterFirstName = reqInfo.SubmitterFirstName,
SubmitterLastName = reqInfo.SubmitterLastName,
RequestStatus = reqInfo.RequestStatus
};
I hope I'm making sense here - I only want the requestapps based on the conditions, not all the request information that is those conditions. RequestApps is a part of the RequestInformation class.
Edit:
Piggybacking off of Roberts comments, I modified his code to look like this, but I am struggling with the last line - I need to be able to make sure that "AppDataChoiceDL" which is a member of RequestAppDataChoices (which is a member of RequestApps) is equal to groupName
var PullData = _context.RequestInformation.Include(x => x.RequestMinistry)
.Include(x => x.RequestApps)
.ThenInclude(x => x.RequestAppDataChoices)
.Where(x => x.RequestStatus != "Cancelled With Reason")
.Where(x => x.RequestApps.SelectMany(app => app.RequestAppDataChoices)
.Where(a => a.AppDataChoiceDl.Contains(groupName))
.Where(b => b.RequestApps.AppProvisioningCompleteTime != null)
.Where(c => c.RequestApps.AppTeamCompleteTime == null)
.Any(n => !n.RequestApps.AppStatus.Contains("Rejected"))).Select(reqInfo => new RequestInformation
{
ProvisioningCompleteTime = reqInfo.ProvisioningCompleteTime,
RequestMinistry = reqInfo.RequestMinistry,
UserFirstName = reqInfo.UserFirstName,
UserLastName = reqInfo.UserLastName,
SubmitterFirstName = reqInfo.SubmitterFirstName,
SubmitterLastName = reqInfo.SubmitterLastName,
RequestStatus = reqInfo.RequestStatus,
RequestApps = reqInfo.RequestApps
.Where(a => a.AppProvisioningCompleteTime != null)
.Where(b => b.AppTeamCompleteTime == null)
.Where(x => !x.AppStatus.Contains("Rejected"))
.Where(b => b.RequestAppDataChoices.Where(a => a.AppDataChoiceDl == groupName))
});
This throws an error not being able to convert IENUmerable to bool.. maybe I'm selecting my RequestAppDataChoices wrong?
This assumes you have a RequestInformationDto class that closely resembles your RequestInformation class:
var PullData = _context.RequestInformation.Include(x => x.RequestMinistry)
.Include(x => x.RequestApps)
.ThenInclude(x => x.RequestAppDataChoices)
.Where(x => x.RequestStatus != "Cancelled With Reason")
.Where(x => x.RequestApps.SelectMany(app => app.RequestAppDataChoices)
.Where(x => x.AppDataChoiceDl.Contains(groupName))
.Where(x => x.RequestApps.AppProvisioningCompleteTime != null)
.Where(x => x.RequestApps.AppTeamCompleteTime == null)
.Where(x => !x.RequestApps.AppStatus.Contains("Rejected")).Any())
.Select(reqInfo => new RequestInformationDto {
ProvisioningCompleteTime = reqInfo.ProvisioningCompleteTime,
RequestMinistry = reqInfo.RequestMinistry,
RequestApps = reqInfo.RequestApps
.Where(app => app.AppProvisioningCompleteTime != null)
.Where(app => app.AppTeamCompleteTime == null)
.Where(app => !app.AppStatus.Contains("Rejected")).Any())
.ToList()
UserFirstName = reqInfo.UserFirstName,
UserLastName = reqInfo.UserLastName,
SubmitterFirstName = reqInfo.SubmitterFirstName,
SubmitterLastName = reqInfo.SubmitterLastName,
RequestStatus = reqInfo.RequestStatus
});
I just noticed this part of your query, which should probably be reworked (or removed) as well:
.Where(x => x.RequestApps.SelectMany(app => app.RequestAppDataChoices)
.Where(x => x.AppDataChoiceDl.Contains(groupName))
.Where(x => x.RequestApps.AppProvisioningCompleteTime != null)
.Where(x => x.RequestApps.AppTeamCompleteTime == null)
.Where(x => !x.RequestApps.AppStatus.Contains("Rejected")).Any())
With Robert McKee's help, I was able to come up with this, that does work:
var PullData = _context.RequestInformation.Include(x => x.RequestMinistry)
.Include(x => x.RequestApps)
.ThenInclude(x => x.RequestAppDataChoices)
.Where(x => x.RequestStatus != "Cancelled With Reason")
.Where(x => x.RequestApps.SelectMany(app => app.RequestAppDataChoices)
.Where(a => a.AppDataChoiceDl.Contains(groupName))
.Where(b => b.RequestApps.AppProvisioningCompleteTime != null)
.Where(c => c.RequestApps.AppTeamCompleteTime == null)
.Any(n => !n.RequestApps.AppStatus.Contains("Rejected"))).Select(reqInfo => new RequestInformation
{
ProvisioningCompleteTime = reqInfo.ProvisioningCompleteTime,
RequestMinistry = reqInfo.RequestMinistry,
UserFirstName = reqInfo.UserFirstName,
UserLastName = reqInfo.UserLastName,
SubmitterFirstName = reqInfo.SubmitterFirstName,
SubmitterLastName = reqInfo.SubmitterLastName,
RequestStatus = reqInfo.RequestStatus,
RequestApps = reqInfo.RequestApps
.Where(a => a.AppProvisioningCompleteTime != null)
.Where(b => b.AppTeamCompleteTime == null)
.Where(x => !x.AppStatus.Contains("Rejected"))
.Where(b => b.RequestAppDataChoices.Any(a => a.AppDataChoiceDl == groupName)).ToList()
});

Dynamic Bool Query for NEST (ElasticSearch)

Good day:
I'm trying to achieve a dynamic Bool Query. Example...
QueryContainer addressQuery = null;
QueryContainer typeQuery = null;
BoolQueryDescriptor <Facility> boolQuery = new BoolQueryDescriptor<Facility>();
QueryContainerDescriptor<Facility> sh = new QueryContainerDescriptor<Facility>();
SearchDescriptor<Facility> mainSh = new SearchDescriptor<Facility>();
if (search.searchTerm != null)
{
addressQuery = sh.Term(f => f.Address, search.searchTerm) ||
sh.Match(m => m.Field(f => f.Address).Query(search.searchTerm) ) ||
sh.Term(f => f.ZipCode, search.searchTerm) ||
sh.Match(m => m.Field(f => f.State).Query(search.searchTerm));
boolQuery = boolQuery.Should(b => addressQuery);
}
if (search.type != null) {
typeQuery = sh.Term(f => f.Types.First(), search.type);
boolQuery = boolQuery.Must(b => typeQuery && addressQuery);
}
if (boolQuery != null)
{
mainSh = mainSh.Query(q => q.Bool(b => boolQuery));
request = await this._elasticClient.SearchAsync<Facility>(s => mainSh.Size(search.size));
}
This query is turning into a OR between AddressQuery and TypeQuery however, I'd like to achieve an And condition between both query. Effectively making the query dynamic. Is this possible? Example: dynamic addressQuery && typeQuery.
Thanks.
Fire it out..you can && or || query containers...sorter like:
queryContainer1=queryContainer1 && queryContainer2
Then .bool(b => b(queryContainer1)).

Reactive ToProperty not setting property

I have the property Sessions in my vm
private ObservableAsPropertyHelper<IEnumerable<SessionViewModel>> _Sessions;
public IEnumerable<SessionViewModel> Sessions
{
get { return _Sessions.Value; }
}
I am trying to set this in the constructor like so
this.WhenAny(x => x.LocationsViewModel.CurrentLocation, x => x.Date, (location, date) => location.Value)
.Where(x => x != null)
.Select(x => FilterSessions(x.Id, Date))
.ToProperty(this, x => x.Sessions);
FilterSessions looks like so
private IEnumerable<SessionViewModel> FilterSessions(Guid locationId, DateTime date)
{
return _allSessions
.Where(s => s.SessionLocationId == locationId && s.StartTime.Date == date.Date)
.Select(s => new SessionViewModel(s));
}
It returns 10 SessionViewModel's but _Sessions is never set.
What platform is this? Certain platforms do not allow setting private fields via reflection. Instead, you can just use the return value of ToProperty:
_Sessions = this.WhenAny(x => x.LocationsViewModel.CurrentLocation, x => x.Date, (location, date) => location.Value)
.Where(x => x != null)
.Select(x => FilterSessions(x.Id, Date))
.ToProperty(this, x => x.Sessions);
Or better, in RxUI 5.x:
this.WhenAny(x => x.LocationsViewModel.CurrentLocation, x => x.Date, (location, date) => location.Value)
.Where(x => x != null)
.Select(x => FilterSessions(x.Id, Date))
.ToProperty(this, x => x.Sessions, out _Sessions);
I worked around this by settings the property in the subscribe method like so:
this.WhenAny(x => x.LocationsViewModel.CurrentLocation, x => x.Date, (location, date) => location.Value)
.Where(x => x != null)
.Select(x => FilterSessions(x.Id, Date))
.Subscribe(x => Sessions = x);

How to get EF Framework to generate this group by query

I tried a few different ways and I am not impressed with the generated output query. It seems so inefficient. It needs to be efficient and shouldn't bring back all the rows into the app layer
select year(datetaken) as yr,
month(datetaken) as mth,
day(datetaken) as dy,
count(*) as totalpics
from photos
where photos.dateTaken <= #cutoffdate
group by year(datetaken), month(datetaken), day(datetaken)
order by yr asc, mth asc, dy asc
LINQ query:
var query = ctx.Photos.Where(p => p.DateTaken <= maxCutOffDate)
.GroupBy(p => new { p.DateTaken.Year, p.DateTaken.Month, p.DateTaken.Day })
.Select(grp => grp);
var results = query.ToList();
Select only the grouping key (grp.Key):
var query = ctx.Photos.Where(p => p.DateTaken <= maxCutOffDate)
.GroupBy(p => new { p.DateTaken.Year, p.DateTaken.Month, p.DateTaken.Day })
.Select(grp => grp.Key);
var results = query.ToList();
Edit
Or, including the count of totalpics:
var query = ctx.Photos.Where(p => p.DateTaken <= maxCutOffDate)
.GroupBy(p => new { p.DateTaken.Year, p.DateTaken.Month, p.DateTaken.Day })
.Select(grp => new
{
Date = grp.Key,
TotalPics = grp.Count()
});
var results = query.ToList();
You can access the key properties through grp.Key and the count through grp.Count().
var query = ctx.Photos.Where(p => p.DateTaken <= maxCutOffDate)
.GroupBy(p => new { p.DateTaken.Year, p.DateTaken.Month, p.DateTaken.Day })
.Select(grp => new
{
grp.Key.Year,
grp.Key.Month,
grp.Key.Day,
Count = grp.Count()
});
var results = query.ToList();

Resources