Speed up large Linq to Entities call MVC3 - asp.net-mvc-3

Below is the code I am using and the database table it is pulling from has about 92000 records in it. The way it is pulling right now it is pulling all 92000 records then doing the filtering.
What I am looking to do is the filtering on the initial pull from the DB so that it does not take aproximately 40 seconds to load the page.
This is something I am still new at so I am lost as to how to do this and make it work with my view
public ViewResult Makes()
{
var items = (from item in DBCacheHelper.recallslist
orderby item.MFGTXT ascending
select item.ToDomainRecall()).GroupBy(item => item.MFGTXT).Select(grp => grp.First());
return View(items);
}
public static IEnumerable<Recall> recallslist
{
get
{
if (c["GetAllRecalls"] == null)
{
c.Insert("GetAllRecalls", GetAllRecalls());
return (IEnumerable<Recall>)c["GetAllRecalls"];
}
else
{
return (IEnumerable<Recall>)c["GetAllRecalls"];
}
}
}
public static IEnumerable<Recall> GetAllRecalls()
{
using (DealerContext context = new DealerContext())
{
var items = from item in context.recalls.ToList<Recall>()
select item.ToDomainRecall();
return items.ToList<Recall>();
}
}
SELECT
[Extent1].[RecallsId] AS [RecallsId],
[Extent1].[RECORD_ID] AS [RECORD_ID],
[Extent1].[CAMPNO] AS [CAMPNO],
[Extent1].[MAKETXT] AS [MAKETXT],
[Extent1].[MODELTXT] AS [MODELTXT],
[Extent1].[YEARTXT] AS [YEARTXT],
[Extent1].[MFGCAMPNO] AS [MFGCAMPNO],
[Extent1].[COMPNAME] AS [COMPNAME],
[Extent1].[MFGNAME] AS [MFGNAME],
[Extent1].[BGMAN] AS [BGMAN],
[Extent1].[ENDMAN] AS [ENDMAN],
[Extent1].[RCLTYPECD] AS [RCLTYPECD],
[Extent1].[POTAFF] AS [POTAFF],
[Extent1].[ODATE] AS [ODATE],
[Extent1].[INFLUENCED_BY] AS [INFLUENCED_BY],
[Extent1].[MFGTXT] AS [MFGTXT],
[Extent1].[RCDATE] AS [RCDATE],
[Extent1].[DATEA] AS [DATEA],
[Extent1].[RPNO] AS [RPNO],
[Extent1].[FMVSS] AS [FMVSS],
[Extent1].[DESC_DEFECT] AS [DESC_DEFECT],
[Extent1].[CONEQUENCE_DEFECT] AS [CONEQUENCE_DEFECT],
[Extent1].[CORRECTIVE_ACTION] AS [CORRECTIVE_ACTION],
[Extent1].[NOTES] AS [NOTES],
[Extent1].[RCL_CMPT_ID] AS [RCL_CMPT_ID]
FROM [dbo].[Recalls] AS [Extent1]
Update:
Ultimately I would like to only pull records from the Recalls Table where the MFGTXT is equal to the
MakeName in the AutoMake Table
public class AutoMake
{
[Key]
public int MakeID { get; set; }
public string MakeName { get; set; }
public AutoMake ToDomainAutoMakes()
{
return new AutoMake
{
MakeID = this.MakeID,
MakeName = this.MakeName
};
}
}
public class Recall
{
[Key]
public int RecallsId { get; set; }
public string RECORD_ID { get; set; }
public string CAMPNO { get; set; }
public string MAKETXT { get; set; }
public string MODELTXT { get; set; }
public string YEARTXT { get; set; }
public string MFGCAMPNO { get; set; }
public string COMPNAME { get; set; }
public string MFGNAME { get; set; }
public string BGMAN { get; set; }
public string ENDMAN { get; set; }
public string RCLTYPECD { get; set; }
public string POTAFF { get; set; }
public string ODATE { get; set; }
public string INFLUENCED_BY { get; set; }
public string MFGTXT { get; set; }
public string RCDATE { get; set; }
public string DATEA { get; set; }
public string RPNO { get; set; }
public string FMVSS { get; set; }
public string DESC_DEFECT { get; set; }
public string CONEQUENCE_DEFECT { get; set; }
public string CORRECTIVE_ACTION { get; set; }
public string NOTES { get; set; }
public string RCL_CMPT_ID { get; set; }
public Recall ToDomainRecall()
{
return new Recall
{
RECORD_ID = this.RECORD_ID,
CAMPNO = this.CAMPNO,
MAKETXT = this.MAKETXT,
MODELTXT = this.MODELTXT,
YEARTXT = this.YEARTXT,
MFGCAMPNO = this.MFGCAMPNO,
COMPNAME = this.COMPNAME,
MFGNAME = this.MFGNAME,
BGMAN = this.BGMAN,
ENDMAN = this.ENDMAN,
RCLTYPECD = this.RCLTYPECD,
POTAFF = this.POTAFF,
ODATE = this.ODATE,
INFLUENCED_BY = this.INFLUENCED_BY,
MFGTXT = this.MFGTXT,
RCDATE = this.RCDATE,
DATEA = this.DATEA,
RPNO = this.RPNO,
FMVSS = this.FMVSS,
DESC_DEFECT = this.DESC_DEFECT,
CONEQUENCE_DEFECT = this.CONEQUENCE_DEFECT,
CORRECTIVE_ACTION = this.CORRECTIVE_ACTION,
NOTES = this.NOTES,
RCL_CMPT_ID = this.RCL_CMPT_ID
};
}
}

If you want to add server side filtering outside of your repository methods, you need to return your types as IQueryable rather than IEnumerable and not call .ToList, .AsEnumerable, or any other method that would cause .GetEnumerator to be called. Additionally, your cast `(IEnumerable)c["GetAllRecalls"];' forces LINQ to Objects to be used for subsequent requests rather than retaining the expression tree and using Entity Framework. That being said, you may need to move your call to ToDomainRecall method to after the additional filter is applied as well because that can't be translated to your database. Here are some of the changes you would need to make:
public ViewResult Makes()
{
var items = (from item in DBCacheHelper.recallslist
orderby item.MFGTXT ascending
select item.ToDomainRecall()).GroupBy(item => item.MFGTXT).Select(grp => grp.First());
return View(items);
}
public static IQueryable<Recall> recallslist
{
get
{
if (c["GetAllRecalls"] == null)
{
c.Insert("GetAllRecalls", GetAllRecalls(context));
}
return c["GetAllRecalls"];
}
}
public static IQueryable<Recall> GetAllRecalls(DealerContext context)
{
var items = context.recalls;
return items;
}

Looks like your DBaccess is done in the call to DBCacheHelper.recallslist.
You need to edit the sql that runs from/in this function.
As Eranga pointed out, you don't show how you are filtering the large number down to a smaller number of records. I assume you want 20 or 100 at a time? If so, please see the accepted answer here:
efficient way to implement paging
Specifically, this part which shows how to only retrieve rows x to y (where x = #p0 + 1 AND y = #p0 + #p1):
SELECT [t1].[CodCity],
[t1].[CodCountry],
[t1].[CodRegion],
[t1].[Name],
[t1].[Code]
FROM (
SELECT ROW_NUMBER() OVER (
ORDER BY [t0].[CodCity],
[t0].[CodCountry],
[t0].[CodRegion],
[t0].[Name],
[t0].[Code]) AS [ROW_NUMBER],
[t0].[CodCity],
[t0].[CodCountry],
[t0].[CodRegion],
[t0].[Name],
[t0].[Code]
FROM [dbo].[MtCity] AS [t0]
) AS [t1]
WHERE [t1].[ROW_NUMBER] BETWEEN #p0 + 1 AND #p0 + #p1
ORDER BY [t1].[ROW_NUMBER]

Related

InvalidCastException: Unable to cast object of type 'System.Collections.Generic.List to System.Collections.Generic.IEnumerable

I am trying to implementation clean architecture in netcore and I have Runtime Error
InvalidCastException: Unable to cast object of type 'System.Collections.Generic.List to System.Collections.Generic.IEnumerable
In the WebUI I have Match controller and ViewAllMatch Action like this
public async Task<IActionResult> ViewAllMatch()
{
var matches = await _mediator.Send(new GetMatchesDetail());
return View(matches);
}
In the application Layer I have A queries like this:
public class GetMatchesDetail : IRequest<IEnumerable<MatchesDetail>>
{
}
public class MatchesDetail
{
public string MatchId { get; set; }
public int MatchNumer { get; set; }
public DateTime DateMatch { get; set; }
public TimeSpan TimeMatch { get; set; }
public int MatchYear { get; set; }
public string SeasonId { get; set; }
public string Round { get; set; }
/// <summary>
/// Set value to Qualified for Qualified and Final for Final Round
/// </summary>
public string Stage { get; set; }
public string SubStage { get; set; }
public string HTeam { get; set; }
public string HTeamCode { get; set; } //For Flag get from Table Team from Foreign Key TeamName
public int HGoal { get; set; }
public int GGoal { get; set; }
public string GTeam { get; set; }
public string GTeamCode { get; set; }
public string WinNote { get; set; }
public string Stadium { get; set; }
public string Referee { get; set; }
public long Visistors { get; set; }
public string Status { get; set; }
}
public class GetMatchesHandler : IRequestHandler<GetMatchesDetail, IEnumerable<MatchesDetail>>
{
private readonly IMatchRepository _matchRepository;
public GetMatchesHandler(IMatchRepository matchRepository)
{
_matchRepository = matchRepository;
}
public async Task<IEnumerable<MatchesDetail>> Handle(GetMatchesDetail request, CancellationToken cancellationToken)
{
var matchlistview = await _matchRepository.GetMatchDetailAsync();
return matchlistview;
}
}
And the code for matchRepository to get all the match in Infastructure like this.
public async Task<IEnumerable<MatchesDetail>> GetMatchDetailAsync()
{
var matchDetailList = (from match in _context.Matches
join team1 in _context.Teams on match.HTeam equals team1.TeamName
join team2 in _context.Teams on match.GTeam equals team2.TeamName
join season in _context.Seasons on match.SeasonId equals season.SeasonId
select new
{
match.MatchId,
match.MatchNumber,
match.DateMatch,
match.TimeMatch,
match.MatchYear,
match.SeasonId,
season.SeasonName,
match.Round,
match.Stage,
match.SubStage,
match.HTeam,
HTeamCode = team1.TeamCode,
match.HGoal,
match.GGoal,
match.GTeam,
GTeamCode = team2.TeamCode,
match.WinNote,
match.Stadium,
match.Referee,
match.Visistors
});
return (IEnumerable<MatchesDetail>)await matchDetailList.ToListAsync();
}
Full code have been upload to Github at https://github.com/nguyentuananh921/Betting.git.
for more detail.
Thanks for your help.
I am so confuse about model in clean architech when i have more entities and the model I want to view in the WebUI contain many entities in domain.
Thanks for your help.
I have Modify public IEnumerable GetMatchDetailAsync() like that.
public IEnumerable<MatchesDetail> GetMatchDetailAsync()
{
#region TryOther way
var matchQuery = (from match in _context.Matches
join team1 in _context.Teams on match.HTeam equals team1.TeamName
join team2 in _context.Teams on match.GTeam equals team2.TeamName
join season in _context.Seasons on match.SeasonId equals season.SeasonId
select new
{
#region selectResult
//Remove to clear Select what I want to get.
#endregion
});
MatchesDetail matchesDetail = new MatchesDetail();
List<MatchesDetail> retList = new List<MatchesDetail>();
//IEnumerable<MatchesDetail> retList;
foreach (var item in matchQuery)
{
#region ManualMapping
matchesDetail.MatchId = item.MatchId;
//other field mapping
#endregion
retList.Add(matchesDetail);
}
#endregion
return retList;
}
And it work

Linq Query Help on outter join against a list in and object

I have the following object Job and Job sor are populated by reading in data from an XML file and Sor is populated from a database.
class Job
{
public int JobID { get; set; }
public string DepartmentCode { get; set; }
public string ClientReference { get; set; }
public string JobDescription { get; set; }
public List<JobSor> JobSorList { get; set; }
}
class JobSor
{
public int JobID { get; set; }
public string SorUserCode { get; set; }
public string SorNotes1 { get; set; }
public string SorNotes2 { get; set; }
}
class Sor
{
[Key]
public string code { get; set; }
public string description { get; set; }
public string contract { get; set; }
}
I want to write a linq query that will show me all the JobSors that do not exist in the Sor object.
This is what I have so far but I can’t reference the SorUserCode property?
var db = new dbContext();
var sor = db.Sors.Where(p => p.contract == "??");
var query =
from j in jobs
join p in sor on j.JobSorList.SorUserCode equals p.code into jp
from a in jp.DefaultIfEmpty()
select j;
How can I do this?
First get list of all JobSor from jobs list.
Then apply condition Where its SorUserCode value does not match with Any code value of the sor list.
Your query will be as below.
var query = jobs.SelectMany(x => x.JobSorList)
.Where(x => !sor.Any(y => y.code == x.SorUserCode));

I try to add entity model class data to my another list ,but after foreach Same rows insert in every row , MVC

This is my Entity model class which was auto generated by Ado.net model
public partial class SubModule
{
public int SubModuleId { get; set; }
public Nullable<int> ModuleId { get; set; }
public string SubModuleName { get; set; }
public Nullable<bool> Active { get; set; }
public Nullable<bool> IsModules { get; set; }
public string url { get; set; }
public string path { get; set; }
public string subform { get; set; }
}
this is my another class
public class ChildModules
{
public int ? SubModuleId { get; set; }
public Nullable<int> ModuleId { get; set; }
public string SubModuleName { get; set; }
public Nullable<bool> Active { get; set; }
public Nullable<bool> IsModules { get; set; }
public string url { get; set; }
public string path { get; set; }
public string subform { get; set; }
}
I want to copy Sub modules data to my Child modules class properties
My code is
List<SubModule> ChildModule = entity.SubModules.Where(x => x.IsModules == false).ToList();
List<ChildModules> listchildmodules = new List<ChildModules>();
ChildModules chmodule = new ChildModules();
foreach (SubModule mod in ChildModule)
{
chmodule.SubModuleId = mod.SubModuleId;
chmodule.ModuleId = mod.ModuleId;
chmodule.SubModuleName = mod.SubModuleName;
chmodule.Active = mod.Active;
chmodule.IsModules = mod.IsModules;
chmodule.url = mod.url;
chmodule.path = mod.path;
chmodule.subform = mod.subform;
listchildmodules.Add(chmodule);
}
but in listchildmodules last row insert in every index.
Why?
Your code always add the same object always. Because you always updating the values of same object and insert that into list.
Keep the below line of code inside foreach.
ChildModules chmodule = new ChildModules();
Your foreach should look like below
foreach (SubModule mod in ChildModule)
{
ChildModules chmodule = new ChildModules();
chmodule.SubModuleId = mod.SubModuleId;
chmodule.ModuleId = mod.ModuleId;
chmodule.SubModuleName = mod.SubModuleName;
chmodule.Active = mod.Active;
chmodule.IsModules = mod.IsModules;
chmodule.url = mod.url;
chmodule.path = mod.path;
chmodule.subform = mod.subform;
listchildmodules.Add(chmodule);
}
Or you could declare ChildModules chmodule; outside foreach and initialize chmodule = new ChildModules(); inside foreach loop.

Convert query expression to lambda in LINQPad4

While coding I had came across a LINQ query that I was able to accomplish in query syntax but not in lamda syntax. While this works fine in the application, I wanted to learn the query syntax for what I was trying to do.
Essentially, I have a database with views, CO_Leather_V and CO_LeatherSizeColor_V. I also have two classes, CuttingOrder and CuttingOrderDetail. CuttingOrderDetail contains entirely string,int and float properties. The CuttingOrder Class contains 2 string properties and a List of CuttingOrderDetails.
public class CuttingOrder
{
public string cuttingOrderNo { get; set; }
public string reserveSalesOrderNo { get; set; }
public List<CuttingOrderDetail> details { get; set; }
}
public class CuttingOrderDetail
{
public string cuttingOrderNo { get; set; }
public string reserveSalesOrderNo { get; set; }
public string itemCode { get; set; }
public string material { get; set; }
public string color { get; set; }
public string size { get; set; }
public int qty { get; set; }
public float squareFeet { get; set; }
public float squareFeetUsed { get; set; }
}
The query expression I used to get a list of all CuttingOrders with a given SalesOrder was
cos = (from l in db.CO_Leather_Vs
where l.orderNo == Globals.orderNo
select new Globals.CuttingOrder
{
cuttingOrderNo = "NOT SET",
reserveSalesOrderNo = "FAKE_SO_NO",
details = (
from d in db.CO_LeatherSizeColor_Vs
select new Globals.CuttingOrderDetail
{
cuttingOrderNo = d.orderNo
}
).ToList()
}).ToList();
I converted this to work in LINQPad with the following query, but I can't get anything to show on the lambda pane.
void Main()
{
var p = (from l in CO_Leather_V
select new CuttingOrder
{
cuttingOrderNo = "NOT SET",
reserveSalesOrderNo = "FAKE_SO_NO",
details = (
from d in CO_LeatherSizeColor_V
select new CuttingOrderDetail
{
cuttingOrderNo = d.OrderNo
}
).ToList()
}).ToList();
p.Dump();
}
// Define other methods and classes here
public class CuttingOrder
{
public string cuttingOrderNo { get; set; }
public string reserveSalesOrderNo { get; set; }
public List<CuttingOrderDetail> details { get; set; }
}
public class CuttingOrderDetail
{
public string cuttingOrderNo { get; set; }
public string reserveSalesOrderNo { get; set; }
public string itemCode { get; set; }
public string material { get; set; }
public string color { get; set; }
public string size { get; set; }
public int qty { get; set; }
public float squareFeet { get; set; }
public float squareFeetUsed { get; set; }
}
If anyone knows how to perform the linq query in lambda form or knows why LINQPad is unable to generate the lamda form it would be greatly appreciated.
This should work:
var p = CO_Leather_V.Select(l=> new CuttingOrder
{
cuttingOrderNo = "NOT SET",
reserveSalesOrderNo = "FAKE_SO_NO",
details = CO_LeatherSizeColor_V.Select(d=>new CuttingOrderDetail {cuttingOrderNo = d.OrderNo}).ToList()
}).ToList();
However, CO_LeatherSizeColor_V does not reference l, so you're going to get everything in that table, every time. You might want something like:
details = l.LeatherSizeColor.Select(d=>new CuttingOrderDetail {cuttingOrderNo = d.OrderNo}).ToList()
for that line instead.

Joining in LINQ to select a sublist within a list

I have two classes as follows:
public class HRM_SERVICE_PERD_BNFT_DTLModel
{
public string SRVC_PERD_BNFT_CODE { get; set; }
public string RTR_BNFT_CODE { get; set; }
public string RTR_BNFT_NAME { get; set; }
public string RTR_BNFT_CALC_MODE { get; set; }
public string SAL_HEAD_CODE { get; set; }
public string SAL_HEAD_NAME { get; set; }
public string RTR_BNFT_IN_PERCENT { get; set; }
public string RTR_BNFT_AMOUNT { get; set; }
public string RTR_BNFT_INACTV_DATE { get; set; }
public short? RTR_BNFT_SLNO { get; set; }
}
public class HRM_RETIREMENT_BENEFITModel : BaseModel
{
public string RTR_BNFT_CODE { get; set; }
public string RTR_BNFT_NAME { get; set; }
public string RTR_BNFT_SRTNM { get; set; }
public string RTR_BNFT_REM { get; set; }
public short? RTR_BNFT_SLNO { get; set; }
}
Now I want to select data from the first model for a specific 'SRVC_PERD_BNFT_CODE', then I need to select the 'RTR_BNFT_NAME' from the second table for all the filtered data along with some other value from the first model. I'm trying kind of joining, but not getting the name. It's showing system.collection.string instead of value.
Here is what I'm trying so far:
public List<HRM_SERVICE_PERD_BNFT_DTLModel> GetBenefitData(string mstcode)
{
var model2 = DataContext.HRM_SERVICE_PERD_BNFT_DTL.AsEnumerable().Where(m => m.SRVC_PERD_BNFT_CODE == mstcode).Select(s=>s).ToList();
var model = DataContext.HRM_SERVICE_PERD_BNFT_DTL.AsEnumerable().Where(m => m.SRVC_PERD_BNFT_CODE == mstcode)
.Select(s => new HRM_SERVICE_PERD_BNFT_DTLModel
{
RTR_BNFT_CODE = s.RTR_BNFT_CODE,
RTR_BNFT_SLNO = s.RTR_BNFT_SLNO,
RTR_BNFT_CALC_MODE = s.RTR_BNFT_CALC_MODE,
RTR_BNFT_AMOUNT = (s.RTR_BNFT_AMOUNT).ToString(),
RTR_BNFT_NAME = (from x in model2
join c in DataContext.HRM_RETIREMENT_BENEFIT on x.RTR_BNFT_CODE equals c.RTR_BNFT_CODE into p
from b in p.AsEnumerable()
select b.RTR_BNFT_NAME).ToList().ToString(),
RTR_BNFT_IN_PERCENT = (s.RTR_BNFT_IN_PERCENT).ToString()
}).ToList();
return model;
}
What I'm doing wrong? Please help.
Following is suspicious.
.ToList().ToString()
Remove the ToString part and replace it with something like FirstorDefault, then it should work.
Reason for getting something like System.Collection.String is that List<T>.ToString() is usually typename until it is overridden in some way.
Following minimal snipped produce System.Collections.Generic.List1[System.String]`
List<String> list = new List<String>();
var str = list.ToString();
Console.WriteLine(str); //System.Collections.Generic.List`1 [ System.String ]

Resources