Linq to SQL not querying memory - linq

I am using Linq for SQL and have always thought that I was querying the results of a Query in memory. I have just looked at the database and it is showing many thousands of queries rather than 1 which is what I expected.
My approach has been to run a query and then use linq to search within the resultset.
IQueryable<mapping> fieldList = from mm in db.mappings
join mi in db.metaItems on mm.secondaryMetaItemId equals mi.metaItemId
join mo in db.metaObjects on mi.metaObjectId equals mo.metaObjectId
where mm.linkageId == 277
select mm;
for (int i=0;i<100;i++)
{
mapping thisVar = fieldList.FirstOrDefault(m => m.primaryItem == info.Name);
}
How can I stop Linq requerying everytime I access my resultset...
Thanks for your help!

When you write a LINQ query, the query doesn't get actually get executed until you perform an action that actually enumerates over it (deferred execution). Calling FirstOrDefault() is an example of one such method that enumerates over the result (the first result has to be found). You'll want to call a method or otherwise enumerate over the results once. That way when you want to refer to those results throughout your program, you do it on a stored copy.
The easiest way you can do that is to convert it to a list. That will put the results in memory as a list. You could then use that.
IQueryable<mapping> fieldList =
from mm in db.mappings
join mi in db.metaItems on mm.secondaryMetaItemId equals mi.metaItemId
join mo in db.metaObjects on mi.metaObjectId equals mo.metaObjectId
where mm.linkageId == 277
select mm;
// save it!
var result = fieldList.ToList(); // query is processed only once here
// do stuff with result
for (int i=0;i<100;i++)
{
// using the stored result
thisVar = result.FirstOrDefault(m => m.primaryItem == info.Name);
}

try this :
var fieldList = (from mm in db.mappings
join mi in db.metaItems on mm.secondaryMetaItemId equals mi.metaItemId
join mo in db.metaObjects on mi.metaObjectId equals mo.metaObjectId
where mm.linkageId == 277
select mm).AsEnumerable();
foreach (int i=0;i<100;i++)
{
mapping thisVar = fieldList.FirstOrDefault(m => m.primaryItem == info.Name);
}

Related

Building LINQ Expression Getting Ignored

I'm trying to build a linq query coming from a table grid from the client side, so im expecting page offset, page start, order and the traditional paging parameters. I have the following code:
[Route("api/settings/logs")]
public Rest.DatatablesResponse GetLogs(int draw, int start, int length) {
var query_string = Request.GetQueryNameValuePairs().ToDictionary(x => x.Key, x => x.Value);
var search = query_string["search.value"];
int order_column = int.Parse(query_string["order[0].column"]);
var order_direction = query_string["order[0].dir"];
var count = db.Logs.Count(q => q.Mode == 2);
var logs = (from l in db.Logs
where l.Mode == 2
select new {
id = l.ID,
mode = l.Mode,
phase_id = l.Phase.ID,
created = l.Created,
user = l.User.Name,
blender_name = l.Blender.Name,
oil_name = l.Oil,
oil_quantity = l.OilQuantity,
production_cycle_name = l.ProductionCycle.Name
});
if (order_direction == "asc") {
if (order_column == 0) logs.OrderBy(q => q.created);
else if (order_column == 2) logs.OrderBy(q => q.production_cycle_name);
} else {
if (order_column == 0) logs.OrderByDescending(q => q.created);
else if (order_column == 2) logs.OrderByDescending(q => q.production_cycle_name);
};
if (!string.IsNullOrEmpty(search)) {
logs.Where(q => q.blender_name.Contains(search) ||
q.oil_name.Contains(search) ||
SqlFunctions.StringConvert((decimal)q.id).Contains(search));
}
logs.Skip(start).Take(length);
DateTime dtDateTime = new DateTime(1970,1,1,0,0,0,0,System.DateTimeKind.Utc);
var steps = from l in logs.ToList()
select new {
id = l.id,
message = StringHelpers.FormatWith(_tpl_message[l.phase_id.ToString() + l.mode.ToString() ], l) ,
created = dtDateTime.AddSeconds(l.created).ToString("h:mmtt - MMMM d, yyyy"),
production_cycle_name = l.production_cycle_name
};
return new Rest.DatatablesResponse {
draw = draw,
recordsTotal = count,
recordsFiltered = count,
data = steps.ToArray()
};
}
My problem is the skip and take and orderby expressions are getting ignored for some reason, and this is the SQL code generated just before converting my linq expressions to a list. From my understanding, the query should not be executed or evaluated until my logs.ToList() call, so the ordering and take/skip should be taken into account, but it's not:
{SELECT
[Extent1].[ID] AS [ID],
[Extent1].[Mode] AS [Mode],
[Extent1].[Phase_ID] AS [Phase_ID],
[Extent1].[Created] AS [Created],
[Extent2].[Name] AS [Name],
[Extent3].[Name] AS [Name1],
[Extent1].[Oil] AS [Oil],
[Extent1].[OilQuantity] AS [OilQuantity],
[Extent4].[Name] AS [Name2]
FROM [dbo].[Steps] AS [Extent1]
LEFT OUTER JOIN [dbo].[Users] AS [Extent2] ON [Extent1].[User_Id] = [Extent2].[Id]
LEFT OUTER JOIN [dbo].[Blenders] AS [Extent3] ON [Extent1].[Blender_ID] = [Extent3].[ID]
LEFT OUTER JOIN [dbo].[ProductionCycles] AS [Extent4] ON [Extent1].[ProductionCycle_ID] = [Extent4].[ID]
WHERE 2 = [Extent1].[Mode]}
Irrelevant P.S. I'm using the not so clever ifs for building the order expression instead of using DynamicLINQ since i have only two sortable columns.
logs.Skip(start).Take(length);
Creates a IQueryable<T> where T is the same anonymous type of which logs is an IQueryable<T> but with start items skipped. Then from that it creates a similarly typed IQueryable<T> where lenght items are the most that will be taken.
Then it throws that away and lets it be garbage collected. (Or ideally the compiler or jitter steps will realise it's thrown away and cut out the whole thing).
Then logs.ToList() goes back to the logs you still have and creates a list from it.
You should replace the Skip and Take line with:
logs = logs.Skip(start).Take(length);
So that you are actually making use of this skipping and taking.
I'm using the not so clever ifs for building the order expression instead of using DynamicLINQ since i have only two sortable columns.
There's nothing particularly not-clever about that, except that you make the same mistake; apply the OrderBy and then throwing away the result instead of using it. Likewise with the Where. You need logs = logs.OrderBy(...) etc.
I'd also question from l in logs.ToList() select new {…} here.
It might be the best approach, if obtaining that list in one step has some advantage. However otherwise:
from l in logs select new {…}
Do the select work on the database, retrieving just what you need.
from l in logs.AsEnumerable() select new {…}
Do the select work in the application, appropriate if part of it cannot be converted to database work, but do it as it comes rather than loading it all into memory first.
from l in await logs.ToListAsync() select new {…}
Has the downside of ToList() but in asynchronous uses, then (assuming your provider has a ToListAsync() method) allows for awaiting.
ToList() is rarely the best option here.

Linq query select count into same entity

I got two tables: comments and commentLikes
in the same query i count the likes users have given on a comment.
I got the following (simplified) query:
var res = (from c in db.Comments
where c.Topic.ID == topicID
select new
{
comment = c,
count = c.CommentLikes.Count()
}).ToList();
But, rather than mapping the likecount into the comment entity again, I'd like to get a list of Comments only with a field LikeCount in it, preferably with an efficient query. Something like this:
var res = (from c in db.Comments
where c.Topic.ID == topicID
select new
{
comment = c,
c.LikeCount = c.CommentLikes.Count()
}).ToList();
This query doesn't compile.
How to do this in linq?
You can't do that. EF does not support to project (= select) data into an entity. You must fill the LikeCount property in memory after the query has been executed. You can write it in a compact way, but it's basically just a foreach loop over the materialized anonymous objects:
IEnumerable<Comment> res =
(from c in db.Comments
where c.Topic.ID == topicID
select new
{
comment = c,
count = c.CommentLikes.Count()
})
.ToList() // DB query runs here, the rest in memory
.Select(a => {
a.comment.LikeCount = a.count;
return a.comment;
});

LINQ to Entities three table join query

I'm having a bit trouble with a query in Linq to Entities which I hope someone can shed a light on :-) What I'm trying to do is to create a query that joins three tables.
So far it works, but since the last table I'm trying to join is empty, the result of the query doesn't contain any records. When I remove the last join, it gives me the right results.
My query looks like this:
var query = from p in db.QuizParticipants
join points in db.ParticipantPoints on p.id
equals points.participantId into participantsGroup
from po in participantsGroup
join winners in db.Winners on p.id
equals winners.participantId into winnersGroup
from w in winnersGroup
where p.hasAttended == 1 && p.weeknumber == weeknumber
select new
{
ParticipantId = p.id,
HasAttended = p.hasAttended,
Weeknumber = p.weeknumber,
UmbracoMemberId = p.umbMemberId,
Points = po.points,
HasWonFirstPrize = w.hasWonFirstPrize,
HasWonVoucher = w.hasWonVoucher
};
What I would like is to get some records even if the Winners table is empty or there is no match in it.
Any help/hint on this is greatly appreciated! :-)
Thanks a lot in advance.
/ Bo
If you set these up as related entities instead of doing joins, I think it will be easier to do what you're trying to do.
var query = from p in db.QuizParticipants
where p.hasAttended == 1 && p.weeknumber == weeknumber
select new
{
ParticipantId = p.id,
HasAttended = p.hasAttended,
Weeknumber = p.weeknumber,
UmbracoMemberId = p.umbMemberId,
Points = p.ParticipantPoints.Sum(pts => pts.points),
HasWonFirstPrize = p.Winners.Any(w => w.hasWonFirstPrize),
HasWonVoucher = p.Winners.Any(w => w.hasWonVoucher)
};
This is assuming hasWonFirstPrize and hasWonVoucher are boolean fields, but you can use any aggregate function to get the results you need, such as p.Winners.Any(w => w.hasWonFirstPrize == 1)
I don't use query syntax a lot but I believe you need to change from w in winnersGroup to from w in winnersGroup.DefaultIfEmpty()

complex orderby that links to another table

I have the following query to start with:
var query = from p in db.Products
from pc in p.NpProductCategories
where pc.CategoryId == categoryId
select p;
I'm applying some more filtering on it and in the end I want to sort the results:
if (orderBy == ProductSortingEnum.Name)
query = query.OrderBy(x => x.Name);
else
query = query.OrderBy(............);
My big problem (coming from not knowing linq too good) is the ELSE here. How can I sort results by a column that is not in the current result set? I would like to somehow link to another linq query in the orderby. The sorting I'm trying to achive is to link to NpProductVariants query using the ProductId to match between NpProductVariant and Products
and sort by the Price of the NpProductVariant
Assuming you have the relationship set up in the dbml...
For one to one (and many to one):
query = query.OrderBy(p => p.NpProductVariant.Price);
For one to many:
query = query.OrderBy(p => p.NpProductVariants.Select(v => v.Price).Max());
Also:
var query =
from p in db.Products
where p.NpProductCategories.Any(pc => pc.CategoryId == categoryId)
select p;
I think you can hook your Join to your query as long as it is returning the same thing. So maybe something like (I'm not 100 % sure since I haven't tried it):
query = from i1 in query
join i2 in query2 on i1.PropertyToJoin equals i2.PropertyToJoin
orderby i1.OrderProp1, i2.OrderProp2
select i1;
But I think it might be a good idea to check the generated sql so it is still effective.

How to do a simple Count in Linq?

I wanted to do a paging style table, but NeerDinner example fetches the entire data into a PaggingList type, and I have more than 10 000 rows to be fetched, so I skipped that part.
so I come up with this query
var r = (from p in db.Prizes
join c in db.Calendars on p.calendar_id equals c.calendar_id
join ch in db.Challenges on c.calendar_id equals ch.calendar_id
join ca in db.ChallengeAnswers on ch.challenge_id equals ca.challenge_id
join cr in db.ChallengeResponses on ca.challenge_answer_id equals cr.challenge_answer_id
where
p.prize_id.Equals(prizeId)
&& ch.day >= p.from_day && ch.day <= p.to_day
&& ca.correct.Equals(true)
&& ch.day.Equals(day)
orderby cr.Subscribers.name
select new PossibleWinner()
{
Name = cr.Subscribers.name,
Email = cr.Subscribers.email,
SubscriberId = cr.subscriber_id,
ChallengeDay = ch.day,
Question = ch.question,
Answer = ca.answer
})
.Skip(size * page)
.Take(size);
Problem is, how can I get the total number of results before the Take part?
I was thinking of:
var t = (from p in db.JK_Prizes
join c in db.JK_Calendars on p.calendar_id equals c.calendar_id
join ch in db.JK_Challenges on c.calendar_id equals ch.calendar_id
join ca in db.JK_ChallengeAnswers on ch.challenge_id equals ca.challenge_id
join cr in db.JK_ChallengeResponses on ca.challenge_answer_id equals cr.challenge_answer_id
where
p.prize_id.Equals(prizeId)
&& ch.day >= p.from_day && ch.day <= p.to_day
&& ca.correct.Equals(true)
&& ch.day.Equals(day)
select cr.subscriber_id)
.Count();
but that will do the query all over again...
anyone has suggestions on how can I do this effectively ?
If you take a query as such:
var qry = (from x in y
select x).Count();
...LINQ to SQL will be clever enough to make this a SELECT COUNT query, which is potentially rather efficient (efficiency will depend more on the conditions in the query). Bottom line is that the count operation happens in the database, not in LINQ code.
Writing my old comments :Well i was facing the same issue some time back and then i came up with LINQ to SP =). Make an SP and drop that into your entities and use it.you can get write Sp according to your need like pulling total record column too. It is more easy and fast as compare to that whet you are using wright now.
You can put count for query logic as well as, see the sample as below:
public int GetTotalCountForAllEmployeesByReportsTo(int? reportsTo, string orderBy = default(string), int startRowIndex = default(int), int maximumRows = default(int))
{
//Validate Input
if (reportsTo.IsEmpty())
return GetTotalCountForAllEmployees(orderBy, startRowIndex, maximumRows);
return _DatabaseContext.Employees.Count(employee => reportsTo == null ? employee.ReportsTo == null : employee.ReportsTo == reportsTo);
}

Resources