My app consolidates data from other DBs for reporting purposes. We can't link the databases, so all the data processing has to be done in code - this is fine as we want to allow manual validation during the imports.
Certain users will be able to start an update through the Silverlight 4 front end.
I have 3 tables in database x that are fed from one EF4 Model (ModelX). I want to join those tables together, select specific columns and return the result as a new entity that exists in a different EF4 Model (ModelY). I'm using this query:
var myQuery = from i in DBx.table1 from it in DBx.table2 from h in DBx.table3 where (i.id==it.id && h.otherid == i.otherid) select new ModelYServer {Name = i.name,Thing = it.thing, Stuff = h.stuff};
The bit i'm stuck on, is how to execute that query, and wait until the Asynchronous call has completed. Normally, i'd use:
DomainContext.Load<T>(myQuery).Completed += (sender,args) =>
{List<T> myList = ((LoadOperation<T>)sender.Entities.ToList();};
but I can't pass myQuery (an IEnumerable) into the DomainContext.Load() as that expects an EntityQuery. The dataset is very large, and is taking up to 30 seconds to return, so I definitely need to wait before continuing.
So can anyone tell me how I can wait for the IEnumerable query to complete, or suggest a better way of doing this (there very likely is one).
Thanks
Mick
One simple way is just to force it to evaluate by calling ToList:
var query = from i in DBx.table1
join it in DBx.table2 on i.id equals it.id
join h in DBx.table3 on i.otherid equals h.otherid
select new ModelYServer {
Name = i.name,
Thing = it.thing,
Stuff = h.stuff
};
// This will block until the results have been fetched
var results = query.ToList();
// Now use results...
(I've changed your where clause into joins on the earlier tables, as that's what you were effectively doing and this is more idiomatic, IMO.)
Related
I am sort of new to this. Curious what happens in the following situation?
var q = //MY LINQ TO SQL QUERY.Select(...)
.........
.........
var c = q.Count();
.........
.........
var x = q.Where(....).Select(....);
var y = x.ToList();//or something such that forces materialization
var d = q.Count();//same as c
var e = x.Count();
var f = y.Count();
How many times did the sql statements make a trip to the db actually? Once at Count(). Again at Where()? Or Linq retains what it materialized during Count()?
Or it also depends on what the Where(..) has? Like if it is again referencing to database vs it just references what's obtained as part of 'q'/ or any other .net collections etc?
Edit:
Updated my code with a couple of other scenarios. Please correct my answers below:
q -no db trip
c -yes, but translates to aggregate qry - select Count(*) and not a result set (as per answer below)
x -no db trip. No matter what is written in the Where(..)
y - yes
d - yes - does not *reuse* c
e - yes - select count(*) EVEN THOUGH x already materized during y
f - no db trip
When you call Count it does not materialize the entire data set. Rather it prepares and executes a query like
SELECT COUNT(*) FROM ...
Using ExecuteScalar to get the result.
When you call Where and Select it does not materialize anything (assuming q is an IQueryable). Instead it's just preparing a query like
SELECT col1, col2, ... FROM ...
But it doesn't actually execute it at that point. It will only execute the query when you call GetEnumerator on q. You'll rarely do this directly, but anything like the following will cause your query to be executed:
var arry = q.ToArray();
var list = q.ToList();
foreach(var rec in q) ...
It will only execute this query once, so having multiple foreach loops will not create multiple database queries. Of course, if you create a new IQueryable based on q (e.g. var q2 = q.Where(...)) it will not be tied to the result set used by q, so it will have to query the database again.
I tested out your code in LINQPad, and it seems all your analyses are correct.
At my job our main application was written long ago before n-tier was really a thing, ergo - it has tons and tons of business logic begin handled in stored procs and such.
So we have finally decided to bite the bullet and make it not suck so bad. I have been tasked with converting a 900+ line sql script to a .NET exe, which I am doing in C#/Linq. Problem is...for the last 5-6 years at another job, I had been doing Linq exclusively, so my SQL has gotten somewhat rusty, and some of thing I am converting I have never tried to do before in Linq, so I'm hitting some roadblocks.
Anyway, enough whining.
I'm having trouble with the following sql statement, I think due to the fact that he is joining on a temp table and a derived table. Here's the SQL:
insert into #processedBatchesPurgeList
select d.pricebatchdetailid
from pricebatchheader h (nolock)
join pricebatchstatus pbs (nolock) on h.pricebatchstatusid = pbs.pricebatchstatusid
join pricebatchdetail d (nolock) on h.pricebatchheaderid = d.pricebatchheaderid
join
( -- Grab most recent REG.
select
item_key
,store_no
,pricebatchdetailid = max(pricebatchdetailid)
from pricebatchdetail _pbd (nolock)
join pricechgtype pct (nolock) on _pbd.pricechgtypeid = pct.pricechgtypeid
where
lower(rtrim(ltrim(pct.pricechgtypedesc))) = 'reg'
and expired = 0
group by item_key, store_no
) dreg
on d.item_key = dreg.item_key
and d.store_no = dreg.store_no
where
d.pricebatchdetailid < dreg.pricebatchdetailid -- Make sure PBD is not most recent REG.
and h.processeddate < #processedBatchesPurgeDateLimit
and lower(rtrim(ltrim(pbs.pricebatchstatusdesc))) = 'processed' -- Pushed/processed batches only.
So that's raising an overall question first: how to handle temp tables in Linq? This script uses about 10 of them. I currently have them as List. The problem is, if I try to .Join() on one in a query, I get the "Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator." error.
I was able to get the join to the derived table to work using 2 queries, just so a single one wouldn't get nightmarishly long:
var dreg = (from _pbd in db.PriceBatchDetails.Where(pbd => pbd.Expired == false && pbd.PriceChgType.PriceChgTypeDesc.ToLower().Trim() == "reg")
group _pbd by new { _pbd.Item_Key, _pbd.Store_No } into _pbds
select new
{
Item_Key = _pbds.Key.Item_Key,
Store_No = _pbds.Key.Store_No,
PriceBatchDetailID = _pbds.Max(pbdet => pbdet.PriceBatchDetailID)
});
var query = (from h in db.PriceBatchHeaders.Where(pbh => pbh.ProcessedDate < processedBatchesPurgeDateLimit)
join pbs in db.PriceBatchStatus on h.PriceBatchStatusID equals pbs.PriceBatchStatusID
join d in db.PriceBatchDetails on h.PriceBatchHeaderID equals d.PriceBatchHeaderID
join dr in dreg on new { d.Item_Key, d.Store_No } equals new { dr.Item_Key, dr.Store_No }
where d.PriceBatchDetailID < dr.PriceBatchDetailID
&& pbs.PriceBatchStatusDesc.ToLower().Trim() == "processed"
select d.PriceBatchDetailID);
So that query gives the expected results, which I am holding in a List, but then I need to join the results of that query to another one selected from the database, which is leading me back to the aforementioned "Local sequence cannot be used..." error.
That query is this:
insert into #pbhArchiveFullListSaved
select h.pricebatchheaderid
from pricebatchheader h (nolock)
join pricebatchdetail d (nolock)
on h.pricebatchheaderid = d.pricebatchheaderid
join #processedBatchesPurgeList dlist
on d.pricebatchdetailid = dlist.pricebatchdetailid -- PBH list is restricted to PBD purge list rows that have PBH references.
group by h.pricebatchheaderid
The join there on #processedBatchesPurgeList is the problem I am running into.
So uh...help? I have never written SQL like this, and certainly never tried to convert it to Linq.
As pointed out by the comments above, this is no longer being rewritten as Linq.
Was hoping to get a performance improvement along with achieving better SOX compliance, which was the whole reason for the rewrite in the first place.
I'm happy with just satisfying the SOX compliance issues.
Thanks, everyone.
I have a piece of code that I don't know how to improve it.
I have two entities: EntityP and EntityC.
EntityP is the parent of EntityC. It is 1 to many relationship.
EntityP has a property depending on a property of all its attached EntityC.
I need to load a list of EntityP with the property set correctly. So I wrote a piece of code to get the EntityP List first.It's called entityP_List. Then as I wrote below, I loop through the entityP_List and for each of them, I query the database with a "any" function which will eventually be translated to "NOT EXIST" sql query. The reason I use this is that I don't want to load all the attached entityC from database to memory, because I only need the aggregation value of their property. But the problem here is, the looping will query the databae many times, for each EntityP!
So I am wondering if anybody can help me improve the code to query the database only once to get all the EntityP.IsAll_C_Complete set, without load EntityC to memory.
foreach(EntityP p in entityP_List)
{
isAnyNotComoplete = entities.entityC.Any(c => c.IsComplete==false && c.parent.ID == p.ID);
p.IsAll_C_Complete = !isAnyNotComoplete;
}
Thank you very much!
In EF 4, you can do:
var ids = entityP_List.Select(p => p.ID);
var q = (from p in entities.entityP
where ids.Contains(p => p.ID)
select new
{
ID = p.ID,
IsAll_C_Complete = !p.entityCs.Any(c => !c.IsComplete)
}).ToList();
foreach (var p in entityP_List)
{
p.IsAll_C_Complete = q.Where(e.ID == p.Id).Single().IsAll_C_Complete;
}
...which will do the whole thing in one DB query. For EF 1, Google BuildContainsExpression for a replacement for the .Contains( part of the above.
I would base EntityP on a SQL View instead of a table. Then I would define the relationship, and aggregate the value for child table within the view.
I'm trying to get Entity Framework to select an object and filter its collection at the same time. I have a JobSeries object which has a collection of jobs, what I need to do is select a jobseries by ID and filter all the jobs by SendDate but I can't believe how difficult this simple query is!
This is the basic query which works:
var q = from c in KnowledgeStoreEntities.JobSeries
.Include("Jobs.Company")
.Include("Jobs.Status")
.Include("Category")
.Include("Category1")
where c.Id == jobSeriesId
select c;
Any help would be appreciated, I've been trying to find something in google and what I want to do is here:http://blogs.msdn.com/bethmassi/archive/2009/07/16/filtering-entity-framework-collections-in-master-detail-forms.aspx
It's in VB.NET though and I couldn't convert it to C#.
EDIT: I've tried this now and it doesn't work!:
var q = from c in KnowledgeStoreEntities.JobSeries
.Include("Jobs")
.Include("Jobs.Company")
.Include("Jobs.Status")
.Include("Category")
.Include("Category1")
where (c.Id == jobSeriesId & c.Jobs.Any(J => J.ArtworkId == "13"))
select c;
Thanks
Dan
Include can introduce performance problems. Lazy loading is guaranteed to introduce performance problems. Projection is cheap and easy:
var q = from c in KnowledgeStoreEntities.JobSeries
where c.Id == jobSeriesId
select new
{
SeriesName = c.Name,
Jobs = from j in c.Jobs
where j.SendDate == sendDate
select new
{
Name = j.Name
}
CategoryName = c.Category.Name
};
Obviously, I'm guessing at the names. But note:
Filtering works.
SQL is much simpler.
No untyped strings anywhere.
You always get the data you need, without having to specify it in two places (Include and elsewhere).
No bandwith penalties for retrieving columns you don't need.
Free performance boost in EF 4.
The key is to think in LINQ, rather than in SQL or in materializing entire entities for no good reason as you would with older ORMs.
I've long given up on .Include() and implemented Lazy loading for Entity Framework
I couldn't get last articles of every writers in this statement.
List<Editor> lstEditors = dataContext.GetTable<Editor>().Where(t => t.M_Active).Select(t => t).ToList();
var lstArticles = from article in DAO.context.GetTable<Article>().ToList()
join editor in lstEditors on article.RefEditorId equals editor.EditorId
select
new
{
article.M_ArticleId,
article.M_Subject,
article.M_Text,
editor.M_EditorId,
editor.M_Member.M_EditorPicture,
M_NameSurname = editor.M_Member.M_Fname + " " + editor.M_Member.M_Lname
};
Be careful, your query is fetching all the contents of both the Editor and the Yazi tables and then performs Linq-to-Objects on it.
I'm not sure what you ask exactly either, do you want to obtain the list of all writers (editors) along with the last article of each one of these writers?
Do you want to get the writers that did not write any articles yet also?
Edit:
explanation of methods causing an immediate query
Any time you call one of the methods listed below on an IQueryable object (tables or other queries), it performs the actual query to SQL server:
ToList(), ToArray(), ToLookup(), ToDictionay()
Count(), Sum(), Avg(), Aggregate(), Min(), Max()
First(), FirstOrDefault(), Last(), LastOrDefault()
getting last article written by each writer
//create a subquery that returns an editor and its last article date
var editorLastArticleDates =
from article in DAO.context.GetTable<Article>()
group article by article.RefEditor into g
let lastArticleDate= g.Max(x => x.Date)
select new
{
Editor = g.Key,
LastArticleDate = lastArticleDate,
};
//Note: We did not do a ToList() here so the query is not executed
// The editorLastArticleDates object is a IQueryable<>
var query =
from article in DAO.context.GetTable<Article>()
join editorLastArticleDate in editorLastArticleDates
on new { article.Editor, article.Date } // 1
equals new { editorLastArticleDate.Editor, // 2
Date = editorLastArticleDate.LastArticleDate } // 3
select new
{
article.M_ArticleId,
article.M_Subject,
article.M_Text,
article.RefEditor.M_EditorId,
article.RefEditor.M_Member.M_EditorPicture,
M_NameSurname = article.RefEditor.M_Member.M_Fname + " "
+ article.RefEditor.M_Member.M_Lname,
};
//Note: We did not do a ToList() yet so the query is not executed
// The query object is a IQueryable<>
Console.WriteLine(query.ToString()); //Displays SQL query on the console
var results = query.ToList(); // SQL query is executed on this line.
In the code above, I left some remarks on things I had problems with:
When using join, the section between new and equals access only variables declared before the join keyword while the section after the equals keyword has access to the variable defined between join and in.
When writing your join condition, make sure you use equals and not ==.
When using new { XXX, YYY } syntax in your join condition, you declare anonymous types. If the property names are not identical on both sides, it will not compile. In order to have identical property names in this sample, I added the Date = before my value.
By the way, you should use LinqPad to test your queries, it is really a nice tool.