Admittedly I don't perform lots of LINQ queries. Therefore I'm uncertain whether the problem I see is due to an obvious LINQ blunder or a legitimate Mongo driver problem (I use 10Gen 1.9.2 C# driver). In the below code I get an error indicating invalid where clause for .where(ques => unAnswered...). Code compiles fine but generates runtime error stating "unsupported where clause". Am I up against a driver limitation or is my LINQ bad?
public IEnumerable<QuestionDataModel> getUnanswered(String username, Category cat)
{
IQueryable<QuestionDataModel> questions =
from e in this.questionCollection.AsQueryable<QuestionDataModel>()
where (e.questionCategory == cat)
select e;
IQueryable<AnswerDataModel> answers =
from e in this.answerCollection.AsQueryable<AnswerDataModel>()
where (e.questionCategory == cat && e.username == username)
select e;
IEnumerable<QuestionDataModel> filteredquestionslist = null;
if (answers.Count()==0) // it's possible the user has not answered anything
filteredquestionslist = questions.ToList();
else
filteredquestionslist = questions.Where(ques => unAnswered(ques, ref answers)).ToList();
return filteredquestionslist;
}
private bool unAnswered(QuestionDataModel qdm, ref IQueryable<AnswerDataModel> answer_queryable)
{
bool retval;
retval = answer_queryable.Any(ans => ans.questionID == qdm.questionID) ? false:true;
return retval;
}
You can't combine two collections in a single query like this with MongoDB - there are no join operations in the database. (You also generally can't use your own method like that in LINQ since they don't translate into SQL (or any other database) but that's a separate issue and even if you fixed that it still wouldn't help here. unAnswered question cannot be translated into Mongo a query).
You must either iterate over one collection, performing the other query and yield return the results you want (i.e. the join happens not in the database but on the computer making the query), or you could denormalize the data in some way such that you can query a single collection to get the results. Of if the number of questions is really small you could possibly load them into a list using .ToList() and then operating on that list in memory.
Related
I am using LinqPad to test my query. This query works when the LInqPad connection is to my database (LInq to SQL) but it does not work when I change the connection to use my Entity Framework 5 Model.dll. (Linq to Entity). This is in C#.
I have two tables called Plan and PlanDetails. Relationship is one Plan to many PlanDetails.
var q = from pd in PlanDetails
select new {
pd.PlanDetailID,
ThePlanName = (from p in this.Plans
where p.PlanID == pd.PlanID
select p.PlanName)
};
var results = q.ToList();
q.Dump(); //This is a linqpad method to output the result.
I get this error "NotSupportedException: Unable to create a constant value of type 'Domain.Data.Plan'. Only primitive types or enumeration types are supported in this context." Any ideas why this only works with Linq to SQL?
basically it means you are using some complex datatype inside the query for comparison.
in your case i suspect from p in this.Plans where p.PlanID == pd.PlanID is the culprit.
And it depends on DataProvider. It might work for Sql Data Provider, but not for SqlCE data Provider and so on.
what you should do is to convert your this.Plans collection into a primitive type collection containing only the Ids i.e.
var integers = PlanDetails.Plans.Select(s=>s.Id).ToList();
and then use this list inside.
var q = from pd in PlanDetails
select new {
pd.PlanDetailID,
ThePlanName = (from p in integers
where p == pd.PlanID
select pd.PlanName)
};
I got this error when i was trying to null check for a navigational property in the entity framework expression
I resolved it by not using the not null check in the expression and just using Any() function only.
protected Expression<Func<Entities.Employee, bool>> BriefShouldAppearInSearchResults(
IQueryable<Entities.Employee> briefs, string username)
{
var trimmedUsername = NameHelper.GetFormattedName(username);
Expression<Func<Entities.Employee, bool>> filterExpression = cse =>
cse.Employee.Cars.All(c =>
c.Employee.Cars!=null && <--Removing this line resolved my issue
c.Employee.Cars.Any(cur => cur.CarMake =="Benz")));
return filterExpression;
}
Hope this helps someone!
This is a Linqpad bug if you like (or a peculiarity). I found similar behaviour myself. Like me, you may find that your query works with an ObjectContext, but not a DbContext. (And it works in Visual Studio).
I think it has to do with Linqpad's inner structure. It adds MergeAs (AppendOnly) to collections and the context is a UserQuery, which probably contains some code that causes this bug.
This is confirmed by the fact that the code does work when you create a new context instance in the Linqpad code and run the query against this instance.
If the relationship already exists.
Why not simply say.
var q = from pd in PlanDetails
select new {
pd.PlanDetailID,
ThePlanName = pd.Plan.PlanName
};
Of course i'm assuming that every PlanDetail will belong to a Plan.
Update
To get better results from LinqPad you could tell it to use your own assembly (which contains your DbContext) instead of the default Datacontext it uses.
I started using compiled queries to increase the performance of some commonly executed linq to entities queries. In one scenario I only boiled the query down to it's most basic form and pre-compiled that, then I tack on additional where clauses based on user input.
I seem to be losing the performance benefit of compiled queries in this particular case. Can someone explain why?
Here's an example of what I'm doing...
IEnumerable<Task> tasks = compiledQuery.Invoke(context, userId);
if(status != null)
{
tasks = tasks.Where(x=x.Status == status);
}
if(category != null)
{
tasks = tasks.Where(x=x.Category == category);
}
return tasks;
I think it's important to understand how Compiled Queries in EF work.
When you execute a query Entity Framework will map your expression tree with the help of your mapping file (EDMX or with code first your model definitions) to a SQL query. This can be a complex and performance intensive task.
Precompiling stores the results of these mapping phase so the next time you hit the query it has the SQL already available and it only has to set the current parameters.
The problem is that a precompiled query will lose it's performance benefit as soon as you modifie the query. Let's say you have the following:
IQueryable query = GetCompiledQuery(); // => db.Tasks.Where(t => t.Id == myId);
var notModifiedResult = query.ToList(); // Fast
int ModifiedResult = query.Count(); // Slow
With the first query you will have all the benefits of precompiling because EF has the SQL already generated for you and can execute this immediatly.
The second query will lose the precompiling because it has to regenerate it's SQL.
If you would now execute a query on notModifiedResult this will be a Linq To Objects one because you have already executed your SQL to the database and fetched all the elements in memory.
You can however chain Compiled Queries (that is, use a compiled query in another compiled query).
But your code would require a series of compiled queries:
- The default
- One where status != null
- One where category != null
- One where both status and category != null
(Note: I haven't done any EF work for ages, and then it was just pottering. This is just an informed guess, really.)
This could be the culprit:
IEnumerable<Task> tasks = compiledQuery.Invoke(context, userId);
Any further querying will have to be done within the .NET process, not in SQL. All the possible results will have to be fetched from the database and filtered locally. Try this instead:
IQueryable<Task> tasks = compiledQuery.Invoke(context, userId);
(Assuming that's valid, of course.)
The compiled query can't be changed, only the parameters can be changed. What you are doing here is actually running the query, and THEN filtering the results.
.Invoke(context, userId); // returns all the results
.Where(....) // filters on that entire collection
You can see if there is a clever way to restate your query, so that the parameters can be included in all cases, but not have any effect. I haven't worked with compiled queries, sorry about that, but does this work (using -1 as the "ignore" value)?
// bunch of code to define the compiled query part, copied from [msdn][1]
(ctx, total) => from order in ctx.SalesOrderHeaders
where (total == -1 || order.TotalDue >= total)
select order);
In SQL, you do this by either using dynamic sql, or having a default value (or null) that you pass in which indicates that parameter should be ignored
select * from table t
where
(#age = 0 or t.age = #age) and
(#weight is null or t.weight = #weight)
I have a non-trivial Linq To Sql query that I'm trying to break down into pieces for the sake of readability / further filtering / reuse.
The refactored code looks like this, where ids is the subquery performed to grab the ids.
var results = from solution in context.csExtendedQAIncident_Docs
join solutionText in context.csNTexts
on solution.chIdNo equals solutionText.chIdNo
where solutionText.chColumnId == "Solution"
//this is a very complicated subquery that returns a short list of the ids we need
&& (ids).Select(s => s.chIdNo)
//the TOP query portion - applied to just the ids
.Take(count ?? Settings.Current.WCFServices().Output.HomePage.MaxRows)
.Contains(solution.chIdNo)
select solution;
The 'ids' is an IOrderedQueryable<csExtendedQAIncident_Docs> which itself has a number of criteria and nested subqueries (i.e. Contains which gets translated into EXISTS style queries on SQL server)
In any event, the issue here is that when the full-blown subquery is included in the results query above, the query works without a hitch.
When the query is pulled out into it's own variable, the query dies at runtime with a NullReferenceException at SqlFactory.Member (partial stack trace below)
System.NullReferenceException: Object reference not set to an instance of an object.
at System.Data.Linq.SqlClient.SqlFactory.Member(SqlExpression expr, MemberInfo member)
at System.Data.Linq.SqlClient.QueryConverter.VisitMemberAccess(MemberExpression ma)
at System.Data.Linq.SqlClient.QueryConverter.VisitInner(Expression node)
at System.Data.Linq.SqlClient.QueryConverter.Visit(Expression node)
at System.Data.Linq.SqlClient.QueryConverter.VisitExpression(Expression exp)
at System.Data.Linq.SqlClient.QueryConverter.VisitMethodCall(MethodCallExpression mc)
at System.Data.Linq.SqlClient.QueryConverter.VisitInner(Expression node)
at System.Data.Linq.SqlClient.QueryConverter.Visit(Expression node)
at System.Data.Linq.SqlClient.QueryConverter.VisitExpression(Expression exp)
at
I suspect this is a bug in Linq to Sql's query evaluation process -- but was wondering if anyone else might have come across such an issue?
I stepped into the Framework source code -- but of course the trouble with that is that while I can hit the exact source line of the exception, the Linq assembly I'm using has been optimized and I can't inspect variables, etc.
Note: Keep in mind that this query, despite being a little long is actually not a great candidate in this scenario for a SPROC (especially since we're stuck on SQL 2000 here and can't parameterize TOP :( )
Edit: Found one other reference to a very similar issue out there that was never resolved http://www.eggheadcafe.com/software/aspnet/31934404/linq-combined-query.aspx
Edit2: Ask and ye shall receive ;0 I figured I would keep the id subquery out of here because I don't think its actually relevant (and as mentioned its complicated, etc). Please don't blame the crap db design on me -- but know that I worked on the SQL for this query for a bit to get acceptable performance, and then translated to Linq To Sql.
var ids = from solutionIds in context.csExtendedQAIncident_Docs
where solutionIds.iIncidentTypeId == 102094
&& solutionIds.tiRecordStatus == 1
&& solutionIds.iLanguage == 102074
&& null != solutionIds.chIdNo
&& (from openTo in context.csOpenTos
where onyxIdentity.GetDocumentAccessLevels().Union(new[] { "BUSG5" }).ToArray().Contains(openTo.vchOpenTo)
select openTo.chIdNo
).Distinct().Contains(solutionIds.chIdNo)
&& (from solutionProductAssocation in context.csProductDocs
where (from allowedProduct in context.KB_User_Allowed_Products
where allowedProduct.UserId == userId
select allowedProduct.ModelCode
).Contains(solutionProductAssocation.chModelCd)
select solutionProductAssocation.chIdNo).Distinct().Contains(solutionIds.chIdNo)
orderby solutionIds.dtUpdateDate descending
select solutionIds;
I'm trying to do a query like so...
query.Where(x => !string.IsNullOrEmpty(x.PropertyName));
but it fails...
so for now I have implemented the following, which works...
query.Where(x => (x.PropertyName ?? string.Empty) != string.Empty);
is there a better (more native?) way that LINQ handles this?
EDIT
apologize! didn't include the provider... This is using LINQ to SQL
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=367077
Problem Statement
It's possible to write LINQ to SQL that gets all rows that have either null or an empty string in a given field, but it's not possible to use string.IsNullOrEmpty to do it, even though many other string methods map to LINQ to SQL.
Proposed Solution
Allow string.IsNullOrEmpty in a LINQ to SQL where clause so that these two queries have the same result:
var fieldNullOrEmpty =
from item in db.SomeTable
where item.SomeField == null || item.SomeField.Equals(string.Empty)
select item;
var fieldNullOrEmpty2 =
from item in db.SomeTable
where string.IsNullOrEmpty(item.SomeField)
select item;
Other Reading:
1. DevArt
2. Dervalp.com
3. StackOverflow Post
This won't fail on Linq2Objects, but it will fail for Linq2SQL, so I am assuming that you are talking about the SQL provider or something similar.
The reason has to do with the way that the SQL provider handles your lambda expression. It doesn't take it as a function Func<P,T>, but an expression Expression<Func<P,T>>. It takes that expression tree and translates it so an actual SQL statement, which it sends off to the server.
The translator knows how to handle basic operators, but it doesn't know how to handle methods on objects. It doesn't know that IsNullOrEmpty(x) translates to return x == null || x == string.empty. That has to be done explicitly for the translation to SQL to take place.
This will work fine with Linq to Objects. However, some LINQ providers have difficulty running CLR methods as part of the query. This is expecially true of some database providers.
The problem is that the DB providers try to move and compile the LINQ query as a database query, to prevent pulling all of the objects across the wire. This is a good thing, but does occasionally restrict the flexibility in your predicates.
Unfortunately, without checking the provider documentation, it's difficult to always know exactly what will or will not be supported directly in the provider. It looks like your provider allows comparisons, but not the string check. I'd guess that, in your case, this is probably about as good of an approach as you can get. (It's really not that different from the IsNullOrEmpty check, other than creating the "string.Empty" instance for comparison, but that's minor.)
... 12 years ago :) But still, some one may found it helpful:
Often it is good to check white spaces too
query.Where(x => !string.IsNullOrWhiteSpace(x.PropertyName));
it will converted to sql as:
WHERE [x].[PropertyName] IS NOT NULL AND ((LTRIM(RTRIM([x].[PropertyName])) <> N'') OR [x].[PropertyName] IS NULL)
or other way:
query.Where(x => string.Compare(x.PropertyName," ") > 0);
will be converted to sql as:
WHERE [x].[PropertyName] > N' '
If you want to go change the type of the collection from nullable type IEnumerable<T?> to non-null type IEnumerable<T> you can use .OfType<T>().
.OfType<T>() will remove null values and return a list of the type T.
Example: If you have a list of nullable strings: List<string?> you can change the type of the list to string by using OfType<string() as in the below example:
List<string?> nullableStrings = new List<string?> { "test1", null, "test2" };
List<string> strings = nullableStrings.OfType<string>().ToList();
// strings now only contains { "test1", "test2" }
This will result in a list of strings only containing test1 and test2.
I've been developing a webapp using Linq to NHibernate for the past few months, but haven't profiled the SQL it generates until now. Using NH Profiler, it now seems that the following chunk of code hits the DB more than 3,000 times when the Linq expression is executed.
var activeCaseList = from c in UserRepository.GetCasesByProjectManagerID(consultantId)
where c.CompletionDate == null
select new { c.PropertyID, c.Reference, c.Property.Address, DaysOld = DateTime.Now.Subtract(c.CreationDate).Days, JobValue = String.Format("£{0:0,0}", c.JobValue), c.CurrentStatus };
Where the Repository method looks like:
public IEnumerable<Case> GetCasesByProjectManagerID(int projectManagerId)
{
return from c in Session.Linq<Case>()
where c.ProjectManagerID == projectManagerId
select c;
}
It appears to run the initial Repository query first, then iterates through all of the results checking to see if the CompletionDate is null, but issuing a query to get c.Property.Address first.
So if the initial query returns 2,000 records, even if only five of them have no CompletionDate, it still fires off an SQL query to bring back the address details for the 2,000 records.
The way I had imagined this would work, is that it would evaluate all of the WHERE and SELECT clauses and simply amalgamate them, so the inital query would be like:
SELECT ... WHERE ProjectManager = #p1 AND CompleteDate IS NOT NULL
Which would yield 5 records, and then it could fire the further 5 queries to obtain the addresses. Am I expecting too much here, or am I simply doing something wrong?
Anthony
Change the declaration of GetCasesByProjectManagerID:
public IQueryable<Case> GetCasesByProjectManagerID(int projectManagerId)
You can't compose queries with IEnumerable<T> - they're just sequences. IQueryable<T> is specifically designed for composition like this.
Since I can't add a comment yet. Jon Skeet is right you'll want to use IQueryable, this is allows the Linq provider to Lazily construct the SQL. IEnumerable is the eager version.