I have a query that has a where clause that checks if the data element is contained within a list.
This query executes fine:
results = awardedStats.Where(r => guidReq.Contains(r.RequirementId) || orgAcr.Contains(r.Division))
.Select(r => r);
however this does not:
results = awardedStats.Where(r => guidReq.Contains(r.RequirementId) || orgAcrId.Contains(r.guidDivisionId))
.Select(r => r);
r.division is a string and orgAcr is a List
r.guidDivisionId is a Guid and orgAcrId is a List
I know that each list get the correct values, I can check the list in the debugger, but if I run the first query, everything runs through fine, if I run the second query I get an error stating that the member "guidDivisionId" has no supported translation to SQL
If orgAcrId is a List<Guid> and r.guidDivisionId is a uniqueidentifier column in SQL Server, this should be fine. Are you sure the column name isn't r.DivisionId?
Get all data from sql and call AsEnumarable() method on it and then apply the where. That way comparison would be done in memory and it won't complain about sql translation.
Another thing is that when you use contains, it's converted to Sql IN clause. All elements in the list are included in the IN clause. If your list has more than 2100 elements, then you would get sql exception saying that sql cannot accept more than 2100 parameters. Doing this kind of comparison in memory is safer.
Related
I have a simple table (SQL server and EF6) Myvalues, with columns Id & Value (double)
I'm trying to get the sum of the natural log of all values in this table. My LINQ statement is:
var sum = db.Myvalues.Select(x => Math.Log(x.Value)).Sum();
It compiles fine, but I'm getting a RTE:
LINQ to Entities does not recognize the method 'Double Log(Double)' method, and this method cannot be translated into a store expression.
What am I doing wrong/how can I fix this?
FWIW, I can execute the following SQL query directly against the database which gives me the correct answer:
select exp(sum(LogCol)) from
(select log(Myvalues.Value) as LogCol From Myvalues
) results
LINQ tries to translate Math.Log into a SQL command so it is executed against the DB.
This is not supported.
The first solution (for SQL Server) is to use one of the existing SqlFunctions. More specifically, SqlFunctions.Log.
The other solution is to retrieve all your items from your DB using .ToList(), and execute Math.Log with LINQ to Objects (not LINQ to Entities).
As EF cannot translate Math.Log() you could get your data in memory and execute the function form your client:
var sum = db.Myvalues.ToList().Select(x => Math.Log(x.Value)).Sum();
I'm using NHibernate 3.2 and I have a repository method that looks like:
public IEnumerable<MyModel> GetActiveMyModel()
{
return from m in Session.Query<MyModel>()
where m.Active == true
select m;
}
Which works as expected. However, sometimes when I use this method I want to filter it further:
var models = MyRepository.GetActiveMyModel();
var filtered = from m in models
where m.ID < 100
select new { m.Name };
Which produces the same SQL as the first one and the second filter and select must be done after the fact. I thought the whole point in LINQ is that it formed an expression tree that was unravelled when it's needed and therefore the correct SQL for the job could be created, saving my database requests.
If not, it means all of my repository methods have to return exactly what is needed and I can't make use of LINQ further down the chain without taking a penalty.
Have I got this wrong?
Updated
In response to the comment below: I omitted the line where I iterate over the results, which causes the initial SQL to be run (WHERE Active = 1) and the second filter (ID < 100) is obviously done in .NET.
Also, If I replace the second chunk of code with
var models = MyRepository.GetActiveMyModel();
var filtered = from m in models
where m.Items.Count > 0
select new { m.Name };
It generates the initial SQL to retrieve the active records and then runs a separate SQL statement for each record to find out how many Items it has, rather than writing something like I'd expect:
SELECT Name
FROM MyModel m
WHERE Active = 1
AND (SELECT COUNT(*) FROM Items WHERE MyModelID = m.ID) > 0
You are returning IEnumerable<MyModel> from the method, which will cause in-memory evaluation from that point on, even if the underlying sequence is IQueryable<MyModel>.
If you want to allow code after GetActiveMyModel to add to the SQL query, return IQueryable<MyModel> instead.
You're running IEnumerable's extension method "Where" instead of IQueryable's. It will still evaluate lazily and give the same output, however it evaluates the IQueryable on entry and you're filtering the collection in memory instead of against the database.
When you later add an extra condition on another table (the count), it has to lazily fetch each and every one of the Items collections from the database since it has already evaluated the IQueryable before it knew about the condition.
(Yes, I would also like to be the extensive extension methods on IEnumerable to instead be virtual members, but, alas, they're not)
My understanding is that the use of scalar or conversion functions causes immediate execution of a LINQ query. It is also my understanding that subqueries are executed upon demand of the outer query which would typically be once per element. For the following example would I be right in saying that the inner query is executed immediately? If so, as this would produce a scalar value how would this affect how the outer query operates?
IEnumerable<string> outerQuery = names.Where ( n => n.Length == names
.OrderBy(n2 => n2.Length).Select(n2 => n2.Length).First());
I would expect the above query to operate in a similar way as below, ie as if there wasn't a subquery.
int val = names.OrderBy(n2 => n2.Length).Select(n2 => n2.Length).First();
IEnumerable<string> outerQuery = names.Where ( n => n.Length == val );
This example was taken from Joseph and Ben Albahari's C# 4.0 in a Nutshell (Chp 8 P331/332) and my confusion stems from the accompanying diagram which appears to show that the subquery is being evaluated each time the outer query iterates through the elements of names.
Could someone clarify how LINQ works in this setup? Any help would be appreciated!
For the following example would I be right in saying that the inner query is executed immediately?
No, the inner query will be executed for each item in names when the outer query is enumerated. If you want it to be executed only once, use the second code sample.
EDIT: as LukeH pointed out, this is only true of Linq to Objects. Other Linq providers (Linq to SQL, Entity Framework...) might be able to optimize this automatically
What is names? If it's collection (and you use LINQ to Objects) then "subquery" will be executed for each outer query item. If it's actually query object then result depends on actual IQueryable.Provider. For example, for LINQ to SQL you will give SQL query with scalar subquery. And in the most cases subquery actually will be executed only once.
I'm trying to do a query like so...
query.Where(x => !string.IsNullOrEmpty(x.PropertyName));
but it fails...
so for now I have implemented the following, which works...
query.Where(x => (x.PropertyName ?? string.Empty) != string.Empty);
is there a better (more native?) way that LINQ handles this?
EDIT
apologize! didn't include the provider... This is using LINQ to SQL
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=367077
Problem Statement
It's possible to write LINQ to SQL that gets all rows that have either null or an empty string in a given field, but it's not possible to use string.IsNullOrEmpty to do it, even though many other string methods map to LINQ to SQL.
Proposed Solution
Allow string.IsNullOrEmpty in a LINQ to SQL where clause so that these two queries have the same result:
var fieldNullOrEmpty =
from item in db.SomeTable
where item.SomeField == null || item.SomeField.Equals(string.Empty)
select item;
var fieldNullOrEmpty2 =
from item in db.SomeTable
where string.IsNullOrEmpty(item.SomeField)
select item;
Other Reading:
1. DevArt
2. Dervalp.com
3. StackOverflow Post
This won't fail on Linq2Objects, but it will fail for Linq2SQL, so I am assuming that you are talking about the SQL provider or something similar.
The reason has to do with the way that the SQL provider handles your lambda expression. It doesn't take it as a function Func<P,T>, but an expression Expression<Func<P,T>>. It takes that expression tree and translates it so an actual SQL statement, which it sends off to the server.
The translator knows how to handle basic operators, but it doesn't know how to handle methods on objects. It doesn't know that IsNullOrEmpty(x) translates to return x == null || x == string.empty. That has to be done explicitly for the translation to SQL to take place.
This will work fine with Linq to Objects. However, some LINQ providers have difficulty running CLR methods as part of the query. This is expecially true of some database providers.
The problem is that the DB providers try to move and compile the LINQ query as a database query, to prevent pulling all of the objects across the wire. This is a good thing, but does occasionally restrict the flexibility in your predicates.
Unfortunately, without checking the provider documentation, it's difficult to always know exactly what will or will not be supported directly in the provider. It looks like your provider allows comparisons, but not the string check. I'd guess that, in your case, this is probably about as good of an approach as you can get. (It's really not that different from the IsNullOrEmpty check, other than creating the "string.Empty" instance for comparison, but that's minor.)
... 12 years ago :) But still, some one may found it helpful:
Often it is good to check white spaces too
query.Where(x => !string.IsNullOrWhiteSpace(x.PropertyName));
it will converted to sql as:
WHERE [x].[PropertyName] IS NOT NULL AND ((LTRIM(RTRIM([x].[PropertyName])) <> N'') OR [x].[PropertyName] IS NULL)
or other way:
query.Where(x => string.Compare(x.PropertyName," ") > 0);
will be converted to sql as:
WHERE [x].[PropertyName] > N' '
If you want to go change the type of the collection from nullable type IEnumerable<T?> to non-null type IEnumerable<T> you can use .OfType<T>().
.OfType<T>() will remove null values and return a list of the type T.
Example: If you have a list of nullable strings: List<string?> you can change the type of the list to string by using OfType<string() as in the below example:
List<string?> nullableStrings = new List<string?> { "test1", null, "test2" };
List<string> strings = nullableStrings.OfType<string>().ToList();
// strings now only contains { "test1", "test2" }
This will result in a list of strings only containing test1 and test2.
Why do I get the error:
Unable to create a constant value of type 'Closure type'. Only
primitive types (for instance Int32, String and Guid) are supported in
this context.
When I try to enumerate the following Linq query?
IEnumerable<string> searchList = GetSearchList();
using (HREntities entities = new HREntities())
{
var myList = from person in entities.vSearchPeople
where upperSearchList.All( (person.FirstName + person.LastName) .Contains).ToList();
}
Update:
If I try the following just to try to isolate the problem, I get the same error:
where upperSearchList.All(arg => arg == arg)
So it looks like the problem is with the All method, right? Any suggestions?
It looks like you're trying to do the equivalent of a "WHERE...IN" condition. Check out How to write 'WHERE IN' style queries using LINQ to Entities for an example of how to do that type of query with LINQ to Entities.
Also, I think the error message is particularly unhelpful in this case because .Contains is not followed by parentheses, which causes the compiler to recognize the whole predicate as a lambda expression.
I've spent the last 6 months battling this limitation with EF 3.5 and while I'm not the smartest person in the world, I'm pretty sure I have something useful to offer on this topic.
The SQL generated by growing a 50 mile high tree of "OR style" expressions will result in a poor query execution plan. I'm dealing with a few million rows and the impact is substantial.
There is a little hack I found to do a SQL 'in' that helps if you are just looking for a bunch of entities by id:
private IEnumerable<Entity1> getByIds(IEnumerable<int> ids)
{
string idList = string.Join(",", ids.ToList().ConvertAll<string>(id => id.ToString()).ToArray());
return dbContext.Entity1.Where("it.pkIDColumn IN {" + idList + "}");
}
where pkIDColumn is your primary key id column name of your Entity1 table.
BUT KEEP READING!
This is fine, but it requires that I already have the ids of what I need to find. Sometimes I just want my expressions to reach into other relations and what I do have is criteria for those connected relations.
If I had more time I would try to represent this visually, but I don't so just study this sentence a moment: Consider a schema with a Person, GovernmentId, and GovernmentIdType tables. Andrew Tappert (Person) has two id cards (GovernmentId), one from Oregon (GovernmentIdType) and one from Washington (GovernmentIdType).
Now generate an edmx from it.
Now imagine you want to find all the people having a certain ID value, say 1234567.
This can be accomplished with a single database hit with this:
dbContext context = new dbContext();
string idValue = "1234567";
Expression<Func<Person,bool>> expr =
person => person.GovernmentID.Any(gid => gid.gi_value.Contains(idValue));
IEnumerable<Person> people = context.Person.AsQueryable().Where(expr);
Do you see the subquery here? The generated sql will use 'joins' instead of sub-queries, but the effect is the same. These days SQL server optimizes subqueries into joins under the covers anyway, but anyway...
The key to this working is the .Any inside the expression.
I have found the cause of the error (I am using Framework 4.5). The problem is, that EF a complex type, that is passed in the "Contains"-parameter, can not translate into an SQL query. EF can use in a SQL query only simple types such as int, string...
this.GetAll().Where(p => !assignedFunctions.Contains(p))
GetAll provides a list of objects with a complex type (for example: "Function"). So therefore, I would try here to receive an instance of this complex type in my SQL query, which naturally can not work!
If I can extract from my list, parameters which are suited to my search, I can use:
var idList = assignedFunctions.Select(f => f.FunctionId);
this.GetAll().Where(p => !idList.Contains(p.FunktionId))
Now EF no longer has the complex type "Function" to work, but eg with a simple type (long). And that works fine!
I got this error message when my array object used in the .All function is null
After I initialized the array object, (upperSearchList in your case), the error is gone
The error message was misleading in this case
where upperSearchList.All(arg => person.someproperty.StartsWith(arg)))