How to bind a value to a variable in a programmatically generated Linq Expression - linq

I am trying to programmatically generate an expression tree (that will eventually get used in Linq-to-entity framework).
I can get the query to work just fine with one exception - it does not parametrize the query - which I want for Sql Server query plan reuse.
I read that in order for the generated sql to be parametrized, the expression needs to compare based on a variable. However, I cannot figure out how to assign the value to the variable in the expression tree. If I just use Expression.Constantit works (but is not parametrized).
So basically:
public Expression<Func<T, bool>> FooEquals<T>(
Expression<Func<T, int>> propertyExpression, int value)
{
ParameterExpression param = propertyExpression.Parameters.Single();
int targetValueForEqualityComparison = 9;
//There's some "special sauce" here which is why I don't
//use propertyExpression directly
var body = Expression.Property(param, GetPropertyName(propertyExpression));
//If I just use Expression.Constant, it works, but doesn't parametrize.
//var equalExpression = ParameterExpression.Equal(body,
// Expression.Constant(targetValueForEqualityComparison, typeof(int)));
var variable = Expression
.Variable(typeof(int), "targetValueForEqualityComparison");
var assigned = Expression.Assign(variable,
Expression.Constant(targetValueForEqualityComparison, typeof(int)));
//throws InvalidOperaitonException: "The parameter was not bound in the
//specified Linq to Entities query expression
var equalExpression = ParameterExpression.Equal(body, variable);
//throws NotSupportedException: "Unknown LINQ expression of type 'Assign'.
var equalExpression = ParameterExpression.Equal(body, assigned);
return Expression.Lambda<Func<T, bool>>(equalExpression, param);
}
How do I properly bind a value to the variable expression so that Linq-to-EntityFramework will parametrize the query?

I went ahead and tried it out because I was curious. The following seem to result in identical SQL, at least when used with Linq-to-SQL (which LINQPad does more readily than EF). I'd imagine it should work just the same with EF though.
Seems like a pretty convoluted way to just pass an integer in, but because this is what the compiler generates for a plain lambda function, I guess this is what the SQL generator looks for.
// Given this class, which is equivalent to the compiler-generated class
class Holder {
public int AnInteger;
}
int id = 1;
// You get the same SQL with a plain lambda function
var query = db.Items.Where(i => i.Id == id);
// or with a handwritten expression:
var arg = Expression.Parameter(typeof(Item), "i");
var paramHolder = new Holder { AnInteger = id };
// essentially, (i) => i.Id == paramHolder.AnInteger
var lambda = Expression.Lambda<Func<Item, bool>>(
Expression.Equal(
Expression.Property(arg, "Id"),
Expression.Field(
Expression.Constant(paramHolder), "AnInteger")),
arg);
// the SQL this translates to is equivalent to that of the first query
var query2 = db.Items.Where(lambda);

Related

Group by Dynamic Aggregate

In the code below I want to replace x.BaseSalary with any other property whose name is stored in feildtoretrive:
var feildtoretrive = "BaseSalary"
var groupcal = db.Employees.GroupBy(x=>x.Region)
.Select(group => new {
Key = group.Key,
Avg = group.Average(x => x.BaseSalary)
})
.ToList();
You need to
create a MemberExpression that accesses that member of an instance
compile a lambda expression that returns that member for a passed instance parameter
(I assume that the elements in Employees are of type Employee)
// the parameter expression for the lambda (your 'x')
var parameter = Expression.Parameter(typeof(Employee));
// the property access expression (your 'x.BaseSalary' or 'x.<feildtoretrive')
var propAccess = Expression.PropertyOrField(parameter, feildtoretrive);
// the build-together and compiled lambda
var expression = (Expression<Func<Employee, int?>>)Expression.Lambda(propAccess, parameter);
You can now use lambda for the x.Average call:
new { Key = group.Key, Avg = group.Average(lambda) }
A caveat: This only works now for members of type int?. I'm lacking a little experience on how to do this more type independent, but what types can you calculate an average over? But if there are int or double members, another cast expression maybe necessary.
EDIT: (changed the return type to int?). According to Ivan Stoev's comment on my follow up question you could try this:
new { Key = group.Key, Avg = group.AsQueryable().Average(expression) }
EF6 should recognize the call to AsQueryable() and then use the correct Average method (note that I use the expression as argument instead of the lambda). EF Core and linq2Sql won't work with that.

How to create programmatically single LINQ query w/ OR between Where() clauses (.Where(fn) OR .Where(fn) OR .Where(fn)) programatically

I'd like to know it is it possible to create programmatically single LINQ query (for EntityFramework 6) with N .Where() clauses, but with OR between these .Where() clauses.
Imagine IQueryable object defined like:
var query = dbContext.MyTable.Where(mt => mt.TimeStamp >= DateBegin);
What I need else is add N (unknown number) of Where clauses, but with OR condition between them.
Image list of some object:
List<MyObject> myObj =
new List<MyObject>({new MyObject {val = "a" }, new MyObject { val = "b"}}); //In real code there is more than 1 property.
then I'd like to add Where() clauses to query like:
myObj.ForEach(mo =>{
// THIS CREATES -AND- BETWEEN WHERE CLAUSES, BUT I NEED -OR-
query.Where(q=>q.MyValue == mo.val); // In real code there is more than 1 property to compare
});
I was thinking about .Union() beteween queries, but It could generate union between separated queries and it's not optimal I think.
Thanks!
Here's the solution: linq-to-entities-combining-predicates
Or course is necessary to use "latest" answer:
Copy/Paste class ParameterRebinder
Copy/Paste static class Utility
Usage:
Expression<Func<Car, bool>> theCarIsRed = c => c.Color == "Red";
Expression<Func<Car, bool>> theCarIsCheap = c => c.Price < 10.0;
Expression<Func<Car, bool>> theCarIsRedOrCheap = theCarIsRed.Or(theCarIsCheap);
var query = carQuery.Where(theCarIsRedOrCheap);
Because in my solution is N of expressions, I take first expression and then append other expressions in ForEach cycle.
var firstExpression = expressionList.First();
expressionList.Skip(1).ToList().ForEach(ex => { firstExpression = firstExpression.Or(ex); });

Equivalent to SQL IN clause

I've got an entity called new_trexmail with a string attribute called new_contextline.
I'm trying to get a list of entities where new_contextlineis in a defined list.
The following code fails with the error : NotSupportedException: Invalid 'where' condition. An entity member is invoking an invalid property or method.
string[] test = new[]{"aaa", "hhh"};
var query = from n in New_trexmailSet
where test.Contains(n.New_contextline)
select n;
I understand why this error is being thrown but I'm wondering if it's possible to do the equiavalent of an IN clause using XRM.
If it is possible then how do I go about getting XRM to execute SELECT * FROM new_trexmail WHERE new_contextline in ('aaa', 'hhh')?
Thanks,
David
Check out the (longer than desired) list of LINQ limitations, particularly the limitation on the where clause:
The left side of the clause must be an attribute name and the right
side of the clause must be a value. You cannot set the left side to a
constant. Both the sides of the clause cannot be constants. Supports
the String functions Contains, StartsWith, EndsWith, and Equals.
So since test isn't a CRM attribute, you can't call Contains on it. However, one way around this is to use "Dynamic Linq" as developed by ScottGu and as demonstrated below:
//must include the below using statements
//using System.Linq;
//using System.Linq.Dynamic;
var trexmailSet = New_trexmailSet;
string[] test = new[] { "aaa", "hhh" };
string whereClause = "";
foreach (string name in test)
{
whereClause += string.Format("new_contextline = \"{0}\" OR ", name);
}
trexmailSet = trexmailSet.Where(whereClause.Substring(0, whereClause.Length - 4));
var query = from n in trexmailSet
select n;

LINQ for LIKE queries of array elements

Let's say I have an array, and I want to do a LINQ query against a varchar that returns any records that have an element of the array anywhere in the varchar.
Something like this would be sweet.
string[] industries = { "airline", "railroad" }
var query = from c in contacts where c.industry.LikeAnyElement(industries) select c
Any ideas?
This is actually an example I use in my "Express Yourself" presentation, for something that is hard to do in regular LINQ; As far as I know, the easiest way to do this is by writing the predicate manually. I use the example below (note it would work equally for StartsWith etc):
using (var ctx = new NorthwindDataContext())
{
ctx.Log = Console.Out;
var data = ctx.Customers.WhereTrueForAny(
s => cust => cust.CompanyName.Contains(s),
"a", "de", "s").ToArray();
}
// ...
public static class QueryableExt
{
public static IQueryable<TSource> WhereTrueForAny<TSource, TValue>(
this IQueryable<TSource> source,
Func<TValue, Expression<Func<TSource, bool>>> selector,
params TValue[] values)
{
return source.Where(BuildTrueForAny(selector, values));
}
public static Expression<Func<TSource, bool>> BuildTrueForAny<TSource, TValue>(
Func<TValue, Expression<Func<TSource, bool>>> selector,
params TValue[] values)
{
if (selector == null) throw new ArgumentNullException("selector");
if (values == null) throw new ArgumentNullException("values");
if (values.Length == 0) return x => true;
if (values.Length == 1) return selector(values[0]);
var param = Expression.Parameter(typeof(TSource), "x");
Expression body = Expression.Invoke(selector(values[0]), param);
for (int i = 1; i < values.Length; i++)
{
body = Expression.OrElse(body,
Expression.Invoke(selector(values[i]), param));
}
return Expression.Lambda<Func<TSource, bool>>(body, param);
}
}
from c in contracts
where industries.Any(i => i == c.industry)
select c;
something like that. use the any method on the collection.
IEnumerable.Contains() translates to SQL IN as in:
WHERE 'american airlines' IN ('airline', 'railroad') -- FALSE
String.Contains() which translates to SQL LIKE %...% as in:
WHERE 'american airlines' LIKE '%airline%' -- TRUE
If you want the contacts where the contact's industry is LIKE (contains) any of the given industries, you want to combine both Any() and String.Contains() into something like this:
string[] industries = { "airline", "railroad" };
var query = from c in contacts
where industries.Any(i => c.Industry.Contains(i))
select c;
However, combining both Any() and String.Contains() like this is NOT supported in LINQ to SQL. If the set of given industries is small, you can try something like:
where c.Industry.Contains("airline") ||
c.Industry.Contains("railroad") || ...
Or (although normally not recommended) if the set of contacts is small enough, you could bring them all from the DB and apply the filter with LINQ to Objects by using contacts.AsEnumerable() or contacts.ToList() as the source of the query above:
var query = from c in contacts.AsEnumerable()
where industries.Any(i => c.Industry.Contains(i))
select c;
it will work if you build up the query as follows:
var query = from c in contacts.AsEnumerable()
select c;
query = query.Where(c=> (c.Industry.Contains("airline")) || (c.Industry.Contains("railroad")));
you just need to programmatically generate the string above if the parameters airline and railroad are user inputs. This was in fact a little more complicated than I was expecting. See article - http://www.albahari.com/nutshell/predicatebuilder.aspx
Unfortunately, LIKE is not supported in LINQ to SQL as per here:
http://msdn.microsoft.com/en-us/library/bb882677.aspx
To get around this, you will have to write a stored procedure which will accept the parameters you want to use in the like statement(s) and then call that from LINQ to SQL.
It should be noted that a few of the answers suggest using Contains. This won't work because it looks to see that the entire string matches the array element. What is being looked for is for the array element to be contained in the field itself, something like:
industry LIKE '%<element>%'
As Clark has mentioned in a comment, you could use a call to IndexOf on each element (which should translate to a SQL call):
string[] industries = { "airline", "railroad" }
var query =
from c in contacts
where
c.industry.IndexOf(industries[0]) != -1 ||
c.industry.IndexOf(industries[1]) != -1
If you know the length of the array and the number of elements, then you could hard-code this. If you don't, then you will have to create the Expression instance based on the array and the field you are looking at.

LINQ to SQL bug (or very strange feature) when using IQueryable, foreach, and multiple Where

I ran into a scenario where LINQ to SQL acts very strangely. I would like to know if I'm doing something wrong. But I think there is a real possibility that it's a bug.
The code pasted below isn't my real code. It is a simplified version I created for this post, using the Northwind database.
A little background: I have a method that takes an IQueryable of Product and a "filter object" (which I will describe in a minute). It should run some "Where" extension methods on the IQueryable, based on the "filter object", and then return the IQueryable.
The so-called "filter object" is a System.Collections.Generic.List of an anonymous type of this structure: { column = fieldEnum, id = int }
The fieldEnum is an enum of the different columns of the Products table that I would possibly like to use for the filtering.
Instead of explaining further how my code works, it's easier if you just take a look at it. It's simple to follow.
enum filterType { supplier = 1, category }
public IQueryable<Product> getIQueryableProducts()
{
NorthwindDataClassesDataContext db = new NorthwindDataClassesDataContext();
IQueryable<Product> query = db.Products.AsQueryable();
//this section is just for the example. It creates a Generic List of an Anonymous Type
//with two objects. In real life I get the same kind of collection, but it isn't hard coded like here
var filter1 = new { column = filterType.supplier, id = 7 };
var filter2 = new { column = filterType.category, id = 3 };
var filterList = (new[] { filter1 }).ToList();
filterList.Add(filter2);
foreach(var oFilter in filterList)
{
switch (oFilter.column)
{
case filterType.supplier:
query = query.Where(p => p.SupplierID == oFilter.id);
break;
case filterType.category:
query = query.Where(p => p.CategoryID == oFilter.id);
break;
default:
break;
}
}
return query;
}
So here is an example. Let's say the List contains two items of this anonymous type, { column = fieldEnum.Supplier, id = 7 } and { column = fieldEnum.Category, id = 3}.
After running the code above, the underlying SQL query of the IQueryable object should contain:
WHERE SupplierID = 7 AND CategoryID = 3
But in reality, after the code runs the SQL that gets executed is
WHERE SupplierID = 3 AND CategoryID = 3
I tried defining query as a property and setting a breakpoint on the setter, thinking I could catch what's changing it when it shouldn't be. But everything was supposedly fine. So instead I just checked the underlying SQL after every command. I realized that the first Where runs fine, and query stays fine (meaning SupplierID = 7) until right after the foreach loop runs the second time. Right after oFilter becomes the second anonymous type item, and not the first, the 'query' SQL changes to Supplier = 3. So what must be happening here under-the-hood is that instead of just remembering that Supplier should equal 7, LINQ to SQL remembers that Supplier should equal oFilter.id. But oFilter is a name of a single item of a foreach loop, and it means something different after it iterates.
I have only glanced at your question, but I am 90% sure that you should read the first section of On lambdas, capture, and mutability (which includes links to 5 similar SO questions) and all will become clear.
The basic gist of it is that the variable oFilter in your example has been captured in the closure by reference and not by value. That means that once the loop finishes iterating, the variable's reference is to the last one, so the value as evaluated at lambda execution time is the final one as well.
The cure is to insert a new variable inside the foreach loop whose scope is only that iteration rather than the whole loop:
foreach(var oFilter in filterList)
{
var filter = oFilter; // add this
switch (oFilter.column) // this doesn't have to change, but can for consistency
{
case filterType.supplier:
query = query.Where(p => p.SupplierID == filter.id); // use `filter` here
break;
Now each closure is over a different filter variable that is declared anew inside of each loop, and your code will run as expected.
Working as designed. The issue you are confronting is the clash between lexical closure and mutable variables.
What you probably want to do is
foreach(var oFilter in filterList)
{
var o = oFilter;
switch (o.column)
{
case filterType.supplier:
query = query.Where(p => p.SupplierID == o.id);
break;
case filterType.category:
query = query.Where(p => p.CategoryID == o.id);
break;
default:
break;
}
}
When compiled to IL, the variable oFilter is declared once and used multiply. What you need is a variable declared separately for each use of that variable within a closure, which is what o is now there for.
While you're at it, get rid of that bastardized Hungarian notation :P.
I think this is the clearest explanation I've ever seen: http://blogs.msdn.com/ericlippert/archive/2009/11/12/closing-over-the-loop-variable-considered-harmful.aspx:
Basically, the problem arises because we specify that the foreach loop is a syntactic sugar for
{
IEnumerator<int> e = ((IEnumerable<int>)values).GetEnumerator();
try
{
int m; // OUTSIDE THE ACTUAL LOOP
while(e.MoveNext())
{
m = (int)(int)e.Current;
funcs.Add(()=>m);
}
}
finally
{
if (e != null) ((IDisposable)e).Dispose();
}
}
If we specified that the expansion was
try
{
while(e.MoveNext())
{
int m; // INSIDE
m = (int)(int)e.Current;
funcs.Add(()=>m);
}
then the code would behave as expected.
The problem is that you're not appending to the query, you're replacing it each time through the foreach statement.
You want something like the PredicateBuilder - http://www.albahari.com/nutshell/predicatebuilder.aspx

Resources