I need to implement the query using expressions syntax (because I don't know types in compile time). For example query like this one:
from customer in Customers
join purchase in Purchases
on customer.ID equals purchase.CustomerID
into outerJoin
from range in outerJoin.DefaultIfEmpty()
where
customer.Name == "SomeName" &&
range.Description.Contains("SomeString") &&
customer.ID == range.CustomerID
select
new { Customer = customer, Purchase = range }
I found way to implement group join part like this:
ITable p = _dataContext.GetTable(typeof(Purchases));
ITable c = _dataContext.GetTable(typeof(Customers));
LambdaExpression outerSelectorLambda = DynamicExpression.ParseLambda(typeof(Customers), null, "ID");
LambdaExpression innerSelectorLambda = DynamicExpression.ParseLambda(typeof(Purchases), null, "CustomerID");
ParameterExpression param1 = Expression.Parameter(typeof(Customers), "customer");
ParameterExpression param2 = Expression.Parameter(typeof(IEnumerable<Purchases>), "purchases");
ParameterExpression[] parameters = new ParameterExpression[] { param1, param2 };
LambdaExpression resultsSelectorLambda = DynamicExpression.ParseLambda(parameters, null, "New(customers as customers, purchases as purchases)");
MethodCallExpression joinCall =
Expression.Call(
typeof(Queryable),
"GroupJoin",
new Type[] {
typeof(Customers),
typeof(Purchases),
outerSelectorLambda.Body.Type,
resultsSelectorLambda.Body.Type
},
c.Expression,
p.Expression,
Expression.Quote(outerSelectorLambda),
Expression.Quote(innerSelectorLambda),
Expression.Quote(resultsSelectorLambda)
);
But I can't figure out how to write rest of query using this syntax.
Does anyone can help me?
I would follow an approach to achieve that:
Get the Expression equivalent of the LINQ query.
Get the ToString() of the Expression extracted from the LINQ query.
Study the Expression, to understand the input parameters, type parameters, Expression Arguments etc...
Get back to me if the approach mentioned is not clear.
I just copied + pasted the "join" implementation in dynamic.cs and made couple of changes to make "GroupJoin" to work.
public static IQueryable GroupJoin(this IQueryable outer, IEnumerable inner, string outerSelector, string innerSelector, string resultsSelector, params object[] values)
{
if (inner == null) throw new ArgumentNullException("inner");
if (outerSelector == null) throw new ArgumentNullException("outerSelector");
if (innerSelector == null) throw new ArgumentNullException("innerSelector");
if (resultsSelector == null) throw new ArgumentNullException("resultsSelctor");
LambdaExpression outerSelectorLambda = DynamicExpression.ParseLambda(outer.ElementType, null, outerSelector, values);
LambdaExpression innerSelectorLambda = DynamicExpression.ParseLambda(inner.AsQueryable().ElementType, null, innerSelector, values);
Type resultType = typeof(IEnumerable<>).MakeGenericType(inner.AsQueryable().ElementType);
ParameterExpression[] parameters = new ParameterExpression[]
{
Expression.Parameter(outer.ElementType, "outer"), Expression.Parameter(resultType, "inner")
};
LambdaExpression resultsSelectorLambda = DynamicExpression.ParseLambda(parameters, null, resultsSelector, values);
return outer.Provider.CreateQuery(
Expression.Call(
typeof(Queryable), "GroupJoin",
new Type[] { outer.ElementType, inner.AsQueryable().ElementType, outerSelectorLambda.Body.Type, resultsSelectorLambda.Body.Type },
outer.Expression, inner.AsQueryable().Expression, Expression.Quote(outerSelectorLambda), Expression.Quote(innerSelectorLambda), Expression.Quote(resultsSelectorLambda)));
}
Related
I am wondering how would the below code work and what would be the performance. I am interested in line #4 basically. How will linq and property info work together?
In other words will (int)prop.GetValue(a) and a.SomeId work the same way or will the reflection need to get everything to memory before checking the value?
var prop = type.GetProperty("SomeId");
if (prop != null)
{
DbSet<T> dbSet = _dbContext.Set<T>();
return dbSet.Where(a => (int)prop.GetValue(a) == 1);
}
In other words will (int)prop.GetValue(a) and a.SomeId work the same way
No.
Depending on the backing store, the context may fail to convert that expression ((int)prop.GetValue(a)) to the underlying query syntax, which in most cases would be SQL.
You could consider building a valid expression manually using the property info.
For example
//Assuming
Type type = typeof(T);
PropertyInfo prop = type.GetProperty("SomeId");
if (prop != null) {
//need to build a => a.SomeId == 1
// a =>
ParameterExpression parameter = Expression.Parameter(type, "a");
// a => a.SomeId
MemberExpression property = Expression.Property(parameter, prop);
// a => a.SomeId == 1
BinaryExpression body = Expression.Equal(property, Expression.Constant(1));
Expression<Func<T, bool>> expression = Expression.Lambda<Func<T, bool>>(body, parameter);
DbSet<T> dbSet = _dbContext.Set<T>();
return dbSet.Where(expression);
}
I have chosen to create an anonymous type for temporary projection in a LINQ Join query. I am using ExpressionTrees, to build the query at runtime. I donot know if the following code would help me with creating a temporary projected sequence.
Here is the code that does the temporary Projection:
private Expression<Func<EntityObject, EntityObject,dynamic>> TempProjectionExpression
{
get
{
return (o, p) => new
{
o = o,
p = p
};
}
}
My Join Query using Expression Trees is given below.
public IQueryable<dynamic> GetQueryExpressionresults3<T, U, V>(IQueryable<T> aEntityCollection1, IQueryable<U> aEntityCollection2, Type[] _TypeArguments ,V _anonymousType, string aPropertyName)
where T : EntityObject
where U : EntityObject
{
ParameterExpression pe = Expression.Parameter(typeof(U), "o");
ParameterExpression pe1 = Expression.Parameter(typeof(T), "p");
//This should be populated from UI
Expression me = Expression.Property(pe1, typeof(T).GetProperty(aPropertyName));
//This should be populated from UI
Expression me1 = Expression.Property(pe, typeof(U).GetProperty(aPropertyName));
LambdaExpression le = Expression.Lambda<Func<T, int>>(me, new ParameterExpression[] { pe1 });
LambdaExpression le1 = Expression.Lambda<Func<U, int>>(me1, new ParameterExpression[] { pe });
_TypeArguments = new Type[] { aEntityCollection1 .ElementType, aEntityCollection2.ElementType, le.Body.Type, typeof(MovieCollections)};
//_TypeArguments = _TypeArguments.Concat(new Type[] { le.Body.Type, typeof(object) }).ToArray();
MethodCallExpression JoinCallExpression = Expression.Call(typeof(Queryable), "Join", _TypeArguments, aEntityCollection1.Expression, aEntityCollection2.Expression
, le, le1, TempProjectionExpression);
var oResult = aEntityCollection1.Provider.CreateQuery(JoinCallExpression) as IQueryable<dynamic>;
return oResult;
}
Now the question is, i would like to determine the return type of the TempProjectionExpression, i.e. typeof(dynamic). Is this possible? If yes, then how?
typeof(dynamic) can't do any better than System.Object (which the compiler won't even try to do) which isn't a very interesting result.
You can use returnedResult.GetType(), though, to get its runtime type.
Since dynamic puts off type resolution to runtime, there's no way to tell what the return is before something is actually returned without doing type analysis on your expression tree worthy of the C# compiler itself.
I built a custom IQueryable provider. The provider transforms query for example
c.PurchaseDate == new DateTime(2011, 11, 29) && c.Name == "Elizabeth Brown"
from underlying code into System.Linq.Expressions.Expression
Now I need to run them against this collection with Linq query
IQueryable<Customer> customers = _customers.AsQueryable();
Can anyone tell me how to query the collection with Expression?
Thanks
//Query = c.PurchaseDate == new DateTime(2011, 11, 29) && c.Name
// == "Elizabeth Brown" )
IQueryable<Customer> customers = _customers.AsQueryable<Customer>();
//Predicate parameter
ParameterExpression parameter = Expression.Parameter(typeof(Customer),
"customer");
//Create left expression
Expression left = Expression.Property(parameter, typeof(Customer)
.GetProperty("PurchaseDate"));
Expression right = Expression.Constant(new DateTime(2011, 11, 29));
Expression leftExp = Expression.Equal(left, right);
//Create right expression tree
left = Expression.Property(parameter, typeof(Customer).GetProperty("Name"));
right = Expression.Constant("Elizabeth Brown", typeof(string));
Expression rightExp = Expression.Equal(left, right);
//Combine the expressions into expression tree
Expression expressionTree = Expression.AndAlso(leftExp, rightExp);
//Create an expression tree that represents the expression
MethodCallExpression methodCall = Expression.Call(
typeof(Queryable),
"Where",
new Type[] { customers.ElementType },
customers.Expression,
Expression
.Lambda<Func<Customer, bool>>
(expressionTree, new ParameterExpression[] { parameter }));
// Create an executable query from the expression tree.
IQueryable<Customer> results =
customers.Provider.CreateQuery<Customer>(methodCall);
// Enumerate the results
foreach (Customer customer in results)
{
Console.WriteLine("{0} {1}", customer.Name, customer.PurchaseDate);
}
Console.ReadLine();
I finished with the task this way. IQueryable is really wonderfull stuff. Enjoy!
I'm building a LINQ-based query generator.
One of the features is being able to specify an arbitrary server-side projection as part of the query definition. For example:
class CustomerSearch : SearchDefinition<Customer>
{
protected override Expression<Func<Customer, object>> GetProjection()
{
return x => new
{
Name = x.Name,
Agent = x.Agent.Code
Sales = x.Orders.Sum(o => o.Amount)
};
}
}
Since the user must then be able to sort on the projection properties (as opposed to Customer properties), I recreate the expression as a Func<Customer,anonymous type> instead of Func<Customer, object>:
//This is a method on SearchDefinition
IQueryable Transform(IQueryable source)
{
var projection = GetProjection();
var properProjection = Expression.Lambda(projection.Body,
projection.Parameters.Single());
In order to return the projected query, I'd love to be able to do this (which, in fact, works in an almost identical proof of concept):
return Queryable.Select((IQueryable<TRoot>)source, (dynamic)properProjection);
TRoot is the type parameter in SearchDefinition. This results in the following exception:
Microsoft.CSharp.RuntimeBinder.RuntimeBinderException:
The best overloaded method match for
'System.Linq.Queryable.Select<Customer,object>(System.Linq.IQueryable<Customer>,
System.Linq.Expressions.Expression<System.Func<Customer,object>>)'
has some invalid arguments
at CallSite.Target(Closure , CallSite , Type , IQueryable`1 , Object )
at System.Dynamic.UpdateDelegates.UpdateAndExecute3[T0,T1,T2,TRet]
(CallSite site, T0 arg0, T1 arg1, T2 arg2)
at SearchDefinition`1.Transform(IQueryable source) in ...
If you look closely, it's inferring the generic parameters incorrectly: Customer,object instead of Customer,anonymous type, which is the actual type of the properProjection expression (double-checked)
My workaround is using reflection. But with generic arguments, it's a real mess:
var genericSelectMethod = typeof(Queryable).GetMethods().Single(
x => x.Name == "Select" &&
x.GetParameters()[1].ParameterType.GetGenericArguments()[0]
.GetGenericArguments().Length == 2);
var selectMethod = genericSelectMethod.MakeGenericMethod(source.ElementType,
projectionBody.Type);
return (IQueryable)selectMethod.Invoke(null, new object[]{ source, projection });
Does anyone know of a better way?
Update: the reason why dynamic fails is that anonymous types are defined as internal. That's why it worked using a proof-of-concept project, where everything was in the same assembly.
I'm cool with that. I'd still like to find a cleaner way to find the right Queryable.Select overload.
The fix is so simple it hurts:
[assembly: InternalsVisibleTo("My.Search.Lib.Assembly")]
Here's my test as requested. This on a Northwind database and this works fine for me.
static void Main(string[] args)
{
var dc = new NorthwindDataContext();
var source = dc.Categories;
Expression<Func<Category, object>> expr =
c => new
{
c.CategoryID,
c.CategoryName,
};
var oldParameter = expr.Parameters.Single();
var parameter = Expression.Parameter(oldParameter.Type, oldParameter.Name);
var body = expr.Body;
body = RebindParameter(body, oldParameter, parameter);
Console.WriteLine("Parameter Type: {0}", parameter.Type);
Console.WriteLine("Body Type: {0}", body.Type);
var newExpr = Expression.Lambda(body, parameter);
Console.WriteLine("Old Expression Type: {0}", expr.Type);
Console.WriteLine("New Expression Type: {0}", newExpr.Type);
var query = Queryable.Select(source, (dynamic)newExpr);
Console.WriteLine(query);
foreach (var item in query)
{
Console.WriteLine(item);
Console.WriteLine("\t{0}", item.CategoryID.GetType());
Console.WriteLine("\t{0}", item.CategoryName.GetType());
}
Console.Write("Press any key to continue . . . ");
Console.ReadKey(true);
Console.WriteLine();
}
static Expression RebindParameter(Expression expr, ParameterExpression oldParam, ParameterExpression newParam)
{
switch (expr.NodeType)
{
case ExpressionType.Parameter:
var parameterExpression = expr as ParameterExpression;
return (parameterExpression.Name == oldParam.Name)
? newParam
: parameterExpression;
case ExpressionType.MemberAccess:
var memberExpression = expr as MemberExpression;
return memberExpression.Update(
RebindParameter(memberExpression.Expression, oldParam, newParam));
case ExpressionType.AndAlso:
case ExpressionType.OrElse:
case ExpressionType.Equal:
case ExpressionType.NotEqual:
case ExpressionType.LessThan:
case ExpressionType.LessThanOrEqual:
case ExpressionType.GreaterThan:
case ExpressionType.GreaterThanOrEqual:
var binaryExpression = expr as BinaryExpression;
return binaryExpression.Update(
RebindParameter(binaryExpression.Left, oldParam, newParam),
binaryExpression.Conversion,
RebindParameter(binaryExpression.Right, oldParam, newParam));
case ExpressionType.New:
var newExpression = expr as NewExpression;
return newExpression.Update(
newExpression.Arguments
.Select(arg => RebindParameter(arg, oldParam, newParam)));
case ExpressionType.Call:
var methodCallExpression = expr as MethodCallExpression;
return methodCallExpression.Update(
RebindParameter(methodCallExpression.Object, oldParam, newParam),
methodCallExpression.Arguments
.Select(arg => RebindParameter(arg, oldParam, newParam)));
default:
return expr;
}
}
Also, dynamic method resolution doesn't really do much for you in this case as there are only two very distinct overloads of Select(). Ultimately you just need to remember that you won't have any static type checking on your results since you don't have any static type information. With that said, this will also work for you (using the above code example):
var query = Queryable.Select(source, expr).Cast<dynamic>();
Console.WriteLine(query);
foreach (var item in query)
{
Console.WriteLine(item);
Console.WriteLine("\t{0}", item.CategoryID.GetType());
Console.WriteLine("\t{0}", item.CategoryName.GetType());
}
Ok, I'll admit that I don't entirely "get" lambda expressions and LINQ expression trees yet; a lot of what I'm doing is cutting and pasting and seeing what works. I've looked over lots of documentation, but I still haven't found the my "aha" moment yet.
With that being said...
I'm attempting to dynamically add a GroupBy expression to my Linq expression. I followed the question here:
Need help creating Linq.Expression to Enumerable.GroupBy
and tried to implement what I saw there.
First off, I've got entity classes for my database, and a table calledObjCurLocViewNormalized
I've got an method that does the initial call,
public IQueryable<ObjCurLocViewNormalized> getLocations()
{
IQueryable<ObjCurLocViewNormalized> res = (from loc in tms.ObjCurLocViewNormalized
select loc);
return res;
}
so I can call:
IQueryable<MetAmericanLinqDataModel.ObjCurLocViewNormalized> locations = american.getLocations();
No problem so far.
Now, I want to group by an arbitrary column, with a call like this:
var grouped = locations.addGroupBy(childLocationFieldName);
Right now, I have a method :
static public System.Linq.IQueryable<System.Linq.IGrouping<string, TResult>> addGroupBy<TResult>(this IQueryable<TResult> query, string columnName)
{
var providerType = query.Provider.GetType();
// Find the specific type parameter (the T in IQueryable<T>)
var iqueryableT = providerType.FindInterfaces((ty, obj) => ty.IsGenericType && ty.GetGenericTypeDefinition() == typeof(IQueryable<>), null).FirstOrDefault();
var tableType = iqueryableT.GetGenericArguments()[0];
var tableName = tableType.Name;
var data = Expression.Parameter(iqueryableT, "query");
var arg = Expression.Parameter(tableType, tableName);
var nameProperty = Expression.PropertyOrField(arg, columnName);
var lambda = Expression.Lambda<Func<TResult, string>>(nameProperty, arg);
var expression = Expression.Call(typeof(Enumerable),
"GroupBy",
new Type[] { tableType, typeof(string) },
data,
lambda);
var predicate = Expression.Lambda<Func<TResult, String>>(expression, arg); // this is the line that produces the error I describe below
var result = query.GroupBy(predicate).AsQueryable();
return result;
}
All this compiles ok, but when I run it, I get the error:
System.ArgumentException: Expression of type 'System.Collections.Generic.IEnumerable`1[System.Linq.IGrouping`2[System.String,MetAmericanLinqDataModel.ObjCurLocViewNormalized]]' cannot be used for return type 'System.String'
and the error comes from this line:
var predicate = Expression.Lambda<Func<TResult, String>>(expression, arg);
I'm copying and adapting this code from successful work I did in dynamically added Where clauses to an expression. So I'm sort of stabbing in the dark here.
If anyone out there can help to shed some light on this, Obviously posting complete working code and doing all my thinking for me would be great :), but if you could just lay out just why this is wrong, or how to wrap my head around these concepts, that would be great. If you can point to documentation that can really help be bridge the gap between the basics of lambda expressions, and building dynamic expression trees, that would be great. There's obviously big holes in my knowledge, but I think this information could be useful to others.
thanks everyone for your time, and of course if I find the answer elsewhere, I'll post it here.
Thanks again.
Don
The solution should be pretty simple:
public static IQueryable<IGrouping<TColumn, T>> DynamicGroupBy<T, TColumn>(
IQueryable<T> source, string column)
{
PropertyInfo columnProperty = typeof(T).GetProperty(column);
var sourceParm = Expression.Parameter(typeof(T), "x");
var propertyReference = Expression.Property(sourceParm, columnProperty);
var groupBySelector = Expression.Lambda<Func<T, TColumn>>(propertyReference, sourceParm);
return source.GroupBy(groupBySelector);
}
Assuming a sample class like this:
public class TestClass
{
public string TestProperty { get; set; }
}
You invoke it like this:
var list = new List<TestClass>();
var queryable = list.AsQueryable();
DynamicGroupBy<TestClass, string>(queryable, "TestProperty");
All that you need to do to make it work is the following:
static public IQueryable<IGrouping<TValue, TResult>> addGroupBy<TValue, TResult>(
this IQueryable<TResult> query, string columnName)
{
var providerType = query.Provider.GetType();
// Find the specific type parameter (the T in IQueryable<T>)
const object EmptyfilterCriteria = null;
var iqueryableT = providerType
.FindInterfaces((ty, obj) => ty.IsGenericType && ty.GetGenericTypeDefinition() == typeof(IQueryable<>), EmptyfilterCriteria)
.FirstOrDefault();
Type tableType = iqueryableT.GetGenericArguments()[0];
string tableName = tableType.Name;
ParameterExpression data = Expression.Parameter(iqueryableT, "query");
ParameterExpression arg = Expression.Parameter(tableType, tableName);
MemberExpression nameProperty = Expression.PropertyOrField(arg, columnName);
Expression<Func<TResult, TValue>> lambda = Expression.Lambda<Func<TResult, TValue>>(nameProperty, arg);
//here you already have delegate in the form of "TResult => TResult.columnName"
return query.GroupBy(lambda);
/*var expression = Expression.Call(typeof(Enumerable),
"GroupBy",
new Type[] { tableType, typeof(string) },
data,
lambda);
var predicate = Expression.Lambda<Func<TResult, String>>(expression, arg); // this is the line that produces the error I describe below
var result = query.GroupBy(predicate).AsQueryable();
return result;*/
}
And you will call you expression in the following manner:
var grouped = locations.addGroupBy<string, ObjCurLocViewNormalized>(childLocationFieldName);
First generic parameter "string" us used for saying explicilty what type of elements you a grouping on. For example you can group by "int" field and method call will be like following:
var grouped = locations.addGroupBy<int, ObjCurLocViewNormalized>(someFieldNameWithTheTypeOfInt);
Edit
Just to finish this solution your way:
//return query.GroupBy(lambda);
MethodCallExpression expression = Expression.Call(typeof (Enumerable),
"GroupBy",
new[] { typeof(TResult), typeof(TValue) },
data,
lambda);
var result = Expression.Lambda(expression, data).Compile().DynamicInvoke(query);
return ((IEnumerable<IGrouping<TValue, TResult>>)result).AsQueryable();