How can I intercept the result of an IQueryProvider query (other than single result) - linq

I'm using Entity Framework and I have a custum IQueryProvider. I use the Execute method so that I can modify the result (a POCO) of a query after is has been executed. I want to do the same for collections. The problem is that the Execute method is only called for single result.
As described on MSDN :
The Execute method executes queries that return a single value
(instead of an enumerable sequence of values). Expression trees that
represent queries that return enumerable results are executed when
their associated IQueryable object is enumerated.
Is there another way to accomplish what I want that I missed?
I know I could write a specific method inside a repository or whatever but I want to apply this to all possible queries.

This is true that the actual signature is:
public object Execute(Expression expression)
public TResult Execute<TResult>(Expression expression)
However, that does not mean that the TResult will always be a single element! It is the type expected to be returned from the expression.
Also, note that there are no constraints over the TResult, not even 'class' or 'new()'.
The TResult is a MyObject when your expression is of singular result, like .FirstOrDefault(). However, the TResult can also be a double when you .Avg() over the query, and also it can be IEnumerable<MyObject> when your query is plain .Select.Where.
Proof(*) - I've just set a breakpoint inside my Execute() implementation, and I've inspected it with Watches:
typeof(TResult).FullName "System.Collections.Generic.IEnumerable`1[[xxxxxx,xxxxx]]"
expression.Type.FullName "System.Linq.IQueryable`1[[xxxxxx,xxxxx]]"
I admit that three overloads, one object, one TResult and one IEnumerable<TResult> would probably be more readable. I think they did not place three of them as extensibility point for future interfaces. I can imagine that in future they came up with something more robust than IEnumerable, and then they'd need to add another overload and so on. With simple this interface can process any type.
Oh, see, we now also have IQueryable in addition to IEnumerable, so it would need at least four overloads:)
The Proof is marked with (*) because I have had a small bug/feature in my IQueryProvider's code that has is obscuring the real behavior of LINQ.
LINQ indeed calls the generic Execute only for singular cases. This is a shortcut, an optimization.
For all other cases, it ... doesn't call Execute() it at all
For those all other cases, the LINQ calls .GetEnumerator on your custom IQueryable<> implementation, that what happens is dictated by .. simply what you wrote there. I mean, assuming that you actually provided custom implementations of IQueryable. That would be strange if you did not - that's just about 15 lines in total, nothing compared to the length of custom provider.
In the project where I got the "proof" from, my implementation looks like:
public System.Collections.IEnumerator GetEnumerator()
{
return Provider.Execute<IEnumerable>( this.Expression ).GetEnumerator();
}
public IEnumerator<TOut> GetEnumerator()
{
return Provider.Execute<IEnumerable<TOut>>( this.Expression ).GetEnumerator();
}
of course, one of them would be explicit due to name collision. Please note that to fetch the enumerator, I actually call the Execute with explicitely stated TResult. This is why in my "proof" those types occurred.
I think that you see the "TResult = Single Element" case, because you wrote i.e. something like this:
public IEnumerator<TOut> GetEnumerator()
{
return Provider.Execute<TOut>( this.Expression ).GetEnumerator();
}
Which really renders your Execute implementation without choice, and must return single element. IMHO, this is just a bug in your code. You could have done it like in my example above, or you could simply use the untyped Execute:
public System.Collections.IEnumerator GetEnumerator()
{
return ((IEnumerable)Provider.Execute( this.Expression )).GetEnumerator();
}
public IEnumerator<TOut> GetEnumerator()
{
return ((IEnumerable<TOut>)Provider.Execute( this.Expression )).GetEnumerator();
}
Of course, your implementation of Execute must make sure to return proper IEnumerables for such queries!

Expression trees that represent queries that return enumerable results are executed when their associated IQueryable object is enumerated.
I recommend enumerating your query:
foreach(T t in query)
{
CustomModification(t);
}
Your IQueryProvider must implement CreateQuery<T>. You get to choose the implemenation of the resulting IQueryable. If you want that IQueryable to do something to each row when enumerated, you get to write that implementation.

The final answer is that it's not possible.

Related

c# generic orderby

In my base-repository class
i wrote this function to make possible to retrive a sorted data collection from the DB.
T is a generic defined at Class level
public abstract class RepositoryBase<T>
where T : class
The code is this:
public IList<T> GetAll<TKey>(Expression<Func<T, bool>> whereCondition, Expression<Func<T, TKey>> sortCondition, bool sortDesc = false)
{
if (sortDesc)
return this.ObjectSet.Where(whereCondition).OrderByDescending(sortCondition).ToList<T>();
return this.ObjectSet.Where(whereCondition).OrderBy(sortCondition).ToList<T>() ;
}
My goal was to introduce a generic sort parameter so that i could call the function in this way:
repo.GetAll (model=>model.field>0, model=>model.sortableField, true)
i mean that i could specify the sorting field directly via anonymous function and so using Intellisense...
Unfortunately this function doesn't work as the last code line generate errors at compile time.
I tried also to call:
repo.GetAll<Model> (model=>model.field>0, model=>model.sortableField, true)
but this don't work.
How should i write the function to meet my goal?
i'm working with EF 5, c#, .NET 4.5
You're using ObjectSet which implements IQueryable<T>. That is extended by methods on System.Linq.Queryable, which accept Expression<Func< parameters. It is correct to use those Expression parameters, as you intend for execution to occur in the database, not locally.
A Func is an anonymous delegate, a .net method.
An Expression is a tree, which may be compiled into a Func, or may be translated into Sql or something else.
You showed us a really abstract use of the method, but not an actual use of the method, or the compiler error. I suspect the error you may be making is confusing the two type parameters.
You said:
repo.GetAll<Model> (model=>model.field>0, model=>model.sortableField, true)
But this generic parameter for this method represents the type of sortableField. If sortableField isn't a Model - this is wrong.
Instead, you should be doing something like this:
Repository<Person> myRepo = new Repository<Person>();
myRepo.GetAll<DateTime>(p => p.Friends.Count() > 3, p => p.DateOfBirth, true);
If specifying the sort type breaks your intended pattern of usage, consider hiding that key by using an IOrderer: Store multi-type OrderBy expression as a property

Design choice: Any good reason for 'ToArray()' LINQ extension to throw an exception for null collections?

Unsurprisingly, following code will throw an ArgumentNullException
IEnumerable<string> collection = null;
string[] collectionViewAsAnArray = collection.ToArray();
This looks obvious at first sight ... but ain't incoherent to argue that returning null may have been a reasonnable alternative (Accounting that ToArray() is an extension method and therefor can be called, even on null).
While I acknowledge, this way, extension behave like a real method, I can't help finding the other approach really smart too ... but this may lead to other issues?
Granted IEnumerable is an interface used in collections but the underlying implementation is an object and you have set that collection variable to point to null, hence the exception.
If on the other hand you initialized collection with:
IEnumerable<string> collection = Enumerable.Empty<string>();
OR
IEnumerable<string> collection = new List<string>();
You would have an empty list object that you could act on. The ArgumentNullException exception is thrown because the collection argument is in fact null and that is what ToArray() is trying to act on. So logically, to me anyways, this was the only design choice.
Edit
On the other hand, in practice, I've made the conscious decision to always return a valid IEnumerable<T> when the return type on my class methods are supposed to return an IEnumerable<T>.
For example; a method signature that looks like IEnumerable<T> GetAll() would always return a valid enumerable and If there was nothing to return then I would return return Enumerable.Empty<T>();
The difference here to me is that GetAll() is not acting on a collection argument. You could really look at this like the collection is really nothing more than a parameter to the method and if you passed in a null parameter to a regular method you would probably throw an ArgumentNullException.
The short answer is that your collection variable is an expected argument (or parameter) to the ToArray() method and it is null so it makes sense to throw an ArgumentNullException.
Why push the error to some other spot in your code? If you expect that something can be null, check it before you use it. Otherwise throw an error exactly at the spot where your assumption was false, namely when you tried to convert it to an array.
I understand it can be debatable when you're chaining operations but in those cases I've found it easier to work with an empty IEnumerable rather than a null one.

Are LINQ-returned IEnumerables "cast-hack" free?

Are IEnumerables returned by LINQ methods such as Select or SelectMany "cast-hack" free? For instance, you can return from a function whose output type is IEnumerable an IList, but if you cast it back to IList you will be able to modify it. Does the same happen with IEnumerables returned by LINQ?
Yes. The LINQ methods return special iterator collection classes that wrap the original data source or employ the yield keyword. The reason is deferred execution.
For example:
Select and Where return an instance of the private class WhereSelectEnumerableIterator<TSource, TResult>.
Except and Distinct use the yield keyword to return the elements from the collections that match the condition.
You can use ILSpy to have a look at this code yourself.

MemberExpression to MemberExpression[]

The objective is to get an array of MemberExpressions from two LambdaExpressions. The first is convertible to a MethodCallExpression that returns the instance of an object (Expression<Func<T>>). The second Lambda expression would take the result of the compiled first expression and return a nested member (Expression<Func<T,TMember>>). We can assume that the second Lambda expression will only make calls to nested properties, but may do several of these calls.
So, the signature of the method I am trying to create is :
MemberExpression[] GetMemberExpressionArray<T,TValue>(Expression<Func<T>> instanceExpression, Expression<Func<T,TValue>> nestedMemberExpression)
where nestedMemberExpression will be assumed to take an argument of the form
parent => parent.ChildProperty.GrandChildProperty
and the resulting array represents the MemberAccess from parent to ChildProperty and from the value of ChildProperty to GrandChildProperty.
I have already returned the last MemberExpression using the following extension method.
public static MemberExpression GetMemberExpression<T, TValue>(Expression<Func<T, TValue>> expression)
{
if (expression == null)
{
return null;
}
if (expression.Body is MemberExpression)
{
return (MemberExpression)expression.Body;
}
if (expression.Body is UnaryExpression)
{
var operand = ((UnaryExpression)expression.Body).Operand;
if (operand is MemberExpression)
{
return (MemberExpression)operand;
}
if (operand is MethodCallExpression)
{
return ((MethodCallExpression)operand).Object as MemberExpression;
}
}
return null;
}
Now, I know there are several ways to accomplish this. The most immediately intuitive to me would be to loop through the .Expression property to get the first expression and capture references to each MemberExpression along the way. This may be the best way to do it, but it may not. I am not extraordinarily familiar with the performance costs I get from using expressions like this. I know a MemberExpression has a MemberInfo and that reflection is supposed to hurt performance.
I've tried to search for information on expressions, but my resources have been very limited in what I've found.
I would appreciate any advice on how to accomplish this task (and this type of task, in general) with optimal performance and reliability.
I'm not sure why this has been tagged performance, but the easiest way I can think of to extract member-expressions from a tree is to subclass ExpressionVisitor. This should be much simpler than manually writing the logic to 'expand' different types of expressions and walk the tree.
You'll probably have to override the VisitMember method so that:
Each member-expression is captured.
Its children are visited.
I imagine that would look something like:
protected override Expression VisitMember(MemberExpression node)
{
_myListOfMemberExpressions.Add(node);
return base.VisitMember(node);
}
I'm slightly unclear about the remainder of your task; it appears like you want to rewrite parameter-expressions, in which case you might want to look at this answer from Marc Gravell.

LINQ Performance

What exactly is happening behind the scenes in a LINQ query against an object collection? Is it just syntactical sugar or is there something else happening making it more of an efficient query?
Do you mean in terms of a query expression, or what the query does behind the scenes?
Query expressions are expanded into "normal" C# first. For example:
var query = from x in source
where x.Name == "Fred"
select x.Age;
is translated to:
var query = source.Where(x => x.Name == "Fred")
.Select(x => x.Age);
The exact meaning of this depends on the type of source of course... in LINQ to Objects, it typically implements IEnumerable<T> and the Enumerable extension methods come into play... but it could be a different set of extension methods. (LINQ to SQL would use the Queryable extension methods, for example.)
Now, suppose we are using LINQ to Objects... after extension method expansion, the above code becomes:
var query = Enumerable.Select(Enumerable.Where(source, x => x.Name == "Fred"),
x => x.Age);
Next the implementations of Select and Where become important. Leaving out error checking, they're something like this:
public static IEnumerable<T> Where<T>(this IEnumerable<T> source,
Func<T, bool> predicate)
{
foreach (T element in source)
{
if (predicate(element))
{
yield return element;
}
}
}
public static IEnumerable<TResult> Select<TSource, TResult>
(this IEnumerable<TSource> source,
Func<TSource, TResult> selector)
{
foreach (TSource element in source)
{
yield return selector(element);
}
}
Next there's the expansion of iterator blocks into state machines, which I won't go into here but which I have an article about.
Finally, there's the conversion of lambda expressions into extra methods + appropriate delegate instance creation (or expression trees, depending on the signatures of the methods called).
So basically LINQ uses a lot of clever features of C#:
Lambda expression conversions (into delegate instances and expression trees)
Extension methods
Type inference for generic methods
Iterator blocks
Often anonymous types (for use in projections)
Often implicit typing for local variables
Query expression translation
However, the individual operations are quite simple - they don't perform indexing etc. Joins and groupings are done using hash tables, but straightforward queries like "where" are just linear. Don't forget that LINQ to Objects usually just treats the data as a forward-only readable sequence - it can't do things like a binary search.
Normally I'd expect hand-written queries to be marginally faster than LINQ to Objects as there are fewer layers of abstraction, but they'll be less readable and the performance difference usually won't be significant.
As ever for performance questions: when in doubt, measure!
If you need better performance, consider trying i4o - Index for Objects. It build in-memory objects for large collections (think 100,000+ rows), which LINQ then uses to speed up queries. You need a lot of data to make this work, but the improvements are impressive.
http://www.codeplex.com/i4o
It's just syntactic sugar - there's no magic involved.
You could write out the equivalent code in "longhand", in C# or whatever, and it would perform the same.
(The compiler will do a good job of producing efficient code, of course, so the code it produces might be a fraction more efficient than the code you would write yourself, simply because you might not know the most performant way to write that code.)

Resources