I have this linq query (using linq 4.3.0):
return ExecuteODataQuery(Db.AdvisorFees, f => f.Advisor);
Implemented as follows:
protected IQueryable<TType> ExecuteODataQuery<TType>(IQueryable<TType> queryFunc, Func<TType, Advisor> advisorFunc) where TType : Entity
{
var systemAccount = GetSystemAccountThrow();
var results = queryFunc.Where(x => x.Active).AsEnumerable().Where(x => CheckAdvisorOrChannel(systemAccount, advisorFunc(x)));
return results;
}
This is my custom method:
public static bool CheckAdvisorOrChannel(SystemAccount systemAccount, Advisor advisor)
{
return (systemAccount.IsChannelSystemAccount ? systemAccount.Channel.Code == advisor.Channel.Code : systemAccount.Advisor.Code == advisor.Code);
}
This currently works. If I remove the AsEnumerable() I get a LinqToEntities not supported exception on my custom method. I cannot use AsEnumerable() as I have too many records and do not want to do the rest of the filtering in memory. Is it possible to call a custom method on IQueryable instead? I've been playin with extension methods and expressions but have not managed to get this working yet. Any pointers?
I would prefer to not use expression trees as much as possible.
Related
I am using the repository pattern with Entity Framework as described in this article: repository pattern with Entity Framework
In the part where the GenericRepository is described (Generic Repository) there is a method which is used to get entities from the database set called Get. It has an orderBy but no groupBy. I am wondering how one might implement a groupBy in the same manner as the orderBy so that you can specify which field to group by dynamically on the entity.
What I have come up with is this:
Func<IQueryable<TEntity>, IGrouping<string, TEntity>> groupBy = null
and then in the method code it should be used something like this:
if(groupBy != null)
{
query = groupBy(query).ToList();
}
But this is not compiling since the IGrouping is not queryable. Does someone know how to point me in the right direction or has a solution to this?
Edit: The reason for doing this instead of using groupby on the returned list is for performance reasons. I want the groupby to be sent as an sql statement to the database and resolved there.
Grouping has no sense without projection. So you have to define new method which returns IEnumerable with new type.
I have added sample of such method. Also removed includeProperties because EF Core ignores Includes during grouping.
Usage sample:
_orderRepostory
.GetGrouped(e => e.UserId, g => new { UserId = g.Key, Count = g.Count()});
And implementation:
public class GenericRepository<TEntity> where TEntity : class
{
... // other code
public virtual IEnumerable<TResult> GetGrouped<TKey, TResult>(
Expression<Func<TEntity, TKey>> groupingKey,
Expression<Func<IGrouping<TKey, TEntity>, TResult>> resultSelector,
Expression<Func<TEntity, bool>>? filter = null)
{
var query = dbSet.AsQueryable();
if (filter != null)
{
query = query.Where(filter);
}
return query.GroupBy(groupingKey).Select(resultSelector);
}
}
Inspired by this post dynamic-repositories-in-lightspeed I am trying to build my own like this.
I have a abstract GenericRepository like this. I have omitted most of the code for simplicity (Its just normal Add/Update/Filtering methods).
public abstract class GenericRepository<TEntity, TContext> :
DynamicObject,
IDataRepository<TEntity>
where TEntity : class, new()
where TContext : DbContext, new()
{
protected TContext context;
protected DbSet<TEntity> DbSet;
}
As you can see, my abstract GenericRepository extends from DynamicObject to support dynamic repositories.
I also have a abstract UnitOfWork implementation which generated a repository for a given entity at runtime like this. Again, base classes and other details are irrelevant for the question, but I'm happy to provide them if you require.
public abstract class UnitOfWorkBase<TContext> : IUnitOfWork
where TContext : DbContext, new()
{
public abstract IDataRepository<T> Repository<T>()
where T : class, IIdentifiableEntity, new();
// Code
}
Following class implements abstract method of the above class.
public class MyUnitOfWorkBase : UnitOfWorkBase<MyDataContext>
{
public override IDataRepository<T> Repository<T>()
{
if (Repositories == null)
Repositories = new Hashtable();
var type = typeof(T).Name;
if (!Repositories.ContainsKey(type))
{
var repositoryType = typeof(GenericRepositoryImpl<,>);
var genericType = repositoryType.MakeGenericType(typeof(T), typeof(InTeleBillContext));
var repositoryInstance = Activator.CreateInstance(genericType);
Repositories.Add(type, repositoryInstance);
}
return (IDataRepository<T>)Repositories[type];
}
}
Now, whenever I want to create a dynamic repository for basic CRUD functions, I can do it like this.
var uow = new MyUnitOfWorkBase();
var settingsRepo = uow.Repository<Settings>();
var settingsList = settingsRepo.Get().ToList();
Now, What I want to do is something like this.
dynamic settingsRepo = uow.Repository<Settings>();
var result = settingsRepo.FindSettingsByCustomerNumber(774278L);
Here, FindSettingsByCustomerNumber() is a dynamic method. I resolve this method using this code.
public class GenericRepositoryImpl<TEntity, TContext> :
GenericRepository<TEntity, TContext>
where TEntity : class, IIdentifiableEntity, new()
where TContext : DbContext, new()
{
public override bool TryInvokeMember(InvokeMemberBinder binder,
object[] args, out object result)
{
// Crude parsing for simplicity
if (binder.Name.StartsWith("Find"))
{
int byIndex = binder.Name.IndexOf("By");
if (byIndex >= 0)
{
string collectionName = binder.Name.Substring(4, byIndex - 4);
string[] attributes = binder.Name.Substring(byIndex + 2)
.Split(new[] { "And" }, StringSplitOptions.None);
var items = DbSet.ToList();
Func<TEntity, bool> predicate = entity => entity.GetType().GetProperty(attributes[0]).GetValue(entity).Equals(args[0]);
result = items.Where(predicate).ToList();
return true;
}
}
return base.TryInvokeMember(binder, args, out result);
}
}
This is the problem I am having.
using this line var items = DbSet.ToList(); works well, but if I were to query a large table with 1000's of data, then performance issues occur.
If I directly try to use the IQueryble interface and call it like this
Func predicate = entity => entity.GetType().GetProperty(attributes[0]).GetValue(entity).Equals(args[0]);
result = DbSet.Where(predicate).ToList();
It gives me an error saying there is no method GetProperty() in LINQ to Entities.
Is it possible to make it work using LINQ to Entities?
You need to know that LINQ-to-Entities needs to convert your expression (given by the predicate) into a SQL query. entity is replaced by the database column. Additionally LINQ2Entities supports various expressions (e.g. EqualExpression, etc.). However it cannot support the whole .NET Framework. Especially: what should GetType() on a database column return?
Therefore you need to use the Expresson API to generate the predicate and use only expressions supported by LINQ2Entities. For example: Use a MemberAccess expression for accessing a property (LINQ2Entities is able to map that to an SQL query).
Hint: we are doing predicate generation for Entity Framework and had to overcome some additional problems which we could solve using the library LinqKit.
If you do not know about the .NET Expression API yet, you need to gather skills in that area before you can resume your dynamic repository idea.
BTW: I don't think that it is a very good idea to have this kind of automatic calls. They are not refactoring safe (i.e. what if you rename the DB column? All your method calls run into problems, and it is not detectable by the compiler).
I would use it only to generate predicates for Where() clauses from Filter-like DTO types.
Unusual pattern - dynamic methods on a repository patterns.But that is another topic.
Dynamic invocation of the repository you have.
So now you need to understand Linq to Entities a little more.
Linq to Entities language reference what you can do with linq to Entities.
Given the expression tree has to be converted in to DB instructions,
it isnt surprising there are restrictions.
In case you are interested The EF provider specs and links to samples
So given you want to Dynamic EF, you have a few options.
I concentrate on dynamic wheres, but you can apply to other EF methods.
Check out
Dynamic Linq on codeplex
which allows things like
public virtual IQueryable<TPoco> DynamicWhere(string predicate, params object[] values) {
return Context.Set<TPoco>().Where(predicate, values);
}
This Where is an IQueryable extension that accepts strings...
Samples of using this string based predicate parser
LinqKit or even PM> Install-Package LinqKit
Linqkit takes dynamic EF to the next level,
Offers amazing features like
public IQueryable<TPoco> AsExpandable() {
return Context.Set<TPoco>().AsExpandable();
}
which allows you build AND and ORs progressively.
Expression trees
Expression Building API is the most powerful tool to support you here .
Learning the API is hard. using the tool harder.
eg Dealing with concatenation very hard. BUT if you can understand the API and how expressions work.
It is possible.
Here is a SIMPLE example. (imagine something complex)
public static Expression<Func<TPoco, bool>> GetContainsPredicate<TPoco>(string propertyName,
string containsValue)
{
// (tpoco t) => t.propertyName.Contains(value ) is built
var parameterExp = Expression.Parameter(typeof(TPoco), #"t");
var propertyExp = Expression.Property(parameterExp, propertyName);
MethodInfo method = typeof(string).GetMethod(#"Contains", new[] { typeof(string) });
var someValue = Expression.Constant(containsValue, typeof(string));
var containsMethodExp = Expression.Call(propertyExp, method, someValue);
return Expression.Lambda<Func<TPoco, bool>>(containsMethodExp, parameterExp);
}
How to use string extension method in linq query:
public NewsType GetNewsType(string name)
{
var newsType = db.NewsTypes.FirstOrDefault(x => x.Name.ToFriendlyUrl() ==
name.ToFriendlyUrl());
return newsType;
}
Above query x.Name.ToFriendlyUrl() is not allowed at the minute. Is anyone know how to achieve with it.
Extension methods are indeed allowed in LINQ queries, moreover the LINQ methods themselves are implemented as extension methods.
It's quite another issue however, to use extension methods (or most other methods) in LINQ-to-SQL or LINQ-to-Entities queries. Those queries are not actually run in the C# code, but they are treated like expressions that are translated to SQL. I.e.
db.News.Where(x => x.Published).Select(x => x.Name)
is translated to the SQL Statement
Select Name
From News
Where Published = 1
and it's results are returned to the C# code.
Since there is not way to transfer the ToFriendlyUrl() method to SQL, your code throws an error.
You have basically, two solutions/workarounds. One is to transform the call to a form could be translated into SQL, e.g. if the ToFriendlyUrl() method was just:
public static string ToFriendlyURL(this string value)
{
return value.ToLower();
}
you can inline that code in the LINQ call, and that would work. If however, the methods is more complex, than your only solution is to just fetch the data from the base and then process it on the C# side:
var newsTypeQuery = db.NewsTypes.Where(x => // other conditions, if any);
var newsTypes = newsTypes.ToList(); //forces execution of the query
// the result is now a C# list
var newsType = newsTypes.FirstOrDefault(x =>
x.Name.ToFriendlyUrl() == name.ToFriendlyUrl());
Assuming the NewsTypes is an IQueryable this is a result of Entity Framework not being able to convert you extension method into SQL (how should it?). Unless you can rewrite your predicate into something that Entity Framework can translate into SQL you will have to perform the query client side:
public NewsType GetNewsType(string name)
{
var newsType = db.NewsTypes.AsEnumerable().FirstOrDefault(x => x.Name.ToFriendlyUrl() == name.ToFriendlyUrl());
return newsType;
}
Notice how AsEnumerable() has been added before FirstOrDefault. Unfortunately this may pull all the rows returned by NewsTypes from the server to client and thus may be quite costly.
This
var newsType = db.NewsTypes.FirstOrDefault(
x => x.Name.ToFriendlyUrl() == name.ToFriendlyUrl());
can't be done in Entity Framework. ToFriendlyUrl is an extension method. It's something that is in the "client" computer. The query will be executed on the SQL server. The SQL server doesn't have a ToFriendlyUrl function.
The "standard" solution is to save in a second column named FriendlyName a precalculated version of the ToFriendlyUrl(), so your query becomes:
var friendlyName = name.ToFriendlyUrl();
var newsType = db.NewsTypes.FirstOrDefault(
x => x.FriendlyName == friendlyName);
Instead try like this
public NewsType GetNewsType(string name)
{
var newsType = db.NewsTypes.FirstOrDefault(x => x.Name == name).ToFriendlyUrl();
return newsType;
}
I am in the middle of a refactoring cycle where I converted some extension methods that used to look like this:
public static IQueryable<Family> FilterOnRoute(this IQueryable<Family> families, WicRoute route)
{
return families.Where(fam => fam.PODs
.Any(pod => pod.Route.RouteID == route.RouteID));
}
to a more fluent implementation like this:
public class SimplifiedFamilyLinqBuilder
{
private IQueryable<Family> _families;
public SimplifiedFamilyLinqBuilder Load(IQueryable<Family> families)
{
_families = families;
return this;
}
public SimplifiedFamilyLinqBuilder OnRoute(WicRoute route)
{
_families = _families.Where(fam => fam.PODs
.Any(pod => pod.Route.RouteID == route.RouteID));
return this;
}
public IQueryable<Family> AsQueryable()
{
return _families;
}
}
which I can call like this: (note this is using Linq-to-Nhibernate)
var families =
new SimplifiedFamilyLinqBuilder()
.Load(session.Query<Family>())
.OnRoute(new WicRoute() {RouteID = 1})
.AsQueryable()
.ToList();
this produces the following SQL which is fine with me at the moment: (of note is that the above Linq is being translated to a SQL Query)
select ... from "Family" family0_
where exists (select pods1_.PODID from "POD" pods1_
inner join Route wicroute2_ on pods1_.RouteID=wicroute2_.RouteID
where family0_.FamilyID=pods1_.FamilyID
and wicroute2_.RouteID=#p0);
#p0 = 1
my next effort in refactoring is to move the query part that deals with the child to another class like this:
public class SimplifiedPODLinqBuilder
{
private IQueryable<POD> _pods;
public SimplifiedPODLinqBuilder Load(IQueryable<POD> pods)
{
_pods = pods;
return this;
}
public SimplifiedPODLinqBuilder OnRoute(WicRoute route)
{
_pods = _pods.Where(pod => pod.Route.RouteID == route.RouteID);
return this;
}
public IQueryable<POD> AsQueryable()
{
return _pods;
}
}
with SimplifiedFamilyLinqBuilder changing to this:
public SimplifiedFamilyLinqBuilder OnRoute(WicRoute route)
{
_families = _families.Where(fam =>
_podLinqBuilder.Load(fam.PODs.AsQueryable())
.OnRoute(route)
.AsQueryable()
.Any()
);
return this;
}
only I now get this error:
Remotion.Linq.Parsing.ParserException : Cannot parse expression 'value(Wic.DataTests.LinqBuilders.SimplifiedPODLinqBuilder)' as it has an unsupported type. Only query sources (that is, expressions that implement IEnumerable) and query operators can be parsed.
I started to implement IQueryable on SimplifiedPODLinqBuilder(as that seemed more logical than implementing IEnumberable) and thought I would be clever by doing this:
public class SimplifiedPODLinqBuilder : IQueryable
{
private IQueryable<POD> _pods;
...
public IEnumerator GetEnumerator()
{
return _pods.GetEnumerator();
}
public Expression Expression
{
get { return _pods.Expression; }
}
public Type ElementType
{
get { return _pods.ElementType; }
}
public IQueryProvider Provider
{
get { return _pods.Provider; }
}
}
only to get this exception (apparently Load is not being called and _pods is null):
System.NullReferenceException : Object reference not set to an instance of an object.
is there a way for me to refactor this code out that will parse properly into an expression that will go to SQL?
The part fam => _podLinqBuilder.Load(fam.PODs.AsQueryable() is never going to work, because the linq provider will try to parse this into SQL and for that it needs mapped members of Family after the =>, or maybe a mapped user-defined function but I don't know if Linq-to-Nhibernate supports that (I never really worked with it, because I still doubt if it is production-ready).
So, what can you do?
To be honest, I like the extension methods much better. You switched to a stateful approach, which doesn't mix well with the stateless paradigm of linq. So you may consider to retrace your steps.
Another option: the expression in .Any(pod => pod.Route.RouteID == route.RouteID)); could be paremeterized (.Any(podExpression), with
OnRoute(WicRoute route, Expression<Func<POD,bool>> podExpression)
(pseudocode).
Hope this makes any sense.
You need to separate methods you intend to call from expressions you intend to translate.
This is great, you want each of those methods to run. They return an instance that implements IQueryable<Family> and operate on that instance.
var families = new SimplifiedFamilyLinqBuilder()
.Load(session.Query<Family>())
.OnRoute(new WicRoute() {RouteID = 1})
.AsQueryable()
.ToList();
This is no good. you don't want Queryable.Where to get called, you want it to be an expression tree which can be translated to SQL. But PodLinqBuilder.Load is a node in that expression tree which can't be translated to SQL!
families = _families
.Where(fam => _podLinqBuilder.Load(fam.PODs.AsQueryable())
.OnRoute(route)
.AsQueryable()
.Any();
You can't call .Load inside the Where expression (it won't translate to sql).
You can't call .Load outside the Where expression (you don't have the fam parameter).
In the name of "separation of concerns", you are mixing query construction methods with query definition expressions. LINQ, by its Integrated nature, encourages you to attempt this thing which will not work.
Consider making expression construction methods instead of query construction methods.
public static Expression<Func<Pod, bool>> GetOnRouteExpr(WicRoute route)
{
int routeId = route.RouteID;
Expression<Func<Pod, bool>> result = pod => pod.Route.RouteID == route.RouteID;
return result;
}
called by:
Expression<Func<Pod, bool>> onRoute = GetOnRouteExpr(route);
families = _families.Where(fam => fam.PODs.Any(onRoute));
With this approach, the question is now - how do I fluidly hang my ornaments from the expression tree?
I am using Entity Framework 4.0 and trying to use the "Contains" function of one the object sets in my context object. to do so i coded a Comparer class:
public class RatingInfoComparer : IEqualityComparer<RatingInfo>
{
public bool Equals(RatingInfo x, RatingInfo y)
{
var a = new {x.PlugInID,x.RatingInfoUserIP};
var b = new {y.PlugInID,y.RatingInfoUserIP};
if(a.PlugInID == b.PlugInID && a.RatingInfoUserIP.Equals(b.RatingInfoUserIP))
return true;
else
return false;
}
public int GetHashCode(RatingInfo obj)
{
var a = new { obj.PlugInID, obj.RatingInfoUserIP };
if (Object.ReferenceEquals(obj, null))
return 0;
return a.GetHashCode();
}
}
when i try to use the comparer with this code:
public void SaveRatingInfo2(int plugInId, string userInfo)
{
RatingInfo ri = new RatingInfo()
{
PlugInID = plugInId,
RatingInfoUser = userInfo,
RatingInfoUserIP = "192.168.1.100"
};
//This is where i get the execption
if (!context.RatingInfoes.Contains<RatingInfo>(ri, new RatingInfoComparer()))
{
//my Entity Framework context object
context.RatingInfoes.AddObject(ri);
context.SaveChanges();
}
}
i get an execption:
"LINQ to Entities does not recognize the method 'Boolean Contains[RatingInfo](System.Linq.IQueryable1[OlafCMSLibrary.Models.RatingInfo], OlafCMSLibrary.Models.RatingInfo,
System.Collections.Generic.IEqualityComparer1[OlafCMSLibrary.Models.RatingInfo])' method, and his method cannot be translated into a store expression."
Since i am not proficient with linQ and Entity Framework i might be making a mistake with my use of the "var" either in the "GetHashCode" function or in general.
If my mistake is clear to you do tell me :) it does not stop my project! but it is essential for me to understand why a simple comparer doesnt work.
Thanks
Aaron
LINQ to Entities works by converting an expression tree into queries against an object model through the IQueryable interface. This means than you can only put things into the expression tree which LINQ to Entities understands.
It doesn't understand the Contains method you are using, so it throws the exception you see. Here is a list of methods which it understands.
Under the Set Methods section header, it lists Contains using an item as supported, but it lists Contains with an IEqualityComparer as not supported. This is presumably because it would have to be able to work out how to convert your IEqualityComparer into a query against the object model, which would be difficult. You might be able to do what you want using multiple Where clauses, see which ones are supported further up the document.