Using an IEqualityComparer with a LINQ to Entities Except clause - linq

I have an entity that I'd like to compare with a subset and determine to select all except the subset.
So, my query looks like this:
Products.Except(ProductsToRemove(), new ProductComparer())
The ProductsToRemove() method returns a List<Product> after it performs a few tasks. So in it's simplest form it's the above.
The ProductComparer() class looks like this:
public class ProductComparer : IEqualityComparer<Product>
{
public bool Equals(Product a, Product b)
{
if (ReferenceEquals(a, b)) return true;
if (ReferenceEquals(a, null) || ReferenceEquals(b, null))
return false;
return a.Id == b.Id;
}
public int GetHashCode(Product product)
{
if (ReferenceEquals(product, null)) return 0;
var hashProductId = product.Id.GetHashCode();
return hashProductId;
}
}
However, I continually receive the following exception:
LINQ to Entities does not recognize
the method
'System.Linq.IQueryable1[UnitedOne.Data.Sql.Product]
Except[Product](System.Linq.IQueryable1[UnitedOne.Data.Sql.Product],
System.Collections.Generic.IEnumerable1[UnitedOne.Data.Sql.Product],
System.Collections.Generic.IEqualityComparer1[UnitedOne.Data.Sql.Product])'
method, and this method cannot be
translated into a store expression.

Linq to Entities isn't actually executing your query, it is interpreting your code, converting it to TSQL, then executing that on the server.
Under the covers, it is coded with the knowledge of how operators and common functions operate and how those relate to TSQL. The problem is that the developers of L2E have no idea how exactly you are implementing IEqualityComparer. Therefore they cannot figure out that when you say Class A == Class B you mean (for example) "Where Person.FirstName == FirstName AND Person.LastName == LastName".
So, when the L2E interpreter hits a method it doesn't recognize, it throws this exception.
There are two ways you can work around this. First, develop a Where() that satisfies your equality requirements but that doesn't rely on any custom method. In other words, test for equality of properties of the instance rather than an Equals method defined on the class.
Second, you can trigger the execution of the query and then do your comparisons in memory. For instance:
var notThisItem = new Item{Id = "HurrDurr"};
var items = Db.Items.ToArray(); // Sql query executed here
var except = items.Except(notThisItem); // performed in memory
Obviously this will bring much more data across the wire and be more memory intensive. The first option is usually the best.

You're trying to convert the Except call with your custom IEqualityComparer into Entity SQL.
Obviously, your class cannot be converted into SQL.
You need to write Products.AsEnumerable().Except(ProductsToRemove(), new ProductComparer()) to force it to execute on the client. Note that this will download all of the products from the server.
By the way, your ProductComparer class should be a singleton, like this:
public class ProductComparer : IEqualityComparer<Product> {
private ProductComparer() { }
public static ProductComparer Instance = new ProductComparer();
...
}

The IEqualityComparer<T> can only be executed locally, it can't be translated to a SQL command, hence the error

Related

Creating a group by parameter for the get method in a generic repository for entity framework

I am using the repository pattern with Entity Framework as described in this article: repository pattern with Entity Framework
In the part where the GenericRepository is described (Generic Repository) there is a method which is used to get entities from the database set called Get. It has an orderBy but no groupBy. I am wondering how one might implement a groupBy in the same manner as the orderBy so that you can specify which field to group by dynamically on the entity.
What I have come up with is this:
Func<IQueryable<TEntity>, IGrouping<string, TEntity>> groupBy = null
and then in the method code it should be used something like this:
if(groupBy != null)
{
query = groupBy(query).ToList();
}
But this is not compiling since the IGrouping is not queryable. Does someone know how to point me in the right direction or has a solution to this?
Edit: The reason for doing this instead of using groupby on the returned list is for performance reasons. I want the groupby to be sent as an sql statement to the database and resolved there.
Grouping has no sense without projection. So you have to define new method which returns IEnumerable with new type.
I have added sample of such method. Also removed includeProperties because EF Core ignores Includes during grouping.
Usage sample:
_orderRepostory
.GetGrouped(e => e.UserId, g => new { UserId = g.Key, Count = g.Count()});
And implementation:
public class GenericRepository<TEntity> where TEntity : class
{
... // other code
public virtual IEnumerable<TResult> GetGrouped<TKey, TResult>(
Expression<Func<TEntity, TKey>> groupingKey,
Expression<Func<IGrouping<TKey, TEntity>, TResult>> resultSelector,
Expression<Func<TEntity, bool>>? filter = null)
{
var query = dbSet.AsQueryable();
if (filter != null)
{
query = query.Where(filter);
}
return query.GroupBy(groupingKey).Select(resultSelector);
}
}

Dynamic Repository With Dynamic Methods

Inspired by this post dynamic-repositories-in-lightspeed I am trying to build my own like this.
I have a abstract GenericRepository like this. I have omitted most of the code for simplicity (Its just normal Add/Update/Filtering methods).
public abstract class GenericRepository<TEntity, TContext> :
DynamicObject,
IDataRepository<TEntity>
where TEntity : class, new()
where TContext : DbContext, new()
{
protected TContext context;
protected DbSet<TEntity> DbSet;
}
As you can see, my abstract GenericRepository extends from DynamicObject to support dynamic repositories.
I also have a abstract UnitOfWork implementation which generated a repository for a given entity at runtime like this. Again, base classes and other details are irrelevant for the question, but I'm happy to provide them if you require.
public abstract class UnitOfWorkBase<TContext> : IUnitOfWork
where TContext : DbContext, new()
{
public abstract IDataRepository<T> Repository<T>()
where T : class, IIdentifiableEntity, new();
// Code
}
Following class implements abstract method of the above class.
public class MyUnitOfWorkBase : UnitOfWorkBase<MyDataContext>
{
public override IDataRepository<T> Repository<T>()
{
if (Repositories == null)
Repositories = new Hashtable();
var type = typeof(T).Name;
if (!Repositories.ContainsKey(type))
{
var repositoryType = typeof(GenericRepositoryImpl<,>);
var genericType = repositoryType.MakeGenericType(typeof(T), typeof(InTeleBillContext));
var repositoryInstance = Activator.CreateInstance(genericType);
Repositories.Add(type, repositoryInstance);
}
return (IDataRepository<T>)Repositories[type];
}
}
Now, whenever I want to create a dynamic repository for basic CRUD functions, I can do it like this.
var uow = new MyUnitOfWorkBase();
var settingsRepo = uow.Repository<Settings>();
var settingsList = settingsRepo.Get().ToList();
Now, What I want to do is something like this.
dynamic settingsRepo = uow.Repository<Settings>();
var result = settingsRepo.FindSettingsByCustomerNumber(774278L);
Here, FindSettingsByCustomerNumber() is a dynamic method. I resolve this method using this code.
public class GenericRepositoryImpl<TEntity, TContext> :
GenericRepository<TEntity, TContext>
where TEntity : class, IIdentifiableEntity, new()
where TContext : DbContext, new()
{
public override bool TryInvokeMember(InvokeMemberBinder binder,
object[] args, out object result)
{
// Crude parsing for simplicity
if (binder.Name.StartsWith("Find"))
{
int byIndex = binder.Name.IndexOf("By");
if (byIndex >= 0)
{
string collectionName = binder.Name.Substring(4, byIndex - 4);
string[] attributes = binder.Name.Substring(byIndex + 2)
.Split(new[] { "And" }, StringSplitOptions.None);
var items = DbSet.ToList();
Func<TEntity, bool> predicate = entity => entity.GetType().GetProperty(attributes[0]).GetValue(entity).Equals(args[0]);
result = items.Where(predicate).ToList();
return true;
}
}
return base.TryInvokeMember(binder, args, out result);
}
}
This is the problem I am having.
using this line var items = DbSet.ToList(); works well, but if I were to query a large table with 1000's of data, then performance issues occur.
If I directly try to use the IQueryble interface and call it like this
Func predicate = entity => entity.GetType().GetProperty(attributes[0]).GetValue(entity).Equals(args[0]);
result = DbSet.Where(predicate).ToList();
It gives me an error saying there is no method GetProperty() in LINQ to Entities.
Is it possible to make it work using LINQ to Entities?
You need to know that LINQ-to-Entities needs to convert your expression (given by the predicate) into a SQL query. entity is replaced by the database column. Additionally LINQ2Entities supports various expressions (e.g. EqualExpression, etc.). However it cannot support the whole .NET Framework. Especially: what should GetType() on a database column return?
Therefore you need to use the Expresson API to generate the predicate and use only expressions supported by LINQ2Entities. For example: Use a MemberAccess expression for accessing a property (LINQ2Entities is able to map that to an SQL query).
Hint: we are doing predicate generation for Entity Framework and had to overcome some additional problems which we could solve using the library LinqKit.
If you do not know about the .NET Expression API yet, you need to gather skills in that area before you can resume your dynamic repository idea.
BTW: I don't think that it is a very good idea to have this kind of automatic calls. They are not refactoring safe (i.e. what if you rename the DB column? All your method calls run into problems, and it is not detectable by the compiler).
I would use it only to generate predicates for Where() clauses from Filter-like DTO types.
Unusual pattern - dynamic methods on a repository patterns.But that is another topic.
Dynamic invocation of the repository you have.
So now you need to understand Linq to Entities a little more.
Linq to Entities language reference what you can do with linq to Entities.
Given the expression tree has to be converted in to DB instructions,
it isnt surprising there are restrictions.
In case you are interested The EF provider specs and links to samples
So given you want to Dynamic EF, you have a few options.
I concentrate on dynamic wheres, but you can apply to other EF methods.
Check out
Dynamic Linq on codeplex
which allows things like
public virtual IQueryable<TPoco> DynamicWhere(string predicate, params object[] values) {
return Context.Set<TPoco>().Where(predicate, values);
}
This Where is an IQueryable extension that accepts strings...
Samples of using this string based predicate parser
LinqKit or even PM> Install-Package LinqKit
Linqkit takes dynamic EF to the next level,
Offers amazing features like
public IQueryable<TPoco> AsExpandable() {
return Context.Set<TPoco>().AsExpandable();
}
which allows you build AND and ORs progressively.
Expression trees
Expression Building API is the most powerful tool to support you here .
Learning the API is hard. using the tool harder.
eg Dealing with concatenation very hard. BUT if you can understand the API and how expressions work.
It is possible.
Here is a SIMPLE example. (imagine something complex)
public static Expression<Func<TPoco, bool>> GetContainsPredicate<TPoco>(string propertyName,
string containsValue)
{
// (tpoco t) => t.propertyName.Contains(value ) is built
var parameterExp = Expression.Parameter(typeof(TPoco), #"t");
var propertyExp = Expression.Property(parameterExp, propertyName);
MethodInfo method = typeof(string).GetMethod(#"Contains", new[] { typeof(string) });
var someValue = Expression.Constant(containsValue, typeof(string));
var containsMethodExp = Expression.Call(propertyExp, method, someValue);
return Expression.Lambda<Func<TPoco, bool>>(containsMethodExp, parameterExp);
}

Writing dynamic LINQ queries as closure methods

I'm trying to write a C# repository with atomic contexts and feel like this is a perfect situation for the usage of a closure, but I can't quite grok how to get it done in C#. I have this as a main method in my repository:
...
protected virtual IQueryable<T> AsQueryable()
{
return _context.ObjectSet<T>().AsQueryable();
}
...
Meanwhile, I have derived classes with methods like:
...
public IQueryable<Arc> ByRun(Run run)
{
IQueryable<Arc> query = from o in AsQueryable()
from r in o.Runs
where r.Id == run.Id
select o;
return query;
}
...
and I want to change my query method to return IEnumerable and to dispose quickly of the context, so want to use (something like) this:
...
protected virtual IEnumerable<T> AsEnumerable()
{
using (IContextUnitOfWork unitOfWork = new EFUnitOfWork())
{
return unitOfWork.ObjectSet<T>().ToList();
}
}
...
The problem, of course, is that once the context is disposed, calling LINQ on the resulting IEnumerable set will fail. Thus, my thought is that I should bundle up the ByRun() method and pass it to AsEnumerable() to be used as a closure.
While not my original language style, I learned closures in Ruby. There, what I'm trying to do would look something like this mixed up pseudo-code:
ByRun(Run run)
AsEnumerable do |query|
from o in query
from r in o.Runs
where r.Id == run.Id
select o;
end
end
where the AsEnumerable method would open the context, perform the operation that was passed in, and return. I'm sure I can do this once I understand the syntax, so I'm looking for my desired AsEnumerable and ByRun methods implemented this way.
If I understand the question correctly, you want to have a wrapper on any query to ensure that AsEnumerable is called at the end and context is disposed just after the query?
If so (assuming that your base class is generic with T parameter), try this:
protected virtual IEnumerable<T> AsEnumerable(Func<ObjectSet<T>, IQueryable<T>> query)
{
using (IContextUnitOfWork unitOfWork = new EFUnitOfWork())
{
return query(unitOfWork.ObjectSet<T>()).AsEnumerable();
}
}
And usage example:
public IEnumerable<Arc> ByRun(Run run)
{
return AsEnumerable(query => from o in query
from r in o.Runs
where r.Id == run.Id
select o);
}
The parameter of AsEnumerable here is the lambda expression containing any delegate that takes ObjectSet<T> as the only parameter and returns IQueryable<T>. So it's logically equivalent to have the following code in the derived class:
public IEnumerable<Arc> ByRun(Run run)
{
using (IContextUnitOfWork unitOfWork = new EFUnitOfWork())
{
return (from o in unitOfWork.ObjectSet<T>()
from r in o.Runs
where r.Id == run.Id
select o).AsEnumerable();
}
}

Entity Framework 4.1 simple dynamic expression for object.property = value

I know there is a way to use Expressions and Lambdas to accomplish this but I having a hard time piecing it all together. All I need is a method that will dynamically query an Entity Framework DBSet object to find the row where the propery with the given name matches the value.
My context:
public class MyContext : DbContext
{
public IDbSet<Account> Accoounts{ get { return Set<Account>(); } }
}
The method that I'm looking to write:
public T Get<T>(string property, object value) : where T is Account
{...}
I would rather not have to use Dynamic SQL to accomplish this so no need to suggest it because I already know it's possible. What I'm really looking for is some help to accomplish this using Expressions and Lambdas
Thanks in advance, I know it's brief but it should be pretty self-explanatory. Comment if more info is needed
I'm trying to avoid dynamic linq as much as possible because the main point of linq is strongly typed access. Using dynamic linq is a solution but it is exactly the oppose of the linq purpose and it is quite close to using ESQL and building the query from sting concatenation. Anyway dynamic linq is sometimes real time saver (especially when it comes to complex dynamic ordering) and I successfully use it in a large project with Linq-to-Sql.
What I usually do is defining some SearchCriteria class like:
public class SearchCriteria
{
public string Property1 { get; set; }
public int? Property2 { get; set; }
}
And helper query extension method like:
public static IQueryable<SomeClass> Filter(this IQueryable<SomeClass> query, SearchCriteria filter)
{
if (filter.Property1 != null) query = query.Where(s => s.Property1 == filter.Property1);
if (filter.Property2 != null) query = query.Where(s => s.Property2 == filter.Property2);
return query;
}
It is not generic solution. Again generic solution is for some strongly typed processing of classes sharing some behavior.
The more complex solution would be using predicate builder and build expression tree yourselves but again building expression tree is only more complex way to build ESQL query by concatenating strings.
Here's my implementation:
public T Get<T>(string property, object value) : where T is Account
{
//p
var p = Expression.Parameter(typeof(T));
//p.Property
var propertyExpression = Expression.Property(p, property);
//p.Property == value
var equalsExpression = Expression.Equal(propertyExpression, Expression.Constant(value));
//p => p.Property == value
var lambda = Expression.Lambda<Func<T,bool>>(equalsExpression, p);
return context.Set<T>().SingleOrDefault(lambda);
}
It uses EF 5's Set<T>() method. If you are using a lower version, you'll need to implement a way of getting the DbSet based on the <T> type.
Hope it helps.
Dynamic Linq may be an option. Specify your criteria as a string and it will get built as an expression and ran against your data;
An example from something I have done;
var context = new DataContext(ConfigurationManager.ConnectionStrings["c"].ConnectionString);
var statusConditions = "Status = 1";
var results = (IQueryable)context.Contacts.Where(statusConditions);
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx

Alternative to use a method in a query to the database in c# using entity framework

i separated my application into a DAL, BL, UI.
I used entity framework code first throw repositories to access the sql database.
public class Person{
...
}
public class PersonRepository{
Create(...){...}
Update(...){...}
Delete(...){...}
GetById(...){...}
Query(...){...}
...
Now the thing is the BL i'm working on a method to get all the Persons who are leaving near an adress
public GetPersonsNear(string Address){
...
}
private bool AddressesAreClose(string address1, string address2)
{
...
}
the thing is linq does'nt let me use my method (in a query passed in the "Query" method of the repository)
...
PersonRepository personRepository = new PersonRepository();
var person = repository.Query(p => AddressAreClose(adress,p.Adress);
...
therefor i needed to get All the elements of the table in a list using a simple foreach loop to make the tests and keeping only the relevant ones
...
PersonRepository personRepository = new PersonRepository();
var persons = personRepository.GetAll;
foreach(person in persons)
{
if(AdressAreClose(adress,person.adress))
...
}
for now i populated the database with only a few elements to test it, but i'm not sure it would work very well with the far more greater number it will contain later specially with all the test i'm planing to add
isn't there a more clever way to do this ??? I'm open to anything
Well first of all, you should use generics in your repository, even if it's constrained to Person. This way you can build pipes/filters off your queries to clean up your LINQ queries and facilitate reuse.
Of course, without seeing the full signature/implementation of your Query method, it's hard to tell. But either way, you need to return IEnumerable<Person> or IQueryable<Person> to make the following work.
So, you could turn AddressesAreClose into a pipe/filter, like this:
public static bool WhereAddressesAreClose(this IQueryable<Person> source, string address)
{
return source.Where(/* your conditions */);
}
Then you can use it in your LINQ query:
var person = repository
.Query() // Should be IQueryable<Person>
.WhereAddressAreClose(adress);
.ToList();
Depending on the size of your data and whether or not your implementing caching, you should limit the results on the server (database), not post-query with a foreach loop.
If the performance isn't great, consider adding indexes, using compiled queries or moving to a stored procedure.

Resources