I have a scenario where a method will take a predicate of type Func< T, bool > because the type T is the one that is exposed to the outer world, but when actually using that predicate I need that method to call another method that will take in Func< U, bool > where properties of T are mapped to properties of U.
A more concrete example would be:
public IEnumerable<ClientEntity> Search(Func<ClientEntity, bool> predicate)
{
IList<ClientEntity> result = new List<ClientEntity>();
// Somehow translate predicate into Func<Client, bool> which I will call realPredicate.
_dataFacade.Clients.Where(realPredicate).ToList().ForEach(c => result.Add(new ClientEntity() { Id = c.Id, Name = c.Name }));
return result.AsEnumerable();
}
Would that be possible?
Please note that ClientEntity is a POCO class that I defined myself while Client is an Entity Framework class created by the model (DB first).
Thanks!
I once attempted this. It resulted in a not-too-bad working expression tree rewriter when the expression tree consist of the simpler operations (equals, larger-then, smaller-then, etc).
It can be found here.
You can use it as:
Expression<Func<Poco1>> where1 = p => p.Name == "fred";
Expression<Func<Poco2>> where2 = ExpressionRewriter.CastParam<Poco1, Poco2>(where1);
EF doesn't use lambdas - it uses Expression Trees
Func<T, bool> lambda = ( o => o.Name == "fred" );
Expression<Func<T, bool>> expressionTree = ( o => o.Name == "fred" );
Expression Trees are in-memory object graphs that represent a given expression.
As they are just objects, you can create or modify them.
Here's another link: MSDN: How to: Modify Expression Trees
What I ended up doing did not require the use of Expression Trees:
public IEnumerable<ClientEntity> Search(Func<ClientEntity, bool> predicate)
{
IList<ClientEntity> result = new List<ClientEntity>();
Func<Client, bool> realPredicate = (c => predicate(ConvertFromClient(c)));
_dataFacade.Clients.Where(realPredicate).ToList().ForEach(c => result.Add(ConvertFromClient(c)));
return result.AsEnumerable();
}
private static ClientEntity ConvertFromClient(Client client)
{
ClientEntity result = new ClientEntity();
if (client != null)
{
// I actually used AutoMapper from http://automapper.org/ here instead of assigning every property.
result.Id = client.Id;
result.Name = client.Name;
}
return result;
}
Related
Is it possible to construct a where clause like this, where predicate is of type Expression<Func<T, bool>> predicate:
var resultQuery = query.Where(q => !q.IsDeleted && predicate).ToList();
I would like to avoid double where clauses like this:
var resultQuery = query.Where(q => !q.IsDeleted).Where(predicate).ToList();
Instead, you could use custom extension methods and filter result using them:
public static class QueryExtensions
{
public static IQueryable<Image> NonDeleted(this IQueryable<Image> queryable)
{
return queryable.Where(x => !x.Deleted);
}
public static IQueryable<Image> LatestOnly(this IQueryable<Image> queryable)
{
return queryable.Where(x => x.CreateDate <= DateTime.UtcNow.AddDays(-7));
}
}
And then combine them in query:
var result = context
.Images
.NonDeleted()
.LatestOnly()
.ToList();
I like this approach cause it's clean and easy to read. You can also use interfaces in your entities and extension which use those interfaces to quickly filter items based on interfaces that implemented on entity. For example:
public interface ICreationDate{
DateTime CreateDate {get;}
}
public class Image: ICreationDate{
public DateTime CreateDate {get; private set;} = DateTime.UtcNow;
}
Then extension can be changed like this:
public static IQueryable<T> LatestOnly<T>(this IQueryable<T> queryable)
where T : ICreationDate
{
return queryable.Where(x => x.CreateDate <= DateTime.UtcNow.AddDays(-7));
}
This approach gives you more flexibility and reusability.
I know these all are too far from your original question, but it may bring you some alternative aproaches
IT is impossible. You need to provide a lambda expression as a parameter to the Where clause which would be compiled to an expression tree and after that translated into some SQL query. In your example
var resultQuery = query.Where(q => !q.IsDeleted && predicate).ToList();
you are combining a lambda expression and a boolean check. The only way to avoid the double Where clauses is to create a helper function that returns a lambda expression for filtering which includes filtering for the IsDeleted flag and the predicate logic i.e.
private System.Linq.Expressions.Expression<Func<T, bool>> filterPredicate(int n)
{
return q => !q.IsDeleted && q.Age > n;
}
Here we are assuming that
q.Age > n
is the logic of your predicate function. And then use the filter predicates like this:
var resultQuery = query.Where(filterPredicate(5)).ToList();
More about lambda expressions and expression trees you can read here https://learn.microsoft.com/en-us/dotnet/api/system.linq.expressions.expression-1?view=net-5.0
I am writing a method in which i am using async prog.
var tasks = new List<Task<List<SomeClass>>>();
tasks.Add(this.Method1());
tasks.Add(this.Method2());
var results = await Task.WhenAll(tasks).ConfigureAwait(false);
i want distinct records from this result. How to achieve that.
currently i have written
return results.SelectMany(s => s).GroupBy(x => x.Name).Select(x => x.FirstOrDefault()).ToList();
But i am not sure with SelectMany, will this give correct result.
SelectMany(s => s) is a "flatten" operation. It takes a sequence of sequences and flattens them to a single sequence.
The LINQ "distinct" operator is called Distinct. If your SomeClass overrides equality to be based on Name, then that's all you need:
return results.SelectMany(s => s).Distinct().ToList();
But if SomeClass doesn't define equality that way, you'll need to do another kind of distinct.
One option is to use the Distinct overload that takes an equality comparer. Then you can pass in an equality comparer that determines equality by Name. To do this, first define an equality comparer:
public sealed class NameEqualityComparer: IEqualityComparer<SomeClass>
{
public int GetHashCode(SomeClass obj) => EqualityComparer<string>.Default.GetHashCode(obj.Name);
public bool Equals(SomeClass x, SomeClass y) => EqualityComparer<string>.Default.Equals(x.Name, y.Name);
}
and then you can invoke the correct overload:
return results.SelectMany(s => s).Distinct(new NameEqualityComparer()).ToList();
I have a library that helps define custom comparers (properly handling all edge cases), which I prefer to use for things like this. With the Nito.Comparers library, you don't need to define a custom NameEqualityComparer; instead, you can define comparers in-line like this:
return results.SelectMany(s => s).Distinct(b => b.EquateBy(x => x.Name)).ToList();
or separately, if desired:
var comparer = EqualityComparerBuilder.For<SomeClass>().EquateBy(x => x.Name);
return results.SelectMany(s => s).Distinct(comparer).ToList();
A completely different option is to add a new "Distinct-By" operator that acts the way you want. This is part of MoreLINQ or you can add it yourself:
public static IEnumerable<T> DistinctBy<T, TKey>(this IEnumerable<T> #this, Func<T, TKey> selector)
{
var keys = new HashSet<TKey>();
foreach (var item in #this)
{
if (keys.Add(selector(item)))
yield return item;
}
}
Then you can use the new operator like this:
return results.SelectMany(s => s).DistinctBy(x => x.Name).ToList();
All of these options are more efficient than grouping.
I'm currently working in a project that involves the encryption of a few columns in an existing database. There is quite a lot of code already written against the current schema, a lot of which is in the form of custom linq-to-sql queries. The number of queries is in the neighbourhood of a 5 figure number, so modifying and re-testing each and everyone of them would be way too expensive.
An alternative we found is to keep the DB schema the same --only altering the columns length slightly, which mean we don't need to change our current entity class definitions-- and instead, changing the expression trees on-the-fly, before they reach the l2sql IQueryProvider, and apply a decryption function on the columns I need. I do this by wrapping the pertinent Table<TEntity> properties of my DataContext with a custom IQueryable<TEntity> implementation, which allows me to preview every single query in the system.
In my current implementation, say I've got this query:
var mydate = new DateTime(2013, 1, 1);
var context = new DataContextFactory.GetClientsContext();
Expression<Func<string>> foo = context.MyClients.First(
c => c.BirthDay < mydate).EncryptedColumn;
but when I catch the query, I change it to read:
Expression<Func<string>> foo = context.Decrypt(
context.MyClients.First(c => c.BirthDay < mydate).EncryptedColumn);
I do this using the ExpressionVisitor class. In the VisitMember method, I check and see whether the current MemberExpression refers to an encrypted column. If it does, I substitute the expression for a method call:
private const string FuncName = "Decrypt";
protected override Expression VisitMember(MemberExpression ma)
{
if (datactx != null && IsEncryptedColumnReference(ma))
return MakeCallExpression(ma);
}
return base.VisitMember(ma);
}
private static bool IsEncryptedColumnReference(MemberExpression ma)
{
return ma.Member.Name == "EncryptedColumn"
&& ma.Member.DeclaringType == typeof(MyClient);
}
private Expression MakeCallExpression(MemberExpression ma)
{
const BindingFlags flags = BindingFlags.Instance | BindingFlags.Public;
var mi = typeof(MyDataContext).GetMethod(FuncName, flags);
return Expression.Call(datactx, mi, ma);
}
datactx is an instance variable with a reference to the expression pointing at the current datacontext (which I look up in a previous pass).
My problem is that if I have a query such as:
var qbeClient = new MyClient { EncryptedColumn = "FooBar" };
Expression<Func<MyClient>> dbquery = () => context.MyClients.First(
c => c.EncryptedColumn == qbeClient.EncryptedColumn);
I want it to be turned into:
Expression<Func<MyClient>> dbquery = () => context.MyClients.First(c =>
context.Decrypt(c.EncryptedColumn) == qbeClient.EncryptedColumn);
instead, what I'm getting is this:
Expression<Func<MyClient>> dbquery = () => context.MyClients.First(c =>
context.Decrypt(c.EncryptedColumn) == context.Decrypt(qbeClient.EncryptedColumn));
Which I don't want, because when I've got an in-memory object, the data is already unencrypted (besides, I don't want a nasty db function call against my objects!)
So, that's basically my question: Having a MemberExpression instance, how can I determine whether it refers to an in-memory object or a row in the database?
Thanks in advance
Edit:
#Shlomo's code actually solves the case I posted, but now one of my previous tests got broken:
var context = new DataContextFactory.GetClientsContext();
Expression<Func<string>> expr = context.MyClients.First().EncryptedColumn;
Expression<Func<string>> expected = context.Decrypt(
context.MyClients.First().EncryptedColumn);
var actual = MyVisitor.Visit(expr);
Assert.AreEqual(expected.ToString(), actual.ToString());
In this case, the reference to EncryptedColumn isn't a parameter, but it should definitely be taken into account by the visitor!
A MemberExpression representing a DB row will be a descendent of a ParameterExpression. In-Memory objects will not, they'll most likely come from some form of a FieldExpression.
In your case, something like this will work for most cases (adding one method to your code, and revising your VisitMember method:
private bool IsFromParameter(MemberExpression ma)
{
if(ma.Expression.NodeType == ExpressionType.Parameter)
return true;
if(ma.Expression is MemberExpression)
return IsFromParameter(ma.Expression as MemberExpression);
return false;
}
protected override Expression VisitMember(MemberExpression ma)
{
if (datactx != null && IsEncryptedColumnReference(ma) && IsFromParameter(ma))
return MakeCallExpression(ma);
}
return base.VisitMember(ma);
}
I am trying to write a really generic way to load EF entities in batches, using the Contains method to generate a SQL IN statement. I've got it working if I pass the entire expression in, but when I try to build the expression dynamically, I am getting a "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities." So I know this means that EF thinks I'm calling an arbitrary method and it can't translate it into SQL, but I can't figure out how to get it to understand the underlying expression.
So If I do something like this (just showing the relevant snippets):
Function declaration:
public static List<T> Load<T>(IQueryable<T> entityQuery, int[] entityIds, Func<T, int> entityKey, int batchSize = 500, Func<T, bool> postFilter = null) where T : EntityObject
{
var retList = new List<T>();
// Append a where clause to the query passed in, that will use a Contains expression, which generates a SQL IN statement. So our SQL looks something like
// WHERE [ItemTypeId] IN (1921,1920,1922)
// See http://rogeralsing.com/2009/05/21/entity-framework-4-where-entity-id-in-array/ for details
Func<int[], Expression<Func<T, bool>>> containsExpression = (entityArray => (expr => entityArray.Contains(entityKey(expr))));
// Build a new query with the current batch of IDs to retrieve and add it to the list we are returning
newQuery = entityQuery.Where<T>(containsExpression(entityIds));
retList.AddRange(newQuery.ToList());
return retList;
}
Call function:
var entities = BatchEntity.Load<ItemType>(from eItemType in dal.Context.InstanceContainer.ItemTypes
select eItemType
, itemTypeData
, (ek => ek.ItemTypeId)
);
I get "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities."
But if I change it to be this:
Function declaration:
public static List<T> Load<T>(IQueryable<T> entityQuery, int[] entityIds, Func<int[], Expression<Func<T, bool>>> containsExpression, int batchSize = 500, Func<T, bool> postFilter = null) where T : EntityObject
{
var retList = new List<T>();
// Build a new query with the current batch of IDs to retrieve and add it to the list we are returning
newQuery = entityQuery.Where<T>(containsExpression(entityIds));
retList.AddRange(newQuery.ToList());
return retList;
}
Call function:
var entities = BatchEntity.Load<ItemType>(from eItemType in dal.Context.InstanceContainer.ItemTypes
select eItemType
, itemTypeData
, (entityArray => (ek => entityArray.Contains(ek.ItemTypeId)))
);
It works fine. Is there any way I can make EF understand the more generic version?
The problem, as you describe, is that the entityKey function in the first example is opaque since it is of type Func rather than Expression. However, you can get the behavior you want by implementing a Compose() method to combine two expressions. I posted the code to implement compose in this question: use Expression<Func<T,X>> in Linq contains extension.
With Compose() implemented, your function can be implemented as below:
public static List<T> Load<T>(this IQueryable<T> entityQuery,
int[] entityIds,
// note that this is an expression now
Expression<Func<T, int>> entityKey,
int batchSize = 500,
Expression<Func<T, bool>> postFilter = null)
where T : EntityObject
{
Expression<Func<int, bool>> containsExpression = id => entityIds.Contains(id);
Expression<Func<T, bool>> whereInEntityIdsExpression = containsExpression.Compose(entityKey);
IQueryable<T> filteredById = entityQuery.Where(whereInEntityIdsExpression);
// if your post filter is compilable to SQL, you might as well do the filtering
// in the database
if (postFilter != null) { filteredById = filteredById.Where(postFilter); }
// finally, pull into memory
return filteredById.ToList();
}
I'm looking to use LINQ to do multiple where conditions on a collection similar to
IEnumerable<Object> items;
items.Where(p => p.FirstName = "John");
items.Where(p => p.LastName = "Smith");
except for rather than having multiple AND conditions (as with this example), I'd like to have multiple OR conditions.
EDIT
Sorry, to clarify I don't know how many of these conditions I will have so
items.Where(p => p.FirstName = "John" || p => p.LastName = "Smith")
won't work.
Basically, here's what I'm trying to do:
foreach(var name in names)
{
items = items.Where(p => p.Name == name);
}
Use PredicateBuilder:
Suppose you want to write a LINQ to SQL or Entity Framework query that implements a keyword-style search. In other words, a query that returns rows whose description contains some or all of a given set of keywords...
The ideal approach is to dynamically construct a lambda expression tree that performs an or-based predicate.
Of all the things that will drive you to manually constructing expression trees, the need for dynamic predicates is the most common in a typical business application. Fortunately, it’s possible to write a set of simple and reusable extension methods that radically simplify this task. This is the role of our PredicateBuilder class...
It sounds like your whitelist of names is only known at runtime. Perhaps try this:
string[] names = new string[] {"John", "foo", "bar"};
var matching = items.Where(x => names.Contains(x.Name));
You can use .Union() to return results that satisfy any condition.
var results = items.Where(p => p.FirstName == "John")
.Union(items.Where(p => p.LastName == "Smith"));
This is inferior to using the || operator. It isn't clear from your edit why that wouldn't work.
public static Expression<Func<T, bool>> OrTheseFiltersTogether<T>(
this IEnumerable<Expression<Func<T, bool>>> filters)
{
Expression<Func<T, bool>> firstFilter = filters.FirstOrDefault();
if (firstFilter == null)
{
Expression<Func<T, bool>> alwaysTrue = x => true;
return alwaysTrue;
}
var body = firstFilter.Body;
var param = firstFilter.Parameters.ToArray();
foreach (var nextFilter in filters.Skip(1))
{
var nextBody = Expression.Invoke(nextFilter, param);
body = Expression.OrElse(body, nextBody);
}
Expression<Func<T, bool>> result = Expression.Lambda<Func<T, bool>>(body, param);
return result;
}
Then, later:
List<Expression<Func<Person, bool>>> filters = names
.Select<string, Expression<Func<Person, bool>>>(name =>
p => p.Name == name
).ToList();
Expression<Func<Person, bool>> filterOfOrs = filters.OrTheseFiltersTogether();
query = query.Where<Person>(filterOfOrs);
You can't make the Where clause dynamic, but you can dynamically create the Lambda Expression you pass to it. Create the right Expression, compile it and pass the resulting lambda expression as a parameter to the Where clause.
EDIT:
Okay, seems like you can skip the part where you have to manually create the Expression and can use PredicateBuilder for it, as already answered by AS-CII.