c# linq combine Expression<Func<T, bool>> predicate in where clause with another condition - linq

Is it possible to construct a where clause like this, where predicate is of type Expression<Func<T, bool>> predicate:
var resultQuery = query.Where(q => !q.IsDeleted && predicate).ToList();
I would like to avoid double where clauses like this:
var resultQuery = query.Where(q => !q.IsDeleted).Where(predicate).ToList();

Instead, you could use custom extension methods and filter result using them:
public static class QueryExtensions
{
public static IQueryable<Image> NonDeleted(this IQueryable<Image> queryable)
{
return queryable.Where(x => !x.Deleted);
}
public static IQueryable<Image> LatestOnly(this IQueryable<Image> queryable)
{
return queryable.Where(x => x.CreateDate <= DateTime.UtcNow.AddDays(-7));
}
}
And then combine them in query:
var result = context
.Images
.NonDeleted()
.LatestOnly()
.ToList();
I like this approach cause it's clean and easy to read. You can also use interfaces in your entities and extension which use those interfaces to quickly filter items based on interfaces that implemented on entity. For example:
public interface ICreationDate{
DateTime CreateDate {get;}
}
public class Image: ICreationDate{
public DateTime CreateDate {get; private set;} = DateTime.UtcNow;
}
Then extension can be changed like this:
public static IQueryable<T> LatestOnly<T>(this IQueryable<T> queryable)
where T : ICreationDate
{
return queryable.Where(x => x.CreateDate <= DateTime.UtcNow.AddDays(-7));
}
This approach gives you more flexibility and reusability.
I know these all are too far from your original question, but it may bring you some alternative aproaches

IT is impossible. You need to provide a lambda expression as a parameter to the Where clause which would be compiled to an expression tree and after that translated into some SQL query. In your example
var resultQuery = query.Where(q => !q.IsDeleted && predicate).ToList();
you are combining a lambda expression and a boolean check. The only way to avoid the double Where clauses is to create a helper function that returns a lambda expression for filtering which includes filtering for the IsDeleted flag and the predicate logic i.e.
private System.Linq.Expressions.Expression<Func<T, bool>> filterPredicate(int n)
{
return q => !q.IsDeleted && q.Age > n;
}
Here we are assuming that
q.Age > n
is the logic of your predicate function. And then use the filter predicates like this:
var resultQuery = query.Where(filterPredicate(5)).ToList();
More about lambda expressions and expression trees you can read here https://learn.microsoft.com/en-us/dotnet/api/system.linq.expressions.expression-1?view=net-5.0

Related

Can I convert a Func<T, bool> to a Func<U, bool> where T and U are POCO classes where I can map properties of one to the other? If so, how?

I have a scenario where a method will take a predicate of type Func< T, bool > because the type T is the one that is exposed to the outer world, but when actually using that predicate I need that method to call another method that will take in Func< U, bool > where properties of T are mapped to properties of U.
A more concrete example would be:
public IEnumerable<ClientEntity> Search(Func<ClientEntity, bool> predicate)
{
IList<ClientEntity> result = new List<ClientEntity>();
// Somehow translate predicate into Func<Client, bool> which I will call realPredicate.
_dataFacade.Clients.Where(realPredicate).ToList().ForEach(c => result.Add(new ClientEntity() { Id = c.Id, Name = c.Name }));
return result.AsEnumerable();
}
Would that be possible?
Please note that ClientEntity is a POCO class that I defined myself while Client is an Entity Framework class created by the model (DB first).
Thanks!
I once attempted this. It resulted in a not-too-bad working expression tree rewriter when the expression tree consist of the simpler operations (equals, larger-then, smaller-then, etc).
It can be found here.
You can use it as:
Expression<Func<Poco1>> where1 = p => p.Name == "fred";
Expression<Func<Poco2>> where2 = ExpressionRewriter.CastParam<Poco1, Poco2>(where1);
EF doesn't use lambdas - it uses Expression Trees
Func<T, bool> lambda = ( o => o.Name == "fred" );
Expression<Func<T, bool>> expressionTree = ( o => o.Name == "fred" );
Expression Trees are in-memory object graphs that represent a given expression.
As they are just objects, you can create or modify them.
Here's another link: MSDN: How to: Modify Expression Trees
What I ended up doing did not require the use of Expression Trees:
public IEnumerable<ClientEntity> Search(Func<ClientEntity, bool> predicate)
{
IList<ClientEntity> result = new List<ClientEntity>();
Func<Client, bool> realPredicate = (c => predicate(ConvertFromClient(c)));
_dataFacade.Clients.Where(realPredicate).ToList().ForEach(c => result.Add(ConvertFromClient(c)));
return result.AsEnumerable();
}
private static ClientEntity ConvertFromClient(Client client)
{
ClientEntity result = new ClientEntity();
if (client != null)
{
// I actually used AutoMapper from http://automapper.org/ here instead of assigning every property.
result.Id = client.Id;
result.Name = client.Name;
}
return result;
}

Generic expression for where clause - "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities."

I am trying to write a really generic way to load EF entities in batches, using the Contains method to generate a SQL IN statement. I've got it working if I pass the entire expression in, but when I try to build the expression dynamically, I am getting a "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities." So I know this means that EF thinks I'm calling an arbitrary method and it can't translate it into SQL, but I can't figure out how to get it to understand the underlying expression.
So If I do something like this (just showing the relevant snippets):
Function declaration:
public static List<T> Load<T>(IQueryable<T> entityQuery, int[] entityIds, Func<T, int> entityKey, int batchSize = 500, Func<T, bool> postFilter = null) where T : EntityObject
{
var retList = new List<T>();
// Append a where clause to the query passed in, that will use a Contains expression, which generates a SQL IN statement. So our SQL looks something like
// WHERE [ItemTypeId] IN (1921,1920,1922)
// See http://rogeralsing.com/2009/05/21/entity-framework-4-where-entity-id-in-array/ for details
Func<int[], Expression<Func<T, bool>>> containsExpression = (entityArray => (expr => entityArray.Contains(entityKey(expr))));
// Build a new query with the current batch of IDs to retrieve and add it to the list we are returning
newQuery = entityQuery.Where<T>(containsExpression(entityIds));
retList.AddRange(newQuery.ToList());
return retList;
}
Call function:
var entities = BatchEntity.Load<ItemType>(from eItemType in dal.Context.InstanceContainer.ItemTypes
select eItemType
, itemTypeData
, (ek => ek.ItemTypeId)
);
I get "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities."
But if I change it to be this:
Function declaration:
public static List<T> Load<T>(IQueryable<T> entityQuery, int[] entityIds, Func<int[], Expression<Func<T, bool>>> containsExpression, int batchSize = 500, Func<T, bool> postFilter = null) where T : EntityObject
{
var retList = new List<T>();
// Build a new query with the current batch of IDs to retrieve and add it to the list we are returning
newQuery = entityQuery.Where<T>(containsExpression(entityIds));
retList.AddRange(newQuery.ToList());
return retList;
}
Call function:
var entities = BatchEntity.Load<ItemType>(from eItemType in dal.Context.InstanceContainer.ItemTypes
select eItemType
, itemTypeData
, (entityArray => (ek => entityArray.Contains(ek.ItemTypeId)))
);
It works fine. Is there any way I can make EF understand the more generic version?
The problem, as you describe, is that the entityKey function in the first example is opaque since it is of type Func rather than Expression. However, you can get the behavior you want by implementing a Compose() method to combine two expressions. I posted the code to implement compose in this question: use Expression<Func<T,X>> in Linq contains extension.
With Compose() implemented, your function can be implemented as below:
public static List<T> Load<T>(this IQueryable<T> entityQuery,
int[] entityIds,
// note that this is an expression now
Expression<Func<T, int>> entityKey,
int batchSize = 500,
Expression<Func<T, bool>> postFilter = null)
where T : EntityObject
{
Expression<Func<int, bool>> containsExpression = id => entityIds.Contains(id);
Expression<Func<T, bool>> whereInEntityIdsExpression = containsExpression.Compose(entityKey);
IQueryable<T> filteredById = entityQuery.Where(whereInEntityIdsExpression);
// if your post filter is compilable to SQL, you might as well do the filtering
// in the database
if (postFilter != null) { filteredById = filteredById.Where(postFilter); }
// finally, pull into memory
return filteredById.ToList();
}

Linq Order by when column name is dynamic and pass as a string to a function

I have a Linq (Entity Framework) Query as
function getData(string col_to_sort , bool IsAscending , int pageNo , int pageSize)
{
context.table_name.Skip(pageNo*pageSize).Take(pageSize).ToArray();
}
What i want is that if i pass the name of the column as a parameter to the function
and the order it will sort my query too.
Since my column name will be a string so we might need to convert it to ObjectQuery.
How can i achieve this?
Any help is appreciated
You can use Dynamic Linq:
string direction = IsAscending ? " ASC" : " DESC";
context.table_name.OrderBy(col_to_sort + direction).Skip(pageNo*pageSize).Take(pageSize).ToArray();
If you are using Dynamic Linq, then the accepted answer will work.
But If you don't want to add an extra library (Dynamic Linq), then you can pick my first approach. I will explain both the approaches where you have or don't have Dynamic Linq. You can select based on your preferences and choice.
First Approach: When you don't have Dynamic Linq:
If you are using using System.Linq; instead of using System.Linq.Dynamic.Core, then you can use this approach:
orderBy is the string and Student is the T (The Entity, in which we want to search).
Create a Utility class, something like this: (you can anytime covert to extension method If you wish.)
public static class LinqUtility
{
public static Expression<Func<T, object>> ToLambda<T>(string propertyName)
{
var parameter = Expression.Parameter(typeof(T));
var property = Expression.Property(parameter, propertyName);
var propAsObject = Expression.Convert(property, typeof(object));
return Expression.Lambda<Func<T, object>>(propAsObject, parameter);
}
}
And you can use like this:
public async Task<IList<Student>> GetStudents(long groupId, string orderBy, Filter filter)
{
return await _context.Students.Where(x => x.StudentGroupId == groupId)
.OrderByDescending(LinqUtility.ToLambda<Student>(orderBy))
.Skip(filter.Skip)
.Take(filter.Take)
.ToListAsync();
}
Second Approach: When you have Dynamic Linq:
The Dynamic LINQ library exposes a set of extension methods on IQueryable corresponding to the standard LINQ methods at Queryable, and which accept strings in a special syntax instead of expression trees.
You need to include the Library separately. Include System.Linq.Dynamic.Core. The Author of this Library is not Microsoft.. As such no harm in using it.
So, this Library, provides you a method, that accepts the string.
You can pass comma separated values as well in string, this can be achieved by above code as well, but some changes would be required.
In the same example, just pass orderBy:
public async Task<IList<Student>> GetStudents(long groupId, string orderBy, Filter filter)
{
return await _context.Students.Where(x => x.StudentGroupId == groupId)
.OrderBy(orderBy)
.Skip(filter.Skip)
.Take(filter.Take)
.ToListAsync();
}

Linq Parsing Error when trying to create seperation of concerns

I am in the middle of a refactoring cycle where I converted some extension methods that used to look like this:
public static IQueryable<Family> FilterOnRoute(this IQueryable<Family> families, WicRoute route)
{
return families.Where(fam => fam.PODs
.Any(pod => pod.Route.RouteID == route.RouteID));
}
to a more fluent implementation like this:
public class SimplifiedFamilyLinqBuilder
{
private IQueryable<Family> _families;
public SimplifiedFamilyLinqBuilder Load(IQueryable<Family> families)
{
_families = families;
return this;
}
public SimplifiedFamilyLinqBuilder OnRoute(WicRoute route)
{
_families = _families.Where(fam => fam.PODs
.Any(pod => pod.Route.RouteID == route.RouteID));
return this;
}
public IQueryable<Family> AsQueryable()
{
return _families;
}
}
which I can call like this: (note this is using Linq-to-Nhibernate)
var families =
new SimplifiedFamilyLinqBuilder()
.Load(session.Query<Family>())
.OnRoute(new WicRoute() {RouteID = 1})
.AsQueryable()
.ToList();
this produces the following SQL which is fine with me at the moment: (of note is that the above Linq is being translated to a SQL Query)
select ... from "Family" family0_
where exists (select pods1_.PODID from "POD" pods1_
inner join Route wicroute2_ on pods1_.RouteID=wicroute2_.RouteID
where family0_.FamilyID=pods1_.FamilyID
and wicroute2_.RouteID=#p0);
#p0 = 1
my next effort in refactoring is to move the query part that deals with the child to another class like this:
public class SimplifiedPODLinqBuilder
{
private IQueryable<POD> _pods;
public SimplifiedPODLinqBuilder Load(IQueryable<POD> pods)
{
_pods = pods;
return this;
}
public SimplifiedPODLinqBuilder OnRoute(WicRoute route)
{
_pods = _pods.Where(pod => pod.Route.RouteID == route.RouteID);
return this;
}
public IQueryable<POD> AsQueryable()
{
return _pods;
}
}
with SimplifiedFamilyLinqBuilder changing to this:
public SimplifiedFamilyLinqBuilder OnRoute(WicRoute route)
{
_families = _families.Where(fam =>
_podLinqBuilder.Load(fam.PODs.AsQueryable())
.OnRoute(route)
.AsQueryable()
.Any()
);
return this;
}
only I now get this error:
Remotion.Linq.Parsing.ParserException : Cannot parse expression 'value(Wic.DataTests.LinqBuilders.SimplifiedPODLinqBuilder)' as it has an unsupported type. Only query sources (that is, expressions that implement IEnumerable) and query operators can be parsed.
I started to implement IQueryable on SimplifiedPODLinqBuilder(as that seemed more logical than implementing IEnumberable) and thought I would be clever by doing this:
public class SimplifiedPODLinqBuilder : IQueryable
{
private IQueryable<POD> _pods;
...
public IEnumerator GetEnumerator()
{
return _pods.GetEnumerator();
}
public Expression Expression
{
get { return _pods.Expression; }
}
public Type ElementType
{
get { return _pods.ElementType; }
}
public IQueryProvider Provider
{
get { return _pods.Provider; }
}
}
only to get this exception (apparently Load is not being called and _pods is null):
System.NullReferenceException : Object reference not set to an instance of an object.
is there a way for me to refactor this code out that will parse properly into an expression that will go to SQL?
The part fam => _podLinqBuilder.Load(fam.PODs.AsQueryable() is never going to work, because the linq provider will try to parse this into SQL and for that it needs mapped members of Family after the =>, or maybe a mapped user-defined function but I don't know if Linq-to-Nhibernate supports that (I never really worked with it, because I still doubt if it is production-ready).
So, what can you do?
To be honest, I like the extension methods much better. You switched to a stateful approach, which doesn't mix well with the stateless paradigm of linq. So you may consider to retrace your steps.
Another option: the expression in .Any(pod => pod.Route.RouteID == route.RouteID)); could be paremeterized (.Any(podExpression), with
OnRoute(WicRoute route, Expression<Func<POD,bool>> podExpression)
(pseudocode).
Hope this makes any sense.
You need to separate methods you intend to call from expressions you intend to translate.
This is great, you want each of those methods to run. They return an instance that implements IQueryable<Family> and operate on that instance.
var families = new SimplifiedFamilyLinqBuilder()
.Load(session.Query<Family>())
.OnRoute(new WicRoute() {RouteID = 1})
.AsQueryable()
.ToList();
This is no good. you don't want Queryable.Where to get called, you want it to be an expression tree which can be translated to SQL. But PodLinqBuilder.Load is a node in that expression tree which can't be translated to SQL!
families = _families
.Where(fam => _podLinqBuilder.Load(fam.PODs.AsQueryable())
.OnRoute(route)
.AsQueryable()
.Any();
You can't call .Load inside the Where expression (it won't translate to sql).
You can't call .Load outside the Where expression (you don't have the fam parameter).
In the name of "separation of concerns", you are mixing query construction methods with query definition expressions. LINQ, by its Integrated nature, encourages you to attempt this thing which will not work.
Consider making expression construction methods instead of query construction methods.
public static Expression<Func<Pod, bool>> GetOnRouteExpr(WicRoute route)
{
int routeId = route.RouteID;
Expression<Func<Pod, bool>> result = pod => pod.Route.RouteID == route.RouteID;
return result;
}
called by:
Expression<Func<Pod, bool>> onRoute = GetOnRouteExpr(route);
families = _families.Where(fam => fam.PODs.Any(onRoute));
With this approach, the question is now - how do I fluidly hang my ornaments from the expression tree?

Dynamic LINQ OR Conditions

I'm looking to use LINQ to do multiple where conditions on a collection similar to
IEnumerable<Object> items;
items.Where(p => p.FirstName = "John");
items.Where(p => p.LastName = "Smith");
except for rather than having multiple AND conditions (as with this example), I'd like to have multiple OR conditions.
EDIT
Sorry, to clarify I don't know how many of these conditions I will have so
items.Where(p => p.FirstName = "John" || p => p.LastName = "Smith")
won't work.
Basically, here's what I'm trying to do:
foreach(var name in names)
{
items = items.Where(p => p.Name == name);
}
Use PredicateBuilder:
Suppose you want to write a LINQ to SQL or Entity Framework query that implements a keyword-style search. In other words, a query that returns rows whose description contains some or all of a given set of keywords...
The ideal approach is to dynamically construct a lambda expression tree that performs an or-based predicate.
Of all the things that will drive you to manually constructing expression trees, the need for dynamic predicates is the most common in a typical business application. Fortunately, it’s possible to write a set of simple and reusable extension methods that radically simplify this task. This is the role of our PredicateBuilder class...
It sounds like your whitelist of names is only known at runtime. Perhaps try this:
string[] names = new string[] {"John", "foo", "bar"};
var matching = items.Where(x => names.Contains(x.Name));
You can use .Union() to return results that satisfy any condition.
var results = items.Where(p => p.FirstName == "John")
.Union(items.Where(p => p.LastName == "Smith"));
This is inferior to using the || operator. It isn't clear from your edit why that wouldn't work.
public static Expression<Func<T, bool>> OrTheseFiltersTogether<T>(
this IEnumerable<Expression<Func<T, bool>>> filters)
{
Expression<Func<T, bool>> firstFilter = filters.FirstOrDefault();
if (firstFilter == null)
{
Expression<Func<T, bool>> alwaysTrue = x => true;
return alwaysTrue;
}
var body = firstFilter.Body;
var param = firstFilter.Parameters.ToArray();
foreach (var nextFilter in filters.Skip(1))
{
var nextBody = Expression.Invoke(nextFilter, param);
body = Expression.OrElse(body, nextBody);
}
Expression<Func<T, bool>> result = Expression.Lambda<Func<T, bool>>(body, param);
return result;
}
Then, later:
List<Expression<Func<Person, bool>>> filters = names
.Select<string, Expression<Func<Person, bool>>>(name =>
p => p.Name == name
).ToList();
Expression<Func<Person, bool>> filterOfOrs = filters.OrTheseFiltersTogether();
query = query.Where<Person>(filterOfOrs);
You can't make the Where clause dynamic, but you can dynamically create the Lambda Expression you pass to it. Create the right Expression, compile it and pass the resulting lambda expression as a parameter to the Where clause.
EDIT:
Okay, seems like you can skip the part where you have to manually create the Expression and can use PredicateBuilder for it, as already answered by AS-CII.

Resources