Dynamic LINQ OR Conditions - linq

I'm looking to use LINQ to do multiple where conditions on a collection similar to
IEnumerable<Object> items;
items.Where(p => p.FirstName = "John");
items.Where(p => p.LastName = "Smith");
except for rather than having multiple AND conditions (as with this example), I'd like to have multiple OR conditions.
EDIT
Sorry, to clarify I don't know how many of these conditions I will have so
items.Where(p => p.FirstName = "John" || p => p.LastName = "Smith")
won't work.
Basically, here's what I'm trying to do:
foreach(var name in names)
{
items = items.Where(p => p.Name == name);
}

Use PredicateBuilder:
Suppose you want to write a LINQ to SQL or Entity Framework query that implements a keyword-style search. In other words, a query that returns rows whose description contains some or all of a given set of keywords...
The ideal approach is to dynamically construct a lambda expression tree that performs an or-based predicate.
Of all the things that will drive you to manually constructing expression trees, the need for dynamic predicates is the most common in a typical business application. Fortunately, it’s possible to write a set of simple and reusable extension methods that radically simplify this task. This is the role of our PredicateBuilder class...

It sounds like your whitelist of names is only known at runtime. Perhaps try this:
string[] names = new string[] {"John", "foo", "bar"};
var matching = items.Where(x => names.Contains(x.Name));

You can use .Union() to return results that satisfy any condition.
var results = items.Where(p => p.FirstName == "John")
.Union(items.Where(p => p.LastName == "Smith"));
This is inferior to using the || operator. It isn't clear from your edit why that wouldn't work.

public static Expression<Func<T, bool>> OrTheseFiltersTogether<T>(
this IEnumerable<Expression<Func<T, bool>>> filters)
{
Expression<Func<T, bool>> firstFilter = filters.FirstOrDefault();
if (firstFilter == null)
{
Expression<Func<T, bool>> alwaysTrue = x => true;
return alwaysTrue;
}
var body = firstFilter.Body;
var param = firstFilter.Parameters.ToArray();
foreach (var nextFilter in filters.Skip(1))
{
var nextBody = Expression.Invoke(nextFilter, param);
body = Expression.OrElse(body, nextBody);
}
Expression<Func<T, bool>> result = Expression.Lambda<Func<T, bool>>(body, param);
return result;
}
Then, later:
List<Expression<Func<Person, bool>>> filters = names
.Select<string, Expression<Func<Person, bool>>>(name =>
p => p.Name == name
).ToList();
Expression<Func<Person, bool>> filterOfOrs = filters.OrTheseFiltersTogether();
query = query.Where<Person>(filterOfOrs);

You can't make the Where clause dynamic, but you can dynamically create the Lambda Expression you pass to it. Create the right Expression, compile it and pass the resulting lambda expression as a parameter to the Where clause.
EDIT:
Okay, seems like you can skip the part where you have to manually create the Expression and can use PredicateBuilder for it, as already answered by AS-CII.

Related

using linq to find if a text field contains any string in a list

im running this in asp.net core v3.1
my question is similar to this question:
How to use Linq to check if a list of strings contains any string in a list
with the specific question relating to the first answer such that
filterTags = ["abc", "cd", "efg"]
var results = db.People
.Where(p => filterTags.Any(tag => p.Tags.Contains(tag)));
so basically saying
give me results from the db of all People
who's Tags field contains any of the filterTags
where Tags = a big text field populated by a bunch of space-delimited tags
This seems straightforward (esp since this has been written before)
but i get an error back
System.InvalidOperationException: The LINQ expression 'DbSet
.Where(p => __filterTags_0
.Any(tag => p.Tags.Contains(tag)))' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync()
does anyone know what this means or what im doing wrong?
This is not possible with pure EF LINQ. You have to create helper which transforms your search list in Expression Tree.
public static class QueryExtensions
{
private static MethodInfo _containsMethodInfo = typeof(string).GetMethod("Contains")!;
public static IQueryable<T> FilterUsingContains<T>(this IQueryable<T> query, Expression<Func<T, string>> prop, IList<string> items)
{
if (items.Count == 0)
return query.Where(e => 1 == 2);
var param = prop.Parameters[0];
var predicate = items.Select(i =>
(Expression)Expression.Call(prop.Body, _containsMethodInfo, Expression.Constant(i, typeof(string))))
.Aggregate(Expression.OrElse);
var lambda = Expression.Lambda<Func<T, bool>>(predicate, param);
return query.Where(lambda);
}
}
Then you can use this extension in your queries
filterTags = ["abc", "cd", "efg"]
var results = db.People
.Where(p => p.Tags.AsQueryable().FilterUsingContains(t => t, filterTags).Any());
Here is a workaround for you:
using System.Linq;
string[] filterTags = {"abc", "cd", "efg"};
var results = db.People.Where(p => filterTags.Contains(p.Tags)).ToList();

Can I convert a Func<T, bool> to a Func<U, bool> where T and U are POCO classes where I can map properties of one to the other? If so, how?

I have a scenario where a method will take a predicate of type Func< T, bool > because the type T is the one that is exposed to the outer world, but when actually using that predicate I need that method to call another method that will take in Func< U, bool > where properties of T are mapped to properties of U.
A more concrete example would be:
public IEnumerable<ClientEntity> Search(Func<ClientEntity, bool> predicate)
{
IList<ClientEntity> result = new List<ClientEntity>();
// Somehow translate predicate into Func<Client, bool> which I will call realPredicate.
_dataFacade.Clients.Where(realPredicate).ToList().ForEach(c => result.Add(new ClientEntity() { Id = c.Id, Name = c.Name }));
return result.AsEnumerable();
}
Would that be possible?
Please note that ClientEntity is a POCO class that I defined myself while Client is an Entity Framework class created by the model (DB first).
Thanks!
I once attempted this. It resulted in a not-too-bad working expression tree rewriter when the expression tree consist of the simpler operations (equals, larger-then, smaller-then, etc).
It can be found here.
You can use it as:
Expression<Func<Poco1>> where1 = p => p.Name == "fred";
Expression<Func<Poco2>> where2 = ExpressionRewriter.CastParam<Poco1, Poco2>(where1);
EF doesn't use lambdas - it uses Expression Trees
Func<T, bool> lambda = ( o => o.Name == "fred" );
Expression<Func<T, bool>> expressionTree = ( o => o.Name == "fred" );
Expression Trees are in-memory object graphs that represent a given expression.
As they are just objects, you can create or modify them.
Here's another link: MSDN: How to: Modify Expression Trees
What I ended up doing did not require the use of Expression Trees:
public IEnumerable<ClientEntity> Search(Func<ClientEntity, bool> predicate)
{
IList<ClientEntity> result = new List<ClientEntity>();
Func<Client, bool> realPredicate = (c => predicate(ConvertFromClient(c)));
_dataFacade.Clients.Where(realPredicate).ToList().ForEach(c => result.Add(ConvertFromClient(c)));
return result.AsEnumerable();
}
private static ClientEntity ConvertFromClient(Client client)
{
ClientEntity result = new ClientEntity();
if (client != null)
{
// I actually used AutoMapper from http://automapper.org/ here instead of assigning every property.
result.Id = client.Id;
result.Name = client.Name;
}
return result;
}

Generic expression for where clause - "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities."

I am trying to write a really generic way to load EF entities in batches, using the Contains method to generate a SQL IN statement. I've got it working if I pass the entire expression in, but when I try to build the expression dynamically, I am getting a "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities." So I know this means that EF thinks I'm calling an arbitrary method and it can't translate it into SQL, but I can't figure out how to get it to understand the underlying expression.
So If I do something like this (just showing the relevant snippets):
Function declaration:
public static List<T> Load<T>(IQueryable<T> entityQuery, int[] entityIds, Func<T, int> entityKey, int batchSize = 500, Func<T, bool> postFilter = null) where T : EntityObject
{
var retList = new List<T>();
// Append a where clause to the query passed in, that will use a Contains expression, which generates a SQL IN statement. So our SQL looks something like
// WHERE [ItemTypeId] IN (1921,1920,1922)
// See http://rogeralsing.com/2009/05/21/entity-framework-4-where-entity-id-in-array/ for details
Func<int[], Expression<Func<T, bool>>> containsExpression = (entityArray => (expr => entityArray.Contains(entityKey(expr))));
// Build a new query with the current batch of IDs to retrieve and add it to the list we are returning
newQuery = entityQuery.Where<T>(containsExpression(entityIds));
retList.AddRange(newQuery.ToList());
return retList;
}
Call function:
var entities = BatchEntity.Load<ItemType>(from eItemType in dal.Context.InstanceContainer.ItemTypes
select eItemType
, itemTypeData
, (ek => ek.ItemTypeId)
);
I get "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities."
But if I change it to be this:
Function declaration:
public static List<T> Load<T>(IQueryable<T> entityQuery, int[] entityIds, Func<int[], Expression<Func<T, bool>>> containsExpression, int batchSize = 500, Func<T, bool> postFilter = null) where T : EntityObject
{
var retList = new List<T>();
// Build a new query with the current batch of IDs to retrieve and add it to the list we are returning
newQuery = entityQuery.Where<T>(containsExpression(entityIds));
retList.AddRange(newQuery.ToList());
return retList;
}
Call function:
var entities = BatchEntity.Load<ItemType>(from eItemType in dal.Context.InstanceContainer.ItemTypes
select eItemType
, itemTypeData
, (entityArray => (ek => entityArray.Contains(ek.ItemTypeId)))
);
It works fine. Is there any way I can make EF understand the more generic version?
The problem, as you describe, is that the entityKey function in the first example is opaque since it is of type Func rather than Expression. However, you can get the behavior you want by implementing a Compose() method to combine two expressions. I posted the code to implement compose in this question: use Expression<Func<T,X>> in Linq contains extension.
With Compose() implemented, your function can be implemented as below:
public static List<T> Load<T>(this IQueryable<T> entityQuery,
int[] entityIds,
// note that this is an expression now
Expression<Func<T, int>> entityKey,
int batchSize = 500,
Expression<Func<T, bool>> postFilter = null)
where T : EntityObject
{
Expression<Func<int, bool>> containsExpression = id => entityIds.Contains(id);
Expression<Func<T, bool>> whereInEntityIdsExpression = containsExpression.Compose(entityKey);
IQueryable<T> filteredById = entityQuery.Where(whereInEntityIdsExpression);
// if your post filter is compilable to SQL, you might as well do the filtering
// in the database
if (postFilter != null) { filteredById = filteredById.Where(postFilter); }
// finally, pull into memory
return filteredById.ToList();
}

How to create a collection of Expression<Func<T, TRelated>>?

I have a repository with the following method:
IEnumerable<T> FindAll<TRelated>(Specification<T> specification,
Expression<Func<T, TRelated>> fetchExpression);
I need to pass in more than one expression. I was thinking about changing the signature to:
IEnumerable<T> FindAll<TRelated>(Specification<T> specification,
IEnumerable<Expression<Func<T, TRelated>>> fetchExpression);
Is this possible?
How do I create an array, say, of expressions to pass into this method?
Currently I'm calling the method from my service layer like this:
var products = productRepository.FindAll(specification,p => p.Variants);
But I'd like to pass p => p.Variants and p => p.Reviews for example. And then in the repository I'd like to iterate through the expression and add them to a query.
For a bit of background on why I am doing this see Ben Foster's blog post on Eager loading with NHibernate.
You could use params to do it:
IEnumerable<T> FindAll(Specification<T> specification,
params Expression<Func<T, object>>[] fetchExpressions)
{
var query = GetQuery(specification);
foreach(var fetchExpression in fetchExpressions)
{
query.Fetch(fetchExpression);
}
return query.ToList();
}
You can call this like so:
var products = productRepository.FindAll(specification,
p => p.Variants, p => p.Reviews );
You can change your call to this:
var products = productRepository.FindAll(specification,
new [] { p => p.Variants,
p => p.Reviews });
But this will only work if the T is the same in both!

How to dynamically add OR operator to WHERE clause in LINQ

I have a variable size array of strings, and I am trying to programatically loop through the array and match all the rows in a table where the column "Tags" contains at least one of the strings in the array. Here is some pseudo code:
IQueryable<Songs> allSongMatches = musicDb.Songs; // all rows in the table
I can easily query this table filtering on a fixed set of strings, like this:
allSongMatches=allSongMatches.Where(SongsVar => SongsVar.Tags.Contains("foo1") || SongsVar.Tags.Contains("foo2") || SongsVar.Tags.Contains("foo3"));
However, this does not work (I get the following error: "A lambda expression with a statement body cannot be converted to an expression tree")
allSongMatches = allSongMatches.Where(SongsVar =>
{
bool retVal = false;
foreach(string str in strArray)
{
retVal = retVal || SongsVar.Tags.Contains(str);
}
return retVal;
});
Can anybody show me the correct strategy to accomplish this? I am still new to the world of LINQ :-)
You can use the PredicateBuilder class:
var searchPredicate = PredicateBuilder.False<Songs>();
foreach(string str in strArray)
{
var closureVariable = str; // See the link below for the reason
searchPredicate =
searchPredicate.Or(SongsVar => SongsVar.Tags.Contains(closureVariable));
}
var allSongMatches = db.Songs.Where(searchPredicate);
LinqToSql strange behaviour
I recently created an extension method for creating string searches that also allows for OR searches. Blogged about here
I also created it as a nuget package that you can install:
http://www.nuget.org/packages/NinjaNye.SearchExtensions/
Once installed you will be able to do the following
var result = db.Songs.Search(s => s.Tags, strArray);
If you want to create your own version to allow the above, you will need to do the following:
public static class QueryableExtensions
{
public static IQueryable<T> Search<T>(this IQueryable<T> source, Expression<Func<T, string>> stringProperty, params string[] searchTerms)
{
if (!searchTerms.Any())
{
return source;
}
Expression orExpression = null;
foreach (var searchTerm in searchTerms)
{
//Create expression to represent x.[property].Contains(searchTerm)
var searchTermExpression = Expression.Constant(searchTerm);
var containsExpression = BuildContainsExpression(stringProperty, searchTermExpression);
orExpression = BuildOrExpression(orExpression, containsExpression);
}
var completeExpression = Expression.Lambda<Func<T, bool>>(orExpression, stringProperty.Parameters);
return source.Where(completeExpression);
}
private static Expression BuildOrExpression(Expression existingExpression, Expression expressionToAdd)
{
if (existingExpression == null)
{
return expressionToAdd;
}
//Build 'OR' expression for each property
return Expression.OrElse(existingExpression, expressionToAdd);
}
}
Alternatively, take a look at the github project for NinjaNye.SearchExtensions as this has other options and has been refactored somewhat to allow other combinations
There is another, somewhat easier method that will accomplish this. ScottGu's blog details a dynamic linq library that I've found very helpful in the past. Essentially, it generates the query from a string you pass in. Here's a sample of the code you'd write:
Dim Northwind As New NorthwindDataContext
Dim query = Northwind.Products _
.Where("CategoryID=2 AND UnitPrice>3") _
.OrderBy("SupplierId")
Gridview1.DataSource = query
Gridview1.DataBind()
More info can be found at scottgu's blog here.
Either build an Expression<T> yourself, or look at a different route.
Assuming possibleTags is a collection of tags, you can make use of a closure and a join to find matches. This should find any songs with at least one tag in possibleTags:
allSongMatches = allSongMatches.Where(s => (select t from s.Tags
join tt from possibleTags
on t == tt
select t).Count() > 0)

Resources