I have a nice Linq challenge. I would like to filter some data on a Date using a FilterFunction. It should work like this:
ApplyDateFilterEx(query,null, DateTime.Today, s => s.CreatedDate);
Should filter the query so that all 'shipments' until today are returned. The query object is of IQueryable and the whole query should be evaluated by EF5 and therefore should convert to SQL.
This is what I have so far (DOES NOT COMPILE):
private static IQueryable<Shipment> ApplyDateFilterEx(IQueryable<Shipment> query, DateTime? minDate, DateTime? maxDate, Expression<Func<Shipment, DateTime?>> dateMember)
{
if (minDate != null)
{
//convert func to expression so EF understands
Expression<Func<Shipment, bool>> where = x => minDate <= dateMember(x);
query = query.Where(where);
}
if (maxDate != null)
{
Expression<Func<Shipment, bool>> where = x => dateMember(x) <= maxDate;
query = query.Where(where);
}
return query;
}
You can see I want to convert the expression s=>DateTime? to s=> Bool to be evaluated. How can I get this to work?
Thank you for reading.
Martijn
UDPATE:
I ended up with this (thanks to rdvanbuuren)
var predicate = Expression.Lambda<Func<Shipment, bool>>(
Expression.GreaterThanOrEqual(
selector.Body,
Expression.Constant(dateFilter.MinDate, typeof (DateTime?))
), selector.Parameters);
Have a look at
How to implement method with expression parameter c#
I think that's exactly what you need, but with an Expression.GreaterThanOrEqual or Expression.LessThanOrEqual. Good luck!
Related
Is it possible to construct a where clause like this, where predicate is of type Expression<Func<T, bool>> predicate:
var resultQuery = query.Where(q => !q.IsDeleted && predicate).ToList();
I would like to avoid double where clauses like this:
var resultQuery = query.Where(q => !q.IsDeleted).Where(predicate).ToList();
Instead, you could use custom extension methods and filter result using them:
public static class QueryExtensions
{
public static IQueryable<Image> NonDeleted(this IQueryable<Image> queryable)
{
return queryable.Where(x => !x.Deleted);
}
public static IQueryable<Image> LatestOnly(this IQueryable<Image> queryable)
{
return queryable.Where(x => x.CreateDate <= DateTime.UtcNow.AddDays(-7));
}
}
And then combine them in query:
var result = context
.Images
.NonDeleted()
.LatestOnly()
.ToList();
I like this approach cause it's clean and easy to read. You can also use interfaces in your entities and extension which use those interfaces to quickly filter items based on interfaces that implemented on entity. For example:
public interface ICreationDate{
DateTime CreateDate {get;}
}
public class Image: ICreationDate{
public DateTime CreateDate {get; private set;} = DateTime.UtcNow;
}
Then extension can be changed like this:
public static IQueryable<T> LatestOnly<T>(this IQueryable<T> queryable)
where T : ICreationDate
{
return queryable.Where(x => x.CreateDate <= DateTime.UtcNow.AddDays(-7));
}
This approach gives you more flexibility and reusability.
I know these all are too far from your original question, but it may bring you some alternative aproaches
IT is impossible. You need to provide a lambda expression as a parameter to the Where clause which would be compiled to an expression tree and after that translated into some SQL query. In your example
var resultQuery = query.Where(q => !q.IsDeleted && predicate).ToList();
you are combining a lambda expression and a boolean check. The only way to avoid the double Where clauses is to create a helper function that returns a lambda expression for filtering which includes filtering for the IsDeleted flag and the predicate logic i.e.
private System.Linq.Expressions.Expression<Func<T, bool>> filterPredicate(int n)
{
return q => !q.IsDeleted && q.Age > n;
}
Here we are assuming that
q.Age > n
is the logic of your predicate function. And then use the filter predicates like this:
var resultQuery = query.Where(filterPredicate(5)).ToList();
More about lambda expressions and expression trees you can read here https://learn.microsoft.com/en-us/dotnet/api/system.linq.expressions.expression-1?view=net-5.0
I am trying to write a really generic way to load EF entities in batches, using the Contains method to generate a SQL IN statement. I've got it working if I pass the entire expression in, but when I try to build the expression dynamically, I am getting a "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities." So I know this means that EF thinks I'm calling an arbitrary method and it can't translate it into SQL, but I can't figure out how to get it to understand the underlying expression.
So If I do something like this (just showing the relevant snippets):
Function declaration:
public static List<T> Load<T>(IQueryable<T> entityQuery, int[] entityIds, Func<T, int> entityKey, int batchSize = 500, Func<T, bool> postFilter = null) where T : EntityObject
{
var retList = new List<T>();
// Append a where clause to the query passed in, that will use a Contains expression, which generates a SQL IN statement. So our SQL looks something like
// WHERE [ItemTypeId] IN (1921,1920,1922)
// See http://rogeralsing.com/2009/05/21/entity-framework-4-where-entity-id-in-array/ for details
Func<int[], Expression<Func<T, bool>>> containsExpression = (entityArray => (expr => entityArray.Contains(entityKey(expr))));
// Build a new query with the current batch of IDs to retrieve and add it to the list we are returning
newQuery = entityQuery.Where<T>(containsExpression(entityIds));
retList.AddRange(newQuery.ToList());
return retList;
}
Call function:
var entities = BatchEntity.Load<ItemType>(from eItemType in dal.Context.InstanceContainer.ItemTypes
select eItemType
, itemTypeData
, (ek => ek.ItemTypeId)
);
I get "The LINQ expression node type 'Invoke' is not supported in LINQ to Entities."
But if I change it to be this:
Function declaration:
public static List<T> Load<T>(IQueryable<T> entityQuery, int[] entityIds, Func<int[], Expression<Func<T, bool>>> containsExpression, int batchSize = 500, Func<T, bool> postFilter = null) where T : EntityObject
{
var retList = new List<T>();
// Build a new query with the current batch of IDs to retrieve and add it to the list we are returning
newQuery = entityQuery.Where<T>(containsExpression(entityIds));
retList.AddRange(newQuery.ToList());
return retList;
}
Call function:
var entities = BatchEntity.Load<ItemType>(from eItemType in dal.Context.InstanceContainer.ItemTypes
select eItemType
, itemTypeData
, (entityArray => (ek => entityArray.Contains(ek.ItemTypeId)))
);
It works fine. Is there any way I can make EF understand the more generic version?
The problem, as you describe, is that the entityKey function in the first example is opaque since it is of type Func rather than Expression. However, you can get the behavior you want by implementing a Compose() method to combine two expressions. I posted the code to implement compose in this question: use Expression<Func<T,X>> in Linq contains extension.
With Compose() implemented, your function can be implemented as below:
public static List<T> Load<T>(this IQueryable<T> entityQuery,
int[] entityIds,
// note that this is an expression now
Expression<Func<T, int>> entityKey,
int batchSize = 500,
Expression<Func<T, bool>> postFilter = null)
where T : EntityObject
{
Expression<Func<int, bool>> containsExpression = id => entityIds.Contains(id);
Expression<Func<T, bool>> whereInEntityIdsExpression = containsExpression.Compose(entityKey);
IQueryable<T> filteredById = entityQuery.Where(whereInEntityIdsExpression);
// if your post filter is compilable to SQL, you might as well do the filtering
// in the database
if (postFilter != null) { filteredById = filteredById.Where(postFilter); }
// finally, pull into memory
return filteredById.ToList();
}
I am trying to get the following method to work:
private static IQueryable<TObject> ApplyOrderBy<TObject, TKey>(IQueryable<TObject> query, OrderByDirection orderByDirection,
Expression<Func<TObject, TKey>> sortExpression, ref bool first)
{
if (orderByDirection == OrderByDirection.None || sortExpression == null) return;
if (orderByDirection != OrderByDirection.Ascending && orderByDirection != OrderByDirection.Descending)
throw new Exception(string.Format("Should never get here! Unknown OrderByDirection enum - '{0}'.", orderByDirection));
if (first)
{
first = false;
query = orderByDirection == OrderByDirection.Ascending
? query.OrderBy(sortExpression)
: query.OrderByDescending(sortExpression);
}
else
{
query = orderByDirection == OrderByDirection.Ascending
? ((IOrderedQueryable<TObject>)query).ThenBy(sortExpression)
: ((IOrderedQueryable<TObject>)query).ThenByDescending(sortExpression);
}
return query;
}
This method works great if you call it like this:
ApplyOrderByToGet(ref query, OrderByDirection.Ascending, x => x.StartDateTime, ref first);
The sort expression then has a strongly typed DateTime as the type and LINQ to SQL is happy. However, if you want to pass an array of these expressions with varying types, you ultimately need a list with "object" as the type. Problem is that LINQ to SQL does not figure out that the type is not object, but instead is DateTime. This works with a regular list using LINQ to objects.
Seeing as it’s possible to navigate the expression tree and find out what the type is, would it be possible to cast/convert the expression before calling ApplyOrderBy?
Cast or convert from:
Expression<Func<T, object>>
to:
Expression<Func<T, DateTime>>
I'm looking to use LINQ to do multiple where conditions on a collection similar to
IEnumerable<Object> items;
items.Where(p => p.FirstName = "John");
items.Where(p => p.LastName = "Smith");
except for rather than having multiple AND conditions (as with this example), I'd like to have multiple OR conditions.
EDIT
Sorry, to clarify I don't know how many of these conditions I will have so
items.Where(p => p.FirstName = "John" || p => p.LastName = "Smith")
won't work.
Basically, here's what I'm trying to do:
foreach(var name in names)
{
items = items.Where(p => p.Name == name);
}
Use PredicateBuilder:
Suppose you want to write a LINQ to SQL or Entity Framework query that implements a keyword-style search. In other words, a query that returns rows whose description contains some or all of a given set of keywords...
The ideal approach is to dynamically construct a lambda expression tree that performs an or-based predicate.
Of all the things that will drive you to manually constructing expression trees, the need for dynamic predicates is the most common in a typical business application. Fortunately, it’s possible to write a set of simple and reusable extension methods that radically simplify this task. This is the role of our PredicateBuilder class...
It sounds like your whitelist of names is only known at runtime. Perhaps try this:
string[] names = new string[] {"John", "foo", "bar"};
var matching = items.Where(x => names.Contains(x.Name));
You can use .Union() to return results that satisfy any condition.
var results = items.Where(p => p.FirstName == "John")
.Union(items.Where(p => p.LastName == "Smith"));
This is inferior to using the || operator. It isn't clear from your edit why that wouldn't work.
public static Expression<Func<T, bool>> OrTheseFiltersTogether<T>(
this IEnumerable<Expression<Func<T, bool>>> filters)
{
Expression<Func<T, bool>> firstFilter = filters.FirstOrDefault();
if (firstFilter == null)
{
Expression<Func<T, bool>> alwaysTrue = x => true;
return alwaysTrue;
}
var body = firstFilter.Body;
var param = firstFilter.Parameters.ToArray();
foreach (var nextFilter in filters.Skip(1))
{
var nextBody = Expression.Invoke(nextFilter, param);
body = Expression.OrElse(body, nextBody);
}
Expression<Func<T, bool>> result = Expression.Lambda<Func<T, bool>>(body, param);
return result;
}
Then, later:
List<Expression<Func<Person, bool>>> filters = names
.Select<string, Expression<Func<Person, bool>>>(name =>
p => p.Name == name
).ToList();
Expression<Func<Person, bool>> filterOfOrs = filters.OrTheseFiltersTogether();
query = query.Where<Person>(filterOfOrs);
You can't make the Where clause dynamic, but you can dynamically create the Lambda Expression you pass to it. Create the right Expression, compile it and pass the resulting lambda expression as a parameter to the Where clause.
EDIT:
Okay, seems like you can skip the part where you have to manually create the Expression and can use PredicateBuilder for it, as already answered by AS-CII.
We have an entity with DateTime property DateDestroyed. A query needs to return results where this value is between nullable DateTimes startDate and endDate.
The where clauses I have are:
.Where(x => startDate.HasValue ? startDate <= x.DateDestroyed : true)
.Where(x => endDate.HasValue ? x.DateDestroyed <= endDate : true);
The query always return no results. I'm pretty sure I haven't written this query properly but don't know how it should be written or why it's not working?
My code requires IQueryable so I adapted the work by #p.campbell at ExtensionMethod.net as follows:
public static IQueryable<TSource> WhereIf<TSource>(this IQueryable<TSource> source, bool condition, Func<TSource, bool> predicate)
{
return condition ? source.Where(predicate).AsQueryable() : source;
}
public static IQueryable<TSource> WhereIf<TSource>(this IQueryable<TSource> source, bool condition, Func<TSource, int, bool> predicate)
{
return condition ? source.Where(predicate).AsQueryable() : source;
}
You could create/use an extension method for WhereIf:
Given a boolean condition, append a Where clause.
var foo = db.Customers.WhereIf(startDate.HasValue,
x => startDate <= x.DateDestroyed)
.WhereIf(endDate.HasValue,
x => x.DateDestroyed <= endDate );
More details at WhereIf at ExtensionMethod.net. You can find the code there for IEnumerable<T> and IQueryable<T>.
let's say you have a variable called "query" that you've stored the beginning part of your linq statement. Try this to dynamically construct the where clause:
if (startDate.HasValue) {
query = query.Where(x => x.DateDestroyed >= startDate);
}
if (endDate.HasValue) {
query = query.Where(x => x.DateDestroyed <= endDate);
}
LINQ works on deferred execution so the WHERE clause will correctly resolve when the code executes.
Are you always re-assigning your query with the Where() filter applied?
This pattern should work as expected:
var query = getResults();
query = query.Where(x => startDate.HasValue ? startDate <= x.DateDestroyed : true)
query = query.Where(x => endDate.HasValue ? x.DateDestroyed <= endDate : true);