I need to group some data after it has arrived from server.
var result = context.GetData();
Assert.Equal(54, result.Count); // OK
var transactions = result.GroupBy(t => new //BeneficiaryGroup
{
BankAcronym = t.BankAcronym.Substring(4, 4),
DebitDate = t.DebitDate,
Key = t.Key
})
.OrderBy(g => g.Key.BankAcronym)
.ThenBy(g => g.Key.DebitDate)
.ThenBy(g => g.Key.Key)
.ToList();
Assert.Equal( 14, transactions.Count ); // OK
When I'm grouping by an anonymous object, the grouping is correctly done.
When I'm grouping by a BeneficiaryGroup object, with the exact same properties
public class BeneficiaryGroup
{
BankAcronym,
DebitDate,
Key
}
the grouping is not correctly done - the group has 54 records, as before grouping.
And I want to group the data by a class so I can return to the API consumer a collection already grouped.
Any ideas why this strange behaviour?
Anonymous types get 'sensible' equality behaviour 'for free', which is why the grouping works as you expect in that case. When you switch to using a named class, it then becomes your responsibility as the class definer to provide equality behaviour, to allow a BeneficiaryGroup to be used as a grouping key in the manner you expect.
As it says in the docs for GroupBy,
The default equality comparer Default is used to compare keys.
where Default is EqualityComparer<T>.Default, which explains:
The Default property checks whether type T implements the
System.IEquatable<T> interface and, if so, returns an
EqualityComparer<T> that uses that implementation. Otherwise, it
returns an EqualityComparer<T> that uses the overrides of
Object.Equals and Object.GetHashCode provided by T.
And for an anonymous type,
Because the Equals and GetHashCode methods on anonymous types are
defined in terms of the Equals and GetHashCode methods of the
properties, two instances of the same anonymous type are equal only if
all their properties are equal.
whereas for a named type that doesn't override Equals and GetHashCode, you will get the Object implementations, which generally aren't useful.
Related
I am trying to group by my custom method. For example, if the group id is something, then I want to return 1 or 0 from the method of GetClientGroup, then I want to group by the value. But I am getting error such as this.
Error
could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
await (from o in _cdsContext.Order
where o.ClienteleId == clienteleId && o.DeliveryDate >= new DateTime(2020, 06, 29).Date
&& o.DeliveryDate != null
group o by new
{
o.ClienteleId,
o.DeliveryDate,
ClientGroup= o.OrderTypeId == 22 ? 259 : GetClientGroup(clienteleId, (int)o.GroupId),
}
into g
select new { ClienteleId = g.Key.ClienteleId}).ToListAsync()
I think you get this error at run time, not at compile time. Am I right?
IEnumerable and IQueryable
You should be aware of the difference between IEnumerable<...> and IQueryable<...>.
Object that implement IEnumerable<...> or IQueryable<...> represents the potentional to give you an enumerable sequence. Once you've got the sequence, you can ask for the first element, and once you've got this, you can ask for the next element as long as there is an element.
This iterating over the elements is usually done using a foreach (var element in sequence) {...}. This translates into the following:
IEnumerable<MyType> sequence = ... // the potential to get iterator
IEnumerator<MyType> enumerator = sequence.GetEnumerator(); // get the iterator
while (enumerator.MoveNext()) // iterate
{ // as long as there are items
MyType item = enumerator.Current; // fetch the item
ProcessItem(item); // and process it.
}
The LINQ methods that don't return IEnumerable<...> or IQueryable<...>, like ToList, ToDictionary, Count, Any, FirstOrDefault, etc internally all use foreach or GetEnumerator
An object that implements IEnumerable<...> is meant to be processed by your local process. The object holds everything to be able to iterate, inclusive calls to local methods.
On the other hand, an object that implements IQueryable<...>, like your _cdsContext.Order is meant to be processed by another process, usually a database management system.
This object holds an Expression and a Provider. The Expression is a generic form of the data that you want to query. The Provider knows who has to execute the query, and what language is used (usually SQL)
Concatenating LINQ statements won't execute the query, they will only change the Expression. When (deep inside) GetEnumerator() is called, the Expression is sent to the Provider, who will translate it into SQL and execute the query at the DBMS. The fetched data is represented as an iterator to your process, who will repeatedly call MoveNext() and Current.
Back to your question
Your GroupBy contains a call to a local method. The GroupBy won't execute the query, it will only change the Expression. In the end you do a ToList. The Tolist will do a GetEnumerator(). The Expression is sent to the Provider who will try to translate it into SQL.
Alas, your provider doesn't know your local method GetClientGroup, and thus can't convert it into SQL. In fact, apart from all your local methods, there are also several LINQ methods that can't be translated into SQL. See Supported and Unsupported LINQ methods (LINQ to entities)
Your compiler doesn't know which methods the provider can translate, so the compiler won't complain. Only at run time, when you do a ToList, the problem is detected.
How to solve the problem
The problem is in parameter KeySelector of Queryable.GroupBy
Expression<Func<TSource,TKey>> keySelector
Alas you forgot to write what GetClientGroup does. It seems that it takes the ClienteleId and the GroupId of an Order, and returns an integer that is similar to a ClientGroup.
The most easy would be to replace the call to GetClientGroup with the code that is in that method. Don't call any other methods
DateTime deliveryLimitDate = new DateTime(2020, 06, 29).Date;
var result = dbContext.Orders
.Where (order => order.ClienteleId == clienteleId
&& order.DeliveryDate != null
&& order.DeliveryDate >= deliveryLimitDate)
.GroupBy(order => new // Parameter KeySelector
{
ClienteleId = order.ClienteleId,
DeliveryDate = order.DeliveryDate,
ClientGroup= order.OrderTypeId == 22 ? 259 :
// formula in GetClientGroup(...)
// for example
(int)order.GroupId << 16 + order.ClienteleId
// parameter ResultSelector
group => new { ClienteleId = group.Key.ClienteleId});
Instead of a separate Select, I used the GroupBy overload with a parameter ResultSelector. Your result is a sequence of objects with only one property ClienteleId. Consider to return only a sequence of ClienteleId:
// parameter ResultSelector
group => group.Key.ClienteleId});
Alas, since I don't know your GetClientGroup, I can't give you parameter KeySelector
I want to add some calculated properties to an EntityObject without loosing the possibility of querying it agains the database.
I created a partial class and added the fields I need in the object. Than I wrote a static function "AttachProperties" that should somehow add some calculated values. I cannot do this on clientside, since several other functions attach some filter-conditions to the query.
The functions should look like this:
return query.Select(o =>
{
o.HasCalculatedProperties = true;
o.Value = 2;
return o;
});
In my case the calculated value depends on several lookups and is not just a simple "2". This sample works with an IEnumerable but, of course, not with an IQueryable
I first created a new class with the EntityObject as property and added the other necessary fields but now I need this extended class to be of the same basetype.
First, in my opinion changing objects in a Select() is a bad idea, because it makes something else happen (state change) than the method name suggests (projection), which is always a recipe for trouble. Linq is rooted in a functional programming (stateless) paradigm, so this kind of usage is just not expected.
But you can extend your class with methods that return a calculation result, like:
partial class EntityObject
{
public int GetValue()
{
return this.MappedProp1 * this.MappedProp2;
}
}
It is a bit hard to tell from your question whether this will work for you. If generating a calculated value involves more than a simple calculation from an object's own properties it may be better to leave your entities alone and create a services that return calculation results from an object graph.
Try something like this:
return from o in collection
select new O()
{
OtherProperty = o.OtherProperty,
HasCalculatedProperties = true,
Value = 2
};
This will create a copy of the original object with the changes you require and avoid all the messiness that come with modifying an entity in a select clause.
I'm working on a custom linq extension for nHibernate by extending the BaseHqlGeneratorForMethod. The technique is documented here:
http://fabiomaulo.blogspot.com/2010/07/nhibernate-linq-provider-extension.html
I've had success with implementing these for various types of operations, but I must say - converting a simple linq expression to its full expression tree is not easy! I'm stuck on one now.
For this example, I have three entities. Employee, Group, and EmployeeGroup. The EmployeeGroup class sets up a many-to-many relationship between Employee and Group. I must specifically create the intermediate class because there are additional properties to track like specific permissions each employee has in each group. So there are two one-to-many relationships, rather than an nHibernate many-to-many relationship.
Now say I want to get all groups that contain a specific employee. I can write this query:
var groups = session.Query<Group>()
.Where(g => g.EmployeeGroups.Any(eg => eg.Employee == employee));
This works fine, but it's a lot to type. I'd much rather be able to do this:
var groups = session.Query<Group>().Where(g => g.HasEmployee(employee));
I start by creating an extension method like so:
public static bool HasEmployee(this Group group, Employee employee)
{
return group.EmployeeGroups.Any(eg => eg.Employee == employee);
}
This works when querying a local list of groups, but not against the nHibernate session. For that, I have to also create a linq extension and register it. Just like in the article (linked above), I create a GroupHasEmployeeGenerator class that extends BaseHqlGeneratorForMethod. I set its .SupportedMethods property to reference my HasEmployee extension method.
Where I get lost is in the override to BuildHql. The expression to build gets complicated pretty fast. I figure since I'm replacing the .Any clause - a good place to start is with the source for the built-in AnyHqlGenerator class. But that doesn't take into account that the source is a property of the original element, and it also doesn't take into account that I don't have a lambda expression to represent the where clause. I need to build these parts manually.
There's no point in posting my attempts so far, as they've all be quite far from anything that would work.
Will someone please help me convert this simple expression into the approprate set of methods for the BuildHql method override?
If there is any better documentation out there for this, please let me know. Thanks.
I know this question is a year old, but I ran into a very similar issue when implementing BaseHqlGeneratorForMethod today.
The input to BuildHql contains a collection of System.Linq.Expressions.Expression arguments that are passed to your extension method. Using these arguments, you can build an expression tree that represents the implementation of your extension method. If the resulting expression is something NHibernate.Linq supports, then you can transform that expression to a subtree of Hql using the provided IHqlExpressionVisitor.
In your example:
public static bool HasEmployee(this Group group, Employee employee)
{
return group.EmployeeGroups.Any(eg => eg.Employee == employee);
}
This would become something similar to this:
public override HqlTreeNode BuildHql(MethodInfo method, Expression targetObject, ReadOnlyCollection<Expression> arguments, HqlTreeBuilder treeBuilder, IHqlExpressionVisitor visitor)
{
var AnyMethod = EnumerableHelper.GetMethod("Any", new[] {typeof(IEnumerable<EmployeeGroup>), typeof(Func<EmployeeGroup, bool>)}, new[] {typeof(EmployeeGroup)});
var EmployeeGroupsProperty = ReflectionHelper.GetProperty<Group>(g => g.EmployeeGroups);
var EmployeeProperty = ReflectionHelper.GetProperty<EmployeeGroup>(eg => eg.Employee);
var EmployeeGroupParameter = Expression.Parameter(typeof(EmployeeGroup));
var EmployeeGroupPredicate = Expression.Lambda(Expression.Equal(Expression.MakeMemberAccess(EmployeeGroupParameter, EmployeeProperty), arguments[1]), EmployeeGroupParameter);
var CallExpression = Expression.Call(AnyMethod, Expression.MakeMemberAccess(arguments[0], EmployeeGroupsProperty), EmployeeGroupPredicate);
return visitor.Visit(CallExpression);
}
I can't really test this specific example, but the same approach worked for me when providing support for my own extension method.
I have two Entity Framework 4 Linq queries I wrote that make use of a custom class method, one works and one does not:
The custom method is:
public static DateTime GetLastReadToDate(string fbaUsername, Discussion discussion)
{
return (discussion.DiscussionUserReads.Where(dur => dur.User.aspnet_User.UserName == fbaUsername).FirstOrDefault() ?? new DiscussionUserRead { ReadToDate = DateTime.Now.AddYears(-99) }).ReadToDate;
}
The linq query that works calls a from after a from, the equivalent of SelectMany():
from g in oc.Users.Where(u => u.aspnet_User.UserName == fbaUsername).First().Groups
from d in g.Discussions
select new
{
UnReadPostCount = d.Posts.Where(p => p.CreatedDate > DiscussionRepository.GetLastReadToDate(fbaUsername, p.Discussion)).Count()
};
The query that does not work is more like a regular select:
from d in oc.Discussions
where d.Group.Name == "Student"
select new
{
UnReadPostCount = d.Posts.Where(p => p.CreatedDate > DiscussionRepository.GetLastReadToDate(fbaUsername, p.Discussion)).Count(),
};
The error I get is:
LINQ to Entities does not recognize the method 'System.DateTime GetLastReadToDate(System.String, Discussion)' method, and this method cannot be translated into a store expression.
My question is, why am I able to use my custom GetLastReadToDate() method in the first query and not the second? I suppose this has something to do with what gets executed on the db server and what gets executed on the client? These queries seem to use the GetLastReadToDate() method so similarly though, I'm wondering why would work for the first and not the second, and most importantly if there's a way to factor common query syntax like what's in the GetLastReadToDate() method into a separate location to be reused in several different other LINQ queries.
Please note all these queries are sharing the same object context.
I think your better of using a Model Defined Function here.
Define a scalar function in your database which returns a DateTime, pass through whatever you need, map it on your model, then use it in your LINQ query:
from g in oc.Users.Where(u => u.aspnet_User.UserName == fbaUsername).First().Groups
from d in g.Discussions
select new
{
UnReadPostCount = d.Posts.Where(p => p.CreatedDate > myFunkyModelFunction(fbaUsername, p.Discussion)).Count()
};
and most importantly if there's a way to factor common query syntax like what's in the GetLastReadToDate() method into a separate location to be reused in several different places LINQ queries.
A stored procedure would probably be one way to store that 'common query syntax"...EF, at least 4.0, works very nicely with SP's.
I have a method to build an expression for a linq query for a given type, property, and value. This works wonderfully as long as the property on the type is NOT nullable. Here is the example I am working from (http://www.marcuswhitworth.com/2009/12/dynamic-linq-with-expression-trees) I am calling the Equals method on the property. However I have discovered that the Equals method for Nullable types takes an Object as a parameter instead of the Nullable type. I attempted to use GetValueOrDefault to hide the null values but EF doesn't support that method. As a simple example the following code will throw an error:
decimal? testVal = new decimal?(2100);
var test = (from i in context.Leases where i.AnnualRental.Equals(testVal) select i).ToList();
However if you use == instead of the Equals() method it will work OK. I am not sure how to convert the code to use == instead of Equals() however. Any suggestions will be greatly appreciated.
If you say ==, you generate a BinaryExpression with NodeType as ExpressionType.Equal. http://msdn.microsoft.com/en-us/library/bb361179.aspx
If you say .Equals(x), you generate a MethodCallExpression. The MethodCallExpressions that LinqToEntities may translate into Sql is a limitted list (for example none of your own undecorated methods are in that list). Nullable<T>.Equals(x) is apparently not on that list.
Don't say .Equals(x) to LinqToEntities.