I am trying to apply additional server-side filtering to entities exposed by an asp.net OData service, using Linq to Entities.
So far so good, but the Linq query that I build for the filter can sometimes get to an enormous length, causing a StackOverflowException.
As a workaround, I want to execute the query in a separate thread, giving that thread a bigger stack size limit. To achieve this, I implemented an IQueryable and IQueryProvider wrappers, so that when the IQueryable's Execute method is called, it would be executed in a separate thread.
To keep it simple and focused on my problem, I will omit the threading part in the following code samples, as this is not the issue here.
Here's the relevant parts of the code:
public class MyQueryable<T> : IQueryable<T> {
private IQueryable<T> _Inner { get; set; }
private IQueryProvider _Provider { get; set; }
public MyQueryable(IQueryable<T> inner) {
_Inner = inner;
_Provider = new MyQueryProvider(inner.Provider);
}
public Type ElementType {
get { return _Inner.ElementType; }
}
public Expression Expression {
get { return _Inner.Expression; }
}
public IQueryProvider {
get { return _Provider; }
}
/* ... Implementations of GetEnumerator ... */
}
public class MyQueryProvider : IQueryProvider {
private IQueryProvider _Inner { get; set; }
public MyQueryProvider(IQueryProvider inner) {
_Inner = inner;
}
public IQueryable<T> CreateQuery<T>(Expression expression) {
return new MyQueryable<T>(_Inner.CreateQuery<T>(expression));
}
public T Execute<T>(Expression expression) {
// The problem occurs on the following line.
return _Inner.Execute<T>(expression);
}
/* ... Implementation of the non-generic versions of CreateQuery, Execute ... */
}
When I run this, I get an InvalidOperationException in MyQueryProvider.Execute method:
Cannot compare elements of type 'System.Collections.Generic.ICollection`1'.
Only primitive types (such as Int32, String, and Guid) and entity types are supported.
This is how I build the original IQueryable that is being passed to MyQueryable:
I get the DbSet from my entities context, apply OData filter query option on it, and call Count(). Something along the lines of:
((odataQueryOptions.Filter.ApplyTo(
context.Entities.AsQueryable(), new ODataQuerySettings()))
as IQueryable<Entity>).Count()
I partially identified that the problem is that the odata filter query option contains a nested 'any' filter, something like:
Entities/$filter=RelatedEntities/any(entity: (entity/ID eq 1))
which adds a check to the IQueryable's Where expression whether the RelatedEntities collection is null. This check is suppostedly what causes the above exception.
What I cannot understand, is why the original IQueryable fails when I try to delegate the Execute method's execution to it from the MyQueryable.Execute method, but it all works fine when I use the original IQueryable directly.
Any help on this would be highly appreciated. Thanks in advance.
Related
I am creating a very simple web api that allows me to search for things:
public IEnumerable<Thing> GetAllThings()
{
// get all the things!
}
My application has to be able to retrieve a single Thing:
public Thing GetThing(string id)
{
// get a single thing
}
I need more details when retrieving a single Thing than I do when I retrieve all Things. Should I have a separate controller that returns a ThingWithDetails instead of having separate models on GetAllThings and GetThing?
No. You do not have to make a separate controller for ThingsWithDetails. Just have a different overload method. All the below methods will be called based on the request
The below method will be called when you are using url: api/{controller}
public IEnumerable<Thing> GetAllThings()
{
// get all the things!
}
The below method will be called when you are using url: api/{controller}/1
public Thing GetThing(string id)
{
// get a single thing
}
The below method will be called when you are using url: api/{controller}?id=1&name=xyz
public Thing GetThing(string id, string name)
{
// get a single thing
}
As a best practice, you should have a Thing entity with all the possible properties, so you keep the code consistent, and by default you load all of them (assuming that they come from DB).
If you then want to "hide" some properties, you can set them to null just before returning them, using an extension method.
public class Thing
{
public string PropertyA { get; set; }
public string PropertyB { get; set; }
public string PropertyC { get; set; }
}
// extension methods
public static class ThingExtensionMethods
{
public static void ToBasicDetails(this Thing thing)
{
// hide properties
thing.PropertyB = null;
thing.PropertyC = null;
}
}
public IEnumerable<Thing> GetAllThings()
{
var things = _db.Things.ToList();
foreach(var item in things)
{
item.ToBasicDetails();
}
return things;
}
public Thing GetThing(string id)
{
var thing = _db.Things.Find(id);
return thing;
}
Defining separate "Thing" and "ThingWithDetails" models makes sense, because you are returning different resource types to the client. The return type defines the response body, so use different models for different response formats.
You don't need to put those methods into separate controllers - it's valid to have an action that returns a ThingWithDetails and another action that returns IEnumerable. It comes down to preference. (I'd probably leave them in the same controller, because they are dealing with the same DB entity.)
This tutorial shows an example.
I have a WebApi 2.1 OData (v 5.1.1) service backed in Entity Framework 6.1. I'm trying to lock it down from a security standpoint, so that users can only query data that is theirs. I have everything working fine, until you get to the $expands option.
For the sake of this discussion, consider the following simplified data model:
public class Scenario
{
public Guid Id { get; set; }
public Guid CreatedById { get; set; }
}
public class Property
{
public Guid Id { get; set }
public Guid CreatedById { get; set; }
public IQueryable<Scenario> Scenarios { get; set; }
}
When I call /Properties(guid'SOMEGUID')?$expand=Scenarios, I need to be able to make sure that only Scenarios where the CreatedById = CurrentUserId are returned. This needs to happen on the server-side and not in the client-side query.
WCF Data Services had QueryInterceptors that would handle this kind of situation... what is the equivalent in WebApi 2.1 OData?
Thanks!
here's a gist with a sample on how can you implement this on your own:
https://gist.github.com/anonymous/9237151
Based on my git, you can use a similar validator and implement your validation logic on a CanAcess method or similar. Let me know if this helps you.
We will have soon an official sample on http://aspnet.codeplex.com
There are two ways to solve your problem if I understood your question correctly.
Call the ApplyTo method of ODataQueryOptions on the IQueryable result
public IQueryable<Property> Get(ODataQueryOptions queryOptions)
{
....
return queryOptions.ApplyTo(properties);
}
Add attribute Queryable on the GetData method and let WebAPI handles the query option
[Queryable]
public IQueryable<Property> Get()
{
...
return properties;
}
I'm with an irritating problem. It might be something stupid, but I couldn't find out.
I'm using Linq to NHibernate, and I would like to count how many items are there in a repository. Here is a very simplified definition of my repository, with the code that matters:
public class Repository {
private ISession session;
/* ... */
public virtual IQueryable<Product> GetAll() {
return session.Linq<Product>();
}
}
All the relevant code in the end of the question.
Then, to count the items on my repository, I do something like:
var total = productRepository.GetAll().Count();
The problem is that total is 0. Always. However there are items in the repository. Furthermore, I can .Get(id) any of them.
My NHibernate log shows that the following query was executed:
SELECT count(*) as y0_ FROM [Product] this_ WHERE not (1=1)
That must be that "WHERE not (1=1)" clause the cause of this problem.
What can I do to be able .Count() the items in my repository?
Thanks!
EDIT: Actually the repository.GetAll() code is a little bit different... and that might change something! It is actually a generic repository for Entities. Some of the entities implement also the ILogicalDeletable interface (it contains a single bool property "IsDeleted"). Just before the "return" inside the GetAll() method I check if if the Entity I'm querying implements ILogicalDeletable.
public interface IRepository<TEntity, TId> where TEntity : Entity<TEntity, TId> {
IQueryable<TEntity> GetAll();
...
}
public abstract class Repository<TEntity, TId> : IRepository<TEntity, TId>
where TEntity : Entity<TEntity, TId>
{
public virtual IQueryable<TEntity> GetAll()
{
if (typeof (ILogicalDeletable).IsAssignableFrom(typeof (TEntity)))
{
return session.Linq<TEntity>()
.Where(x => (x as ILogicalDeletable).IsDeleted == false);
}
else
{
return session.Linq<TEntity>();
}
}
}
public interface ILogicalDeletable {
bool IsDeleted {get; set;}
}
public Product : Entity<Product, int>, ILogicalDeletable
{ ... }
public IProductRepository : IRepository<Product, int> {}
public ProductRepository : Repository<Product, int>, IProductRepository {}
Edit 2: actually the .GetAll() is always returning an empty result-set for entities that implement the ILogicalDeletable interface (ie, it ALWAYS add a WHERE NOT (1=1) clause.
I think Linq to NHibernate does not like the typecast.
It looks like you've got a soft delete model going on here, and you're trying to filter these out from being returned by the GetAll() method. I agree with your analysis that NHibernate.Linq doesn't properly process the typecast, but you may want to try replacing this with a query filter.
public virtual IQueryable<TEntity> GetAll()
{
if (typeof(ILogicalDeletable).IsAssignableFrom(typeof(TEntity)))
{
return session.Linq<TEntity>().OfType<ILogicalDeletable>()
.Where(x => !x.IsDeleted).Cast<TEntity>();
}
return session.Linq<TEntity>();
}
Try that.
How do you deal with validation on complex aggregates in a domain driven design? Are you consolidating your business rules/validation logic?
I understand argument validation and I understand property validation which can be attached to the models themselves and do things like check that an email address or zipcode is valid or that a first name has a minimum and maximum length.
But what about complex validation that involves multiple models? Where do you typically place these rules & methods within your architecture? And what patterns if any do you use to implement them?
Instead of relying on IsValid(xx) calls all over your application, consider taking some advice from Greg Young:
Don't ever let your entities get into
an invalid state.
What this basically means is that you transition from thinking of entities as pure data containers and more about objects with behaviors.
Consider the example of a person's address:
person.Address = "123 my street";
person.City = "Houston";
person.State = "TX";
person.Zip = 12345;
Between any of those calls your entity is invalid (because you would have properties that don't agree with each other. Now consider this:
person.ChangeAddress(.......);
all of the calls relating to the behavior of changing an address are now an atomic unit. Your entity is never invalid here.
If you take this idea of modeling behaviors rather than state, then you can reach a model that doesn't allow invalid entities.
For a good discussion on this, check out this infoq interview: http://www.infoq.com/interviews/greg-young-ddd
I like Jimmy Bogard's solution to this problem. He has a post on his blog titled "Entity validation with visitors and extension methods" in which he presents a very elegant approach to entity validation that suggest the implementation of a separate class to store validation code.
public interface IValidator<T>
{
bool IsValid(T entity);
IEnumerable<string> BrokenRules(T entity);
}
public class OrderPersistenceValidator : IValidator<Order>
{
public bool IsValid(Order entity)
{
return BrokenRules(entity).Count() == 0;
}
public IEnumerable<string> BrokenRules(Order entity)
{
if (entity.Id < 0)
yield return "Id cannot be less than 0.";
if (string.IsNullOrEmpty(entity.Customer))
yield return "Must include a customer.";
yield break;
}
}
I usualy use a specification class,
it provides a method (this is C# but you can translate it in any language) :
bool IsVerifiedBy(TEntity candidate)
This method performs a complete check of the candidate and its relations.
You can use arguments in the specification class to make it parametrized, like a check level...
You can also add a method to know why the candidate did not verify the specification :
IEnumerable<string> BrokenRules(TEntity canditate)
You can simply decide to implement the first method like this :
bool IsVerifiedBy(TEntity candidate)
{
return BrokenRules(candidate).IsEmpty();
}
For broken rules, I usualy write an iterator :
IEnumerable<string> BrokenRules(TEntity candidate)
{
if (someComplexCondition)
yield return "Message describing cleary what is wrong...";
if (someOtherCondition)
yield return
string.Format("The amount should not be {0} when the state is {1}",
amount, state);
}
For localization, you should use resources, and why not pass a culture to the BrokenRules method.
I place this classes in the model namespace with names that suggest their use.
Multiple model validation should be going through your aggregate root. If you have to validate across aggregate roots, you probably have a design flaw.
The way I do validation for aggregates is to return a response interface that tells me if validation pass/fail and any messages about why it failed.
You can validate all the sub-models on the aggregate root so they remain consistent.
// Command Response class to return from public methods that change your model
public interface ICommandResponse
{
CommandResult Result { get; }
IEnumerable<string> Messages { get; }
}
// The result options
public enum CommandResult
{
Success = 0,
Fail = 1
}
// My default implementation
public class CommandResponse : ICommandResponse
{
public CommandResponse(CommandResult result)
{
Result = result;
}
public CommandResponse(CommandResult result, params string[] messages) : this(result)
{
Messages = messages;
}
public CommandResponse(CommandResult result, IEnumerable<string> messages) : this(result)
{
Messages = messages;
}
public CommandResult Result { get; private set; }
public IEnumerable<string> Messages { get; private set; }
}
// usage
public class SomeAggregateRoot
{
public string SomeProperty { get; private set; }
public ICommandResponse ChangeSomeProperty(string newProperty)
{
if(newProperty == null)
{
return new CommandResponse(CommandResult.Fail, "Some property cannot be changed to null");
}
SomeProperty = newProperty;
return new CommandResponse(CommandResult.Success);
}
}
This questions a bit old now but in case anyone is interested here's how I implement validation in my service classes.
I have a private Validate method in each of my service classes that takes an entity instance and action being performed, if validation fails a custom exception is thrown with the details of the broken rules.
Example DocumentService with built in validation
public class DocumentService : IDocumentService
{
private IRepository<Document> _documentRepository;
public DocumentService(IRepository<Document> documentRepository)
{
_documentRepository = documentRepository;
}
public void Create(Document document)
{
Validate(document, Action.Create);
document.CreatedDate = DateTime.Now;
_documentRepository.Create(document);
}
public void Update(Document document)
{
Validate(document, Action.Update);
_documentRepository.Update(document);
}
public void Delete(int id)
{
Validate(_documentRepository.GetById(id), Action.Delete);
_documentRepository.Delete(id);
}
public IList<Document> GetAll()
{
return _documentRepository
.GetAll()
.OrderByDescending(x => x.PublishDate)
.ToList();
}
public int GetAllCount()
{
return _documentRepository
.GetAll()
.Count();
}
public Document GetById(int id)
{
return _documentRepository.GetById(id);
}
// validation
private void Validate(Document document, Action action)
{
var brokenRules = new List<string>();
if (action == Action.Create || action == Action.Update)
{
if (string.IsNullOrWhiteSpace(document.Title))
brokenRules.Add("Title is required");
if (document.PublishDate == null)
brokenRules.Add("Publish Date is required");
}
if (brokenRules.Any())
throw new EntityException(string.Join("\r\n", brokenRules));
}
private enum Action
{
Create,
Update,
Delete
}
}
I like this approach because it allows me to put all my core validation logic in one place which keeps things simple.
In my code I'd like to make my repositories IQueryable. This way, the criteria for selection will be a linq expression tree.
Now if I want to mock my repository in theorie this is very easy : just implement the interface of my repository (which is also a IQueryable object).
My mock repository implementation would be only a in memory collection, but my question is : Do you know an easy way to implement the IQueryable interface of my mock, to transfer the query to my in-memory collection (IEnumerable) ?
Thanks for your response,
Some precision
The client object of my repository will use my repository this way :
var result = from entry in MyRepository where entry.Product == "SomeProduct" select entry;
What does ToList or AsEnumerable is to execute the query and return the result as a List or as a IEnumerable. But I have to implement the IQueryable interface on my repository, with a IQueryProvider which transform the expression in a call to a IEnumerable object.
Solution
The implementation of the solution is delegating call to IQueryable to my inmemory collection with AsQueryable.
public class MockRepository : IQueryable<DomainObject>
{
private List<DomainObject> inMemoryList = new List<DomainObject>();
#region IEnumerable<DomainObject> Members
public IEnumerator<DomainObject> GetEnumerator()
{
return inMemoryList.GetEnumerator();
}
#endregion
#region IEnumerable Members
IEnumerator IEnumerable.GetEnumerator()
{
return inMemoryList.GetEnumerator();
}
#endregion
#region IQueryable Members
public Type ElementType
{
get
{
return inMemoryList.AsQueryable().ElementType;
}
}
public Expression Expression
{
get
{
return inMemoryList.AsQueryable().Expression;
}
}
public IQueryProvider Provider
{
get
{
return inMemoryList.AsQueryable().Provider;
}
}
#endregion
}
Use AsQueryable on your mocks. Now they're queryable and you can treat them like any other queryable.
Can you use the extension method
.ToList<>