EF4, Lambda, Repository pattern and DTOs - linq

I have a semi complicated question regarding Entity Framework4, Lambda expressions, and Data Transfer Objects (DTO).
So I have a small EF4 project, and following established OO principles, I have a DTO to provide a layer of abstraction between the data consumers (GUI) and the data model.
VideoDTO = DTO with getters/setters, used by the GUI
VideoEntity = Entity generated by EF4
My question revolves around the use of the DTO by the GUI (and not having the GUI use the Entity at all), combined with a need to pass a lambda to the data layer. My data layer is a basic repository pattern with Add. Change, Delete, Get, GetList, etc.
Trying to implement a Find method with a signature like so:
public IEnumerable<VideoDTO> Find(Expression<Func<VideoEntity, bool>> exp)
...
_dataModel.Videos.Where(exp).ToList<Video>()
---
My problem/concern is the "exp" needing to be of type VideoEntity instead of VideoDTO. I want to preserve the separation of concerns so that the GUI does not know about the Entity objects. But if I try to pass in
Func<VideoDTO, bool>
I cannot then do a LINQ Where on that expression using the actual data model.
Is there a way to convert a Func<VideoDTO,bool> to a Func<VideoEntity, bool>
Ideally my method signature would accept Func<VideoDTO, bool> and that way the GUI would have no reference to the underlying data entity.
Is this clear enough? Thanks for your help
Thanks for the repliesto both of you.
I'll try the idea of defining the search criteria in an object and using that in the LINQ expression. Just starting out with both EF4 and L2S, using this as a learning project.
Thanks again!

In architectures like CQRS there isn't need for such a conversion at all cause read & write sides of app are separated.
But in Your case, You can't runaway from translation.
First of all - You should be more specific when defining repositories. Repository signature is thing You want to keep explicit instead of generic.
Common example to show this idea - can You tell what indexes You need in Your database when You look at Your repository signature (maybe looking at repository implementation, but certainly w/o looking at client code)? You can't. Cause it's too generic and client side can search by anything.
In Your example it's a bit better cause expression genericness is tied with dto instead of entity.
This is what I do (using NHibernate.Linq, but the idea remains)
public class Application{
public Project Project {get;set;}
}
public class ApplicationRepository{
public IEnumerable<Application> Search(SearchCriteria inp){
var c=Session.Linq<Application>();
var q=c.AsQueryable();
if(!string.IsNullOrEmpty(inp.Acronym))
q=q.Where(a=>a.Project.Acronym.Contains(inp.Acronym));
/*~20 lines of similar code snipped*/
return q.AsQueryable();
}
}
//used by client
public class SearchCriteria{
public string Acronym{get;set;}
/*some more fields that defines how we can search Applications*/
}
If You do want to keep Your expressions, one way would be to define dictionary manually like this:
var d=new Dictionary<Expression<Func<VideoDTO,object>>,
Expression<Func<VideoEntity,object>>{
{x=>x.DtoPropNumberOne,x=>x.EntityPropNumberOne} /*, {2}, {3}, etc.*/
};
And use it later:
//can You spot it?
//client does not know explicitly what expressions dictionary contains
_dataModel.Videos.Where(d[exp]).ToList<Video>();
//and I'm not 100% sure checking expression equality would actually work
If You don't want to write mapping dictionary manually, You will need some advanced techniques. One idea would be to translate dto expression to string and then back to entity expression. Here are some ideas (sorting related though) that might help. Expressions are quite complicated beasts.
Anyway - as I said, You should avoid this. Otherwise - You will produce really fragile code.

Perhaps your design goal is to prevent propagation of the data model entities to the client tier rather than to prevent a dependency between the presentation layer and data model. If viewed that way then there would be nothing wrong with the query being formed the way you state.
To go further you could expose the searchable fields from VideoEntity via an interface (IVideoEntityQueryFields) and use that as the type in the expression.
If you don't want to add an interface to your entities then the more complicated option is to use a VideoEntityQuery object and something that translates an Expression<Func<VideoEntityQuery,bool>> to an Expression<Func<VideoEntity,bool>>.

Related

OData accent-insensitive filter

How can I apply a filter accent-insensitive? In OData the "eq" operator is case and accent sensitive. The case is easy to fix, because the "tolower" but relative to the accent I'm not getting a simple solution. I know contains is supposed to be accent-insensitive but if I use contains filtering by "São José" I am only getting these responses "São José" and "São José dos Campos", it is missing "Sao Jose".
The following example filtering by "Florianopolis" is expected to return "Florianópolis", but it does not:
url: api/cidades/get?$filter=contains(nome, 'Florianopolis')
[HttpGet]
[EnableQuery]
public ActionResult<IQueryable<CidadeDTO>> Get()
{
try
{
return Ok(_mapper.Map<IEnumerable<CidadeDTO>>(_db.Cidades));
}
catch (System.Exception e)
{
return BadRequest(e.GetBaseException().Message);
}
}
It should bring aswell, like entity framework.
If your OData model was mapped directly to EF models AND an IQueryable<T> expression was passed into OK() then the query is explicitly passed through to the database engine as SQL:
SELECT * FROM Cidades WHERE nome LIKE '%Florianopolis%'
When that occurs, the Collation settings in the database connection will determine the comparison matching logic.
When your database collation is case and accent insensitive, but your data is still filtered as if it was not, then this is an indication that an IEnumerable<T> has been passed into OK() and the comparison logic is being evaluated in C# which by default in insensitive to both case and accent. Unfortunately this means that it is very likely that the entire data table has been loaded into memory first so that the filter can be applied.
In your case the OData model is mapped to DTO expressions that are mapped to the EF models via AutoMapper and that is where the generated query can break down. By calling Map() you are loading ALL records from the EF table and leaving the $filter criteria to be applied by the EnableQueryAttribute
For OData query conventions to be applied automatically you must return an IQueryable<T> from your method, or atleast pass an IQueryable<T> into the OK() response handler. With AutoMapper, you can use the Queryable Extensions to satisfy the IQueryable<T> requirement:
Queryable Extensions
When using an ORM such as NHibernate or Entity Framework with AutoMapper’s standard mapper.Map functions, you may notice that the ORM will query all the fields of all the objects within a graph when AutoMapper is attempting to map the results to a destination type.
...
ProjectTo must be the last call in the chain. ORMs work with entities, not DTOs. So apply any filtering and sorting on entities and, as the last step, project to DTOs.
In OData the last requirement (about ProjectTo) is still problematic because the EnableQueryAttribute will append the query options to the IQueryable<T> response, which will still end up materializing the entire table into memory first (IEnumerable<T>) and then apply the filter, which is still incredibly inefficient. It is this behaviour that is generally observed when someone complains about poor performance from an OData implementation, it is not always AutoMapper, but usually the pattern that the data source is loaded into memory in its entirety and then filtered. Following the default guidance for AutoMapper will lead you in this direction.
Instead we need to use an additional package: AutoMapper.Extensions.ExpressionMapping that will give us access to the UseAsDataSource extension method.
UseAsDataSource
Mapping expressions to one another is a tedious and produces long ugly code.
UseAsDataSource().For<DTO>() makes this translation clean by not having to explicitly map expressions. It also calls ProjectTo<TDO>() for you as well, where applicable.
This changes your implementation to the following:
[HttpGet]
[EnableQuery]
public ActionResult<IQueryable<CidadeDTO>> Get()
{
return Ok(_db.Cidades.UseAsDataSource().For<CidadeDTO>());
}
Don't fall into the trap of assuming that AutoMapper is necessary or best practice for an OData API implementation. If you are not using the unique features that AutoMapper provides then adding an additional abstraction layer can end up over-complicating your solution.
I'm not against AutoMapper, I use it a lot for Integrations, ETL, GraphQL and non-DDD style data schemas where the DTO models are significantly different to the underlying EF/data storage models. But it is a maintenance and performance overhead that a simple DDD data model and OData API based solution can easily do without.
Don't hire an excavator when a simple shovel will do the job.
AutoMapper is a convention based ORM that can be useful when you want to change the structure between implementation layers in your code, traditionally you might map Business Domain models that may represent aggregates or have flattened structures to highly normalised Database models.
OData is also a convention based ORM. It was designed to facilitate many of the same operations that AutoAmpper provides with the exception of Flattening and Unflattening models. These operations are deferred to the EF engine. The types exposed via OData mapping are DTOs
If your DTO models are the same relational structure as your EF models, then you would generally not use AutoMapper at all, the OData Edm mapping is optimised specifically to manage this type of workload and is designed to be and has been integrated directly into the serialization layer, making the Edm truely Data Transfer Objects that only exist over the wire and in the client.
This did the job
[HttpGet]
public ActionResult<IQueryable<PessoaDTO>> Get(ODataQueryOptions<Pessoa> options)
{
try
{
var queryResult = options.ApplyTo(_db.Pessoas);
return Ok(queryResult);
}
catch (System.Exception e)
{
return BadRequest(e.GetBaseException().Message);
}
}

'Existing Entity' constraint

I'm reading some data from an excel file, and hydrating it into an object of class A. Now I have to make sure that one of the fields of the data corresponds to the Id of a specific Entity. i.e:
class A{
protected $entityId;
}
I have to make sure that $entityId is an existing id of a specific entity (let's call it Foo). Now this can be achieved using the choice constraint, by supplying the choices option as all of the existing ids of Foo. However this will obviously cause a performance overhead. Is there a standard/better way to do this?
I'm a bit confused about what you are doing, since you seem to talk about Excel parsing, but at the same time you mention choices, which in my opinion relate to Forms.
IMO you should handle directly the relationship to your entity, instead of only its id. Most of the time it is always better to have directly the related entity as attribute of your class A than only the id, and Symfony manipulates such behaviours pretty well.
Then just have your Excel parser do something like this:
$relatedEntity = $this->relatedEntityRepository->find($entityId);
if (!$relatedEntity) {
throw new \Exception();
}
$entity->setRelatedEntity($relatedEntity);
After doing this, since you were talking about Forms, you can then use an EntityType field which will automatically perform the request in database. Use query_builder if you need to filter the results.

Workarounds for using custom methods/extension methods in LINQ to Entities

I have defined a GenericRepository class which does the db interaction.
protected GenericRepository rep = new GenericRepository();
And in my BLL classes, I can query the db like:
public List<Album> GetVisibleAlbums(int accessLevel)
{
return rep.Find<Album>(a => a.AccessLevel.BinaryAnd(accessLevel)).ToList();
}
BinaryAnd is an extension method which checks two int values bit by bit. e.g. AccessLevel=5 => AccessLevel.BinaryAnd(5) and AccessLevel.binaryAnd(1) both return true.
However I cannot use this extension method in my LINQ queries. I get a runtime error as follows:
LINQ to Entities does not recognize the method 'Boolean BinaryAnd(System.Object, System.Object)' method, and this method cannot be translated into a store expression.
Also tried changing it to a custom method but no luck. What are the workarounds?
Should I get all the albums and then iterate them through a foreach loop and pick those which match the AccessLevels?
I realize this already has an accepted answer, I just thought I'd post this in case someone wanted to try writing a LINQ expression interceptor.
So... here is what I did to make translatable custom extension methods: Code Sample
I don't believe this to be a finished solution, but it should hopefully provide a good starting point for anyone brave enough to see it through to completion.
You can only use the core extension methods and CLR methods defined for your EF provider when using Entity Framework and queries on IQueryable<T>. This is because the query is translated directly to SQL code and run on the server.
You can stream the entire collection (using .ToEnumerable()) then query this locally, or convert this to a method that is translatable directly to SQL by your provider.
That being said, basic bitwise operations are supported:
The bitwise AND, OR, NOT, and XOR operators are also mapped to canonical functions when the operand is a numeric type.
So, if you rewrite this to not use a method, and just do the bitwise operation on the value directly, it should work as needed. Try something like the following:
public List<Album> GetVisibleAlbums(int accessLevel)
{
return rep.Find<Album>(a => (a.AccessLevel & accessLevel > 0)).ToList();
}
(I'm not sure exactly how your current extension method works - the above would check to see if any of the flags come back true, which seems to match your statement...)
There are ways to change the linq query just before EF translates it to SQL, at that moment you'd have to translate your ''foreign'' method into a construct translatable by EF.
See an previous question of mine How to wrap Entity Framework to intercept the LINQ expression just before execution? and mine EFWrappableFields extension which does just this for wrapped fields.

Linq to SQL inheritance patterns

Caveat emptor, I'm new to Linq To SQL.
I am knocking up a prototype to convert an existing application to use Linq To SQL for its model (it's an MVVM app). Since the app exists, I can not change its data model.
The database includes information on events; these are either advertising events or prize events. As such, the data model includes a table (Event) with two associated tables (AdvertisingEvent and PrizeEvent). In my old C# code, I had a base class (Event) with two subclasses (AdvertisingEvent and PrizeEvent) and used a factory method to create the appropriate flavour.
This can not be done under Linq to SQL, it does not support this inheritance strategy.
What I was thinking of doing is creating an interface (IEvent) to includes the base, shared functionality (for example, a property "Description' which is implemented in each subclass). I thought I'd then add a propery to the superclass, for example SharedStuff, that would either return an AdvertisingEvent or PrizeEvent as a IEvent. From WPF I could then bind to MyEvent.SharedStuff.Description.
Does this make sense? Is there a better way to do this?
BTW: I'd rather not have to move to Linq to Entities.
You could always use interface inheritance to accomplish this. Instead of working with subclasses, have your IEvent interface, with the IPrizeEvent and IAdvertisingEvent interfaces deriving from that.
Then, work in terms of the interfaces.
You could then have separate implementations that don't derive from each other, but implement the appropriate interfaces.
Also, the nice side effect of working with interface inheritance in LINQ-to-SQL is if you have methods that operate on IQueryable<T> where the constraint on T is IEvent, you can do something like this:
// Get an IQueryable<AdvertisingEvent>
IQueryable<AdvertisingEvent> events = ...;
// A function to work on anything of type IEvent.
static IQueryable<T> FilteredEvents<T>(this IQueryable<T> query,
string description)
where T : class, IEvent
{
// Return the filtered event.
return query.Where(e => e.Description == description);
}
And then make the call like this:
events = events.FilteredEvents("my description");

LINQ To SQL entity objects as domain objects

Clearly separation of concerns is a desirable trait in our code and the first obvious step most people take is to separate data access from presentation. In my situation, LINQ To SQL is being used within data access objects for the data access.
My question is, where should the use of the entity object stop? To clarify, I could pass the entity objects up to the domain layer but I feel as though an entity object is more than just a data object - it's like passing a bit of the DAL up to the next layer too.
Let's say I have a UserDAL class, should it expose an entity User object to the domain when a method GetByID() is called, or should it spit out a plain data object purely for storing the data and nothing more? (seems like wasteful duplication in this case)
What have you guys done in this same situation? Is there an alternative method to this?
Hope that wasn't too vague.
Thanks a lot,
Martin.
I return IQueryable of POCOs from my DAL (which uses LINQ2SQL), so no Linq entity object ever leaves the DAL. These POCOs are returned to the service and UI layers, and are also used to pass data back into the DAL for processing. Linq handles this very well:
IQueryable<MyObjects.Product> products = from p in linqDataContext.Products
select new MyObjects.Product //POCO
{
ProductID = p.ProductID
};
return products;
For most projects, we use LINQ to SQL entities as our business objects.
The LINQ to SQL designer allows you to control the accessibility of the classes and properties that it generates, so you can restrict access to anything that would allow the consumer to violate the business rules and provide suitable public alternatives (that respect the business rules) in partial classes.
There's even an article on implementing your business logic this way on the MSDN.
This saves you from writing a lot of tedious boilerplate code and you can even make your entities serialisable if you want to return them from a web service.
Whether or not you create a separate layer for the business logic really depends on the size of your project (with larger projects typically having greater variation between the business logic and data access layers).
I believe LINQ to Entities attempts to provide a one-stop solution to this conundrum by maintaining two separate models (a conceptual schema for your business logic and a storage schema for your data access).
I personally don't like my entities to spread accross the layers. My DAL return POCO's (of course, it often means extra work, but I found this much cleaner - maybe that this will be simpler in the next .NET version ;-)).
The question is not so simple and there are lots of different thinking of the subject (I keep on asking myself the same question that you are).
Maybe you could take a look at the MVC Storefront sample app : I like the essence of the concept (the mapping that occurs in the data layer especially).
Hope this helps.
There is a similar post here, however, I see your question is more about what you should do, rather than how you should do it.
In small applications I find a second POCO implementation to be wasteful, in larger applications (particularly those that implement web services) the POCO object (usually a Data Transfer Object) is useful.
If your app falls into the later case, you may want to look at ADO.Net Data Services.
Hope that helps!
I have actually struggled with this, as well. Using plain vanilla LINQ to SQL, I quickly abandoned the DBML tooling, because it bound the entities to tightly to the DAL. I was striving for a higher level of persistence ignorance, although Microsoft didn't make it very easy.
What I ended up doing was hand-writing the persistence ignorance layer, by having the DAL inherit from my POCOs. The inherited objects exposed the same properties of the POCO it is inheriting from, so while inside the persistence ignorance layer, I could use attributes to map to the objects. The called then could cast the inherited object back to its base type, or have the DAL do that for them. I preferred the latter case, because it lessened the amount of casting that needed to be done. Granted, this was a primarily read-only implementation, so I would have to revisit it for more complex update scenarios.
The amount of manual coding for this is rather large, because I also have to manually maintain (after coding, to begin with) the context and provider for each data source, on top of the object inheritance and mappings. If this project was being deprecated, I would definitely move to a more robust solution.
Looking forward to the Entity Framework, persistence ignorance is a commonly requested feature according to the design blogs for the EF team. In the meantime, if you decide to go the EF route, you could always look at a pre-rolled persistence ignorance tool, like the EFPocoAdapter project on MSDN, to help.
I use a custom LinqToSQL generator, built upon one I found in the Internet, in place of the default MSLinqToSQLGenerator.
To make my upper layers independent of such Linq objects, I create interfaces to represent each one of them and then use such interfaces in these layers.
Example:
public interface IConcept {
long Code { get; set; }
string Name { get; set; }
bool IsDefault { get; set; }
}
public partial class Concept : IConcept { }
[Table(Name="dbo.Concepts")]
public partial class Concept
{
private long _Code;
private string _Name;
private bool _IsDefault;
partial void OnCreated();
public Concept() { OnCreated(); }
[Column(Storage="_Code", DbType="BigInt NOT NULL IDENTITY", IsPrimaryKey=true)]
public long Code
{
//***
}
[Column(Storage="_Name", DbType="VarChar(50) NOT NULL")]
public string Name
{
//***
}
[Column(Storage="_IsDefault", DbType="Bit NOT NULL")]
public bool IsDefault
{
//***
}
}
Of course there is much more than this, but that's the idea.
Please keep in mind that Linq to SQL is not a forward looking technology. It was released, it's fun to play with, but Microsoft is not taking it anywhere. I have a feeling it won't be supported forever either. Take a look at the Entity Framework (EF) by Microsoft which incorporates some of the Linq to SQL goodness.

Resources