I have the projects
Domain.Model (contains code first POCOs)
Data.Context (contains the context & migrations only)
Data.Access (contains IGenericRepository & GenericRepository)
Service (contains BL service classes and UnitsOfWork)
Presentation.Admin (an Asp.Net Webforms web application)
I am using my POCOs as business objects across all the layers. I know there is some debate about this but this also fairly accepted.
So I have the Presentation calling Service > Getting a POCO via the Repository > Returning to Presentation and displaying, for example a HTML table and saving edits back to the DB - great.
Now I have a more complex page which I think requires a business object. This is a made-up by analogous example.
POCO
public class Book
{
BookId
string ExternalReference
}
public class Movie
{
int MovieId
string ExternalReference
}
Suggested Business Object
public MovieAdaptation
{
Book book;
Movie movie;
}
So ExternalReference is external and can not be a common foreign key in my database so therefore i cannot just do Book.Movie using a navigation property. I need to do a LINQ join (probably).
So my questions are:
1) Where should I define this business object. Currently it is just in the Service layer as only things that reference the service layer will use it.
2) Where should I construct this business object? Should it be in repositories which sit in Data.Access or further up?
3) How do I construct it using LINQ. Here is my best shot so far, but it seems pretty inefficient, especially if I am returning a list of these.
namespace MyProject.Services
{
public class AdaptationsService
{
AdaptationUnitOfWork _unitOfWork;
public AdaptationService
{
unitOfWork = new AdaptationUnitOfWork();
}
public Adaption GetAdaptations(string externalReference)
{
//Can anyone improve this maybe using LINQ join (as maybe it won't be getting books/movies by SingleOrDefault but by where
Book book= _unitOfWork.BookRepository.Get.SingleOrDefault(b=>b.ExternalReference==externalReference);
Movie movie= _unitOfWork.MovieRepository.Get.SingleOrDefault(m=>m.ExternalReference==externalReference);
Adaptation adaptation = new Adaptation();
adaptation.Book=book;
adaptation.Movie=movie;
}
}
}
1) Where should I define this business object. Currently it is just in the Service layer as only things that reference the service layer will use it.
I would probably keep it in the Service Layer. It's not relevant to your DAL as it's a combination of your POCO's, not the POCO's themselves. Have the Service layer construct/destruct it (see below).
2) Where should I construct this business object? Should it be in repositories which sit in Data.Access or further up?
Construct and destruct it in the Service Layer. The DAL should only send and receive your data access objects (your POCOs). Constructing and destructing business objects is not part of it's job description. For all intents and purposes whenever you use the term business object it should be above the DAL.
3) How do I construct it using LINQ. Here is my best shot so far, but it seems pretty inefficient, especially if I am returning a list of these.
I don't have a better answer than the example you gave. It sounds like you have to perform 2 queries then construct it yourself as you're doing.
Related
After a comment of this question. I started to research but I am still confused.
Entities should convert to Dto before return to controller? To me it sounds not really practical.
We are talking about software architecture and as always when we are talking about software architecture there are a thousand ways of doing something and many opinions about what is the best way. But there is no best way, everything has advantages and disadvantages. Keep this in mind!
Typically you have different layers:
A persistence layer to store data
Business layer to operate on data
A presentation layer to expose data
Typically, each layer would use its own kind of objects:
Persistence Layer: Repositories, Entities
Business Layer: Services, Domain Objects
Presentation Layer: Controllers, DTOs
This means each layer would only work with its own objects and never ever pass them to another layer.
Why? Because you want each layer to be separated from the other layers. If you would use entities in your controller, your presentation would depend on how your data is stored. That's really bad. Your view has nothing to do with how the data is stored. It shouldn't even know that or how data is stored.
Think of that: You change your database model, e.g. you add a new column to one of your database tables. If you pass the entities to your controller (or worse: your controller exposes them as JSON), a change at the database would result in a change in your presentation. If the entities are directly exposed as JSON, this might even result in changes in JavaScript or some other clients which are using the JSON. So a simple change in the database might require a change in the JavaScript front end, because you couple your layers very tight. You definitely don't want that in a real project.
How? You doubt that this is practical, so just a small example of how to do that in (pseudo) code:
class Repository {
public Person loadById(Long id) {
PersonEntity entity = loadEntityById(id);
Person person = new Person();
person.setId(entity.getId());
person.setName(entity.getFirstName + " " + entity.getLastName());
return person;
}
}
In this example, your repository would use entities internally. No other layer knows or uses this entities! They are an implementation detail of this particular layer. So if the repository is asked to return a "person", it works on the entity, but it will return a domain object. So the domain layer which works with the repo is save in the case the entities need to be changed. And as you can see in the case of the name, the domain and the database might be different. While the database stores the name in first name and last name, the domain only know a single name. It's a detail of the persistence how it stores the name.
The same goes for controllers and DTOs, just another layer.
I was wondering, should the entities have the capability to save changes to the context? Or have business logic that relates to that particular entity? For example:
ActionResult ResetPassword(UserViewModel viewModel)
{
var user = userCollection.GetUser(viewModel.Username);
user.ResetPassword();
}
where:
class User : Entity
{
public string Password{ get; set; }
public ResetPassword()
{
Password = ""
context.SaveChanges();
}
}
I find this a bit weird since the entity would have a reference to the context. I am not sure whether this would work either - or whether this is recommended. But I want to work on a domain where I do not have to worry about saving changes, at a higher level etc. How can I accomplish this?
Thanks!
Update
I have updated my example - hope its a bit clearer now :)
According to Domain-Driven Design domain objects should have behavior.
You should definately read this book:
I would keep my Entities as POCO's(Plain Old Class Objects, classes with only properties) and have a Repositary do methods like Insert / Update.
I can keep my Entities in a separate class library and use it in different places ( even in a different project), with a different Repository implementation.
In this Tutorial It is nicely explained, how to do a Repositary & Unit Of Work Patterns on an MVC project which uses Entity Framework
You may want to consider the UnitOfWork pattern between your controller and your entities.
http://martinfowler.com/eaaCatalog/unitOfWork.html
I keep hearing about EF 4.0, POCO, IObjectSet, UnitOfWork (by the way, UoW is atleast more than 17 years old when I first heard it) etc.
So some folks talk about Repository "pattern". etc. There are numerous bloggers showcasing their concoction of a "wrapper" or repository or something similar.
But they all require IObjectSets (or in some cases - IQueryables) to be hanging off their POCOs. Expectation seems to be that you can write queries against them.
So if one needs IObjectSet and not just IList or some other simpler collection, why are we saying this is POCO and free from EF?
If I want to swap EF from underneath, I need to make sure my "other" O/R Mapper (I know I know.. EF is not just an O/R Mapper) understands IObjectSet and be able to parse the ExpressionTrees from the queries, execute and otherwise behave similar to EF.
IObjectSet is not the interface that makes an Entity POCO, it's just the persistence container IObjectSet. The point of POCO is to prevent you from having to derive your Model classes from an EF type, which the T4 POCO template in EF4 provides.
The Repository pattern is an optional additional layer of abstraction from your ORM to allow easier implementation of a different one if the need arose. Separation of concerns etc etc.
Take a look at Entity Framework Code First: http://weblogs.asp.net/scottgu/archive/2010/08/03/using-ef-code-first-with-an-existing-database.aspx
In response to the phrase: "If I want to swap EF from underneath":
In my business, it is more likely that I would swap out the database, say from Oracle to SQL Server (or vice versa), than that I would swap out the data access framework. On the other hand, there do exist options that make EF a favorable choice.
There are other LINQ providers than those provided by EF (e.g. LLBLGen). Sure, swapping out an EF data tier for NHibernate or EasyObjects would be difficult, because the frameworks do not have sufficient feature parity to ease the transition; however, LINQ was designed to open the way for other LINQ providers to step in and provide their own solution.
Your question contains a wrong statement: Correct is that POCOs do not depend on IObjectSet.
POCOs themselves are independent from EF. Or better: They are supposed to be independent from EF. Since YOU are implementing the POCO classes you are finally responsible to make this sure. (Otherwise the term POCO would be the wrong one.)
If you are using the standard T4 template to create POCO classes from a model description instead of writing the classes on your own the template ensures that the classes do not depend on EF - they are not derived from Entity and collections as members of a class are generated with ICollection by this template, not with IObjectSet.
Repository pattern is another question. The POCO T4 template does not create a Repository as an abstract interface to act on a database with POCOs. It creates a derived ObjectContext which is rather an EF specific implementation of a possible repository interface (or at least helps to easily implement a possible repository interface).
If you want to have a repository interface which doesn't depend on EF or LINQ you have to define it this way. Nothing forces you to use IObjectSet or IQueryable in that interface. Perhaps the examples of implementing the Repository pattern you saw didn't intend to be independent from Entity Framework or LINQ.
An example:
Suppose, in your business layer you need a list of all products of a given category returned from the persistance layer. What would this layer expose to fulfill the request?
If you only have databases in mind which offer a LINQ provider you might design the repository interface like so:
public interface IProductsRepository
{
IQueryable<Product> AllProducts { get; } // Product is the POCO class
}
A concrete implementation of this repository based on EF would simply return an ObjectSet<Product> from the ObjectContext which the T4 template did create.
And your business layer runs a query this way:
IProductsRepository rep = new SomeConcreteImplementationOfProductsRepository();
IList productsOfCategory =
rep.AllProducts.Select(p => p.Category == "stuff").ToList();
But if you want to be more open what kind of persistance storage you like to support it might be better to design the repository independent from IQueryable. The consequence could be that your abstract repository interface needs more specific methods to answer requests from the business layer, for instance you need now:
public interface IProductsRepository
{
IList<Product> GetProductsOfCategory(string category);
}
and the business layer does this:
IProductsRepository rep = new SomeConcreteImplementationOfProductsRepository();
IList productsOfCategory = rep.GetProductsOfCategory("stuff");
A concrete implementation of this Repository using EF (or another data framework supporting LINQ) could still leverage a LINQ query like the business layer did in the first example. But other implementations could work in another way (say: you have a "database" which stores products in one text file per category. Then the implementation for that interface method would read one specific file from disk. Or your repository implementation asks a webservice for the data, and so on...)
Key point is: If you are using POCO classes you are open for all those kinds of repositories. EF with POCO support doesn't force you to build repository interfaces based on IQueryable or even IObjectSet. It finally depends on what kind of persistance layers you have in mind. The more different they are the more specific methods you might need to support in your repository interface and the more work you'll have to implement those methods. Using IQueryable is a comfortable compromise which allows to define a simple repository interface while enabling simple implementations by EF but also other databases with LINQ provider. I think that's the only reason why you see examples of repository pattern implementations with IQueryable so often. It's not an inherent restriction imposed by EF with POCOs.
(That's how I think about it, not being an expert in design patterns, so heavy attacks and corrections in the comments are welcome.)
What is a Data Transfer Object?
In MVC are the model classes DTO, and if not what are the differences and do we need both?
A Data Transfer Object is an object that is used to encapsulate data, and send it from one subsystem of an application to another.
DTOs are most commonly used by the Services layer in an N-Tier application to transfer data between itself and the UI layer. The main benefit here is that it reduces the amount of data that needs to be sent across the wire in distributed applications. They also make great models in the MVC pattern.
Another use for DTOs can be to encapsulate parameters for method calls. This can be useful if a method takes more than four or five parameters.
When using the DTO pattern, you would also make use of DTO assemblers. The assemblers are used to create DTOs from Domain Objects, and vice versa.
The conversion from Domain Object to DTO and back again can be a costly process. If you're not creating a distributed application, you probably won't see any great benefits from the pattern, as Martin Fowler explains here.
The definition for DTO can be found on Martin Fowler's site. DTOs are used to transfer parameters to methods and as return types. A lot of people use those in the UI, but others inflate domain objects from them.
A DTO is a dumb object - it just holds properties and has getters and setters, but no other logic of any significance (other than maybe a compare() or equals() implementation).
Typically model classes in MVC (assuming .net MVC here) are DTOs, or collections/aggregates of DTOs
In general Value Objects should be Immutable. Like Integer or String objects in Java. We can use them for transferring data between software layers. If the software layers or services running in different remote nodes like in a microservices environment or in a legacy Java Enterprise App. We must make almost exact copies of two classes. This is the where we met DTOs.
|-----------| |--------------|
| SERVICE 1 |--> Credentials DTO >--------> Credentials DTO >-- | AUTH SERVICE |
|-----------| |--------------|
In legacy Java Enterprise Systems DTOs can have various EJB stuff in it.
I do not know this is a best practice or not but I personally use Value Objects in my Spring MVC/Boot Projects like this:
|------------| |------------------| |------------|
-> Form | | -> Form | | -> Entity | |
| Controller | | Service / Facade | | Repository |
<- View | | <- View | | <- Entity / Projection View | |
|------------| |------------------| |------------|
Controller layer doesn't know what are the entities are. It communicates with Form and View Value Objects. Form Objects has JSR 303 Validation annotations (for instance #NotNull) and View Value Objects have Jackson Annotations for custom serialization. (for instance #JsonIgnore)
Service layer communicates with repository layer via using Entity Objects. Entity objects have JPA/Hibernate/Spring Data annotations on it. Every layer communicates with only the lower layer. The inter-layer communication is prohibited because of circular/cyclic dependency.
User Service ----> XX CANNOT CALL XX ----> Order Service
Some ORM Frameworks have the ability of projection via using additional interfaces or classes. So repositories can return View objects directly. There for you do not need an additional transformation.
For instance this is our User entity:
#Entity
public final class User {
private String id;
private String firstname;
private String lastname;
private String phone;
private String fax;
private String address;
// Accessors ...
}
But you should return a Paginated list of users that just include id, firstname, lastname. Then you can create a View Value Object for ORM projection.
public final class UserListItemView {
private String id;
private String firstname;
private String lastname;
// Accessors ...
}
You can easily get the paginated result from repository layer. Thanks to spring you can also use just interfaces for projections.
List<UserListItemView> find(Pageable pageable);
Don't worry for other conversion operations BeanUtils.copy method works just fine.
To me the best answer to the question what is a DTO is that DTO's are simple objects that should not contain any business logic or methods implementation that would require testing.
Normally your model (using the MVC pattern) are intelligent models, and they can contain a lot of/some methods that do some different operations for that model specifically (not business logic, this should be at the controllers). However, when you transfer data (eg. calling a REST (GET/POST/whatever) endpoint from somewhere, or consuming a webservice using SOA, etc...) you do not want to transmit the big sized object with code that is not necessary for the endpoint, will consume data, and slow down the transfer.
With MVC data transfer objects are often used to map domain models to simpler objects that will ultimately get displayed by the view.
From Wikipedia:
Data transfer object (DTO), formerly known as value objects or VO, is
a design pattern used to transfer data between software application
subsystems. DTOs are often used in conjunction with data access
objects to retrieve data from a database.
All credits goes to Rick-Andreson
Production apps typically limit the data that's input and returned using a subset of the model. There are multiple reasons behind this and security is a major one. The subset of a model is usually referred to as a Data Transfer Object (DTO), input model, or view model.
A DTO may be used to:
Prevent over-posting.
Hide properties that clients are not supposed to view.
Omit some properties in order to reduce payload size.
Flatten object graphs that contain nested objects.
Flattened object graphs can be more convenient for clients.
Practical implementation of a DTO approach, by Rick-Andreson on Microsoft Web APIs best tutorials and practices using C# and ASP .Net Core 5:
The principle behind Data Transfer Object is to create new Data Objects that only include the necessary properties you need for a specific data transaction.
Benefits include:
Make data transfer more secure
Reduce transfer size if you remove all unnecessary data.
Read More: https://www.codenerd.co.za/what-is-data-transfer-objects
Some programmers use DTO to distinguish their final object data that is going to be passed through an API. So, it is basically a payload object to an endpoint. Like, you could name your contact form values object that you pass to the server as contactFormDto or contactFromPayload, then you or any other programmer know what you have in that object is final shape of the data, that is going to travel through network.
I would explain DTO to my kid as
My son, Data Transfer Object (aka DTO) **is used to encapsulate data we send from one endpoint to another.
Use DTO to define interfaces for input and output for endpoints in your system
In this context think of a system as a collection of endpoints. And endpoints can be anything between (mobile app, web app, backend API) that talk with each other.
Data transfer object (DTO) describes “an object that carries data
between processes” (Wikipedia) or an “object that is used to encapsulate data,
and send it from one subsystem of an application to another” (Stack Overflow
answer).
DefN
A DTO is a hardcoded data model. It only solves the problem of modeling a data record handled by a hardcoded production process, where all fields are known at compile-time and therefore accessed via strongly typed properties.
In contrast, a dynamic model or "property bag" solves the problem of modeling a data record when the production process is created at runtime.
The Cvar
A DTO can be modeled with fields or properties, but someone invented a very useful data container called the Cvar. It is a reference to a value. When a DTO is modeled with what I call reference properties, modules can be configured to share heap memory and thereby collaboratively work on it. This completely eliminates parameter passing and O2O communication from your code. In other words, DTOs having reference properties allow code to achieve zero coupling.
class Cvar { ... }
class Cvar<T> : Cvar
{
public T Value { get; set; }
}
class MyDTO
{
public Cvar<int> X { get; set; }
public Cvar<int> Y { get; set; }
public Cvar<string> mutableString { get; set; } // >;)
}
Source: http://www.powersemantics.com/
Dynamic DTOs are a necessary component for dynamic software. To instantiate a dynamic process, one compiler step is to bind each machine in the script to the reference properties the script defines. A dynamic DTO is built by adding the Cvars to a collection.
// a dynamic DTO
class CvarRegistry : Dictionary<string, Cvar> { }
Contentions
Note: because Wix labeled the use of DTOs for organizing parameters as an "anti-pattern", I will give an authoritative opinion.
return View(model); // MVC disagrees
My collaborative architecture replaces design patterns. Refer to my web articles.
Parameters provide immediate control of a stack frame machine. If you use continuous control and therefore do not need immediate control, your modules do not need parameters. My architecture has none. In-process configuration of machines (methods) adds complexity but also value (performance) when the parameters are value types. However, reference type parameters make the consumer cause cache misses to get the values off the heap anyway -- therefore, just configure the consumer with reference properties. Fact from mechanical engineering: reliance on parameters is a kind of preoptimization, because processing (making components) itself is waste. Refer to my W article for more information. http://www.powersemantics.com/w.html.
Fowler and company might realize the benefits of DTOs outside of distributed architecture if they had ever known any other architecture. Programmers only know distributed systems. Integrated collaborative systems (aka production aka manufacturing) are something I had to claim as my own architecture, because I am the first to write code this way.
Some consider the DTO an anemic domain model, meaning it lacks functionality, but this assumes an object must own the data it interacts with. This conceptual model then forces you to deliver the data between objects, which is the model for distributed processing. However on a manufacturing line, each step can access the end product and change it without owning or controlling it. That's the difference between distributed and integrated processing. Manufacturing separates the product from operations and logistics.
There's nothing inherently wrong with modeling processing as a bunch of useless office workers who e-mail work to one another without keeping an e-mail trail, except for all the extra work and headache it creates in handling logistics and return problems. A properly modeled distributed process attaches a document (active routing) to the product describing what operations it came from and will go to. The active routing is a copy of the process source routing, which is written before the process begins. In the event of a defect or other emergency change, the active routing is modified to include the operation steps it will be sent to. This then accounts for all the labor which went into production.
Clearly separation of concerns is a desirable trait in our code and the first obvious step most people take is to separate data access from presentation. In my situation, LINQ To SQL is being used within data access objects for the data access.
My question is, where should the use of the entity object stop? To clarify, I could pass the entity objects up to the domain layer but I feel as though an entity object is more than just a data object - it's like passing a bit of the DAL up to the next layer too.
Let's say I have a UserDAL class, should it expose an entity User object to the domain when a method GetByID() is called, or should it spit out a plain data object purely for storing the data and nothing more? (seems like wasteful duplication in this case)
What have you guys done in this same situation? Is there an alternative method to this?
Hope that wasn't too vague.
Thanks a lot,
Martin.
I return IQueryable of POCOs from my DAL (which uses LINQ2SQL), so no Linq entity object ever leaves the DAL. These POCOs are returned to the service and UI layers, and are also used to pass data back into the DAL for processing. Linq handles this very well:
IQueryable<MyObjects.Product> products = from p in linqDataContext.Products
select new MyObjects.Product //POCO
{
ProductID = p.ProductID
};
return products;
For most projects, we use LINQ to SQL entities as our business objects.
The LINQ to SQL designer allows you to control the accessibility of the classes and properties that it generates, so you can restrict access to anything that would allow the consumer to violate the business rules and provide suitable public alternatives (that respect the business rules) in partial classes.
There's even an article on implementing your business logic this way on the MSDN.
This saves you from writing a lot of tedious boilerplate code and you can even make your entities serialisable if you want to return them from a web service.
Whether or not you create a separate layer for the business logic really depends on the size of your project (with larger projects typically having greater variation between the business logic and data access layers).
I believe LINQ to Entities attempts to provide a one-stop solution to this conundrum by maintaining two separate models (a conceptual schema for your business logic and a storage schema for your data access).
I personally don't like my entities to spread accross the layers. My DAL return POCO's (of course, it often means extra work, but I found this much cleaner - maybe that this will be simpler in the next .NET version ;-)).
The question is not so simple and there are lots of different thinking of the subject (I keep on asking myself the same question that you are).
Maybe you could take a look at the MVC Storefront sample app : I like the essence of the concept (the mapping that occurs in the data layer especially).
Hope this helps.
There is a similar post here, however, I see your question is more about what you should do, rather than how you should do it.
In small applications I find a second POCO implementation to be wasteful, in larger applications (particularly those that implement web services) the POCO object (usually a Data Transfer Object) is useful.
If your app falls into the later case, you may want to look at ADO.Net Data Services.
Hope that helps!
I have actually struggled with this, as well. Using plain vanilla LINQ to SQL, I quickly abandoned the DBML tooling, because it bound the entities to tightly to the DAL. I was striving for a higher level of persistence ignorance, although Microsoft didn't make it very easy.
What I ended up doing was hand-writing the persistence ignorance layer, by having the DAL inherit from my POCOs. The inherited objects exposed the same properties of the POCO it is inheriting from, so while inside the persistence ignorance layer, I could use attributes to map to the objects. The called then could cast the inherited object back to its base type, or have the DAL do that for them. I preferred the latter case, because it lessened the amount of casting that needed to be done. Granted, this was a primarily read-only implementation, so I would have to revisit it for more complex update scenarios.
The amount of manual coding for this is rather large, because I also have to manually maintain (after coding, to begin with) the context and provider for each data source, on top of the object inheritance and mappings. If this project was being deprecated, I would definitely move to a more robust solution.
Looking forward to the Entity Framework, persistence ignorance is a commonly requested feature according to the design blogs for the EF team. In the meantime, if you decide to go the EF route, you could always look at a pre-rolled persistence ignorance tool, like the EFPocoAdapter project on MSDN, to help.
I use a custom LinqToSQL generator, built upon one I found in the Internet, in place of the default MSLinqToSQLGenerator.
To make my upper layers independent of such Linq objects, I create interfaces to represent each one of them and then use such interfaces in these layers.
Example:
public interface IConcept {
long Code { get; set; }
string Name { get; set; }
bool IsDefault { get; set; }
}
public partial class Concept : IConcept { }
[Table(Name="dbo.Concepts")]
public partial class Concept
{
private long _Code;
private string _Name;
private bool _IsDefault;
partial void OnCreated();
public Concept() { OnCreated(); }
[Column(Storage="_Code", DbType="BigInt NOT NULL IDENTITY", IsPrimaryKey=true)]
public long Code
{
//***
}
[Column(Storage="_Name", DbType="VarChar(50) NOT NULL")]
public string Name
{
//***
}
[Column(Storage="_IsDefault", DbType="Bit NOT NULL")]
public bool IsDefault
{
//***
}
}
Of course there is much more than this, but that's the idea.
Please keep in mind that Linq to SQL is not a forward looking technology. It was released, it's fun to play with, but Microsoft is not taking it anywhere. I have a feeling it won't be supported forever either. Take a look at the Entity Framework (EF) by Microsoft which incorporates some of the Linq to SQL goodness.