This is basically asking the same question as in How to handle views in a multilayer-application. However, that post didn't receive much feedback.
Here's the problem: we have built a 3-tiered web application with the following tiers:
-Data Access (using repositories)
-Service
-UI (MVC 3)
DTO's are passed between the UI (Controller) Layer and Service Layer. Heavier Domain Models, containing a lot of domain-level logic, are passed between the Service and Data Access Layers. Everything is decoupled using IOC and the app follows SOLID principals (or tries too) --a big happy decoupled family!
Currently the DTO->Domain Model and Domain Model->DTO conversion happens all in the service layer.
So, finally to my question:
We are going to need to start displaying more complex read-only subsets of information, (i.e. summary views joining multiple entities doing rollup totals, etc). So what is the best practice for representing this type of read-only data in the n-tiered system? Having to map read-only Domain Model types to DTO types in this case doesn't make sense to me. In most cases, there would be no difference between the 2 types anyway. My thought would be to "break" the layering boundaries for these read-only types, having the Data Access Layer serve up the DTO's directly and pass those through to the Service Layer and on to the UI.
Can anyone point me in the right direction?
Much Thanks!
Your thought on breaking the layering for reading and then displaying values make sense completely. After all, the architecture/design of the system should help you and not the other way around.
Displaying report-like data to the user should be queried simply from the database and pushed to the view; no domain/dto conversion, especially if you're in a web app. You will save yourself a lot of trouble by doing this.
Personally, I had some attempts to go through these mappings just to display some read only data and it worked poorly; the performance, the unnecessary mappings, the odd things I had to do just to display some kind of report-like views. In this case, you'll likely have your domain model and a read model. You can look up CQRS pattern, it might guide you away from thinking that you want to use the same data model for both writes and reads.
So, to answer you question, I believe that in this case the best way would be to skip layering and read DTOs directly from the database through a thin layer.
Related
I guess traditionally, one would for a RESTful web service use one type of DTO objects for POJO/JSON conversion, and a separate DTO object for database entity/POJO conversion?
Spring Boot should be more opinionated and easier to use, but would you still use different DTO object types for JSON and database entity representation, or do you convert entity objects directly to JSON?
Let me share my opinion.
At first, I think your question has nothing to do with spring boot. Spring boot just provides a fancy and lightweight way to start the application and allows to build the app in an easier manner.
But still you have your rest controller there and from that point it doesn't differ much for any other type of application.
So what you're actually asking is whether it makes sense to maintain an abstraction of JSON objects and converting them to the Business Logic Entity objects and later on converting them once again to Database objects or its enough to maintain only 2 levels and ditch the Json level.
I think the answer is "it depends".
First of all, In general currently the trend is a simplification. So maybe its enough to maintain only 1 level of objects.
There are a lot of advantages of such an approach:
Obviously less code to maintain
Speed of development and testing (POJOs should be checked, converters should be tested and so forth)
Speed of execution - you don't need to waste the CPU time on conversion. A kind of obvious implication.
Less obvious: Memory consumption. Lets say you work with a big bulk of data returned by your DAO. Let's say it occupies 10MB of memory (just for the sake of example). Now if you start to convert, to Business Entities, you'll spend yet another 10MB and now if its A JSon objects, well its again 10MB. The point is that all these objects may co-exist in memory simultaneously. Of course GC will probably take care of them if you implemented everything right, but this is a different story.
However there is one drawback of such a simplification.
In one word I would call it a Commitment
There are three Types of APIs in the application.
The API you're committed to at the level of Web Service - The JSon structure.
The chances are that various clients (not necessary using the JVM at all) are running against your Web service and consume the data. So they really expect you to provide a JSon objects of the given structure.
The API of your business. If your Business logic layer is pretty complicated, you probably have an entire team that develops that logic. So you usually work at the level of APIs between the teams.
The level of DAO - the same story as Business Logic actually.
So now, what happens if you, say, change that API at one level. Does it mean that all the levels will be broken?
Example
Lets say, we don't maintain "JSon" level. In this case, if we change the API at the level of Business Logic, the JSON will also change automatically. All the rest frameworks will happily convert the object for us, and the chances are that the user will get another data.
Another example
Lets say, your BL layer provides a Person entity that looks like this:
class Person {
String firstName;
String lastName;
List<Language> languages;
}
class Language {
...
}
Now, let's say you have a UI that consumes your REST service that provides a list of Persons upon request. What if there are 2 different pages in UI. One that shows only the Persons (in this case it doesn't make sense to provide a list of language, spoken by a person).
In the second page however you want to get the full information.
So, you'll end up exposing 2 web services or complicating the existing one by some parameters (the more params like this you have, the less it resembles the rest :) )
Maybe separation would help a little here? I don't know.
Bottom line.
I would say that as long as you can live without such a separation - do it. It can work even for quite big projects. And of course it can work for small or middle-sized projects.
If you find yourself struggling around fixes and you feel like such a separation would solve the issues - do the separation.
Hope this helps to understand the implications and chose what works for you
I've been reading a lot of blogs which advocate the fat models and skinny controllers approach, esp. the Rails camp. As a result the routers is basically just figuring out what method to call on what controller and all the controller method does is call the corresponding method on the model and then bring up the view. So I've two concerns here which I don't understand:
The controller and router are really not doing much different tasks other than just calling a method on the God-like model based on the route.
Models are doing too much. Sending emails, creating relationships, deleting and modifying other models, queuing tasks, etc. Basically now you have God-like objects that are supposed to do everything that may or may not concern with modeling and dealing with data.
Where do you draw the line? Isn't this just falling into the God pattern?
It might not be the best idea to look at Rails as a staple of MVC design pattern. Said framework was made with some inherent shortcomings (I kinda elaborated on it in a different post) and the community only just now has begun addressing the fallout. You could look at DataMapper2 development as the first major step.
Some theory
People giving that advice seem to be afflicted by a quite common misconception. So let me begin by clearing it up: Model, in modern MVC design pattern, is NOT a class or object. Model is a layer.
The core idea behind MVC pattern is Separation of Concerns and the first step in it is the division between presentation layer and model layers. Just like the presentation layer breaks down into controllers (instances, responsible for dealing with user input), views (instances, responsible for UI logic) and templates/layouts, so does the model layer.
The major parts that the model layer consists of are:
Domain Objects
Also known as domain entities, business objects, or model objects (I dislike that latter name because it just adds to the confusion). These structures are what people usually mistakenly call "models". They are responsible for containing business rules (all the math and validation for specific unit of domain logic).
Storage Abstractions:
Usually implemented using data mapper pattern (do not confuse with ORMs, which have abused this name). These instances usually are tasked with information storage-from and retrieval-into the domain objects. Each domain object can have several mappers, just like there are several forms of storage (DB, cache, session, cookies, /dev/null).
Services:
Structures responsible for application logic (that is, interaction between domain objects and interaction between domain objects and storage abstractions). They should act like the "interface" through which the presentation layer interacts with the model layer. This is usually what in Rails-like code ends up in the controllers.
There are also several structures that might be in the spaces between these groups: DAOs, units of work and repositories.
Oh ... and when we talk (in context of web) about a user that interacts with MVC application, it is not a human being. The "user" is actually your web browser.
So what about deities?
Instead of having some scary and monolithic model to work with, controllers should interact with services. You pass data from user input to a specific service (for example MailService or RecognitionService). This way the controller changes the state of model layer, but it is done by using a clear API and without messing with internal structures (which would cause a leaky abstraction).
Such changes can either cause some immediate reaction, or only affect the data that the view instance requests from model layer, or both.
Each service can interact with any number (though, it's usually only a handful) of domain object and storage abstractions. For example, the RecogitionService could not care less about storage abstractions for the articles.
Closing notes
This way you get an application that can be unit-tested at any level, has low coupling (if correctly implemented) and has clearly understandable architecture.
Though, keep in mind: MVC is not meant for small applications. If you are writing a guestbook page using MVC pattern, you are doing it wrong. This pattern is meant for enforcing law and order on large scale applications.
For people who are using PHP as primary language, this post might be relevant. It's a bit longer description of the model layer with a few snippets of code.
If the "model" classes are implemented poorly yes, your concern is relevant.
A model class shouldnt be doing Email (infrastructure tasks).
The real question is what does model in MVC imply.
It isnt restricted to POCO classes with a few methods.
Model in MVC means Data and Business logic. Treat it as a superset of classic core POCO models.
View ==== Controller ==== Model ---> Business Process layer --> Core models
Throw in Infrastructure assemblies and Data Access layers and use injection to hand that into the BPL then your a process is using MVC as intended.
BPL may invoke UoW / Respository patterns, and execute business rules and call Infrastructure features by way of injected Objects or interface patters.
So the recommendation to keep a controller skinny doesnt mean the "person" class in a classic Core model should have 50 methods, and call Email directly. You are right to think this is wrong.
The Controller May still be required to instantiate and inject Infrastructure classes into the BPL or core layer if called directly. There should be a business layer or at least classes orchestrating calls across Classic Object model classes.
Well thats my "view" anyway ;-)
For generic take on MVC the wiki description http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller
A Little Blog that talks about the "M" in MVC. http://www.thedeveloperday.com/skinny-controllers/
I think you can make a distinction between a single fat model (possibly named App or Application), and several fat models broken down into logical groups (Business, Customer, Order, Message). The latter is how I structure my apps, and each model roughly corresponds to a database table in a relational database or collection in a document database. These models handle all aspects of creating, updating, and manipulating the data that makes up the model, whether it is talking to the database or calling an API. The controller is very thinm responsible for little mor that calling the appropriate model and selecting a template.
The model of the domain are my entities used as POCOs which means no base class, no interfaces around and no Attributes.
So the business logic like validation rules must be outside of the entities. (Anemic Domain Model)
Would this comply with Domain Driven Design?
No. Not really.
Main aim of domain driven design is to capture and encapsulate business domain in model explicitly as possible. Business would always contain behavior, therefore - Your objects are supposed to have behavior too.
The model of the domain are my entities used as POCOs which means no base class, no interfaces around and no Attributes.
...and no c#, .net clr should be used. that's infrastructure, right? ;)
Those are tools to express Your model. You should try to keep noise level down, separate Your model, but You won't be able to runaway completely because it's a model of real life expressed in programming language and technology around it.
Btw, You might want to investigate idea of never allowing domain object to be in invalid state. And if it feels that this particular kind of validation does not relate to business - it is not supposed to be in domain model first of all.
That's a really philosophical question. I really want to give an equally philosophical answer, so here goes:
As I have understood domain driven design, the most important thing is that whoever knows something, does things with that knowledge. I believe this to be intertwined with this article.
With this in mind, your plain old objects should have the means of performing their "life or death" - important tasks (which makes your solution wrong).
However, another way of looking at it would be that these plain old objects are the tiniest available sets of data, almost like primitives. What happens then is that the objects owning these data objects, they are the actual model objects within the domain driven design. and they don't have to correlate perfectly (which would make your solution correct).
This could easily happen if the model and the data layer are designed by two completely independent designers, or if one person is capable of switching hats. Or maybe be a little.... wohoooooo D: i'm thinking this could be a good thing though! let me give an example:
A forum
What do we need? We need users, boards, threads, and posts. The last 3 all have a "one to many" relationship in the data layer. One board has many threads, and one thread has many posts. One user also has many posts, and one user starts many threads (could be derived by finding the author of the first post in a thread, so might not have to be stored in the data layer). But what is going on in the presentation layer?
When viewing a board, we will want to see all available threads in that board. but we won't be satisfied with seeing the name of the thread, and the name of the user who started it. We also want to see the number of posts in each thread, plus the name of the last poster in the thread, and the time of that posting.
We are now looking at a model object which is somewhat out of sync with the data layer. It will contain business logic to calculate the needed data from the given data objects, and then it will be able to load some sort of view with the data that the view wants. No getters or setters will be needed in the model, so capsulation is never broken. The model object conforms to the domain, which should be dependant on the usability demands, not the limitations of data storage. The data objects conform to the old data storing style.
This would give us a data abstraction layer (with the pocos), mvc, and domain driven design. win? :)
So I was searching the web looking for best practices when implementing the repository pattern with multiple data stores when I found my entire way of looking at the problem turned upside down. Here's what I have...
My application is a BI tool pulling data from (as of now) four different databases. Due to internal constraints, I am currently using LINQ-to-SQL for data access but require a design that will allow me to change to Entity Framework or NHibernate or the next data access du jour. I also hold steadfast to decoupled layers in my apps using an IoC framework (Castle Windsor in this case).
As such, I've used the Repository pattern to abstract the actual data access code from my business layer. As a result, my business object is coded against some I<Entity>Repository interface and the IoC Container is used to manage the actual implementation. In this case, I would expect to have a concrete Linq<Entity>Repository that implements the interface using LINQ-to-SQL to do the work. Later I could replace this with an EF<Entity>Repository with no changes required to my business layer.
Also, because I'm coding against the interface, I can easily mock the repository for unit testing purposes.
So the first question that I have as I begin coding the application is whether I should have one repository per DataContext or per entity (as I've typically done)? Let's say one database contains Customers and Sales with the expected relationship. Should I have a single OrderTrackingRepository with methods that work with both entities or have a separate CustomerRepository and a different SalesRepository?
Next, as a BI tool, the primary interface is for reporting, charting, etc and often will require a "mashup" of data across multiple sources. For instance, the reality is that one database contains customer information while another handles sales information and a third holds other financial information but one of my requirements is to display aggregated information that spans all three. Plus, I have to support dynamic filtering in the UI. Obviously working directly against the LINQ-to-SQL or EF DataContext objects (Table<Entity>, for instance) will allow me to pretty much do anything. What's the best approach to expose that same functionality to my business logic when abstracting the DAL with a repository interface?
This article: link text indicates that EF4 has turned this approach around and that the repository is nothing more than an IQueryable returned from the EF DataContext which brings up a whole other set of questions.
But, I think I've rambled on enough...
UPDATE (Thanks, Steven!)
Okay, let me put a more tangible (for me, at least) example on the table and clarify a few points that will hopefully lead to an approach I can better wrap my head around.
While I understand what Steven has proposed, I have a team of developers I have to consider when implementing such things and I'm afraid they will get lost in the complexity (yes, a real problem here!).
So, let's remove any direct tie-in with Linq-to-Sql because I don't want a solution that is dependant upon the way L2S works - or even EF, for that matter. My intent has been to abstract away the data access technology being used so that I can change it as needed without requiring collateral changes to the consuming code in my business layer. I've accomplished this in the past by presenting the business layer with IRepository interfaces to work against. Perhaps these should have been named IUnitOfWork or, more to my liking, IDataService, but the goal is the same. These interfaces typically exposed methods such as Add, Remove, Contains and GetByKey, for example.
Here's my situation. I have three databases to work with. One is DB2 and contains all of the business information for a customer (franchise) such as their info and their Products, Orders, etc. Another, SQL Server database contains their financial history while a third SQL Server database contains application-specific information. The first two databases are shared by multiple applications.
Through my application, the customer may enter/upload their financial information for a given time period. When entered, I have to perform the following steps:
1.Validate the entered data against a set of static rules. For example, the data must contain a legitimate customer ID value (in the case of an upload). This requires a lookup in the DB2 database to verify that the supplied customer ID exists and is current.
2.Next I have to validate the data against a set of dynamic rules which are contained in the third (SQL Server) database. An example may be that a given value cannot exceed a certain percentage of another value.
3.Once validated, I persist the data to the second SQL Server database containing the financial data.
All the while, my code must have loosely-coupled dependencies so I may mock them in my unit tests.
As part of the analysis, I know that I have three distinct data stores to work with and about a half-dozen or so entities (at this time) that I am working with. In generic terms, I presume that I would have three DataContexts in my application, one per data store, with the entities exposed by the appropriate data context.
I could then create a separate I{repository|unit of work|service} for each entity that would be consumed by my business logic with a concrete implementation that knows which data context to use. But this seems to be a risky proposition as the number of entities increases, so does the number of individual repository|UoW|service types.
Then, take the case of my validation logic which works with multiple entities and, thereby, multiple data contexts. I'm not sure this is the most efficient way to do this.
The other requirement that I have yet to mention is on the reporting side where I will need to execute some complex queries on the data stores. As of right now, these queries will be limited to a single data store at a time, but the possibility is there that I might need to have the ability to mash data together from multiple sources.
Finally, I am considering the idea of pulling out all of the data access stuff for the first two (shared) databases into their own project and have been looking at WCF Data Services as a possible approach. This would give me the basis for a consistent approach for any application making use of this data.
How does this change your thinking?
In your case I would recommend returning IEnummerables's for your data queries for the repo. I usually aggregate calls from multiple repo's through a service class that represents the domain problem and encapsulates my business logic. To keep it clean I try keep my repros focused on the domain problem. I liken my Datacontext to a repo, and extract an interface using a T4 template to make life easier for mocking. But there is nothing stopping you using a traditional repo that encapsulates your calls. Doing it this way will allow you to switch ORM's at any stage.
EDIT: IQueryable IS NOT THE ANSWER! :-)
I have also done a lot of work in this area, and INITIALLY came to the same conclusion, however it is NOT a good solution. The point of the Repo is to abstract queries into discrete chunks of work. Exposing IQueryable is too adhoc and raises some issues later down the line. You loose your ability to scale. You loose your ability to optimize queries (Lets say I want to move to a highly optimized stored proc). You loose your ability to use IoC for the repo to switch out data access layers (switch the project from SQL to Mongo). You loose your ability to provide effective data caching in the Repo (Which is a major strength in the Repo pattern). I would recommend taking a CLOSE look as to WHY we have a Repo pattern. It isn't simply an "ORM" mapping layer. What made this really clear to me was the CQRS pattern.
Further to this allowing the ad-hoc nature of IQueryable opens you to misfitting reuse of queries. It is GENERALLY not a good idea to reuse queries, since query to query you see slight deviations, which ends up with 2 byproducts: Queries become too broad and inefficient. Queries become riddled with unmaintainable IF THEN statements to cater for the deviations.
IQueryable is easy, but opens you up to an unmaintainable mess.
Look at this SO answer. I think it shows a simplified model of what you want. IQueryable<T> is indeed our new Repository :-). DataContext and ObjectContext are our Unit of Work.
UPDATE 2:
Here is a blog post that describes the model you might be looking for.
UPDATE 3
It would be wise to hide the shared databases behind a service. This will solve several problems:
This will make the database private to the service, which makes it much easier to change the implementation when needed.
You can put the needed validation logic (for database 1) in that service and can create tests for that validation logic in that project.
Clients accessing that service can assume correctness of the service, and its validation logic.
The result of this is that your application will send data to the service to validate it. Call the service to fetch data. Query its own private database (database 3) and join the data of the three data source locally together. I've never been a fan of using cross-database or even cross-server (in your situation) database calls and letting the database join everything together. Transactions will be promoted to distributed-transactions and it's hard to predict how many data the servers will exchange.
When you abstract the shared databases behind the service, things get easier (at least from your application's point of view). Your application calls services it trusts which limits the amount of code in that application and the amount of tests. You still want to mock the calls to such a service, but that would be pretty easy. It should also solve the problem of validating over multiple data sources.
Validation is always a hard part. I'm very familiar with Validation Application block, and love it for it's flexibility. It isn't however an easy framework, but you might take a peek at what you can do with it. For instance, I've written several articles about integration with O/RM tools and how to 'embed' a context (context as in DataContext/Unit of Work) in Validation Application Block.
Please have a look at my IRepository pattern implementation using EF 4.0.
My solution has the following features:
supports connections to multiple dbs
One repository per entity
Support for execution of queries
Unit of work pattern implementation
Support for validating entities using VAB guidance
Common operations are kept at base class level. High use of OOPS techniques for code re-usability and ease of maintenance.
I just cannot seem to get my head around what exactly is the MODEL in MVP.
If I have a layered architecture PRESENTATION / APPLICATION / DOMAIN / INFRASTRUCTURE, what exactly is the MODEL?
DOMAIN objects accessed through
lower layers?
A separate object defined in the
PRESENTATION layer that maps to the
UI and uses data obtained from a
lower layer?
If someone could clear my understanding on what is the MODEL it would be greatly appreciated.
The Model is normally the group of classes/types/components that represent the core domain (business or otherwise) that your application operates within. These are the classes that perform the key logic required, often in the form of business rules, and also consume/manipulate data.
In your layered example, the Model would mostly be found in the Domain layer but could also be in the Application layer.
I think you're having difficulty understanding it because you are trying to combine two separate architectural patterns, or ways of looking at the application, being n-tier/n-layer versus MVP.
It's completely reasonable (and quite common) to use some sort of Model/View approach while at the same time applying layering in your application.
Maybe you should focus on them one at a time to start with and then overlay them when you are more familiar with both.
In any of the Model-View-* architectures, the Model is what describes the data in your application (and, if they fit the need, are passed in to the View for rendering).
If your application already has Domain objects, it very well may be the case that you could use them for your Model.
It doesn't matter what architectural guidelines you're following, M is always going to be the same thing. The Model is the piece that is specific to your domain. It's the part that really is what you're application is trying to do. The Model is supposed to represent your business domain. This goes for MVP, MVC, MVVM, etc.
If you were making a inventory system, then an Inventory class would most likely be in your Model, a Product would probably be there, an Order, you get the idea. These are the things that compose your domain logic.
The model is the data. This might just be data out of a database in DataSets, or it might be a complete domain model with objects representing your field of business.
The view is the UI, whether web pages or a Windows application or a mobile device application.
The presenter is the glue between the two and the brains of the whole outfit. Actions initiated by the view take place in the presenter. Generally in a WinForms application, for instance, a Button.Click event in my View simply calls a method on the Presenter, which then takes whatever action is necessary (and it may just be doing something back in the View).
The presenter holds a reference to the view (through an interface), and to the model. The view has a reference to the presenter (usually I strongly-type this, but it can be an interface as well). The model doesn't know about the presenter or the view.