I am learning DDD and making use of the CQRS pattern. I don't understand how to validate business rules in a command handler without reading from the data store.
For example, Chris wants to give Ashley a gift.
The command might be GiveGiftCommand.
At what point would I verify Chris actually owns the gift he wants to give? And how would I do that without reading from the database?
There are different views and opinions about validation in command handlers.
Command can be rejected, we can say No to the command if it is not valid.
Typically you would have a validation that occurs on the UI and that might be duplicated inside the command handler (some people tend to put it in the domain too). Command handler then can run simple validation that can occur outside of the entity like is data in correct format, are there expected values, etc.
Business logic, on the other hand, should not be in a command handler. It should be in your domain.
So I think that the underlying question is...
Should I query the read side from Command Handlers?
I would say no. Do not use the read model in the command handlers or domain logic. But you can always query your read model from the client to fetch the data you need in for your command and to validate the command. You would query the read side on the client to check would if Chris actually owns the gift he wants to give. Of course, the validation involving a read model is likely to be eventually consistent, which is of course another reason a command could be rejected from the aggregate, inside the command handler.
Some people disagree saying that if you require your commands to contain the data the handler needs to validate your command, than you can never change the validation logic within your handler/domain without affecting the client as well. This exposes too much of the domain knowledge to the client and go against the fact that the client only wants to express an intent. So they would tend to provide an GiftService interface (which are part of the ubiquitous language) to your command handler and then implement the interface as needed - which might include querying the read side.
I think that the client should always assume that the commands it issues will succeed. Calling the read side to validate the command shouldn't be needed. Getting two contradictory commands is very unlikely (users creating accounts with the same email address). Then you should have a mean to issue a corrective action, something like for example a Saga/Process Manager. So instead making a corrective action would be less problematic that if the command could have been validated and not dispatched in the first place.
It depends if the operation is async or not i.e does the user expect some immediate response. Gift ownership is basically a security feature and that can be done as a 'prep' operation before invoking the actual service or sending the GiveGiftCommand.
The only command validation you can do is to make sure it contains the data in the required format (UI validation) and that the user has the permissions to do that action. But after the command is sent it's up to the Domain to decide if other business constraints are respected.
If the user expects some immediate feedback, you can actually 'wait' until the command is completed and for that you can use an approach where a command handler can provide a result to the sender using a mediator . But this implies that at least some commands are executed in an immediate manner and that might not be the case in your app. However, this is the simplest approach if you want to just return a message error as opposed to implementing compensations and other stuff. Some use cases are simple.
About command handlers and business logic, I disagree with Tomasz Jaskuλa . A command handler is a function, a technical detail. You can put business logic in a command handler or a static function, it doesn't matter. Messages and their handlers are infrastructural components that can be used to implement functionality. For example, in an app you can have Domain Events, Application Events etc . They're all events i.e notification that something changed and you can have event handlers that reside in Domain or in other places.
There's no rule preventing you to 'read' from the db, however at least the read model theoretically is stale. However this might not be such a problem in 99% of cases. For the rest 1% you need very specific solutions.
I just asked the exact same question from a knowledgeable friend of mine and his answer was that it is OK to do this validation inside the command handler (in my case inside an akka persistent actor that interprets commands and writes events to the journal).
However if that is not possible for performance reasons (because processing validation inside a persistent actor would block the actor and that would be a scalability bottleneck for the whole app) then optimistic locking (OCC) can be used.
In other words the validation can be performed in an other actor (lets call it the validator actor - which is not in the persistent actor) , this will not block the persistent actor but it might happen that the data that is used for the validation has changed in the persistent actor while the validation was running in the validator actor) .
If the validator actor returns with an OK and all the data that has been used for the validation is still the same in the persistent actor (has the same version - version as in OCC) as it has been at the point when the command arrived to the command handler (persistent actor) then the command is accepted by the persistent actor , otherwise the validation needs to be resubmitted for re-evaluation to the validator actor.
Related
Up until now I have been handling authorization in the CommandHandlers.
An example is I have an aggregate "Team" containing a list of managers (AggregateIdentifier from a User). All command handlers in the Team aggregate then verify the user executing the command is manager of the team.
The userId is injected as metadata in a CommandHandlerInterceptor based on the SecurityContext.
My main concern is, when I use sagas, it becomes an additional overhead to maintain the user context across the commands issued against different aggregates. Aside from that, the manager association can expire in the period the saga is running and subsequent failing commands, leading to an incomplete state which also needs to be handled with some rollback functionality.
Is it better to do the authorization in my controller layer to avoid the additional overhead or should I see it more as good practice to let my CommandHandlers decide whether the command is valid for the aggregate?
Authorization to perform certain operations/commands is something which I'd argue isn't domain specific logic. Instead, it is more a form of cross cutting concern which you need throughout your application. Thus, placing it in the #CommandHandler annotated method is not the ideal place in my head. However, placing it close by makes a lot of sense.
You have pointed out you are already using a CommandHandlerInterceptor to populate the Spring SecurityContext, thus I am assuming you are using a CommandDispatchInterceptor to populate the command's MetaData with information when you send a command out. This is a great use of the interceptor logic indeed, so I'd keep that in place. This however set's the information, it doesn't validate it.
To that end, you could build your own Handler Enhancer, which validates security metadata on a command. You could even build a dedicated annotation you'd add next to the #CommandHandler annotation, which describes the required roles. That way, the method still portrays what roles you need for the given command, but the actual validation can be done in this Handler Enhancer for you.
Now, let's circle back to your question:
Is it better to do the authorization in my controller layer to avoid the additional overhead or should I see it more as good practice to let my CommandHandlers decide whether the command is valid for the aggregate?
I think it's fine to do it in the aggregate, potentially making it cleaner through use of a Handler Enhancer. When it comes to your concern in the Saga, well, I think you should see that separate. The Saga handles events, facts that something has happened. Ignoring that fact because somebody whom initiated the operations which led to this fact doesn't have the rights doesn't resolve the point that it still has happened. Added, you are indeed not guaranteed on the timing of the Saga at all. Maybe your Saga deals with historical events, meaning it is completely out of scope.
If possible within your system, I would regard any command the Saga wants to publish as being sent by a "system user". The Saga is not something your users (which have specific roles) will directly influence; it is all indirect. The Saga is internal to your system, hence it is the system describing the intent to perform an operation.
That's my two cents to the situation, hope this helps you out #Vincent!
So, I'm working on a CQRS/ES project in which we are having some doubts about how to handle trivial problems that would be easy to handle in other architectures
My scenario is the following:
I have a customer CRUD REST API and each customer has unique document(number), so when I'm registering a new customer I have to verify if there is another customer with that document to avoid duplicity, but when it comes to a CQRS/ES architecture where we have eventual consistency, I found out that this kind of validations can be very hard to address.
It is important to notice that my problem is not across microservices, but between the command application and the query application of the same microservice.
Also we are using eventstore.
My current solution:
So what I do today is, in my command application, before saving the CustomerCreated event, I ask the query application (using PostgreSQL) if there is a customer with that document, and if not, I allow the event to go on. But that doesn't guarantee 100%, right? Because my query can be desynchronized, so I cannot trust it 100%. That's when my second validation kicks in, when my query application is processing the events and saving them to my PostgreSQL, I check again if there is a customer with that document and if there is, I reject that event and emit a compensating event to undo/cancel/inactivate the customer with the duplicated document, therefore finishing that customer stream on eventstore.
Altough this works, there are 2 things that bother me here, the first thing is my command application relying on the query application, so if my query application is down, my command is affected (today I just return false on my validation if query is down but still...) and second thing is, should a query/read model really be able to emit events? And if so, what is the correct way of doing it? Should the command have some kind of API for that? Or should the query emit the event directly to eventstore using some common shared library? And if I have more than one view/read? Which one should I choose to handle this?
Really hope someone could shine a light into these questions and help me this these matters.
For reference, you may want to be reviewing what Greg Young has written about Set Validation.
I ask the query application (using PostgreSQL) if there is a customer with that document, and if not, I allow the event to go on. But that doesn't guarantee 100%, right?
That's exactly right - your read model is stale copy, and may not have all of the information collected by the write model.
That's when my second validation kicks in, when my query application is processing the events and saving them to my PostgreSQL, I check again if there is a customer with that document and if there is, I reject that event and emit a compensating event to undo/cancel/inactivate the customer with the duplicated document, therefore finishing that customer stream on eventstore.
This spelling doesn't quite match the usual designs. The more common implementation is that, if we detect a problem when reading data, we send a command message to the write model, telling it to straighten things out.
This is commonly referred to as a process manager, but you can think of it as the automation of a human supervisor of the system. Conceptually, a process manager is an event sourced collection of messages to be sent to the command model.
You might also want to consider whether you are modeling your domain correctly. If documents are supposed to be unique, then maybe the command model should be using the document number as a key in the book of record, rather than using the customer. Or perhaps the document id should be a function of the customer data, rather than being an arbitrary input.
as far as I know, eventstore doesn't have transactions across different streams
Right - one of the things you really need to be thinking about in general is where your stream boundaries lie. If set validation has significant business value, then you really need to be thinking about getting the entire set into a single stream (or by finding a way to constrain uniqueness without using a set).
How should I send a command message to the write model? via API? via a message broker like Kafka?
That's plumbing; it doesn't really matter how you do it, so long as you are sure that the command runs within its own transaction/unit of work.
So what I do today is, in my command application, before saving the CustomerCreated event, I ask the query application (using PostgreSQL) if there is a customer with that document, and if not, I allow the event to go on. But that doesn't guarantee 100%, right? Because my query can be desynchronized, so I cannot trust it 100%.
No, you cannot safely rely on the query side, which is eventually consistent, to prevent the system to step into an invalid state.
You have two options:
You permit the system to enter in a temporary, pending state and then, eventually, you will bring it into a valid permanent state; for this you could allow the command to pass, yield CustomerRegistered event and using a Saga/Process manager you verify against a uniquely-indexed-by-document-collection and issue a compensating command (not event!), i.e. UnregisterCustomer.
Instead of sending a command, you create&start a Saga/Process that preallocates the document in a uniquely-indexed-by-document-collection and if successfully then send the RegisterCustomer command. You can model the Saga as an entity.
So, in both solution you use a Saga/Process manager. In order for the system to be resilient you should make sure that RegisterCustomer command is idempotent (so you can resend it if the Saga fails/is restarted)
You've butted up against a fairly common problem. I think the other answer by VoicOfUnreason is worth reading. I just wanted to make you aware of a few more options.
A simple approach I have used in the past is to create a lookup table. Your command tries to register the key in a unique constraint table. If it can reserve the key the command can go ahead.
Depending on the nature of the data and the domain you could let this 'problem' occur and raise additional events to mark it. If it is something that's important to the business/the way the application works then you can deal with it either manually or at the time via compensating commands. if the latter then it would make sense to use a process manager.
In some (rare) cases where speed/capacity is less of an issue then you could consider old-fashioned locking and transactions. Admittedly these are much better suited to CRUD style implementations but they can be used in CQRS/ES.
I have more detail on this in my blog post: How to Handle Set Based Consistency Validation in CQRS
I hope you find it helpful.
Let's assume we host two microservices: RealEstate and Candidate.
The RealEstate service is responsible for managing rental properties, landlords and so forth.
The Candidate service provides commands to apply for a rental property.
There would be a CandidateForRentalProperty command which requires the RentalPropertyId and all necessary Candidate information.
Now the crucial point: Different types of RentalPropertys require a different set of Candidate information.
Therefore the commands and aggregates got splitten up:
Commands: CandidateForParkingLot, CandidateForFlat, and so forth.
Aggregates: ParkingLotCandidature, FlatCandidature, and so forth.
The UI asks the read model to decide which command has to be called.
It's reasonable for me to validate the Candidate information and all the business logic involved with that in the Candidate domain layer, but leave out validation whether the correct command got called based on the given RentalPropertyId. Reason: Multiple aggregates are involved in this validation.
The microservice should be autonomous and it's read model consumes events from the RealEstate domain, hence it's not guaranteed to be up to date. We don't want to reject candidates based on that but rather use eventual consistency.
Yes, this could lead to inept Candidate information used for a certain kind of RentalProperty. Someone could just call the CandidateForFlat command with a parking lot rental property id.
But how do we handle the cases in which this happens?
The RealEstate domain does not know anything about Candidates.
Would there be an event handler which checks if there is something wrong and execute an appropriate command to compensate?
On the other hand, this "mapping" is domain logic and I'd like to accomodate it in the domain layer. But I don't know who's accountable for this kind of compensating measures. Would the Candidate aggregate be informed, like IneptApplicationTypeUsed or something like that?
As an aside - commands are usually imperative verbs. ApplyForFlat might be a better spelling than CandidateForFlat.
The pattern you are probably looking for here is that of an exception report; when the candidate service matches a CandidateForFlat message with a ParkingLot identifier, then the candidate service emits as an output a message saying "hey, we've got a problem here".
If a follow up message fixes the problem -- the candidate service gets an updated message that fixes the identifier in the CandidateForFlat message, or the candidate service gets an update from real estate announcing that the identifier actually points to a Flat, then the candidate service can emit another message "never mind, the problem has been fixed"
I tend to find in this pattern that the input commands to the service are really all just variations of handle(Event); the user submitted, the http request arrived; the only question is whether or not the microservice chooses to track that event. In other words, the "command" stream is just another logical event source that the microservice is subscribed to.
As you said, validation of commands should be performed at the point of command generation - at client side - where read models are available.
Command processing is performed by aggregate, so it cannot and should not check validity or existence of other aggregates. So it should trust a command issuer.
If commands comes from an untrusted environment like public API, then your API gateway becomes a client, and it should have necessary read models to validate references.
If you want to accept a command fast and check it later, then log events like ClientAppliedForParkingLot, and have a Saga/Process manager handle further workflow by keeping its internal state, and issuing commands like AcceptApplication or RejectApplication.
I understand the need for validation but I don't think the example you gave calls for cross-Aggregate (or cross-microservice for that matter) compensating measures as stated in the Q title.
Verifications like checking that the ID the client gave along with the flat rental command matches a flat and not a parking lot, that the client has permission to do that, and so forth, are legitimate. But letting the client create such commands in the wild and waiting for an external actor to come around and enforce these rules seems subpar because the rules could be made intrinsic properties of the object originating the process.
So what I'd recommend is to change the entry point into the operation - to create the Candidature Aggregate Root as part of another Aggregate Root's behavior. If that other Aggregate (RentalProperty in our case) lives in another Bounded Context/microservice, you can maintain a list of RentalProperties in the Candidate Bounded Context with just the amount of info needed, and initiate the Candidature from there.
So you would have
FlatCandidatureHandler ==loads==> RentalProperty ==creates==> FlatCandidature
or
FlatCandidatureHandler ==checks existence==> local RentalProperty data
==creates==> FlatCandidature
As a side note, what could actually necessitate compensating actions are factors extrinsic to the root object of the process. For instance, if the property becomes unavailable in the mean time. Then whatever Aggregate holds that information should emit an event when that happens and the compensation should be initiated.
I'm developing small CQRS+ES framework and develop applications with it. In my system, I should log some action of the client and use it for analytics, statistics and maybe in the future do something in domain with it. For example, client (on web) download some resource(s) and I need save date, time, type (download, partial,...), from region or country (maybe IP), etc. after that in some view client can see count of download or some complex report. I'm not sure how to implement this feather.
First solution creates analytic context and some aggregate, in each client action send some command like IncreaseDownloadCounter(resourced) them handle the command and raise domain event's and updating view, but in this scenario first download occurred and after that, I send command so this is not really command and on other side version conflict increase.
The second solution is raising event, from client side and update the view model base on it, but in this type of handling my event not store in event store because it's not raise by command and never change any domain context. If is store it in event store, no aggregate to handle it after fetch for some other use.
Third solution is raising event, from client side and I store it on other database may be for each type of event have special table, but in this manner of event handle I have multiple event storage with different schema and difficult on recreating view models and trace events for recreating contexts states so in future if I add some domain for use this type of event's it's difficult to use events.
What is the best approach and solution for this scenario?
First solution creates analytic context and some aggregate
Unquestionably the wrong answer; the event has already happened, so it is too late for the domain model to complain.
What you have is a stream of events. Putting them in the same event store that you use for your aggregate event streams is fine. Putting them in a separate store is also fine. So you are going to need some other constraint to make a good choice.
Typically, reads vastly outnumber writes, so one concern might be that these events are going to saturate the domain store. That might push you towards storing these events separately from your data model (prior art: we typically keep the business data in our persistent book of record, but the sequence of http requests received by the server is typically written instead to a log...)
If you are supporting an operational view, push on the requirement that the state be recovered after a restart. You might be able to get by with building your view off of an in memory model of the event counts, and use something more practical for the representations of the events.
Thanks for your complete answer, so I should create something like the ES schema without some field (aggregate name or type, version, etc.) and collect client event in that repository, some offline process read and update read model or create command to do something on domain space.
Something like that, yes. If the view for the client doesn't actually require any validation by your model at all, then building the read model from the externally provided events is fine.
Are you recommending save some claim or authorization token of the user and sender app for validation in another process?
Maybe, maybe not. The token describes the authority of the event; our own event handler is the authority for the command(s) that is/are derived from the events. It's an interesting question that probably requires more context -- I'd suggest you open a new question on that point.
I am new to CQRS and I am tying to make sense of business rule validation within the write side (domain). I know that client side validation should be done in terms of valid date (required field, string length, valid email, etc) and business rule/business domain related validation should be done in domain side. Actually, same client side validation rules should also be applied to command in the domain since we don't trust the users.
So, we have a valid command (AddEmailToCustomer) and the command handler is invoked on the command. Here is my approach to validation.
Create instances of two command validators in the command handler.
First one validates the command data same as client side validation (required field, valid email, etc.)
Second validator validates the data based on logic within the second validator. Something like "is this customer active", or what ever. I know the changing email doesn't fit here but it is not important. Important thing is there is a business validation here.
We look at the ValidationResult returned by the Validator.Validate(ICommand cmd) and we find out there are errors
We will not get a customer from repository to call on the UpdateEmail method on the AR. So what do we do at this point?
Do I throw and exception in the command handler and add these errors there?
Do I send the command to error queue or somewhere else?
Do I respond with something like Bus.Reply and return error code? If so, what do I do with the error messages?
How do I communicate these errors to the user? I know I can email them later but in a web scenario I can send a request id in the command (or use the message id), and poll for response with the request id and display the error messages to user.
Your guidance is appreciated.
Thanks
It's important to know that commands can be rejected after they are sent to the handler.
At the very least, you could encounter a concurrency violation that cannot be detected until the aggregate root is touched.
But also, the validation that can occur outside of the entity is simple validation. Not only string lengths, numeric ranges, regex matching, etc. but also validation that can be reasonably satisfied through a query or view, like uniqueness in a collection. It's important to remember that validation involving a materialized view is likely to be eventually consistent, which is another reason a command could be rejected from the aggregate, inside the command handler. That said, to preempt the situation, I often use read models to drive UI choices which only allow valid actions.
Validation that cannot occur outside an entity is your business logic validation. This validation is dependent upon the context in which it is run (see Udi Dahan's Clarified CQRS).
Business logic should not be in a separate validation service. It should be in your domain.
Also, I'm of the opinion that validation which occurs in the UI should be re-checked not in the command handler, but in the domain too. That validation is there to prevent corruption to the domain -- if it is not performed outside the domain then the domain is still subject to invalid parameters.
Using command handlers to duplicate this validation is only a convention. If no other front end is sending commands, then it is a useless duplicate. If there are multiple front ends, it is just one choice of where to place the then-necessary duplicate validation, and in those cases I prefer to handle it in the domain.
Lastly, you will need to bubble up commands rejected from within the handler. I accomplish this with exceptions as much as possible.