I’ve been studying the actor model for some time and trying to figure out how to correctly combine it with a RESTful API. I’m struggling how to separate responsibilities of both layers, either by using the ask-pattern or the actor-per-request. With both patterns request-reply semantics are leaking into the actor model, which seems like an anti-pattern. Most messages, initiated by an HTTP-requests, send to an actor require a reply. The receiving actors have multiple conditionals where it needs signals the API it cannot fulfil the request.
Furthermore, what is considered good practice in regard to input validation; should this be implemented as part of the HTTP (for example if field X is a valid email address, if field Y holds an integer). And for complex domain logic, how/should the actor notify the sender when a (pre-)condition fails?
While request/reply is an anti-pattern in inter-actor communication, nothing stands on your way to use it from outside of an actor system. You can use Ask from there and by using combination of Forward + Tell back to original sender send the reply without necessarily using request/reply model inside of an actor.
When it comes to input validation, ofc the simple validation (field presence, email format etc.) can be easily done on the web framework's level. However more advanced cases (like permission management) will probably make use of actors - at least if your business logic uses them as well.
For complex scenarios - try to think with protocols. Describe set of contracts between actors and/or external services, and use messages to control flow of your logic. It's usually hard to describe that kind of reasoning, but it's usually very easy to draw it with pencil ;)
I.e. you may decide to use some kind of AuthorizationGate actor, which given an unathorized request, will validate it: on auth failure it sends back some RequestFailed message back to original sender (the asker), on success it could transform that message into ValidRequest and send it to actor responsible for handling that message type. Then an actor (which handles only valid requests), processes it, sends RequestSucceed or RequestFailed back to original sender (remember to either store that sender as message field, or use actorRef.Forward instead of actorRef.Tell so you don't override it).
Related
I am prettty new to microservices, and I am trying to figure out how to set a micro-service architecture in which my publisher that emits an event, can receive a response with data from the consumer within the publisher?
From what i have read about message-brokers and message-queues, it seems like it's one-way communication. The producer emits an event (or rather, sends a message) which is handled by the message broker, and then the consumer consumes that event and performs some action.
This allows for decoupled code, which is part of what im looking for, but i dont understand if the consumer is able to return any data to the caller.
Say for example I have a microservice that communicates with an external API to fetch data. I want to be able to send a message or emit an event from my front-facing server, which then calls the service that fetches data, parses the data, and then returns that data back to my servver1 (front-facing server)
Is there a way to make message brokers or queues bidirectional? Or is it only useable in one direction. I keep reading message brokers allow services to communicate with each other, but I only find examples in which data flow goes one way.
Even reading rabbitMQ documentation hasn't really made it very clear to me how i could do this
In general, when talking about messaging, it's one-way.
When you send a letter to someone you're not opening up a mind-meld so that they telepathically communicate their response to you.
Instead, you include a return address (or some other means of contacting you).
So to map a request-response interaction when communicating with explicit messaging (e.g. via a message queue), the solution is the same: you include some directions which the recipient can/will interpret as "send a response here". That could, for instance be, "publish a message on this queue with this correlation ID".
Your publisher then, after sending this message, subscribes to the queue it's designated and waits for a message with the expected correlation ID.
Needless to say, this is fairly elaborate: you are, in some sense, reimplementing a decent portion of a session protocol like TCP on top of a datagram protocol like IP (albeit in this case, we may have some stronger reliability guarantees than we'd get from IP). It's worth noting that this sort of request-response interaction intrinsically couples the two parties (we can't really say "sender and receiver": each is the other's audience), so we're basically putting in some effort to decouple the two sides and then some more effort to recouple them.
With that in mind, if the actual business use case calls for a request-response interaction like this, consider implementing it with an actual request-response protocol (e.g. REST over HTTP or gRPC...) and accept that you have this coupling.
Alternatively, if you really want to pursue loose coupling, go for broke and embrace the asynchronicity at the heart of the universe (maybe that way lies true enlightenment?). Have your publisher return success with that correlation ID as soon as its sent its message. Meanwhile, have a different service be tracking the state of those correlation IDs and exposing a query interface (CQRS, hooray!). Your client can then check at any time whether the thing it wanted succeeded, even if its connection to your publisher gets interrupted.
Queues are the wrong level of abstraction for request-reply. You can build an application out of them, but it would be nontrivial to support and operate.
The solution is to use an orchestration system like temporal.io or AWS Step Functions. These services out of the box provide state management, asynchronous communication, and automatic recovery in case of various types of failures.
Let's consider the flow below:
API client calls [POST] /api/v1/invitation/:InvitationId/confirm
Confirm the invitation within a SAGA
Eventually raise an InvitationConfirmed event to indicate success
We are some troubles finding a good place to validate the "event" we pass to the SAGA. For instance, we want to make sure that:
- The specified InvitationId exists
- The corresponding invitation is not expired or already processed
We tried a couple of things:
Fire a command:
Fire a command RequestInvitationConfirmation
Handle synchronously this command and return an error if the command is not valid OR otherwise raise the InvitationConfirmationRequested event.
The rest of the flow is the same
CONS:
- Requires us to follow a "request/response" pattern (synchronous within the HTTP request lifetime)
Raise an event:
Raise an event InvitationConfirmationRequested
Within the SAGA, query the Invitation service and perform the validations. If the command is not valid, we publish an event InvitationConfirmationFailed
(...)
CONS:
- As far as I understand SAGA should be used to orchestrate the flow. Here we are introducing the concept of "validation". I'm not sure it's the recommended approach.
Validation is a very common concept. How would you handle it in a distributed fully asynchronous system?
Important point in the design of this system is: "Who is the client of this API?".
If this client is an internal Service or Application that's one thing (as in a distributed app, microservices etc.).
If the API is used by third party client's, that's another thing.
Short answer
If the API is used internally between Services, sending a command with invalid Id in the system is a fault, so it should be logged and examined by the system developers. Also cases like these should be accounted for by having a manual way of fixing them (by some administrative backend). Log these kinds of stuff and notify developers.
If the API is used from third party apps, then it matters how responsibilities are separated between the API and the other part of the system that it uses. Make the API responsible for validation and don't send commands with invalid id's. Treat command with invalid ID's like fault, as in the first case. In this case if you use asynchronous flow, you will need a way to communicate with the third party app to notify it. You can use something like WebHooks.
For the second part of the validations check these series of blog posts and the original paper.
Long answer
If you search around you will see a lot of discussions on errors and validations, so here's my take on that.
Since we do separation of other parts of our systems, it's seems natural to separate the types of error that we have. You can check this paper on that topic.
Let's define some error types.
Domain Errors
Application Errors
Technical Errors (database connections lost etc.)
Because we have different types of errors, the validation should be performed from different parts of our systems.
Also the communication of these errors can be accomplished by different mechanisms depending on:
the requester of the operation and the receiver
the communication channel used
the communication type: synchronous or asynchronous
Now the validations that you have are:
Validate that an Invitation with the specified Id exists
Validate that the Invitation has not expired
Validate that the Invitation is not already processed (accepted, rejected etc.)
How this is handled will depend on how we separate the responsibilities in our application. Let's use the DesignByContract principle and define clear rules what each layer (Domain, Application etc.) should expect from the other ones.
Let's define a rule that a Command containing an InvitationId that doesn't correspond to an existing Invitation should not be created and dispatched.
NOTE the terminology used here can vary vastly depending of what type of architecture is used on the project (Layered Architecture, Hexagonal etc.)
This forces the CommandCreator to validate that an Invitation exists with the specified Id before dispatching the command.
In the case with the API, the RouteHandler (App controller etc.) that will accept the request will have to:
perform this validation himself
delegate to someone else to do the validation
Let's further define that this is part of our ApplicationLayer (or module, components etc. doesn't matter how it's called, so I'll use Layer) and make this an ApplicationError. From here we can do it in many different ways.
One way is to have a DispatchConfirmInvitationCommandApplicationService that will ask the DomainLayer if an Invitation with the requested Id exists and raise an error (throw exception for example) if it doesn't. This error will be handled by the RouteHandler and will be send back to the requester.
You can use both a sync and async communication. If it's async you will need to create a mechanism for that. You can refer to EnterpriseIntegrationPatterns for more information on this.
The main point here is: It's not part of the Domain
From here on, everyone else in our system should consider that the invitation with the specified Id in the ConfirmInvitationCommand exists. If it doesn't, it's treated like a fault in the system and should be checked by developers and/or administrators. There should be a manual way (an administrative backend) to cancel such invalid commands, so this must be taken into account when developing the system, bu treated like a fault in the system.
The other two validations are part of the Domain.
So let's say you have a
Invitation aggregate
InvitationConfirmationSaga
Let's make them these aggregates communicate with messages. Let's define these types of messages:
RequestConfirmInvitation
InvitationExpired
InvitationAlreadyProcessed
Here's the basic flow:
ConfirmInvitationCommand starts a InvitationConfirmationSaga
InvitationConfirmationSaga send RequestConfirmInvitation message to Invitation
And then:
If the Invitation is expired it sends InvitationExpired message to InvitationConfirmationSaga
If the Invitation is processed it sends InvitationAlreadyProcessed message to InvitationConfirmationSaga
If the Invitation is not expired it, it's accepted and it sends InvitationAccepted message to InvitationConfirmationSaga
Then:
InvitationConfirmationSaga will receive these messages and raise events accordingly.
This way you keep the domain logic in the Domain, in this case the Invitation Aggregate.
You have a command ConfirmInvitation containing InvitationId. You send it to your Invitation domain from InvaitationAppService. Your Invitation domain should look like this
...
public void ConfirmInvitation()
{
if (this.Status == InvitationStatus.Confirmed)
throw new InvalidInvitationException("Requested invitation has already been confirmed");
//check more business logic here
this.Status = InvitationStatus.Confirmed;
Publish(new InviationConfirmedEvent(...));
}
...
Your InvitationAppService should have something like below:
...
public void ConfirmInvitation(Guid invitationId)
{
// rehydrate your domain from eventstore
var invitation = repo.GetById<Invitation>(invitationId);
if (invitation == null)
throw new InvalidInvitationException("Invalid Invitation requested");
invitation.ConfirmInvitation(new ConfirmInvitation(...));
}
You don't need to introduce a new event InvitationConfirmationRequested. DDD is an approach in which your domain/business validation should reside inside domains. Don't try to fit other patterns or technologies in your domain. Validating your domain inside saga(which is used to orchestrate distribute transactions across the services) might create complexities and chaos
Let's say we have two services A and B. B has a relation to A so it needs to know about the existing entities of A.
Service A publishes events every time an entity is created or updated. Service B subscribes to the events published by A and therefore knows about the entities existing in service A.
Problem: The client (UI or other micro services) creates a new entity 'a' and right away creates a new entity 'b' with a reference to 'a'. This is done without much delay so what happens if service B did not receive/handle the event from B before getting the create request with a reference to 'b'?
How should this be handled?
Service B must fail and the client should handle this and possibly do retry.
Service B accepts the entity and over time expect the relation to be fulfilled when the expected event is received. Service B provides a state for the entity that ensures it cannot be trusted before the relation have been verified.
It is poor design that the client can/has to do these two calls in the same transaction. The design should be different. How?
Other ways?
I know that event platforms like Kafka ensures very fast event transmittance but there will always be a delay and since this is an asynchronous process there will be kind of a race condition.
What you're asking about falls under the general category of bridging the gap between Eventual Consistency and good User Experience which is a well-documented challenge with a distributed architecture. You have to choose between availability and consistency; typically you cannot have both.
Your example raises the question as to whether service boundaries are appropriate. It's a common mistake to define microservice boundaries around Entities, but that's an anti-pattern. Microservice boundaries should be consistent with domain boundaries related to the business use case, not how entities are modeled within those boundaries. Here's a good article that discusses decomposition, but the TL;DR; is:
Microservices should be verbs, not nouns.
So, for example, you could have a CreateNewBusinessThing microservice that handles this specific case. But, for now, we'll assume you have good and valid reasons to have the services divided as they are.
The "right" solution in your case depends on the needs of the consuming service/application. If the consumer is an application or User Interface of some sort, responsiveness is required and that becomes your overriding need. If the consumer is another microservice, it may well be that it cares more about getting good "finalized" data rather than being responsive.
In either of those cases, one good option is a facade (aka gateway) service that lives between your client and the highly-dependent services. This service can receive and persist the request, then respond however you'd like. It can give the consumer a 200 - OK response with an endpoint to call back to check status of the request - very responsive. Or, it could receive a URL to use as a webhook when the response is completed from both back-end services, so it could notify the client directly. Or it could publish events of its own (it likely should). Essentially, you can tailor the facade service to provide to as many consumers as needed in the way each consumer wants to talk.
There are other options too. You can look into Task-Based UI, the Saga pattern, or even just Faking It.
I think you would like to leverage the flexibility of a broker and the confirmation of a synchronous call . Both of them can be achieved by this
https://www.rabbitmq.com/tutorials/tutorial-six-dotnet.html
I'm new to EDA and I've read a lot about benefits and would probably be interested to apply it during my next project but still haven't understood something.
When raising an event, which pattern is the most suited:
Name the event "CustomerUpdate" and include all information (updated or not) about the customer
Name the event "CustomerUpdate" and include only information that have really been updated
Name the event "CustomerUpdate" and include minimum information (Identifier) and/or a URI to let the consumer retrieves information about this Customer.
I ask the question because some of our events could be heavy and frequent.
Thx for your answers and time.
Name the event "CustomerUpdate"
First let's start with your event name. The purpose of an event is to describe something which has already happenned. This is different from a command, which is to issue an instruction for something yet to happen.
Your event name "CustomerUpdate" sounds ambiguous in this respect, as it could be describing something in the past or something in the future.
CustomerUpdated would be better, but even then, Updated is another ambiguous term, and is nonspecific in a business context. Why was the customer updated in this instance? Was it because they changed their payment details? Moved home? Were they upgraded from silver to gold status? Events can be made as specific as needed.
This may seem at first to be overthinking, but event naming becomes especially relevant as you remove data and context from the event payload, moving more toward skinny events (the "option 3" from your question, which I discuss below).
That is not to suggest that it is always appropriate to define events at this level of granularity, only that it is an avenue which is open to you early on in the project which may pay dividends later on (or may swamp you with thousands of event types).
Going back to your actual question, let's take each of your options in turn:
Name the event "CustomerUpdate" and include all information (updated
or not) about the customer
Let's call this "pattern" the Fat message.
Fat messages (also called snapshots) represent the state of the described entity at a given point in time with all the event context present in the payload. They are interesting because the message itself represents the contract between service and consumer. They can be used for communicating changes of state between business domains, where it may be preferred that all event context be present during message processing by the consumer.
Advantages:
Self consistent - can be consumed entirely without knowledge of other systems.
Simple to consume (upsert).
Disadvantages:
Brittle - the contract between service and consumer is coupled to the message itself.
Easy to overwrite current data with old data if messages arrive in the wrong order (hint: you can mitigate this by using the event sourcing pattern)
Large.
Name the event "CustomerUpdate" and include only information that have
really been updated
Let's call this pattern the Delta message.
Deltas are similar to fat messages in many ways, though they are generally more complex to generate and consume. A good example here is the JSONPatch standard.
Because they are only a partial description of the event entity, deltas also come with a built-in assumption that the consumer knows something about the event being described. For this reason they may be less suitable for sending outside a business domain, where the event entity may not be well known.
Deltas really shine when synchronising data between systems sharing the same entity model, ideally persisted in non-relational storage (eg, no-sql). In this instance an entity can be retrieved, the delta applied, and then persisted again with minimal effort.
Advantages:
Smaller than Fat messages
Excels in use cases involving shared entity models
Portable (if based on a standard such as jsonpatch, or to a lesser extent, diffgram)
Disadvantages:
Similar to the Fat message, assumes complete knowledge of the data entity.
Easy to overwrite current data with old data.
Complex to generate and consume (except for specific use cases)
Name the event "CustomerUpdate" and include minimum information
(Identifier) and/or a URI to let the consumer retrieves information
about this Customer.
Let's call this the Skinny message.
Skinny messages are different from the other message patterns you have defined, in that the service/consumer contract is no longer explicit in the message, but implied in that at some later time the consumer will retrieve the event context. This decouples the contract and the message exchange, which is a good thing.
This may or may not lend itself well to cross-business domain communication of events, depending on how your enterprise is set up. Because the event payload is so small (usually an ID with some headers), there is no context other than the name of the event on which the consumer can base processing decisions; therefore it becomes more important to make sure the event is named appropriately, especially if there are multiple ways a consumer could handle a CustomerUpdated message.
Additionally it may not be good practice to include an actual resource address in the event data - because events are things which have already happened, event messages are generally immutable and therefore any information in the event should be true forever in case the events need to be replayed. In this instance a resource address could easily become obsolete and events would not be re-playable.
Advantages:
Decouples service contract from message.
Information about the event contained in the event name.
Naturally idempotent (with time-stamp).
Generally tiny.
Simple to generate and consume.
Disadvantages:
Consumer must make additional call to retrieve event context - requires explicit knowledge of other systems.
Event context may have become obsolete at the point where the consumer retrieves it, making this approach generally unsuitable for some real-time applications.
When raising an event, which pattern is the most suited?
I think the answer to this is: it depends on lots of things, and there is probably no one right answer.
Update from comments: Also worth reading, a very old, classic, blog post on messaging: https://learn.microsoft.com/en-gb/archive/blogs/nickmalik/killing-the-command-message-should-we-use-events-or-documents (also here: http://vanguardea.com/killing-the-command-message-should-we-use-events-or-documents/)
Martin Fowler gave a great talk about "The Many Meanings of Event-Driven Architecture" (the content is based on this paper) in which he mentioned the Event-Carried State Transfer pattern.
It seems to be close to your second option "Delta message" with the difference that it doesn't try to describe an entity, but instead describe a named business fact that happened and carry over all the necessary data to understand this fact.
I don't think it matters how you have modeled your persistence layer when it comes to designing domain events. Likewise, I don't think it matters how your consumer has modeled its own persistence layer when designing domain events.
Thus, I don't think it's wise to put as an advantage the fact that you can apply the event as a patch directly on your data (from a consumer point of view), because it pushes the producer to design their events given the persistence model of a consumer.
In that case, I would tend to think that you're designing persistence patches, instead of domain events.
What do you think?
I am learning DDD and making use of the CQRS pattern. I don't understand how to validate business rules in a command handler without reading from the data store.
For example, Chris wants to give Ashley a gift.
The command might be GiveGiftCommand.
At what point would I verify Chris actually owns the gift he wants to give? And how would I do that without reading from the database?
There are different views and opinions about validation in command handlers.
Command can be rejected, we can say No to the command if it is not valid.
Typically you would have a validation that occurs on the UI and that might be duplicated inside the command handler (some people tend to put it in the domain too). Command handler then can run simple validation that can occur outside of the entity like is data in correct format, are there expected values, etc.
Business logic, on the other hand, should not be in a command handler. It should be in your domain.
So I think that the underlying question is...
Should I query the read side from Command Handlers?
I would say no. Do not use the read model in the command handlers or domain logic. But you can always query your read model from the client to fetch the data you need in for your command and to validate the command. You would query the read side on the client to check would if Chris actually owns the gift he wants to give. Of course, the validation involving a read model is likely to be eventually consistent, which is of course another reason a command could be rejected from the aggregate, inside the command handler.
Some people disagree saying that if you require your commands to contain the data the handler needs to validate your command, than you can never change the validation logic within your handler/domain without affecting the client as well. This exposes too much of the domain knowledge to the client and go against the fact that the client only wants to express an intent. So they would tend to provide an GiftService interface (which are part of the ubiquitous language) to your command handler and then implement the interface as needed - which might include querying the read side.
I think that the client should always assume that the commands it issues will succeed. Calling the read side to validate the command shouldn't be needed. Getting two contradictory commands is very unlikely (users creating accounts with the same email address). Then you should have a mean to issue a corrective action, something like for example a Saga/Process Manager. So instead making a corrective action would be less problematic that if the command could have been validated and not dispatched in the first place.
It depends if the operation is async or not i.e does the user expect some immediate response. Gift ownership is basically a security feature and that can be done as a 'prep' operation before invoking the actual service or sending the GiveGiftCommand.
The only command validation you can do is to make sure it contains the data in the required format (UI validation) and that the user has the permissions to do that action. But after the command is sent it's up to the Domain to decide if other business constraints are respected.
If the user expects some immediate feedback, you can actually 'wait' until the command is completed and for that you can use an approach where a command handler can provide a result to the sender using a mediator . But this implies that at least some commands are executed in an immediate manner and that might not be the case in your app. However, this is the simplest approach if you want to just return a message error as opposed to implementing compensations and other stuff. Some use cases are simple.
About command handlers and business logic, I disagree with Tomasz Jaskuλa . A command handler is a function, a technical detail. You can put business logic in a command handler or a static function, it doesn't matter. Messages and their handlers are infrastructural components that can be used to implement functionality. For example, in an app you can have Domain Events, Application Events etc . They're all events i.e notification that something changed and you can have event handlers that reside in Domain or in other places.
There's no rule preventing you to 'read' from the db, however at least the read model theoretically is stale. However this might not be such a problem in 99% of cases. For the rest 1% you need very specific solutions.
I just asked the exact same question from a knowledgeable friend of mine and his answer was that it is OK to do this validation inside the command handler (in my case inside an akka persistent actor that interprets commands and writes events to the journal).
However if that is not possible for performance reasons (because processing validation inside a persistent actor would block the actor and that would be a scalability bottleneck for the whole app) then optimistic locking (OCC) can be used.
In other words the validation can be performed in an other actor (lets call it the validator actor - which is not in the persistent actor) , this will not block the persistent actor but it might happen that the data that is used for the validation has changed in the persistent actor while the validation was running in the validator actor) .
If the validator actor returns with an OK and all the data that has been used for the validation is still the same in the persistent actor (has the same version - version as in OCC) as it has been at the point when the command arrived to the command handler (persistent actor) then the command is accepted by the persistent actor , otherwise the validation needs to be resubmitted for re-evaluation to the validator actor.