Our Events microservice is a dumb SNS topic-based consumer who takes whatever is fed. If microservice A creates an event 'USER D added to Group GR', We may need to get some reference data for this event from microservice B before sending it to the Events microservice.
What's the best way to enrich this event with reference data when I would like to keep my microservices decoupled?
Here are the options I have so far:
Create a partial event from microservice A which would be consumed by microservice B and then be consumed by Events microservice
Make the front-end send the reference data to the backend - even when it is not needed for anything outside of creating events (doesn't sound neat to me)
Do I have any other options? Is option 1 the way to go?
Related
I'm currently studying Saga pattern. Most examples seem to focus on Orchestration Sagas where we have one central saga execution coordinator service that dispatches and receives messages/events. Unfortunately information on how to implement Choreography Sagas seem to be lacking a bit.
In domain driven design, we have multiple bounded contexts, ideally, where each bounded context is a self contained microservice. If microservice A wants to communicate with another microservice B we use Integration Events. Integration Events are published and subscribed to using some asynchronous communication - RabbitMQ, Azure Service Bus.
Assuming we want to start some Saga, for example, where we have to run transactions on Order Service and Customer Service - how exactly do services communicate with each other? Is it just regular Integration Events or something entirely different?
The way I see it and given picture below (source), Saga would be executed this way:
A new order is created. Status is set to "Pending" and OrderSubmittedDomainEvent domain event is emitted.
Domain event handler receives OrderSubmittedDomainEvent domain event, it then creates and dispatches ReserveCreditIntegrationEvent integration event.
Customer Service receives ReserveCreditIntegrationEvent integration event.
It attempts to reserve customer credit.
If credit is successfully reserved, CustomerCreditReservedDomainEvent domain event is emitted.
Domain event handler received CustomerCreditReservedDomainEvent domain event, it creates and dispatches CreditReservedIntegrationEvent integration event.
Order Service receives CreditReservedIntegrationEvent integration event and sets Order Status to "Confirmed".
saga is completed.
Is this the right approach?
I think using Choreography rather then Orchestration for distributed transactions makes sense if you chose it for the right reasons. For instance, if you need to spare the usually higher effort of implementing a central choreography as you don't need to know what state a transaction is in until it has finished. Or because you know that the order of the transaction workflow is stable and is unlikely to change which would also be on the plus side of choreography. But would be a drawback for Choreography if the order changes frequently because you would need to adapt all microservices in that case...
So you need to know the advantages and drawbacks of the two approaches.
If you chose Choreography for the right reasons I would say that I am missing the compensation logic in your considerations. What if the credit was reserved but then the order fails in the order service? Compensation events need to be considered as well in such cases...
Other than that there is the usual suspects:
such as making sure that each service will reliably send the next event after it processed the received event. For this you could look into the Transactional Outbox pattern.
or making sure that you have deduplication of events implemented in each of the services as for reliable sending of events accross distributed transactions you cannot be a hundred percent sure that an event will only be sent once.
And if you are even interested in an alternative to the Saga pattern you can look into the Routing Slip pattern. It is well suited for distributed transaction workflows that will differ depending on the current use case by avoiding that each service needs to know each route. The sequence of the workflow is attached to the initial message of the transaction and all subsequent messages. Then each service receiving a message with the routing slip performs its tasks and passes the next message including the routing slip to the next station (service) on the list.
Note: I am not sure what exactly you mean by ...IntegrationEvent. I would not differentiate between domain and integration events, all events are relevant from the business perspective in your example otherwise they would not be relevant to other Microservices.
I've been aware of event sourcing, CQRS, DDD and micro services for a little while and I'm now at that point where I want to try and start implementing stuff and giving something a go.
I've been looking into the technical side of CQRS and I understand the DDD concepts in there. How both the write side handles commands from the UI and publishes events from it, and how the read side handles events and creates projections on them.
The difficulty I'm having is the communication & a handling events from service-to-service (both from a write to read service and between micro services).
So I want to focus on eventstore (this one: https://eventstore.com/ to be less ambiguous). This is what I want to use as I understand it is a perfect for event sourcing and the simple nature of storing the events means I can use this for a message bus as well.
So my issue falls into two questions:
Between the write and the read, in order for the read side to receive/fetch the events created from the write side, am i right in thinking something like a catch up subscription can be used to subscribe to a stream to receive any events written to it or do i use something like polling to fetch events from a given point?
Between micro services, I am having an even harder time... So when looking at CQRS tutorials/talks etc... they always seem to talk with an example of an isolated service which receives commands from the UI/API. This is fine. I understand the write side will have an API attached to it so the user can interact with it to perform commands. E.g. create a customer. However... say if I have two micro services, e.g. a order micro service and an shipping micro service, how does the shipping micro service get the events published from the order micro service. Specifically, how does those customer events, translate to commands for the shipping service.
So let's take a simple example of: - Command created from the order's API to place an order. - A OrderPlacedEvent is published to the event store. How does the shipping service listen and react to this is it need to then DispatchOrder and create ain turn an OrderDispatchedEvent.
Does the write side of the shipping microservice then need to poll or also have a catch up subscription to the order stream? If so how does an event get translated to an command using DDD approach?
something like a catch up subscription can be used to subscribe to a stream to receive any events written to it
Yes, using catch-up subscriptions is the right way of doing it. You need to keep the stream position of your subscription persisted somewhere as well.
Here you can find some sample code that works. I am not posting the whole snippet since it is too long.
The projection service startup flow is:
Load the checkpoint (first time ever it would be the stream start)
Subscribe to the stream from that checkpoint
The runtime flow will then be:
The subscription will then call the function you provide when it receives an event. There's some plumbing there to do, like if you subscribe to $all, you need to filter out system events (it will be easier in the next version of Event Store)
Project the event
Store the new checkpoint
If you make your projections idempotent, you can store the checkpoint from time to time and save some IO.
how does the shipping micro service get the events published from the order micro service
When you build a brand new system and you have a small team working on all the components, you can make a shortcut and subscribe to domain events from another service, as you'd do with projections. Within the integration context (between the boxes), ordering should not be important so you can use persistent subscriptions so you won't need to think about checkpoints. Event Store will do it for you.
Be aware that it introduces tight coupling on the domain event schema of the originating service. Your contexts will have the Partnership relationship or the downstream service will be a Conformist.
When you move forward with your system, you might decide to decouple those contexts properly. So, you introduce a stable event API for the service that publishes events for others to consume. The same subscription that you used for integration can now instead take care of translating domain (internal) events to integration (external) events. The consuming context would then use the stable API and the domain model of the upstream service will be free in iterating on their domain model, as soon as they keep the conversion up-to-date.
It won't be necessary to use Event Store for the downstream context, they could just as well use a message broker. Integration events usually don't need to be persisted due to their transient nature.
We are running a webinar series about Event Sourcing at Event Store, check our web site to get on-demand access to previous webinars and you might find interesting to join future ones.
The difficulty I'm having is the communication & a handling events from service-to-service (both from a write to read service and between micro services).
The difficulty is not your fault - the DDD literature is really weak when it comes to discussing the plumbing.
Greg Young discusses some of the issues of subscription in the latter part of his Polygot Data talk.
Eventide Project has documentation that does a decent job of explaining the principles behind how the plumbing fits things together.
Between micro services, I am having an even harder time...
The basic idea: your message store is fundamentally a database; when the host of your microservice wakes up, it queries the message store for messages after some checkpoint, and then feeds them to your domain logic (updating its own local copy of the checkpoint as needed).
So the host pulls a document with events in it from the store, and transforms that document into a stream of handle(Event) commands that ultimately get passed to your domain component.
Put another way, you build a host that polls the database for information, parses the response, and then passes the parsed data to the domain model, and writes its own checkpoints.
Say I design a microservice architecture for a system of payment and a user one click the pay button in its UI screen send request for an API gateway and the gateway start chain of api calls to the few microservices. how to handle to case when one of the micorservices is down or not responding in the middle of the chain call?
I want the user to think that his payment has been successful and no to return him "try again later" can I save the state of the chain somewhere?
For such cases using asynchronous communication is more preferable rather than synchronous communication.
In this case;
When client sends a request to the system, API gateway takes the request and delegates to corresponding microservice. After that this microservice sends an event to other related microservice. Generally a message broker is used for this, messages are stored in broker and even the consumer (subcsriber) microservice is down, the message would not be lost.
You can also send events directly from api gateway to message broker. (see: https://microservices.io/patterns/apigateway.html)
For achiving atomicity and consistency (eventual consistency of course) SAGA pattern can be applied. You can check this for more information.
But if your requirement is calling a microservice for getting some data immediately and if this required microservice is down, then this solution would not work. You should avoid this kind of coupling between microservices by design. In my opinion, this is one of the most challenging part in microservices architecture. Domain Driven Design techniques can be used to determine bounded contexts.
I was wondering which of my two methods is more appropriate, or is there event another one?
(1) Direct
Direct communication between GATEWAY and μSERVICE A
UI sends HTTP request to GATEWAY
GATEWAY sends HTTP request to μSERVICE A
μSERVICE A returns either SUCCESS or ERROR
Event is stored in EVENT STORE and published to QUEUE
PROJECTION DATABASE is updated
Other μSERVICES might consume event
(2) Events
Event-based communication via a message queue
UI sends HTTP request to GATEWAY
GATEWAY published event to QUEUE
μSERVICE A consumes event
Event is stored in EVENT STORE and published to QUEUE
PROJECTION DATABASE is updated
Other μSERVICES might consume event
GATEWAY consumes event and sends response (SUCCESS or ERROR) to UI
I am really sorry if I misunderstood some concept, I am relatively new to this style of architecture.
Thanks in advance for every help! :)
Second approach is a preferred way and is async approach.
Direct
In first approach your microsvc B and C wait for the event to get published . The scalability of this system is directly dependent on microsvc A. what if microsvc A is down or falling behind writing events to queue? it's like single point of failure and bottleneck. you can't scale system easily.
Events
In microservices we keep system async so they can scale.
Gateway should be writing to the queue using pub/sub and all these microservices can use events at same time. system over all is more robust and can be scaled.
I am implementing an event-driven microservice architecture. Imagine the following scenario:
Chat service: Ability to see conversations and send messages. Conversations can have multiple participants.
Registration-login service: Deals with the registration of new users, and login.
User service: Getting/updating user profiles.
The registration-login service emits the following event with the newly created user object:
registration-new
login-success
logout-success
The chat service then listens on registration-new and stores some fields of user in its own redis cache. It also listens on login-success and stores the token, and on logout-success to delete the token.
The user service has the following event: user-updated. When this is fired, a listener in the chat service updates the data corresponding to the user id in redis. Like the chat service, the user service also listens on login-success and logout-success and does the same thing as what the chat service does.
My question is the following: is this a good way to do this? It feels a bit counterintuitive to be sharing data everywhere. I need some advice on this. Thank you!
Seems that there's no other way. Microservices architecture puts lots of stress in avoiding data sharing so as to not create dependencies. That means that each microservice will have some data duplicated. That also means that there must exist a way of getting data from other contexts. The preferred methods strive for eventual consistency, such as sending messages to event sourcing or AMQP systems and subscribing to them. You can also use synchronous methods (RPC calls, distributed transactions). That creates additional technologic dependencies, but if you cannot accept eventual consistency it could be the only way.