Let's say we have two microservices: MeetingProposal and Meeting.
Process of creating a meeting:
To create a meeting, users should propose their request, after their request, we create a proposal and show it to the admin.
If admin accepts the proposal then we should change the status of proposal to accepted and also create a meeting in the meeting microservice.
Question:
How we should connect these two processes to each other (after proposal accepted => create meeting automatically)?
It's called business processes management, long running flows, sagas, or similar(see this)
We should use some orchestration engines like Cadence or Netflix Conductor or any other orchestration engine.
Related
i'm trying to learn Microservices by implementing a sample project, tried to pick a semi-complex one to face real world challenges in Microservice architecture.
this is a simplified version of the project flow that I designed till here:
the flow
as you can see in the image I'm trying to get the list of appointments for a specific company, but since the required data is inside different Microservices, for getting the appointments I have to follow these steps:
the API gateway (bff) will get the request from frontend that contains a token
bff will authorize the jwt token by sending it to the users ms
appointments are separated by companies ids, so before getting the appointments, I need to get the user company
company id will be sent to appointments to get the appointments for the company
appointments will check to see if actor is authorized to get the list of appointments by its role (came earlier from the user ms)
appointments will return the list of appointments
inside appointments as you can see in the entities, I do have the id of both sides (sideA, sideB)
bff will get those users details by the ids from users
inside appointments data that is returned, there is a customer_id that is the id of a customer inside the company ms so bff send another request to the company to get the customer details
inside customer details, there is an id of a project that the customer is eager to visit so, bff will send a request to get the project from the projects ms
at the end, bff will join the data and return it to the frontend application.
this is also the simplified version of entities inside Microservices:
entities
right now, i'm using composition API approach to get the data I need, but as you can see the flow is complicated, and I can't think of a way to implement pagination, since I might need to sort, filter and then paginate the data, so I think in this situation, this might be a good idea to use CQRS pattern, but the problem is since I have many situations like this, I have to implement lots of CQRS services.
I'm wondering if:
is it possible to create a single CQRS service to have all the data for read purpose, instead of CQRS for each situation?
for some situations like this, the CQRS read database will becomes almost identical to a monolith architecture db. is. this okay?!
is there any alternative way to scape the complexity of creating and managing multiple CQRSs with partial repetitive data?
CQRS will help you get all the required details in one call. A CQRS service will have multiple tables that are part of different microservices.
An example will be like "OrderViewService" will require to listen and store events from "OrderService", "DeliveryService", "AccountingService". But it wont be listening to multiple other services which are not of concern for "OrderViewService".
So the point I am making here is the database won't become so similar to the monolithic database as it would have a lot more details.
For your project you might require a single CQRS that may deliver your requirements.
As it seems your requirements have dependency on all of the microservices and so a single CQRS service could help you solve the requirements.
Also if you are concerned about the space make sure what details would be required were only be saved to the view/read CQRS database. Thereby ensuring that we are not overwhelming the db with all the details from all the services.
As application grows there can be multiple CQRS services listening to different services or a combination of services and thereby serving their responsibilities.
Reference - https://microservices.io/patterns/data/cqrs.html
I think this explains and these are my thoughts about CQRS. Let me know if you have any questions post it as comments.
If my company sole purpose is processing a specific payload, but there is lot of orchestration for it. Should the orchestration, be in a separate domain. Lets say, payment is what the company does, but there is a workflow service, for that payment payload? If that is in a seperate domain, how should the workflow service domain talk to payment service domain?
It's better to use Event Driven Design which powered by message services like RabitMQ (or Kafka, MSMQ, or ..). It's not recommended to speech microservice each other directly via APIs. On the other hand to aggregate, some information from multiple services you can use 2 techniques, first using a BFF (back end for frontend layer), Second use a materialized view to gather information from many services.
Suppose that I have a microservice for messaging. The microservice knows how to send emails. The service have templates of emails that have some sort of "template engine" like pugjs, and can replace data in the body of the message.
I have an user service (used for authentication/authorization for example), and a bank account service (each user have one). Between the User microservice and Bank Account microservice it's clear that we don't have to duplicate any data than de user's uuid.
But I want now to send every day a message to each user with their account statement. The Messaging microservice needs data from the User microservice and the Bank Account microservice.
Okay... This is a small case of the real world. Now I know that to have the benefits of decoupled microservices I must follow some rules:
I can't share databases between microservices
I can't make synchronous requests between microservices
Okay... I can use a broker and each time a new user is created/updated the Messaging microservice can store that data. But really, this is a stupid thing:
I don't want to have inconsistency with this data, and keeping things sync is hard
The development time and complexity of the Messaging Microservice must now consider: listen and extract the relevant data from events, keep data consistent about other domains/services, managing the saved data on database
And think about a Messaging microservice. Really I must store all the data needed to parse the templates?
I read a lot about microservices and people creating rules for their simple examples. But I never really saw a good explanation and real-world examples like above.
So how to have the microservices above without data duplication?
In your domain example I would not let the message service know anything about bank or user details. Instead the message service should just receive instructions to send messages to recipients along with the given content. I would use a dedicated scheduled job (maybe implemented as an account notification service) that performs the work of acquiring the user and account data from the corresponding services, compiles the information for the message service and instructs it to actually send the messages. This introduces another "higher level, business purpose entity/service" but allows you to keep a clear separation of concerns.
In general it will happen frequently that your "basic" domain services are used by another service that represents a specific business purpose and requires their data. Dependency in itself is not a bad thing as long as concerns are seperated clearly and interfaces versioned, changes communicated etc.
Don't forget the whole idea of microservices is for allowing teams to have dedicated responsibilities with clear interfacing. It is about organization as much as it is about architecture.
We are evaluating a move to microservices. Each microservice would be its own project developed in isolation. During planning, we have determined that some of the microservices will communicate with other via REST calls, pub/sub, messaging (ie. a order service needs product information from product service).
If a microservice depends on retrieving data from another microservice, how can it be run in isolation during development? For example, what happens when your order service requests product details, but there is nothing to answer that request?
What you probably need is an stub rest service. Create a webapp that takes the expected output using a path that is not part of the public api. When you invoke the public api it sends what it just received
If a microservice depends on retrieving data from another microservice, how can it be run in isolation during development?
It should be always temporally isolated from other services during development and production as well.
For example, what happens when your order service requests product details, but there is nothing to answer that request?
This is a place where design flaw reveals itself: order service should not request product details from another service. Product details should be stored in the message (event) that order service will be subscribed to. Order service should be getting this message in an asynchronous manner using publish-subscribe pattern and saving it in its own database. Data about the product will be stored in 2 places as the result of that.
Please consider reading this series of articles about microservices for more details. But in a nutshell: your services should be temporally decoupled, so when your product service is down - order service can continue its operations without interruptions. This is the key thing to understand about good distributed systems design in general.
I've been triying to migrate part of our soa architecture (Mule ESB) to microservices (Spring Boot stack), but I'm facing a problem related to large flows where we have several orchestations.
Basically We a have a flow which has an user id as input and the response is compounded of user account, creditcards data, stocks and loans.
In this flow we have, at the beginning, a splitter (allows to send concurrent requests) and we send requests to account backend, 3 different credicard partners, stock partner and loans partner, at the end there is an agregattor (wait to all responses and merge all of them) and finally a node for prepare the response (apply business logic).
During the migration we have developed an account microservice, loan microservice, stock microservie and creditcard microservices (1 for each partner). The problem here is the orchestation, We can't use and event model approach because we need to get all responses in a certain point. We considered the choreography approach too, but we don't want to add logic related ot how orchestrate calls to our microservices because that would be a stepback to heavy coupled services (N*N connections).
We are thinking on make a new microservice that will be used as an orchestrator, but we don't know if this will be a good solution for microservices concepts.
Note: The front end can't make the orchestrations because it is a closed product and we can't touch it.
Thanks in advance.
We are thinking on make a new microservice that will be used as an orchestrator, but we don't know if this will be a good solution for microservices concepts.
From all you described that sounds like the most reasonable thing to do. You describe this service as having its own business purpose which indicates to me the potential need for a dedicated service. And the fact that it requires input from other (more basic) services would not be unusual for a complex domain service. Also you already listed the alternative of aggregating on the front end as something that doesn't work in your domain.
Something to consider is just making sure that the development teams for the basic services treat their APIs as customer facing (with the customer being your other services). That means they have to do clean work in terms of versioning/deprecating/etc.
And the downstream services need to treat the consumed APIs like they would a 3rd party API. For example Google went so far to allow internal service consumption be charged real money to incentivize optimizing the implementation of dependent services.