Should Microsercices be reusable?
With reusable I do NOT mean sharing Domain specific Models.
I mean should a microservice created for one application be reuseable in another application?
Is it sufficient if they are reusable within an application?
What is the best way to decouple microservices.
From my point of view as soon a microservices calls another microservice it is tightly coupled, means it can not easily (without modifications) be extracted and put into another microservice application that does not have the same service it refers to/from.
To decouple them, in my opinion, there are following ways:
microservice A needs to talk to the other microservice B with a
standard contract eg. a specfic protocol.
another Microservice C acts as a gateway and asks microservice B for the data and passes it as input to microservice A.
A concrete example for nr. 2 would be:
Coupled:
client -> API GateWay -> UserProfileService -> Authorization Service
Decoupled:
Client -> API Gateway -> Authorization Service -> API Gateway -> UserProfileService
Am I right assuming that this all boils down to the goal of the microservice? There is no wrong and right?
Are there any other strategies i'm missing to decouple a microservice?
I think the responses you're likely to get will represent opinions more than answers, but I'll go ahead and give mine!
The literature for microservices has long said, "decouple, decouple, decouple", but frankly I don't find this to be reality. When someone has created a useful API that would empower the functions of your own (auth, payments, and obviously databases come to mind), is it wrong to suggest that those need to be run alongside yours? Most people don't go through complicated, logic-filled gateways in order to make payments via Stripe or send text messages via Twilio, so why should privately hosted APIs be any different?
It is great to design your own service to be a reusable, easily consumable/deployable component. That shouldn't mean it can't have dependencies, but rather that we should be mindful of the bloat those dependencies introduce. This mindfulness is something devs should practice whenever they introduce dependencies, regardless of whether they are app packages or dependent services/APIs.
**disclosure: I build and run a framework/platform, Architect.io, to help cloud-native teams collaborate and build upon each others services. I've seen first-hand how company's like Facebook use similar tactics to enable service re-use and consumption, and wanted to build a microservices dependency resolver for the general public.
It completely depends on what microservice you are building. For ex; say you are building an email notification service. That can be reused by different applications. Another example say you are building a recommendation system. It's very specific for a single application. It hardly makes sense to design it in such a way that it can reused in different applications.
Choose according to the context. There is no right way. It all depends on the application.
Related
As a part of our DDD design, we are working on a bounded context and have identified two microservices A and B.
Service A needs to make calls to Service B via REST API. Service B already provides open API spec on how to get any data. We use openapi generator to auto generate client side DTOs.
There is exactly a one-to-one mapping between B DTO to A's domain object.
As far as I understand, we should use anti corruption layer (using hexagonal architecture) if our service is communicating with an untrusted third party service. But what should we do, if the communication is between internal services, like in the case above ?
Should microservice B API DTOs directly used as domain objects in microservice A ? This would mean that we will not create separate domain class/object in service A. We treat B's DTO as domain object in service A.
Should we create an adapter layer to convert B's DTO to domain objects in microservice A ?
Should I create domain objects in service A which would also be manual output DTO for service B ?. This is opposite of point 1, where DTOs are treated as domain objects, where as in this case, we are treating domain objects as DTOs
In my opinion, you are mixing two separate concepts:
The "anticorruption layer", which is a strategic DDD pattern
The layers in a layered or onion architecture, which are tactical patterns to separate concerns within an application
The goal of the anticorruption layer is to translate the ubiquitous language from another Bounded Context into your ubiquitous language so that your code doesn't get polluted with concepts not belonging to your Bounded Context.
The goal of the layers in a layered or onion architecture and specifically the contracts between them (the DTOs you are talking about) is to avoid changes in one part of the code (for example, adding or removing a property in a domain object in the Core) to cause issues somewhere else (like accidentally modifying the public API contract).
If I understand it correctly, your two microservices belong to the same Bounded Context. If that is the case, you shouldn't need any anticorruption layer because both microservices share the same ubiquitous language.
Now regarding the options that you propose, I'm not sure I fully understand options 2 and 3, but what I'll say is that if you are doing real microservices, you can't use option 1, as these microservices wouldn't be autonomous, and independently deployable as a change in microservice B's API would require a change and coordinated deployment of microservice A.
So, design your microservices so that you have control over how their parts evolve (be able to make non-breaking changes in their APIs, change storage strategies without having to change the Core, change the core without having to change the API, etc). Of course, if a microservice is very simple, don't over-engineer it.
The answer here is simple - There is no magic to make breaking changes non-breaking. So treat internal dependencies like you would external. That requires a high discipline on versioning of the APIs and moving versions ahead. The Service A needs to publish new API version specifications ahead of time and Service B needs to be given time to implement those. Depending on the size of your organization it can be useful to support multiple API versions concurrently (for a while that is). You may also want Team A to be monitoring usage of their API versions (e.g. how many customers are still using an old version of the API etc.).
I would not put a high focus on how exactly your API specification is documented or which technology is used to generate client bindings (if at all). This should be up to the service A owners and service B should be able to adopt whatever being offered. In general I would put the focus on simplicity and avoid too many additional layers if they don't provide clear benefit. And team B should be free to design their internals as they see best fit (even with respect to data that comes from Service A). The whole point of microservices architecture is to be able to move forward independently.
This is probably a bit opinion-based question, but I will try to be technical to still be relevant.
Consider having several microservices: a, b, c.
To make this available on frontend, these could be made available as:
https://host/services/a
https://host/services/b
https://host/services/c
However, the fact that the endpoints are split between differents services are kind of irrelevant for frontend and basically if we can guarantee the endpoints don't clash, it would be great to have these available directly:
a/endpoint1 -> https://host/services/endpoint1
a/endpoint2 -> https://host/services/endpoint2
b/endpoint3 -> https://host/services/endpoint3
c/endpoint4 -> https://host/services/endpoint4
To implement such mapping, one needs to list all endpoint or at least write some matching pattern within the proxy service. This is very nice for the Frontend team to work with, however it is unfortunately very easy to brake.
What are the best practices for mapping the urls of microservices? Only thing which comes to my mind are some exports of OpenApi, which could be handled by FE to get the right path. However, every service generates its own OpenApi json, so we are basically back to the original problem.
are you sure the Frontend team needs ALL the exposed endpoints? Usually, frontends talk with an API Gateway, or, as cool kids call them these days, "Backend for Frontends".
In a nutshell, it's a special service that takes care of exposing only the functionalities/endpoints needed by the frontend. It will forward calls to the relevant services or, if necessary, call multiple services and aggregate the results.
In most cases these API Gateway don't have a db, as they're retrieving all the data from other services. They might however make use of a caching layer to speedup things.
You can even have multiple API Gateway, one per Frontend (eg. desktop, mobile).
I just finished reading Uncle Bob's "Clean Architecture" and now wondering how to apply it in the context of microservices!
On one hand, I think that microservices fall in the "Framework-Drivers" layer since it's an implementation on top of use-cases (they are ways to serve use-cases.) This way, we focus on the core of the app (Entities and Use-cases) and stay flexible in the implementation of the outer layers (including microservices). But since each microservice can be maintained by a different developer/team of developers, they will have a bad time when use-cases change (harder to predict who will be impacted).
On the other hand, we can split our app into multiple microservices, decoupled from each other, and apply Clean Architecture inside each microservice. The pro of this approach is that we can focus on each microservice doing one thing, and doing it well. But the problem is that we started designing using technical separations (microservices) which violates the main Clean Architecture principle of focusing on the business. Also, it will be hard to not duplicate code if two microservices uses the same entity or use-case!
I think the first scenario is the best, but I would like to have feedback from fellow developers on the long-term benefits of both scenarios, and potential troubles.
My two cents:
From Uncle Bob's words, "Micro-services are deployment option, not an architecture". Each micro-service should be deployable, maintainable by different teams (which can be in different geographical locations). Each team can choose their own architecture, programming language, tools, frameworks etc... And forcing each team to use single/same programming language or tool or architecture does not sound good. So each micro-service team must be able to pick their architecture.
How can each team code/maintain/deploy their own micro-service without conflicting with other teams code? This question brings us to how to separate micro-services. IMHO it should be separated on feature based (same principle applies to modularization of mobile application projects where independent teams should be able to work on separate modules/micro-services).
After separating micro-services, the communication between them is implementation detail. It can be done through web-socket/REST API etc... Inside each micro-service, if team decides to follow Clean Architecture, they can have multiple layers based on Clean Arch Principles (Domain/Core - Interface Adapters - Presentation/API & Data & Infrastructure). There can/will be duplicate codes on micro-services, which are OK for micro-services.
As #lww-pai-long said in his answer here splitting based on the Domain responsibilities and DDD is in most cases the best solution.
Still if you worked with a system using micro-services you soon realize that there are other things involved here as well.
DDD Bounded Context as base for micro-services
In most cases splitting your application to micro-services based on Bounded Context is the safe way to go here. From experience I would even say that in some parts of Domain you could go even further and have multiple micro-services per Bounded Context. Example would be if you have quite big part of Domain which represents one Bounded Context. Other example would be if you use CQRS for a particular Domain. Then you can end up having a Write/Domain and Views/Read micro-service.
You can read in this answer how you can split your Domain to micro-services.
It would be advisable as you said to "apply Clean Architecture inside each microservice".
Also, it will be hard to not duplicate code if two microservices uses
the same entity or use-case!
This is something that you have to deal with when working with micro-services in most cases. Duplicating code and/or data across multiple micro-service is common drawback of working with micro-services. You have to take this into account as you on the other hand get isolation and independence of the micro-service and its database. This problem can be partly solved by using shared libraries as some sort of packages. Be careful this is not the best approach for all cases. Here you can read about using common code and libraries across micro-services. Unfortunately not all advice's and principles from Uncle Bob's "Clean Architecture" can be applied when using micro-services.
Non Domain or technical operation micro-services
Usually if your solution is using micro-services you will more or less have micro-services which are not Domain specific but rather some kind of technical task's or non business operations directly. Example could be something like:
micro-service for report generation
micro-service for email generation and forwarding
micro-service for authorization/permission management
micro-service for secret management
micro-service for notification management
These are not services which you will get by splitting your solution based on DDD principles but you still need them as general solution as they could be consumed by multiple other services.
Conclusion
When working with micro-services you will most of the time have a mixture of Domain specific and Domain agnostic micro-services. I think the Clean Architecture could be looked from a little different prospective when working with micro-services.
On one hand, I think that microservices fall in the
"Framework-Drivers" layer since it's an implementation on top of
use-cases (they are ways to serve use-cases.)
It kind of does but it also falls into the other layers like Entities and Use Cases. I think it goes in the direction that if you work on Domain specific services this Diagram becomes the Architecture of each micro-service but not a concept above all micro-services. In the applications where I worked with micro-services each micro-service(the ones which are based on the DDD Bounded Context) had most of this layers if not all of them. The Domain agnostic services are an exception to this as they are not based on Domain Entities but rather on some tasks or operations like 'Create an Email', 'Create a PDF report from html template' or similar'.
I think this question may be better on Sofware Engineering but I'll answer anyway.
My approach would be to use DDD and define each microservice as a Domain Services grouping Use Cases semantically, then link Domain Services with Bounded Context.
Sam newman talk about the importance of separating microservice by domain abraction and not technical one in Building Microservices
The point he makes basically is that defining scaling strategies for microservice based on subdomain will better match the "real live" constraints observed on the production system than using technically based microservice and try to defined a abstract strategy.
And if you look at how something like Kubernetes works it seems to push to that direction. A pod end up being a microservice with multiple containers defined as a complete stack matching a sub-domain if the overhaul application.
It then gets easier in an e-commerce application, for example, to scale the Payment service independently of the Cart service based on customer activity than to scale the web services independently of the job queues in an abstract way.
The way those Bounded Contexts will communicate, i.e request based or event based, depends on the the specific relation between them. To use the same example a Cart may generate an event that will trigger the Payment, while the same Cart may need to request the Inventory before validating the order.
And at the end of a day those Domain Services* and Bounded Contexts can be implemented the same when starting with a monolith, even the Bounded Contexts communication can be. The underlying communication protocol becomes an implementation detail that can easily(kinda) be switch when transitioning to a distributed a.k.a microservices architecture.
I am in the process of learning about microservices and there's one thing that I can't seem to figure out and I can't find any resources that give me a direct answer to this. The question is:
Do microservices involve only business logic and database interactions
or do they involve UI components like JS, CSS, HTML as well ?
There really isn't a notion of what tiers are included in a microservice itself, however generally speaking, when referring to microservices most of the time you're referring to some back end based app. That begin said - there are companies out there - two that I've talked to directly that are building "microapps" - basically the backend service, facade and a portlet like interface that can be plugged into a federated portal through a layer of orchestration.
For the Java folks - think fully encapsulated JSR-268 like components that are run as independent processes running within a portal itself.
A microservice can be a complete application including its own UI, business logic and data.
I try to work with go-kit (gokit.io) and to build real-work application with it.
I look through examples. These examples are great. But I do not understand how to do services to service communications / data transfers in go-kit framework.
I can see "real-world" shipping app, but I do not understand how it could be "real world" micro-services. I can see in sources that, for example, they build the booking service just passing foreign repositories into service
type service struct {
cargoRepository cargo.Repository
locationRepository location.Repository
routingService routing.Service
handlingEventRepository cargo.HandlingEventRepository
}
and later they get data from repositories (this repository belongs to foreign micro-service) just calling the method:
locationRepository.Find(...)
Could someone please explain me:
how to build micro-service to micro-service communications in go-kit framework? Just show me the way / pattern, please. I do not understand it at all.
I see it as they just share direct access to data. But in real world micro-services, I expected that micro-services will communicate to each other to get needed data. And I do not understand how to do it in go-kit framework.
I'm the author of the shipping example. Sorry for not seeing your question earlier.
This particular example needs a bit of explanation. It is an example based on tactical patterns from Domain Driven Design, which means that we need to understand what we are talking about when we're referring to a service.
There are application services that deal with the use cases offered by the application, e.g. booking.Service. There are domain services that reside in the domain layer and provides your domain with concepts that aren't necessarily bound to a domain object. In the shipping example, routing.Service is a domain service whose implementation actually queries another application, in this case it talks to this routing service.
Application and domain services are merely ways of organizing our code. Putting it differently, these services communicate within a process, while microservices typically communicate over a network using some form of common transport, e.g. JSON, gRPC and so on.
Coming back to your question, what I believe you are looking for is the implementation of the routing.Service which you can find here.
The proxy service used here is explained under Client-side endpoints on this page, and is used to make requests from your application to another.
If you want more detail, I wrote a blog post on the subject a while ago.