For microservices, the common design pattern used is API-Gateway. I am a bit confused about its implementation and implications. My questions/concerns are as follows:
Why other patterns for microservices are not generally discussed? If they are, then did I miss them out?
If we deploy a gateway server, isn't it a bottleneck?
Isn't the gateway server vulnerable to crashes/failures due to excessive requests at a single point? I believe that the load would be enormous at this point (and keeping in mind that Netflix is doing something like this). Correct me if I am wrong in understanding.
Stream/download/upload data (like files, videos, images) will also be passing through the gateway server with other middleware services?
Why can't we use the proxy pattern instead of Gateway?
From my understanding, in an ideal environment, a gateway server would be entertaining the requests from clients and responding back after the Microservices has performed the due task.
Additionally, I was looking at Spring Cloud Gateway. It seems to be something that I am looking for in a gateway server but the routing functionality of it confuses me if it's just a routing (redirect) service and the microservice would be directly responsible for the response to the client.
The gateway pattern is used to provide a single interface to a bunch of different microservices. If you have multiple microservices providing data for your API, you don't want to expose all of these to your clients. Much better for them to have just a single point of entry, without having to think about which service to poll for which data. It's also nice to be able to centralise common processing such as authentication. Like any design pattern, it can be applied very nicely to some solutions and doesn't work well for others.
If throughput becomes an issue, the gateway is very scalable. You can just add more gateways and load balance them
There are some subtle differences between proxy pattern and API gateway pattern. I recommend this article for a pretty straightforward explanation
https://blog.akana.com/api-proxy-or-gateway/
In the area of microservices the API-Gateway is a proven Pattern. It has several advantages e.g:
It encapsulate several edge functionalities (like authentication, authorization, routing, monitoring, ...)
It hides all your microservices and controls the access to them (I don't think, you want that your clients should be able to access your microservices directly).
It may encapsulate the communication protocols requested by your microservices (sometimes the service may have a mixture of protocols internally which even are only allowed within a firewall).
An API-Gateway may also provide "API composition" (orchestrating the calls to several services an merge their results to one). It s not recommended, to implement a such composition in a microservice.
and so on
Implementing all these feature in a proxy is not trivial. There is a couple of API-Gateways which provide all these functionalities and more like the Netflix-Zuul, Spring-Gateway or the Akana Gateway.
Furthermore, in order to avoid your API-Gateway from being a bottleneck you may :
Scale your API-Gateway and load balance it (as mentioned above by Arran_Duff)
Your API-Gateway should not provide a single one-size-fits-all API for all your clients. Doing so you will, in the case of huge request amount (or large files to down/up load) for sure encounter the problems you mentioned in questions 3 and 4. Therefore in order to mitigate a such situation your Gateway e.g may provide each client with a client specific API (a API-Gateway instance serves only a certain client type or business area..). This is exactly what Netflix has done to resolve this problem (see https://medium.com/netflix-techblog/embracing-the-differences-inside-the-netflix-api-redesign-15fd8b3dc49d)
1.Why other patterns for microservices are not generally discussed? If they are, then did I miss them out?
There are many microservice pattern under different categories such as database , service etc .This is a very good article https://microservices.io/patterns/index.html
2.If we deploy a gateway server, isn't it a bottleneck?
Yes to some extent .Q3's answers image will answer this.
3.Isn't the gateway server vulnerable to crashes/failures due to excessive requests at a single point? I believe that the load would be enormous at this point (and keeping in mind that Netflix is doing something like this). Correct me if I am wrong in understanding.
4.Stream/download/upload data (like files, videos, images) will also be passing through the gateway server with other middleware services?
Why can't we use the proxy pattern instead of Gateway?
The use case for an API Proxy versus an API Gateway depends on what kinds of capabilities you require and where you are in the API Lifecycle. If you already have an existing API that doesn’t require the advanced capabilities that an API Gateway can offer than an API Proxy would be a recommended route.
You can save valuable engineering bandwidth because proxies are much easier to maintain and you won’t suffer any negligible performance loss. If you need specific capabilities that a proxy doesn’t offer you could also develop an in-house layer to accommodate your use case. If you are earlier in the API lifecycle or need the extra features that an API Gateway can provide, then investing in one would pay dividends.
Related
I am stuck in choosing One API gateway from the three API gateways mentioned below:
KrakenD (https://www.krakend.io/)
Kong (https://konghq.com/kong/)
Spring Cloud Gateway (https://cloud.spring.io/spring-cloud-gateway/reference/html/)
My requirements are:
Good performance and must have majority of the API gateway features.
Supports aggregating data from two Different Micro-services API's.
All the three of them, looks good from the feature list and the performance wise.
I am thinking of relaxing the second requirement, as I am not sure, whether that is a good practice or not.
API Gateway is a concept that is used in all kind of products, I really think the industry should start sub-categorizing these products as most of them are completely different from each other.
I'll try to summarize here the main highlights according to your requirements.
Both Kong and KrakenD offer the "majority" of API gateway functionalities. Although the word is fuzzy, at least all of them cover stuff like routing, rate limiting, authorization, and such.
Kong
Kong is basically an Nginx proxy that adds a lot of functionality on top of it using Lua.
When using Kong your endpoints have a 1:1 relationship with your backends. Meaning that you declare an endpoint in Kong that exposes data from one backend, and does the magic in the middle (authorization, limiting, etc). This magic is the essence of Kong and is based on Lua plugins (unfortunately, these are not written in C as Nginx is).
If you want to aggregate data from several backends into one single endpoint, Kong does not fit in your scenario.
Finally, Kong is stateful (it's impressive how they try to sell it the other way around, but this is out of the scope of this question). The configuration lives inside a database, and changes to the configuration are through an API that ends up modifying its internal Postgres or equivalent.
Performance is also inevitably linked to the existence of this database (and Lua), and going multi-region can be a real pain.
Kong functionality can be extended with Lua code.
In summary:
Proxy with cross cutting concerns
Nodes require coordination and synchronization
Mutable configuration
The database is the source of truth
More pieces, more complexity
Multi-region lag
Requires powerful hardware to run
Customizations in Lua
KrakenD
KrakenD is a service written from the ground up using Go, taking advantage of the language features for concurrency, speed, and small footprint. In terms of performance, this is the winning racehorse.
KrakenD's natural positioning is as a Gateway with aggregation. It's meant to connect lots of backend services to a single endpoint. It's mostly adopted by companies for feeding Mobile applications, Webapps and other clients. It implements the pattern Backend for Frontend, allowing you to define exactly and with a declarative configuration how is the API that you want to expose to the clients. You can choose which fields are taken from responses, aggregate them, validate them, transform them, etc.
KrakenD is stateless, you version your API the same way you do with the rest of the code, using git. And you deploy it in the same way you do with your application (e.g: a CI/CD pipeline that pushes a new container with the new configuration and is rolled out). As everything is on the config, there is no need to have a central database, neither nodes need communication with each other.
As per the customizations, with KrakenD you can create middlewares, plugins or just scripting in several languages: Go, Lua, Common Expression Language (CEL) -sort of JS- and Martian DSL.
In summary:
On the-fly API creation using upstream services, with cross-cutting concerns (api gateway).
Not a proxy, although it can be used as one.
No node coordination
No synchronization needed
Zero complexity (docker container with a configuration file)
No challenges for Multi-region
Declarative configuration
Immutable infrastructure
Runs on micro and small machines in production without issues.
Customizations in Go, Lua, CEL, and Martian DSL
Spring Cloud Gateway
(As well as Zuul) is used mostly by Java developers that want to stick in the JVM space. I am less familiar with this one, but it's design is also for proxying to existing services, adds also the cross-concerns of the API gateway.
I see it more as a framework that you use to deliver your API. With this product you need to code the transformations yourself in Java. The included gateway functionalitites are declarative as well.
--
I am hoping this sheds some light
My only and major blocker from using Kong for me is you can only extend Kong with LUA. Only small percentages of developers in the world familiar with LUA. That's why I choose KrakenD.
I'm working on a project that has multiple microservices behind a API Gateway and some of them expose WebSockets API.
The WebApp needs to be able to interact with those APIs.
.
Those WebSocket API can be built with frameworks that have their own protocols, using socket.io or not etc.
The main goal of this reflexion is to be able to scale and keep flexibility on my WebSockets APIs implementation.
I thought about two solutions :
The first one is to simply proxy requests on the gateway, the webapp will have to open a websocket for each microservice, this looks like a design flaw to me.
The other one is to create a "Notifier Service" that will be a WebSocket Server and that will keep outgoing connections with users and be able to bridge the incomming messages and outgoing ones based on a custom protocol. The drawback is that i need to implement a pub/sub system (or find a solution for). I didn't dig a lot into it but it looks like a lot of work and a homemade solution, i'm not a fan of it.
I didn't find articles that give feedback after exposing such an architecture and websockets in production, i was hoping to find some here.
I agree with you that a simple proxy solution seems to be inadequate as it exposes the internal interfaces to the clients. I found two articles which I think are addressing your problem:
The API Gateway Pattern
Pattern: API Gateway / Backends for Frontends
Both talk about the issues that you have already mentioned in your question: Protocol translation is an advantage of this architecture as it decouples the external API from the protocols internally used. On the other hand, increased complexity due to having another component to be maintained is mentioned as a drawback.
The second article also suggests some existing libraries (namley Netty, Spring Reactor, NodeJS) that can be used to implement an API gateway, so you might want to spend some time evaluating these for your project.
I have a micro-service project with multiple services in .NET Core. When it comes to placing the controllers, there are 2 approaches:
Place the controllers in respective Micro Services, with Startup.cs in each micro-service.
Place all controllers in a separate project and have them call the individual services.
I think the 1st approach will involve less coding effort but the 2nd one separates controllers from actual services using interfaces etc.
Is there a difference in terms of how they are created and managed in Fabric using both approaches.
This is very broad topic and can raise points for discussion because it all depends on preferences, experiences and tech stacks. I will add my two cents, but do not consider it as a rule, just my view for both approaches.
First approach (APIs for each service isolated from each other):
the services will expose their public APIs themselves and you will need to put a service discovery approach in place to enable clients to call each microservice, a simple one is using the reverse proxy to forward the calls using the service name.
Each service and it's APIs scales independently
This approach is better to deploy individual updates without taking down other microservices.
This approach tends to have more code repetition to handle authorization, authentication, and other common aspects, from there you will end up doing shared libraries using on all services.
This approach increase the points of failures, it is good because failures will affect less services, if one API is failing, other services won't be impacted (if the failure does not affect the machine like memory leak or high CPU usage).
The second approach (Single API to forward the calls to right services):
You have a single endpoint and the service discovery will happen in the API, all work will be handled by each services.
The API must scale for everyone even though one service consumes much more resources than others. just the service will scale independently.
This approach, to add or modify api endpoints, you will likely update the API and the service, taking down the API will affect other services.
This approach reduces the code duplication and you can centralize many common aspects like Authorization, request throttling and so on.
This approach has less points of failures, if one microservices goes down, and a good amount of calls depend on this service, the API will handle more connection and pending requests, this will affect other services and performance. If it goes down, every services will be unavailable. Compared to the first approach, the first approach will offloaded the resilience to the proxy or to the client.
In summary,
both approaches will have a similar effort, the difference is that the effort will be split into different areas, you should evaluate both and consider which one to maintain. Don't consider just code in the comparison, because code has very little impact on the overall solution when compared with other aspects like release, monitoring, logging, security, performance.
In our current project we have a public facing API. We have several individual microservice projects for each domain. Being individual allows us to scale according to the resources each microservice use. For example we have an imaging service that consumes a lot of resources, so scaling this is easier. You also have the chance to deploy them individually and if any service fails it doesn't break the whole application.
In front of all the microservices we have an API Gateway that handles all the authentication, throttles, versioning, health checks, metrics, logging etc. We have interfaces for each microservice, and keep the Request and Response models seperately for each context. There is no business logic on this layer, and you also have the chance to aggregate responses where several services need to be called.
If you would like to ask anything about this structure please feel free to ask.
As a mechanism to connect microservices together and make them work it is usually suggested to use APIs and Service Discovery. But they usually work as their own microservices, but these ones should apparently be "hard-coded" into others, as every microservice is supposed to register with them and query for other microservices' locations. Doesn't this break the idea of loose coupling, since a loss of a discovery service implies others being unable to communicate?
Pretty much yes. If one microservice "knows" about another microservice - it means that they are highly coupled. It doesn't matter where specifically this knowledge about other service is coming from: hardcoded, config file or maybe some service discovery, it still leads to high coupling.
What's important to understand is that many microservice evangelists are just preaching about how to build monolith apps on top of Web APIs. Some of them probably think that the more buzz words they use - the better ... not quite sure why this happens. It is probably easier to fake a language and become an "expert" by using buzzword salad instead of really building fault tolerant and horizontally scalable app.
But there is another way to look at service discovery: when it is used by service clients like SPA application or an API Gateway it may be very useful. Clients and gateways should know about service APIs, otherwise, the whole thing doesn't make sense. But they can use a registry to make it more flexible/dynamic.
So, to summarize:
if services are using discovery to get more information about each other - this is probably a bad thing and design flaw (pretty sure there are corner cases where this may be a valid scenario, please post a comment if you know some)
if discovery is used by other parts of the app, like SPA or API Gateway, this may be useful, but not necessarily it always is.
PS: to avoid high coupling, consider reading series of articles by Jeppe Cramon that illustrate the problem and possible solutions very nicely.
In a distributed system, you will always have some amount of coupling, what you want to do is to reduce all aspects of coupling to a minimum.
I'd argue that is does matter how you design your service location. if your code knows of the other service i.e. OrderService.Send(SubmitOrderMessage); (where 'OrderService' is an instance of the other service's proxy)
as opposed to transportAgent.Send(SubmitOrderMessage); (where 'transportAgent' is an instance of the transport's proxy i.e. the queuing service/agent and the actual address of the queue can be in your config), this reduces the coupling and your business logic code (Service) and delegates the routing to your infrastructure.
Make Sense?
Each microservice is expected to be functionally independent. For interacting with other microservices, it should rely on rest api calls only. Service Discovery plays a role to keep the services fairly loose coupled with each other. Also due to dynamic nature of the service urls, the hard dependencies are removed. Hope this helps
Reduced coupling or fairly loose coupling still has one thing in common; coupling. In my opinion, coupling to any degree will always create rigid communication patterns that are difficult to maintain and difficult to troubleshoot as a platform grows into a large distributed platform. Isn't the idea behind microservices to allow consumers to engage in "Permissionless innovation?" I would suggest that this is only possible by decomposing to microservices that have high cohesion and low coupling and then let the consumer decide how to route, orchestrate or aggregate.
I have an existing web service that supports ordering and it has multiple operations (approximately 20). This is a single webservice that support the ordering function. It interacts with multiple other services to provide ordering capability.
Since there is a lot of business functionality within this app and it is supported by a 10 member team , I believe it is a monolith (though I assume there is no hard and fast rule to define what a monolith is).
We are planning to get the application deployed in cloud foundry environment and we are planning to split the app into 2-3 microservices , primarily to enable them scale independently.
The first few apis which enable searching for a product typically have more number of hits whereas the api that support actual order submission receives less that 5% of the hits. So the product search api should have significantly larger number of instances as compared to order submission api.
Though I am not sure if we could split is based on sub-domains (which I have read should be the basis) , we are thinking of splitting them based on the call sequence as explained earlier.
I have also read that microservices should be choreographed and not orchestrated. However in order to ensure our existing consumers are not impacted , I believe we should expose a api layer which would orchestrate the calls to these microservices. Is providing an api gateway , the normal approach that is followed to ensure consumers do not end up calling multiple microservices and also provides a layer of abstraction?
This seems to be orchestration more than choreography - though I am not hung up on the theoretical aspects , I would like to understand the different solutions that are pursued for this problem statement in an enterprise world.
The Benefits of Microservices
Deploy & Scale Independently
Easier to 'Reason About'
Separation of Concerns
Single Responsibility
(Micro)Service-Oriented Architecture
I would suggest splitting your services based on domain. This is a logical and efficient approach which makes it an easy starting point. Your monolithic package structure may already be organized in this manner, which simplifies the refactoring even more.
API Gateway
The typical Spring Cloud approach for this would be to use a Zuul Proxy on the edge of your network which receives the requests from your clients (web, mobile, etc.) and routes them to the microservices located behind your firewall. The client only interfaces with a single domain, and it handles CORS out of the box.
Resources:
API Gateway Pattern
Routing and Filtering