Background
I came from HAproxy background and recently there is a lot of hype around "Service Mesh" Architecture. Long story short, I began to learn "Envoy" and "Consul".
I develop an understanding that Envoy is a proxy software but using sidecar to abstract in-out network with "xDS" as Data Plane for the source of truth (Cluster, Route, Filter, etc). Consul is Service Discovery, Segmentation, etc. It also abstracts network and has Data Plane but Consul can't do complex Load Balancing, filter routing as Envoy does.
As Standalone, I can understand how they work and set up them since documentation relatively good. But it can quickly became a headache if I want to integrate Envoy and Consul, since documentation for both Envoy & Consul lacks specific for integration, use-cases, and best practice.
Schematic
Consider the following simple infrastructure design:
Legends:
CS: Consul Server
CA: Consul Agent
MA: Microservice A
MB: Microservice B
MC: Microservice C
EF: Envoy Front Facing / Edge Proxy
Questions
Following are my questions:
In the event of Multi-Instance Microservices, Consul (as
stand-alone) will randomize round-robin. With Envoy & Consul
Integration, How consul handle multi-instance microservice? which
software does the load balance?
Consul has Consul Server to store its data, however, it seems Envoy
does not have "Envoy Server" to store its data, so where are its
data being stored and distributed across multiple instances?
What about Envoy Cluster (Logical Group of Envoy Front Facing Proxy
and NOT Cluster of Services)? How the leader elected?
As I mentioned above, Separately, Consul and Envoy have their
sidecar/agent on each Machine. I read that when integrated, Consul
injects Envoy Sidecar, but no further information on how this works?
If Envoy uses Consul Server as "xDS", what if for example I want to
add an advanced filter so that for certain URL segment it must
forward to a certain instance?
If Envoy uses Consul Server as "xDS", what if I have another machine
and services (for some reason) not managed by Consul Server. How I
configure Envoy to add filter, cluster, etc for that machine and
services?
Thank You, I'm so excited I hope this thread can be helpful to others too.
Apologies for the late reply. I figure its better late than never. :-)
If you are only using Consul for service discovery, and directly querying it via DNS then Consul will randomize the IP addresses returned to the client. If you're querying the HTTP interface, it is up to the client to implement a load balancing strategy based on the hosts returned in the response. When you're using Consul service mesh, the load balancing function will be entirely handled by Envoy.
Consul is an xDS server. The data is stored within Consul and distributed to the agents within the cluster. See the Connect Architecture docs for more information.
Envoy clusters are similar to backend server pools. Proxies contain Clusters for each upstream service. Within each cluster, there are Endpoints which represent the individual proxy instances for the upstream services.
Consul can inject the Envoy sidecar when it is deployed on Kubernetes. It does this through a Kubernetes mutating admission webhook. See Connect Sidecar on Kubernetes: Installation and Configuration for more information.
Consul supports advanced layer 7 routing features. You can configure a service-router to route requests to different destinations by URL paths, headers, query params, etc.
Consul has an upcoming feature in version 1.8 called Terminating Gateways which may enable this use case. See the GitHub issue "Connect: Terminating (External Service) Gateways" (hashicorp/consul#6357) for more information.
Related
HI folks my question might be silly at first but I am new to microservices architecure and want to understand about service discovey in microservices architecure.
in my understanding if there are multiple services ( serviec A, Service B,Service c) and all of this services are needs to communicate with each other they must know the IP and Port of each other services.
and If ther are multiple instances are running of each services then it would be difficult for other services, as IP and Port might gets changed in cloud.
please correct me if I am wrong
So we can use service discovey like eureka for client side etcd to register the service name corresponding to IP and port, and other services now can make use of this service name to connect with each other.
if above statement is correct I am confused with Inter process communication (IPC), in my view the above which I understood is also a Inter process communication.
but in article enter link description here, I got confused between this two service discovey and IPC.
I have a question regarding how Load balancing works when you have a single node Docker Swarm combined with Spring Eureka using Spring Cloud gateway. I have successfully configured a non Eureka swarm and can see Swarm load balancing between replicas for a service:
Cloud Gateway route config
.route(r -> r.path("/api/**")
.uri("http://my-service:8081")
.id("my-service"))
If I then configure this to use Eureka I now have this:
.route(r -> r.path("/api/**")
.uri("lb://MY-SERVICE")
.id("my-service"))
I believe I'm right in assuming that the gateway will know the IP/Ports and load balance accordingly, however when a request hits an IP will swarm then also decide to load balance between the replicas?
I appreciate that Eureka may be overkill for a small single node swarm but feel it could be beneficial as the app expands and possibly becomes more distributed. Obviously I want to avoid a situation where load balancing happens twice.
I assume I could just use http instead of lb to stop the Gateway from load balancing.
The eureka discovery service will provide the api gateway all available addresses for a given service. Each service registered within eureka will have a unique (container) IP and port and if the api gateway is configured to load balance the requests then yes, each replica of the service will be used, swarm doesn't need to do anything for load balancing, since you are targeting a specific running service (task) and not a node, for example.
However, for a multi-node scenario, Docker swarm has a routing mesh functionality that basically removes the need of having a discovery service. Imagine you have multiple nodes and replicas distributed across them. With swarm's routing mesh you don't even have to know which nodes have specific services running. The api gateway can route the incoming request to literally any node, and if that node happens to lack the requested service, it will automatically balance the request to nodes that do have the task (the name given to a running service).
So, that means that the load balancer doesn't need any sort of discovery service such as Eureka in order to balance the requests to certain container's IPs or nodes, it can simply round-robin all available nodes and that's it.
As for internal requests between the services that have replicas, swarm also provides load balancing capabillities.
I have several Kubernetes clusters. Due to the company's security issues, only A 'service in Cluster A should be allowed to access B' Service in Cluster B. Can you handle such a case with istio?
Although it is possible to control the traffic using the header information in istio's virtualservice, the http header information can be manipulated at any time, which does not satisfy the security issue.
Istio has a different federation with a single control plane or multiple control plane. you can check out below. the communication across network supported by MTLS so you can be assured it can't have tampered.
Shared control plane
https://istio.io/docs/setup/kubernetes/install/multicluster/shared-gateways/
Multiple control planes
https://istio.io/docs/setup/kubernetes/install/multicluster/gateways/
This is pretty new and under heavy development, so you can try them or simply use HTTPS communication when connecting across the network.
How do I enable a port on Google Kubernetes Engine to accept websocket connections? Is there a way of doing so other than using an ingress controller?
Web sockets are supported by Google's global load balancer, so you can use a k8s Service of type LoadBalancer to expose such a service beyond your cluster.
Do be aware that load balancers created and managed outside Kubernetes in this way will have a default connection duration of 30 seconds, which interferes with web socket operation and will cause the connection to be closed frequently. This is almost useless for web sockets to be used effectively.
Until this issue is resolved, you will either need to modify this timeout parameter manually, or (recommended) consider using an in-cluster ingress controller (e.g. nginx) which affords you more control.
As per this article in the GCP documentation, there are 4 ways that you may expose a Service to external applications.
It can be exposed with a ClusterIP, a NodePort, a (TCP/UDP) Load Balancer, or an External Name.
I have two different service fabric applications. Both are stateless web api models. I do have a situation that from service 1 inside application 1, I need to invoke service 2 which is part of application 2. I am deploying both applications in the same cluster. Can someone advise the best practice here. What could be best way to communicate. Please provide some sample as well.
Fabric Transport (aka Service Remoting) is the sdk built-in communication model. Compared to communication over HTTP or WCF it does a little more, especially on the client side of the communication.
When it comes to communicating with Service Fabric services (or really, any distributed systems service) your communication should take into account that the connection could be fail to established on an initial try, or be interrupted mid communication and that you really shouldn't build your solution to expect it to always work flawlessly. The reason for this is in the nature of how Service Fabric at any time can decide to move primaries from a node to another node, the nodes themselves can go down and the services can crash. Nothing strange about he great thing with Service Fabric is that it does a lot of the heavy lifting for you when it comes to maintaining your services and nodes over time.
So, in terms of communication this means that a client needs to be able to do three things (for it to truly work in a distributed environment);
resolve the address to the service (figure out which node it is on, which port it is listening on, which partition id and replica to target and so on)
connect to the service, package and send requests and then recieve and unpack responses
retry the resolve and connect if the communication fails
Fabric Transport does all this when you are using the Service Remoting clients (like ServiceProxy) and service side listeners.
Thats the good part with Fabric Transport, you get all that out of the box and most of the time you don't have to change the default setup either. The bad part is that it only works for communication inside the cluster, i.e. you cannot communicate from the outside to a service running in the cluster using Fabric Transport. For that you need HTTP or WCF.
HTTP(s) and WCF (over HTTP(s)) communication allow you to build your own clients and handle the communication yourself. There are a number of samples on how you can do the resolve, connect and retry for HTTP clients, this one for instance
According to Microsoft there are three built-in communication options. It's up to you to decide which one works best for you. I'm personally using service remoting which is easy to quickly set up. It also allows you to exception handling in your client service.