We are exposing a stateless Owin WebAPI hosted on all nodes in our service fabric cluster (instance count -1) on Azure. The WebAPI is meant for public consumption and should be highly available even in the face of upgrades to the internal services and the WebAPI itself. We have the Azure loadbalancer (LB) in front of the cluster that probes the cluster on port 80 using a TCP probe every 5 sec in order to determine which nodes can receive http traffic.
We are experiencing issues when upgrading the WebAPI, namely that the LB directs to a node that is upgrading but is not yet registered as offline by the probe. Service Fabric does not coordinate the upgrade process with the LB so there are no chance (and no API on the Azure LB) to take the node out of rotation while upgrading.
We are wondering how people are achieving highly available http services on Service Fabric on Azure. I'm hoping someone would comment on their general approach.
How about using HTTP probing in Azure LB and adding a health check endpoint like http://node:80/_health in Web API? This way you can controller if a node should handle traffic.
Related
I am new to Kubernetes.
If there is any service deployed using EKS having 4 replicas A,B,C,D.
Usually loadbalancer directs requests to these replicas
But if I want that my request should go to replica A only or B only...
How can we achieve it.
Request to share some links or steps for guidance
What you could use are the Headless Services:
Sometimes you don't need load-balancing and a single Service IP. In
this case, you can create what are termed "headless" Services, by
explicitly specifying "None" for the cluster IP (.spec.clusterIP).
You can use a headless Service to interface with other service
discovery mechanisms, without being tied to Kubernetes'
implementation.
For headless Services, a cluster IP is not allocated, kube-proxy
does not handle these Services, and there is no load balancing or
proxying done by the platform for them. How DNS is automatically
configured depends on whether the Service has selectors defined:
With selectors
For headless Services that define selectors, the endpoints controller
creates Endpoints records in the API, and modifies the DNS
configuration to return records (addresses) that point directly to the
Pods backing the Service.
Without selectors
For headless Services that do not define selectors, the endpoints
controller does not create Endpoints records. However, the DNS
system looks for and configures either:
CNAME records for ExternalName-type Services.
A records for any Endpoints that share a name with the Service, for all other types.
So, a Headless service is the same as default ClusterIP service, but without load balancing or proxying and therefore allowing you to connect to a Pod directly.
You can also reference below guides for further assistance:
Building a headless service in Kubernetes
Kubernetes Headless service vs ClusterIP and traffic distribution
Basic and Naive question, I saw a demo on youtube how consul server and agent can work together to deliver a web service hosted at port 80 based on its availbilty. i.e. from which server is up , it will make that service availble. However for scenario I am asking if we can specifically redirect user to an endpoint of similar web service based on the location where the user is requesting from ?
if its is possible what confuguration for consul would it take to do ?
Consul uses a network tomography system to compute network coordinates for nodes in the cluster. Inside a data center this can be used in combination with Prepared Queries (in addition to other methods) to discover service instances near a given agent in the data center.
For traffic from external users, you'll want to a use a DNS global traffic manager like NS1 or F5's BIG-IP DNS to direct end-users to the closest data center. Once inside the data center, you can utilize Consul to route the connection to the nearest service instance.
See this blog post from NS1 about their integration with Consul, and ability route traffic based on service location. https://ns1.com/blog/hashicorp-and-ns1-automating-application-networking-for-microservices
I have a service fabric cluster with multiple microservices, and I want to set up Azure Front Door, however it asks for a healthcheck endpoint in the backend but I don't know how I am supposed to set it up since the cluster doesn't have an endpoint for that.
Could anyone point me in the right direction?
You could implement a health check on your service, by introducing a watchdog service. Optionally tapping into the built-in health system of SF. It could look like this:
Create an ASP .NET Core Web API, and implement some health checks. For example, a custom check if your SF Service is alive (and well). Here's how to get started. Return 200 OK from the API, if the watched SF Service is running correctly.
Run this Web API as an SF Service. Expose it through the Load Balancer.
Use its URL as the health endpoint for your Service(s).
We're planning to develop some microservices based on the play framework. They will provide rest apis and lots of them will be using akka cluster/cluster-sharding under the hood.
We would like to have an api gateway that exposes the apis of our internal services, but we're facing one big issue:
- Multiple instances of each service will be running under some ip and port.
- How will the api gateway know where the services instances are running?
- Is there maybe something load-balancer-like for play that keeps track of all running services?
Which solution(s) could possibly fill the spot for the "API Gateway"/"Load Balancer"?
The question you're asking is not really related to play framework. And there is no single answer that would solve what you need.
You could start by reading akka Service Discovery and then make your choice based what fits you more.
We're building services with akka-http and use akka-cluster but use unrelated technologies to expose and run the services.
Check out
Kong for API Gateway
Consul for DNS based service discovery
docker swarm for running containers with mesh network for load balancing
You are looking for following components,
Service Registry : The whole point of this component is to keep track of "what service are running on what addresses". This can be as simple as a simple database which keeps entries for all the running services and their instances. Generally the orchestration service is responsible to register new service instances with Service Registry. Other choice can be to have instances themselves notify the service registry about their existence.
Service Health Checker : This component is mostly responsible for doing periodic runtime checks on the registered service instances and tell service registry if any of them is not working. The service registry implementation can then either mark these instances as "inactive" till they are found to be working by Service Health Checker in future (if ever).
Service Resolution : This is the conceptual component responsible for enabling a client to somehow get to the running service instances.
The whole of above components is called Service Discovery.
In your case, you have load-balancers which can act as a form of ServiceDiscovery.
I don't think load-balancers are going to change much over time unless you require a very advanced architecture, so your API gateway can simply "know" the url's to load-balancers for all your services. So, you don't really need service registry layer.
Now, your load-balancers inherently provide a health-check and quarantine mechanism for instances. So, you don't need an extra health check layer.
So, the only piece missing is to register your instances with the load balancer. This part you will have to figure out based on what your load-balancers are and what ecosystem they live in.
If you live in AWS ecosystem and your load balancers are ELB, then you should have things sorted out in that respect.
Based on Ivan's and Sarvesh's answers we did some research and discovered the netflix OSS projects.
Eureka can be used as service locator that integrates well with the Zuul api gateway. Sadly there's not much documentation on the configuration, so we looked further...
We've now finally choosen Kubernetes as Orchestator.
Kubernetes knows about all running containers, so there's no need for an external service locator like Eureka.
Traefik is an api gateway that utilizes the kuberentes api to discover all running microservices instances and does load balancing
Akka management finds all nodes via the kubernetes api and does the bootstrapping of the cluster for us.
Currently I'm working on a real-time online game. First I implemented a go server with socket.io for handling messages between client and my game world and it works fine. Now for user data managing I need a http api for some functionality like login. I want to use awesome http/net package for that purpose. Should I serve the http server on different Port?
My next question is for deploying I want to use google container engine. Can I use pods with two ports open?
As far as I understood from your explanation, you need two ports open for two different APIs running in your application. Regarding Exposing two ports in Google Container Engine, you can read the discussion here that describes ways to expose ports in a pod.
Moreover, I invite you read this tutorial that involves deploying an API in a GKE cluster with a containerPort in a pod, Creating a Kubernetes service to allow internal cluster traffic to your pods (routing requests on an incoming port to your API targetPort), and creating an Ingress service to define what traffic is allowed into your cluster and where it goes. You can define different APIs with different targetPorts and run them on different pods. You can try it as an alternative. For more documentation on Exposing Applications using Services, you can read this GKE doc.