I want to display performance metrics of the docker compose application with prometheus. Application is runnin on AWS EC2 instance. I am planning to use cAdvisor to scare them. My question is: Should I add all the services of the application in prometheus.yml file with target ports? or should I only configure cAdvisor?
Assuming I have two services: customerService and orderService. Both are Springboot applications contained in its own dockerfile.
In production they should be managed using Kubernetes. So there could be mutliple instances of each service. As the services should be able to call each other via REST I want to use ServiceDiscovery to get the destination of a service instance of the respective other type (e.g. customerService wants to get host, port, etc. of a running orderService instance).
I want to achieve this by using an injected DiscoveryClient in my Spring Boot services.
I understand how this works in production as there is the Kubernetes cluster the DiscoveryClient communicates with.
But how does this work in a local environment where there is no Kubernetes but only Docker running?
I think you should be looking into Kubernetes services instead of using an injected Discovery client.
For Service discovery and load-balancing, you can use Services in Kubernetes. From Kubernetes documentation:
An abstract way to expose an application running on a set of Pods as a
network service. With Kubernetes, you don't need to modify your
application to use an unfamiliar service discovery mechanism.
Kubernetes gives Pods their own IP addresses and a single DNS name for
a set of Pods and can load-balance across them.
By this, you can also avoid the overhead of maintaining discovery server instances and their availability.
How does this work in a local environment where there is no Kubernetes but only Docker running?
You can use spring profiles for picking this URL based on your environment.
For example, you'll have two spring configuration files (application-dev, application-prod) and in the dev file the URL for the second application would be localhost relative and in the prod file, the URL would be the DNS which you'll get as a part of Kubernetes service setup. So use the profile dev while running in local and prod while running in production(or an appropriate profile for your Kubernetes environment).
I have some questions about terminology that I should use. I have searched up the components that I am using, but I am still not sure. Can you please check if these are right way to describe each component? If not, can you please revise it?
I develop microservices using Spring Framework (Spring boot).
I deploy components on cloud via Kubernetes and Docker. (I understand Docker is used for containerizing and Kubernetes is for orchestrating multiple containers. Is it right to say "I deploy on Kubernetes or via Kubernetes"?)
CI/CD pipeline using Jenkins and Azure DevOps. (Project uses Azure DevOps for triggering CI/CD in Jenkins)
Please note that this project was already there when I joined the team, and I am new to all of these concepts. I understand what they do briefly and I know what each terminology means on entry level, but I just want to make sure if I am saying these in right ways.
I would say that you deploy services, not components, but your team might have its own terminology.
You do deploy on Kubernetes.
Docker is used to create and manage containers and container images. Kubernetes does not use Docker but can use images created via Docker to deploy containers in Pods (via the OCI format)
The rest seems right to me :)
In Kubernetes an app as a unit is called Service. It has components reachable under it, for example: deployment, pod, etc. This set of components could be for microservice or service, but terminology just a service.
You can list all services using kubectl get services
The Docker is related here only because pod has a container or sometimes few of them. So Kubernetes supports Docker and its images. It's most popular container runtime in 2021, but not the only one for Kubernetes.
To combine everything, the right sentence to me would be:
I deploy a service on Kubernetes
I deploy a service on Amazon EKS
I deploy a service on Azure Kubernetes
etc.
Adding "cloud" doesn't add so much value as Kubernetes platform is same regardless of cloud provider chosen. Of course, there are some differences in cloud, but not in Kubernetes core.
Docker doesn't fit here and usually discussed when going into details, especially about images, building them, etc.
Also off topic, but even pod is more used than container, because in Kubernetes you access the pod (while in Docker Compose, you access the container). Of course, you can access Docker container if there are multiple of them setup in a pod and container engine is Docker in this context.
I develop microservices using Spring Framework (Spring boot).
To understand in the most basic form - Microservices is an architectural pattern for developing applications, so there can be many ways to implement this pattern (which is nothing but developing applications) and Spring Boot is one of them. More precisely you should say "I develop microservices using Spring Boot"
I deploy components on cloud via Kubernetes and Docker. (I understand
Docker is used for containerizing and Kubernetes is for orchestrating
multiple containers. Is it right to say "I deploy on Kubernetes or via
Kubernetes"?)
Precisely speaking you are not correct here - you understand the purpose of Docker and Kubernetes but putting it wrongly w.r.t. application and its deployment.
Do read this, where it is clearly mentioned that K8s doesn't build or deploy code. When the official documentation is clearly saying that K8s doesn't deploy code, then how you can say that I deploy via K8s?
I would say you shouldn't put it using either of the statement you mentioned, rather you should say - "My application runs (or is deployed) in Cloud on a Kubernetes cluster (or GKE or EKS or AKS)".
CI/CD pipeline using Jenkins and Azure DevOps. (Project uses Azure
DevOps for triggering CI/CD in Jenkins)
Looks fine to me.
Please note that in general you can say anything as long as the other person is able to understand, so in general all your statements might sell well. But I guess you were looking for "precise" statements, and I gave you my thoughts on the same.
Guys here are right but I would like to add that when speaking of Kubernetes components you usually think of Control Plane Components or Node Components and when speaking of smaller units the Services and Pods comes to mind first. Note that a Kubernetes service is an abstract way to expose an application running on a set of Pods as a network service. So addressing your main question:
Is it correct to say I am deploying components via Kubernetes?
It would be better to say that you are deploying a service/application (running) on Kubernetes (on AKS/AWS/GCP/etc).
Are there any document for deploying Spring Cloud Data Flow on AWS EKS? I am looking for example or guide video about this problem.
There are no special instructions for running SCDF on EKS or any other K8s distribution. As far as you have a K8s cluster and the necessary service accounts, you will be able to provision SCDF on a given namespace.
You can either use the release-tagged deployment YAMLs directly or the SCDF's Bitnami Helm chart. Depending on your customization needs, you'll find the deployment YAMLs more flexible, though.
We are currently working on a approach which will work with collaboration of multiple microservices to provide us with solution.
Dataflow example:
These services will be deployed in ECS with ALB in front of it. I want to stitch together a workflow for the system and needed help with the following:
We plan to use Route 53 DNS service for the Service Discovery. I know that conductor uses Eureka client for service discovery. Is it possible just to swap out of use DNS based implementation rather than Eureka??
Can Conductor be deployed as docker images in an ECS cluster? Can I create this internal to ECS cluster and not deploy it on EC2 servers? Any documentation that will help? Do we maintain Docker Images or we need to create our own?