We are currently working on a approach which will work with collaboration of multiple microservices to provide us with solution.
Dataflow example:
These services will be deployed in ECS with ALB in front of it. I want to stitch together a workflow for the system and needed help with the following:
We plan to use Route 53 DNS service for the Service Discovery. I know that conductor uses Eureka client for service discovery. Is it possible just to swap out of use DNS based implementation rather than Eureka??
Can Conductor be deployed as docker images in an ECS cluster? Can I create this internal to ECS cluster and not deploy it on EC2 servers? Any documentation that will help? Do we maintain Docker Images or we need to create our own?
Related
I have some questions about terminology that I should use. I have searched up the components that I am using, but I am still not sure. Can you please check if these are right way to describe each component? If not, can you please revise it?
I develop microservices using Spring Framework (Spring boot).
I deploy components on cloud via Kubernetes and Docker. (I understand Docker is used for containerizing and Kubernetes is for orchestrating multiple containers. Is it right to say "I deploy on Kubernetes or via Kubernetes"?)
CI/CD pipeline using Jenkins and Azure DevOps. (Project uses Azure DevOps for triggering CI/CD in Jenkins)
Please note that this project was already there when I joined the team, and I am new to all of these concepts. I understand what they do briefly and I know what each terminology means on entry level, but I just want to make sure if I am saying these in right ways.
I would say that you deploy services, not components, but your team might have its own terminology.
You do deploy on Kubernetes.
Docker is used to create and manage containers and container images. Kubernetes does not use Docker but can use images created via Docker to deploy containers in Pods (via the OCI format)
The rest seems right to me :)
In Kubernetes an app as a unit is called Service. It has components reachable under it, for example: deployment, pod, etc. This set of components could be for microservice or service, but terminology just a service.
You can list all services using kubectl get services
The Docker is related here only because pod has a container or sometimes few of them. So Kubernetes supports Docker and its images. It's most popular container runtime in 2021, but not the only one for Kubernetes.
To combine everything, the right sentence to me would be:
I deploy a service on Kubernetes
I deploy a service on Amazon EKS
I deploy a service on Azure Kubernetes
etc.
Adding "cloud" doesn't add so much value as Kubernetes platform is same regardless of cloud provider chosen. Of course, there are some differences in cloud, but not in Kubernetes core.
Docker doesn't fit here and usually discussed when going into details, especially about images, building them, etc.
Also off topic, but even pod is more used than container, because in Kubernetes you access the pod (while in Docker Compose, you access the container). Of course, you can access Docker container if there are multiple of them setup in a pod and container engine is Docker in this context.
I develop microservices using Spring Framework (Spring boot).
To understand in the most basic form - Microservices is an architectural pattern for developing applications, so there can be many ways to implement this pattern (which is nothing but developing applications) and Spring Boot is one of them. More precisely you should say "I develop microservices using Spring Boot"
I deploy components on cloud via Kubernetes and Docker. (I understand
Docker is used for containerizing and Kubernetes is for orchestrating
multiple containers. Is it right to say "I deploy on Kubernetes or via
Kubernetes"?)
Precisely speaking you are not correct here - you understand the purpose of Docker and Kubernetes but putting it wrongly w.r.t. application and its deployment.
Do read this, where it is clearly mentioned that K8s doesn't build or deploy code. When the official documentation is clearly saying that K8s doesn't deploy code, then how you can say that I deploy via K8s?
I would say you shouldn't put it using either of the statement you mentioned, rather you should say - "My application runs (or is deployed) in Cloud on a Kubernetes cluster (or GKE or EKS or AKS)".
CI/CD pipeline using Jenkins and Azure DevOps. (Project uses Azure
DevOps for triggering CI/CD in Jenkins)
Looks fine to me.
Please note that in general you can say anything as long as the other person is able to understand, so in general all your statements might sell well. But I guess you were looking for "precise" statements, and I gave you my thoughts on the same.
Guys here are right but I would like to add that when speaking of Kubernetes components you usually think of Control Plane Components or Node Components and when speaking of smaller units the Services and Pods comes to mind first. Note that a Kubernetes service is an abstract way to expose an application running on a set of Pods as a network service. So addressing your main question:
Is it correct to say I am deploying components via Kubernetes?
It would be better to say that you are deploying a service/application (running) on Kubernetes (on AKS/AWS/GCP/etc).
What are the benefits to use "spring service discovery kubernetes" instead of using directly the Service DNS coming from Kubernetes?
I mean, If I deploy in kubernetes 2 services (service-a and service-b), and service-b exposes a Rest API.
service-a can easily connect to service-b using the url "http://service-b/...".
Question #1. In order to let service-a be able to connect to service-b using the service DNS, service-b has to be deployed before service-a?
Question #2. What are the pros/cons using the spring discovery?
Question #1:
No, the order in which you deploy the services is not important to use the kubernetes DNS services to resolve the ips, the only thing here is that if you deploy serviceA after serviceB, you will have in serviceA as an environment variable the ip of serviceB but not the inverse.
Question #2:
The spring service discovery is an alternative to the native kubernetes service discovery and it is used by other spring cloud projects like spring-cloud-eureka to perform the service discovery. The only pros I see in this approach is that you can custom the load balancing algorithm tath you can use to spread the load among the different instances
Are there any document for deploying Spring Cloud Data Flow on AWS EKS? I am looking for example or guide video about this problem.
There are no special instructions for running SCDF on EKS or any other K8s distribution. As far as you have a K8s cluster and the necessary service accounts, you will be able to provision SCDF on a given namespace.
You can either use the release-tagged deployment YAMLs directly or the SCDF's Bitnami Helm chart. Depending on your customization needs, you'll find the deployment YAMLs more flexible, though.
I am migrating a Spring Boot application from PCF to AWS ECS which is currently using cloud config server reading properties from git repo, and aws RDS. Now in ECS is there a way we can implement config servers along with refresh in ECS AWS.
I think ECS operates on a different level.
Spring Cloud Config server is a solution that works especially good with spring boot based applications. For example, a refresh option that you've mentioned is implemented as a special Scope which is purely a spring (applicative) thing.
On the other hand, AWS ECS (stands for Elastic Container Service) provides a way to work with containers in a general sense (with scaling and everything). It doesn't require the containers to be spring-based or even java based.
So, I think you might want to consider keeping a spring boot driven microservice for config server just like you have now, but wrap it into docker container and deploy it in AWS ECS
I am trying to learn and create a microservice using Spring Cloud. I am using Spring mvc and Spring Boot for development. I added separate Spring Boot application for both Eureka server and Zuul client. I have three Spring Boot application now. One Sureka server, one for Zuul routing and third one is for my microservice application. I run the Eureka server and the microservice and Zuul found to be running properly in Eureka server UI.
My doubt is that, I am planned to use routing and service discovery in my microservice. So I have a total of three Spring Boot applications. Can I deploy these three applications into AWS Elastic Beanstalk?
When I exploring I found that Pivotal Cloud Foundry topic. Since I am planing to use AWS Elastic Beanstalk. So what is the role of Pivotal Cloud Foundry in my application?
I am still confused about microservice development using Spring Cloud.
So there are two elements in your confusion. Is Amazon Beanstalk suitable?
So answer is yes. but you have overhead of orchestrating various services of amazon over then. Basically Amazon Elastic Beanstalk is more form based which means you need to tell what exactly you want to do with application for example route setup , scaling mechanism , application portability, application health management , Integration with external log aggregators etc. This is big learning curve to many people (including me :) ). Where as these with PCF these things are easy and straightforward and sometimes implicitly done.
Now Can I use pivotal cloud foundry with amazon? answer will be absolutely yes.
Pivotal cloud foundry is Wrapper written over open source Cloud Foundry project.
It is more succinct PaaS option than Amazon Beanstalk. You can host PCF on amazon ec2 ( IaaS )boxes using BOSH software. To answer you PCF is a alternative to Amazon Beanstalk. You don't need PCF is you want to use Amazon Beanstalk and vice versa.