I have a functionning zookeeper stateful set, i have manually created.
I want to migrate to a helm installation of zookeeper.
My kubernetes cluster run on aws.
How to migrate volume from one stateful to the other ?
There is a tool called "chartify", that can generate Helm Charts from an existing Kubernetes api objects. It works both as a Helm plugin or as a stand-alone tool.
Please mind that there is already a Helm chart for zookeeper in the incubator repository.
Related
I'm new to Kubernetes and I'm learning about statefulsets. For stateful applications, where the identity of pods matter, we use statefulsets instead of simple deployments so each pod can have its own persistent volume. The writes need to be pointed to the master pod, while the reading operations can be pointed to the slaves. So pointing to the ClusterIP service attached to the statefulset won't guarantee the replication, instead we need to use a headless service that will be pointing to the master.
My questions are the following :
How to edit the application.properties in spring boot project to use the slaves for reading operations ( normal ClusterIP service ) and the master for writing/reading operations ( Headless service )?
In case that is unnecessary and the headless service does this work for us, how does it work exactly since it's pointing to the master ?
How can we run the Hazelcast management center on Kubernetes?
What i did so far is - Deployed the hazelcast/management-center Docker image on our Kubernetes cluster. However as seen on the management-center Pod log, it has started on localhost:8080 by default. I did this with a 'Deployment' on K8s pointing to Docker hazelcast/management-center image. Hazelcast Server is running as the Spring Boot application, also other applications are able to connect to this as Hazelcast client.
Question is how can we run/connect the management center to our Hazelcast cluster running on the same namespace in the Kubernetes (with 2 members in the cluster)?
I suggest using the official helm chart, it includes Management Center by default.
If it doesn't work for you, first I would check if there is a k8s Service defined for the deployment, and if yes, if there is a k8s Ingress exposing that service.
Is there a way to run Elastic cloud on Kubernetes (ECK) with helm3?
As much as i know, there is no helm chart for ECK operator however for the Elasticsearch stack there is a helm chart available.
Elastic stack helm chart : https://github.com/elastic/helm-charts/tree/master/elasticsearch
ECK is operate you can extend Kubernetes orchestration by YAML files or else you can create own helm chart as per need if required.
ECK quick deploy : https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html
ECK is providing official support in GKE however i think you have idea about the advantages & disadvantages of using ECK.
Yes, starting ECK 1.3.0, there is an official Helm chart for deploying the operator.
I have a kubernetes cluster hosted on azure cloud. I had installed kafka resources using below helm chart https://github.com/confluentinc/cp-helm-charts/tree/master/charts/cp-kafka. This helm chart successfully deployed zoopkeeper pods and broker pods etc. Now I want to write a golang based application which connects with any of the kafka broker installed on my kubernetes cluster and creates a new producer and publishes messages. Any help would be highly appreciated.
You can use the following string in bootstrap.servers to communicate with the brokers <helm-release-name>-cp-kafka-headless.<namespace>:9092 or bootstrap service which is created as part of confluent helm chart <helm-release-name>-cp-kafka. When you hit this service, it will randomly got to any of the brokers for the first time and get all the metadata information which is synced through zookeeper.
The subsequent requests will be made to individual brokers based on information found in metadata.
You would deploy your Golang code in a container, in k8s, then set bootstrap.servers to the Kafka Deployment's Service name, ideally via an environment variable
There are a lot of examples of the SMACK stack, but in my infrastructure I would like to use ElasticSearch and Confluent Kafka Connect and Kafka Streams.
There is a great tutorial on deploying a CloudFormation-based SMACK stack environment and another in creating an IoT pipeline with SMACK as well.
Since I am working on a Lambda architecture, I am first starting with my batch data using ElasticSearch (not Cassandra) and would like to know if there are CloudFormation templates that use Kafka Connect, ElasticSearch. Eventually we want to use Kafka Streams with InfluxDB?
DC/OS has AWS CloudFormation templates and install instructions. Once you have DC/OS installed you can install ElasticSearch and Kafka from the Mesosphere Universe as DC/OS packages.