There're already some CRDs defined in my kubernetes cluster.
kubectl can create/update/delete the resources well.
When I tried to do those operations with program, the way I found by searching is to generate code with below tool:
https://github.com/kubernetes/code-generator
I'm wondering why kubectl can do it out-of-box without generating code for CRDs.
Is it necessary to generate code in order to add or delete a CRD resource?
Thanks!
First lets understand what CRD is.
The CustomResourceDefinition API resource allows you to define custom resources. Defining a CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource. The name of a CRD object must be a valid DNS subdomain name.
This frees you from writing your own API server to handle the custom resource, but the generic nature of the implementation means you have less flexibility than with API server aggregation.
Why would one create Custom Resources:
A resource is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind. For example, the built-in pods resource contains a collection of Pod objects.
A custom resource is an extension of the Kubernetes API that is not necessarily available in a default Kubernetes installation. It represents a customization of a particular Kubernetes installation. However, many core Kubernetes functions are now built using custom resources, making Kubernetes more modular.
Custom resources can appear and disappear in a running cluster through dynamic registration, and cluster admins can update custom resources independently of the cluster itself. Once a custom resource is installed, users can create and access its objects using kubectl, just as they do for built-in resources like Pods.
So to answer your question, if you need a functionality that is missing from Kubernetes you need to create it yourself using CRDs. Without it cluster won't know what you want and how to get it.
If you are looking for examples of usage of Kubernetes Client-go you can find them on the official GitHub Client-go/examples
Related
I'm using the k8s client-go library to mainly perform CRUD operations for CRDs.
Also, in kubernetes we have a unique UID for each resource.
So is there a way to query resources in kubernetes using this UID. If so can you please share any resources, code snippets which helpful for this??
We need to create a Dataflow job that ingests from PubSub to Elasticsearch but the job can not make outbound internet connections to reach Elastic Cloud.
Is there a way to pass proxy parameters to the Dataflow vm on creation time?
Found this article but proxy parameters are part of a maven app, I'm not sure how to use it here.
https://leifengblog.net/blog/run-dataflow-jobs-in-a-shared-vpc-on-gcp/
Thanks
To reach an external endpoint you’ll need to configure internet access and firewall settings, depending on your use case, your VMs may also need access to other resources you can check in this document which method you’ll need to configure for Dataflow. Before selecting the method that you’ll choose please check the document how to specify a network or a subnetwork.
In GCP, in subnetwork, you can enable Google Private Access, and the VMs in that subnetwork will be able to reach all the GCP endpoints (Dataflow, BigQuery, etc), even if they have private IPs only. There is no need to set up a proxy. See this document.
For instance, for Java pipelines, I normally use private IPs only for the Dataflow workers, and they are able to reach Pubsub, BigQuery, Bigtable, etc.
For Python pipelines, if you have external dependencies, the workers will need to reach the PyPi, and for that, you need Internet connectivity. If you want to use private IPs in Python pipelines, you can ship those external dependencies in a custom container, so the workers don't need to download them.
You can use a maven file right after you write your pipeline, you must create and stage your template file(mvn) you can follow this example.
I'm able to create a GKE cluster using the golang container lib here.
Now for my golang k8s client to be able to deploy my k8s deployment files there, I need to get the kubeconfig from the GKE cluster. However I can't find the relevant api for that in the container lib above. Can anyone please point out what am I missing ?
As per #Subhash suggestion I am posting the answer from this question:
The GKE API does not have a call that outputs a kubeconfig file (or
fragment). The specific processing between fetching a full cluster
definition and updating the kubeconfig file are implemented in python
in the gcloud tooling. It isn't part of the Go SDK so you'd need to
implement it yourself.
You can also try using kubectl config set-credentials (see
this) and/or see if you can vendor the libraries that implement
that function if you want to do it programmatically.
We are trying to spin up a Stateful MQ manager with Azure File System as persistent storage mounted for data in an Azure Kubernetes cluster. Here is the link which we followed. We exposed the service type as LoadBalancer as shown in below command.
helm install stable/ibm-mqadvanced-server-dev --version 3.0.1 --set service.type=LoadBalancer,security.initVolumeAsRoot=true,license=accept
By default, it takes default storage class as Azure disk. Here I want to use the Azure File System as Persistence storage.so, How should I pass my Azure file System name? and the other thing is, we can able to run the pod successfully without any restarts, but unable to access the web interface of it. so, we don't know where might be the exact issue raises while accessing the service?
Github repo you've linked specifically mentions dataPVC.storageClassName under configuration. This is used to define storage class, if you dont have a storage class for Azure Files (i think it doesnt exist by default), you'd need to create it and then reference it, so it would use that class.
How to set it up: here
I have setup the Elasticsearch Certified by Bitnami on GCP
Which I would link to put behind the HTTP(S) Load Balancing on GCP for auto scaling propose. What I have done is create snapshot and use it to create image for instance template. But the Instance group continuous return "instance in being verified" and "Recreated instance" for long time do I don't know where the problem is so I design to use the default instance template from GCP instead.
My question is, when the new node created of when the data in elasticsearch updated how can I sync data between node in the GCP load balancer? Think about when there is high traffic and load balancer created the new coming node, and when the query come in from load balance how the new node have the exactly same data with the existing node or when the new index come in, all the node get the new index.
PS I dont mind for the delay if it less than 5 mins it is acceptable.
Thanks in advance for helping out.
In GCP, if you want to sync your data between nodes, we recommend using a centralized location to store your data. You can use Cloud Storage, Cloud SQL, Cloud File System etc. You can check this link to find more options for the data storage. Then you can create an instance template that specifies that when any instance is created it will use the custom image and has access to that centralized database. This is a recommended workaround rather than replicate new instances with data. You can find this link for the similar kind of thread.
For your Elasticsearch setup, I'll recommend deploying an Elasticsearch Cluster that provides multiple VMs that you can customize the configuration. If deploying cluster, this other Stackoverflow post suggest that is not not necessary to use a load balancer as Elasticsearch handles the load between the nodes.