I'm trying to create a pod using Kubernetes client api in Go and I've been getting below error in TravisCI,
ERRO Running error: buildir: analysis skipped: errors in package: [/home/travis/gopath/src/github.com/pravarag/test-repo/check_pod.go:25:70: cannot use desiredPod (variable of type *"k8s.io/api/core/v1".Pod) as context.Context value in argument to s.kubeClient.CoreV1().Pods(desiredPod.Namespace).Create: missing method Deadline /home/travis/gopath/src/github.com/pravarag/test-repo/check_pod.go:25:80: too few arguments in call to s.kubeClient.CoreV1().Pods(desiredPod.Namespace).Create
Below is the code,
import (
"fmt"
"go.uber.org/zap"
core "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func (s *server) createPod() {
// build the pod definition
desiredPod := getPodObjet()
pod, err := s.kubeClient.CoreV1().Pods(desiredPod.Namespace).Create(desiredPod)
if err != nil {
s.log.Fatal("Failed to create the static pod", zap.Error(err))
}
fmt.Println("Created Pod: ", pod.Name)
}
func getPodObjet() *core.Pod {
pod := &core.Pod{
ObjectMeta: metav1.ObjectMeta{
Name: "test-pod",
Namespace: "default",
Labels: map[string]string{
"app": "test-pod",
},
},
Spec: core.PodSpec{
Containers: []core.Container{
{
Name: "busybox",
Image: "busybox",
ImagePullPolicy: core.PullIfNotPresent,
Command: []string{
"sleep",
"3600",
},
},
},
},
}
return pod
}
I tried to check what that error is pointing to and it seems, the actual pod Interface in K8s client code here: https://godoc.org/k8s.io/client-go/kubernetes/typed/core/v1#PodInterface
is expecting 3 arguments: one is "context.Context", "pod *v1.Pod" and "opts metav1.CreateOptions"
I tried to pass the values as:
pod, err :=s.kubeClient.CoreV1().Pods(desiredPod.Namespace).Create(context.Context, desiredPod, opts metav1.CreateOptions{})
But that doesn't work also. Even in the IDE, the code lint is pointing to missing arguments but I've seen couple of examples used to create a pod in above mentioned way that worked previously.
just use context.TODO() as argument to pass context.
try this one.
pod, err := s.kubeClient.CoreV1().Pods(desiredPod.Namespace).Create( context.TODO(), desiredPod , metav1.CreateOptions{})
here is the updated code:
import (
"fmt"
"context"
"go.uber.org/zap"
core "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func (s *server) createPod() {
// build the pod definition
desiredPod := getPodObjet()
pod, err := s.kubeClient.CoreV1().Pods(desiredPod.Namespace).Create( context.TODO(), desiredPod , metav1.CreateOptions{})
if err != nil {
s.log.Fatal("Failed to create the static pod", zap.Error(err))
}
fmt.Println("Created Pod: ", pod.Name)
}
func getPodObjet() *core.Pod {
pod := &core.Pod{
ObjectMeta: metav1.ObjectMeta{
Name: "test-pod",
Namespace: "default",
Labels: map[string]string{
"app": "test-pod",
},
},
Spec: core.PodSpec{
Containers: []core.Container{
{
Name: "busybox",
Image: "busybox",
ImagePullPolicy: core.PullIfNotPresent,
Command: []string{
"sleep",
"3600",
},
},
},
},
}
return pod
}
Related
Anyone know how to create a Custom Resource using go-client. basically equivalent of kubectl apply -f 'yaml path'
apiVersion: k6.io/v1alpha1
kind: K6
metadata:
name: k6-sample
spec:
parallelism: 1
#arguments: --out statsd
#cleanup: post
script:
configMap:
name: "staging-stress-test"
file: "staging.js"
Have a go-client code to generate a Custom Resource below
func createk6CR(clientset *kubernetes.Clientset) (string, error) {
k6plugin := &v1alpha1.K6{
TypeMeta: metav1.TypeMeta{
APIVersion: "k6.io/v1alpha1",
Kind: "K6",
},
ObjectMeta: metav1.ObjectMeta{
Name: "k6-sample-1",
Namespace: "default",
},
Spec: v1alpha1.K6Spec{
Parallelism: 3,
Script: v1alpha1.K6Script{
ConfigMap: v1alpha1.K6Configmap{
Name: "staging-stress-test",
File: "staging.js",
},
},
},
// Status: v1alpha1.K6Status{
// Stage: "started",
// },
}
body, err := json.Marshal(k6plugin)
if err != nil {
fmt.Printf("error getting Kubernetes config: %v\n", err)
os.Exit(1)
}
data, err := clientset.RESTClient().
Post().
AbsPath("/apis/k6.io/v1alpha1/namespaces/default/k6").
Body(body).
DoRaw(context.TODO())
if data != nil {
str := string(data[:])
fmt.Printf("return data: %v\n", str)
//os.Exit(1)
}
return "success", err
}
But I get Page 404 not found on AbsPath("/apis/k6.io/v1alpha1/namespaces/default/k6").
Found what was wrong with it, when you do kubectl apply pass it with -v 8 to see the Abspath check for POST
kubectl apply -f 'resource path'
I0816 09:20:56.239402 15535 round_trippers.go:463] POST https://0.0.0.0:43117/apis/k6.io/v1alpha1/namespaces/default/k6s?fieldManager=kubectl-client-side-apply&fieldValidation=Strict
I0816 09:20:56.239428 15535 round_trippers.go:469] Request Headers:
so the code with corrected AbsPath should be below,
data, err := clientset.RESTClient().
Post().
AbsPath("/apis/k6.io/v1alpha1/namespaces/default/k6s").
Body(body).
DoRaw(context.TODO())
When I get the kubernetes resources from api with client-go, but I can't found the apiversion and kind in the response, the apiversion and kind is empty. How can I get the apiversion and kind of the resource?
below is my code:
package main
import (
"flag"
"k8s.io/client-go/tools/clientcmd"
"log"
"k8s.io/client-go/kubernetes"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"fmt"
)
var clientset *kubernetes.Clientset
func main() {
k8sconfig := flag.String("k8sconfig","./k8sconfig","kubernetes config file path")
flag.Parse()
config , err := clientcmd.BuildConfigFromFlags("",*k8sconfig)
if err != nil {
log.Println(err)
}
clientset , err = kubernetes.NewForConfig(config)
if err != nil {
log.Fatalln(err)
} else {
fmt.Println("connect k8s success")
}
pods,err := clientset.CoreV1().Pods("").List(metav1.ListOptions{})
if err != nil {
log.Println(err.Error())
}
for _, pod := range pods.Items{
fmt.Println("apiversion: ", pod.APIVersion, "kind: ", pod.Kind)
}
}
The output:
apiversion: kind:
apiversion: kind:
apiversion: kind:
apiversion: kind:
apiversion: kind:
apiversion: kind:
apiversion: kind:
apiversion: kind:
......
......
I think the issue is that you are getting the list of pods using the List() API so it is not a pod as you expect and doesn't have a Kind field.
You need to iterate over the list of pods to access individual pods:
for _, pod := range pods.Items {
fmt.Printf("%s %s\n", pod.GetName(), pod.GetCreationTimestamp())
}
The Kind field is present as part of Pod's Metadata and can be accessed using pod.ObjectMeta.Kind.
You are not getting the APIVersion and Kind because it was ignored from the code.
If you take a look at the API server JSON response, it will be something like below:
{
"kind":"PodList",
"apiVersion":"v1",
"metadata":{
"resourceVersion":"2397"
},
"items":[
{
... ... ...
So the response does contain the APIVersion and Kind. But when the response is decoded to k8s object, here
out, _, err := r.decoder.Decode(r.body, nil, obj)
Here you can see that the second output parameter is ignored, which is schema.GroupVersionKind.
func (c *Something) Decode([]byte, *schema.GroupVersionKind, runtime.Object) (runtime.Object, *schema.GroupVersionKind, error) {
}
N.B.: When you are making an API call using the client-go (unless dynamic client), you already know the APIVersion (ie. CoreV1() ) and the Kind ( ie. List()).
In a custom Kubernetes operator implemented with the operator-sdk in golang is it possible to call the custom API directly and retrieve the object as YAML?
For example. I have a custom resource
apiVersion: test.com/v1alpha1
kind: TEST
metadata::
name: example-test
spec:
replicas: 3
randomname: value
I don't know ahead of time what the fields in the spec are going to be apart from replicas. So I am not able to create a go type that includes structs to hold the entries.
So rather than doing:
instance := &testv1alpha1.Test{}
err := r.client.Get(context.TODO(), nameSpaceName, instance)
I want to be able to do something like:
instanceYAML := genericContainer{}
err := r.client.GetGeneric(context.TODO(), nameSpaceName, instance)
and then parse the instanceYAML to check the entries.
This is called the "unstructured" client. The docs are pretty light so I recommend looking over the tests as examples https://github.com/kubernetes-sigs/controller-runtime/blob/ea32729106c995d9df310ac4731c2061490addfb/pkg/client/client_test.go#L1536-L1566
Using the unstructured type client all the operation done on the CoreAPI resource can be done on the CustomResourceDefinition(CRD)
package util
import (
"context"
"encoding/json"
"strconv"
"github.com/go-logr/logr"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime/schema"
types "k8s.io/apimachinery/pkg/types"
_ "k8s.io/client-go/plugin/pkg/client/auth"
"sigs.k8s.io/controller-runtime/pkg/client"
)
var reqLogger logr.Logger
func CRUDonCRD(c client.Client) {
u := createUnstructuredObject(c)
GetCR(u, c)
PatchCR(u, c, 1)
DeleteCR(u, c)
}
func createUnstructuredObject(c client.Client) *unstructured.Unstructured {
u := &unstructured.Unstructured{}
u.SetGroupVersionKind(schema.GroupVersionKind{
// Group: "<crd group name>",
// Kind: "<crd kind>",
// Version: "<crd version>",
Group: "com.cumulus.netq.operator.transport",
Kind: "TransportOperator",
Version: "v1",
})
_ = c.Get(context.Background(), client.ObjectKey{
// Namespace: "<namespace>",
// Name: "cr name",
Namespace: "default",
Name: "transport-operator",
}, u)
return u
}
type patchStringValue struct {
Op string `json:"op"`
Path string `json:"path"`
Value string `json:"value"`
}
func PatchCR(u *unstructured.Unstructured, c client.Client, replicaCount int) error {
payload := []patchStringValue{{
Op: "replace",
Path: "/spec/replicas/replicasOpta",
Value: strconv.Itoa(replicaCount),
}}
payloadBytes, _ := json.Marshal(payload)
err := c.Patch(context.Background(), u, client.RawPatch(types.JSONPatchType, payloadBytes))
if err != nil {
reqLogger.Error(err, "error occured while patching")
}
reqLogger.Info("Patching is successful", "Patched Transport Operator Object", u)
return err
}
func GetCR(u *unstructured.Unstructured, c client.Client) {
// key := client.ObjectKey{Namespace: "default", Name: "<cr-name>"}
key := client.ObjectKey{Namespace: "default", Name: "transport-operator"}
err := c.Get(context.Background(), key, u)
if err != nil {
reqLogger.Error(err, "error occured while getting the resource")
}
reqLogger.Info("Got the resource", "Resource Object", u)
}
func DeleteCR(u *unstructured.Unstructured, c client.Client) {
err := c.Delete(context.Background(), u)
if err != nil {
reqLogger.Error(err, "error occured while deleting the resource")
}
reqLogger.Info("Resource deleted", "Resource ", u)
}
Here is what I did in the Operator SDK to get a object from kubernetes.
Ref: https://github.com/kubernetes-sigs/controller-runtime/blob/master/pkg/client/example_test.go#L57
func (r *ConfigDeploymentReconciler) findObjectsForConfigMap(configMap client.Object) []reconcile.Request {
pod := &corev1.Pod{}
_ = r.Client.Get(context.Background(), client.ObjectKey{
Namespace: "prometheus",
Name: "prometheus-node-exporter-7vxqs",
}, pod)
fmt.Println("Got this pod", pod.String())
return nil
}
I'm looking for the go equivalent of:
kubectl get some-custom-resource-kind some-custom-resource -o yaml > file.yaml
Modify the yaml file...
kubectl apply -f file.yaml
Kubernetes has a client go library for standard resource kinds.
And various vendors have client go libraries for their custom resources.
But I need to get/update a resource kind that doesn't have a publicly available client go library. The logic is implemented in bash script today and I'm trying to move that function to a go controller.
Seems like it there should be a straightforward way in go to do the equivalent of kubectl.
Thanks, Paul
For any type, including your CRDs, use client.Client.
From the documentation:
// Using a typed object.
pod := &corev1.Pod{}
// c is a created client.
_ = c.Get(context.Background(), client.ObjectKey{
Namespace: "namespace",
Name: "name",
}, pod)
pod.SetFinalizers(append(pod.GetFinalizers(), "new-finalizer"))
_ = c.Update(context.Background(), pod)
// Using a unstructured object.
u := &unstructured.Unstructured{}
u.SetGroupVersionKind(schema.GroupVersionKind{
Group: "apps",
Kind: "Deployment",
Version: "v1",
})
_ = c.Get(context.Background(), client.ObjectKey{
Namespace: "namespace",
Name: "name",
}, u)
u.SetFinalizers(append(u.GetFinalizers(), "new-finalizer"))
_ = c.Update(context.Background(), u)
You could just as easily swap in SomeCustomResourceKind:
myCR := &v1alpha1.SomeCustomResourceKind{}
// c is a created client.Client
_ = c.Get(context.TODO(), client.ObjectKey{
Namespace: "namespace",
Name: "some-custom-resource", }, myCR)
myCR.MyProperty = "NewValue"
_ = c.Update(context.TODO(), myCR)
You mentioned you're trying to move this functionality from a bash script to a Go controller, so it would be worth checking out the Kubebuilder project, which can scaffold out a controller for you (and any additional APIs you might need). It creates fully functional controllers with the controller-runtime Client and wires up all the reconciliation logic to manage your CRDs.
I am creating a CRD (securitycontextconstraint) as follows:
import(
v1 "github.com/openshift/api/security/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func (c *ClusterClient) CreateSCC() {
name := "ssc-test"
scc := v1.SecurityContextConstraints{
TypeMeta: metav1.TypeMeta{
Kind: "SecurityContextConstraints",
APIVersion: "security.openshift.io/v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: name,
},
RequiredDropCapabilities: []corev1.Capability{"KILL", "MKNOD", "SETUID", "SETGID"},
AllowHostPorts: false,
AllowPrivilegedContainer: false,
AllowHostDirVolumePlugin: false,
AllowHostIPC: false,
ReadOnlyRootFilesystem: false,
DefaultAddCapabilities: nil,
AllowHostPID: false,
AllowHostNetwork: false,
AllowedCapabilities: nil,
Volumes: []v1.FSType{v1.FSTypeConfigMap, v1.FSTypeDownwardAPI, v1.FSTypeEmptyDir, v1.FSTypePersistentVolumeClaim, v1.FSProjected, v1.FSTypeSecret},
SELinuxContext: v1.SELinuxContextStrategyOptions{
Type: v1.SELinuxStrategyMustRunAs,
},
RunAsUser: v1.RunAsUserStrategyOptions{
Type: v1.RunAsUserStrategyMustRunAsRange,
},
SupplementalGroups: v1.SupplementalGroupsStrategyOptions{
Type: v1.SupplementalGroupsStrategyRunAsAny,
},
Users: []string{},
Groups: []string{"system:authenticated"},
}
// To check if scc is already created or not
_, err := c.kubeClient.CoreV1().RESTClient().
Get().
AbsPath("/apis/security.openshift.io/v1/securitycontextconstraints/ssc-test").
DoRaw(context.Background())
// scc is not present in cluster
if err != nil {
//Creating SCC
_, err = c.kubeClient.CoreV1().RESTClient().
Post().
AbsPath("/apis/security.openshift.io/v1").
Resource("securitycontextconstraints").
// VersionedParams(&metav1.CreateOptions{}, scheme.ParameterCodec).
Body(&scc).
DoRaw(context.Background())
if err != nil {
fmt.Printf("Failed to create SecurityContextConstraints: %v\n", err)
// return Failure, err
}
fmt.Printf("Successfully created SecurityContextConstraints %s\n", name)
}
}
How can I deserialize a Kubernetes YAML file into an Go struct? I took a look into the kubectl code, but somehow I get an error for every YAML file:
no kind "Deployment" is registered for version "apps/v1beta1"
This is an MWE:
package main
import (
"fmt"
"k8s.io/client-go/pkg/api"
)
var service = `
apiVersion: apps/v1beta1
kind: Deployment
metadata:
name: my-nginx
spec:
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
`
func main() {
decode := api.Codecs.UniversalDecoder().Decode
//decode := api.Codecs.UniversalDeserializer().Decode
obj, _, err := decode([]byte(service), nil, nil)
if err != nil {
panic(err)
}
fmt.Printf("%#v\n", obj)
}
I am using client version 2.0.0. The glide.yaml looks like this:
package: test/stackoverflow
import:
- package: k8s.io/client-go
version: ^2.0.0
These are the references to kubectl:
https://github.com/kubernetes/kubernetes/blob/43ac38e29e6ecf83e78bc7c5d9f804310b051c95/pkg/kubectl/cmd/apply.go#L637
https://github.com/kubernetes/kubernetes/blob/43ac38e29e6ecf83e78bc7c5d9f804310b051c95/pkg/kubectl/cmd/util/factory_client_access.go#L205-L213
Unfortunately, the docs are very confusing to me, so I have no idea how to tackle this problem.
Edit:
This problem also exists with other resource types:
no kind "Service" is registered for version "v1"
You need to import _ "k8s.io/client-go/pkg/apis/extensions/install" otherwise the schema is empty, see also docs.
The complete working example is:
$ go get -u github.com/golang/dep/cmd/dep
$ dep init
$ go run main.go
With the following main.go:
package main
import (
"fmt"
"k8s.io/client-go/pkg/api"
_ "k8s.io/client-go/pkg/api/install"
_ "k8s.io/client-go/pkg/apis/extensions/install"
)
var deployment = `
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: my-nginx
spec:
replicas: 2
template:
metadata:
labels:
run: my-nginx
spec:
containers:
- name: my-nginx
image: nginx
ports:
- containerPort: 80
`
func main() {
// decode := api.Codecs.UniversalDecoder().Decode
decode := api.Codecs.UniversalDeserializer().Decode
obj, _, err := decode([]byte(deployment), nil, nil)
if err != nil {
fmt.Printf("%#v", err)
}
fmt.Printf("%#v\n", obj)
}
Note that I also imported _ "k8s.io/client-go/pkg/api/install" for you so that you can use objects in v1 such as pods or services.
EDIT: Kudos to my colleague Stefan Schimanski who proposed the initial solution.
I've been using api machinery'sk8s.io/apimachinery/pkg/util/yaml to decode kubernete's deployment and service manifests.
import (
"fmt"
"bytes"
appsv1 "k8s.io/api/apps/v1"
k8Yaml "k8s.io/apimachinery/pkg/util/yaml"
)
...
d := &appsv1.Deployment{}
dec := k8Yaml.NewYAMLOrJSONDecoder(bytes.NewReader([]byte(deploymentManifest)), 1000)
if err := dec.Decode(&d); err != nil {
return nil, err
}
fmt.Printf("%+v", d)
import (
"fmt"
"gopkg.in/yaml.v2"
"log"
//corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/runtime/serializer/json"
"k8s.io/client-go/kubernetes/scheme"
v1alpha1 "github.com/argoproj/argo-cd/pkg/apis/application/v1alpha1"
)
....
func ParseYaml2(yaml []byte) (v1alpha1.Application, error) {
// Create a YAML serializer. JSON is a subset of YAML, so is supported too.
s := json.NewYAMLSerializer(json.DefaultMetaFactory, scheme.Scheme,
scheme.Scheme)
// Decode the YAML to an object.
var app v1alpha1.Application
_, _, err := s.Decode(yaml, nil, &app)
if err != nil {
panic(err)
}
//fmt.Printf("%#v\n", app)
return app, err
}
---
go.mod
// https://github.com/argoproj/argo-cd/issues/4055
replace github.com/argoproj/argo-cd => github.com/argoproj/argo-cd v1.5.5
var yaml2 = []byte(`
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
...
var app v1alpha1.Application
app,err := ParseYaml2(yaml2)
// Types from https://github.com/argoproj/argo-cd/blob/master/pkg/apis/application/v1alpha1/types.go
fmt.Printf("--- t:\n%s\n\n", app.Spec.Source.Path)
fmt.Printf("--- t:\n%s\n\n", app.Spec.Source.Helm.ValueFiles)
----