I am very new at Golang AWS SDK V2, I had similar code work without AWS results, but this one is getting me issues since the types are different. Also, I have search and none of the examples is with the code pipeline aws-sdk-v2 with the type of JSON I have to unmarshal.
I hope some of you can help me.
— This below is main.go I have the structs in another file called un-marshal.go that I created with the result of getting the same output I need from awscli tool and passing it by https://mholt.github.io/json-to-go/
package main
import (
"context"
"fmt"
"log"
"encoding/json"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/codepipeline"
)
func main() {
pipeline_name := "frontend"
// Load the Shared AWS Configuration (~/.aws/config)
cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("eu-central-1"))
if err != nil {
log.Fatal(err)
}
if err != nil {
log.Fatalf("unable to load SDK config, %v", err)
}
client := codepipeline.NewFromConfig(cfg)
pipeJson, err := client.GetPipelineState(context.TODO(), &codepipeline.GetPipelineStateInput{
Name: &pipeline_name,
})
if err != nil {
log.Println("Error getting Pipeline")
}
var cookie PipeLineResult
json.Unmarshal(pipeJson, &cookie)
fmt.Println("The name of the pipeline is: %s",cookie.PipelineName)
}
The error I am getting is:
/main.go:39:17: cannot use pipeJson (variable of type *codepipeline.GetPipelineStateOutput) as type []byte in argument to json.Unmarshal
Here I am lost because it is a new type, and not sure if I should convert, how to convert or work with the native type etc.
Thanks in advance.
Related
I am trying to run a very simple operation just to list the clusters in a project, using the google cloud sdk for go
package main
import (
"context"
"flag"
"fmt"
"log"
"os"
"google.golang.org/api/container/v1"
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp" // register GCP auth provider
)
var fProjectId = flag.String("projectId", "", "specify a project id to examine")
func main() {
flag.Parse()
if *fProjectId == "" {
log.Fatal("must specific -projectId")
}
ctx := context.TODO()
svc, err := container.NewService(ctx)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
// Ask Google for a list of all kube clusters in the given project.
_, err = svc.Projects.Zones.Clusters.List(*fProjectId, "-").Context(ctx).Do()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
}
the code fails as follows
▶ go run main.go --projectId my-project-id
Get "https://container.googleapis.com/v1/projects/my-project-id/zones/-/clusters?alt=json&prettyPrint=false": oauth2: cannot fetch token: 400 Bad Request
Response: {
"error": "invalid_grant",
"error_description": "reauth related error (rapt_required)",
"error_uri": "https://support.google.com/a/answer/9368756",
"error_subtype": "rapt_required"
}
exit status 1
However the command
gcloud container clusters list
succeeds?
What might be the issue here?
The answer in the link is not very informative.
EDIT: The problem was solved once I run
gcloud auth application-default login
Why is this needed?
I've recently shifted from python to golang. I had been using python to work with GCP.
I used to pass in the scopes and mention the discovery client I wanted to create like this :
def get_client(scopes, api, version="v1"):
service_account_json = os.environ.get("SERVICE_ACCOUNT_KEY_JSON", None)
if service_account_json is None:
sys.exit("Exiting !!! No SSH_KEY_SERVICE_ACCOUNT env var found.")
credentials = service_account.Credentials.from_service_account_info(
json.loads(b64decode(service_account_json)), scopes=scopes
)
return discovery.build(api, version, credentials=credentials, cache_discovery=False)
And this would create my desired discovery client, whether it be compute engine service or sqladmin
However in go I don't seem to find this.
I found this : https://pkg.go.dev/google.golang.org/api/discovery/v1
For any client that I want to create I would've to import that and then create that, like this :
https://cloud.google.com/resource-manager/reference/rest/v1/projects/list#examples
package main
import (
"fmt"
"log"
"golang.org/x/net/context"
"golang.org/x/oauth2/google"
"google.golang.org/api/cloudresourcemanager/v1"
)
func main() {
ctx := context.Background()
c, err := google.DefaultClient(ctx, cloudresourcemanager.CloudPlatformScope)
if err != nil {
log.Fatal(err)
}
cloudresourcemanagerService, err := cloudresourcemanager.New(c)
if err != nil {
log.Fatal(err)
}
req := cloudresourcemanagerService.Projects.List()
if err := req.Pages(ctx, func(page *cloudresourcemanager.ListProjectsResponse) error {
for _, project := range page.Projects {
// TODO: Change code below to process each `project` resource:
fmt.Printf("%#v\n", project)
}
return nil
}); err != nil {
log.Fatal(err)
}
}
So I've to import each client library to get the client for that.
"google.golang.org/api/cloudresourcemanager/v1"
There's no dynamic creation of it.
Is it even possible, cause go is strict type checking 🤔
Thanks.
No, this is not possible with the Golang Google Cloud library.
You've nailed the point on the strict type checking, as it would definitely defeat the benefits of compile time type checking. It would also be a bad Golang practice to return different objects with different signatures, as we don't do duck typing and instead we rely on interface contracts.
Golang is boring and verbose, and it's like that by design :)
The kubernetes go client has tons of methods and I can't find how I can get the current CPU & RAM usage of a specific (or all pods).
Can someone tell me what methods I need to call to get the current usage for pods & nodes?
My NodeList:
nodes, err := clientset.CoreV1().Nodes().List(metav1.ListOptions{})
Kubernetes Go Client: https://github.com/kubernetes/client-go
Metrics package: https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/metrics
As far as I got the metrics server implements the Kubernetes metrics package in order to fetch the resource usage from pods and nodes, but I couldn't figure out where & how they do it: https://github.com/kubernetes-incubator/metrics-server
It is correct that go-client does not have support for metrics type, but in the metrics package there is a pregenerated client that can be used for fetching metrics objects and assign them right away to the appropriate structure. The only thing you need to do first is to generate a config and pass it to metrics client. So a simple client for metrics would look like this:
package main
import (
"k8s.io/client-go/tools/clientcmd"
metrics "k8s.io/metrics/pkg/client/clientset/versioned"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
func main() {
var kubeconfig, master string //empty, assuming inClusterConfig
config, err := clientcmd.BuildConfigFromFlags(master, kubeconfig)
if err != nil{
panic(err)
}
mc, err := metrics.NewForConfig(config)
if err != nil {
panic(err)
}
mc.MetricsV1beta1().NodeMetricses().Get("your node name", metav1.GetOptions{})
mc.MetricsV1beta1().NodeMetricses().List(metav1.ListOptions{})
mc.MetricsV1beta1().PodMetricses(metav1.NamespaceAll).List(metav1.ListOptions{})
mc.MetricsV1beta1().PodMetricses(metav1.NamespaceAll).Get("your pod name", metav1.GetOptions{})
}
Each of the above methods from metric client returns an appropriate structure (you can check those here) and an error (if any) which you should process according to your requirements.
here is an example.
package main
import (
"fmt"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/tools/clientcmd"
metrics "k8s.io/metrics/pkg/client/clientset/versioned"
)
func main() {
var kubeconfig, master string //empty, assuming inClusterConfig
config, err := clientcmd.BuildConfigFromFlags(master, kubeconfig)
if err != nil {
panic(err)
}
mc, err := metrics.NewForConfig(config)
if err != nil {
panic(err)
}
podMetrics, err := mc.MetricsV1beta1().PodMetricses(metav1.NamespaceAll).List(metav1.ListOptions{})
if err != nil {
fmt.Println("Error:", err)
return
}
for _, podMetric := range podMetrics.Items {
podContainers := podMetric.Containers
for _, container := range podContainers {
cpuQuantity, ok := container.Usage.Cpu().AsInt64()
memQuantity, ok := container.Usage.Memory().AsInt64()
if !ok {
return
}
msg := fmt.Sprintf("Container Name: %s \n CPU usage: %d \n Memory usage: %d", container.Name, cpuQuantity, memQuantity)
fmt.Println(msg)
}
}
}
The API you're looking for in new versions of Kubernetes (tested on mine as of 1.10.7) is the metrics.k8s.io/v1beta1 API route.
You can see it locally if you run a kubectl proxy and check http://localhost:8001/apis/metrics.k8s.io/v1beta1/pods and /nodes on your localhost.
I see where your confusion is though. At the time of writing, it does not look like the metrics/v1beta1 has a generated typed package (https://godoc.org/k8s.io/client-go/kubernetes/typed), and doesn't appear in the kubernetes.ClientSet object.
You can hit all available endpoints directly though the rest.RestClient object, and just specify metrics/v1beta1 as the versionedAPIPath, which will be more work and less convenient than the nicely wrapped ClientSet, but I'm not sure how long it'll take before that API shows up in that interface.
I am using the atom IDE, and for some reason whenever I add this to my imports:
"github.com/nlopes/slack"
And save the file, it removes the import. So I'm not sure why but it isn't finding the InteractionCallback type in the library?
I copied this code from the example:
func unmarshalSuggestionCallback(j string) (*InteractionCallback, error) {
callback := &InteractionCallback{}
if err := json.Unmarshal([]byte(j), &callback); err != nil {
return nil, err
}
return callback, nil
}
I am getting this error:
undefined: InteractionCallback
How can I tell if my library I just downloaded has the type defined? Or am I referencing the type incorrectly?
Please use this command in your terminal:
go get -u github.com/nlopes/slack
After that try to run this code:
package main
import (
"encoding/json"
"fmt"
"github.com/nlopes/slack"
)
func unmarshalSuggestionCallback(j string) (*slack.InteractionCallback, error) {
callback := &slack.InteractionCallback{}
if err := json.Unmarshal([]byte(j), &callback); err != nil {
return nil, err
}
return callback, nil
}
func main() {
callback,_:=unmarshalSuggestionCallback(`{"type":"callback"}`)
fmt.Println(callback.Type)
}
Everything should work fine, I have checked in my PC
You need to specify from which package InteractionCallback comes from, in your case its slack package - slack.InteractionCallback
I want to access metrics from kubernetes with golang. Something like cpu and memory per node as well as the same for pods and/or namespaces.
I am kind of lost here because the documentation is not as clear as it could be.
I have learned that there is heapster (which is deprecated according to the github repo). There is also metric server and a rest api.
Where can I find some examples to get started? I do not want to install another app, package or service in kubernetes. I'd like to get the information as native as possible.
What is the preferred way to access these information with client-go and golang?
There's a much better API for this: https://github.com/kubernetes/metrics. Using this, you don't have to create the data structs or handle row byte slices.
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
metricsv "k8s.io/metrics/pkg/client/clientset/versioned"
...
)
...
clientset, err := metricsv.NewForConfig(config)
podMetricsList, err := clientset.MetricsV1beta1().PodMetricses("").List(metav1.ListOptions{})
Here's an example of using the REST API to query node metrics and return a []byte in JSON format. Replace "nodes" with "pods" to get pod/container metrics.
data, err := clientset.RESTClient().Get().AbsPath("apis/metrics.k8s.io/v1beta1/nodes").DoRaw()
As explained in the question, the documentations are not clear for a beginner. Even go-client examples retrieve the data, I wanted to get Type support.
As it explained by above answer, you can get the data in []byte in JSON format. This is how I did it.
package main
import (
"encoding/json"
"fmt"
"time"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/rest"
)
// PodMetricsList : PodMetricsList
type PodMetricsList struct {
Kind string `json:"kind"`
APIVersion string `json:"apiVersion"`
Metadata struct {
SelfLink string `json:"selfLink"`
} `json:"metadata"`
Items []struct {
Metadata struct {
Name string `json:"name"`
Namespace string `json:"namespace"`
SelfLink string `json:"selfLink"`
CreationTimestamp time.Time `json:"creationTimestamp"`
} `json:"metadata"`
Timestamp time.Time `json:"timestamp"`
Window string `json:"window"`
Containers []struct {
Name string `json:"name"`
Usage struct {
CPU string `json:"cpu"`
Memory string `json:"memory"`
} `json:"usage"`
} `json:"containers"`
} `json:"items"`
}
func getMetrics(clientset *kubernetes.Clientset, pods *PodMetricsList) error {
data, err := clientset.RESTClient().Get().AbsPath("apis/metrics.k8s.io/v1beta1/pods").DoRaw()
if err != nil {
return err
}
err = json.Unmarshal(data, &pods)
return err
}
func main() {
// creates the in-cluster config
// https://github.com/kubernetes/client-go/tree/master/examples#configuration
config, err := rest.InClusterConfig()
if err != nil {
panic(err.Error())
}
// creates the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
var pods PodMetricsList
err = getMetrics(clientset, &pods)
if err != nil {
panic(err.Error())
}
for _, m := range pods.Items {
fmt.Println(m.Metadata.Name, m.Metadata.Namespace, m.Timestamp.String())
}
}
Install following Go packages: go get -u k8s.io/client-go/kubernetes k8s.io/client-go/rest
You can use following endpoints to retrieve the data as you want;
Nodes: apis/metrics.k8s.io/v1beta1/nodes
Pods: apis/metrics.k8s.io/v1beta1/pods
Pods of default namespace: apis/metrics.k8s.io/v1beta1/namespaces/default/pods
Specific Pod: /apis/metrics.k8s.io/v1beta1/namespaces/default/pods/<POD-NAME>
NOTE: You may need to change the Type before json.Unmarshal. You can define the Type only for the field which you are interested with.
From the latest version of client-go you have to add a Context to DoRaw(). For example use context.TODO() importing the context library.