GCP - Remove Obsolete Buckets
Remove Obsolete Buckets in GCP in golang
How can I implement this thing in golang ?
You may try to delete the bucket through BucketHandle.Delete
import (
"context"
"cloud.google.com/go/storage"
)
func main() {
ctx := context.Background()
client, err := storage.NewClient(ctx)
if err != nil {
// TODO: handle error.
}
if err := client.Bucket("my-bucket").Delete(ctx); err != nil {
// TODO: handle error.
}
}
Related
I am going through the documentation of cloudresourcemanager package and trying to build a simple example to list the folders of my GCP project.
The following example however fails
package main
import (
"context"
"fmt"
"log"
cloudresourcemanager "google.golang.org/api/cloudresourcemanager/v2"
)
func main() {
ctx := context.Background()
svc, err := cloudresourcemanager.NewService(ctx)
if err != nil {
log.Fatal(err)
}
foldersService := cloudresourcemanager.NewFoldersService(svc)
fmt.Println(foldersService)
foldersListCall := foldersService.List()
resp, err := foldersListCall.Do()
if err != nil {
fmt.Println("Here")
log.Fatal(err)
}
for _, fld := range resp.Folders {
fmt.Println(fld.Name)
}
}
It fails in
resp, err := foldersListCall.Do()
and the error is
googleapi: Error 400: Request contains an invalid argument., badRequest
I have the following environment variables set
GOOGLE_CLOUD_PROJECT=my-project-id
GOOGLE_APPLICATION_CREDENTIALS=/path/to/application_default_credentials.json
and gcloud cli works fine.
Any suggestions what I might be missing?
The error message is not helpful at all...
The problem is I was not setting the Parent parameter in the request, i.e. the organization (switching to v3 helped a bit)
package main
import (
"context"
"fmt"
"log"
cloudresourcemanager "google.golang.org/api/cloudresourcemanager/v3"
)
func main() {
ctx := context.Background()
svc, err := cloudresourcemanager.NewService(ctx)
if err != nil {
log.Fatal(err)
}
foldersService := cloudresourcemanager.NewFoldersService(svc)
foldersListCall := foldersService.List()
foldersListCall.Parent("organizations/12345678910")
resp, err := foldersListCall.Do()
if err != nil {
log.Fatal(err)
}
for _, fld := range resp.Folders {
fmt.Println(fld.DisplayName)
}
}
I am trying to get all documents from a Firestore database and things were working fine.
But then I decided to make the context and client variable global, so that I won't have to deal with passing them as parameters everytime.
Things broke after that.
The error I get is:
panic: runtime error: invalid memory address or nil pointer dereference
and according to the stack trace, it occurs when I try to:
client.Collection("dummy").Documents(ctx)
What can I do to resolve this?
And how can I efficiently work with global variables in my case?
My code for reference:
package main
import (
"context"
"fmt"
"log"
"cloud.google.com/go/firestore"
firebase "firebase.google.com/go"
"google.golang.org/api/iterator"
"google.golang.org/api/option"
)
var (
ctx context.Context
client *firestore.Client
)
func init() {
ctx := context.Background()
keyFile := option.WithCredentialsFile("serviceAccountKey.json")
app, err := firebase.NewApp(ctx, nil, keyFile)
if err != nil {
log.Fatalln(err)
}
client, err = app.Firestore(ctx)
if err != nil {
log.Fatalln(err)
}
fmt.Println("Connection to Firebase Established!")
}
func getDocuments(collectionName string) {
iter := client.Collection("dummy").Documents(ctx)
for {
doc, err := iter.Next()
if err == iterator.Done {
break
}
if err != nil {
log.Fatalf("Failed to iterate: %v", err)
}
fmt.Println(doc.Data()["question"])
}
}
func main() {
getDocuments("dummy")
defer client.Close()
}
You get that error because you never assign anything to the package level ctx variable, so it remains nil.
Inside init() you use short variable declaration which creates a local variable:
ctx := context.Background()
If you change to to simple assignment, it will assign a value to the existing, package level ctx variable:
ctx = context.Background()
Although using "global" variables to store something that's not global is bad practice. You should just pass ctx where it's needed.
I'm trying to learn Go by managing Google Cloud Platform. I didn't understand how to use related functions about Compute. The goal is listing instances with some go code.
This is https://godoc.org/google.golang.org/api/compute/v1#InstancesService.List the related function.
func (r *InstancesService) List(project string, zone string) *InstancesListCall
There are two structs, InstancesService and InstancesListCall
As far as i understand i should define these structs but it's not clear the things should be defined in the structs. I've searched for examples but many of them using rest calls instead of golang api. Have any idea how to list instances with go?
i had to write something like this today and googling for examples turned up surprisingly little. i've written up what i learned below, however, i'm quite new to golang so maybe smarter people can suggest improvements.
my work in progress is at: https://github.com/grenade/rubberneck
if you want to run your go program from a development pc that is not on the google compute platform:
set up the gcloud cli to run on your pc (instructions: https://cloud.google.com/sdk/gcloud)
create a service account for your go application to run under (instructions: https://cloud.google.com/docs/authentication/production#creating_a_service_account)
grant permissions to the service account (use the same instructions link above)
create a local key file containing your new service account credentials (use the same instructions link above)
set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of your local key file
write your go application. something like this:
package main
import (
"golang.org/x/net/context"
"google.golang.org/api/compute/v1"
"golang.org/x/oauth2/google"
"fmt"
"strings"
)
func main() {
projects := [...]string{
"my-project-one",
"my-project-two",
}
filters := [...]string{
"status = RUNNING",
"name != my-uninteresting-instance-one",
"name != my-uninteresting-instance-two",
}
ctx := context.Background()
client, err := google.DefaultClient(ctx,compute.ComputeScope)
if err != nil {
fmt.Println(err)
}
computeService, err := compute.New(client)
for _, project := range projects {
zoneListCall := computeService.Zones.List(project)
zoneList, err := zoneListCall.Do()
if err != nil {
fmt.Println("Error", err)
} else {
for _, zone := range zoneList.Items {
instanceListCall := computeService.Instances.List(project, zone.Name)
instanceListCall.Filter(strings.Join(filters[:], " "))
instanceList, err := instanceListCall.Do()
if err != nil {
fmt.Println("Error", err)
} else {
for _, instance := range instanceList.Items {
if workerType, isWorker := instance.Labels["worker-type"]; isWorker {
m := strings.Split(instance.MachineType, "/")
fmt.Printf("cloud: gcp, zone: %v, name: %v, instance id: %v, machine type: %v, worker type: %v, launch time: %v\n",
zone.Name,
instance.Name,
instance.Id,
m[len(m)-1],
workerType,
instance.CreationTimestamp)
}
}
}
}
}
}
}
You can also use Aggregated List which will search every zone for you. This saves you having to do nested loops or figuring out what the zones are.
https://pkg.go.dev/cloud.google.com/go/compute/apiv1#InstancesClient.AggregatedList
The below assumes you have logged into gcloud and set your ADC.
$ gcloud init
$ gcloud auth application-default login
Using a service account key is also possible but not demonstrated below.
package main
import (
"context"
"fmt"
"log"
compute "cloud.google.com/go/compute/apiv1"
"google.golang.org/api/iterator"
protobuf "google.golang.org/genproto/googleapis/cloud/compute/v1"
)
func main() {
ctx := context.Background()
c, err := compute.NewInstancesRESTClient(ctx)
if err != nil {
log.Fatalln(err)
}
defer c.Close()
project := "my-project"
req := &protobuf.AggregatedListInstancesRequest{
Project: project,
}
it := c.client.AggregatedList(ctx, req)
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
log.Fatalln(err)
}
fmt.Println(resp)
}
}
You can also use AggregatedList and cloud cred in golang and can retrieve all instance information
package main
import (
"context"
"flag"
"fmt"
"log"
"github.com/binxio/gcloudconfig"
"golang.org/x/oauth2/google"
"google.golang.org/api/compute/v1"
"google.golang.org/api/option"
)
func main() {
//gcp session
var credentials *google.Credentials
name := flag.String("configuration", "", "`kunets` of the configuration to use")
project := flag.String("project", "", "`kunets` of the project to query")
flag.Parse()
credentials, _ = gcloudconfig.GetCredentials(*name)
if project == nil || *project == "" {
project = &credentials.ProjectID
}
if *project == "" {
log.Printf("%v", credentials)
log.Fatal("no -project specified")
}
computeService, err := compute.NewService(context.Background(), option.WithCredentials(credentials))
if err != nil {
log.Fatal(err)
}
token := ""
var list *compute.InstanceAggregatedList
if list, err = computeService.Instances.AggregatedList(*project).PageToken(token).Do(); err != nil {
log.Fatal(err)
}
for _, instances := range list.Items {
for _, instance := range instances.Instances {
EXTERNAL_IP := instance.NetworkInterfaces[0].AccessConfigs[0].NatIP
fmt.Printf("%s \n", EXTERNAL_IP)
INTERNAL_IP := instance.NetworkInterfaces[0].NetworkIP
fmt.Printf("%s \n", INTERNAL_IP)
fmt.Printf("%s \n", instance.Name)
}
}
}
I am trying to use Elasticsearch for GO with this well-known repo
However, when I am trying to create an index (docs, and also given as an example here):
// Define an elastic client
client, err := elastic.NewClient(elastic.SetURL("host1"))
if err != nil {
client, err := elastic.NewClient(elastic.SetURL("host2"))
if err != nil {
fmt.Println("Error when connecting Elasticsearch host");
}
}
// Create an index
_, err = client.CreateIndex("events").Do()
if err != nil {
fmt.Println("Error when creating Elasticsearch index");
panic(err)
}
I got the following error, which I do not understand:
not enough arguments in call to client.CreateIndex("events").Do
Why is that? What do I miss here?
The IndicesCreateService.Do() function expects a context.Context to be passed.
So, you need to import "golang.org/x/net/context" and then change your call to this:
import (
... your other imports...
"golang.org/x/net/context"
)
...
_, err := client.CreateIndex("events").Do(context.TODO())
^
|
add this
You can also check the indices_create_test.go test case in order to see how it's done.
I'm struggling with the Kubernetes Go library. The docs--at least the ones I found--appear out-of-date with the library itself. The example provided does not build because of issues with the imports. I'm just trying to do something simple: get a Service object by name and print some attributes (like nodePort). I just need a simple example of library usage to get me going.
I could easily do this using the RESTful API but that feels like re-inventing the wheel.
So after a little experimentation and a hint from the k8s Slack channel, I have this example. Perhaps someone can update the example with a proper import path.
package main
import (
"fmt"
"log"
"github.com/kubernetes/kubernetes/pkg/api"
client "github.com/kubernetes/kubernetes/pkg/client/unversioned"
)
func main() {
config := client.Config{
Host: "http://my-kube-api-server.me:8080",
}
c, err := client.New(&config)
if err != nil {
log.Fatalln("Can't connect to Kubernetes API:", err)
}
s, err := c.Services(api.NamespaceDefault).Get("some-service-name")
if err != nil {
log.Fatalln("Can't get service:", err)
}
fmt.Println("Name:", s.Name)
for p, _ := range s.Spec.Ports {
fmt.Println("Port:", s.Spec.Ports[p].Port)
fmt.Println("NodePort:", s.Spec.Ports[p].NodePort)
}
}
Here's how to do it with the latest Go client.
If you're inside the k8s cluster:
package main
import (
"fmt"
"k8s.io/client-go/1.5/kubernetes"
"k8s.io/client-go/1.5/pkg/api/v1"
"k8s.io/client-go/1.5/rest"
)
func main() {
config, err = rest.InClusterConfig()
if err != nil {
return nil, err
}
c, err := kubernetes.NewForConfig(config)
if err != nil {
return nil, err
}
// Get Pod by name
pod, err := c.Pods(v1.NamespaceDefault).Get("my-pod")
if err != nil {
fmt.Println(err)
return
}
// Print its creation time
fmt.Println(pod.GetCreationTimestamp())
}
And if you're outside of the cluster:
package main
import (
"fmt"
"k8s.io/client-go/1.5/kubernetes"
"k8s.io/client-go/1.5/pkg/api/v1"
"k8s.io/client-go/1.5/tools/clientcmd"
)
func main() {
config, err := clientcmd.BuildConfigFromFlags("", <kube-config-path>)
if err != nil {
return nil, err
}
c, err := kubernetes.NewForConfig(config)
if err != nil {
return nil, err
}
// Get Pod by name
pod, err := c.Pods(v1.NamespaceDefault).Get("my-pod")
if err != nil {
fmt.Println(err)
return
}
// Print its creation time
fmt.Println(pod.GetCreationTimestamp())
}
I have gone into more detail on this in a blog post.
With kubernetes go client, it could be done this way:
package main
import (
"flag"
"fmt"
"k8s.io/client-go/kubernetes"
"k8s.io/client-go/pkg/api/v1"
"k8s.io/client-go/tools/clientcmd"
)
var (
kubeconfig = flag.String("kubeconfig", "./config", "absolute path to the kubeconfig file")
)
func main() {
flag.Parse()
// uses the current context in kubeconfig
config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
if err != nil {
panic(err.Error())
}
// creates the clientset
clientset, err := kubernetes.NewForConfig(config)
if err != nil {
panic(err.Error())
}
services, err := clientset.Core().Services("").List(v1.ListOptions{})
if err != nil {
panic(err.Error())
}
fmt.Printf("There are %d pods in the cluster\n", len(services.Items))
for _, s := range services.Items {
for p, _ := range s.Spec.Ports {
fmt.Println("Port:", s.Spec.Ports[p].Port)
fmt.Println("NodePort:", s.Spec.Ports[p].NodePort)
}
}
}