I'm trying to implement a google cloud function to test Google Logging client library. below is my code
// Package p contains an HTTP Cloud Function.
package loggingclient
import (
"cloud.google.com/go/logging"
"net/http"
"context"
"fmt"
)
// HelloWorld prints the JSON encoded "message" field in the body
// of the request or "Hello, World!" if there isn't one.
func HelloWorld(w http.ResponseWriter, r *http.Request) {
label := map[string]string{"priority": "High"}
var projectName = "my-project-id"
ctx := context.Background()
client, err := logging.NewClient(ctx, projectName)
if err != nil {
fmt.Printf("client not created: %v", err)
}
lg := client.Logger("MY-LOGGER")
lg.Log(logging.Entry{
Payload: "Hello, This is error!!",
Severity: logging.Error,
Labels: label,
})
client.Close()
}
Here, I'm expecting a log entry with a message:"Hello, This is error!!" and with a lable:"priority": "High" and severirty "ERROR"
But actually, when I trigger this Cloud Function, I didn't get any new log entries. Therefore don't client logging libraries work inside cloud functions?, How to resolve this?
Thanks
It works on cloud functions. I have done the exact same thing in a cloud function before. You can use google's official documenation with cloud function logging here
Also ensure that the service account have necessary permissions for logging
https://cloud.google.com/logging/docs/access-control
Related
The golang operator started writing an error.
failed to list v1.Secret: secrets is forbidden: User "system:serviceaccount:operator-*****" cannot list resource "secrets" in API group "" in the namespace "namespace-name"
The error appeared after we enabled restrictions on list secret (set resource Names).
Without restrictions, everything works fine.
I am not familiar with golang, but after looking at the source code, I came to the conclusion that the error occurs in this place
if err := g.client.Get(ctx, client.ObjectKey{Name: tokens.Name, Namespace: g.namespace}, &tokens); err != nil {
return nil, errors.WithMessage(err, "failed to query tokens")
Here is a more complete part of the code
package initgeneration
import (
"context"
"encoding/json"
dynatracev1beta1 "github.com/Dynatrace/dynatrace-operator/src/api/v1beta1"
"github.com/Dynatrace/dynatrace-operator/src/config"
"github.com/pkg/errors"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/labels"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
)
// InitGenerator manages the init secret generation for the user namespaces.
type InitGenerator struct {
client client.Client
apiReader client.Reader
namespace string
canWatchNodes bool
dynakubeQuery kubeobjects.DynakubeQuery
}
...
func (g *InitGenerator) createSecretConfigForDynaKube(ctx context.Context, dynakube *dynatracev1beta1.DynaKube, kubeSystemUID types.UID, hostMonitoringNodes map[string]string) (*standalone.SecretConfig, error) {
var tokens corev1.Secret
if err := g.client.Get(ctx, client.ObjectKey{Name: tokens.Name, Namespace: g.namespace}, &tokens); err != nil {
return nil, errors.WithMessage(err, "failed to query tokens")
The Get method is used in the package "asus.k8s.io/controller-runtime/pkg/client " , when accessing the K8S API, it requests all the secrets and only then imposes a filter.
I need to get around this limitation
How to get secrets, how to access the K8S API through the client package with a restriction on resourceName?
Controller-runtime client ( "sigs.k8s.io/controller-runtime/pkg/client") goes to its cache when receiving a request, and it updates the cache through the watch operation, i.e. it needs access to all secrets
The solution that helped: disable cache for secrets in "Options"
https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/manager#Options
via ClientDisableCacheFor
func (provider operatorManagerProvider) createOptions(namespace string) ctrl.Options {
return ctrl.Options{
Namespace: namespace,
Scheme: scheme.Scheme,
MetricsBindAddress: metricsBindAddress,
Port: operatorManagerPort,
...
ClientDisableCacheFor: []client.Object{
&corev1.Secret{},
},
}
}
I am trying to implement iamcredentials Go API client to generate an Access Token to access some Google APIs via REST API, I am using this code
package main
import (
"context"
"log"
"google.golang.org/api/iamcredentials/v1"
)
func main() {
iamcredentialsService, err := iamcredentials.NewService(context.Background())
if err != nil {
log.Println("error initialize iamcredential Service ", err)
return
}
accessTokenCall := iamcredentialsService.Projects.ServiceAccounts.GenerateAccessToken(
"projects/-/serviceAccounts/some-sa#some-project-id.iam.gserviceaccount.com:generateAccessToken",
&iamcredentials.GenerateAccessTokenRequest{
Scope: []string{
iamcredentials.CloudPlatformScope,
},
},
)
iamResp, err := accessTokenCall.Do()
if err != nil {
log.Println("error generate access token", err)
return
}
log.Println(iamResp)
}
But when I tried to run the above snippet, I got this message
go run main.go
error generate access token googleapi: Error 400: Request contains an invalid argument., badRequest
Is there any way to check which one is causing the above response? I am not sure since there isn't any good example of implementation. Any help would be appreciated, Thanks.
Notes :
I have checked following documentation on this topic https://cloud.google.com/iam/docs/creating-short-lived-service-account-credentials and this https://pkg.go.dev/google.golang.org/api/iamcredentials/v1#pkg-overview
I have already setup the Service account using Service Account Token Creator role on IAM and also enabled the IAM API from the console
Also I have added GOOGLE_APPLICATION_CREDENTIALS to the environment variables as suggested
#DanielFarrell is right, you need to remove the :generateAccessToken at the end. Here the documentation in the code. Don't hesitate to explore it, it's open source ;)
// GenerateAccessToken: Generates an OAuth 2.0 access token for a
// service account.
//
// - name: The resource name of the service account for which the
// credentials are requested, in the following format:
// `projects/-/serviceAccounts/{ACCOUNT_EMAIL_OR_UNIQUEID}`. The `-`
// wildcard character is required; replacing it with a project ID is
// invalid.
func (r *ProjectsServiceAccountsService) GenerateAccessToken(name string, generateaccesstokenrequest *GenerateAccessTokenRequest) *ProjectsServiceAccountsGenerateAccessTokenCall {
c := &ProjectsServiceAccountsGenerateAccessTokenCall{s: r.s, urlParams_: make(gensupport.URLParams)}
c.name = name
c.generateaccesstokenrequest = generateaccesstokenrequest
return c
}
I am building a multilingual SAAS website builder in Golang which is run per client. Each client can have their own website and can translate their website in the desired language.
Since the feature is per client, so I collected an API key from client, which I used to translate their site content.
Here is the code,
V2
package main
import (
"context"
"fmt"
"cloud.google.com/go/translate"
"google.golang.org/api/option"
)
func main() {
translationStrings := []string{"hello"}
ctx := context.Background()
opts := option.WithAPIKey(APIKEY)
c, err := translate.NewClient(ctx, opts)
if err != nil {
fmt.Println(err)
}
defer c.Close()
resp, err := c.Translate(ctx, translationStrings, language.French,
&translate.Options{
Source: language.English,
Format: translate.Text,
})
if err != nil {
fmt.Println(err)
}
fmt.Println(resp)
}
V3
translate "cloud.google.com/go/translate/apiv3"
translatepb "google.golang.org/genproto/googleapis/cloud/translate/v3"
c, err := translate.NewTranslationClient(ctx, opts)
if err != nil {
fmt.Println(err)
}
defer c.Close()
req := &translatepb.TranslateTextRequest{
Contents: translationStrings,
TargetLanguageCode: "sr-Latn",
}
resp, err := c.TranslateText(ctx, req)
The code with V2 works well but the same code with V3 does not work. It gives error:
API keys are not supported for gRPC APIs. Remove the WithAPIKey option from your client-creating call.
As stated in the error, it is asking me to remove WithAPIKey options. But if I remove this then how will I use the api key for each client.
I have chosen to work with V3 apis because the will be translated as a whole so it will be a large request. I have read in the docs that V3 api can work in batches.
So my questions are:
how can I use per client api key structure with api V3?
Is it okay to go with the api V2 for the purpose as stated above?
Cloud Translation API v3 does not currently support API keys. It is recommended that you create a service account for Cloud Translation API v3 requests. For information on creating a service account, see Creating and managing service accounts. Your service account must be added to one of the IAM roles added for Cloud Translation API v3.
I'd like to create Signed URLs to Google Cloud Storage resources from an app deployed using CloudRun.
I set up CloudRun with a custom Service Account with the GCS role following this guide.
My intent was to use V4 Signing to create Signed URLs from CloudRun. There is a guide for this use-case where a file service_account.json is used to generate JWT config. This works for me on localhost when I download the file from google's IAM. I'd like to avoid having this file committed in the repository use the one that I provided in CloudRun UI.
I was hoping that CloudRun injects this service account file to the app container and makes it accessible in GOOGLE_APPLICATION_CREDENTIALS variable but that's not the case.
Do you have a recommendation on how to do this? Thank you.
As you say, Golang Storage Client Libraries require a service account json file to sign urls.
There is currently a feature request open in GitHub for this but you should be able to work this around with this sample that I found here:
import (
"context"
"fmt"
"time"
"cloud.google.com/go/storage"
"cloud.google.com/go/iam/credentials/apiv1"
credentialspb "google.golang.org/genproto/googleapis/iam/credentials/v1"
)
const (
bucketName = "bucket-name"
objectName = "object"
serviceAccount = "[PROJECTNUMBER]-compute#developer.gserviceaccount.com"
)
func main() {
ctx := context.Background()
c, err := credentials.NewIamCredentialsClient(ctx)
if err != nil {
panic(err)
}
opts := &storage.SignedURLOptions{
Method: "GET",
GoogleAccessID: serviceAccount,
SignBytes: func(b []byte) ([]byte, error) {
req := &credentialspb.SignBlobRequest{
Payload: b,
Name: serviceAccount,
}
resp, err := c.SignBlob(ctx, req)
if err != nil {
panic(err)
}
return resp.SignedBlob, err
},
Expires: time.Now().Add(15*time.Minute),
}
u, err := storage.SignedURL(bucketName, objectName, opts)
if err != nil {
panic(err)
}
fmt.Printf("\"%v\"", u)
}
Cloud Run (and other compute platforms) does not inject a service account key file. Instead, they make access_tokens available on the instance metadata service. You can then exchange this access token with a JWT.
However, often times, Google’s client libraries and gcloud works out of the box on GCP’s compute platforms without explicitly needing to authenticate. So if you use the instructions on the page you linked (gcloud or code samples) it should be working out-of-the-box.
I have been looking for an example GAE script in go to get my image that I got from the resulted screenshot of PageSpeed Insights and saved it as json_decode object using Kohana/Cache to Google Cloud Storage (GCS).
The reason of this method is simply because I found this Kohana model is the most convenient way writing files to GCS, although I am seeking also other way like this to write files to GCS using Blobstore to serve them while the Go API Files has been deprecate as documented here.
Here is the form of stored object containing the screenshot image data (base64) which is saved as public in default application bucket with object name images/thumb/mythumb.jpg:
stdClass Object
(
[screenshot] => stdClass Object
(
[data] => _9j_4AAQSkZJRgABAQAAAQABAAD_...= // base64 data
[height] => 240
[mime_type] => image/jpeg
[width] => 320
)
[otherdata] => Array
(
[..] => ..
[..] => ..
)
)
I want to get this image that set as public using my customized url as below that to be proceed through go module and also I need it to be expired in a certain time because I have managed to update the image content itself regularly:
http://myappId.appspot.com/image/thumb/mythumb.jpg
I have set in disptach.yaml to send all image request to my go module as below:
- url: "*/images/*"
module: go
and set the handler in go.yaml to proceed the image request as below:
handlers:
- url: /images/thumb/.*
script: _go_app
- url: /images
static_dir: images
Using this directive I have got that all /images/ request (other than /images/thumb/ request) serve images from the static directory and that /images/thumb/mythumb.jpg goes to the module application.
So left what code I have to use (see ????) in my application file named thumb.go as below:
package thumb
import(
//what to import
????
????
)
const (
googleAccessID = "<serviceAccountEmail>#developer.gserviceaccount.com"
serviceAccountPEMFilename = "YOUR_SERVICE_ACCOUNT_KEY.pem"
bucket = "myappId.appspot.com"
)
var (
expiration = time.Now().Add(time.Second * 60) //expire in 60 seconds
)
func init() {
http.HandleFunc("/images/thumb/", handleThumb)
}
func handleThumb(w http.ResponseWriter, r *http.Request) {
ctx := cloud.NewContext(appengine.AppID(c), hc)
???? //what code to get the string of 'mythumb.jpg' from url
???? //what code to get the image stored data from GCS
???? //what code to encoce base64 data
w.Header().Set("Content-Type", "image/jpeg;")
fmt.Fprintf(w, "%v", mythumb.jpg)
}
I have taken many codes from some examples like this, this or this but could not get one works so far. I have also tried a sample from this which is almost close to my case but also found no luck.
So in generally t was mainly due to lack on what are the correct code to be put on the line that I marked by ???? as well the relevant library or path to be imported. I have also checked the GCS permission if something have been missing as described here and here.
I shall thank you much for your help and advise.
From what I've read in your description, it seems that the only relevant parts are the ???? lines in the actual Go code. Let me know if that's not the case.
First ????: "what code to get the string of 'mythumb.jpg' from url"?
From reading the code, you're looking to extract mythumb.jpg from a url like http://localhost/images/thumb/mythumb.jpg. A working example is available at the Writing Web Applications tutorial:
package main
import (
"fmt"
"net/http"
)
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hi there, I love %s!", r.URL.Path[1:])
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
}
Such that
http://localhost:8080/monkeys
Prints
Hi there, I love monkeys!
Second ????: "what code to get the image stored data from GCS"?
The API method you're probably looking to use is storage.objects.get.
You did link to one of the JSON API Go Examples for Google Cloud Storage, which is a good general reference, but is not related to the problem you're trying to solve. That particular example is put together for Client-side applications (hence the redirectURL = "urn:ietf:wg:oauth:2.0:oob" line). Additionally, this sample uses deprecated/out-of-date oauth2 and storage packages.
One of the cleanest (and non-deprecated) ways to do this for an application which wants to access its own buckets on behalf of itself would be to use the golang/oauth2 and Google APIs Client Library for Go packages.
An example of how to authenticate with JSON Web Token auth with the golang/oauth2 package is available in the repo:
func ExampleJWTConfig() {
conf := &jwt.Config{
Email: "xxx#developer.com",
// The contents of your RSA private key or your PEM file
// that contains a private key.
// If you have a p12 file instead, you
// can use `openssl` to export the private key into a pem file.
//
// $ openssl pkcs12 -in key.p12 -out key.pem -nodes
//
// It only supports PEM containers with no passphrase.
PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
Subject: "user#example.com",
TokenURL: "https://provider.com/o/oauth2/token",
}
// Initiate an http.Client, the following GET request will be
// authorized and authenticated on the behalf of user#example.com.
client := conf.Client(oauth2.NoContext)
client.Get("...")
}
Next, instead of using the oauth2 client directly, use that client with the Google APIs Client Library for Go mentioned earlier:
service, err := storage.New(client)
if err != nil {
fatalf(service, "Failed to create service %v", err)
}
Notice the similarity to the out-of-date JSON API Go Examples?
In your handler, you'll want to go out and get the related object using func ObjectsService.Get. Assuming that you know the name of the object and bucket, that is.
Straight from the previous example, you can use code similar to what's below to retrieve the download link:
if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
} else {
fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
}
Then, fetch the file, or do whatever you want with it. Full example:
import (
"golang.org/x/oauth2"
"golang.org/x/oauth2/jwt"
"google.golang.org/api/storage/v1"
"fmt"
)
...
const (
bucketName = "YOUR_BUCKET_NAME"
objectName = "mythumb.jpg"
)
func main() {
conf := &jwt.Config{
Email: "xxx#developer.com",
PrivateKey: []byte("-----BEGIN RSA PRIVATE KEY-----..."),
Subject: "user#example.com",
TokenURL: "https://provider.com/o/oauth2/token",
}
client := conf.Client(oauth2.NoContext)
service, err := storage.New(client)
if err != nil {
fatalf(service, "Failed to create service %v", err)
}
if res, err := service.Objects.Get(bucketName, objectName).Do(); err == nil {
fmt.Printf("The media download link for %v/%v is %v.\n\n", bucketName, res.Name, res.MediaLink)
} else {
fatalf(service, "Failed to get %s/%s: %s.", bucketName, objectName, err)
}
// Go fetch the file, etc.
}
Third ????: "what code to encoce base64 data"?
Pretty simple with the encoding/base64 package. SO simple, that they've included an example:
package main
import (
"encoding/base64"
"fmt"
)
func main() {
data := []byte("any + old & data")
str := base64.StdEncoding.EncodeToString(data)
fmt.Println(str)
}
Hope that helps.