google cloud functions default environment variables not set - go

Are there any conditions to the default environment variables being set on google cloud function?
I have the following code:
func init() {
projectID := os.Getenv("GCP_PROJECT")
log.Printf("projectID: %s\n", projectID)
functionName := os.Getenv("FUNCTION_NAME")
log.Printf("functoinName: %s\n", functionName)
region := os.Getenv("FUNCTION_REGION")
log.Printf("region: %s\n", region)
}
and the values are empty.
Even if I do:
func GameUpdate(ctx context.Context, e FirestoreEvent) error {
functionName := os.Getenv("FUNCTION_NAME")
log.Printf("functoinName: %s\n", functionName)
}
They are still empty.
According to documentation, I would expect them to be set and available. But they are not :|
EDIT:
I am using go 1.13 as runtime and as Armatorix mentioned, these env variables are not available in that runtime...
Why I needed them was to write a wrapper for cloud.google.com/go/logging to be able to tag the severity of the logs.
I ended up prepending my stdout logs with [INFO]/[ERROR], and creating a tag from it \[([A-Z]+)\].*. Bonus is that I don't have to do a network call in my function to ship the logs.
Still disappointing that these environment variables are not available.

So I've read the same documentation.
Here you've got the info that it works like this with go1.11 (And it works, I tested it out).
BUT for go1.13 these are not set. You can still do it manually.
Also I've checked which envs are set on 1.13 version.
From os.Envrion()
PATH=/layers/google.go.build/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
FUNCTION_SIGNATURE_TYPE=http
DEBIAN_FRONTEND=noninteractive
HOME=/root
K_REVISION=9
FUNCTION_TARGET=HelloWorld
PORT=8080
GOROOT=/usr/local/go/
CGO_ENABLED=1
PWD=/srv
K_SERVICE=function-1
So the env that you probably want to use is K_SERVICE

I have created a Feature Request on your behalf, in order for Cloud Functions Engineering team to implement the automatic set of these Environment Variables to the newer Runtime Versions, such as Node.js 10 and Go1.13.
You may "star" the issue so that it gets visibility and also include yourself in the "CC" section, in order to receive further updates posted on this thread.
I hope this helps.

I created a library for that very purpose:
github.com/ncruces/go-gcf/logging
But you're right, on the Go 1.13 runtime, those environment variables are missing. On the migration guide they suggest setting them when you deploy.
Later I found that the recommended way of doing this is to use structured logging.
// Structured logging can be used to set severity levels.
// See https://cloud.google.com/logging/docs/structured-logging.
fmt.Println(`{"message": "This has ERROR severity", "severity": "error"}`)
So now, I'm in the process of "deprecating" my library, and creating a new one, with a simpler approach:
github.com/ncruces/go-gcp/glog
This is simple enough that a library isn't really required, but it helps to correctly JSON escape message.

Related

Configuring OTLP exporter through environment variables

Currently I am trying to configure my OTLP exporter using environment variables. This is supposed to be possible as per the official docs.
In particular, I want to focus on the OTEL_EXPORTER_OTLP_ENDPOINT one, which is allowed for the OTLPtrace exporter. According to the comments in their code, the environment variable takes precedence over any other value set in the code.
I wrote a very basic HTTP application in Go, which is instrumented with OpenTelemetry. When I specify the exporter endpoint explicitly in the code like:
exporter, err := otlptrace.New(
context.Background(),
otlptracegrpc.NewClient(
otlptracegrpc.WithInsecure(),
otlptracegrpc.WithEndpoint("My Endpoint"),
),
)
The instrumentation works just fine like that. However if I remove the otlptracegrpc.NewClient configuration from the code, it does not pick up the values set in the environment, which are set like:
OTEL_EXPORTER_OTLP_ENDPOINT="my endpoint"
So when I run this application in my debugger I can see that the exporter client has an empty value as the endpoint, yet I can pick them up within my program as:
exporterEndpoint := os.Getenv("OTEL_EXPORTER_OTLP_ENDPOINT")
This I interpret as the variables being actually present at the time the code is being executed, which was my main fear.
Why is this? Am I missing something here? Should I populate the environment variable differently (I see there are "options" for the environment variable in the official docs, but no examples)?
From what I see from your code, you're trying to contact the OTLP exporter through a gRPC call. If you see, in their documentation they wrote this in line 71:
This option has no effect if WithGRPCConn is used.
This means that you can completely avoid passing this variable at all to the otlptracegrpc.NewClient function. I instantiate a gRPC client with this code and it works:
func newOtlpExporter(ctx context.Context) (trace.SpanExporter, error) {
client := otlptracegrpc.NewClient(otlptracegrpc.WithInsecure(), otlptracegrpc.WithDialOption(grpc.WithBlock()))
exporter, err := otlptrace.New(ctx, client)
if err != nil {
panic(err)
}
return exporter, err
}
Back to your question, you're right with your guess but only if you're sending metrics, traces, and so on through HTTPS calls.
Let me know if this helps to solve the issue or if anything else is needed!
Edit 1
I overlooked this. The comment you linked in the question is taken from the wrong file. The correct line is this: https://github.com/open-telemetry/opentelemetry-go/blob/48a05478e238698e02b4025ac95a11ecd6bcc5ad/exporters/otlp/otlptrace/otlptracegrpc/options.go#L71
As you can see, the comment is clearer and you have only two options:
Provide your own endpoint address
Use the default one which is localhost:0.0.0.0:4317
Let me know if helps!

Reading CloudWatch log query status in go SDK v2

I'm running a CloudWatch log query through the v2 SDK for Go. I've successfully submitted the query using the StartQuery method, however I can't seem to process the results.
I've got my query ID in a variable (queryID) and am using the GetQueryResults method as follows:
results, err := svc.GetQueryResults(context.TODO(), &cloudwatchlogs.GetQueryResultsInput{QueryId: queryId,})
How do I actually read the contents? Specifically, I'm looking at the Status field. If I run the query at the command line, this comes back as a string description. According to the SDK docs, this is a bespoke type "QueryStatus", which is defined as a string with enumerated constants.
I've tried comparing to the constant names, e.g.
if results.Status == cloudwatchlogs.GetQueryResultsOutput.QueryStatus.QueryStatusComplete
but the compiler doesn't accept this. How do I either reference the constants or get to the string value itself?
The QueryStatus type is defined in the separate types package. The Go SDK services are all organised this way.
import "github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs/types"
if res.Status == types.QueryStatusComplete {
fmt.Println("complete!")
}

Writing new key to configuration file with Viper

First easy project with Go here.
Based on user input, I need to add new keys to my existing configuration file.
I manage to read it correctly with Viper and use it throughout the application, but WriteConfig doesn't seem to work.
Here's a snippet:
oldConfig := viper.AllSettings()
fmt.Printf("All settings #1 %+v\n\n", oldConfig)
viper.Set("setting1", chosenSetting1)
viper.Set("setting2", chosenSetting2)
newConfig := viper.AllSettings()
fmt.Printf("All settings #2 %+v\n\n", newConfig)
err := viper.WriteConfig()
if err != nil {
log.Fatalln(err)
}
newConfig includes new settings as expected, but WriteConfig doesn't apply changes to the config file.
I've read in Viper repo that writing functions are quite controversial and a bit buggy in terms of treating existing or non-existing files, but I expect them to work in simple cases like this.
I also tried other functions (i.e. SafeWriteConfig) with no success.
I'm using Go 1.16.2 and Viper 1.7.1.
What am I doing wrong?
viper.WriteConfig() // writes current config to predefined path set by 'viper.AddConfigPath()' and 'viper.SetConfigName'
you first need to specify the path to the config file
or try this method bellow
viper.SafeWriteConfigAs("/path/to/my/.config") // will error since it has already been written
try WriteConfigAs(filename) ; you will be able to name the file to write to.
If there is no error in WriteConfig, it's probably that the changes are not written to the file you expect.
viper.ConfigFileUsed() should return the path used by default.

Golang Google Admin SDK API Client not setting boolean properties correctly

I am using golang package google.golang.org/api/admin/directory/v1 v0.28.0 to build an administrator CLI for Google G Suite.
The User type (and other types as well) has boolean properties with omitempty json tags. I can set these properties to a value of true, but when I try to set them to false, nothing happens. I believe that this is because the omitempty option means that the value false is ignored when passed.
After a bit of web research, it seems that the answer might be to use pointers and change the type of the boolean properties from bool to *bool. Then set the value by using a bool pointer like this:
user := new(*admin.User)
f := new(bool)
*f = false
user.Suspended = f
I'm not sure why this might work, but it was suggested and confirmed as working in respect of another API.
These API client libraries are auto-generated and I'm not sure how I can test my theory. If anyone has come across this issue in the past, is there a remedy or workaround for it? Could anyone suggest how I can generate my own version of the golang client?

Different const value for dev and prod env in go

For example I'm developing guestbook. It allows to add records, that need to be approved by moderator. But when developing further functionality, it's a pain to approve each record, that is added during testing.
So, is it possible to build dev version of application that creates such records with appropriate flag set?
For example prod build is compiled with the following fucntion:
func NewRecord() Record {
return Record{Moderation: Awaiting}
}
And in dev build is compiled with:
func NewRecord() Record {
return Record{Moderation: Approved}
}
I know in frontend, when you building some JS app, it's a common practice to set NODE_ENV=production environment variable when building for production.
I'm looking for something similar in Go.
I see two ways, but don't like any of them:
I can just set Awaiting = Approved while developing and then change it back to actual value when building prod version. But I afraid that one day I will forget about this mock, will commit it to repo or something like that.
Change function to something like
func NewRecord() Record {
if os.Getenv(mykey) == "production" {
return Record{Moderation: Awaiting}
} else {
return Record{Moderation: Approved}
}
}
But I don't like that this condition is evaluated in runtime for each new record. It just seems to be wrong for compiled language.
As a bonus it would be nice, if such application can show warning (to stdout/stderr) if it's build as dev version.
Thanks.
Personally I think using environment variables as you've done in your example is the "correct" way to do this. This allows you to tweak the behavior without rebuilding the application, which may be very useful while debugging. However, if you really want this done at compile time it can be accomplished with build constraints.
A built constraint is a special comment placed at the top of a file that says "only build this file if a specific condition is met". Conditions can be things like the machine architecture we're building on, the OS we're running, or custom build tags that the user specifies when building.
For example, you could tweak your function to something like:
func NewRecord() Record {
return Record{Moderation: ModLevel}
}
and then define ModLevel twice, once in a file modlevel_prod.go that looks like this (Note that the syntax for build constraints is changing, the second line is the new syntax and the first one wont' be valid for much longer. See the link at the end of this post for more info):
// +build !dev
//go:build !dev
package mypackage
const ModLevel = Awaiting
and one in modlevel_dev.go (or whever, the filename doesn't matter in this case) that looks like this:
// +build dev
//go:build dev
package mypackage
const ModLevel = Approved
Now, when you build, the production version of the file is the default and if you want to build the dev version of the file you have to explicitly include the build tag:
$ go build -tags dev
EDIT: the syntax for build constraints is changing. I have updated the examples to include both the old and new versions (which currently coexist). See https://golang.org/issues/41184 for more info and a timeline for deprecating the old one.
Evaluate the expression once at package initialization:
var defaultModeration int
func init() {
if os.Getenv(mykey) == "production" {
defaultModeration = Awaiting
} else {
defaultModeration = Approved
}
}
func NewRecord() Record { return Record{Moderation: defaultModeration} }

Resources