How do I write a pre/post traffic hook function in go? - go

I started using AWS SAM and for now I only have some unit tests, but I want to try to run integration tests in a pre traffic hook function.
Unfortunately there seems to be no code example for Golang, all I could find was for Javascript.
From this example I pieced together that I have to use the code deploy SDK and call PutLifecycleEventHookExecutionStatus, but the specifics remain unclear. The aws code example repo for go has no examples for code deploy either.
More Information about the topic that I am looking for is available here https://github.com/awslabs/serverless-application-model/blob/master/docs/safe_lambda_deployments.rst#pretraffic-posttraffic-hooks.
I want to start out by testing a lambda function that simply queries DynamoDB.

Something like this works:
package main
import (
"context"
"encoding/json"
"github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/codedeploy"
)
type CodeDeployEvent struct {
DeploymentId string `json:"deploymentId"`
LifecycleEventHookExecutionId string `json:"lifecycleEventHookExecutionId"`
}
func HandleRequest(ctx context.Context, event CodeDeployEvent) (string, error) {
// add some tests here and change status flag as needed . . .
client := codedeploy.New(session.New())
params := &codedeploy.PutLifecycleEventHookExecutionStatusInput{
DeploymentId: &event.DeploymentId,
LifecycleEventHookExecutionId: &event.LifecycleEventHookExecutionId,
Status: "Succeeded",
}
req, _ := client.PutLifecycleEventHookExecutionStatusRequest(params)
_ = req.Send()
}

I got around to implement this and want to share my complete solution.
After figuring out how to use it, I decided against using it, because there are a couple of drawbacks.
there is no way to expose a new version of the canary to a dedicated portion of the user base, that means sometimes they'll hit the new or the old version
invoking functions that publish to sns will trigger all downstream actions, which might get the new or the old version of the downstream services, which would cause a lot of problems in case of breaking APIs
IAM changes affect both version immediately, possibly breaking the old version.
Instead, I deploy everything to a pre prod account, run my integration and e2e tests and if they succeed I'll deploy to prod
the cdk code to create a canary deployment:
const versionAlias = new lambda.Alias(this, 'Alias', {
aliasName: "alias",
version: this.lambda.currentVersion,
})
const preHook = new lambda.Function(this, 'LambdaPreHook', {
description: "pre hook",
code: lambda.Code.fromAsset('dist/upload/convert-pre-hook'),
handler: 'main',
runtime: lambda.Runtime.GO_1_X,
memorySize: 128,
timeout: cdk.Duration.minutes(1),
environment: {
FUNCTION_NAME: this.lambda.currentVersion.functionName,
},
reservedConcurrentExecutions: 5,
logRetention: RetentionDays.ONE_WEEK,
})
// this.lambda.grantInvoke(preHook) // this doesn't work, I need to grant invoke to all functions :s
preHook.addToRolePolicy(new iam.PolicyStatement({
actions: [
"lambda:InvokeFunction",
],
resources: ["*"],
effect: iam.Effect.ALLOW,
}))
const application = new codedeploy.LambdaApplication(this, 'CodeDeployApplication')
new codedeploy.LambdaDeploymentGroup(this, 'CanaryDeployment', {
application: application,
alias: versionAlias,
deploymentConfig: codedeploy.LambdaDeploymentConfig.ALL_AT_ONCE,
preHook: preHook,
autoRollback: {
failedDeployment: true,
stoppedDeployment: true,
deploymentInAlarm: false,
},
ignorePollAlarmsFailure: false,
// alarms:
// autoRollback: codedeploy.A
// postHook:
})
My go code of the pre hook function. PutLifecycleEventHookExecutionStatus tells code deploy if the pre hook succeeded or not. Unfortunately in case you fail the deployment message, the message you get in the cdk deploy output is utterly useless, so you need to check the pre/post hook logs.
In order to actually run the integration test I simply invoke the lambda and check if an error occurred.
package main
import (
"encoding/base64"
"fmt"
"log"
"os"
"github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/aws/aws-sdk-go/service/codedeploy"
lambdaService "github.com/aws/aws-sdk-go/service/lambda"
)
var svc *codedeploy.CodeDeploy
var lambdaSvc *lambdaService.Lambda
type codeDeployEvent struct {
DeploymentId string `json:"deploymentId"`
LifecycleEventHookExecutionId string `json:"lifecycleEventHookExecutionId"`
}
func handler(e codeDeployEvent) error {
params := &codedeploy.PutLifecycleEventHookExecutionStatusInput{
DeploymentId: &e.DeploymentId,
LifecycleEventHookExecutionId: &e.LifecycleEventHookExecutionId,
}
err := handle()
if err != nil {
log.Println(err)
params.Status = aws.String(codedeploy.LifecycleEventStatusFailed)
} else {
params.Status = aws.String(codedeploy.LifecycleEventStatusSucceeded)
}
_, err = svc.PutLifecycleEventHookExecutionStatus(params)
if err != nil {
return fmt.Errorf("failed putting the lifecycle event hook execution status. the status was %s", *params.Status)
}
return nil
}
func handle() error {
functionName := os.Getenv("FUNCTION_NAME")
if functionName == "" {
return fmt.Errorf("FUNCTION_NAME not set")
}
log.Printf("function name: %s", functionName)
// invoke lambda via sdk
input := &lambdaService.InvokeInput{
FunctionName: &functionName,
Payload: nil,
LogType: aws.String(lambdaService.LogTypeTail), // returns the log in the response
InvocationType: aws.String(lambdaService.InvocationTypeRequestResponse), // synchronous - default
}
err := input.Validate()
if err != nil {
return fmt.Errorf("validating the input failed: %v", err)
}
resp, err := lambdaSvc.Invoke(input)
if err != nil {
return fmt.Errorf("failed to invoke lambda: %v", err)
}
decodeString, err := base64.StdEncoding.DecodeString(*resp.LogResult)
if err != nil {
return fmt.Errorf("failed to decode the log: %v", err)
}
log.Printf("log result: %s", decodeString)
if resp.FunctionError != nil {
return fmt.Errorf("lambda was invoked but returned error: %s", *resp.FunctionError)
}
return nil
}
func main() {
sess, err := session.NewSession()
if err != nil {
return
}
svc = codedeploy.New(sess)
lambdaSvc = lambdaService.New(sess)
lambda.Start(handler)
}

Related

How to created an unbounded input for Beam in Go?

I'm trying to use the Go Beam Sdk to create a pipeline processing pubsub messages.
github.com/apache/beam/sdks/v2/go/pkg/beam
I understand that the pubsubio connector is doing external calls working only on dataflow runner.
What if I want to test my pipeline locally ? How would you do that ?
I need to understand what is preventing me to write my own pubsub unbounded source ? (I may not understand how Beam works under the hood, like how does it serialize user defined code to send it to the runner ?)
Tried to do something like that:
package pubsubio
import (
"context"
"fmt"
cloud_pubsub "cloud.google.com/go/pubsub"
"github.com/apache/beam/sdks/v2/go/pkg/beam"
"github.com/apache/beam/sdks/v2/go/pkg/beam/log"
"github.com/apache/beam/sdks/v2/go/pkg/beam/register"
"github.com/apache/beam/sdks/v2/go/pkg/beam/util/pubsubx"
)
func init() {
register.DoFn3x1[context.Context, string, func(*cloud_pubsub.Message), error](&readFn{})
register.Emitter1[*cloud_pubsub.Message]()
}
type ReadConfig struct {
ProjectID string
TopicName string
SubscriptionName string
}
func Read(
scope beam.Scope,
cfg ReadConfig,
) beam.PCollection {
scope = scope.Scope("pubsubio.Read")
col := beam.Create(scope, cfg.SubscriptionName)
return beam.ParDo(scope, newReadFn(cfg.ProjectID, cfg.TopicName), col)
}
type readFn struct {
pubsubFn
TopicName string
}
func newReadFn(projectID, topicName string) *readFn {
return &readFn{
pubsubFn: pubsubFn{
ProjectID: projectID,
},
TopicName: topicName,
}
}
func (fn *readFn) ProcessElement(
ctx context.Context,
subscriptionName string,
emit func(message *cloud_pubsub.Message),
) error {
log.Info(ctx, "[pubsubio.ProcessElement] Reading from pubsub")
_, err := pubsubx.EnsureTopic(ctx, fn.client, fn.TopicName)
if err != nil {
return fmt.Errorf("cannot get topic: %w", err)
}
sub, err := pubsubx.EnsureSubscription(ctx, fn.client, fn.TopicName, subscriptionName)
if err != nil {
return fmt.Errorf("cannot get subscription: %w", err)
}
return sub.Receive(ctx, func(ctx context.Context, message *cloud_pubsub.Message) {
emit(message)
log.Debugf(ctx, "[pubsubio.ProcessElement] Emit msg: %s", message.ID)
message.Ack()
})
}
So basically I created a Read fn that never return, but the rest of my pipeline is never triggered (I must miss something)

aws-sdk-go-v2 PutObject api error AccessDenied

In our Staging environment, we have credential-less access to our private S3 buckets. Access is granted to individual Docker containers. I am trying to upload a file using PutObject using the aws-sdk-go-v2 SDK library, but I'm continually getting a 403 AccessDenied api error.
My upload code looks like this:
var uploadFileFunc = func(s3Details S3Details, key string, payload []byte, params MetadataParams) (*s3.PutObjectOutput, error) {
client := getS3Client(s3Details)
return client.PutObject(context.TODO(), &s3.PutObjectInput{
Bucket: aws.String(s3Details.Bucket),
Key: aws.String(key),
Body: bytes.NewReader(payload),
ContentType: aws.String("text/xml"),
})
}
func getS3Client(s3Details S3Details) *s3.Client {
endpointResolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
if s3Details.EndpointUrl != "" {
return aws.Endpoint{
PartitionID: "aws",
URL: s3Details.EndpointUrl,
SigningRegion: s3Details.Region,
SigningMethod: s3Details.SignatureVersion,
}, nil
}
return aws.Endpoint{}, &aws.EndpointNotFoundError{}
})
cfg, _ := config.LoadDefaultConfig(context.TODO(),
config.WithEndpointDiscovery(aws.EndpointDiscoveryEnabled),
config.WithEndpointResolverWithOptions(endpointResolver))
return s3.NewFromConfig(cfg, func(o *s3.Options) {
o.Region = s3Details.Region
o.Credentials = aws.AnonymousCredentials{}
o.UsePathStyle = true
})
}
I am using aws.AnonymousCredentials{} (as our access is credential-less) but this is only to be used for unsigned requests. I cannot use NewStaticCredentialsProvider with empty values for AccessKeyID and/or SecretAccessKey as this will throw a StaticCredentialsEmptyError during the Retrieve(). Adding dummy credentials will throw an error that they are not on record. I am assuming that this is the cause of my 403 AccessDenied.
How do I sign requests without providing credentials in the Go SDK? Is it even possible? In the boto3 Python library this works fine.
First of all, I'll strongly suggest you use the v2 of the AWS SDK of Go. I'll present here how I do this so far.
First, I get the AWS config to use with this code (only relevant parts are shown):
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
Log.Fatal(err)
}
Here the package used is github.com/aws/aws-sdk-go-v2/config.
Then, I instantiate an s3Client to use for contacting AWS S3 service:
s3Client := s3.NewFromConfig(*cfg)
Here, we use this package github.com/aws/aws-sdk-go-v2/service/s3. Finally, to post your object you have to run this code:
input := &s3.PutObjectInput{
Key: aws.String("test"),
Bucket: aws.String("test"),
Body: bytes.NewReader([]byte("test")),
ACL: types.ObjectCannedACLPrivate,
}
if _, err := s3Client.PutObject(context.TODO(), input); err != nil {
return "", fmt.Errorf("fn UploadFile %w", err)
}
The new package used here is github.com/aws/aws-sdk-go-v2/service/s3/types.
This code is a simplification but you should able to achieve what you need. Furthermore, it should take very little time to update the version of the SDK and you can rely on both of them simultaneously if you've to work with a huge codebase.
Let me know if this helps!
Edit
I updated my solution by using the aws.AnonymousCredentials{} option. Now I was successfully able to upload a file into an s3 bucket with these options. Below you can find the entire solution:
package main
import (
"bytes"
"context"
"crypto/tls"
"net/http"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/s3"
"github.com/aws/aws-sdk-go-v2/service/s3/types"
)
func GetAwsConfig() (*aws.Config, error) {
cfg, err := config.LoadDefaultConfig(context.TODO(),
// config.WithClientLogMode(aws.LogRequestWithBody|aws.LogResponseWithBody),
config.WithRegion("eu-west-1"),
config.WithHTTPClient(&http.Client{Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}}),
config.WithEndpointResolverWithOptions(
aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
return aws.Endpoint{
PartitionID: "aws",
URL: "http://127.0.0.1:4566",
SigningRegion: "eu-west-1",
HostnameImmutable: true,
}, nil
}),
))
if err != nil {
return nil, err
}
return &cfg, err
}
func main() {
cfg, _ := GetAwsConfig()
s3Client := s3.NewFromConfig(*cfg, func(o *s3.Options) {
o.Credentials = aws.AnonymousCredentials{}
})
if _, err := s3Client.PutObject(context.Background(), &s3.PutObjectInput{
Bucket: aws.String("mybucket"),
Key: aws.String("myfile"),
Body: bytes.NewReader([]byte("hello")),
ACL: types.ObjectCannedACLPrivate,
}); err != nil {
panic(err)
}
}
Before running the code, you've to create the bucket. I used the below command:
aws --endpoint-url=http://localhost:4566 s3 mb s3://mybucket
Thanks to this you can upload the file into the mybucket s3 bucket. To check for the file existence you can issue this command:
aws --endpoint-url=http://localhost:4566 s3 ls s3://mybucket --recursive --human-readable
Hope this helps in solving your issue!

Automated integration testing in lambda support (golang, serverless)

So I have a web app running in serverless. It spins up a bunch of lambdas and then a 'test' lambda is invoked at a later stage of our pipeline to run some api tests against the other lambbas. It passes or fails code pipeline depending on the result of the tests.
My concern is the implementation of the tests portion itself.
We're using golang, and I wasn't able to successfully find a way to have a bunch of go test files and run them, record the results and determine pass/fail or not.. but I wanted to use the test suite and library in order to run my assertions. So I came up with this solution which was to break about the go test library and run it myself using MainStart(), ex:
// T is used to manage our test cases manually using MainStart
type T struct{}
func (*T) ImportPath() string { return "" }
func (*T) MatchString(pat, str string) (bool, error) { return true, nil }
func (*T) SetPanicOnExit0(bool) {}
func (*T) StartCPUProfile(io.Writer) error { return nil }
func (*T) StopCPUProfile() {}
func (*T) StartTestLog(io.Writer) {}
func (*T) StopTestLog() error { return nil }
func (*T) WriteHeapProfile(io.Writer) error { return nil }
func (*T) WriteProfileTo(string, io.Writer, int) error { return nil }
func (h *Handler) Handle(ctx context.Context, event events.CodePipelineEvent) (interface{}, error) {
job := event.CodePipelineJob
ok := runTest()
if ok {
logger.Info("Tests Passed!")
input := &codepipeline.PutJobSuccessResultInput{
JobId: &job.ID,
success, err := h.service.PutJobSuccessResult(input)
return success, err
} else {
logger.Info("Tests Failed :(")
input := &codepipeline.PutJobFailureResultInput{
JobId: &job.ID,
FailureDetails: &codepipeline.FailureDetails{
Message: aws.String("tests failed"),
Type: aws.String("JobFailed"),
},
failure, err := h.service.PutJobFailureResult(input)
return failure, err
}
}
}
// NewHandler returns a pointer to a Handler struct
func NewHandler(service *codepipeline.CodePipeline) *Handler {
return &Handler{
service,
}
}
// runTest returns the test results
func runTest() bool {
testSuite := []testing.InternalTest{
{Name: "Test Service", F: TestService},
}
logger.Info("Running Tests")
errors := testing.MainStart(&T{}, testSuite, nil, nil).Run()
if errors == 0 {
return true
} else {
return false
}
}
So the test looks like this:
func TestServices(t *testing.T) {
logger.Info("blah blah do tests")
}
And for reference, main.go contains:
// Initial code pipeline session
cpSession := session.Must(session.NewSession())
// Create the credentials from AssumeRoleProvider to assume the role
// referenced by the "putJobResultRoleArn" ARN.
creds := stscreds.NewCredentials(cpSession, "putJobResultRoleArn", func(p *stscreds.AssumeRoleProvider) {
p.RoleARN = roleArn
p.RoleSessionName = "put_job_result_session"
})
// Create a new instance of the CodePipeline client with a session
svc := codepipeline.New(cpSession, &aws.Config{Credentials: creds})
intTesthandler := inttest.NewHandler(
svc,
)
lambda.Start(func(ctx context.Context, event events.CodePipelineEvent) (interface{}, error) {
return intTesthandler.Handle(ctx, event)
})
This works quite well, but what I don't like is:
It's using an unsupported approach from golang that could break in the future
I have to handhold any failures, meaning if one fails, I need to write code to stop the rest of the suite (if there were more tests) from running.
Bit of a learning curve
To note the tests I have mainly do an http request to an api endpoint and validate the response + parse/store the data.
Anyone have any other creative solution? Is there a repo out there that makes it simple to run integration tests in golang?
Cheers

rpc error: code = Unimplemented desc = RPC method not implemented

I have been trying to create a grpc client in Go and I have followed the correct instructions as shown in the official grpc site. When I start my grpc server written in node.js, the connection works well but upon compiling the protocol buffer in Go and creating a client interface with the correct grpc client configurations, I run into an error.
Here is my what I have in my identity.pb.go.
type IdentityServiceClient interface {
CreateUser(ctx context.Context, in *GoogleIdToken, opts ...grpc.CallOption) (error, *UserInfo)
}
type simpleServerClient struct {
connection *grpc.ClientConn
}
func NewSimpleServerClient(connection *grpc.ClientConn) IdentityServiceClient {
return &simpleServerClient{connection}
}
func (simpleClient *simpleServerClient) CreateUser(ctx context.Context, in *GoogleIdToken, opts ...grpc.CallOption) (error, *UserInfo) {
out := new(UserInfo)
err := simpleClient.connection.Invoke(ctx, "/protobuf.IdentityService/CreateUser", in, out, opts...)
if err != nil {
return err, nil
}
return nil, out
}
here is the identity.proto
syntax="proto3";
package protobuf;
service IdentityService {
rpc CreateUser (GoogleIdToken) returns (UserInfo) {}
}
message GoogleIdToken {
string token = 1;
}
message UserInfo {
string name = 1;
string email = 2;
message Profile {
string imageUrl = 1;
string lastUpdated = 2;
};
Profile profile = 3;
string token = 4;
}
here is my main.go
import pb "github.com/Duncanian/iam-gateway/server/protobuf"
func grpcConnection() {
conn, err := grpc.Dial("localhost:50051", grpc.WithInsecure())
if err != nil {
log.Fatalf("Failed to start gRPC connection: %v", err)
}
defer conn.Close()
client := pb.NewSimpleServerClient(conn)
err, _ = client.CreateUser(context.Background(), &pb.GoogleIdToken{Token: "tevgvybububvvg"})
if err != nil {
log.Fatalf("Failed to create user: %v", err)
}
log.Println("Created user!")
}
I expect the output of passing a correct google auth token to get me the correct user details which are
name: user,
email: user.email#user.com,
profile: {
imageUrl: myimageUrl,
lastUpdated: mylastUpdatedTime,
},
token,
but got
rpc error: code = Unimplemented desc = RPC method not implemented /protobuf.IdentityService/CreateUser
Here are my github repos:
Go grpc client &
Node JS grpc server
I had the same problem.
Here is my solution:
After compiling .proto I created 2 files: client.go and server.go.
In client.go I implemented my methods (rpc in terms of protobuffers) and had a main function.
In server.go I defined a server struct server with one field: Unimplemented*ServiceName*Server. After that I also implemented the mentioned above methods, but those had a receiver type: func (s *server) Foo(ctx context.Context, *other params*)
That worked for me, hopefully it will help you!
The error indicates that the /protobuf.IdentityService/CreateUser method is not registered at the server side. And I didn't see any service registration code in your linked server code. Please take a look at the node.js guide here.
Using GUI of grpc try to send your protos to your both servers and check for upcoming errors/correct endpoints.
In my case java proto had a package inside which was added to the endpoint.
Had package com.example.grpc;
instead of
option java_package = "com.example.grpc";

zap logging with 1) customized config and 2) lumberjack

I'm trying to build a customized zap logger with 1) customized *zap.Config and 2) lumberjack, but can't find proper example to apply both configurations.
Since config.Build does not accept WriteSync as an input. Do you know how to achieve this?
func genBaseLoggerZap() Logger {
ex, err := os.Executable()
if err != nil {
Fatalf("Failed to get os.Executable, err: %v", err)
}
zlManager.outputPath = path.Join(filepath.Dir(ex), zlManager.outputPath)
// Want to add sync here..
zapcore.AddSync(&lumberjack.Logger{
Filename: zlManager.outputPath + "123",
MaxSize: 500,
MaxBackups: 10,
MaxAge: 28,
})
return genLoggerZap(BaseLogger, genDefaultConfig())
}
// genLoggerZap creates a zapLogger with given ModuleID and Config.
func genLoggerZap(mi ModuleID, cfg *zap.Config) Logger {
logger, err := cfg.Build()
if err != nil {
Fatalf("Failed to generate zap logger, err: %v", err)
}
newLogger := &zapLogger{mi, cfg, logger.Sugar()}
newLogger.register()
return newLogger
}
You can add custom log destinations using the zap.RegisterSink function and the Config.OutputPaths field. RegisterSink maps URL schemes to Sink constructors, and OutputPaths configures log destinations (encoded as URLs).
Conveniently, *lumberjack.Logger implements almost all of the zap.Sink interface already. Only the Sync method is missing, which can be easily added with a thin wrapper type.
package main
import (
"net/url"
"go.uber.org/zap"
lumberjack "gopkg.in/natefinch/lumberjack.v2"
)
type lumberjackSink struct {
*lumberjack.Logger
}
// Sync implements zap.Sink. The remaining methods are implemented
// by the embedded *lumberjack.Logger.
func (lumberjackSink) Sync() error { return nil }
func main() {
zap.RegisterSink("lumberjack", func(u *url.URL) (zap.Sink, error) {
return lumberjackSink{
Logger: &lumberjack.Logger{
Filename: u.Opaque,
// Use query parameters or hardcoded values for remaining
// fields.
},
}, nil
})
config := zap.NewProductionConfig()
// Add a URL with the "lumberjack" scheme.
config.OutputPaths = append(config.OutputPaths, "lumberjack:foo.log")
log, _ := config.Build()
log.Info("test", zap.String("foo", "bar"))
}

Resources