Integrating Sentry and Elastic APM in Buffalo - go

I am trying to integrate Elastic APM and Sentry into my website using Buffalo. The interesting files are as follows:
handlers/sentryHandler.go
package handlers
import (
sentryhttp "github.com/getsentry/sentry-go/http"
"github.com/gobuffalo/buffalo"
)
func SentryHandler(next buffalo.Handler) buffalo.Handler {
handler := buffalo.WrapBuffaloHandler(next)
sentryHandler := sentryhttp.New(sentryhttp.Options{})
return buffalo.WrapHandler(sentryHandler.Handle(handler))
}
handlers/elasticAPMHandler.go
package handlers
import (
"fmt"
"github.com/gobuffalo/buffalo"
"go.elastic.co/apm/module/apmhttp"
)
func ElasticAPMHandler(next buffalo.Handler) buffalo.Handler {
fmt.Println("AAA")
handler := apmhttp.Wrap(buffalo.WrapBuffaloHandler(next))
return buffalo.WrapHandler(handler)
}
actions/app.go
package actions
import (
"github.com/gobuffalo/buffalo"
"github.com/gobuffalo/envy"
forcessl "github.com/gobuffalo/mw-forcessl"
paramlogger "github.com/gobuffalo/mw-paramlogger"
"github.com/unrolled/secure"
"my_website/handlers"
"my_website/models"
"github.com/gobuffalo/buffalo-pop/pop/popmw"
csrf "github.com/gobuffalo/mw-csrf"
i18n "github.com/gobuffalo/mw-i18n"
"github.com/gobuffalo/packr/v2"
)
func App() *buffalo.App {
if app == nil {
app = buffalo.New(buffalo.Options{
Env: ENV,
SessionName: "_my_website_session",
})
// Automatically redirect to SSL
app.Use(forceSSL())
// Catch errors and send them to Sentry.
app.Use(handlers.SentryHandler)
// Get tracing information and send it to Elastic.
app.Use(handlers.ElasticAPMHandler)
// Other Buffalo middleware stuff goes here...
// Routing stuff goes here...
}
return app
}
The problem I'm running into is if I have the Sentry/APM handlers at the top, then I get errors like application.html: line 24: "showPagePath": unknown identifier. However, if I move it to just before I set up the routes, then I get a no transaction found error. So, I'm guessing that the handler wrappers are dropping the buffalo.Context information. So, what would I need to do to be able to integrate Sentry and Elastic in Buffalo asides from trying to reimplement their wrappers?

So, I'm guessing that the handler wrappers are dropping the buffalo.Context information.
That's correct. The problem is that buffalo.WrapHandler (Source) throws away all of the context other than the underlying http.Request/http.Response:
// WrapHandler wraps a standard http.Handler and transforms it
// into a buffalo.Handler.
func WrapHandler(h http.Handler) Handler {
return func(c Context) error {
h.ServeHTTP(c.Response(), c.Request())
return nil
}
}
So, what would I need to do to be able to integrate Sentry and Elastic in Buffalo asides from trying to reimplement their wrappers?
I can see two options:
Reimplement buffalo.WrapHandler/buffalo.WrapBuffaloHandler to stop throwing away the buffalo.Context. This would involve storing the buffalo.Context in the underlying http.Request's context, and then pulling it out again on the other side instead of creating a whole new context.
Implement Buffalo-specific middleware for Sentry and Elastic APM without using the Wrap* functions.
There's an open issue in the Elastic APM agent for the latter option: elastic/apm#39.

Related

Logging to stderr and stdout golang Google Cloud Platform

Currently running a go service on GCP however in the logs viewer every message is treated as an error.
Is there a generally advised way of logging to stderr and stdout depending on the log level. Ie errors to stderr and anything else to stdout.
I'm currently using the logrus package and have come across this implementation. Other ways i see achieving this while still using the same package is to pass the logger to each package that needs it or to create a global log object, neither of which i am too keen on.
https://github.com/microsoft/fabrikate/pull/252/commits/bd24d62d7c2b851ad6e7b36653eb0a6dc364474b#diff-ed0770fdbf87b0c6d536e33a99a8df9c
You can use Stackdriver library package for GoLang:
go get -u cloud.google.com/go/logging
Then you can use StandardLogger:
// Sample stdlogging writes log.Logger logs to the Stackdriver Logging.
package main
import (
"context"
"log"
"cloud.google.com/go/logging"
)
func main() {
ctx := context.Background()
// Sets your Google Cloud Platform project ID.
projectID := "YOUR_PROJECT_ID"
// Creates a client.
client, err := logging.NewClient(ctx, projectID)
if err != nil {
log.Fatalf("Failed to create client: %v", err)
}
defer client.Close()
// Sets the name of the log to write to.
logName := "my-log"
logger := client.Logger(logName).StandardLogger(logging.Info)
// Logs "hello world", log entry is visible at
// Stackdriver Logs.
logger.Println("hello world")
}
Here you can find documentation on Google Cloud website
Update:
Alternatively you could give a try GCP formatter for logrus
This will not tie your app to Google Cloud Platform. However, it does not mean that on another platform you will not need to change your code to format output.
Using StackDriver library is the recommended solution for Google Cloud.
We use https://github.com/rs/zerolog with the following method called in our Init() method to setup the logging options at global level:
package main
import (
"github.com/rs/zerolog"
"github.com/rs/zerolog/log"
"os"
)
// initializeLogging sets up the logging configuration for the service.
// Invoke this method in your Init() method.
func initializeLogging() {
// Set logging options for production development
if os.Getenv("ENV") != "DEV" {
// change the level field name to ensure these are parsed correctly in Stackdriver
zerolog.LevelFieldName = "severity"
// UNIX Time is faster and smaller than most timestamps
zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
} else {
// Set logging options for local development
log.Logger = log.Output(zerolog.ConsoleWriter{Out: os.Stderr})
zerolog.SetGlobalLevel(zerolog.DebugLevel)
}
// Example log
log.Info().Msg("This is how you log at Info level")
}
I recommend the https://github.com/apsystole/log. You can swap either log and logrus imports for it. It is a small zero-dependency module, hence much lighter than logrus.

List custom resources from caching client with custom fieldSelector

I'm using the Operator SDK to build a custom Kubernetes operator. I have created a custom resource definition and a controller using the respective Operator SDK commands:
operator-sdk add api --api-version example.com/v1alpha1 --kind=Example
operator-sdk add controller --api-version example.com/v1alpha1 --kind=Example
Within the main reconciliation loop (for the example above, the auto-generated ReconcileExample.Reconcile method) I have some custom business logic that requires me to query the Kubernetes API for other objects of the same kind that have a certain field value. It's occurred to me that I might be able to use the default API client (that is provided by the controller) with a custom field selector:
func (r *ReconcileExample) Reconcile(request reconcile.Request) (reconcile.Result, error) {
ctx := context.TODO()
listOptions := client.ListOptions{
FieldSelector: fields.SelectorFromSet(fields.Set{"spec.someField": "someValue"}),
Namespace: request.Namespace,
}
otherExamples := v1alpha1.ExampleList{}
if err := r.client.List(ctx, &listOptions, &otherExamples); err != nil {
return reconcile.Result{}, err
}
// do stuff...
return reconcile.Result{}, nil
}
When I run the operator and create a new Example resource, the operator fails with the following error message:
{"level":"info","ts":1563388786.825384,"logger":"controller_example","msg":"Reconciling Example","Request.Namespace":"default","Request.Name":"example-test"}
{"level":"error","ts":1563388786.8255732,"logger":"kubebuilder.controller","msg":"Reconciler error","controller":"example-controller","request":"default/example-test","error":"Index with name field:spec.someField does not exist","stacktrace":"..."}
The most important part being
Index with name field:spec.someField does not exist
I've already searched the Operator SDK's documentation on the default API client and learned a bit about the inner workings of the client, but no detailed explanation on this error or how to fix it.
What does this error message mean, and how can I create this missing index to efficiently list objects by this field value?
The default API client that is provided by the controller is a split client -- it serves Get and List requests from a locally-held cache and forwards other methods like Create and Update directly to the Kubernetes API server. This is also explained in the respective documentation:
The SDK will generate code to create a Manager, which holds a Cache and a Client to be used in CRUD operations and communicate with the API server. By default a Controller's Reconciler will be populated with the Manager's Client which is a split-client. [...] A split client reads (Get and List) from the Cache and writes (Create, Update, Delete) to the API server. Reading from the Cache significantly reduces request load on the API server; as long as the Cache is updated by the API server, read operations are eventually consistent.
To query values from the cache using a custom field selector, the cache needs to have a search index for this field. This indexer can be defined right after the cache has been set up.
To register a custom indexer, add the following code into the bootstrapping logic of the operator (in the auto-generated code, this is done directly in main). This needs to be done after the controller manager has been instantiated (manager.New) and also after the custom API types have been added to the runtime.Scheme:
package main
import (
k8sruntime "k8s.io/apimachinery/pkg/runtime"
"example.com/example-operator/pkg/apis/example/v1alpha1"
// ...
)
function main() {
// ...
cache := mgr.GetCache()
indexFunc := func(obj k8sruntime.Object) []string {
return []string{obj.(*v1alpha1.Example).Spec.SomeField}
}
if err := cache.IndexField(&v1alpha1.Example{}, "spec.someField", indexFunc); err != nil {
panic(err)
}
// ...
}
When a respective indexer function is defined, field selectors on spec.someField will work from the local cache as expected.

How to set broker.SubscriberOptions in go-micro

I am trying to configure a RabbitMQ broker using the go-micro framework. I have noticed that the broker interface in go-micro has a broker.SubscriberOptions struct which allows configuring the parameters I am looking for (AutoAck, Queue name and so on) however I am unable to figure out how to pass this when starting a broker.
This is how a simple rabbit go-micro setup would look like
package main
import (
"log"
"github.com/micro/go-micro/server"
"github.com/micro/go-plugins/broker/rabbitmq"
micro "github.com/micro/go-micro"
)
func main() {
// Create a new service. Optionally include some options here.
service := micro.NewService(
micro.Name("go-micro-rabbit"),
micro.Broker(rabbitmq.NewBroker()),
)
// Init will parse the command line flags.
service.Init()
// Register handler
proto.RegisterGreeterHandler(service.Server(), new(Greeter))
micro.RegisterSubscriber("micro-exchange", service.Server(), myFunc, server.SubscriberQueue("my-queue"))
// Run the server
if err := service.Run(); err != nil {
log.Fatal(err)
}
}
The micro.RegisterSubscriber method takes in a list of server.SubscriberOption but does not allow me to set the broker.SubscriberOptions and the rabbitmq.NewBroker allows setting broker.Options but once again, not broker.SubscriberOptions
I have dug in the code of go-micro but have been unable to figure out how the broker.Subscribe method (Which exposes the correct struct) is called or by who.
Is this possible at all? Is it maybe something not yet fully fleshed out in the API?

Golang service/daos implementation

Coming from a Java background, I have some questions on how things are typically done in Golang. I am specifically talking about services and dao's/repositories.
In java, I would use dependency injection (probably as singleton/application-scoped), and have a Service injected into my rest endpoint / resource.
To give a bit more context. Imagine the following Golang code:
func main() {
http.ListenAndServe("localhost:8080", nil)
}
func init() {
r := httptreemux.New()
api := r.NewGroup("/api/v1")
api.GET("/blogs", GetAllBlogs)
http.Handle("/", r)
}
Copied this directly from my code, main and init are split because google app engine.
So for now I have one handler. In that handler, I expect to interact with a BlogService.
The question is, where, and in what scope should I instantiate a BlogService struct and a dao like datastructure?
Should I do it everytime the handler is triggered, or make it constant/global?
For completeness, here is the handler and blogService:
// GetAllBlogs Retrieves all blogs from GCloud datastore
func GetAllBlogs(w http.ResponseWriter, req *http.Request, params map[string]string) {
c := appengine.NewContext(req)
// need a reference to Blog Service at this point, where to instantiate?
}
type blogService struct{}
// Blog contains the content and meta data for a blog post.
type Blog struct {...}
// newBlogService constructs a new service to operate on Blogs.
func newBlogService() *blogService {
return &blogService{}
}
func (s *blogService) ListBlogs(ctx context.Context) ([]*Blog, error) {
// Do some dao-ey / repository things, where to instantiate BlogDao?
}
You can use context.Context to pass request scoped values into your handlers (available in Go 1.7) , if you build all your required dependencies during the request/response cycle (which you should to avoid race conditions, except for dependencies that manage concurrency on their own like sql.DB). Put all your services into a single container for instance, then query the context for that value :
container := request.Context.Value("container").(*Container)
blogs,err := container.GetBlogService().ListBlogs()
read the following material :
https://golang.org/pkg/context/
https://golang.org/pkg/net/http/#Request.Context

Overriding ResponseWriter interface to catch HTTP errors

I'm coding a web application in Go, and while various mux libraries provide a way to set a custom 404 error handler, there's nothing for other 4xx and 5xx error codes.
One suggestion is to override the WriteHeader method in the ResponseWriter interface and check the status code, but I'm confused as to how this would actually be written (the overriding of ResponseWriter methods, before outputting). One possible example can be found from the negroni package.
Would this be the correct way to serve a custom template for 4xx and 5xx errors? Could anyone provide an example of how this could be implemented?
Update
Big thanks to David and elithrar for their responses and code. The Interceptor struct that David coded can be used in a wrapper for the server mux, as elithrar shows in the comments. For those looking for further explanation as to why and how this works, this section from astaxie's book gives some very good info on the workings of the net/http package, along with viewing the server.go source code from the net/http package.
The mux libraries only have a means of setting the not found handler as a means of giving you a way to intercept requests where the mux can't resolve the URL to it's known mappings.
For example, you do:
mux.Handle("/foo",fooFunc)
mux.Handle("/bar",barFunc)
But the client accesses /baz to which the mux has no mapping.
They do not actually intercept a 404 going to the client, they just invoke the not found handler when it runs in to this problem.
Also, if your /foo handler sends a 404 response, the not found does not get invoked.
If you want custom pages for various return responses from your mapped URLs, simply make the various handlers write the correct response.
If you don't control that logic (ie: the framwork is writing something and has no means to override), then you may want to intercept all requests and override the http.ResposeWriter with response code detection logic.
Here's an example interceptor that basically does what you want. On Play
package main
import (
"fmt"
"log"
)
import "net/http"
type Interceptor struct {
origWriter http.ResponseWriter
overridden bool
}
func (i *Interceptor) WriteHeader(rc int) {
switch rc {
case 500:
http.Error(i.origWriter, "Custom 500 message / content", 500)
case 404:
http.Error(i.origWriter, "Custom 404 message", 404)
case 403:
i.origWriter.WriteHeader(403)
fmt.Fprintln(i.origWriter, "Custom 403 message")
default:
i.origWriter.WriteHeader(rc)
return
}
// if the default case didn't execute (and return) we must have overridden the output
i.overridden = true
log.Println(i.overridden)
}
func (i *Interceptor) Write(b []byte) (int, error) {
if !i.overridden {
return i.origWriter.Write(b)
}
// Return nothing if we've overriden the response.
return 0, nil
}
func (i *Interceptor) Header() http.Header {
return i.origWriter.Header()
}

Resources