I'm trying to find a way to add a correlation/request id for logs in our project to make it easier to navigate through them and debug when some issues occur. I found this article. From the example there, there is a middleware to add the correlationID and then retrieve it in some handler function.
Middleware function:
const ContextKeyRequestID ContextKey = "requestID"
func reqIDMiddleware1(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
id := uuid.New()
ctx = context.WithValue(ctx, ContextKeyRequestID, id.String())
r = r.WithContext(ctx)
log.Debugf("Incoming request %s %s %s %s", r.Method, r.RequestURI, r.RemoteAddr, id.String())
next.ServeHTTP(w, r)
log.Debugf("Finished handling http req. %s", id.String())
})
}
Handler:
const LogFieldKeyRequestID = "requestID"
func handleSomeRequest() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
reqIDRaw := ctx.Value(ContextKeyRequestID) // reqIDRaw at this point is of type 'interface{}'
reqID, ok := reqIDRaw.(string)
if !ok {
// handler error
}
// if reached here, reqID is ready to be used
// let's use it with logrus FieldLogger!
logger := log.WithField(LogFieldKeyRequestID, reqID)
// Do something, then log what you did
logger.Debugf("What I just did!")
// Do more, log more. Handle this request seriously
}
}
But I was wondering if there is a way to achieve this without having to refactor all the existing handlers and changing the logging functionality, through some automatic configuration that would add id for each log, in my case our project is quite big, and doing it in the way described above would require a lot of changes.
Have you looked at WithContext and Hooks?
You still have to modify your code but you can centralize some behaviours.
https://go.dev/play/p/4YxJMK6Zl5D
Related
Using this https://github.com/praveen001/go-passport
Path with passsport middleware:
app.Put("/api/auth/login", p.Authenticate("local", MyHandler))
The passport authenticate function:
// Authenticate calls `Strategy.Authenticate` method of registered strategies, and checks the `passport.Result` returned by it.
//
// The result is stored in the request context with `passport.CtxKey` as key.
//
func (p *Passport) Authenticate(name string, h http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
s, ok := p.Options.strategies[name]
if !ok {
w.WriteHeader(404)
return
}
s.Authenticate(w, r, func(res *Result) {
res.StrategyName = name
ctx := context.WithValue(r.Context(), CtxKey, res)
h.ServeHTTP(w, r.WithContext(ctx))
})
}
}
So this attach the response from my auth method and returns a http.HandleFunc which I can define myself (MyHandler).
func MyHandler(w http.ResponseWriter, r *http.Request) {
// Where is the data attached from the Authenticate func?
w.Header().Add("Content-Type", "application/json")
w.WriteHeader(401)
io.WriteString(w, `{"status":"ok"}`)
}
I dont understand where I can reach the attached data from the Authenticate func. The comment says: "//The result is stored in the request context with passport.CtxKey as key".
I don't find it in the r *http.Request.
The http.Request object has a Context() method which gives you the context. You access the key through that.
E.g.
ctx := r.Context()
value := ctx.Value(passport.CtxKey)
I am designing my handlers to return a http.Handler. Here's the design of my handlers:
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
})
}
My middleware is designed to accept an http.Handler and then call the handler once the middleware has finished performing its operations. Here's the design of my middleware:
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
next.ServeHTTP(w, r)
})
}
Considering the design of my middleware and handlers, what is the proper way of passing information from the middleware to the handler? The information that I am trying to pass from my middleware to the handlers is a JSON web token parsed from the request body. If I do not pass the parsed JWT to the handler, then I will need to parse the JWT again in my handlers. Parsing the request body for a JWT in both the middleware and handler seems wasteful. Just in case this information is relevant, I am using the standard net/http library with gorilla mux.
Since you're already using Gorilla take a look at the context package.
(This is nice if you don't want to change your method signatures.)
import (
"github.com/gorilla/context"
)
...
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
// Parse body/get token.
context.Set(r, "token", token)
next.ServeHTTP(w, r)
})
}
...
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token := context.Get(r, "token")
})
}
Update
The Gorilla context package is now in maintenance mode
per the repo:
Note gorilla/context, having been born well before context.Context existed, does not play well with the shallow copying of the request that http.Request.WithContext (added to net/http Go 1.7 onwards) performs.
Using gorilla/context may lead to memory leaks under those conditions, as the pointers to each http.Request become "islanded" and will not be cleaned up when the response is sent.
You should use the http.Request.Context() feature in Go 1.7.
The proper way to pass request scoped data would now be the context package in the standard library.
https://golang.org/pkg/context/
You can access it with request.Context on an http.Request.
A first approach, similar to the question, is in codemodus/chain by Daved.
Package chain aids the composition of Handler wrapper chains that carry request-scoped data.
It uses the notion of Context, coupled with a Context handler:
func ctxHandler(ctx context.Context, w http.ResponseWriter, r *http.Request) {
// ...
if s, ok := getMyString(ctx); ok {
// s = "Send this down the line."
}
// ...
}
Another approach: You can have a look at "Custom Handlers and Avoiding Globals in Go Web Applications", by Matt Silverlock (elithrar). (full example here)
The idea is to define ServeHTTP on a type which include the relevant context.
// We've turned our original appHandler into a struct with two fields:
// - A function type similar to our original handler type (but that now takes an *appContext)
// - An embedded field of type *appContext
type appHandler struct {
*appContext
h func(*appContext, http.ResponseWriter, *http.Request) (int, error)
}
// Our ServeHTTP method is mostly the same, and also has the ability to
// access our *appContext's fields (templates, loggers, etc.) as well.
func (ah appHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Updated to pass ah.appContext as a parameter to our handler type.
status, err := ah.h(ah.appContext, w, r)
if err != nil {
log.Printf("HTTP %d: %q", status, err)
switch status {
case http.StatusNotFound:
http.NotFound(w, r)
// And if we wanted a friendlier error page, we can
// now leverage our context instance - e.g.
// err := ah.renderTemplate(w, "http_404.tmpl", nil)
case http.StatusInternalServerError:
http.Error(w, http.StatusText(status), status)
default:
http.Error(w, http.StatusText(status), status)
}
}
}
In the appContext struct, you would put any data you want to pass around.
I am building a Go web application that supports various middleware functions when handling routing. I'm trying to stick to net/http as much as possible and was wondering how I might accomplish this without using middleware libraries like negroni.
Essentially what I would like to do is to be able to provide a slice of middleware functions, say one for logging, one for checking for a valid JWT, and then finally the handler to handle the request.
I am able to do this with negroni fairly simply by defining the following struct:
// Route ..
type Route struct {
Method string
Path string
Middleware []negroni.Handler
Handler http.HandlerFunc
}
and then defining a route like:
var commonRoutes = []Route{
{
Method: "GET",
Path: "/info",
Middleware: []negroni.Handler{negroni.HandlerFunc(middleware.CheckCache), negroni.HandlerFunc(middleware.Authenticated), negroni.NewLogger()},
Handler: handlers.APIInfo,
},
}
Finally when I boot up my server, I import the list of routes and register them like so:
for _, r := range routes {
handler := append(r.Middleware, negroni.Wrap(r.Handler))
router.Handle(r.Path, negroni.New(handler...)).Methods(r.Method)
}
And this works perfectly.
Any idea how I might be able to do this with just the standard net/http signature and way of defining middleware handlers that look like this:
http.Handle("/", middlewareOne(middlewareTwo(finalHandler)))
Thank you :)
func Auth(n http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
log.Printf("Start")
n.ServeHTTP(w, r)
log.Printf("End")
})
}
func processReq(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Success"))
}
func main() {
handler := http.HandlerFunc(processReq)
http.Handle("/",Auth(handler))
http.ListenAndServe(":8000", nil)
}
can be done using http.handler
Simple. You define each handler like so:
// So I don't have to type it over and over...
type HTTPHandler func(w http.ResponseWriter, r *http.Request)
func Handler1(next HTTPHandler) HTTPHandler {
return func(w http.ResponseWriter, r *http.Request){
// Do stuff
if next != nil {
next(w, r)
}
}
}
// Handler2 ... HandlerN defined in the same basic way.
// Chaining:
http.Handle("/", Handler1(Handler2(nil)))
Each handler takes the next handler and returns a closure that does whatever you want plus calling the next handler. If you need lots of these it may make sense to write a helper similar to this one:
func MakeHandler(worker, next HTTPHandler) HTTPHandler {
return func(w http.ResponseWriter, r *http.Request){
// Maybe have to worker return an error and do standard error
// handling here? Could simplify your code some depending on
// what you are doing.
worker(w, r)
if next != nil {
next(w, r)
}
}
}
I've got a pretty solid grip on how to pass data from a handler to the handler it wraps, but is there a go idiomatic way to get something back from the wrapped handler? Here's a motivating example: I have an accessLogHandler and an authHandler. accessLogHandler logs every http request, with timings and other request info such as the currently logged in user's ID (if there is one). authHandler is for routes that need a logged in user, it 403's when a user isn't logged in. I want to wrap some (but perhaps not all) of my routes with the authHandler, and wrap all of my routes with the accessLogHandler. If a user is logged in, I would like my accessLogHandler to log the user info along with the access log.
Now, I have a solution I've come up with that I don't like. I'll add the code and then explain some of my issues with it.
// Log the timings of each request optionally including user data
// if there is a logged in user
func accessLogHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
accessLog := newAccessLog()
ctx := context.WithValue(r.Context(), accessLogKey, accessLog)
fn.ServeHTTP(w, r.WithContext(ctx))
// Logs the http access, ommit user info if not set
accessLog.Log()
}
}
// pull some junk off the request/cookies/whatever and check if somebody is logged in
func authHandler(fn http.HandlerFunc) http.HandlerFunc {
return func (w http.ResponseWriter, r *http.Request) {
//Do some authorization
user, err := auth(r)
if err != nil{
//No userId, don't set anything on the accesslogger
w.WriteHeader(http.StatusForbiddend)
return
}
//Success a user is logged in, let's make sure the access logger knows
acessLog := r.Context().Value(accessLogKey).(*AccessLog)
accessLog.Set("userID", user.ID)
fn.ServeHTTP(w, r)
}
}
Basically, what I'm doing here is attaching an accessLog struct to my context inside the accessLogHandler and inside the authHandler I'm reading accessLog from the context and calling accessLog.Set to inform the logger that a userID is present.
Some things I don't like about this approach:
context is immutable, but I'm sticking a mutable struct on it and mutating said struct elsewhere downstream. Feels like a hack.
My authHandler now has a package level dependency on the accessLog package, since I'm type asserting to *AccessLog.
Ideally my authHandler would have some way of informing any part of the request stack about user data without tightly coupling itself to said parts.
Context itself is an interface, so you could create a new logger context in the logger middleware that has the methods you would need to get the behavior you are after.
Something like this:
type Logger struct{}
func (l *Logger) SetLogField(key string, value interface{}) {// set log field }
func (l *Logger) Log(){// log request}
type LoggerCtx struct {
context.Context
*Logger
}
func newAccessLog() *Logger {
return &Logger{}
}
func accessLogHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// create new logger context
ctx := &LoggerCtx{}
ctx.Context = r.Context()
ctx.Logger = newAccessLog()
fn.ServeHTTP(w, r.WithContext(ctx))
// Logs the http access, ommit user info if not set
ctx.Log()
}
}
// pull some junk off the request/cookies/whatever and check if somebody is logged in
func authHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
//Do some authorization
user, err := auth(r)
if err != nil {
//No userId, don't set anything on the accesslogger
w.WriteHeader(http.StatusForbiddend)
return
}
//Success a user is logged in, let's make sure the access logger knows
ctx := r.Context()
// this could be moved - here for clarity
type setLog interface {
SetLogField(string, interface{})
}
if lctx, ok := ctx.(setLog); ok {
lctx.SetLogField("userID", user.ID)
}
fn.ServeHTTP(w, r.WithContext(ctx))
}
}
I am designing my handlers to return a http.Handler. Here's the design of my handlers:
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
})
}
My middleware is designed to accept an http.Handler and then call the handler once the middleware has finished performing its operations. Here's the design of my middleware:
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
next.ServeHTTP(w, r)
})
}
Considering the design of my middleware and handlers, what is the proper way of passing information from the middleware to the handler? The information that I am trying to pass from my middleware to the handlers is a JSON web token parsed from the request body. If I do not pass the parsed JWT to the handler, then I will need to parse the JWT again in my handlers. Parsing the request body for a JWT in both the middleware and handler seems wasteful. Just in case this information is relevant, I am using the standard net/http library with gorilla mux.
Since you're already using Gorilla take a look at the context package.
(This is nice if you don't want to change your method signatures.)
import (
"github.com/gorilla/context"
)
...
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
// Parse body/get token.
context.Set(r, "token", token)
next.ServeHTTP(w, r)
})
}
...
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token := context.Get(r, "token")
})
}
Update
The Gorilla context package is now in maintenance mode
per the repo:
Note gorilla/context, having been born well before context.Context existed, does not play well with the shallow copying of the request that http.Request.WithContext (added to net/http Go 1.7 onwards) performs.
Using gorilla/context may lead to memory leaks under those conditions, as the pointers to each http.Request become "islanded" and will not be cleaned up when the response is sent.
You should use the http.Request.Context() feature in Go 1.7.
The proper way to pass request scoped data would now be the context package in the standard library.
https://golang.org/pkg/context/
You can access it with request.Context on an http.Request.
A first approach, similar to the question, is in codemodus/chain by Daved.
Package chain aids the composition of Handler wrapper chains that carry request-scoped data.
It uses the notion of Context, coupled with a Context handler:
func ctxHandler(ctx context.Context, w http.ResponseWriter, r *http.Request) {
// ...
if s, ok := getMyString(ctx); ok {
// s = "Send this down the line."
}
// ...
}
Another approach: You can have a look at "Custom Handlers and Avoiding Globals in Go Web Applications", by Matt Silverlock (elithrar). (full example here)
The idea is to define ServeHTTP on a type which include the relevant context.
// We've turned our original appHandler into a struct with two fields:
// - A function type similar to our original handler type (but that now takes an *appContext)
// - An embedded field of type *appContext
type appHandler struct {
*appContext
h func(*appContext, http.ResponseWriter, *http.Request) (int, error)
}
// Our ServeHTTP method is mostly the same, and also has the ability to
// access our *appContext's fields (templates, loggers, etc.) as well.
func (ah appHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Updated to pass ah.appContext as a parameter to our handler type.
status, err := ah.h(ah.appContext, w, r)
if err != nil {
log.Printf("HTTP %d: %q", status, err)
switch status {
case http.StatusNotFound:
http.NotFound(w, r)
// And if we wanted a friendlier error page, we can
// now leverage our context instance - e.g.
// err := ah.renderTemplate(w, "http_404.tmpl", nil)
case http.StatusInternalServerError:
http.Error(w, http.StatusText(status), status)
default:
http.Error(w, http.StatusText(status), status)
}
}
}
In the appContext struct, you would put any data you want to pass around.