I've got a pretty solid grip on how to pass data from a handler to the handler it wraps, but is there a go idiomatic way to get something back from the wrapped handler? Here's a motivating example: I have an accessLogHandler and an authHandler. accessLogHandler logs every http request, with timings and other request info such as the currently logged in user's ID (if there is one). authHandler is for routes that need a logged in user, it 403's when a user isn't logged in. I want to wrap some (but perhaps not all) of my routes with the authHandler, and wrap all of my routes with the accessLogHandler. If a user is logged in, I would like my accessLogHandler to log the user info along with the access log.
Now, I have a solution I've come up with that I don't like. I'll add the code and then explain some of my issues with it.
// Log the timings of each request optionally including user data
// if there is a logged in user
func accessLogHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
accessLog := newAccessLog()
ctx := context.WithValue(r.Context(), accessLogKey, accessLog)
fn.ServeHTTP(w, r.WithContext(ctx))
// Logs the http access, ommit user info if not set
accessLog.Log()
}
}
// pull some junk off the request/cookies/whatever and check if somebody is logged in
func authHandler(fn http.HandlerFunc) http.HandlerFunc {
return func (w http.ResponseWriter, r *http.Request) {
//Do some authorization
user, err := auth(r)
if err != nil{
//No userId, don't set anything on the accesslogger
w.WriteHeader(http.StatusForbiddend)
return
}
//Success a user is logged in, let's make sure the access logger knows
acessLog := r.Context().Value(accessLogKey).(*AccessLog)
accessLog.Set("userID", user.ID)
fn.ServeHTTP(w, r)
}
}
Basically, what I'm doing here is attaching an accessLog struct to my context inside the accessLogHandler and inside the authHandler I'm reading accessLog from the context and calling accessLog.Set to inform the logger that a userID is present.
Some things I don't like about this approach:
context is immutable, but I'm sticking a mutable struct on it and mutating said struct elsewhere downstream. Feels like a hack.
My authHandler now has a package level dependency on the accessLog package, since I'm type asserting to *AccessLog.
Ideally my authHandler would have some way of informing any part of the request stack about user data without tightly coupling itself to said parts.
Context itself is an interface, so you could create a new logger context in the logger middleware that has the methods you would need to get the behavior you are after.
Something like this:
type Logger struct{}
func (l *Logger) SetLogField(key string, value interface{}) {// set log field }
func (l *Logger) Log(){// log request}
type LoggerCtx struct {
context.Context
*Logger
}
func newAccessLog() *Logger {
return &Logger{}
}
func accessLogHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// create new logger context
ctx := &LoggerCtx{}
ctx.Context = r.Context()
ctx.Logger = newAccessLog()
fn.ServeHTTP(w, r.WithContext(ctx))
// Logs the http access, ommit user info if not set
ctx.Log()
}
}
// pull some junk off the request/cookies/whatever and check if somebody is logged in
func authHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
//Do some authorization
user, err := auth(r)
if err != nil {
//No userId, don't set anything on the accesslogger
w.WriteHeader(http.StatusForbiddend)
return
}
//Success a user is logged in, let's make sure the access logger knows
ctx := r.Context()
// this could be moved - here for clarity
type setLog interface {
SetLogField(string, interface{})
}
if lctx, ok := ctx.(setLog); ok {
lctx.SetLogField("userID", user.ID)
}
fn.ServeHTTP(w, r.WithContext(ctx))
}
}
Related
I'm trying to find a way to add a correlation/request id for logs in our project to make it easier to navigate through them and debug when some issues occur. I found this article. From the example there, there is a middleware to add the correlationID and then retrieve it in some handler function.
Middleware function:
const ContextKeyRequestID ContextKey = "requestID"
func reqIDMiddleware1(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
id := uuid.New()
ctx = context.WithValue(ctx, ContextKeyRequestID, id.String())
r = r.WithContext(ctx)
log.Debugf("Incoming request %s %s %s %s", r.Method, r.RequestURI, r.RemoteAddr, id.String())
next.ServeHTTP(w, r)
log.Debugf("Finished handling http req. %s", id.String())
})
}
Handler:
const LogFieldKeyRequestID = "requestID"
func handleSomeRequest() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
reqIDRaw := ctx.Value(ContextKeyRequestID) // reqIDRaw at this point is of type 'interface{}'
reqID, ok := reqIDRaw.(string)
if !ok {
// handler error
}
// if reached here, reqID is ready to be used
// let's use it with logrus FieldLogger!
logger := log.WithField(LogFieldKeyRequestID, reqID)
// Do something, then log what you did
logger.Debugf("What I just did!")
// Do more, log more. Handle this request seriously
}
}
But I was wondering if there is a way to achieve this without having to refactor all the existing handlers and changing the logging functionality, through some automatic configuration that would add id for each log, in my case our project is quite big, and doing it in the way described above would require a lot of changes.
Have you looked at WithContext and Hooks?
You still have to modify your code but you can centralize some behaviours.
https://go.dev/play/p/4YxJMK6Zl5D
My original question here was flagged as a duplicate of this question. I had no luck implementing it and suspect my problem is misunderstood, so with my question closed, I'm starting fresh with a more specific question.
I'm trying to set a cookie based on a response header from within middleware in request that is reverse proxied.
Here's the workflow:
User requests http://example.com/foo/bar
Go app uses ReverseProxy to proxy that request to http://baz.com
baz.com sets a response header X-FOO
Go app modifies response by setting a MYAPPFOO cookie with the value of the X-FOO response header
The cookie is written to the user's browser
It was suggested that a custom http.ResponseWriter will work, but after trying and searching for more information, it is not clear how to approach this.
Since I'm failing to grasp the concept of a custom ResponseWriter for my use case, I'll post code that demonstrates more precisely what I was trying to do at the point I got stuck:
package main
import (
"github.com/gorilla/mux"
"log"
"net/http"
"net/http/httputil"
"net/url"
)
func setCookie(w http.ResponseWriter, name string, value string) {
...
http.SetCookie(w, &cookie)
}
func handler(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// setCookie() works here
// but I cannot access w.Header().Get("X-FOO")
next.ServeHTTP(w, r)
// I can access w.Header().Get("X-FOO") here
// but setCookie() does not cookie the user's browser
// If I could do it all in one place, this is what I would do:
if r.Method == "POST" && r.URL.String() == "/login" {
foo := w.Header().Get("X-FOO")
setCookie(w, "MYAPPFOO", foo)
}
})
}
func main() {
r := mux.NewRouter()
r.Use(handler)
proxy := httputil.NewSingleHostReverseProxy("https://baz.example.com/")
r.PathPrefix("/").Handler(proxy)
log.Fatal(http.ListenAndServe(":9001", r))
}
As a side note, I was able to make this work with ReverseProxy.ModifyResponse as recommended in the comments of my last question but I'd really like to achieve this with middleware to keep the code that dynamically creates proxies from config clean. (not in the example code)
From the documentation on http.ResponseWriter methods:
(emphasis added)
Header() http.Header:
Changing the header map after a call to WriteHeader (or Write) has no
effect unless the modified headers are trailers.
WriteHeader(statusCode int):
WriteHeader sends an HTTP response header with the provided status
code.
Write([]byte) (int, error):
If WriteHeader has not yet been called, Write calls
WriteHeader(http.StatusOK) before writing the data.
This should highlight the reason why, you can't set a cookie after the next.ServeHTTP(w, r) call, which is that one of the handlers in the middleware chain executed by that call is calling either WriteHeader or Write directly or indirectly.
So to be able set the cookie after the next.ServeHTTP(w, r) call you need to make sure that none of the handlers in the middleware chain calls WriteHeader or Write on the original http.ResponseWriter instance. One way to do this is to wrap the original instance in a custom http.ResponseWriter implementation that will postpone the writing of the response until after you're done with setting the cookie.
For example something like this:
type responsewriter struct {
w http.ResponseWriter
buf bytes.Buffer
code int
}
func (rw *responsewriter) Header() http.Header {
return rw.w.Header()
}
func (rw *responsewriter) WriteHeader(statusCode int) {
rw.code = statusCode
}
func (rw *responsewriter) Write(data []byte) (int, error) {
return rw.buf.Write(data)
}
func (rw *responsewriter) Done() (int64, error) {
if rw.code > 0 {
rw.w.WriteHeader(rw.code)
}
return io.Copy(rw.w, &rw.buf)
}
And you would use it like this in your middleware:
func handler(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
rw := &responsewriter{w: w}
next.ServeHTTP(rw, r)
if r.Method == "POST" && r.URL.String() == "/login" {
foo := rw.Header().Get("X-FOO")
setCookie(rw, "MYAPPFOO", foo)
}
if _, err := rw.Done(); err != nil {
log.Println(err)
}
})
}
I am designing my handlers to return a http.Handler. Here's the design of my handlers:
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
})
}
My middleware is designed to accept an http.Handler and then call the handler once the middleware has finished performing its operations. Here's the design of my middleware:
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
next.ServeHTTP(w, r)
})
}
Considering the design of my middleware and handlers, what is the proper way of passing information from the middleware to the handler? The information that I am trying to pass from my middleware to the handlers is a JSON web token parsed from the request body. If I do not pass the parsed JWT to the handler, then I will need to parse the JWT again in my handlers. Parsing the request body for a JWT in both the middleware and handler seems wasteful. Just in case this information is relevant, I am using the standard net/http library with gorilla mux.
Since you're already using Gorilla take a look at the context package.
(This is nice if you don't want to change your method signatures.)
import (
"github.com/gorilla/context"
)
...
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
// Parse body/get token.
context.Set(r, "token", token)
next.ServeHTTP(w, r)
})
}
...
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token := context.Get(r, "token")
})
}
Update
The Gorilla context package is now in maintenance mode
per the repo:
Note gorilla/context, having been born well before context.Context existed, does not play well with the shallow copying of the request that http.Request.WithContext (added to net/http Go 1.7 onwards) performs.
Using gorilla/context may lead to memory leaks under those conditions, as the pointers to each http.Request become "islanded" and will not be cleaned up when the response is sent.
You should use the http.Request.Context() feature in Go 1.7.
The proper way to pass request scoped data would now be the context package in the standard library.
https://golang.org/pkg/context/
You can access it with request.Context on an http.Request.
A first approach, similar to the question, is in codemodus/chain by Daved.
Package chain aids the composition of Handler wrapper chains that carry request-scoped data.
It uses the notion of Context, coupled with a Context handler:
func ctxHandler(ctx context.Context, w http.ResponseWriter, r *http.Request) {
// ...
if s, ok := getMyString(ctx); ok {
// s = "Send this down the line."
}
// ...
}
Another approach: You can have a look at "Custom Handlers and Avoiding Globals in Go Web Applications", by Matt Silverlock (elithrar). (full example here)
The idea is to define ServeHTTP on a type which include the relevant context.
// We've turned our original appHandler into a struct with two fields:
// - A function type similar to our original handler type (but that now takes an *appContext)
// - An embedded field of type *appContext
type appHandler struct {
*appContext
h func(*appContext, http.ResponseWriter, *http.Request) (int, error)
}
// Our ServeHTTP method is mostly the same, and also has the ability to
// access our *appContext's fields (templates, loggers, etc.) as well.
func (ah appHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Updated to pass ah.appContext as a parameter to our handler type.
status, err := ah.h(ah.appContext, w, r)
if err != nil {
log.Printf("HTTP %d: %q", status, err)
switch status {
case http.StatusNotFound:
http.NotFound(w, r)
// And if we wanted a friendlier error page, we can
// now leverage our context instance - e.g.
// err := ah.renderTemplate(w, "http_404.tmpl", nil)
case http.StatusInternalServerError:
http.Error(w, http.StatusText(status), status)
default:
http.Error(w, http.StatusText(status), status)
}
}
}
In the appContext struct, you would put any data you want to pass around.
I have REST services:
each request has a header with JWT token
each controller get parameters from request (variables, body..) and pass them to data layer
I need to pass JWT token from header of each request into corresponding data layer method like this:
func (a *App) UpdateOrder(_ http.ResponseWriter, r *http.Request) (interface{}, error) {
bodyData := new(models.Order)
err = json.NewDecoder(r.Body).Decode(&bodyData)
if err != nil {
return nil, err
}
user, err := a.Saga.GetUserByToken(r.Header.Get("Authorization")) // here
// error handling ...
a.DbLayer.UpdateOrder(id, bodyData, user) // and there
}
In this case I must write the same code for each controller to get the user by token, and pass this user to database layer explicitly.
Is there a way to pass this user for each request without writing this code in each controller ?
I know about middleware and I can get user by token in my middleware. But how can I pass this user from middleware to corresponding database level method ?
May be I am looking for something like "global variables" for goroutine ? I can get user in my middleware and set it to something like "global variable". I can get the value of this "global variable" in the database layer. But it must be "global variable" for the current web request and concurrent web requests mustn't affect to each other.
Is there a some mechanism in Go, http module or gorilla\mux to implement what I have called "global variables" ?
You are describing contexts.
Originally there was the gorilla context package, which provides a pseudoglobal context object - essentially a map[interface{}]interface{} with a reference intrinsicly available to all players in the middleware/controller/datalayer stack.
See this except from an excellent guide to the package (all credit to the author, Matt Silverlock).
type contextKey int
// Define keys that support equality.
const csrfKey contextKey = 0
const userKey contextKey = 1
var ErrCSRFTokenNotPresent = errors.New("CSRF token not present in the request context.")
// We'll need a helper function like this for every key:type
// combination we store in our context map else we repeat this
// in every middleware/handler that needs to access the value.
func GetCSRFToken(r *http.Request) (string, error) {
val, ok := context.GetOk(r, csrfKey)
if !ok {
return "", ErrCSRFTokenNotPresent
}
token, ok := val.(string)
if !ok {
return "", ErrCSRFTokenNotPresent
}
return token, nil
}
// A bare-bones example
func CSRFMiddleware(h http.Handler) http.Handler {
return func(w http.ResponseWriter, r *http.Request) {
token, err := GetCSRFToken(r)
if err != nil {
http.Error(w, "No good!", http.StatusInternalServerError)
return
}
// The map is global, so we just call the Set function
context.Set(r, csrfKey, token)
h.ServeHTTP(w, r)
}
}
After the gorilla package's inception, a context package was added to the standard library. It's slightly different, in that contexts are no longer pseudoglobal, but instead passed from method to method. Under this, the context comes attached to the initial request - available via request.Context. Layers below the handler can accept a context value as a part of their signature, and read values from it.
Here's a simplified example:
type contextKey string
var (
aPreSharedKey = contextKey("a-preshared-key")
)
func someHandler(w http.ResponseWriter, req *http.Request) {
ctx := context.WithValue(req.Context, aPreSharedKey, req.Header.Get("required-header"))
data, err := someDataLayerFunction(ctx)
if err != nil {
fmt.Fprintf(w, "uhoh", http.StatusBadRequest)
return
}
fmt.Fprintf(w, data, http.StatusOK)
}
func someDataLayerFunction(ctx context.Context) (string, error) {
val, ok := ctx.Value(aPreSharedKey).(string)
if !ok {
return nil, errors.New("required context value missing")
}
return val
}
For more details and a less contrived example, check out google's excellent blog on the context package's use.
I am designing my handlers to return a http.Handler. Here's the design of my handlers:
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
})
}
My middleware is designed to accept an http.Handler and then call the handler once the middleware has finished performing its operations. Here's the design of my middleware:
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
next.ServeHTTP(w, r)
})
}
Considering the design of my middleware and handlers, what is the proper way of passing information from the middleware to the handler? The information that I am trying to pass from my middleware to the handlers is a JSON web token parsed from the request body. If I do not pass the parsed JWT to the handler, then I will need to parse the JWT again in my handlers. Parsing the request body for a JWT in both the middleware and handler seems wasteful. Just in case this information is relevant, I am using the standard net/http library with gorilla mux.
Since you're already using Gorilla take a look at the context package.
(This is nice if you don't want to change your method signatures.)
import (
"github.com/gorilla/context"
)
...
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
// Parse body/get token.
context.Set(r, "token", token)
next.ServeHTTP(w, r)
})
}
...
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token := context.Get(r, "token")
})
}
Update
The Gorilla context package is now in maintenance mode
per the repo:
Note gorilla/context, having been born well before context.Context existed, does not play well with the shallow copying of the request that http.Request.WithContext (added to net/http Go 1.7 onwards) performs.
Using gorilla/context may lead to memory leaks under those conditions, as the pointers to each http.Request become "islanded" and will not be cleaned up when the response is sent.
You should use the http.Request.Context() feature in Go 1.7.
The proper way to pass request scoped data would now be the context package in the standard library.
https://golang.org/pkg/context/
You can access it with request.Context on an http.Request.
A first approach, similar to the question, is in codemodus/chain by Daved.
Package chain aids the composition of Handler wrapper chains that carry request-scoped data.
It uses the notion of Context, coupled with a Context handler:
func ctxHandler(ctx context.Context, w http.ResponseWriter, r *http.Request) {
// ...
if s, ok := getMyString(ctx); ok {
// s = "Send this down the line."
}
// ...
}
Another approach: You can have a look at "Custom Handlers and Avoiding Globals in Go Web Applications", by Matt Silverlock (elithrar). (full example here)
The idea is to define ServeHTTP on a type which include the relevant context.
// We've turned our original appHandler into a struct with two fields:
// - A function type similar to our original handler type (but that now takes an *appContext)
// - An embedded field of type *appContext
type appHandler struct {
*appContext
h func(*appContext, http.ResponseWriter, *http.Request) (int, error)
}
// Our ServeHTTP method is mostly the same, and also has the ability to
// access our *appContext's fields (templates, loggers, etc.) as well.
func (ah appHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Updated to pass ah.appContext as a parameter to our handler type.
status, err := ah.h(ah.appContext, w, r)
if err != nil {
log.Printf("HTTP %d: %q", status, err)
switch status {
case http.StatusNotFound:
http.NotFound(w, r)
// And if we wanted a friendlier error page, we can
// now leverage our context instance - e.g.
// err := ah.renderTemplate(w, "http_404.tmpl", nil)
case http.StatusInternalServerError:
http.Error(w, http.StatusText(status), status)
default:
http.Error(w, http.StatusText(status), status)
}
}
}
In the appContext struct, you would put any data you want to pass around.