Dealing with middleware that requires access to the database in Go - go

I am having an issue with creating a middleware that will be chained to other routes and requires access to the database and am not sure how to approach this problem.
I store all of my app context, including the database in a struct called AppContext. I want to create a function handler that looks something like this:
func SomeHandler(appC *AppContext, next http.Handler) http.Handler {
fn := func(w http.ResponseWriter, r *http.Request) {
// Access the database using appC.db
// Logic that requires access to the database.
next.ServeHTTP(w, r)
}
return http.HandlerFunc(fn)
}
}
In main.go, I have tried:
someHandler := middleware.SomeHandler(&appC)
However, I get the error not enough arguments in call to middleware.SomeHandler. What would be the best way to approach this problem?

The error you're getting is due to not providing the second argument, next http.Handler.
In the case of how to go about the middleware, I recommend taking a look at the implementation of the http.ServeMux https://golang.org/src/net/http/server.go?s=57308:57433#L1890 it essentially does what you're trying to do (and then some) for routing though. So using a http.Handler struct might be more easy here than using a Handler function, this way the Handler(s) you'd have as the next http.Handler argument in your function are just a field within the struct that the parent handler can call from within its ServeHTTP().
So to wrap up my point, you might want to use a struct that implements the http.Handler interface. That way it can have child handlers and the DB access. That way you don't have to keep passing this AppContext either.

I'd not do a premature setup of the context. It is specifically there to be request scoped.
I'd rather create a small and dedicated middleware for passing a database session out of the pool into the context and retrieve said session created for the request in the main handler.
func DBSession(sess *mgo.Session, next http.Handler) http.Handler {
return http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
// Create a new session for the scope of the request
s := sess.Copy()
// Make it close when exiting scope
defer s.Close()
// Add the new session (actually a pointer to it)
// acessible via the name "_sess"
ctx = context.WithValue(r.Context(), "_sess", s)
// execute the next handler
next(w, r.WithContext(ctx))
})
}
Now you can use this middleware in you main.go
package main
import (
"net/http"
"gopkg.in/mgo.v2"
)
func sayHello(w http.ResponseWriter, r *http.Request) http.Handler{
return http.HandlerFunc(
func (w http.ResponseWriter, r *http.Request) {
s := r.Context().Value("_sess").(*mgo.Session)
// Do something with your database here.
}
)
}
func main() {
// Ignoring err for brevity
sess, _ := mgo.Dial("localhost:27017")
// Magic happens here: the sayHello Handler is wrapped in the
// DBSession middleware, which makes the session available
// as context param _sess and closes the session after sayHello
// executed. No clutter in your controller for setting up and tearing down
// the session, no way you can forget to close it.
http.Handle("/", middleware.DBSession(sess, sayHello))
}

Related

Tracing golang http request

I want to trace complete execution of an HTTP request in golang. For any non trivial operation the request will eventually call many different functions. I would like to have logs from the entire execution stack tagged with a unique request id. e.g.
http.Handle("/my-request", myrequesthandler)
func myrequestHandler(w http.ResponseWriter, r *http.Request) {
//debug print1
log.Printf("....")
myfunc()
}
func myfunc() {
//debug print2
log.Printf("....")
}
Here I need a way to identify print1 and print2 as part of same request. It looks like zerolog does have something like this, as described here. Like so:
....
c = c.Append(hlog.RequestIDHandler("req_id", "Request-Id"))
h := c.Then(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hlog.FromRequest(r).Info().
Msg("Something happened")
}))
http.Handle("/", h)
But if I understand it right, it will involve passing request object to each and every function. Is this the idiomatic way to solve this problem? What are the alternatives?
Set a unique id on a context.Context as soon as the request is received, and pass that context down the call stack. This is what contexts are for.
[Context] carries deadlines, cancellation signals, and other request-scoped values across API boundaries and between processes.
Example:
// You could place this in a separate helper package to improve encapsulation
type ctxKey struct{}
func myRequestHandler(w http.ResponseWriter, r *http.Request) {
uniqueID := // generate a unique identifier
// create a context with the http.Request context as parent
ctx := context.WithValue(r.Context(), ctxKey{}, uniqueID)
foo(ctx, ...)
bar(ctx, ...)
}
func foo(ctx context.Context, ...) {
uniqueID := ctx.Value(ctxKey{})
// do something with the unique id
baz(ctx, ...)
}
In particular:
Create the context with *http.Request.Context() as parent. This way if the request is canceled, e.g. due to client disconnection, cancellation will propagate to your sub-context
Consider setting the context value using an unexported struct as key. Unexported structs defined in your package will never conflict with other keys. If you use strings as keys instead, any package could in theory use the same key and overwrite your value (or you could overwrite others' values). YMMV.
Pass your context as the first argument of any function in your call stack, as the package documentation recommends.
For tracing and logging across applications, you might want to look into opentracing. Propagation of tracers is still done with Contexts as outlined above.
you can use context.Context and set request id on it via middleware function.
example:
type requestIDKey struct{}
func requestIDSetter(next http.HandlerFunc) http.HandlerFunc {
return func(rw http.ResponseWriter, r *http.Request) {
// use provided request id from incoming request if any
reqID := r.Header.Get("X-Request-ID")
if reqID == "" {
// or use some generated string
reqID = uuid.New().String()
}
ctx := context.WithValue(r.Context(), requestIDKey{}, reqID)
next(rw, r.WithContext(ctx))
}
}
then you need to modify your logger to accept context.Context
example:
func printfWithContext(ctx context.Context, format string, v ...interface{}) {
reqID := ctx.Value(requestIDKey{}).(string)
log.Printf(reqID+": "+format, v...)
}
and finally apply to your code
http.HandleFunc("/my-request", requestIDSetter(myrequestHandler))
func myrequestHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
//debug print1
printfWithContext(ctx, "....1")
myfunc(ctx)
}
func myfunc(ctx context.Context) {
//debug print2
printfWithContext(ctx, "....2")
}

gorilla mux middleware: wrapping a handler

I need to wrap the GetAssetsCompute function inside a middleware
r.Handle("/api/v1/assets/ComputeBlade", GetAssetsCompute(assetService)).Methods("GET")
func GetAssetsCompute(assetService ServiceType) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// stuff here
})
}
but because middlewares take HTTP handlers as an argument and my function is not a handler, I can't.
I was thinking of doing something like this.
func GetAssetsCompute(assetService ServiceType) http.Handler {
return MyMiddleware(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// stuff here
}))
}
func MyMiddleware(next http.Handler) http.Handler {
}
Is this correct? Or is there a better way to do this.
Also inside the middleware, I need to access the URL endpoint, do some processing and store this processed value and then again access that in the main handler. How can I do that?
EDIT: I want to apply this middleware to only a subset(>1) of endpoints I have. Not all
I also require the assetService variable used in GetAssetsCompute(assetService ServiceType) function in the handler. So, I need this closure too.
It seems you are trying to do 2 things. 1 - Apply a middleware to only some of your request handlers. 2 - Pass data from your middleware to your request handlers.
For the first one, I can think of three options. The first is what you are doing now, having a Middleware function in which you wrap your handler functions when you pass them to r.Handle. Pseudocode:
r.Handle("/path1", Mware(Handler1())).Methods("GET")
r.Handle("/path2", Mware(Handler2())).Methods("GET")
r.Handle("/path3-nomiddleware", Handler3()).Methods("GET")
The second thing you could do is to add code to your middleware to filter based on URI path and then register your middleware using r.Use. Pseudocode:
const mwarePaths []string = ...
func Mware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.RequestURI is in mwarePaths {
// do the middleware
}
}
}
r.Use(Mware)
Thirdly, you could put the code in a function which you call directly in your handlers and not register it like a middleware. Pseudocode:
func myUtil(w http.ResponseWriter, r *http.Request){ ... }
func GetAssetsCompute(assetService ServiceType) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
myUtil(w, r)
// stuff here
})
}
For the second thing - passing data from middleware to request handlers - here are some ideas.
First, if you go with the regular-function, no-middleware setup above, this problem disappears because anything you need in your handler can simply be a return value from your function.
If you do use a middleware, you can use the context library (also from gorilla) to tie variables to an http.Request instance for passing to your handler: http://www.gorillatoolkit.org/pkg/context . Using that looks like this:
import "github.com/gorilla/context"
func middleware(...) {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
context.Set(r, "myKey", "bar")
}
}
func handler(w http.ResponseWriter, r *http.Request) {
val, ok := context.GetOk(r, "myKey") // returns "bar", true
}
Which of these options you choose to use is up to you (you know your needs). But, as mentioned in the comments, a good rule of thumb would be that code which handles unrelated concerns to what your request handlers do can be middleware. Code which handles concerns that are directly related to what your request handlers are doing can go directly in the handlers.

Passing along data with request [duplicate]

I am designing my handlers to return a http.Handler. Here's the design of my handlers:
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
})
}
My middleware is designed to accept an http.Handler and then call the handler once the middleware has finished performing its operations. Here's the design of my middleware:
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
next.ServeHTTP(w, r)
})
}
Considering the design of my middleware and handlers, what is the proper way of passing information from the middleware to the handler? The information that I am trying to pass from my middleware to the handlers is a JSON web token parsed from the request body. If I do not pass the parsed JWT to the handler, then I will need to parse the JWT again in my handlers. Parsing the request body for a JWT in both the middleware and handler seems wasteful. Just in case this information is relevant, I am using the standard net/http library with gorilla mux.
Since you're already using Gorilla take a look at the context package.
(This is nice if you don't want to change your method signatures.)
import (
"github.com/gorilla/context"
)
...
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
// Parse body/get token.
context.Set(r, "token", token)
next.ServeHTTP(w, r)
})
}
...
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token := context.Get(r, "token")
})
}
Update
The Gorilla context package is now in maintenance mode
per the repo:
Note gorilla/context, having been born well before context.Context existed, does not play well with the shallow copying of the request that http.Request.WithContext (added to net/http Go 1.7 onwards) performs.
Using gorilla/context may lead to memory leaks under those conditions, as the pointers to each http.Request become "islanded" and will not be cleaned up when the response is sent.
You should use the http.Request.Context() feature in Go 1.7.
The proper way to pass request scoped data would now be the context package in the standard library.
https://golang.org/pkg/context/
You can access it with request.Context on an http.Request.
A first approach, similar to the question, is in codemodus/chain by Daved.
Package chain aids the composition of Handler wrapper chains that carry request-scoped data.
It uses the notion of Context, coupled with a Context handler:
func ctxHandler(ctx context.Context, w http.ResponseWriter, r *http.Request) {
// ...
if s, ok := getMyString(ctx); ok {
// s = "Send this down the line."
}
// ...
}
Another approach: You can have a look at "Custom Handlers and Avoiding Globals in Go Web Applications", by Matt Silverlock (elithrar). (full example here)
The idea is to define ServeHTTP on a type which include the relevant context.
// We've turned our original appHandler into a struct with two fields:
// - A function type similar to our original handler type (but that now takes an *appContext)
// - An embedded field of type *appContext
type appHandler struct {
*appContext
h func(*appContext, http.ResponseWriter, *http.Request) (int, error)
}
// Our ServeHTTP method is mostly the same, and also has the ability to
// access our *appContext's fields (templates, loggers, etc.) as well.
func (ah appHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Updated to pass ah.appContext as a parameter to our handler type.
status, err := ah.h(ah.appContext, w, r)
if err != nil {
log.Printf("HTTP %d: %q", status, err)
switch status {
case http.StatusNotFound:
http.NotFound(w, r)
// And if we wanted a friendlier error page, we can
// now leverage our context instance - e.g.
// err := ah.renderTemplate(w, "http_404.tmpl", nil)
case http.StatusInternalServerError:
http.Error(w, http.StatusText(status), status)
default:
http.Error(w, http.StatusText(status), status)
}
}
}
In the appContext struct, you would put any data you want to pass around.

HTTP handler function

I saw some http handler function declarations are varied.
Two of them I found are the standard function and the one returning anonymous function inside the handler.
For example:
Using standard way:
func helloworld(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "Hello World")
}
This the most straight way to declare a handler for an http api.
Another way is using anonym/closure function inside the handler function:
func helloworld2() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request){
fmt.Fprintln(w, "Hello World")
})
}
What are the differences and the benefit? When to use one of them? What's the best practice?
Pattern
func Middleware(next http.Handler) http.Handler{
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Do something
next.ServeHTTP(w, r)
})
}
often used to construct middleware chain like
http.Handle("/", middlewareOne(middlewareTwo(finalHandler)))
Returning an anonymous function is the only way to work with handlers that require additional arguments, by returning a closure. Example:
func fooHandler(db *someDatabase) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// do something with `db` variable
}
}
Otherwise, there's typically no practical difference between the approaches. One may choose to use the anonymous function universally for consistency.
One of the most popular source of information about structure returning anonymous functions is a blog post from Mat Ryer How I write HTTP services after eight years
I sure it will be good to provide some quotes from his article here:
... handler functions don’t actually handle the requests, they return a function that does. This gives us a closure environment in which our handler can operate:
func (s *server) handleSomething() http.HandlerFunc {
thing := prepareThing()
return func(w http.ResponseWriter, r *http.Request) {
// use thing
}
}
The prepareThing is called only once, so you can use it to do one-time
per-handler initialisation, and then use the thing in the handler.
Also,
If an endpoint has its own request and response types, usually they’re only useful for that particular handler. If that’s the case, you can define them inside the function.
func (s *server) handleSomething() http.HandlerFunc {
// you have these handy structs always visible to your handler and eyes
// and invisible to code that don't use them
type request struct {
Name string
}
type response struct {
Greeting string `json:"greeting"`
}
return func(w http.ResponseWriter, r *http.Request) {
// decode into request struct
// validate
// call business-logic
// encode response from business-logic into response struct
}
}
In practice, writing RESTy APIs you have handler named after resource, e.g. you have /maps resource and appropriate handler struct mapsHandler with injected dependencies (repositories, services containing some business-logic, loggers) into it. But sometimes you will also need to pass an additional dependency exclusively per one handle and suddenly realized that handler has strict signature, so you should wrap it. Then you have something like this
// RESTy routes for "maps" resource
router.Route("/maps", func(r chi.Router) {
adHocDependency := newAdHocDependency(options)
r.Post("/", mapsHandler.handleCreateMap(adHocDependency))
})
making your ad hoc dependency visible to your handler.
Hope it helps!

How can I pass data from middleware to handlers?

I am designing my handlers to return a http.Handler. Here's the design of my handlers:
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
})
}
My middleware is designed to accept an http.Handler and then call the handler once the middleware has finished performing its operations. Here's the design of my middleware:
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
next.ServeHTTP(w, r)
})
}
Considering the design of my middleware and handlers, what is the proper way of passing information from the middleware to the handler? The information that I am trying to pass from my middleware to the handlers is a JSON web token parsed from the request body. If I do not pass the parsed JWT to the handler, then I will need to parse the JWT again in my handlers. Parsing the request body for a JWT in both the middleware and handler seems wasteful. Just in case this information is relevant, I am using the standard net/http library with gorilla mux.
Since you're already using Gorilla take a look at the context package.
(This is nice if you don't want to change your method signatures.)
import (
"github.com/gorilla/context"
)
...
func Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Middleware operations
// Parse body/get token.
context.Set(r, "token", token)
next.ServeHTTP(w, r)
})
}
...
func Handler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
token := context.Get(r, "token")
})
}
Update
The Gorilla context package is now in maintenance mode
per the repo:
Note gorilla/context, having been born well before context.Context existed, does not play well with the shallow copying of the request that http.Request.WithContext (added to net/http Go 1.7 onwards) performs.
Using gorilla/context may lead to memory leaks under those conditions, as the pointers to each http.Request become "islanded" and will not be cleaned up when the response is sent.
You should use the http.Request.Context() feature in Go 1.7.
The proper way to pass request scoped data would now be the context package in the standard library.
https://golang.org/pkg/context/
You can access it with request.Context on an http.Request.
A first approach, similar to the question, is in codemodus/chain by Daved.
Package chain aids the composition of Handler wrapper chains that carry request-scoped data.
It uses the notion of Context, coupled with a Context handler:
func ctxHandler(ctx context.Context, w http.ResponseWriter, r *http.Request) {
// ...
if s, ok := getMyString(ctx); ok {
// s = "Send this down the line."
}
// ...
}
Another approach: You can have a look at "Custom Handlers and Avoiding Globals in Go Web Applications", by Matt Silverlock (elithrar). (full example here)
The idea is to define ServeHTTP on a type which include the relevant context.
// We've turned our original appHandler into a struct with two fields:
// - A function type similar to our original handler type (but that now takes an *appContext)
// - An embedded field of type *appContext
type appHandler struct {
*appContext
h func(*appContext, http.ResponseWriter, *http.Request) (int, error)
}
// Our ServeHTTP method is mostly the same, and also has the ability to
// access our *appContext's fields (templates, loggers, etc.) as well.
func (ah appHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Updated to pass ah.appContext as a parameter to our handler type.
status, err := ah.h(ah.appContext, w, r)
if err != nil {
log.Printf("HTTP %d: %q", status, err)
switch status {
case http.StatusNotFound:
http.NotFound(w, r)
// And if we wanted a friendlier error page, we can
// now leverage our context instance - e.g.
// err := ah.renderTemplate(w, "http_404.tmpl", nil)
case http.StatusInternalServerError:
http.Error(w, http.StatusText(status), status)
default:
http.Error(w, http.StatusText(status), status)
}
}
}
In the appContext struct, you would put any data you want to pass around.

Resources