Right now i am using Gorilla context package to pass data around in my middlewares & controllers, but what i want to do is pass the data directly to my Pongo2 template so later in my controller i don't have to get the data from the Gorilla context and manually pass it to the template context, for those of you familiar with express.js it would be like
var user = {
name: "Name",
age: 0
}
response.locals = user
Edit: So every pongo2 template needs access to a User object, right now i fetch the user from database using middleware and using Gorilla context pass the data to my controller, from there on to my template on each controller but what i want to do is pass the User object to template from my middleware instead of using Gorilla context.
func UserMiddleware(next http.HandlerFunc) http.HandlerFunc {
return http.HandlerFunc(func(res http.ResponseWriter, req *http.Request) {
user := &User{} // user will normally be fetched from database
context.Set(req, "user", user)
next.ServeHTTP(res, req)
})
}
Then In My Request Handler
func Handler(res http.ResponseWriter, req *http.Request) {
tpl, _ := pongo2.FromFile("view/template.html")
user := context.Get(req, "user").(*User)
data := pongo2.Context{
"user": user,
}
out, _ := tpl.Execute(data)
res.Write([]byte(out))
}
For all of my handlers i have to pass in user to template like that, but i want to pass it in from my middleware so that i don't have to do it in each of my handlers.
invoke MyExecute(req, tpl) instead of tpl.Execute(data)
func MyExecute(req *http.Request, tpl TemplateSet) (string, error){
gorillaObj := context.GetAll(req)
pongoObj := make(map[string]interface{})
for key, value := range gorillaObj {
if str, ok := key.(string); ok{
pongoObj[str] = value
}
}
return tpl.Execute(pongo2.Context(pongoObj))
}
not tested, it should work.
the most problem is that gorilla use map[interface{}]interface{} as store, but pongo use map[string]interface{}, note not to use non-string as key in gorilla context.
Related
I want to trace complete execution of an HTTP request in golang. For any non trivial operation the request will eventually call many different functions. I would like to have logs from the entire execution stack tagged with a unique request id. e.g.
http.Handle("/my-request", myrequesthandler)
func myrequestHandler(w http.ResponseWriter, r *http.Request) {
//debug print1
log.Printf("....")
myfunc()
}
func myfunc() {
//debug print2
log.Printf("....")
}
Here I need a way to identify print1 and print2 as part of same request. It looks like zerolog does have something like this, as described here. Like so:
....
c = c.Append(hlog.RequestIDHandler("req_id", "Request-Id"))
h := c.Then(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hlog.FromRequest(r).Info().
Msg("Something happened")
}))
http.Handle("/", h)
But if I understand it right, it will involve passing request object to each and every function. Is this the idiomatic way to solve this problem? What are the alternatives?
Set a unique id on a context.Context as soon as the request is received, and pass that context down the call stack. This is what contexts are for.
[Context] carries deadlines, cancellation signals, and other request-scoped values across API boundaries and between processes.
Example:
// You could place this in a separate helper package to improve encapsulation
type ctxKey struct{}
func myRequestHandler(w http.ResponseWriter, r *http.Request) {
uniqueID := // generate a unique identifier
// create a context with the http.Request context as parent
ctx := context.WithValue(r.Context(), ctxKey{}, uniqueID)
foo(ctx, ...)
bar(ctx, ...)
}
func foo(ctx context.Context, ...) {
uniqueID := ctx.Value(ctxKey{})
// do something with the unique id
baz(ctx, ...)
}
In particular:
Create the context with *http.Request.Context() as parent. This way if the request is canceled, e.g. due to client disconnection, cancellation will propagate to your sub-context
Consider setting the context value using an unexported struct as key. Unexported structs defined in your package will never conflict with other keys. If you use strings as keys instead, any package could in theory use the same key and overwrite your value (or you could overwrite others' values). YMMV.
Pass your context as the first argument of any function in your call stack, as the package documentation recommends.
For tracing and logging across applications, you might want to look into opentracing. Propagation of tracers is still done with Contexts as outlined above.
you can use context.Context and set request id on it via middleware function.
example:
type requestIDKey struct{}
func requestIDSetter(next http.HandlerFunc) http.HandlerFunc {
return func(rw http.ResponseWriter, r *http.Request) {
// use provided request id from incoming request if any
reqID := r.Header.Get("X-Request-ID")
if reqID == "" {
// or use some generated string
reqID = uuid.New().String()
}
ctx := context.WithValue(r.Context(), requestIDKey{}, reqID)
next(rw, r.WithContext(ctx))
}
}
then you need to modify your logger to accept context.Context
example:
func printfWithContext(ctx context.Context, format string, v ...interface{}) {
reqID := ctx.Value(requestIDKey{}).(string)
log.Printf(reqID+": "+format, v...)
}
and finally apply to your code
http.HandleFunc("/my-request", requestIDSetter(myrequestHandler))
func myrequestHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
//debug print1
printfWithContext(ctx, "....1")
myfunc(ctx)
}
func myfunc(ctx context.Context) {
//debug print2
printfWithContext(ctx, "....2")
}
I'm trying to write a "Binder" middleware that will validate any request query using a struct type with gin bindings/validators
So for example, let's say I have an endpoint group called /api/subject which requires the query string to have a subject code and an ID that will be validated using the following struct (called entity.Subject):
type Subject struct {
Code string `binding:"required,alphanum"`
ID string `binding:"required,alphanum,len=4"`
}
That's just one example, but I'd like to be able to pass any struct type to this middleware, because I'd like to access the query data on future handlers without worrying about query validation.
So I tried something like this:
func Binder(t reflect.Type) gin.HandlerFunc {
return func(c *gin.Context) {
obj := reflect.New(t).Elem().Interface()
if err := c.BindQuery(&obj); err != nil {
c.AbortWithStatus(http.StatusBadRequest)
return
}
c.Set(t.Name(), obj)
}
}
And added this middleware like so:
apiGroup := router.Group("/api")
{
// other subgroups/endpoints
// ...
subjectGroup := apiGroup.Group("/subject", middleware.Binder(reflect.TypeOf(entity.Subject{})))
}
And later on, in another handler function, let's say GetSubject, I want to access the subject data passed by doing c.MustGet("Subject").(entity.Subject)
But this isn't working =(, when I print obj, it's just an empty interface, how would I do this?
I managed to do something similar!
I created the following middleware
var allowedTypes = []binding.Binding{
binding.Query,
binding.Form,
binding.FormPost,
binding.FormMultipart,
}
func Bind(name string, data interface{}, bindingType binding.Binding) gin.HandlerFunc {
return func(ctx *gin.Context) {
ok := false
for _, b := range allowedTypes {
if b == bindingType {
ok = true
}
}
if !ok {
ctx.AbortWithError(
http.StatusInternalServerError,
fmt.Errorf("Bind function only allows %v\n", allowedTypes),
)
}
_ = ctx.MustBindWith(data, bindingType)
ctx.Set(name, data)
}
}
Remember to pass a pointer to your desired type in the call, like so:
router.GET("/something", Bind("Object", &myObject, binding.Query))
I restricted only to a few binding types because they allow ShouldBind to be called multiple times, whereas JSON, XML and others consume the Request body.
This way you can pass multiple Bind middlewares and if the validation fails it automatically aborts with http.StatusBadRequest
I have REST services:
each request has a header with JWT token
each controller get parameters from request (variables, body..) and pass them to data layer
I need to pass JWT token from header of each request into corresponding data layer method like this:
func (a *App) UpdateOrder(_ http.ResponseWriter, r *http.Request) (interface{}, error) {
bodyData := new(models.Order)
err = json.NewDecoder(r.Body).Decode(&bodyData)
if err != nil {
return nil, err
}
user, err := a.Saga.GetUserByToken(r.Header.Get("Authorization")) // here
// error handling ...
a.DbLayer.UpdateOrder(id, bodyData, user) // and there
}
In this case I must write the same code for each controller to get the user by token, and pass this user to database layer explicitly.
Is there a way to pass this user for each request without writing this code in each controller ?
I know about middleware and I can get user by token in my middleware. But how can I pass this user from middleware to corresponding database level method ?
May be I am looking for something like "global variables" for goroutine ? I can get user in my middleware and set it to something like "global variable". I can get the value of this "global variable" in the database layer. But it must be "global variable" for the current web request and concurrent web requests mustn't affect to each other.
Is there a some mechanism in Go, http module or gorilla\mux to implement what I have called "global variables" ?
You are describing contexts.
Originally there was the gorilla context package, which provides a pseudoglobal context object - essentially a map[interface{}]interface{} with a reference intrinsicly available to all players in the middleware/controller/datalayer stack.
See this except from an excellent guide to the package (all credit to the author, Matt Silverlock).
type contextKey int
// Define keys that support equality.
const csrfKey contextKey = 0
const userKey contextKey = 1
var ErrCSRFTokenNotPresent = errors.New("CSRF token not present in the request context.")
// We'll need a helper function like this for every key:type
// combination we store in our context map else we repeat this
// in every middleware/handler that needs to access the value.
func GetCSRFToken(r *http.Request) (string, error) {
val, ok := context.GetOk(r, csrfKey)
if !ok {
return "", ErrCSRFTokenNotPresent
}
token, ok := val.(string)
if !ok {
return "", ErrCSRFTokenNotPresent
}
return token, nil
}
// A bare-bones example
func CSRFMiddleware(h http.Handler) http.Handler {
return func(w http.ResponseWriter, r *http.Request) {
token, err := GetCSRFToken(r)
if err != nil {
http.Error(w, "No good!", http.StatusInternalServerError)
return
}
// The map is global, so we just call the Set function
context.Set(r, csrfKey, token)
h.ServeHTTP(w, r)
}
}
After the gorilla package's inception, a context package was added to the standard library. It's slightly different, in that contexts are no longer pseudoglobal, but instead passed from method to method. Under this, the context comes attached to the initial request - available via request.Context. Layers below the handler can accept a context value as a part of their signature, and read values from it.
Here's a simplified example:
type contextKey string
var (
aPreSharedKey = contextKey("a-preshared-key")
)
func someHandler(w http.ResponseWriter, req *http.Request) {
ctx := context.WithValue(req.Context, aPreSharedKey, req.Header.Get("required-header"))
data, err := someDataLayerFunction(ctx)
if err != nil {
fmt.Fprintf(w, "uhoh", http.StatusBadRequest)
return
}
fmt.Fprintf(w, data, http.StatusOK)
}
func someDataLayerFunction(ctx context.Context) (string, error) {
val, ok := ctx.Value(aPreSharedKey).(string)
if !ok {
return nil, errors.New("required context value missing")
}
return val
}
For more details and a less contrived example, check out google's excellent blog on the context package's use.
My basic main setup:
muxRouter := mux.NewRouter()
v1Router.Router(muxRouter.PathPrefix("/v1").Subrouter())
http.Handle("/", muxRouter)
n := negroni.Classic()
n.Use(negroni.HandlerFunc(apiRouter.Middleware))
n.UseHandler(muxRouter)
s := &http.Server{
Addr: ":6060",
Handler: n,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
log.Fatal(s.ListenAndServe())
Inside the apiRouter.Middleware I have set the following context:
context.Set(req, helperKeys.DomainName, "some-value")
However, in some handlerFunc within v1Router.Router when trying to Get the context's value, the result is nil:
domain := context.Get(req, helperKeys.DomainName)
fmt.Println("DomainName", domain)
Prints: DomainName <nil>
I know that the Set method is correct as getting the value immediately after setting it in the apiRouter.Middleware will return the correct string value.
I ended up using Go 1.7's built in Context:
context.Set(req, helperKeys.DomainName, "some-value")
// Replaced with:
ctx := req.Context()
ctx = context.WithValue(ctx, helperKeys.DomainName, "some-value")
req = req.WithContext(ctx)
AND
domain := context.Get(req, helperKeys.DomainName)
// Replaced with:
domain := req.Context().Value(helperKeys.DomainName).(string)
Based on your answer, it looks like you are trying to store a database in the context. I wouldn't suggest doing that. Instead try something like this:
type Something struct {
DB *sql.DB // or some other DB object
}
func (s *Something) CreateUser(w http.ResponseWriter, r *http.Request) {
// use s.DB to access the database
fmt.Fprintln(w, "Created a user...")
}
func main() {
db := ...
s := Something{db}
http.HandleFunc("/", s.CreateUser)
// ... everything else is pretty much like normal.
}
This gives your handlers access to the database while not having to set it on the context every single time. Context values should be reserved for things that you can't possibly set until runtime. For example, a request ID that is specific to that web request. Things that outlive the request don't typically fall into this category, and your DB connection will outlive the request.
If you do actually need context values, you should:
Use getters and setters that are typed
"packages should define keys as an unexported type to avoid collisions." - From the Go source code
An example of this is shown below, and I talk more about context values in general in this blog post:
type userCtxKeyType string
const userCtxKey userCtxKeyType = "user"
func WithUser(ctx context.Context, user *User) context.Context {
return context.WithValue(ctx, userCtxKey, user)
}
func GetUser(ctx context.Context) *User {
user, ok := ctx.Value(userCtxKey).(*User)
if !ok {
// Log this issue
return nil
}
return user
}
I'm using Gin, https://gin-gonic.github.io/gin/, to build a simple RESTful JSON API with Golang.
The routes are setup with something like this:
func testRouteHandler(c *gin.Context) {
// do smth
}
func main() {
router := gin.Default()
router.GET("/test", testRouteHandler)
router.Run(":8080")
}
My question is how can I pass down an argument to the testRouteHandler function? For example a common database connection could be something that one wants to reuse among routes.
Is the best way to have this in a global variable? Or is there some way in Go to pass along an extra variable to the testRouteHandler function? Are there optional arguments for functions in Go?
PS. I'm just getting started in learning Go, so could be something obvious that I'm missing :)
I would avoid stuffing 'application scoped' dependencies (e.g. a DB connection pool) into a request context. Your two 'easiest' options are:
Make it a global. This is OK for smaller projects, and *sql.DB is thread-safe.
Pass it explicitly in a closure so that the return type satisfies gin.HandlerFunc
e.g.
// SomeHandler returns a `func(*gin.Context)` to satisfy Gin's router methods
// db could turn into an 'Env' struct that encapsulates all of your
// app dependencies - e.g. DB, logger, env vars, etc.
func SomeHandler(db *sql.DB) gin.HandlerFunc {
fn := func(c *gin.Context) {
// Your handler code goes in here - e.g.
rows, err := db.Query(...)
c.String(200, results)
}
return gin.HandlerFunc(fn)
}
func main() {
db, err := sql.Open(...)
// handle the error
router := gin.Default()
router.GET("/test", SomeHandler(db))
router.Run(":8080")
}
Using the link i posted on comments, I have created a simple example.
package main
import (
"log"
"github.com/gin-gonic/gin"
"github.com/jinzhu/gorm"
_ "github.com/mattn/go-sqlite3"
)
// ApiMiddleware will add the db connection to the context
func ApiMiddleware(db gorm.DB) gin.HandlerFunc {
return func(c *gin.Context) {
c.Set("databaseConn", db)
c.Next()
}
}
func main() {
r := gin.New()
// In this example, I'll open the db connection here...
// In your code you would probably do it somewhere else
db, err := gorm.Open("sqlite3", "./example.db")
if err != nil {
log.Fatal(err)
}
r.Use(ApiMiddleware(db))
r.GET("/api", func(c *gin.Context) {
// Don't forget type assertion when getting the connection from context.
dbConn, ok := c.MustGet("databaseConn").(gorm.DB)
if !ok {
// handle error here...
}
// do your thing here...
})
r.Run(":8080")
}
This is just a simple POC. But i believe it's a start.
Hope it helps.
Late to the party, so far here is my proposal. Encapsulate methods into the object with private/public vars in it:
package main
import (
"log"
"github.com/gin-gonic/gin"
"github.com/jinzhu/gorm"
_ "github.com/mattn/go-sqlite3"
)
type HandlerA struct {
Db gorm.DB
}
func (this *HandlerA) Get(c *gin.Context) {
log.Info("[%#f]", this.Db)
// do your thing here...
}
func main() {
r := gin.New()
// Init, should be separate, but it's ok for this sample:
db, err := gorm.Open("sqlite3", "./example.db")
if err != nil {
log.Fatal(err)
}
Obj := new(HandlerA)
Obj.Db = db // Or init inside Object
r := gin.New()
Group := r.Group("api/v1/")
{
Group.GET("/storage", Obj.Get)
}
r.Run(":8080")
}
Handler closures are a good option, but that works best when the argument is used in that handler alone.
If you have route groups, or long handler chains, where the same argument is needed in multiple places, you should set values into the Gin context.
You can use function literals, or named functions that return gin.HandlerFunc to do that in a clean way.
Example injecting configs into a router group:
Middleware package:
func Configs(conf APIV1Config) gin.HandlerFunc {
return func(c *gin.Context) {
c.Set("configKey", conf) // key could be an unexported struct to ensure uniqueness
}
}
Router:
conf := APIV1Config{/* some api configs */}
// makes conf available to all routes in this group
g := r.Group("/api/v1", middleware.Configs(conf))
{
// ... routes that all need API V1 configs
}
This is also easily unit-testable. Assuming that you test the single handlers, you can set the necessary values into the mock context:
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
c.Set("configKey", /* mock configs */)
apiV1FooHandler(c)
Now in the case of application-scoped dependencies (db connections, remote clients, ...), I agree that setting these directly into the Gin context is a poor solution.
What you should do then, is to inject providers into the Gin context, using the pattern outlined above:
Middleware package:
// provider could be an interface for easy mocking
func DBProvider(provider database.Provider) gin.HandlerFunc {
return func(c *gin.Context) {
c.Set("providerKey", provider)
}
}
Router:
dbProvider := /* init provider with db connection */
r.Use(DBProvider(dbProvider)) // global middleware
// or
g := r.Group("/users", DBProvider(dbProvider)) // users group only
Handler (you can greatly reduce the boilerplate code by putting these context getters in some helper function):
// helper function
func GetDB(c *gin.Context) *sql.DB {
provider := c.MustGet("providerKey").(database.Provider)
return provider.GetConn()
}
func createUserHandler(c *gin.Context) {
db := GetDB(c) // same in all other handlers
// ...
}
I like wildneuro's example but would do a one liner to setup the handler
package main
import (
"log"
"github.com/gin-gonic/gin"
"github.com/jinzhu/gorm"
_ "github.com/mattn/go-sqlite3"
)
type HandlerA struct {
Db gorm.DB
}
func (this *HandlerA) Get(c *gin.Context) {
log.Info("[%#f]", this.Db)
// do your thing here...
}
func main() {
r := gin.New()
// Init, should be separate, but it's ok for this sample:
db, err := gorm.Open("sqlite3", "./example.db")
if err != nil {
log.Fatal(err)
}
r := gin.New()
Group := r.Group("api/v1/")
{
Group.GET("/storage", (&HandlerA{Db: db}).Get)
}
r.Run(":8080")
}
Let me try to explain in detail so that you won't get confused.
Depending on the incoming route, you want to call a controller function. Lets say your incoming route is /books and your controller is BooksController
Your BooksController will try to fetch the books from the database and returns a response.
Now, you want this a handler within your BooksController so that you can access database.
I would do something like this. Let's assume that you are using dynamoDB and the aws sdk provides *dynamodb.DynamoDB. Depending on your db, change this variable.
Create a struct as below.
type serviceConnection struct {
db *dynamoDB.DynamoDB
// You can have all services declared here
// which you want to use it in your controller
}
In your main function, get the db connection information. Let's say you already have a function initDatabaseConnection which returns a handler to db, something like below.
db := initDatabaseConnection() -> returns *dynamodb.DynamoDB
Set db to a struct variable.
conn := new(serviceConnection)
conn.db = db
Call the gin request method with a receiver handler as below.
r := gin.Default()
r.GET("/books", conn.BooksController)
As you see, the gin handler is a controller method which has your struct instance as a receiver.
Now, create a controller method with serviceConnection struct receiver.
func (conn *serviceConnection) BooksController(c *gin.Context) {
books := getBooks(conn.db)
}
As you see here, you have access to all the serviceConnection struct variables and you can use them in your controller.
Alright, I have given you a simple example. It should work. You can extend it as per your need
func main() {
router := gin.Default()
router.GET("/test/:id/:name", testRouteHandler)
router.Run(":8080")
}
func testRouteHandler(c *gin.Context) {
id := c.Params.ByName("id")
name := c.Params.ByName("name")
}
Now you will have to call your handler as below
http://localhost:8080/test/1/myname