Go and Pointers in http middleware - go

I'm trying to log some data on my web server, so I created a loggingMiddleware that serves the next request and then logs the data, I thought this way I would have all the necessary data inside the r *http.Request pointer
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// call next handler
next.ServeHTTP(o, r)
// get requestID
reqID := middleware.GetReqID(r.Context())
log.WithFields(
log.Fields{
"request_id": reqID, // drops requestID
"method": r.Method, // http method
"remote_address": r.RemoteAddr, // remote address
"url": r.URL.String(), // url used by the client to access the service
"referer": r.Referer(), // Referer header if present
"user_agent": r.UserAgent(), // User-Agent header
"content_length": r.ContentLength, // Content-Length header if present"
},
).Info()
})
However for the RequestID this is true only if the RequestID middleware is mounted before the loggingMiddleware
Non-Working
...
// get a new chi rotuter
router = chi.NewRouter()
// MIDDLEWARES
// log
// use logrus to log requests
router.Use(LoggingMiddleware)
// requestID
// Generate a unique id for every request
// ids are grouped based on client hostname
router.Use(middleware.RequestID)
...
Working
...
// get a new chi rotuter
router = chi.NewRouter()
// MIDDLEWARES
// requestID
// Generate a unique id for every request
// ids are grouped based on client hostname
router.Use(middleware.RequestID)
// log
// use logrus to log requests
router.Use(LoggingMiddleware)
...
Is this the expected behavior? Should the r *http.Request pointer point to the "updated" version of the request? Is there a way to get around this?
Because if I want, for example, extract a username from a JWT token and put it in the r.Context() so I can log it later, this would require a separate middleware to be mounted before the loggingMiddleware.
Sorry for my english, please ask if there's something not clear.
Thanks

middleware.RequestID adds the request ID to the request context by using http.Request.WithContext:
ctx = context.WithValue(ctx, RequestIDKey, requestID)
next.ServeHTTP(w, r.WithContext(ctx))
Per the documentation:
WithContext returns a shallow copy of r with its context changed to
ctx. The provided ctx must be non-nil.
Therefore, because it "returns a shallow copy of r", and it is (a pointer to) this shallow copy which it passes to next.ServeHTTP, if middleware.RequestID is mounted second, then the r in LoggingMiddleware is pointing to a different *http.Request than the one which contains the modified context, and so its context will not contain the request ID. If, on the other hand, middleware.RequestID is mounted first, then LoggingMiddleware will receive a pointer to the shallow copy that r.WithContext returned, and everything will work as expected.

Related

how to pass parameter of the destination to middleware in gin/golang

My problem in short is:
I send my auth token as a parameter to my destination api and it seems like middleware can not access that. How can I access the parameter since the middleware needs that to check the auth conditions?
I am trying to implement a simple authentication/authorization application.
I know that it is common to set auth token in coockies, however, in my use-case, I need it to be implemented differently.
The implementation is: login returns auth token in response body and anytime authentication token is required, it is sent as a parameter "authorization" to the application.
here is the code for my user routers :
func UserRoute(router *gin.Engine) {
user := router.Group("/user")
{
user.POST("/signup", controllers.SignUp)
user.POST("/login", controllers.Login)
user.GET("/validate", middleware.RequireAuth, controllers.Validate)
}
}
validate function in usercontrollers.go:
func Validate(c *gin.Context) {
user, _ := c.Get("user")
c.IndentedJSON(http.StatusOK, gin.H{
"message": user,
})
}
here is the request I send
http://localhost:6000/user/validate?authorization=[My-JWT-Token]
Now when I try to read my auth parameter and use it in my middleware it seems like it does not actually exist:
func RequireAuth(c *gin.Context) {
confs, _ := configs.LoadConfig()
tokenString := c.Param("authorization")
if tokenString == "" {
// this abort case always happens
c.AbortWithStatus(http.StatusUnauthorized)
}
}
1. ctx.Request.URL.Query().Get("authorization")
2. ctx.Query("authorization")

Go GRPC Refresh token for a bidirectional stream

TLDR: I am looking for a way to update headers on an open stream for each call to stream.Send(msg) without closing the stream and opening a new one.
Summary
I have a GRPC client and server built to handle bidirectional streams. To authenticate with the server the client must send a JWT in the request headers, set as "authorization". The token is valid for 30 minutes. After the token has expired, the server will terminate the connection.
I am looking for a way to refresh my authorization token from the client, and keep the stream open. The client should run in a loop executing a new request every 30 minutes with the updated token, and the updated payload. I have not seen a way to update a header from the client side for an already opened stream.
Let's look at some code to get an idea of what the client side looks like. The code below has a function to create a new instance of the client, and another function to establish the connection to the GRPC server.
func NewWatchClient(config *Config, logger *logrus.Logger) (*WatchClient, error) {
cc, err := newConnection(config, logger)
if err != nil {
return nil, err
}
service := proto.NewWatchServiceClient(cc)
return &WatchClient{
config: config,
conn: cc,
logger: entry,
service: service,
}, nil
}
func newConnection(config *Config, logger *logrus.Logger) (*grpc.ClientConn, error) {
address := fmt.Sprintf("%s:%d", config.Host, config.Port)
// rpcCredential implements credentials.PerRPCCredentials
rpcCredential := newTokenAuth(config.Auth, config.TenantID)
return grpc.Dial(
address,
grpc.WithPerRPCCredentials(rpcCredential),
)
}
Looking at the newConnection function above I can see that there is a call to another function, newTokenAuth, to create an auth token. This func returns a struct that implements the PerRPCCredentials interface.
There are two ways to set the authorization for a request.
Use grpc.WithPerRPCCredentials to add the authorization at the time of creating the connection to the server.
Use grpc.PerRPCCredentials to add the authorization to each stream opened on the connection to the server.
In this case, I am using grpc.WithPerRPCCredentials to attach the token at the time of creating the connection to the server.
Now, let's take a look at the definition of PerRPCCredentials.
type PerRPCCredentials interface {
// GetRequestMetadata gets the current request metadata, refreshing
// tokens if required. This should be called by the transport layer on
// each request, and the data should be populated in headers or other
// context. If a status code is returned, it will be used as the status
// for the RPC. uri is the URI of the entry point for the request.
// When supported by the underlying implementation, ctx can be used for
// timeout and cancellation. Additionally, RequestInfo data will be
// available via ctx to this call.
// TODO(zhaoq): Define the set of the qualified keys instead of leaving
// it as an arbitrary string.
GetRequestMetadata(ctx context.Context, uri ...string) (map[string]string, error)
// RequireTransportSecurity indicates whether the credentials requires
// transport security.
RequireTransportSecurity() bool
}
The interface requires that you define two methods. The documentation of GetRequestMetadata says
GetRequestMetadata gets the current request metadata, refreshing tokens if required
So, it looks like my implementation of PerRPCCredentials should be able to handle a token refresh for my stream or connection. Let's take a look at my implementation of PerRPCCredentials.
// tokenAuth implements the PerRPCCredentials interface
type tokenAuth struct {
tenantID string
tokenRequester auth.PlatformTokenGetter
token string
}
// RequireTransportSecurity leave as false for now
func (tokenAuth) RequireTransportSecurity() bool {
return false
}
// GetRequestMetadata sets the http header prior to transport
func (t tokenAuth) GetRequestMetadata(_ context.Context, _ ...string) (map[string]string, error) {
token, err := t.tokenRequester.GetToken()
if err != nil {
return nil, err
}
t.token = token
go func() {
time.Sleep(25 * time.Minute)
token, _ := t.tokenRequester.GetToken()
t.token = token
}()
return map[string]string{
"tenant-id": t.tenantID,
"authorization": "Bearer " + t.token,
}, nil
}
As you can see, the call to GetRequestMetadata will establish a go routine that will attempt to refresh a token every 25 minutes. Adding a go routine right here is probably not the right way to do it. It was an attempt to get the auth header to refresh, which doesn't work.
Let's take a look at the stream.
func (w WatchClient) CreateWatch() error {
topic := &proto.Request{SelfLink: w.config.TopicSelfLink}
stream, err := w.service.CreateWatch(context.Background())
if err != nil {
return err
}
for {
err = stream.Send(topic)
if err != nil {
return err
}
time.Sleep(25 * time.Minute)
}
}
The client sends a message on the stream every 25 minutes. All I'm looking to get here is that when stream.Send is called, the updated token is also sent.
This function, GetRequestMetadata only gets called once, regardless if I am setting the auth through grpc.WithPerRPCCredentials or grpc.PerRPCCredsCallOption so there appears to be no way to update the authorization header.
If you have any idea what I have missed in my attempt to utilize the PerRPCCredentials for token refresh then please let me know.
Thank you.
Headers are sent at the beginning of an RPC, and cannot be updated during the RPC. If you need to send data during the life of a stream, it needs to be part of the request message in your proto definition.

307 redirect with Authorization header

In looking at the Go docs for http it looks like the Authorization header is removed when a response is a 307. Obviously it makes sense for almost every case but is there a way not to remove the Authorization header?
You can modify your http.Client to add the header again after it has been removed using CheckRedirect:
CheckRedirect func(req *Request, via []*Request) error
Since req is the upcoming request, it can be modified before it is sent. After making the changes, return nil to indicate that the request should still be sent.
Since this is a change to the http client instead of the request, you should check that this redirect is only used for the one URL where you need it (in case you use that client to do other requests).
You client definition could look like this:
http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
// you can check old responses for a status code
if len(via) != 0 && via[0].Response.StatusCode == http.StatusTemporaryRedirect {
req.Header.Add("Authorization", "some-value")
}
return nil
},
}

how to proxy GET request in golang with URL manipulation

I want to build a golang service which will listen for GET request, do some URL manipulation and then proxy a new request (to the manipulated URL) back to the browser:
(from browser -> server) GET http://www.example.com/7fbsjfhfh93hdkwhfbf398fhkef93..
(server manipulates URL - decrypts "7fbsjfhfh93hdkwhfbf398fhkef93.." -> "my-super-resource")
(server -> URL resource) GET http://www.somewhereelse.com/my-super-resource
(server -> browser) Response from http://www.somewhereelse.com/my-super-resource passed on to browser (using cors)
The whole chain will need to be synchronous which is ok. Is there a decent proxy library which allows for this sort of thing?
You can do something like this in less than 10 lines of code with the Sling package:
type Foo struct {
Bar string `json:"bar"`
}
func Handler(w http.ResponseWriter, r *http.Request) {
// Get the URL and decrypt it
url := getUrl(r)
decryptedUrl := decryptUrl(url)
// Use the decrypted URL to make a request to another service
var data *Foo
req, err := sling.New().Get(decryptedUrl).Receive(data)
if err != nil {
// Handle error...
}
// Respond to the original request using the data from the other service
respond(w, http.StatusOK, data)
}

Negroni continues to call other handlers after request is completed

My web-application in Go (using Gorilla mux and negroni) has about 20 handlers split into three groups based on what Middleware functions should be applied. Specifically:
Group 1: Static requests (no middleware at all)
GET /favicon.ico
GET /files
GET /files/index.html
GET /files/favicon.ico
Group 2: Requests that should have CORS middleware only, no authentication:
GET /
GET /login
POST /login
GET /auth-configuration
GET /service-status
Group 3: Requests that should have both CORS and authentication middleware applied:
GET /articles
POST /articles
PUT /articles/etc
PATCH /articles/etc
This is my code that sets-up the HTTP server:
func run() {
negroniStack := setUpNegroni()
bindAddr := // ...
http.ListenAndServe(bindAddr, negroniStack)
}
func setUpNegroni() negroni.Negroni {
negroniStack := negroni.Negroni{}
staticNegroni := setUpRoutesAndMiddlewareForStaticRequests()
loginNegroni := setUpRoutesAndMiddlewareForLogin()
serviceNegroni = setUpRoutesAndMiddlewareForService()
negroniStack.UseHandler(&staticNegroni)
negroniStack.UseHandler(&loginNegroni)
negroniStack.UseHandler(&serviceNegroni)
return negroniStack
}
func setUpRoutesAndMiddlewareForStaticRequests() negroni.Negroni {
staticNegroni := negroni.Negroni{}
staticRouter := mux.NewRouter()
staticRouter.PathPrefix("/files").HandlerFunc(staticHandler)
staticRouter.Path("/favicon.ico").HandlerFunc(staticHandler)
staticNegroni.UseHandler(staticRouter)
return staticNegroni
}
func setUpRoutesAndMiddlewareForLogin() negroni.Negroni {
authNegroni := negroni.Negroni{}
corsMiddleware := cors.New(cors.Options{
AllowedMethods: []string{"GET", "HEAD", "POST", "PUT", "PATCH", "DELETE"},
AllowCredentials: true,
OptionsPassthrough: false,
})
authNegroni.Use(corsMiddleware)
authRouter := mux.NewRouter()
authRouter.HandleFunc("/login", HandlePostAuth).Methods("POST")
authRouter.HandleFunc("/login", HandleGetAuth) // GET
authNegroni.UseHandler(authRouter)
return authNegroni
}
func setUpRoutesAndMiddlewareForService() negroni.Negroni {
serviceNegroni := negroni.Negroni{}
corsMiddleware := cors.New(cors.Options{
AllowedMethods: []string{"GET", "HEAD", "POST", "PUT", "PATCH", "DELETE"},
AllowCredentials: true,
OptionsPassthrough: false,
})
serviceNegroni.Use(corsMiddleware)
serviceNegroni.UseFunc(jwtMiddleware)
serviceRouter := mux.NewRouter()
serviceRouter.HandleFunc("/articles", HandleGetArticles).Methods("GET")
serviceRouter.HandleFunc("/articles", HandlePostArticles).Methods("POST")
// etc
serviceNegroni.UseHandler(serviceRouter)
return serviceNegroni
}
I believe this is correct based on the "Route Specific Middleware" section in Negroni's documentation where it says:
If you have a route group of routes that need specific middleware to be executed, you can simply create a new Negroni instance and use it as your route handler.
However, when I make requests and use the debugger, I see that (*Negroni).ServeHTTP is called multiple times. For example, if I request GET /favicon.ico then the staticHandler function is called correctly and calls WriteHeader(200), but after that it then calls into the next mux.Router which calls WriteHeader(404) which prints out a warning in the terminal because the header was written twice (http: multiple response.WriteHeader calls)
If it's for a route that doesn't exist then the Gorilla default NotFoundHandler is invoked 3 times (one for each mux.Router).
How do I get Negroni to stop invoking other handlers after the request was completed?
...and if I have misconfigured my Negroni instance, why doesn't it perform checks during initialization to warn me about an invalid configuration?
My understanding is that negroni.Use and UseFunc are for setting-up middleware (which are all invoked for every request), while UseHandler is to set-up the terminal handler (only 1 is invoked for each request, or fallback to 404). If I understand the situation correctly then for some reason it's treating my terminal handlers as middlewares.
From the UseHandler documentation (https://godoc.org/github.com/urfave/negroni#Negroni.UseHandler)
UseHandler adds a http.Handler onto the middleware stack. Handlers are invoked in the order they are added to a Negroni.
So it seems what you are seeing here is the expected behaviour.
You are basically creating different negroni instances and chaining them, so your final negroniStack is a middleware itself which will execute the other middlewares you added.
I believe what you want to do is create routes using an actual router, then add the appropriate middleware (using negroni) to each route.
If you look at the example you linked from the docs, that's what they are doing in that section (https://github.com/urfave/negroni#route-specific-middleware).
router.PathPrefix("/admin").Handler(negroni.New(
Middleware1,
Middleware2,
negroni.Wrap(adminRoutes),
))
See that they are not nesting negroni instances but rather creating just one which is applied to the desired routes.

Resources