Logging http responses (in addition to requests) - go

I am using Go and the Gorilla web toolkit's mux and handler packages to build a complex application, part of which requires a http server. Gorilla's mux and handler packages work wonderfully and I am able to successfully get the http server up and running and it has been quite simple to log requests.
However, I am unable to determine how I may log responses. Ideally, I would like a mechanism, similar to Gorilla's LoggingHandler, that "wraps" the logging mechanism easily.
Is there a Go package that does easily wraps / logs responses? Is there a way to use Go or Gorilla's capabilities in this fashion that I have not considered?

Thanks for the great suggestions. I tried a few of the suggestions and landed on a rather simple solution that uses a minimalist wrapper. Here is the solution that worked for me (feel free to offer comments, or better yet, other solutions):
import (
"fmt"
"log"
"net/http"
"net/http/httptest"
"net/http/httputil"
"github.com/gorilla/mux"
)
:
func logHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
x, err := httputil.DumpRequest(r, true)
if err != nil {
http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
return
}
log.Println(fmt.Sprintf("%q", x))
rec := httptest.NewRecorder()
fn(rec, r)
log.Println(fmt.Sprintf("%q", rec.Body))
}
}
func MessageHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "A message was received")
}
And the following code will use the aforementioned handler:
:
router := mux.NewRouter()
router.HandleFunc("/", logHandler(MessageHandler))
:
Output from the above code will be something along the lines of:
:
2016/07/20 14:44:29 "GET ... HTTP/1.1\r\nHost: localhost:8088\r\nAccept: */*\r\nUser-Agent: curl/7.43.0\r\n\r\n"
2016/07/20 14:44:29 ...[response body]
:

The accepted answer by Eric Broda won't help much if you want to actually send your response to the client. I've made a modification to that code that will actually work:
func logHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
x, err := httputil.DumpRequest(r, true)
if err != nil {
http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
return
}
log.Println(fmt.Sprintf("%q", x))
rec := httptest.NewRecorder()
fn(rec, r)
log.Println(fmt.Sprintf("%q", rec.Body))
// this copies the recorded response to the response writer
for k, v := range rec.HeaderMap {
w.Header()[k] = v
}
w.WriteHeader(rec.Code)
rec.Body.WriteTo(w)
}
}

edit sorry, I didn't notice your mention of gorilla-mux, I have only tried this with gin, but if it uses middlewares this should still work.
the trick is, c.Next() in a middleware blocks until all subsequent middlewares return. Here's a logrus solution. Put this as your first middleware:
func Logrus(logger *logrus.Logger) gin.HandlerFunc {
return func(c *gin.Context) {
start := time.Now().UTC()
path := c.Request.URL.Path
c.Next()
end := time.Now().UTC()
latency := end.Sub(start)
logger.WithFields(logrus.Fields{
"status": c.Writer.Status(),
"method": c.Request.Method,
"path": path,
"ip": c.ClientIP(),
"duration": latency,
"user_agent": c.Request.UserAgent(),
}).Info()
}
}
GinEngine.Use(Logrus(logrus.StandardLogger()))

Related

How to handle superfluous response.WriteHeader call in order to return 500

I am aware that the WriteHeader method of http.ResponseWriter can only be called once per HTTP response, can only have a single response status code and can only send the headers once. This is all perfectly fine.
The question is, how should I refactor my code in order to override 201 and return 500 if http.ResponseWriter.Write returns an error? As you can see below I force panic on purpose to see how httprouter.Router.PanicHandler handles it. As expected, logs show http: superfluous response.WriteHeader call from ... and the response is 201 because it is too late as explained above.
package server
import (
"github.com/julienschmidt/httprouter"
"log"
"net/http"
)
func Serve() {
rtr := httprouter.New()
rtr.GET("/", home.Welcome)
handle500(rtr)
err := http.ListenAndServe(":8080", rtr)
if err != nil {
log.Fatalf("server crash")
}
}
func handle500(r *httprouter.Router) {
r.PanicHandler = func(res http.ResponseWriter, req *http.Request, err interface{}) {
res.WriteHeader(http.StatusInternalServerError)
// http: superfluous response.WriteHeader call from line above
}
}
package home
import (
"github.com/julienschmidt/httprouter"
"net/http"
)
func Welcome(res http.ResponseWriter, _ *http.Request, _ httprouter.Params) {
// doing a few bits and building the body
res.Header().Set("Content-Type", "application/json")
res.WriteHeader(201)
_, err := res.Write("body goes here")
if err == nil { // I am doing this deliberately to test 500
panic("assume that something has gone wrong with res.Write and an error occurred")
}
}
There is no way to "override" the status code as it's sent to the browser immediately.
You're checking for the return value of http.ResponseWriter.Write(). I'm not sure this is a good strategy. If writing the response failed then chances are that writing more will probably fail, too.
Logging the failure seems more appropriate, but I would expect most failures to be dropped connections and other errors which won't require action.

How to remove a named route from Gorilla mux in go? [duplicate]

I am fairly new to Go and have not been able to find any information on this, maybe it is just not possible at this time.
I am trying to delete or replace a mux route (using http.NewServeMux, or gorilla's mux.Router). My end goal is to be able to enable/disable a route or set of routes without having to restart the program.
I can probably accomplish this on a handler to handler basis and just return 404 if that feature is "disabled", but I would rather find a more general way to do this since I would like to implement it for every route in my application.
Or would I be better off just keeping track of disabled url patterns and using some middleware to prevent handler execution?
If someone can at least point me in the right direction, I will absolutely post code examples of a solution assuming there is one. Thanks!
There's no built in way, but it is easy enough to implement play.
type HasHandleFunc interface { //this is just so it would work for gorilla and http.ServerMux
HandleFunc(pattern string, handler func(w http.ResponseWriter, req *http.Request))
}
type Handler struct {
http.HandlerFunc
Enabled bool
}
type Handlers map[string]*Handler
func (h Handlers) ServeHTTP(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if handler, ok := h[path]; ok && handler.Enabled {
handler.ServeHTTP(w, r)
} else {
http.Error(w, "Not Found", http.StatusNotFound)
}
}
func (h Handlers) HandleFunc(mux HasHandleFunc, pattern string, handler http.HandlerFunc) {
h[pattern] = &Handler{handler, true}
mux.HandleFunc(pattern, h.ServeHTTP)
}
func main() {
mux := http.NewServeMux()
handlers := Handlers{}
handlers.HandleFunc(mux, "/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("this will show once"))
handlers["/"].Enabled = false
})
http.Handle("/", mux)
http.ListenAndServe(":9020", nil)
}
Yes you can.
One way to do it is to have a sturct that implement http.Handle interface with the method
ServeHTTP.
Then have the struct contain another muxer like gorilla's
and finally have an atomic Switch to enable/ disable the subrouting
This is a working example of what I mean:
package main
import (
"fmt"
"github.com/gorilla/mux"
"net/http"
"sync/atomic"
)
var recording int32
func isRecording() bool {
return atomic.LoadInt32(&recording) != 0
}
func setRecording(shouldRecord bool) {
if shouldRecord {
atomic.StoreInt32(&recording, 1)
} else {
atomic.StoreInt32(&recording, 0)
}
}
type SwitchHandler struct {
mux http.Handler
}
func (s *SwitchHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if isRecording() {
fmt.Printf("Switch Handler is Recording\n")
s.mux.ServeHTTP(w, r)
return
}
fmt.Printf("Switch Handler is NOT Recording\n")
w.WriteHeader(http.StatusNotFound)
fmt.Fprintf(w, "NOT Recording\n")
}
func main() {
router := mux.NewRouter()
router.HandleFunc("/success/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Recording\n")
})
handler := &SwitchHandler{mux: router}
setRecording(false)
http.Handle("/", handler)
http.ListenAndServe(":8080", nil)
}
According to https://github.com/gorilla/mux/issues/82 it is suggested to swap the router instead of deleting routes. Existing connections will stay open.

Gorilla Mux router from inside handler only works once then gives 404 page not found

I'm using Gorilla mux as my router and I'm having a very strange behaviour. On the first request to the server, I get a valid response. But on subsequent requests, I receive a 404 page not found. There are no errors in the console.
My code is pretty straightforward (it can be copy-pasted to test it right out):
package main
import (
"fmt"
"github.com/gorilla/mux"
"log"
"net/http"
)
func main() {
router := mux.NewRouter()
router.HandleFunc("/", RootHandler).Name("root")
http.Handle("/", router)
log.Println("Listening on port 1337...")
if err := http.ListenAndServe(":1337", nil); err != nil {
log.Fatal("http.ListenAndServe: ", err)
}
}
func RootHandler(w http.ResponseWriter, r *http.Request) {
content := "Welcome to "
rootUrl, err := mux.CurrentRoute(r).Subrouter().Get("root").URL()
if err != nil {
log.Printf("mux.CurrentRoute(r).Subrouter().Get(\"root\").URL(): ", err)
}
response := content + rootUrl.String()
fmt.Fprintf(w, response)
}
After some code commenting and tests, it seems the following line is the culprit:
rootUrl, err := mux.CurrentRoute(r).Subrouter().Get("root").URL()
This method of getting the router inside the handler using the current request comes from another StackOverflow post: How to call a route by its name from inside a handler?
But for a strange reason, it only works once:
shell-1$ go run servertest.go
2014/10/30 13:31:34 Listening on port 1337...
shell-2$ curl http://127.0.0.1:1337
Welcome to /
shell-2$ curl http://127.0.0.1:1337
404 page not found
As you can see, there are no errors in the console.
Does someone have an idea of why it only works once ?
The problem is Subrouter() isn't made to return the router, but to create one, thus it changes the matcher of the router it is called on, making you lose the handler.
You could try passing the router to the handler using closures instead.
func RootHandler(router *mux.Router) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
...
}
}
i run to this problem and fixe it by re initiating the methods
//create a subrouter separately \
subRoute := mux.CurrentRoute(req).Subrouter() \
//Call the Route u want and store the URL
url, err := subRoute.Get("check_authorization").URL("id", key, "password", token)
// re-initiate the method to GET or whatever u had before
subRoute.Methods(http.MethodGet)

Golang http mux change handler function

I am fairly new to Go and have not been able to find any information on this, maybe it is just not possible at this time.
I am trying to delete or replace a mux route (using http.NewServeMux, or gorilla's mux.Router). My end goal is to be able to enable/disable a route or set of routes without having to restart the program.
I can probably accomplish this on a handler to handler basis and just return 404 if that feature is "disabled", but I would rather find a more general way to do this since I would like to implement it for every route in my application.
Or would I be better off just keeping track of disabled url patterns and using some middleware to prevent handler execution?
If someone can at least point me in the right direction, I will absolutely post code examples of a solution assuming there is one. Thanks!
There's no built in way, but it is easy enough to implement play.
type HasHandleFunc interface { //this is just so it would work for gorilla and http.ServerMux
HandleFunc(pattern string, handler func(w http.ResponseWriter, req *http.Request))
}
type Handler struct {
http.HandlerFunc
Enabled bool
}
type Handlers map[string]*Handler
func (h Handlers) ServeHTTP(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path
if handler, ok := h[path]; ok && handler.Enabled {
handler.ServeHTTP(w, r)
} else {
http.Error(w, "Not Found", http.StatusNotFound)
}
}
func (h Handlers) HandleFunc(mux HasHandleFunc, pattern string, handler http.HandlerFunc) {
h[pattern] = &Handler{handler, true}
mux.HandleFunc(pattern, h.ServeHTTP)
}
func main() {
mux := http.NewServeMux()
handlers := Handlers{}
handlers.HandleFunc(mux, "/", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("this will show once"))
handlers["/"].Enabled = false
})
http.Handle("/", mux)
http.ListenAndServe(":9020", nil)
}
Yes you can.
One way to do it is to have a sturct that implement http.Handle interface with the method
ServeHTTP.
Then have the struct contain another muxer like gorilla's
and finally have an atomic Switch to enable/ disable the subrouting
This is a working example of what I mean:
package main
import (
"fmt"
"github.com/gorilla/mux"
"net/http"
"sync/atomic"
)
var recording int32
func isRecording() bool {
return atomic.LoadInt32(&recording) != 0
}
func setRecording(shouldRecord bool) {
if shouldRecord {
atomic.StoreInt32(&recording, 1)
} else {
atomic.StoreInt32(&recording, 0)
}
}
type SwitchHandler struct {
mux http.Handler
}
func (s *SwitchHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if isRecording() {
fmt.Printf("Switch Handler is Recording\n")
s.mux.ServeHTTP(w, r)
return
}
fmt.Printf("Switch Handler is NOT Recording\n")
w.WriteHeader(http.StatusNotFound)
fmt.Fprintf(w, "NOT Recording\n")
}
func main() {
router := mux.NewRouter()
router.HandleFunc("/success/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Recording\n")
})
handler := &SwitchHandler{mux: router}
setRecording(false)
http.Handle("/", handler)
http.ListenAndServe(":8080", nil)
}
According to https://github.com/gorilla/mux/issues/82 it is suggested to swap the router instead of deleting routes. Existing connections will stay open.

Set http headers for multiple handlers in go

I'm trying to set an http header for multiple handlers. My first thought was to make a custom write function that would set the header before writing the response like the code sample at the bottom.
However, when I pass a pointer to the http.ResponseWriter and try to access it from my function it tells me that "type *http.ResponseWriter has no Header method".
What is the best way to set headers for multiple handlers, and also why isn't the pointer working the way I want it to?
func HelloServer(w http.ResponseWriter, req *http.Request) {
type Message struct {
Name string
Body string
Time int64
}
m := Message{"Alice", "Hello", 1294706395881547000}
b, _ := json.Marshal(m)
WriteJSON(&w, b)
}
func WriteJSON(wr *http.ResponseWriter, rawJSON []byte) {
*wr.Header().Set("Content-Type", "application/json")
io.WriteString(*wr, string(rawJSON))
}
func main() {
http.HandleFunc("/json", HelloServer)
err := http.ListenAndServe(":9000", nil)
if err != nil {
log.Fatal("ListenAndServer: ", err)
}
}
I'm not sure about the multiple handlers thing, but I do know why the code you wrote is failing. The key is that the line:
*wr.Header().Set("Content-Type", "application/json")
is being interpreted, because of operator precedence, as:
*(wr.Header().Set("Content-Type", "application/json"))
Since wr has the type *http.ResponseWriter, which is a pointer to and interface, rather than the interface itself, this won't work. I assume that you knew that, which is why you did *wr. I assume what you meant to imply to the compiler is:
(*wr).Header().Set("Content-Type", "application/json")
If I'm not mistaken, that should compile and behave properly.
You don't need to use *wr as it already references a pointer.
wr.Header().Set("Content-Type", "application/json") should be sufficient.
If you want to set "global" headers for every request, you can create a function that satisfies http.HandleFunc (go.auth has a good example) and then wrap your handlers like so:
http.HandleFunc("/hello", Defaults(helloHandler))
Also take a look at the net/http documentation, which has further examples.
I wrap my handlers with an error handler
which calls my AddSafeHeader function.
I based it on http://golang.org/doc/articles/error_handling.html
but it doesn't use ServeHTTP so it works with appstats:
http.Handle("/", appstats.NewHandler(util.ErrorHandler(rootHandler)))
Here:
package httputil
import (
"appengine"
"net/http"
"html/template"
)
func AddSafeHeaders(w http.ResponseWriter) {
w.Header().Set("X-Content-Type-Options", "nosniff")
w.Header().Set("X-XSS-Protection", "1; mode=block")
w.Header().Set("X-Frame-Options", "SAMEORIGIN")
w.Header().Set("Strict-Transport-Security", "max-age=2592000; includeSubDomains")
}
// Redirect to a fixed URL
type redirectHandler struct {
url string
code int
}
func (rh *redirectHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
Redirect(w, r, rh.url, rh.code)
}
func Redirect(w http.ResponseWriter, r *http.Request, urlStr string, code int) {
AddSafeHeaders(w)
http.Redirect(w, r, urlStr, code)
}
// RedirectHandler returns a request handler that redirects
// each request it receives to the given url using the given
// status code.
func RedirectHandler(url string, code int) http.Handler {
return &redirectHandler{url, code}
}
func ErrorHandler(fn func(appengine.Context, http.ResponseWriter, *http.Request)) func(appengine.Context, http.ResponseWriter, *http.Request) {
return func(c appengine.Context, w http.ResponseWriter, r *http.Request) {
defer func() {
if err, ok := recover().(error); ok {
c.Errorf("%v", err)
w.WriteHeader(http.StatusInternalServerError)
errorTemplate.Execute(w, err)
}
}()
AddSafeHeaders(w)
fn(c, w, r)
}
}
// Check aborts the current execution if err is non-nil.
func Check(err error) {
if err != nil {
panic(err)
}
}
var errorTemplate = template.Must(template.New("error").Parse(errorTemplateHTML))
const errorTemplateHTML = `
<html>
<head>
<title>XXX</title>
</head>
<body>
<h2>An error occurred:</h2>
<p>{{.}}</p>
</body>
</html>
`
http.ResponseWriter is an interface.
You should probably not be using a pointer to an interface. In net/http/server.go, the unexported response struct is the actual type that implements ResponseWriter when the server calls your handler, and importantly, when the server actually calls the handler's ServeHTTP, it passes a *response. It's already a pointer, but you don't see that because ResonseWriter is an interface. (the response pointer is created here, by (c *conn).readRequest. (The links will likely be for the wrong lines the future, but you should be able to locate them).
That's why the ServeHTTP function required to implement Handler is:
ServeHTTP(w ResponseWriter, r *Request)
i.e. not a pointer to ResponseWriter, as this declaration already permits a pointer to a struct that implements the ResponseWriter interface.
As I am new to Go, I created a minimal contrived example, based on elithrar's answer, which shows how to easily add headers to all your routes / responses. We do so, by creating a function that satisfies the http.HandlerFunc interface, then wraps the route handler functions:
package main
import (
"encoding/json"
"log"
"net/http"
"github.com/gorilla/mux"
)
// Hello world.
func Hello(w http.ResponseWriter, r *http.Request) {
json.NewEncoder(w).Encode("Hello World")
}
// HelloTwo world
func HelloTwo(w http.ResponseWriter, r *http.Request) {
json.NewEncoder(w).Encode("Hello Two World")
}
// JSONHeaders conforms to the http.HandlerFunc interface, and
// adds the Content-Type: application/json header to each response.
func JSONHeaders(handler http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
handler(w, r)
}
}
func main() {
router := mux.NewRouter()
// Now, instead of calling your handler function directly, pass it into the wrapper function.
router.HandleFunc("/", JSONHeaders(Hello)).Methods("GET")
router.HandleFunc("/hellotwo", JSONHeaders(HelloTwo)).Methods("GET")
log.Fatal(http.ListenAndServe(":3000", router))
}
Results:
$ go run test.go &
$ curl -i localhost:3000/
HTTP/1.1 200 OK
Content-Type: application/json
Date: Thu, 28 Feb 2019 22:27:04 GMT
Content-Length: 14
"Hello World"
What i end up doing:
// Accepts a user supplied http.HandlerFunc and then modifies it in various ways. In this case, it adds two new headers.
func CommonlHandler(h http.HandlerFunc) http.HandlerFunc {
return func (rs http.ResponseWriter, rq *http.Request) {
rs.Header().Add("Server", "Some server")
rs.Header().Add("Cache-Control", "no-store")
h(rs, rq)
}
// somewhere down the line, where you're setting up http request handlers
serveMux := http.NewServeMux()
serveMux.HandleFunc("/", CommonHandler(func(rs http.ResponseWriter, rq *http.Request) {
// Handle request as usual. Since it's wrapped in the CommonHandler and we've set some headers there, responses to requests to "/" will contain those headers.
// Be mindful what modifications you're doing here. If, for ex., you decide you want to apply different caching strategy than the Common one, since this will be passed to the CommonHandler, your changes will be overwritten and lost. So it may be a good idea to introduce checks in CommonHandler to determine whether the header exists, before you decide to create it.
}))
serveMux.HandleFunc("/contact", CommonHandler(func(rs http.ResponseWriter, rq *http.Request) {
// handle another request as usual
}))

Resources