I need to log the response body in a middleware of gin, but I don't find how to get the response body. Can anyone help?
I am using a middleware like this:
func Logger() gin.HandlerFunc {
return func(c *gin.Context) {
c.Next()
statusCode := c.Writer.Status()
if statusCode >= 400 {
//ok this is an request with error, let's make a record for it
//log body here
}
}
}
My question is, how to get response body from Context in middleware?
You need to intercept writing of response and store it somewhere first. Then you can log it. And to do that you need to implement your own Writer intercepting Write() calls.
For example, as follows:
type bodyLogWriter struct {
gin.ResponseWriter
body *bytes.Buffer
}
func (w bodyLogWriter) Write(b []byte) (int, error) {
w.body.Write(b)
return w.ResponseWriter.Write(b)
}
func ginBodyLogMiddleware(c *gin.Context) {
blw := &bodyLogWriter{body: bytes.NewBufferString(""), ResponseWriter: c.Writer}
c.Writer = blw
c.Next()
statusCode := c.Writer.Status()
if statusCode >= 400 {
//ok this is an request with error, let's make a record for it
// now print body (or log in your preferred way)
fmt.Println("Response body: " + blw.body.String())
}
}
Then use this middleware like this:
router.Use(ginBodyLogMiddleware)
Note that this sill won't work for static files as gin does not seem to use c.Writer for them. But in most cases, that's what you want anyway.
If you want to intercept all files, you need to use a slightly more complicated approach. Instead of Middleware, you'll need to implement a wrapper http.Handler that will wrap gin.Engine and will use same approach as shown above to intercept and log whatever is written to http.ResponseWriter. Then run gin server like this:
ginRouter := gin.New()
// configure your middleware and routes as required
// Run http server as follows, where bodyLogHandler is your wrapper handler
http.ListenAndServe(bindAddress, &bodyLogHandler{wrappedHandler: ginRouter}
FYI
Note: implement WriteString() if using c.String() for writing response body
type bodyLogWriter struct {
gin.ResponseWriter
body *bytes.Buffer
}
func (w bodyLogWriter) Write(b []byte) (int, error) {
w.body.Write(b)
return w.ResponseWriter.Write(b)
}
func (w bodyLogWriter) WriteString(s string) (int, error) {
w.body.WriteString(s)
return w.ResponseWriter.WriteString(s)
}
func ginBodyLogMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
blw := &bodyLogWriter{body: bytes.NewBufferString(""), ResponseWriter: c.Writer}
c.Writer = blw
c.Next()
fmt.Println("Response body: " + blw.body.String())
}
}
...
// register
router := r.Group("/", ginBodyLogMiddleware())
Related
Premise: I've found a similar issue but not working in my case, so please do not mark this as a duplicate.
I've a HTTP server in Go and I've created a middleware to log the request, the response time and I would like to log the response too.
I've used httputil.DumpRequest in a function called HTTPRequest under the package log.
How can I correctly get the response body and status and headers from the w http.ResponseWriter and log them together with the other data?
My ISSUE is: I would like to intercept the Response Headers, Status and Body and to log the together with the Request and Response Time
Here's the code:
log "core/logger"
...
func RequestLoggerMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
defer func() {
log.Info(
fmt.Sprintf(
"[Request: %s] [Execution time: %v] [Response: %s]",
log.HTTPRequest(r),
time.Since(start),
// RESPONSE DATA HERE !!!!!!!
))
}()
next.ServeHTTP(w, r)
})
}
Thanks, #Sivachandran for the response. It was almost perfect, only it didn't implement the http.ResponseWriter because of the pointers.
For the sake of completeness, I post here the correct solution code, because it's not easy to find any documentation on it, even if this question has been given a negative score.
Stackoverflow is a good place to exchange questions and this, in my opinion, was a very good and difficult question, either for a middle lever Golang programmer, so it didn't deserve a negative score at all!
That's the solution, enjoy:
// RequestLoggerMiddleware is the middleware layer to log all the HTTP requests
func RequestLoggerMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
rww := NewResponseWriterWrapper(w)
w.Header()
defer func() {
log.Info(
fmt.Sprintf(
"[Request: %s] [Execution time: %v] [Response: %s]",
log.HTTPRequest(r),
time.Since(start),
rww.String(),
))
}()
next.ServeHTTP(rww, r)
})
}
// ResponseWriterWrapper struct is used to log the response
type ResponseWriterWrapper struct {
w *http.ResponseWriter
body *bytes.Buffer
statusCode *int
}
// NewResponseWriterWrapper static function creates a wrapper for the http.ResponseWriter
func NewResponseWriterWrapper(w http.ResponseWriter) ResponseWriterWrapper {
var buf bytes.Buffer
var statusCode int = 200
return ResponseWriterWrapper{
w: &w,
body: &buf,
statusCode: &statusCode,
}
}
func (rww ResponseWriterWrapper) Write(buf []byte) (int, error) {
rww.body.Write(buf)
return (*rww.w).Write(buf)
}
// Header function overwrites the http.ResponseWriter Header() function
func (rww ResponseWriterWrapper) Header() http.Header {
return (*rww.w).Header()
}
// WriteHeader function overwrites the http.ResponseWriter WriteHeader() function
func (rww ResponseWriterWrapper) WriteHeader(statusCode int) {
(*rww.statusCode) = statusCode
(*rww.w).WriteHeader(statusCode)
}
func (rww ResponseWriterWrapper) String() string {
var buf bytes.Buffer
buf.WriteString("Response:")
buf.WriteString("Headers:")
for k, v := range (*rww.w).Header() {
buf.WriteString(fmt.Sprintf("%s: %v", k, v))
}
buf.WriteString(fmt.Sprintf(" Status Code: %d", *(rww.statusCode)))
buf.WriteString("Body")
buf.WriteString(rww.body.String())
return buf.String()
}
You need to wrap the ResponseWriter to capture the response data.
type ResponseWriterWrapper struct {
w http.ResponseWriter
body bytes.Buffer
statusCode int
}
func (i *ResponseWriterWrapper) Write(buf []byte) (int, error) {
i.body.Write(buf)
return i.w.Write(buf)
}
func (i *ResponseWriterWrapper) WriteHeader(statusCode int) {
i.statusCode = statusCode
i.w.WriteHeader(statusCode)
}
func (i *ResponseWriterWrapper) String() {
var buf bytes.Buffer
buf.WriteString("Response:")
buf.WriteString("Headers:")
for k, v := range i.w.Header() {
buf.WriteString(fmt.Sprintf("%s: %v", k, v))
}
buf.WriteString(fmt.Sprintf("Status Code: %d", i.statusCode))
buf.WriteString("Body")
buf.WriteString(i.body.String())
}
Pass the wrapper to ServeHTTP and log captured response data.
func RequestLoggerMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
start := time.Now()
rww := ResponseWriterWrapper{ w: w }
defer func() {
log.Info(
fmt.Sprintf(
"[Request: %s] [Execution time: %v] [Response: %s]",
log.HTTPRequest(r),
time.Since(start),
log.Info(rww.String())
))
}()
next.ServeHTTP(rww, r)
})
}
I have a unary interceptor that contains the following code:
func (m Middlewares) LocationInterceptor(c context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
ctx := c.(HarmonyContext)
location, ok := req.(interface{ GetLocation() *corev1.Location })
if !ok {
panic("location middleware used on message without a location")
}
ctx.Location := location.GetLocation()
return handler(c, req)
}
How would I be able to convert this to a stream interceptor, if I know that the stream will definitely only stream from server to client? In addition, is there any way to make it only intercept when the moment the stream begins?
func (m Middlewares) LocationInterceptorStream(srv interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
wrappedStream := ss.(HarmonyWrappedServerStream)
return handler(srv, wrappedStream)
}
For a "server streaming RPC", the client sends one message (the request), and the server responds with multiple messages. In the stream interceptor, you need to call ServerStream.RecvMsg(...) once to get the request from the client. You will then need to pass a "wrapped stream" into handler which will later return this message on the first call to RecvMsg. For gRPC using protobuf messages, you can do something like the following. This serverStreamWrapper will allow you to call peekRequest(...) in the interceptor to get the request, then pass the stream on to the handler.
For this specific example, you may also be able to implement the logic in the wrapped RecvMsg(...) function itself. This function gets called by gRPC to read the request from the client to the server. This may be simpler than trying to store the request.
type serverStreamWrapper struct {
peekedRequest proto.Message
wrappedStream grpc.ServerStream
}
func newServerStreamWrapper(stream grpc.ServerStream) *serverStreamWrapper {
return &serverStreamWrapper{nil, stream}
}
func (s *serverStreamWrapper) peekRequest(msg interface{}) error {
protoMsg := msg.(proto.Message)
if protoMsg == nil {
panic("BUG: msg must not be nil")
}
if s.peekedRequest != nil {
panic("BUG: Must only called peekRequest once")
}
err := s.wrappedStream.RecvMsg(protoMsg)
if err == nil {
s.peekedRequest = protoMsg
}
return err
}
func (s *serverStreamWrapper) RecvMsg(msg interface{}) error {
if s.peekedRequest != nil {
protoMsg := msg.(proto.Message)
proto.Reset(protoMsg)
proto.Merge(protoMsg, s.peekedRequest)
s.peekedRequest = nil
return nil
}
return s.wrappedStream.RecvMsg(msg)
}
My original question here was flagged as a duplicate of this question. I had no luck implementing it and suspect my problem is misunderstood, so with my question closed, I'm starting fresh with a more specific question.
I'm trying to set a cookie based on a response header from within middleware in request that is reverse proxied.
Here's the workflow:
User requests http://example.com/foo/bar
Go app uses ReverseProxy to proxy that request to http://baz.com
baz.com sets a response header X-FOO
Go app modifies response by setting a MYAPPFOO cookie with the value of the X-FOO response header
The cookie is written to the user's browser
It was suggested that a custom http.ResponseWriter will work, but after trying and searching for more information, it is not clear how to approach this.
Since I'm failing to grasp the concept of a custom ResponseWriter for my use case, I'll post code that demonstrates more precisely what I was trying to do at the point I got stuck:
package main
import (
"github.com/gorilla/mux"
"log"
"net/http"
"net/http/httputil"
"net/url"
)
func setCookie(w http.ResponseWriter, name string, value string) {
...
http.SetCookie(w, &cookie)
}
func handler(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// setCookie() works here
// but I cannot access w.Header().Get("X-FOO")
next.ServeHTTP(w, r)
// I can access w.Header().Get("X-FOO") here
// but setCookie() does not cookie the user's browser
// If I could do it all in one place, this is what I would do:
if r.Method == "POST" && r.URL.String() == "/login" {
foo := w.Header().Get("X-FOO")
setCookie(w, "MYAPPFOO", foo)
}
})
}
func main() {
r := mux.NewRouter()
r.Use(handler)
proxy := httputil.NewSingleHostReverseProxy("https://baz.example.com/")
r.PathPrefix("/").Handler(proxy)
log.Fatal(http.ListenAndServe(":9001", r))
}
As a side note, I was able to make this work with ReverseProxy.ModifyResponse as recommended in the comments of my last question but I'd really like to achieve this with middleware to keep the code that dynamically creates proxies from config clean. (not in the example code)
From the documentation on http.ResponseWriter methods:
(emphasis added)
Header() http.Header:
Changing the header map after a call to WriteHeader (or Write) has no
effect unless the modified headers are trailers.
WriteHeader(statusCode int):
WriteHeader sends an HTTP response header with the provided status
code.
Write([]byte) (int, error):
If WriteHeader has not yet been called, Write calls
WriteHeader(http.StatusOK) before writing the data.
This should highlight the reason why, you can't set a cookie after the next.ServeHTTP(w, r) call, which is that one of the handlers in the middleware chain executed by that call is calling either WriteHeader or Write directly or indirectly.
So to be able set the cookie after the next.ServeHTTP(w, r) call you need to make sure that none of the handlers in the middleware chain calls WriteHeader or Write on the original http.ResponseWriter instance. One way to do this is to wrap the original instance in a custom http.ResponseWriter implementation that will postpone the writing of the response until after you're done with setting the cookie.
For example something like this:
type responsewriter struct {
w http.ResponseWriter
buf bytes.Buffer
code int
}
func (rw *responsewriter) Header() http.Header {
return rw.w.Header()
}
func (rw *responsewriter) WriteHeader(statusCode int) {
rw.code = statusCode
}
func (rw *responsewriter) Write(data []byte) (int, error) {
return rw.buf.Write(data)
}
func (rw *responsewriter) Done() (int64, error) {
if rw.code > 0 {
rw.w.WriteHeader(rw.code)
}
return io.Copy(rw.w, &rw.buf)
}
And you would use it like this in your middleware:
func handler(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
rw := &responsewriter{w: w}
next.ServeHTTP(rw, r)
if r.Method == "POST" && r.URL.String() == "/login" {
foo := rw.Header().Get("X-FOO")
setCookie(rw, "MYAPPFOO", foo)
}
if _, err := rw.Done(); err != nil {
log.Println(err)
}
})
}
I'm trying to log the response body of a request that has been redirected.
func main() {
r := gin.Default()
eanAPI := api.NewEanAPI()
v1 := r.Group("/v1")
v1.POST("/*action", eanAPI.Redirect, middleware.SaveRequest())
port := os.Getenv("PORT")
if len(port) == 0 {
port = "8000"
}
r.Run(":" + port)
}
func (api *eanAPI) Redirect(ctx *gin.Context) {
forwardToHost := "https://jsonplaceholder.typicode.com"
url := ctx.Request.URL.String()
ctx.Redirect(http.StatusTemporaryRedirect, forwardToHost)
}
type bodyLogWriter struct {
gin.ResponseWriter
body *bytes.Buffer
}
func (w bodyLogWriter) Write(b []byte) (int, error) {
w.body.Write(b)
return w.ResponseWriter.Write(b)
}
func SaveRequest() gin.HandlerFunc {
return func(c *gin.Context) {
c.Next()
blw := &bodyLogWriter{body: bytes.NewBufferString(""), ResponseWriter: c.Writer}
c.Writer = blw
c.Next()
statusCode := c.Writer.Status()
fmt.Println("status: ", statusCode)
if statusCode <= 400 {
//ok this is an request with error, let's make a record for it
// now print body (or log in your preferred way)
fmt.Println("Response body: " + blw.body.String())
}
}
Unfortunately, the body response is always empty. Maybe the redirection or the middleware is messing with my body response
When I tried with postman I can see the body response coming! You can try it by a POST on https://jsonplaceholder.typicode.com/posts. It should return a payload with an id in the body response
What am I doing wrong here?
Thanks in advance
The response body you're seeing in the browser/postman is from the page you're being redirected to. The page doing the actual redirecting likely has no response body, just a status and a Location header. This is the expected behavior of a redirect. If you want to try to capture the body of the page you're redirecting to, you can't actually use redirects for that (the redirect handler isn't involved in the final request); you'd have to fully proxy the request rather than redirecting it. Proxying is very different from redirecting though, so make sure that's really the behavior you're after.
gRPC's Go library provides interfaces for creating your own custom interceptors (i.e. middleware functions), and I'm attempting to write two logging interceptors. The first is a Unary Server Interceptor where I'm easily able to log the request parameters using the object passed into the interceptor function.
func loggingUnary(context context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
start := time.Now()
resp, err := handler(context, req)
printLogMessage(err, info.FullMethod, context, time.Since(start), req)
return resp, err
}
How can I do the same with the Stream Server Interceptor which doesn't conveniently pass the request object as a parameter? Is there another way to access the request?
func loggingStream(srv interface{}, stream grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
start := time.Now()
err := handler(srv, stream)
printLogMessage(err, info.FullMethod, stream.Context(), time.Since(start), "")
return err
}
This is a bit old now, but the easiest way to extend your interception into the stream is to create a grpc.ServerStream wrapper, then wrap the real ServerStream in your interceptor. In that way, your intercepting code can handle the received and sent messages in the stream.
// A wrapper for the real grpc.ServerStream
type LoggingServerStream struct {
inner grpc.ServerStream
}
func (l LoggingServerStream) SetHeader(m metadata.MD) error {
return l.SetHeader(m)
}
func (l LoggingServerStream) SendHeader(m metadata.MD) error {
return l.SendHeader(m)
}
func (l LoggingServerStream) SetTrailer(m metadata.MD) {
l.SetTrailer(m)
}
func (l LoggingServerStream) Context() context.Context {
return l.Context()
}
func (l LoggingServerStream) SendMsg(m interface{}) error {
fmt.Printf("Sending Message: type=%s\n", reflect.TypeOf(m).String())
return l.SendMsg(m)
}
func (l LoggingServerStream) RecvMsg(m interface{}) error {
fmt.Printf("Receiving Message: type=%s\n", reflect.TypeOf(m).String())
return l.RecvMsg(m)
}
The interceptor:
func LoggingStreamInterceptor() grpc.StreamServerInterceptor {
return func(srv interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
return handler(srv, LoggingServerStream{inner:ss})
}
}
Any state you need to keep and go in your wrapper.
The stream handler can be called multiple times during the lifetime of the request that created the stream, that's why the request is not part of the arguments to the handler (and to any interceptor, either). You could place the request (or better, a copy of the data that you want logged and not a reference to the request itself) in the stream context (assuming you are in control of the code that creates the ServerStream object). I would rather log the request parameters once, when the stream is created and not on every call to the handler (so each request is logged once only).