I have a unary interceptor that contains the following code:
func (m Middlewares) LocationInterceptor(c context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
ctx := c.(HarmonyContext)
location, ok := req.(interface{ GetLocation() *corev1.Location })
if !ok {
panic("location middleware used on message without a location")
}
ctx.Location := location.GetLocation()
return handler(c, req)
}
How would I be able to convert this to a stream interceptor, if I know that the stream will definitely only stream from server to client? In addition, is there any way to make it only intercept when the moment the stream begins?
func (m Middlewares) LocationInterceptorStream(srv interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
wrappedStream := ss.(HarmonyWrappedServerStream)
return handler(srv, wrappedStream)
}
For a "server streaming RPC", the client sends one message (the request), and the server responds with multiple messages. In the stream interceptor, you need to call ServerStream.RecvMsg(...) once to get the request from the client. You will then need to pass a "wrapped stream" into handler which will later return this message on the first call to RecvMsg. For gRPC using protobuf messages, you can do something like the following. This serverStreamWrapper will allow you to call peekRequest(...) in the interceptor to get the request, then pass the stream on to the handler.
For this specific example, you may also be able to implement the logic in the wrapped RecvMsg(...) function itself. This function gets called by gRPC to read the request from the client to the server. This may be simpler than trying to store the request.
type serverStreamWrapper struct {
peekedRequest proto.Message
wrappedStream grpc.ServerStream
}
func newServerStreamWrapper(stream grpc.ServerStream) *serverStreamWrapper {
return &serverStreamWrapper{nil, stream}
}
func (s *serverStreamWrapper) peekRequest(msg interface{}) error {
protoMsg := msg.(proto.Message)
if protoMsg == nil {
panic("BUG: msg must not be nil")
}
if s.peekedRequest != nil {
panic("BUG: Must only called peekRequest once")
}
err := s.wrappedStream.RecvMsg(protoMsg)
if err == nil {
s.peekedRequest = protoMsg
}
return err
}
func (s *serverStreamWrapper) RecvMsg(msg interface{}) error {
if s.peekedRequest != nil {
protoMsg := msg.(proto.Message)
proto.Reset(protoMsg)
proto.Merge(protoMsg, s.peekedRequest)
s.peekedRequest = nil
return nil
}
return s.wrappedStream.RecvMsg(msg)
}
Related
I have a Go server defined like this in proto file:
syntax = "proto3";
package go-package;
option go_package = "github.com/path/to/go-package";
import "vibe.proto";
service MyService {
rpc Function (stream TheRequest) returns (stream TheResponse) {}
}
message TheRequest {
oneof Payload {
Config config = 1;
Messages messages = 2;
}
}
message Config {
// config some fields here
}
message Messages {
repeated string messages = 1;
}
I'm using https://github.com/grpc-ecosystem/go-grpc-middleware as an interceptor engine. I need to implement logging of the request or Messages when I have a panic, because right now the code is panicked, but i don't have any information about the request.
Just add interceptor with recovery, simple example:
func RecoveryUnaryInterceptor(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (_ interface{}, err error) {
defer func() {
if r := recover(); r != nil {
err = status.Error(codes.Internal, fmt.Sprintf("Panic: `%s` %s", info.FullMethod, string(debug.Stack())))
}
}()
return handler(ctx, req)
}
In order to perform Authorization, some attributes from the request is to be read so that input for Authorization Server can be made
For example, this is the interceptor. Here prepareAuthZInput is called to preparing the input
func AuthInterceptor(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
input := prepareAuthZInput(info.FullMethod, req)
}
In this function, there's a big if-else part which checks for the actual type for the request, type casts it and then performs the input preparation.
func prepareAuthZInput(method string, req interface{}) {
var input Input
if methodName = "/Data/Call" {
callRequest, ok := req.(CallRequest)
if ok {
// prepare input from callRequest
}
} else if methodName = "/Data/Receive" {
receiveRequest, ok := req.(ReceiveRequest)
if ok {
// prepare input from receiveRequest
}
}
return input
}
How can I improve this code?
When doing something like this, it's typical to add auth data to the metadata instead of the request messages. This way the server doesn't need to inspect all the possible request payload types.
If you must use the request payload, it would be more idiomatic to use a type switch instead:
switch r := req.(type) {
case CallRequest: // r is a CallRequest...
case ReceiveRequest: // r is a ReceiveRequest...
default:
return status.Errorf(codes.Unimplemented, "unknown request type: %T", req)
}
gRPC's Go library provides interfaces for creating your own custom interceptors (i.e. middleware functions), and I'm attempting to write two logging interceptors. The first is a Unary Server Interceptor where I'm easily able to log the request parameters using the object passed into the interceptor function.
func loggingUnary(context context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {
start := time.Now()
resp, err := handler(context, req)
printLogMessage(err, info.FullMethod, context, time.Since(start), req)
return resp, err
}
How can I do the same with the Stream Server Interceptor which doesn't conveniently pass the request object as a parameter? Is there another way to access the request?
func loggingStream(srv interface{}, stream grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
start := time.Now()
err := handler(srv, stream)
printLogMessage(err, info.FullMethod, stream.Context(), time.Since(start), "")
return err
}
This is a bit old now, but the easiest way to extend your interception into the stream is to create a grpc.ServerStream wrapper, then wrap the real ServerStream in your interceptor. In that way, your intercepting code can handle the received and sent messages in the stream.
// A wrapper for the real grpc.ServerStream
type LoggingServerStream struct {
inner grpc.ServerStream
}
func (l LoggingServerStream) SetHeader(m metadata.MD) error {
return l.SetHeader(m)
}
func (l LoggingServerStream) SendHeader(m metadata.MD) error {
return l.SendHeader(m)
}
func (l LoggingServerStream) SetTrailer(m metadata.MD) {
l.SetTrailer(m)
}
func (l LoggingServerStream) Context() context.Context {
return l.Context()
}
func (l LoggingServerStream) SendMsg(m interface{}) error {
fmt.Printf("Sending Message: type=%s\n", reflect.TypeOf(m).String())
return l.SendMsg(m)
}
func (l LoggingServerStream) RecvMsg(m interface{}) error {
fmt.Printf("Receiving Message: type=%s\n", reflect.TypeOf(m).String())
return l.RecvMsg(m)
}
The interceptor:
func LoggingStreamInterceptor() grpc.StreamServerInterceptor {
return func(srv interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
return handler(srv, LoggingServerStream{inner:ss})
}
}
Any state you need to keep and go in your wrapper.
The stream handler can be called multiple times during the lifetime of the request that created the stream, that's why the request is not part of the arguments to the handler (and to any interceptor, either). You could place the request (or better, a copy of the data that you want logged and not a reference to the request itself) in the stream context (assuming you are in control of the code that creates the ServerStream object). I would rather log the request parameters once, when the stream is created and not on every call to the handler (so each request is logged once only).
I am trying to add context to Authorization middleware. The ContextHandler is a handler which will be passed to api handlers to take care of connections and config variables. A struct Method ServeHTTP also has been added to the ContextHandler so that it satisfies the net/Http interface for handling requests properly.
CheckAuth is the middle ware which takes in the request to check token validation etc, If token is valid, executes the ServeHTTP method and if not, Returns the appropriate error in the response.
Code compiles, but i am getting error in the ServeHTTP method.
type ContextHandler struct {
*AppContext
Handler func(*AppContext, http.ResponseWriter, *http.Request)(int, error)
}
type AppContext struct {
Db *mgo.Session
Config *simplejson.Json
}
func (ah *ContextedHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
status, err := ah.handler(ah.AppContext, w, r)
if err != nil {
switch status {
case http.StatusNotFound:
http.NotFound(w, r)
case http.StatusInternalServerError:
http.Error(w, http.StatusText(status), status)
default:
http.Error(w, http.StatusText(405), 405)
}}}
func CheckAuth(h http.Handler) http.Handler {
log.Println("Entered in CheckAuth")
f := func( w http.ResponseWriter, r *http.Request) {
authorizationToken := r.Header.Get("Authorization")
if authorizationToken != ""{
secret := []byte("somejunk")
var credentials authorization
token, err := jwt.ParseWithClaims(authorizationToken, &credentials, func(t *jwt.Token) (interface{}, error) {
return []byte(secret), nil
})
if err == nil && token.Valid {
//If everything is fine serve the Http request
h.ServeHTTP( w, r)
return
} else {
//Some response returned
json.NewEncoder(w).Encode(response)
return
}
//Check if user exists in the database
if dberr != nil {
//SOmeresponse to be returned
json.NewEncoder(w).Encode(response)
return
}
}else{
response := simplejson.New()
//response authorization header is missing
json.NewEncoder(w).Encode(response)
return
}
}
return http.HandlerFunc(f)
}
func Initdb(configfile *simplejson.Json) *mgo.Session {
//return mongodbsession, copy and close while using it
}
In main.go file in the parent package
func main() {
var FileUploadContextHandler *ContextedHandler = &ContextedHandler{&context, filesystem.FileUpload}
router.Methods("POST").Path("/decentralizefilesystem/fileupload").Name("FileUpload").Handler(CheckAuth(FileUploadContextHandler))
}
I am getting this error
2018/07/08 20:45:38 http: panic serving 127.0.0.1:52732: runtime error: invalid memory address or nil pointer dereference
goroutine 35 [running]:
net/http.(*conn).serve.func1(0xc4202ce140)
/usr/local/go/src/net/http/server.go:1726 +0xd0
panic(0x6fe680, 0x92cb10)
/usr/local/go/src/runtime/panic.go:502 +0x229
gitlab.com/mesha/Gofeynmen/vendor/gopkg.in/mgo%2ev2.(*Session).Copy(0x0, 0x7ff9485fb060)
/home/feynman/goworkspace/src/gitlab.com/mesha/Gofeynmen/vendor/gopkg.in/mgo.v2/session.go:1589 +0x22
gitlab.com/mesha/Gofeynmen/appsettings.CheckAuth.func1(0x7ff9485fb060, 0xc420276300, 0xc4202e4200)
/home/feynman/goworkspace/src/gitlab.com/mesha/Gofeynmen/appsettings/appsettings.go:115 +0x361
net/http.HandlerFunc.ServeHTTP(0xc420290180, 0x7ff9485fb060, 0xc420276300, 0xc4202e4200)
/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/gorilla/mux.(*Router).ServeHTTP(0xc42024a310, 0x7ff9485fb060, 0xc420276300, 0xc4202e4200)
/home/feynman/goworkspace/src/github.com/gorilla/mux/mux.go:162 +0xed
github.com/gorilla/handlers.loggingHandler.ServeHTTP(0x7a8120, 0xc42000e018, 0x7a7b20, 0xc42024a310, 0x7aad60, 0xc4202f0000, 0xc4202e4000)
/home/feynman/goworkspace/src/github.com/gorilla/handlers/handlers.go:69 +0x123
github.com/gorilla/handlers.(*cors).ServeHTTP(0xc4202c4090, 0x7aad60, 0xc4202f0000, 0xc4202e4000)
/home/feynman/goworkspace/src/github.com/gorilla/handlers/cors.go:52 +0xa3b
net/http.serverHandler.ServeHTTP(0xc4202da0d0, 0x7aad60, 0xc4202f0000, 0xc4202e4000)
/usr/local/go/src/net/http/server.go:2694 +0xbc
net/http.(*conn).serve(0xc4202ce140, 0x7ab120, 0xc42025e100)
/usr/local/go/src/net/http/server.go:1830 +0x651
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2795 +0x27b
It's likely an attempt to dereference ah from (ah *ContextedHandler), when ah is not a pointer to a ContextedHandler.
The types in this assignment don't match up:
var FileUploadContextHandler *ContextedHandler =
ContextedHandler{&context, filesystem.FileUpload}
On the left side you have type *ContextedHandler. On the right side you have type ContextedHandler.
Did you mean
var FileUploadContextHandler *ContextedHandler =
&ContextedHandler{&context, filesystem.FileUpload}
Or did you mean
var FileUploadContextHandler ContextedHandler =
ContextedHandler{&context, filesystem.FileUpload}
?
The argument passed to the CheckAuth function appears to not match the function signature either:
CheckAuth(FileUploadContextHandler)
FileUploadContextHandler is type *ContextedHandler. The function signature is:
func CheckAuth(h contextHandlerFunc) contextHandlerFunc
The type definition of contextHandlerFunc does not appear to be part of the code you shared.
A problem with this line:
router.Methods("POST").Path("/decentralizefilesystem/fileupload").Name("FileUpload").Handler(CheckAuth(FileUploadContextHandler))
...would be easier to track down if you broke it up into variable assignments on several lines and then figured out which line the panic pointed to.
I need to log the response body in a middleware of gin, but I don't find how to get the response body. Can anyone help?
I am using a middleware like this:
func Logger() gin.HandlerFunc {
return func(c *gin.Context) {
c.Next()
statusCode := c.Writer.Status()
if statusCode >= 400 {
//ok this is an request with error, let's make a record for it
//log body here
}
}
}
My question is, how to get response body from Context in middleware?
You need to intercept writing of response and store it somewhere first. Then you can log it. And to do that you need to implement your own Writer intercepting Write() calls.
For example, as follows:
type bodyLogWriter struct {
gin.ResponseWriter
body *bytes.Buffer
}
func (w bodyLogWriter) Write(b []byte) (int, error) {
w.body.Write(b)
return w.ResponseWriter.Write(b)
}
func ginBodyLogMiddleware(c *gin.Context) {
blw := &bodyLogWriter{body: bytes.NewBufferString(""), ResponseWriter: c.Writer}
c.Writer = blw
c.Next()
statusCode := c.Writer.Status()
if statusCode >= 400 {
//ok this is an request with error, let's make a record for it
// now print body (or log in your preferred way)
fmt.Println("Response body: " + blw.body.String())
}
}
Then use this middleware like this:
router.Use(ginBodyLogMiddleware)
Note that this sill won't work for static files as gin does not seem to use c.Writer for them. But in most cases, that's what you want anyway.
If you want to intercept all files, you need to use a slightly more complicated approach. Instead of Middleware, you'll need to implement a wrapper http.Handler that will wrap gin.Engine and will use same approach as shown above to intercept and log whatever is written to http.ResponseWriter. Then run gin server like this:
ginRouter := gin.New()
// configure your middleware and routes as required
// Run http server as follows, where bodyLogHandler is your wrapper handler
http.ListenAndServe(bindAddress, &bodyLogHandler{wrappedHandler: ginRouter}
FYI
Note: implement WriteString() if using c.String() for writing response body
type bodyLogWriter struct {
gin.ResponseWriter
body *bytes.Buffer
}
func (w bodyLogWriter) Write(b []byte) (int, error) {
w.body.Write(b)
return w.ResponseWriter.Write(b)
}
func (w bodyLogWriter) WriteString(s string) (int, error) {
w.body.WriteString(s)
return w.ResponseWriter.WriteString(s)
}
func ginBodyLogMiddleware() gin.HandlerFunc {
return func(c *gin.Context) {
blw := &bodyLogWriter{body: bytes.NewBufferString(""), ResponseWriter: c.Writer}
c.Writer = blw
c.Next()
fmt.Println("Response body: " + blw.body.String())
}
}
...
// register
router := r.Group("/", ginBodyLogMiddleware())