I need to embed the default http.Server in my own server struct and customize the Serve method.
The server needs to short circuit the go c.serve() call and only run that line if it has the computing resources available to respond within 50ms. Otherwise the server is just going to send a 204 and move on.
This is almost straightforward.
type PragmaticServer struct {
http.Server
Addr string
Handler http.Handler
}
func (srv *PragmaticServer) Serve(l net.Listener) error {
defer l.Close()
var tempDelay time.Duration // how long to sleep on accept failure
for {
// SNIP for clarity
c, err := srv.newConn(rw)
if err != nil {
continue
}
c.setState(c.rwc, StateNew) // before Serve can return
go c.serve()
}
}
So, again. This almost works. Except that srv.newConn is an unexported method, as is c.serve and c.setState, which means that I end up having to copy and paste pretty much the entirety of net/http in order for this to compile. Which is basically a fork. Is there any better way to do this?
Unfortunately, you're not going to be able to do that without reimplementing most of the Server code. Short of that, we usually intercept the call either just before at conn.Accept, or just after at Handler.ServerHTTP.
The first method is to create a custom net.Listener that filters out connections before they are even handed off to the http.Server. While this can respond faster, and consume fewer resources, it however makes it less convenient to write http responses, and precludes you from limiting requests on already open connections.
The second way to handle this, is to just wrap the handlers and intercept the request before any real work has been done. You most likely want to create a http.Handler to filter the requests, and pass them through to your main handler. This can also be more flexible, since you can filter based on the route, or other request information if you so choose.
Related
I’m currently maintaining a few HTTP APIs based on the standard library and gorilla mux and running in kubernetes (GKE).
We’ve adopted the http.TimeoutHandler as our “standard” way to have a consistent timeout error management.
A typical endpoint implementation will use the following “chain”:
MonitoringMiddleware => TimeoutMiddleware => … => handler
so that we can monitor a few key metrics per endpoint.
One of our API is typically used in a “fire and forget” mode meaning that clients will push some data and not care for the API response. We are facing the issue that
the Golang standard HTTP server will cancel a request context when the client connection is no longer active (godoc)
the TimeoutHandler will return a “timeout” response whenever the request context is done (see code)
This means that we are not processing requests to completion when the client disconnects which is not what we want and I’m therefore looking for solutions.
The only discussion I could find that somewhat relates to my issue is https://github.com/golang/go/issues/18527; however
The workaround is your application can ignore the Handler's Request.Context()
would mean that the monitoring middleware would not report the "proper" status since the Handler would perform the request processing in its goroutine but the TimeoutHandler would be enforcing the status and observability would be broken.
For now, I’m not considering removing our middlewares as they’re helpful to have consistency across our APIs both in terms of behaviours and observability. My conclusion so far is that I need to “fork” the TimeoutHandler and use a custom context for when an handler should not depend on the client waiting for the response or not.
The gist of my current idea is to have:
type TimeoutHandler struct {
handler Handler
body string
dt time.Duration
// BaseContext optionally specifies a function that returns
// the base context for controling if the server request processing.
// If BaseContext is nil, the default is req.Context().
// If non-nil, it must return a non-nil context.
BaseContext func(*http.Request) context.Context
}
func (h *TimeoutHandler) ServeHTTP(w ResponseWriter, r *Request) {
reqCtx := r.Context()
if h.BaseContext != nil {
reqCtx = h.BaseContext(r)
}
ctx, cancelCtx := context.WithTimeout(reqCtx, h.dt)
defer cancelCtx()
r = r.WithContext(ctx)
...
case <-reqCtx.Done():
tw.mu.Lock()
defer tw.mu.Unlock()
w.WriteHeader(499) // write status for monitoring;
// no need to write a body since no client is listening.
case <-ctx.Done():
tw.mu.Lock()
defer tw.mu.Unlock()
w.WriteHeader(StatusServiceUnavailable)
io.WriteString(w, h.errorBody())
tw.timedOut = true
}
The middleware BaseContext callback would return context.Background() for requests to the “fire and forget” endpoint.
One thing I don’t like is that in doing so I’m losing any context keys written so this new middleware would have strong usage constraints. Overall I feel like this is more complex than it should be.
Am I completely missing something obvious?
Any feedback on API instrumentation (maybe our middlewares are an antipattern) /fire and forget implementations would be welcomed!
EDIT: as most comments are that a request for which the client does not wait for a response has unspecified behavior, I checked for more information on typical clients for which this happens.
From our logs, this happens for user agents that seem to be mobile devices. I can imagine that connections can be much more unstable and the problem will likely not disappear.
I would therefore not conclude that I shouldn't find a solution since this is currently creating false-positive alerts.
When serving a bidirectional stream in gRPC in golang, the canonical stream handler looks something like this:
func (s *MyServer) MyBidiRPC(stream somepb.MyServer_MyBidiServer) error {
for {
data, err := stream.Recv()
if err == io.EOF {
return nil // clean close
}
if err != nil {
return err // some other error
}
// do things with data here
}
}
Specifically, when the handler for the bidi RPC returns, that is the signal to consider the server side closed.
This is a synchronous programming model -- the server stays blocked inside this goroutine (created by the grpc library) while waiting for messages from the client.
Now, I would like to unblock this Recv() call (which ends up calling RecvMsg() on an underlying grpc.ServerStream,) and return/close the stream, because the server process has decided that it is done with this client.
Unfortunately, I can find no obvious way to do this:
There's no Close() or CloseSend() or CloseRecv() or Shutdown()-like function on the bidi server interface generated for my service
The context inside the stream, which I can get at with stream.Context(), doesn't expose user-accessible the cancel function
I can't find a way to pass in a context on the "starting side" for a new connection accepted by the grpc.Server, where I could inject my own cancel function
I could close the entire grpc.Server by calling Stop(), but that's not what I want to do -- only this particular client connection (grpc.ServerStream) should be finished.
I could send a message to the client that makes the client in turn shut down the conection. However, this doesn't work if the client has fallen off the network, which would be solved with a timeout, which has to be pretty long to be generally robust. I want it now because I'm impatient, and, more importantly, at scale, dangling unresponsive clients can be a high cost.
I could (perhaps) dig through the grpc.ServerStream with reflection until I find the transportStream, and then dig out the cancel function out of that and call it. Or dig through the stream.Context() with reflection, and make my own cancel function reference to call. Neither of these seem well advised for future maintainers.
But surely these can't be the only options? Deciding that a particular client no longer needs to be connected is not magic space-alien science. How do I close this stream such that the Recv() call un-blocks, from the server process side, without involving a round-trip to the client?
Unfortunately I don't think there is a great way to do what you are asking. Depending on your goal, I think you have two options:
Run Recv in a goroutine and return from the bidi handler when you need it to return. This will close the context and unblock Recv. This is obviously suboptimal, as it requires care because you now have code executing outside the scope of the handler's execution. It is, however, the closest answer I can seem to find.
If you are trying to mitigate the impact of misbehaving clients by instituting timeouts, you might be able to offload the work of this to the framework with KeepaliveEnforcementPolicy and/or KeepaliveParams. This is probably preferable if this aligns with the reason you are hoping to close the connection, but otherwise isn't of much use.
I'm learning Go and I'm trying to understand how to properly deal with panics from external packages.
Here is a runnable example, say a package defines the doFoo method. (It's located in the same package here for the sake of the example )
package main
import (
"log"
"net/http"
"sync"
"time"
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
)
// Method from External package
func doFoo() {
var wg sync.WaitGroup
wg.Add(1)
// Do some cool async stuff
go func() {
time.Sleep(500)
wg.Done()
panic("Oops !")
}()
}
func router() *mux.Router {
var router = mux.NewRouter().StrictSlash(true)
router.HandleFunc("/doFoo", index).Methods("GET")
return router
}
func main() {
log.Fatal(http.ListenAndServe(":8080", handlers.RecoveryHandler()(router())))
}
func index(w http.ResponseWriter, r *http.Request) {
defer func() {
recover()
w.WriteHeader(http.StatusInternalServerError)
}()
doFoo()
w.WriteHeader(http.StatusOK)
}
Invoking the doFoo method will crash the server, I appreciate that is correct behavior, since the application is now in an undermined state. And it's best to crash and have subsequent requests forwarded to a different processes trough some load balancer.
But, my api server might still be serving other clients, it might be maintaining websockets, and I Might would also want to return a 500 error here.
Coming from nodejs, I am used to the concept of uncaughtException, for handeling uncaptured synchronous exceptions and unhandledRejection to handle uncaptured asynchronous exceptions. These two process constructs give the developer the choice to either crash the program right away ( if it makes sense ), or log the error, return a proper http code, and then maybe shutdown gracefully if needed.
In my online research I find a lot of resources saying, panic's are not like exceptions, they are unusual, you don't need to worry about them. But it seems like it's actually very easy to cause a panic when writing code. It's completely up to the developer to ensure his library does not panic, the human factor is 100% involved here.
This leads me to wonder, do I need to audit the entire code base of every single package I'm going to use, including all the package dependencies as well ? just because I have no means of safeguarding against a missed recover in some external package that will take down my whole server and ruin my user's experience ?
Or is there some strategy I am not aware of that I can fail gracefully when an asynchronous panic occurs in library code ?
I noticed there is graceful shutdown since 1.8, but I can't use this because my program has already crashed.
https://golang.org/pkg/net/http/#Server.Shutdown
There is the gorilla recovery handler, but again, this only protects against synchronous panics.
http://www.gorillatoolkit.org/pkg/handlers#RecoveryHandler
Update:
I am aware that panics are not exceptions. Restating that does not answer the question, panics and exceptions is not what this question is about. This question is about understanding what tools the language may provide to enforce boundaries without bestowing the need to read every single line in the entire package tree onto the developer. If it's not possible in the language then stating that is a valid answer. I just don't know if it is or not.
Panics are not exceptions. Do not treat them like exceptions and you will be fine.
First things first: Package APIs should never panic, they should always return an error except in certain very rare cases, and then they must be clearly documented as to when and why they can panic (regexp.MustCompile is a good example of something that may panic). Any package that panics if it hits an error (and doesn't have a very good reason to do so) is bad, don't use it.
If you do bounds checking, make sure not to acess nil pointers, etc you should never have to worry about panics.
As for recovering panics in a goroutine, unless the goroutine has its own recovery handler you can't.
If the goroutine in from a third party library, don't use that library! If they are lax enough not to check edge cases and/or are lazy enough to just panic on error, why are you using their code? Who knows what other mines it holds?
If the goroutine is your own code, try to eliminate things that can panic, then add a recovery handler to catch the ones you can't prevent if needed.
In order to test a server that I am writing, I want to be able to start and stop it in the testing framework.
To do so, I am hoping I can integrate the context package in with the http.Server struct. I want to be able to stop the server when I call the Done function and the ctx.Done() channel returns something.
What I would love to do would be to just modify the http.Server.Serve() method, to accept a context.Context and check if it is done, on each iteration of the for loop, like this:
func (srv *server) Serve(ctx context.Context, l net.Listener) error {
defer l.Close()
var tempDelay time.Duration // how long to sleep on accept failure
for {
select {
case <-ctx.Done():
return nil
default:
}
... rest is same as original
However it seems like if I wanna add that check inside the for loop, I would have to rewrite a lot of the methods, because this method calls other private methods (like http.server.srv), which in turn call other private methods....
I also notice that the for loop will stop when the Accept() method on the listener returns an error.
However, I can't seem to figure out a way to get a listener to output an error from it's accept method without accessing its private methods as well.
It seems like I am doing something very stupid and wrong if I have to copy and paste half the http library just to let the server stop using the context package.
I know there are lots of solutions around for supporting context canceling for the ServeHTTP function, but that isn't what I am talking about. I wanna pass a context to the whole server, not just to each incoming request.
Is this just impossible?
Use httptest.Server to create a server that you can start and stop in tests.
If you do use the http.Server directly, you can break the Serve loop by closing the net.Listener. When a net.Listener is closed, any blocked Accept operations are unblocked and return errors. The Serve function returns if Accept returns a permanent error.
In Go using the standard net/rpc functionality, I would like to determine what the IP address an inbound RPC request is coming from. The underlying http functionality appears to provide this in the http.Request object, but I cannot see any way of getting at that from the default RPC handler (set using rpc.HandleHTTP).
Is there some hidden mechanism for getting at the underlying http.Request, or do I have to do something fancier with setting up a different HTTP responder?
As far as I know, it is not possible to grab the address from somewhere in the default server.
The service call method, which calls the request receiving function, does not provide any access to the remote data stored in the codec.
If http handlers could be registered twice (which they can't), you could have overwritten the DefaultRPCPath for the HTTP Handler setup by HandleHTTP. But that's simply not possible today.
What you can do, without much fuss, is to build a RPC server based on the default one with your own ServeHTTP method:
import (
"log"
"net"
"net/http"
"net/rpc"
)
type myRPCServer struct {
*rpc.Server
}
func (r *myRPCServer) ServeHTTP(w http.ResponseWriter, req *http.Request) {
log.Println(req.RemoteAddr)
r.Server.ServeHTTP(w, req)
}
func (r *myRPCServer) HandleHTTP(rpcPath, debugPath string) {
http.Handle(rpcPath, r)
}
func main() {
srv := &myRPCServer{rpc.NewServer()}
srv.HandleHTTP(rpc.DefaultRPCPath, rpc.DefaultDebugPath)
// ...http listen code...
}
The downside of this, is of course, that you can't use rpc.Register anymore. You have to write srv.Register.
Edit: I forgot that you'd need to write your own HandleHTTP as well. The reason for this is, that if you embed the RPC server and you write srv.HandleHTTP it is called on the embedded instance, passing the embedded instance to http.Handle(), ignoring your own definition of ServeHTTP. This has the drawback, that you won't have the ability to debug your RPC server using the debug path, as the server's HandleHTTP uses a private debug handler (rpc.debugHTTP) which you can't access.
You can also use https://github.com/valyala/gorpc instead of net/rpc, which passes client address to RPC server handler - see http://godoc.org/github.com/valyala/gorpc#HandlerFunc for details.
The net/rpc package is at a higher level of abstraction than tcp or http. Since it can use multiple codecs it doesn't make sense for it to offer a way to get at the ip address of the inbound rpc. It's theoretically possible someone could implement a code that talks on unix sockets instead or using radio transmitters.
If you want access to specifics of the transport layer you will have to drop a level in the stack and use net or net/http directory to make your rpc service.
It seems that there is currently no way to do this in rpc function.
See this link for more info
Here is a summary.
Q:
Right now RemoteAddr() method can be called to get the RPC client's address only on
net.Conn when the client dials to server, but suppose that your server has multiple
clients connected and each of this clients are calling an RPC exported method. Is there
a way to implement a method to get the caller's remote address from inside the RPC
method?
func (t *Type) Method(args *Args, reply *string) error {
//something like
*reply = Caller.RemoteAddr().String()
// who called the method now?
return nil
}
A:
I'm skeptical. It would take an API change (not necessarily a backwards incompatible one
but still a bit of a redesign) to supply this.