In order to test a server that I am writing, I want to be able to start and stop it in the testing framework.
To do so, I am hoping I can integrate the context package in with the http.Server struct. I want to be able to stop the server when I call the Done function and the ctx.Done() channel returns something.
What I would love to do would be to just modify the http.Server.Serve() method, to accept a context.Context and check if it is done, on each iteration of the for loop, like this:
func (srv *server) Serve(ctx context.Context, l net.Listener) error {
defer l.Close()
var tempDelay time.Duration // how long to sleep on accept failure
for {
select {
case <-ctx.Done():
return nil
default:
}
... rest is same as original
However it seems like if I wanna add that check inside the for loop, I would have to rewrite a lot of the methods, because this method calls other private methods (like http.server.srv), which in turn call other private methods....
I also notice that the for loop will stop when the Accept() method on the listener returns an error.
However, I can't seem to figure out a way to get a listener to output an error from it's accept method without accessing its private methods as well.
It seems like I am doing something very stupid and wrong if I have to copy and paste half the http library just to let the server stop using the context package.
I know there are lots of solutions around for supporting context canceling for the ServeHTTP function, but that isn't what I am talking about. I wanna pass a context to the whole server, not just to each incoming request.
Is this just impossible?
Use httptest.Server to create a server that you can start and stop in tests.
If you do use the http.Server directly, you can break the Serve loop by closing the net.Listener. When a net.Listener is closed, any blocked Accept operations are unblocked and return errors. The Serve function returns if Accept returns a permanent error.
Related
I'm currently working on migrating our code from global sign package to go mongo-driver, not sure where should I use context.TODO() and context.Background(), it’s really confusing, I know both it returns non-nil empty, so should I use context.Background() in the main function & init() functions? And use context.TODO() in other places? Can anyone help with this?
Trying to check to see which param should I use context.TODO() or context.Background().
Check their documentation:
context.Background():
func Background() Context
Background returns a non-nil, empty Context. It is never canceled, has no values, and has no deadline. It is typically used by the main function, initialization, and tests, and as the top-level Context for incoming requests.
context.TODO():
func TODO() Context
TODO returns a non-nil, empty Context. Code should use context.TODO when it's unclear which Context to use or it is not yet available (because the surrounding function has not yet been extended to accept a Context parameter).
According to the doc, when you need a context but you don't have one (yet) and don't know what to use, use context.TODO(). This is exactly your case, using context.TODO() properly documents this. Also, static analysis tools and IDEs may give support discovering these contexts and give you warning later to address the issue.
Also note that if you do have a context when you have to use the mongo driver, then consider using that context, or deriving a new one from it.
For example, if you are writing an HTTP handler where you need to query some documents, the HTTP request http.Request already has a context which you can access using Request.Context(). This is a prime candidate to use when you call other API functions that require a context.
Why, you may ask? Because the request's context is cancelled when the HTTP client abandons / aborts the request, which means whatever you do, the client will not receive it, so for example if the HTTP handler just serves information, you may as well abort generating it, saving some resources. So if the request context is cancelled while you're executing a MongoDB query, you may as well cancel the query too, because you won't need (won't process) the result anyway. Cancelling the context passed to the query execution (Collection.Find() for example) is the way to tell the MongoDB server to cancel the query if possible (because you won't need the result).
So using the request context in this case can save you some CPU time and memory both on the HTTP server and on the MongoDB server as well.
Another example: let's say you have to produce the HTTP response within 10 seconds (may be a platform limitation), during which you have to execute a MongoDB operation, and you have to post process the results which takes 4 seconds. In this scenario if the MongoDB operation takes more than 6 seconds, you would not be able to complete the post processing to fit in the 10-second limit. So you might as well cancel the MongoDB operation if it doesn't complete in 6 seconds.
An easy way to solve this is to derive a context with 6 seconds timeout:
ctx, cancel := context.WithTimeout(r.Context(), 6 * time.Second)
defer cancel()
// ctx automatically times out after 6 seconds
curs, err := c.Find(ctx, bson.M{"some": "filter"})
In this example ctx will time out after 6 seconds, so if the c.Find() operation doesn't get finished by that, the passed ctx will signal that it can be cancelled (in which case an error will be returned). Since ctx is derived from r.Context(), if the request's context get's cancelled earlier (e.g. the HTTP client aborts), ctx will also be cancelled.
Now let's say you're not writing an HTTP handler but some other code (standalone app, background worker etc.), where you may not have a context, but you have your standards that you don't want to wait more than 30 seconds for a query. This is a prime example where you may derive a context from context.Background() using a 30-sec timeout:
ctx, cancel := context.WithTimeout(context.Background(), 30 * time.Second)
defer cancel()
// ctx automatically times out after 30 seconds
curs, err := c.Find(ctx, bson.M{"some": "filter"})
If the query finishes within 30 seconds, all good, you may use the results. If not, the context gets cancelled after 30 sec, and so c.Find() will also (likely) return an error.
context.TODO:
Code should use context.TODO when it's unclear which Context to use or it is not yet available (because the surrounding function has not yet been extended to accept a Context parameter).
context.Background:
Background returns a non-nil, empty Context. It is never canceled, has no values, and has no deadline. It is typically used by the main function, initialization, and tests, and as the top-level Context for incoming requests.
TODO is a placeholder to make it clear that the code should be updated with a more appropriate context in the future, but none is available yet. Background is used when such an indication isn't necessary and you simply need an empty root context with no values or cancellation. The values they return are identical; the only difference is semantic, in that one might scan code for TODO contexts (or use a linter to do so) as places that need to be updated/addressed.
I'm writing a custom terraform provider. I need to wrap the calls to the backed API in a "session", opening it before any calls are made and then closing it once all the terraform calls have completed.
Opening the session is straightforward, I can do that in the ConfigureContextFunc of the schema.Provider. Is there a way to set up a callback (or something) at the end of the application so I can close/"finalize" the session? I can imagine something specific to my resources, but that seems hacky. In my dream world I'd also be able to fail the apply if the close had an error.
Absent a nice finalize call is there a way to access the plan that I could use to determine that the current call is the last needed for the apply?
Update: I thought I could use a StopContext:
stopCtx, ok := schema.StopContext(ctx)
...
go func(ctx context.Context) {
// Wait for stop context cancellation
<-stopCtx.Done()
...
}
However, this is both deprecated and seems to only get called when stopping due to some outside trigger, like SIGINT, and not a regular exit (at least that's what I've been seeing).
After a fair amount of floundering I have what I believe to be a reasonable solution, and it even matches #Ben Hoyt's comment. We can defer and teardown in main.
func main() {
var prov *schema.Provider
defer func() {
xyz.PrividerTeardown(prov)
}()
plugin.Serve(&plugin.ServeOpts{
ProviderFunc: func() *schema.Provider {
prov = xyz.Provider()
return prov
},
})
}
I'll note that my comment on the question about not being around when the provider is made is incorrect. The provider is made in a main function in the provider code. In my (flimsy) defense I used the example scaffolding so that code came pre-written.
One thing that threw me was the docs for plugin.Serve
Serve serves a plugin. This function never returns and should be the final function called in the main function of the plugin.
It turns out that when the terraform action is done and the plugin is no longer needed, Serve does return and allows our defer to run. I did need to keep track of success or failure of all the calls that were made while Serve was active to know the status in my ProviderTeardown, but it is working.
When serving a bidirectional stream in gRPC in golang, the canonical stream handler looks something like this:
func (s *MyServer) MyBidiRPC(stream somepb.MyServer_MyBidiServer) error {
for {
data, err := stream.Recv()
if err == io.EOF {
return nil // clean close
}
if err != nil {
return err // some other error
}
// do things with data here
}
}
Specifically, when the handler for the bidi RPC returns, that is the signal to consider the server side closed.
This is a synchronous programming model -- the server stays blocked inside this goroutine (created by the grpc library) while waiting for messages from the client.
Now, I would like to unblock this Recv() call (which ends up calling RecvMsg() on an underlying grpc.ServerStream,) and return/close the stream, because the server process has decided that it is done with this client.
Unfortunately, I can find no obvious way to do this:
There's no Close() or CloseSend() or CloseRecv() or Shutdown()-like function on the bidi server interface generated for my service
The context inside the stream, which I can get at with stream.Context(), doesn't expose user-accessible the cancel function
I can't find a way to pass in a context on the "starting side" for a new connection accepted by the grpc.Server, where I could inject my own cancel function
I could close the entire grpc.Server by calling Stop(), but that's not what I want to do -- only this particular client connection (grpc.ServerStream) should be finished.
I could send a message to the client that makes the client in turn shut down the conection. However, this doesn't work if the client has fallen off the network, which would be solved with a timeout, which has to be pretty long to be generally robust. I want it now because I'm impatient, and, more importantly, at scale, dangling unresponsive clients can be a high cost.
I could (perhaps) dig through the grpc.ServerStream with reflection until I find the transportStream, and then dig out the cancel function out of that and call it. Or dig through the stream.Context() with reflection, and make my own cancel function reference to call. Neither of these seem well advised for future maintainers.
But surely these can't be the only options? Deciding that a particular client no longer needs to be connected is not magic space-alien science. How do I close this stream such that the Recv() call un-blocks, from the server process side, without involving a round-trip to the client?
Unfortunately I don't think there is a great way to do what you are asking. Depending on your goal, I think you have two options:
Run Recv in a goroutine and return from the bidi handler when you need it to return. This will close the context and unblock Recv. This is obviously suboptimal, as it requires care because you now have code executing outside the scope of the handler's execution. It is, however, the closest answer I can seem to find.
If you are trying to mitigate the impact of misbehaving clients by instituting timeouts, you might be able to offload the work of this to the framework with KeepaliveEnforcementPolicy and/or KeepaliveParams. This is probably preferable if this aligns with the reason you are hoping to close the connection, but otherwise isn't of much use.
I’m just getting started with Golang and the net/rpc package. I’m trying to understand when you might use the asynchronous client.Go() call over the client.Call() method most examples online use. Would calling client.Call asynchronously via something like
go client.Call(...)
essentially be the same as using the client.Go call? I’ve seen this example online too (for example when calling multiple simultaneous RPCs).
As documented:
Go invokes the function asynchronously. It returns the Call structure representing the invocation. The done channel will signal when the call is complete by returning the same Call object. If done is nil, Go will allocate a new channel. If non-nil, done must be buffered or Go will deliberately crash.
What this means is that it issues the command, but does not wait for it to finish.
By contrast:
Call invokes the named function, waits for it to complete, and returns its error status.
Neither method executes in a goroutine directly*--this is left as an exercise for the caller (so an argument might be made that Go is a misnomer).
If you look at the source to Call, perhaps it is more clear:
func (client *Client) Call(serviceMethod string, args interface{}, reply
interface{}) error {
call := <-client.Go(serviceMethod, args, reply, make(chan *Call, 1)).Done
return call.Error
}
So in actuality, Call is a wrapper around Go, which waits for the operation to complete, whereas Go is the underlying function, which leaves waiting up to the caller.
*Clearly, in the background, a goroutine is involved somewhere, since this is a non-blocking operation.
I need to embed the default http.Server in my own server struct and customize the Serve method.
The server needs to short circuit the go c.serve() call and only run that line if it has the computing resources available to respond within 50ms. Otherwise the server is just going to send a 204 and move on.
This is almost straightforward.
type PragmaticServer struct {
http.Server
Addr string
Handler http.Handler
}
func (srv *PragmaticServer) Serve(l net.Listener) error {
defer l.Close()
var tempDelay time.Duration // how long to sleep on accept failure
for {
// SNIP for clarity
c, err := srv.newConn(rw)
if err != nil {
continue
}
c.setState(c.rwc, StateNew) // before Serve can return
go c.serve()
}
}
So, again. This almost works. Except that srv.newConn is an unexported method, as is c.serve and c.setState, which means that I end up having to copy and paste pretty much the entirety of net/http in order for this to compile. Which is basically a fork. Is there any better way to do this?
Unfortunately, you're not going to be able to do that without reimplementing most of the Server code. Short of that, we usually intercept the call either just before at conn.Accept, or just after at Handler.ServerHTTP.
The first method is to create a custom net.Listener that filters out connections before they are even handed off to the http.Server. While this can respond faster, and consume fewer resources, it however makes it less convenient to write http responses, and precludes you from limiting requests on already open connections.
The second way to handle this, is to just wrap the handlers and intercept the request before any real work has been done. You most likely want to create a http.Handler to filter the requests, and pass them through to your main handler. This can also be more flexible, since you can filter based on the route, or other request information if you so choose.