Does it make sense to additionally use http.TimeoutHandler if I already set the server's ReadTimeout and WriteTimeout? It seems as if this scenario is mutually exclusive?
These two deal with different aspects of http request/response lifecycle.
http.TimeoutHandler is used to limit execution time of the http.Handler. it will return 503 status code to the client, if http.Handler doesn't finish in stipulated time.
While, ReadTimeout and WriteTimeout deals with network I/O timeout, i.e time required to read/write request/response body to your client respectively.
So, http.TimeoutHandler handles the case where your handler (code block that handles http request) need to be complete in set amount of time, by wrapping your original handler. while http.WriteTimeout or http.ReadTimeout is used when you dont want to wait for network read/write of request/response indefinitely.
This blog article does a good job of explaining, but essentially they fulfil slightly different roles, as demonstrated here:
Link to blog article
You've not included what your usecase is, but hopefully this is enough information to be able to make a decision.
If you are going to expose your app naked (directly) no HAproxy/Nginx in front, changing the defaults may help, for example (fine-tune them based on your requirements):
// configure server
srv := &http.Server{
Addr: ":8080",
Handler: router,
ReadTimeout: 5 * time.Second,
WriteTimeout: 7 * time.Second,
MaxHeaderBytes: 1 << 20,
}
log.Fatal(srv.ListenAndServe())
Here is a very nice article explaining more about the topic: https://blog.cloudflare.com/the-complete-guide-to-golang-net-http-timeouts/
Also exists important difference between them if you are use Context.
In case if you are use http.TimeoutHandler, you will get cancel signal from request context <-ctx.Done() with which you can terminate running processes.
In case if you are use WriteTimeout and if given time limit is reached you will not get cancel signal from request context.
Here is one good sample Diving into Go's HTTP server timeouts
There are two cases when TimeoutHandler returns 503.
The first is, of course, when the deadline is set on the context fires (it could have been set somewhere else, in theory).
The other is if the context was canceled for some other reason (such as the client leaving). They are distinguishable from the client-side because there’s no response body in the latter case
Configuring ReadHeaderTimeout would protect directly against this attack by closing the connection once the deadline is reached, to Mitigate a Slowloris Attack
ReadTimeout is the one we would need to configure here since it covers the header and the body of the HTTP requests, to Mitigate R.U.D.Y attack
Let’s work through the timeout math
Let’s say we want, generally, a 10-second request timeout. So we set TimeoutHandler’s timeout to 10 seconds.
We need to pick a ReadHeaderTimeout that is basically independent from that (because the handler timeout doesn’t start until after the header read is complete). It seems reasonable to pick 5 seconds.
As discussed above, we prefer the ReadTimeout to be longer than the handler timeout, so the client has a chance of getting the response. Because ReadTimeout ticks away during the header read, the calculation for this is something like:
ReadTimeout := handler_timeout + ReadHeaderTimeout + wiggle_room
Related
I want to send a fairly large number (several thousand) of HTTP requests ASAP, without putting too much load on the CDN (has an https: URL, and ALPN selects HTTP/2 during the TLS phase) So, staggering (i.e. time shifting) the requests is an option, but I don't want to wait TOO long (minimize errors AND total round-trip time) and I'm not being rate limited by the server at the scale I'm operating yet.
The problem I'm seeing originates from h2_bundle.go and specifically in either writeFrame or onWriteTimeout when about 500-1k requests are in-flight, which manifests during io.Copy(fileWriter, response.Body) as:
http2ErrCodeInternal = "INTERNAL_ERROR" // also IDs a Stream number
// ^ then io.Copy observes the reader encountering "unexpected EOF"
I'm fine sticking with HTTP/1.x for now, but I would love an explanation re: what's going on. Clearly, people DO use Go to make a lot of round-trips happen per unit time, but most advice I can find is from the perspective of the server, not clients. I've already tried specifying all the relevant time-outs I can find, and cranking up connection pool max sizes.
Here's my best guess at what's going on:
The rate of requests is overwhelming a queue of connections or some other resource in the HTTP/2 internals. Maybe this is fix-able in general or possible to fine tune for my specific use case, but the fastest way to overcome this kind of problem is to rely on HTTP/1.1 entirely, as well as implement limited retry + rate limiting mechanisms.
Aside, I am now using a single retry and rate.Limiter from https://pkg.go.dev/golang.org/x/time/rate#Limiter in addition to the "ugly hack" of disabled HTTP/2, so that outbound requests are able send an initial "burst" of M requests, and then "leak more gradually" at a given rate of N/sec. Ultimately, the errors from h2_bundle.go are just too ugly for end-users to parse. An expected/unexpected EOF should result in the client "giving it another try" or two, which is more pragmatic anyway.
As per the docs, the easiest way to disable h2 in Go's http.Client at runtime is env GODEBUG=http2client=0 ... which I can also achieve in other ways as well. Especially important to understand is that the "next protocol" is pre-negotiated "early" during TLS, so Go's http.Transport must manage that configuration along with a cache/memo to provide its functionality in a performant way. Therefore, use your own httpClient to .Do(req) (and don't forget to give your Request a context.Context so that it's easy to cancel) using a custom http.RoundTripper for Transport. Here's some example code:
type forwardRoundTripper struct {
rt http.RoundTripper
}
func (my *forwardRoundTripper) RoundTrip(r *http.Request) (*http.Response, error) {
return my.rt.RoundTrip(r) // adjust URLs, or transport as necessary per-request
}
// httpTransport is the http.RoundTripper given to a Client as Transport
// (don't forget to set up a reasonable Timeout and other behavior as desired)
var httpTransport = &customRoundTripper{rt: http.DefaultTransport}
func h2Disabled(rt *http.Transport) *http.Transport {
log.Println("--- only using HTTP/1.x ...")
rt.ForceAttemptHTTP2 = false // not good enough
// at least one of the following is ALSO required:
rt.TLSClientConfig.NextProtos = []string{"http/1.1"}
// need to Clone() or replace the TLSClientConfig if a request already occurred
// - Why? Because the first time the transport is used, it caches certain structures.
// (if you do this replacement, don't forget to set a minimum TLS version)
rt.TLSHandshakeTimeout = longTimeout // not related to h2, but necessary for stability
rt.TLSNextProto = make(map[string]func(authority string, c *tls.Conn) http.RoundTripper)
// ^ some sources seem to think this is necessary, but not in all cases
// (it WILL be required if an "h2" key is already present in this map)
return rt
}
func init() {
h2ok := ...
if t, ok := httpTransport.rt.(*http.Transport); ok && !h2ok {
httpTransport.rt = h2Disabled(t.Clone())
}
// tweak rate limits here
}
This lets me make the volume of requests that I need to OR get more-reasonable errors in edge cases.
I'm currently working on migrating our code from global sign package to go mongo-driver, not sure where should I use context.TODO() and context.Background(), it’s really confusing, I know both it returns non-nil empty, so should I use context.Background() in the main function & init() functions? And use context.TODO() in other places? Can anyone help with this?
Trying to check to see which param should I use context.TODO() or context.Background().
Check their documentation:
context.Background():
func Background() Context
Background returns a non-nil, empty Context. It is never canceled, has no values, and has no deadline. It is typically used by the main function, initialization, and tests, and as the top-level Context for incoming requests.
context.TODO():
func TODO() Context
TODO returns a non-nil, empty Context. Code should use context.TODO when it's unclear which Context to use or it is not yet available (because the surrounding function has not yet been extended to accept a Context parameter).
According to the doc, when you need a context but you don't have one (yet) and don't know what to use, use context.TODO(). This is exactly your case, using context.TODO() properly documents this. Also, static analysis tools and IDEs may give support discovering these contexts and give you warning later to address the issue.
Also note that if you do have a context when you have to use the mongo driver, then consider using that context, or deriving a new one from it.
For example, if you are writing an HTTP handler where you need to query some documents, the HTTP request http.Request already has a context which you can access using Request.Context(). This is a prime candidate to use when you call other API functions that require a context.
Why, you may ask? Because the request's context is cancelled when the HTTP client abandons / aborts the request, which means whatever you do, the client will not receive it, so for example if the HTTP handler just serves information, you may as well abort generating it, saving some resources. So if the request context is cancelled while you're executing a MongoDB query, you may as well cancel the query too, because you won't need (won't process) the result anyway. Cancelling the context passed to the query execution (Collection.Find() for example) is the way to tell the MongoDB server to cancel the query if possible (because you won't need the result).
So using the request context in this case can save you some CPU time and memory both on the HTTP server and on the MongoDB server as well.
Another example: let's say you have to produce the HTTP response within 10 seconds (may be a platform limitation), during which you have to execute a MongoDB operation, and you have to post process the results which takes 4 seconds. In this scenario if the MongoDB operation takes more than 6 seconds, you would not be able to complete the post processing to fit in the 10-second limit. So you might as well cancel the MongoDB operation if it doesn't complete in 6 seconds.
An easy way to solve this is to derive a context with 6 seconds timeout:
ctx, cancel := context.WithTimeout(r.Context(), 6 * time.Second)
defer cancel()
// ctx automatically times out after 6 seconds
curs, err := c.Find(ctx, bson.M{"some": "filter"})
In this example ctx will time out after 6 seconds, so if the c.Find() operation doesn't get finished by that, the passed ctx will signal that it can be cancelled (in which case an error will be returned). Since ctx is derived from r.Context(), if the request's context get's cancelled earlier (e.g. the HTTP client aborts), ctx will also be cancelled.
Now let's say you're not writing an HTTP handler but some other code (standalone app, background worker etc.), where you may not have a context, but you have your standards that you don't want to wait more than 30 seconds for a query. This is a prime example where you may derive a context from context.Background() using a 30-sec timeout:
ctx, cancel := context.WithTimeout(context.Background(), 30 * time.Second)
defer cancel()
// ctx automatically times out after 30 seconds
curs, err := c.Find(ctx, bson.M{"some": "filter"})
If the query finishes within 30 seconds, all good, you may use the results. If not, the context gets cancelled after 30 sec, and so c.Find() will also (likely) return an error.
context.TODO:
Code should use context.TODO when it's unclear which Context to use or it is not yet available (because the surrounding function has not yet been extended to accept a Context parameter).
context.Background:
Background returns a non-nil, empty Context. It is never canceled, has no values, and has no deadline. It is typically used by the main function, initialization, and tests, and as the top-level Context for incoming requests.
TODO is a placeholder to make it clear that the code should be updated with a more appropriate context in the future, but none is available yet. Background is used when such an indication isn't necessary and you simply need an empty root context with no values or cancellation. The values they return are identical; the only difference is semantic, in that one might scan code for TODO contexts (or use a linter to do so) as places that need to be updated/addressed.
I’m currently maintaining a few HTTP APIs based on the standard library and gorilla mux and running in kubernetes (GKE).
We’ve adopted the http.TimeoutHandler as our “standard” way to have a consistent timeout error management.
A typical endpoint implementation will use the following “chain”:
MonitoringMiddleware => TimeoutMiddleware => … => handler
so that we can monitor a few key metrics per endpoint.
One of our API is typically used in a “fire and forget” mode meaning that clients will push some data and not care for the API response. We are facing the issue that
the Golang standard HTTP server will cancel a request context when the client connection is no longer active (godoc)
the TimeoutHandler will return a “timeout” response whenever the request context is done (see code)
This means that we are not processing requests to completion when the client disconnects which is not what we want and I’m therefore looking for solutions.
The only discussion I could find that somewhat relates to my issue is https://github.com/golang/go/issues/18527; however
The workaround is your application can ignore the Handler's Request.Context()
would mean that the monitoring middleware would not report the "proper" status since the Handler would perform the request processing in its goroutine but the TimeoutHandler would be enforcing the status and observability would be broken.
For now, I’m not considering removing our middlewares as they’re helpful to have consistency across our APIs both in terms of behaviours and observability. My conclusion so far is that I need to “fork” the TimeoutHandler and use a custom context for when an handler should not depend on the client waiting for the response or not.
The gist of my current idea is to have:
type TimeoutHandler struct {
handler Handler
body string
dt time.Duration
// BaseContext optionally specifies a function that returns
// the base context for controling if the server request processing.
// If BaseContext is nil, the default is req.Context().
// If non-nil, it must return a non-nil context.
BaseContext func(*http.Request) context.Context
}
func (h *TimeoutHandler) ServeHTTP(w ResponseWriter, r *Request) {
reqCtx := r.Context()
if h.BaseContext != nil {
reqCtx = h.BaseContext(r)
}
ctx, cancelCtx := context.WithTimeout(reqCtx, h.dt)
defer cancelCtx()
r = r.WithContext(ctx)
...
case <-reqCtx.Done():
tw.mu.Lock()
defer tw.mu.Unlock()
w.WriteHeader(499) // write status for monitoring;
// no need to write a body since no client is listening.
case <-ctx.Done():
tw.mu.Lock()
defer tw.mu.Unlock()
w.WriteHeader(StatusServiceUnavailable)
io.WriteString(w, h.errorBody())
tw.timedOut = true
}
The middleware BaseContext callback would return context.Background() for requests to the “fire and forget” endpoint.
One thing I don’t like is that in doing so I’m losing any context keys written so this new middleware would have strong usage constraints. Overall I feel like this is more complex than it should be.
Am I completely missing something obvious?
Any feedback on API instrumentation (maybe our middlewares are an antipattern) /fire and forget implementations would be welcomed!
EDIT: as most comments are that a request for which the client does not wait for a response has unspecified behavior, I checked for more information on typical clients for which this happens.
From our logs, this happens for user agents that seem to be mobile devices. I can imagine that connections can be much more unstable and the problem will likely not disappear.
I would therefore not conclude that I shouldn't find a solution since this is currently creating false-positive alerts.
Think about that I have a critical function, that should run all or nothing.
TakeMoneyFromSomeone()
GiveMoneyToSomeoneElse()
My question is: Can I trust Go that the function will not panicing between line1 & line2 when using `net/http.HandleFunc? (The two function are inside my http handler)
I'm focusing only on networking issues. For example: if the client disconnect, or timeout or too long body, or anything else. There is any networking issue that can make the server panic between line1, and line2? (Those two lines not using network)
If the answer is no. What happened if I try to ResponseWriter.write to a client that closed the connection. Is it will panic?
The HandlerFunc itself doesn't related to any potential networking issues, it just registers handlers to url patterns.
HandlerFunc
With 0 timeouts it will hang not panic. The better way is to use timeouts and handle errors instead of using default timeouts.
More control over the server's behavior is available by creating a custom Server:
s := &http.Server{
Addr: ":8080",
Handler: myHandler,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
log.Fatal(s.ListenAndServe())
And you shouldn't apprehend a panic from a built-in libs.
Go By Example:
A panic typically means something went unexpectedly wrong. Mostly we use it to fail fast on errors that shouldn’t occur during normal operation, or that we aren’t prepared to handle gracefully.
Note that unlike some languages which use exceptions for handling of many errors, in Go it is idiomatic to use error-indicating return values wherever possible.
I have a simple multipart form which uploads to a Go app. I wanted to set a restriction on the upload size, so I did the following:
func myHandler(rw http.ResponseWriter, request *http.Request){
request.Body = http.MaxBytesReader(rw, request.Body, 1024)
err := request.ParseMultipartForm(1024)
if err != nil{
// Some response.
}
}
Whenever an upload exceeds the maximum size, I get a connection reset like the following:
and yet the code continues executing. I can't seem to provide any feedback to the user. Instead of severing the connection I'd prefer to say "You've exceeded the size limit". Is this possible?
This code works as intended. Description of http.MaxBytesReader
MaxBytesReader is similar to io.LimitReader but is intended for
limiting the size of incoming request bodies. In contrast to
io.LimitReader, MaxBytesReader's result is a ReadCloser, returns a
non-EOF error for a Read beyond the limit, and closes the underlying
reader when its Close method is called.
MaxBytesReader prevents clients from accidentally or maliciously
sending a large request and wasting server resources.
You could use io.LimitReader to read just N bytes and then do the handling of the HTTP request on your own.
The only way to force a client to stop sending data is to forcefully close the connection, which is what you're doing with http.MaxBytesReader.
You could use a io.LimitReader wrapped in a ioutil.NopCloser, and notify the client of the error state. You could then check for more data, and try and drain the connection up to another limit to keep it open. However, clients that aren't responding correctly to MaxBytesReader may not work in this case either.
The graceful way to handle something like this is using Expect: 100-continue, but that only really applies to clients other than web browsers.