I have net/http handlers that have defer req.Body.Close() in each on web server side.
What is the correct place to put this in? Should I put it in the end of the function or does it matter at all, and I can put it in the beginning?
A request body does not need to be closed in the handler. From the http.Request documentation
// The Server will close the request body. The ServeHTTP
// Handler does not need to.
net/http
Server requests
Clarified at 2ede818, net/http states that:
For server requests, the Request Body is always non-nil but will return EOF immediately when no body is present.
The Server will close the request body. The ServeHTTP Handler does not need to.
What is the correct place to put this in? Should I put it in the end of the function or does it matter at all, and I can put it in the beginning?
Neither nor. Both are terribly wrong.
The defer req.Body.Close() has become cult.
Hard facts first:
If the request failed (non nil error returned) there is no Body to close, neither in a defered way nor a direct way.
You must close the Body (if present) on all code paths you might take.
You probably do not want to close the Body before having processed it (or at least parts of it).
Back to the options in your question:
"at the beginning [of the function]": Totally wrong as Body might be nil (fact 1).
"at the end [of the function]": Totally wrong because of A) it is dangerous as you might miss a code path leaving your function (fact 2) and B) even if you equip all your function ends (i.e. returns) with defer Bod.Close() it is totally useless to defer it instead of simply closing it via Body.Close().
The only sensible way of defered closing the request body is once, right after having established that Body is non-nil what means the request did not return an error.
As mentioned in the documentation, no need to explicit close it both in client and server side.
// Body is the request's body.
//
// For client requests, a nil body means the request has no
// body, such as a GET request. The HTTP Client's Transport
// is responsible for calling the Close method.
//
// For server requests, the Request Body is always non-nil
// but will return EOF immediately when no body is present.
// The Server will close the request body. The ServeHTTP
// Handler does not need to.
According to the Go documentation it is up to you to close the body once you are done with it.
I usually put the defer line right after the line where I check the request for errors.
Related
When I read the source code of the Golang net/http package, it always mentioned the difference between client requests and server requests. I want to know when these two are generally used, (of course, client requests are well understood and are widely used in websites with front-end and back-end separation architectures)
e.g.
//
// For client requests, a nil body means the request has no
// body, such as a GET request. The HTTP Client's Transport
// is responsible for calling the Close method.
//
// For server requests, the Request Body is always non-nil
// but will return EOF immediately when no body is present.
// The Server will close the request body. The ServeHTTP
// Handler does not need to.
//
// Body must allow Read to be called concurrently with Close.
// In particular, calling Close should unblock a Read waiting
// for input.
Body io.ReadCloser
You can create a Request using http.NewRequest, then set its fields, and call Client.Do to issue the request. In this case, the Request.Body is a variable the client has to set if the request has a body.
When this request is handled on the server side, a new instance of http.Request is created, and that is a server request. For this use, the Body is never nil.
I have a simple multipart form which uploads to a Go app. I wanted to set a restriction on the upload size, so I did the following:
func myHandler(rw http.ResponseWriter, request *http.Request){
request.Body = http.MaxBytesReader(rw, request.Body, 1024)
err := request.ParseMultipartForm(1024)
if err != nil{
// Some response.
}
}
Whenever an upload exceeds the maximum size, I get a connection reset like the following:
and yet the code continues executing. I can't seem to provide any feedback to the user. Instead of severing the connection I'd prefer to say "You've exceeded the size limit". Is this possible?
This code works as intended. Description of http.MaxBytesReader
MaxBytesReader is similar to io.LimitReader but is intended for
limiting the size of incoming request bodies. In contrast to
io.LimitReader, MaxBytesReader's result is a ReadCloser, returns a
non-EOF error for a Read beyond the limit, and closes the underlying
reader when its Close method is called.
MaxBytesReader prevents clients from accidentally or maliciously
sending a large request and wasting server resources.
You could use io.LimitReader to read just N bytes and then do the handling of the HTTP request on your own.
The only way to force a client to stop sending data is to forcefully close the connection, which is what you're doing with http.MaxBytesReader.
You could use a io.LimitReader wrapped in a ioutil.NopCloser, and notify the client of the error state. You could then check for more data, and try and drain the connection up to another limit to keep it open. However, clients that aren't responding correctly to MaxBytesReader may not work in this case either.
The graceful way to handle something like this is using Expect: 100-continue, but that only really applies to clients other than web browsers.
I need to embed the default http.Server in my own server struct and customize the Serve method.
The server needs to short circuit the go c.serve() call and only run that line if it has the computing resources available to respond within 50ms. Otherwise the server is just going to send a 204 and move on.
This is almost straightforward.
type PragmaticServer struct {
http.Server
Addr string
Handler http.Handler
}
func (srv *PragmaticServer) Serve(l net.Listener) error {
defer l.Close()
var tempDelay time.Duration // how long to sleep on accept failure
for {
// SNIP for clarity
c, err := srv.newConn(rw)
if err != nil {
continue
}
c.setState(c.rwc, StateNew) // before Serve can return
go c.serve()
}
}
So, again. This almost works. Except that srv.newConn is an unexported method, as is c.serve and c.setState, which means that I end up having to copy and paste pretty much the entirety of net/http in order for this to compile. Which is basically a fork. Is there any better way to do this?
Unfortunately, you're not going to be able to do that without reimplementing most of the Server code. Short of that, we usually intercept the call either just before at conn.Accept, or just after at Handler.ServerHTTP.
The first method is to create a custom net.Listener that filters out connections before they are even handed off to the http.Server. While this can respond faster, and consume fewer resources, it however makes it less convenient to write http responses, and precludes you from limiting requests on already open connections.
The second way to handle this, is to just wrap the handlers and intercept the request before any real work has been done. You most likely want to create a http.Handler to filter the requests, and pass them through to your main handler. This can also be more flexible, since you can filter based on the route, or other request information if you so choose.
In order to test a server that I am writing, I want to be able to start and stop it in the testing framework.
To do so, I am hoping I can integrate the context package in with the http.Server struct. I want to be able to stop the server when I call the Done function and the ctx.Done() channel returns something.
What I would love to do would be to just modify the http.Server.Serve() method, to accept a context.Context and check if it is done, on each iteration of the for loop, like this:
func (srv *server) Serve(ctx context.Context, l net.Listener) error {
defer l.Close()
var tempDelay time.Duration // how long to sleep on accept failure
for {
select {
case <-ctx.Done():
return nil
default:
}
... rest is same as original
However it seems like if I wanna add that check inside the for loop, I would have to rewrite a lot of the methods, because this method calls other private methods (like http.server.srv), which in turn call other private methods....
I also notice that the for loop will stop when the Accept() method on the listener returns an error.
However, I can't seem to figure out a way to get a listener to output an error from it's accept method without accessing its private methods as well.
It seems like I am doing something very stupid and wrong if I have to copy and paste half the http library just to let the server stop using the context package.
I know there are lots of solutions around for supporting context canceling for the ServeHTTP function, but that isn't what I am talking about. I wanna pass a context to the whole server, not just to each incoming request.
Is this just impossible?
Use httptest.Server to create a server that you can start and stop in tests.
If you do use the http.Server directly, you can break the Serve loop by closing the net.Listener. When a net.Listener is closed, any blocked Accept operations are unblocked and return errors. The Serve function returns if Accept returns a permanent error.
I have a function that just makes a get request to check the status code. It does not read anything from the body. Should I still end the function with resp.Body.Close() ?
Callers should close resp.Body when done reading from it. If resp.Body is not closed, the Client's underlying RoundTripper (typically Transport) may not be able to re-use a persistent TCP connection to the server for a subsequent "keep-alive" request.
Yes. When you call http.Get, the function returns a response as soon as all the HTTP headers have been read. The body of the response has not been read yet. The Response.Body is a wrapper around the network connection to the server. When you read from it, it downloads the body of the response.
.Close() tells the system that you're done with the network connection. If you have not read the response body, the default http transport closes the connection. (The transport can only re-use the connection if the body has been read, because if it reused a connection with an unread body the next request made using that connection would receive the previous request's response!)
So reading the Body is often more efficient than simply Close()ing if you're making more than one request - especially with TLS connections which are relatively expensive to create.
If you don't need the body of the response, you should use Head instead of Get. Head doesn't require reading or closing the response body.