Given that I want to use a different proxy per request I did the following:
var proxies chan *url.URL
var anonymousClient = &http.Client{Transport: &http.Transport{Proxy: func(r *http.Request) (*url.URL, error) {
fmt.Println("Called")
p := <-proxies
proxies <- p
return p, nil
}}}
If I make 10 get requests using the above client Called gets printed once, shouldn't it be printed out with every request?
It looks to me that the result of the first call to that function gets cached and its called only once but I can be wrong, any ideas?
From the net/http package documentation:
By default, Transport caches connections for future re-use. This may leave many open connections when accessing many hosts. This behavior can be managed using Transport's CloseIdleConnections method and the MaxIdleConnsPerHost and DisableKeepAlives fields.
Transports should be reused instead of created as needed. Transports are safe for concurrent use by multiple goroutines.
Related
I’m currently maintaining a few HTTP APIs based on the standard library and gorilla mux and running in kubernetes (GKE).
We’ve adopted the http.TimeoutHandler as our “standard” way to have a consistent timeout error management.
A typical endpoint implementation will use the following “chain”:
MonitoringMiddleware => TimeoutMiddleware => … => handler
so that we can monitor a few key metrics per endpoint.
One of our API is typically used in a “fire and forget” mode meaning that clients will push some data and not care for the API response. We are facing the issue that
the Golang standard HTTP server will cancel a request context when the client connection is no longer active (godoc)
the TimeoutHandler will return a “timeout” response whenever the request context is done (see code)
This means that we are not processing requests to completion when the client disconnects which is not what we want and I’m therefore looking for solutions.
The only discussion I could find that somewhat relates to my issue is https://github.com/golang/go/issues/18527; however
The workaround is your application can ignore the Handler's Request.Context()
would mean that the monitoring middleware would not report the "proper" status since the Handler would perform the request processing in its goroutine but the TimeoutHandler would be enforcing the status and observability would be broken.
For now, I’m not considering removing our middlewares as they’re helpful to have consistency across our APIs both in terms of behaviours and observability. My conclusion so far is that I need to “fork” the TimeoutHandler and use a custom context for when an handler should not depend on the client waiting for the response or not.
The gist of my current idea is to have:
type TimeoutHandler struct {
handler Handler
body string
dt time.Duration
// BaseContext optionally specifies a function that returns
// the base context for controling if the server request processing.
// If BaseContext is nil, the default is req.Context().
// If non-nil, it must return a non-nil context.
BaseContext func(*http.Request) context.Context
}
func (h *TimeoutHandler) ServeHTTP(w ResponseWriter, r *Request) {
reqCtx := r.Context()
if h.BaseContext != nil {
reqCtx = h.BaseContext(r)
}
ctx, cancelCtx := context.WithTimeout(reqCtx, h.dt)
defer cancelCtx()
r = r.WithContext(ctx)
...
case <-reqCtx.Done():
tw.mu.Lock()
defer tw.mu.Unlock()
w.WriteHeader(499) // write status for monitoring;
// no need to write a body since no client is listening.
case <-ctx.Done():
tw.mu.Lock()
defer tw.mu.Unlock()
w.WriteHeader(StatusServiceUnavailable)
io.WriteString(w, h.errorBody())
tw.timedOut = true
}
The middleware BaseContext callback would return context.Background() for requests to the “fire and forget” endpoint.
One thing I don’t like is that in doing so I’m losing any context keys written so this new middleware would have strong usage constraints. Overall I feel like this is more complex than it should be.
Am I completely missing something obvious?
Any feedback on API instrumentation (maybe our middlewares are an antipattern) /fire and forget implementations would be welcomed!
EDIT: as most comments are that a request for which the client does not wait for a response has unspecified behavior, I checked for more information on typical clients for which this happens.
From our logs, this happens for user agents that seem to be mobile devices. I can imagine that connections can be much more unstable and the problem will likely not disappear.
I would therefore not conclude that I shouldn't find a solution since this is currently creating false-positive alerts.
I'm writing a TCP server which simultaneously accepts multiple connections from mobile devices and some WiFi devices (IOT). The connections needs to be maintained once established, with the 30 seconds timeout if there is no heartbeat received. So it is something like the following:
// clientsMap map[string] conn
func someFunction() {
conn, err := s.listener.Accept()
// I store the conn in clientsMap
// so I can access it, for brevity not
// shown here, then:
go serve(connn)
}
func serve(conn net.Conn) {
timeoutDuration := 30 * time.Second
conn.SetReadDeadline(time.Now().Add(timeoutDuration))
for {
msgBuffer := make([]byte, 2048)
msgBufferLen, err := conn.Read(msgBuffer)
// do something with the stuff
}
}
So there is one goroutine for each client. And each client, once connected to the server, is pending on the read. The server then processes the stuff read.
The problem is that I sometimes need to read things off one client, and then pass data to another (Between a mobile device and a WiFi device). I have stored the connections in clientsMap. So I can always access that. But since each client is handled by one goroutine, shall I be passing the data from one client to another by using a channel? But if the goroutine is blocked waiting for a pending read, how do I make it also wait for data from a channel? Or shall I just obtain the connection for the other party from the clientsMap and write to it?
The documentation for net.Conn clearly states:
Multiple goroutines may invoke methods on a Conn simultaneously.
So yes, it is okay to simply Write to the connections. You should take care to issue a single Write call per message you want to send. If you call Write more than once you risk interleaving messages from different mobile devices. This implies calling Write directly and not via some other API (in other words don't wrap the connection). For instance, the following would not be safe:
json.NewEncoder(conn).Encode(myValue) // use json.Marshal(myValue) instead
io.Copy(conn, src) // use io.ReadAll(src) instead
I built a http.Client for HTTP2 connection, what do I need to do to release the client and resource used?
http.Client does not require any special way to free "used" resources. When it becomes unreachable, memory used by it will be reclaimed by the garbage collector.
http.Client does not store connection or state information. The documentation even states that http.Client should be reused:
The Client's Transport typically has internal state (cached TCP connections), so Clients should be reused instead of created as needed. Clients are safe for concurrent use by multiple goroutines.
If you build your own client using (e.g. embedding) http.Client and you allocate resources that must be released explicitly, provide your own Close() method on it and document that anyone who uses your own implementation must call Close() if it is not needed anymore.
Note:
What you might confuse it with is that if you use an http.Client to do HTTP operations (like Client.Do(), Client.Get(), Client.Post() etc.), they return a value of *http.Response, and that response does hold a connection, state and other resources, which does need to be freed, typically via Response.Body.Close(). Quoting from the package doc of http:
The client must close the response body when finished with it:
resp, err := http.Get("http://example.com/")
if err != nil {
// handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
// ...
It is also documented at Client.Get():
When err is nil, resp always contains a non-nil resp.Body. Caller should close resp.Body when done reading from it.
The CloseIdleConnections() method was added in Go 1.12.
https://golang.org/pkg/net/http/#Client.CloseIdleConnections
CloseIdleConnections closes any connections on its Transport which were previously connected from previous requests but are now sitting idle in a "keep-alive" state. It does not interrupt any connections currently in use.
Hypothetically speaking, is it good practice to connect to a database for each request and close in when the request has completed?
I'm using mongodb with mgo for the database.
In my project, I would like to connect to a certain database by getting the database name from the request header (of course, this is combined with an authentication mechanism, e.g. JWT in my app). The flow goes something like:
User authentication:
POST to http://api.app.com/authenticate
// which checks the user in a "global" database,
// authenticates them and returns a signed JWT token
// The token is stored in bolt.db for the authentication mechanism
Some RESTful operations
POST to http://api.app.com/v1/blog/posts
// JWT middleware for each request to /v1* is set up
// `Client-Domain` in header is set to a database's name, e.g 'app-com'
// so we open a connection to that database and close when
// request finishes
So my questions are:
Is this feasible? - I've read about connection pools and reusing them but I haven't read much about them yet
Is there a better way of achieving the desired functionality?
How do I ensure the session is only closed when the request has completed?
The reason why I need to do this is because we have multiple vendors that have the same database collections with different entries with restricted access to their own databases.
Update / Solution
I ended up using Go's built in Context by Copying a session and using it anywhere I need to do any CRUD ops
Something like:
func main() {
...
// Configure connection and set in global var
model.DBSession, err = mgo.DialWithInfo(mongoDBDialInfo)
defer model.DBSession.Close()
...
n := negroni.Classic()
n.Use(negroni.HandlerFunc(Middleware))
...
}
func Middleware(res http.ResponseWriter, req *http.Request, next http.HandlerFunc) {
...
db := NewDataStore(clientDomain)
// db.Close() is an alias for ds.session.Close(), code for this function is not included in this post
// Im still experimenting with this, I need to make sure the session is only closed after a request has completed, currently it does not always do so
defer db.Close()
ctx := req.Context()
ctx = context.WithValue(ctx, auth.DataStore, db)
req = req.WithContext(ctx)
...
}
func NewDataStore(db string) *DataStore {
store := &DataStore{
db: DBSession.Copy().DB(db),
session: DBSession.Copy(),
}
return store
}
And then use it in a HandlerFunc, example /v1/system/users:
func getUsers(res http.ResponseWriter, req *http.Request) {
db := req.Context().Value(auth.DataStore).(*model.DataStore)
users := make([]SystemUser{}, 0)
// db.C() is an alias for ds.db.C(), code for this function is not included in this post
db.C("system_users").Find(nil).All(&users)
}
40% response time decrease over the original method I experimented with.
Hypothetically speaking is not a good practice because:
The database logic is scattered among several packages.
It's difficult to test
You can't apply DI (mainly it will be hard to maintain the code)
Replying to your questions:
Yes is feasible BUT you will not use the connection pool inside them go package (take a look to the code here if you want know more about Connection Pool)
A better way is to create a global variable that contains the database connection and close when the application is going to stop (and not close the connection every request)
How do I ensure the session is only closed when the request has complete<- you should checkout the answer fro your db query and then close the connection (but I don't recommend to close the connection after a request because you'll need to open again for another request and close again etc...)
I need to embed the default http.Server in my own server struct and customize the Serve method.
The server needs to short circuit the go c.serve() call and only run that line if it has the computing resources available to respond within 50ms. Otherwise the server is just going to send a 204 and move on.
This is almost straightforward.
type PragmaticServer struct {
http.Server
Addr string
Handler http.Handler
}
func (srv *PragmaticServer) Serve(l net.Listener) error {
defer l.Close()
var tempDelay time.Duration // how long to sleep on accept failure
for {
// SNIP for clarity
c, err := srv.newConn(rw)
if err != nil {
continue
}
c.setState(c.rwc, StateNew) // before Serve can return
go c.serve()
}
}
So, again. This almost works. Except that srv.newConn is an unexported method, as is c.serve and c.setState, which means that I end up having to copy and paste pretty much the entirety of net/http in order for this to compile. Which is basically a fork. Is there any better way to do this?
Unfortunately, you're not going to be able to do that without reimplementing most of the Server code. Short of that, we usually intercept the call either just before at conn.Accept, or just after at Handler.ServerHTTP.
The first method is to create a custom net.Listener that filters out connections before they are even handed off to the http.Server. While this can respond faster, and consume fewer resources, it however makes it less convenient to write http responses, and precludes you from limiting requests on already open connections.
The second way to handle this, is to just wrap the handlers and intercept the request before any real work has been done. You most likely want to create a http.Handler to filter the requests, and pass them through to your main handler. This can also be more flexible, since you can filter based on the route, or other request information if you so choose.