Context timeout implementation on every request using golang - go

I am trying to handle a context timeout for every request. We have the following server structures:
Flow overview:
Go Server: Basically, acts as a [Reverse-proxy].2
Auth Server: Check for requests Authentication.
Application Server: Core request processing logic.
Now if Authorization server isn't able to process a request in stipulated time, then I want to close the goroutine from memory.
Here is what I tried:
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
req, _ := http.NewRequest("GET", authorizationServer, nil)
req.Header = r.Header
req.WithContext(ctx)
res, error := client.Do(req)
select {
case <-time.After(10 * time.Second):
fmt.Println("overslept")
case <-ctx.Done():
fmt.Println(ctx.Err()) // prints "context deadline exceeded"
}
Over here, context returns as "deadline exceeded", if the request is not processed in stipulated time. But it continues to process that request and return response in more than the specified time. So, how can I stop the request flow (goroutine), when time is exceeded.
I've also the complete request needs to be processed in 60 seconds with this code:
var netTransport = &http.Transport{
Dial: (&net.Dialer{
Timeout: 60 * time.Second,
}).Dial,
TLSHandshakeTimeout: 60 * time.Second,
}
client := &http.Client{
Timeout: time.Second * 60,
Transport: netTransport,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
So do I need any separate context implementations as well?
Note1: It would be awesome, if we can manage the timeout on every requests (goroutine) created by HTTP server, using context.

What happens in your code is very correct and behaves as expected.
You create a context with 5 seconds timeout. You pass it to the request and make that request. Let's say that request returns in 2 seconds. You then do a select and either wait 10 seconds or wait for the context to finish. Context will always finish in the initial 5 seconds from when it was created and will also give that error every time it reaches the end.
The context is independent of the request and it will reach its deadline unless cancelled previously. You cancel the request when the function finishes using defer.
In your code the request takes your timeout in consideration. But the ctx.Err() will return deadline exceeded every time it reaches the timeout. Since that's what happens inside the context. calling ctx.Err() multiple times will return the same error.
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
go func () {
select {
case <-time.After(10 * time.Second):
fmt.Println("overslept")
case <-ctx.Done():
fmt.Println(ctx.Err()) // prints "context deadline exceeded"
}
}()
req, _ := http.NewRequest("GET", authorizationServer, nil)
req.Header = r.Header
req = req.WithContext(ctx)
res, error := client.Do(req)
From the context documentation:
// Err returns a non-nil error value after Done is closed. Err returns
// Canceled if the context was canceled or DeadlineExceeded if the
// context's deadline passed. No other values for Err are defined.
// After Done is closed, successive calls to Err return the same value.
In your code, the timeout will always be reached and not cancelled, that is why you receive DeadlineExceeeded. Your code is correct except the select part which will block until either 10 seconds pass or context timeout is reached. In your case always the context timeout is reached.
You should check the error returned by the client.Do call and not worry about the context error in here. You are the one controlling the context. If the request times out, a case you should test of course, then a proper error would be returned for you to verify.

Related

Context timeout not working as expected in golang

Helo All,
New to golang and was debugging timeout issues in a production environment. Before making a call to the server we add a timeout of 50ms to context and fire a server call. If the response is not received within 50 ms we expect the application to move on and not wait for the response.
But while debugging, I capture the duration between we fire a server call and the response received (or error out), to my surprise the value at the time is much higher than 50 ms.
Client syntax -
ctx, cancel := context.WithTimeout(ctx, e.opts.Timeout)
defer cancel()
fireServerCall(ctx)
..
..
def fireServerCall(ctx context:Context){
startTime:=time.Now()
//call to the server
res, err:=callToServer(ctx)
if err!=nil{
//capture failure latency
return ....
}
//capture success latency
return ....
}
Has anyone ever faced any similar issue? Is this expected behaviour? How did you handle such cases?
Am I doing something incorrectly? Suggestions are welcome :)
Edit:
I am passing context in my original code but forgot to mention it here, just added it. That mean, I am passing the same context on which my client is waiting for server to respond within 50 ms.
You should pass created context to fireServerCall and callToServer functions
callToServer should consider passed context and monitor ctx.Done() channel to stop its execution accordingly
Answering to comment by #Bishnu:
Don't think this is needed. Did a test and even without passing ctx to callToServer() it works. The behaviour is not as expected under high load. Can you kindly share some document/test what you have pointed here?
Context timeout just can't work without context passing and checking its Done() channel. Context is not some kind of magic — simplifying it is just a struct with done channel which is closed by calling cancel function or when timeout occurs. Monitoring this channel — is responsibility of the innermost function that accepts it.
Example:
package main
import (
"context"
"fmt"
"time"
)
func callToServer(ctx context.Context) {
now := time.Now()
select {
case <-ctx.Done(): // context cancelled via cancel or deadline
case <-time.After(1 * time.Second): // emulate external call
}
fmt.Printf("callToServer: %v\n", time.Since(now))
}
func callToServerContextAgnostic(ctx context.Context) {
now := time.Now()
select {
case <-time.After(2 * time.Second): // emulate external call
}
fmt.Printf("callToServerContextAgnostic: %v\n", time.Since(now))
}
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel()
callToServer(ctx)
ctx2, cancel2 := context.WithTimeout(context.Background(), 100*time.Millisecond)
defer cancel2()
callToServerContextAgnostic(ctx2)
}
Results:
callToServer: 100ms
callToServerContextAgnostic: 2s
You can launch it on Go Playground: https://go.dev/play/p/tIxjHxUzYfh
Note that many of the clients (from standard or third party libraries) monitors Done channel by themselves.
For example standard HTTP client:
c := &http.Client{} // client for all requests
ctx, cancel := context.WithTimeout(context.Background(), time.Duration(time.Millisecond*100))
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://google.com", nil)
if err != nil {
log.Fatal(err)
}
resp, err := c.Do(req) // will monitor `Done` channel for you
Some docs and articles:
https://pkg.go.dev/context
https://www.digitalocean.com/community/tutorials/how-to-use-contexts-in-go

Context.WithTimeout() and os.exit in gorilla/mux

I'm using the Golang gorilla/mux package, and one of the examples is as follows:
func main() {
var wait time.Duration
flag.DurationVar(&wait, "graceful-timeout", time.Second * 15, "the duration for which the server gracefully wait for existing connections to finish - e.g. 15s or 1m")
flag.Parse()
r := mux.NewRouter()
// Add your routes as needed
srv := &http.Server{
Addr: "0.0.0.0:8080",
// Good practice to set timeouts to avoid Slowloris attacks.
WriteTimeout: time.Second * 15,
ReadTimeout: time.Second * 15,
IdleTimeout: time.Second * 60,
Handler: r, // Pass our instance of gorilla/mux in.
}
// Run our server in a goroutine so that it doesn't block.
go func() {
if err := srv.ListenAndServe(); err != nil {
log.Println(err)
}
}()
c := make(chan os.Signal, 1)
// We'll accept graceful shutdowns when quit via SIGINT (Ctrl+C)
// SIGKILL, SIGQUIT or SIGTERM (Ctrl+/) will not be caught.
signal.Notify(c, os.Interrupt)
// Block until we receive our signal.
<-c
// Create a deadline to wait for.
ctx, cancel := context.WithTimeout(context.Background(), wait)
defer cancel()
// Doesn't block if no connections, but will otherwise wait
// until the timeout deadline.
srv.Shutdown(ctx)
// Optionally, you could run srv.Shutdown in a goroutine and block on
// <-ctx.Done() if your application should wait for other services
// to finalize based on context cancellation.
log.Println("shutting down")
os.Exit(0)
}
This seems simple enough, but my understanding is that defers do not run when os.Exit() is called (as per https://gobyexample.com/exit). I notice that there's a CancelFunc() returned by context.WithTimeout(), which is then deferred. My guess is that this is supposed to cancel the context created above if main() finishes before the deadline, but I don't see how that can happen with a call to os.Exit() at the end. Could anyone help me see what I'm missing?
You right, the deferred cancel function is never called. The author want probably point out that in real application, you should never forget to cancel your context.

Using rate.NewLimiter rate limiter with channels

I'm using a rate limiter to throttle the number of requests that are routed
The requests are sent to a channel, and I want to limit the number that are processed per second but i'm struggling to understand if i'm setting this correctly, I don't get an error, but i'm unsure if i'm even using the rate limiter
This is what is being added to the channel:
type processItem struct {
itemString string
}
Here's the channel and limiter:
itemChannel := make(chan processItem, 5)
itemThrottler := rate.NewLimiter(4, 1) //4 a second, no more per second (1)
var waitGroup sync.WaitGroup
Items are added to the channel:
case "newItem":
waitGroup.Add(1)
itemToExec := new(processItem)
itemToExec.itemString = "item string"
itemChannel <- *itemToExec
Then a go routine is used to process everything that is added to the channel:
go func() {
defer waitGroup.Done()
err := itemThrottler.Wait(context.Background())
if err != nil {
fmt.Printf("Error with limiter: %s", err)
return
}
for item := range itemChannel {
execItem(item.itemString) // the processing function
}
defer func() { <-itemChannel }()
}()
waitGroup.Wait()
Can someone confirm that the following occurs:
The execItem function is run on each member of the channel 4 times a second
I don't understand what "err := itemThrottler.Wait(context.Background())" is doing in the code, how is this being invoked?
... i'm unsure if i'm even using the rate limiter
Yes, you are using the rate-limiter. You are rate-limiting the case "newItem": branch of your code.
I don't understand what "err := itemThrottler.Wait(context.Background())" is doing in the code
itemThrottler.Wait(..) will just stagger requests (4/s i.e. every 0.25s) - it does not refuse requests if the rate is exceeded. So what does this mean? If you receive a glut of 1000 requests in 1 second:
4 requests will be handled immediately; but
996 requests will create a backlog of 996 go-routines that will block
The 996 will unblock at a rate of 4/s and thus the backlog of pending go-routines will not clear for another 4 minutes (or maybe longer if more requests come in). A backlog of go-routines may or may not be what you want. If not, you may want to use Limiter.Allow - and if false is returned, then refuse the request (i.e. don't create a go-routine) and return a 429 error (if this is a HTTP request).
Finally, if this is a HTTP request, you should use it's imbedded context when calling Wait e.g.
func (a *app) myHandler(w http.ResponseWriter, r *http.Request) {
// ...
err := a.ratelimiter(r.Context())
if err != nil {
// client http request most likely canceled (i.e. caller disconnected)
}
}

Go HTTP Request timeout

I have a client that connects to a server to read some response. The server takes around 5 minutes to respond to a particular request when I use Postman to execute the request.
I am writing this client in Go language and executing the following code to set a timeout of 10 minutes.
_client := &http.Client{
Timeout: 10 * time.Minute,
}
resp, err := _client.Post(c.Url, "application/json", r)
However, the request terminates after 2 minutes with an error. The error just says EOF.
I tried setting the timeout to 15 seconds to check if the config works and the request terminates in 15 seconds as expected.
How can I make sure that the timeout is 10 minutes?
I've had the same problem but there my code was running in GAE which canceled the request after 2 min with no operation. So if you have total control over where your code runs you should be able to specify the timeout time in client.Timeout
See here
Here’s a simple way to set timeouts for an HTTP request in Go:
client := &http.Client{Timeout: 10*time.Second}
resp, err := client.Get("https://freshman.tech")
The above snippet sets a timeout of 10 seconds on the HTTP client. So any request made with the client will timeout after 10 seconds if it does not complete in that time.
Before using the response from the request, you can check for a timeout error using the os.IsTimeout() method. It returns a boolean indicating whether the error is known to report a timeout that occurred.
client := &http.Client{Timeout: 10 * time.Second}
resp, err := client.Get(url)
if err != nil {
if os.IsTimeout(err) {
// A timeout error occurred
return
}
// This was an error, but not a timeout
}
// use the response
Another method involves using the net.Error interface along with a type assertion, but I prefer os.IsTimeout() due to its simplicity.
client := &http.Client{Timeout: 10 * time.Second}
resp, err := client.Get(url)
if err, ok := err.(net.Error); ok && err.Timeout() {
// A timeout error occurred
} else if err != nil {
// This was an error, but not a timeout
}
The real problem here is that you are using HTTP in a way implementers do not want you to. You can read more about your problem and the best practice to solve it here. The short answer is that you are better off responding quickly to the client saying the job is running along with the url to find the result when it is done.

Golang how can I stop an Http request from continuing to fire

I was doing some load testing earlier today and found something peculiar, sometimes an Http request doesn't not die and keeps on firing . How can I correct that in my Golang code for instance see the image below . I am load testing loading 1,000 HTTP request but if you notice on the 1,000th request below it takes 392,999 milliseconds or 392 seconds while the rest of the request takes 2.2 seconds on average . I have done the test multiple times and sometimes it hangs . This is my code
func Home_streams(w http.ResponseWriter, r *http.Request) {
var result string
r.ParseForm()
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
defer wg.Done()
db.QueryRow("select json_build_object('Locations', array_to_json(array_agg(t))) from (SELECT latitudes,county,longitudes,"+
"statelong,thirtylatmin,thirtylatmax,thirtylonmin,thirtylonmax,city"+
" FROM zips where city='Orlando' ORDER BY city limit 5) t").Scan(&result)
}()
wg.Wait()
fmt.Fprintf(w,result)
}
and I connect to the database with this code
func init() {
var err error
db, err = sql.Open("postgres","Postgres Connection String")
if err != nil {
log.Fatal("Invalid DB config:", err)
}
if err = db.Ping(); err != nil {
log.Fatal("DB unreachable:", err)
}
}
I would say that about 10 % of the time I load test this issue happens and the only way it stops is if I stop the requests manually otherwise it keeps on going indefinitely . I wonder if maybe this issue is addressed here https://medium.com/#nate510/don-t-use-go-s-default-http-client-4804cb19f779#.83uzpsp24 I am still learning my way around Golang .
and the only way it stops is if I stop the requests manually otherwise
it keeps on going indefinitely
You're not showing your full code, but it seems like you're using the http.ListenAndServe convenience function, which doesn't set a default timeout. So what I assume is happening is you're overloading your database server and your http server isn't set to timeout so it's just waiting for your database to respond.
Assuming all of this is correct try doing something like this instead:
srv := &http.Server{
ReadTimeout: 5 * time.Second,
WriteTimeout: 10 * time.Second,
}
srv.ListenAndServe()
There's a nice reference here.

Resources