Timeout handler moves ServeHTTP execution on a new goroutine, but not able to kill that goroutine after the timer ends. On every request, it creates two goroutines, but ServeHTTP goroutines never kill with context.
Not able to find a way to kill goroutines.
Edit For-loop with time.Sleep function, represents huge computation which goes beyond our timer. Can replace it with any other function.
package main
import (
"fmt"
"io"
"net/http"
"runtime"
"time"
)
type api struct{}
func (a api) ServeHTTP(w http.ResponseWriter, req *http.Request) {
// For-loop block represents huge computation and usually takes more time
// Can replace with any code
i := 0
for {
if i == 500 {
break
}
fmt.Printf("#goroutines: %d\n", runtime.NumGoroutine())
time.Sleep(1 * time.Second)
i++
}
_, _ = io.WriteString(w, "Hello World!")
}
func main() {
var a api
s := http.NewServeMux()
s.Handle("/", a)
h := http.TimeoutHandler(s, 1*time.Second, `Timeout`)
fmt.Printf("#goroutines: %d\n", runtime.NumGoroutine())
_ = http.ListenAndServe(":8080", h)
}
ServeHTTP goroutine should kill along with request context, normally which does not happen.
Use context.Context to instruct go-routines to abort their function. The go-routines, of course, have to listen for such cancelation events.
So for your code, do something like:
ctx := req.Context() // this will be implicitly canceled by your TimeoutHandler after 1s
i := 0
for {
if i == 500 {
break
}
// for any long wait (1s etc.) always check the state of your context
select {
case <-time.After(1 * time.Second): // no cancelation, so keep going
case <-ctx.Done():
fmt.Println("request context has been canceled:", ctx.Err())
return // terminates go-routine
}
i++
}
Playground: https://play.golang.org/p/VEnW0vsItXm
Note: Context are designed to be chained - allowing for multiple levels of sub-tasks to be canceled in a cascading manner.
In a typical REST call one would initiate a database request. So, to ensure such a blocking and/or slow call completes in a timely manner, instead of using Query one should use QueryContext - passing in the http request's context as the first argument.
I found, if you do not have any way to reach to your channel then there is no way to kill or stop goroutine when it is running.
In the large computational task, you have to watch the channel on a specific interval or after specific task completion.
Related
In my application I need async/await pattern with context cancellation support. In practice I have a function like:
func longRunningTask() <-chan int32 {
r := make(chan int32)
go func() {
defer close(r)
// Simulate a workload.
time.Sleep(time.Second * 3)
r <- rand.Int31n(100)
}()
return r
}
However, it will not support context cancellation. In order to fix this, I can add an argument and modify function to wait for ctx.Done() channel signal in a select statement, to abort an operation if context is cancelled.
If done this way, the function will not properly abort if run twice or more times (because the context pointer will be shared), since context cancellation channel only receives one signal:
ctx := ...
go func() { r := <-longRunningTask(ctx) } // Done() works
go func() { r := <-longRunningTask(ctx) } // ?
// cancel() ...
Here is what I see about Done:
// go/context.go
357 func (c *cancelCtx) Done() <-chan struct{} {
358 c.mu.Lock()
359 if c.done == nil {
360 c.done = make(chan struct{})
361 }
362 d := c.done
363 c.mu.Unlock()
364 return d
365 } // Done() returns the same channel for all callers, and cancellation signal is sent once only
Does the go source mean context does not really support abortion of a function that calls other "long-running" functions, "a chained cancellation"?
What are options to write asynchronious functions that will support context cancellation in an unlimited recursion of .Done() usage?
Does the go source mean context does not really support abortion of a
function that calls other "long-running" functions, "a chained
cancellation"?
No. A task can call other long-running tasks, passing a context down the call chain. This is a standard practice. And if a context is canceled, a nested call will error and bubble-up the cancelation error along the call stack
What are options to write asynchronious functions that will support context cancellation in an unlimited recursion of .Done() usage?
Recursion is no different that a couple of nested calls that take a context. Provided the recursive calls take a context input parameter and return an error (that is check), a recursive call chain will bubble-up a cancelation event just like a set of non-recursive nested calls.
First, let's update your wrapper function to support context.Context:
func longRunningTask(ctx context.Context) <-chan int32 {
r := make(chan int32)
go func() {
defer close(r)
// workload
i, err := someWork(ctx)
if err != nil {
return
}
r <- i
}()
return r
}
And then someWork - to use the sleep workload would look like this:
func someWork(ctx context.Context) (int32, error) {
tC := time.After(3*time.Second) // fake workload
// we can check this "workload" *AND* the context at the same time
select {
case <-tC:
return rand.Int31n(100), nil
case <-ctx.Done():
return 0, ctx.Err()
}
}
The important thing to note here is, we can alter the fake "workload" (time.Sleep) in a way that it becomes a channel - and thus watch it and our context via a select statement. Most workloads are of course not this trivial...
Fortunately the Go standard library is full of support for context.Context. So if your workload consists of lots of potentially long-running SQL query, each query can be passed a context. Same with HTTP requests or gRPC calls. If your workload consists of any of these calls, passing in the parent context will cause any of these potentially blocking calls to return with an error when the context is canceled - and thus your workload will return with a cancellation error, letting the caller know what happened.
If your workload does not fit neatly into this model e.g. computing a large Mandelbrot-Set image. Checking the context for cancelation after every pixel can have negative performance impact as polling selects are not free:
select {
case <-ctx.Done(): // polling select when `default` is included
return ctx.Err()
default:
}
In cases like this, tuning could be applied and if say pixels are calculated at a rate of 10,000/s - polling the context every 10,000 pixels will ensure the task will return no later than 1 second from the point of cancelation.
I have a Go RPC server that serves client requests. A client requests work (or task) from the server and the server assigns a task to the client. The server expects workers (or clients) to finish any task within a time limit. Therefore a timeout event callback mechanism is required on the server-side.
Here is what I tried so far.
func (l *Listener) RequestHandler(request string, reply string) error {
// some other work
// ....
_timer := time.NewTimer(time.Second * 5) // timer for 2 seconds
go func() {
// simulates a client not replying case, with timeout of 2 sec
y := <-_timer.C
fmt.Println("TimeOut for client")
// revert state changes becasue of client fail
}()
// set reply
// update some states
return nil
}
In the above snippet for each request from a worker (or a client) the handler in the server-side starts a timer and a goroutine. The goroutine reverts the changes done by the handler function before sending a reply to the client.
Is there any way of creating a "set of timers" and blocking wait on the "set of timers" ? Further, whenever a timer expires the blocking wait wakes up and provides us with the timer handles. Depending on the timer type we can perform different expiry handler functions in the runtime.
I am trying to implement a similar mechanism in Go that we can implement in C++ with timerfd with epoll.
Full code for the sample implementation of timers in Go. server.go and client.go.
I suggest you to explored the context package
it can be be done like this:
func main() {
c := context.Background()
wg := &sync.WaitGroup{}
f(c, wg)
wg.Wait()
}
func f(c context.Context, wg *sync.WaitGroup) {
c, _ = context.WithTimeout(c, 3*time.Second)
wg.Add(1)
go func(c context.Context) {
defer wg.Done()
select {
case <-c.Done():
fmt.Println("f() Done:", c.Err())
return
case r := <-time.After(5 * time.Second):
fmt.Println("f():", r)
}
}(c)
}
basically you initiate a base context and then derive other contexts from it, when a context is terminated, either by passing the time or a call to its close, it closes its Done channel and the Done channel of all the contexts that are derived from it.
Got some help here already that made me move forward in this concept I'm trying, but it still isn't quite working and I hit a conflict that I can't seem to get around.
I have tried here to illustrate what I want in a flowchart - please note that the client can be many clients that will send in with printjobs therefore we cannot reply on the worker to be processing our job at that moment, but for most it will (peak times it won't as the processing job of printing can take time).
type QueueElement struct {
jobid string
rw http.ResponseWriter
doneChan chan struct{}
}
type GlobalVars struct {
db *sql.DB
wg sync.WaitGroup
jobs chan QueueElement
}
func (gv *GlobalVars) ServeHTTP(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/StartJob":
fmt.Printf ("incoming\r\n")
doneC := make(chan struct{}, 1) //Buffered channel in order not to block the worker routine
newPrintJob := QueueElement{
doneChan: doneC,
jobid: "jobid",
}
gv.jobs <- newPrintJob
func(doneChan chan struct{},w http.ResponseWriter) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
select {
//If this triggers first, then this waiting goroutine would exit
//and nobody would be listeding the 'doneChan'. This is why it has to be buffered.
case <-ctx.Done():
fmt.Fprintf(w, "job is taking more than 5 seconds to complete\r\n")
fmt.Printf ("took longer than 5 secs\r\n")
case <-doneChan:
fmt.Fprintf(w, "instant reply from serveHTTP\r\n")
fmt.Printf ("instant\r\n")
}
}(doneC,w)
default:
fmt.Fprintf(w, "No such Api")
}
}
func worker(jobs <-chan QueueElement) {
for {
job := <-jobs
processExec ("START /i /b try.cmd")
fmt.Printf ("job done")
// processExec("START /i /b processAndPrint.exe -" + job.jobid)
job.doneChan <- struct{}{}
}
}
func main() {
db, err := sql.Open("sqlite3", "jobs.sqlite")
if err := db.Ping(); err != nil {
log.Fatal(err)
}
db.SetMaxOpenConns(1) // prevents locked database error
_, err = db.Exec(setupSQL)
if err != nil {
log.Fatal(err)
}
// create a GlobalVars instance
gv := GlobalVars{
db : db,
jobs: make(chan QueueElement),
}
go worker (gv.jobs)
// create an http.Server instance and specify our job manager as
// the handler for requests.
server := http.Server{
Handler: &gv,
Addr : ":8888",
}
// start server and accept connections.
log.Fatal(server.ListenAndServe())
}
The above code is of the serveHTTP and the worker with help from here, initially the func inside the ServeHTTP was a go routine and it's here the whole conflict arises for me - the concept is that in the serveHTTP it spawns a process that will get reply from the worker if the worker were able to process the job in time, within 5 seconds.
If the job was able to finish in 1 second, I want to reply back instantly after that 1 second to the client, if it takes 3 I want to reply after 3, if it takes more than 5 I will send reply back after 5 seconds (if the job takes 13 seconds I still want to reply after 5 seconds). That the client has to poll on the job from now on - but the conflict is:
a) When ServeHTTP exits - then the ResponseWriter closes - and to be able to reply back to the client, we have to write the answer to the ResponseWriter.
b) if I block serveHTTP (like in the code example below where I don't call the func as a go routine) then it affects not only that single API call but it seems that all others after that will be affected, so the first call in will be served correctly in time but the calls that would be coming in at the same time after the first will be sequentially delayed by the blocking operation.
c) if I don't block it ex - and change it to a go routine :
gv.jobs <- newPrintJob
go func(doneChan chan struct{},w http.ResponseWriter) {
Then there's no delay - can call in many APIs - but problem is here serveHTTP exists right away and thereby kills ResponseWriter and then I'm not able to reply back to the client.
I am not sure how I can get around this conflict where I don't cause any blockage to serveHTTP so I can handle all requests parallel, but still able to reply back to the ResponseWriter in question.
Is there any way of preventing serveHTTP of shutting down a responsewriter even though the function exits?
Yes, you are right with your point "c) if i don't block it ex".
In order to save the response writer, you shouldn't call a go routine inside it. Rather you should call ServeHTTP as a go-routine, which most of the http server implementations do.
This way you won't be blocking any api calls, each api call will run in a different go-routine, blocked by their functionalities.
Since your "jobs chan QueueElement" is a single channel (not a buffered channel), so all your processes get blocked at "gv.jobs <- newPrintJob".
You should rather use a buffered channel so that all api calls can add it to the queue and get response depending on work completion or time out.
Having a buffered channel simulates the real world memory limit of printers too. (queue length 1 being a special case)
I've add some updates to your code. Now it works as you've described.
package main
import (
"database/sql"
"fmt"
"log"
"math/rand"
"net/http"
"sync"
"time"
)
type QueueElement struct {
jobid string
rw http.ResponseWriter
doneChan chan struct{}
}
type GlobalVars struct {
db *sql.DB
wg sync.WaitGroup
jobs chan QueueElement
}
func (gv *GlobalVars) ServeHTTP(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/StartJob":
fmt.Printf("incoming\r\n")
doneC := make(chan struct{}, 1) //Buffered channel in order not to block the worker routine
go func(doneChan chan struct{}, w http.ResponseWriter) {
gv.jobs <- QueueElement{
doneChan: doneC,
jobid: "jobid",
}
}(doneC, w)
select {
case <-time.Tick(time.Second * 5):
fmt.Fprintf(w, "job is taking more than 5 seconds to complete\r\n")
fmt.Printf("took longer than 5 secs\r\n")
case <-doneC:
fmt.Fprintf(w, "instant reply from serveHTTP\r\n")
fmt.Printf("instant\r\n")
}
default:
fmt.Fprintf(w, "No such Api")
}
}
func worker(jobs <-chan QueueElement) {
for {
job := <-jobs
fmt.Println("START /i /b try.cmd")
fmt.Printf("job done")
randTimeDuration := time.Second * time.Duration(rand.Intn(7))
time.Sleep(randTimeDuration)
// processExec("START /i /b processAndPrint.exe -" + job.jobid)
job.doneChan <- struct{}{}
}
}
func main() {
// create a GlobalVars instance
gv := GlobalVars{
//db: db,
jobs: make(chan QueueElement),
}
go worker(gv.jobs)
// create an http.Server instance and specify our job manager as
// the handler for requests.
server := http.Server{
Handler: &gv,
Addr: ":8888",
}
// start server and accept connections.
log.Fatal(server.ListenAndServe())
}
select statement should be outside the goroutine function and blocks request till the end of the job execution or reaching timeout.
I want to run my function InsertRecords for 30 seconds and test how many records I can insert in a given time.
How can I stop processing InsertRecords after x seconds and then return a result from my handler?
func benchmarkHandler(w http.ResponseWriter, r *http.Request) {
counter := InsertRecords()
w.WriteHeader(200)
io.WriteString(w, fmt.Sprintf("counter is %d", counter))
}
func InsertRecords() int {
counter := 0
// db code goes here
return counter
}
Cancellations and timeouts are often done with a context.Context.
While this simple example could be done with a channel alone, using the context here makes it more flexible, and can take into account the client disconnecting as well.
func benchmarkHandler(w http.ResponseWriter, r *http.Request) {
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
defer cancel()
counter := InsertRecords(ctx)
w.WriteHeader(200)
io.WriteString(w, fmt.Sprintf("counter is %d", counter))
}
func InsertRecords(ctx context.Context) int {
counter := 0
done := ctx.Done()
for {
select {
case <-done:
return counter
default:
}
// db code goes here
counter++
}
return counter
}
This will run for at least 30 seconds, returning the number of complete database iterations. If you want to be sure that the handler always returns immediately after 30s, even if the DB call is blocked, then you need to push the DB code into another goroutine and let it return later. The shortest example would be to use a similar pattern as above, but synchronize access to the counter variable, since it could be written by the DB loop while returning.
func InsertRecords(ctx context.Context) int {
counter := int64(0)
done := ctx.Done()
go func() {
for {
select {
case <-done:
return
default:
}
// db code goes here
atomic.AddInt64(&counter, 1)
}
}()
<-done
return int(atomic.LoadInt64(&counter))
}
See #JoshuaKolden's answer for an example with a producer and a timeout, which could also be combined with the existing request context.
As JimB pointed out cancelation for limiting the time taken by an http requests can be handled with context.WithTimeout, however since you asked for the purposes of benchmarking you may want to use a more direct method.
The purpose of context.Context is to allow for numerous cancelation events to occur and have the same net effect of gracefully stopping all downstream tasks. In JimB's example it's possible that some other process will cancel the context before the 30 seconds have elapsed, and this is desirable from the resource utilization point of view. For example, if the connection is terminated prematurely there is no point in doing any more work on building a response.
If benchmarking is your goal you'd want to minimized the effect of superfluous events on the code being benchmarked. Here is an example of how to do that:
func InsertRecords() int {
stop := make(chan struct{})
defer close(stop)
countChan := make(chan int)
go func() {
defer close(countChan)
for {
// db code goes here
select {
case countChan <- 1:
case <-stop:
return
}
}
}()
var counter int
timeoutCh := time.After(30 * time.Second)
for {
select {
case n := <-countChan:
counter += n
case <-timeoutCh:
return counter
}
}
}
Essentially what we are doing is creating an infinite loop over discrete db operations, and counting iterations through the loop, we stop when time.After is triggered.
A problem in JimB's example is that despite checking ctx.Done() in the loop the loop can still block if the "db code" blocks. This is because ctx.Done() is only evaluated inline with the "db code" block.
To avoid this problem we separate the timing function and the benchmarking loop so that nothing can prevent us from receiving the timeout event when it occurs. Once the time out even occurs we immediately return the result of the counter. The "db code" may still be in mid execution but InsertRecords will exit and return its results anyway.
If the "db code" is in mid-execution when InsertRecords exits, the goroutine will be left running, so to clean this up we defer close(stop) so that on function exit we'll be sure to signal the goroutine to exit on the next iteration. When the goroutine exits, it cleans up the channel it was using to send the count.
As a general pattern the above is an example of how you can get precise timing in Go without regard to the actual execution time of the code being timed.
sidenote: A somewhat more advanced observation is that my example does not attempt to synchronize the start times between the timer and the goroutine. It seemed a bit pedantic to address that issue here. However, you can easily synchronize the two threads by creating a channel that blocks the main thread until the goroutine closes it just before starting the loop.
I thought I'd found an easy way to return an http response immediately then do some work in the background without blocking. However, this doesn't work.
func MyHandler(w http.ResponseWriter, r *http.Request) {
//handle form values
go doSomeBackgroundWork() // this will take 2 or 3 seconds
w.WriteHeader(http.StatusOK)
}
It works the first time--the response is returned immediately and the background work starts. However, any further requests hang until the background goroutine completes. Is there a better way to do this, that doesn't involve setting up a message queue and a separate background process.
I know this question was posted 4 years ago, but I hope someone can find this useful.
Here is a way to do that
There are something called worker pool https://gobyexample.com/worker-pools Using go routines and channels
But in the following code I adapt it to a handler. (Consider for simplicity I'm ignoring the errors and I'm using jobs as global variable)
package main
import (
"fmt"
"net/http"
"time"
)
var jobs chan int
func worker(jobs <-chan int) {
fmt.Println("Register the worker")
for i := range jobs {
fmt.Println("worker processing job", i)
time.Sleep(time.Second * 5)
}
}
func handler(w http.ResponseWriter, r *http.Request) {
jobs <- 1
fmt.Fprintln(w, "hello world")
}
func main() {
jobs = make(chan int, 100)
go worker(jobs)
http.HandleFunc("/request", handler)
http.ListenAndServe(":9090", nil)
}
The explanation:
main()
Runs the worker in background using a go routine
Start the service with my handler
note that the worker in this moment is ready to receive a job
worker()
it is a go routine that receives a channel
the for loop never ends because the channel is never closed
when a the channel contain a job, do some work (e.g. waits for 5 seconds)
handler()
writes to the channel to active a job
immediately returns printing "hello world" to the page
the cool thing is that you can send as many requests you want and because this scenario only contains 1 worker. the next request will wait until the previous one finished.
This is awesome Go!
Go multiplexes goroutines onto the available threads which is determined by the GOMAXPROCS environment setting. As a result if this is set to 1 then a single goroutine can hog the single thread Go has available to it until it yields control back to the Go runtime. More than likely doSomeBackgroundWork is hogging all the time on a single thread which is preventing the http handler from getting scheduled.
There are a number of ways to fix this.
First, as a general rule when using goroutines, you should set GOMAXPROCS to the number of CPUs your system has or to whichever is bigger.
Second, you can yield control in a goroutine by doing any of the following:
runtime.Gosched()
ch <- foo
foo := <-ch
select { ... }
mutex.Lock()
mutex.Unlock()
All of these will yield back to the Go runtime scheduler giving other goroutines a chance to work.