Trying to understand how go context cancellation will abort execution of subsequent code
Details of experiment:
main func has a context that times out in 2sec
main func calls another func sum in a separate go-routine - which sleeps for 1sec for test-run-1 & 4sec for test-run-2
letting main sleep for 3sec to let spun go-routine complete execution
package main
import (
"context"
"fmt"
"log"
"time"
)
func main() {
c := context.Background()
childCtx, cancel := context.WithTimeout(c, 2*time.Second)
defer cancel()
ch := make(chan int, 1)
go sum(5, 6, ch)
var msg string
select {
case <-childCtx.Done():
msg = "return from ctx done channel"
case res := <-ch:
msg = fmt.Sprintf("return from go routine: %v", res)
}
log.Print(msg)
time.Sleep(3 * time.Second) //sleeping here to test if go-routine is still running
}
func sum(x int, y int, c chan<- int) {
time.Sleep(1 * time.Second)
//testcase-1: sleep - 1s
//testcase-2: sleep - 4s
result := x + y
log.Printf("print from sum fn: %v", result)
c <- result
}
Response for testcase-1 : sleep sum function for 1 sec:
2021/04/12 01:06:58 print from sum fn: 11
2021/04/12 01:06:58 return from go routine: 11
Response for testcase-2 : sleep sum function for 4 sec:
2021/04/12 01:08:25 return from ctx done channel
2021/04/12 01:08:27 print from sum fn: 11
In testcase-2 when sum func sleeps for 4 secs, context is already cancelled by timeout after 2secs, why is it still executing the sum func in diff go-routine and printing print from sum fn: 1 ?
From documentation: Canceling this context releases resources associated with it.
My assumption is that all the computation will be aborted immediately after 2 secs including the spun go-routine
Let me know how to do this right, thanks in adavance
As #AndySchweig has noted, context signals a cancelation event, but does not enforce cancelation. It's up to any potentially blocking goroutine to do its best at trying to cancel/clean-up after it detects a cancelation.
To update your sum function to support cancelation you could try:
// add context parameter as the first argument
// add a return error - to indicate any errors (i.e. function was interrupted due to cancelation)
func sum(ctx context.Context, x int, y int, c chan<- int) (err error) {
wait := 1 * time.Second // testcase-1
//wait := 4 * time.Second // testcase-2
// any blocking called - even sleeps - should be interruptible
select {
case <-time.After(wait):
case <-ctx.Done():
err = ctx.Err()
return
}
result := x + y
log.Printf("print from sum fn: %v", result)
select {
case c <- result:
case <-ctx.Done(): // check for ctx cancelation here - as no one may be listening on result channel
err = ctx.Err()
}
return
}
https://play.golang.org/p/DuIACxPvHYJ
The context is not gonna abort the go routine. In your case you just don't print the result from the go routine if context's time is out. The go routine doesn't know anything about the context.
Related
In the below program, there is a select sequence in both C1 & C2. A deadline context for 1.5 seconds is defined in C1, which after the deadline, cancels the context, leads to ctx.Done() and prevents the reading of integers in C2 which would further cancel the context and lead to ctx.Done() in C1.
If that is the case then what use is cancelCtx() in C1 when the context is already being cancelled?
package main
import (
"context"
"fmt"
"time"
)
func main() {
ctx := context.Background()
C1(ctx)
}
func C1(ctx context.Context) {
deadline := time.Now().Add(1500 * time.Millisecond)
ctx, cancelCtx := context.WithDeadline(ctx, deadline)
printCh := make(chan int)
go C2(ctx, printCh)
for num := 1; num <= 3; num++ {
select {
case printCh <- num:
time.Sleep(1 * time.Second)
case <-ctx.Done():
fmt.Printf("C1: Finished\n")
}
}
cancelCtx()
time.Sleep(100 * time.Millisecond)
}
func C2(ctx context.Context, printCh <-chan int) {
for {
select {
case num := <-printCh:
fmt.Printf("C2: %d\n", num)
case <-ctx.Done():
if err := ctx.Err(); err != nil {
fmt.Printf("C2 Error: %s\n", err)
}
fmt.Printf("C2: Finished\n")
return
}
}
}
As per the docs for context.WithDeadline():
Canceling this context releases resources associated with it, so code should call cancel as soon as the operations running in this Context complete.
So best practice is to call cancelCtx() (or whatever you call the function) as soon as you no longer need it (it's fairly common to defer cancel() even if this means it's called multiple times).
In your example there is another reason that this might be beneficial. Lets assume that the deadline is configurable and has been increased to 5 seconds; now cancelling the context leads to the termination of C2 being triggered by the call cancelCtx() instead of waiting until the deadline (playground).
Trying to get 2 Goroutines to play nicely within my project. Even though this is all sorta working, I am 100% sure it is being done very poorly... maybe even wrong...ly (is that even a word?).
Anyway, the basic concept of this project is to run some repeated work over a pre-selected amount of time while allowing the ability to abort the run before the time is up.
Here's the mess I have so far (I've only included the important parts for everyone's sanity... and to hide my horrible coding):
Two Goroutines:
BenchTimer() is a simple run-time countdown timer that does some repeating work for a set amount of time.
AbortTest() is a keyboard listener used to catch an 'ESC' (or whatever else I want) keypress from the keyboard to act as a "User Abort".
Each Goroutine, upon a successful run (i.e. BenchTimer() completes the countdown OR AbortTest() catches an abort keypress), sends a message down a common channel testAction. I use the one channel since this is an OR kinda thing (i.e. You can't get a completed countdown and an abort at the same time.). If BenchTimer() completes, then it sends "Complete" down the channel. If AbortTest() "completes" it sends "Abort" down the channel. [So far this all seems to be working...]
The next problem I ran into with this setup is how to kill the Goroutine that wasn't the "winner"... (i.e. If BenchTimer() completes normally, then I need to somehow kill AbortTest()... and vice-versa.) After a bunch of searching, I found that it isn't possible to kill a Goroutine externally, but it can be done internally... so I came up with using a second channel for each Goroutine to act as a sort of "kill signal" line: killAbortTest and killBenchTimer.
To tie this all together, I evaluate the result of the testAction channel. Because this channel will tell me which Goroutine "won", I can use this knowledge to send the correct (i.e. opposite) "kill signal" to have the "loser" Goroutine self-terminate.
Note: ... just means other code exists, but was removed due to not being needed for this post.
func main() {
...
testAction := make(chan string) // Action Result (Timer "Complete" or User "Abort")
killAbortTest := make(chan bool) // Kill AbortTest() Goroutine when BenchTimer() completes.
killBenchTimer := make(chan bool) // Kill BenchTimer() Goroutine when AbortTest() completes.
go BenchTimer(testAction, killBenchTimer) // Run BenchTimer() as Goroutine
go AbortTest(testAction, killAbortTest) // Run AbortTest() as Goroutine
// Program should wait here until it receives something on testAction channel.
actionVal := <-testAction
// Evaluate the testAction to kill the "loser" Goroutine
switch actionVal {
case "Abort":
killBenchTimer <- true // Abort received, signal BenchTimer() Goroutine to Quit
fmt.Println()
fmt.Println("Test Aborted")
case "Complete":
killAbortTest <- true // Countdown finished, signal AbortTest() Goroutine to Quit
fmt.Println()
fmt.Println("Test Completed")
}
...
}
// AbortTest - Listen for User Abort
func AbortTest(c chan<- string, k <-chan bool) {
if err := keyboard.Open(); err != nil {
panic(err)
}
defer func() {
_ = keyboard.Close()
}()
for {
select {
case <-k:
return
default:
_, key, err := keyboard.GetKey() // Poll for keypress
if err != nil {
panic(err)
}
if key == keyboard.KeyEsc { // ESC key was pressed
c <- "Abort"
return
}
}
}
}
// BenchTimer - Countdown Timer for BenchTest
func BenchTimer(c chan<- string, k <-chan bool) {
seconds := 0
switch testTime {
case "2-minute (fast)":
seconds = 120
case "5-minute (short)":
seconds = 300
case "10-minute (long)":
seconds = 600
case "20-minute (slow)":
seconds = 1200
}
ticker := time.Tick(time.Second)
for i := seconds; i >= 0; i-- {
select {
case <-k: // Kill Signal Received
return
default:
<-ticker
...
}
}
c <- "Complete"
}
There it is. My mess. There are many like it, but this one is my own. Like I said, it sorta works now, but I'm looking to make it better.
Am I just overthinking this whole process and making it way more complex than it needs to be?
Any help would be great.
This is the simplest example I can think of. I left out the keyboard part, but its basically the idea behind your code :
As mentioned by mh-cbon, this is safe :
package main
import (
"context"
"fmt"
"math/rand"
"sync"
"time"
)
var wg sync.WaitGroup
func main() {
rand.Seed(time.Now().UnixNano())
ctx, cancel := context.WithCancel(context.Background())
wg.Add(2)
go DoSomeTask(ctx, cancel)
go CancelTask(ctx, cancel)
wg.Wait()
}
func DoSomeTask(ctx context.Context, cancel func()) {
defer wg.Done()
defer cancel() // force cancellation
for i := 1; i < 10; i++ {
select {
case <-ctx.Done():
fmt.Println("context cancelled")
return
case <-time.After(time.Second):
}
fmt.Println("Done something!", i)
}
}
func CancelTask(ctx context.Context, cancel func()) {
defer wg.Done()
defer cancel() // force cancellation
duration := time.Second * time.Duration((rand.Intn(20-1) + 1))
fmt.Println("Will cancel in ", duration, " seconds!")
select {
case <-ctx.Done():
fmt.Println("context cancelled")
case <-time.After(duration):
}
}
I am trying to understand context in golang. I copied an example from https://golang.org/pkg/context/#example_WithCancel and changed it a bit:
Playgroud: https://play.golang.org/p/Aczc2CqcVZR
package main
import (
"context"
"fmt"
"time"
)
func main() {
// gen generates integers in a separate goroutine and
// sends them to the returned channel.
// The callers of gen need to cancel the context once
// they are done consuming generated integers not to leak
// the internal goroutine started by gen.
gen := func(ctx context.Context) <-chan int {
dst := make(chan int)
n := 1
go func() {
for {
select {
case <-ctx.Done():
fmt.Println("DONE")
return // returning not to leak the goroutine
case dst <- n:
n++
}
}
fmt.Println("END")
}()
return dst
}
ctx, cancel := context.WithCancel(context.Background())
defer time.Sleep(1 * time.Second)
defer fmt.Println("Before cancel")
defer cancel() // cancel when we are finished consuming integers
defer fmt.Println("After cancel")
channel := gen(ctx)
for n := range channel {
fmt.Println(n)
if n == 5 {
break
}
}
fmt.Println( <-channel)
}
When commenting out the
defer time.Sleep(1 * time.Second)
the "DONE" never gets printed. Playgroud: (https://play.golang.org/p/K0OcyZaj_xK)
I would expect the go routine which was started in the anonymous function still to be active. Once cancel() is called due to being deferred, the select should no longer block as
case <-ctx.Done():
should be available. However it seems to just end, unless I wait for 1 second and give it time. This behavior seems very wrong.
This behavior seems very wrong.
It's not. That's how program execution is specified. After main and its deferred functions return, the program exits.
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
https://golang.org/ref/spec#Program_execution
I have a goroutine which can generate an infinite number of values (each more suitable than the last), but it takes progressively longer to find each values. I'm trying to find a way to add a time limit, say 10 seconds, after which my function does something with the best value received so far.
This is my current "solution", using a channel and timer:
// the goroutine which runs infinitely
// (or at least a very long time for high values of depth)
func runSearch(depth int, ch chan int) {
for i := 1; i <= depth; i++ {
fmt.Printf("Searching to depth %v\n", i)
ch <- search(i)
}
}
// consumes progressively better values until the channel is closed
func awaitBestResult(ch chan int) {
var result int
for result := range ch {
best = result
}
// do something with best result here
}
// run both consumer and producer
func main() {
timer := time.NewTimer(time.Millisecond * 2000)
ch := make(chan int)
go runSearch(1000, ch)
go awaitBestResult(ch)
<-timer.C
close(ch)
}
This mostly works - the best result is processed after the timer ends and the channel is closed. However, I then get a panic (panic: send on closed channel) from the runSearch goroutine, since the channel has been closed by the main function.
How can I stop the first goroutine running after the timer has completed? Any help is very appreciated.
You need to ensure that the goroutine knows when it is done processing, so that it doesn't attempt to write to a closed channel, and panic.
This sounds like a perfect case for the context package:
func runSearch(ctx context.Context, depth int, ch chan int) {
for i := 1; i <= depth; i++ {
select {
case <- ctx.Done()
// Context cancelled, return
return
default:
}
fmt.Printf("Searching to depth %v\n", i)
ch <- search(i)
}
}
Then in main():
// run both consumer and producer
func main() {
ctx := context.WithTimeout(context.Background, 2 * time.Second)
ch := make(chan int)
go runSearch(ctx, 1000, ch)
go awaitBestResult(ch)
close(ch)
}
You are getting a panic because your sending goroutine runSearch apparently outlives the timer and it is trying to send a value on the channel which is already closed by your main goroutine. You need to devise a way to signal the sending go routine not to send any values once your timer is lapsed and before you close the channel in main. On the other hand if your search gets over sooner you also need to communicate to main to move on. You can use one channel and synchronize so that there are no race conditions. And finally you need to know when your consumer has processed all the data before you can exit main.
Here's something which may help.
package main
import (
"fmt"
"sync"
"time"
)
var mu sync.Mutex //To protect the stopped variable which will decide if a value is to be sent on the signalling channel
var stopped bool
func search(i int) int {
time.Sleep(1 * time.Millisecond)
return (i + 1)
}
// (or at least a very long time for high values of depth)
func runSearch(depth int, ch chan int, stopSearch chan bool) {
for i := 1; i <= depth; i++ {
fmt.Printf("Searching to depth %v\n", i)
n := search(i)
select {
case <-stopSearch:
fmt.Println("Timer over! Searched till ", i)
return
default:
}
ch <- n
fmt.Printf("Sent depth %v result for processing\n", i)
}
mu.Lock() //To avoid race condition with timer also being
//completed at the same time as execution of this code
if stopped == false {
stopped = true
stopSearch <- true
fmt.Println("Search completed")
}
mu.Unlock()
}
// consumes progressively better values until the channel is closed
func awaitBestResult(ch chan int, doneProcessing chan bool) {
var best int
for result := range ch {
best = result
}
fmt.Println("Best result ", best)
// do something with best result here
//and communicate to main when you are done processing the result
doneProcessing <- true
}
func main() {
doneProcessing := make(chan bool)
stopSearch := make(chan bool)
// timer := time.NewTimer(time.Millisecond * 2000)
timer := time.NewTimer(time.Millisecond * 12)
ch := make(chan int)
go runSearch(1000, ch, stopSearch)
go awaitBestResult(ch, doneProcessing)
select {
case <-timer.C:
//If at the same time runsearch is also completed and trying to send a value !
//So we hold a lock before sending value on the channel
mu.Lock()
if stopped == false {
stopped = true
stopSearch <- true
fmt.Println("Timer expired")
}
mu.Unlock()
case <-stopSearch:
fmt.Println("runsearch goroutine completed")
}
close(ch)
//Wait for your consumer to complete processing
<-doneProcessing
//Safe to exit now
}
On playground. Change the value of timer to observe both the scenarios.
I am trying to measure execution time of funcWithUnpredictiveExecutionTime function.
func measureTime(expectedMs float64) (ok bool) {
t1 := time.Now()
funcWithUnpredictiveExecutionTime()
t2 := time.Now()
diff := t2.Sub(t1)
The measuring is fine when funcWithUnpredictiveExecutionTime works faster than I expected. But if it works slower than expectedMs the measuring will not stop right after expected amount of milliseconds passed.
Is it possible to stop time measuring when funcWithUnpredictiveExecutionTime works longer than expectedMs without waiting funcWithUnpredictiveExecutionTime to finish?
In other words, measureTime(200) should return in 200 ms anyway with a good or bad result.
I guess I should use channels and then somehow cancel waiting for a channel. But how to do it exactly?
Full code:
package main
import (
"fmt"
"math/rand"
"time"
)
// random number between min and max
func random(min, max int) int {
rand.Seed(time.Now().Unix())
return rand.Intn(max-min) + min
}
// sleeps for a random milliseconds amount between 200 and 1000
func funcWithUnpredictiveExecutionTime() {
millisToSleep := random(200, 1000)
fmt.Println(fmt.Sprintf("Sleeping for %d milliseconds", millisToSleep))
time.Sleep(time.Millisecond * time.Duration(millisToSleep))
}
// measures execution time of a function funcWithUnpredictiveExecutionTime
// if expectedMs < actual execution time, it's ok.
// if expectedMs milliseconds passed and funcWithUnpredictiveExecutionTime
// still did not finish execution it should return
// without waiting for funcWithUnpredictiveExecutionTime
func measureTime(expectedMs float64) (ok bool) {
t1 := time.Now()
funcWithUnpredictiveExecutionTime()
t2 := time.Now()
diff := t2.Sub(t1)
actualMs := diff.Seconds() * 1000
ok = actualMs < expectedMs
fmt.Println(actualMs)
return
}
// prints results: Ok or too late
func printTimeResults(ok bool) {
if ok {
fmt.Println("Ok")
} else {
fmt.Println("Too late")
}
}
func main() {
printTimeResults(measureTime(200)) // expect it to finish in 200 ms anyway
printTimeResults(measureTime(1000)) // expect it to finish in 1000 ms anyway
}
Output:
Sleeping for 422 milliseconds
424.11895200000004
Too late
Sleeping for 422 milliseconds
425.27274900000003
Ok
Playground
You can't cancel a goroutine, unless you design it to be canceled. You can short circuit your timing function, by using a channel to signal the completion of the function being timed:
func measureTime(expectedMs float64) (ok bool) {
done := make(chan struct{})
t1 := time.Now()
go func() {
funcWithUnpredictiveExecutionTime()
close(done)
}()
select {
case <-done:
ok = true
case <-time.After(time.Duration(expectedMs) * time.Millisecond):
}
fmt.Println(time.Since(t1))
return ok
}
Extending JimB's example a little with the design I've personally followed for async background workers. I would say in most cases it's unacceptable to launch a go routine without passing an abort channel... All of your async methods should accept one as an argument or have one defined on their receiving type so you can actually control execution. fyi there are libraries for this, here's a simple one an old colleague of mine made; https://github.com/lytics/squaredance
If your program does not have an abort path for every goroutine you're probably going to face significant quality issues sooner or later. Also, for applications that are doing any heavy lifting in a goroutine, you will likely not be able to gracefully stop and start your application.
func measureTime(expectedMs float64) (ok bool) {
done := make(chan struct{})
abort := make(chan struct{})
t1 := time.Now()
go func() {
funcWithUnpredictiveExecutionTime(abort)
close(done)
}()
select {
case <-done:
ok = true
case <-time.After(time.Duration(expectedMs) * time.Millisecond):
// after signals here when the duration is reached so I close abort
close(abort)
}
fmt.Println(time.Since(t1))
return ok
}
funcWithUnpredictiveExecutionTime(abort) {
for {
select {
// doing work in here
case abort:
// except here, we clean up and return
}
}
}