Consider this (https://play.golang.org/p/zvDiwul9QR0):
package main
import (
"context"
"fmt"
"time"
)
func main() {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
for {
select {
case <-ctx.Done():
fmt.Println("Done")
break
default:
for {
fmt.Println("loop")
time.Sleep(500 * time.Millisecond)
}
}
}
}
So here the contexts returns a "Done()" channel after 2 seconds. And I want to catch this and cancel my infinite for loop. The code example above does not do this, it never exits the loop.
How can I achieve this?
Context cancelation is not magic - they are just a signal mechanism. To abort work, you need to monitor the state of the context from your worker goroutine:
for {
fmt.Println("loop")
select {
case <-time.After(500 * time.Millisecond):
case <-ctx.Done():
return
}
}
https://play.golang.org/p/L6-nDpo9chb
also as Eli pointed out, break will only break out of the select statement - so you need something more precise to break out of a loop. Refactoring into functions make return's much more intuitive for task abortion.
Following up from comments. I would refactor your task like so:
// any potentially blocking task should take a context
// style: context should be the first passed in parameter
func myTask(ctx context.Context, poll time.Duration) error {
for {
fmt.Println("loop")
select {
case <-time.After(poll):
case <-ctx.Done():
return ctx.Err()
}
}
}
https://play.golang.org/p/I3WDVd1uHbz
Related
I have the following code for a module I'm developing and I'm not sure why the provider.Shutdown() function is never called when I called .Stop()
The main process does stop but I'm confused why this doesn't work?
package pluto
import (
"context"
"fmt"
"log"
"sync"
)
type Client struct {
name string
providers []Provider
cancelCtxFunc context.CancelFunc
}
func NewClient(name string) *Client {
return &Client{name: name}
}
func (c *Client) Start(blocking bool) {
log.Println(fmt.Sprintf("Starting the %s service", c.name))
ctx, cancel := context.WithCancel(context.Background())
c.cancelCtxFunc = cancel // assign for later use
var wg sync.WaitGroup
for _, p := range c.providers {
wg.Add(1)
provider := p
go func() {
provider.Setup()
select {
case <-ctx.Done():
// THIS IS NEVER CALLED?!??!
provider.Shutdown()
return
default:
provider.Run(ctx)
}
}()
}
if blocking {
wg.Wait()
}
}
func (c *Client) RegisterProvider(p Provider) {
c.providers = append(c.providers, p)
}
func (c *Client) Stop() {
log.Println("Attempting to stop service")
c.cancelCtxFunc()
}
Client code
package main
import (
"pluto/pkgs/pluto"
"time"
)
func main() {
client := pluto.NewClient("test-client")
testProvider := pluto.NewTestProvider()
client.RegisterProvider(testProvider)
client.Start(false)
time.Sleep(time.Second * 3)
client.Stop()
}
Because it's already chosen the other case before the context is cancelled. Here is your code, annotated:
// Start a new goroutine
go func() {
provider.Setup()
// Select the first available case
select {
// Is the context cancelled right now?
case <-ctx.Done():
// THIS IS NEVER CALLED?!??!
provider.Shutdown()
return
// No? Then call provider.Run()
default:
provider.Run(ctx)
// Run returned, nothing more to do, we're not in a loop, so our goroutine returns
}
}()
Once provider.Run is called, cancelling the context isn't going to do anything in the code shown. provider.Run also gets the context though, so it is free to handle cancellation as it sees fit. If you want your routine to also see cancellation, you could wrap this in a loop:
go func() {
provider.Setup()
for {
select {
case <-ctx.Done():
// THIS IS NEVER CALLED?!??!
provider.Shutdown()
return
default:
provider.Run(ctx)
}
}
}()
This way, once provider.Run returns, it will go through the select again, and if the context has been cancelled, that case will be called. However, if the context hasn't been cancelled, it'll call provider.Run again, which may or may not be what you want.
EDIT:
More typically, you'd have one of a couple scenarios, depending on how provider.Run and provider.Shutdown work, which hasn't been made clear in the question, so here are your options:
Shutdown must be called when the context is cancelled, and Run must only be called once:
go func() {
provider.Setup()
go provider.Run(ctx)
go func() {
<- ctx.Done()
provider.Shutdown()
}()
}
Or Run, which already receives the context, already does the same thing as Shutdown when the context is cancelled, and therefore calling Shutdown when the context is cancelled is wholly unnecessary:
go provider.Run(ctx)
I'm trying to write a program in go that is similar to cron with the addition that jobs are given a max runtime and if a function exceeds this duration, the job should exit. Here is my my whole code:
package main
import (
"fmt"
"log"
"sync"
"time"
)
type Job struct {
ID string
MaxRuntime time.Duration
Frequency time.Duration
Function func()
}
func testFunc() {
log.Println("OPP11")
time.Sleep(7 * time.Second)
log.Println("OP222")
}
func New(ID, frequency, runtime string, implementation func()) Job {
r, err := time.ParseDuration(runtime)
if err != nil {
panic(err)
}
f, err := time.ParseDuration(frequency)
if err != nil {
panic(err)
}
j := Job{ID: ID, MaxRuntime: r, Frequency: f, Function: implementation}
log.Printf("Created job %#v with frequency %v and max runtime %v", ID, f, r)
return j
}
func (j Job) Run() {
for range time.Tick(j.Frequency) {
start := time.Now()
log.Printf("Job %#v executing...", j.ID)
done := make(chan int)
//quit := make(chan int)
//var wg sync.WaitGroup
//wg.Add(1)
go func() {
j.Function()
done <- 0
}()
select {
case <-done:
elapsed := time.Since(start)
log.Printf("Job %#v completed in %v \n", j.ID, elapsed)
case <-time.After(j.MaxRuntime):
log.Printf("Job %#v halted after %v", j.ID, j.MaxRuntime)
// here should exit the above goroutine
}
}
}
func main() {
// create a new job given its name, frequency, max runtime
// and the function it should run
testJob := New("my-first-job", "3s", "5s", func() {
testFunc()
})
testJob.Run()
}
What I'm trying to do is that in the second case in the select of the Run() function, it should exit the goroutine which is running the function. I tried to do this by wrapping the function in a for loop with a select statement which listens on a quit channel like this:
go func() {
for {
select {
case <-quit:
fmt.Println("quiting goroutine")
return
default:
j.Function()
done <- 0
}
}
}()
And then having quit <- 1 in the Run() function, but that doesnt seem to be doing anything. Is there a better of doing this?
As explained in the comments, the whole problem is that you want to cancel the execution of a function (j.Function) that isn't cancellable.
There's no way to "kill a goroutine". Goroutines work in a cooperative fashion. If you want to be able to "kill it", you need to ensure that the function running in that Goroutine has a mechanism for you to signal that it should stop what it's doing and return, letting the Goroutine that was running it finally terminate.
The standard way of indicating that a function is cancellable is by having it take a context.Context as its first param:
type Job struct {
// ...
Function func(context.Context)
}
Then you create the context and pass it to the j.Function. Since your cancellation logic is simply based on a timeout, there's no need to write all that select ... case <-time.After(...), as that is provided as built-in functionality with a context.Context:
func (j Job) Run() {
for range time.Tick(j.Frequency) {
go j.ExecuteOnce()
}
}
func (j Job) ExecuteOnce() {
log.Printf("Job %#v executing...", j.ID)
ctx, cancel := context.WithTimeout(context.Background(), j.MaxRuntime)
defer cancel()
j.Function(ctx)
}
Now, to finish, you have to rewrite the functions that you're going to be passing to your job scheduler so that they take context.Context and, very importantly, that they use it properly and cancel whatever they're doing when the context is cancelled.
This means that if you're writing the code for those funcs and they will somehow block, you'll be responsible for writing stuff like:
select {
case <-ctx.Done():
return ctx.Err()
case ...your blocking case...:
}
If your funcs are invoking 3rd party code, then that code needs to be aware of context and cancellation, and you'll need to pass down the ctx your funcs receive.
I'm relatively new to Golang and am trying to incorporate Contexts into my code.
I see the benefits in terms of cancelling from the parent as well as sharing context-specific stuff (loggers, for example).
Beyond that, I might be missing something, but I can't see a way for a child to cancel the context. The example here would be if one of the child routines encounters an error that means the whole context is done.
Here's some sample code:
package main
import (
"context"
"fmt"
"math/rand"
"os"
"os/signal"
"sync"
"time"
)
func main() {
ctx, cancel := context.WithCancel(context.Background())
// handle SIGINT (control+c)
go func() {
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
<-c
fmt.Println("main: interrupt received. cancelling context.")
cancel()
}()
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
child1DoWork(ctx)
wg.Done()
}()
wg.Add(1)
go func() {
child2DoWork(ctx)
wg.Done()
}()
fmt.Println("main: waiting for children to finish")
wg.Wait()
fmt.Println("main: children done. exiting.")
}
func child1DoWork(ctx context.Context) {
// pretend we're doing something useful
tck := time.NewTicker(5 * time.Second)
for {
select {
case <-tck.C:
fmt.Println("child1: still working")
case <-ctx.Done():
// context cancelled
fmt.Println("child1: context cancelled")
return
}
}
}
func child2DoWork(ctx context.Context) {
// pretend we're doing something useful
tck := time.NewTicker(2 * time.Second)
for {
select {
case <-tck.C:
if rand.Intn(5) < 4 {
fmt.Println("child2: did some work")
} else {
// pretend we encountered an error
fmt.Println("child2: error encountered. need to cancel but how do I do it?!?")
// PLACEHOLDER: HOW TO CANCEL FROM HERE?
return
}
case <-ctx.Done():
// context cancelled
fmt.Println("child2: context cancelled")
return
}
}
}
Here you have an example of cancelling from the parent (due to SIGINT) which works great. However, there's a placeholder in child2DoWork where an error is encountered and I want to then cancel the whole context, but I can't see a way to do that with the vanilla context capabilities.
Is this out-of-scope for contexts? Clearly I could communicate from child2 back to the parent which could then cancel, but I'm wondering if there isn't an easier way.
If communication back to the parent is the proper way, is there an idiomatic way of doing this? It does seem like a common problem.
Thanks!
A child can't and shouldn't cancel a context, it's the parent's call. What a child may do is return an error, and the parent should decide if the error requires cancelling the context.
Just because a "subtask" fails, it doesn't mean all other subtasks need to be cancelled. Often, a failing subtask may have a meaning that other subtasks become more important. Think of a parallel search: you may use multiple subtasks to search for the same thing in multiple sources. You may use the fastest result and may wish to cancel the slower ones. If a search fails, you do want the rest to continue.
Obviously if you pass the cancel function to the child, the child will have the power to cancel the context. But instead leave that power at the parent.
Is this out-of-scope for contexts? Clearly I could communicate from child2 back to the parent which could then cancel, but I'm wondering if there isn't an easier way.
Yes, this is exactly backwards for contexts. They are explicitly for a caller to cancel. The correct mechanism here is the simplest and most obvious: when child2DoWork encounters an error, it should return an error, and when the caller gets an error back, if the correct response is to cancel other tasks, it can then cancel the appropriate context(s).
Essentially, the child is a task, and it should be isolated from any other tasks. It shouldn't be trying to manage its siblings; the parent should be managing all of its children.
In the case that
parent spawn multiple child goroutines to achieve one goal
if one child failed, parent need to stop its siblings
you can use a channel to communicate, parent can listen to the channel, once there is an error, parent can cancel all children task.
I have modified your code
package main
import (
"context"
"fmt"
"math/rand"
"os"
"os/signal"
"sync"
"time"
)
func main() {
ctx, cancel := context.WithCancel(context.Background())
errChan := make(chan error)
// handle SIGINT (control+c)
go func() {
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
select {
case <-c:
fmt.Println("main: interrupt received. cancelling context.")
case err := <-errChan:
fmt.Printf("main: child goroutine returns error. cancelling context. %s\n", err)
}
cancel()
}()
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
child1DoWork(ctx)
wg.Done()
}()
wg.Add(1)
go func() {
child2DoWork(ctx, errChan)
wg.Done()
}()
fmt.Println("main: waiting for children to finish")
wg.Wait()
fmt.Println("main: children done. exiting.")
}
func child1DoWork(ctx context.Context) {
// pretend we're doing something useful
tck := time.NewTicker(5 * time.Second)
for {
select {
case <-tck.C:
fmt.Println("child1: still working")
case <-ctx.Done():
// context cancelled
fmt.Println("child1: context cancelled")
return
}
}
}
func child2DoWork(ctx context.Context, errChan chan error) {
// pretend we're doing something useful
tck := time.NewTicker(2 * time.Second)
for {
select {
case <-tck.C:
if rand.Intn(5) < 4 {
fmt.Println("child2: did some work")
} else {
// pretend we encountered an error
fmt.Println("child2: error encountered")
// PLACEHOLDER: HOW TO CANCEL FROM HERE?
errChan <- fmt.Errorf("error in child2")
return
}
case <-ctx.Done():
// context cancelled
fmt.Println("child2: context cancelled")
return
}
}
}
I have a func that receives context and does some cpubound operations like bellow.
func DoSomeOperation(ctx context.Context){
CPUBoundWork1()
CPUBoundWork2()
CPUBoundWork3()
CPUBoundWork4()
}
What I want to do is to check if Context has been cancelled or not before making each CPUBound func call. If cancelled I want to return immidiately without making next func call. Is there any way to do this?
Use ctx.Err()
if ctx.Err() == context.Canceled {
return
}
You also can use select statement with slice of functions.
For Example:
ctx := ...
executors := []func(){...}
Loop:
for _,executor := range executors{
select {
case <-ctx.Done():
if ctx.Err() == context.Canceled {
break Loop
}
if ctx.Err() == context.DeadlineExceeded {
//do something else
}
default:
executor()
}
}
PLAYGROUND
I am attempting to create a poller in Go that spins up and every 24 hours executes a function.
I want to also be able to stop the polling, I'm attempting to do this by having a done channel and passing down an empty struct to stop the for loop.
In my tests, the for just loops infinitely and I can't seem to stop it, am I using the done channel incorrectly? The ticker case works as expected.
Poller struct {
HandlerFunc HandlerFunc
interval *time.Ticker
done chan struct{}
}
func (p *Poller) Start() error {
for {
select {
case <-p.interval.C:
err := p.HandlerFunc()
if err != nil {
return err
}
case <-p.done:
return nil
}
}
}
func (p *Poller) Stop() {
p.done <- struct{}{}
}
Here is the test that's exeuting the code and causing the infinite loop.
poller := poller.NewPoller(
testHandlerFunc,
time.NewTicker(1*time.Millisecond),
)
err := poller.Start()
assert.Error(t, err)
poller.Stop()
Seems like problem is in your use case, you calling poller.Start() in blocking maner, so poller.Stop() is never called. It's common, in go projects to call goroutine inside of Start/Run methods, so, in poller.Start(), i would do something like that:
func (p *Poller) Start() <-chan error {
errc := make(chan error, 1 )
go func() {
defer close(errc)
for {
select {
case <-p.interval.C:
err := p.HandlerFunc()
if err != nil {
errc <- err
return
}
case <-p.done:
return
}
}
}
return errc
}
Also, there's no need to send empty struct to done channel. Closing channel like close(p.done) is more idiomatic for go.
There is no explicit way in Go to broadcast an event to go routines for something like cancellation. Instead its idiomatic to create a channel that when closed signifies a message such as cancelling any work it has to do. Something like this is a viable pattern:
var done = make(chan struct{})
func cancelled() bool {
select {
case <-done:
return true
default:
return false
}
}
Go-routines can call cancelled to poll for a cancellation.
Then your main loop can respond to such an event but make sure you drain any channels that might cause go-routines to block.
for {
select {
case <-done:
// Drain whatever channels you need to.
for range someChannel { }
return
//.. Other cases
}
}