different about withcancel and withtimeout in golang's context - go

I'm new in Golang and I have some confused about WithCancel and WithTimeout when I learn golang's context part.
Show the code.
package main
import (
"context"
"fmt"
"time"
)
func someHandler() {
//ctx, cancel := context.WithCancel(context.Background())
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
go doSth(ctx)
time.Sleep(3 * time.Second)
cancel()
}
func doSth(ctx context.Context) {
var i = 1
for {
time.Sleep(1 * time.Second)
select {
case <-ctx.Done():
fmt.Println("done")
return
default:
fmt.Printf("work %d seconds: \n", i)
}
i++
}
}
func main() {
fmt.Println("start...")
someHandler()
fmt.Println("end.")
}
Result:
// when use WithCancel
//
start...
work 1 seconds:
work 2 seconds:
end.
// when use WithTimeout
start...
work 1 seconds:
done
end.
My question is: why doesn't it print 'done' when I use withCancel but withTimeout does print it?

"Understanding the context package in golang" from Parikshit Agnihotry
mentions:
context.WithCancel(parent Context) (ctx Context, cancel CancelFunc)
This function creates a new context derived from the parent context that is passed in.
The parent can be a background context or a context that was passed into the function.
This returns a derived context and the cancel function.
Only the function that creates this should call the cancel function to cancel this context.
You can pass around the cancel function if you wanted to, but, that is highly not recommended. This can lead to the invoker of cancel not realizing what the downstream impact of canceling the context may be. There may be other contexts that are derived from this which may cause the program to behave in an unexpected fashion. In short, NEVER pass around the cancel function.
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(2 * time.Second))
context.WithDeadline(parent Context, d time.Time) (ctx Context, cancel CancelFunc)
This function returns a derived context from its parent that gets cancelled when the deadline exceeds or cancel function is called.
For example, you can create a context that will automatically get canceled at a certain time in future and pass that around in child functions.
When that context gets canceled because of deadline running out, all the functions that got the context get notified to stop work and return.
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(2 * time.Second))
context.WithTimeout(parent Context, timeout time.Duration) (ctx Context, cancel CancelFunc)
This function is similar to context.WithDeadline.
The difference is that it takes in time duration as an input instead of the time object.
This function returns a derived context that gets canceled if the cancel function is called or the timeout duration is exceeded.
ctx, cancel := context.WithDeadline(context.Background(), time.Now().Add(2 * time.Second))
why dones't print 'done' when use withCancel but withTimeout does
Probably because the Go program already exited before the goroutine has time to acknowledge the "Done" part.

We can find the relationship of cancelContext, timeOutContext and deadlineContext in the source code as below:
// cancelCtx declaration
type cancelCtx struct {
Context
mu sync.Mutex // protects following fields
done chan struct{} // created lazily, closed by first cancel call
children map[canceler]struct{} // set to nil by the first cancel call
err error // set to non-nil by the first cancel call
}
// timerCtx declaration
type timerCtx struct {
cancelCtx
timer *time.Timer // Under cancelCtx.mu.
deadline time.Time
}
func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc) {
return WithDeadline(parent, time.Now().Add(timeout))
}
cancelCtx is embeded inside the timerCtx, which realize inheritance in golang.

Related

How to detect a timeout occurred using Go's WithTimeout

I have the following Go code:
func MyFunc(ctx context.Context, cfg *Config) (packedevent []byte, err error, publishEvent bool) {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, cfg.ScanTimeout)
defer cancel()
event := GetEvent(ctx, cfg)
packedevent, err = PackEvent(event)
publishEvent = shouldSendToIoT(event)
return
}
I am trying to cause this function to timeout using context.WithTimeout.
What I have not been able to figure out is how to set err if the timeout occurs.
I looked at the example in the Go docs but I did not really understand it. Does the <-ctx.Done() case always mean that the timeout was reached? This example seems to suggest the opposite - that <-ctx.Done() means the code ran to completion without timing out.
I am looking for clarification on how to detect when code run with context.WithTimeout has or has not timed out.
Additionally, I would like to understand where in my code I should check if the timeout occurred. My first thought was to put this check at the end of the function, but would that be putting the check too late?
To detect if the context has timed out, check ctx.Error(). If the error is context.Canceled, then the context has been canceled using the cancel() function. If it is context.DeadlineExceeded, then it timed out.
To check if the context has been canceled or timed out, use:
select {
case <-ctx.Done():
// canceled or timed out
default:
// So the select will not block
}
ctx.Done() fires when the timeout is reached or the cancel func is called.
Per the docs:
Done returns a channel that's closed when work done on behalf of this context should be canceled.
You can try this code:
package main
import (
"context"
"fmt"
"time"
)
func main() {
ctx, cancel := context.WithTimeout(context.Background(), time.Second*2)
defer cancel()
for {
select {
case <-ctx.Done():
fmt.Println(ctx.Err())
return
default:
fmt.Println("do something")
time.Sleep(time.Second)
}
}
}
The output should be:
do something
do something
context deadline exceeded

Golang cli application - how to use context properly?

I'm new to golang and somewhat confused about context and how to use the context in golang applications.
Specifically im working on the cli application and just need to access mongo, for example.
Like - is this correct that I just create single shared ctx context variable, then use it for any operations that need context?
Would any operation that needs context restart the 5-second timer? or is this a shared timer?
package main
import (
"context"
"log"
"os"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
)
func main() {
log.SetOutput(os.Stdout)
// Create a context with a timeout of 5 seconds
//This defines a timeout context that will be canceled after 5 seconds.
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
// always defer in case func returns early
defer cancel()
//Creates a new ClientOptions instance.
clientOptions := options.Client()
clientOptions = clientOptions.ApplyURI("mongodb+srv://127.0.0.1?retryWrites=true&w=majority")
//Connect to mongo
client, err := mongo.Connect(ctx, clientOptions)
defer client.Disconnect(ctx)
if err != nil {
log.Fatal(err)
}
//Test connection to the database
log.Println("I: test mongo connection using ping")
err = client.Ping(ctx, readpref.Primary())
if err != nil {
log.Fatal(err)
}
log.Println("I: Fin")
}
If you think about it, it makes no sense that a context.Context could be shared "horizontally" (meaning between operations not part of the same call stack). A golang Context provides the context within which an operation (including any nested operations below it in the call stack) are to be performed - such as "within X seconds," to protect against hanging due to communications delays, etc. So if you issue 10 requests in parallel, you should give each one its own context - you probably don't want the tenth one to fail because the first one did. If you are just using a context.Background() or context.TODO(), without further decoration, you probably don't need to store the Context in a variable the first time you create it - you can just create when you pass it to the first function in the call stack, and properly constructed code will pass it down the stack as appropriate, applying necessary decorations along the way:
func Execute() {
DoThing(context.Background())
// Other stuff
}
func DoThing(pctx context.Context) {
ctx, cancel := context.WithTimeout(pctx, 10 * time.Second) // Timeout after 10 seconds
defer cancel()
DoThingThatMayTakeAWhile(ctx)
select {
// You may want other stuff here
case <-ctx.Done():
// Context timed out, maybe log an error?
}
}
func DoThingThatMayTakeAWhile(pctx context.Context) {
DoThingNeedingInfoInContext(context.WithValue(pctx, "thisisakey", "value"))
}
func DoThingNeedingInfoInContext(ctx context.Context) {
val := ctx.Value("thisisakey")
// Do something with val, check that it isn't nil, etc.
}
If I were to make multiple calls to DoThingThatMayTakeAWhile(), I'd want to give each one a separate child context - I would not want to share ctx with each of them.
So in your code, every call to mongo.Connect() should receive a freshly created context.Context instance.
You create a new context variable for each operations that need context.
The timer of a context will never restart.
In your example, try to add time.Sleep(6*time.Second) after context.WithTimeout, you will see all operations return error context deadline exceeded.

Why golang g.cancel() is required in func (g *Group) Wait() error

golang's error group is very useful, below is the implementation
func (g *Group) Go(f func() error) {
g.wg.Add(1)
go func() {
defer g.wg.Done()
if err := f(); err != nil {
g.errOnce.Do(func() {
g.err = err
if g.cancel != nil {
g.cancel()
}
})
}
}()
}
I understand that once one go-routine has error, the error group will cancel all other go-routines. However what I am confused is the Wait function below:
func (g *Group) Wait() error {
g.wg.Wait()
if g.cancel != nil {
g.cancel()
}
return g.err
}
Why we need to cancel again in this function? Could you provide me a case that this cancel is necessary?
I mean in success case, all go-routines finished task so no need cancel again, on the other hand, if one go-routine failed, no need as well because we canceled in the first code block.
Quick review of errgroup:
An errgroup.Group unifies error propagation and context cancelation.
The calling routines Wait() for subtasks to complete; if any subtask returns an error, Wait() returns that error back to the caller.
If any subtask returns an error, the group Context is canceled, which early-terminates all subtasks.
Regarding the group's Context:
If the parent Context (say, an http request context) is canceled, the group Context is also canceled. This helps avoid unnecessary work. For example, if the user navigates away from the page and cancels the http request, we can stop immediately.
You can see from the code that when an errgroup is created, a new context is created:
// WithContext returns a new Group and an associated Context derived from ctx.
//
// The derived Context is canceled the first time a function passed to Go
// returns a non-nil error or the first time Wait returns, whichever occurs
// first.
func WithContext(ctx context.Context) (*Group, context.Context) {
ctx, cancel := context.WithCancel(ctx)
return &Group{cancel: cancel}, ctx
}
The cancel function that you see on the group is a return value of the withCancel function, and belongs to this new context, not directly to the group.
This context needs to be cancelled as soon as one function returns an error, but also if the operation completes, as you can see from the code's documentation: "or the first time Wait Returns"
Why?
The wait is a wait on the entire wait group.
// Wait blocks until all function calls from the Go method have returned, then
// returns the first non-nil error (if any) from them.
func (g *Group) Wait() error {
g.wg.Wait()
if g.cancel != nil {
g.cancel()
}
return g.err
}
This means that the entire operation is complete. Its context needs to be cancelled to clear up the context's resources. Perhaps the operation even started a child context which is still running and no longer needed, and the cancellation needs to be propagated to it.

Go ctx.Done() never firing in select statement

I have the following code for a module I'm developing and I'm not sure why the provider.Shutdown() function is never called when I called .Stop()
The main process does stop but I'm confused why this doesn't work?
package pluto
import (
"context"
"fmt"
"log"
"sync"
)
type Client struct {
name string
providers []Provider
cancelCtxFunc context.CancelFunc
}
func NewClient(name string) *Client {
return &Client{name: name}
}
func (c *Client) Start(blocking bool) {
log.Println(fmt.Sprintf("Starting the %s service", c.name))
ctx, cancel := context.WithCancel(context.Background())
c.cancelCtxFunc = cancel // assign for later use
var wg sync.WaitGroup
for _, p := range c.providers {
wg.Add(1)
provider := p
go func() {
provider.Setup()
select {
case <-ctx.Done():
// THIS IS NEVER CALLED?!??!
provider.Shutdown()
return
default:
provider.Run(ctx)
}
}()
}
if blocking {
wg.Wait()
}
}
func (c *Client) RegisterProvider(p Provider) {
c.providers = append(c.providers, p)
}
func (c *Client) Stop() {
log.Println("Attempting to stop service")
c.cancelCtxFunc()
}
Client code
package main
import (
"pluto/pkgs/pluto"
"time"
)
func main() {
client := pluto.NewClient("test-client")
testProvider := pluto.NewTestProvider()
client.RegisterProvider(testProvider)
client.Start(false)
time.Sleep(time.Second * 3)
client.Stop()
}
Because it's already chosen the other case before the context is cancelled. Here is your code, annotated:
// Start a new goroutine
go func() {
provider.Setup()
// Select the first available case
select {
// Is the context cancelled right now?
case <-ctx.Done():
// THIS IS NEVER CALLED?!??!
provider.Shutdown()
return
// No? Then call provider.Run()
default:
provider.Run(ctx)
// Run returned, nothing more to do, we're not in a loop, so our goroutine returns
}
}()
Once provider.Run is called, cancelling the context isn't going to do anything in the code shown. provider.Run also gets the context though, so it is free to handle cancellation as it sees fit. If you want your routine to also see cancellation, you could wrap this in a loop:
go func() {
provider.Setup()
for {
select {
case <-ctx.Done():
// THIS IS NEVER CALLED?!??!
provider.Shutdown()
return
default:
provider.Run(ctx)
}
}
}()
This way, once provider.Run returns, it will go through the select again, and if the context has been cancelled, that case will be called. However, if the context hasn't been cancelled, it'll call provider.Run again, which may or may not be what you want.
EDIT:
More typically, you'd have one of a couple scenarios, depending on how provider.Run and provider.Shutdown work, which hasn't been made clear in the question, so here are your options:
Shutdown must be called when the context is cancelled, and Run must only be called once:
go func() {
provider.Setup()
go provider.Run(ctx)
go func() {
<- ctx.Done()
provider.Shutdown()
}()
}
Or Run, which already receives the context, already does the same thing as Shutdown when the context is cancelled, and therefore calling Shutdown when the context is cancelled is wholly unnecessary:
go provider.Run(ctx)

Stopping Go routine from tests

I have a node that I want to spin up until I decide I want to stop it. I currently have a Start method that blocks on the Contexts Done channel, I then have a Stop function that calls the cancel, in my tests my Start seems to hang forever and the Stop is never called. I can't work out why the Done signal isn't being called and stopping my node.
var (
// Ctx is the node's main context.
Ctx, cancel = context.WithCancel(context.Background())
// Cancel is a function used to initiate graceful shutdown.
Cancel = cancel
)
type (
Node struct {
database *store.Store
}
)
// Starts run the Node.
func (n *Node) Start() error {
var nodeError error
defer func() {
err := n.database.Close()
if err != nil {
nodeError = err
}
}()
<-Ctx.Done()
return nodeError
}
// Stop stops the node.
func (n *Node) Stop() {
Cancel()
}
And my test is:
func TestNode_Start(t *testing.T) {
n, _ := node.NewNode("1.0")
err := n.Start()
n.Stop()
assert.NoError(t, err)
}
There are several problems with your code. Let's break it down.
var (
// Ctx is the node's main context.
Ctx, cancel = context.WithCancel(context.Background())
// Cancel is a function used to initiate graceful shutdown.
Cancel = cancel
)
These should not be package variables. They should be instance variables--that is to say, members or the Node struct. By making these package variables, if you have multiple tests that use Node, they will all step on each others toes, cause race conditions, and crashes. So instead, do this:
type Node struct {
database *store.Store
ctx context.Context
cancel context.CancelFunc
}
Next, we see that you have a deferred function in your Start() method:
// Starts run the Node.
func (n *Node) Start() error {
var nodeError error
defer func() {
err := n.database.Close()
if err != nil {
nodeError = err
}
}()
/* snip */
This does not do what you expect. It closes the database connection as soon as Start() returns--before anything possibly has a chance to use it.
Instead, you should close the database connection as part of your Stop() method:
// Stop stops the node.
func (n *Node) Stop() error {
n.cancel()
return n.database.Close()
}
And finally, your Start() method blocks, because it waits for the context to cancel, which cannot possibly be canceled until Stop() is called, which is only ever called after Start() returns:
func (n *Node) Start() error {
/* snip */
<-Ctx.Done()
return nodeError
}
I cannot think of any reason to have <-Ctx.Done in Start() at all, so I would just remove it.
With all of my suggested changes, you should have something like this:
type Node struct {
database *store.Store
ctx context.Context
cancel context.CancelFunc
}
// Starts run the Node.
func (n *Node) Start() {
n.ctx, n.cancel = context.WithCancel(context.Background())
}
// Stop stops the node.
func (n *Node) Stop() error {
n.cancel()
return n.database.Close()
}
Of course, this still leaves open the question of if/where/how ctx is used. Since your original code didn't include that, I didn't either.

Resources