calling wait group done right after go routine starts? - go

https://go.dev/play/p/YVYRWSgcp4u
I'm seeing this code in "Concurrency in Go Tools and Techniques for Developers", where it's mentioned about the usage for broadcast, the context is to use broadcast to wake three goroutings.
package main
import (
"fmt"
"sync"
"time"
)
func main() {
type Button struct {
Clicked *sync.Cond
}
button := Button{Clicked: sync.NewCond(&sync.Mutex{})}
subscribe := func(c *sync.Cond, fn func()) {
var goroutineRunning sync.WaitGroup
goroutineRunning.Add(1)
go func() {
goroutineRunning.Done() // <---- why here?
//fmt.Println("wg already done")
c.L.Lock()
defer c.L.Unlock()
c.Wait()
fn()
//goroutineRunning.Done(), if put here will result in deadlock, why?
}()
goroutineRunning.Wait()
}
var clickRegistered sync.WaitGroup
clickRegistered.Add(3)
subscribe(button.Clicked, func() {
fmt.Println("Maximizing window.")
clickRegistered.Done()
})
subscribe(button.Clicked, func() {
fmt.Println("Displaying annoying dialog box!")
clickRegistered.Done()
})
subscribe(button.Clicked, func() {
fmt.Println("Mouse clicked.")
clickRegistered.Done()
})
time.Sleep(time.Second * 3)
button.Clicked.Broadcast()
clickRegistered.Wait()
}
I'm trying to understand the subscribe part
subscribe := func(c *sync.Cond, fn func()) {
var goroutineRunning sync.WaitGroup
goroutineRunning.Add(1)
go func() {
goroutineRunning.Done()
//fmt.Println("wg already done")
c.L.Lock()
defer c.L.Unlock()
c.Wait()
fn()
//goroutineRunning.Done()
//fmt.Println("fn executed")
}()
goroutineRunning.Wait()
}
The author says:
Here we define a convenience function that will allow us to register functions to handle signals from a condition. Each handler is run on its own goroutine, and subscribe will not exit until that goroutine is confirmed to be running.
My understanding is that we should defer goroutingRunning.Done() inside gorouting so that the subsequent code (including waiting for Cond and fn() call, will have opportunities
to run), but in this case it seems the goroutingRunning.Done() has to be in the beginning of gorouting, otherwise it would result in deadlock error, but why?
------UPDATE------
We could actually get rid of the waitgroup in subscribe function this way:
subscribe := func(c *sync.Cond, fn func(), wg *sync.WaitGroup) {
c.L.Lock()
defer c.L.Unlock()
c.Wait()
fn()
wg.Done()
}
var ClickRegistered sync.WaitGroup
ClickRegistered.Add(3)
go subscribe(button.Clicked, func() {
fmt.Println("do 1")
}, &ClickRegistered)
go subscribe(button.Clicked, func() {
fmt.Println("do 2")
}, &ClickRegistered)
go subscribe(button.Clicked, func() {
fmt.Println("do 3")
}, &ClickRegistered)
time.Sleep(time.Millisecond * 50)
fmt.Println("some process in main go routine finished")
button.Clicked.Broadcast()
ClickRegistered.Wait()

This is a mechanism that ensures that when subscribe returns, the goroutine has started running. When goroutine starts, it calls Done to signal to the waiting caller that the goroutine is running. Without this mechanism, it is possible that when subscribe returns the goroutine has not been scheduled yet.
A deferred Done will not work, because that will only run once the goroutine returns, and that will not happen until the condition variable is signaled.
The scheme does not guarantee that the new goroutine locks the mutex. It is debatable whether this schema is really necessary.

Related

goroutine not seeing context cancel?

I have two goroutines running at the same time.
At some point, I want my program to exit gracefully so I use the cancel() func to notify my goroutines that they need to be stopped, but only one of the two receive the message.
here is my main (simplified):
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
done := make(chan os.Signal, 1)
signal.Notify(done, os.Interrupt, syscall.SIGINT, syscall.SIGTERM)
wg := &sync.WaitGroup{}
wg.Add(2)
go func() {
err := eng.Watcher(ctx, wg)
if err != nil {
cancel()
}
}()
go func() {
err := eng.Suspender(ctx, wg)
if err != nil {
cancel()
}
}()
<-done // wait for SIGINT / SIGTERM
log.Print("receive shutdown")
cancel()
wg.Wait()
log.Print("controller exited properly")
The Suspender goroutine exist successfully (here is the code):
package main
import (
"context"
"sync"
"time"
log "github.com/sirupsen/logrus"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/util/retry"
)
func (eng *Engine) Suspender(ctx context.Context, wg *sync.WaitGroup) error {
contextLogger := eng.logger.WithFields(log.Fields{
"go-routine": "Suspender",
})
contextLogger.Info("starting Suspender goroutine")
now := time.Now().In(eng.loc)
for {
select {
case n := <-eng.Wl:
//dostuff
case <-ctx.Done():
// The context is over, stop processing results
contextLogger.Infof("goroutine Suspender canceled by context")
return nil
}
}
}
and here is the func that is not receiving the context cancellation:
package main
import (
"context"
"sync"
"time"
log "github.com/sirupsen/logrus"
)
func (eng *Engine) Watcher(ctx context.Context, wg *sync.WaitGroup) error {
contextLogger := eng.logger.WithFields(log.Fields{
"go-routine": "Watcher",
"uptime-schedule": eng.upTimeSchedule,
})
contextLogger.Info("starting Watcher goroutine")
ticker := time.NewTicker(time.Second * 30)
for {
select {
case <-ctx.Done():
contextLogger.Infof("goroutine watcher canceled by context")
log.Printf("toto")
return nil
case <-ticker.C:
//dostuff
}
}
}
}
Can you please help me ?
Thanks :)
Did you try it with an errgroup? It has context cancellation baked in:
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
defer cancel()
done := make(chan os.Signal, 1)
signal.Notify(done, os.Interrupt, syscall.SIGINT, syscall.SIGTERM)
// "golang.org/x/sync/errgroup"
wg, ctx := errgroup.WithContext(ctx)
wg.Go(func() error {
return eng.Watcher(ctx, wg)
})
wg.Go(func() error {
return eng.Suspender(ctx, wg)
})
wg.Go(func() error {
defer cancel()
<-done
return nil
})
err := wg.Wait()
if err != nil {
log.Print(err)
}
log.Print("receive shutdown")
log.Print("controller exited properly")
On the surface the code looks good. The only thing I can think is that it's busy in "dostuff". It can be tricky to step through timing related code in the debugger so try adding some logging:
case <-ticker.C:
log.Println("doing stuff")
//dostuff
log.Println("done stuff")
(I also assume you are calling wg.Done() in your go-routines somewhere though if they are missing that would not be the cause of the problem you describe.)
The code in Suspender and in Watcher doesn't decrement the waitgroup counter through the Done() method call - the reason behind the infinite execution.
And to be honest it's quite normal to forget such small things. That's why as a standard general practice in Go, it is suggested to use defer and handle things that are critical (and should be handled inside the function/method ) at the very beginning.
The updated implementation might look like
func (eng *Engine) Suspender(ctx context.Context, wg *sync.WaitGroup) error {
defer wg.Done()
// ------------------------------------
func (eng *Engine) Watcher(ctx context.Context, wg *sync.WaitGroup) error {
defer wg.Done()
contextLogger := eng.logger.WithFields(log.Fields{
Also, another suggestion, looking at the main routine, it is always suggested to pass context by value to any go-routine or method calls (lambda) that are being invoked.
This approach saves developers from a lot of program-related bugs that can't be noticed very easily.
go func(ctx context.Context) {
err := eng.Watcher(ctx, wg)
if err != nil {
cancel()
}
}(ctx)
Edit-1: (the exact solution)
Try passing the context using the value in the go routines as I mentioned earlier. Otherwise, both of the go routine will use a single context (because you are referencing it) and only one ctx.Done() will be fired.
By passing ctx as a value 2 separate child contexts are created in Go. And while closing parent with cancel() - both children independently fires ctx.Done().

Why Go channel is blocking but not quitting

I have those two versions to implement context cancellation over signals using signal.NotifyContext
Version 1 https://play.golang.org/p/rwOnYEgPecE
func main() {
ch := run()
<-ch
}
func run() chan bool {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM, syscall.SIGQUIT)
var done = make(chan bool)
//go func() {
select {
case <-ctx.Done():
fmt.Println("Quitting")
stop()
done <- true
}
//}()
return done
}
Version 2 https://play.golang.org/p/oijbICeSrNT
func main() {
ch := run()
<-ch
}
func run() chan bool {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM, syscall.SIGQUIT)
var done = make(chan bool)
go func() {
select {
case <-ctx.Done():
fmt.Println("Quitting")
stop()
done <- true
}
}()
return done
}
Why is the first version printing the line Quitting but does not exit, while the second version prints and quits properly?
The reason the first case doesn't behave as you expect is because everything is running in a single (main) goroutine.
select {
case <-ctx.Done():
fmt.Println("Quitting")
stop()
done <- true // this blocks because no one is listening
}
}
in your second example because the above logic is run in a goroutine, then main() will be listening on the channel.
The second method is preferred, since any signal handler - which is running for the lifetime of a program - should be running in its own goroutine.

context ctx.Done not being executed even though context was passed to the function in golang

I just don't understand why ctx.Done() is not being executed even though I am passing context and calling the cancel from the main? What am I doing wrong here?
var c = make(chan string)
func A(ctx context.Context) {
for {
select {
case <-ctx.Done():
fmt.Println("killing AAAA")
return // kill A at least
default:
fmt.Println("in A1.. .. again")
c <- "yesss"
}
}
}
//func B(ctx context.Context) {
func main() {
ctx, cancel := context.WithCancel(context.Background())
fmt.Println("BEFORE Number of active goroutines ", runtime.NumGoroutine())
go A(ctx)
time.Sleep(2 * time.Second)
valueReceived := <-c
cancel()
fmt.Println("AFTER Number of active goroutines ", runtime.NumGoroutine())
}
The goroutine executes the default branch twice and blocks on send to c. The <-ctx.Done() case is not executed because the goroutine is stuck in the default branch.
Fix the problem by sending from the select case instead of the branch statements.
func A(ctx context.Context) {
for {
select {
case <-ctx.Done():
fmt.Println("killing AAAA")
return // kill A at least
case c <- "yesss":
fmt.Println("in A1.. .. again")
}
}
}
You may not see the the killing AAAA with this change alone because the program can exit before the goroutine runs to completion.
Wait for the goroutine to complete to see the message:
var wg sync.WaitGroup
func A(ctx context.Context) {
defer wg.Done()
for {
select {
case <-ctx.Done():
fmt.Println("killing AAAA")
return // kill A at least
case c <- "yesss":
fmt.Println("in A1.. .. again")
}
}
}
...
wg.Add(1)
go A(ctx)
time.Sleep(2 * time.Second)
valueReceived := <-c
cancel()
wg.Wait()
Run it on the Go playground.

close(channel) used to implement the observer pattern

I need to stop an HTTP server on demand besides calling other functions as well when receiving the "quit" signal in no specific order.
In my try to implement something like the observer pattern, I found "handy" to create a channel (quit := make(chan struct{}), let's say the "subject" and then on each of the goroutines "observers" listen on that channel <-quit waiting until a change for then to continue.
The way I trigger all the functions at once is by closing the channel close(quit) not by writing into it, I have tried this and so far working, but wondering if there are some cons with this approach or if there are better/idiomatic ways of implementing similar behavior/pattern.
package main
import (
"log"
"net/http"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup
srv := &http.Server{Addr: ":8080"}
wg.Add(1)
go func() {
log.Println(srv.ListenAndServe())
wg.Done()
}()
quit := make(chan struct{})
go func() {
<-quit
if err := srv.Close(); err != nil {
log.Printf("HTTP server Shutdown: %v", err)
}
}()
wg.Add(1)
go func() {
<-quit
log.Println("just waiting 1")
wg.Done()
}()
wg.Add(1)
go func() {
<-quit
log.Println("just waiting 2")
wg.Done()
}()
<-time.After(2 * time.Second)
close(quit)
wg.Wait()
}
https://play.golang.org/p/uIfMJfN6xQy
I would say your way is good enough but lacks some elegance.
You could implement required behavior using sync.Cond:
https://golang.org/pkg/sync/#Cond
How to correctly use sync.Cond?

How to signal all go routines that are waiting?

The problem I have is to "signal all go routines that are waiting".
Some go routines may be performing other tasks (simulated by time.Sleep in code below), before waiting for the signal. Others go routines may wait for the signal immediately after starting. In some case there may not be any go routines that are waiting (not simulated in the code below)
I came with the following approach:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
wg := sync.WaitGroup{}
wg.Add(3)
cond := sync.NewCond(&sync.Mutex{})
signalled := false
producer := func() {
time.Sleep(1)
cond.L.Lock()
signalled = true
cond.Broadcast()
cond.L.Unlock()
wg.Done()
}
go producer()
go func() {
time.Sleep(3 * time.Second)
cond.L.Lock()
for !signalled {
cond.Wait()
}
cond.L.Unlock()
wg.Done()
}()
go func() {
cond.L.Lock()
for !signalled {
cond.Wait()
}
cond.L.Unlock()
wg.Done()
}()
wg.Wait()
fmt.Println("done once")
reset := func() {
cond.L.Lock()
signalled = false
cond.L.Unlock()
}
// a reset function
reset()
wg.Add(2)
go producer()
go func() {
cond.L.Lock()
for !signalled {
cond.Wait()
}
cond.L.Unlock()
wg.Done()
}()
wg.Wait()
fmt.Println("done twice")
}
https://play.golang.org/p/uNmyofH2hs
I can use a channel and close the channel as a signal. But this means that in case there is a need for "reset" (where in the "signal" can be triggered again), the channel would have to be created again.
Is there a better (concise) way to achieve this?

Resources