The problem I have is to "signal all go routines that are waiting".
Some go routines may be performing other tasks (simulated by time.Sleep in code below), before waiting for the signal. Others go routines may wait for the signal immediately after starting. In some case there may not be any go routines that are waiting (not simulated in the code below)
I came with the following approach:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
wg := sync.WaitGroup{}
wg.Add(3)
cond := sync.NewCond(&sync.Mutex{})
signalled := false
producer := func() {
time.Sleep(1)
cond.L.Lock()
signalled = true
cond.Broadcast()
cond.L.Unlock()
wg.Done()
}
go producer()
go func() {
time.Sleep(3 * time.Second)
cond.L.Lock()
for !signalled {
cond.Wait()
}
cond.L.Unlock()
wg.Done()
}()
go func() {
cond.L.Lock()
for !signalled {
cond.Wait()
}
cond.L.Unlock()
wg.Done()
}()
wg.Wait()
fmt.Println("done once")
reset := func() {
cond.L.Lock()
signalled = false
cond.L.Unlock()
}
// a reset function
reset()
wg.Add(2)
go producer()
go func() {
cond.L.Lock()
for !signalled {
cond.Wait()
}
cond.L.Unlock()
wg.Done()
}()
wg.Wait()
fmt.Println("done twice")
}
https://play.golang.org/p/uNmyofH2hs
I can use a channel and close the channel as a signal. But this means that in case there is a need for "reset" (where in the "signal" can be triggered again), the channel would have to be created again.
Is there a better (concise) way to achieve this?
Related
https://go.dev/play/p/YVYRWSgcp4u
I'm seeing this code in "Concurrency in Go Tools and Techniques for Developers", where it's mentioned about the usage for broadcast, the context is to use broadcast to wake three goroutings.
package main
import (
"fmt"
"sync"
"time"
)
func main() {
type Button struct {
Clicked *sync.Cond
}
button := Button{Clicked: sync.NewCond(&sync.Mutex{})}
subscribe := func(c *sync.Cond, fn func()) {
var goroutineRunning sync.WaitGroup
goroutineRunning.Add(1)
go func() {
goroutineRunning.Done() // <---- why here?
//fmt.Println("wg already done")
c.L.Lock()
defer c.L.Unlock()
c.Wait()
fn()
//goroutineRunning.Done(), if put here will result in deadlock, why?
}()
goroutineRunning.Wait()
}
var clickRegistered sync.WaitGroup
clickRegistered.Add(3)
subscribe(button.Clicked, func() {
fmt.Println("Maximizing window.")
clickRegistered.Done()
})
subscribe(button.Clicked, func() {
fmt.Println("Displaying annoying dialog box!")
clickRegistered.Done()
})
subscribe(button.Clicked, func() {
fmt.Println("Mouse clicked.")
clickRegistered.Done()
})
time.Sleep(time.Second * 3)
button.Clicked.Broadcast()
clickRegistered.Wait()
}
I'm trying to understand the subscribe part
subscribe := func(c *sync.Cond, fn func()) {
var goroutineRunning sync.WaitGroup
goroutineRunning.Add(1)
go func() {
goroutineRunning.Done()
//fmt.Println("wg already done")
c.L.Lock()
defer c.L.Unlock()
c.Wait()
fn()
//goroutineRunning.Done()
//fmt.Println("fn executed")
}()
goroutineRunning.Wait()
}
The author says:
Here we define a convenience function that will allow us to register functions to handle signals from a condition. Each handler is run on its own goroutine, and subscribe will not exit until that goroutine is confirmed to be running.
My understanding is that we should defer goroutingRunning.Done() inside gorouting so that the subsequent code (including waiting for Cond and fn() call, will have opportunities
to run), but in this case it seems the goroutingRunning.Done() has to be in the beginning of gorouting, otherwise it would result in deadlock error, but why?
------UPDATE------
We could actually get rid of the waitgroup in subscribe function this way:
subscribe := func(c *sync.Cond, fn func(), wg *sync.WaitGroup) {
c.L.Lock()
defer c.L.Unlock()
c.Wait()
fn()
wg.Done()
}
var ClickRegistered sync.WaitGroup
ClickRegistered.Add(3)
go subscribe(button.Clicked, func() {
fmt.Println("do 1")
}, &ClickRegistered)
go subscribe(button.Clicked, func() {
fmt.Println("do 2")
}, &ClickRegistered)
go subscribe(button.Clicked, func() {
fmt.Println("do 3")
}, &ClickRegistered)
time.Sleep(time.Millisecond * 50)
fmt.Println("some process in main go routine finished")
button.Clicked.Broadcast()
ClickRegistered.Wait()
This is a mechanism that ensures that when subscribe returns, the goroutine has started running. When goroutine starts, it calls Done to signal to the waiting caller that the goroutine is running. Without this mechanism, it is possible that when subscribe returns the goroutine has not been scheduled yet.
A deferred Done will not work, because that will only run once the goroutine returns, and that will not happen until the condition variable is signaled.
The scheme does not guarantee that the new goroutine locks the mutex. It is debatable whether this schema is really necessary.
I just don't understand why ctx.Done() is not being executed even though I am passing context and calling the cancel from the main? What am I doing wrong here?
var c = make(chan string)
func A(ctx context.Context) {
for {
select {
case <-ctx.Done():
fmt.Println("killing AAAA")
return // kill A at least
default:
fmt.Println("in A1.. .. again")
c <- "yesss"
}
}
}
//func B(ctx context.Context) {
func main() {
ctx, cancel := context.WithCancel(context.Background())
fmt.Println("BEFORE Number of active goroutines ", runtime.NumGoroutine())
go A(ctx)
time.Sleep(2 * time.Second)
valueReceived := <-c
cancel()
fmt.Println("AFTER Number of active goroutines ", runtime.NumGoroutine())
}
The goroutine executes the default branch twice and blocks on send to c. The <-ctx.Done() case is not executed because the goroutine is stuck in the default branch.
Fix the problem by sending from the select case instead of the branch statements.
func A(ctx context.Context) {
for {
select {
case <-ctx.Done():
fmt.Println("killing AAAA")
return // kill A at least
case c <- "yesss":
fmt.Println("in A1.. .. again")
}
}
}
You may not see the the killing AAAA with this change alone because the program can exit before the goroutine runs to completion.
Wait for the goroutine to complete to see the message:
var wg sync.WaitGroup
func A(ctx context.Context) {
defer wg.Done()
for {
select {
case <-ctx.Done():
fmt.Println("killing AAAA")
return // kill A at least
case c <- "yesss":
fmt.Println("in A1.. .. again")
}
}
}
...
wg.Add(1)
go A(ctx)
time.Sleep(2 * time.Second)
valueReceived := <-c
cancel()
wg.Wait()
Run it on the Go playground.
I want to know if there are any guarantees regarding the return order upon Context cancellation in golang.
I want to create a context with cancellation and once all the listeners are done with processing catching and reacting to "<-ctx.Done()" from this context, I want to call os.Exit safely.
A concrete example to explain the idea of what I want is following. I want to catch a signal, trigger all cancellations, and then call os.Exit().
I create a context and listen for a signal:
ctx, cancel := context.WithCancel(context.Background())
go func() {
c := make(chan os.Signal)
signal.Notify(c, os.Interrupt)
defer signal.Stop(c)
select {
case <-c:
cancel()
}
}()
In other places I "sign up" for this request several times:
res := NewRes()
go func() {
<-ctx.Done():
res.Close()
}()
But then I want to call os.Exit at the point when all the listeners are done.
For that I plan to create either parent or child context like this:
parent, pCancel := context.WithCancel(context.Background())
child, _ := context.WithCancel(parent)
go func() {
c := make(chan os.Signal)
signal.Notify(c, os.Interrupt)
defer signal.Stop(c)
select {
case <-c:
pCancel()
case <-child.Done():
os.Exit(0)
}
}()
Unfortunately, I did not find the documentation describing the order how context are canceled, so I cannot come up with the correct solution for now.
You have to wait all routines before exiting. Calling pCancel() doesn't mean everything will stop. I recommend to do in routine all jobs, but on the main thread to wait for os.Interrupt signal.
Check example below
package main
import (
"context"
"fmt"
"os"
"os/signal"
"sync"
"time"
)
func main() {
parent, pCancel := context.WithCancel(context.Background())
child, _ := context.WithCancel(parent)
wg := &sync.WaitGroup{}
for i := 0; i < 10; i++ {
go work(wg, child)
}
c := make(chan os.Signal)
signal.Notify(c, os.Interrupt)
defer signal.Stop(c)
select {
case <-c:
pCancel()
fmt.Println("Waiting everyone to finish...")
wg.Wait()
fmt.Println("Exiting")
os.Exit(0)
}
}
func work(wg *sync.WaitGroup, ctx context.Context) {
done := false
wg.Add(1)
for !done {
fmt.Println("Doing something...")
time.Sleep(time.Second)
select {
case <-ctx.Done():
fmt.Println("Done")
done = true
default:
}
}
wg.Done()
}
Although, It's recommended to use principle "Share Memory By Communicating".
Here is another example without using WaitGroup.
package main
import (
"context"
"fmt"
"os"
"os/signal"
"time"
)
func main() {
parent, pCancel := context.WithCancel(context.Background())
child, _ := context.WithCancel(parent)
done := make(chan struct{})
jobsCount := 10
for i := 0; i < jobsCount; i++ {
go work(child, done)
}
c := make(chan os.Signal)
signal.Notify(c, os.Interrupt)
defer signal.Stop(c)
select {
case <-c:
pCancel()
fmt.Println("Waiting everyone to finish...")
for i := 0; i < jobsCount; i++ {
<-done
}
fmt.Println("Exiting")
os.Exit(0)
}
}
func work(ctx context.Context, doneChan chan struct{}) {
done := false
for !done {
fmt.Println("Doing something...")
time.Sleep(time.Second)
select {
case <-ctx.Done():
fmt.Println("Done")
done = true
default:
}
}
doneChan <- struct{}{}
}
I need to stop an HTTP server on demand besides calling other functions as well when receiving the "quit" signal in no specific order.
In my try to implement something like the observer pattern, I found "handy" to create a channel (quit := make(chan struct{}), let's say the "subject" and then on each of the goroutines "observers" listen on that channel <-quit waiting until a change for then to continue.
The way I trigger all the functions at once is by closing the channel close(quit) not by writing into it, I have tried this and so far working, but wondering if there are some cons with this approach or if there are better/idiomatic ways of implementing similar behavior/pattern.
package main
import (
"log"
"net/http"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup
srv := &http.Server{Addr: ":8080"}
wg.Add(1)
go func() {
log.Println(srv.ListenAndServe())
wg.Done()
}()
quit := make(chan struct{})
go func() {
<-quit
if err := srv.Close(); err != nil {
log.Printf("HTTP server Shutdown: %v", err)
}
}()
wg.Add(1)
go func() {
<-quit
log.Println("just waiting 1")
wg.Done()
}()
wg.Add(1)
go func() {
<-quit
log.Println("just waiting 2")
wg.Done()
}()
<-time.After(2 * time.Second)
close(quit)
wg.Wait()
}
https://play.golang.org/p/uIfMJfN6xQy
I would say your way is good enough but lacks some elegance.
You could implement required behavior using sync.Cond:
https://golang.org/pkg/sync/#Cond
How to correctly use sync.Cond?
In Go I can write such code for creating a gorouting that sleeps 5 sec.
func sleep(link chan interface{}){
time.Sleep(5 * time.Second)
fmt.Println("I've slept for 5 sec")
link <- struct {}{}
}
func main() {
var link = make(chan interface{})
go sleep(link)
time.Sleep(1 * time.Second)
// here I want to change the remaining sleeping time of `sleep` goroutine to 0.5 sec
<- link
}
What if in main function I change my mind and decide that the sleeper should sleep not 5 sec but 3. How can it done if goroutine already started to sleep (and sleeping, for example, for 1 sec)?
UPDATE
I mean is there something whereby I can manage that unique goroutine while it sleeps. Like giving commands to it about decreasing or increasing time of sleep:
func main() {
// ...
time.Sleep(1)
something.ManageRemainingTime(10)
time.Sleep(5)
something.ManageRemainingTime(100)
time.Sleep(8)
something.ManageRemainingTime(0.5)
// ...
}
If you just need a way to "wakeup" a sleeping goroutine, you could use sync.Once to ensure your function only gets called once, and then return a channel so you can set a sooner "trigger time", so something this:
func sleep(callback func(), seconds int) chan int {
once := sync.Once{}
wakeup := make(chan int)
go func() {
for sleep := range wakeup {
go func() {
time.Sleep(time.Duration(sleep) * time.Second)
once.Do(callback)
}()
}
}()
wakeup <- seconds
return wakeup
}
func main() {
wg := sync.WaitGroup{}
wg.Add(1)
t := time.Now()
wakeup := sleep(func() {
fmt.Println("Hello, playground")
wg.Done()
}, 5)
wakeup <- 2
wg.Wait()
fmt.Println(time.Since(t))
}
https://play.golang.org/p/BRNtaBPKpLW
One method is to use a context object. Specifically one created with WithTimeout.
The context object provides a way of sending a "cancel" signal to a worker. In this case your worker is not really doing anything, but still fits the paradigm.
Example would be:
package main
import (
"context"
"fmt"
"sync"
"time"
)
func sleep(ctx context.Context, wg *sync.WaitGroup) {
sctx, _ := context.WithTimeout(ctx, 5*time.Second)
tStart := time.Now()
<-sctx.Done() // will sit here until the timeout or cancelled
tStop := time.Now()
fmt.Printf("Slept for %s\n", tStop.Sub(tStart))
wg.Done()
}
func main() {
ctx, cancelFunc := context.WithCancel(context.Background())
wg := sync.WaitGroup{}
wg.Add(1)
go sleep(ctx, &wg)
if true {
time.Sleep(3 * time.Second)
cancelFunc()
}
wg.Wait()
}
https://play.golang.org/p/2Krx4PsxFKL
Change the if true { to if false { and you can see the Slept for ... change.
One method is to use a timer. You can call Stop() on a timer, but this stops it without waking up whatever is waiting on it. So you then use Reset() to set it to a new value, specifically 0, which triggers it immediately.
Example:
package main
import (
"fmt"
"sync"
"time"
)
func sleep(wg *sync.WaitGroup) *time.Timer {
timer := time.NewTimer(5 * time.Second)
go func() {
tStart := time.Now()
<-timer.C
tStop := time.Now()
fmt.Printf("Slept for %s\n", tStop.Sub(tStart))
wg.Done()
}()
return timer
}
func main() {
wg := sync.WaitGroup{}
wg.Add(1)
timer := sleep(&wg)
if true {
time.Sleep(3 * time.Second)
if timer.Stop() {
timer.Reset(0)
}
}
wg.Wait()
}
https://play.golang.org/p/b3I65kAujR4
Change the if true { to if false { and you can see the Slept for ... change.