This question already has answers here:
golang sync.WaitGroup never completes
(3 answers)
Best way of using sync.WaitGroup with external function
(2 answers)
GO - Pointer or Variable in WaitGroups reference
(1 answer)
Golang WaitGroup.Done() being skipped
(1 answer)
Closed 11 months ago.
Here's a simple example of what I mean
package main
import (
"sync"
"testing"
"time"
)
func TestWaitGroup(t *testing.T) {
var wg sync.WaitGroup
quitSig := make(chan struct{})
go func(wg sync.WaitGroup, quitChan, chan struct{}) {
defer func() {
t.Log("Done...")
wg.Done()
t.Log("Done!")
}()
t.Log("waiting for quit channel signal...")
<-quitChan
t.Log("signal received")
}(wg, quitSig)
time.Sleep(5*time.Second)
t.Log("Done sleeping")
close(quitSig)
t.Log("closed quit signal channel")
wg.Wait()
t.Log("goroutine shutdown")
}
When I run this, I get the following
=== RUN TestWaitGroup
main.go:18: waiting for quit channel signal...
main.go:23: Done sleeping
main.go:25: closed quit signal channel
main.go:20: signal received
main.go:14: Done...
main.go:16: Done!
Where it just hangs until it timesout. If you just do defer wg.Done() the same behaviour is observed. I'm running go1.18. Is this a bug or am I using not using WaitGroups properly in this context?
Two issues:
don't copy sync.WaitGroup: from the docs:
A WaitGroup must not be copied after first use.
you need a wg.Add(1) before launching your work - to pair with the wg.Done()
wg.Add(1) // <- add this
go func (wg *sync.WaitGroup ...) { // <- pointer
}(&wg, quitSig) // <- pointer to avoid WaitGroup copy
https://go.dev/play/p/UmeI3TdGvhg
You are passing a copy of the waitgroup, so the goroutine does not affect the waitgroup declared in the outer scope. Fix it by:
go func(wg *sync.WaitGroup, quitChan, chan struct{}) {
...
}(&wg, quitSig)
Related
This question already has answers here:
Buffered/Unbuffered channel
(3 answers)
fatal error: all goroutines are asleep - deadlock! when channel is not buffered
(1 answer)
Why is the order of channels receiving causing/resolving a deadlock in Golang?
(3 answers)
In the Go select construct, can I have send and receive to unbuffered channel in two case statements?
(2 answers)
Closed 7 days ago.
package main
import (
"fmt"
"sync"
)
func main() {
ch := make(chan struct{})
wg := &sync.WaitGroup{}
wg.Add(1)
go func() {
defer wg.Done()
ch <- struct{}{}
}()
wg.Wait()
<-ch
fmt.Println("finished")
}
result
fnished will not be printed
package main
import (
"fmt"
"sync"
)
func main() {
ch := make(chan struct{}, 1)
wg := &sync.WaitGroup{}
wg.Add(1)
go func() {
defer wg.Done()
ch <- struct{}{}
}()
wg.Wait()
<-ch
fmt.Println("finished")
}
result
finished can be printed immediately
The only difference between the two code is that the first code use no buffer channel, but the second code use channel with a buffer.
In the first code, ch <- struct{}{} can not be run. message can not be sent to the channel.
Why does that happen? Is there any official explanation or material?
In the first example wg.Done() can only be called after a value on the channel has been sent. But since the channel is unbuffered, the send can only happen if there's a ready receiver. The main goroutine would be the receiver, but it only receives after wg.Wait(). But wg.Wait() blocks until wg.Done() is called, so this will never happen, it's a deadlock.
This is in Spec: Send statements:
Communication blocks until the send can proceed. A send on an unbuffered channel can proceed if a receiver is ready.
When using non-buffered channels, sending or receiving commands from the channel are blocking.
This means with non-buffered channel, your go routine that sends the message is not done, hence it won't call wg.Done() which will keep the wg.Wait() method in blocking state.
If you run your code on the play server https://go.dev/play/p/oDpyL6-dARP, it will panic with an informative message: fatal error: all goroutines are asleep - deadlock!
When your using buffered channels, the go routine runs because the channel has a buffer, then wg.Done() is called, which release wg.Wait() afterwards.
This question already has an answer here:
Is there buffered lock pattern?
(1 answer)
Closed 2 years ago.
I would like to place concurrency limit for a piece of code in a goroutine.
I looked at the mutex, channels but could not achieve this.
m.Lock()
// make request to server
log.Info("Calling server for " + name)
resp, err := netClient.Do(req)
m.Unlock()
log.Info("outside lock for " + name)
Using the above code snippet I could restrict the server call one after the other.
Instead is there a way I make this call to be executed by 4 goroutines. Another goroutine can execute it only after any of those 4 responds back.
So at a time I need 4 goroutines inside the block.
Thanks in advance.
The answer here is a semaphore.
Here is a snippet:
package main
import (
"fmt"
"sync"
)
func main() {
numOfGoroutines := 4
sem := make(chan struct{}, numOfGoroutines)
wg := sync.WaitGroup{}
for i := 0; i < 100; i++ {
sem <- struct{}{}
wg.Add(1)
go func(i int) {
defer func() { <-sem }()
defer wg.Done()
fmt.Println(i)
}(i)
}
wg.Wait()
fmt.Println("done")
}
This snippet has an important detail that is missing in the link that Peter mentioned. The snippet waits for all goroutines to complete before moving forward.
Also, Go implements a weighted semaphore.
This question already has answers here:
No output from goroutine
(3 answers)
Closed 3 years ago.
In this piece of code, a goroutine is created and tries to read from a buffered channel.
But the buffer is empty and should block the receiver, but it didn't. It will run but output nothin.
package main
import "fmt"
func fun(c chan int) {
fmt.Println(<-c)
}
func main() {
ch := make(chan int, 1)
//ch <- 1
go fun(ch)
}
First, the buffer is not empty, because you send a value on it:
ch <- 1
This send operation will not block because the channel is buffered.
Then you launch a goroutine, and the main() function ends. And with that, your program ends as well, it does not wait for other non-main goroutines to complete. For details, see No output from goroutine in Go.
Note that the same thing happens even if you comment out the send operation in main(): you launch a goroutine and then main() ends along with your app: it does not wait for the other goroutine.
To wait for other goroutines, often a sync.WaitGroup is used like this:
var wg = &sync.WaitGroup{}
func fun(c chan int) {
defer wg.Done()
fmt.Println(<-c)
}
func main() {
ch := make(chan int, 1)
ch <- 1
wg.Add(1)
go fun(ch)
wg.Wait()
}
Then output will be (try it on the Go Playground):
1
For details, see Prevent the main() function from terminating before goroutines finish in Golang
This question already has answers here:
how to listen to N channels? (dynamic select statement)
(7 answers)
Closed 3 years ago.
Suppose I have a slice of go receiving channels. Is there a way I can listen to all of them at once? For example:
channels := make([]<-chan int, 0, N)
// fill the slice with channels
for _, channel := range channels {
<-channel
}
Is the closest I can get to doing that. However, this implementation is dependent on the order of the elements of the slice.
For clarity, I don't need to know the values of the go channel. I just need to know they all finished.
You can use sync package to create a waitgroup.
Start a goroutine for each channel after adding to waitGroup.
Finally when channel is done it decrements waitGroup. The caller just waits on waitgroup. Here is the playground link.
package main
import (
"fmt"
"sync"
"time"
)
func main() {
wg := sync.WaitGroup{}
numChans := 10
chans := make([]chan interface{}, numChans)
for i:=0;i<numChans;i++ {
chans[i] = make(chan interface{})
wg.Add(1)
go func(c chan interface{}) {
defer wg.Done()
<-c
}(chans[i])
}
//simulate closing channels
go func(){
time.Sleep(time.Second)
fmt.Println("closing")
for _, ch := range chans{
close(ch)
}
}()
fmt.Println("waiting")
wg.Wait()
}
This question already has answers here:
How to wait for all goroutines to finish without using time.Sleep?
(5 answers)
Closed 7 years ago.
let's say I want to start a group of goroutines and than wait for all of them to finish (e.g. return).
I can think of some channel-based solution (e.g. create a channel and listen to it counting received messages which shall be sent by each of goroutines and quiting after receive corresponding number of msgs) but maybe there's a more elegant/efficient solution because this case seems very common.
Yes; you want a *sync.WaitGroup, which you can use by calling waitGroup.Add(1) before starting each task, waitGroup.Done() when it finishes, and waitGroup.Wait() once you have started everything and want to wait for it all to finish, like this:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
wg := new(sync.WaitGroup)
for i := 1; i < 3; i++ {
wg.Add(1)
go func(i int) { // ensures each run gets distinct i
fmt.Println("Sleeping", i, "seconds")
time.Sleep(time.Duration(i) * time.Second)
fmt.Println("Slept", i, "seconds")
wg.Done()
}(i)
}
wg.Wait()
fmt.Println("All done")
}