This question already has answers here:
How to wait for all goroutines to finish without using time.Sleep?
(5 answers)
Closed 7 years ago.
let's say I want to start a group of goroutines and than wait for all of them to finish (e.g. return).
I can think of some channel-based solution (e.g. create a channel and listen to it counting received messages which shall be sent by each of goroutines and quiting after receive corresponding number of msgs) but maybe there's a more elegant/efficient solution because this case seems very common.
Yes; you want a *sync.WaitGroup, which you can use by calling waitGroup.Add(1) before starting each task, waitGroup.Done() when it finishes, and waitGroup.Wait() once you have started everything and want to wait for it all to finish, like this:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
wg := new(sync.WaitGroup)
for i := 1; i < 3; i++ {
wg.Add(1)
go func(i int) { // ensures each run gets distinct i
fmt.Println("Sleeping", i, "seconds")
time.Sleep(time.Duration(i) * time.Second)
fmt.Println("Slept", i, "seconds")
wg.Done()
}(i)
}
wg.Wait()
fmt.Println("All done")
}
Related
This question already has answers here:
golang sync.WaitGroup never completes
(3 answers)
Best way of using sync.WaitGroup with external function
(2 answers)
GO - Pointer or Variable in WaitGroups reference
(1 answer)
Golang WaitGroup.Done() being skipped
(1 answer)
Closed 11 months ago.
Here's a simple example of what I mean
package main
import (
"sync"
"testing"
"time"
)
func TestWaitGroup(t *testing.T) {
var wg sync.WaitGroup
quitSig := make(chan struct{})
go func(wg sync.WaitGroup, quitChan, chan struct{}) {
defer func() {
t.Log("Done...")
wg.Done()
t.Log("Done!")
}()
t.Log("waiting for quit channel signal...")
<-quitChan
t.Log("signal received")
}(wg, quitSig)
time.Sleep(5*time.Second)
t.Log("Done sleeping")
close(quitSig)
t.Log("closed quit signal channel")
wg.Wait()
t.Log("goroutine shutdown")
}
When I run this, I get the following
=== RUN TestWaitGroup
main.go:18: waiting for quit channel signal...
main.go:23: Done sleeping
main.go:25: closed quit signal channel
main.go:20: signal received
main.go:14: Done...
main.go:16: Done!
Where it just hangs until it timesout. If you just do defer wg.Done() the same behaviour is observed. I'm running go1.18. Is this a bug or am I using not using WaitGroups properly in this context?
Two issues:
don't copy sync.WaitGroup: from the docs:
A WaitGroup must not be copied after first use.
you need a wg.Add(1) before launching your work - to pair with the wg.Done()
wg.Add(1) // <- add this
go func (wg *sync.WaitGroup ...) { // <- pointer
}(&wg, quitSig) // <- pointer to avoid WaitGroup copy
https://go.dev/play/p/UmeI3TdGvhg
You are passing a copy of the waitgroup, so the goroutine does not affect the waitgroup declared in the outer scope. Fix it by:
go func(wg *sync.WaitGroup, quitChan, chan struct{}) {
...
}(&wg, quitSig)
This question already has an answer here:
Is there buffered lock pattern?
(1 answer)
Closed 2 years ago.
I would like to place concurrency limit for a piece of code in a goroutine.
I looked at the mutex, channels but could not achieve this.
m.Lock()
// make request to server
log.Info("Calling server for " + name)
resp, err := netClient.Do(req)
m.Unlock()
log.Info("outside lock for " + name)
Using the above code snippet I could restrict the server call one after the other.
Instead is there a way I make this call to be executed by 4 goroutines. Another goroutine can execute it only after any of those 4 responds back.
So at a time I need 4 goroutines inside the block.
Thanks in advance.
The answer here is a semaphore.
Here is a snippet:
package main
import (
"fmt"
"sync"
)
func main() {
numOfGoroutines := 4
sem := make(chan struct{}, numOfGoroutines)
wg := sync.WaitGroup{}
for i := 0; i < 100; i++ {
sem <- struct{}{}
wg.Add(1)
go func(i int) {
defer func() { <-sem }()
defer wg.Done()
fmt.Println(i)
}(i)
}
wg.Wait()
fmt.Println("done")
}
This snippet has an important detail that is missing in the link that Peter mentioned. The snippet waits for all goroutines to complete before moving forward.
Also, Go implements a weighted semaphore.
This question already has answers here:
No output from goroutine
(3 answers)
Closed 3 years ago.
In this piece of code, a goroutine is created and tries to read from a buffered channel.
But the buffer is empty and should block the receiver, but it didn't. It will run but output nothin.
package main
import "fmt"
func fun(c chan int) {
fmt.Println(<-c)
}
func main() {
ch := make(chan int, 1)
//ch <- 1
go fun(ch)
}
First, the buffer is not empty, because you send a value on it:
ch <- 1
This send operation will not block because the channel is buffered.
Then you launch a goroutine, and the main() function ends. And with that, your program ends as well, it does not wait for other non-main goroutines to complete. For details, see No output from goroutine in Go.
Note that the same thing happens even if you comment out the send operation in main(): you launch a goroutine and then main() ends along with your app: it does not wait for the other goroutine.
To wait for other goroutines, often a sync.WaitGroup is used like this:
var wg = &sync.WaitGroup{}
func fun(c chan int) {
defer wg.Done()
fmt.Println(<-c)
}
func main() {
ch := make(chan int, 1)
ch <- 1
wg.Add(1)
go fun(ch)
wg.Wait()
}
Then output will be (try it on the Go Playground):
1
For details, see Prevent the main() function from terminating before goroutines finish in Golang
This question already has answers here:
how to listen to N channels? (dynamic select statement)
(7 answers)
Closed 3 years ago.
Suppose I have a slice of go receiving channels. Is there a way I can listen to all of them at once? For example:
channels := make([]<-chan int, 0, N)
// fill the slice with channels
for _, channel := range channels {
<-channel
}
Is the closest I can get to doing that. However, this implementation is dependent on the order of the elements of the slice.
For clarity, I don't need to know the values of the go channel. I just need to know they all finished.
You can use sync package to create a waitgroup.
Start a goroutine for each channel after adding to waitGroup.
Finally when channel is done it decrements waitGroup. The caller just waits on waitgroup. Here is the playground link.
package main
import (
"fmt"
"sync"
"time"
)
func main() {
wg := sync.WaitGroup{}
numChans := 10
chans := make([]chan interface{}, numChans)
for i:=0;i<numChans;i++ {
chans[i] = make(chan interface{})
wg.Add(1)
go func(c chan interface{}) {
defer wg.Done()
<-c
}(chans[i])
}
//simulate closing channels
go func(){
time.Sleep(time.Second)
fmt.Println("closing")
for _, ch := range chans{
close(ch)
}
}()
fmt.Println("waiting")
wg.Wait()
}
I wish to know what is the proper way of waiting for a go routine to finish before exiting the program. Reading some other answers it seems that a bool chan will do the trick, as in Playground link
func do_stuff(done chan bool) {
fmt.Println("Doing stuff")
done <- true
}
func main() {
fmt.Println("Main")
done := make(chan bool)
go do_stuff(done)
<-done
//<-done
}
I have two questions here:
why the <- done works at all?
what happens if I uncomment the last line? I have a deadlock error. Is this because the channel is empty and there is no other function sending values to it?
Listening to channel <- done, is a blocking operation, so your program won't continue until true or false is sent i.e. done <- true.
Your question can have a few different answers depending on the circumstance.
For instance, suppose you wanted to parallelize a series of function calls that take a long time.
I would use the sync package for this
package main
import (
"fmt"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func() {
longOp()
wg.Done()
}()
}
// will wait until wg.Done is called 10 times
// since we made wg.Add(1) call 10 times
wg.Wait()
}
func longOp() {
time.Sleep(time.Second * 2)
fmt.Println("long op done")
}
Why the <- done works at all?
It works because the runtime detects that you're writing something to the channel somewhere else.
what happens if I uncomment the last line?
The runtime is smart enough to know that there's nothing else being written and it deadlocks.
Bonus, if you're extremely limited on memory, you can use done := make(chan struct{}) and done <- struct{}{}, struct{} is guaranteed to use 0 memory.