I'm spawning some goroutines and want to hand them a channel to send back errors. In the parent goroutine, I select the first error and return that, or the wg.Done() condition, which is synchronized with closing a done channel.
The closing of errc is deferred to avoid a goroutine leak; but it causes a race condition.
package main
import (
"log"
"sync"
"time"
)
func f(ch chan<- bool, wg *sync.WaitGroup) {
defer wg.Done()
time.Sleep(1 * time.Second)
log.Println("f sending a value")
ch <- true
log.Println("f sent a value")
}
func g(ch chan<- bool, wg *sync.WaitGroup) {
defer wg.Done()
time.Sleep(2 * time.Second)
log.Println("g sending a value")
ch <- true
log.Println("g sent a value")
}
func main() {
var wg sync.WaitGroup
ch := make(chan bool)
bufc := make(chan bool, 2)
defer func() {
log.Println("Closing bufc")
close(bufc)
log.Println("Closed bufc")
time.Sleep(5 * time.Second)
}()
wg.Add(2)
go f(bufc, &wg)
go g(bufc, &wg)
go func() {
wg.Wait()
close(ch)
}()
select {
case done, ok := <-bufc:
log.Printf("bufc closed: %v %v", done, ok)
case <-ch:
log.Println("ch was closed")
}
}
Result:
❗ ~/c/scrap
(i) go run test.go
2018/05/01 20:28:03 f sending a value
2018/05/01 20:28:03 f sent a value
2018/05/01 20:28:03 bufc closed: true true
2018/05/01 20:28:03 Closing bufc
2018/05/01 20:28:03 Closed bufc
2018/05/01 20:28:04 g sending a value
panic: send on closed channel
goroutine 19 [running]:
main.g(0xc42009c000, 0xc42008a010)
/Users/yangmillstheory/code/scrap/test.go:23 +0xb2
created by main.main
/Users/yangmillstheory/code/scrap/test.go:42 +0x11e
exit status 2
Is there any way to do proper cleanup of the errc channel without causing a panic? Do I even need to close errc? Given that it's buffered, senders on that channel wouldn't block, so I would guess the answer is no?
Your error is clear enough--the channel bufc (which I assume you refer to as errc) is closed before g can send the value to it because the select statement receives only once from bufc and it's closed by defer. Instead of deferring the closing of bufc you'd have to make some synchronization, possibly using a sync.WaitGroup to make sure all the values are sent before closing it, for example by just moving close(bufc) to after wg.Wait():
go func() {
wg.Wait()
close(ch)
close(bufc)
}()
In your case since bufc is buffered you don't have to close it because it's not blocking on the receiving end, but once you have more than two goroutines sending you'll still need to close it to signal properly.
I ended up with the following implementation, which drains the buffered channel bufc and works correctly in all cases.
var (
err error
wg sync.WaitGroup
)
ch := make(chan bool)
bufc := make(chan error, 2)
wg.Add(2)
go f(bufc, &wg)
go g(bufc, &wg)
go func() {
wg.Wait()
close(ch)
close(bufc)
}()
<-ch
log.Println("Goroutines are done")
for err = range bufc {
log.Printf("Got an error: %v", err)
}
log.Println("Returning.")
return err
Related
Consider the following example:
package main
import (
"fmt"
"time"
)
func main() {
ticker := time.NewTicker(2 * time.Second)
done := make(chan bool)
defer func() {
fmt.Println("exiting..")
done <- true
close(done)
}()
go func(ticker *time.Ticker, done chan bool) {
for {
select {
case <-done:
fmt.Println("DONE!")
break
case <-ticker.C:
fmt.Println("TICK!...")
}
}
}(ticker, done)
time.Sleep(7 * time.Second)
}
The goroutine waiting to receive from done never receives as (I am guessing) the main goroutine finished beforehand. However if I change the sleep time of the main goroutine to 8 seconds it receives a message; Why is there this dependency on the sleep time?
Is it because there is that second difference that keeps the goroutine alive and the there isn't enough time to kill it?
How would I than kill the goroutine gracefully?
You need to ensure that main does not return before the goroutine finishes.
The simplest way to do this is using a WaitGroup:
var wg sync.WaitGroup
defer wg.Wait()
wg.Add(1)
go func() {
defer wg.Done()
// …
Note that defers run in reverse order, so you must put defer wg.Wait() before defer close(done), otherwise it will deadlock.
I am having 2 go-routines reading from a single channel. After 4 seconds, I cancel the context and terminate the select loop. Before terminating the loop I call close on the channel, since there are 2 go-routines the close gets called twice and causes a panic because one of the go-routines would have already closed the channel. Currently I am using a recover to recover from the panic, is there a better way of doing this?
package main
import (
"context"
"fmt"
"sync"
"time"
)
func numberGen(ctx context.Context, numChan chan int) {
num := 0
doneCh := ctx.Done()
defer func() {
if r := recover(); r != nil {
fmt.Println("recovered from ", r)
}
}()
for {
select {
case <-doneCh:
fmt.Println("done generating...")
close(numChan)
return
default:
num++
numChan <- num
}
}
}
func main() {
ctx, cancelFn := context.WithCancel(context.Background())
numChan := make(chan int)
var wg sync.WaitGroup
wg.Add(2)
go numberGen(ctx, numChan)
go numberGen(ctx, numChan)
go func(cfn context.CancelFunc) {
time.Sleep(10 * time.Millisecond)
cfn()
}(cancelFn)
for n := range numChan {
fmt.Println("received value ", n)
}
time.Sleep(2 * time.Second)
}
Close the channel after the goroutines are done sending values.
var wg sync.WaitGroup
wg.Add(2)
go numberGen(ctx, numChan, &wg)
go numberGen(ctx, numChan, &wg)
go func() {
wg.Wait()
close(numChan)
}()
Update numberGen to call Done() on the wait group. Also, remove the call to close.
func numberGen(ctx context.Context, numChan chan int, wg *sync.WaitGroup) {
defer wg.Done()
...
func main() {
c := make(chan os.Signal, 1)
signal.Notify(c)
ticker := time.NewTicker(time.Second)
stop := make(chan bool)
go func() {
defer func() { stop <- true }()
for {
select {
case <-ticker.C:
fmt.Println("Tick")
case <-stop:
fmt.Println("Goroutine closing")
return
}
}
}()
<-c
ticker.Stop()
stop <- true
<-stop
fmt.Println("Application stopped")
}
No matter how many times I run the code above, I got the same result. That is, "Goroutine closing" is always printed before "Application stopped" after I press Ctrl+C.
I think, theoretically, there is a chance that "Goroutine closing" won't be printed at all. Am I right? Unfortunately, I never get this theoretical result.
BTW: I know reading and writing a channel in one routine should be avoided. Ignore that temporarily.
In your case, Goroutine closing will always be executed and it will always be printed before Application stopped, because your stop channel is not buffered. This means that the sending will block until the result is received.
In your code, the stop <- true in your main will block until the goroutine has received the value, causing the channel to be empty again. Then the <-stop in your main will block until another value is sent to the channel, which happens when your goroutine returns after printing Goroutine closing.
If you would initialize your channel in a buffered fashion
stop := make(chan bool, 1)
then Goroutine closing might not be executed. To see this, you can add a time.Sleep right after printing Tick, as this makes this case more likely (it will occur everytime you press Ctrl+C during the sleep).
Using a sync.WaitGroup to wait for goroutines to finish is a good alternative, especially if you have to wait for more than one goroutine. You can also use a context.Context to stop goroutines. Reworking your code to use these two methods could look something like this:
func main() {
c := make(chan os.Signal, 1)
signal.Notify(c)
ticker := time.NewTicker(time.Second)
ctx, cancel := context.WithCancel(context.Background())
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer func() { wg.Done() }()
for {
select {
case <-ctx.Done():
fmt.Println("Goroutine closing")
return
case <-ticker.C:
fmt.Println("Tick")
time.Sleep(time.Second)
}
}
}()
<-c
ticker.Stop()
cancel()
wg.Wait()
fmt.Println("Application stopped")
}
I have 3 channels that get data from together . My code run for first time perfect but when after that I send another data , there is error : send on closed channel.In func FillNotCheckedDeliveryCh that line I closed channel .If I do not close it app stopped and not continue.
func main() {
receiveBulkIdsCh := make(chan int64, 100)
NotCheckedDeliveryCh := make(chan CommonType.BasicRequestParameters, 100)
ResultCh := make(chan CommonType.MessageStateResult, 100)
var wg sync.WaitGroup
wg.Add(4)
/* Every BulkIds that receive from queue , save in receiveBulkIdsChan channel*/
go func() {
defer wg.Done()
PopQueue(receiveBulkIdsCh)
}()
go func() {
defer wg.Done()
for BulkId := range receiveBulkIdsCh {
FillNotCheckedDeliveryCh(data, NotCheckedDeliveryCh)
}
}()
go func() {
defer wg.Done()
for item := range NotCheckedDeliveryCh {
for msgStatus := range DoFuncGetMessageState(item) {
ResultCh <- msgStatus
}
close(ResultCh)
}
}()
go func() {
defer wg.Done()
for Result := range ResultCh {
ReadResultCh(Result)
}
}()
wg.Wait()
}
func FillNotCheckedDeliveryCh(data IntegrateRowsFields, ch chan<- CommonType.BasicRequestParameters) {
ch <- PackerForNotFinishedCh(data)
defer close(ch)
}
can help what's wrong ?
When you work with channels in Go always the sender should close the channel. Because that signals that no more data will be send over the channel.
In your code the receiver is closing it. Just leave the channel here opened and remove the close(ch) there.
func FillNotCheckedDeliveryCh(data IntegrateRowsFields, ch chan<- CommonType.BasicRequestParameters) {
ch <- PackerForNotFinishedCh(data)
defer close(ch)
}
If something is blocking at that part you could use select:
func FillNotCheckedDeliveryCh(data IntegrateRowsFields, ch chan<- CommonType.BasicRequestParameters) {
select {
case ch <- PackerForNotFinishedCh(data):
default:
}
}
Closing the channels on the receiver side, and then trying to send on the same channel gives you the error. Better is to close the channel outside the go routine only after all values are sent on the channels.
Close the channel after waiting for all go routines to be finished.
wg.Wait()
close(ch)
Always close the channel when there are no more values to be sent on the channel.
Receivers can test whether a channel has been closed by assigning a
second parameter to the receive expression: after
v, ok := <-ch
In your case too you can check on the receiver side by using ok if there are more values in the channel or if it is closed.
Channels aren't like files; you don't usually need to close them.
Closing is only necessary when the receiver must be told there are no
more values coming, such as to terminate a range loop.
Go playground example for generating the error when not closing a channel and range over it.
Playground example for closing the channels when all values are send.
I have some issues with the following code:
package main
import (
"fmt"
"sync"
)
// This program should go to 11, but sometimes it only prints 1 to 10.
func main() {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(2)
go Print(ch, wg) //
go func(){
for i := 1; i <= 11; i++ {
ch <- i
}
close(ch)
defer wg.Done()
}()
wg.Wait() //deadlock here
}
// Print prints all numbers sent on the channel.
// The function returns when the channel is closed.
func Print(ch <-chan int, wg sync.WaitGroup) {
for n := range ch { // reads from channel until it's closed
fmt.Println(n)
}
defer wg.Done()
}
I get a deadlock at the specified place. I have tried setting wg.Add(1) instead of 2 and it solves my problem. My belief is that I'm not successfully sending the channel as an argument to the Printer function. Is there a way to do that? Otherwise, a solution to my problem is replacing the go Print(ch, wg)line with:
go func() {
Print(ch)
defer wg.Done()
}
and changing the Printer function to:
func Print(ch <-chan int) {
for n := range ch { // reads from channel until it's closed
fmt.Println(n)
}
}
What is the best solution?
Well, first your actual error is that you're giving the Print method a copy of the sync.WaitGroup, so it doesn't call the Done() method on the one you're Wait()ing on.
Try this instead:
package main
import (
"fmt"
"sync"
)
func main() {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(2)
go Print(ch, &wg)
go func() {
for i := 1; i <= 11; i++ {
ch <- i
}
close(ch)
defer wg.Done()
}()
wg.Wait() //deadlock here
}
func Print(ch <-chan int, wg *sync.WaitGroup) {
for n := range ch { // reads from channel until it's closed
fmt.Println(n)
}
defer wg.Done()
}
Now, changing your Print method to remove the WaitGroup of it is a generally good idea: the method doesn't need to know something is waiting for it to finish its job.
I agree with #Elwinar's solution, that the main problem in your code caused by passing a copy of your Waitgroup to the Print function.
This means the wg.Done() is operated on a copy of wg you defined in the main. Therefore, wg in the main could not get decreased, and thus a deadlock happens when you wg.Wait() in main.
Since you are also asking about the best practice, I could give you some suggestions of my own:
Don't remove defer wg.Done() in Print. Since your goroutine in main is a sender, and print is a receiver, removing wg.Done() in receiver routine will cause an unfinished receiver. This is because only your sender is synced with your main, so after your sender is done, your main is done, but it's possible that the receiver is still working. My point is: don't leave some dangling goroutines around after your main routine is finished. Close them or wait for them.
Remember to do panic recovery everywhere, especially anonymous goroutine. I have seen a lot of golang programmers forgetting to put panic recovery in goroutines, even if they remember to put recover in normal functions. It's critical when you want your code to behave correctly or at least gracefully when something unexpected happened.
Use defer before every critical calls, like sync related calls, at the beginning since you don't know where the code could break. Let's say you removed defer before wg.Done(), and a panic occurrs in your anonymous goroutine in your example. If you don't have panic recover, it will panic. But what happens if you have a panic recover? Everything's fine now? No. You will get deadlock at wg.Wait() since your wg.Done() gets skipped because of panic! However, by using defer, this wg.Done() will be executed at the end, even if panic happened. Also, defer before close is important too, since its result also affects the communication.
So here is the code modified according to the points I mentioned above:
package main
import (
"fmt"
"sync"
)
func main() {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(2)
go Print(ch, &wg)
go func() {
defer func() {
if r := recover(); r != nil {
println("panic:" + r.(string))
}
}()
defer func() {
wg.Done()
}()
for i := 1; i <= 11; i++ {
ch <- i
if i == 7 {
panic("ahaha")
}
}
println("sender done")
close(ch)
}()
wg.Wait()
}
func Print(ch <-chan int, wg *sync.WaitGroup) {
defer func() {
if r := recover(); r != nil {
println("panic:" + r.(string))
}
}()
defer wg.Done()
for n := range ch {
fmt.Println(n)
}
println("print done")
}
Hope it helps :)