How to make this golang for-select code work? - go

Question
How to make the below code print "QUIT" after 3 seconds?
Code
package main
import (
"fmt"
"time"
)
func main() {
quit := make(chan struct{})
tasks := make(chan struct{})
go func(){
time.Sleep(1 * time.Second)
tasks <- struct{}{}
}()
go func(){
time.Sleep(3 * time.Second)
quit <- struct{}{}
}()
for {
select {
case <-quit:
fmt.Println("QUIT")
return
case <-tasks:
fmt.Println("Doing")
// do some long time jobs more than 10 seconds
time.Sleep(10 * time.Second)
}
}
}
Observations
The above code prints "Doing". and sleep 10 seconds, then print "QUIT".
How to interrupt this sleep, let it receive quit channel after 3 seconds and print "QUIT"?
It seems select is blocked by case tasks, and it will not receive from the quit channel after 3 seconds.

To signal end of an asynchronous task it is a best practice to close the channel, this is rather important to prevent many missuse that leads to various deadlocks.
In your original code I would have written close(quite) rather than quit <- struct{}{}
Remember, read on a closed does not block and always return the zero value, that is the trick.
Anyways, appart from this, an elegant way to solve your problem is to use a combination of both context.Context and time.After.
time.After will help you block on a selectable task set.
context.Context is better suited to handle this kind of signals.
https://play.golang.org/p/ZVsZw3P-YHd
package main
import (
"context"
"log"
"time"
)
func main() {
log.Println("start")
defer log.Println("end")
ctx, cancel := context.WithCancel(context.Background())
tasks := make(chan struct{})
go func() {
time.Sleep(1 * time.Second) // a job
tasks <- struct{}{}
}()
go func() {
time.Sleep(3 * time.Second) // a job
cancel()
}()
for {
select {
case <-ctx.Done():
log.Println("QUIT")
return
case <-tasks:
log.Println("Doing")
// do some long time jobs more than 10 seconds
select {
case <-ctx.Done():
return
case <-time.After(time.Second * 10):
}
}
}
}

The second job is running for 10 seconds, so it will block the loop for that time, and the signal you sent into the quit channel will not be received until this job is complete.
To interrupt the time-consuming job, maybe you can split it into another goroutine. Something like this would do:
go func() {
for {
select {
case <-tasks:
fmt.Println("Doing")
// do some long time jobs more than 10 seconds
time.Sleep(10 * time.Second)
}
}
}()
<-quit
fmt.Println("QUIT")

It would be easier to use [sync.WaitGroup.
package main
import (
"fmt"
"sync"
"time"
)
func worker(msg string, duration time.Duration, doing bool, wg *sync.WaitGroup) {
defer wg.Done()
time.Sleep(duration)
fmt.Println(msg)
if doing {
time.Sleep(10 * time.Second)
}
}
func main() {
var wg sync.WaitGroup
msgs := [2]string{"QUIT", "Doing"}
durations := [2]time.Duration{3 * time.Second, 1 * time.Second}
doing := [2]bool{false, true}
for i, msg := range msgs {
wg.Add(1)
go worker(msg, durations[i], doing[i], &wg)
}
wg.Wait()
}

Related

How can one close a channel in a defer block safely?

Consider the following example:
package main
import (
"fmt"
"time"
)
func main() {
ticker := time.NewTicker(2 * time.Second)
done := make(chan bool)
defer func() {
fmt.Println("exiting..")
done <- true
close(done)
}()
go func(ticker *time.Ticker, done chan bool) {
for {
select {
case <-done:
fmt.Println("DONE!")
break
case <-ticker.C:
fmt.Println("TICK!...")
}
}
}(ticker, done)
time.Sleep(7 * time.Second)
}
The goroutine waiting to receive from done never receives as (I am guessing) the main goroutine finished beforehand. However if I change the sleep time of the main goroutine to 8 seconds it receives a message; Why is there this dependency on the sleep time?
Is it because there is that second difference that keeps the goroutine alive and the there isn't enough time to kill it?
How would I than kill the goroutine gracefully?
You need to ensure that main does not return before the goroutine finishes.
The simplest way to do this is using a WaitGroup:
var wg sync.WaitGroup
defer wg.Wait()
wg.Add(1)
go func() {
defer wg.Done()
// …
Note that defers run in reverse order, so you must put defer wg.Wait() before defer close(done), otherwise it will deadlock.

GO routine never exits based on stop condition - unable to find the reason

In this example, we have a worker. The idea here is simulate clean shutdown of all go routines based on a condition.
In this case, go routines get spun - based on workers count. Each go routine reads the channel, does some work and sends output to the outputChannel.
The main go routine reads this output and prints it. To simulate a stop condition, the doneChannel is closed. Expected outcome is that select inside each go routine will pick this up and execute return which in turn will call the defer println. The actual output is that it never gets called and main exits.
Not sure what's the reason behind this.
package main
import (
"log"
"time"
)
const jobs = 100
const workers = 1
var timeout = time.After(5 * time.Second)
func main() {
doneChannel := make(chan interface{})
outputChannel := make(chan int)
numberStream := generator()
for i := 1; i <= workers; i++ {
go worker(doneChannel, numberStream, outputChannel)
}
// listen for output
loop:
for {
select {
case i := <-outputChannel:
log.Println(i)
case <-timeout:
// before you timeout cleanup go routines
break loop
}
}
close(doneChannel)
time.Sleep(5 * time.Second)
log.Println("main exited")
}
func generator() <-chan int {
defer log.Println("generator completed !")
c := make(chan int)
go func() {
for i := 1; i <= jobs; i++ {
c <- i
}
defer close(c)
}()
return c
}
func worker(done <-chan interface{}, c <-chan int, output chan<- int) {
// this will be a go routine
// Do some work and send results to output Channel.
// Incase if the done channel is called kill the go routine.
defer log.Println("go routines exited")
for {
select {
case <-done:
log.Println("here")
return
case i := <-c:
time.Sleep(1 * time.Second) // worker delay
output <- i * 100
}
}
}
When your main loop finishes during the timeout, you continue your program and
Close done channel
Print message
Exit
There is no reason to wait for any goroutine to process the signal of this channel.
If you add a small sleep you will see some messages
In real scenarios we use a waitgroup to be sure all goroutine finish properly

How to detect goroutine leaks?

In the below code:
package main
import (
"context"
"fmt"
"time"
)
func cancellation() {
duration := 150 * time.Millisecond
ctx, cancel := context.WithTimeout(context.Background(), duration)
defer cancel()
ch := make(chan string)
go func() {
time.Sleep(time.Duration(500) * time.Millisecond)
ch <- "paper"
}()
select {
case d := <-ch:
fmt.Println("work complete", d)
case <-ctx.Done():
fmt.Println("work cancelled")
}
time.Sleep(time.Second)
fmt.Println("--------------------------------------")
}
func main() {
cancellation()
}
Because of unbuffered channel(ch := make(chan string)), go-routine leaks due to block on send(ch <- "paper"), if main goroutine is not ready to receive.
Using buffered channel ch := make(chan string, 1) does not block send(ch <- "paper")
How to detect such go-routine leaks?
There are some packages that let you do that. Two that I've used in the past:
https://github.com/fortytw2/leaktest
https://github.com/uber-go/goleak
Generally, they use functionality from the runtime package to examine the stack before and after your code runs and report suspected leaks. It's recommended to use them in tests. I found this works well in practice and used it in a couple of projects.

How to handle multiple go-routines closing the same channel?

I am having 2 go-routines reading from a single channel. After 4 seconds, I cancel the context and terminate the select loop. Before terminating the loop I call close on the channel, since there are 2 go-routines the close gets called twice and causes a panic because one of the go-routines would have already closed the channel. Currently I am using a recover to recover from the panic, is there a better way of doing this?
package main
import (
"context"
"fmt"
"sync"
"time"
)
func numberGen(ctx context.Context, numChan chan int) {
num := 0
doneCh := ctx.Done()
defer func() {
if r := recover(); r != nil {
fmt.Println("recovered from ", r)
}
}()
for {
select {
case <-doneCh:
fmt.Println("done generating...")
close(numChan)
return
default:
num++
numChan <- num
}
}
}
func main() {
ctx, cancelFn := context.WithCancel(context.Background())
numChan := make(chan int)
var wg sync.WaitGroup
wg.Add(2)
go numberGen(ctx, numChan)
go numberGen(ctx, numChan)
go func(cfn context.CancelFunc) {
time.Sleep(10 * time.Millisecond)
cfn()
}(cancelFn)
for n := range numChan {
fmt.Println("received value ", n)
}
time.Sleep(2 * time.Second)
}
Close the channel after the goroutines are done sending values.
var wg sync.WaitGroup
wg.Add(2)
go numberGen(ctx, numChan, &wg)
go numberGen(ctx, numChan, &wg)
go func() {
wg.Wait()
close(numChan)
}()
Update numberGen to call Done() on the wait group. Also, remove the call to close.
func numberGen(ctx context.Context, numChan chan int, wg *sync.WaitGroup) {
defer wg.Done()
...

WaitGroup goroutines with channel

I am learning WaitGroup from the blog https://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-all-goroutines-to-finish-executing-before-continuing/
the code:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
messages := make(chan int)
var wg sync.WaitGroup
// you can also add these one at
// a time if you need to
wg.Add(3)
go func() {
defer wg.Done()
time.Sleep(time.Second * 3)
messages <- 1
}()
go func() {
defer wg.Done()
time.Sleep(time.Second * 2)
messages <- 2
}()
go func() {
defer wg.Done()
time.Sleep(time.Second * 1)
messages <- 3
}()
go func() {
for i := range messages {
fmt.Println(i)
}
}()
wg.Wait()
}
I think it should print 3, 2 and 1 in order. But it only prints 3, 2 but 1 is missing, what's the problem?
You can tree it on https://play.golang.org/p/kZCvDhykYM
Right after the latest messages <- 1, the deferred wg.Done() is invoked which releases wg.Wait() in the end of the program and the program quits. When a program quits all the goroutines get killed, so the printing goroutine does not have a chance to print the latest value.
If you put something like time.Sleep(time.Second * 1) right after wg.Done() you would be able to see all the output lines.
package main
import (
"fmt"
"sync"
"time"
)
func main() {
messages := make(chan int)
var wg sync.WaitGroup
wg.Add(3)
// created this goroutine to wait for other
// goroutines to complete and to close the channel
go func() {
wg.Wait()
close(messages)
}()
go func() {
defer wg.Done()
time.Sleep(time.Second * 3)
messages <- 1
}()
go func() {
defer wg.Done()
time.Sleep(time.Second * 2)
messages <- 2
}()
go func() {
defer wg.Done()
time.Sleep(time.Second * 1)
messages <- 3
}()
// this for loop is blocked in main goroutine.
// till it reads all messages and channel is closed.
for v := range messages {
fmt.Print(v)
}
}
The mentioned blog starts with following comment:
EDIT: As pointed out by effenn in this Reddit comment, a lot of information in this article is “dangerously inaccurate”. OOPS! I’ve written a followup/correction article here for your viewing pleasure, but I’m leaving this article up for “historical purposes”.
The Reddit comment and the followup article both describe the problem and give a solution to your problem. (Adding time.Sleep(...) to make the program work the way you expect is really hacky...)
package main
import (
"fmt"
"sync"
"time"
)
func main() {
messages := make(chan int)
var wg sync.WaitGroup
// you can also add these one at
// a time if you need to
wg.Add(3)
go func() {
defer wg.Done()
time.Sleep(time.Second * 3)
messages <- 1
}()
go func() {
defer wg.Done()
time.Sleep(time.Second * 2)
messages <- 2
}()
go func() {
defer wg.Done()
time.Sleep(time.Second * 1)
messages <- 3
}()
exit:
for {
select {
case i, ok := <-messages:
if !ok {
break exit
}
fmt.Println(i)
default:
time.Sleep(time.Second)
}
}
wg.Wait()
}

Resources