select in a for loop deadlock's - go

https://play.golang.org/p/FyIUPkwq0R
Why does the following deadlock?
package main
import (
"fmt"
)
var quit chan bool
var buffer chan string
func main() {
buffer = make(chan string)
quit = make(chan bool)
go func() {
i:=0
for {
select {
case <- quit:
fmt.Println("Bye!")
return
default:
fmt.Println(<-buffer)
}
i++
fmt.Println(i)
}
}()
buffer <- "Go!"
quit <- true // This line dead locks
//buffer <- "Hello" // When I do this instead it works?
//quit <- true // Also when I don't quit it still exit's?
}

This code isn't guaranteed to work properly. You might get lucky, but apparently you've been getting unlucky. You might get lucky and the following might happen:
Let's say we have two goroutines, A and B, where A is the goroutine running main and B is the goroutine running the anonymous function. The following might happen:
B: Execute select; there's nobody writing on the quit channel, so execute the default case
B: Execute <-buffer, so begin blocking, waiting for somebody to write to buffer
A: Write "Go!" to buffer
B: Receive "Go!" and print it. Continue looping.
A: Write true to quit
B: Execute select; A is trying to write to quit, so execute that case. Print "Bye!" and return
A: Since write has finished, continue and return from main
However, this isn't guaranteed to happen. In particular, after reading from buffer, B might keep executing, execute the select, and fall into the default case before A has a chance to write to quit. That's what's probably happening, and would look like this:
B: Execute select; there's nobody writing on the quit channel, so execute the default case
B: Execute <-buffer, so begin blocking, waiting for somebody to write to buffer
A: Write "Go!" to buffer
B: Receive "Go!" and print it. Continue looping
B: Execute select; there's nobody writing on the quit channel, so execute the default case
B: Execute <-buffer, so begin blocking, waiting for somebody to write to buffer
A: Write true to quit, so begin blocking, waiting for somebody to read from quit
Now both A and B are blocked, and since there are no other goroutines in the system, no event can ever unblock either of them, so the system is stuck.
Solution
One way of fixing this would be to make it so that goroutine B reads from the buffer as one of the select cases instead of inside a select case. This way, the select will simply block until either channel becomes available for an action, and your code will behave as you probably wanted it to:
select {
case <-quit:
fmt.Println("Bye!")
return
case str := <-buffer:
fmt.Println(str)
}
See it here on the Go Playground.
Note, however, that since the main goroutine is returning as soon as it writes to the quit channel, and the entire Go program exits as soon as that happens, you may (and probably will) get unlucky and fmt.Println("Bye!") will not execute before the program quits.

Change the select default clause to
default:
fmt.Println("waiting on <-buffer")
fmt.Println(<-buffer)
to see what's going on.
The issue is that the goroutine executes the default branch in the select before main executes quit <- true.
The goroutine blocks at fmt.Println(<-buffer) and the main function blocks at quit <- true.
To prevent the deadlock, receive in the case statement:
select {
case <-quit:
fmt.Println("Bye!")
return
case msg := <-buffer:
fmt.Println(msg)
}
playground example

package main
import (
"fmt"
)
var quit chan bool
var buffer chan string
func main() {
buffer = make(chan string)
quit = make(chan bool)
go func() {
i := 0
for {
select {
case <-quit:
fmt.Println("Bye!")
return
case v := <-buffer:
fmt.Println(v)
default:
// 这里也可能 block
// fmt.Println(<-buffer)
}
i++
fmt.Println(i)
}
}()
buffer <- "Go!"
quit <- true // This line dead locks
//buffer <- "Hello" // When I do this instead it works?
//quit <- true // Also when I don't quit it still exit's?
}

Related

"Select" statement inside of goroutine with for loop

Could somebody please explain, why if goroutine has endless for loop and select inside of the loop, a code in the loop is run only one time?
package main
import (
"time"
)
func f1(quit chan bool){
go func() {
for {
println("f1 is working...")
time.Sleep(1 * time.Second)
select{
case <-quit:
println("stopping f1")
break
}
}
}()
}
func main() {
quit := make(chan bool)
f1(quit)
time.Sleep(4 * time.Second)
}
Output:
f1 is working...
Program exited.
But if "select" is commented out:
package main
import (
"time"
)
func f1(quit chan bool){
go func() {
for {
println("f1 is working...")
time.Sleep(1 * time.Second)
//select{
//case <-quit:
//println("stopping f1")
//break
//}
}
}()
}
func main() {
quit := make(chan bool)
f1(quit)
time.Sleep(4 * time.Second)
}
Output:
f1 is working...
f1 is working...
f1 is working...
f1 is working...
f1 is working...
https://play.golang.org/p/MxKy2XqQlt8
A select statement without a default case is blocking until a read or write in at least one of the case statements can be executed. Accordingly, your select will block until a read from the quit channel is possible (either of a value or the zero value if the channel is closed). The language spec provides a concrete description of this behavior, specifically:
If one or more of the communications [expressed in the case statements] can proceed, a single one that can proceed is chosen via a uniform pseudo-random selection. Otherwise, if there is a default case, that case is chosen. If there is no default case, the "select" statement blocks until at least one of the communications can proceed.
Other code issues
Caution! break applies to the select statement
However, even if you did close the quit channel to signal shutdown of your program, your implementation would likely not have the desired effect. A (unlabelled) call to break will terminate execution of the inner-most for, select or switch statement within the function. In this case, the select statement will break and the for loop will run through again. If quit was closed, it will run forever until something else stops the program, otherwise it will block again in select (Playground example)
Closing quit will (probably) not immediately stop the program
As noted in the comments, the call to time.Sleep blocks for a second on each iteration of the loop, so any attempt to stop the program by closing quit will be delayed for up to approximately a second before the goroutine checks quit and escapes. It is unlikely this sleep period must complete in its entirety before the program stops.
More idiomatic Go would block in the select statement on receiving from two channels:
The quit channel
The channel returned by time.After – this call is an abstraction around a Timer which sleeps for a period of time and then writes a value to the provided channel.
Resolution
Solution with minimal changes
A resolution to fix your immediate issue with minimal changes to your code would be:
Make the read from quit non-blocking by adding a default case to the select statement.
Ensuring the goroutine actually returns when the read from quit succeeds:
Label the for loop and use a labelled call to break; or
return from the f1 function when it is time to quit (preferred)
Depending on your circumstances, you may find it more idiomatic Go to use a context.Context to signal termination, and to use a sync.WaitGroup to wait for the goroutine to finish before returning from main.
package main
import (
"fmt"
"time"
)
func f1(quit chan bool) {
go func() {
for {
println("f1 is working...")
time.Sleep(1 * time.Second)
select {
case <-quit:
fmt.Println("stopping")
return
default:
}
}
}()
}
func main() {
quit := make(chan bool)
f1(quit)
time.Sleep(4 * time.Second)
close(quit)
time.Sleep(4 * time.Second)
}
Working example
(Note: I have added an additional time.Sleep call in your main method to avoid this returning immediately after the call to close and terminating the program.)
Fixing the sleep blocking issue
To fix the additional issue regarding the blocking sleep preventing immediate quit, move the sleep to a timer in the select block. Modifying your for loop as per this playground example from the comments does exactly this:
for {
println("f1 is working...")
select {
case <-quit:
println("stopping f1")
return
case <-time.After(1 * time.Second):
// repeats loop
}
}
Related literature
Go-by-example: Non-blocking channel operations
Dave Cheney's Channel axioms, specifically the fourth axiom "A receive from a closed channel returns the zero value immediately."

go routine: does select really pick a random case?

Context: https://tour.golang.org/concurrency/5
Hello everyone, I am learning Go following the above link.
The description says "It chooses one at random if multiple are ready."
However, after making the main routine waiting for 2 second, before calling func fibonacci. The channels should be the following after 2 sec:
c: 10 calls to get value from the channel
quit: 0
It looks to me both channels are ready. If "It chooses one at random if multiple are ready" is true, then there is a 50% chance that the first call on the case in fibonacci will get the 0 from the quit channel. However, it is not the case. All 10 numbers will always get printed out before quitting. Hence it does not look like the selection is random. Am I missing something?
package main
import "fmt"
import "time"
func fibonacci(c, quit chan int) {
x, y := 0, 1
for {
select {
case c <- x:
x, y = y, x+y
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
time.Sleep(2 * time.Second)
fibonacci(c, quit)
}
In addition, the next page:
https://tour.golang.org/concurrency/6
It looks like the default code should print out either tick. or BOOM! at 500 milli second. However, only BOOM! is printed, always. If I changed the time in the default from 50 to 55, then both tick and BOOM get printed. Why is this? Does a After take precedence over a Tick in a select?
package main
import (
"fmt"
"time"
)
func main() {
tick := time.Tick(100 * time.Millisecond)
boom := time.After(500 * time.Millisecond)
for {
select {
case <-tick:
fmt.Println("tick.")
case <-boom:
fmt.Println("BOOM!")
return
default:
fmt.Println(" .")
time.Sleep(55 * time.Millisecond)
}
}
}
I think you're mistaken about what's happening in your main method... I'm gonna break down what I think is going on for clarity
func main() {
c := make(chan int)
quit := make(chan int) // make a couple channels
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c) // blocked here in goroutine
}
quit <- 0
}() // go func { ... }() - you're both writing this closure and invoking it as a goroutine
time.Sleep(2 * time.Second) // sleep for 2 seconds
fibonacci(c, quit) // call fibonacci passing in the channels
}
so what's actually happened here is you've called this closure as a goroutine then wait 2 seconds during which your goroutine is still sitting in the body of the for loop waiting to receive on c, you call fibonacci which executes as you expect going into the for-select, at which point you keep hitting that code on every iteration of the loop c <- x (it receives, i gets incremented, you receive again, next value until the loop is over due to i == 10). then you proceed to the next line and send on the quit channel, the select executes that condition and your program exits.
As far as what executes first the language spec says;
Execution of a "select" statement proceeds in several steps:
1) For all the cases in the statement, the channel operands of receive
operations and the channel and right-hand-side expressions of send
statements are evaluated exactly once, in source order, upon entering
the "select" statement. The result is a set of channels to receive
from or send to, and the corresponding values to send. Any side
effects in that evaluation will occur irrespective of which (if any)
communication operation is selected to proceed. Expressions on the
left-hand side of a RecvStmt with a short variable declaration or
assignment are not yet evaluated. 2) If one or more of the
communications can proceed, a single one that can proceed is chosen
via a uniform pseudo-random selection. Otherwise, if there is a
default case, that case is chosen. If there is no default case, the
"select" statement blocks until at least one of the communications can
proceed. Unless the selected case is the default case, the respective
communication operation is executed. 3) If the selected case is a
RecvStmt with a short variable declaration or an assignment, the
left-hand side expressions are evaluated and the received value (or
values) are assigned. 4) The statement list of the selected case is
executed.
It's only psuedo random in race conditions, problem is you're not creating a race condition.
In the first snippet of your code, quit is never ready before the for loop. And, in every iteration of the loop, c is ready, and will block until a number is sent through. So there is really nothing select can do but write to c. It does not matter at all if you sleep two seconds.
The second piece of code is not bugged. The select does randomly pick a case indeed. However, Go Playground has a fixed random generator, which means, on Playground, select will always pick one certain case in every run.
You created an unbuffered chan:
c := make(chan int)
This means that anything reading from the chan is going to block until something is written to it. And anything writing to the chan is going to block until something reads from it.
Thus in this code:
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
the <-c blocks until something is put on c. So it sits there waiting until after your time.Sleep() completes, and gets to:
case c <- x:
upon which it alternates blocking back and forth reading one value, writing one value, until it reads 10 values, and then it sends 0 to quit.
To create a buffered chan, you need to specify a size for the buffer.
c := make(chan int, 10)
But note, that even if you do this, it's still not going to behave as you expect, as you're reading all 10 values before you write to quit. You need to put the writers in the same place, and the readers in the same place, not mixing them up.

Go: Act on Signal while not Blocking Operations

I want to fire the ps command continuously in a goroutine to monitor mem and cpu usages. I didn't use top because top doesn't allow me to select columns as ps does. This goroutine needs to receive a stop signal to stop the command, but I don't know how not to block running the command while waiting the signal. For top I can do:
top := exec.Command("top")
<-stop // blocking
top.Process.Signal(os.Kill)
But for ps if I do:
ps := exec.Command("ps")
for {
ps.Run()
<-stop
}
The above code will block on stop. I want to keep firing ps.Run(), while being able to stop when a stop signal is ready. Thanks.
One way that you could achieve this is by utilizing the for/select timeout idiom, there are a couple of similar methods of doing this. Take this trivial example:
package main
import (
"fmt"
"time"
)
func main() {
abort := make(chan struct{})
go func() {
for {
select {
case <-abort:
return
case <-time.After(1 * time.Second):
// replace fmt.Println() with the command you wish to run
fmt.Println("tick")
}
}
}()
// replace time.Sleep() with code waiting for 'abort' command input
time.Sleep(10 * time.Second)
abort <- struct{}{}
}
To modify this example to work in your circumstance place the code that you want to run in the <-time.After(): case, which (in this instance) will run once per second, if no other case is available to receive for that duration. And instead of time.Sleep() which I placed at the end, put the code that will interrupt the <-time.After(): case, sending <- struct{}{} on the abort channel (or whatever you name it).
NOTE: In an earlier version of this answer I had abort as a chan bool, as I like the clarity of <-abort true and don't consider chan struct{} to be as clear, I opted to change it in this example however, as <- struct{}{} isn't unclear, especially once you've gotten used to the pattern.
Also, if you want the command to execute on each iteration of the for loop and not wait for a timeout then you can make that case default:, remove <-time.After() and it will run on each iteration of the loop where another channel is not ready to receive.
You can play with this example in the playground if you'd like, although it will not allow syscalls, or the default: case example to be run in that environment.
To run it over and over, without sleep:
func run(stop <-chan struct{}) {
ps := exec.Command("ps")
for {
select {
case <-stop:
return
default:
ps.Run()
//processing the output
}
}
}
And then go run(stop). And to run it every 3 seconds (for example):
func run(stop <-chan struct{}) {
ps := exec.Command("ps")
tk := time.Tick(time.Second * 3)
for {
select {
case <-stop:
return
case <-tk:
ps.Run()
//processing the output
}
}
}

go tutorial select statement

I'm working through the examples at tour.golang.org, and I've encountered this code I don't really understand:
package main
import "fmt"
func fibonacci(c, quit chan int) {
x, y := 0, 1
for {
select {
case c <- x: // case: send x to channel c?
x, y = y, x+y
case <-quit: // case: receive from channel quit?
fmt.Println("quit")
return
}
}
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() { // when does this get called?
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
fibonacci(c, quit)
}
I understand the basics of how channels work, but what I don't get is how the above select statement works. The explanation on the tutorial says:
"The select statement lets a goroutine wait on multiple communication operations.
A select blocks until one of its cases can run, then it executes that case. It chooses one at random if multiple are ready."
But how are the cases getting executed? From what I can tell, they're saying:
case: send x to channel c
case: receive from quit
I think I understand that the second one executes only if quit has a value, which is done later inside the go func(). But what is the first case checking for? Also, inside the go func(), we're apparently printing values from c, but c shouldn't have anything in it at that point? The only explanation I can think of is that the go func() somehow executes after the call to fibonacci(). I'm guessing it's a goroutine which I don't fully understand either, it just seems like magic.
I'd appreciate if someone could go through this code and tell me what it's doing.
Remember that channels will block, so the select statement reads:
select {
case c <- x: // if I can send to c
// update my variables
x, y = y, x+y
case <-quit: // If I can receive from quit then I'm supposed to exit
fmt.Println("quit")
return
}
The absence of a default case means "If I can't send to c and I can't read from quit, block until I can."
Then in your main process you spin off another function that reads from c to print the results
for i:=0; i<10; i++ {
fmt.Println(<-c) // read in from c
}
quit <- 0 // send to quit to kill the main process.
The key here is to remember that channels block, and you're using two unbuffered channels. Using go to spin off the second function lets you consume from c so fibonacci will continue.
Goroutines are so-called "green threads." Starting a function call with the keyword go spins it off into a new process that runs independent of the main line of execution. In essence, main() and go func() ... are running simultaneously! This is important since we're using a producer/consumer pattern in this code.
fibonacci produces values and sends them to c, and the anonymous goroutine that's spawned from main consumes values from c and processes them (in this case, "processing them" just means printing to the screen). We can't simply produce all the values and then consume them, because c will block. Furthermore fibonacci will produce more values forever (or until integer overflow anyway) so even if you had a magic channel that had an infinitely long buffer, it would never get to the consumer.
There are two key things to understanding this code example:
First, let's review how unbuffered channels work. From the documentation
If the channel is unbuffered, the sender blocks until the receiver has
received the value.
Note that both channels in the code example, c and quit are unbuffered.
Secondly, when we use the go keyword to start a new goroutine, the execution will happen in parallel with other routines. So in the example, we have two go routines running: the routine started by func main(), and the routine started by go func()... inside the func main().
I added some inline comments here which should make things clearer:
package main
import "fmt"
func fibonacci(c, quit chan int) {
x, y := 0, 1
for { // this is equivalent to a while loop, without a stop condition
select {
case c <- x: // when we can send to channel c, and because c is unbuffered, we can only send to channel c when someone tries to receive from it
x, y = y, x+y
case <-quit: // when we can receive from channel quit, and because quit is unbuffered, we can only receive from channel quit when someone tries to send to it
fmt.Println("quit")
return
}
}
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() { // this runs in another goroutine, separate from the main goroutine
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
fibonacci(c, quit) // this doesn't start with the go keyword, so it will run on the go routine started by func main()
}
You've pretty much got it.
inside the go func(), we're apparently printing values from c, but c shouldn't have anything in it at that point? The only explanation I can think of is that the go func() somehow executes after the call to fibonacci(). I'm guessing it's a goroutine
Yes, the go keyword starts a goroutine, so the func() will run at the same time as the fibonacci(c, quit). The receive from the channel in the Println simply blocks until there is something to receive

How does make(chan bool) behave differently from make(chan bool, 1)?

My question arises from trying to read a channel, if I can, or write it, if I can, using a select statement.
I know that channels specified like make(chan bool, 1) are buffered, and part of my question is what is the difference between that, and make(chan bool) -- which this page says is the same thing as make(chan bool, 0) --- what is the point of a channel that can fit 0 values in it?
See playground A:
chanFoo := make(chan bool)
for i := 0; i < 5; i++ {
select {
case <-chanFoo:
fmt.Println("Read")
case chanFoo <- true:
fmt.Println("Write")
default:
fmt.Println("Neither")
}
}
A output:
Neither
Neither
Neither
Neither
Neither
(Removing the default case results in a deadlock!!)
Now see playground B:
chanFoo := make(chan bool, 1) // the only difference is the buffer size of 1
for i := 0; i < 5; i++ {
select {
case <-chanFoo:
fmt.Println("Read")
case chanFoo <- true:
fmt.Println("Write")
default:
fmt.Println("Neither")
}
}
B output:
Write
Read
Write
Read
Write
In my case, B output is what I want. What good are unbuffered channels? All the examples I see on golang.org appear to use them to send one signal/value at a time (which is all I need) -- but as in playground A, the channel never gets read or written. What am I missing here in my understanding of channels?
what is the point of a channel that can fit 0 values in it
First I want to point out that the second parameter here means buffer size, so that is simply a channel without buffers (un-buffered channel).
Actually that's the reason why your problem is generated. Un-buffered channels are only writable when there's someone blocking to read from it, which means you shall have some coroutines to work with -- instead of this single one.
Also see The Go Memory Model:
A receive from an unbuffered channel happens before the send on that channel completes.
An unbuffered channel means that read and writes from and to the channel are blocking.
In a select statement:
the read would work if some other goroutine was currently blocked in writing to the channel
the write would work if some other goroutine was currently blocked in reading to the channel
otherwise the default case is executed, which happens in your case A.
Unbuffered channels (created without capacity) will block the sender until somebody can read from them, so to make it work as you expect, you should use two goroutines to avoid the deadlock in the same thread.
For example, with this code: http://play.golang.org/p/KWJ1gbdSqf
It also includes a mechanism for the main func to detect when both goroutines have finished.
package main
import "fmt"
func send(out, finish chan bool) {
for i := 0; i < 5; i++ {
out <- true
fmt.Println("Write")
}
finish <- true
close(out)
}
func recv(in, finish chan bool) {
for _ = range in {
fmt.Println("Read")
}
finish <- true
}
func main() {
chanFoo := make(chan bool)
chanfinish := make(chan bool)
go send(chanFoo, chanfinish)
go recv(chanFoo, chanfinish)
<-chanfinish
<-chanfinish
}
It won't show the alternate Read and Write, because as soon as send writes to the channel, it is blocked, before it can print the "Write", so the execution moves to recv that receives the channel and prints "Read". It will try to read the channel again but it will be blocked and the execution moves to send. Now send can write the first "Write", send to the channel (without blocking because now there is a receiver ready) and write the second "Write". In any case, this is not deterministic and the scheduler may move the execution at any point to any other running goroutine (at least in the latest 1.2 release).

Resources