Following on from this answer, if a goroutine is selecting on two channels, is it guaranteed that the channels are selected in the same order that they are sent on? I'm particularly interested in the case where the sender is single threaded.
As an example, is it guaranteed that this code always produces the same output:
package main
import (
"fmt"
"time"
)
var x, y chan int
func worker() {
for {
select {
case v := <- x:
fmt.Println(v)
case v := <- y:
fmt.Println(v)
}
}
}
func main() {
x = make(chan int)
y = make(chan int)
go worker()
x <- 1
y <- 2
<- time.After(1 * time.Second)
}
My experiments would seem to indicate that this is guaranteed for unbuffered channels, as shown, but not guaranteed for buffered channels. Can someone confirm that please?
No. It's not guaranteed. In fact, it's randomized. From the Go language spec (emphasis added):
If one or more of the communications can proceed, a single one that can proceed is chosen via a uniform pseudo-random selection. Otherwise, if there is a default case, that case is chosen. If there is no default case, the "select" statement blocks until at least one of the communications can proceed.
Although in your example it's guaranteed that you'll see 1 printed first, but not because of how select works.
It's because x is an unbuffered channel, and send on it will block until someone receives from it, and only then can you send on channel y.
So your select inside worker() won't see 2 ready channels at the same time.
It if would, then one is chosen pseudo-randomly. For details, see How does select work when multiple channels are involved?
So note that if you'd use buffered channels:
x = make(chan int, 1)
y = make(chan int, 1)
Then send on x and y wouldn't block, and now it's up to the goroutine scheduler how goroutines are executed. It's possible the main goroutine can send both values before the worker() goroutine reaches and evaluates the channel operations, and so you could see 1 then 2 printed, just as well as 2 and then 1.
Related
Here is the code from A tour of Go to demonstrate the use of select statements in Go.
func fibonacci(c, quit chan int) {
x, y := 0, 1
for {
select {
case c <- x:
x, y = y, x+y
case <-quit:
return
}
}
}
My question is :
what is case c <-x doing here? is some data x being sent to the channel or we are looking for the condition when some data is being sent to the channel.
If some data is being sent to the loop, the loop should never end and if the condition of a data sent to channel is evaluated I cannot see any data being sent to the channel.
The main function is:
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
fibonacci(c, quit)
}
That case c <- x: will be successful if channel c is free to receive data only. And it successfully send that value of x, then x's value is changed to y's previous value with line x, y = y, x+y and populate fibonacci.
In this way, If there is no receiver to the channel c, there is no blocking because the loop always continuing and quit when the quit channel receives a value.
In the main function, inside the goroutine channel receives messages sent from the select case and free up the channel for a new message.
Please refer to this example for more understanding.
i have been following the go tour examples, and i don't understand how this works
https://tour.golang.org/concurrency/5
package main
import "fmt"
func fibonacci(c, quit chan int) {
x, y := 0, 1
for {
select {
case c <- x:
x, y = y, x+y
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
fibonacci(c, quit)
}
how is this working?
and while i'm trying to understand.
package main
import "fmt"
func b(c,quit chan int) {
c <-1
c <-2
c <-3
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
b(c,quit)
}
this will sometimes print 1,2 sometimes print 1,2,3, why?
First, in the func fibonacci, the select statement tries select the first thing of the two following to finish:
c <- x
<- quit
It is fairly easy to understand <- quit, which is tries to receive a value from a channel called quit (and ignores the value received).
c <- x means sending a value that equals (is a copy of) x. It seems like unblocking, but in Go, sending over an unbuffered channel (which is explained in the Go tour) blocks when there is no receiver.
So here it means, waiting for a receiver to be ready to receive the value (or a space in the buffer, if it were a buffered channel), which in this code means fmt.Println(<-c), and then send the value to the receiver.
So this statement unblocks (finishes) whenever <-c is evaluated. That is, every iteration of the loop.
And for your code, while all value 1, 2, 3, is guaranteed to be sent over the channel (and received), func b returns and thus func main retains without guaranteeing the fmt.Println(3) finishes.
In Go, when the func main returns, the program terminates, and unfinished goroutine does not get a chance to finish its work - thus sometimes it prints 3 and soetimes it does not.
To better understand the Fibonacci example let's analyze the different parts.
First the anonymous function
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
The "go" keyword will start a new goroutine, so now we have the "main" goroutine and this one, they're gonna be running concurrently.
The loop is telling us that we are going to execute this 10 times:
fmt.Println(<-c)
This call will block until we receive an integer from the channel and print it. Once this happens 10 times we are gonna signal that this goroutine has finished
quit <- 0
Now let's go back to the main goroutine, while the other goroutine was starting it has invoked the function "finbonacci", there's no "go" keyword so this call is blocking here.
fibonacci(c, quit)
Now let's analyze the function
x, y := 0, 1
for {
select {
case c <- x:
x, y = y, x+y
case <-quit:
fmt.Println("quit")
return
}
}
We start an infinite loop and on each iteration, we will try to execute one case from the select
The first case will try to send Fibonacci's sequence values indefinitely to the channel, at some point, once the other goroutine reaches the fmt.Println(<-c) statement, this case will be executed, the values will be recalculated and the next iteration of the loop will happen.
case c <- x:
x, y = y, x+y
The second case won't be able to run yet, since nobody is sending anything to the "quit" channel.
case <-quit:
fmt.Println("quit")
return
}
After 10 iterations the first goroutine will stop receiving new values, thus the first select case won't be able to run
case c <- x: // this will block after 10 times, nobody is reading
x, y = y, x+y
At this point, no case in the "select" can be executed, the main goroutine is "blocked" (more like paused from a logical point of view).
Finally, at some point, the first goroutine will signal that it has finished using the "quit" channel
quit <- 0
This allows the execution of the second "select" case, which will break the infinite loop and allow the "fibonacci" function to return and the main function will be able to finish in a clean way.
Hopefully, this helps you understand the Fibonacci example.
Now, moving to your code, you are not waiting for the anonymous goroutine to finish, in fact, it's not even finishing. Your "b" method is sending "1,2,3" and returns immediately.
Since it's the last part of the main function, the program terminates.
If you only see "1,2" sometimes is because the "fmt.Println" statement is too slow and the program terminates before it can print the 3.
I am learning basic Go via a Udemy course. In the goroutines section, there is an example of throttling which has thrown my understanding of how wait groups work.
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
func main() {
c1 := make(chan int)
c2 := make(chan int)
go populate(c1)
go fanOutIn(c1, c2)
for v := range c2 {
fmt.Println(v)
}
fmt.Println("about to exit")
}
func populate(c chan int) {
for i := 0; i < 100; i++ {
c <- i
}
close(c)
}
func fanOutIn(c1, c2 chan int) {
var wg sync.WaitGroup
const goroutines = 10
wg.Add(goroutines)
for i := 0; i < goroutines; i++ {
go func() {
for v := range c1 {
func(v2 int) {
c2 <- timeConsumingWork(v2)
}(v)
}
wg.Done()
}()
}
wg.Wait()
close(c2)
}
func timeConsumingWork(n int) int {
time.Sleep(time.Microsecond * time.Duration(rand.Intn(500)))
return n + rand.Intn(1000)
}
The part that isn't inline with my understanding is in the function fanOutIn where the we set up the WaitGroup, and Add(10).
Why am I getting 100 values printed out? Only a single value (i := 0) can be put onto c1, and the value is never explicitly removed from the channel. The code then hits wg.Done(), and the wait group queue is reduced to 9 and so on.
In my current understanding, I would expect to see 10 values, of 0 + rand.Intn(1000).
The function that is spun off reads as follows (including the go at the front and the parentheses to call it):
go func() {
for v := range c1 {
func(v2 int) {
c2 <- timeConsumingWork(v2)
}(v)
}
wg.Done()
}()
This code is a little weird and bizarre. Let's shrink it down even further, discarding the wg.Done and keeping only the for loop itself:
for v := range c1 {
func(v2 int) {
c2 <- timeConsumingWork(v2)
}(v)
}
There is an inner unnamed function that is pretty useless here; we can discard it without changing the behavior of the program, to get:
for v := range c1 {
c2 <- timeConsumingWork(v)
}
which is at last a simple loop. One key question is now this: How many iterations do you expect from this loop? Note: It is not necessarily any constant number. Perhaps a better way to phrase the question is: When does this loop end?
The for loop reads a channel. This kind of loop ends when a read from the channel indicates that there is no more data, i.e., that the channel is closed and its queue is empty. (See the Go specification section on for loops.)
So this innermost loop, for v := range c1, is not going to terminate until channel c1 is closed and there is no more data in its queue. This channel was created with:
c1 := make(chan int)
so it has no queue, so we need not even think about that: it terminates after a close(c1) closes it. You should now look for a close that closes c1.
Where is our close?
Here's the place that closes c1:
func populate(c chan int) {
for i := 0; i < 100; i++ {
c <- i
}
close(c)
}
We call this with c1 as its argument, so its final close(c) closes c1. Now you can ask: When do we reach this close call? The answer is obvious: after i >= 100 in the loop, i.e., after we have sent 100 values, zero through 99 respectively, into channel c1.
What fanOutIn does is spin off 10 goroutines. Each of the 10 goroutines runs the first anonymous function I quoted above. That anonymous function has a loop that runs an indeterminate number of times, repeating until channel c1 is closed. Each trip through the loop takes a value of the channel, so initially, if the ten goroutines all manage to start before there are any values available, all ten goroutines will be waiting for values.
When the producer function puts one value into the channel, one of the ten waiting goroutines will get it and begin using it. If that goroutine takes a long time to get back to the top of its own for loop, another goroutine will take the next-produced value. So what happens here is that up to ten produced values propagate through the channel to up to ten goroutines.1 Each of those (up to ten) goroutines spends some nontrivial amount of time using its value, then sends a final-product-value to channel c2 and goes back to the top of its own indefinite for loop.
Only when the producer has closed its channel c (which is our c1 here) will the ten goroutines see a closed-channel-empty-queue, allowing them to exit their for loops. When they do exit their for loops, each of them will call wg.Done() (once each) and terminate.
So, once the close(c1) has occurred (via the close(c) in populate), eventually all ten of these anonymous goroutines will have called wg.Done(). At that point, the wg.Wait() in fanOutIn will return. This will call close(c2) and return from fanOutIn, terminating that goroutine as well.
Meanwhile, in main, we use for v := range c2 to read from channel c2. This for loop will run when values are written into c2 by any of the ten goroutines. It will exit only when c2 itself is closed (its queue must also be empty but again c2 has a zero-length queue). So main will not proceed past the for loop until c2 is closed, which cannot happen until wg.Wait() returns, which cannot happen until ten wg.Done() calls have happened, which cannot happen until channel c1 is closed.
This means that main cannot get past its own for loop until populate has called close(c), and that happens only after generating exactly 100 values.
1As discussed in comments below, the phrase up to here can be important: we don't really know how many goroutines will really consume values. A lot depends on how much work each goroutine does, what kind of work that is, and how many CPUs your Go runtime has available.
I am stuck in a strange situation where the write operation to the channel never happens.
package main
import (
"fmt"
"time"
)
func main() {
c := make(chan int)
s := make(chan bool)
k := make(chan bool)
fmt.Println("start")
go func() {
fmt.Println("routine 1")
s <- true
}()
go func() {
fmt.Println("routine 2")
for {
select {
case <-s :
for i := 0; i < 3; i++ {
fmt.Println("before")
c <- i
fmt.Println("after")
}
case <- k :
fmt.Println("k ready")
break
}
}
}()
go func() {
fmt.Println("routine 3")
for {
select {
case x := <- c :
fmt.Println("x=", x)
k <- true
fmt.Println("k done")
}
}
}()
time.Sleep(1000 * time.Millisecond)
}
And here is the output:
start
routine 1
routine 2
before
routine 3
x= 0
after
before
I wonder why the write to the channel k blocks, but the log statement fmt.Println("k ready") is never printed.
Here is what I think :
go routine 1 writes true to channel s
go routine 2 writes 0 to
channel c and waits because buffer size is 0, it will not be able to
write '1' to it unless someone reads channel c
go routine 3 comes into the picture, reads channel c (now go routine 2
can write to c once go routine 2 resumes) prints the value of x. NOW IT SHOULD BE ABLE TO WRITE TO CHANNEL K but that is not happening
According to me it should be able to write to channel k then case 2 of goroutine should execute and print "k ready"
Can anyone explain me why write to the channel blocked?
As a fix I know I can increase the buffer size of channel c and everything will get printed but I am not interested in fixing this, instead I want to understand this scenario.
A nice blog to understand the above case.
You have a deadlock.
goroutine 1 writes to s then quits
goroutine 2 reads from s, and writes to c
goroutine 3 reads from c, and writes to k, and this blocks because nothing is reading from k, because goroutine 2 is blocked in the write to k above.
goroutine 2 writes to c again which blocks as goroutine 3 is still trying to write to k and thus is not reading from c
Contrary to what you say, you don't have a buffer size of 1. You have a buffer size of zero (i.e. an unbuffered channel), so a write will block until something reads. This is probably the source of your misunderstanding. Per the language specification:
A new, initialized channel value can be made using the built-in function make, which takes the channel type and an optional capacity as arguments:
make(chan int, 100)
The capacity, in number of elements, sets the size of the buffer in the channel. If the capacity is zero or absent, the channel is unbuffered and communication succeeds only when both a sender and receiver are ready. Otherwise, the channel is buffered and communication succeeds without blocking if the buffer is not full (sends) or not empty (receives). A nil channel is never ready for communication.
I am following the examples on tour.golang.org.
I understand the example mostly, the only issue I have is why does it stop when we pass 0 to quit channel? Regardless of whether 0 was passed to quit, there is always a value for x. So shouldn't select always fall on case 'c <- x' ?
func fibonacci(c chan int, quit chan int) {
x, y := 0, 1
for {
select {
case c <- x:
x, y = y, x+y
case <-quit:
return
}
}
close(c)
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
fibonacci(c, quit)
}
there is always a value for x. So shouldn't select always fall on case 'c <- x' ?
No, because this channel is unbuffered, the send will block until someone can receive from it.
Read about channels on Effective Go:
Receivers always block until there is data to receive. If the channel is unbuffered, the sender blocks until the receiver has received the value. If the channel has a buffer, the sender blocks only until the value has been copied to the buffer; if the buffer is full, this means waiting until some receiver has retrieved a value.
Additionally, if 2 cases in a select statement could proceed, one is picked pseudo-randomly.
If one or more of the communications can proceed, a single one that can proceed is chosen via a uniform pseudo-random selection. Otherwise, if there is a default case, that case is chosen. If there is no default case, the "select" statement blocks until at least one of the communications can proceed.