I try to understand logic behind concurrency of goroutine ,select and channel. The sample code is below. The base code is from tour go. I add some Printf to help me understand better.
package main
import "fmt"
func fibonacci(c, quit chan int) {
x, y := 0, 1
for {
select {
case c <- x:
fmt.Printf("(%v, %v)\n", x ,y)
x, y = y, x+y
fmt.Printf("(%v, %v)\n", x ,y)
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 4; i++ {
fmt.Println(<-c)
fmt.Printf("%v from main\n",i)
}
quit <- 0
}()
fibonacci(c, quit)
}
Output is
0
0 from main
(0, 1)
(1, 1)
(1, 1)
(1, 2)
1
1 from main
1
2 from main
(1, 2)
(2, 3)
(2, 3)
(3, 5)
2
3 from main
quit
There are concurrency behind goroutine and channel operation. My question is why the output is not
0
0 from main
(0, 1)
(1, 1)
1
1 from main
(1, 1)
(1, 2)
1
2 from main
(1, 2)
(2, 3)
2
3 from main
(2, 3)
(3, 5)
quit
It might clarify things to rewrite the body of the for loop in the goroutine slightly:
x := <-c
// block until a value has been read, then continue
fmt.Println(x)
fmt.Printf("%v from main\n",i)
And similarly the body of the select loop in fibonacci() is effectively:
c <- x
// block until a value has been written, then continue
fmt.Printf("(%v, %v)\n", x ,y)
x, y = y, x+y
fmt.Printf("(%v, %v)\n", x ,y)
In both cases you have a guarantee that the second print statement will print after the first.
When you run the program, say the goroutine starts immediately. It gets up to this read, but there is no value in the channel, so it blocks. Then the main program calls fibonacci(). It gets to the select statement. There is a reader for the channel c, so it sends the number x there.
Once this has happened both the goroutine and the main program are free to go. The select statement has triggered one of its branches and sent its value; the read has completed. Both goroutines are free to run, and can do so in any order (so long as each executes its own statements in order). You'll eventually reach a point where either the goroutine is blocked on the read or fibonacci() is blocked on the select, and once both catch up to each other, both will be free to execute again.
The order you propose would require the reader to "wake up" before the writer did, but nothing in Go requires this, and indeed on a multi-core system both can be running at the same time.
Related
i have been following the go tour examples, and i don't understand how this works
https://tour.golang.org/concurrency/5
package main
import "fmt"
func fibonacci(c, quit chan int) {
x, y := 0, 1
for {
select {
case c <- x:
x, y = y, x+y
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
fibonacci(c, quit)
}
how is this working?
and while i'm trying to understand.
package main
import "fmt"
func b(c,quit chan int) {
c <-1
c <-2
c <-3
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
b(c,quit)
}
this will sometimes print 1,2 sometimes print 1,2,3, why?
First, in the func fibonacci, the select statement tries select the first thing of the two following to finish:
c <- x
<- quit
It is fairly easy to understand <- quit, which is tries to receive a value from a channel called quit (and ignores the value received).
c <- x means sending a value that equals (is a copy of) x. It seems like unblocking, but in Go, sending over an unbuffered channel (which is explained in the Go tour) blocks when there is no receiver.
So here it means, waiting for a receiver to be ready to receive the value (or a space in the buffer, if it were a buffered channel), which in this code means fmt.Println(<-c), and then send the value to the receiver.
So this statement unblocks (finishes) whenever <-c is evaluated. That is, every iteration of the loop.
And for your code, while all value 1, 2, 3, is guaranteed to be sent over the channel (and received), func b returns and thus func main retains without guaranteeing the fmt.Println(3) finishes.
In Go, when the func main returns, the program terminates, and unfinished goroutine does not get a chance to finish its work - thus sometimes it prints 3 and soetimes it does not.
To better understand the Fibonacci example let's analyze the different parts.
First the anonymous function
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
The "go" keyword will start a new goroutine, so now we have the "main" goroutine and this one, they're gonna be running concurrently.
The loop is telling us that we are going to execute this 10 times:
fmt.Println(<-c)
This call will block until we receive an integer from the channel and print it. Once this happens 10 times we are gonna signal that this goroutine has finished
quit <- 0
Now let's go back to the main goroutine, while the other goroutine was starting it has invoked the function "finbonacci", there's no "go" keyword so this call is blocking here.
fibonacci(c, quit)
Now let's analyze the function
x, y := 0, 1
for {
select {
case c <- x:
x, y = y, x+y
case <-quit:
fmt.Println("quit")
return
}
}
We start an infinite loop and on each iteration, we will try to execute one case from the select
The first case will try to send Fibonacci's sequence values indefinitely to the channel, at some point, once the other goroutine reaches the fmt.Println(<-c) statement, this case will be executed, the values will be recalculated and the next iteration of the loop will happen.
case c <- x:
x, y = y, x+y
The second case won't be able to run yet, since nobody is sending anything to the "quit" channel.
case <-quit:
fmt.Println("quit")
return
}
After 10 iterations the first goroutine will stop receiving new values, thus the first select case won't be able to run
case c <- x: // this will block after 10 times, nobody is reading
x, y = y, x+y
At this point, no case in the "select" can be executed, the main goroutine is "blocked" (more like paused from a logical point of view).
Finally, at some point, the first goroutine will signal that it has finished using the "quit" channel
quit <- 0
This allows the execution of the second "select" case, which will break the infinite loop and allow the "fibonacci" function to return and the main function will be able to finish in a clean way.
Hopefully, this helps you understand the Fibonacci example.
Now, moving to your code, you are not waiting for the anonymous goroutine to finish, in fact, it's not even finishing. Your "b" method is sending "1,2,3" and returns immediately.
Since it's the last part of the main function, the program terminates.
If you only see "1,2" sometimes is because the "fmt.Println" statement is too slow and the program terminates before it can print the 3.
I have an approach to solve the find prime number problem using Go like this:
package main
import (
"fmt"
)
// Generate natural seri number: 2,3,4,...
func GenerateNatural() chan int {
ch := make(chan int)
go func() {
for i := 2; ; i++ {
ch <- i
}
}()
return ch
}
// Filter: delete the number which is divisible by a prime number to find prime number
func PrimeFilter(in <-chan int, prime int) chan int {
out := make(chan int)
go func() {
for {
if i := <-in; i%prime != 0 {
out <- i
}
}
}()
return out
}
func main() {
ch := GenerateNatural()
for i := 0; i < 100; i++ {
prime := <-ch
fmt.Printf("%v: %v\n", i+1, prime)
ch = PrimeFilter(ch, prime)
}
}
I have no idea what happen in this approach:
I know that can not print the content of channel without interrupt: Can not print content of channel
Size of channel: Default buffer channel size is 1, that mean:
By default channels are unbuffered, which states that they will only
accept sends (chan <-) if there is a corresponding receive (<- chan)
which are ready to receive the sent value
I can not image how above Go program run!
Could anybody please help to show me the step by step flow of above Go program for first 10 number or so?
This is a pretty convoluted example. In both functions, go func(){...}() creates an anonymous goroutine and runs it asynchronously, then returns the channel which will receive values from the goroutine. PrimeFilter returns a channel which will receive numbers not divisible by a certain candidate.
The idea is that prime := <-ch always takes the first element from the channel. So, to visualize the flow:
GenerateNatural() starts by sending numbers 2, 3, 4... to ch.
First loop iteration:
a. prime := <-ch reads the first (prime) number 2.
b. PrimeFilter(ch, 2) then continues receiving the rest of the numbers (3, 4, 5, ...), and sends numbers not divisible by 2 to the output channel. So, channel returned by PrimeFilter(ch, 2) will receive numbers (3, 5, 7, ...).
c. ch = PrimeFilter(ch, prime) in the main function now replaces the local ch variable with the output of PrimeFilter(ch, 2) from the previous step.
Second loop iteration:
a. prime := <-ch reads the first (prime) number from the current ch instance (this first number is 3).
b. PrimeFilter(ch, 3) then continues receiving the (already filtered) numbers, except for the first one (so, 5, 7, 9, ...), and sends numbers not divisible by 3 to the output channel. So, channel returned by PrimeFilter(ch, 2) will receive numbers 5, 7, 11, ..., because 9 is divisible by 3.
c. ch = PrimeFilter(ch, prime) in the main function now replaces the local ch variable with the output of PrimeFilter(ch, 3) from the previous step.
...
I'm new to the Go language and am currently going through the Go tour. I have a question regarding the concurrency example 5 on the select statement.
The code below has been edited with print statements to trace statements execution.
package main
import "fmt"
func fibonacci(c, quit chan int) {
x, y := 0, 1
fmt.Printf("Run fib with c: %v, quit: %v\n", c, quit)
for {
select {
case c <- x:
fmt.Println("Run case: c<-x")
x, y = y, x+y
fmt.Printf("x: %v, y: %v\n", x, y)
case <-quit:
fmt.Println("Run case: quit")
fmt.Println("quit")
return
}
}
}
func runForLoop(c, quit chan int) {
fmt.Println("Run runForLoop()")
for i := 0; i < 10; i++ {
fmt.Printf("For loop with i: %v\n", i)
fmt.Printf("Returned from c: %v\n", <-c)
}
quit <- 0
}
func main() {
c := make(chan int)
quit := make(chan int)
go runForLoop(c, quit)
fibonacci(c, quit)
}
The following is printed to the console.
Run fib with c: 0xc00005e060, quit: 0xc00005e0c0
Run runForLoop()
For loop with i: 0
Returned from c: 0 // question 1
For loop with i: 1
Run case: c<-x // question 2
x: 1, y: 1
Run case: c<-x // question 2
x: 1, y: 2
Returned from c: 1
For loop with i: 2
Returned from c: 1
For loop with i: 3
// ...
My questions are
The value of c is received here is 0 even though none of the select blocks have been executed. Could I confirm that this is the zero value of the c variable that has int type?
Why is case c<-x executed twice?
For 1: It prints the result of <-c, which will block until another goroutine writes to it. So your statement is not correct: the select case for c<-x ran, with x=0. It is not the zero value of the chan variable. You will only read the zero-value of the chan type from a channel if the channel is closed, or if you use the two-value form of channel read: value,ok := <-c. When ok=false, value is the zero-value of the channel value type.
For 2: c<-x will execute 10 times, because you read from it 10 times in the for-loop, and only then you are writing to quit, which will enable the second case of the select. What you observe here is the second iteration of the loop.
The following code implements the yield pattern in golang. As an experiment I was implementing an all permutations generator. However, when I return the slice A to channel, if I do not create a new copy of the array I get an incorrect result.
Please see the code around "???". Can someone explain what happens under the covers here? I thought that since the channel is not buffered, I was guaranteed that after publishing the array's slice to the channel I was ensured that the result would be consumed before continuing.
package main
import (
"fmt"
)
func swap(A []int, i int, j int) {
t := A[i]
A[i] = A[j]
A[j] = t
}
func recurse(A []int, c chan []int, depth int) {
if depth == len(A) {
// ??? Why do I need to copy the data?
// If I do c <- A I get an incorrect answer.
ra := make([]int, len(A))
copy(ra, A)
c <- ra
return
}
for i := depth; i < len(A); i++ {
swap(A, depth, i)
recurse(A, c, depth+1)
swap(A, depth, i)
}
}
func yieldPermutations(A []int, c chan []int) {
recurse(A, c, 0)
close(c)
}
func main() {
A := []int{1, 2, 3}
c2 := make(chan []int)
go yieldPermutations(A, c2)
for v := range c2 {
fmt.Println(v)
}
}
If I do not copy the data, I get the following result:
[1 3 2]
[1 3 2]
[2 3 1]
[2 3 1]
[3 1 2]
[3 1 2]
Obviously, the correct result (which we get with data copy) is:
[1 2 3]
[1 3 2]
[2 1 3]
[2 3 1]
[3 2 1]
[3 1 2]
It's a mistake to think this code is like generators/yield in Python, and that's what's causing your error.
In Python, when you request the next item from a generator, the generator starts executing and stops when the next yield <value> statement is reached. There is no parallelism in Python's generators: the consumer runs until it wants a value, then the generator runs until it produces a value, then the consumer gets the value and continues execution.
In your go code, the goroutine executes concurrently with the code that's consuming items. As soon as an item is read from the channel from the main code, the goroutine works concurrently to produce the next. The goroutine and the consumer both run until they reach the channel send/receive, then the value is sent from the goroutine to the consumer, then they both continue execution.
That means the backing array of A gets modified concurrently as the goroutine works to generate the next item. And that's a race condition which causes your unexpected output. To demonstrate that this is a race, insert time.Sleep(time.Second) after the channel send. Then the code produces the correct results (albeit slowly): https://play.golang.org/p/uEa_k6Brcc
I am learning how channels work in Go and have stumbled upon a problem with closing the channels. This is a modified example from A Tour of Go, which generates n-1 fibonacci numbers and sends them through the channels, leaving the last "element" of the channel capacity unused.
func fibonacci(n int, c chan int) {
x, y := 0, 1
for i := 0; i < n-1; i++ {
c <- x
x, y = y, x+y
}
// close(c) // It's commented out on purpose
}
func main() {
n := 10
c := make(chan int, n)
go fibonacci(n, c)
for i := 0; i < n; i++ {
_, ok := <-c
fmt.Println(ok)
}
}
The problem is that I get:
fatal error: all goroutines are asleep - deadlock!
when I do not close the channel. What exactly is causing the deadlock? Why can't I receive from the channel in its capacity boundaries when I don't close it?
You're writing n values into the channel (from 0 to n-1), but are trying to read n+1 values from the channel (from 0 to n). Without explicitly closing the channel, the main function will wait forever for this last value.
What exactly is causing the deadlock?
After n iterations, the goroutine running the fibonacci function will exit. After this goroutine has exited, the only remaining goroutine in your program is the main goroutine, and this goroutine is waiting for some data to be written to the c channel -- and since there is no other goroutine left that might ever write data into this channel, it will wait forever. This is exactly what the error message is trying to tell you: "all goroutines ("all" is just "one", here) are asleep".
The _, ok := <- c call in the main function will only stop blocking as soon as the c channel is closed (as reading from a channel is blocking, this needs to be done from another goroutine). When the channel is closed, the main function will read remaining data from the channel (when it's a buffered channel)
For loop in main expect n communication in channel, but you produce only n-1 in func fibonacci
func fibonacci(n int, c chan int) {
x, y := 0, 1
for i := 0; i < n; i++ { //here
c <- x
x, y = y, x+y
}
// close(c) // It's commented out on purpose
}
should work
http://play.golang.org/p/zdRuy14f9x