The following code keeps printing 0.
package main
import (
"fmt"
)
func main() {
c := make(chan int)
go func() {
for i := 0; i < 10; i++ {
c <- i // INPUT
}
close(c)
}()
for {
fmt.Print(<-c) // RECEIVER; OUTPUT
}
}
From my understanding, it should print 0 to 9 and then go on an infinite loop. Why does it keep printing zero?
Your understanding is (almost) correct, except that the infinite loop in the main goroutine will not block but will receive and print relentlessly.
The goroutine you start sends the numbers 0, 1, ... 9 on the channel, then closes it. And receiving from a closed channel does not block, on the contrary, it can proceed immediately, and it yields the zero value of the element type of the channel, which is 0 for the type int. This is stated in Spec: Receive operator:
Receiving from a nil channel blocks forever. A receive operation on a closed channel can always proceed immediately, yielding the element type's zero value after any previously sent values have been received.
So you see exactly what you should. First it prints the numbers 0..9, then keeps printing 0 very fast (without any delay), so probably you don't even notice the initial 0..9 numbers.
Slightly modifying your example so that the loop in the main goroutine exits after 15 iterations will immediately show what happens:
c := make(chan int)
go func() {
for i := 0; i < 10; i++ {
c <- i // INPUT
}
close(c)
}()
for i := 0; i < 15; i++ {
fmt.Print(<-c) // RECEIVER; OUTPUT
}
Output (try it on the Go Playground):
012345678900000
If your goal is to quit once all sent numbers have been processed (printed), use the for range on the channel:
for i := range c {
fmt.Print(i) // RECEIVER; OUTPUT
}
If your goal is to block until new numbers are available once those 10 numbers are processed, then don't close the channel. That way the next receive operation will block. In this case both your original loop and the for range loop would work, but for range is better as that always exits if / when the channel is closed.
Check out this question and memorize the channel axioms: How does a non initialized channel behave?
Related
i have been following the go tour examples, and i don't understand how this works
https://tour.golang.org/concurrency/5
package main
import "fmt"
func fibonacci(c, quit chan int) {
x, y := 0, 1
for {
select {
case c <- x:
x, y = y, x+y
case <-quit:
fmt.Println("quit")
return
}
}
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
fibonacci(c, quit)
}
how is this working?
and while i'm trying to understand.
package main
import "fmt"
func b(c,quit chan int) {
c <-1
c <-2
c <-3
}
func main() {
c := make(chan int)
quit := make(chan int)
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
b(c,quit)
}
this will sometimes print 1,2 sometimes print 1,2,3, why?
First, in the func fibonacci, the select statement tries select the first thing of the two following to finish:
c <- x
<- quit
It is fairly easy to understand <- quit, which is tries to receive a value from a channel called quit (and ignores the value received).
c <- x means sending a value that equals (is a copy of) x. It seems like unblocking, but in Go, sending over an unbuffered channel (which is explained in the Go tour) blocks when there is no receiver.
So here it means, waiting for a receiver to be ready to receive the value (or a space in the buffer, if it were a buffered channel), which in this code means fmt.Println(<-c), and then send the value to the receiver.
So this statement unblocks (finishes) whenever <-c is evaluated. That is, every iteration of the loop.
And for your code, while all value 1, 2, 3, is guaranteed to be sent over the channel (and received), func b returns and thus func main retains without guaranteeing the fmt.Println(3) finishes.
In Go, when the func main returns, the program terminates, and unfinished goroutine does not get a chance to finish its work - thus sometimes it prints 3 and soetimes it does not.
To better understand the Fibonacci example let's analyze the different parts.
First the anonymous function
go func() {
for i := 0; i < 10; i++ {
fmt.Println(<-c)
}
quit <- 0
}()
The "go" keyword will start a new goroutine, so now we have the "main" goroutine and this one, they're gonna be running concurrently.
The loop is telling us that we are going to execute this 10 times:
fmt.Println(<-c)
This call will block until we receive an integer from the channel and print it. Once this happens 10 times we are gonna signal that this goroutine has finished
quit <- 0
Now let's go back to the main goroutine, while the other goroutine was starting it has invoked the function "finbonacci", there's no "go" keyword so this call is blocking here.
fibonacci(c, quit)
Now let's analyze the function
x, y := 0, 1
for {
select {
case c <- x:
x, y = y, x+y
case <-quit:
fmt.Println("quit")
return
}
}
We start an infinite loop and on each iteration, we will try to execute one case from the select
The first case will try to send Fibonacci's sequence values indefinitely to the channel, at some point, once the other goroutine reaches the fmt.Println(<-c) statement, this case will be executed, the values will be recalculated and the next iteration of the loop will happen.
case c <- x:
x, y = y, x+y
The second case won't be able to run yet, since nobody is sending anything to the "quit" channel.
case <-quit:
fmt.Println("quit")
return
}
After 10 iterations the first goroutine will stop receiving new values, thus the first select case won't be able to run
case c <- x: // this will block after 10 times, nobody is reading
x, y = y, x+y
At this point, no case in the "select" can be executed, the main goroutine is "blocked" (more like paused from a logical point of view).
Finally, at some point, the first goroutine will signal that it has finished using the "quit" channel
quit <- 0
This allows the execution of the second "select" case, which will break the infinite loop and allow the "fibonacci" function to return and the main function will be able to finish in a clean way.
Hopefully, this helps you understand the Fibonacci example.
Now, moving to your code, you are not waiting for the anonymous goroutine to finish, in fact, it's not even finishing. Your "b" method is sending "1,2,3" and returns immediately.
Since it's the last part of the main function, the program terminates.
If you only see "1,2" sometimes is because the "fmt.Println" statement is too slow and the program terminates before it can print the 3.
I am learning basic Go via a Udemy course. In the goroutines section, there is an example of throttling which has thrown my understanding of how wait groups work.
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
func main() {
c1 := make(chan int)
c2 := make(chan int)
go populate(c1)
go fanOutIn(c1, c2)
for v := range c2 {
fmt.Println(v)
}
fmt.Println("about to exit")
}
func populate(c chan int) {
for i := 0; i < 100; i++ {
c <- i
}
close(c)
}
func fanOutIn(c1, c2 chan int) {
var wg sync.WaitGroup
const goroutines = 10
wg.Add(goroutines)
for i := 0; i < goroutines; i++ {
go func() {
for v := range c1 {
func(v2 int) {
c2 <- timeConsumingWork(v2)
}(v)
}
wg.Done()
}()
}
wg.Wait()
close(c2)
}
func timeConsumingWork(n int) int {
time.Sleep(time.Microsecond * time.Duration(rand.Intn(500)))
return n + rand.Intn(1000)
}
The part that isn't inline with my understanding is in the function fanOutIn where the we set up the WaitGroup, and Add(10).
Why am I getting 100 values printed out? Only a single value (i := 0) can be put onto c1, and the value is never explicitly removed from the channel. The code then hits wg.Done(), and the wait group queue is reduced to 9 and so on.
In my current understanding, I would expect to see 10 values, of 0 + rand.Intn(1000).
The function that is spun off reads as follows (including the go at the front and the parentheses to call it):
go func() {
for v := range c1 {
func(v2 int) {
c2 <- timeConsumingWork(v2)
}(v)
}
wg.Done()
}()
This code is a little weird and bizarre. Let's shrink it down even further, discarding the wg.Done and keeping only the for loop itself:
for v := range c1 {
func(v2 int) {
c2 <- timeConsumingWork(v2)
}(v)
}
There is an inner unnamed function that is pretty useless here; we can discard it without changing the behavior of the program, to get:
for v := range c1 {
c2 <- timeConsumingWork(v)
}
which is at last a simple loop. One key question is now this: How many iterations do you expect from this loop? Note: It is not necessarily any constant number. Perhaps a better way to phrase the question is: When does this loop end?
The for loop reads a channel. This kind of loop ends when a read from the channel indicates that there is no more data, i.e., that the channel is closed and its queue is empty. (See the Go specification section on for loops.)
So this innermost loop, for v := range c1, is not going to terminate until channel c1 is closed and there is no more data in its queue. This channel was created with:
c1 := make(chan int)
so it has no queue, so we need not even think about that: it terminates after a close(c1) closes it. You should now look for a close that closes c1.
Where is our close?
Here's the place that closes c1:
func populate(c chan int) {
for i := 0; i < 100; i++ {
c <- i
}
close(c)
}
We call this with c1 as its argument, so its final close(c) closes c1. Now you can ask: When do we reach this close call? The answer is obvious: after i >= 100 in the loop, i.e., after we have sent 100 values, zero through 99 respectively, into channel c1.
What fanOutIn does is spin off 10 goroutines. Each of the 10 goroutines runs the first anonymous function I quoted above. That anonymous function has a loop that runs an indeterminate number of times, repeating until channel c1 is closed. Each trip through the loop takes a value of the channel, so initially, if the ten goroutines all manage to start before there are any values available, all ten goroutines will be waiting for values.
When the producer function puts one value into the channel, one of the ten waiting goroutines will get it and begin using it. If that goroutine takes a long time to get back to the top of its own for loop, another goroutine will take the next-produced value. So what happens here is that up to ten produced values propagate through the channel to up to ten goroutines.1 Each of those (up to ten) goroutines spends some nontrivial amount of time using its value, then sends a final-product-value to channel c2 and goes back to the top of its own indefinite for loop.
Only when the producer has closed its channel c (which is our c1 here) will the ten goroutines see a closed-channel-empty-queue, allowing them to exit their for loops. When they do exit their for loops, each of them will call wg.Done() (once each) and terminate.
So, once the close(c1) has occurred (via the close(c) in populate), eventually all ten of these anonymous goroutines will have called wg.Done(). At that point, the wg.Wait() in fanOutIn will return. This will call close(c2) and return from fanOutIn, terminating that goroutine as well.
Meanwhile, in main, we use for v := range c2 to read from channel c2. This for loop will run when values are written into c2 by any of the ten goroutines. It will exit only when c2 itself is closed (its queue must also be empty but again c2 has a zero-length queue). So main will not proceed past the for loop until c2 is closed, which cannot happen until wg.Wait() returns, which cannot happen until ten wg.Done() calls have happened, which cannot happen until channel c1 is closed.
This means that main cannot get past its own for loop until populate has called close(c), and that happens only after generating exactly 100 values.
1As discussed in comments below, the phrase up to here can be important: we don't really know how many goroutines will really consume values. A lot depends on how much work each goroutine does, what kind of work that is, and how many CPUs your Go runtime has available.
I am stuck in a strange situation where the write operation to the channel never happens.
package main
import (
"fmt"
"time"
)
func main() {
c := make(chan int)
s := make(chan bool)
k := make(chan bool)
fmt.Println("start")
go func() {
fmt.Println("routine 1")
s <- true
}()
go func() {
fmt.Println("routine 2")
for {
select {
case <-s :
for i := 0; i < 3; i++ {
fmt.Println("before")
c <- i
fmt.Println("after")
}
case <- k :
fmt.Println("k ready")
break
}
}
}()
go func() {
fmt.Println("routine 3")
for {
select {
case x := <- c :
fmt.Println("x=", x)
k <- true
fmt.Println("k done")
}
}
}()
time.Sleep(1000 * time.Millisecond)
}
And here is the output:
start
routine 1
routine 2
before
routine 3
x= 0
after
before
I wonder why the write to the channel k blocks, but the log statement fmt.Println("k ready") is never printed.
Here is what I think :
go routine 1 writes true to channel s
go routine 2 writes 0 to
channel c and waits because buffer size is 0, it will not be able to
write '1' to it unless someone reads channel c
go routine 3 comes into the picture, reads channel c (now go routine 2
can write to c once go routine 2 resumes) prints the value of x. NOW IT SHOULD BE ABLE TO WRITE TO CHANNEL K but that is not happening
According to me it should be able to write to channel k then case 2 of goroutine should execute and print "k ready"
Can anyone explain me why write to the channel blocked?
As a fix I know I can increase the buffer size of channel c and everything will get printed but I am not interested in fixing this, instead I want to understand this scenario.
A nice blog to understand the above case.
You have a deadlock.
goroutine 1 writes to s then quits
goroutine 2 reads from s, and writes to c
goroutine 3 reads from c, and writes to k, and this blocks because nothing is reading from k, because goroutine 2 is blocked in the write to k above.
goroutine 2 writes to c again which blocks as goroutine 3 is still trying to write to k and thus is not reading from c
Contrary to what you say, you don't have a buffer size of 1. You have a buffer size of zero (i.e. an unbuffered channel), so a write will block until something reads. This is probably the source of your misunderstanding. Per the language specification:
A new, initialized channel value can be made using the built-in function make, which takes the channel type and an optional capacity as arguments:
make(chan int, 100)
The capacity, in number of elements, sets the size of the buffer in the channel. If the capacity is zero or absent, the channel is unbuffered and communication succeeds only when both a sender and receiver are ready. Otherwise, the channel is buffered and communication succeeds without blocking if the buffer is not full (sends) or not empty (receives). A nil channel is never ready for communication.
I am learning how channels work in Go and have stumbled upon a problem with closing the channels. This is a modified example from A Tour of Go, which generates n-1 fibonacci numbers and sends them through the channels, leaving the last "element" of the channel capacity unused.
func fibonacci(n int, c chan int) {
x, y := 0, 1
for i := 0; i < n-1; i++ {
c <- x
x, y = y, x+y
}
// close(c) // It's commented out on purpose
}
func main() {
n := 10
c := make(chan int, n)
go fibonacci(n, c)
for i := 0; i < n; i++ {
_, ok := <-c
fmt.Println(ok)
}
}
The problem is that I get:
fatal error: all goroutines are asleep - deadlock!
when I do not close the channel. What exactly is causing the deadlock? Why can't I receive from the channel in its capacity boundaries when I don't close it?
You're writing n values into the channel (from 0 to n-1), but are trying to read n+1 values from the channel (from 0 to n). Without explicitly closing the channel, the main function will wait forever for this last value.
What exactly is causing the deadlock?
After n iterations, the goroutine running the fibonacci function will exit. After this goroutine has exited, the only remaining goroutine in your program is the main goroutine, and this goroutine is waiting for some data to be written to the c channel -- and since there is no other goroutine left that might ever write data into this channel, it will wait forever. This is exactly what the error message is trying to tell you: "all goroutines ("all" is just "one", here) are asleep".
The _, ok := <- c call in the main function will only stop blocking as soon as the c channel is closed (as reading from a channel is blocking, this needs to be done from another goroutine). When the channel is closed, the main function will read remaining data from the channel (when it's a buffered channel)
For loop in main expect n communication in channel, but you produce only n-1 in func fibonacci
func fibonacci(n int, c chan int) {
x, y := 0, 1
for i := 0; i < n; i++ { //here
c <- x
x, y = y, x+y
}
// close(c) // It's commented out on purpose
}
should work
http://play.golang.org/p/zdRuy14f9x
I'm a Go learner. In order better to understand the care and feeding of channels and goroutines, I'm trying to build a Sieve of Eratosthenes as a set of goroutines connected into a pipeline by channels.
Here's what I have so far:
// esieve implements a Sieve of Eratosthenes
// as a series of channels connected together
// by goroutines
package main
import "fmt"
func sieve(mine int, inch chan int) {
start := true // First-number switch
ouch := make(chan int) // Output channel for this instance
fmt.Printf("%v\n", mine) // Print this instance's prime
for next := <-inch; next > 0; next = <-inch { // Read input channel
fmt.Printf("%v <- %v\n",mine,next) // (Trace)
if (next % mine) > 0 { // Divisible by my prime?
if start { // No; is it the first number through?
go sieve(next, ouch) // First number - create instance for it
start = false // First time done
} else { // Not first time
ouch <- next // Pass it to the next instance
}
}
}
}
func main() {
lim := 30 // Let's do up to 30
fmt.Printf("%v\n", 2) // Treat 2 as a special case
ouch := make(chan int) // Create the first segment of the pipe
go sieve(3, ouch) // Create the instance for '3'
for prime := 3; prime < lim; prime += 2 { // Generate 3, 5, ...
fmt.Printf("Send %v\n", prime) // Trace
ouch <- prime // Send it down the pipe
}
}
And as far as it goes, it works nicely.
However, when I finish the main loop, main exits before all the numbers still in the pipeline of sieve instances have propagated down to the end.
What is the simplest, most elegant, or generally accepted way to make a main routine wait for a set of goroutines (about which it only 'knows' of the first one) to complete?
As for your title question, killing worker goroutines when you don't need them anymore:
You could use the Done idiom. Reads from a closed channel yield the zero value.
Make a new channel done. When reads from this channel succeed, the goroutines know they should quit. Close the channel in main when you have all the values you need.
Check if you can read from a channel done, and exit by returning, or read from next when that's available. This partially replaces the assignment to next in you for loop:
select {
case <-done:
return
case next = <- inch:
}
Ranging over a channel also works, since closing that channel exits the loop.
As for the reverse, your body question, waiting for a set of goroutines to finish:
Use sync.WaitGroup.
var wg sync.WaitGroup
wg.Add(goroutineCount)
And when each goroutine finishes:
wg.Done()
Or use defer:
defer wg.Done()
To wait for all of them to report as Done:
wg.Wait()
In your example, simply call wg.Add(1) when you start a new goroutine, before you call wg.Done() and return. As long as you only reach zero once, wg.Wait() works as expected, so wg.Add(1) before wg.Done.
After #izca unblocked my logjam, and after a few false starts involving deadlocks when everything finished, here's my solution working correctly:
// esieve implements a Sieve of Eratosthenes
// as a series of channels connected together
// by goroutines
package main
import "fmt"
func sieve(mine int, // This instance's own prime
inch chan int, // Input channel from lower primes
done chan int, // Channel for signalling shutdown
count int) { // Number of primes - counter
start := true // First-number switch
ouch := make(chan int) // Output channel, this instance
fmt.Printf("%v ", mine) // Print this instance's prime
for next := <-inch; next > 0; next = <-inch { // Read input channel
if (next % mine) > 0 { // Divisible by my prime?
if start { // No; first time through?
go sieve(next, ouch, done, count+1) // First number,
// create instance for it
start = false // First time done
} else { // Not first time
ouch <- next // Pass to next instance
}
}
}
if start { // Just starting?
close(done) // Yes - we're last in pipe - signal done
print("\n",count," primes\n") // Number of primes/goroutines
} else {
close(ouch) // No - send the signal down the pipe
}
}
func main() {
lim := 100 // Let's do up to 100
done := make(chan int) // Create the done return channel
ouch := make(chan int) // Create the first segment of the pipe
go sieve(2, ouch, done, 1) // Create the first instance for '2'
for prime := 3; prime < lim; prime += 1 { // Generate odd numbers
ouch <- prime // Send numbers down the pipe
}
close(ouch) // Send the done signal down the pipe
<- done // and wait for it to come back
}
I'm tremendously impressed with the elegance and simplicity of Go for this kind of programming, when compared with many other languages. Of course, the warts I claim for myself.
If appropriate here, I'd welcome critical comments.