Is gorouines ignore channel's buffer size - go

Environment: OS X 10.8, Go 1.0.2
I make a channel with buffer-size 2, then if I write channel three times, it will throw error:
throw: all goroutines are asleep - deadlock!
Of course, it's correct.
BUT if I write channel four or more times in the goroutines, it works fine, why?
The channel's capacity is 2, why goroutines ignore that or forget the capacity setting?
I comment the read-channel codes, so no one will read channel and save the capacity. I also use time.Sleep to waiting for all goroutines finish their work.
please review following codes:
package main
//import "fmt"
func main() {
c := make(chan int, 2)
/*c <- 1
c <- 2
c <- 3*/
for i:=0; i<4; i++ {
go func(i int) {
c <- i
c <- 9
c <- 9
c <- 9
}(i)
}
time.Sleep(2000 * time.Millisecond)
/*for i:=0; i<4*2; i++ {
fmt.Println(<-c)
}*/
}
Would anyone please give some hits? thanks, guys.

When a channel is buffered, this means it will not block until the buffer is full. Once the buffer is full, the sending goroutine will block as it tries to add things to the channel.
This means that this will block:
c := make(chan int)
c <- 1 // Block here, this is unbuffered !
println(<-c)
And this will also block:
c := make(chan int, 2)
c <- 1
c <- 2
c <- 3 // Block here, buffer is full !
println(<-c)
But the point of goroutines and channel is precisely to run things concurrently, so this will work:
c := make(chan int)
go func() { c <- 1; }() // This will block in the spawned goroutine until...
println(<-c) // ... this line is reached in the main goroutine
And similarly:
c := make(chan int, 2)
go func() { // `go ...` spawns a goroutine
c <- 1 // Buffer is not full, no block
c <- 2 // Buffer is not full, no block
c <- 3 // Buffer is full, spawned goroutine is blocking until...
}()
println(<-c) // ... this line is reached in the main goroutine
In your example, you spawn four different goroutines, which all write four numbers to the same buffered channel. As the buffer is 2 < 16, they will end up blocking
But the crux of the matter is that the Go policy is to wait only for the main goroutine:
Program execution begins by initializing the main package and then invoking the function main. When the function main returns, the program exits. It does not wait for other (non-main) goroutines to complete.
This means that in your first example, the main goroutine was blocking when it reached line c <- 3. As no other goroutine was able to do anything which could potentially unblock it, the runtime detected that the program was deadlocked and reported an error.
In your second example however, spawned goroutines block, while the main continues quietly until it reaches the end of its execution, at which point all the (blocked) spawned goroutines are silently killed, and no error is reported.

val has given a good answer. I'd like to add an extra tip I've found useful.
When learning to use goroutines, use zero-buffered channels to start with. This way, when you make a mistake, you'll immediately get a deadlock, which you can learn from. You need to learn how to write code that doesn't deadlock, which means learning tricks like not having cyclic dependencies in the client-server relationships (assuming your goroutines are written as clients or servers).
Reasoning about the network without buffering is simpler, although this might not be obvious at first.
Buffering is really useful, but should be considered as a means to enhance performance.

Related

Why is the order of channels receiving causing/resolving a deadlock in Golang?

I've boiled my issue down to this simple example below. I am invoking a goroutine that takes two channels and sends one message to each. Then I am attempting to receive those messages further along. However, the order of channels receiving matters. If I use the same order I sent the messages, the program runs. If I switch, it does not.
I would have expected the goroutine to run independently from retrieving the messages, allowing me to receive from whichever channel I wanted to first.
I can solve this by sending messages to a single channel per goroutine (2 goroutines).
Could someone explain why there is an order dependence here and why 2 separate goroutines resolves that dependence?
package main
import "fmt"
func main() {
chanA := make(chan string)
chanB := make(chan string)
go func() {
chanA <- "el"
chanB <- "el"
}()
// if B is received before A, fatal error
// if A is received before B, completes
<-chanB
<-chanA
fmt.Println("complete")
}
You will need to buffer your channels. A buffered channel can store so many elements before it will block.
chanA := make(chan string, 1)
chanA <- "el" // This will not block
fmt.Println("Hello World")
When you do chanA <- "el" on the buffered channel above, the element gets placed into the buffer and the thread does not block. If you add a second element, it will then block as there is no room in the buffer:
chanA := make(chan string, 1)
chanA <- "el"
chanA <- "el" // <- This will block, as the buffer is full
In your example, you have a buffer of 0. So the first write to the channel is blocked, and requires another thread to read the value to unblock.
https://go.dev/play/p/6GbsVW4d0Mg
chanA := make(chan string)
go func() {
time.Sleep(time.Second)
fmt.Println("Pop:", <-chanA) // Unblock the writer
}()
chanA <- "el"
Extra knowledge
If you do not want a thread to block, you can wrap a channel insert in a select. This will ensure if the channel is full, your application does not deadlock. One cheap way of fixing this is a larger buffer...
https://go.dev/play/p/kKR-lrCO4FX
select {
case chanA <- "el":
default:
return fmt.Errorf("value not written: %s", value)
}
this is how goroutine works:
a goroutine will be blocked read/write on a channel unless if find another goroutine which write/read from the same channel.
Pay attention to read/write and write/read in the above blocked quote.
In your case, your anon goroutine(which you kicked off with go) waits to write on channelA until it finds a goroutine which reads from channelA.
The main goroutine waits to read from channelB unless it finds a goroutine that reads from it.
You can think it this way, any line written after read/write to channel won't be considered unless go finds another routine which write/read from the same channel.
So, if you change either read or write order you will not have deadlock or as you said another goroutine will do the job too.
Hope it's clear.
A write to or read from an unbuffered channel will block until there is a goroutine to write to or read from that channel. When the goroutine writes to a, it will block until the main goroutine can read from a, but main goroutine is also blocked waiting to read from b, hence deadlock.

Go workflow when only sending values to channel

I am learning channels in Go by following this tutorial.
When I only send value to a channel it gives error. Here is the example code.
package main
import "fmt"
func main() {
ch := make(chan int)
ch <- 1
fmt.Println("Does not work")
}
Here I am just sending value to the channel but not receiving anything. It give an error
fatal error: all goroutines are asleep - deadlock!
But when I run following code it doesn't give any error
package main
import "fmt"
func sum(s []int, c chan int) {
sum := 0
for _, v := range s {
sum += v
}
c <- sum // send sum to c
}
func main() {
s := []int{7, 2, 8, -9, 4, 0}
c := make(chan int)
go sum(s[:len(s)/2], c)
go sum(s[len(s)/2:], c)
fmt.Println("did not receive but still works")
}
and prints
did not receive but still works
I couldn't understand why it works in second case while it does not work in first case. Even though I have not received any value for the channel in both cases. Also what is causing the deadlock in first case and how it gets avoided in second case?
Neither example works. In fact, no example that only sends to a channel will ever work, in any traditional sense of "work".
But here's a step through of each, to make it clear:
First example
ch := make(chan int)
This creates an unbuffered channel. Unbuffered channels don't hold any data, they only act as a conduit for communication--all data sent must be received by something else, before the program execution proceeds--on either side of the channel.
ch <- 1
Here you send data on the channel, but nothing is waiting to receive it, so the program waits. In this case, it waits forever, because you never created a receiver for the channel, thus your deadlock.
Second example
c := make(chan int)
Again, create an unbuffered channel.
go sum(s[:len(s)/2], c)
Call the sum function, which, incidentally, will also block forever, for the reasons described above--nothing is receiving on the channel, so it will wait forever. However, in this case, you've called it in a goroutine. The goroutine will run in a separate execution thread, while other bits of the program run. Although, due to never receiving data from the channel, this goroutine will never exit, until the main program exits.
go sum(s[len(s)/2:], c)
Again, you call sum, and again, in a goroutine. So at this point, you have three gorotuines: One running main(), and one each running an invocation of sum(). The latter two will never exit.
Then your program exits. When the program exits, all goroutines (including the two that are stuck forever waiting on your channel) exit.
Since this program exits immediately, the deadlock is never reported, but it does assuredly exist, same as in your first example.

golang channels deadlock before receiving values

I'm not understanding why this doesn't work https://play.golang.org/p/_ALPii0pXV6 but this https://play.golang.org/p/vCOjAr-o54e works.
As I understand the goroutine asynchronously sends to value true to a and 12 to b. While in the main function, a is blocked, until it receives a value. Why is it that when I rearrange it to have b is blocked before a, it results in a deadlock?
Go channels are unbuffered by default. That means that it cannot send on a channel until the receiver is reading the channel. This is actually the Go preferred mode. It's more efficient than buffered channels in most cases.
What that means for your first code is that the goroutine cannot proceed to write to channel b until it completes the write to channel a. It cannot do that until the main goroutine reads a.
Go by Example explains that, by default, channel sending and receiving waits until both the sending routine and the receiving routine are ready. This blocking is made obvious by the following example:
func main() {
ch := make(chan int)
ch <- 1
fmt.Println(<-ch)
}
This code results in a deadlock because the only goroutine (the main one) is stuck at ch <- 1, waiting for another goroutine to receive. Little does it know that we are expecting it to be the receiver at the next line.
This explains why your first example does not work, because the other goroutine doesn't send on b until its send operation on a has completed. But the main routine won't receive on a until it's received on b! So both are stuck waiting forever.
To read more about this kind of operation (called a synchronous operation), check out this explanation.
If you rewrite the code the way it is going to be executed sequentially then it becomes clearer what's going on.
Original code:
func main() {
a := make(chan bool)
b := make(chan int64)
go func(a chan bool, b chan int64) {
fmt.Println("Here1")
a <- true
b <- 12
} (a,b)
fmt.Println("Here2")
fmt.Println(fmt.Sprintf("%d", <-b))
fmt.Println(fmt.Sprintf("%v", <-a))
}
Close representation of sequential execution of the same code:
a := make(chan bool)
b := make(chan int64)
fmt.Println("Here2") // Prints
// Pass control to new goroutine
fmt.Println("Here1")
a <- true // Write to channel a and block goroutine here and pass control to main
fmt.Println(fmt.Sprintf("%d", <-b)) // Tries to read from b but nothing has been written to it so blocks. At this point all your goroutines are blocked hence the deadlock.
fmt.Println(fmt.Sprintf("%v", <-a)) // doesn't even reach here.
b <- 12
}

"Consume or put back" Go channels [duplicate]

This question already has answers here:
Go channels and deadlock
(3 answers)
Closed 7 years ago.
I am trying to have two separate consumer go routines, that would filter out even and odd numbers from the input channel. This is just a toy example in order to see if it is possible to have consumers do something with the message read from the input channel if it matches certain condition, otherwise put back onto the input channel.
My current code is as follows:
package main
func filterOdd(ch chan int, out chan int) {
val := <- ch
if val % 2 == 0 {
ch <- val
} else {
out <- val
}
}
func filterEven(ch chan int, out chan int) {
val := <- ch
if val % 2 != 0 {
ch <- val
} else {
out <- val
}
}
func main() {
even := make(chan int)
odd := make(chan int)
input := make(chan int)
go filterOdd(input, odd)
go filterEven(input, even)
for i:=1; i <= 10; i++ {
input <- i
}
println("Even...")
for i := range even {
println(i)
}
println("Odd...")
for i := range odd {
println(i)
}
}
However, this produces the following output:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/tmp/sandbox594577124/main.go:27 +0x140
goroutine 4 [chan send]:
main.filterOdd(0x10336100, 0x103360c0)
/tmp/sandbox594577124/main.go:8 +0xc0
created by main.main
/tmp/sandbox594577124/main.go:24 +0xc0
Link to the Go Playground: https://play.golang.org/p/9RIvFsGKI-
You have a deadlock because your even and odd goroutines are blocked on sending to out because nothing is reading from it. Why is nothing reading out? Because the main goroutine is blocked on sending to input because nothing is reading from it. Why is nothing reading from input? Because the two goroutines that would read from it are blocked.
Also, both filterEven and filterOdd will only run once unless you wrap their contents in for { } (but then they'll never stop until you break). On the other hand, range even will block (and range odd never happens) when nothing is left to write to even , because range over a channel only stops when the channel is closed or break is called.
In general, these aren't hard problems to solve as long as you know when you can close a channel. With what you're describing, that gets more difficult. None of the goroutines know when it's OK to close input, because all three write to it and two also read from it. You can use a sync.WaitGroup to make sure that everything you've put into the input channel has been processed before closing it. Once it's closed, the two other goroutines can use that as a signal to close their own channels and break or return to finish running.
However, writes to the in and out channels will still block until there is a corresponding read, because they are unbuffered. However, if you buffer them by specifying a size as the second argument to make, writes won't block until the channel is full. Since you know neither even or odd will have more written to them than what main send to input, you can use that as a safe buffer capacity.
Here's an example of using a WaitGroup with buffered channels for your code: https://play.golang.org/p/VXqfwUwRcx
If you don't want buffered channels, you can also use another pair of goroutines to capture the values and send them back to main as slices once finished. This way writes on the even and odd channels don't block: https://play.golang.org/p/i5vLDcsK1v
Otherwise, if don't need to print the contents of each channel all at once, you can use those two extra goroutines to read from the channels and print right away: https://play.golang.org/p/OCaUTcJkKB

Buffered/Unbuffered channel

Could someone explain, why if the channel is buffered the program doesn't exit with a fatal_error?
Unbuffered channel
package main
func main() {
c := make(chan int)
c <- 3
}
fatal error: all goroutines are asleep - deadlock!
Buffered channel
package main
func main() {
c := make(chan int, 1)
c <- 3
}
[no output]
Program exited.
Thank you!
Writing to a buffered channel doesn't block if there is room in the buffer.
If you try to put two items in the channel with a buffer size of one, you get the same error:
package main
func main() {
c := make(chan int, 1)
c <- 3
c <- 4
}
gives you:
fatal error: all goroutines are asleep - deadlock!
It's a core concept of Go's channels (or other CSP implementations such as Clojure's core.async library) that they are blocking. In general, as you already mentioned, there're two types of channels:
buffered which block if the buffer is full.
unbuffered which block if there's no "rendezvous", i.e. there must be someone who puts (c <-) to and someone who takes (<- c) from the channel.
In your particular case the Go runtime is smart enough to detect that there's no one who will ever take 3 from channel c. Hence, it's a deadlock and (thankfully) an error is thrown.
What you typically do when you're working with channels is using goroutines (checkout this introduction) which spawn a lightweight thread—managed by the Go runtime—to execute the body concurrently:
c := make(chan int)
go func() { c <- 3 }() // Create a new gorountine that puts 3 to the channel
fmt.Println(<- c) // Take 3 from the channel and print it in the main thread
Thanks #Matt
I found the answer in this post How does make(chan bool) behave differently from make(chan bool, 1)? :
Actually that's the reason why your problem is generated. Un-buffered channels are only writable when there's someone blocking to read from it, which means you shall have some coroutines to work with -- instead of this single one.

Resources