I am trying to work with go streams and I have a few "stupid" question.
I have done a primitive stream example with the byte limit range and here is the working code and here are my questions.
1 - why this code shows 1 and 2 at the new line? Why doesn't it show 12 at the one? Does the first peace of bytes removed from the byte-limited stream? (But how we can push the 2 number into the stream when we have already pushed the 1 number?) I just can't understand it
package main
import "fmt"
func main() {
ch := make(chan int, 2)
ch <- 1
ch <- 2
fmt.Println(<-ch)
fmt.Println(<-ch)
}
It shows:
1
2
2 question - I have tried to play with this code to understand how it works and I have removed the byte range and I have got the deadlock error. Why does it happen? Thanks!
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/tmp/sandbox557775903/main.go:7 +0x60
The deadlock error code:
package main
import "fmt"
func main() {
ch := make(chan int)
ch <- 1
ch <- 2
fmt.Println(<-ch)
fmt.Println(<-ch)
}
Thanks for any help! Sorry for the primitive questions.
Because channel operator <- takes only 1 element from the channel. If you want them printed together try: fmt.Println("%v%v", <-c, <-c)"
Last number in channel creation make(chan int, 2) means channel buffer - its capacity to store items. So you can easily push 2 items to channel. But what happens if you try to push one more item? The operation would be blocked because there's no space until another goroutine will read from the channel and free space.
The same applies to all channels - unbuffeted ones get blocked at first element writing. Until some goroutine reads from the channel.
Because there's no goroutine to read, writing one locks forever. You may solve it starting a reading goroutine before.
c := make(chan int)
go func () {
fmt.Println(<-c)
fmt.Println(<-c)
}()
ch <- 1
ch <- 2
This way would not get locked but start transmitting items.
1 - why this code shows 1 and 2 at the new line ?
Answer: because you use Println() method. If you want them on one line use Print()
2 I have tried to play with this code to understand how it works and I have removed the byte rande and I have got the deadlock error. Why it happens?
As far as the code shows, you are never firing up a concurrent reader for the channel you create. Because it is unbuffered, any writes to it, will block until someone, somewhere reads from the other end.
Related
This seems to challenge my understanding of unbuffered channel, which is that it can only take one value and then it would block for a reader to read it.
How in the following code writeToChan is able to write 3 values?
More so surprisingly, how are those values available to be read later albeit not in same order?
An excerpt from https://golang.org/doc/effective_go#channels
Receivers always block until there is data to receive. If the channel is unbuffered, the sender blocks until the receiver has received the value. If the channel has a buffer, the sender blocks only until the value has been copied to the buffer; if the buffer is full, this means waiting until some receiver has retrieved a value.
package main
import (
"fmt"
"time"
)
func main() {
ch := make(chan int)
go writeToChan(ch)
go rdFrmChan(ch)
x := <- ch
fmt.Println("main read 1 -:",x)
fmt.Println("main read 2 -:",<-ch)
}
func writeToChan(c chan int) {
time.Sleep(time.Second * 1)
c <- 42
c <- 27
c <- 9
}
func rdFrmChan(c chan int) {
fmt.Println("Go routine read :", <-c)
}
Output:
Go routine read : 27
main read 1 -: 42
main read 2 -: 9
Playground link: https://play.golang.org/p/DYzfYh-kXnC
Each line of the excerpt pasted is proven by your example code, if you understand the sequence of events happening.
After the goroutines are started, your main routine is blocked reading from the channel c, as it is yet to see a value to read. The writeToChan routine waits for a second before writing the first value to the channel
The goroutine rdFrmChan is also blocked, because it is waiting to read on the channel ch
After 1s, when the sleep on writeToChan expires, the first write (c <- 42) will unblock your main routine first, causing the value to be stored in x i.e. 42
Next the rdFrmChan is unblocked on the next write to the channel (c <- 27) and sees the value 27. The routine terminates at this point after printing the value
At this point, there is only value to be written and one to be read. The third write (c <- 9) from the goroutine allows the main routine to read the value as part of <-ch and print it
The following code logged an error:
fatal error: all goroutines are asleep - deadlock!
package main
import "fmt"
func main() {
ch := make(chan int)
ch <- 1
fmt.Println(<-ch)
}
But when I changed the code into this:
package main
import "fmt"
func assign (ch chan int) {
ch <- 1
}
func main() {
ch := make(chan int)
go assign (ch)
fmt.Println(<-ch)
}
"1" was printed out.
Then I used buffered channels:
package main
import "fmt"
func main() {
ch := make(chan int, 2)
ch <- 1
ch <- 2
fmt.Println(<-ch)
fmt.Println(<-ch)
}
"1" and "2" can also be printed out.
I'm a little confused about the situation. Thanks in advance!
Why the deadlock happened:
In the first code snippet you have only one main goroutine and it is blocked when you are trying to write into the channel here:
ch <- 1
Because nobody reads from the channel and the main goroutine is waiting for this to continue.
See Effective Go -> Channels
If the channel is unbuffered, the sender blocks until the receiver has received the value.
The sender is main function, the receiver is also main function.
How to avoid the deadlock:
In order to solve this, you have two options:
Option 1: make the ch channel buffered like this:
ch := make(chan int, 1) // buffer length is set to 1
From A Tour of Go
Sends to a buffered channel block only when the buffer is full.
So, you can write to the channel until the buffer is full. Then somebody has to start reading from the channel.
Option 2: write to the channel from a goroutine, like you did in the second code snippet:
func assign(ch chan int) {
ch <- 1
}
func main() {
ch := make(chan int)
go assign(ch) // does not block the main goroutine
fmt.Println(<-ch) // waiting to read from the channel
}
In this case main function will be executed until fmt.Println(<-ch) and continues as soon as it can read from the channel.
When you are using unbuffered channel, goroutine is blocked during write until someone does the read.
In your first snippet, there is an unbuffered channel and single goroutine (main goroutine).
So when you are trying to write:
ch <- 1
Nobody reads from the channel yet. The main goroutine is blocked and this line is never executed:
fmt.Println(<-ch)
That's why you've got the deadlock error.
In the second example, you still using unbuffered channel, which means write operation blocks the goroutine.
But by using go you are running the second goroutine.
It means even if this new goroutine will be blocked during write (in your assign function), the main goroutine will continue to work and fmt.Println(<-ch) will be executed and do the read (which in turn unblock background goroutine and assign function will finally reach the end).
To get more understanding about channels and goroutines, this snippet will give the same result (as your second snippet):
package main
import "fmt"
func print(ch chan int) {
fmt.Println(<-ch)
}
func main() {
ch := make(chan int)
go print(ch)
ch <- 1
}
When you are working with a buffered channel (third snippet), you can do N write operations without blocking goroutine (where N is the size of the buffer).
That's why in your example you did 2 writes without blocking and is able to read them later. But if your buffer is less than the count of write operations, and nobody do the read, you will fall into the same blocking issues (see the explanation of 1&2 snippets).
This question already has answers here:
Go channels and deadlock
(3 answers)
Closed 7 years ago.
I am trying to have two separate consumer go routines, that would filter out even and odd numbers from the input channel. This is just a toy example in order to see if it is possible to have consumers do something with the message read from the input channel if it matches certain condition, otherwise put back onto the input channel.
My current code is as follows:
package main
func filterOdd(ch chan int, out chan int) {
val := <- ch
if val % 2 == 0 {
ch <- val
} else {
out <- val
}
}
func filterEven(ch chan int, out chan int) {
val := <- ch
if val % 2 != 0 {
ch <- val
} else {
out <- val
}
}
func main() {
even := make(chan int)
odd := make(chan int)
input := make(chan int)
go filterOdd(input, odd)
go filterEven(input, even)
for i:=1; i <= 10; i++ {
input <- i
}
println("Even...")
for i := range even {
println(i)
}
println("Odd...")
for i := range odd {
println(i)
}
}
However, this produces the following output:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/tmp/sandbox594577124/main.go:27 +0x140
goroutine 4 [chan send]:
main.filterOdd(0x10336100, 0x103360c0)
/tmp/sandbox594577124/main.go:8 +0xc0
created by main.main
/tmp/sandbox594577124/main.go:24 +0xc0
Link to the Go Playground: https://play.golang.org/p/9RIvFsGKI-
You have a deadlock because your even and odd goroutines are blocked on sending to out because nothing is reading from it. Why is nothing reading out? Because the main goroutine is blocked on sending to input because nothing is reading from it. Why is nothing reading from input? Because the two goroutines that would read from it are blocked.
Also, both filterEven and filterOdd will only run once unless you wrap their contents in for { } (but then they'll never stop until you break). On the other hand, range even will block (and range odd never happens) when nothing is left to write to even , because range over a channel only stops when the channel is closed or break is called.
In general, these aren't hard problems to solve as long as you know when you can close a channel. With what you're describing, that gets more difficult. None of the goroutines know when it's OK to close input, because all three write to it and two also read from it. You can use a sync.WaitGroup to make sure that everything you've put into the input channel has been processed before closing it. Once it's closed, the two other goroutines can use that as a signal to close their own channels and break or return to finish running.
However, writes to the in and out channels will still block until there is a corresponding read, because they are unbuffered. However, if you buffer them by specifying a size as the second argument to make, writes won't block until the channel is full. Since you know neither even or odd will have more written to them than what main send to input, you can use that as a safe buffer capacity.
Here's an example of using a WaitGroup with buffered channels for your code: https://play.golang.org/p/VXqfwUwRcx
If you don't want buffered channels, you can also use another pair of goroutines to capture the values and send them back to main as slices once finished. This way writes on the even and odd channels don't block: https://play.golang.org/p/i5vLDcsK1v
Otherwise, if don't need to print the contents of each channel all at once, you can use those two extra goroutines to read from the channels and print right away: https://play.golang.org/p/OCaUTcJkKB
Could someone explain, why if the channel is buffered the program doesn't exit with a fatal_error?
Unbuffered channel
package main
func main() {
c := make(chan int)
c <- 3
}
fatal error: all goroutines are asleep - deadlock!
Buffered channel
package main
func main() {
c := make(chan int, 1)
c <- 3
}
[no output]
Program exited.
Thank you!
Writing to a buffered channel doesn't block if there is room in the buffer.
If you try to put two items in the channel with a buffer size of one, you get the same error:
package main
func main() {
c := make(chan int, 1)
c <- 3
c <- 4
}
gives you:
fatal error: all goroutines are asleep - deadlock!
It's a core concept of Go's channels (or other CSP implementations such as Clojure's core.async library) that they are blocking. In general, as you already mentioned, there're two types of channels:
buffered which block if the buffer is full.
unbuffered which block if there's no "rendezvous", i.e. there must be someone who puts (c <-) to and someone who takes (<- c) from the channel.
In your particular case the Go runtime is smart enough to detect that there's no one who will ever take 3 from channel c. Hence, it's a deadlock and (thankfully) an error is thrown.
What you typically do when you're working with channels is using goroutines (checkout this introduction) which spawn a lightweight thread—managed by the Go runtime—to execute the body concurrently:
c := make(chan int)
go func() { c <- 3 }() // Create a new gorountine that puts 3 to the channel
fmt.Println(<- c) // Take 3 from the channel and print it in the main thread
Thanks #Matt
I found the answer in this post How does make(chan bool) behave differently from make(chan bool, 1)? :
Actually that's the reason why your problem is generated. Un-buffered channels are only writable when there's someone blocking to read from it, which means you shall have some coroutines to work with -- instead of this single one.
Environment: OS X 10.8, Go 1.0.2
I make a channel with buffer-size 2, then if I write channel three times, it will throw error:
throw: all goroutines are asleep - deadlock!
Of course, it's correct.
BUT if I write channel four or more times in the goroutines, it works fine, why?
The channel's capacity is 2, why goroutines ignore that or forget the capacity setting?
I comment the read-channel codes, so no one will read channel and save the capacity. I also use time.Sleep to waiting for all goroutines finish their work.
please review following codes:
package main
//import "fmt"
func main() {
c := make(chan int, 2)
/*c <- 1
c <- 2
c <- 3*/
for i:=0; i<4; i++ {
go func(i int) {
c <- i
c <- 9
c <- 9
c <- 9
}(i)
}
time.Sleep(2000 * time.Millisecond)
/*for i:=0; i<4*2; i++ {
fmt.Println(<-c)
}*/
}
Would anyone please give some hits? thanks, guys.
When a channel is buffered, this means it will not block until the buffer is full. Once the buffer is full, the sending goroutine will block as it tries to add things to the channel.
This means that this will block:
c := make(chan int)
c <- 1 // Block here, this is unbuffered !
println(<-c)
And this will also block:
c := make(chan int, 2)
c <- 1
c <- 2
c <- 3 // Block here, buffer is full !
println(<-c)
But the point of goroutines and channel is precisely to run things concurrently, so this will work:
c := make(chan int)
go func() { c <- 1; }() // This will block in the spawned goroutine until...
println(<-c) // ... this line is reached in the main goroutine
And similarly:
c := make(chan int, 2)
go func() { // `go ...` spawns a goroutine
c <- 1 // Buffer is not full, no block
c <- 2 // Buffer is not full, no block
c <- 3 // Buffer is full, spawned goroutine is blocking until...
}()
println(<-c) // ... this line is reached in the main goroutine
In your example, you spawn four different goroutines, which all write four numbers to the same buffered channel. As the buffer is 2 < 16, they will end up blocking
But the crux of the matter is that the Go policy is to wait only for the main goroutine:
Program execution begins by initializing the main package and then invoking the function main. When the function main returns, the program exits. It does not wait for other (non-main) goroutines to complete.
This means that in your first example, the main goroutine was blocking when it reached line c <- 3. As no other goroutine was able to do anything which could potentially unblock it, the runtime detected that the program was deadlocked and reported an error.
In your second example however, spawned goroutines block, while the main continues quietly until it reaches the end of its execution, at which point all the (blocked) spawned goroutines are silently killed, and no error is reported.
val has given a good answer. I'd like to add an extra tip I've found useful.
When learning to use goroutines, use zero-buffered channels to start with. This way, when you make a mistake, you'll immediately get a deadlock, which you can learn from. You need to learn how to write code that doesn't deadlock, which means learning tricks like not having cyclic dependencies in the client-server relationships (assuming your goroutines are written as clients or servers).
Reasoning about the network without buffering is simpler, although this might not be obvious at first.
Buffering is really useful, but should be considered as a means to enhance performance.