If I pass more than 12 for totalValues to getResults(), no values are read from channel resultCh and I get fatal error: all goroutines are asleep - deadlock!:
package main
type result struct {
index int
count int
}
func getResults(indexCh chan int, resultCh chan result, totalValues int) {
allFields := make([]int, totalValues)
for i := range allFields {
indexCh <- i
println("Sent value", i)
}
println("Sent all values")
for i := 0; i < totalValues; i++ {
value := <-resultCh
println("Received result", value.index)
allFields[value.index] = value.count
}
println("Done processing")
}
func processValue(indexCh chan int, ch chan result) {
for index := range indexCh {
println("Received value", index)
ch <- result{
index: index,
count: index * index,
}
println("Sent result", index)
}
}
func main() {
bufferDepth := 4
indexCh := make(chan int, bufferDepth)
resultCh := make(chan result, bufferDepth)
for i := 0; i < 4; i++ {
go processValue(indexCh, resultCh)
}
// Value > 12 results in goroutine asleep
getResults(indexCh, resultCh, 12)
}
The above code works by launching four goroutines using function processValue() that read values from buffered channel indexCh. Integers 0 to totalValues are inserted into indexCh by getResults(). processValue() processes the value and puts the result onto buffered channel resultCh, which is read by a getResults().
I only only observe this problem when totalValues is greater than 12. No values are read from resultCh because "Received result..." is not printed.
If I increase bufferDepth to five, the program completes successfully for totalValues > 12 and < 15.
The program also completes successfully if the buffer depth of resultCh matches totalValues.
getResults starts by writing all values to the channel. There are 4 goroutines listening, so each picks up those values, and they are blocked writing to the result channel. Channels have buffer sizes of 4. So 4 goroutines each can write a total of 4 values to result channel before blocking.
4 goroutines + 4 index channel size + 4 result channel size = 12 entries
After that, the processValue is blocked at writing to the channel, because getResults are also blocked writing to the indexCh.
Related
https://play.golang.org/p/5A-dnZVy2aA
func closeChannel(stream chan int) {
fmt.Println("closing the channel")
close(stream)
}
func main() {
chanOwner := func() <-chan int {
resultStream := make(chan int, 5)
go func() {
defer closeChannel(resultStream)
for i := 0; i <= 5; i++ {
resultStream <- i
}
}()
return resultStream
}
resultStream := chanOwner()
for result := range resultStream { //this blocks until the channel is closed
fmt.Printf("Received: %d\n", result)
}
fmt.Println("done receiving")
}
Output of program
closing the channel
Received: 0
Received: 1
Received: 2
Received: 3
Received: 4
Received: 5
done receiving
why is the closing the channel statement printed before any Received in the above program. since the channel is buffered with a capacity of 5, and we are inserting 6 elements into it, I was expecting it to block at the resultStream <- i before a value was read and cleared up space in the buffer.
The generator goroutine fills the channel to its capacity and blocks. The receiver for loops receives the first item from the channel, which enables the generator goroutine again. The generator goroutine runs to completion before the receiver for-loop can print the Received message, printing the closing the channel message. Then the receiver loop receives all remaining messages.
Here is my program.
package main
import "fmt"
func init() {
fmt.Println("init function")
}
func main() {
// gen number to naturals channel
naturals := gen(1, 2, 3)
// write the squars in a channel
squar := dosquar(naturals)
for p := range squar {
fmt.Println("printer shows----- ", p)
}
}
func gen(nums ...int) chan int {
naturals := make(chan int)
go func() {
for _, n := range nums {
fmt.Println("generating number ", n)
naturals <- n
fmt.Println("generated number ", n)
}
//close(naturals)
}()
return naturals
}
func dosquar(naturals chan int) chan int {
// write the squars the values
squar := make(chan int)
go func() {
for number := range naturals {
fmt.Println("recieved number ", number)
squar <- number * number
fmt.Println("sent squar of number ", number*number)
}
//close(squar)
}()
return squar
}
It gives following error.
init function
generating number 1
generated number 1
generating number 2
recieved number 1
sent squar of number 1
recieved number 2
generated number 2
generating number 3
printer shows----- 1
printer shows----- 4
sent squar of number 4
recieved number 3
sent squar of number 9
printer shows----- 9
generated number 3
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/Users/siyaram/go/src/github.com/SrmHitter9062/go-pipeline/main.go:15 +0x127
goroutine 19 [chan receive]:
I expect it to be running as channels are being ranged.That leads me to a question that do we need to close the channel every time, when we are finished with them ?
Please somebody help clarifying why program is not working as expected.
You can do whatever you want, but a range loop over a channel won't exit until the channel is closed.
https://play.golang.org/p/AyKy5odhfZw
In my sight, prime := <- ch this line is before go Filter () , every time data put into ch will be got out by prime directly.
// A concurrent prime sieve
package main
import "fmt"
// Send the sequence 2, 3, 4, ... to channel 'ch'.
func Generate(ch chan<- int) {
for i := 2; ; i++ {
ch <- i // Send 'i' to channel 'ch'.
}
}
// Copy the values from channel 'in' to channel 'out',
// removing those divisible by 'prime'.
func Filter(in <-chan int, out chan<- int, prime int) {
for {
i := <-in // Receive value from 'in'.
if i%prime != 0 {
out <- i // Send 'i' to 'out'.
}
}
}
// The prime sieve: Daisy-chain Filter processes.
func main() {
ch := make(chan int) // Create a new channel.
go Generate(ch) // Launch Generate goroutine.
for i := 0; i < 10; i++ {
prime := <-ch
fmt.Println(prime)
ch1 := make(chan int)
go Filter(ch, ch1, prime)
ch = ch1
}
}
Filter does not receive data first.
The way the code is written means that the variable prime will always receive the first value on the output of the filter goroutine created in the previous loop (or the generator on the first loop).
The next time around the main loop the channel variable ch will have changed to output of the next filter due to the line ch=ch1 at the end of the loop. All further integers beyond the first output by a filter will pass to the next filter in the chain.
Placing a few print statements in the filter goroutine will let you see what is happening:
func Filter(in <-chan int, out chan<- int, prime int) {
for {
i := <-in // Receive value from 'in'.
if i%prime != 0 {
fmt.Printf("Prime filter (%d): passing %d\n", prime, i)
out <- i // Send 'i' to 'out'.
} else {
fmt.Printf("Prime filter (%d): filtered %d\n", prime, i)
}
}
}
It's a little messy that the goroutines in the pipeline aren't shut down elgantly by closing the generator but that's another story :)
I am trying the fan in - fan out pattern with a factorial problem. But I am getting:
fatal error: all goroutines are asleep - deadlock!
and unable to identify the reason for deadlock.
I am trying to concurrently calculate factorial for 100 numbers using the fan-in fan-out pattern.
package main
import (
"fmt"
)
func main() {
_inChannel := _inListener(generator())
for val := range _inChannel {
fmt.Print(val, " -- ")
}
}
func generator() chan int { // NEED TO CALCULATE FACTORIAL FOR 100 NUMBERS
ch := make(chan int) // CREATE CHANNEL TO INPUT NUMBERS
go func() {
for i := 1; i <= 100; i++ {
ch <- i
}
close(ch) // CLOSE CHANNEL WHEN ALL NUMBERS HAVE BEEN WRITTEM
}()
return ch
}
func _inListener(ch chan int) chan int {
rec := make(chan int) // CHANNEL RECEIVED FROM GENERATOR
go func() {
for num := range ch { // RECEIVE THE INPUT NUMBERS FROM GENERATOR
result := factorial(num) // RESULT IS A NEW CHANNEL CREATED
rec <- <-result // MERGE INTO A SINGLE CHANNEL; rec
close(result)
}
close(rec)
}()
return rec // RETURN THE DEDICATED CHANNEL TO RECEIVE ALL OUTPUTS
}
func factorial(n int) chan int {
ch := make(chan int) // MAKE A NEW CHANNEL TO OUTPUT THE RESULT
// OF FACTORIAL
total := 1
for i := n; i > 0; i-- {
total *= i
}
ch <- total
return ch // RETURN THE CHANNEL HAVING THE FACTORIAL CALCULATED
}
I have put in comments, so that it becomes easier to follow the code.
I'm no expert in channels. I've taking on this to try and get more familiar with go.
Another issue is the int isn't large enough to take all factorials over 20 or so.
As you can see, I added a defer close as well as a logical channel called done in the generator func. The rest of the changes probably aren't needed. With channels you need to make sure something is ready to take off a value on the channel when you put something on a channel. Otherwise deadlock. Also, using
go run -race main.go
helps at least see which line(s) are causing problems.
I hope this helps and isn't removed for being off topic.
I was able to remove the deadlock by doing this:
package main
import (
"fmt"
)
func main() {
_gen := generator()
_inChannel := _inListener(_gen)
for val := range _inChannel {
fmt.Print(val, " -- \n")
}
}
func generator() chan int { // NEED TO CALCULATE FACTORIAL FOR 100 NUMBERS
ch := make(chan int) // CREATE CHANNEL TO INPUT NUMBERS
done := make(chan bool)
go func() {
defer close(ch)
for i := 1; i <= 100; i++ {
ch <- i
}
//close(ch) // CLOSE CHANNEL WHEN ALL NUMBERS HAVE BEEN WRITTEM
done <- true
}()
// this function will pull off the done for each function call above.
go func() {
for i := 1; i < 100; i++ {
<-done
}
}()
return ch
}
func _inListener(ch chan int) chan int {
rec := make(chan int) // CHANNEL RECEIVED FROM GENERATOR
go func() {
for num := range ch { // RECEIVE THE INPUT NUMBERS FROM GENERATOR
result := factorial(num) // RESULT IS A NEW CHANNEL CREATED
rec <- result // MERGE INTO A SINGLE CHANNEL; rec
}
close(rec)
}()
return rec // RETURN THE DEDICATED CHANNEL TO RECEIVE ALL OUTPUTS
}
func factorial(n int) int {
// OF FACTORIAL
total := 1
for i := n; i > 0; i-- {
total *= i
}
return total // RETURN THE CHANNEL HAVING THE FACTORIAL CALCULATED
}
I'm reading/working through Go Concurrency Patterns: Pipelines and cancellation, but i'm having trouble understanding the Stopping short section. We have the following functions:
func sq(in <-chan int) <-chan int {
out := make(chan int)
go func() {
for n := range in {
out <- n * n
}
close(out)
}()
return out
}
func gen(nums ...int) <-chan int {
out := make(chan int)
go func() {
for _, n := range nums {
out <- n
}
close(out)
}()
return out
}
func merge(cs ...<-chan int) <-chan int {
var wg sync.WaitGroup
out := make(chan int, 1) // enough space for the unread inputs
// Start an output goroutine for each input channel in cs. output
// copies values from c to out until c is closed, then calls wg.Done.
output := func(c <-chan int) {
for n := range c {
out <- n
}
wg.Done()
}
wg.Add(len(cs))
for _, c := range cs {
go output(c)
}
// Start a goroutine to close out once all the output goroutines are
// done. This must start after the wg.Add call.
go func() {
wg.Wait()
close(out)
}()
return out
}
func main() {
in := gen(2, 3)
// Distribute the sq work across two goroutines that both read from in.
c1 := sq(in)
c2 := sq(in)
// Consume the first value from output.
out := merge(c1, c2)
fmt.Println(<-out) // 4 or 9
return
// Apparently if we had not set the merge out buffer size to 1
// then we would have a hanging go routine.
}
Now, if you notice line 2 in merge, it says we make the out chan with buffer size 1, because this is enough space for the unread inputs. However, I'm almost positive that we should allocate a chan with buffer size 2. In accordance with this code sample:
c := make(chan int, 2) // buffer size 2
c <- 1 // succeeds immediately
c <- 2 // succeeds immediately
c <- 3 // blocks until another goroutine does <-c and receives 1
Because this section implies that a chan of buffer size 3 would not block. Can anyone please clarify/assist my understanding?
The program sends two values to the channel out and reads one value from the channel out. One of the values is not received.
If the channel is unbuffered (capacity 0), then one of the sending goroutines will block until the program exits. This is a leak.
If the channel is created with a capacity of 1, then both goroutines can send to the channel and exit. The first value sent to the channel is received by main. The second value remains in the channel.
If the main function does not receive a value from the channel out, then a channel of capacity 2 is required to prevent the goroutines from blocking indefinitely.