Channels concurrency issue - go

I am experimenting with channel concept in Go. I wrote the below program playground to implement counter using channels. But I am not getting any output, although I am doing some printing in the goroutine body.
func main() {
wg := sync.WaitGroup{}
ch := make(chan int)
count := func(ch chan int) {
var last int
last = <-ch
last = last + 1
fmt.Println(last)
ch <- last
wg.Done()
}
wg.Add(10)
for i := 1; i <= 10; i++ {
go count(ch)
}
}
I expect at least some output but I am getting none at all.

When the main() function (the main goroutine) ends, your program ends as well (it doesn't wait for other non-main goroutines to finish). You must add a wg.Wait() call to the end. See No output from goroutine in Go.
Once you do this, you'll hit a deadlock. This is because all goroutines start with attempting to receive a value from the channel, and only then would they send something.
So you should first send something on the channel to let at least one of the goroutines to proceed.
Once you do that, you'll see numbers printed 10 times, and again deadlock. This is because when the last goroutine tries to send its incremented number, there will be no one to receive that. An easy way to fix that is to give a buffer to the channel.
Final, working example:
wg := sync.WaitGroup{}
ch := make(chan int, 2)
count := func(ch chan int) {
var last int
last = <-ch
last = last + 1
fmt.Println(last)
ch <- last
wg.Done()
}
wg.Add(10)
for i := 1; i <= 10; i++ {
go count(ch)
}
go func() {
ch <- 0
}()
wg.Wait()
Outputs (try it on the Go Playground):
1
2
3
4
5
6
7
8
9
10
Also note that since we made the channel buffered, it's not necessary to use another goroutine to send an initial value, we can do that in the main goroutine:
ch <- 0
wg.Wait()
This will output the same. Try it on the Go Playground.

func main() {
wg := sync.WaitGroup{}
ch := make(chan int)
count := func(ch chan int) {
var last int
last, ok := <-ch // 这里做一层保护
if !ok {
return
}
last = last + 1
fmt.Println(last)
go func(ch chan int, res int) {
ch <- res
}(ch, last)
wg.Done()
}
go func() {
ch <- 0
}()
wg.Add(10)
for i := 1; i <= 10; i++ {
go count(ch)
}
wg.Wait()
fmt.Println("main finish")
close(ch)
}

Related

Deadlocks with buffered channels in Go

I am encountering for the below code fatal error: all goroutines are asleep - deadlock!
Am I right in using a buffered channel? I would appreciate it if you can give me pointers. I am unfortunately at the end of my wits.
func main() {
valueChannel := make(chan int, 2)
defer close(valueChannel)
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go doNothing(&wg, valueChannel)
}
for {
v, ok := <- valueChannel
if !ok {
break
}
fmt.Println(v)
}
wg.Wait()
}
func doNothing(wg *sync.WaitGroup, numChan chan int) {
defer wg.Done()
time.Sleep(time.Duration(rand.Intn(1000)) * time.Millisecond)
numChan <- 12
}
The main goroutine blocks on <- valueChannel after receiving all values. Close the channel to unblock the main goroutine.
func main() {
valueChannel := make(chan int, 2)
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go doNothing(&wg, valueChannel)
}
// Close channel after goroutines complete.
go func() {
wg.Wait()
close(valueChannel)
}()
// Receive values until channel is closed.
// The for / range loop here does the same
// thing as the for loop in the question.
for v := range valueChannel {
fmt.Println(v)
}
}
Run the example on the playground.
The code above works independent of the number of values sent by the goroutines.
If the main() function can determine the number of values sent by the goroutines, then receive that number of values from main():
func main() {
const n = 10
valueChannel := make(chan int, 2)
for i := 0; i < n; i++ {
go doNothing(valueChannel)
}
// Each call to doNothing sends one value. Receive
// one value for each call to doNothing.
for i := 0; i < n; i++ {
fmt.Println(<-valueChannel)
}
}
func doNothing(numChan chan int) {
time.Sleep(time.Duration(rand.Intn(1000)) * time.Millisecond)
numChan <- 12
}
Run the example on the playground.
The main problem is on the for loop of channel receiving.
The comma ok idiom is slightly different on channels, ok indicates whether the received value was sent on the channel (true) or is a zero value returned because the channel is closed and empty (false).
In this case the channel is waiting a data to be sent and since it's already finished sending the value ten times : Deadlock.
So apart of the design of the code if I just need to do the less change possible here it is:
func main() {
valueChannel := make(chan int, 2)
defer close(valueChannel)
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go doNothing(&wg, valueChannel)
}
for i := 0; i < 10; i++ {
v := <- valueChannel
fmt.Println(v)
}
wg.Wait()
}
func doNothing(wg *sync.WaitGroup, numChan chan int) {
defer wg.Done()
time.Sleep(time.Duration(rand.Intn(1000)) * time.Millisecond)
numChan <- 12
}

Simple parallel example with channel, leading to deadlock

I'm trying to implement a simple parallelization example in golang using channels. The code tries to implement a parallelized map function. Made channels buffered too for easing constraints on blocking nature of channel. but code still leads to a deadlock.
func pmmap(inp []int, f func(int) int, p int) {
var wg sync.WaitGroup
var output []chan int
tot := len(inp)
out := make(chan int, tot)
slice := tot / p
for i := 0; i < p; i++ {
temp := make(chan int, slice)
output = append(output, temp)
start_ind := slice * i
end_ind := start_ind + slice
fmt.Println(start_ind, end_ind)
wg.Add(1)
go func(si, ei int, out chan int, wg *sync.WaitGroup) {
fmt.Println("goroutine started with ", si, ei)
for ind := si; ind < ei; ind++ {
out <- f(inp[ind])
}
wg.Done()
}(start_ind, end_ind, output[i], &wg)
}
wg.Add(1)
go func(wg *sync.WaitGroup) {
for i := 0; i < p; i++ {
for val := range output[i] {
out <- val
}
close(output[i])
}
wg.Done()
}(&wg)
wg.Add(1)
go func(wg *sync.WaitGroup) {
for i := range out {
fmt.Println(i)
}
wg.Done()
}(&wg)
time.Sleep(time.Second * 6)
wg.Wait()
close(out)
}
func add4(i int) int {
return i + 4
}
func main() {
temp := []int{}
for i := 1; i <= 20; i++ {
temp = append(temp, i)
}
pmmap(temp, add4, 2)
}
from the output of above code, I get the deadlock is because channel output[1] is never read. but I'm not sure why
0 10
10 20
goroutine started with 0 10
goroutine started with 10 20
5
6
7
8
9
10
11
12
13
14
fatal error: all goroutines are asleep - deadlock!
The problem is that range over a channel keeps trying to receive from the channel until the channel is closed, but you didn't close channels after finished sending.
Add close(out) before wg.Done in your code will fix it.
Playground: https://play.golang.org/p/NbKTx6Lke7X
Edit: fixed a bug of closing closed channel.

Dead lock in goroutines

can someone give me some insight about this code, Why this get deadlock error in for x:=range c
func main() {
c:=make(chan int,10)
for i:=0;i<5;i++{
go func(chanel chan int,i int){
println("i",i)
chanel <- 1
}(c,i)
}
for x:=range c {
println(x)
}
println("Done!")
}
Because this:
for x:=range c {
println(x)
}
will loop until the channel c closes, which is never done here.
Here is one way you can fix it, using a WaitGroup:
package main
import "sync"
func main() {
var wg sync.WaitGroup
c := make(chan int, 10)
for i := 0; i < 5; i++ {
wg.Add(1)
go func(chanel chan int, i int) {
defer wg.Done()
println("i", i)
chanel <- 1
}(c, i)
}
go func() {
wg.Wait()
close(c)
}()
for x := range c {
println(x)
}
println("Done!")
}
Try it on Go Playground
You create five goroutines, each sending an integer value to the channel. Once all those five values are written, there's no other goroutine left that writes to the channel.
The main goroutine reads those five values from the channel. But there are no goroutines that can possibly write the sixth value or close the channel. So, you're deadlocked waiting data from the channel.
Close the channel once all the writes are completed. It should be an interesting exercise to figure out how you can do that with this code.
Channel needs to be closed to indicate task is complete.
Coordinate with a sync.WaitGroup:
c := make(chan int, 10)
var wg sync.WaitGroup // here
for i := 0; i < 5; i++ {
wg.Add(1) // here
go func(chanel chan int, i int) {
defer wg.Done()
println("i", i)
chanel <- 1
}(c, i)
}
go func() {
wg.Wait() // and here
close(c)
}()
for x := range c {
println(x)
}
println("Done!")
https://play.golang.org/p/VWcBC2YGLvM

All goroutines are asleep

I write some code which aims to synchronize using channel.
var counter int64 // shared resource
var wg sync.WaitGroup
func main() {
ch := make(chan int64)
wg.Add(2)
go incCounter(ch)
go incCounter(ch)
ch <- counter
wg.Wait()
fmt.Println("Final Counter:", counter) // expected value is 4
}
func incCounter(ch chan int64) {
defer wg.Done()
for count := 0; count < 2; count++ {
value := <-ch
value++
counter = value
ch <- counter
}
}
When I ran this program, an error happened: all goroutines are asleep - deadlock!. However I can't fix the problem and I don't know what is wrong. Could anyone help?
Channels make(chan int) has implicit size zero ( ref: https://golang.org/ref/spec#Making_slices_maps_and_channels)
A channel of size zero is unbuffered. A channel of specified size make(chan int, n) is buffered. See http://golang.org/ref/spec#Send_statements for a discussion on buffered vs. unbuffered channels. The example at http://play.golang.org/p/VZAiN1V8-P illustrates the difference.
Here, channel <-ch or ch <- will be blocked until someone processes it (concurrently). If you try the flow of this program in pen and paper, you will figure it out why it is blocked. Below figure shows possible data flow through channel ch:
So if you make your ch := make(chan int64) to ch := make(chan int64,1), it will work.
var counter int64 // shared resource
var wg sync.WaitGroup
func main() {
ch := make(chan int64, 1)
wg.Add(2)
go incCounter(ch)
go incCounter(ch)
ch <- counter
wg.Wait()
fmt.Println("Final Counter:", counter) // expected value is 4
}
func incCounter(ch chan int64) {
defer wg.Done()
for count := 0; count < 2; count++ {
value := <-ch
value++
counter = value
ch <- counter
}
}
If we analyse how program works when you are using ch := make(chan int64), we can see that one go routine is blocked in this program(another one is exited). With the help of time.Sleep(n) and receiving the last data from the channel in the blocked go routine, we can overcome the deadlock. See the code below:
var counter int64 // shared resource
var wg sync.WaitGroup
func main() {
ch := make(chan int64)
wg.Add(2)
go incCounter(ch)
go incCounter(ch)
ch <- counter
// to ensure one go routine 'incCounter' is completed and one go routine is blocked for unbuffered channel
time.Sleep(3*time.Second)
<-ch // to unblock the last go routine
wg.Wait()
fmt.Println("Final Counter:", counter) // expected value is 4
}
func incCounter(ch chan int64) {
defer wg.Done()
for count := 0; count < 2; count++ {
value := <-ch
value++
counter = value
ch <- counter
}
}

Go: transformed channel

Let's say I have an int channel in Go:
theint := make(chan int)
I want to wrap this channel in a new channel called incremented
incremented := make(chan int)
Such that:
go func() { theint <- 1 }
<- incremented // 2
appended can be assumed to be the only one that reads from the int.
It will work if a run a goroutine in the background
go func() {
for num := range theint {
incremented <- num + 1
}
}
However, I prefer to do it without a goroutine since I can't control it in my context.
Is there a simpler way to do it?
One thing that came to mind is python's yield:
for num in theint:
yield num + 1
Is something like this possible in go?
Generator pattern
What you are trying to implement is generator pattern. To use channels and goroutines for implementation of this pattern is totally common practice.
However, I prefer to do it without a goroutine since I can't control it in my context.
I believe the problem is deadlock
fatal error: all goroutines are asleep - deadlock!
To avoid deadlocks and orphaned (not closed) channels use sync.WaitGroup. This is an idiomatic way to control goroutines.
Playground
package main
import (
"fmt"
"sync"
)
func incGenerator(n []int) chan int {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(len(n))
for _, i := range n {
incremented := i + 1
go func() {
wg.Done()
ch <- incremented
}()
}
go func() {
wg.Wait()
close(ch)
}()
return ch
}
func main() {
n := []int{1, 2, 3, 4, 5}
for x := range incGenerator(n) {
fmt.Println(x)
}
}
One thing you can also consider is having a select on the int channel and an exit channel - in an infinite for loop. You can choose a variable increment value too. Please see code below:
package main
import (
"fmt"
"sync"
"time"
)
func main() {
var accum int //accumulator of incremented values
var wg sync.WaitGroup
c1 := make(chan int)
exChan := make(chan bool)
wg.Add(1)
go func() {
time.Sleep(time.Second * 1)
c1 <- 1
wg.Done()
}()
wg.Add(1)
go func() {
time.Sleep(time.Second * 2)
c1 <- 2
wg.Done()
}()
wg.Add(1)
go func() {
time.Sleep(time.Second * 2)
c1 <- 5
wg.Done()
}()
go func() {
wg.Wait()
close(exChan)
}()
for {
var done bool
select {
case incBy := <-c1: //Increment by value in channel
accum += incBy
fmt.Println("Received value to increment:", incBy, "; Accumulated value is", accum)
case d := <-exChan:
done = !(d)
}
if done == true {
break
}
}
fmt.Println("Final accumulated value is", accum)
}
Playground: https://play.golang.org/p/HmdRmMCN7U
Exit channel not needed, if we are having non-zero increments always. I like #I159 's approach too!
Anyways, hope this helps.

Resources