Problem Understanding Golang Concurrency Code - go

I just started learning golang, while I was going through concurrency I accidentally wrote this code:
import (
"fmt"
)
func squares(c chan int) {
for i := 0; i < 4; i++ {
num := <- c
fmt.Println(num * num)
}
}
func main() {
fmt.Println("main start")
c := make(chan int)
go squares(c)
c <- 1
c <- 2
c <- 3
c <- 4
go squares(c)
c <- 5
c <- 6
c <- 7
c <- 8
fmt.Println("main stop")
}
Originally I was suppose to assign c := make(chan int, 3).
I am having trouble understanding the output of the code I've written.
main start
1
4
9
25
36
49
16
main stop
I would like to understand how the code is executed.
I was expecting error: all goroutines are asleep - deadlock!
Many thanks!

I am not sure to really understand what you wanted to achieve, especially the reason of that weird loop :
for i := 0; i < 4; i++ {
num := <- c
fmt.Println(num * num)
}
But anyway.
First have all, some explanations on how works channels and goroutines.
A Channel is a thread safe messaging pipe used to share data accross differents executions contexts. A channel is creating with make instruction, then
c := make(chan int, 3)
means create a channel of int type with a "buffer" size of 3. This element is very important to understand. Channel do follow a producer / consumer pattern with that bases rules :
For producer :
if I try to push some data in a channel with some "free space", it doesn't block and next instructions are executed
If I try to push some data in a channel without "free space", it blocks till previous has been treated
For consumer :
If I try to take element from an empty channel, it block untill some data come
If I try to take element from a non empty channel, it take the first (Channel is a FIFO)
Please note that all blocking operations may be turned non-blocking using some special pattern available through select instruction, but that's another subject :).
Goroutine are light sub-processes, routines (No thread). It is no place here to explain all in details, but what is important is 1 goroutine === 1 execution stack.
So in your case, the output is quite predictable till there is only one consumer. Once you start the second Goroutine, the consumption order is still predictable (the size of the channel is only one), but the execution order not !
One thing is noticable: you have only 7 output... That because your main goroutine ends before it happends, terminating the whole program execution. That point explain why you didn't have the all goroutines are asleep - deadlock!.
If you want to have that, you just should add <- c somewhere at the end of the main :
package main
import (
"fmt"
)
func squares(c chan int) {
for i := 0; i < 4; i++ {
num := <-c
fmt.Println(num * num)
}
}
func main() {
fmt.Println("main start")
c := make(chan int)
go squares(c)
c <- 1
c <- 2
c <- 3
c <- 4
go squares(c)
c <- 5
c <- 6
c <- 7
c <- 8
<-c
fmt.Println("main stop")
}
You will have the behavior I think you expect :
main start
1
4
9
25
36
49
64
16
fatal error: all goroutines are asleep - deadlock!
Edit : on step by step, execution :
// a goroutine is created and c is empty. because
// the code of `squares` act as a consumer, the goroutine
// "block" at instruction `num := <-c`, but not the main goroutine
go squares(c)
// 1 is pushed to the channel. Till "c" is not full the main goroutine
// doesn't block but the other goroutine may be "released"
c <- 1
// if goroutine of squares has not consume 1 yet, main goroutine block
// untill so, but is released just after
c <- 2
// it continues with same logic
c <- 3
c <- 4
// till main goroutine encountered `<- c` (instruction I added) .
// Here, it behave as a consumer of "c". At this point all
// goroutine are waiting as consuler on "c" => deadlock

Related

Closing channel on multiple coroutines

If I have multiple goroutines reading and writing to the same channel, are there any ways to close the channel at certain point, for example after a fix amount of elements are written.
func foo(c chan int) {
for i := range c {
// how to close the channel after 100 integers are written?
c <- i + 1
c <- i + 2
}
}
func bar() {
c := make(chan int, 200)
c <- 0
go foo(c)
go foo(c)
go foo(c)
}
Give conditional case would work for it:
for i := 0; i <= cap(c); i++ {
// how to close the channel after 100 integers are written?
c <- i + 1
// since you send 2 values to channel for each i loop, you need to half of it.
if i == 49 { //if you asking why 49, you have sent one value on bar function. Also to make it perfectly 100 integer in total.
close(ch)
break // terminating the function after the closing channel, it will run in panic because the function still running and keep sending value on channel which was already closed.
}
c <- i + 2
}
Output :
0
1
2
2
3
3
...
50
50
Just give some information about channel, don't use range on channel if you havent sent anything on it...
If you put "i range channel" before you have value on channel, it will not read anything.
Back to again to your code
If you run this code :
func bar() {
c := make(chan int, 200)
c <- 0
go foo(c)
go foo(c) // run in panic
go foo(c) // run in panic
}
It will run in panic, since the channel already closed which mean your channel not accepting any values again.
Take a look at this picture, let say you make a buffered channel with 3 capacity. And you going to close after receiving 2 values to send it will run in panic if you try to send values on it again.

Why is the following program deadlock?

package main
import "fmt"
func main() {
c := make(chan int)
c <- 5
a := <- c
fmt.Println(a)
}
Here both the data is being sent and recieved to/from the channel but still the error is being displayed:
fatal error: all goroutines are asleep - deadlock!
The channel is unbuffered. So if one writes to it - and no other goroutine is reading from it - it is a guaranteed deadlock.
To fix, either make it unbuffered:
c := make(chan int, 1)
or write asynchronously from another goroutine:
go func() {
c <- 5
}()
https://play.golang.org/p/RRJILbuZKTQ
Since channels are used to communicate between goroutines, its extremely rare one writes to a channel from the same goroutine as the reads. An exception may be, if using a channel as a resource pool and one wants to pre-populate the (buffered) channel.
Your channel c doesn't have any buffer (even a single cell), which means that the main goroutine locks at the c <- 5 line. Since it is the only goroutine in your application, it creates a deadlock.
Channels make(chan int) has implicit size zero ( ref: https://golang.org/ref/spec#Making_slices_maps_and_channels)
A channel of size zero is unbuffered. A channel of specified size make(chan int, n) is buffered. See http://golang.org/ref/spec#Send_statements for a discussion on buffered vs. unbuffered channels. The example at http://play.golang.org/p/VZAiN1V8-P illustrates the difference.
Here, channel <-c or c <- will be blocked until someone processes it (concurrently). So, change c := make(chan int) to c := make(chan int,1)
package main
import "fmt"
func main() {
c := make(chan int, 1)
c <- 5
a := <- c
fmt.Println(a)
}

Write to channel blocked forever

I am stuck in a strange situation where the write operation to the channel never happens.
package main
import (
"fmt"
"time"
)
func main() {
c := make(chan int)
s := make(chan bool)
k := make(chan bool)
fmt.Println("start")
go func() {
fmt.Println("routine 1")
s <- true
}()
go func() {
fmt.Println("routine 2")
for {
select {
case <-s :
for i := 0; i < 3; i++ {
fmt.Println("before")
c <- i
fmt.Println("after")
}
case <- k :
fmt.Println("k ready")
break
}
}
}()
go func() {
fmt.Println("routine 3")
for {
select {
case x := <- c :
fmt.Println("x=", x)
k <- true
fmt.Println("k done")
}
}
}()
time.Sleep(1000 * time.Millisecond)
}
And here is the output:
start
routine 1
routine 2
before
routine 3
x= 0
after
before
I wonder why the write to the channel k blocks, but the log statement fmt.Println("k ready") is never printed.
Here is what I think :
go routine 1 writes true to channel s
go routine 2 writes 0 to
channel c and waits because buffer size is 0, it will not be able to
write '1' to it unless someone reads channel c
go routine 3 comes into the picture, reads channel c (now go routine 2
can write to c once go routine 2 resumes) prints the value of x. NOW IT SHOULD BE ABLE TO WRITE TO CHANNEL K but that is not happening
According to me it should be able to write to channel k then case 2 of goroutine should execute and print "k ready"
Can anyone explain me why write to the channel blocked?
As a fix I know I can increase the buffer size of channel c and everything will get printed but I am not interested in fixing this, instead I want to understand this scenario.
A nice blog to understand the above case.
You have a deadlock.
goroutine 1 writes to s then quits
goroutine 2 reads from s, and writes to c
goroutine 3 reads from c, and writes to k, and this blocks because nothing is reading from k, because goroutine 2 is blocked in the write to k above.
goroutine 2 writes to c again which blocks as goroutine 3 is still trying to write to k and thus is not reading from c
Contrary to what you say, you don't have a buffer size of 1. You have a buffer size of zero (i.e. an unbuffered channel), so a write will block until something reads. This is probably the source of your misunderstanding. Per the language specification:
A new, initialized channel value can be made using the built-in function make, which takes the channel type and an optional capacity as arguments:
make(chan int, 100)
The capacity, in number of elements, sets the size of the buffer in the channel. If the capacity is zero or absent, the channel is unbuffered and communication succeeds only when both a sender and receiver are ready. Otherwise, the channel is buffered and communication succeeds without blocking if the buffer is not full (sends) or not empty (receives). A nil channel is never ready for communication.

Multiple goroutines listening on one channel

I have multiple goroutines trying to receive on the same channel simultaneously. It seems like the last goroutine that starts receiving on the channel gets the value. Is this somewhere in the language spec or is it undefined behaviour?
c := make(chan string)
for i := 0; i < 5; i++ {
go func(i int) {
<-c
c <- fmt.Sprintf("goroutine %d", i)
}(i)
}
c <- "hi"
fmt.Println(<-c)
Output:
goroutine 4
Example On Playground
EDIT:
I just realized that it's more complicated than I thought. The message gets passed around all the goroutines.
c := make(chan string)
for i := 0; i < 5; i++ {
go func(i int) {
msg := <-c
c <- fmt.Sprintf("%s, hi from %d", msg, i)
}(i)
}
c <- "original"
fmt.Println(<-c)
Output:
original, hi from 0, hi from 1, hi from 2, hi from 3, hi from 4
NOTE: the above output is outdated in more recent versions of Go (see comments)
Example On Playground
Yes, it's complicated, But there are a couple of rules of thumb that should make things feel much more straightforward.
prefer using formal arguments for the channels you pass to go-routines instead of accessing channels in global scope. You can get more compiler checking this way, and better modularity too.
avoid both reading and writing on the same channel in a particular go-routine (including the 'main' one). Otherwise, deadlock is a much greater risk.
Here's an alternative version of your program, applying these two guidelines. This case demonstrates many writers & one reader on a channel:
c := make(chan string)
for i := 1; i <= 5; i++ {
go func(i int, co chan<- string) {
for j := 1; j <= 5; j++ {
co <- fmt.Sprintf("hi from %d.%d", i, j)
}
}(i, c)
}
for i := 1; i <= 25; i++ {
fmt.Println(<-c)
}
http://play.golang.org/p/quQn7xePLw
It creates the five go-routines writing to a single channel, each one writing five times. The main go-routine reads all twenty five messages - you may notice that the order they appear in is often not sequential (i.e. the concurrency is evident).
This example demonstrates a feature of Go channels: it is possible to have multiple writers sharing one channel; Go will interleave the messages automatically.
The same applies for one writer and multiple readers on one channel, as seen in the second example here:
c := make(chan int)
var w sync.WaitGroup
w.Add(5)
for i := 1; i <= 5; i++ {
go func(i int, ci <-chan int) {
j := 1
for v := range ci {
time.Sleep(time.Millisecond)
fmt.Printf("%d.%d got %d\n", i, j, v)
j += 1
}
w.Done()
}(i, c)
}
for i := 1; i <= 25; i++ {
c <- i
}
close(c)
w.Wait()
This second example includes a wait imposed on the main goroutine, which would otherwise exit promptly and cause the other five goroutines to be terminated early (thanks to olov for this correction).
In both examples, no buffering was needed. It is generally a good principle to view buffering as a performance enhancer only. If your program does not deadlock without buffers, it won't deadlock with buffers either (but the converse is not always true). So, as another rule of thumb, start without buffering then add it later as needed.
Late reply, but I hope this helps others in the future like Long Polling, "Global" Button, Broadcast to everyone?
Effective Go explains the issue:
Receivers always block until there is data to receive.
That means that you cannot have more than 1 goroutine listening to 1 channel and expect ALL goroutines to receive the same value.
Run this Code Example.
package main
import "fmt"
func main() {
c := make(chan int)
for i := 1; i <= 5; i++ {
go func(i int) {
for v := range c {
fmt.Printf("count %d from goroutine #%d\n", v, i)
}
}(i)
}
for i := 1; i <= 25; i++ {
c<-i
}
close(c)
}
You will not see "count 1" more than once even though there are 5 goroutines listening to the channel. This is because when the first goroutine blocks the channel all other goroutines must wait in line. When the channel is unblocked, the count has already been received and removed from the channel so the next goroutine in line gets the next count value.
I've studied existing solutions and created simple broadcast library https://github.com/grafov/bcast.
group := bcast.NewGroup() // you created the broadcast group
go bcast.Broadcasting(0) // the group accepts messages and broadcast it to all members
member := group.Join() // then you join member(s) from other goroutine(s)
member.Send("test message") // or send messages of any type to the group
member1 := group.Join() // then you join member(s) from other goroutine(s)
val := member1.Recv() // and for example listen for messages
It is complicated.
Also, see what happens with GOMAXPROCS = NumCPU+1. For example,
package main
import (
"fmt"
"runtime"
)
func main() {
runtime.GOMAXPROCS(runtime.NumCPU() + 1)
fmt.Print(runtime.GOMAXPROCS(0))
c := make(chan string)
for i := 0; i < 5; i++ {
go func(i int) {
msg := <-c
c <- fmt.Sprintf("%s, hi from %d", msg, i)
}(i)
}
c <- ", original"
fmt.Println(<-c)
}
Output:
5, original, hi from 4
And, see what happens with buffered channels. For example,
package main
import "fmt"
func main() {
c := make(chan string, 5+1)
for i := 0; i < 5; i++ {
go func(i int) {
msg := <-c
c <- fmt.Sprintf("%s, hi from %d", msg, i)
}(i)
}
c <- "original"
fmt.Println(<-c)
}
Output:
original
You should be able to explain these cases too.
For multiple goroutine listen on one channel, yes, it's possible. the key point is the message itself, you can define some message like that:
package main
import (
"fmt"
"sync"
)
type obj struct {
msg string
receiver int
}
func main() {
ch := make(chan *obj) // both block or non-block are ok
var wg sync.WaitGroup
receiver := 25 // specify receiver count
sender := func() {
o := &obj {
msg: "hello everyone!",
receiver: receiver,
}
ch <- o
}
recv := func(idx int) {
defer wg.Done()
o := <-ch
fmt.Printf("%d received at %d\n", idx, o.receiver)
o.receiver--
if o.receiver > 0 {
ch <- o // forward to others
} else {
fmt.Printf("last receiver: %d\n", idx)
}
}
go sender()
for i:=0; i<reciever; i++ {
wg.Add(1)
go recv(i)
}
wg.Wait()
}
The output is random:
5 received at 25
24 received at 24
6 received at 23
7 received at 22
8 received at 21
9 received at 20
10 received at 19
11 received at 18
12 received at 17
13 received at 16
14 received at 15
15 received at 14
16 received at 13
17 received at 12
18 received at 11
19 received at 10
20 received at 9
21 received at 8
22 received at 7
23 received at 6
2 received at 5
0 received at 4
1 received at 3
3 received at 2
4 received at 1
last receiver 4
Quite an old question, but nobody mentioned this, I think.
First, the outputs of both examples can be different if you run the codes many times. This is not related to the Go version.
The output of the 1st example can be goroutine 4, goroutine 0, goroutine 1,... actually all the goroutine can be a one who sends the string to the main goroutine.
Main goroutine is one of the goroutines, so it's also waiting for data from the channel.
Which goroutine should receive the data? Nobody knows. It's not in the language spec.
Also, the output of the 2nd example also can be anything:
(I added the square brackets just for clarity)
// [original, hi from 4]
// [[[[[original, hi from 4], hi from 0], hi from 2], hi from 1], hi from 3]
// [[[[[original, hi from 4], hi from 1], hi from 0], hi from 2], hi from 3]
// [[[[[original, hi from 0], hi from 2], hi from 1], hi from 3], hi from 4]
// [[original, hi from 4], hi from 1]
// [[original, hi from 0], hi from 4]
// [[[original, hi from 4], hi from 1], hi from 0]
// [[[[[original, hi from 4], hi from 1], hi from 0], hi from 3], hi from 2]
// [[[[original, hi from 0], hi from 2], hi from 1], hi from 3]
//
// ......anything can be the output.
This is not magic, nor a mysterious phenomenon.
If there are multiple threads being executed, no one knows exactly which thread will acquire the resource. The language doesn't determine it. Rather, OS takes care of it. This is why multithread programming is quite complicated.
Goroutine is not OS thread, but it behaves somewhat similarly.
Use sync.Cond is a good choice.
ref: https://pkg.go.dev/sync#Cond

Go channels and deadlock

I'm trying to understand the Go language. I tried to create two goroutines
that chain the flow between them using two channels:
func main() {
c1 := make(chan int)
c2 := make(chan int)
go func() {
for i := range c1{
println("G1 got", i)
c2 <- i
}
}()
go func() {
for i := range c2 {
println("G2 got", i)
c1 <- i
}
}()
c1 <- 1
time.Sleep(1000000000 * 50)
}
As expected this code prints:
G1 got 1
G2 got 1
G1 got 1
G2 got 1
....
Until the main function exits.
But if I send another value to one of the channels from main, it suddenly blocks:
func main() {
c1 := make(chan int)
c2 := make(chan int)
go func() {
for i := range c1{
println("G1 got", i)
c2 <- i
}
}()
go func() {
for i := range c2 {
println("G2 got", i)
c1 <- i
}
}()
c1 <- 1
time.Sleep(1000000000 * 1)
c1 <- 2
time.Sleep(1000000000 * 50)
}
It outputs
G1 got 1
G2 got 1
G1 got 1
G2 got 1
G1 got 2
and then blocks until the main ends.
The value "2" sent to c1 arrives to the first goroutie, which sends it to c2, but the second
goroutine never receives.
(Using buffered channels with size 1 (either c1 or c2) works in this example)
Why does it happen? When this happens in real code, how can I debug it?
nmichaels is right on with his answer, but I thought I'd add that there are ways to figure out where you're deadlocking when debugging a problem like this.
A simple one is if you're on a Unix-like OS, run the command
kill -6 [pid]
This will kill the program and give a stack trace for each goroutine.
A slightly more involved way is to attach gdb.
gdb [executable name] [pid]
You can examine the stack and variables of the active goroutine as normal, but there's no easy way to switch goroutines that I know of. You can switch OS threads in the usual way, but that might not be enough to help.
Go channels created with make(chan int) are not buffered. If you want a buffered channel (that won't necessarily block), make it with make(chan int, 2) where 2 is the size of the channel.
The thing about unbuffered channels is that they are also synchronous, so they always block on write as well as read.
The reason it deadlocks is that your first goroutine is waiting for its c2 <- i to finish while the second one is waiting for c1 <- i to finish, because there was an extra thing in c1. The best way I've found to debug this sort of thing when it happens in real code is to look at what goroutines are blocked and think hard.
You can also sidestep the problem by only using synchronous channels if they're really needed.
to prevent the channel from overflowing, you can ask for the channel's current capacity and dry it before writing again.
in my case, the game takes place at 60fps and the mouse moves much faster, so it is always good to check that the channel has been cleared before writing again.
notice that the previous data is lost
package main
import (
"fmt"
)
func main() {
// you must specify the size of the channel,
// even for just one element, or the code doesn't work
ch := make( chan int, 1 )
fmt.Printf("len: %v\n", len(ch))
fmt.Printf("cap: %v\n\n", cap(ch))
ch <- 1
for i := 0; i != 100; i += 1 {
fmt.Printf("len: %v\n", len(ch))
fmt.Printf("cap: %v\n\n", cap(ch))
if cap( ch ) == 1 {
<- ch
}
ch <- i
fmt.Printf("len: %v\n", len(ch))
fmt.Printf("cap: %v\n\n", cap(ch))
}
fmt.Printf("end!\n")
}

Resources