I am trying to program a simple Go script that calculates the sum of the natural numbers up to 8:
package main
import "fmt"
func sum(nums []int, c chan int) {
var sum int = 0
for _, v := range nums {
sum += v
}
c <- sum
}
func main() {
allNums := []int{1, 2, 3, 4, 5, 6, 7, 8}
c1 := make(chan int)
c2 := make(chan int)
sum(allNums[:len(allNums)/2], c1)
sum(allNums[len(allNums)/2:], c2)
a := <- c1
b := <- c2
fmt.Printf("%d + %d is %d :D", a, b, a + b)
}
However, running this program produces the following output.
throw: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.sum(0x44213af00, 0x800000004, 0x420fbaa0, 0x2f29f, 0x7aaa8, ...)
main.go:9 +0x6e
main.main()
main.go:16 +0xe6
goroutine 2 [syscall]:
created by runtime.main
/usr/local/go/src/pkg/runtime/proc.c:221
exit status 2
Why is my code deadlocking? I am confused because I am using 2 separate channels to calculate the sub-sums. How are the two channels dependent at all?
Your channels are unbuffered, so the c <- sum line in sum() will block until some other routine reads from the other end.
One option would be to add buffers to the channels, so you can write a value to the channel without it blocking:
c1 := make(chan int, 1)
c2 := make(chan int, 1)
Alternatively, if you run the sum() function as a separate goroutine, then it can block while your main() function continues to the point where it reads from the channels.
Yes, you need to add go like
go sum(allNums[:len(allNums)/2], c1)
go sum(allNums[len(allNums)/2:], c2)
or
c1 := make(chan int,1)
c2 := make(chan int,1)
add channel cache.
I haven't used Go in a while, so this may not be the case, but from what I remember you need go to get another goroutine started, so:
go sum(allNums[:len(allNums)/2], c1)
go sum(allNums[len(allNums)/2:], c2)
If sum isn't running on another goroutine, it tries to execute:
c <- sum
But nothing's reading c; the code reading c has not been reached yet because it's waiting for sum to finish, and sum won't finish because it needs to give it to that code first!
Related
I have this code from Go tour:
func sum(s []int, c chan int) {
sum := 0
for _, v := range s {
sum += v
}
fmt.Printf("Sending %d to chan\n", sum)
c <- sum // send sum to c
}
func main() {
s := []int{2, 8, -9, 4, 0, 99}
c := make(chan int)
go sum(s[len(s)/2:], c)
go sum(s[:len(s)/2], c)
x, y := <-c, <-c // receive from c
fmt.Println(x, y, x+y)
}
Produces this output:
Sending 1 to chan
Sending 103 to chan
1 103 104
In this, x gets the second sum and y gets the first sum. Why is the order reversed?
Similar to goroutines order of execution
If you run it multiple times, it may give different result. When I run this I get:
Sending 103 to chan
Sending 1 to chan
103 1 104
If you want result to be deterministic. You may use two channels:
func main() {
s := []int{2, 8, -9, 4, 0, 99}
c1 := make(chan int)
c2 := make(chan int)
go sum(s[len(s)/2:], c1)
go sum(s[:len(s)/2], c2)
x, y := <-c1, <-c2 // receive from c
fmt.Println(x, y, x+y)
}
There are no guarantees in the execution ordering of goroutines. When you start multiple goroutines, they may or may not execute in the order you expect them unless there are explicit synchronizations between them, such as channels or other synchronization primitives.
In your case, the second goroutine writes the channel before the first can, because there is no mechanism to enforce ordering between the two goroutines.
The golang spec says about the channels:
Channels act as first-in-first-out queues. For example, if one
goroutine sends values on a channel and a second goroutine receives
them, the values are received in the order sent.
If you combine the above statement with the arbitrary order of the goroutines execution it could lead to arbitrary order of enqueuing items to channel.
NB: A channel is an abstraction of the CSP.
I'm trying to understand this piece of code, not sure why the 2nd go is executed before the 1st one. It'd be great if someone can really help me out with this!
func sum(a []int, c chan int) {
fmt.Println("summing: ", a)
total := 0
for _, v := range a {
total += v
}
//fmt.Println("send to c",total)
c <- total // send total to c
}
func main() {
//a := []int{7, 2, 8,134,23,23,1,23,1234,143, -9, 4, 0, 1234}
c := make(chan int)
go sum([]int{1,2,3}, c)
go sum([]int{4,5,6}, c)
x := <-c
fmt.Println(x)
x = <-c
fmt.Println(x)
}
OUTPUT:
summing: [4 5 6]
15
summing: [1 2 3]
6
You have nothing explicitly synchronizing the order of the two goroutines. If you run this enough times, you will see the calls to fmt.Println print in different sequences. When executing goroutines, as they are concurrent operations, you have no guarantees when they will execute and/or complete. You need to use various standard library packages, or channels themselves to synchronize the execution of concurrently running goroutines.
For example (by leveraging the blocking nature of channels, you could do something like):
func main() {
c := make(chan int)
go sum([]int{1, 2, 3}, c)
//use the channel to block until it receives a send
x := <-c
fmt.Println(x)
//then execute the next routine
go sum([]int{4, 5, 6}, c)
x = <-c
fmt.Println(x)
}
Another example (significantly less practical, but here to look at other common go synchronization features) you could introduce a wait group, and a range over a channel:
func sum(a []int, c chan int, wg *sync.WaitGroup) {
defer wg.Done()
fmt.Println("summing: ", a)
total := 0
for _, v := range a {
total += v
}
//fmt.Println("send to c",total)
c <- total // send total to c
}
func main() {
c := make(chan int)
wg := new(sync.WaitGroup)
//concurrently call the concurrent calls to sum, allowing execution to continue to the range of the channel
go func() {
//increment the wait group, and pass it to the sum func to decrement it when it is complete
wg.Add(1)
go sum([]int{1, 2, 3}, c, wg)
//wait for the above call to sum to complete
wg.Wait()
//and repeat...
wg.Add(1)
go sum([]int{4, 5, 6}, c, wg)
wg.Wait()
//all calls are complete, close the channel to allow the program to exit cleanly
close(c)
}()
//range of the channel
for theSum := range c {
x := theSum
fmt.Println(x)
}
}
I'm trying to use Go's concurrency to speed up my code,
Here's what I have:
for i:=7; i>-1; i-- {
go func (ch chan int32, ch2 chan int32, i int, arx int32, ary int32, dirf []int8, dirg []int8) {
nx := arx + int32(dirf[i])
ny := ary + int32(dirg[i])
ch <- nx
ch2 <- ny
}(ch, ch2, i, arx,ary,dirf,dirg)
}
for i:=7; i>-1; i-- {
nxx := <- ch
nyx := <- ch2
ind := nyx*w+nxx
if imData[ind] == e[i]{
process[c]=nxx
process[c+1]=nyx
c+=2
matrix[ind]=1
}
}
After running this I'm not getting the matrix slice that I expected, it's full of zeros.
But if I run the code below, it gives the matrix slice just as the code without channels but it's too slow.
for i:=7; i>-1; i-- {
go func (ch chan int32, ch2 chan int32, i int, arx int32, ary int32, dirf []int8, dirg []int8) {
nx := arx + int32(dirf[i])
ny := ary + int32(dirg[i])
ch <- nx
ch2 <- ny
}(ch, ch2, i, arx,ary,dirf,dirg)
nxx := <- ch
nyx := <- ch2
ind := nyx*w+nxx
if imData[ind] == e[i]{
process[c]=nxx
process[c+1]=nyx
c+=2
matrix[ind]=1
}
}
What's wrong with the first one? Any ideas? I'm very new at Go. So, please be clear when you are suggesting something.
Edit:
I edited the code to have values in proper order,
type data struct {
i int
nx int32
ny int32
}
for i:=7; i>-1; i-- {
go func (ch chan data, i int, arx int32, ary int32, dirf []int8, dirg []int8) {
nx := arx + int32(dirf[i])
ny := ary + int32(dirg[i])
ch <- data{i,nx,ny}
}(ch, i, arx,ary,dirf,dirg)
}
for i:=7; i>-1; i-- {
d := <- ch
nxx := d.nx
nyx := d.ny
j := d.i
ind := nyx*w+nxx
if imData[ind] == e[j]{
process[c]=nxx
process[c+1]=nyx
c+=2
matrix[ind]=1
}
}
It works now but it's still too slow.
I'm trying to speed up this main code:
for i:=7; i>-1; i-- {
nx := arx + int32(dirf[i])
ny := ary + int32(dirg[i])
ind := ny*w+nx
if imData[ind] == e[i]{
process[c]=nx
process[c+1]=ny
c+=2
matrix[ind]=1
}
}
what do you suggest with that?
In the second case, you're certain that the goroutines are executed "in the right order", since you wait for goroutines to complete before you continue on to the next one.
An example would be this minimal example on the golang playground. To fix this, you probably want pass a struct of three members, your nx, ny and i values across the channel.
I'm suspecting your "if imData[ind] == e[i]" conditional is failing in the former case, but it's hard to tell without the setup code for the channels, and more details on what those various slices hold. Have you tried running it with a print statement to see what you get from the channels?
Also, note that if the channels in question are buffered, there's no guarantee that the values in ch and ch2 are going to be in the same order. This is very possibly your issue.
Goroutine 1 could put a value on ch, but Goroutine 2 could have put a value on ch2 before Goroutine 1 gets to it. If you have 7 goroutines, it's perfectly possible to see the following ordering on the channels (or any number of others):
ch: 1, 2, 3, 4, 5, 6, 7
ch2: 1, 3, 4, 5, 6, 7, 2
If they aren't buffered, this isn't possible with your code, but it's still technically unsafe (edit: actually, it still won't match up with i in the second loop). If the data is a set of ordered pairs, you should be sending each pair as a structure over a single channel.
Incidentally, you only need to pass variables to the go func() call if they are expected to change outside that call. ch, ch2, arx, ary, dirf, and dirg all appear to be effectively constant for this block of code, and thus don't need to be passed into that go func(). You only need to pass in i, because the loop changes it immediately after firing the enclosure into a goroutine.
Now, from a pure speed perspective, you're probably better off moving the first loop inside the go func() call. Instead of creating 7 goroutines while looping in the main routine, you can fire a single routine, and it will loop over the values and send them on the channels. If the channels are buffered to at least that size, this becomes a VERY fast operation. Incidentally, this also solves the issue with channel ordering (though it's still better to send ordered pairs as a structure on a single channel), since you only have a single goroutine trying to send on the channels.
I have multiple goroutines trying to receive on the same channel simultaneously. It seems like the last goroutine that starts receiving on the channel gets the value. Is this somewhere in the language spec or is it undefined behaviour?
c := make(chan string)
for i := 0; i < 5; i++ {
go func(i int) {
<-c
c <- fmt.Sprintf("goroutine %d", i)
}(i)
}
c <- "hi"
fmt.Println(<-c)
Output:
goroutine 4
Example On Playground
EDIT:
I just realized that it's more complicated than I thought. The message gets passed around all the goroutines.
c := make(chan string)
for i := 0; i < 5; i++ {
go func(i int) {
msg := <-c
c <- fmt.Sprintf("%s, hi from %d", msg, i)
}(i)
}
c <- "original"
fmt.Println(<-c)
Output:
original, hi from 0, hi from 1, hi from 2, hi from 3, hi from 4
NOTE: the above output is outdated in more recent versions of Go (see comments)
Example On Playground
Yes, it's complicated, But there are a couple of rules of thumb that should make things feel much more straightforward.
prefer using formal arguments for the channels you pass to go-routines instead of accessing channels in global scope. You can get more compiler checking this way, and better modularity too.
avoid both reading and writing on the same channel in a particular go-routine (including the 'main' one). Otherwise, deadlock is a much greater risk.
Here's an alternative version of your program, applying these two guidelines. This case demonstrates many writers & one reader on a channel:
c := make(chan string)
for i := 1; i <= 5; i++ {
go func(i int, co chan<- string) {
for j := 1; j <= 5; j++ {
co <- fmt.Sprintf("hi from %d.%d", i, j)
}
}(i, c)
}
for i := 1; i <= 25; i++ {
fmt.Println(<-c)
}
http://play.golang.org/p/quQn7xePLw
It creates the five go-routines writing to a single channel, each one writing five times. The main go-routine reads all twenty five messages - you may notice that the order they appear in is often not sequential (i.e. the concurrency is evident).
This example demonstrates a feature of Go channels: it is possible to have multiple writers sharing one channel; Go will interleave the messages automatically.
The same applies for one writer and multiple readers on one channel, as seen in the second example here:
c := make(chan int)
var w sync.WaitGroup
w.Add(5)
for i := 1; i <= 5; i++ {
go func(i int, ci <-chan int) {
j := 1
for v := range ci {
time.Sleep(time.Millisecond)
fmt.Printf("%d.%d got %d\n", i, j, v)
j += 1
}
w.Done()
}(i, c)
}
for i := 1; i <= 25; i++ {
c <- i
}
close(c)
w.Wait()
This second example includes a wait imposed on the main goroutine, which would otherwise exit promptly and cause the other five goroutines to be terminated early (thanks to olov for this correction).
In both examples, no buffering was needed. It is generally a good principle to view buffering as a performance enhancer only. If your program does not deadlock without buffers, it won't deadlock with buffers either (but the converse is not always true). So, as another rule of thumb, start without buffering then add it later as needed.
Late reply, but I hope this helps others in the future like Long Polling, "Global" Button, Broadcast to everyone?
Effective Go explains the issue:
Receivers always block until there is data to receive.
That means that you cannot have more than 1 goroutine listening to 1 channel and expect ALL goroutines to receive the same value.
Run this Code Example.
package main
import "fmt"
func main() {
c := make(chan int)
for i := 1; i <= 5; i++ {
go func(i int) {
for v := range c {
fmt.Printf("count %d from goroutine #%d\n", v, i)
}
}(i)
}
for i := 1; i <= 25; i++ {
c<-i
}
close(c)
}
You will not see "count 1" more than once even though there are 5 goroutines listening to the channel. This is because when the first goroutine blocks the channel all other goroutines must wait in line. When the channel is unblocked, the count has already been received and removed from the channel so the next goroutine in line gets the next count value.
I've studied existing solutions and created simple broadcast library https://github.com/grafov/bcast.
group := bcast.NewGroup() // you created the broadcast group
go bcast.Broadcasting(0) // the group accepts messages and broadcast it to all members
member := group.Join() // then you join member(s) from other goroutine(s)
member.Send("test message") // or send messages of any type to the group
member1 := group.Join() // then you join member(s) from other goroutine(s)
val := member1.Recv() // and for example listen for messages
It is complicated.
Also, see what happens with GOMAXPROCS = NumCPU+1. For example,
package main
import (
"fmt"
"runtime"
)
func main() {
runtime.GOMAXPROCS(runtime.NumCPU() + 1)
fmt.Print(runtime.GOMAXPROCS(0))
c := make(chan string)
for i := 0; i < 5; i++ {
go func(i int) {
msg := <-c
c <- fmt.Sprintf("%s, hi from %d", msg, i)
}(i)
}
c <- ", original"
fmt.Println(<-c)
}
Output:
5, original, hi from 4
And, see what happens with buffered channels. For example,
package main
import "fmt"
func main() {
c := make(chan string, 5+1)
for i := 0; i < 5; i++ {
go func(i int) {
msg := <-c
c <- fmt.Sprintf("%s, hi from %d", msg, i)
}(i)
}
c <- "original"
fmt.Println(<-c)
}
Output:
original
You should be able to explain these cases too.
For multiple goroutine listen on one channel, yes, it's possible. the key point is the message itself, you can define some message like that:
package main
import (
"fmt"
"sync"
)
type obj struct {
msg string
receiver int
}
func main() {
ch := make(chan *obj) // both block or non-block are ok
var wg sync.WaitGroup
receiver := 25 // specify receiver count
sender := func() {
o := &obj {
msg: "hello everyone!",
receiver: receiver,
}
ch <- o
}
recv := func(idx int) {
defer wg.Done()
o := <-ch
fmt.Printf("%d received at %d\n", idx, o.receiver)
o.receiver--
if o.receiver > 0 {
ch <- o // forward to others
} else {
fmt.Printf("last receiver: %d\n", idx)
}
}
go sender()
for i:=0; i<reciever; i++ {
wg.Add(1)
go recv(i)
}
wg.Wait()
}
The output is random:
5 received at 25
24 received at 24
6 received at 23
7 received at 22
8 received at 21
9 received at 20
10 received at 19
11 received at 18
12 received at 17
13 received at 16
14 received at 15
15 received at 14
16 received at 13
17 received at 12
18 received at 11
19 received at 10
20 received at 9
21 received at 8
22 received at 7
23 received at 6
2 received at 5
0 received at 4
1 received at 3
3 received at 2
4 received at 1
last receiver 4
Quite an old question, but nobody mentioned this, I think.
First, the outputs of both examples can be different if you run the codes many times. This is not related to the Go version.
The output of the 1st example can be goroutine 4, goroutine 0, goroutine 1,... actually all the goroutine can be a one who sends the string to the main goroutine.
Main goroutine is one of the goroutines, so it's also waiting for data from the channel.
Which goroutine should receive the data? Nobody knows. It's not in the language spec.
Also, the output of the 2nd example also can be anything:
(I added the square brackets just for clarity)
// [original, hi from 4]
// [[[[[original, hi from 4], hi from 0], hi from 2], hi from 1], hi from 3]
// [[[[[original, hi from 4], hi from 1], hi from 0], hi from 2], hi from 3]
// [[[[[original, hi from 0], hi from 2], hi from 1], hi from 3], hi from 4]
// [[original, hi from 4], hi from 1]
// [[original, hi from 0], hi from 4]
// [[[original, hi from 4], hi from 1], hi from 0]
// [[[[[original, hi from 4], hi from 1], hi from 0], hi from 3], hi from 2]
// [[[[original, hi from 0], hi from 2], hi from 1], hi from 3]
//
// ......anything can be the output.
This is not magic, nor a mysterious phenomenon.
If there are multiple threads being executed, no one knows exactly which thread will acquire the resource. The language doesn't determine it. Rather, OS takes care of it. This is why multithread programming is quite complicated.
Goroutine is not OS thread, but it behaves somewhat similarly.
Use sync.Cond is a good choice.
ref: https://pkg.go.dev/sync#Cond
I'm trying to understand the Go language. I tried to create two goroutines
that chain the flow between them using two channels:
func main() {
c1 := make(chan int)
c2 := make(chan int)
go func() {
for i := range c1{
println("G1 got", i)
c2 <- i
}
}()
go func() {
for i := range c2 {
println("G2 got", i)
c1 <- i
}
}()
c1 <- 1
time.Sleep(1000000000 * 50)
}
As expected this code prints:
G1 got 1
G2 got 1
G1 got 1
G2 got 1
....
Until the main function exits.
But if I send another value to one of the channels from main, it suddenly blocks:
func main() {
c1 := make(chan int)
c2 := make(chan int)
go func() {
for i := range c1{
println("G1 got", i)
c2 <- i
}
}()
go func() {
for i := range c2 {
println("G2 got", i)
c1 <- i
}
}()
c1 <- 1
time.Sleep(1000000000 * 1)
c1 <- 2
time.Sleep(1000000000 * 50)
}
It outputs
G1 got 1
G2 got 1
G1 got 1
G2 got 1
G1 got 2
and then blocks until the main ends.
The value "2" sent to c1 arrives to the first goroutie, which sends it to c2, but the second
goroutine never receives.
(Using buffered channels with size 1 (either c1 or c2) works in this example)
Why does it happen? When this happens in real code, how can I debug it?
nmichaels is right on with his answer, but I thought I'd add that there are ways to figure out where you're deadlocking when debugging a problem like this.
A simple one is if you're on a Unix-like OS, run the command
kill -6 [pid]
This will kill the program and give a stack trace for each goroutine.
A slightly more involved way is to attach gdb.
gdb [executable name] [pid]
You can examine the stack and variables of the active goroutine as normal, but there's no easy way to switch goroutines that I know of. You can switch OS threads in the usual way, but that might not be enough to help.
Go channels created with make(chan int) are not buffered. If you want a buffered channel (that won't necessarily block), make it with make(chan int, 2) where 2 is the size of the channel.
The thing about unbuffered channels is that they are also synchronous, so they always block on write as well as read.
The reason it deadlocks is that your first goroutine is waiting for its c2 <- i to finish while the second one is waiting for c1 <- i to finish, because there was an extra thing in c1. The best way I've found to debug this sort of thing when it happens in real code is to look at what goroutines are blocked and think hard.
You can also sidestep the problem by only using synchronous channels if they're really needed.
to prevent the channel from overflowing, you can ask for the channel's current capacity and dry it before writing again.
in my case, the game takes place at 60fps and the mouse moves much faster, so it is always good to check that the channel has been cleared before writing again.
notice that the previous data is lost
package main
import (
"fmt"
)
func main() {
// you must specify the size of the channel,
// even for just one element, or the code doesn't work
ch := make( chan int, 1 )
fmt.Printf("len: %v\n", len(ch))
fmt.Printf("cap: %v\n\n", cap(ch))
ch <- 1
for i := 0; i != 100; i += 1 {
fmt.Printf("len: %v\n", len(ch))
fmt.Printf("cap: %v\n\n", cap(ch))
if cap( ch ) == 1 {
<- ch
}
ch <- i
fmt.Printf("len: %v\n", len(ch))
fmt.Printf("cap: %v\n\n", cap(ch))
}
fmt.Printf("end!\n")
}