I am trying hard to understand concurrency in Go.
package main
import "fmt"
func sendValues(myIntChannel chan int) {
for i := 0; i < 5; i++ {
myIntChannel <- i //sending value
}
}
func main() {
myIntChannel := make(chan int)
defer close(myIntChannel)
go sendValues(myIntChannel)
for value := range myIntChannel {
fmt.Println(value) //receiving value
}
}
Above code is giving below output:
0
1
2
3
4
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
/Users/spikki/Desktop/GoLearning/go_channel.go:51 +0x10b
As per my understanding, defer function will execute after completion of its surrounding functions. I am not able to interpret it.
If I use for loop to receive the values from the channel its working as below.
for i := 0; i < 5; i++ {
fmt.Println(<-myIntChannel) //receiving value
}
Can anyone help me to understand this concept?
for ... range over a channel terminates only once all values have been received from the channel and the channel has been closed.
In your example you wish to close the channel in a deferred function, but that would only run when main() returns. But main() would only return if the loop would end. This is the cause of deadlock. The for loop waits for the channel to be closed, and closing the channel waits for the for loop to end.
When you use a loop to receive exactly 5 values form the channel, it works because the launched goroutines sends 5 values on it. This loop doesn't wait for the channel to be closed, so the loop can end, and so can the main() function.
That's why the sender should close the channel (not the receiver), and problem is solved immediately:
func sendValues(myIntChannel chan int) {
for i := 0; i < 5; i++ {
myIntChannel <- i //sending value
}
close(myIntChannel)
}
func main() {
myIntChannel := make(chan int)
go sendValues(myIntChannel)
for value := range myIntChannel {
fmt.Println(value) //receiving value
}
}
Output (try it on the Go Playground):
0
1
2
3
4
To explain it in a bit different terms, what your code does is:
func main() {
...
while myIntChannel is not closed {
...
}
close myIntChannel
}
Now you can see where the deadlock comes from.
While the above answer is valid, you could also try this, if you prefer using defer:
func sendValues(myIntChannel chan int) {
defer close(myIntChannel)
for i := 0; i < 5; i++ {
myIntChannel <- i //sending value
}
}
func main() {
myIntChannel := make(chan int)
go sendValues(myIntChannel)
for value := range myIntChannel {
fmt.Println(value) //receiving value
}
}
Related
why there is deadlock after printing all values ?
what i understand
as from receiving part code channel is waiting which letting to block or pause main go routine although i tried with waitgroup doesn't work
package main
import (
"fmt"
//"sync"
)
//output from 10 20 30 ... - 100
func main() {
//wg := sync.WaitGroup{}
done := make(chan int)
for i := 1; i <= 10; i++ {
//wg.Add(1)
go func(i int) {
done <- i * 10
}(i)
// close(done)
}
// close(done)
//wg.Wait()
// for item := range done{
// fmt.Println(item)}
for {
if value, ok := <-done; ok {
fmt.Println("received is ", value)
} else {
return
//os.Exit(1)
}
}
}
As per the answer from #aureliar value, ok := <-done will block until a value is received on the channel or the channel is closed (and, once your goroutines complete, neither of these happen). From your question and the comments in your code it looks like you were close to working out how to solve this by waiting for the goroutines to complete and closing the channel.
Because you know the number of goroutines in advance (and each goroutine always sends one value on the channel) there is a simple solution (playground):
func main() {
noOfGoRoutines := 10
done := make(chan int)
for i := 1; i <= noOfGoRoutines; i++ {
go func(i int) {
done <- i * 10
}(i)
}
for noOfGoRoutines > 0 {
value := <-done
fmt.Println("received is ", value)
noOfGoRoutines--
}
}
Things get a bit more complicated when you don't know how many values will be received in advance. In that case closing the channel is a good way of letting the receiver know you have finished. In your case this means closing the channel after the goroutines have completed (this is important because sending to a closed channel leads to a panic).
To do this using a WaitGroup you will need three functions:
wg.Add(delta int) to set the counter. The call you have commented out is fine but an alternative is to call wg.Add(10) before entering the loop.
wg.Done() "decrements the WaitGroup counter by one" - you need to call this before each goroutine ends (its common to defer the call). This was missing from your code.
wg.Wait() "blocks until the WaitGroup counter is zero".
With the above in place you can safely call close(done) knowing that nothing further will be sent to the channel. However there is a complication - if you just add this code after your first loop you will hit another deadlock because your goroutines will all block at done <- i * 10. This happens because main will be blocked at wg.Wait() meaning nothing is receiving from the channel and as per the spec:
If the capacity is zero or absent, the channel is unbuffered and communication succeeds only when both a sender and receiver are ready.
This can be solved by waiting/closing within another goroutine.
You can try this in the playground.
package main
import (
"fmt"
"sync"
)
//output from 10 20 30 ... - 100
func main() {
wg := sync.WaitGroup{}
done := make(chan int)
for i := 1; i <= 10; i++ {
wg.Add(1)
go func(i int) {
done <- i * 10
wg.Done()
}(i)
}
go func() {
wg.Wait()
close(done)
}()
for value := range done {
fmt.Println("received is ", value)
}
fmt.Println("channel closed")
/* I have simplified your loop but this would also work
for {
if value, ok := <-done; ok {
fmt.Println("received is ", value)
} else {
fmt.Println("channel closed")
return
}
}
*/
}
Because you never close the channel. So the value, ok := <-done part is always waiting for the 11th value that will never come.
Replacing this part should do the trick:
for i := 1; i <= 10; i++ {
//wg.Add(1)
go func(i int) {
done <- i * 10
}(i)
// close(done)
}
new:
go func(){
for i := 1; i <= 10; i++ {
done <- i * 10
}
close(done)
}()
This question already has answers here:
What is channel buffer size?
(3 answers)
Closed 7 years ago.
I went through a series of definitions to figure out how the buffer works but I just don't get it. Here is an example below, I changed the value of the buffer but I have no clue about what it does. Can some one explain it to me based on this example and provide some test cases of how/why it's working? Thanks.
package main
import (
"fmt"
"time"
)
func send(out, finish chan bool) {
for i := 0; i < 5; i++ {
out <- true
time.Sleep(1 * time.Second)
fmt.Println("Fin d'une écriture")
}
finish <- true
close(out)
}
func recv(in, finish chan bool) {
for _ = range in {
fmt.Println("Fin d'une lecture")
time.Sleep(10 * time.Second)
}
finish <- true
}
func main() {
chanFoo := make(chan bool, 3)
chanfinish := make(chan bool)
go send(chanFoo, chanfinish)
go recv(chanFoo, chanfinish)
<-chanfinish
<-chanfinish
}
If a channel does not have a buffer then only a single item can be sent on it at a time. This means code that sends on it will block until some receiver reads the item out of the channel. Here's a contrived example; https://play.golang.org/p/HM8jdIFqsN
package main
import (
"fmt"
)
func main() {
blocker := make(chan bool)
nonBlocker := make(chan bool, 5)
for i := 0; i < 5; i++ {
nonBlocker <- true
fmt.Println("We keep going")
}
go func () {
for i := 0; i < 5; i++ {
blocker <- true
fmt.Println("We block cause that channel is full")
} }()
}
There's plenty of other things I could do to demonstrate the same but the basic idea is, if you pass a channel into some goroutine and the channel is not buffered, the goroutine which sends on the channel will block until the item it sent is received. With a buffered channel you can send as long as the buffer isn't at capacity. Basically, if you spin up goroutines which are doing work and returning the results and they're moving faster than the code that spawned them, you may want to use a buffered channel to open up that bottle neck.
EDIT: If it's still not obvious what's happening look at this; https://play.golang.org/p/9SXc4M1to4
package main
import (
"fmt"
)
func main() {
blocker := make(chan bool)
nonBlocker := make(chan bool, 5)
for i := 0; i < 5; i++ {
nonBlocker <- true
fmt.Println("We keep going")
}
go func () {
for i := 0; i < 5; i++ {
blocker <- true
fmt.Println("Now we see this cause the reciever keeps opening the channel up again!")
} }()
for i := 0; i < 5; i++ {
<-blocker
}
}
I'm trying to spin off a set of goroutines, and then wait for them all to finish.
import "sync"
func doWork(wg sync.WaitGroup) error {
defer wg.Done()
// Do some heavy lifting... request URL's or similar
return nil
}
func main() {
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go doWork(wg)
}
wg.Wait()
}
However when I run this code I get the following error:
fatal error: all goroutines are asleep - deadlock!
goroutine 16 [semacquire]:
sync.runtime_Semacquire(0xc20818c658)
/usr/local/Cellar/go/1.3/libexec/src/pkg/runtime/sema.goc:199 +0x30
sync.(*WaitGroup).Wait(0xc2080544e0)
/usr/local/Cellar/go/1.3/libexec/src/pkg/sync/waitgroup.go:129 +0x14b
main.main()
/Users/kevin/code/vrusability/scripts/oculus_share_ratings.go:150 +0x398
I'm confused because I wrote it pretty much exactly as the documentation example demonstrates.
You need to pass a pointer to the WaitGroup, and not the WaitGroup object. When you pass the actual WaitGroup, Go makes a copy of the value, and calls Done() on the copy. The result is the original WaitGroup will have ten Add's and no Done's, and each copy of the WaitGroup will have one Done() and however many Add's were there when the WaitGroup was passed to the function.
Pass a pointer instead, and every function will reference the same WaitGroup.
import "sync"
func doWork(wg *sync.WaitGroup) error {
defer wg.Done()
// Do some heavy lifting... request URL's or similar
return nil
}
func main() {
wg := &sync.WaitGroup{}
for i := 0; i < 10; i++ {
wg.Add(1)
go doWork(wg)
}
}
As #Kevin mentioned, you will need to pass a reference to your WaitGroup. This is actually the one thing I do not like about WaitGroup because you would be mixing your concurrency logic with your business logic.
So I came up with this generic function to solve this problem for me:
// Parallelize parallelizes the function calls
func Parallelize(functions ...func()) {
var waitGroup sync.WaitGroup
waitGroup.Add(len(functions))
defer waitGroup.Wait()
for _, function := range functions {
go func(copy func()) {
defer waitGroup.Done()
copy()
}(function)
}
}
Here is an example:
func1 := func() {
for char := 'a'; char < 'a' + 3; char++ {
fmt.Printf("%c ", char)
}
}
func2 := func() {
for number := 1; number < 4; number++ {
fmt.Printf("%d ", number)
}
}
Parallelize(func1, func2) // a 1 b 2 c 3
If you would like to use it, you can find it here https://github.com/shomali11/util
I am trying to create a simple program to learn channels in Go.
But I´m running in to a deadlock error, which I can´t figure out
package main
import (
"fmt"
"time"
)
func printer(c chan int) {
for i := 0; i < 10; i++ {
c <- i
time.Sleep(time.Second)
}
}
func reciever(c chan int) {
for {
recievedMsg := <-c
fmt.Println(recievedMsg)
}
}
func main() {
newChanel := make(chan int)
printer(newChanel)
reciever(newChanel)
}
My initial thoughts was something about the Sleep function, but even if I don´t include this I still run into this error and exit message.
Can anyone give some hints on how to solve this?
Thanks in advance
You need two execution threads because now there is no way for the reciever function to be called as you never leave the printer function. You need to execute one of them on a separate goroutine.
You should also close the channel and use the range operator in your loop, so that it ends when the channel is closed.
So I propose you this code :
func printer(c chan int) {
for i := 0; i < 10; i++ {
c <- i
time.Sleep(time.Second)
}
close(c)
}
func reciever(c chan int) {
for recievedMsg := range c {
fmt.Println(recievedMsg)
}
}
func main() {
newChanel := make(chan int)
go printer(newChanel)
reciever(newChanel)
}
I try to write a simple code in GO, in which two go routine (Send and Receive) send each other integers. I give the code below. Can anybody help me why the output of this program is [no output]. Is there any silly mistake (sorry, I am new in GO) ?
package main
func Send (in1 <-chan int, out1 chan<- int) {
i := 2
out1 <- i
print(i, "\n")
}
func Receive (in <-chan int, out chan<- int) {
i := <-in
print(i, "\n")
out <- i
}
func main() {
for i := 0; i < 10; i++ {
ch1 := make(chan int)
ch := make(chan int)
go Send (ch1 , ch)
go Receive (ch , ch1)
ch = ch1
ch1 = ch
}
}
How about this:
package main
func Send (ch chan<- int) {
for i := 0; i < 10; i++ {
print(i, " sending\n")
ch <- i
}
}
func Receive (ch <-chan int) {
for i := 0; i < 10; i++ {
print(<-ch, " received\n")
}
}
func main() {
ch := make(chan int)
go Receive(ch)
Send(ch)
}
The output of this when I run it on golang.org is:
0 sending
0 received
1 sending
2 sending
1 received
2 received
3 sending
4 sending
3 received
4 received
5 sending
6 sending
5 received
6 received
7 sending
8 sending
7 received
8 received
9 sending
I'm not sure why the 9 was never received. There should be some way to sleep the main thread until the Receive goroutine has completed. Also, it's inelegant that the receiver and the sender both know that they are going to send 10 numbers. One of the goroutines should just shut off when the other has finished its work. I'm not sure how to do that.
EDIT1:
Here is an implementation with two channels, and the go routines send ints back and forth between eachother. One is designated as the responder, who only sends and int after it receives an int. The responder simply adds two to the int he receives, and then sends it back.
package main
func Commander(commands chan int, responses chan int) {
for i := 0; i < 10; i++ {
print(i, " command\n")
commands <- i
print(<-responses, " response\n");
}
close(commands)
}
func Responder(commands chan int, responses chan int) {
for {
x, open := <-commands
if !open {
return;
}
responses <- x + 2
}
}
func main() {
commands := make(chan int)
responses := make(chan int)
go Commander(commands, responses)
Responder(commands, responses)
}
The output when I run it on golang.org is:
0 command
2 response
1 command
3 response
2 command
4 response
3 command
5 response
4 command
6 response
5 command
7 response
6 command
8 response
7 command
9 response
8 command
10 response
9 command
11 response
David Grayson's fix works, but the explanation of why 9 is not always received wouldn't fit into a comment!
When the sending goroutine sends the value on the channel, it blocks until the receiving goroutine receives it. At that point, it's unblocked. The go scheduler might return from main almost immediately before it gives the receiving goroutine a chance to print anything. If you set GOMAXPROCS to something greater than 1, this happens less frequently because the receiving goroutine won't block while the main goroutine is active.
If you want to stop main from returning until all goroutines are finished, you can return a channel from Receive(), like this:
package main
import "fmt"
func Send(ch chan<- int) {
for i := 0; i < 10; i++ {
fmt.Println(i, " sending")
ch <- i
}
close(ch)
}
func Receive(ch <-chan int) (<-chan bool) {
done := make(chan bool)
go func() {
for {
i, ok := <-ch
if !ok {
break
}
fmt.Println(i, " received")
}
done <- true
}()
return done
}
func main() {
ch := make(chan int)
d := Receive(ch)
Send(ch)
_ = <-d
}
As for why your output was empty in your original example: you make two goroutines, and thus main is totally unblocked, so it just returns and the program exits! I've definitely made this mistake too, but the runtime only joins with the main goroutine, not with ALL goroutines, like you may expect.