I just started to learn Go so please bear with me, I've tried to play around with Go routines and channels but are getting a deadlock somehow.
Here's the example
package main
import (
"fmt"
"sync"
)
func main() {
total := 2
var wg sync.WaitGroup
wg.Add(total)
ch := make(chan int)
for idx := 0; idx < total; idx++ {
fmt.Printf("Processing idx %d\n", idx)
go func(idx int) {
defer wg.Done()
ch <- idx
}(idx)
}
for val := range ch {
fmt.Println(val)
}
fmt.Println("Wait")
wg.Wait()
}
which throws the error
Processing idx 0
Processing idx 1
1
0
fatal error: all goroutines are asleep - deadlock!
range ch reads from the channel until it is closed.
How many times do you call close(ch)? When will the for val := range ch loop terminate?
When should you close the channel? You have a lot of options here, but one way to do it is to add another goroutine:
go func() {
wg.Wait()
close(ch)
}()
e.g., after spinning off all routines that will write-to-channel-then-call-wg.Done(), so that the channel is closed once all the writers are done writing. (You can run this goroutine as soon as you've increased the wg count to account for all writers.)
Related
I am experimenting with channel concept in Go. I wrote the below program playground to implement counter using channels. But I am not getting any output, although I am doing some printing in the goroutine body.
func main() {
wg := sync.WaitGroup{}
ch := make(chan int)
count := func(ch chan int) {
var last int
last = <-ch
last = last + 1
fmt.Println(last)
ch <- last
wg.Done()
}
wg.Add(10)
for i := 1; i <= 10; i++ {
go count(ch)
}
}
I expect at least some output but I am getting none at all.
When the main() function (the main goroutine) ends, your program ends as well (it doesn't wait for other non-main goroutines to finish). You must add a wg.Wait() call to the end. See No output from goroutine in Go.
Once you do this, you'll hit a deadlock. This is because all goroutines start with attempting to receive a value from the channel, and only then would they send something.
So you should first send something on the channel to let at least one of the goroutines to proceed.
Once you do that, you'll see numbers printed 10 times, and again deadlock. This is because when the last goroutine tries to send its incremented number, there will be no one to receive that. An easy way to fix that is to give a buffer to the channel.
Final, working example:
wg := sync.WaitGroup{}
ch := make(chan int, 2)
count := func(ch chan int) {
var last int
last = <-ch
last = last + 1
fmt.Println(last)
ch <- last
wg.Done()
}
wg.Add(10)
for i := 1; i <= 10; i++ {
go count(ch)
}
go func() {
ch <- 0
}()
wg.Wait()
Outputs (try it on the Go Playground):
1
2
3
4
5
6
7
8
9
10
Also note that since we made the channel buffered, it's not necessary to use another goroutine to send an initial value, we can do that in the main goroutine:
ch <- 0
wg.Wait()
This will output the same. Try it on the Go Playground.
func main() {
wg := sync.WaitGroup{}
ch := make(chan int)
count := func(ch chan int) {
var last int
last, ok := <-ch // 这里做一层保护
if !ok {
return
}
last = last + 1
fmt.Println(last)
go func(ch chan int, res int) {
ch <- res
}(ch, last)
wg.Done()
}
go func() {
ch <- 0
}()
wg.Add(10)
for i := 1; i <= 10; i++ {
go count(ch)
}
wg.Wait()
fmt.Println("main finish")
close(ch)
}
When writing data into same channel using multiple go routines with waitgroup after waiting wg.Wait() getting exception saying all go routines are asleep or deedlock.
package main
import (
"fmt"
"runtime"
"sync"
)
var wg sync.WaitGroup
func CreateMultipleRoutines() {
ch := make(chan int)
for i := 0; i < 10; i++ { // creates 10 go routines and adds to waitgroup
wg.Add(1)
go func() {
for j := 0; j < 10; j++ {
ch <- j
}
wg.Done() // indication of go routine is done to main routine
}()
}
fmt.Println(runtime.NumGoroutine())
wg.Wait() //wait for all go routines to complete
close(ch) // closing channel after completion of wait fo go routines
for v := range ch { // range can be used since channel is closed
fmt.Println(v)
}
fmt.Println("About to exit program ...")
}
When tried to implement this without waitgroup I am able to read data from channel by looping exact number of times data pushed to channel but i cant range since there will be panic when we close channel. here is the example code
package main
import (
"fmt"
"runtime"
)
func main() {
ch := make(chan int)
for i := 0; i < 10; i++ { // creates 10 go routines and adds to waitgroup
go func(i int) {
for j := 0; j < 10; j++ {
ch <- j * i
}
}(i)
}
fmt.Println(runtime.NumGoroutine())
for v := 0; v < 100; v++ {
fmt.Println(<-ch)
}
fmt.Println("About to exit program ...")
}
I want to understand why waitgroup in wait state is still waiting even though all go routines are signalled Done() which inturn makes number of go routines to zero
I think your original code has some problems.
You are closing the channel before reading from it.
You are not getting the advantage of using 10 goroutines because of your channel is 1 "sized". So one goroutine is producing one result per once.
My solution would be to spawn a new goroutine to monitor if the 10 goroutines finished its jobs. There you will use your WaitGroup.
Then the code would be like:
package main
import (
"fmt"
"runtime"
"sync"
)
var wg sync.WaitGroup
func main() {
ch := make(chan int, 10)
for i := 0; i < 10; i++ { // creates 10 go routines and adds to waitgroup
wg.Add(1)
go func() {
for j := 0; j < 10; j++ {
ch <- j
}
wg.Done() // indication of go routine is done to main routine
}()
}
go func(){
wg.Wait()
close(ch)
}()
fmt.Println(runtime.NumGoroutine())
for v := range ch { // range can be used since channel is closed
fmt.Println(v)
}
fmt.Println("About to exit program ...")
}
By default a chan holds no items, so all go routines are blocked on sending, until something reads from it. They never actually reach the wg.Done() statement.
A solution would be to close the channel in it's own go routine. Wrap your wg.Wait() and close(ch) lines like this:
go func() {
wg.Wait() //wait for all go routines to complete
close(ch) // closing channel after completion of wait fo go routines
}()
Then you can range over the channel, which will only close after all of the sending go routines have finished (and implicitly all values have been received).
I write some code which aims to synchronize using channel.
var counter int64 // shared resource
var wg sync.WaitGroup
func main() {
ch := make(chan int64)
wg.Add(2)
go incCounter(ch)
go incCounter(ch)
ch <- counter
wg.Wait()
fmt.Println("Final Counter:", counter) // expected value is 4
}
func incCounter(ch chan int64) {
defer wg.Done()
for count := 0; count < 2; count++ {
value := <-ch
value++
counter = value
ch <- counter
}
}
When I ran this program, an error happened: all goroutines are asleep - deadlock!. However I can't fix the problem and I don't know what is wrong. Could anyone help?
Channels make(chan int) has implicit size zero ( ref: https://golang.org/ref/spec#Making_slices_maps_and_channels)
A channel of size zero is unbuffered. A channel of specified size make(chan int, n) is buffered. See http://golang.org/ref/spec#Send_statements for a discussion on buffered vs. unbuffered channels. The example at http://play.golang.org/p/VZAiN1V8-P illustrates the difference.
Here, channel <-ch or ch <- will be blocked until someone processes it (concurrently). If you try the flow of this program in pen and paper, you will figure it out why it is blocked. Below figure shows possible data flow through channel ch:
So if you make your ch := make(chan int64) to ch := make(chan int64,1), it will work.
var counter int64 // shared resource
var wg sync.WaitGroup
func main() {
ch := make(chan int64, 1)
wg.Add(2)
go incCounter(ch)
go incCounter(ch)
ch <- counter
wg.Wait()
fmt.Println("Final Counter:", counter) // expected value is 4
}
func incCounter(ch chan int64) {
defer wg.Done()
for count := 0; count < 2; count++ {
value := <-ch
value++
counter = value
ch <- counter
}
}
If we analyse how program works when you are using ch := make(chan int64), we can see that one go routine is blocked in this program(another one is exited). With the help of time.Sleep(n) and receiving the last data from the channel in the blocked go routine, we can overcome the deadlock. See the code below:
var counter int64 // shared resource
var wg sync.WaitGroup
func main() {
ch := make(chan int64)
wg.Add(2)
go incCounter(ch)
go incCounter(ch)
ch <- counter
// to ensure one go routine 'incCounter' is completed and one go routine is blocked for unbuffered channel
time.Sleep(3*time.Second)
<-ch // to unblock the last go routine
wg.Wait()
fmt.Println("Final Counter:", counter) // expected value is 4
}
func incCounter(ch chan int64) {
defer wg.Done()
for count := 0; count < 2; count++ {
value := <-ch
value++
counter = value
ch <- counter
}
}
I'm new to concurrency in Go and I'm trying to figure out how to use channels to control concurrency. What I would like to do have a loop where I can call out to a function using a new go routine and continue looping while that function processes and I would like to limit the number of routines that run to 3. My first attempt to do this was the code below:
func write(val int, ch chan bool) {
fmt.Println("Processing:", val)
time.Sleep(2 * time.Second)
ch <- val % 3 == 0
}
func main() {
ch := make(chan bool, 3) // limit to 3 routines?
for i := 0; i< 10; i++ {
go write(i, ch)
resp := <- ch
fmt.Println("Divisible by 3:", resp)
}
time.Sleep(20 * time.Second)
}
I was under the impression that this would basically make calls to write 3 at a time and then hold off on processing the next 3 until the first 3 had finished. Based on what is logging it appears to only be processing one at a time. The code can be found and executed here.
What would I need to change in this example to get the functionality that I described above?
The problem here is very simple:
for i := 0; i< 10; i++ {
go write(i, ch)
resp := <- ch
fmt.Println("Divisible by 3:", resp)
}
You spin up a goroutine, then wait for it to respond, before you continue around the loop and spin up the next goroutine. They can't run in parallel because you never run two of them at the same time.
To fix this, you need to spin up all 10 goroutines, then wait on all 10 responses (playground):
for i := 0; i< 10; i++ {
go write(i, ch)
}
for i := 0; i<10; i++ {
resp := <- ch
fmt.Println("Divisible by 3:", resp)
}
Now you do have 7 goroutines blocking on the channel—but it's so brief that you can't see it happening, so the output won't be very interesting. If you try adding a Processed message at the end of the goroutine, and sleeping between each channel read, you'll see that 3 of them finish immediately (well, after waiting 2 seconds), and then the others unblock and finish one by one (playground).
There is one more way to run go routines in parallel with wait for all of them to return the value on channel is using Wait groups. It also helps to synchronize go routines. If you are working with go routines to wait for all of them to finish before executing another function better approach is to use wait group.
package main
import (
"fmt"
"time"
"sync"
)
func write(val int, wg *sync.WaitGroup, ch chan bool) {
defer wg.Done()
fmt.Println("Processing:", val)
time.Sleep(2 * time.Second)
ch <- val % 3 == 0
}
func main() {
wg := &sync.WaitGroup{}
ch := make(chan bool, 3)
for i := 0; i< 10; i++ {
wg.Add(1)
go write(i, wg, ch)
}
for i := 0; i< 10; i++ {
fmt.Println("Divisible by 3: ", <-ch)
}
close(ch)
wg.Wait()
time.Sleep(20 * time.Second)
}
Playground example
Here is my code at Go Playground
package main
import (
"fmt"
)
func sum_up(my_int int, cs chan int) {
my_sum := 0
for i := 0; i < my_int; i++ {
my_sum += i
}
cs <- my_sum
}
func main() {
my_channel := make(chan int)
for i := 2; i < 5; i++ {
go sum_up(i, my_channel)
}
for ele := range my_channel {
fmt.Println(ele)
}
//fatal error: all goroutines are asleep - deadlock!
fmt.Println("Done")
}
Which results in:
1
3
6
fatal error: all goroutines are asleep - deadlock!
And I don't understand what causes the error. My understanding is that in my function sum_up I am adding new values to my_channel. Why does the problem occur after I try to print out the values? Since I see 1,3,6 are printed it means that all goroutines have successfully finished.
Moreover, if the block that attempts to print the values of the channel
for ele := range my_channel {
fmt.Println(ele)
}
Is removed, then I don't get the error. So it is including the block that causes the error, but why?
A for-range on a empty channel will block until there are elements to read from the channel or until the channel is closed.
Here is a version that uses sync.WaitGroup to account for how many goroutines remain active. After all goroutines have finished, the channel is closed and the for-range loop exists.
https://play.golang.org/p/ZnLYxLMNdF
package main
import (
"fmt"
"sync"
)
func sum_up(my_int int, cs chan int, wg *sync.WaitGroup) {
my_sum := 0
for i := 0; i < my_int; i++ {
my_sum += i
}
cs <- my_sum
wg.Done()
}
func main() {
wg := &sync.WaitGroup{}
my_channel := make(chan int)
for i := 2; i < 5; i++ {
wg.Add(1)
go sum_up(i, my_channel, wg)
}
// Run a goroutine that will monitor how many sum_up are running.
go func(cs chan int, wg *sync.WaitGroup) {
wg.Wait()
close(cs)
}(my_channel, wg)
for ele := range my_channel {
fmt.Println(ele)
}
//fatal error: all goroutines are asleep - deadlock!
fmt.Println("Done")
}
When you use range on a channel, it will wait for values forever or until the channel is closed. It's deadlocking because when the last value is written to my_channel, it will wait forever for a value that will never come.
Here's a slightly modified variant that shows how to cleanly leave the range: https://play.golang.org/p/YDlM8EcRnx
for range chan exit when chan receive close signal. You must close(my_channel) somewhere, or loop will wait forever.