Here is my program which is producing deadlock, how do I avoid it and what is the recommended pattern to handle this kind of situation.
The problem is after timeout how do I detect that there is no reader on my channel ?
var wg sync.WaitGroup
func main() {
wg.Add(1)
c := make(chan int)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
c <- 10
wg.Wait()
}
func readFromChannel(c chan int, ti <-chan time.Time) {
select {
case x := <-c:
fmt.Println("Read", x)
case <-ti:
fmt.Println("TIMED OUT")
}
wg.Done()
}
So, lets look at what's really going on in your source. You have two goroutines (there's more than two, but we're going to focus on the explicit ones), main and readFromChannel.
Lets look at what readFromChannel does:
if channel `c` is not empty before `ti` has expired, print its contents and return, after signalling its completion to wait group.
if `ti` has expired before `c` is not empty, print "TIMED OUT" and return, after signalling its completion to wait group.
now Main:
adds to waitgroup
make a channel `c`
start a goroutine `readFromChannel`
sleep for 5 seconds
send 10 to channel `c`
call wait for waitgroup
Now, lets go through the flow of execution for your code, concurrently (your code may/ may not execute in this order every time, keep that in mind)
1) wg.Add(1)
2) c := make(chan int)
3) go readFromChannel(c, time.After(time.Duration(2)*time.Second))
#timer ti starts#
4) time.Sleep(time.Duration(5) * time.Second)
#MAIN Goroutine begins sleep
#timer ti expires#
5) case <-ti:
6) fmt.Println("TIMED OUT")
7) wg.Done()
# readFromChannel Goroutine returns #
#MAIN Goroutine exits sleep#
8) c<-10
9) ......#DEADLOCK#
Now you can guess why you got a deadlock. In go, non buffered channels will block until something happens on the other end of the channel, regardless of whether you're sending or receiving. So c <- 10 will block until something reads from the other end of c, but the goroutine you had for that has dropped out of the picture 2 seconds ago. Therefore, c blocks forever, and since main is the last goroutine left, you get a Deadlock.
How to prevent it? When using channels, ensure that there's always a receive at the other end of the channel for every send.
Using a buffered channel in this scenario can serve as a quick fix, but may fuel potential gotchas in larger repositories. For example, assuming you wrote more data to c afterward and ran go readFromChannel(c, time.After(time.Duration(2)*time.Second)) a second time. You might see:
Read D1
Read D2
or
TIMED OUT
Read D1
solely based on chance. That's probably not the behavior you'd want.
Here's how I'd resolve the deadlock:
func main() {
wg.Add(1)
c := make(chan int)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
c <- 10
wg.Wait()
}
func readFromChannel(c chan int, ti <-chan time.Time) {
// the forloop will run forever
loop: // **
for {
select {
case x := <-c:
fmt.Println("Read", x)
break loop // breaks out of the for loop and the select **
case <-ti:
fmt.Println("TIMED OUT")
}
}
wg.Done()
}
** see this answer for details
You have an unbuffered channel. According to the docs:
If the channel is unbuffered, the sender blocks until the receiver has
received the value. If the channel has a buffer, the sender blocks
only until the value has been copied to the buffer
By changing the channel to being buffered, we can avoid deadlock.
c := make(chan int, 10) // holds 10 ints
I also suggest reading https://golang.org/doc/effective_go.html#channels, it's got some good stuff in there related to channels.
Your problem is that you are using select statement but you are not using inside a goroutine.
go func() {
for {
select {
case x := <-c:
fmt.Println("Read", x)
case <-ti:
fmt.Println("TIMED OUT")
}
}
}()
Getting the values out of different concurrently executing goroutines can be accomplished with the select keyword, which closely resembles the switch control statement and is sometimes called the communications switch.
Using a send operation in a select statement with a default case guarantees that the send will be non-blocking! If there are no cases, the select blocks execution forever.
https://play.golang.org/p/Ai1ggveb4s
This is an older question, but i'm diving deep into learning channels myself and found this here.
I think you just need to close the channel after your done sending on it?
Code:
func main() {
wg.Add(1)
c := make(chan int)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
c <- 10
close(c) // <- CLOSE IT HERE
wg.Wait()
}
To avoid deadlock here is some general advice from Deadlocks: the dark side of concurrency
Channel deadlock, Don’t send and receive to the same channel in the same goroutine
Read lock, Don’t take a read lock twice in the same goroutine
Release the lock as soon as possible
Objects which contain locks that call each other methods are a deadlock waiting to happen
Care total lock ordering
Access only read variables outside the lock
You can add a buffer to the c chan, like this:
var wg sync.WaitGroup
func main() {
wg.Add(1)
c := make(chan int, 1)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
c <- 10 // never blocks, because of the buffer
wg.Wait()
}
func readFromChannel(c chan int, ti <-chan time.Time) {
select {
case x := <-c:
fmt.Println("Read", x)
case <-ti:
fmt.Println("TIMED OUT")
}
wg.Done()
}
a write operation(c <- 10) to an unbuffered channel will be blocked until it there is a read operation(case x := <-c:).
Your timeout(2s) is smaller than sleep duration(5s) and because of this channel c will never be read in the select block.
I would recommend you to read chapter 8 & 9 of this book
func main() {
wg.Add(1)
c := make(chan int)
go func() {
c <- 10
}()
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
wg.Wait()
}//This will read without deadlock
func main() {
wg.Add(1)
c := make(chan int)
go readFromChannel(c, time.After(time.Duration(2)*time.Second))
time.Sleep(time.Duration(5) * time.Second)
go func() {
c <- 10
}()
wg.Wait()
}//Time out without deadlock
Related
i am new to the Go language, and looking for some help for the concurrency model.
say I want to make 2 http calls concurrently, and wait for both of them to finish then process/merge the response data.
here is the code i have
func main() {
var wg sync.WaitGroup
wg.Add(2)
c1 := make(chan string)
c2 := make(chan string)
go foo(c1, &wg)
go bar(c2, &wg)
wg.Wait()
foo := <-c1
bar := <-c2
fmt.Println("foo: ", foo)
fmt.Println("bar: ", bar)
}
func foo(c chan string, wg *sync.WaitGroup) {
defer wg.Done()
c <- "foo"
}
func bar(c chan string, wg *sync.WaitGroup) {
defer wg.Done()
c <- "bar"
}
however when i run it, it gives the error fatal error: all goroutines are asleep - deadlock!
I can get it working without the WaitGroup, but just curious why this gets to the deadlock, and whats the best way of doing it
The goroutines will block waiting to write to the channel, because the channel read happens after goroutines end (after wg.Done), thus the deadlock.
Simples solution here is to get rid of the waitgroup. The channel read operations will block until the goroutines write to the channel, so after both channels are read, there is no need to wait.
Indeed the waitgroup is not really needed here. But if you want to continue experimenting with goroutines and channels, then you can also try and make the channels buffered, like:
c1 := make(chan string, 1)
c2 := make(chan string, 1)
What happens then is that you can write a single entry to each channel without blocking.
Reading/writing to an unbuffered channel is a blocking call, meaning that these lines:
c <- "foo"
c <- "bar"
will hang until you reach calls that pull the values from the channels, i.e., these lines:
foo := <-c1
bar := <-c2
The cause of your deadlock is calling wg.Wait() before these two lines. In order to move past wg.Wait(), all waitgroups must complete, but your waitgroups cannot because wg.Done() is deferred until c <- "foo"/c <- "bar" unblock. As mentioned before, these cannot unblock until you reach foo := <-c1/bar := <-c2, and these cannot be reached because of wg.Wait(). You see the how no progress is possible, hence the deadlock.
The best practice is to avoid using waitgroups and mutex until absolutely necessary. Often, (like in this case) a solution is possible in pure Go and these packages only complicate the code.
I am trying to implement a simple logic where a Producer sends data to a channel ch with an forever for loop and a Consumer reads from the channel ch.
The Producer stops producing and exit the forever loop when it receives a signal on the channel quit.
The code is this (see also this playground)
func main() {
ch := make(chan int)
quit := make(chan bool)
var wg sync.WaitGroup
wg.Add(1)
go produce(ch, quit, &wg)
go consume(ch)
time.Sleep(1 * time.Millisecond)
fmt.Println("CLOSE")
close(quit)
wg.Wait()
}
func produce(ch chan int, quit chan bool, wg *sync.WaitGroup) {
for i := 0; ; i++ {
select {
case <-quit:
close(ch)
fmt.Println("exit")
wg.Done()
return //we exit
default:
ch <- i
fmt.Println("Producer sends", i)
}
}
}
func consume(ch chan int) {
for {
runtime.Gosched() // give the opportunity to the main goroutine to close the "quit" channel
select {
case i, more := <-ch:
if !more {
fmt.Println("exit consumer")
return
}
fmt.Println("Consumer receives", i)
}
}
}
If I run this piece of code on my machine (a Mac with 4 cores) everything works fine. If I try the same code on the Go Playgroud it always times out. I guess that this because the Go Playground is a single core and so the infinite loop does not give the chance to other goroutines to run, but then I do not understand why the instruction runtime.Gosched() does not have any effect.
Just to complete the picture I have seen that, if I set GOMAXPROCS=1 on my Mac, the program still works fine and exits as expected. If I set GOMAXPROCS=1 on my Mac and remove the runtime.Gosched() instruction, the behavior gets brittle: sometimes the program terminates as expected, some other times it seems to never exit the infinite loop.
You created a pathological situation that shouldn't happen in a real program, so the scheduler is not optimized to handle this. Combined with the fake time implementation in the playground, and you get far too many cycles of the producer and consumer before hitting a timeout.
The producer goroutine is creating values as fast as possible, while the consumer is always ready to receive them. With GOMAPXPROCS=1, the scheduler spends all its time bouncing between the two before it is forced to preempt the available work to check on the main goroutine, which takes longer than the playground will allow.
If we add something for the producer-consumer pair to do, we can limit the amount of time they have to monopolize the scheduler. For example, adding a time.Sleep(time.Microsecond) to the consumer will cause the playground to print 1000 values. This also goes to show how "accurate" the simulated time is in the playground, since that would not be possible with normal hardware which takes a non-zero amount time to process each message.
While an interesting case, this has little bearing on real programs.
A few notes, you can range over a channel to receive all values, you should always defer wg.Done at the start of the goroutine when possible, you can send values in the select case which allows you to actually cancel the for-select loop when the send isn't ready, and if you want the "exit consumer" message you need to send the WaitGroup to the consumer as well.
https://play.golang.org/p/WyPmpY9pFl7
func main() {
ch := make(chan int)
quit := make(chan bool)
var wg sync.WaitGroup
wg.Add(2)
go produce(ch, quit, &wg)
go consume(ch, &wg)
time.Sleep(50 * time.Microsecond)
fmt.Println("CLOSE")
close(quit)
wg.Wait()
}
func produce(ch chan int, quit chan bool, wg *sync.WaitGroup) {
defer wg.Done()
for i := 0; ; i++ {
select {
case <-quit:
close(ch)
fmt.Println("exit")
return
case ch <- i:
fmt.Println("Producer sends", i)
}
}
}
func consume(ch chan int, wg *sync.WaitGroup) {
defer wg.Done()
for i := range ch {
fmt.Println("Consumer receives", i)
time.Sleep(time.Microsecond)
}
fmt.Println("exit consumer")
return
}
In the following code
What happens to goroutine1?(at the end of program we have three goroutine goroutine1 without any functionality)
What happens to channel?(when we make a channel in loop it release the previous channel memory? close it? or something else?)
func main() {
for i := 1; i <= 3; i += 1 {
ch := make(chan int)
// gorutine1
go func() {
time.Sleep(3 * time.Second)
ch <- i
fmt.Println("gorutine1 end")
}()
// gorutine2
go func() {
time.Sleep(1 * time.Second)
ch <- i+1000
fmt.Println("gorutine2 end")
}()
fmt.Println("loop", <-ch)
}
time.Sleep(10 * time.Second)
fmt.Println("main end")
}
Run above code here
For i=1, the loop creates two goroutines, and starts waiting to read from the channel. goroutine2 writes first and terminates. The channel is read, and then i becomes 2. goroutine1 will wait forever, because nobody will read from the channel again. You create a new channel, and do the same thing. When everything is said and done, you have three instances of goroutine1 waiting to write to three different channels.
I was just experimenting with Go channels on my Ubuntu 64 bit environment and got confused with the output the following program produced.
I got the output:
0
1
2
3
Exit
The output when I uncommented the two commented lines:
0
1
2
3
4
Exit
Please explain the behavior.
TIA.
package main
import (
"fmt"
//"time"
)
func main() {
ch := make(chan int)
done := make(chan bool)
go func() {
for i := 0; i < 5; i++ {
ch <- i
}
//time.Sleep(1 * time.Second)
done <- false
}()
go func() {
for {
select {
case message := <-ch:
fmt.Println(message)
case <-done:
return
}
}
}()
<-done
fmt.Println("Exit")
}
Your main thread is waiting on done, then exiting. Meanwhile your first go function pipes 5 values into ch, then sends to done.
The value in done then gets read from the main thread and happens to occur before the second go function reads the last value from ch. When it does so, it exits the program.
Note that if your second thread did happen to both read from ch and done, then your program would deadlock since the main thread would never receive on done and all running go threads would be blocked waiting to receive on channels.
You're not waiting for both goroutines, and only sending a single value over done to 2 receivers, which will deadlock if the second receiver happens to be main.
Using a WaitGroup simplifies the code, and allows you to easily wait for as many goroutines as needed. https://play.golang.org/p/MWknv_9AFKp
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
defer close(ch)
for i := 0; i < 5; i++ {
ch <- i
}
}()
wg.Add(1)
go func() {
defer wg.Done()
for message := range ch {
fmt.Println(message)
}
}()
wg.Wait()
fmt.Println("Exit")
You have two go routines running in parallel. One inserts 5 numbers into the channel and then signals the main thread to exit, and another one reads the numbers from the channel.
Note that once the go routine that is responsible for enqueuing the numbers into the channel finishes, it signals the main thread to exit, regardless of whether the go routine that reads the numbers finished or not. So you could end up with a case where the enqueueing go routine finished before the dequeuing finished, and the main thread exited.
By adding the sleep, you make the enqueueing go routine live a bit longer and give a chance to the dequeueing go routine to read and print all the numbers before the enqueueing go routine signals the main thread to exit.
To solve that, you could just run the dequeuing code in the main thread. No need to run it in a go routine in this case.
I have issue when using Go routine with channel. The code looks like this:
func main() {
c := make(chan int)
var wg sync.WaitGroup
wg.Add(1)
go func (c chan int, x int) {
c <- x
fmt.Println(x)
close(c)
defer wg.Done()
}(c,10)
wg.Wait()
}
When run the code I got this error:
fatal error: all goroutines are asleep - deadlock!
I cannot understand why this issue happens. Please help me to understand
You have 2 goroutines in your example: the main goroutine running the main() function, and the other you launch inside it. The main goroutine waits for the other to complete (to call wg.Done()), and the other goroutine blocks in the line where it attempts to send a value on channel c. Since nobody is receiving from that channel, and because that channel is unbuffered, this goroutine will never advance, so all your 2 goroutines will block forever.
Note that defer wg.Done() should be the first statement in the goroutine. If it's the last, defer won't make any difference.
If the channel would have a buffer of at least one, the send operation could proceed:
c := make(chan int, 1)
And output will be (try it on the Go Playground):
10
If we leave the channel unbuffered, there must be another goroutine that receives from the channel, e.g.:
wg.Add(1)
go func() {
defer wg.Done()
x := <-c
fmt.Println("Received:", x)
}()
Then output will be (try it on the Go Playground):
10
Received: 10