package main
import (
"fmt"
"time"
)
func main() {
c := make(chan int)
go func() {
fmt.Println("hello")
c <- 10
}()
time.Sleep(2 * time.Second)
}
In the above program, I have created a Go routine which is writing to channel c but there is no other go routine which is reading from the channel. Why isnt there a deadlock in this case?
A deadlock implies all goroutines being blocked, not just one arbitrary goroutine of your choosing.
The main goroutine is simply in a sleep, once that is over, it can continue to run.
If you switch the sleep with a select{} blocking forever operation, you'll get your deadlock:
c := make(chan int)
go func() {
fmt.Println("hello")
c <- 10
}()
select {}
Try it on the Go Playground.
See related: Why there is no error that receiver is blocked?
Related
golang: Why there's no deadlock in this code?
Please go through this following code:
package main
import (
"fmt"
"time"
)
func f1(done chan bool) {
done <- true
fmt.Printf("this's f1() goroutine\n")
}
func f2(done chan bool) {
val := <-done
fmt.Printf("this's f2() goroutine. val: %t\n", val)
}
func main() {
done1 := make(chan bool)
done2 := make(chan bool)
go f1(done1)
go f2(done2)
fmt.Printf("main() go-routine is waiting to see deadlock.\n")
time.Sleep(5 * time.Second)
}
As one can see, go-routine f1() is sending a value onto the channel. And go-routine f2() is receiving a value from a channel.
However, there's no go-routine receiving from the channel which is being sent onto by go-routine f1().
Similarly, there's no go-routine sending onto the channel which is being received from by go-routine f2().
As #icza's comment correctly states, a deadlock happens when all goroutines are stuck and can't make progress. In your case f1 and f2 are stuck, but the main goroutine is not - so this isn't a deadlock.
However, it is a goroutine leak! Goroutine leaks happen when some code finishes its logical existence but leaves goroutines running (unterminated). I've found tools like github.com/fortytw2/leaktest useful for detecting goroutine leaks, and it would detect the issue in your code - give it a try.
Here's a modified code sample:
import (
"testing"
"time"
"github.com/fortytw2/leaktest"
)
func f1(done chan bool) {
done <- true
fmt.Printf("this's f1() goroutine\n")
}
func f2(done chan bool) {
val := <-done
fmt.Printf("this's f2() goroutine. val: %t\n", val)
}
func TestTwoGoroutines(t *testing.T) {
defer leaktest.CheckTimeout(t, time.Second)()
done1 := make(chan bool)
done2 := make(chan bool)
go f1(done1)
go f2(done2)
}
When you run this test, you'll see something like:
--- FAIL: TestTwoGoroutines (1.03s)
leaktest.go:132: leaktest: timed out checking goroutines
leaktest.go:150: leaktest: leaked goroutine: goroutine 7 [chan send]:
leaktest-samples.f1(0xc000016420)
leaktest-samples/leaktest1_test.go:45 +0x37
created by leaktest-samples.TestTwoGoroutines
leaktest-samples/leaktest1_test.go:60 +0xd1
leaktest.go:150: leaktest: leaked goroutine: goroutine 8 [chan receive]:
leaktest-samples.f2(0xc000016480)
leaktest-samples/leaktest1_test.go:50 +0x3e
created by leaktest-samples.TestTwoGoroutines
leaktest-samples/leaktest1_test.go:61 +0xf3
As #icza said, the main goroutine can continue and ultimately terminate.
If you remove the go keyword from either of the two function calls (make them occur on the main goroutine) then the application has a deadlock. See this go playground (it highlights the deadlock for you) https://play.golang.org/p/r9qo2sc9LQA
There's one more thing to it.
func f1(done chan bool) {
fmt.Printf("f1()\n")
}
func main() {
done := make(chan bool)
go f1(done)
done <- true
}
Here, the caller go-routine clearly gets stuck since there's no go-routine in existence that's receiving from the channel.
Here, not that all the go-routines are blocked (f1() slips through), but there's still a dead-lock. The reason being, there has to be at least 1 go routine that should be receiving from the channel.
But this clearly contradicts the above comment and not all go-routines are blocked here.
Consider two examples:
First one:
package main
import "fmt"
func main() {
c := make(chan int)
go func(){
c <- 1
}()
go func(){
c <- 2
}()
select {
case <-c:
fmt.Println("<-c:", <-c)
fmt.Println("<-c:", <-c)
}
}
As far as I know, what should happen is: Two goroutines spawn and write to the same channel, one of them blocks and waits for the main goroutine to read.
But I don't understand what happens with the second goroutine. Does it block because the channel is unbuffered, and it can't hold two values by design.
With this piece I am getting a deadlock. I found a similar question here.
And if so, how this code is running then?
package main
import "fmt"
func main() {
c := make(chan int)
go func(){
c <- 1
}()
go func(){
c <- 2
}()
select {
case forget := <-c:
fmt.Println("forget:", forget)
fmt.Println("<-c:", <-c)
}
}
Ok, It looks like using case <-c actually reads from the channel and does not simply check it for having a value as I thought.
I'm having trouble figuring out how to correctly use sync.Cond. From what I can tell, a race condition exists between locking the Locker and invoking the condition's Wait method. This example adds an artificial delay between the two lines in the main goroutine to simulate the race condition:
package main
import (
"sync"
"time"
)
func main() {
m := sync.Mutex{}
c := sync.NewCond(&m)
go func() {
time.Sleep(1 * time.Second)
c.Broadcast()
}()
m.Lock()
time.Sleep(2 * time.Second)
c.Wait()
}
[Run on the Go Playground]
This causes an immediate panic:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [semacquire]:
sync.runtime_Syncsemacquire(0x10330208, 0x1)
/usr/local/go/src/runtime/sema.go:241 +0x2e0
sync.(*Cond).Wait(0x10330200, 0x0)
/usr/local/go/src/sync/cond.go:63 +0xe0
main.main()
/tmp/sandbox301865429/main.go:17 +0x1a0
What am I doing wrong? How do I avoid this apparent race condition? Is there a better synchronization construct I should be using?
Edit: I realize I should have better explained the problem I'm trying to solve here. I have a long-running goroutine that downloads a large file and a number of other goroutines that need access to the HTTP headers when they are available. This problem is harder than it sounds.
I can't use channels since only one goroutine would then receive the value. And some of the other goroutines would be trying to retrieve the headers long after they are already available.
The downloader goroutine could simply store the HTTP headers in a variable and use a mutex to safeguard access to them. However, this doesn't provide a way for the other goroutines to "wait" for them to become available.
I had thought that both a sync.Mutex and sync.Cond together could accomplish this goal but it appears that this is not possible.
OP answered his own, but did not directly answer the original question, I am going to post how to correctly use sync.Cond.
You do not really need sync.Cond if you have one goroutine for each write and read - a single sync.Mutex would suffice to communicate between them. sync.Cond could useful in situations where multiple readers wait for the shared resources to be available.
var sharedRsc = make(map[string]interface{})
func main() {
var wg sync.WaitGroup
wg.Add(2)
m := sync.Mutex{}
c := sync.NewCond(&m)
go func() {
// this go routine wait for changes to the sharedRsc
c.L.Lock()
for len(sharedRsc) == 0 {
c.Wait()
}
fmt.Println(sharedRsc["rsc1"])
c.L.Unlock()
wg.Done()
}()
go func() {
// this go routine wait for changes to the sharedRsc
c.L.Lock()
for len(sharedRsc) == 0 {
c.Wait()
}
fmt.Println(sharedRsc["rsc2"])
c.L.Unlock()
wg.Done()
}()
// this one writes changes to sharedRsc
c.L.Lock()
sharedRsc["rsc1"] = "foo"
sharedRsc["rsc2"] = "bar"
c.Broadcast()
c.L.Unlock()
wg.Wait()
}
Playground
Having said that, using channels is still the recommended way to pass data around if the situation permitting.
Note: sync.WaitGroup here is only used to wait for the goroutines to complete their executions.
You need to make sure that c.Broadcast is called after your call to c.Wait. The correct version of your program would be:
package main
import (
"fmt"
"sync"
)
func main() {
m := &sync.Mutex{}
c := sync.NewCond(m)
m.Lock()
go func() {
m.Lock() // Wait for c.Wait()
c.Broadcast()
m.Unlock()
}()
c.Wait() // Unlocks m, waits, then locks m again
m.Unlock()
}
https://play.golang.org/p/O1r8v8yW6h
package main
import (
"fmt"
"sync"
"time"
)
func main() {
m := sync.Mutex{}
m.Lock() // main gouroutine is owner of lock
c := sync.NewCond(&m)
go func() {
m.Lock() // obtain a lock
defer m.Unlock()
fmt.Println("3. goroutine is owner of lock")
time.Sleep(2 * time.Second) // long computing - because you are the owner, you can change state variable(s)
c.Broadcast() // State has been changed, publish it to waiting goroutines
fmt.Println("4. goroutine will release lock soon (deffered Unlock")
}()
fmt.Println("1. main goroutine is owner of lock")
time.Sleep(1 * time.Second) // initialization
fmt.Println("2. main goroutine is still lockek")
c.Wait() // Wait temporarily release a mutex during wating and give opportunity to other goroutines to change the state.
// Because you don't know, whether this is state, that you are waiting for, is usually called in loop.
m.Unlock()
fmt.Println("Done")
}
http://play.golang.org/p/fBBwoL7_pm
Looks like you c.Wait for Broadcast which would never happens with your time intervals.
With
time.Sleep(3 * time.Second) //Broadcast after any Wait for it
c.Broadcast()
your snippet seems to work http://play.golang.org/p/OE8aP4i6gY .Or am I missing something that you try to achive?
I finally discovered a way to do this and it doesn't involve sync.Cond at all - just the mutex.
type Task struct {
m sync.Mutex
headers http.Header
}
func NewTask() *Task {
t := &Task{}
t.m.Lock()
go func() {
defer t.m.Unlock()
// ...do stuff...
}()
return t
}
func (t *Task) WaitFor() http.Header {
t.m.Lock()
defer t.m.Unlock()
return t.headers
}
How does this work?
The mutex is locked at the beginning of the task, ensuring that anything calling WaitFor() will block. Once the headers are available and the mutex unlocked by the goroutine, each call to WaitFor() will execute one at a time. All future calls (even after the goroutine ends) will have no problem locking the mutex, since it will always be left unlocked.
Yes you can use one channel to pass Header to multiple Go routines.
headerChan := make(chan http.Header)
go func() { // This routine can be started many times
header := <-headerChan // Wait for header
// Do things with the header
}()
// Feed the header to all waiting go routines
for more := true; more; {
select {
case headerChan <- r.Header:
default: more = false
}
}
This can be done with channels pretty easily and the code will be clean. Below is the example. Hope this helps!
package main
import (
"fmt"
"net/http"
"sync"
)
func main() {
done := make(chan struct{})
var wg sync.WaitGroup
// fork required number of goroutines
for i := 0; i < 5; i++ {
wg.Add(1)
go func() {
defer wg.Done()
<-done
fmt.Println("read the http headers from here")
}()
}
time.Sleep(1) //download your large file here
fmt.Println("Unblocking goroutines...")
close(done) // this will unblock all the goroutines
wg.Wait()
}
In the excellent book "Concurrency in Go" they provide the following easy solution while leveraging the fact that a channel that is closed will release all waiting clients.
package main
import (
"fmt"
"time"
)
func main() {
httpHeaders := []string{}
headerChan := make(chan interface{})
var consumerFunc= func(id int, stream <-chan interface{}, funcHeaders *[]string)
{
<-stream
fmt.Println("Consumer ",id," got headers:", funcHeaders )
}
for i:=0;i<3;i++ {
go consumerFunc(i, headerChan, &httpHeaders)
}
fmt.Println("Getting headers...")
time.Sleep(2*time.Second)
httpHeaders=append(httpHeaders, "test1");
fmt.Println("Publishing headers...")
close(headerChan )
time.Sleep(5*time.Second)
}
https://play.golang.org/p/cE3SiKWNRIt
in the next example, I don't understand why end value not printed when received
package main
import "fmt"
func main() {
start := make(chan int)
end := make(chan int)
go func() {
fmt.Println("Start")
fmt.Println(<-start)
}()
go func() {
fmt.Println("End")
fmt.Println(<-end)
}()
start <- 1
end <- 2
}
I know sync.WaitGroup can solve this problem.
Because the program exits when it reaches the end of func main, regardless of whether any other goroutines are running. As soon as the second function receives from the end channel, main's send on that channel is unblocked and the program finishes, before the received value gets a chance to be passed to Println.
The end value is not printed because as soon as the main goroutine (the main function is actually a goroutine) is finished (in other terms get unblocked) the other non-main goroutines does not have the chance to get completed.
When the function main() returns, the program exits. Moreover goroutines are independent units of execution and when a number of them starts one after the other you cannot depend on when a goroutine will actually be started. The logic of your code must be independent of the order in which goroutines are invoked.
One way to solve your problem (and the easiest one in your case) is to put a time.Sleep at the end of your main() function.
time.Sleep(1e9)
This will guarantee that the main goroutine will not unblock and the other goroutines will have a change to get executed.
package main
import (
"fmt"
"time"
)
func main() {
start := make(chan int)
end := make(chan int)
go func() {
fmt.Println("Start")
fmt.Println(<-start)
}()
go func() {
fmt.Println("End")
fmt.Println(<-end)
}()
start <- 1
end <- 2
time.Sleep(1e9)
}
Another solution as you mentioned is to use waitgroup.
Apart from sleep where you have to specify the time, you can use waitgroup to make you program wait until the goroutine completes execution.
package main
import "fmt"
import "sync"
var wg sync.WaitGroup
func main() {
start := make(chan int)
end := make(chan int)
wg.Add(2)
go func() {
defer wg.Done()
fmt.Println("Start")
fmt.Println(<-start)
}()
go func() {
defer wg.Done()
fmt.Println("End")
fmt.Println(<-end)
}()
start <- 1
end <- 2
wg.Wait()
}
package main
import(
"fmt"
"time"
)
func main(){
fmt.Println("1")
defer fmt.Println("-1")
go func() {
fmt.Println("2")
defer fmt.Println("-2")
time.Sleep(9 * time.Second)
}()
time.Sleep(1 * time.Second)
fmt.Println("3")
}
Produces the output: 1 2 3 -1
But I would've thought the goroutine's defer would have been called to produce: 1 2 3 -2 -1
In my actual code my goroutine is blocked on a websocket... I suppose I could send a shutdown signal but I haven't yet figured out how to do a wait-on-multiple-objects kind of thing (if it can actually be done in go). I'm currently solving my problem by hoisting the deferred -2 into the main scope.
Is there some trick to defer placement I'm not doing right?
http://play.golang.org/p/qv8UEuF2Rb
(I don't have enough reps to comment?)
Yes, the issue is with goroutine scheduling and is platform dependent. Use runtime.Gosched at the end of the function with the goroutine to ensure it gets some cputime. Presently on x86_64 linux, the code below will produce "3" but without Gosched it produces "1":
No. That's not the issue, and that only solves it by co-incidence(via an implementation detail). As has already been hinted at in goroutine's defer not called when main scope ends . Your code is racy and you must therefore explicitly synchronize the goroutines. The simplest way is to to use a channel or a waitgroup http://golang.org/pkg/sync/#WaitGroup . Which option is more appropriate depends on what you're doing. If all you want to do is wait for some result, then have the gorutine send its result on the channel and you simply wait for it. If you just want to wait for a number of goroutines, then use a waitgroup.
The process is ending, so the internal function is terminated. Try sleeping at the end of main:
func main(){
fmt.Println("1")
defer fmt.Println("-1")
go func() {
fmt.Println("2")
defer fmt.Println("-2")
time.Sleep(9 * time.Second)
}()
time.Sleep(1 * time.Second)
fmt.Println("3")
time.Sleep(9 * time.Second)
}
http://play.golang.org/p/Qrtz_0xiRs
EDIT: as mentioned in the comments, this is not the right approach in general. You should explicitly wait for the goroutine.
Yes, the issue is with goroutine scheduling and is platform dependent. Use runtime.Gosched at the end of the function with the goroutine to ensure it gets some cputime. Presently on x86_64 linux, the code below will produce "3" but without Gosched it produces "1":
q = 1
go func() {
q = 2
defer func() { q = 3 }()
}()
runtime.Gosched()
fmt.Println(q)
http://play.golang.org/p/dNFQv9IxXv
WaitGroup can help you resolving the issue:
package main
import(
"fmt"
"sync"
"time"
)
func main(){
fmt.Println("-start main-")
defer fmt.Println("-defer main-")
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
fmt.Println("-start goroutine-")
defer fmt.Println("-defer goroutine-")
time.Sleep(2 * time.Second)
wg.Done()
}()
fmt.Println("-end main-")
wg.Wait()
}
Prints:
-start main-
-end main-
-start goroutine-
-defer goroutine-
-defer main-
https://play.golang.org/p/ycmPJJ6pvC