goroutine's defer not called when main scope ends - go

package main
import(
"fmt"
"time"
)
func main(){
fmt.Println("1")
defer fmt.Println("-1")
go func() {
fmt.Println("2")
defer fmt.Println("-2")
time.Sleep(9 * time.Second)
}()
time.Sleep(1 * time.Second)
fmt.Println("3")
}
Produces the output: 1 2 3 -1
But I would've thought the goroutine's defer would have been called to produce: 1 2 3 -2 -1
In my actual code my goroutine is blocked on a websocket... I suppose I could send a shutdown signal but I haven't yet figured out how to do a wait-on-multiple-objects kind of thing (if it can actually be done in go). I'm currently solving my problem by hoisting the deferred -2 into the main scope.
Is there some trick to defer placement I'm not doing right?
http://play.golang.org/p/qv8UEuF2Rb

(I don't have enough reps to comment?)
Yes, the issue is with goroutine scheduling and is platform dependent. Use runtime.Gosched at the end of the function with the goroutine to ensure it gets some cputime. Presently on x86_64 linux, the code below will produce "3" but without Gosched it produces "1":
No. That's not the issue, and that only solves it by co-incidence(via an implementation detail). As has already been hinted at in goroutine's defer not called when main scope ends . Your code is racy and you must therefore explicitly synchronize the goroutines. The simplest way is to to use a channel or a waitgroup http://golang.org/pkg/sync/#WaitGroup . Which option is more appropriate depends on what you're doing. If all you want to do is wait for some result, then have the gorutine send its result on the channel and you simply wait for it. If you just want to wait for a number of goroutines, then use a waitgroup.

The process is ending, so the internal function is terminated. Try sleeping at the end of main:
func main(){
fmt.Println("1")
defer fmt.Println("-1")
go func() {
fmt.Println("2")
defer fmt.Println("-2")
time.Sleep(9 * time.Second)
}()
time.Sleep(1 * time.Second)
fmt.Println("3")
time.Sleep(9 * time.Second)
}
http://play.golang.org/p/Qrtz_0xiRs
EDIT: as mentioned in the comments, this is not the right approach in general. You should explicitly wait for the goroutine.

Yes, the issue is with goroutine scheduling and is platform dependent. Use runtime.Gosched at the end of the function with the goroutine to ensure it gets some cputime. Presently on x86_64 linux, the code below will produce "3" but without Gosched it produces "1":
q = 1
go func() {
q = 2
defer func() { q = 3 }()
}()
runtime.Gosched()
fmt.Println(q)
http://play.golang.org/p/dNFQv9IxXv

WaitGroup can help you resolving the issue:
package main
import(
"fmt"
"sync"
"time"
)
func main(){
fmt.Println("-start main-")
defer fmt.Println("-defer main-")
wg := sync.WaitGroup{}
wg.Add(1)
go func() {
fmt.Println("-start goroutine-")
defer fmt.Println("-defer goroutine-")
time.Sleep(2 * time.Second)
wg.Done()
}()
fmt.Println("-end main-")
wg.Wait()
}
Prints:
-start main-
-end main-
-start goroutine-
-defer goroutine-
-defer main-
https://play.golang.org/p/ycmPJJ6pvC

Related

How can one close a channel in a defer block safely?

Consider the following example:
package main
import (
"fmt"
"time"
)
func main() {
ticker := time.NewTicker(2 * time.Second)
done := make(chan bool)
defer func() {
fmt.Println("exiting..")
done <- true
close(done)
}()
go func(ticker *time.Ticker, done chan bool) {
for {
select {
case <-done:
fmt.Println("DONE!")
break
case <-ticker.C:
fmt.Println("TICK!...")
}
}
}(ticker, done)
time.Sleep(7 * time.Second)
}
The goroutine waiting to receive from done never receives as (I am guessing) the main goroutine finished beforehand. However if I change the sleep time of the main goroutine to 8 seconds it receives a message; Why is there this dependency on the sleep time?
Is it because there is that second difference that keeps the goroutine alive and the there isn't enough time to kill it?
How would I than kill the goroutine gracefully?
You need to ensure that main does not return before the goroutine finishes.
The simplest way to do this is using a WaitGroup:
var wg sync.WaitGroup
defer wg.Wait()
wg.Add(1)
go func() {
defer wg.Done()
// …
Note that defers run in reverse order, so you must put defer wg.Wait() before defer close(done), otherwise it will deadlock.

Go channel deadlock is not happening

package main
import (
"fmt"
"time"
)
func main() {
c := make(chan int)
go func() {
fmt.Println("hello")
c <- 10
}()
time.Sleep(2 * time.Second)
}
In the above program, I have created a Go routine which is writing to channel c but there is no other go routine which is reading from the channel. Why isnt there a deadlock in this case?
A deadlock implies all goroutines being blocked, not just one arbitrary goroutine of your choosing.
The main goroutine is simply in a sleep, once that is over, it can continue to run.
If you switch the sleep with a select{} blocking forever operation, you'll get your deadlock:
c := make(chan int)
go func() {
fmt.Println("hello")
c <- 10
}()
select {}
Try it on the Go Playground.
See related: Why there is no error that receiver is blocked?

Gorutine doesn't finish executing after returning from main. Why?

I am trying to understand context in golang. I copied an example from https://golang.org/pkg/context/#example_WithCancel and changed it a bit:
Playgroud: https://play.golang.org/p/Aczc2CqcVZR
package main
import (
"context"
"fmt"
"time"
)
func main() {
// gen generates integers in a separate goroutine and
// sends them to the returned channel.
// The callers of gen need to cancel the context once
// they are done consuming generated integers not to leak
// the internal goroutine started by gen.
gen := func(ctx context.Context) <-chan int {
dst := make(chan int)
n := 1
go func() {
for {
select {
case <-ctx.Done():
fmt.Println("DONE")
return // returning not to leak the goroutine
case dst <- n:
n++
}
}
fmt.Println("END")
}()
return dst
}
ctx, cancel := context.WithCancel(context.Background())
defer time.Sleep(1 * time.Second)
defer fmt.Println("Before cancel")
defer cancel() // cancel when we are finished consuming integers
defer fmt.Println("After cancel")
channel := gen(ctx)
for n := range channel {
fmt.Println(n)
if n == 5 {
break
}
}
fmt.Println( <-channel)
}
When commenting out the
defer time.Sleep(1 * time.Second)
the "DONE" never gets printed. Playgroud: (https://play.golang.org/p/K0OcyZaj_xK)
I would expect the go routine which was started in the anonymous function still to be active. Once cancel() is called due to being deferred, the select should no longer block as
case <-ctx.Done():
should be available. However it seems to just end, unless I wait for 1 second and give it time. This behavior seems very wrong.
This behavior seems very wrong.
It's not. That's how program execution is specified. After main and its deferred functions return, the program exits.
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
https://golang.org/ref/spec#Program_execution

How to correctly use sync.Cond?

I'm having trouble figuring out how to correctly use sync.Cond. From what I can tell, a race condition exists between locking the Locker and invoking the condition's Wait method. This example adds an artificial delay between the two lines in the main goroutine to simulate the race condition:
package main
import (
"sync"
"time"
)
func main() {
m := sync.Mutex{}
c := sync.NewCond(&m)
go func() {
time.Sleep(1 * time.Second)
c.Broadcast()
}()
m.Lock()
time.Sleep(2 * time.Second)
c.Wait()
}
[Run on the Go Playground]
This causes an immediate panic:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [semacquire]:
sync.runtime_Syncsemacquire(0x10330208, 0x1)
/usr/local/go/src/runtime/sema.go:241 +0x2e0
sync.(*Cond).Wait(0x10330200, 0x0)
/usr/local/go/src/sync/cond.go:63 +0xe0
main.main()
/tmp/sandbox301865429/main.go:17 +0x1a0
What am I doing wrong? How do I avoid this apparent race condition? Is there a better synchronization construct I should be using?
Edit: I realize I should have better explained the problem I'm trying to solve here. I have a long-running goroutine that downloads a large file and a number of other goroutines that need access to the HTTP headers when they are available. This problem is harder than it sounds.
I can't use channels since only one goroutine would then receive the value. And some of the other goroutines would be trying to retrieve the headers long after they are already available.
The downloader goroutine could simply store the HTTP headers in a variable and use a mutex to safeguard access to them. However, this doesn't provide a way for the other goroutines to "wait" for them to become available.
I had thought that both a sync.Mutex and sync.Cond together could accomplish this goal but it appears that this is not possible.
OP answered his own, but did not directly answer the original question, I am going to post how to correctly use sync.Cond.
You do not really need sync.Cond if you have one goroutine for each write and read - a single sync.Mutex would suffice to communicate between them. sync.Cond could useful in situations where multiple readers wait for the shared resources to be available.
var sharedRsc = make(map[string]interface{})
func main() {
var wg sync.WaitGroup
wg.Add(2)
m := sync.Mutex{}
c := sync.NewCond(&m)
go func() {
// this go routine wait for changes to the sharedRsc
c.L.Lock()
for len(sharedRsc) == 0 {
c.Wait()
}
fmt.Println(sharedRsc["rsc1"])
c.L.Unlock()
wg.Done()
}()
go func() {
// this go routine wait for changes to the sharedRsc
c.L.Lock()
for len(sharedRsc) == 0 {
c.Wait()
}
fmt.Println(sharedRsc["rsc2"])
c.L.Unlock()
wg.Done()
}()
// this one writes changes to sharedRsc
c.L.Lock()
sharedRsc["rsc1"] = "foo"
sharedRsc["rsc2"] = "bar"
c.Broadcast()
c.L.Unlock()
wg.Wait()
}
Playground
Having said that, using channels is still the recommended way to pass data around if the situation permitting.
Note: sync.WaitGroup here is only used to wait for the goroutines to complete their executions.
You need to make sure that c.Broadcast is called after your call to c.Wait. The correct version of your program would be:
package main
import (
"fmt"
"sync"
)
func main() {
m := &sync.Mutex{}
c := sync.NewCond(m)
m.Lock()
go func() {
m.Lock() // Wait for c.Wait()
c.Broadcast()
m.Unlock()
}()
c.Wait() // Unlocks m, waits, then locks m again
m.Unlock()
}
https://play.golang.org/p/O1r8v8yW6h
package main
import (
"fmt"
"sync"
"time"
)
func main() {
m := sync.Mutex{}
m.Lock() // main gouroutine is owner of lock
c := sync.NewCond(&m)
go func() {
m.Lock() // obtain a lock
defer m.Unlock()
fmt.Println("3. goroutine is owner of lock")
time.Sleep(2 * time.Second) // long computing - because you are the owner, you can change state variable(s)
c.Broadcast() // State has been changed, publish it to waiting goroutines
fmt.Println("4. goroutine will release lock soon (deffered Unlock")
}()
fmt.Println("1. main goroutine is owner of lock")
time.Sleep(1 * time.Second) // initialization
fmt.Println("2. main goroutine is still lockek")
c.Wait() // Wait temporarily release a mutex during wating and give opportunity to other goroutines to change the state.
// Because you don't know, whether this is state, that you are waiting for, is usually called in loop.
m.Unlock()
fmt.Println("Done")
}
http://play.golang.org/p/fBBwoL7_pm
Looks like you c.Wait for Broadcast which would never happens with your time intervals.
With
time.Sleep(3 * time.Second) //Broadcast after any Wait for it
c.Broadcast()
your snippet seems to work http://play.golang.org/p/OE8aP4i6gY .Or am I missing something that you try to achive?
I finally discovered a way to do this and it doesn't involve sync.Cond at all - just the mutex.
type Task struct {
m sync.Mutex
headers http.Header
}
func NewTask() *Task {
t := &Task{}
t.m.Lock()
go func() {
defer t.m.Unlock()
// ...do stuff...
}()
return t
}
func (t *Task) WaitFor() http.Header {
t.m.Lock()
defer t.m.Unlock()
return t.headers
}
How does this work?
The mutex is locked at the beginning of the task, ensuring that anything calling WaitFor() will block. Once the headers are available and the mutex unlocked by the goroutine, each call to WaitFor() will execute one at a time. All future calls (even after the goroutine ends) will have no problem locking the mutex, since it will always be left unlocked.
Yes you can use one channel to pass Header to multiple Go routines.
headerChan := make(chan http.Header)
go func() { // This routine can be started many times
header := <-headerChan // Wait for header
// Do things with the header
}()
// Feed the header to all waiting go routines
for more := true; more; {
select {
case headerChan <- r.Header:
default: more = false
}
}
This can be done with channels pretty easily and the code will be clean. Below is the example. Hope this helps!
package main
import (
"fmt"
"net/http"
"sync"
)
func main() {
done := make(chan struct{})
var wg sync.WaitGroup
// fork required number of goroutines
for i := 0; i < 5; i++ {
wg.Add(1)
go func() {
defer wg.Done()
<-done
fmt.Println("read the http headers from here")
}()
}
time.Sleep(1) //download your large file here
fmt.Println("Unblocking goroutines...")
close(done) // this will unblock all the goroutines
wg.Wait()
}
In the excellent book "Concurrency in Go" they provide the following easy solution while leveraging the fact that a channel that is closed will release all waiting clients.
package main
import (
"fmt"
"time"
)
func main() {
httpHeaders := []string{}
headerChan := make(chan interface{})
var consumerFunc= func(id int, stream <-chan interface{}, funcHeaders *[]string)
{
<-stream
fmt.Println("Consumer ",id," got headers:", funcHeaders )
}
for i:=0;i<3;i++ {
go consumerFunc(i, headerChan, &httpHeaders)
}
fmt.Println("Getting headers...")
time.Sleep(2*time.Second)
httpHeaders=append(httpHeaders, "test1");
fmt.Println("Publishing headers...")
close(headerChan )
time.Sleep(5*time.Second)
}
https://play.golang.org/p/cE3SiKWNRIt

why I can not see output when received value by channel

in the next example, I don't understand why end value not printed when received
package main
import "fmt"
func main() {
start := make(chan int)
end := make(chan int)
go func() {
fmt.Println("Start")
fmt.Println(<-start)
}()
go func() {
fmt.Println("End")
fmt.Println(<-end)
}()
start <- 1
end <- 2
}
I know sync.WaitGroup can solve this problem.
Because the program exits when it reaches the end of func main, regardless of whether any other goroutines are running. As soon as the second function receives from the end channel, main's send on that channel is unblocked and the program finishes, before the received value gets a chance to be passed to Println.
The end value is not printed because as soon as the main goroutine (the main function is actually a goroutine) is finished (in other terms get unblocked) the other non-main goroutines does not have the chance to get completed.
When the function main() returns, the program exits. Moreover goroutines are independent units of execution and when a number of them starts one after the other you cannot depend on when a goroutine will actually be started. The logic of your code must be independent of the order in which goroutines are invoked.
One way to solve your problem (and the easiest one in your case) is to put a time.Sleep at the end of your main() function.
time.Sleep(1e9)
This will guarantee that the main goroutine will not unblock and the other goroutines will have a change to get executed.
package main
import (
"fmt"
"time"
)
func main() {
start := make(chan int)
end := make(chan int)
go func() {
fmt.Println("Start")
fmt.Println(<-start)
}()
go func() {
fmt.Println("End")
fmt.Println(<-end)
}()
start <- 1
end <- 2
time.Sleep(1e9)
}
Another solution as you mentioned is to use waitgroup.
Apart from sleep where you have to specify the time, you can use waitgroup to make you program wait until the goroutine completes execution.
package main
import "fmt"
import "sync"
var wg sync.WaitGroup
func main() {
start := make(chan int)
end := make(chan int)
wg.Add(2)
go func() {
defer wg.Done()
fmt.Println("Start")
fmt.Println(<-start)
}()
go func() {
defer wg.Done()
fmt.Println("End")
fmt.Println(<-end)
}()
start <- 1
end <- 2
wg.Wait()
}

Resources