in the next example, I don't understand why end value not printed when received
package main
import "fmt"
func main() {
start := make(chan int)
end := make(chan int)
go func() {
fmt.Println("Start")
fmt.Println(<-start)
}()
go func() {
fmt.Println("End")
fmt.Println(<-end)
}()
start <- 1
end <- 2
}
I know sync.WaitGroup can solve this problem.
Because the program exits when it reaches the end of func main, regardless of whether any other goroutines are running. As soon as the second function receives from the end channel, main's send on that channel is unblocked and the program finishes, before the received value gets a chance to be passed to Println.
The end value is not printed because as soon as the main goroutine (the main function is actually a goroutine) is finished (in other terms get unblocked) the other non-main goroutines does not have the chance to get completed.
When the function main() returns, the program exits. Moreover goroutines are independent units of execution and when a number of them starts one after the other you cannot depend on when a goroutine will actually be started. The logic of your code must be independent of the order in which goroutines are invoked.
One way to solve your problem (and the easiest one in your case) is to put a time.Sleep at the end of your main() function.
time.Sleep(1e9)
This will guarantee that the main goroutine will not unblock and the other goroutines will have a change to get executed.
package main
import (
"fmt"
"time"
)
func main() {
start := make(chan int)
end := make(chan int)
go func() {
fmt.Println("Start")
fmt.Println(<-start)
}()
go func() {
fmt.Println("End")
fmt.Println(<-end)
}()
start <- 1
end <- 2
time.Sleep(1e9)
}
Another solution as you mentioned is to use waitgroup.
Apart from sleep where you have to specify the time, you can use waitgroup to make you program wait until the goroutine completes execution.
package main
import "fmt"
import "sync"
var wg sync.WaitGroup
func main() {
start := make(chan int)
end := make(chan int)
wg.Add(2)
go func() {
defer wg.Done()
fmt.Println("Start")
fmt.Println(<-start)
}()
go func() {
defer wg.Done()
fmt.Println("End")
fmt.Println(<-end)
}()
start <- 1
end <- 2
wg.Wait()
}
Related
The following piece of code try to send to the channel on the main goroutine and receive from another goroutine but a few times it returns as expected but a few times it exits without printing any on the console screen
package main
import "fmt"
func main() {
ch := make(chan bool)
go func() {
data := <-ch
fmt.Printf("Received: %t", data)
}()
ch <- true
}
At the same time, the following piece of code works as expected everytime, one difference is that an additional check has been added to check if the channel is closed or not which always throws the same expected output.
Does this ensure that a check on the channel is a must than optional ? or anything wrong with the code
package main
import "fmt"
func main() {
ch := make(chan bool)
go func() {
data, ok := <-ch
if !ok {
fmt.Println("Channel closed")
return
}
fmt.Printf("Received: %t", data)
}()
ch <- true
}
You should wait for goroutine to complete before main routine exit.
package main
import (
"fmt"
"sync"
)
func main() {
ch := make(chan bool)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
data := <-ch
fmt.Printf("Received: %t", data)
}()
ch <- true
wg.Wait()
}
The thing is your second piece of code doesn't print Received: true every time. I tested it several times.
As #jub0bs mentioned there is no guarantee that your goroutine finishes before the main routine. You must control it yourself.
package main
import (
"fmt"
"time"
)
func main() {
c := make(chan int)
go func() {
fmt.Println("hello")
c <- 10
}()
time.Sleep(2 * time.Second)
}
In the above program, I have created a Go routine which is writing to channel c but there is no other go routine which is reading from the channel. Why isnt there a deadlock in this case?
A deadlock implies all goroutines being blocked, not just one arbitrary goroutine of your choosing.
The main goroutine is simply in a sleep, once that is over, it can continue to run.
If you switch the sleep with a select{} blocking forever operation, you'll get your deadlock:
c := make(chan int)
go func() {
fmt.Println("hello")
c <- 10
}()
select {}
Try it on the Go Playground.
See related: Why there is no error that receiver is blocked?
While SayHello() executes as expected, the goroutine prints nothing.
package main
import "fmt"
func SayHello() {
for i := 0; i < 10 ; i++ {
fmt.Print(i, " ")
}
}
func main() {
SayHello()
go SayHello()
}
When your main() function ends, your program ends as well. It does not wait for other goroutines to finish.
Quoting from the Go Language Specification: Program Execution:
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
See this answer for more details.
You have to tell your main() function to wait for the SayHello() function started as a goroutine to complete. You can synchronize them with channels for example:
func SayHello(done chan int) {
for i := 0; i < 10; i++ {
fmt.Print(i, " ")
}
if done != nil {
done <- 0 // Signal that we're done
}
}
func main() {
SayHello(nil) // Passing nil: we don't want notification here
done := make(chan int)
go SayHello(done)
<-done // Wait until done signal arrives
}
Another alternative is to signal the completion by closing the channel:
func SayHello(done chan struct{}) {
for i := 0; i < 10; i++ {
fmt.Print(i, " ")
}
if done != nil {
close(done) // Signal that we're done
}
}
func main() {
SayHello(nil) // Passing nil: we don't want notification here
done := make(chan struct{})
go SayHello(done)
<-done // A receive from a closed channel returns the zero value immediately
}
Notes:
According to your edits/comments: if you want the 2 running SayHello() functions to print "mixed" numbers randomly: you have no guarantee to observe such behaviour. Again, see the aforementioned answer for more details. The Go Memory Model only guarantees that certain events happen before other events, you have no guarantee how 2 concurrent goroutines are executed.
You might experiment with it, but know that the result will not be deterministic. First you have to enable multiple active goroutines to be executed with:
runtime.GOMAXPROCS(2)
And second you have to first start SayHello() as a goroutine because your current code first executes SayHello() in the main goroutine and only once it finished starts the other one:
runtime.GOMAXPROCS(2)
done := make(chan struct{})
go SayHello(done) // FIRST START goroutine
SayHello(nil) // And then call SayHello() in the main goroutine
<-done // Wait for completion
Alternatively (to icza's answer) you can use WaitGroup from sync package and anonymous function to avoid altering original SayHello.
package main
import (
"fmt"
"sync"
)
func SayHello() {
for i := 0; i < 10; i++ {
fmt.Print(i, " ")
}
}
func main() {
SayHello()
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
SayHello()
}()
wg.Wait()
}
In order to print numbers simultaneously run each print statement in separate routine like the following
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func(fnScopeI int) {
defer wg.Done()
// next two strings are here just to show routines work simultaneously
amt := time.Duration(rand.Intn(250))
time.Sleep(time.Millisecond * amt)
fmt.Print(fnScopeI, " ")
}(i)
}
wg.Wait()
}
A Go program exits when the main function returns.
One option is to use something like sync.WaitGroup to wait on the other goroutines that main has spawned before returning from main.
Another option is to call runtime.Goexit() in main. From the godoc:
Goexit terminates the goroutine that calls it. No other goroutine is affected. Goexit runs all deferred calls before terminating the goroutine. Because Goexit is not a panic, any recover calls in those deferred functions will return nil.
Calling Goexit from the main goroutine terminates that goroutine without func main returning. Since func main has not returned, the program continues execution of other goroutines. If all other goroutines exit, the program crashes.
This allows main goroutine to stop executing while the background routines continue to execute. For example:
package main
import (
"fmt"
"runtime"
"time"
)
func f() {
for i := 0; ; i++ {
fmt.Println(i)
time.Sleep(10 * time.Millisecond)
}
}
func main() {
go f()
runtime.Goexit()
}
This can be cleaner than blocking forever in the main function, especially for programs that are infinite. One downside is that if all of the goroutines of a process return or exit (including the main goroutine), Go will detect this as an error and panic:
fatal error: no goroutines (main called runtime.Goexit) - deadlock!
To avoid this, at least one goroutine must call os.Exit before it returns. Calling os.Exit(0) immediately terminates the program and indicates that it did so without error. For example:
package main
import (
"fmt"
"os"
"runtime"
"time"
)
func f() {
for i := 0; i < 10; i++ {
fmt.Println(i)
time.Sleep(10 * time.Millisecond)
}
os.Exit(0)
}
func main() {
go f()
runtime.Goexit()
}
I'm having trouble figuring out how to correctly use sync.Cond. From what I can tell, a race condition exists between locking the Locker and invoking the condition's Wait method. This example adds an artificial delay between the two lines in the main goroutine to simulate the race condition:
package main
import (
"sync"
"time"
)
func main() {
m := sync.Mutex{}
c := sync.NewCond(&m)
go func() {
time.Sleep(1 * time.Second)
c.Broadcast()
}()
m.Lock()
time.Sleep(2 * time.Second)
c.Wait()
}
[Run on the Go Playground]
This causes an immediate panic:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [semacquire]:
sync.runtime_Syncsemacquire(0x10330208, 0x1)
/usr/local/go/src/runtime/sema.go:241 +0x2e0
sync.(*Cond).Wait(0x10330200, 0x0)
/usr/local/go/src/sync/cond.go:63 +0xe0
main.main()
/tmp/sandbox301865429/main.go:17 +0x1a0
What am I doing wrong? How do I avoid this apparent race condition? Is there a better synchronization construct I should be using?
Edit: I realize I should have better explained the problem I'm trying to solve here. I have a long-running goroutine that downloads a large file and a number of other goroutines that need access to the HTTP headers when they are available. This problem is harder than it sounds.
I can't use channels since only one goroutine would then receive the value. And some of the other goroutines would be trying to retrieve the headers long after they are already available.
The downloader goroutine could simply store the HTTP headers in a variable and use a mutex to safeguard access to them. However, this doesn't provide a way for the other goroutines to "wait" for them to become available.
I had thought that both a sync.Mutex and sync.Cond together could accomplish this goal but it appears that this is not possible.
OP answered his own, but did not directly answer the original question, I am going to post how to correctly use sync.Cond.
You do not really need sync.Cond if you have one goroutine for each write and read - a single sync.Mutex would suffice to communicate between them. sync.Cond could useful in situations where multiple readers wait for the shared resources to be available.
var sharedRsc = make(map[string]interface{})
func main() {
var wg sync.WaitGroup
wg.Add(2)
m := sync.Mutex{}
c := sync.NewCond(&m)
go func() {
// this go routine wait for changes to the sharedRsc
c.L.Lock()
for len(sharedRsc) == 0 {
c.Wait()
}
fmt.Println(sharedRsc["rsc1"])
c.L.Unlock()
wg.Done()
}()
go func() {
// this go routine wait for changes to the sharedRsc
c.L.Lock()
for len(sharedRsc) == 0 {
c.Wait()
}
fmt.Println(sharedRsc["rsc2"])
c.L.Unlock()
wg.Done()
}()
// this one writes changes to sharedRsc
c.L.Lock()
sharedRsc["rsc1"] = "foo"
sharedRsc["rsc2"] = "bar"
c.Broadcast()
c.L.Unlock()
wg.Wait()
}
Playground
Having said that, using channels is still the recommended way to pass data around if the situation permitting.
Note: sync.WaitGroup here is only used to wait for the goroutines to complete their executions.
You need to make sure that c.Broadcast is called after your call to c.Wait. The correct version of your program would be:
package main
import (
"fmt"
"sync"
)
func main() {
m := &sync.Mutex{}
c := sync.NewCond(m)
m.Lock()
go func() {
m.Lock() // Wait for c.Wait()
c.Broadcast()
m.Unlock()
}()
c.Wait() // Unlocks m, waits, then locks m again
m.Unlock()
}
https://play.golang.org/p/O1r8v8yW6h
package main
import (
"fmt"
"sync"
"time"
)
func main() {
m := sync.Mutex{}
m.Lock() // main gouroutine is owner of lock
c := sync.NewCond(&m)
go func() {
m.Lock() // obtain a lock
defer m.Unlock()
fmt.Println("3. goroutine is owner of lock")
time.Sleep(2 * time.Second) // long computing - because you are the owner, you can change state variable(s)
c.Broadcast() // State has been changed, publish it to waiting goroutines
fmt.Println("4. goroutine will release lock soon (deffered Unlock")
}()
fmt.Println("1. main goroutine is owner of lock")
time.Sleep(1 * time.Second) // initialization
fmt.Println("2. main goroutine is still lockek")
c.Wait() // Wait temporarily release a mutex during wating and give opportunity to other goroutines to change the state.
// Because you don't know, whether this is state, that you are waiting for, is usually called in loop.
m.Unlock()
fmt.Println("Done")
}
http://play.golang.org/p/fBBwoL7_pm
Looks like you c.Wait for Broadcast which would never happens with your time intervals.
With
time.Sleep(3 * time.Second) //Broadcast after any Wait for it
c.Broadcast()
your snippet seems to work http://play.golang.org/p/OE8aP4i6gY .Or am I missing something that you try to achive?
I finally discovered a way to do this and it doesn't involve sync.Cond at all - just the mutex.
type Task struct {
m sync.Mutex
headers http.Header
}
func NewTask() *Task {
t := &Task{}
t.m.Lock()
go func() {
defer t.m.Unlock()
// ...do stuff...
}()
return t
}
func (t *Task) WaitFor() http.Header {
t.m.Lock()
defer t.m.Unlock()
return t.headers
}
How does this work?
The mutex is locked at the beginning of the task, ensuring that anything calling WaitFor() will block. Once the headers are available and the mutex unlocked by the goroutine, each call to WaitFor() will execute one at a time. All future calls (even after the goroutine ends) will have no problem locking the mutex, since it will always be left unlocked.
Yes you can use one channel to pass Header to multiple Go routines.
headerChan := make(chan http.Header)
go func() { // This routine can be started many times
header := <-headerChan // Wait for header
// Do things with the header
}()
// Feed the header to all waiting go routines
for more := true; more; {
select {
case headerChan <- r.Header:
default: more = false
}
}
This can be done with channels pretty easily and the code will be clean. Below is the example. Hope this helps!
package main
import (
"fmt"
"net/http"
"sync"
)
func main() {
done := make(chan struct{})
var wg sync.WaitGroup
// fork required number of goroutines
for i := 0; i < 5; i++ {
wg.Add(1)
go func() {
defer wg.Done()
<-done
fmt.Println("read the http headers from here")
}()
}
time.Sleep(1) //download your large file here
fmt.Println("Unblocking goroutines...")
close(done) // this will unblock all the goroutines
wg.Wait()
}
In the excellent book "Concurrency in Go" they provide the following easy solution while leveraging the fact that a channel that is closed will release all waiting clients.
package main
import (
"fmt"
"time"
)
func main() {
httpHeaders := []string{}
headerChan := make(chan interface{})
var consumerFunc= func(id int, stream <-chan interface{}, funcHeaders *[]string)
{
<-stream
fmt.Println("Consumer ",id," got headers:", funcHeaders )
}
for i:=0;i<3;i++ {
go consumerFunc(i, headerChan, &httpHeaders)
}
fmt.Println("Getting headers...")
time.Sleep(2*time.Second)
httpHeaders=append(httpHeaders, "test1");
fmt.Println("Publishing headers...")
close(headerChan )
time.Sleep(5*time.Second)
}
https://play.golang.org/p/cE3SiKWNRIt
I have some issues with the following code:
package main
import (
"fmt"
"sync"
)
// This program should go to 11, but sometimes it only prints 1 to 10.
func main() {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(2)
go Print(ch, wg) //
go func(){
for i := 1; i <= 11; i++ {
ch <- i
}
close(ch)
defer wg.Done()
}()
wg.Wait() //deadlock here
}
// Print prints all numbers sent on the channel.
// The function returns when the channel is closed.
func Print(ch <-chan int, wg sync.WaitGroup) {
for n := range ch { // reads from channel until it's closed
fmt.Println(n)
}
defer wg.Done()
}
I get a deadlock at the specified place. I have tried setting wg.Add(1) instead of 2 and it solves my problem. My belief is that I'm not successfully sending the channel as an argument to the Printer function. Is there a way to do that? Otherwise, a solution to my problem is replacing the go Print(ch, wg)line with:
go func() {
Print(ch)
defer wg.Done()
}
and changing the Printer function to:
func Print(ch <-chan int) {
for n := range ch { // reads from channel until it's closed
fmt.Println(n)
}
}
What is the best solution?
Well, first your actual error is that you're giving the Print method a copy of the sync.WaitGroup, so it doesn't call the Done() method on the one you're Wait()ing on.
Try this instead:
package main
import (
"fmt"
"sync"
)
func main() {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(2)
go Print(ch, &wg)
go func() {
for i := 1; i <= 11; i++ {
ch <- i
}
close(ch)
defer wg.Done()
}()
wg.Wait() //deadlock here
}
func Print(ch <-chan int, wg *sync.WaitGroup) {
for n := range ch { // reads from channel until it's closed
fmt.Println(n)
}
defer wg.Done()
}
Now, changing your Print method to remove the WaitGroup of it is a generally good idea: the method doesn't need to know something is waiting for it to finish its job.
I agree with #Elwinar's solution, that the main problem in your code caused by passing a copy of your Waitgroup to the Print function.
This means the wg.Done() is operated on a copy of wg you defined in the main. Therefore, wg in the main could not get decreased, and thus a deadlock happens when you wg.Wait() in main.
Since you are also asking about the best practice, I could give you some suggestions of my own:
Don't remove defer wg.Done() in Print. Since your goroutine in main is a sender, and print is a receiver, removing wg.Done() in receiver routine will cause an unfinished receiver. This is because only your sender is synced with your main, so after your sender is done, your main is done, but it's possible that the receiver is still working. My point is: don't leave some dangling goroutines around after your main routine is finished. Close them or wait for them.
Remember to do panic recovery everywhere, especially anonymous goroutine. I have seen a lot of golang programmers forgetting to put panic recovery in goroutines, even if they remember to put recover in normal functions. It's critical when you want your code to behave correctly or at least gracefully when something unexpected happened.
Use defer before every critical calls, like sync related calls, at the beginning since you don't know where the code could break. Let's say you removed defer before wg.Done(), and a panic occurrs in your anonymous goroutine in your example. If you don't have panic recover, it will panic. But what happens if you have a panic recover? Everything's fine now? No. You will get deadlock at wg.Wait() since your wg.Done() gets skipped because of panic! However, by using defer, this wg.Done() will be executed at the end, even if panic happened. Also, defer before close is important too, since its result also affects the communication.
So here is the code modified according to the points I mentioned above:
package main
import (
"fmt"
"sync"
)
func main() {
ch := make(chan int)
var wg sync.WaitGroup
wg.Add(2)
go Print(ch, &wg)
go func() {
defer func() {
if r := recover(); r != nil {
println("panic:" + r.(string))
}
}()
defer func() {
wg.Done()
}()
for i := 1; i <= 11; i++ {
ch <- i
if i == 7 {
panic("ahaha")
}
}
println("sender done")
close(ch)
}()
wg.Wait()
}
func Print(ch <-chan int, wg *sync.WaitGroup) {
defer func() {
if r := recover(); r != nil {
println("panic:" + r.(string))
}
}()
defer wg.Done()
for n := range ch {
fmt.Println(n)
}
println("print done")
}
Hope it helps :)