I started to learn go language days ago. When I tried to start writing some fun codes, I am stuck by a strange behavior.
package main
import "fmt"
func recv(value int) {
if value < 0 {
return
}
fmt.Println(value)
go recv(value-1)
}
func main() {
recv(10)
}
when I run the above code, only 10 is printed. When I remove the go before the call to recv, 10 to 0 are printed out. I believe I am misusing go routine here, but I can not understand why it failed start a go routine this way.
When the main function returns, Go will not wait for any still existing goroutines to finish but instead just exit.
recv will return to main after the first "iteration" and because main has nothing more to do, the program will terminate.
One solution to this problem is to have a channel that signals that all work is done, like the following:
package main
import "fmt"
func recv(value int, ch chan bool) {
if value < 0 {
ch <- true
return
}
fmt.Println(value)
go recv(value - 1, ch)
}
func main() {
ch := make(chan bool)
recv(10, ch)
<-ch
}
Here, recv will send a single boolean before returning, and main will wait for that message on the channel.
For the logic of the program, it does not matter what type or specific value you use. bool and true are just a straightforward example. If you want to be more efficient, using a chan struct{} instead of a chan bool will save you an additional byte, since empty structs do not use any memory.
A sync.Waitgroup is another solution and specifically intended for the purpose of waiting for an arbitrary amount of goroutines to run their course.
package main
import (
"fmt"
"sync"
)
func recv(value int, wg *sync.WaitGroup) {
if value < 0 {
return
}
fmt.Println(value)
wg.Add(1) // Add 1 goroutine to the waitgroup.
go func() {
recv(value-1, wg)
wg.Done() // This goroutine is finished.
}()
}
func main() {
var wg sync.WaitGroup
recv(10, &wg)
// Block until the waitgroup signals
// all goroutines to be finished.
wg.Wait()
}
I did so and also worked. How come?
package main
import "fmt"
func recv(value int) {
if value < 0 {
return
}
fmt.Println(value)
recv(value - 1)
}
func main() {
recv(10)
}
Related
i am new in golang recently. i have a question about goroutine when use time.sleep function. here the code.
package main
import (
"fmt"
"time"
)
func go1(msg_chan chan string) {
for {
msg_chan <- "go1"
}
}
func go2(msg_chan chan string) {
for {
msg_chan <- "go2"
}
}
func count(msg_chan chan string) {
for {
msg := <-msg_chan
fmt.Println(msg)
time.Sleep(time.Second * 1)
}
}
func main() {
var c chan string
c = make(chan string)
go go1(c)
go go2(c)
go count(c)
var input string
fmt.Scanln(&input)
}
and output is
go1
go2
go1
go2
go1
go2
i think when count function is execute sleep function, go1 and go2 will execute in random sequence. so the out put maybe like
go1
go1
go2
go2
go2
go1
when i delete the sleep code in count function. the result as i supposed , it's random.
i am stucked in this issue.
thanks.
First thing to notice is that there are three go routines and all of them are independent of each other. The only thing that combines the two go routines with count routine is the channel on which both go routines are sending the values.
time.Sleep is not making the go routines synchronous. On using time.Sleep you are actually letting the count go routine to wait for that long which let other go routine to send the value on the channel which is available for the count go routine to be able to receive.
One more thing that you can do to check it is increase the number of CPU's which will give you the random result.
func GOMAXPROCS(n int) int
GOMAXPROCS sets the maximum number of CPUs that can be executing
simultaneously and returns the previous setting. If n < 1, it does not
change the current setting. The number of logical CPUs on the local
machine can be queried with NumCPU. This call will go away when the
scheduler improves.
The number of CPUs available simultaneously to executing goroutines is
controlled by the GOMAXPROCS shell environment variable, whose default
value is the number of CPU cores available. Programs with the
potential for parallel execution should therefore achieve it by
default on a multiple-CPU machine. To change the number of parallel
CPUs to use, set the environment variable or use the similarly-named
function of the runtime package to configure the run-time support to
utilize a different number of threads. Setting it to 1 eliminates the
possibility of true parallelism, forcing independent goroutines to
take turns executing.
Considering the part where output of the go routine is random, it is always random. But channels most probably work in queue which is FIFO(first in first out) as it depends on which value is available on the channel to b received. So whichever be the value available on the channel to be sent is letting the count go routine to wait and print that value.
Take for an example even if I am using time.Sleep the output is random:
package main
import (
"fmt"
"time"
)
func go1(msg_chan chan string) {
for i := 0; i < 10; i++ {
msg_chan <- fmt.Sprintf("%s%d", "go1:", i)
}
}
func go2(msg_chan chan string) {
for i := 0; i < 10; i++ {
msg_chan <- fmt.Sprintf("%s%d", "go2:", i)
}
}
func count(msg_chan chan string) {
for {
msg := <-msg_chan
fmt.Println(msg)
time.Sleep(time.Second * 1)
}
}
func main() {
var c chan string
c = make(chan string)
go go1(c)
go go2(c)
go count(c)
time.Sleep(time.Second * 20)
fmt.Println("finished")
}
This sometimes leads to race condition which is why we use synchronization either using channels or wait.groups.
package main
import (
"fmt"
"sync"
"time"
)
var wg sync.WaitGroup
func go1(msg_chan chan string) {
defer wg.Done()
for {
msg_chan <- "go1"
}
}
func go2(msg_chan chan string) {
defer wg.Done()
for {
msg_chan <- "go2"
}
}
func count(msg_chan chan string) {
defer wg.Done()
for {
msg := <-msg_chan
fmt.Println(msg)
time.Sleep(time.Second * 1)
}
}
func main() {
var c chan string
c = make(chan string)
wg.Add(1)
go go1(c)
wg.Add(1)
go go2(c)
wg.Add(1)
go count(c)
wg.Wait()
fmt.Println("finished")
}
Now coming to the part where you are using never ending for loop to send the values on a channel. So if you remove the time.Sleep your process will hang since the loop will never stop to send the values on the channel.
While SayHello() executes as expected, the goroutine prints nothing.
package main
import "fmt"
func SayHello() {
for i := 0; i < 10 ; i++ {
fmt.Print(i, " ")
}
}
func main() {
SayHello()
go SayHello()
}
When your main() function ends, your program ends as well. It does not wait for other goroutines to finish.
Quoting from the Go Language Specification: Program Execution:
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
See this answer for more details.
You have to tell your main() function to wait for the SayHello() function started as a goroutine to complete. You can synchronize them with channels for example:
func SayHello(done chan int) {
for i := 0; i < 10; i++ {
fmt.Print(i, " ")
}
if done != nil {
done <- 0 // Signal that we're done
}
}
func main() {
SayHello(nil) // Passing nil: we don't want notification here
done := make(chan int)
go SayHello(done)
<-done // Wait until done signal arrives
}
Another alternative is to signal the completion by closing the channel:
func SayHello(done chan struct{}) {
for i := 0; i < 10; i++ {
fmt.Print(i, " ")
}
if done != nil {
close(done) // Signal that we're done
}
}
func main() {
SayHello(nil) // Passing nil: we don't want notification here
done := make(chan struct{})
go SayHello(done)
<-done // A receive from a closed channel returns the zero value immediately
}
Notes:
According to your edits/comments: if you want the 2 running SayHello() functions to print "mixed" numbers randomly: you have no guarantee to observe such behaviour. Again, see the aforementioned answer for more details. The Go Memory Model only guarantees that certain events happen before other events, you have no guarantee how 2 concurrent goroutines are executed.
You might experiment with it, but know that the result will not be deterministic. First you have to enable multiple active goroutines to be executed with:
runtime.GOMAXPROCS(2)
And second you have to first start SayHello() as a goroutine because your current code first executes SayHello() in the main goroutine and only once it finished starts the other one:
runtime.GOMAXPROCS(2)
done := make(chan struct{})
go SayHello(done) // FIRST START goroutine
SayHello(nil) // And then call SayHello() in the main goroutine
<-done // Wait for completion
Alternatively (to icza's answer) you can use WaitGroup from sync package and anonymous function to avoid altering original SayHello.
package main
import (
"fmt"
"sync"
)
func SayHello() {
for i := 0; i < 10; i++ {
fmt.Print(i, " ")
}
}
func main() {
SayHello()
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
SayHello()
}()
wg.Wait()
}
In order to print numbers simultaneously run each print statement in separate routine like the following
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func(fnScopeI int) {
defer wg.Done()
// next two strings are here just to show routines work simultaneously
amt := time.Duration(rand.Intn(250))
time.Sleep(time.Millisecond * amt)
fmt.Print(fnScopeI, " ")
}(i)
}
wg.Wait()
}
A Go program exits when the main function returns.
One option is to use something like sync.WaitGroup to wait on the other goroutines that main has spawned before returning from main.
Another option is to call runtime.Goexit() in main. From the godoc:
Goexit terminates the goroutine that calls it. No other goroutine is affected. Goexit runs all deferred calls before terminating the goroutine. Because Goexit is not a panic, any recover calls in those deferred functions will return nil.
Calling Goexit from the main goroutine terminates that goroutine without func main returning. Since func main has not returned, the program continues execution of other goroutines. If all other goroutines exit, the program crashes.
This allows main goroutine to stop executing while the background routines continue to execute. For example:
package main
import (
"fmt"
"runtime"
"time"
)
func f() {
for i := 0; ; i++ {
fmt.Println(i)
time.Sleep(10 * time.Millisecond)
}
}
func main() {
go f()
runtime.Goexit()
}
This can be cleaner than blocking forever in the main function, especially for programs that are infinite. One downside is that if all of the goroutines of a process return or exit (including the main goroutine), Go will detect this as an error and panic:
fatal error: no goroutines (main called runtime.Goexit) - deadlock!
To avoid this, at least one goroutine must call os.Exit before it returns. Calling os.Exit(0) immediately terminates the program and indicates that it did so without error. For example:
package main
import (
"fmt"
"os"
"runtime"
"time"
)
func f() {
for i := 0; i < 10; i++ {
fmt.Println(i)
time.Sleep(10 * time.Millisecond)
}
os.Exit(0)
}
func main() {
go f()
runtime.Goexit()
}
This code selects all xml files in the same folder, as the invoked executable and asynchronously applies processing to each result in the callback method (in the example below, just the name of the file is printed out).
How do I avoid using the sleep method to keep the main method from exiting? I have problems wrapping my head around channels (I assume that's what it takes, to synchronize the results) so any help is appreciated!
package main
import (
"fmt"
"io/ioutil"
"path"
"path/filepath"
"os"
"runtime"
"time"
)
func eachFile(extension string, callback func(file string)) {
exeDir := filepath.Dir(os.Args[0])
files, _ := ioutil.ReadDir(exeDir)
for _, f := range files {
fileName := f.Name()
if extension == path.Ext(fileName) {
go callback(fileName)
}
}
}
func main() {
maxProcs := runtime.NumCPU()
runtime.GOMAXPROCS(maxProcs)
eachFile(".xml", func(fileName string) {
// Custom logic goes in here
fmt.Println(fileName)
})
// This is what i want to get rid of
time.Sleep(100 * time.Millisecond)
}
You can use sync.WaitGroup. Quoting the linked example:
package main
import (
"net/http"
"sync"
)
func main() {
var wg sync.WaitGroup
var urls = []string{
"http://www.golang.org/",
"http://www.google.com/",
"http://www.somestupidname.com/",
}
for _, url := range urls {
// Increment the WaitGroup counter.
wg.Add(1)
// Launch a goroutine to fetch the URL.
go func(url string) {
// Decrement the counter when the goroutine completes.
defer wg.Done()
// Fetch the URL.
http.Get(url)
}(url)
}
// Wait for all HTTP fetches to complete.
wg.Wait()
}
WaitGroups are definitely the canonical way to do this. Just for the sake of completeness, though, here's the solution that was commonly used before WaitGroups were introduced. The basic idea is to use a channel to say "I'm done," and have the main goroutine wait until each spawned routine has reported its completion.
func main() {
c := make(chan struct{}) // We don't need any data to be passed, so use an empty struct
for i := 0; i < 100; i++ {
go func() {
doSomething()
c <- struct{}{} // signal that the routine has completed
}()
}
// Since we spawned 100 routines, receive 100 messages.
for i := 0; i < 100; i++ {
<- c
}
}
sync.WaitGroup can help you here.
package main
import (
"fmt"
"sync"
"time"
)
func wait(seconds int, wg * sync.WaitGroup) {
defer wg.Done()
time.Sleep(time.Duration(seconds) * time.Second)
fmt.Println("Slept ", seconds, " seconds ..")
}
func main() {
var wg sync.WaitGroup
for i := 0; i <= 5; i++ {
wg.Add(1)
go wait(i, &wg)
}
wg.Wait()
}
Although sync.waitGroup (wg) is the canonical way forward, it does require you do at least some of your wg.Add calls before you wg.Wait for all to complete. This may not be feasible for simple things like a web crawler, where you don't know the number of recursive calls beforehand and it takes a while to retrieve the data that drives the wg.Add calls. After all, you need to load and parse the first page before you know the size of the first batch of child pages.
I wrote a solution using channels, avoiding waitGroup in my solution the the Tour of Go - web crawler exercise. Each time one or more go-routines are started, you send the number to the children channel. Each time a go routine is about to complete, you send a 1 to the done channel. When the sum of children equals the sum of done, we are done.
My only remaining concern is the hard-coded size of the the results channel, but that is a (current) Go limitation.
// recursionController is a data structure with three channels to control our Crawl recursion.
// Tried to use sync.waitGroup in a previous version, but I was unhappy with the mandatory sleep.
// The idea is to have three channels, counting the outstanding calls (children), completed calls
// (done) and results (results). Once outstanding calls == completed calls we are done (if you are
// sufficiently careful to signal any new children before closing your current one, as you may be the last one).
//
type recursionController struct {
results chan string
children chan int
done chan int
}
// instead of instantiating one instance, as we did above, use a more idiomatic Go solution
func NewRecursionController() recursionController {
// we buffer results to 1000, so we cannot crawl more pages than that.
return recursionController{make(chan string, 1000), make(chan int), make(chan int)}
}
// recursionController.Add: convenience function to add children to controller (similar to waitGroup)
func (rc recursionController) Add(children int) {
rc.children <- children
}
// recursionController.Done: convenience function to remove a child from controller (similar to waitGroup)
func (rc recursionController) Done() {
rc.done <- 1
}
// recursionController.Wait will wait until all children are done
func (rc recursionController) Wait() {
fmt.Println("Controller waiting...")
var children, done int
for {
select {
case childrenDelta := <-rc.children:
children += childrenDelta
// fmt.Printf("children found %v total %v\n", childrenDelta, children)
case <-rc.done:
done += 1
// fmt.Println("done found", done)
default:
if done > 0 && children == done {
fmt.Printf("Controller exiting, done = %v, children = %v\n", done, children)
close(rc.results)
return
}
}
}
}
Full source code for the solution
Here is a solution that employs WaitGroup.
First, define 2 utility methods:
package util
import (
"sync"
)
var allNodesWaitGroup sync.WaitGroup
func GoNode(f func()) {
allNodesWaitGroup.Add(1)
go func() {
defer allNodesWaitGroup.Done()
f()
}()
}
func WaitForAllNodes() {
allNodesWaitGroup.Wait()
}
Then, replace the invocation of callback:
go callback(fileName)
With a call to your utility function:
util.GoNode(func() { callback(fileName) })
Last step, add this line at the end of your main, instead of your sleep. This will make sure the main thread is waiting for all routines to finish before the program can stop.
func main() {
// ...
util.WaitForAllNodes()
}
I am trying to create a simple program to learn channels in Go.
But I´m running in to a deadlock error, which I can´t figure out
package main
import (
"fmt"
"time"
)
func printer(c chan int) {
for i := 0; i < 10; i++ {
c <- i
time.Sleep(time.Second)
}
}
func reciever(c chan int) {
for {
recievedMsg := <-c
fmt.Println(recievedMsg)
}
}
func main() {
newChanel := make(chan int)
printer(newChanel)
reciever(newChanel)
}
My initial thoughts was something about the Sleep function, but even if I don´t include this I still run into this error and exit message.
Can anyone give some hints on how to solve this?
Thanks in advance
You need two execution threads because now there is no way for the reciever function to be called as you never leave the printer function. You need to execute one of them on a separate goroutine.
You should also close the channel and use the range operator in your loop, so that it ends when the channel is closed.
So I propose you this code :
func printer(c chan int) {
for i := 0; i < 10; i++ {
c <- i
time.Sleep(time.Second)
}
close(c)
}
func reciever(c chan int) {
for recievedMsg := range c {
fmt.Println(recievedMsg)
}
}
func main() {
newChanel := make(chan int)
go printer(newChanel)
reciever(newChanel)
}
I've a long running server written in Go. Main fires off several goroutines where the logic of the program executes. After that main does nothing useful. Once main exits, the program will quit. The method I am using right now to keep the program running is just a simple call to fmt.Scanln(). I'd like to know how others keep main from exiting. Below is a basic example. What ideas or best practices could be used here?
I considered creating a channel and delaying exit of main by receiving on said channel, but I think that could be problematic if all my goroutines become inactive at some point.
Side note: In my server (not the example), the program isn't actually running connected to a shell, so it doesn't really make sense to interact with the console anyway. For now it works, but I'm looking for the "correct" way, assuming there is one.
package main
import (
"fmt"
"time"
)
func main() {
go forever()
//Keep this goroutine from exiting
//so that the program doesn't end.
//This is the focus of my question.
fmt.Scanln()
}
func forever() {
for ; ; {
//An example goroutine that might run
//indefinitely. In actual implementation
//it might block on a chanel receive instead
//of time.Sleep for example.
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
Block forever. For example,
package main
import (
"fmt"
"time"
)
func main() {
go forever()
select {} // block forever
}
func forever() {
for {
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
The current design of Go's runtime assumes that the programmer is responsible for detecting when to terminate a goroutine and when to terminate the program. The programmer needs to compute the termination condition for goroutines and also for the entire program. A program can be terminated in a normal way by calling os.Exit or by returning from the main() function.
Creating a channel and delaying exit of main() by immediately receiving on said channel is a valid approach of preventing main from exiting. But it does not solve the problem of detecting when to terminate the program.
If the number of goroutines cannot be computed before the main() function enters the wait-for-all-goroutines-to-terminate loop, you need to be sending deltas so that main function can keep track of how many goroutines are in flight:
// Receives the change in the number of goroutines
var goroutineDelta = make(chan int)
func main() {
go forever()
numGoroutines := 0
for diff := range goroutineDelta {
numGoroutines += diff
if numGoroutines == 0 { os.Exit(0) }
}
}
// Conceptual code
func forever() {
for {
if needToCreateANewGoroutine {
// Make sure to do this before "go f()", not within f()
goroutineDelta <- +1
go f()
}
}
}
func f() {
// When the termination condition for this goroutine is detected, do:
goroutineDelta <- -1
}
An alternative approach is to replace the channel with sync.WaitGroup. A drawback of this approach is that wg.Add(int) needs to be called before calling wg.Wait(), so it is necessary to create at least 1 goroutine in main() while subsequent goroutines can be created in any part of the program:
var wg sync.WaitGroup
func main() {
// Create at least 1 goroutine
wg.Add(1)
go f()
go forever()
wg.Wait()
}
// Conceptual code
func forever() {
for {
if needToCreateANewGoroutine {
wg.Add(1)
go f()
}
}
}
func f() {
// When the termination condition for this goroutine is detected, do:
wg.Done()
}
Go's runtime package has a function called runtime.Goexit that will do exactly what you want.
Calling Goexit from the main goroutine terminates that goroutine
without func main returning. Since func main has not returned,
the program continues execution of other goroutines.
If all other goroutines exit, the program crashes.
Example in the playground
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
go func() {
time.Sleep(time.Second)
fmt.Println("Go 1")
}()
go func() {
time.Sleep(time.Second * 2)
fmt.Println("Go 2")
}()
runtime.Goexit()
fmt.Println("Exit")
}
Nobody mentioned signal.Notify(c chan<- os.Signal, sig ...os.Signal)
Example:
package main
import (
"fmt"
"time"
"os"
"os/signal"
"syscall"
)
func main() {
go forever()
quitChannel := make(chan os.Signal, 1)
signal.Notify(quitChannel, syscall.SIGINT, syscall.SIGTERM)
<-quitChannel
//time for cleanup before exit
fmt.Println("Adios!")
}
func forever() {
for {
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
Here is a simple block forever using channels
package main
import (
"fmt"
"time"
)
func main() {
done := make(chan bool)
go forever()
<-done // Block forever
}
func forever() {
for {
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
You could daemonize the process using Supervisor (http://supervisord.org/). Your function forever would just be a process that it runs, and it would handle the part of your function main. You would use the supervisor control interface to start/shutdown/check on your process.