I'm going through 'A Tour of Go' and have been editing most of the lessons to make sure I fully understand them. I have a question regarding: https://tour.golang.org/concurrency/1
package main
import (
"fmt"
"time"
)
func say(s string) {
for i := 0; i < 5; i++ {
time.Sleep(100 * time.Millisecond)
fmt.Println(s)
}
}
func main() {
go say("world")
say("hello")
}
Leaving main the way it is produces a random ordering of hellos and worlds because the threads are executing in different orders each time the program runs. I have two questions:
If I remove go from the line with world and add it to the line with hello, world is printed 5 times and hello is not printed at all. Why is that?
If I add go in front of both lines, nothing is printed at all. Why is that?
I have some experience with concurrency using C++ (although it was a while ago) and some more recent experience with Python, but would describe my overall experience with concurrency fairly novice-level.
Thanks!
The program terminates before you get a chance to see the results.
You can fix this by adding a statement that ensures main doesn't exit before the other routines are finished.
From Go - Concurrency:
With a goroutine we return immediately to the next line and don't wait for the function to complete.
They give a code example:
package main
import "fmt"
func f(n int) {
for i := 0; i < 10; i++ {
fmt.Println(n, ":", i)
}
}
func main() {
go f(0)
var input string
fmt.Scanln(&input)
}
In regards to the code above:
This is why the call to the Scanln function has been included; without it the program would exit before being given the opportunity to print all the numbers.
If I remove the go keyword from the line say("world") and add it to the line say("hello"), world is printed 5 times and hello is not printed at all. Why is that?
If I add the go in front of both lines, nothing is printed at all. Why is that?
In both cases the same problem occurs, you are executing unsynchronized operations. It results in your program returning from main before all started work was processed. The specification states that When that [main] function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
When you use the go keyword, the runtime starts the execution of a function call as an independent concurrent thread of control. You need to use the synchronization language primitives to re synchronize the exit order of operations following the call to main with the remaining asynchronous jobs.
The language offers channels or otherwise WaitGroups via the sync package to help you implement that behavior.
For example
package main
import (
"fmt"
"time"
"sync"
)
var wg sync.WaitGroup
func say(s string) {
for i := 0; i < 5; i++ {
time.Sleep(100 * time.Millisecond)
fmt.Println(s)
wg.Done()
}
}
func main() {
wg.Add(5)
say("world")
wg.Add(5)
say("hello")
wg.Wait()
}
Related
Like most of you, I'm familiar with the fact that Go reuses the iterator variable in a for-loop, so closures for each goroutine will capture the same variable. A typical example of this the following code which will always print the final value of the loop from each goroutine:
func main() {
for i := 0; i < 5; i++ {
go func() {
fmt.Println(i) // prints 5 each time
}()
}
time.Sleep(100 * time.Millisecond)
}
What I haven't been able to find an explanation of, is why the goroutines do not execute until after the loop is completed. Even the following code produces the value of ii as 10 which is set after the goroutine is called:
func main() {
for i := 0; i < 5; i++ {
ii := i
go func() {
fmt.Println(ii) // prints 10 each time...!
}()
ii = 10
}
time.Sleep(100 * time.Millisecond)
}
The closest thing to an explanation I've read is that for-loops typically execute faster than goroutines. This raises two questions for me: 1. Is that true? 2. Why?
Never assume the order of execution when dealing with more than one goroutine - no matter how tempting it may be. Don't put in sleeps; don't call runtime.Gosched; don't assume anything.
The only guaranteed way to ensure order of execution is via synchronization methods such as channels, sync.Mutex, sync.WaitGroups etc.
Anything is possible because the program has a data race.
Setting aside the data race, there's no evidence that the goroutines execute after the for loop is completed. The value 10 is assigned to ii in the statement after the goroutine is started, not after the for loop.
The memory model allows for the compiler to optimize the two ii assignments to a single assignment at the start of the for loop body. It could be that the goroutines execute immediately, but the goroutines see the value of 10 from the optimization.
you should use the goroutine like this:
package main
import "fmt"
import "time"
func main() {
for i := 0; i < 5; i++ {
ii := i
go func(k int) {
fmt.Println(k) // prints 10 each time...!
}(ii)
ii = 10
}
time.Sleep(100 * time.Millisecond)
}
I am trying to run this below piece of code
package main
import (
"fmt"
"time"
)
func main() {
time.Sleep(time.Millisecond*6000)
fmt.Println("Done")
}
As one expects, it waits for 6 seconds, print "done" and then exits
But if I remove the print statement,
package main
import (
"time"
)
func main() {
time.Sleep(time.Millisecond*6000)
}
it doesn't wait and exits immediately. Why?
Consequently, look at the code below
package main
import (
"fmt"
"time"
)
func main() {
c := make(chan int)
go count(6, c)
time.Sleep(time.Millisecond*5000)
}
func count(num int, c chan int) {
for i := 1; i <= num; i++ {
fmt.Println(i)
c <- i
time.Sleep(time.Millisecond*2000)
}
close(c)
}
Here the count goroutine will get blocked trying to send i to a channel when no receiver is there to read it and the main function immediately exits even though there is a sleep statement after it. But when I remove the statement
c <- i
the count goroutine gets to count till 3 since the main function does wait for those 5 seconds as stated.
What is going on here?
Run it locally, and it will wait. The output on the Go Playground is cached. If there is no output, it doesn't make you wait 6 seconds for nothing. If there is output, the timing of the output is preserved.
Read blog post: The Go Blog: Inside the Go Playground:
We capture the timing of each write to standard output and standard error and provide it to the client. Then the client can "play back" the writes with the correct timing, so that the output appears just as if the program were running locally.
I've been playing around with the following code from A Tour of Go, but I don't understand what is going on when I apply some minor changes. The original code is this
package main
import (
"fmt"
"time"
)
func say(s string) {
for i := 0; i < 5; i++ {
time.Sleep(100 * time.Millisecond)
fmt.Println(s)
}
}
func main() {
go say("world")
say("hello")
}
and it produces this
world
hello
hello
world
world
hello
hello
world
world
hello
which is OK: five times hello, five times world. I starts to get strange when I call
say("world")
go say("hello")
Now the output is just
world
world
world
world
world
No hello whatsoever. It gets even weirder with two goroutines
go say("world")
go say("hello")
Now there is no output at all. When I change i < 5 to i < 2 and call
go say("world")
say("hello")
I get
world
hello
hello
What am I missing here?
In the case of
say("world")
go say("hello")
The "world" call must complete before the "hello" goroutine is started. The "hello" goroutine does not run or complete because main returns.
For
go say("world")
go say("hello")
the goroutines do not run or complete because main returns.
Use sync.WaitGroup to prevent main from exiting before the goroutines complete:
func say(wg *sync.WaitGroup, s string) {
defer wg.Done()
for i := 0; i < 5; i++ {
time.Sleep(100 * time.Millisecond)
fmt.Println(s)
}
}
func main() {
var wg sync.WaitGroup
wg.Add(2)
go say(&wg, "world")
go say(&wg, "hello")
wg.Wait()
}
playground example
Congratulations for learning Go. As someone new, it is nice to understand concurrency and how it is different from parallelism.
Concurrency
Concurrency is like a juggler juggling several balls in the air with one hand. No matter how many balls he is juggling, only one ball touch his hand at any moment.
Parallelism
When the juggler starts juggling more balls with another hand in parallel, we have two concurrent processes running at the same time.
Goroutines are great because they're both concurrent and auto-parallel, depending on the computing cores available and the GOMAXPROCS variable being set.
The One-handed Juggler
Back to the one-handed, single-cored, concurrent juggler. Imagine him juggling three balls named "hello", "world", and "mars" respectively with the hand being the main routine.
var balls = []string{"hello", "world", "mars"}
func main() {
go say(balls[0])
go say(balls[1])
go say(balls[2])
}
Or more appropriately,
func main() {
for _, ball := range balls {
go say(ball)
}
}
Once the three balls are thrown up into the air sequentially, the juggler simply retreats his hand right away. That is, the main routine exits before the first ball thrown can even land on his hand. Shame, the balls just drop to the ground. Bad show.
In order to get the balls back in his hand, the juggler has to make sure he waits for them. This means his hand needs to be able to keep track of and count the balls he threw and learn when each is landing.
The most straightforward way is to use sync.WaitGroup:
import (
"fmt"
"time"
"sync"
)
var balls = []string{"hello", "world", "mars"}
var wg sync.WaitGroup
func main() {
for _, ball := range balls {
// One ball thrown
wg.Add(1)
go func(b string) {
// Signals the group that this routine is done.
defer wg.Done()
// each ball hovers for 1 second
time.Sleep(time.Duration(1) * time.Second)
fmt.Println(b)
// wg.Done() is called before goroutine exits
}(ball)
}
// The juggler can do whatever he pleases while the
// balls are hanging in the air.
// This hand will come back to grab the balls after 1s.
wg.Wait()
}
WaitGroup is simple. When a goroutine is spawned, one adds to a "backlog counter" with WaitGroup.Add(1) and call WaitGroup.Done() to decrease the counter. Once the backlog becomes 0, it means that all goroutines are done and WaitGroup should stop waiting (and grab the balls!).
While using channel(s) for synchronization is fine, it is encouraged to use available concurrent tools as appropriate especially when the use of channels make the code more complex and hard to comprehend.
It is because the main function has been exited.
When main function return, all goroutines are abruptly terminated, then program exits.
You add a statment:
time.Sleep(100 * time.Second)
before main function return, and everything goes well.
But a good practice in Go is to use channel, which is used to communicate between goroutines. You can use it to let main function wait for background goroutines to finish.
With
func main() {
go say("world")
say("hello")
}
You are creating two separate goroutines, one is the main functions goroutine and one is the go say("world"). Normally when functions are executed the programs jumps to that function, execute all code inside and then jumps to the line after where the function was called from.
With goroutines you are not jumping inside the function but you are starting the goroutine in a separate thread and continuing to execute the line just after the call without waiting for it.
Therefore the goroutine will not have time to finish before the main goroutine is done.
I already set runtime.GOMAXPROCS(2), but this program still hang when output some numbers. I can see high cpu used by this program(more than 100%), but I can't understand why the for loop goroutine can make my program not work.
go version is 1.4.2 on linux/amd64, and my pc has 4 CPUs.
here is the code:
package main
import "fmt"
import "runtime"
import "time"
func forever() {
for {
}
}
func show() {
for number := 1; number < 999999; number++ {
time.Sleep(1000)
fmt.Println(number)
}
}
func main() {
runtime.GOMAXPROCS(2)
go show()
go forever()
for {
time.Sleep(1000)
}
}
There's no need to ever have a busy loop that does nothing except burn CPU time. Not only does it consume an entire OS thread, but goroutines are cooperatively scheduled, and it will interfere with the runtime's goroutines. For example, on Go1.5 this will usually block the stop-the-world phase of the GC, (which you can test by setting GOGC=off).
To make this program run, you could insert a scheduling point in the for loop, but it would be better to remove it altogether.
func forever() {
for {
runtime.Gosched()
}
}
From code, it looks like you want to print the number in for loop in go routine. In this case why not use channel to indicate when go routine is completed with for loop and accordingly exit the main function. Something like this
package main
import "fmt"
import "runtime"
import "time"
func show(result chan bool) {
for number := 1; number < 999999; number++ {
time.Sleep(1000)
fmt.Println(number)
}
result <- true
}
func main() {
runtime.GOMAXPROCS(2)
result := make(chan bool)
go show(result)
<- result
}
It's good to use runtime.Gosched().
But in golang, time.Duration is in Nanoseconds, so time.Sleep(1000) is nearly no sleep. mostly you consider it as milliseconds like things in Java. you can try
time.Sleep( 1000 * time.MilliSecond )
I've a long running server written in Go. Main fires off several goroutines where the logic of the program executes. After that main does nothing useful. Once main exits, the program will quit. The method I am using right now to keep the program running is just a simple call to fmt.Scanln(). I'd like to know how others keep main from exiting. Below is a basic example. What ideas or best practices could be used here?
I considered creating a channel and delaying exit of main by receiving on said channel, but I think that could be problematic if all my goroutines become inactive at some point.
Side note: In my server (not the example), the program isn't actually running connected to a shell, so it doesn't really make sense to interact with the console anyway. For now it works, but I'm looking for the "correct" way, assuming there is one.
package main
import (
"fmt"
"time"
)
func main() {
go forever()
//Keep this goroutine from exiting
//so that the program doesn't end.
//This is the focus of my question.
fmt.Scanln()
}
func forever() {
for ; ; {
//An example goroutine that might run
//indefinitely. In actual implementation
//it might block on a chanel receive instead
//of time.Sleep for example.
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
Block forever. For example,
package main
import (
"fmt"
"time"
)
func main() {
go forever()
select {} // block forever
}
func forever() {
for {
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
The current design of Go's runtime assumes that the programmer is responsible for detecting when to terminate a goroutine and when to terminate the program. The programmer needs to compute the termination condition for goroutines and also for the entire program. A program can be terminated in a normal way by calling os.Exit or by returning from the main() function.
Creating a channel and delaying exit of main() by immediately receiving on said channel is a valid approach of preventing main from exiting. But it does not solve the problem of detecting when to terminate the program.
If the number of goroutines cannot be computed before the main() function enters the wait-for-all-goroutines-to-terminate loop, you need to be sending deltas so that main function can keep track of how many goroutines are in flight:
// Receives the change in the number of goroutines
var goroutineDelta = make(chan int)
func main() {
go forever()
numGoroutines := 0
for diff := range goroutineDelta {
numGoroutines += diff
if numGoroutines == 0 { os.Exit(0) }
}
}
// Conceptual code
func forever() {
for {
if needToCreateANewGoroutine {
// Make sure to do this before "go f()", not within f()
goroutineDelta <- +1
go f()
}
}
}
func f() {
// When the termination condition for this goroutine is detected, do:
goroutineDelta <- -1
}
An alternative approach is to replace the channel with sync.WaitGroup. A drawback of this approach is that wg.Add(int) needs to be called before calling wg.Wait(), so it is necessary to create at least 1 goroutine in main() while subsequent goroutines can be created in any part of the program:
var wg sync.WaitGroup
func main() {
// Create at least 1 goroutine
wg.Add(1)
go f()
go forever()
wg.Wait()
}
// Conceptual code
func forever() {
for {
if needToCreateANewGoroutine {
wg.Add(1)
go f()
}
}
}
func f() {
// When the termination condition for this goroutine is detected, do:
wg.Done()
}
Go's runtime package has a function called runtime.Goexit that will do exactly what you want.
Calling Goexit from the main goroutine terminates that goroutine
without func main returning. Since func main has not returned,
the program continues execution of other goroutines.
If all other goroutines exit, the program crashes.
Example in the playground
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
go func() {
time.Sleep(time.Second)
fmt.Println("Go 1")
}()
go func() {
time.Sleep(time.Second * 2)
fmt.Println("Go 2")
}()
runtime.Goexit()
fmt.Println("Exit")
}
Nobody mentioned signal.Notify(c chan<- os.Signal, sig ...os.Signal)
Example:
package main
import (
"fmt"
"time"
"os"
"os/signal"
"syscall"
)
func main() {
go forever()
quitChannel := make(chan os.Signal, 1)
signal.Notify(quitChannel, syscall.SIGINT, syscall.SIGTERM)
<-quitChannel
//time for cleanup before exit
fmt.Println("Adios!")
}
func forever() {
for {
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
Here is a simple block forever using channels
package main
import (
"fmt"
"time"
)
func main() {
done := make(chan bool)
go forever()
<-done // Block forever
}
func forever() {
for {
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
You could daemonize the process using Supervisor (http://supervisord.org/). Your function forever would just be a process that it runs, and it would handle the part of your function main. You would use the supervisor control interface to start/shutdown/check on your process.