Take the following code snippet:
func main() {
ch := make(chan int)
quit := make(chan int)
go func() {
for {
ch <- querySomePeriodicThing()
}
}()
// ...
loop:
for {
select {
case <-ch: handlePeriodicThing()
case <-quit: break loop
}
}
}
The goroutine should run for the duration of execution. When the select statement receives something from the quit channel, it breaks out of the loop and the program ends, without any attempt to stop the goroutine.
My question: will this have any intermittent adverse effects that are not obvious from running it once or twice? I know that in other languages threads should be cleaned up (i.e., exited) before the program ends, but is go different? Assume that querySomePeriodicThing() does not open file descriptors or sockets or anything that would be bad to leave open.
As mentioned in the spec, your program will exit when the main function completes:
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
So the fact you have other goroutines still running is not a problem from a language point of view. It may still be a problem depending on what your program is doing.
If the goroutine has created some resources that should be cleaned up before program exit, then having execution stop mid-way through might be a problem: in this case, you should make your main function wait for them to complete first. There is no equivalent to pthread_join, so you will need to code this yourself (e.g. by using a channel or sync.WaitGroup).
Note that for some resources are automatically cleaned up by the operating system on process exit (e.g. open files, file locks, etc), so in some cases no special clean up will be necessary
Goroutines aren't threads, they are very lightweight and the runtime automatically cleans them up when they are no longer running, or if the program exits.
Related
I'm reading The Little Go Book.
Page 76 demonstrates how you can deadlock with a single lock:
var (
lock sync.Mutex
)
func main() {
go func() { lock.Lock() }()
time.Sleep(time.Millisecond * 10)
lock.Lock()
}
Running this results in a deadlock as explained by the author. However, what I don't understand is why.
I changed the program to this:
var (
lock sync.Mutex
)
func main() {
go func() { lock.Lock() }()
lock.Lock()
}
My expectation was that a deadlock would still be thrown. But it wasn't.
Could someone explain to me what's happening here please?
The only scenario I can think of that explains this is the following (but this is guesswork):
First example
The lock is acquired in the first goroutine
The call to time.Sleep() ensures the lock is acquired
The main function attempts to acquire the lock resulting in a deadlock
Program exits
Second example
The lock is acquired in the first goroutine but this takes some time to happen (??)
Since there is no delay the main function acquires the lock before the goroutine can
Program exits
In the first example, main sleeps long enough to give the child goroutine the opportunity to start and acquire the lock. That goroutine then will merrily exit without releasing the lock.
By the time main resumes its control flow, the shared mutex is locked so the next attempt to acquire it will block forever. Since at this point main is the only routine left alive, blocking forever results in a deadlock.
In the second example, without the call to time.Sleep, main proceeds straight away to acquire the lock. This succeeds, so main goes ahead and exits. The child goroutine would then block forever, but since main has exited, the program just terminates, without deadlock.
By the way, even if main didn't exit, as long as there is at least one goroutine which is not blocking, there's no deadlock. For this purpose, time.Sleep is not a blocking operation, it simply pauses execution for the specified time duration.
go shows the deadlock error when all goroutines (including the main) are asleep.
in your first example, the inside goroutine is executed and terminated after he call mutex.Lock(). then the main goroutine tries to lock again but it goes asleep waiting for the opportunity to occupy the lock. so now we have all the goroutines(the main one) in the program in asleep mode which will cause a deadlock error !
it is important to understand this because a deadlock may happen but it will not always show an error if there still a running goroutine. which mostly what will happen in production. error will be reported only when the whole program get in a deadlock.
Trying to understand the flow of goroutines so i wrote this code only one thing which i am not able to understand is that how routine-end runs between the other go routines and complete a single go routines and print the output from the channel at the end.
import(
"fmt"
)
func add(dataArr []int,dataChannel chan int,i int ){
var sum int
fmt.Println("GOROUTINE",i+1)
for i:=0;i<len(dataArr);i++{
sum += dataArr[i]
}
fmt.Println("wRITING TO CHANNEL.....")
dataChannel <- sum
fmt.Println("routine-end")
}
func main(){
fmt.Println("main() started")
dataChannel := make(chan int)
dataArr := []int{1,2,3,4,5,6,7,8,9}
for i:=0;i<len(dataArr);i+=3{
go add(dataArr[i:i+3],dataChannel,i)
}
fmt.Println("came to blocking statement ..........")
fmt.Println(<-dataChannel)
fmt.Println("main() end")
}
output
main() started
came to blocking statement ..........
GOROUTINE 1
wRITING TO CHANNEL.....
routine-end
GOROUTINE 4
wRITING TO CHANNEL.....
6
main() end
Your for loop launches 3 goroutines that invoke the add function.
In addition, main itself runs in a separate "main" goroutine.
Since goroutines execute concurrently, the order of their run is typically unpredictable and depends on timing, how busy your machine is, etc. Results may differ between runs and between machines. Inserting time.Sleep calls in various places may help visualize it. For example, inserting time.Sleep for 100ms before "came to blocking statement" shows that all add goroutines launch.
What you may see in your run typically is that one add goroutine launches, adds up its slice to its sum and writes sum to dataChannel. Since main launches a few goroutines and immediately reads from the channel, this read gets the sum written by add and then the program exists -- because by default main won't wait for all goroutines to finish.
Moreover, since the dataChannel channel is unbuffered and main only reads one value, the other add goroutines will block on the channel indefinitely while writing.
I do recommend going over some introductory resources for goroutines and channels. They build up the concepts from simple principles. Some good links for you:
Golang tour
https://gobyexample.com/ -- start with the Goroutines example and do the next several ones.
Just have a question, what is happening here?
forever := make(chan bool)
log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
<-forever
That code creates an unbuffered channel, and attempts to receive from it.
And since no one ever sends anything on it, it's essentially a blocking forever operation.
The purpose of this is to keep the goroutine from ending / returning, most likely because there are other goroutines which do some work concurrently or they wait for certain events or incoming messages (like your log message says).
And the need for this is that without this, the application might quit without waiting for other goroutines. Namely, if the main goroutine ends, the program ends as well. Quoting from Spec: Program execution:
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
Check out this answer for similar and more techniques: Go project's main goroutine sleep forever?
For an introduction about channels, see What are channels used for?
The code in the question is probably coming from the RabbitMQ golang tutorial here.
Here's a more extended chunk of it with some commends of my own:
...
// create an unbuffered channel for bool types.
// Type is not important but we have to give one anyway.
forever := make(chan bool)
// fire up a goroutine that hooks onto msgs channel and reads
// anything that pops into it. This essentially is a thread of
// execution within the main thread. msgs is a channel constructed by
// previous code.
go func() {
for d := range msgs {
log.Printf("Received a message: %s", d.Body)
}
}()
log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
// We need to block the main thread so that the above thread stays
// on reading from msgs channel. To do that just try to read in from
// the forever channel. As long as no one writes to it we will wait here.
// Since we are the only ones that know of it it is guaranteed that
// nothing gets written in it. We could also do a busy wait here but
// that would waste CPU cycles for no good reason.
<-forever
I am working with a piece of code that has an intentional infinite loop, I can't modify that code. I want to write some tests on that method (e.g. make sure it triggers actions at the right times) but I don't want to orphan a bunch of go routines. So I am trying to find a way that I can kill/interrupt that goroutine.
I was thinking of trying to wrap it in a wrapper function that would kill it after a signal. Like this (doesn't work).
func wrap(inf func()) func() {
return func() {
select {
case inf():
case <-time.After(5 * time.Second):
}
}
}
func main() {
go wrap(inf())()
// do things
}
All the variations I can think of don't really work. I was thinking of wrapping the inf in a function that writes to a channel (that will never get called), or something with a return statement. And then the select can read from that. The problem is then you have to launch that. If you do it in this routine you're never getting to the select. If you do it in another routine, you've just made the problem worse.
So, is there a way that I can kill that routine?
(yes - I would rather change the infinite loop code, but can't here)
If you can't change the loop code, you can't kill the loop. Go is rather intentionally designed such that there's no way to kill a goroutine from outside of the goroutine, short of actually terminating the program itself.
If you can change the loop itself, the typical method of killing a routine is to provide a quit channel to the goroutine, and then close (or send on) that channel to tell the loop to exit. Example:
quitCh := make(chan struct{})
go func() {
for {
select {
case <-quitCh:
return
// other cases to handle what you need to do
}
}
}()
// Once you're done
close(quitCh) // goroutine exits
But without some way to coding that closure behavior into the loop itself, there's no way (to my knowledge) of specifically killing that goroutine or terminating the loop within it (unless you can trigger a panic in it, but that's a terrible way to handle that issue)
To settle some misunderstandings I have about goroutines, I went to the Go playground and ran this code:
package main
import (
"fmt"
)
func other(done chan bool) {
done <- true
go func() {
for {
fmt.Println("Here")
}
}()
}
func main() {
fmt.Println("Hello, playground")
done := make(chan bool)
go other(done)
<-done
fmt.Println("Finished.")
}
As I expected, Go playground came back with an error: Process took too long.
This seems to imply that the goroutine created within other runs forever.
But when I run the same code on my own machine, I get this output almost instantaneously:
Hello, playground.
Finished.
This seems to imply that the goroutine within other exits when the main goroutine finishes. Is this true? Or does the main goroutine finish, while the other goroutine continues to run in the background?
Edit: Default GOMAXPROCS has changed on the Go Playground, it now defaults to 8. In the "old" days it defaulted to 1. To get the behavior described in the question, set it to 1 explicitly with runtime.GOMAXPROCS(1).
Explanation of what you see:
On the Go Playground, GOMAXPROCS is 1 (proof).
This means one goroutine is executed at a time, and if that goroutine does not block, the scheduler is not forced to switch to other goroutines.
Your code (like every Go app) starts with a goroutine executing the main() function (the main goroutine). It starts another goroutine that executes the other() function, then it receives from the done channel - which blocks. So the scheduler must switch to the other goroutine (executing other() function).
In your other() function when you send a value on the done channel, that makes both the current (other()) and the main goroutine runnable. The scheduler chooses to continue to run other(), and since GOMAXPROCS=1, main() is not continued. Now other() launches another goroutine executing an endless loop. The scheduler chooses to execute this goroutine which takes forever to get to a blocked state, so main() is not continued.
And then the timeout of the Go Playground's sandbox comes as an absolution:
process took too long
Note that the Go Memory Model only guarantees that certain events happen before other events, you have no guarantee how 2 concurrent goroutines are executed. Which makes the output non-deterministic.
You are not to question any execution order that does not violate the Go Memory Model. If you want the execution to reach certain points in your code (to execute certain statements), you need explicit synchronization (you need to synchronize your goroutines).
Also note that the output on the Go Playground is cached, so if you run the app again, it won't be run again, but instead the cached output will be presented immediately. If you change anything in the code (e.g. insert a space or a comment) and then you run it again, it then will be compiled and run again. You will notice it by the increased response time. Using the current version (Go 1.6) you will see the same output every time though.
Running locally (on your machine):
When you run it locally, most likely GOMAXPROCS will be greater than 1 as it defaults to the number of CPU cores available (since Go 1.5). So it doesn't matter if you have a goroutine executing an endless loop, another goroutine will be executed simultaneously, which will be the main(), and when main() returns, your program terminates; it does not wait for other non-main goroutines to complete (see Spec: Program execution).
Also note that even if you set GOMAXPROCS to 1, your app will most likely exit in a "short" time as the scheduler imlementation will switch to other goroutines and not just execute the endless loop forever (however, as stated above, this is non-deterministic). And when it does, it will be the main() goroutine, and so when main() finishes and returns, your app terminates.
Playing with your app on the Go Playground:
As mentioned, by default GOMAXPROCS is 1 on the Go Playground. However it is allowed to set it to a higher value, e.g.:
runtime.GOMAXPROCS(2)
Without explicit synchronization, execution still remains non-deterministic, however you will observe a different execution order and a termination without running into a timeout:
Hello, playground
Here
Here
Here
...
<Here is printed 996 times, then:>
Finished.
Try this variant on the Go Playground.
What you will see on screen is nondeterministic. Or more precisely if by any chance the true value you pass to channel is delayed you would see some "Here".
But usually the Stdout is buffered, it means it's not printed instantaneously but the data gets accumulated and after it gets to maximum buffer size it's printed. In your case before the "here" is printed the main function is already finished thus the process finishes.
The rule of thumb is: main function must be alive otherwise all other goroutines gets killed.