I can run some cleanup on interruption (when I press ctrlc).
$ go build
$ ./exit
^Creceived interrupt signal
Is it possible to trap the same way os.Exit call and run some code before program exits? The code:
package main
import (
"fmt"
"os"
"os/signal"
"syscall"
"time"
)
func main() {
handleInterrupt(2)
time.Sleep(2 * time.Second)
os.Exit(1) // how to trap it?
}
func handleInterrupt(intrptChSize int) {
s := make(chan os.Signal, intrptChSize)
signal.Notify(s,
syscall.SIGABRT,
syscall.SIGALRM,
syscall.SIGBUS,
syscall.SIGCHLD,
syscall.SIGCONT,
syscall.SIGEMT,
syscall.SIGFPE,
syscall.SIGHUP,
syscall.SIGILL,
syscall.SIGINFO,
syscall.SIGINT,
syscall.SIGIO,
syscall.SIGIOT,
syscall.SIGKILL,
syscall.SIGPIPE,
syscall.SIGPROF,
syscall.SIGQUIT,
syscall.SIGSEGV,
syscall.SIGSTOP,
syscall.SIGSYS,
syscall.SIGTERM,
syscall.SIGTRAP,
syscall.SIGTSTP,
syscall.SIGTTIN,
syscall.SIGTTOU,
syscall.SIGURG,
syscall.SIGUSR1,
syscall.SIGUSR2,
syscall.SIGVTALRM,
syscall.SIGWINCH,
syscall.SIGXCPU,
syscall.SIGXFSZ)
go func() {
for sig := range s {
fmt.Printf("received %s signal\n", sig)
//cleanup()
}
}()
}
I know I can just run cleanup before each os.Exit() code manually in this example:
cleanup()
os.Exit(1)
But in my real project I am importing code which I cannot edit. This code contains os.Exit calls and I would like to do some cleanup before the program exits without editing imported code.
You can't. from TFM:
The program terminates immediately; deferred functions are not run.
Related
When running tests from command line, capturing SIGINT works fine. However, is there a way to pass SIGINT signal to code when running tests from GoLand IDE?
When running from command line:
go test -v -run TestSth and then calling Ctrl + C it captures fine.
Example code:
EDIT: it seems my example code now captures SIGINT as intended (Goland 2022.3.2)
package main
import (
"fmt"
"os"
"os/signal"
"syscall"
"testing"
"time"
)
func TestMain(m *testing.M) {
terminate := make(chan os.Signal)
signal.Notify(terminate, syscall.SIGINT)
go func() {
<-terminate
fmt.Println()
fmt.Println("CAPTURED!!!") // want to get here when running tests from IDE
}()
exitCode := m.Run()
os.Exit(exitCode)
}
func TestSth(t *testing.T) {
time.Sleep(time.Second * 5)
}
Get the current process information by calling FindProcess on the current PID and signal the interrupt to it using Process.Signal
func TestSth(t *testing.T) {
go func() {
// Sleep added for demonstrative purposes, can be removed
time.Sleep(time.Second * 1)
p, err := os.FindProcess(os.Getpid())
if err != nil {
panic(err)
}
p.Signal(syscall.SIGINT)
}()
time.Sleep(time.Second * 5)
}
main.go
func main() {
fmt.Println("hello")
ch := make(chan struct{}, 1)
<-ch
}
main_test.go
func Test_Main(t *testing.T) {
main()
}
go run main.go
hello
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
but
go test -v main_test.go -run=Test_Main
=== RUN Test_Main
hello
go test will not report an error and will always run.
After consulting a lot of information, I didn't find an answer to explain this phenomenon. Maybe my way is wrong?This channel method is used in projects.
Thanks.
When you run a regular program, it waits for input from channel. And because there is only one goroutine there is no way to receive the input from channel (no other thread to send to it). Thus deadlock is reported.
On the other hand test runner uses goroutines to execute tests. So there is more then one goroutine spawned and the deadlock is not detected (runtime assumes that other goroutine could send to channel).
To answer your question from comment: go run and go test are not supposed to achieve the same effects. go run executes your program, go test executes procedures that test your code. These commands executes two different programs.
I am not sure if you can detect this kind of errors (deadlocks) with tests.
Edit:
go test waits for test to finish (you can configure how long with -timeout d option). So I assume it spawns goroutine that waits for timer.Timer to expire, so there is no deadlock (there is always one goroutine that has a chance to be executed).
Edit2:
Try this program:
package main
import (
"fmt"
"time"
)
func main() {
go func() {
t := time.NewTimer(10 * time.Second)
<-t.C
}()
fmt.Println("hello")
ch := make(chan struct{}, 1)
<-ch
}
It waits 10 seconds before reporting deadlock.
Edit3:
Or take a look at flowing code that illustrates how test runner works:
package main
import (
"fmt"
"time"
)
func original_main_func() {
fmt.Println("hello")
ch := make(chan struct{}, 1)
<-ch
}
func test() {
original_main_func()
}
func test_runner() {
ch := make(chan struct{}, 1)
go func() {
test()
close(ch)
}()
t := time.NewTimer(10 * time.Second)
select {
case <-t.C:
panic("timeout")
case <-ch:
fmt.Println("test executed")
}
}
func main() {
test_runner()
}
While SayHello() executes as expected, the goroutine prints nothing.
package main
import "fmt"
func SayHello() {
for i := 0; i < 10 ; i++ {
fmt.Print(i, " ")
}
}
func main() {
SayHello()
go SayHello()
}
When your main() function ends, your program ends as well. It does not wait for other goroutines to finish.
Quoting from the Go Language Specification: Program Execution:
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
See this answer for more details.
You have to tell your main() function to wait for the SayHello() function started as a goroutine to complete. You can synchronize them with channels for example:
func SayHello(done chan int) {
for i := 0; i < 10; i++ {
fmt.Print(i, " ")
}
if done != nil {
done <- 0 // Signal that we're done
}
}
func main() {
SayHello(nil) // Passing nil: we don't want notification here
done := make(chan int)
go SayHello(done)
<-done // Wait until done signal arrives
}
Another alternative is to signal the completion by closing the channel:
func SayHello(done chan struct{}) {
for i := 0; i < 10; i++ {
fmt.Print(i, " ")
}
if done != nil {
close(done) // Signal that we're done
}
}
func main() {
SayHello(nil) // Passing nil: we don't want notification here
done := make(chan struct{})
go SayHello(done)
<-done // A receive from a closed channel returns the zero value immediately
}
Notes:
According to your edits/comments: if you want the 2 running SayHello() functions to print "mixed" numbers randomly: you have no guarantee to observe such behaviour. Again, see the aforementioned answer for more details. The Go Memory Model only guarantees that certain events happen before other events, you have no guarantee how 2 concurrent goroutines are executed.
You might experiment with it, but know that the result will not be deterministic. First you have to enable multiple active goroutines to be executed with:
runtime.GOMAXPROCS(2)
And second you have to first start SayHello() as a goroutine because your current code first executes SayHello() in the main goroutine and only once it finished starts the other one:
runtime.GOMAXPROCS(2)
done := make(chan struct{})
go SayHello(done) // FIRST START goroutine
SayHello(nil) // And then call SayHello() in the main goroutine
<-done // Wait for completion
Alternatively (to icza's answer) you can use WaitGroup from sync package and anonymous function to avoid altering original SayHello.
package main
import (
"fmt"
"sync"
)
func SayHello() {
for i := 0; i < 10; i++ {
fmt.Print(i, " ")
}
}
func main() {
SayHello()
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
SayHello()
}()
wg.Wait()
}
In order to print numbers simultaneously run each print statement in separate routine like the following
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
wg.Add(1)
go func(fnScopeI int) {
defer wg.Done()
// next two strings are here just to show routines work simultaneously
amt := time.Duration(rand.Intn(250))
time.Sleep(time.Millisecond * amt)
fmt.Print(fnScopeI, " ")
}(i)
}
wg.Wait()
}
A Go program exits when the main function returns.
One option is to use something like sync.WaitGroup to wait on the other goroutines that main has spawned before returning from main.
Another option is to call runtime.Goexit() in main. From the godoc:
Goexit terminates the goroutine that calls it. No other goroutine is affected. Goexit runs all deferred calls before terminating the goroutine. Because Goexit is not a panic, any recover calls in those deferred functions will return nil.
Calling Goexit from the main goroutine terminates that goroutine without func main returning. Since func main has not returned, the program continues execution of other goroutines. If all other goroutines exit, the program crashes.
This allows main goroutine to stop executing while the background routines continue to execute. For example:
package main
import (
"fmt"
"runtime"
"time"
)
func f() {
for i := 0; ; i++ {
fmt.Println(i)
time.Sleep(10 * time.Millisecond)
}
}
func main() {
go f()
runtime.Goexit()
}
This can be cleaner than blocking forever in the main function, especially for programs that are infinite. One downside is that if all of the goroutines of a process return or exit (including the main goroutine), Go will detect this as an error and panic:
fatal error: no goroutines (main called runtime.Goexit) - deadlock!
To avoid this, at least one goroutine must call os.Exit before it returns. Calling os.Exit(0) immediately terminates the program and indicates that it did so without error. For example:
package main
import (
"fmt"
"os"
"runtime"
"time"
)
func f() {
for i := 0; i < 10; i++ {
fmt.Println(i)
time.Sleep(10 * time.Millisecond)
}
os.Exit(0)
}
func main() {
go f()
runtime.Goexit()
}
Is there any way to exit a Go program, but execute all the pending defer statements?
I've been clearing up temporary files by using defer, but the deferred statements aren't executed when the program is interrupted with Ctrl+C or even os.Exit.
After exiting this program with Ctrl+C, both foo.txt and bar.txt are left over:
package main
import (
"fmt"
"io/ioutil"
"os"
"os/signal"
"syscall"
)
func main() {
ioutil.WriteFile("./foo.txt", []byte("foo"), 0644)
defer os.RemoveAll("./foo.txt")
go func() {
ioutil.WriteFile("./bar.txt", []byte("bar"), 0644)
defer os.RemoveAll("./bar.txt")
for {
// various long running things
}
}()
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
signal.Notify(c, syscall.SIGTERM)
go func() {
<-c
fmt.Println("Received OS interrupt - exiting.")
os.Exit(0)
}()
for {
// various long running things
}
}
From golang reference:
A "defer" statement invokes a function whose execution is deferred to
the moment the surrounding function returns
When you call os.Exit(0) you bypass the normal return procedure and your deferred functions are not executed.
Also, even if the deferred worked inside the main goroutine, the defers in other goroutines would not work since they would die before returning.
A better code architecture would allow you to get something similar. You need to think about your long running processes as workers. Export every long running process in workers and defer any clean up right after calling the worker. Use a select in the main goroutine to capture signals and synchronise work
package main
import (
"fmt"
"io/ioutil"
"os"
"os/signal"
"syscall"
"time"
)
func check(e error) {
if e != nil {
panic(e)
}
}
func main() {
ioutil.WriteFile("./foo.txt", []byte("foo"), 0644)
defer os.RemoveAll("./foo.txt")
// Worker 1
done := make(chan bool, 1)
go func(done chan bool) {
fmt.Println("worker 1 with bar ...")
ioutil.WriteFile("./bar.txt", []byte("bar"), 0644)
// various long running things
time.Sleep(3 * time.Second)
done <- true
}(done)
defer os.RemoveAll("./bar.txt")
// End worker1
s := make(chan os.Signal, 1)
signal.Notify(s, os.Interrupt)
signal.Notify(s, syscall.SIGTERM)
// Worker 2
done2 := make(chan bool, 1)
go func(done chan bool) {
fmt.Println("worker 2 ...")
time.Sleep(6 * time.Second)
done <- true
}(done2)
// End worker 2
select {
case <-s:
fmt.Println("Quiting with signal - exit")
case <-done:
<-done2
case <-done2:
<-done
}
return
}
This select is a quick and dirty way to handle two workers, a better way would be to use sync.WaitGroup
I would recommend not relying on defer, but defining a reusable function that can be used in a defer or in the signal block. Something like this:
package main
import (
"fmt"
"io/ioutil"
"os"
"os/signal"
"syscall"
)
func main() {
ioutil.WriteFile("./foo.txt", []byte("foo"), 0644)
cleanup := func(){
os.RemoveAll("./foo.txt")
os.RemoveAll("./bar.txt")
}
defer cleanup() //for normal return
go func() {
ioutil.WriteFile("./bar.txt", []byte("bar"), 0644)
for {
// various long running things
}
}()
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
signal.Notify(c, syscall.SIGTERM)
go func() {
<-c
fmt.Println("Received OS interrupt - exiting.")
cleanup()
os.Exit(0)
}()
for {
// various long running things
}
}
I've a long running server written in Go. Main fires off several goroutines where the logic of the program executes. After that main does nothing useful. Once main exits, the program will quit. The method I am using right now to keep the program running is just a simple call to fmt.Scanln(). I'd like to know how others keep main from exiting. Below is a basic example. What ideas or best practices could be used here?
I considered creating a channel and delaying exit of main by receiving on said channel, but I think that could be problematic if all my goroutines become inactive at some point.
Side note: In my server (not the example), the program isn't actually running connected to a shell, so it doesn't really make sense to interact with the console anyway. For now it works, but I'm looking for the "correct" way, assuming there is one.
package main
import (
"fmt"
"time"
)
func main() {
go forever()
//Keep this goroutine from exiting
//so that the program doesn't end.
//This is the focus of my question.
fmt.Scanln()
}
func forever() {
for ; ; {
//An example goroutine that might run
//indefinitely. In actual implementation
//it might block on a chanel receive instead
//of time.Sleep for example.
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
Block forever. For example,
package main
import (
"fmt"
"time"
)
func main() {
go forever()
select {} // block forever
}
func forever() {
for {
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
The current design of Go's runtime assumes that the programmer is responsible for detecting when to terminate a goroutine and when to terminate the program. The programmer needs to compute the termination condition for goroutines and also for the entire program. A program can be terminated in a normal way by calling os.Exit or by returning from the main() function.
Creating a channel and delaying exit of main() by immediately receiving on said channel is a valid approach of preventing main from exiting. But it does not solve the problem of detecting when to terminate the program.
If the number of goroutines cannot be computed before the main() function enters the wait-for-all-goroutines-to-terminate loop, you need to be sending deltas so that main function can keep track of how many goroutines are in flight:
// Receives the change in the number of goroutines
var goroutineDelta = make(chan int)
func main() {
go forever()
numGoroutines := 0
for diff := range goroutineDelta {
numGoroutines += diff
if numGoroutines == 0 { os.Exit(0) }
}
}
// Conceptual code
func forever() {
for {
if needToCreateANewGoroutine {
// Make sure to do this before "go f()", not within f()
goroutineDelta <- +1
go f()
}
}
}
func f() {
// When the termination condition for this goroutine is detected, do:
goroutineDelta <- -1
}
An alternative approach is to replace the channel with sync.WaitGroup. A drawback of this approach is that wg.Add(int) needs to be called before calling wg.Wait(), so it is necessary to create at least 1 goroutine in main() while subsequent goroutines can be created in any part of the program:
var wg sync.WaitGroup
func main() {
// Create at least 1 goroutine
wg.Add(1)
go f()
go forever()
wg.Wait()
}
// Conceptual code
func forever() {
for {
if needToCreateANewGoroutine {
wg.Add(1)
go f()
}
}
}
func f() {
// When the termination condition for this goroutine is detected, do:
wg.Done()
}
Go's runtime package has a function called runtime.Goexit that will do exactly what you want.
Calling Goexit from the main goroutine terminates that goroutine
without func main returning. Since func main has not returned,
the program continues execution of other goroutines.
If all other goroutines exit, the program crashes.
Example in the playground
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
go func() {
time.Sleep(time.Second)
fmt.Println("Go 1")
}()
go func() {
time.Sleep(time.Second * 2)
fmt.Println("Go 2")
}()
runtime.Goexit()
fmt.Println("Exit")
}
Nobody mentioned signal.Notify(c chan<- os.Signal, sig ...os.Signal)
Example:
package main
import (
"fmt"
"time"
"os"
"os/signal"
"syscall"
)
func main() {
go forever()
quitChannel := make(chan os.Signal, 1)
signal.Notify(quitChannel, syscall.SIGINT, syscall.SIGTERM)
<-quitChannel
//time for cleanup before exit
fmt.Println("Adios!")
}
func forever() {
for {
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
Here is a simple block forever using channels
package main
import (
"fmt"
"time"
)
func main() {
done := make(chan bool)
go forever()
<-done // Block forever
}
func forever() {
for {
fmt.Printf("%v+\n", time.Now())
time.Sleep(time.Second)
}
}
You could daemonize the process using Supervisor (http://supervisord.org/). Your function forever would just be a process that it runs, and it would handle the part of your function main. You would use the supervisor control interface to start/shutdown/check on your process.