I am currently building a CLI using Go and am trying to disable any backtrace that is produced as a result of a panic. I believe my code has great error handling, but would now like to suppress any panic messages (fairly new to Go).
I currently put in the following in my main.go function (to purposely cause a panic):
var myarr [2]string
myarr[0] = "Foo"
myarr[1] = "Bar"
for i := range myarr {
fmt.Println(myarr[i+1])
}
And I get this as a result:
goroutine 1 [running]:
Bar
panic: runtime error: index out of range [2] with length 2
main.main()
{directory where main.go is located}/main.go:23 +0x207
How can I suppress this error such that anyone with the executable binary file will not be able to see this error?
I've tried utilizing the GOBACKTRACE environment variable when building my binary and setting its value to GOBACKTRACE=none, but this has no effect on other operating systems I've tested on.
I've tried utilizing the GOBACKTRACE environment variable when building my binary and setting its value to GOBACKTRACE=none, but this has no effect on other operating systems I've tested on.
The environment variable is called GOTRACEBACK, not GOBACKTRACE.
Also, you can use debug.SetTraceback("none") to achieve the same effect, although this can still be overridden via the GOTRACEBACK environment variable.
If you use the correct naming, it should work. If it does not work, congratulations: you found a bug in golang and you should likely report it.
As #Burak mentioned you want to use the built-in Go function recover. There's a good Go blog post on all the subtleties of panic and recovery.
If you want to blanket cover your entire application stack, then simply register recover via a defer function at the main level:
func main() {
defer func() {
if r := recover(); r != nil {
fmt.Println("unexpected problem encountered - aborting")
// optionally log `r` to an exception log - so users may email to developer
os.Exit(1)
}
}()
run(os.Args...)
}
https://play.golang.org/p/b8qYnlNZsr5
Another question How to read/write from/to file using Go? got into safe closing of file descriptors in a comment.
Note that these examples aren't checking the error return from
fo.Close(). From the Linux man pages close(2): Not checking the return
value of close() is a common but nevertheless serious programming
error. It is quite possible that errors on a previous write(2)
operation are first reported at the final close(). Not checking the
return value when closing the file may lead to silent loss of data.
This can especially be observed with NFS and with disk quota. – Nick
Craig-Wood Jan 25 '13 at 7:12
The solution that updated the post used a panic:
// close fo on exit and check for its returned error
defer func() {
if err := fo.Close(); err != nil {
panic(err)
}
}()
I want to hand this error as a value instead of panicking.
If we are afraid of writes not being completed close isn't enough, so updating the error is still not correct.
The correct solution if you want to not hit this is to fsync the file(s):
defer(fd.Close())
// Do stuff
return fd.Sync()
It's easier to read then returning a non-nil modified error either through defer or maintaining throughout the function.
This will be a performance hit, but will catch both close errors for writing to buffers and the physical write to disk.
Think of a large project which deals with tons of concurrent requests handled by its own goroutine. It happens that there is a bug in the code and one of these requests will cause panic due to a nil reference.
In Java, C# and many other languages, this would end up in a exception which would stop the request without any harm to other healthy requests. In go, that would crash the entire program.
AFAIK, I'd have to have recover() for every single new go routine creation. Is that the only way to prevent entire program from crashing?
UPDATE: adding recover() call for every gorouting creation seems OK. What about third-party libraries? If third party creates goroutines without recover() safe net, it seems there is NOTHING to be done.
If you go the defer-recover-all-the-things, I suggest investing some time to make sure that a clear error message is collected with enough information to promptly act on it.
Writing the panic message to stderr/stdout is not great as it will be very hard to find where the problem is. In my experience the best approach is to invest a bit of time to get your Go programs to handle errors in a reasonable way. errors.Wrap from "github.com/pkg/errors" for instance allows you to wrap all errors and get a stack-trace.
Recovering panic is often a necessary evil. Like you say, it's not ideal to crash the entire program just because one requested caused a panic. In most cases recovering panics will not back-fire, but it is possible for a program to end up in a undefined not-recoverable state that only a manual restart can fix. That being said, my suggestion in this case is to make sure your Go program exposes a way to create a core dump.
Here's how to write a core dump to stderr when SIGQUIT is sent to the Go program (eg. kill pid -QUIT)
go func() {
// Based on answers to this stackoverflow question:
// https://stackoverflow.com/questions/19094099/how-to-dump-goroutine-stacktraces
sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGQUIT)
for {
<-sigs
fmt.Fprintln(os.Stderr, "=== received SIGQUIT ===")
fmt.Fprintln(os.Stderr, "*** goroutine dump...")
var buf []byte
var bufsize int
var stacklen int
// Create a stack buffer of 1MB and grow it to at most 100MB if
// necessary
for bufsize = 1e6; bufsize < 100e6; bufsize *= 2 {
buf = make([]byte, bufsize)
stacklen = runtime.Stack(buf, true)
if stacklen < bufsize {
break
}
}
fmt.Fprintln(os.Stderr, string(buf[:stacklen]))
fmt.Fprintln(os.Stderr, "*** end of dump")
}
}()
there is no way you can handle panic without recover function, a good practice would be using a middleware like function for your safe function, checkout this snippet
https://play.golang.org/p/d_fQWzXnlAm
I have Java background, and I love to use signal QUIT to inspect Java thread dump.
How to let Golang print out all goroutines stack trace?
To print the stack trace for the current goroutine, use PrintStack() from runtime/debug.
PrintStack prints to standard error the stack trace returned by Stack.
For example:
import(
"runtime/debug"
)
...
debug.PrintStack()
To print the stack trace for all goroutines use Lookup and WriteTo from runtime/pprof.
func Lookup(name string) *Profile
// Lookup returns the profile with the given name,
// or nil if no such profile exists.
func (p *Profile) WriteTo(w io.Writer, debug int) error
// WriteTo writes a pprof-formatted snapshot of the profile to w.
// If a write to w returns an error, WriteTo returns that error.
// Otherwise, WriteTo returns nil.
Each Profile has a unique name. A few profiles are predefined:
goroutine - stack traces of all current goroutines
heap - a sampling of all heap allocations
threadcreate - stack traces that led to the creation of new OS threads
block - stack traces that led to blocking on synchronization primitives
For example:
pprof.Lookup("goroutine").WriteTo(os.Stdout, 1)
There is an HTTP frontend for the runtime/pprof package mentioned in Intermernet's answer. Import the net/http/pprof package to register an HTTP handler for /debug/pprof:
import _ "net/http/pprof"
import _ "net/http"
Start an HTTP listener if you do not have one already:
go func() {
log.Println(http.ListenAndServe("localhost:6060", nil))
}()
Then point a browser to http://localhost:6060/debug/pprof for a menu, or http://localhost:6060/debug/pprof/goroutine?debug=2 for a full goroutine stack dump.
There are other fun things you can learn about your running code this way too. Check out the blog post for examples and more details:
http://blog.golang.org/profiling-go-programs
Similar to Java, SIGQUIT can be used to print a stack trace of a Go program and its goroutines.
A key difference, however, is that by default sending SIGQUIT to Java programs do not terminate them, while Go programs do exit.
This approach requires no code change to print a stack trace of all goroutines of existing programs.
The environment variable GOTRACEBACK (see documentation of the runtime package) controls the amount of output generated. For example, to include all goroutines, set GOTRACEBACK=all.
The printing of the stack trace is triggered by an unexpected runtime condition (unhandled signal), originally documented in this commit, making it available since at least Go 1.1.
Alternatively, if modifying source code is an option, see other answers.
Note that in a Linux terminal, SIGQUIT can be conveniently sent with the key combination Ctrl+\.
To mimic the Java behaviour of stack-dump on SIGQUIT but still leaving the program running:
go func() {
sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGQUIT)
buf := make([]byte, 1<<20)
for {
<-sigs
stacklen := runtime.Stack(buf, true)
log.Printf("=== received SIGQUIT ===\n*** goroutine dump...\n%s\n*** end\n", buf[:stacklen])
}
}()
You can use runtime.Stack to get the stack trace of all goroutines:
buf := make([]byte, 1<<16)
runtime.Stack(buf, true)
fmt.Printf("%s", buf)
From the documentation:
func Stack(buf []byte, all bool) int
Stack formats a stack trace of the calling goroutine into buf and
returns the number of bytes written to buf. If all is true, Stack
formats stack traces of all other goroutines into buf after the trace
for the current goroutine.
Press CTRL+\
(If you run it in a terminal and just want to kill your program and dump the go routines etc)
I found this question looking for the key sequence. Just wanted a quick and easy way to tell if my program is leaking go routines :)
On *NIX systems (including OSX) send a signal abort SIGABRT:
pkill -SIGABRT program_name
By default, press ^\ keys ( CTRL+\ ) to dump the stack traces of all goroutines.
Otherwise, for more granular control, you can use panic. The simple way as of Go 1.6+:
go func() {
s := make(chan os.Signal, 1)
signal.Notify(s, syscall.SIGQUIT)
<-s
panic("give me the stack")
}()
Then, run your program like so:
# Press ^\ to dump the stack traces of all the user-created goroutines
$ GOTRACEBACK=all go run main.go
If you also want to print go runtime goroutines:
$ GOTRACEBACK=system go run main.go
Here are all the GOTRACEBACK options:
GOTRACEBACK=none omits the goroutine stack traces entirely.
GOTRACEBACK=single (the default) behaves as described above.
GOTRACEBACK=all adds stack traces for all user-created goroutines.
GOTRACEBACK=system is like all but adds stack frames for run-time functions and shows goroutines created internally by the run-time.
GOTRACEBACK=crash is like system but crashes in an operating system-specific manner instead of exiting. For example, on Unix systems, the crash raises SIGABRT to trigger a core dump.
Here is the documentation
The GOTRACEBACK variable controls the amount of output generated when a Go program fails due to an unrecovered panic or an unexpected runtime condition.
By default, a failure prints a stack trace for the current goroutine, eliding functions internal to the run-time system, and then exits with exit code 2. The failure prints stack traces for all goroutines if there is no current goroutine or the failure is internal to the run-time.
For historical reasons, the GOTRACEBACK settings 0, 1, and 2 are synonyms for none, all, and system, respectively.
The runtime/debug package's SetTraceback function allows increasing the amount of output at run time, but it cannot reduce the amount below that specified by the environment variable. See https://golang.org/pkg/runtime/debug/#SetTraceback.
It's necessary to use the length returned by runtime.Stack() to avoid printing a bunch of empty lines after your stack trace. The following recovery function prints a nicely formatted trace:
if r := recover(); r != nil {
log.Printf("Internal error: %v", r))
buf := make([]byte, 1<<16)
stackSize := runtime.Stack(buf, true)
log.Printf("%s\n", string(buf[0:stackSize]))
}
You can use this:
kill -3 YOUR_PROCESS_PID_ID
I am running a program in Go which sends data continuously after reading a file /proc/stat.
Using ioutil.ReadFile("/proc/stat")
After running for about 14 hrs i got err: too many files open /proc/stat
Click here for snippet of code.
I doubt that defer f.Close is ignored by Go sometimes or it is skipping it.
The snippet of code (in case play.golang.org dies sooner than stackoverflow.com):
package main
import ("fmt";"io/ioutil")
func main() {
for {
fmt.Println("Hello, playground")
fData,err := ioutil.ReadFile("/proc/stat")
if err != nil {
fmt.Println("Err is ",err)
}
fmt.Println("FileData",string(fData))
}
}
The reason probably is that somewhere in your program:
you are forgetting to close files, or
you are leaning on the garbage collector to automatically close files on object finalization, but Go's conservative garbage collector fails to do so. In this case you should check your program's memory consumption (whether it is steadily increasing while the program is running).
In either case, try to check the contents of /proc/PID/fd to see whether the number of open files is increasing while the program is running.
If you are sure you Do the f.Close(),it still has the proble,Maybe it is because your other connection,for example the connection to MYSQL,also will be cause the problem,especially,in a loop,and you forget to close the connection.
Always do :
db.connection....
**defer db.Close()**
If it is in loop
loop
db.connection....
**defer db.Close()**
end
Do not put the db.connection before the loop