I am writing a game in Go. I was told my code was leaking memory. What would be the right approach to pinpoint the cause of the leaks?
I know pprof exists but as far as I know it can be used when your app uses the http package and my app is a simple game and it does not make use of any netowrk connectivity.
You can certainly create profiles without an http server using the runtime/pprof package. There's also github.com/pkg/profile by Dave Cheney, a wrapper for runtime/pprof with a simpler API. You can use signal handlers to enable and disable profiling, for instance.
That being said, nothing stops you from starting an HTTP server, just for pprof. I find that much more convenient because then you can easily create any profile on demand.
// +build debug
package main
import (
"log"
"net/http"
_ "net/http/pprof/"
)
func init() {
log.Println("Launching pprof server")
go func() { log.Fatal(http.ListenAndServe("localhost:3030", nil)) }()
}
This uses a build tag so you can enable the server in development but exclude it from production builds. Compile with -tags debug to enable the server. See Build Constraints for details.
You can use https://golang.org/pkg/runtime/pprof/ package to write memory profiles on disk (you can do it let's say once in a minute). After that just run go tool pprof (inuse_space mode) on few of them and look if there are some parts where inuse_space is growing
Apart from the pprof discussed before, it might also be useful to check heap allocation by doing an escape analysis of your code (e.g. if file is main.go) by:
go build -gcflags=-m main.go
This immediately gives you an idea of things like if you are passing pointers around unnecessarily which can be avoided.
Here is a short article which goes through various ways you can make your program more efficient.
Related
Since importing net/http/pprof package is enough to enable the golang profiler in a program, my question is, is the profiler enabled by default for all golang programs, and importing the package just exposes the metrics endpoint? If that is the case, how could I disable the profiler to eliminate performance overhead?
Is profiling enabled by default for all go programs?
No.
is the profiler enabled by default for all golang programs, and importing the package just exposes the metrics endpoint?
No.
If that is the case, how could I disable the profiler to eliminate performance overhead?
No, that's not the case.
If you look at the source code of the pprof.go file, you can see in its handlers, that the profiling, tracing etc. are running only when you do a GET on any of its endpoints.
If you specify a time, for example: http://localhost:6060/debug/pprof/trace?seconds=10 the server will take 10 seconds to respond with the trace data. So the profiling is happening only if you call on an endpoint.
You can find some examples in the first comment block of the pprof.go file.
I'm writing a package in go that creates a connection to Kafka.
I create the connection in the init() function in the main file in the package. After the program that uses the package stops, I want to call a function called Close()
Is there a way to enforce it in the package level, instead of giving this responsibility to the user?
Assume that the connection should be available throughout the run of the user's program, I don't want to initialize it every time.
As for just about any resource (and a connection is no different), a user should call the functions to return resources to the system after use. Most users understand this across languages and architectures, so I don't see why your package users should have any concern. Go addresses this problem very elegantly by including the defer statement which makes it very easy to provide for releasing a resource.
I want to profile a simple webserver that I wrote in Go. It accepts requests, maps the request to avro object and sends it to Kafka in a go routine. The requirement is that it answers immediately and sends an object to the Kafka later. Locally it answers in under 1 ms on average. I have been trying to profile it by starting the script with davecheney/profile package and sending test requests with jmeter. I can see in the output that the profile file is generated but it remains empty, no matter how long jemeter is sending the requests. I'm running it on Mac El Capitan. I read that there were some issues with profiling on Mac but it would be working on El Capitan. Do you have any tips?
Firstly, I'm not sure whether you're trying to do latency profiling. If so, be aware that Go's CPU profiler only reports time spent by a function executing on the CPU and doesn't include time spent sleeping, etc. If CPU Profiling really is what you're looking for, read on.
If you're running a webserver, just include the following in your imports (in the file where main() is) and rebuild:
import _ "net/http/pprof"
Then, while applying load through jmeter, run:
go tool pprof /path/to/webserver/binary http://<server>/debug/pprof/profile
The net/http/pprof package provides profiling hooks that allow you to profile your webserver on demand any time, even while it's running in production. You may want to use a different, firewalled port for it, though, if your webserver is meant to be exposed publicly.
I have a couple of go programs that have a unused inport of net/http/pprof in them.
import _ "net/http/pprof"
...
//http.ListenAndServe("127.0.0.1:6060", nil)
I was wondering what the overhead of this import is in term of CPU and Mem. Aka. Should I remove then in prod (yes), but what would be the impact if I forgot?
Related: what are the exact sideeffects of this import? It registers some http handlers, but does it also inject things in go's malloc functions?
The overhead of importing the net/http/pprof package is pretty limited: it just installs some handlers for the http server. See the source code at:
http://golang.org/src/pkg/net/http/pprof/pprof.go
CPU profiling is not activated at initialization time, it is only activated for a period of time (30 seconds by default) when the /debug/pprof/profile REST service is called. So compiling with this package should not impact much the performance of the application (except that extra goroutines for the http server are needed).
Now, during the execution of /debug/pprof/profile, CPU sampling is activated, so a performance overhead is expected. I guess it can be used for production code provided the access to the corresponding port is restricted to the administrators of the application.
I've created a simple Go application on a Mac for writing and reading data to and from a TCP connection. I've used the GAE Go version. Later, I ported that program to Windows, and I got this error :
Connection.SetReadTimeout undefined (type *net.TCPConn has no field or method SetReadTimeout)
I guess the net package information on the Golang website describes the package only for the GAE version. How would I properly set the timeout in a non-GAE Go version?
With latest weekly (aka Go 1 RC2) one has to use the various Set*Deadline methods of the net.Conn type. Note that the old timeouts were relative to some event, deadlines are absolute times. The background for this change is roughly: setting a [relative] timeout of 1 s seems like a good idea in some scenario, but it applied to every event, like receiving a single byte, thus allowing crafted transfers to avoid timeouts forever (with the respective DOS nearby).