I want to profile a Go program's performance between different runs with different OS-level settings. I'm aware that I can get profiles for single runs via $ go test -cpuprofile cpu.prof -memprofile mem.prof -bench .. However I don't know how to aggregate the information in such a way that I can compare the results either visually or programmatically.
To present a sketch in Xonsh scripting language, which is a creole between Python and Bash. However I'm happy to accept suggestion written in pure Bash as well.
for i in range(n):
change_system_settings()
# Run 'go test' and save the results in cpu0.prof, cpu1.prof, cpu2.prof etc.
#(f'go test -cpuprofile cpu{i}.prof -memprofile mem{i}.prof -bench .'.split())
The script changes the system settings and runs the program through profiler n times. Now, after the process I'm left with possibly dozens of individual .prof files. I would like to have a holistic view of them, compare the memory and CPU usage between runs and even run numeric tests to see which run was optimal.
If you use GoLang's pprof to profile your Go program, the library has a Merge method that merges multiple pprof output files into one.
The library is github.com/google/pprof, so you just import it in a Go script:
import ('github.com/google/pprof/profile')
Then you'll need to load all your pprof files into one array. If we consider that you did that and you have all your files loaded (using os.Open()) in an array called allFiles, you merge them using the following method:
result, err := profile.Merge(allFiles)
Then you output the merged data into a new file, using os.OpenFile(...), writing to this file, then closing it.
I haven't tested this right now honestly, but I remember this is how we did it a long time ago. So technically, you could invoke this golang script after your for loop is done in your test script.
Documentation: https://github.com/google/pprof/blob/master/doc/README.md
Related
I'd like some advice on what vocabulary to use to describe the following. Having the right vocabulary will allow me to search for tools and ideas related to the concept
I'd like to say a script is SomeWord if it is expected to produce the same output no matter where it is run.
For example, the following script is not SomeWord:
#!/bin/bash
ls ~
because of course it depends on where it is executed.
Whereas the following (if it runs without error) is expected to always produce the same output:
#!/bin/bash
echo "hello, world"
A more useful example would be something that loads and runs a docker or singularity container in a way that guarantees that a very particular container image is being used. For example by retrieving the singularity image by its content-hash.
The advantages of SomeWord scripts are: (a) they may be safely run on a remote system without worrying about the environment and (b) their outputs may be cached.
The best I can think of would be "deterministic" or some variation on "environment independent" or "reproducible".
Any container should be able to do this, as that is a big part of why the tech developed in the first place. Environment managers like conda can also do this to a certain extent, but because it's just modifying the host environment it's possible to be using non-conda binaries without realizing it.
I am looking for a possibility to generate a call graph for Go projects. Something similar to Doxygen's diagram functionality for C++ classes (with the option CALL_GRAPH=YES).
So far I found
http://saml.rilspace.org/profiling-and-creating-call-graphs-for-go-programs-with-go-tool-pprof
or
http://blog.golang.org/profiling-go-programs
This samples the call stack of your program 100 times per second while the program is running and creates a graph useful for profiling. If your program spends most of its time in functions not relevant to you, I found this solution not very usefull.
Then there is this:
https://godoc.org/golang.org/x/tools/go/callgraph/static
which from its description sounds like what I would need, but there seem to be no docs and I don't understand how to use it.
I also found
https://github.com/davecheney/graphpkg/blob/master/README.md
and
https://github.com/paetzke/go-dep-graph/blob/master/README.org
but they create only dependency graphs.
Take a look here:
http://dave.cheney.net/2014/10/22/simple-profiling-package-moved-updated
https://github.com/pkg/profile
func main() {
defer profile.Start(profile.CPUProfile, profile.ProfilePath(".")).Stop()
// Rest of program
}
Build and run your program as per normal. You'll see the profiling hook
mentioned:
2015/07/12 09:02:02 profile: cpu profiling enabled, cpu.pprof
Run your program (bench it, run through it, etc) to generate the profile during
runtime. Once you've hit what you want, quit and then generate the call-graph:
go tool pprof -pdf -output cgraph.pdf $YOURPROGBINARY cpu.pprof
You can also run go tool pprof $YOURPROGBINARY cpu.pprof to get an
interactive prompt where you can call top10 or web to generate an svg. Type
help at the pprof prompt to get a list of commands.
e.g. - here's the CPU profile for a buffer pool implementation I wrote:
~/Desktop go tool pprof poolio cpu.pprof
Entering interactive mode (type "help" for commands)
(pprof) top5
24770ms of 35160ms total (70.45%)
Dropped 217 nodes (cum <= 175.80ms)
Showing top 5 nodes out of 74 (cum >= 650ms)
flat flat% sum% cum cum%
12520ms 35.61% 35.61% 12520ms 35.61% runtime.mach_semaphore_wait
9300ms 26.45% 62.06% 9360ms 26.62% syscall.Syscall
1380ms 3.92% 65.98% 2120ms 6.03% encoding/json.(*encodeState).string
1030ms 2.93% 68.91% 1030ms 2.93% runtime.kevent
540ms 1.54% 70.45% 650ms 1.85% runtime.mallocgc
And here's a quick way to generate a PNG from the prompt:
(pprof) png > graph.png
Generating report in graph.png
Which outputs this:
You were close with …/x/tools/go/callgraph/static. I'm pretty sure go install golang.org/x/tools/cmd/callgraph is what you want. Once installed run it without arguments to see it's full help/usage.
(In general, the things under …/x/tools/ are somewhat reusable packages with command line front-ends living under …/x/tools/cmd, you can install them all with go install golang.org/x/tools/cmd/..., the literal /... matches all sub-packages).
E.g. running just callgraph produces usage output that starts with:
callgraph: display the the call graph of a Go program.
Usage:
callgraph [-algo=static|cha|rta|pta] [-test] [-format=...] <args>...
Flags:
-algo Specifies the call-graph construction algorithm, one of:
static static calls only (unsound)
cha Class Hierarchy Analysis
rta Rapid Type Analysis
pta inclusion-based Points-To Analysis
The algorithms are ordered by increasing precision in their
treatment of dynamic calls (and thus also computational cost).
RTA and PTA require a whole program (main or test), and
include only functions reachable from main.
-test Include the package's tests in the analysis.
-format Specifies the format in which each call graph edge is displayed.
One of:
digraph output suitable for input to
golang.org/x/tools/cmd/digraph.
graphviz output in AT&T GraphViz (.dot) format.
It can produce arbitrary formatted output (using Go's template syntax) or graphviz or digraph output. The last is a tool you can install with go install golang.org/x/tools/cmd/digraph (and once again, full/help usage is seen by running it without arguments) and can answer queries about arbitrary directed graphs (including call graphs obviously).
Another approach, which does use golang.org/x/tools go/callgraph is the ofabry/go-callvis project:
(Go 1.13+)
The purpose of this tool is to provide developers with a visual overview of a Go program using data from call graph and its relations with packages and types.
support for Go modules!
focus specific package in the program
click on package to quickly switch the focus using interactive viewer
group functions by package and/or methods by type
filter packages to specific import path prefixes
ignore funcs from standard library
omit various types of function calls
Example (based on this Go code project):
How it works
It runs pointer analysis to construct the call graph of the program and uses the data to generate output in dot format, which can be rendered with Graphviz tools.
To use the interactive view provided by a web server that serves SVG images of focused packages, you can simply run:
go-callvis <target package>
HTTP server is listening on http://localhost:7878/ by default, use option -http="ADDR:PORT" to change HTTP server address.
I used golang callgraph recently, and I build a web tool with python + callgraph called CallingViewer here: https://github.com/fiefdx/CallingViewer , it maybe rough, but it work, the screenshot at below:
screenshot of CallingViewer
If I have different packages and each have a test file (pkg_test.go) is there a way for me to make sure that they run in a particular order ?
Say pkg1_test.go gets executed first and then the rest.
I tried using go channels but it seems to hang.
It isn't obvious, considering a go test ./... triggers test on all packages... but runs in parallel: see "Go: how to run tests for multiple packages?".
go test -p 1 would run the tests sequentially, but not necessarily in the order you would need.
A simple script calling go test on the packages listed in the right expected order would be easier to do.
Update 6 years later: the best practice is to not rely on test order.
So much so issue 28592 advocates for adding -shuffle and -shuffleseed to shuffle tests.
CL 310033 mentions:
This CL adds a new flag to the testing package and the go test command
which randomizes the execution order for tests and benchmarks.
This can be useful for identifying unwanted dependencies
between test or benchmark functions.
The flag is off by default.
If -shuffle is set to on then the system
clock will be used as the seed value.
If -shuffle is set to an integer N, then N will be used as the seed value.
In both cases, the seed will be reported for failed runs so that they can reproduced later on.
Picked up for Go 1.17 (Aug. 2021) in commit cbb3f09.
See more at "Benchmarking with Go".
I found a hack to get around this.
I named my test files as follow:
A_{test_file1}_test.go
B_{test_file2}_test.go
C_{test_file3}_test.go
The A B C will ensure they are run in order.
I am not asking to make golang do some sort of "eval" in the current context, just need it to take a input string (say, received from network) and execute it as a separate golang program.
In theory, when you run go run testme.go, it will read the content of testme.go into a string and parse, compile and execute it. Wonder if it is possible to call a go function to execute a string directly. Note that I have a requirement not to write the string into a file.
UPDATE1
I really want to find out if there is a function (in some go package) that serves as an entry point of go, in another word, I can call this function with a string as parameter, it will behave like go run testme.go, where testme.go has the content of that string.
AFAIK it cannot be done, and the go compiler has to write intermediate files, so even if you use go run and not go build, files are created for the sake of running the code, they are just cleaned up if necessary. So you can't run a go program without touching the disk, even if you manage to somehow make the compiler take the source not from a file.
For example, running strace on calling go run on a simple hello world program, shows among other things, the following lines:
mkdir("/tmp/go-build167589894", 0700)
// ....
mkdir("/tmp/go-build167589894/command-line-arguments/_obj/exe/", 0777)
// ... and at the end of things
unlink("/tmp/go-build167589894/command-line-arguments/_obj/exe/foo")
// ^^^ my program was called foo.go
// ....
// and eventually:
rmdir("/tmp/go-build167589894")
So you can see that go run does a lot of disk writing behind the scenes, just cleans up afterwards.
I suppose you can mount some tmpfs and build in it if you wish, but otherwise I don't believe it's possible.
I know that this question is (5 years) old but I wanted to say that actually, it is possible now, for anyone looking for an up-to-date answer.
The Golang compiler is itself written in Go so can theoretically be embedded in a Go program. This would be quite complicated though.
As a better alternative, there are projects like yaegi which are effectively a Go interpreter which can be embedded into Go programs.
I have close to a million files over which I want to run a shell script and append the result to a single file.
For example suppose I just want to run wc on the files.
So that it runs fast I can parallelize it with xargs. But I do not want the scripts to step over each other when writing the output. It is probably better to write to a few separate files rather than one and then cat them later. But I still want the number of such temporary output files to be significantly smaller than the number of input files. Is there a way to get the kind of locking I want, or is it the case that is always ensured by default?
Is there any utility that will recursively cat two files in parallel?
I can write a script to do that, but have to deal with the temporaries and clean up. So was wondering if there is an utility which does that.
GNU parallel claims that it:
makes sure output from the commands is
the same output as you would get had
you run the commands sequentially
If that's the case, then I presume it should be safe to simple pipe the output to your file and let parallel handle the intermediate data.
Use the -k option to maintain the order of the output.
Update: (non-Perl solution)
Another alternative would be prll, which is implemented with shell functions with some C extensions. It is less feature-rich compared to GNU parallel but should the the job for basic use cases.
The feature listing claims:
Does internal buffering and locking to
prevent mangling/interleaving of
output from separate jobs.
so it should meet your needs as long as order of output is not important
However, note on the following statement on this page:
prll generates a lot of status
information on STDERR which makes it
harder to use the STDERR output of the
job directly as input for another
program.
Disclaimer: I've tried neither of the tools and am merely quoting from their respective docs.