I know its possible to capture code coverage metrics when running unit tests. However, we would like to know what the coverage is when we run integrations tests (plural) against the binary itself, like:
go build
./mybin somefile1
./mybin somefile2
# ... test a bunch more files and input flags
Is it possible to do this? The binary can be built just for the purpose of testing, so any compile options as needed.
The Go coverage tool only works in conjunction with the testing package. But not all hope is lost.
If you can coerce your integration tests into the go testing framework, you should have all you need. This shouldn't be as hard as it sounds.
Basically:
Write a test file that executes your main() function in a go routine:
func TestMainApp(t *testing.T) {
go main()
// .. then start your integration tests
}
With your real app running from within a test, start your integration tests--likely with the help of exec.Cmd.
Gather your coverage stats.
Profit.
This article, Go coverage with external tests, from a year ago, outlines a similar approach.
Related
I've been trying to generate code coverage for a vanilla Typescript (no react or any web frameworks) project on Webpack 5 using Cypress for unit testing. I'm not able to generate code coverage for it as if I'm failing to tell Cypress that it needs to use the source instrumented by nyc.
In order to come up with ideas on solving this, I need to know: how exactly nyc passes the instrumented source to test runners?
In this documentation, it mentioned using nyc to instrument the code but it doesn't really show how the instrumented code is going to be picked up by Cypress. Following the provided command line does not allow your cypress test to generate code.
I have tried instrumenting by nyc but it did not work. What worked was transpiling the typescript code and then instrumenting. Once it is instrumented you can deploy by the same way you currently do and then run your tests against it.
I know its possible to capture code coverage metrics when running unit tests. However, we would like to know what the coverage is when we run integrations tests (plural) against the binary itself, like:
go build
./mybin somefile1
./mybin somefile2
# ... test a bunch more files and input flags
Is it possible to do this? The binary can be built just for the purpose of testing, so any compile options as needed.
The Go coverage tool only works in conjunction with the testing package. But not all hope is lost.
If you can coerce your integration tests into the go testing framework, you should have all you need. This shouldn't be as hard as it sounds.
Basically:
Write a test file that executes your main() function in a go routine:
func TestMainApp(t *testing.T) {
go main()
// .. then start your integration tests
}
With your real app running from within a test, start your integration tests--likely with the help of exec.Cmd.
Gather your coverage stats.
Profit.
This article, Go coverage with external tests, from a year ago, outlines a similar approach.
I am building a Go provider for Pruner (a CLI that runs only the tests that ran through the lines you changed, saving you time).
For that, I need to be able to see per-test coverage. Not just a full coverage report after all tests have run, but I need a way to know which tests ran through what line.
Is that possible in Go?
I tried using -func, but it just gives me the method names of the original code, not the test code. In other words, I can't know what code each individual test runs through.
I need a way to know which tests ran through what line.
Is that possible in Go?
It's not supported by the tools. But you can do it. It's just very inefficient.
The way to do this is to run:
go test -cover -run=TheName/OfSome/SpecificTest
Then run this for each test in your suite.
Naturally, this will make your tests much more cumersome to manage, and incredibly slow.
So I would consider whether this is truly a requrement for your use case.
Go is optimized, from the ground-up, to compile quickly. If you have a Go project so long, that running all the tests is too slow, you may want to consider other alternatives. Some suggestions:
Run more tests in parallel, so the total runtime is reduced.
Take advantage of Short mode, and only run short tests by default, saving long-running tests for special cases.
If you really need to run only a subset of tests, do it on a per-package basis, not on a per-test basis.
It is possible to obtain coverage information when running a test in Go by using go test -cover ..., but this coverage doesn't extend to main programs which are invoked by the test. Is there some way of building a Go executable so that it is instrumented for coverage?
I've got some tests for my go project which are written in ruby/cucumber and I'm therefore unable to take advantage of the "go test"-based coverage tools to get coverage numbers for those tests (I also have unit tests and I'm happily generating coverage numbers for those).
Is there a way to create an instrumented build of go code such that I can generate coverage numbers for those non-go tests?
No. You would have to write a ruby/cucumber interpreter in GO, write/adapt a coverage testing framework and then integrate that with the go test framework.
Just rewrite the legacy ruby code in Go :P
NO, but you can run go test in a way that creates coverage html pages.
like this:
go test -v -coverprofile cover.out ./...
go tool cover -html=cover.out -o cover.html
open cover.html