Can someone suggest a way to me if we can get branch coverage for my golang tests? Let's say I have a golang code which looks like below:
package main
import (
"fmt"
)
func HelloWorld(name string, printv int) string {
if name == "tuk" || printv == 1 {
fmt.Println("Hello tuk")
return "Hello tuk"
} else {
fmt.Println("Who are you?")
return "Who are you?"
}
}
The corresponding test file looks like below:
package main
import "testing"
func TestHelloWorld(t *testing.T) {
r := HelloWorld("tuk", 0)
if r != "Hello tuk" {
t.Error("Test Failed")
}
}
If I execute the below test command, it is just giving me the statement coverage:
go test -coverprofile=cover.out .
ok test 0.007s coverage: 60.0% of statements
Is there a way I can get the branch coverage as well?
As of Go 1.11 (November 2018), the generated machine code for the coverage only covers each block of statements. It should be possible to take that code and adjust it to cover every branch of an expression.
It looks as if the gobco project has done exactly this, although only for single files and not for whole packages (again, as of November 2018).
I filed an issue to see whether branch coverage is considered to be worthwhile by the Go team. The decision is to keep the current code as simple as it is now, and that decision had already been documented in the code itself.
In the end I forked the gobco project and added all the features I need.
Related
I have a method that I am testing, and everything seems fine. However, when I run the tests in GoLand, I can see in the output that the tests "PASS" but the test runner says "no tests were run".
Here's the sample method in calculator.go
package calculator
import (
"fmt"
)
type Calculator struct {}
func New() Calculator {
return Calculator{}
}
func (s *Calculator) AddTwoNumbers(num_one, num_two int) int {
fmt.Printf("adding")
return num_one + num_two
}
Here's the test in calculator_test.go:
package calculator
import (
"fmt"
"testing"
)
func Test_Calculator_AddTwoNumbers(t *testing.T) {
// Arrange
calculator := New()
// Act
total := calculator.AddTwoNumbers(1,2)
// Assert
if total != 3 {
msg := fmt.Sprintf("total should have been %d but instead was %d", 3, total)
t.Error(msg)
}
}
What am I doing wrong?
Instead of fmt.Printf(... in AddTwoNumbers try either fmt.Println(... or fmt.Printf("foo\n')
The absence of the newline in the output of your AddTwoNumbers method is is causing the format of the test execution outputs to not have each test in a new line. The test runner is not being able to interpret that a test was run. Adding that newline, keeps a clean output.
I would suggest using to check the verbose output of the test run.
go test -v
-v
Verbose output: log all tests as they are run. Also print all
text from Log and Logf calls even if the test succeeds.
Is it possible to call a test Function from a non test go file to start the execution of tests?
e.g., I have a test function:
package API
import "testing"
func TestAPI(t *testing.T) {
...
}
I need to call this from a non test go file.
package main
import "../API"
API.TestAPI()
Can I do that?
There are some great testing patterns in Go. Unfortunately, the way that you're trying to test is not going to work.
Your unit tests should be run using the command go test <pkg. Tests should never be called from within your code itself.
Generally there are two main forms of unit testing in Go - black and white box unit testing.
White-box testing: Testing unexported functions from within the
package itself.
Black-box testing: Testing exported functions from outside the
package, emulating how other packages will interact with it.
If I have a package, example, that I'd like to test. There is a simple exported function that sums a list of numbers that are provided. There is also an unexported utility function that's used by the Sum function.
example.go
package example
func Sum(nums ...int) int {
sum := 0
for _, num := range nums {
sum = add(sum, num)
}
return sum
}
func add(a, b int) int {
return a + b
}
example_test.go : black-box testing
Notice that I'm testing the example package, but the test sits in the example_test package. Adding the _test keeps the Go compiler happy and lets it know that this is a testing package for example. Here, we may only access exported variables and functions which lets us test the behaviour that external packages will experience when importing and using our example package.
package example_test
import (
"testing"
"example"
)
func TestSum(t *testing.T) {
tests := []struct {
nums []int
sum int
}{
{nums: []int{1, 2, 3}, sum: 6},
{nums: []int{2, 3, 4}, sum: 9},
}
for _, test := range tests {
s := example.Sum(test.nums...)
if s != test.sum {
t.FailNow()
}
}
}
example_internal_test.go : white-box testing
Notice that I'm testing the example package, and the test also sits in the example package. This allows the unit tests to access unexported variables and functions.
package example
import "testing"
func TestAdd(t *testing.T) {
tests := []struct {
a int
b int
sum int
}{
{a: 1, b: 2, sum: 3},
{a: 3, b: 4, sum: 7},
}
for _, test := range tests {
s := add(test.a, test.b)
if s != test.sum {
t.FailNow()
}
}
}
I hope that this you a better understanding on how Go's testing framework is set up and should be used. I haven't used any external libraries in the examples for simplicity sake, although a very popular and powerful package to help with unit testing is github.com/stretchr/testify.
What you are attempting is possible, but not advised, as it depends on internal details of Go's testing library, which are not covered by the Go 1 Compatibly Guarantee, meaning you'll likely have to tweak this approach for every version of Go.
You should rarely, if ever, attempt this. I did this once, for a suite of API integration tests that I wanted to be able to run as normal tests, but also from a CLI tool, to test third-party implementations of the API. Now that I've done it, I'm not sure I'd do it again. I'd probably look for a different approach.
But if you can't be dissuaded, here's what you need to do:
Your tests must not be defined in a file named *_test.go. Such files are excluded from normal compilation.
You must trigger the tests to run from a file appropriately named *_test.go, for your normal test case.
You must manually orchestrate the tests in your non-test case.
In detail:
Put tests in non-test files. The testing package is just a standard package, so it's easy to include wherever you need it:
foo.go
package foo
import "testing"
func SomethingTest(t *testing.T) {
// your tests
}
Run the tests from a *_test.go file:
foo_test.go
func TestSomething(t *testing.T) {
SomethingTest(t)
}
And run the tests from your non-test files. Here you are in risky territory, as it requires using methods that are not protected by the Go 1 compatibility guarantee. You need to call the testing.MainStart method, with a manually-crafted list of tests.
This also requires implementing your own version of testing.testDeps, which is private, but an interface, so easy to implement. You can see my Go 1.9 implementation here.
tests := []testing.InternalTest{
{
Name: "TestSomething",
f: SomethingTest,
},
}
m := testing.MainStart(MyTestDeps, tests, nil, nil)
os.Exit(m.Run())
This is a very simplified example, not meant to be complete. But I hope it puts you on the right path.
I have a production golang code and functional tests for it written not in golang. Functional tests run compiled binary. Very simplified version of my production code is here: main.go:
package main
import (
"fmt"
"math/rand"
"os"
"time"
)
func main() {
rand.Seed(time.Now().UTC().UnixNano())
for {
i := rand.Int()
fmt.Println(i)
if i%3 == 0 {
os.Exit(0)
}
if i%2 == 0 {
os.Exit(1)
}
time.Sleep(time.Second)
}
}
I want to build coverage profile for my functional tests. In order to do it I add main_test.go file with content:
package main
import (
"os"
"testing"
)
var exitCode int
func Test_main(t *testing.T) {
go main()
exitCode = <-exitCh
}
func TestMain(m *testing.M) {
m.Run()
// can exit because cover profile is already written
os.Exit(exitCode)
}
And modify main.go:
package main
import (
"flag"
"fmt"
"math/rand"
"os"
"runtime"
"time"
)
var exitCh chan int = make(chan int)
func main() {
rand.Seed(time.Now().UTC().UnixNano())
for {
i := rand.Int()
fmt.Println(i)
if i%3 == 0 {
exit(0)
}
if i%2 == 0 {
fmt.Println("status 1")
exit(1)
}
time.Sleep(time.Second)
}
}
func exit(code int) {
if flag.Lookup("test.coverprofile") != nil {
exitCh <- code
runtime.Goexit()
} else {
os.Exit(code)
}
}
Then I build coverage binary:
go test -c -coverpkg=. -o myProgram
Then my functional tests run this coverage binary, like this:
./myProgram -test.coverprofile=/tmp/profile
6507374435908599516
PASS
coverage: 64.3% of statements in .
And I build HTML output showing coverage:
$ go tool cover -html /tmp/profile -o /tmp/profile.html
$ open /tmp/profile.html
Problem
Method exit will never show 100% coverage because of condition if flag.Lookup("test.coverprofile") != nil. So line os.Exit(code) is kinda blind spot for my coverage results, although, in fact, functional tests go on this line and this line should be shown as green.
On the other hand, if I remove condition if flag.Lookup("test.coverprofile") != nil, the line os.Exit(code) will terminate my binary without building coverage profile.
How to rewrite exit() and maybe main_test.go to show coverage without blind spots?
The first solution that comes into mind is time.Sleep():
func exit(code int) {
exitCh <- code
time.Sleep(time.Second) // wait some time to let coverprofile be written
os.Exit(code)
}
}
But it's not very good because will cause production code slow down before exit.
As per our conversation in the comments our coverage profile will never include that line of code because it will never be executed.
With out seeing your full code it is hard to come up with a proper solutions however there a few things you can do to increase the coverage with out sacrificing too much.
func Main and TestMain
Standard practice for GOLANG is to avoid testing the main application entry point so most professionals extract as much functionality into other classes so they can be easily tested.
GOLANG testing framework allows you to test your application with out the main function but in it place you can use the TestMain func which can be used to test where the code needs to be run on the main thread. Below is a small exert from GOLANG Testing.
It is sometimes necessary for a test program to do extra setup or teardown before or after testing. It is also sometimes necessary for a test to control which code runs on the main thread. To support these and other cases, if a test file contains a function: func TestMain(m *testing.M)
Check GOLANG Testing for more information.
Working example
Below is an example (with 93.3% coverage that we will make it 100%) that tests all the functionality of your code. I made a few changes to your design because it did not lend itself very well for testing but the functionality still the same.
package main
dofunc.go
import (
"fmt"
"math/rand"
"time"
)
var seed int64 = time.Now().UTC().UnixNano()
func doFunc() int {
rand.Seed(seed)
var code int
for {
i := rand.Int()
fmt.Println(i)
if i%3 == 0 {
code = 0
break
}
if i%2 == 0 {
fmt.Println("status 1")
code = 1
break
}
time.Sleep(time.Second)
}
return code
}
dofunc_test.go
package main
import (
"testing"
"flag"
"os"
)
var exitCode int
func TestMain(m *testing.M) {
flag.Parse()
code := m.Run()
os.Exit(code)
}
func TestDoFuncErrorCodeZero(t *testing.T) {
seed = 2
if code:= doFunc(); code != 0 {
t.Fail()
}
}
func TestDoFuncErrorCodeOne(t *testing.T) {
seed = 3
if code:= doFunc(); code != 1 {
t.Fail()
}
}
main.go
package main
import "os"
func main() {
os.Exit(doFunc());
}
Running the tests
If we build our application with the cover profile.
$ go test -c -coverpkg=. -o example
And run it.
$ ./example -test.coverprofile=/tmp/profile
Running the tests
1543039099823358511
2444694468985893231
6640668014774057861
6019456696934794384
status 1
PASS
coverage: 93.3% of statements in .
So we see that we got 93% coverage we know that is because we don't have any test coverage for main to fix this we could write some tests for it (not a very good idea) since the code has os.Exit or we can refactor it so it is super simple with very little functionality we can exclude it from our tests.
To exclude the main.go file from the coverage reports we can use build tags by placing tag comment at the first line of the main.go file.
//+build !test
For more information about build flags check this link: http://dave.cheney.net/2013/10/12/how-to-use-conditional-compilation-with-the-go-build-tool
This will tell GOLANG that the file should be included in the build process where the tag build is present butNOT where the tag test is present.
See full code.
//+build !test
package main
import "os"
func main() {
os.Exit(doFunc());
}
We need need to build the the coverage application slightly different.
$ go test -c -coverpkg=. -o example -tags test
Running it would be the same.
$ ./example -test.coverprofile=/tmp/profile
We get the report below.
1543039099823358511
2444694468985893231
6640668014774057861
6019456696934794384
status 1
PASS
coverage: 100.0% of statements in .
We can now build the coverage html out.
$ go tool cover -html /tmp/profile -o /tmp/profile.html
In my pkglint project I have declared a package-visible variable:
var exit = os.Exit
In the code that sets up the test, I overwrite it with a test-specific function, and when tearing down a test, I reset it back to os.Exit.
This is a simple and pragmatic solution that works well for me, for at least a year of extensive testing. I get 100% branch coverage because there is no branch involved at all.
Having issue creating unit test of one of my source file ( commonutil.go )
package util
import "github.com/nu7hatch/gouuid"
// GenerateUUID Returns generated UUID sequence
func GenerateUniqueID(hostname string) (string, error) {
var result, err = uuid.NewV4()
return hostname + ":" + result.String(), err
}
For the above source, I created the test file "commonutil_test.go" ( in the same package )
package util
import "testing"
func TestCommonUtil(t *testing.T) {
t.Run("Test generate UUID", func(t *testing.T) {
var uuid, _ = GenerateUniqueID ("test")
//fmt.Printf("UUID isa %v \n", uuid)
if uuid == "" {
t.Errorf("UUID expected, but result is empty ")
}
})
However when trying executing "go test util/commonutil_test.go" it shows :
util\commonutil_test.go:8: undefined: GenerateUniqueID
FAIL command-line-arguments [build failed]
Changing to util.GenerateUniqueID in the test solve the problem, however when running coverage using Goconvey will cause build failure :
can't load package: import cycle not allowed
package rudygunawan.com/MyProject/HFLC-Go/util
imports rudygunawan.com/MyProject/HFLC-Go/util
Any idea to solve this issue? I am confused.
Go version is go1.7.1 windows/386
I've run into a similar problem, when I was trying to run a single test file.
I wanted that, as it was a kind of test driven development thing, where I wanted to run tests only for the code I was working on at the moment, and not all the x-minutes running tests.
The solution turned out to be not running tests from a file, but rather running a specific test by name (actually a regex). So in your case I guess it would be:
go test ./util -run TestCommonUtil
An alternative seems to be listing all the files needed to build your test code:
go test util/commonutil_test.go util/commonutil.go
Just realize it is a silly mistake. The package for the test should be "util_test". Putting the test in the separate package ( but still in the same folder) help solve import cycle issue, yet still allow to solve the undefined error.
The way I normally write Go unit tests is to have one (or more) ..._test.go files, in the same package as the code being tested, with one Test... function for each broad set of tests to be done.
package util
import "testing
func TestGenerateUniqueID(t *testing.T) {
var uuid1, uuid2 string
uuid1, err = GenerateUniqueID("test")
if err != nil {
t.Errorf("Expected no error, got %s", err) // Maybe Fatalf?
}
if uuid1 == "" {
t.Errorf("Expected non-empty string, got empty string (uuid1)")
}
uuid2, err = GenerateUniqueID("test")
if err != nil {
t.Errorf("Expected no error, got %s", err) // Maybe Fatalf?
}
if uuid2 == "" {
t.Errorf("Expected non-empty string, got empty string (uuid2)")
}
if uuid1 == uuid2 {
t.Errorf("Expected uuid1 and uuid2 to be different, both are %s", uuid1)
}
}
One of the reasons I tend towards whitebox testing (where I can do "blackbox testing" by carefully not accessing package internals) is that there's usually a whole slew of non-exported code that really should be tested as well. In this specific, small, example, there's no massive argument for one over the other, since all the functionality that can be tested is already exported.
I want in init function to execute one of those two lines
func init() {
log.SetPrefix(">>>>>>>>>>>>> ")
log.SetOutput(ioutil.Discard)
}
How to achieve something similiar to C and macro definition in make file ?
I build go program like go build my_project and I don't have any custom make file. Can I pass to build command flag and read inside code ?
Create a string variable that you can test. Then set that string variable using -X passed to the linker. For example:
package main
var MODE string
func main() {
if MODE == "PROD" {
println("I'm in prod mode")
} else {
println("I'm NOT in prod mode")
}
}
Now, if you build this with just go build, MODE will be "". But you can also build it this way:
go build -ldflags "-X main.MODE PROD"
And now MODE will be "PROD". You can use that to modify your logic based on build settings. I generally use this technique to set version numbers in the binary; but it can be used for all kinds of build-time configuration. Note that it can only set strings. So you couldn't make MODE a bool or integer.
You can also achieve this with tags, which are slightly more complicated to set up, but much more powerful. Create three files:
main.go
package main
func main() {
doThing()
}
main_prod.go
// +build prod
package main
func doThing() {
println("I'm in prod mode")
}
main_devel.go
// +build !prod
package main
func doThing() {
println("I'm NOT in prod mode")
}
Now when you build with go build, you'll get your main_devel.go version (the filename doesn't matter; just the // +build !prod line at the top). But you can get a production build by building with go build -tags prod.
See the section on Build Constraints in the build documentation.