I have several tests using testify/suite package and I execute them in parallel as follows
type IntegrationSuite struct {
suite.Suite
}
func TestIntegrationSuite(t *testing.T) {
suite.Run(t, &IntegrationSuite{})
}
func (is *IntegrationSuite) TestSomething() {
is.T().Log("\tIntegration Testing something")
for i := range myTestTable {
i := i
is.T().Run("Testing "+myTestTable[i].scenarioName, func(_ *testing.T) {
is.T().Parallel()
...
func (is *IntegrationSuite) TestSomethingElse() {
is.T().Log("\tIntegration Testing something else")
for i := range myOtherTestTable {
i := i
is.T().Run("Testing "+myOtherTestTable[i].scenarioName, func(_ *testing.T) {
is.T().Parallel()
...
})
However this panics with
panic: testing: t.Parallel called multiple times [recovered]
panic: testing: t.Parallel called multiple times
How can one leverage parallelism with the specific package?
You're calling t.Parallel() on the wrong instance of testing.T.
You're spinning up multiple subtests using:
is.T().Run("Testing "+myOtherTestTable[i].scenarioName, func(_ *testing.T) {...}
But instead of marking the subtest as parallel, you're marking the outer parent test as parallel for every subtest you start:
is.T().Parallel() // marks outer test as parallel
Instead, bind the subtest func's testing.T param to a variable and call Parallel on the subtest:
is.T().Run("Testing "+myOtherTestTable[i].scenarioName, func(t *testing.T) {
t.Parallel() // this is the subtest's T instance
// test code
})
This marks all subtests as parallel within the parent test. If you also want to mark the parent test as parallel with other tests in the package, call is.T().Parallel() once at the beginning of TestSomethingElse (not within the subtest)
For more details on parallelism of subtests, see: Control of Parallelism
Please note that the testify suite is currently not "thread safe" and therefore does not support concurrency. There are cases in which tests pass successfully even though they should fail.
Please find the current status as well the reported issues here: https://github.com/stretchr/testify/issues/187
Related
I am running a parallel test using go test. It's a scenario-based test and test runs based on provided JSON data.
Sample JSON data looks like below
[
{
ID: 1,
dependency: []
...
},
{
ID: 2
dependency: [1]
...
},
{
ID: 3
dependency: [2]
...
},
{
ID: 4
dependency: [1,2]
...
}
]
Here ID is identifier of a testcase and dependency is referring to the testcases which should run before a testcase.
Circular dependency checker is implemented and it's making sure that no circular dependency is on JSON.
Test code look like below.
t.Run(file, func(t *testing.T) {
t.Parallel()
var tcs []TestCase
bytes := ReadFile(file, t)
json.Unmarshal(bytes, &tcs)
CheckCircularDependencies(tcs, t) // check circular dependency graph
for idx, tc := range tcs {
queues = append(queues, queitem{
idx: idx
stepID: step.ID,
status: NotStarted,
....
})
}
for idx, tc := range tcs {
UpdateWorkQueStatus(tc, InProgress, t)
ProcessQueItem(tc, t)
}
})
func ProcessQueItem(tc TestCase, t *testing.T) {
WaitForDependencies(tc, t) // this waits for all the dependency test cases to be finished
...
// DO TEST code which takes about 2-3s
// Provide a command to microservices
// Wait for 1-2s for the command to be processed
// If not processed within 1-2s then wait again for 1-2s
// Once command is processed, just check the result and finish test case
UpdateWorkQueStatus(tc, Done, t)
})
}
func WaitForDependencies(tc TestCase, t *testing.T) {
if !GoodToGoForTestCase(tc, t) {
SleepForAWhile()
WaitForDependencies(tc, t)
}
}
func SleepForAWhile() {
time.Sleep(100 * time.Millisecond)
}
Parallel test Behavior:
This is working as expected on my personal computer which has multiple processors.
But when it is working on Google Cloud (maybe 1 cpu), it's stuck and does not go ahead at all for 10mins.
Current solution:
Running tests without parallelism and doing by the order of IDs. (just removed t.Parallel() function call to avoid parallelism)
Problem:
This serial solution is taking more than 3 times than parallel solution, since most of testing time is based on sleeping time.
The test code itself also contains sleep function call.
My question
1. Is this architecture correct and can work correctly on 1 processor CPU?
2. If this architecture can work, what will be the problem of stuck?
Is it possible to call a test Function from a non test go file to start the execution of tests?
e.g., I have a test function:
package API
import "testing"
func TestAPI(t *testing.T) {
...
}
I need to call this from a non test go file.
package main
import "../API"
API.TestAPI()
Can I do that?
There are some great testing patterns in Go. Unfortunately, the way that you're trying to test is not going to work.
Your unit tests should be run using the command go test <pkg. Tests should never be called from within your code itself.
Generally there are two main forms of unit testing in Go - black and white box unit testing.
White-box testing: Testing unexported functions from within the
package itself.
Black-box testing: Testing exported functions from outside the
package, emulating how other packages will interact with it.
If I have a package, example, that I'd like to test. There is a simple exported function that sums a list of numbers that are provided. There is also an unexported utility function that's used by the Sum function.
example.go
package example
func Sum(nums ...int) int {
sum := 0
for _, num := range nums {
sum = add(sum, num)
}
return sum
}
func add(a, b int) int {
return a + b
}
example_test.go : black-box testing
Notice that I'm testing the example package, but the test sits in the example_test package. Adding the _test keeps the Go compiler happy and lets it know that this is a testing package for example. Here, we may only access exported variables and functions which lets us test the behaviour that external packages will experience when importing and using our example package.
package example_test
import (
"testing"
"example"
)
func TestSum(t *testing.T) {
tests := []struct {
nums []int
sum int
}{
{nums: []int{1, 2, 3}, sum: 6},
{nums: []int{2, 3, 4}, sum: 9},
}
for _, test := range tests {
s := example.Sum(test.nums...)
if s != test.sum {
t.FailNow()
}
}
}
example_internal_test.go : white-box testing
Notice that I'm testing the example package, and the test also sits in the example package. This allows the unit tests to access unexported variables and functions.
package example
import "testing"
func TestAdd(t *testing.T) {
tests := []struct {
a int
b int
sum int
}{
{a: 1, b: 2, sum: 3},
{a: 3, b: 4, sum: 7},
}
for _, test := range tests {
s := add(test.a, test.b)
if s != test.sum {
t.FailNow()
}
}
}
I hope that this you a better understanding on how Go's testing framework is set up and should be used. I haven't used any external libraries in the examples for simplicity sake, although a very popular and powerful package to help with unit testing is github.com/stretchr/testify.
What you are attempting is possible, but not advised, as it depends on internal details of Go's testing library, which are not covered by the Go 1 Compatibly Guarantee, meaning you'll likely have to tweak this approach for every version of Go.
You should rarely, if ever, attempt this. I did this once, for a suite of API integration tests that I wanted to be able to run as normal tests, but also from a CLI tool, to test third-party implementations of the API. Now that I've done it, I'm not sure I'd do it again. I'd probably look for a different approach.
But if you can't be dissuaded, here's what you need to do:
Your tests must not be defined in a file named *_test.go. Such files are excluded from normal compilation.
You must trigger the tests to run from a file appropriately named *_test.go, for your normal test case.
You must manually orchestrate the tests in your non-test case.
In detail:
Put tests in non-test files. The testing package is just a standard package, so it's easy to include wherever you need it:
foo.go
package foo
import "testing"
func SomethingTest(t *testing.T) {
// your tests
}
Run the tests from a *_test.go file:
foo_test.go
func TestSomething(t *testing.T) {
SomethingTest(t)
}
And run the tests from your non-test files. Here you are in risky territory, as it requires using methods that are not protected by the Go 1 compatibility guarantee. You need to call the testing.MainStart method, with a manually-crafted list of tests.
This also requires implementing your own version of testing.testDeps, which is private, but an interface, so easy to implement. You can see my Go 1.9 implementation here.
tests := []testing.InternalTest{
{
Name: "TestSomething",
f: SomethingTest,
},
}
m := testing.MainStart(MyTestDeps, tests, nil, nil)
os.Exit(m.Run())
This is a very simplified example, not meant to be complete. But I hope it puts you on the right path.
I am a new in programming. I have two examples code in Go and its about loop using range. This is the first example:
Program A
type Test struct {
Text string
}
func main() {
tests := []Test{
Test{"Test1"},
Test{"Test2"},
}
var a Test
for _, test := range tests {
a = test
fmt.Println(a)
}
}
This is the second example:
Program B
type Test struct {
Text string
}
func main() {
tests := []Test{
Test{"Test1"},
Test{"Test2"},
}
for _, test := range tests {
a := test
fmt.Println(a)
}
}
In the first example 'a' is declared outside the loop, but in the second example 'a' is declared inside the loop. Like in the other programming language, what is the difference between two example program? Is there any optimization difference? Thank you.
The variables have different scopes. It is usually best practice to use the smallest scope possible as in the second example.
There should be no optimization difference.
I am building unit testing of the application that uses os.Exit(1), once os.Exit(1) executes remaining test file are skipped with go test,
I am thinking about suppressing os.Exit during the unit test execution only. I wonder how I can determine that app has been bootstrap from go test runner?
You will find various ways to unit test method with os.Exit() in "Testing os.Exit scenarios in Go with coverage information (coveralls.io/Goveralls)".
It uses a function which is:
os.Exit() when you are not testing
var osExit = os.Exit
and "yourOsExit" when you are testing.
func TestCrasher(t *testing.T) {
// Save current function and restore at the end:
oldOsExit := osExit
defer func() { osExit = oldOsExit }()
osExit = myExit
I am trying to repeat example from golang testing
package main
import (
"testing"
)
func TestSomeTest(t *testing.T) {}
func TestMain(m *testing.M) { // cleaning after each test}
I want TestMain function to run after every test.
Running command go test
And the compiler says
./testingM_test.go:9: undefined: testing.M
So how to clean after executing every test?
Check you go version output: this is for go 1.4+ only.
The testing package has a new facility to provide more control over running a set of tests. If the test code contains a function
func TestMain(m *testing.M)
that function will be called instead of running the tests directly.
The M struct contains methods to access and run the tests.
You can see that feature used here:
The introduction of TestMain() made it possible to run these migrations only once. The code now looks like this:
func TestSomeFeature(t *testing.T) {
defer models.TestDBManager.Reset()
// Do the tests
}
func TestMain(m *testing.M) {
models.TestDBManager.Enter()
// os.Exit() does not respect defer statements
ret := m.Run()
models.TestDBManager.Exit()
os.Exit(ret)
}
While each test must still clean up after itself, that only involves restoring the initial data, which is way faster than doing the schema migrations.