Run tests programmatically in Go - go

I have some tests that I'd like to run programmatically in Go. I'm trying to use testing.RunTests but it's raising a runtime error. I can't figure out what's wrong with the code either.
This is what it looks like:
package main
import (
"testing"
)
func TestSomething(t *testing.T) {
if false {
t.Error("This is a mocked failed test")
}
}
func main() {
testing.RunTests(func(pat, str string) (bool, error) { return true, nil },
[]testing.InternalTest{
{"Something", TestSomething}},
)
}
Playground link: https://play.golang.org/p/BC5MG8WXYGD
The error I'm getting is:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4b5948]

First of all, running tests should be done via the go test command.
Certain types and functions exported in the testing package are for the testing framework, not for you. Quoting from testing.RunTests():
RunTests is an internal function but exported because it is cross-package; it is part of the implementation of the "go test" command.
It "had" to be exported because it predates "internal" packages.
There. You've been warned.
If you still want to do it, call testing.Main() instead of testing.RunTests().
For example:
func TestGood(t *testing.T) {
}
func TestBad(t *testing.T) {
t.Error("This is a mocked failed test")
}
func main() {
testing.Main(
nil,
[]testing.InternalTest{
{"Good", TestGood},
{"Bad", TestBad},
},
nil, nil,
)
}
Which will output (try it on the Go Playground):
--- FAIL: Bad (0.00s)
prog.go:11: This is a mocked failed test
FAIL
If you want to capture the success of the testing, use the "newer" testing.MainStart() function.
First we need a helper type (which implements an unexported interface):
type testDeps struct{}
func (td testDeps) MatchString(pat, str string) (bool, error) { return true, nil }
func (td testDeps) StartCPUProfile(w io.Writer) error { return nil }
func (td testDeps) StopCPUProfile() {}
func (td testDeps) WriteProfileTo(string, io.Writer, int) error { return nil }
func (td testDeps) ImportPath() string { return "" }
func (td testDeps) StartTestLog(io.Writer) {}
func (td testDeps) StopTestLog() error { return nil }
func (td testDeps) SetPanicOnExit0(bool) {}
And now using it:
m := testing.MainStart(testDeps{},
[]testing.InternalTest{
{"Good", TestGood},
{"Bad", TestBad},
},
nil, nil,
)
result := m.Run()
fmt.Println(result)
Which outputs (try it on the Go Playground):
--- FAIL: Bad (0.00s)
prog.go:13: This is a mocked failed test
FAIL
1
If all tests pass, result will be 0.

There is a way to fire a command
go test -v -json
inside the code through exec.Command, initialise a reader that will read from stdout and then parse the output.
This answer was given by https://github.com/AlekSi
Very grateful, thank you Aleksi!
https://github.com/FerretDB/dance/blob/main/internal/gotest/gotest.go

Related

Go - errors.As failing to unwrap custom error type

I'm trying to use the Go stdlib package errors to unwrap a custom error type using errors.As, however it seems as though the check is failing and I cannot extract the underlying error.
I've extracted a minimal reproducible example:
package main
import (
"errors"
"fmt"
)
type myError struct {
err error
}
func (m myError) Error() string {
return fmt.Sprintf("my error: %s", m.err)
}
func retError() error {
return &myError{errors.New("wrapped")}
}
func main() {
var m myError
if err := retError(); errors.As(err, &m) {
fmt.Println("unwrapped", m.err)
} else {
fmt.Println(err)
}
}
https://go.dev/play/p/I7BNk4-rDIB - the example on the Go playground. If launched, it will print "my error: wrapped" instead of the expected "unwrapped wrapped".
The example from the errors.As documentation works, and I can't seem to understand what am I doing incorrectly - I'm passing a *myError to errors.As, which seems to be correct (since passing a myError raises a panic: target must be a non-nil pointer, which is expected).
Instead of:
func retError() error {
return &myError{errors.New("wrapped")}
}
Do:
func retError() error {
return myError{errors.New("wrapped")}
}

Automated integration testing in lambda support (golang, serverless)

So I have a web app running in serverless. It spins up a bunch of lambdas and then a 'test' lambda is invoked at a later stage of our pipeline to run some api tests against the other lambbas. It passes or fails code pipeline depending on the result of the tests.
My concern is the implementation of the tests portion itself.
We're using golang, and I wasn't able to successfully find a way to have a bunch of go test files and run them, record the results and determine pass/fail or not.. but I wanted to use the test suite and library in order to run my assertions. So I came up with this solution which was to break about the go test library and run it myself using MainStart(), ex:
// T is used to manage our test cases manually using MainStart
type T struct{}
func (*T) ImportPath() string { return "" }
func (*T) MatchString(pat, str string) (bool, error) { return true, nil }
func (*T) SetPanicOnExit0(bool) {}
func (*T) StartCPUProfile(io.Writer) error { return nil }
func (*T) StopCPUProfile() {}
func (*T) StartTestLog(io.Writer) {}
func (*T) StopTestLog() error { return nil }
func (*T) WriteHeapProfile(io.Writer) error { return nil }
func (*T) WriteProfileTo(string, io.Writer, int) error { return nil }
func (h *Handler) Handle(ctx context.Context, event events.CodePipelineEvent) (interface{}, error) {
job := event.CodePipelineJob
ok := runTest()
if ok {
logger.Info("Tests Passed!")
input := &codepipeline.PutJobSuccessResultInput{
JobId: &job.ID,
success, err := h.service.PutJobSuccessResult(input)
return success, err
} else {
logger.Info("Tests Failed :(")
input := &codepipeline.PutJobFailureResultInput{
JobId: &job.ID,
FailureDetails: &codepipeline.FailureDetails{
Message: aws.String("tests failed"),
Type: aws.String("JobFailed"),
},
failure, err := h.service.PutJobFailureResult(input)
return failure, err
}
}
}
// NewHandler returns a pointer to a Handler struct
func NewHandler(service *codepipeline.CodePipeline) *Handler {
return &Handler{
service,
}
}
// runTest returns the test results
func runTest() bool {
testSuite := []testing.InternalTest{
{Name: "Test Service", F: TestService},
}
logger.Info("Running Tests")
errors := testing.MainStart(&T{}, testSuite, nil, nil).Run()
if errors == 0 {
return true
} else {
return false
}
}
So the test looks like this:
func TestServices(t *testing.T) {
logger.Info("blah blah do tests")
}
And for reference, main.go contains:
// Initial code pipeline session
cpSession := session.Must(session.NewSession())
// Create the credentials from AssumeRoleProvider to assume the role
// referenced by the "putJobResultRoleArn" ARN.
creds := stscreds.NewCredentials(cpSession, "putJobResultRoleArn", func(p *stscreds.AssumeRoleProvider) {
p.RoleARN = roleArn
p.RoleSessionName = "put_job_result_session"
})
// Create a new instance of the CodePipeline client with a session
svc := codepipeline.New(cpSession, &aws.Config{Credentials: creds})
intTesthandler := inttest.NewHandler(
svc,
)
lambda.Start(func(ctx context.Context, event events.CodePipelineEvent) (interface{}, error) {
return intTesthandler.Handle(ctx, event)
})
This works quite well, but what I don't like is:
It's using an unsupported approach from golang that could break in the future
I have to handhold any failures, meaning if one fails, I need to write code to stop the rest of the suite (if there were more tests) from running.
Bit of a learning curve
To note the tests I have mainly do an http request to an api endpoint and validate the response + parse/store the data.
Anyone have any other creative solution? Is there a repo out there that makes it simple to run integration tests in golang?
Cheers

How to create an object of interface value type in go using reflection

UPDATED
I want to make helper function for testing reading env vars function. It uses envconfig.
func Test_T2(t *testing.T) {
os.Setenv("APP_PARAM_STR", "string value")
os.Setenv("APP_PARAM_INT", "12")
os.Setenv("APP_PARAM_DURATION", "15s")
os.Setenv("APP_PARAM_INT", "44")
c := ConfigTwo{}
d := ConfigTwo{
ParamDuration: 15*time.Second,
ParamInt: 44,
}
helper(t, &c, &d)
}
func helper(t *testing.T, confObject, expValue interface{}) {
t.Helper()
err := getParams(&confObject)
if !assert.NoError(t, err) {
return
}
assert.Equal(t, expValue, confObject)
}
func getParams(cfg interface{}) error {
return envconfig.Process("APP", cfg)
}
** UPDATE 2 **
It works. Thanks everyone.
It works if I have getPrams function only. But if I add helper (that I need to test different structs) I get an error:
specification must be a struct pointer
envconfig performs two checks here:
Use this code. The argument is a pointer to the expected value.
func helper(t *testing.T, pexpected interface{}) {
t.Helper()
pactual := reflect.New(reflect.TypeOf(pexpected).Elem()).Interface()
err := getParams(pactual)
if !assert.NoError(t, err) {
return
}
assert.Equal(t, pexpected, pactual)
}
The expression reflect.New(reflect.TypeOf(pexeceted).Elem()).Interface() returns a pointer to a new empty value with the same type as what pexpected points to.
Call it like this:
helper(t, &ConfigTwo{A: "expected A Field", B: "expected B field"}

Define type and method local in a func?

My code uses the interface with one function:
func InsertRow(rec []string) error
There are different types with different implementation of this interface. Now I would like to test this using "go test". In this case the implementation of InsertRow should do nothing:
func (t TestInserter) InsertRow(rec []string) error {
return nil
}
I can define a type internal in a test function. But now I would also like to define a dummy method for this type:
func TestInserter01(t *testing.T) {
type TestMyInserter struct {} <-- Test type
func (t TestMyInserter) InsertRow(rec []string) error { <-- Dummy function on this type.
return nil
}
... using InsertRow as a parameter in another function ...
}
But this produces compile errors:
expected operand, found ']'
expected ']', found 'return'
The same code works, if I define both the type and the method outside the test function.
Is it possible to hide the test implementation in the test function and do not define it outside the function? I need many of them, so that I would prefer to have them defined locally in the test function.
No, it's not possible. Method declarations may only be at the top level (outside of any function).
Spec: Declarations and scope:
Declaration = ConstDecl | TypeDecl | VarDecl .
TopLevelDecl = Declaration | FunctionDecl | MethodDecl .
See related: Is it possible to define an anonymous interface implementation in Go?
Note howerer that it's possible to supply "dynamic" implementations with a helper type. Meaning you'll provide the method implementations inside a function, and with the help of a helper type that implements the interface, you can get a "dynamic" implementation.
For example:
type Inserter interface {
InsertRow(rec []string) error
}
type helper func(rec []string) error
func (h helper) InsertRow(rec []string) error {
return h(rec)
}
func main() {
testInsert := func(rec []string) error {
return fmt.Errorf("rec: %v", rec)
}
var i Inserter = helper(testInsert)
err := i.InsertRow([]string{"one", "two"})
fmt.Println(err)
}
This will output (try it on the Go Playground):
rec: [one two]
A variant may be a struct holding fields of function types for the methods. It may be used to cover multiple methods:
type helper struct {
insertRow func(rec []string) error
}
func (h helper) InsertRow(rec []string) error {
return h.insertRow(rec)
}
func main() {
h := helper{
insertRow: func(rec []string) error {
return fmt.Errorf("rec: %v", rec)
},
}
var i Inserter = h
err := i.InsertRow([]string{"one", "two"})
fmt.Println(err)
}
This outputs the same. Try it on the Go Playground.

How can I instantiate a nil error using Golang's reflect?

I am writing a function using reflect.MakeFunc. That function can return an error. When it succeeds, I want it to return nil for its error-typed return value. How can I do that using reflect? Currently I have this:
package main
import (
"fmt"
"reflect"
"errors"
)
func main() {
fmt.Println("Hello, playground")
f := func() error {return nil}
fn := reflect.MakeFunc(reflect.TypeOf(f), func(args []reflect.Value) []reflect.Value {
return []reflect.Value{reflect.New(reflect.TypeOf(errors.New("")))}
}).Interface().(func() error)
fmt.Printf("err: %v", fn())
}
I get panic: reflect: function created by MakeFunc using closure returned wrong type: have **errors.errorString for error. I also tried adding a .Elem() after reflect.New(reflect.TypeOf(errors.New(""))), but I got panic: reflect: function created by MakeFunc using closure returned wrong type: have *errors.errorString for error. I tried .Elem().Elem(), and I got a segmentation fault.
How can I get a reflect.Value representing a nil error?
Use the following:
var nilError = reflect.Zero(reflect.TypeOf((*error)(nil)).Elem())
func main() {
fmt.Println("Hello, playground")
f := func() error { return nil }
fn := reflect.MakeFunc(reflect.TypeOf(f), func(args []reflect.Value) []reflect.Value {
return []reflect.Value{nilError}
}).Interface().(func() error)
fmt.Printf("err: %v", fn())
}
Let's break this down. The first step is to get a reflect.Type for error: reflect.TypeOf((*error)(nil)).Elem(). The simpler reflect.TypeOf((error)(nil)) does not work because the concrete value of the argument is nil. There's no type for nil and it's not the type we want anyway. The workaround is to pass a pointer to error and then call Elem() on the type to get the relfect.Type for error.
The second step is to create a zero value for the type.
I found one way - reflect.ValueOf(f).Call(nil). Maybe there's a better one.

Resources