How to create test code that needs to use the database - go

Question

I want to publish my first go library on GitHub.
I created a scan_test.go which has many tests that connect to a postgresql database. It doesn't need any data, only a valid connection since it tests result of static query result for example select 1 union select 2.
So I to release the package and that the tests would work, how do I allow configuration for the database for the tests? one idea that comes up is to use env variables? but what's the official way? how to properly create a test for my project?
example of my test file:
const (
host = "localhost"
port = 5432
user = "ufk"
password = "your-password"
dbname = "mycw"
)
type StructInt struct {
Moshe int
Moshe2 int
Moshe3 []int
Moshe4 []*int
Moshe5 []string
}
func TestVarsInStructInJsonArrayWithOneColumn(t *testing.T) {
if conn, err := GetDbConnection(); err != nil {
t.Errorf("could not connect to database: %v", err)
} else {
sqlQuery := `select json_build_array(json_build_object('moshe',55,'moshe2',66,'moshe3','{10,11}'::int[],'moshe4','{50,51}'::int[],
'moshe5','{kfir,moshe}'::text[]),
json_build_object('moshe',56,'moshe2',67,'moshe3','{41,42}'::int[],'moshe4','{21,22}'::int[],
'moshe5','{kfirrrr,moshrre}'::text[])) as moshe;`
var foo []StructInt
if isEmpty, err := Query(context.Background(), conn, &foo, sqlQuery); err != nil {
t.Errorf("failed test: %v", err)
} else if isEmpty {
log.Fatal("failed test with empty results")
}
if foo[0].Moshe != 55 {
t.Errorf("int slice test failed 21 <> %v", foo[0].Moshe)
}
if foo[1].Moshe2 != 67 {
t.Errorf("int slice failed with 82 <> %v", foo[1].Moshe2)
}
if len(foo[1].Moshe3) != 2 {
t.Errorf("int silice failed, array size should be 2 <> %v", len(foo[1].Moshe3))
}
if foo[1].Moshe3[1] != 42 {
t.Errorf("int slice failed, moshe3[0] not 2 <=> %v", foo[1].Moshe3[1])
}
if len(foo[1].Moshe4) != 2 {
t.Errorf("int silice failed, array size should be 2 <> %v", len(foo[1].Moshe4))
}
if *foo[1].Moshe4[1] != 22 {
t.Errorf("int slice failed, moshe4[1] not 4 <=> %v", foo[1].Moshe4[1])
}
}
}
func GetDbConnection() (*pgxpool.Pool, error) {
psqlInfo := fmt.Sprintf("host=%s port=%d user=%s "+
"password=%s dbname=%s sslmode=disable",
host, port, user, password, dbname)
return pgxpool.Connect(context.Background(), psqlInfo)
}
thanks

Answer 1

A typical approach is to combine usage of TestMain with a documented set of environment variables and/or command-line options which can be passed to the testing binary built to run tests of a particular package.
Basically, TestMain reads the environment and/or command-line options, validates them, may be populates some exported variables available to the test suite and then runs the suite.
Here at my $dayjob we use the described approach and a helper function named like SkipIfNoDatabase(t *testing.T) which inspects the state set up by TestMain and skips the test in the prologue of which it is called (after logging the reason for that) if the required configuration related to DB connectivity was not provided. This allows running a test suite w/o setting up a database—all the tests which require one will be skipped.
An example for running tests in Gitlab CI follows.
The .gitlab-ci.yml in a project contains, among other things, something like
stages:
- test
.golang_stage:
image: internal-corporate-registry:5000/go-test:1.14
variables:
MONGODB_URI: 'mongodb://test-mongodb:27017'
services:
- name: mongo:3.6-xenial
alias: test-mongodb
test_golang:
extends: .golang_stage
stage: test
tags:
- common
only:
- pushes
- schedules
script:
- make test
- make build
This configuration makes sure when our Go program which makes use of a MongoDB instance is tested, the mongo:3.6-xenial Docker image is pulled and run, and assigned a hostname test-mongodb; the MONGODB_URI environment variable is set to refer to that running MongoDB instance.
Now the program featues the package lib/testing/testdb which contains
package testdb
import (
"io"
"os"
"testing"
)
// DBName is the MongoDB database name to be used in tests.
const DBName = "blah_blah_test"
var (
// MongoDBURI identifies a MongoDB instance to use by testing suite.
MongoDBURI string
// Enabled is only set to true if the MongoDB URI was made available
// to the test suite. It can be used by individual tests to skip
// execution if an access to a MongoDB instance is required to perform
// the test.
Enabled bool
)
// Initialize initializes the package's global state.
//
// Initialize is intended to be called once per package being tested -
// typically from the package's TestMain function.
func Initialize() {
MongoDBURI = os.Getenv("MONGODB_URI")
if MongoDBURI == "" {
Enabled = false
io.WriteString(os.Stderr,
"Empty or missing environment variable MONGODB_URI; related tests will be skipped\n")
return
}
Enabled = true
}
// SkipIfDisabled skips the current test if it appears there is no
// MongoDB instance to use.
func SkipIfDisabled(t *testing.T) {
if !Enabled {
t.Skip("Empty or missing MONGODB_URI environment variable; skipped.")
}
}
…and then each package which makes use of the database contains a file named main_test.go which reads
package whatever_test
import (
"testing"
"acme.com/app/lib/testing/testdb"
"acme.com/app/lib/testing/testmain"
)
func TestMain(m *testing.M) {
testdb.Initialize()
testmain.Run(m)
}
and the tests themselves roll like this
package whatever_test
import (
"testing"
"acme.com/app/lib/testing/testdb"
)
func TestFoo(t *testing.T) {
testdb.SkipIfDisabled(t)
# Do testing
}
testmain is another internal package exporting the Run function
which runs the tests but before doing that it takes care of additional initialization: sets up logging for the app's code and figures out whether it was requested to run stress tests (which are only run at night, scheduled).
package testmain
import (
"flag"
"os"
"testing"
"acme.com/app/lib/logging"
)
// Stress is true if the stress tests are enabled in this run
// of the test suite.
var Stress bool
// Run initializes the package state and then runs the test suite the
// way `go test` does by default.
//
// Run is expected to be called from TestMain functions of the test suites
// which make use of the testmain package.
func Run(m *testing.M) {
initialize()
os.Exit(m.Run())
}
// SkipIfNotStress marks the test currently executed by t as skipped
// unless the current test suite is running with the stress tests enabled.
func SkipIfNotStress(t *testing.T) {
if !Stress {
t.Skip("Skipped test: not in stress-test mode.")
}
}
func initialize() {
if flag.Parsed() {
return
}
var logFileName string
flag.BoolVar(&Stress, "stress", false, "Run stress tests")
flag.StringVar(&logFileName, "log", "", "Name of the file to redirect log output into")
flag.Parse()
logging.SetupEx(logging.Params{
Path: logFileName,
Overwrite: true,
Mode: logging.DefaultFileMode,
})
}
The relevant bits of the project's Makefile which run stess tests look like this:
STRESS_LOG ?= stress.log
.PHONY: all test build stress install/linter
all: test build
build:
go build -ldflags='$(LDFLAGS)'
test:
go test ./...
stress:
go test -v -run=Stress -count=1 ./... -stress -log="$(STRESS_LOG)"
…and the CI configuration to run stress tests reads
stress:
extends: .golang_stage
stage: test
tags:
- go
only:
- schedules
variables:
STRESS_LOG: "$CI_PROJECT_DIR/stress.log"
artifacts:
paths:
- "$STRESS_LOG"
when: on_failure
expire_in: 1 week
script:
- make stress