How to use gomock with sqlc - go

I'd like to write unit tests for functions generated by sqlc but I don't understand how to use gomock. I'm new to mocking.
Below I'll describe a bit how sqlc generation works.
So, sqlc generates an interface DBTX:
type DBTX interface {
Exec(context.Context, string, ...interface{}) (pgconn.CommandTag, error)
Query(context.Context, string, ...interface{}) (pgx.Rows, error)
QueryRow(context.Context, string, ...interface{}) pgx.Row
}
func New(db DBTX) *Queries {
return &Queries{db: db}
}
type Queries struct {
db DBTX
}
func (q *Queries) WithTx(tx pgx.Tx) *Queries {
return &Queries{
db: tx,
}
}
And an interface Querier with all functions that talks to db (also generated by sqlc). In my case:
type Querier interface {
CreateCustomer(ctx context.Context, customerID int64) (int64, error)
CreateExecutor(ctx context.Context, arg CreateExecutorParams) (Executor, error)
CreateUser(ctx context.Context, arg CreateUserParams) (User, error)
DeleteUser(ctx context.Context, userID int64) error
GetUser(ctx context.Context, userID int64) (User, error)
ListUsers(ctx context.Context) ([]User, error)
}
var _ Querier = (*Queries)(nil)
Example of generated function GetUser:
const getUser = `-- name: GetUser :one
SELECT user_id, email, password FROM "user"
WHERE user_id = $1
`
func (q *Queries) GetUser(ctx context.Context, userID int64) (User, error) {
row := q.db.QueryRow(ctx, getUser, userID)
var i User
err := row.Scan(
&i.UserID,
&i.Email,
&i.Password,
)
return i, err
}
To use transactions I created an interface Storage (which contains Querier and all transaction functions) as described here. It looks like this:
type Storage interface {
Querier
CreateUserTx(ctx context.Context, arg CreateUserTxParams) (User, error)
}
type SQLStorage struct {
db *pgxpool.Pool
*Queries
}
func NewStorage(db *pgxpool.Pool) Storage {
return &SQLStorage{
db: db,
Queries: New(db),
}
}
So for example I created a function CreateUserTx that uses Storage.db to make transactions. And now all interactions with database are done through Storage.
And the question is how to mock database to test all functions in Storage. All mocking examples in web seem aren't that complicated. Could you elaborate on how to do that?

Related

Go does not recognize a func as a valid interface

I have the func
func(context.Context, *domain.Scorecard) (*domain.Scorecard, error)
And i want pass this as a param that receive a interface
warehouse interface {
Get(context.Context, *domain.Scorecard) (*domain.Scorecard, error)
}
Ex:
warehouseMock :=
usecase.WithScenarioFinder(
func(player *domain.Player) (*domain.Scenario, error) {
return nil,nil
)
The traditional form is create a struct that have the method Get, but i have curiosity if exist the way to tell "hey, that is a simple func, is the same firm (with no name), accept it"
Functions do not implement interfaces with only a single method of that same signature. See this Go issue for a discussion on the topic.
Create an adaptor type to convert a function to an interface:
type WarehouseFunc func(context.Context, *domain.Scorecard) (*domain.Scorecard, error)
func (f WarehouseFunc) Get(c context.Context, d *domain.Scorecard) (*domain.Scorecard, error) {
return f(c, d)
}
Convert an anonymous function to an interface like this:
itf = WarehouseFunc(func(c context.Context, d *domain.Scorecard) (*domain.Scorecard, error) {
return nil, nil
})

Calling the same function on two different structs

I've got two functions that accept different pointers pointing to different structs, but the structs have the same underlying function.
func Save(db *sql.DB) error {
db.Prepare(query)
}
func TxSave(tx *sql.Tx) error {
tx.Prepare(query)
}
I don't want to have to make changes in both functions when I need to extend this function in the future. How do I adhere to DRYness in golang with this scenario?
Create an interface such as:
type SQLRunner interface{
Prepare(query string) (*sql.Stmt, error)
PrepareContext(ctx context.Context, query string) (*sql.Stmt, error)
Query(query string, args ...interface{}) (*Rows, error)
QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
// add as many method shared by both sql.Tx qnd sql.DB
// ...
}
And then create a single method taking that interface:
func Save(s SQLRunner) error {
s.Prepare()
}
In go interface implementation is implicit, so you just have to pass *sql.Tx or *sql.DB to the save function:
Save(tx)
Save(db)
Here a good blog post about interfaces in go:http://jordanorelli.com/post/32665860244/how-to-use-interfaces-in-go
Wow, I think I'm in love with Go. I can actually just do this by creating my own interface.
type Saver interface {
Prepare(query string) (*sql.Stmt, error)
}
func Save(db *sql.DB) error {
return GenericSave(db)
}
func TxSave(tx *sql.Tx) error {
return GenericSave(tx)
}
func GenericSave(saver Saver) error {
stmt := saver.Prepare(query)
// Do rest with saver
}

How to mock functions in golang

I have written a simple package, which basically consists of a lot of getter functions. Each file in this package corresponds to a service, so for instance the product file, contains functions relating to the product service/db, order file to order service etc. Each function takes as parameters a db resource to the underlying db, and parameters for the sql, eg. productid, name, orderid. Each of the functions returns a struct (eg. Order, product) or an error:
// product.go
package lib
type Product struct {
ID int
Name string
Price float
}
func GetProductById(DB *sql.DB, ID int) (p Product, err error) {
q := "SELECT * FROM product WHERE id = " + ID
...
}
func GetProductByName(DB *sql.DB, name string) (p Product, err error) {
...
}
// order.go
package lib
type Order struct {
ID int
Date string
Items []items
}
func GetOrderById(DB *sql.DB, ID int) (o Order, err error) {
...
}
The problem is that I'm not able to mock these functions from my main package. What I really like to do, is to rewrite the package, so I somehow can pass to function to a type instead. But I'm not sure how to do this. Especially not when the functions take different input parameters and return different structs. Is there a way to do this?
In Go you can NOT mock a function, or a method declared on a concrete type.
For example:
func F() { ... }
func (T) M() { ... }
F and M are not mockable in Go.
However you can mock function values, whether they are variables, fields on a struct, or parameters passed to other functions.
For example:
var Fn = func() { ... }
type S struct {
Fn func()
}
func F(Fn func())
Fn in all three cases is mockable.
The other thing that you can mock in Go, and the prefered option most of the time, is an interface.
For example:
type ProductRepository interface {
GetProductById(DB *sql.DB, ID int) (p Product, err error)
}
// the real implementater of the interface
type ProductStore struct{}
func (ProductStore) GetProductById(DB *sql.DB, ID int) (p Product, err error) {
q := "SELECT * FROM product WHERE id = ?"
// ...
}
// the mock implementer
type ProductRepositoryMock struct {}
func (ProductRepositoryMock) GetProductById(DB *sql.DB, ID int) (p Product, err error) {
// ...
}
Now any piece of code that depends on ProductRepository can be passed a value of type ProductStore when you're in "normal mode" and a value of type ProductRepositoryMock when you're testing.
Another option using interfaces, which allows you to keep your functions mostly unchaged is to define an interface that mimics the methods of *sql.DB then use that interface type as the type to be passed to your functions, implement a mock version of that interface and use that during testing.
For example:
type DBIface interface {
Query(query string, args ...interface{}) (*sql.Rows, error)
// ...
// It's enough to implement only those methods that
// the functions that depend on DBIface actually use.
// If none of your functions ever calls SetConnMaxLifetime
// you don't need to declare that method on the DBIface type.
}
type DBMock struct {}
func (DBMock) Query(query string, args ...interface{}) (*sql.Rows, error) {
// ...
}
func GetProductByName(DB DBIface, name string) (p Product, err error) {
...
}
DB parameter to GetProductByName is now mockable.

Mocking a Go database SDK

I am attempting to create a wrapper for test emulating around the Go Flex SDK for Google Cloud Datastore. While I am currently successfully running the localhost emulator using
gcloud beta emulators datastore start --no-store-on-disk
in a separate terminal from my testing window, I would prefer to create a mock database emulator that runs as part of the test process itself (without execing the above) so that I can run multiple tests in parallel, each with its own database emulator.
I have run into a problem with the Google SDK not implementing my interface.
My wrapper contains this code:
package google
import (
"context"
"cloud.google.com/go/datastore"
)
type (
// Datastore is a wrapper for the Google Cloud Datastore Client.
Datastore datastore.Client
// Datastorer represents things that can operate like a datastore.Client.
Datastorer interface {
Delete(context.Context, *datastore.Key) error
Get(context.Context, *datastore.Key, interface{}) error
GetAll(context.Context, *datastore.Query, interface{}) ([]*datastore.Key, error)
Put(context.Context, *datastore.Key, interface{}) (*datastore.Key, error)
PutMulti(context.Context, []*datastore.Key, interface{}) ([]*datastore.Key, error)
RunInTransaction(context.Context, func(Transactioner) error, ...datastore.TransactionOption) (*datastore.Commit, error)
}
// Transactioner represents things that can operate like a datastore.Transaction.
Transactioner interface {
Commit() (*datastore.Commit, error)
Delete(*datastore.Key) error
DeleteMulti([]*datastore.Key) error
Get(*datastore.Key, interface{}) error
GetMulti([]*datastore.Key, interface{}) error
Put(*datastore.Key, interface{}) (*datastore.PendingKey, error)
PutMulti([]*datastore.Key, interface{}) ([]*datastore.PendingKey, error)
Rollback() error
}
)
// Delete deletes the entity for the given key.
func (d *Datastore) Delete(ctx context.Context, key *datastore.Key) error {
return (*datastore.Client)(d).Delete(ctx, key)
}
// Get retrieves the entity for the given key.
func (d *Datastore) Get(ctx context.Context, key *datastore.Key, dst interface{}) error {
return (*datastore.Client)(d).Get(ctx, key, dst)
}
// GetAll retrieves all entities for the given query.
func (d *Datastore) GetAll(ctx context.Context, q *datastore.Query, dst interface{}) ([]*datastore.Key, error) {
return (*datastore.Client)(d).GetAll(ctx, q, dst)
}
// Put stores an entity for the given key.
func (d *Datastore) Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error) {
return (*datastore.Client)(d).Put(ctx, key, src)
}
// PutMulti is a batch version of Put.
func (d *Datastore) PutMulti(ctx context.Context, keys []*datastore.Key, src interface{}) ([]*datastore.Key, error) {
return (*datastore.Client)(d).PutMulti(ctx, keys, src)
}
// RunInTransaction runs the given function in a transaction.
func (d *Datastore) RunInTransaction(ctx context.Context, f func(tx Transactioner) error, opts ...datastore.TransactionOption) (*datastore.Commit, error) {
return (*datastore.Client)(d).RunInTransaction(ctx, func(t *datastore.Transaction) error {
return f(t)
}, opts...)
}
Note that these interfaces do not emulate the complete SDK. I am only including functions that I actually call in my code. I'll add new ones as needed later.
When I try to use an instance of *datastore.Client as a Datastorer, I get the following error:
cannot use client (type *"cloud.google.com/go/datastore".Client) as type Datastorer in field value:
*"cloud.google.com/go/datastore".Client does not implement Datastorer (wrong type for RunInTransaction method)
have RunInTransaction(context.Context, func(*"cloud.google.com/go/datastore".Transaction) error, ..."cloud.google.com/go/datastore".TransactionOption) (*"cloud.google.com/go/datastore".Commit, error)
want RunInTransaction(context.Context, func(Transactioner) error, ..."cloud.google.com/go/datastore".TransactionOption) (*"cloud.google.com/go/datastore".Commit, error)
because *datastore.Client requires a function that takes a func(*datastore.Transaction) error and my interface wants a func(Transactioner) error.
Is there any way to change this so that it compiles?
If I can get it working, I plan to create types that implement my Datastorer and Transactioner interfaces and use maps to mock the real database. As far as tranactions go, for testing I can use sync.Mutex if I need them, but since each test is a single thread and will get its own database object, I may not need to lock them.
I've gotten it to compile by using this code:
package google
import (
"context"
"cloud.google.com/go/datastore"
)
type (
// Datastore is a wrapper for the Google Cloud Datastore Client.
Datastore struct {
*datastore.Client
}
// Datastorer represents things that can operate like a datastore.Client.
Datastorer interface {
Delete(context.Context, *datastore.Key) error
Get(context.Context, *datastore.Key, interface{}) error
GetAll(context.Context, interface{}, interface{}) ([]*datastore.Key, error)
Put(context.Context, *datastore.Key, interface{}) (*datastore.Key, error)
PutMulti(context.Context, []*datastore.Key, interface{}) ([]*datastore.Key, error)
RunInTransaction(context.Context, func(Transactioner) error, ...datastore.TransactionOption) (*datastore.Commit, error)
}
// Querier represents things that can operate like a datastore.Query.
Querier interface {
Filter(string, interface{}) Querier
}
// Transactioner represents things that can operate like a datastore.Transaction.
Transactioner interface {
Commit() (*datastore.Commit, error)
Delete(*datastore.Key) error
DeleteMulti([]*datastore.Key) error
Get(*datastore.Key, interface{}) error
GetMulti([]*datastore.Key, interface{}) error
Put(*datastore.Key, interface{}) (*datastore.PendingKey, error)
PutMulti([]*datastore.Key, interface{}) ([]*datastore.PendingKey, error)
Rollback() error
}
)
// Delete deletes the entity for the given key.
func (d *Datastore) Delete(ctx context.Context, key *datastore.Key) error {
return d.Client.Delete(ctx, key)
}
// Get retrieves the entity for the given key.
func (d *Datastore) Get(ctx context.Context, key *datastore.Key, dst interface{}) error {
return d.Client.Get(ctx, key, dst)
}
// GetAll retrieves all entities for the given query.
func (d *Datastore) GetAll(ctx context.Context, q interface{}, dst interface{}) ([]*datastore.Key, error) {
return d.Client.GetAll(ctx, q.(*datastore.Query), dst)
}
// Put stores an entity for the given key.
func (d *Datastore) Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error) {
return d.Client.Put(ctx, key, src)
}
// PutMulti is a batch version of Put.
func (d *Datastore) PutMulti(ctx context.Context, keys []*datastore.Key, src interface{}) ([]*datastore.Key, error) {
return d.Client.PutMulti(ctx, keys, src)
}
// RunInTransaction runs the given function in a transaction.
func (d *Datastore) RunInTransaction(ctx context.Context, f func(tx Transactioner) error, opts ...datastore.TransactionOption) (*datastore.Commit, error) {
return d.Client.RunInTransaction(ctx, func(t *datastore.Transaction) error {
return f(t)
}, opts...)
}
I changed DataStore to a struct containing the datastore.Client and added a new interface Querier that contains the functions that I am using from datastore.Query. I also updated GetAll to accept an interface{} instead of a *datastore.Query and then type-assert it to be a *datastore.Query. I cannot have it accept a Querier because then I cannot pass variables of type *datastore.Query because they do not satisfy the Querier interface (Filter returns a Querier instead of a *datastore.Query).
All existing tests using the emulator running in a separate process are passing.
UPDATE:
I changed Datastore to
Datastore datastore.Client
and added a wrapper Query around datastore.Query:
Query datastore.Query
Now, the Datastorer interface contains
GetAll(context.Context, Querier, interface{}) ([]*datastore.Key, error)
and the GetAll function is defined as
func (d *Datastore) GetAll(ctx context.Context, q Querier, dst interface{}) ([]*datastore.Key, error) {
return (*datastore.Client)(d).GetAll(ctx, (*datastore.Query)(q.(*Query)), dst)
}
and Query.Filter is defined as
func (q *Query) Filter(filterStr string, value interface{}) Querier {
return (*Query)((*datastore.Query)(q).Filter(filterStr, value))
}
In calling code, I use
q := datastore.NewQuery(entity).Filter("Deleted =", false)
_, err := r.client.GetAll(ctx, (*Query)(q), data)
This compiles and all tests are passing.
I know that question has been asked a long time ago but in case one still may wondering how to Mock Google Datastore Client and Transaction here is a snapshot of how I got it to work.
type Client interface {
Get(ctx context.Context, key *datastore.Key, dst interface{}) (err error)
NewTransaction(ctx context.Context, opts ...datastore.TransactionOption) (t Transaction, err error)
Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error)
}
type Transaction interface {
Commit() (c *datastore.Commit, err error)
Rollback() (err error)
Get(key *datastore.Key, dst interface{}) (err error)
Put(key *datastore.Key, src interface{}) (*datastore.PendingKey, error)
Delete(key *datastore.Key) error
}
type Datastore struct {
*datastore.Client
}
func (d *Datastore) NewTransaction(ctx context.Context, opts ...datastore.TransactionOption) (t Transaction, err error) {
return d.Client.NewTransaction(ctx, opts...)
}
Of course, If you are using different method against Datastore it's up to you to implement them.
In my tests, I can now Mock Datastore Client and Transaction like :
mockedClient := mock_gcloud.NewMockClient(ctrl)
mockedTransaction := mock_gcloud.NewMockTransaction(ctrl)
...
mockedClient.EXPECT().NewTransaction(context.Background()).Return(mockedTransaction, nil).Times(1)
Et voilĂ .

Idiomatic way to run several funcs with *sql.DB object in one transaction in Go lang

Let we have 2 methods
func createClient(db *sql.DB, ...) error // creates a new client
func createOrder(db *sql.DB, ...) error // creates a new order
Each of these method can be run on some *sql.DB, for example,
var mainDb *sql.DB // initialized somewhere in main() method
func orderHandler(r,w) {
...
err := createOrder(mainDb, ...)
...
}
But what if i want to run both these methods in one transaction. For example,
func importOrdersHandler(r,w) {
...
tx, err:= mainDb.Begin()
...
err = createClient(tx, clientData) // but method defined on *sql.DB, not *sql.Tx ?!
err = createOrder(tx, orderData)
...
My solution:
Define a wrapper around *sql.DB, *sql.Tx:
type Database struct {
db *sql.DB
tx *sql.Tx
}
// Redefine all standart methods from sql package, such as
// Exec(...)
// Query(..)
// QueryRow(...)
// Also method to run commands in one transaction:
func TransactionDo(db *sql.DB, body func(*Database) error) error {
tx, err := db.Begin()
...
d, err := NewDb(nil, tx)
....
err = body(d)
...
return tx.Commit()
}
In this way our ordersImportHandler can be realized like that:
func importOrdersHandler(r,w) {
for row := range parseFile(formFile(r)) {
...
err := TransactionDo(mainDb, func(d *Database) error {
err = createClient(d, clientData)
...
err = createOrder(d, orderData)
// if an error occurs in TransactionDo then transaction wiil be
// rollbacked, else commit it
createClient, createOrder must be rewrited to use *Database object insted of *sql.DB
What do you think about such solution? May be there another better and idiomatic
way to do that
I used an interface (that the library squirrel also uses)
type Database interface {
Exec(query string, args ...interface{}) (sql.Result, error)
Query(query string, args ...interface{}) (*sql.Rows, error)
QueryRow(query string, args ...interface{}) *sql.Row
}
Then you can just pass write the functions
func createClient(db *Database, ...) error // creates a new client
func createOrder(db *Database, ...) error // creates a new order
Simpler way is to use interfaces.
type Execer interface {
Exec(query string, args ...interface{}) (sql.Result, error)
}
Both sql.Tx and sql.Db satisfy this interface so you can redefine you createClient and createOrder funtions to take an Execer as an argument and there, you can now pass either Tx or Db to them.

Resources