I am attempting to create a wrapper for test emulating around the Go Flex SDK for Google Cloud Datastore. While I am currently successfully running the localhost emulator using
gcloud beta emulators datastore start --no-store-on-disk
in a separate terminal from my testing window, I would prefer to create a mock database emulator that runs as part of the test process itself (without execing the above) so that I can run multiple tests in parallel, each with its own database emulator.
I have run into a problem with the Google SDK not implementing my interface.
My wrapper contains this code:
package google
import (
"context"
"cloud.google.com/go/datastore"
)
type (
// Datastore is a wrapper for the Google Cloud Datastore Client.
Datastore datastore.Client
// Datastorer represents things that can operate like a datastore.Client.
Datastorer interface {
Delete(context.Context, *datastore.Key) error
Get(context.Context, *datastore.Key, interface{}) error
GetAll(context.Context, *datastore.Query, interface{}) ([]*datastore.Key, error)
Put(context.Context, *datastore.Key, interface{}) (*datastore.Key, error)
PutMulti(context.Context, []*datastore.Key, interface{}) ([]*datastore.Key, error)
RunInTransaction(context.Context, func(Transactioner) error, ...datastore.TransactionOption) (*datastore.Commit, error)
}
// Transactioner represents things that can operate like a datastore.Transaction.
Transactioner interface {
Commit() (*datastore.Commit, error)
Delete(*datastore.Key) error
DeleteMulti([]*datastore.Key) error
Get(*datastore.Key, interface{}) error
GetMulti([]*datastore.Key, interface{}) error
Put(*datastore.Key, interface{}) (*datastore.PendingKey, error)
PutMulti([]*datastore.Key, interface{}) ([]*datastore.PendingKey, error)
Rollback() error
}
)
// Delete deletes the entity for the given key.
func (d *Datastore) Delete(ctx context.Context, key *datastore.Key) error {
return (*datastore.Client)(d).Delete(ctx, key)
}
// Get retrieves the entity for the given key.
func (d *Datastore) Get(ctx context.Context, key *datastore.Key, dst interface{}) error {
return (*datastore.Client)(d).Get(ctx, key, dst)
}
// GetAll retrieves all entities for the given query.
func (d *Datastore) GetAll(ctx context.Context, q *datastore.Query, dst interface{}) ([]*datastore.Key, error) {
return (*datastore.Client)(d).GetAll(ctx, q, dst)
}
// Put stores an entity for the given key.
func (d *Datastore) Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error) {
return (*datastore.Client)(d).Put(ctx, key, src)
}
// PutMulti is a batch version of Put.
func (d *Datastore) PutMulti(ctx context.Context, keys []*datastore.Key, src interface{}) ([]*datastore.Key, error) {
return (*datastore.Client)(d).PutMulti(ctx, keys, src)
}
// RunInTransaction runs the given function in a transaction.
func (d *Datastore) RunInTransaction(ctx context.Context, f func(tx Transactioner) error, opts ...datastore.TransactionOption) (*datastore.Commit, error) {
return (*datastore.Client)(d).RunInTransaction(ctx, func(t *datastore.Transaction) error {
return f(t)
}, opts...)
}
Note that these interfaces do not emulate the complete SDK. I am only including functions that I actually call in my code. I'll add new ones as needed later.
When I try to use an instance of *datastore.Client as a Datastorer, I get the following error:
cannot use client (type *"cloud.google.com/go/datastore".Client) as type Datastorer in field value:
*"cloud.google.com/go/datastore".Client does not implement Datastorer (wrong type for RunInTransaction method)
have RunInTransaction(context.Context, func(*"cloud.google.com/go/datastore".Transaction) error, ..."cloud.google.com/go/datastore".TransactionOption) (*"cloud.google.com/go/datastore".Commit, error)
want RunInTransaction(context.Context, func(Transactioner) error, ..."cloud.google.com/go/datastore".TransactionOption) (*"cloud.google.com/go/datastore".Commit, error)
because *datastore.Client requires a function that takes a func(*datastore.Transaction) error and my interface wants a func(Transactioner) error.
Is there any way to change this so that it compiles?
If I can get it working, I plan to create types that implement my Datastorer and Transactioner interfaces and use maps to mock the real database. As far as tranactions go, for testing I can use sync.Mutex if I need them, but since each test is a single thread and will get its own database object, I may not need to lock them.
I've gotten it to compile by using this code:
package google
import (
"context"
"cloud.google.com/go/datastore"
)
type (
// Datastore is a wrapper for the Google Cloud Datastore Client.
Datastore struct {
*datastore.Client
}
// Datastorer represents things that can operate like a datastore.Client.
Datastorer interface {
Delete(context.Context, *datastore.Key) error
Get(context.Context, *datastore.Key, interface{}) error
GetAll(context.Context, interface{}, interface{}) ([]*datastore.Key, error)
Put(context.Context, *datastore.Key, interface{}) (*datastore.Key, error)
PutMulti(context.Context, []*datastore.Key, interface{}) ([]*datastore.Key, error)
RunInTransaction(context.Context, func(Transactioner) error, ...datastore.TransactionOption) (*datastore.Commit, error)
}
// Querier represents things that can operate like a datastore.Query.
Querier interface {
Filter(string, interface{}) Querier
}
// Transactioner represents things that can operate like a datastore.Transaction.
Transactioner interface {
Commit() (*datastore.Commit, error)
Delete(*datastore.Key) error
DeleteMulti([]*datastore.Key) error
Get(*datastore.Key, interface{}) error
GetMulti([]*datastore.Key, interface{}) error
Put(*datastore.Key, interface{}) (*datastore.PendingKey, error)
PutMulti([]*datastore.Key, interface{}) ([]*datastore.PendingKey, error)
Rollback() error
}
)
// Delete deletes the entity for the given key.
func (d *Datastore) Delete(ctx context.Context, key *datastore.Key) error {
return d.Client.Delete(ctx, key)
}
// Get retrieves the entity for the given key.
func (d *Datastore) Get(ctx context.Context, key *datastore.Key, dst interface{}) error {
return d.Client.Get(ctx, key, dst)
}
// GetAll retrieves all entities for the given query.
func (d *Datastore) GetAll(ctx context.Context, q interface{}, dst interface{}) ([]*datastore.Key, error) {
return d.Client.GetAll(ctx, q.(*datastore.Query), dst)
}
// Put stores an entity for the given key.
func (d *Datastore) Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error) {
return d.Client.Put(ctx, key, src)
}
// PutMulti is a batch version of Put.
func (d *Datastore) PutMulti(ctx context.Context, keys []*datastore.Key, src interface{}) ([]*datastore.Key, error) {
return d.Client.PutMulti(ctx, keys, src)
}
// RunInTransaction runs the given function in a transaction.
func (d *Datastore) RunInTransaction(ctx context.Context, f func(tx Transactioner) error, opts ...datastore.TransactionOption) (*datastore.Commit, error) {
return d.Client.RunInTransaction(ctx, func(t *datastore.Transaction) error {
return f(t)
}, opts...)
}
I changed DataStore to a struct containing the datastore.Client and added a new interface Querier that contains the functions that I am using from datastore.Query. I also updated GetAll to accept an interface{} instead of a *datastore.Query and then type-assert it to be a *datastore.Query. I cannot have it accept a Querier because then I cannot pass variables of type *datastore.Query because they do not satisfy the Querier interface (Filter returns a Querier instead of a *datastore.Query).
All existing tests using the emulator running in a separate process are passing.
UPDATE:
I changed Datastore to
Datastore datastore.Client
and added a wrapper Query around datastore.Query:
Query datastore.Query
Now, the Datastorer interface contains
GetAll(context.Context, Querier, interface{}) ([]*datastore.Key, error)
and the GetAll function is defined as
func (d *Datastore) GetAll(ctx context.Context, q Querier, dst interface{}) ([]*datastore.Key, error) {
return (*datastore.Client)(d).GetAll(ctx, (*datastore.Query)(q.(*Query)), dst)
}
and Query.Filter is defined as
func (q *Query) Filter(filterStr string, value interface{}) Querier {
return (*Query)((*datastore.Query)(q).Filter(filterStr, value))
}
In calling code, I use
q := datastore.NewQuery(entity).Filter("Deleted =", false)
_, err := r.client.GetAll(ctx, (*Query)(q), data)
This compiles and all tests are passing.
I know that question has been asked a long time ago but in case one still may wondering how to Mock Google Datastore Client and Transaction here is a snapshot of how I got it to work.
type Client interface {
Get(ctx context.Context, key *datastore.Key, dst interface{}) (err error)
NewTransaction(ctx context.Context, opts ...datastore.TransactionOption) (t Transaction, err error)
Put(ctx context.Context, key *datastore.Key, src interface{}) (*datastore.Key, error)
}
type Transaction interface {
Commit() (c *datastore.Commit, err error)
Rollback() (err error)
Get(key *datastore.Key, dst interface{}) (err error)
Put(key *datastore.Key, src interface{}) (*datastore.PendingKey, error)
Delete(key *datastore.Key) error
}
type Datastore struct {
*datastore.Client
}
func (d *Datastore) NewTransaction(ctx context.Context, opts ...datastore.TransactionOption) (t Transaction, err error) {
return d.Client.NewTransaction(ctx, opts...)
}
Of course, If you are using different method against Datastore it's up to you to implement them.
In my tests, I can now Mock Datastore Client and Transaction like :
mockedClient := mock_gcloud.NewMockClient(ctrl)
mockedTransaction := mock_gcloud.NewMockTransaction(ctrl)
...
mockedClient.EXPECT().NewTransaction(context.Background()).Return(mockedTransaction, nil).Times(1)
Et voilà.
Related
Let's say I want to decouple github.com/go-redis/redis/v8 from my app, for the usual reasons.
What I have until now is this:
import (
"context"
"github.com/go-redis/redis/v8"
)
type Pubsub interface {
Publish(ctx context.Context, topic string, message interface{}) *redis.IntCmd
Subscribe(ctx context.Context, topics ...string) *redis.PubSub
}
type PubsubClient struct {
Client Pubsub
}
func (psClient *PubsubClient) Publish(ctx context.Context, topic string,
message interface{}) error {
err := psClient.Client.Publish(ctx, topic, message).Err()
if err != nil {
return err
}
return nil
}
func (psClient *PubsubClient) Subscribe(ctx context.Context,
topics ...string) <-chan *redis.Message {
redisPubsub := psClient.Client.Subscribe(ctx, topics...)
return redisPubsub.Channel()
}
How do I avoid exposing *redis.Message to the users of PubsubClient?
I'm a newbie in Go, so I thought using the following approach:
type ThirdPartyType struct{}
func ThirdPartyFunc() ThirdPartyType{
return ThirdPartyType{}
}
type MyType ThirdPartyType
type MyAbstractStuff interface {
ThirdPartyFunc() ThirdPartyType
}
type MyConcreteStuff struct{
Client MyAbstractStuff
}
func (mcs *MyConcreteStuff) MyFunc() MyType {
tpt := mcs.Client.ThirdPartyFunc()
return MyType(tpt)
}
So I moved forward and did this:
type PubsubMsgChan <-chan *redis.Message
func (psClient *PubsubClient) Subscribe(ctx context.Context,
topics ...string) PubsubMsgChan {
redisPubsub := psClient.Client.Subscribe(ctx, topics...)
return PubsubMsgChan(redisPubsub.Channel())
}
But this doesn't seem right. If I was a user of this code, I would understand it better if there was a <-chan in my Subscribe() return type signature, so the user wouldn't have to dig in to find out that PubsubMsgChan is a read-only channel.
Considering redisPubsub.Channel() returns <-chan *redis.Message, how can I keep the <-chan in my method signature and at the same time hide *redis.Message?
For starters, is my approach to decoupling in Go actually valid/good/workable?
I'd like to write unit tests for functions generated by sqlc but I don't understand how to use gomock. I'm new to mocking.
Below I'll describe a bit how sqlc generation works.
So, sqlc generates an interface DBTX:
type DBTX interface {
Exec(context.Context, string, ...interface{}) (pgconn.CommandTag, error)
Query(context.Context, string, ...interface{}) (pgx.Rows, error)
QueryRow(context.Context, string, ...interface{}) pgx.Row
}
func New(db DBTX) *Queries {
return &Queries{db: db}
}
type Queries struct {
db DBTX
}
func (q *Queries) WithTx(tx pgx.Tx) *Queries {
return &Queries{
db: tx,
}
}
And an interface Querier with all functions that talks to db (also generated by sqlc). In my case:
type Querier interface {
CreateCustomer(ctx context.Context, customerID int64) (int64, error)
CreateExecutor(ctx context.Context, arg CreateExecutorParams) (Executor, error)
CreateUser(ctx context.Context, arg CreateUserParams) (User, error)
DeleteUser(ctx context.Context, userID int64) error
GetUser(ctx context.Context, userID int64) (User, error)
ListUsers(ctx context.Context) ([]User, error)
}
var _ Querier = (*Queries)(nil)
Example of generated function GetUser:
const getUser = `-- name: GetUser :one
SELECT user_id, email, password FROM "user"
WHERE user_id = $1
`
func (q *Queries) GetUser(ctx context.Context, userID int64) (User, error) {
row := q.db.QueryRow(ctx, getUser, userID)
var i User
err := row.Scan(
&i.UserID,
&i.Email,
&i.Password,
)
return i, err
}
To use transactions I created an interface Storage (which contains Querier and all transaction functions) as described here. It looks like this:
type Storage interface {
Querier
CreateUserTx(ctx context.Context, arg CreateUserTxParams) (User, error)
}
type SQLStorage struct {
db *pgxpool.Pool
*Queries
}
func NewStorage(db *pgxpool.Pool) Storage {
return &SQLStorage{
db: db,
Queries: New(db),
}
}
So for example I created a function CreateUserTx that uses Storage.db to make transactions. And now all interactions with database are done through Storage.
And the question is how to mock database to test all functions in Storage. All mocking examples in web seem aren't that complicated. Could you elaborate on how to do that?
I have the func
func(context.Context, *domain.Scorecard) (*domain.Scorecard, error)
And i want pass this as a param that receive a interface
warehouse interface {
Get(context.Context, *domain.Scorecard) (*domain.Scorecard, error)
}
Ex:
warehouseMock :=
usecase.WithScenarioFinder(
func(player *domain.Player) (*domain.Scenario, error) {
return nil,nil
)
The traditional form is create a struct that have the method Get, but i have curiosity if exist the way to tell "hey, that is a simple func, is the same firm (with no name), accept it"
Functions do not implement interfaces with only a single method of that same signature. See this Go issue for a discussion on the topic.
Create an adaptor type to convert a function to an interface:
type WarehouseFunc func(context.Context, *domain.Scorecard) (*domain.Scorecard, error)
func (f WarehouseFunc) Get(c context.Context, d *domain.Scorecard) (*domain.Scorecard, error) {
return f(c, d)
}
Convert an anonymous function to an interface like this:
itf = WarehouseFunc(func(c context.Context, d *domain.Scorecard) (*domain.Scorecard, error) {
return nil, nil
})
I've got two functions that accept different pointers pointing to different structs, but the structs have the same underlying function.
func Save(db *sql.DB) error {
db.Prepare(query)
}
func TxSave(tx *sql.Tx) error {
tx.Prepare(query)
}
I don't want to have to make changes in both functions when I need to extend this function in the future. How do I adhere to DRYness in golang with this scenario?
Create an interface such as:
type SQLRunner interface{
Prepare(query string) (*sql.Stmt, error)
PrepareContext(ctx context.Context, query string) (*sql.Stmt, error)
Query(query string, args ...interface{}) (*Rows, error)
QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
// add as many method shared by both sql.Tx qnd sql.DB
// ...
}
And then create a single method taking that interface:
func Save(s SQLRunner) error {
s.Prepare()
}
In go interface implementation is implicit, so you just have to pass *sql.Tx or *sql.DB to the save function:
Save(tx)
Save(db)
Here a good blog post about interfaces in go:http://jordanorelli.com/post/32665860244/how-to-use-interfaces-in-go
Wow, I think I'm in love with Go. I can actually just do this by creating my own interface.
type Saver interface {
Prepare(query string) (*sql.Stmt, error)
}
func Save(db *sql.DB) error {
return GenericSave(db)
}
func TxSave(tx *sql.Tx) error {
return GenericSave(tx)
}
func GenericSave(saver Saver) error {
stmt := saver.Prepare(query)
// Do rest with saver
}
The package 'gopkg.in/redis.v3' contains some code
type Client struct {
}
func (*client) Eval (string, []string, []string) *Cmd {
}
type Cmd struct {
}
func (*Cmd) Result () (interface{}, error) {
}
Which works successfully in the following way
func myFunc (cli *redis.Client) {
result, err := cli.Eval('my script').Result()
}
The problem is that sometimes the Redis cluster gets hammered, has a moment, and the interface returned as a result is nil.
This is reasonably easy to handle but I wish to put a test in place that will ensure that it is actually handled and no type assertion panic occurs.
Traditionally I would insert a mock Redis client into myFunc that can ultimately return nil.
type redisClient interface {
Eval(string, []string, []string) redisCmd
}
type redisCmd interface {
Result() (interface{}, error)
}
func myFunc (cli redisClient) {
result, err := cli.Eval('my script').Result()
}
The problem I am facing is the compiler doesn't recognise that redis.Client satisfies the interface redisClient because it doesn't recognise that the redis.Cmd returned from Eval satisfies redisCmd.
> cannot use client (type *redis.Client) as type redisClient in argument to myFunc:
> *redis.Client does not implement redisClient (wrong type for Eval method)
> have Eval(sting, []string, []string) *redis.Cmd
> want Eval(sting, []string, []string) redisCmd
The problem is that your interface does not match the redis client. If you change the interface to:
type redisClient interface {
Eval(string, []string, []string) *redis.Cmd
}
it will compile. That being said, it looks like you really want rediscmd, so you will need to make a wrapper around the redis client:
type wrapper struct{
c *redis.Client
}
func (w wrapper) Eval(x sting, y []string, z []string) redisCmd {
return w.c.Eval(x,y,z) // This assumes that *redis.Cmd implements rediscmd
}