I have this function which I need to mock in test,
I was able to mock it as expected with http mock package , but now I’ve function that are calling
To the HttpReq method and here I cannot use http mock package
I read about dependency injection and tried something but I wasn’t able to fully do it,
This is the function
type params struct {
cs string
ci string
method string
url string
}
// I added this struct but not sure if it's needed ... probably for test purpose but not sure how to use it.
type Impl struct {
client *http.Client
}
func (i *Impl) HttpReq(p *params) ([]byte, error) {
httpClient := i.client
req, err := http.NewRequest(p.method, p.url, nil)
if err != nil {
fmt.Sprintf(err)
}
req.SetBasicAuth(p.cs, p.ci)
res, err := httpClient.Do(req)
if err != nil {
fmt.Sprintf(err)
}
t, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Sprintf(err)
}
defer res.Body.Close()
return t, nil
}
This is what I tried
I’ve created interface
type Req interface {
HttpReq(params) ([]byte, error)
}
Now I’ve created a struct which contain the interface
type Service struct {
req Req
}
This is the function which I need to tests
func (c *Service) execute(cli Connection , args []string) (error, []byte) {
sk, err := c.doSomthing(cli, args)
sc, err := c.doSometing2(serviceK, []string{"url", "cl", "ct"})
cc := strings.Fields(serviceCredentials)
// ----------Here is what I need to mock ----------
t, err := c.req.HttpReq(params{cs: cc[1],
ci: cc[2],
method: http.MethodPost,
url: cc[0],})
return err, t
}
Any idea how I can run test for this function ??? Im struggling with it a lot.
Independent of the original question, you should not create new HTTP clients for each request. Client's maintain a connection pool and should be reused as much as possible.
You can fix that, and continue using your existing mock server by injecting the HTTP client.
Note also that the interface definition in the question doesn't match the implementation. These two method signatures are not the same:
HttpReq(params) ([]byte, error) // Req
HttpReq(*params) ([]byte, error) // Impl
Pick one. I would probably go with the non-pointer type here. And upper case initials are idiomatic in Go (HTTPReq, not HttpReq).
Add the client to the Impl type and use it in HTTPReq:
type Impl struct {
client *http.Client
}
func (i *Impl) HTTPReq(p params) ([]byte, error) {
req, err := http.NewRequest(p.method, p.url, nil)
if err != nil {
return nil, err
}
req.SetBasicAuth(p.cs, p.ci)
res, err := i.client.Do(req)
if err != nil {
return nil, err
}
defer res.Body.Close()
return ioutil.ReadAll(res.Body)
}
The Service type doesn't have to change.
In the tests, simply inject a test client into the Impl value:
import (
"context"
"net"
"net/http"
"net/http/httptest"
"testing"
)
func TestService_execute(t *testing.T) {
var testHandler http.Handler // TODO
srv := httptest.NewServer(testHandler)
defer srv.Close()
client := srv.Client()
tp := client.Transport.(*http.Transport)
// Direct all requests to the test server, no matter what the request URL is.
tp.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {
// Note that we ignore the network and addr parameters, since these are
// derived from the request URL and are unrelated to the test server.
srvAddr := srv.Listener.Addr()
return (&net.Dialer{}).DialContext(ctx, srvAddr.Network(), srvAddr.String())
}
svc := &Service{
req: &Impl{
client: client,
},
}
svc.execute(/* ... */)
// assert request, response, etc.
}
Since Service struct already has an req interface, during tests initialise service object with mock that satisfies req interface.
Something like this
https://stackoverflow.com/a/53805535/3968921
Related
Context
I have a simple gin based Rest API implementing CRUD over a redis DB. I want to mock the redis client (github.com/go-redis/redis/v9) in my unit tests. Coming from OOP, I feel that I am transposing some patterns wrongly.
Here is the route I want to test (internal/api.go)
func GetRouter() *gin.Engine {
router := gin.Default()
...
router.GET("/session/:sessionid", getSession)
return router
}
Its controller is defined in internal/controlers.go
func getSession(c *gin.Context) {
client := redis_client.GetClient()
sessionid := c.Param("sessionid")
val, err := client.Get(c, sessionid).Result()
if err == redis.Nil {
fmt.Printf("key %s not exist", sessionid)
c.JSON(404, fmt.Sprintf("session %s not found", sessionid))
} else if err != nil {
fmt.Printf("unkown error %s", err)
c.JSON(500, fmt.Sprintf(`{"error":"%s"}`, err))
} else {
c.JSON(200, val)
}
}
redis_client.GetClient comes from one of my pkg. It's a singleton. pkg/redis/redis.go
var redisClient *redis.Client
func GetClient() *redis.Client {
if redisClient == nil {
redisClient = redis.NewClient(&redis.Options{
Addr: os.Getenv("REDIS_HOST"),
Password: os.Getenv("REDIS_PWD"),
DB: 0,
})
}
return redisClient
}
Testing
Spontaneously, I tried to mock either redis.Client.Get or redis.StringCmd.Result, for it is the place where the actual call to Redis occurs. In both cases, I am using github.com/stretchr/testify/mock
Moking redis.Client.Get
import (
....
"testing"
"github.com/stretchr/testify/mock"
)
type MockedRedisConn struct {
mock.Mock
}
func (m *MockedRedisConn) Get(ctx context.Context, key string) *redis.StringCmd {
ret := m.Called(ctx, key)
var r0 *redis.StringCmd
if ret.Get(0) != nil {
r0 = ret.Get(0).(*redis.StringCmd)
}
return r0
}
func Test_Sessions(t *testing.T) {
r := api.GetRouter()
redisConn := new(MockedRedisConn)
var ctx = context.Background()
t.Run("Should return the session", func(t *testing.T) {
//mocks
strCmd := redis.NewStringCmd(ctx)
strCmd.SetVal(`{"session":""}`)
redisConn.On("Get", mock.AnythingOfType("*gin.Context"), "00000000").Return(strCmd)
//call
req, _ := http.NewRequest("GET", "/session/00000000", nil)
w := httptest.NewRecorder()
r.ServeHTTP(w, req)
responseData, _ := ioutil.ReadAll(w.Body)
assert.Equal(t, `{"session":""}`, string(responseData))
assert.Equal(t, http.StatusOK, w.Code)
})
}
Same logic with redis.StringCmd.Result: I end up on a call to localhost:6379. What puzzles me is that, in most examples I could find, the mocked object is an argument of the function to be tested (I do not quite understand the point of it, by the way). In my case, it is not.
I typically miss a link to set the client in getSession to be my MockedRedisConn rather than the client given by GetClient() (i.e. some kind of monkey patching, if I am not mistaken). Is my singleton based setting of the client compatible with this approach? Should I rather load the client from *gin.Context so I could easily replace it when calling GetRouter in my tests?
I use the following function, and I need to raise the coverage of it (if possible to 100%), the problem is that typically I use interface to handle such cases in Go and for this specific case not sure how to do it, as this is a bit more tricky, any idea?
The package https://pkg.go.dev/google.golang.org/genproto/googleapis/cloud/compute/v1
Which I use doesn't have interface so not sure how can I mock it?
import (
"context"
"errors"
"fmt"
"os"
compute "cloud.google.com/go/compute/apiv1"
"google.golang.org/api/iterator"
"google.golang.org/api/option"
computev1 "google.golang.org/genproto/googleapis/cloud/compute/v1"
)
func Res(ctx context.Context, project string, region string,vpc string,secret string) error {
c, err := compute.NewAddressesRESTClient(ctx, option.WithCredentialsFile(secret))
if err != nil {
return err
}
defer c.Close()
addrReq := &computev1.ListAddressesRequest{
Project: project,
Region: region,
}
it := c.List(ctx, addrReq)
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
if *(resp.Status) != "IN_USE" {
return ipConverter(*resp.Name, vpc)
}
}
return nil
}
Whenever I find myself in this scenario, I found that the easiest solution is to create missing interfaces myself. I limit these interfaces to the types and functions that I am using, instead of writing interfaces for the entire library. Then, in my code, instead of accepting third-party concrete types, I accept my interfaces for those types. Then I use gomock to generate mocks for these interfaces as usual.
The following is a descriptive example inspired by your code.
type RestClient interface {
List(context.Context, *computev1.ListAddressesRequest) (ListResult, error) // assuming List returns ListResult type.
Close() error
}
func newRestClient(ctx context.Context, secret string) (RestClient, error) {
return compute.NewAddressesRESTClient(ctx, option.WithCredentialsFile(secret))
}
func Res(ctx context.Context, project string, region string, vpc string, secret string) error {
c, err := newRestClient(ctx, secret)
if err != nil {
return err
}
defer c.Close()
return res(ctx, project, region, vpc, c)
}
func res(ctx context.Context, project string, region string, vpc string, c RestClient) error {
addrReq := &computev1.ListAddressesRequest{
Project: project,
Region: region,
}
it, err := c.List(ctx, addrReq)
if err != nil {
return err
}
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
if *(resp.Status) != "IN_USE" {
return ipConverter(*resp.Name, vpc)
}
}
return nil
}
Now you can test the important bits of the Res function by injecting a mock RestClient to the internal res function.
One obstacle to testability here is that you instantiate a client inside your Res function rather than injecting it. Because
the secret doesn't change during the lifetime of the programme,
the methods of *compute.AddressesClient (other than Close) are concurrency-safe,
you could create one client and reuse it for each invocation or Res. To inject it into Res, you can declare some Compute type and turn Res into a method on that type:
type Compute struct {
Lister Lister // some appropriate interface type
}
func (cp *Compute) Res(ctx context.Context, project, region, vpc string) error {
addrReq := &computev1.ListAddressesRequest{
Project: project,
Region: region,
}
it := cp.Lister.List(ctx, addrReq)
for {
resp, err := it.Next()
if err == iterator.Done {
break
}
if err != nil {
return err
}
if *(resp.Status) != "IN_USE" {
return ipConverter(*resp.Name, vpc)
}
}
return nil
}
One question remains: how should you declare Lister? One possibility is
type Lister interface {
List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) *compute.AddressIterator
}
However, because compute.AddressIterator is a struct type with some unexported fields and for which package compute provides no factory function, you can't easily control how the iterator returned from List behaves in your tests. One way out is to declare an additional interface,
type Iterator interface {
Next() (*computev1.Address, error)
}
and change the result type of List from *compute.AddressIterator to Iterator:
type Lister interface {
List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator
}
Then you can declare another struct type for the real Lister and use that on the production side:
type RealLister struct {
Client *compute.AddressesClient
}
func (rl *RealLister) List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator {
return rl.Client.List(ctx, req, opts...)
}
func main() {
secret := "don't hardcode me"
ctx, cancel := context.WithCancel(context.Background()) // for instance
defer cancel()
c, err := compute.NewAddressesRESTClient(ctx, option.WithCredentialsFile(secret))
if err != nil {
log.Fatal(err) // or deal with the error in some way
}
defer c.Close()
cp := Compute{Lister: &RealLister{Client: c}}
if err := cp.Res(ctx, "my-project", "us-east-1", "my-vpc"); err != nil {
log.Fatal(err) // or deal with the error in some way
}
}
For your tests, you can declare another struct type that will act as a configurable test double:
type FakeLister func(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator
func (fl FakeLister) List(ctx context.Context, req *computev1.ListAddressesRequest, opts ...gax.CallOption) Iterator {
return fl(ctx, req, opts...)
}
To control the behaviour of the Iterator in your test, you can declare another configurable concrete type:
type FakeIterator struct{
Err error
Status string
}
func (fi *FakeIterator) Next() (*computev1.Address, error) {
addr := computev1.Address{Status: &fi.Status}
return &addr, fi.Err
}
A test function may look like this:
func TestResStatusInUse(t *testing.T) {
// Arrange
l := func(_ context.Context, _ *computev1.ListAddressesRequest, _ ...gax.CallOption) Iterator {
return &FakeIterator{
Status: "IN_USE",
Err: nil,
}
}
cp := Compute{Lister: FakeLister(l)}
dummyCtx := context.Background()
// Act
err := cp.Res(dummyCtx, "my-project", "us-east-1", "my-vpc")
// Assert
if err != nil {
// ...
}
}
I have a server implementation. Now I am writing unit test to check it's functionalities.
I cannot prepare request, that would unmarshall on the server side well. Code below results with InvalidUnmarshallError. I don't know, how to debug it further.
Client side code:
body := PatchCatRequest{Adopted: true}
bodyBuf := &bytes.Buffer{}
err := json.NewEncoder(bodyBuf).Encode(body)
assert.NoError(t, err)
req, err := http.NewRequest("PATCH", URL+"/"+catId, bodyBuf)
recorder := httptest.NewRecorder()
handler.PatchCat(recorder, req.WithContext(ctx))
Server side code:
type PatchCatRequest struct {
Adopted bool `json:"adopted"`
}
func (h *Handler) PatchCat (rw http.ResponseWriter, req *http.Request) {
var patchRequest *PatchCatRequest
if err := json.NewDecoder(req.Body).Decode(patchRequest); err != nil {
rw.WriteHeader(http.StatusBadRequest)
logger.WithField("error", err.Error()).Error(ErrDocodeRequest.Error())
return
}
...
}
You are unmarshaling into a nil pointer, as the error message says:
package main
import (
"encoding/json"
"fmt"
)
type PatchCatRequest struct {
Adopted bool
}
func main() {
var patchRequest *PatchCatRequest // nil pointer
err := json.Unmarshal([]byte(`{"Adopted":true}`), patchRequest)
fmt.Println(err) // json: Unmarshal(nil *main.PatchCatRequest)
}
https://play.golang.org/p/vt7t5BgT3lA
Initialize the pointer before unmarshaling:
func main() {
patchRequest := new(PatchCatRequest) // non-nil pointer
err := json.Unmarshal([]byte(`{"Adopted":true}`), patchRequest)
fmt.Println(err) // <nil>
}
https://play.golang.org/p/BqliguktWmr
I have this simple generic Request struct to make get requests in my app:
package api
import (
"net/http"
"time"
"log"
"app/errors"
)
type Request struct {
Url string
}
func (request *Request) Run(responseObject *AppStatusInfo) *errors.Error {
req, requestErr := http.NewRequest(http.MethodGet, request.Url, nil)
req.Header.Set("Content-Type", "application/json")
timeout := time.Duration(5 * time.Second)
client := &http.Client{
Timeout: timeout,
}
resp, requestErr := client.Do(req)
if requestErr != nil {
return &errors.UnknownError
}
decodeError := DecodeJsonRequestBody(resp, &responseObject)
if (decodeError != nil) {
return &errors.UnknownError
}
defer resp.Body.Close()
return nil
}
Here responseObject has pointer of type AppStatusInfo which is a struct with some fields.
I run it like this to get app status information and put it inside appStatusInfo object:
var appStatusInfo AppStatusInfo
req := Request{
Url:config.Config.ApiUrl,
}
req.Run(&appStatusInfo)
So, this code runs fine.
But, when I want to generalize Request to accept other types of responses, like UserProducts, I don't know how to do it without replacing responseObject *AppStatusInfo with responseObject interface{}, then casting it with responseObject.(UserProducts) which I think can be improved.
So, as soon as there are no generics, how do I make Request.Run() accept different types and return respective objects?
Assuming that DecodeJsonRequestBody passes the second argument to json.Unmarshal or json.Decoder.Decode, then write it like this. I show the changed lines only:
func (request *Request) Run(responseObject interface{}) *errors.Error {
...
resp, requestErr := client.Do(req)
if requestErr != nil {
return &errors.UnknownError
}
defer resp.Body.Close() // defer close before doing anything else
...
decodeError := DecodeJsonRequestBody(resp, responseObject) // don't take address of responseObject
...
}
You can call it like this:
var up UserProducts
err = r.Run(&up)
var asi AppStatusInfo
err = r.Run(&asi)
Type assertions and type conversions are not required.
I'm designing a API server in Go. I have many database tables, each with a matching struct. Each has a route and handler:
type Thing1 struct {
ID int64
Name string
...
}
func main() {
...
router := mux.NewRouter()
apiRouter := router.PathPrefix("/v1").Subrouter()
apiRouter.HandleFunc("/thing1/{id}", Thing1ShowHandler).Methods("GET")
}
func Thing1ShowHandler(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
id, err := strconv.ParseInt(vars["id"], 10, 64)
if err != nil {
errorHandler(w, err)
return
}
thing1 := Thing1{ID: id}
err = db.First(&thing1, id).Error
if thing1.ID > 0 {
jsonHeaders(w, http.StatusOK)
if err := json.NewEncoder(w).Encode(thing1); err != nil {
errorHandler(w, err)
}
return
}
notFoundHandler(w, r)
}
The code for Thing2 is pretty much identical, as it is for Thing3 and so on. I will end up with hundreds of things, and therefore lots of duplicated code. It feels like I'm doing something horribly wrong. What's the best way to make this more DRY?
Why not create a factory function for the http.Handler used with each Thing? This allows you to write the showHandler logic once and parameterize the instantiation of individual things.
// A ThingFactory returns a Thing struct configured with the given ID.
type ThingFactory func(id int64) interface{}
// The createShowHandler function is a factory function for creating a handler
// which uses the getThing factory function to obtain an instance of a
// thing to use when generating a view.
func createShowHandler(getThing ThingFactory) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
id, err := strconv.ParseInt(vars["id"], 10, 64)
if err != nil {
errorHandler(w, err)
return
}
thing := getThing(id)
err = db.First(&thing, id).Error
if err != nil {
errorHandler(w, err)
}
if thing1.ID > 0 {
jsonHeaders(w, http.StatusOK)
if err := json.NewEncoder(w).Encode(thing1); err != nil {
errorHandler(w, err)
}
return
}
notFoundHandler(w, r)
}
}
This function can be used to systematically create routes for a given router. For instance, I can create an explicit registry which keeps track of the path for each thing as well as a ThingFactory instance which is used when calling the createShowHandler factory function.
router := mux.NewRouter()
apiRouter := router.PathPrefix("/v1").Subrouter()
registry := []struct {
path string
handler ThingFactory
}{
{"/thing1/{id}", func(id int64) interface{} { return Thing1{ID: id} }},
{"/thing2/{id}", func(id int64) interface{} { return Thing2{ID: id} }},
{"/thing3/{id}", func(id int64) interface{} { return Thing3{ID: id} }},
}
for _, registrant := range registry {
apiRouter.HandleFunc(registrant.path, createShowHandler(registrant.handler)).Methods("GET")
}
Naturally, you would want to define interfaces for the various interaction points in a program like this to gain more type safety when dealing with a large number of instances. A more robust registry could be implemented that provided an interface for Things to register themselves with.