This question already has answers here:
Pointer to an interface vs interface holding a pointer when unmarshaling JSON
(1 answer)
How to tell json.Unmarshal to use struct instead of interface
(1 answer)
Closed 3 months ago.
I have constructed a small example of what I am trying to do:
type Service interface {
Transform()
}
type XYZ struct {
ID string `json:"id"`
}
func (s XYZ) Transform() {}
func main() {
x := "could be anything"
msg := []byte(`{"id": "123"}`)
var msgS Service
switch x {
case "could be anything":
msgS = XYZ{}
break
case "could be something else":
// another type that implements Transform()
break
}
err := json.Unmarshal(msg, &msgS)
if err != nil {
fmt.Println(err)
}
// msgS.Transform()
}
Go Playground
This returns an error:
json: cannot unmarshal object into Go value of type main.Service
Essentially, I need to be flexible with whatever x might hold, hence the switch. If I instantitate msgSto the actual type that implements Transform(), all works as expected.
I realise I might need to reconsider my whole implementation - Any ideas would be welcome!
You should unmarshal to the concrete type and then you can assign the result to the interface type.
A simple example:
var service Service
var err error
switch x {
case "something":
var something Something // this implements Service
err = json.Unmarshal(data, &something)
service = something
case "something else":
var somethingElse SomethingElse // this implements Service
err = json.Unsmarshal(data, &somethingElse)
service = somethingElse
// etc...
}
if err != nil {
// handle error (here or inside of each case if you need/want to)
}
// use service as needed here
Just started learning generics. I'm making a command processor and I honestly don't know how to word this so I'm just going to show an example problem:
var ErrInvalidCommand = errors.New("invalid command")
type TransactionalFn[T any] func(ctx context.Context, db T) error
func NewTransactionalCommand[T any](fn TransactionalFn[T]) *TransactionalCommand[T] {
return &TransactionalCommand[T]{
fn: fn,
}
}
type TransactionalCommand[T any] struct {
fn TransactionalFn[T]
}
func (cmd *TransactionalCommand[T]) StartTransaction() error {
return nil
}
func (cmd *TransactionalCommand[T]) Commit() error {
return nil
}
func (cmd *TransactionalCommand[T]) Rollback() error {
return nil
}
type CMD interface{}
type CommandManager struct{}
func (m *CommandManager) Handle(ctx context.Context, cmd CMD) error {
switch t := cmd.(type) {
case *TransactionalCommand[any]:
return m.handleTransactionalCommand(ctx, t)
default:
fmt.Printf("%T\n", cmd)
return ErrInvalidCommand
}
}
func (m *CommandManager) handleTransactionalCommand(ctx context.Context, cmd *TransactionalCommand[any]) error {
if err := cmd.StartTransaction(); err != nil {
return err
}
if err := cmd.fn(ctx, nil); err != nil {
if err := cmd.Rollback(); err != nil {
return err
}
}
if err := cmd.Commit(); err != nil {
return err
}
return nil
}
// tests
type db struct{}
func (*db) Do() {
fmt.Println("doing stuff")
}
func TestCMD(t *testing.T) {
ctx := context.Background()
fn := func(ctx context.Context, db *db) error {
fmt.Println("test cmd")
db.Do()
return nil
}
tFn := bus.NewTransactionalCommand(fn)
mng := &bus.CommandManager{}
err := mng.Handle(ctx, tFn)
if err != nil {
t.Fatal(err)
}
}
mng.handle returns ErrInvalidCommand so the test fails because cmd is *TransactionalCommand[*db] and not *TransactionalCommand[any]
Let me give another, more abstract example:
type A[T any] struct{}
func (*A[T]) DoA() { fmt.Println("do A") }
type B[T any] struct{}
func (*B[T]) DoB() { fmt.Println("do B") }
func Handle(s interface{}) {
switch x := s.(type) {
case *A[any]:
x.DoA()
case *B[any]:
x.DoB()
default:
fmt.Printf("%T\n", s)
}
}
func TestFuncSwitch(t *testing.T) {
i := &A[int]{}
Handle(i) // expected to print "do A"
}
Why doesn't this switch statement case *A[any] match *A[int]?
How to make CommandManager.Handle(...) accept generic Commands?
Why does the generic type switch fail to compile?
This is in fact the result of an intentional decision of the Go team. It turned out that allowing type switches on parametrized types can cause confusion
In an earlier version of this design, we permitted using type assertions and type switches on variables whose type was a type parameter, or whose type was based on a type parameter. We removed this facility because it is always possible to convert a value of any type to the empty interface type, and then use a type assertion or type switch on that. Also, it was sometimes confusing that in a constraint with a type set that uses approximation elements, a type assertion or type switch would use the actual type argument, not the underlying type of the type argument (the difference is explained in the section on identifying the matched predeclared type)
From the Type Parameters Proposal
Let me turn the emphasized statement into code. If the type constraint uses type approximation (note the tildes)...
func PrintStringOrInt[T ~string | ~int](v T)
...and if there also was a custom type with int as the underlying type...
type Seconds int
...and if PrintOrString() is called with a Seconds parameter...
PrintStringOrInt(Seconds(42))
...then the switch block would not enter the int case but go right into the default case, because Seconds is not an int. Developers might expect that case int: matches the type Seconds as well.
To allow a case statement to match both Seconds and int would require a new syntax, like, for example,
case ~int:
As of this writing, the discussion is still open, and maybe it will result in an entirely new option for switching on a type parameter (such as, switch type T).
More details, please refer to proposal: spec: generics: type switch on parametric types
Trick: convert the type into 'any'
Luckily, we do not need to wait for this proposal to get implemented in a future release. There is a super simple workaround available right now.
Instead of switching on v.(type), switch on any(v).(type).
switch any(v).(type) {
...
This trick converts v into an empty interface{} (a.k.a. any), for which the switch happily does the type matching.
Source: A tip and a trick when working with generics
*A[any] does not match *A[int] because any is a static type, not a wildcard. Therefore instantiating a generic struct with different types yields different types.
In order to correctly match a generic struct in a type switch, you must instantiate it with a type parameter:
func Handle[T any](s interface{}) {
switch x := s.(type) {
case *A[T]:
x.DoA()
case *B[T]:
x.DoB()
default:
panic("no match")
}
}
Though in absence of other function arguments to infer T, you will have to call Handle with explicit instantiation. T won't be inferred from the struct alone.
func main() {
i := &A[int]{}
Handle[int](i) // expected to print "do A"
}
Playground: https://go.dev/play/p/2e5E9LSWPmk
However when Handle is actually a method, as in your database code, this has the drawback of choosing the type parameter when instantiating the receiver.
In order to improve the code here you can make Handle a top-level function:
func Handle[T any](ctx context.Context, cmd CMD) error {
switch t := cmd.(type) {
case *TransactionalCommand[T]:
return handleTransactionalCommand(ctx, t)
default:
fmt.Printf("%T\n", cmd)
return ErrInvalidCommand
}
}
Then you have the problem of how to supply the argument db T to the command function. For this, you might:
simply pass an additional *db argument to Handle and handleTransactionalCommand, which also helps with type parameter inference. Call as Handle(ctx, &db{}, tFn). Playground: https://go.dev/play/p/6WESb86KN5D
pass an instance of CommandManager (like solution above but *db is wrapped). Much more verbose, as it requires explicit instantiation everywhere. Playground: https://go.dev/play/p/SpXczsUM5aW
use a parametrized interface instead (like below). So you don't even have to type-switch. Playground: https://go.dev/play/p/EgULEIL6AV5
type CMD[T any] interface {
Exec(ctx context.Context, db T) error
}
I am leaning to write unit tests and I was wondering the correct way to unit test a basic http.get request.
I found an API online that returns fake data and wrote a basic program that gets some user data and prints out an ID:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
)
type UserData struct {
Meta interface{} `json:"meta"`
Data struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Gender string `json:"gender"`
Status string `json:"status"`
} `json:"data"`
}
func main() {
resp := sendRequest()
body := readBody(resp)
id := unmarshallData(body)
fmt.Println(id)
}
func sendRequest() *http.Response {
resp, err := http.Get("https://gorest.co.in/public/v1/users/1841")
if err != nil {
log.Fatalln(err)
}
return resp
}
func readBody(resp *http.Response) []byte {
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
return body
}
func unmarshallData(body []byte) int {
var userData UserData
json.Unmarshal(body, &userData)
return userData.Data.ID
}
This works and prints out 1841. I then wanted to write some tests that validate that the code is behaving as expected, e.g. that it correctly fails if an error is returned, that the data returned can be unmarshalled. I have been reading online and looking at examples but they are all far more complex that what I feel I am trying to achieve.
I have started with the following test that ensures that the data passed to the unmarshallData function can be unmarshalled:
package main
import (
"testing"
)
func Test_unmarshallData(t *testing.T) {
type args struct {
body []byte
}
tests := []struct {
name string
args args
want int
}{
{name: "Unmarshall", args: struct{ body []byte }{body: []byte("{\"meta\":null,\"data\":{\"id\":1841,\"name\":\"Piya\",\"email\":\"priya#gmai.com\",\"gender\":\"female\",\"status\":\"active\"}}")}, want: 1841},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := unmarshallData(tt.args.body); got != tt.want {
t.Errorf("unmarshallData() = %v, want %v", got, tt.want)
}
})
}
}
Any advise on where to go from here would be appreciated.
before moving on to the testing, your code has a serious flow, which will become a problem if you don't take care about it in your future programming tasks.
https://pkg.go.dev/net/http See the second example
The client must close the response body when finished with it
Let's fix that now (we will have to come back on this subject later), two possibilities.
1/ within main, use defer to Close that resource after you have drained it;
func main() {
resp := sendRequest()
defer body.Close()
body := readBody(resp)
id := unmarshallData(body)
fmt.Println(id)
}
2/ Do that within readBody
func readBody(resp *http.Response) []byte {
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalln(err)
}
return body
}
Using a defer is the expected manner to close the resource. It helps the reader to identify the lifetime span of the resource and improve readability.
Notes : I will not be using much of the table test driven pattern, but you should, like you did in your OP.
Moving on to the testing part.
Tests can be written under the same package or its fellow version with a trailing _test, such as [package target]_test. This has implications in two ways.
Using a separate package, they will be ignored in the final build. Which will help to produce smaller binaries.
Using a separate package, you test the API in a black box manner, you can access only the identifiers it explicitly exposes.
Your current tests are white boxed, meaning you can access any declaration of main, public or not.
About sendRequest, writing a test around this is not very interesting because it does too little, and your tests should not be written to test the std library.
But for the sake of the demonstration, and for good reasons we might want to not rely on external resources to execute our tests.
In order to achieve that we must make the global dependencies consumed within it, an injected dependency. So that later on, it is possible to replace the one thing it depends on to react, the http.Get method.
func sendRequest(client interface{Get() (*http.Response, error)}) *http.Response {
resp, err := client.Get("https://gorest.co.in/public/v1/users/1841")
if err != nil {
log.Fatalln(err)
}
return resp
}
Here i use an inlined interface declaration interface{Get() (*http.Response, error)}.
Now we can add a new test which injects a piece of code that will return exactly the values that will trigger the behavior we want to test within our code.
type fakeGetter struct {
resp *http.Response
err error
}
func (f fakeGetter) Get(u string) (*http.Response, error) {
return f.resp, f.err
}
func TestSendRequestReturnsNilResponseOnError(t *testing.T) {
c := fakeGetter{
err: fmt.Errorf("whatever error will do"),
}
resp := sendRequest(c)
if resp != nil {
t.Fatal("it should return a nil response when an error arises")
}
}
Now run this test and see the result. It is not conclusive because your function contains a call to log.Fatal, which in turns executes an os.Exit; We cannot test that.
If we try to change that, we might think we might call for panic instead because we can recover.
I don't recommend doing that, in my opinion, this is smelly and bad, but it exists, so we might consider. This is also the least possible change to the function signature. Returning an error would break even more the current signatures. I want to minimize this for that demonstration. But, as a rule of thumb, return an error and always check them.
In the sendRequest function, replace this call log.Fatalln(err) with panic(err) and update the test to capture the panic.
func TestSendRequestReturnsNilResponseOnError(t *testing.T) {
var hasPanicked bool
defer func() {
_ = recover() // if you capture the output value or recover, you get the error gave to the panic call. We have no use of it.
hasPanicked = true
}()
c := fakeGetter{
err: fmt.Errorf("whatever error will do"),
}
resp := sendRequest(c)
if resp != nil {
t.Fatal("it should return a nil response when an error arises")
}
if !hasPanicked {
t.Fatal("it should have panicked")
}
}
We can now move on to the other execution path, the non error return.
For that we forge the desired *http.Response instance we want to pass into our function, we will then check its properties to figure out if what the function does is inline with what we expect.
We will consider we want to ensure it is returned unmodified : /
Below test only sets two properties, and I will do it to demonstrate how to set the Body with a NopCloser and strings.NewReader as it is often needed later on using the Go language;
I also use reflect.DeepEqual as brute force equality checker, usually you can be more fine grained and get better tests. DeepEqual does the job in this case but it introduces complexity that does not justify systematic use of it.
func TestSendRequestReturnsUnmodifiedResponse(t *testing.T) {
c := fakeGetter{
err: nil,
resp: &http.Response{
Status: http.StatusOK,
Body: ioutil.NopCloser(strings.NewReader("some text")),
},
}
resp := sendRequest(c)
if !reflect.DeepEqual(resp, c.resp) {
t.Fatal("the response should not have been modified")
}
}
At that point you may have figured that this small function sendRequest is not good, if you did not I ensure you it is not. It does too little, it merely wraps the http.Get method and its testing is of little interest for the survival of the business logic.
Moving on to readBody function.
All remarks that applied for sendRequest apply here too.
it does too little
it os.Exits
One thing does not apply. As the call to ioutil.ReadAll does not rely on external resources, there is no point in attempting to inject that dependency. We can test around.
Though, for the sake of the demonstration, it is the time to talk about the missing call to defer resp.Body.Close().
Let us assume we go for the second proposition made in introduction and test for that.
The http.Response struct adequately exposes its Body recipient as an interface.
To ensure the code calls for the `Close, we can write a stub for it.
That stub will record if that call was made, the test can then check for that and trigger an error if it was not.
type closeCallRecorder struct {
hasClosed bool
}
func (c *closeCallRecorder) Close() error {
c.hasClosed = true
return nil
}
func (c *closeCallRecorder) Read(p []byte) (int, error) {
return 0, nil
}
func TestReadBodyCallsClose(t *testing.T) {
body := &closeCallRecorder{}
res := &http.Response{
Body: body,
}
_ = readBody(res)
if !body.hasClosed {
t.Fatal("the response body was not closed")
}
}
Similarly, and for the sake of the demonstration, we might want to test if the function has called for Read.
type readCallRecorder struct {
hasRead bool
}
func (c *readCallRecorder) Read(p []byte) (int, error) {
c.hasRead = true
return 0, nil
}
func TestReadBodyHasReadAnything(t *testing.T) {
body := &readCallRecorder{}
res := &http.Response{
Body: ioutil.NopCloser(body),
}
_ = readBody(res)
if !body.hasRead {
t.Fatal("the response body was not read")
}
}
We an also verify the body was not modified in betwen,
func TestReadBodyDidNotModifyTheResponse(t *testing.T) {
want := "this"
res := &http.Response{
Body: ioutil.NopCloser(strings.NewReader(want)),
}
resp := readBody(res)
if got := string(resp); want != got {
t.Fatal("invalid response, wanted=%q got %q", want, got)
}
}
We have almost done, lets move one to the unmarshallData function.
You have already wrote a test about it. It is okish, though, i would write it this way to make it leaner:
type UserData struct {
Meta interface{} `json:"meta"`
Data Data `json:"data"`
}
type Data struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
Gender string `json:"gender"`
Status string `json:"status"`
}
func Test_unmarshallData(t *testing.T) {
type args struct {
body []byte
}
tests := []UserData{
UserData{Data: Data{ID: 1841}},
}
for _, u := range tests {
want := u.ID
b, _ := json.Marshal(u)
t.Run("Unmarshal", func(t *testing.T) {
if got := unmarshallData(b); got != want {
t.Errorf("unmarshallData() = %v, want %v", got, want)
}
})
}
}
Then, the usual apply :
don't log.Fatal
what are you testing ? the marshaller ?
Finally, now that we have gathered all those pieces, we can refactor to write a more sensible function and re use all those pieces to help us testing such code.
I won't do it, but here is a starter, which still panics, and I still don't recommend, but the previous demonstration has shown everything needed to test a version of it that returns an error.
type userFetcher struct {
Requester interface {
Get(u string) (*http.Response, error)
}
}
func (u userFetcher) Fetch() int {
resp, err := u.Requester.Get("https://gorest.co.in/public/v1/users/1841") // it does not really matter that this string is static, using the requester we can mock the response, its body and the error.
if err != nil {
panic(err)
}
defer resp.Body.Close() //always.
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}
var userData UserData
err = json.Unmarshal(body, &userData)
if err != nil {
panic(err)
}
return userData.Data.ID
}
Can somebody explain how can this happened?
I put interface as parameter in a function. While invoking this function I pass struct into it, but it didn't give me error. Here's the code
package main
import (
"fmt"
"github.com/myusername/gomodel/domain"
"github.com/myusername/gomodel/model"
)
func main() {
db := model.InitDB()
newFunc(db)
}
func newFunc(db domain.IUser) {
r, err := db.CreateUserTable()
if err != nil {
fmt.Println("error", err)
}
fmt.Println(r)
}
I've implemented the interface somewhere else in the code, because the program just work as the implemented interface expected to be.
IUser is an interface whose member is:
type IUser interface {
CreateUserTable() (sql.Result, error)
}
InitDB is a function to open the database and return struct of database:
type DB struct {
*sql.DB
}
//InitDB initializes the database
func InitDB() *DB {
db, err := sql.Open(dbDriver, dbName)
if err != nil {
log.Fatal("failed to initialize database: ",err)
}
err2 := db.Ping()
if err2 != nil {
log.Fatal(err2)
}
return &DB{db}
}
My question is: how can a function with a parameter type interface be passed a different type of parameter? And how is this working under the hood?
As per Golang Spec
An interface type specifies a method set called its interface. A
variable of interface type can store a value of any type with a method
set that is any superset of the interface. Such a type is said to
implement the interface.
This is because interface can be implemented as a wrapper to every type. Interface actually points to two things mainly one is the underlying type which is a struct here and other one is the value of that type which is a pointer to DB
You see newFunc is actually taking interface{} as an argument, So you can pass anything to it of type T which can be of primitive types too.
func main() {
db := model.InitDB()
newFunc(db)
}
So In case you want to get the underlying value you need to type assert. Interface works like a wrapper to the struct here and save its type and value which can be get using type assertion.
I'm trying refactor some code to use dependency injection for the Docker client library I use in my code. I created an interface with the method I want to be able to mock
type DockerClient interface {
Ping(context.Context) (types.Ping, error)
}
func NewDockerUtil() (*DockerUtil, error) {
var dockerClient *DockerClient
var err error
dockerClient, err = client.NewEnvClient() //Reports incompatible types in binary and unary expressions.
if err != nil {
return nil, err
}
return &DockerUtil{
Client: dockerClient,
}, nil
}
type DockerUtil struct{
Client *DockerClient
}
But when I try to assign it I get Reports incompatible types in binary and unary expressions. What exactly do I need to change?
Let's start with using interface{}, when you define interface don't use pointer definition, good read here.
type DockerUtil struct{
Client DockerClient
}
And moby Client implements lot of methods and you would like to do interface for selective methods.
Right way to do is via Type assertion. Good read Effective Go - Type assertions and Spec - Type assertion.
Note: try this code, I don't have docker env in my machine to test.
func NewDockerUtil() (*DockerUtil, error) {
dockerClient, err := client.NewEnvClient()
if err != nil {
return nil, err
}
return &DockerUtil{
Client: dockerClient.(DockerClient),
}, nil
}
Note:
Using DockerUtil.Client, you can call only Ping method since your interface DockerClient has definition of Ping method.
If you would like to call all the methods supported by client.Client later on then you have to do type assertion-
dockerClient := DockerUtil.Client.(*client.Client)