I am reading the source code for Go's io package and I came across a snippet I don't fully understand. Here it is
func WriteString(w Writer, s string) (n int, err error) {
if sw, ok := w.(stringWriter); ok {
return sw.WriteString(s)
}
return w.Write([]byte(s))
}
where
type stringWriter interface {
WriteString(s string) (n int, err error)
}
type Writer interface {
Write(p []byte) (n int, err error)
}
The type assertion w.(stringWriter) asserts that the dynamic type of w (i.e. Writer) implements the stringWriter interface. I don't see how this is possible given the type definitions of stringWriter and Writer. Assuming that this code is correct, what is it that I am missing?
You can easily build a type which implements Writer as well as stringWriter:
type StringWriter struct{}
func (s StringWriter) Write(in []byte) (int, error) {
return 0, nil
}
func (s StringWriter) WriteString(s string) (int, error) {
return s.Write([]byte(s))
}
So passing an instance of StringWriter to io.WriteString results in StringWriter.WriteString
being called.
The logic behind io.WriteString is: Use WriteString on the writer if it has such a method
or fall back to string to []byte conversion otherwise. To test whether the instance implements
a method or not, interfaces are used, as these are just method sets and can be tested for easily.
Related
I'm looking for an appropriate way to inject dependencies.
Say I have this code where the FancyWrite and FancyRead functions have a dependency on the WriteToFile and ReadFromFile functions. Since these have side effects I'd like to be able to inject them so I can replace them in tests.
package main
func main() {
FancyWrite()
FancyRead()
}
////////////////
func FancyWrite() {
WriteToFile([]byte("content..."))
}
func FancyRead() {
ReadFromFile("/path/to/file")
}
////////////////
func WriteToFile(content []byte) (bool, error) {
return true, nil
}
func ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
One thing I tried is just put them as parameters into the functions:
package main
func main() {
FancyWrite(WriteToFile)
FancyRead(ReadFromFile)
}
////////////////
func FancyWrite(writeToFile func(content []byte) (bool, error)) {
writeToFile([]byte("content..."))
}
func FancyRead(readFromFile func(file string) ([]byte, error)) {
readFromFile("/path/to/file")
}
////////////////
func WriteToFile(content []byte) (bool, error) {
return true, nil
}
func ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
So, this actually works great, but I could see this becoming harder to maintain for more dependencies. I also tried a factory pattern like the following so that the main function doesn't have to concern itself with building the FancyWrite function. But, the syntax is getting a little hard to read and with even more functions would be hard to maintain.
func FancyWriteFactory(writeToFile func(content []byte) (bool, error)) func() {
return func() {
FancyWrite(writeToFile)
}
}
So next I tried housing the functions as methods in a struct:
package main
func main() {
dfu := DefaultFileUtil{}
ffm := FancyFileModule{
FileUtil: &dfu,
}
ffm.FancyWrite()
ffm.FancyRead()
}
////////////////
type FileUtil interface {
WriteToFile(content []byte) (bool, error)
ReadFromFile(file string) ([]byte, error)
}
type FancyFileModule struct {
FileUtil
}
func (fm *FancyFileModule) FancyWrite() {
fm.FileUtil.WriteToFile([]byte("content..."))
}
func (fm *FancyFileModule) FancyRead() {
fm.FileUtil.ReadFromFile("/path/to/file")
}
////////////////
type DefaultFileUtil struct{}
func (fu *DefaultFileUtil) WriteToFile(content []byte) (bool, error) {
return true, nil
}
func (fu *DefaultFileUtil) ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
Now, this actually works well and is cleaner. However, I'm worried I am just shoehorning my functions as methods now and something just felt odd about that. I guess I can reason about it because structs are good when you have some state, and I guess I can count the dependencies as state?
Those are the things I tried. So my question is, what is the proper way to do dependency injection in this case when the only reason to put functions as methods is to make them be a collection of dependencies elsewhere?
Thanks!
The simple answer is that you cannot cleanly use dependency injection with functions, only with methods. Technically, you could make the functions global vars instead (ex. var WriteToFile = func(content []byte) (bool, error) { [...] }), but this is rather brittle code.
The more proper solution, from an idiomatic perspective, is to make any behavior you want to replace, inject, or wrap into a method that is then wrapped in an interface.
For example:
type (
FancyReadWriter interface {
FancyWrite()
FancyRead()
}
fancyReadWriter struct {
w Writer
r Reader
}
Writer interface {
Write([]byte) (bool, error)
}
Reader interface {
Read() ([]byte, error)
}
fileWriter struct {
path string
// or f *os.File
}
fileReader struct {
path string
// or f *os.File
}
)
func (w fileWriter) Write([]byte) (bool, error) {
// Write to the file
return true, nil
}
func (r fileReader) Read() ([]byte, error) {
// Read from the file
return nil, nil
}
func (f fancyReadWriter) FancyWrite() {
// I like to be explicit when I'm ignoring return values,
// hence the underscores.
_, _ = f.w.Write([]byte("some content..."))
}
func (f fancyReadWriter) FancyRead() {
_, _ = f.r.Read()
}
func NewFancyReadWriter(w Writer, r Reader) FancyReadWriter {
// NOTE: Returning a pointer to the struct type, but it is actually
// returned as an interface instead, abstracting the underlying
// implementation.
return &fancyReadWriter{
w: w,
r: r,
}
}
func NewFileReader(path string) Reader {
// Same here, returning a pointer to the struct as the interface
return &fileReader {
path: path
}
}
func NewFileWriter(path string) Writer {
// Same here, returning a pointer to the struct as the interface
return &fileWriter {
path: path
}
}
func Main() {
w := NewFileWriter("/var/some/path")
r := NewFileReader("/var/some/other/path")
f := NewFancyReadWriter(w, r)
f.FancyWrite()
f.FancyRead()
}
And then in the test file (or wherever you want to do the dependency injection):
type MockReader struct {}
func (m MockReader) Read() ([]byte, error) {
return nil, fmt.Errorf("test error 1")
}
type MockWriter struct {}
func (m MockWriter) Write([]byte) (bool, error) {
return false, fmt.Errorf("test error 2")
}
func TestFancyReadWriter(t *testing.T) {
var w MockWriter
var r MockReader
f := NewFancyReadWriter(w, r)
// Now the methods on f will call the mock methods instead
f.FancyWrite()
f.FancyRead()
}
You could then go a step further and make the mock or injection framework functional and thus flexible. This is my preferred style for mocks for tests, actually, as it lets me define the behavior of the mocked dependency within the test using that behavior. Example:
type MockReader struct {
Readfunc func() ([]byte, error)
ReadCalled int
}
func (m *MockReader) Read() (ret1 []byte, ret2 error) {
m.ReadCalled++
if m.Readfunc != nil {
// Be *very* careful that you don't just call m.Read() here.
// That would result in an infinite recursion.
ret1, ret2 = m.Readfunc()
}
// if Readfunc == nil, this just returns the zero values
return
}
type MockWriter struct {
Writefunc func([]byte) (bool, error)
WriteCalled int
}
func (m MockWriter) Write(arg1 []byte) (ret1 bool, ret2 error) {
m.WriteCalled++
if m.Writefunc != nil {
ret1, ret2 = m.Writefunc(arg1)
}
// Same here, zero values if the func is nil
return
}
func TestFancyReadWriter(t *testing.T) {
var w MockWriter
var r MockReader
// Note that these definitions are optional. If you don't provide a
// definition, the mock will just return the zero values for the
// return types, so you only need to define these functions if you want
// custom behavior, like different returns or test assertions.
w.Writefunc = func(d []byte) (bool, error) {
// Whatever tests you want, like assertions on the input or w/e
// Then whatever returns you want to test how the caller handles it.
return false, nil
}
r.Readfunc = func() ([]byte, error) {
return nil, nil
}
// Since the mocks now define the methods as *pointer* receiver methods,
// so the mock can keep track of the number of calls, we have to pass in
// the address of the mocks rather than the mocks as struct values.
f := NewFancyReadWriter(&w, &r)
// Now the methods on f will call the mock methods instead
f.FancyWrite()
f.FancyRead()
// Now you have a simple way to assert that the calls happened:
if w.WriteCalled < 1 {
t.Fail("Missing expected call to Writer.Write().")
}
if r.ReadCalled < 1 {
t.Fail("Missing expected call to Reader.Read().")
}
}
Since all of the types involved here (the Reader, Writer, and the FancyReadWriter) are all handed around as interfaces rather than concrete types, it also becomes trivial to wrap them with middleware or similar (ex. logging, metrics/tracing, timeout aborts, etc).
This is hands down the most power strength of Go's interface system. Start thinking of types as bags of behavior, attach your behavior to types that can hold them, and pass all behavior types around as interfaces rather than concrete structs (data structs that are just used to organize specific bits of data are perfectly fine without interfaces, else you have to define Getters and Setters for everything and it's a real chore without much benefit). This lets you isolate, wrap, or entirely replace any particular bit of behavior you want at any time.
Function below describes how to mock using testify. args.Bool(0), args.Error(1) are mocked positional return values.
func (m *MyMockedObject) DoSomething(number int) (bool, error) {
args := m.Called(number)
return args.Bool(0), args.Error(1)
}
Is it possible to return anything other than args.Int(), args.Bool(), args.String()? What if I need to return int64, or a custom struct. Is there a method or am I missing something?
For example:
func (m *someMock) doStuff(p *sql.DB, id int) (res int64, err error)
Yes, it is possible by using args.Get and type assertion.
From the docs:
// For objects of your own type, use the generic Arguments.Get(index) method and make a type assertion:
//
// return args.Get(0).(*MyObject), args.Get(1).(*AnotherObjectOfMine)
So, your example would be:
func (m *someMock) doStuff(p *sql.DB, id int) (res int64, err error) {
args := m.Called(p, id)
return args.Get(0).(int64), args.Error(1)
}
Additionaly, if your return value is a pointer (e.g. pointer to struct), you should check if it is nil before performing type assertion.
I am trying to write a simple custom marshaler and failing. Notice I have an interface that has three functions. Both Happy and Sad structs implement this interface by embedding the emotion struct which implements all the three required functions.
The problem is UnmarshalJSON does not get invoked when I call json.Unmarshal() on the pointer to either Happy or Sad and I can't understand why. You can reproduce the exact codebase in Go Playground or just look below. You will notice that while MarshalJSON is correctly called, UnmarshalJSON isn't.
type Emotion interface {
String() string
MarshalJSON() ([]byte, error)
UnmarshalJSON(data []byte) error
}
type emotion struct {
status string
}
func (s emotion) String() string {
return s.status
}
func (s emotion) MarshalJSON() ([]byte, error) {
fmt.Println("MarshalJSON is overriden: I am called fine")
x := struct {
Status string
}{
Status: s.String(),
}
return json.Marshal(x)
}
func (s *emotion) UnmarshalJSON(data []byte) error {
fmt.Println("MarshalJSON is overriden: I am never called")
y := struct {
Status string
}{
Status: "",
}
err := json.Unmarshal(data, &y)
if err != nil {
return err
}
s.status = y.Status
return nil
}
type Happy struct {
*emotion
}
// Job is not in any detention
type Sad struct {
*emotion
}
func main() {
x := Happy{&emotion{status: "happy"}}
jsonX, _ := json.Marshal(x)
var y Emotion
err := json.Unmarshal(jsonX, &y)
fmt.Printf("%v", err)
}
You cannot unmarshal into an abstract interface type.
An interface value is just a pointer to a type (associating that types methods) - it has no storage behind it - because an abstract type cannot know the exact size of any concrete value it may have in the future.
Using a concrete value type (that also implements that interface) will work:
y2 := emotion{}
err = json.Unmarshal(jsonX, &y2)
Playground: https://play.golang.org/p/8aCEjLgfKVQ
MarshalJSON is overriden: I am called fine
EXPECTED ERROR, Can't unmarshal into a non-concrete value: json: cannot unmarshal object into Go value of type main.Emotion
MarshalJSON is overriden: I am (fixed) and now called
SHOULD NOT ERROR: <nil>
VALUE: happy
The package 'gopkg.in/redis.v3' contains some code
type Client struct {
}
func (*client) Eval (string, []string, []string) *Cmd {
}
type Cmd struct {
}
func (*Cmd) Result () (interface{}, error) {
}
Which works successfully in the following way
func myFunc (cli *redis.Client) {
result, err := cli.Eval('my script').Result()
}
The problem is that sometimes the Redis cluster gets hammered, has a moment, and the interface returned as a result is nil.
This is reasonably easy to handle but I wish to put a test in place that will ensure that it is actually handled and no type assertion panic occurs.
Traditionally I would insert a mock Redis client into myFunc that can ultimately return nil.
type redisClient interface {
Eval(string, []string, []string) redisCmd
}
type redisCmd interface {
Result() (interface{}, error)
}
func myFunc (cli redisClient) {
result, err := cli.Eval('my script').Result()
}
The problem I am facing is the compiler doesn't recognise that redis.Client satisfies the interface redisClient because it doesn't recognise that the redis.Cmd returned from Eval satisfies redisCmd.
> cannot use client (type *redis.Client) as type redisClient in argument to myFunc:
> *redis.Client does not implement redisClient (wrong type for Eval method)
> have Eval(sting, []string, []string) *redis.Cmd
> want Eval(sting, []string, []string) redisCmd
The problem is that your interface does not match the redis client. If you change the interface to:
type redisClient interface {
Eval(string, []string, []string) *redis.Cmd
}
it will compile. That being said, it looks like you really want rediscmd, so you will need to make a wrapper around the redis client:
type wrapper struct{
c *redis.Client
}
func (w wrapper) Eval(x sting, y []string, z []string) redisCmd {
return w.c.Eval(x,y,z) // This assumes that *redis.Cmd implements rediscmd
}
I have created a new declared type and added a method to marshal the value into JSON
type TextOutput string
func (t *TextOutput) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`{"data": "%s"}`, t)), nil
}
When I try to marshal an instance of the type I get the raw value returned. What am I missing?
var t TextOutput
t = `Test test`
output, err := json.Marshal(t)
if err != nil {
fmt.Println(err)
} else {
fmt.Println(string(output))
}
// prints Test Test. Expected {"data": "Test test"}
You have to define the MarshalJSON interface as a non-pointer.
func (t TextOutput) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`{"data": "%s"}`, t)), nil
}
Play Link: https://play.golang.org/p/lLK6zsAkOi
Output:
{"data":"Test test"}
The root of the problem stems from how interfaces in Go are implicitly satisfied.
In this particular case, the json.Marshal method uses type assertion at runtime to see if the given value implements json.Marshaler. Effective Go mentions this very case.
You could have satisfied the json.Marshaler for the *TextOutput type using a pointer-receiver like so:
func (t *TextOutput) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`{"data":"%s"}`, *t)), nil
}
And for this to work properly, pass the reference to the json.Marshal function:
var t TextOutput
t = `Test test`
output, err := json.Marshal(&t)
However, implementing it using a value-receiver ensures that both TextOutput and *TextOutput types implement json.Marshaler
func (t TextOutput) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`{"data": "%s"}`, t)), nil
}