I'm writing a parser in Go and I have a following code for ast construction:
type NODE interface {
GetPosition() (int, int)
}
type EXPRESSION_NODE interface {
NODE
expressionNode()
}
// binary node
type BINARY_EXPRESSION struct {
Operator string
Left, Right EXPRESSION_NODE
Position
}
// implementing the EXPRESSION_NODE interface for BINARY_EXPRESSION
func (s BINARY_EXPRESSION) expressionNode()
func (s BINARY_EXPRESSION) GetPosition() (int, int) { return s.Line, s.Column }
So I have an interface EXPRESSION_NODE and BINARY_EXPRESSION struct and the struct implements all the interface methods
And here is my confusion:
func (self *Parser) ParseBinaryExpression(
operators []string,
parser ExpressionParser,
) (*EXPRESSION_NODE, error) {
self.SetPosition()
result, err := parser()
if err != nil {
return nil, ChainErrs(self.Err(BINARY_EXPRESSION_ERROR), err)
}
for Includes(operators, self.stream.Peek().Literal) {
operator := self.stream.Next().Literal
right, err := parser()
if err != nil {
return nil, ChainErrs(self.Err(BINARY_EXPRESSION_ERROR), err)
//ERROR: cannot use &(BINARY_EXPRESSION literal) (value of type *BINARY_EXPRESSION) as *EXPRESSION_NODE value in assignment
result = &BINARY_EXPRESSION{operator, result, right, self.position}
}
}
return result, nil
}
So this line
result = &BINARY_EXPRESSION{operator, result, right, self.position}
gives me the following error: cannot use &(BINARY_EXPRESSION literal) (value of type *BINARY_EXPRESSION) as *EXPRESSION_NODE value in assignment
So when I do
var a EXPRESSION_NODE = &BINARY_EXPRESSION{}
without pointer everything seems ok, but with pointers...
And the thing is I actually need to return *EXPRESSION_NODE from my function
[The code in your question is partial, and thus the following is a sketch, not intended as full working code]
Generally, you almost certainly never want to use a pointer to an interface. What you want instead is a pointer to your type to implement an interface.
First, make sure the pointer to your type implements the interface:
type BINARY_EXPRESSION struct {
Operator string
Left, Right EXPRESSION_NODE
Position
}
// Note that pointer receivers
func (s *BINARY_EXPRESSION) expressionNode()
func (s *BINARY_EXPRESSION) GetPosition() (int, int) { return s.Line, s.Column }
And then your parse function would be something like:
func (self *Parser) ParseBinaryExpression(
operators []string,
parser ExpressionParser,
) (EXPRESSION_NODE, error) {
// ... stuff ...
result = &BINARY_EXPRESSION{operator, result, right, self.position}
return result, nil
}
I recommend you learn more about Go interfaces and how to define methods that implement them:
https://golang.org/doc/faq#methods_on_values_or_pointers
https://tour.golang.org/methods/4
https://golangbyexample.com/pointer-vs-value-receiver-method-golang/
Related
I'm doing some experimentation with type parameters to come up with a generic way of wiring up structs that generate a response to JSON HTTP requests.
The Method interface which the structs must implement has a SetParams method. This will work as expected as long as the implementation uses a pointer receiver.
My question: Is there any way of making this a compile time error if SetParams has a value receiver?
Here is an example demonstrating the problem with a SetParams that has a value receiver:
package main
import (
"encoding/json"
"fmt"
"log"
)
type PingParams struct {
Name string
}
type PingResponse struct {
Message string
}
func (p PingParams) Greeting() string {
if p.Name != "" {
return fmt.Sprintf("Hello, %s", p.Name)
}
return fmt.Sprintf("Hello, nobody!")
}
type GoodPing struct {
Params PingParams
}
// SetParams has a pointer receiver.
func (m *GoodPing) SetParams(p PingParams) {
fmt.Printf("assign %v with pointer receiver, Good!\n", p)
m.Params = p
}
func (m GoodPing) Run() (*PingResponse, error) {
return &PingResponse{Message: fmt.Sprintf("%T %s", m, m.Params.Greeting())}, nil
}
type BadPing struct {
Params PingParams
}
// SetParams has a value receiver.
func (m BadPing) SetParams(p PingParams) {
fmt.Printf("assign %v with value receiver, Bad!\n", p)
m.Params = p
}
func (m BadPing) Run() (*PingResponse, error) {
return &PingResponse{Message: fmt.Sprintf("%T %s", m, m.Params.Greeting())}, nil
}
type Method[M, RQ, RS any] interface {
// Run builds the RPC result.
Run() (*RS, error)
// SetParams is intended to set the request parameters in the struct implementing the RPC method.
// This then allows the request parameters to be easily available to all methods of the Method struct.
// The method MUST have a pointer receiver. This is NOT enforced at compile time.
SetParams(p RQ)
// The following line requires the implementing type is a pointer to M.
*M
// https://stackoverflow.com/a/72090807
}
func HandlerMethod[M, RQ, RS any, T Method[M, RQ, RS]](in json.RawMessage) (*RS, error) {
// A real implementation of this would return a func for wiring into a request router
var req RQ
err := json.Unmarshal(in, &req)
if err != nil {
return nil, err
}
var m T = new(M)
m.SetParams(req)
return m.Run()
}
func main() {
payload := []byte(`{"Name": "Mark"}`)
bad, err := HandlerMethod[BadPing, PingParams, PingResponse](payload)
if err != nil {
log.Fatal(err)
}
fmt.Println(bad.Message)
good, err := HandlerMethod[GoodPing, PingParams, PingResponse](payload)
if err != nil {
log.Fatal(err)
}
fmt.Println(good.Message)
}
https://go.dev/play/p/Eii8ADkmDxE
You can't do that.
When in your code you do this:
var m T = new(M)
even if T's type set includes only *M as a type term, *M's method set includes methods declared on M. The compiler can't check for you how the method ends up in *M's method set.
It is your responsibility when declaring the method SetParam on BadPing to make sure that the method doesn't attempt to unfruitfully modify the receiver.
Consider following simplified example:
package main
import (
"fmt"
)
type IMessenger interface {
Message()
}
type TMyMessenger struct {
}
func (m TMyMessenger) Message() {}
func MessengerFactory() IMessenger {
return getInternalMessengerVariant()
}
func getInternalMessengerVariant() *TMyMessenger {
return nil
}
func main() {
e := MessengerFactory()
fmt.Println(" e == nil", e == nil) // *TMyMessenger(nil)
if e != nil {
e.Message()
}
}
And it's output:
e == nil false
panic: runtime error: invalid memory address or nil pointer dereference
Question 1:
Is there an idiomatic Go way to check if e points to a nil pointer?
Preferably an inline snippet.
Basically make the e != nil to be false even in the example case.
What I have considered:
There would not be this issue if getInternalMessengerVariant() would return Interface type instead of concrete pointer, but it requires refactor and may still go undetected and yield itself as a panic at runtime (if e != nil).
func getInternalMessengerVariant() IMessenger {
return nil
}
Rewrite MessengerFactory() to intercept the internal returns:
func MessengerFactory() IMessenger {
if m := getInternalMessengerVariant(); m != nil {
return m
}
return nil
}
Be very specific on type checking, but what if there are many types:
if e != nil && e != (*TMyMessenger)(nil) {
e.Message()
}
This problem exists whenever you return an interface from a function: if the interface contains a typed nil-pointer, interface itself is not nil. There is no easy way to check that.
A good way to deal with this is to return a nil for the interface:
func MessengerFactory() IMessenger {
x:= getInternalMessengerVariant()
if x==nil {
return nil
}
return x
}
Then you will not need to check if the return value points to a nil pointer.
Burak Serdar explains well in his answer why if x == nil returns false for you.
But that is not the reason why you get the panic.
Go is happy to invoke a receiver function on a nil pointer, as long as the receiver doesn't dereference the pointer.
This does not panic:
type TMyMessenger struct {
}
func (m *TMyMessenger) Message() {}
func main() {
var t *TMyMessenger = nil
t.Message()
}
and that's because you don't dereference the pointer m inside the Message receiver function.
Your example only panics because you have defined the receiver function m on the type TMyMessenger (not a pointer). Because of that, Go will have to dereference the nil pointer to TMyMessenger that is inside the IMessenger interface value, in order to invoke the receiver function.
If you change one line in your code, it will no longer panic:
func (m *TMyMessenger) Message() {}
(change (m TMyMessenger) to (m *TMyMessenger))
I'm looking for an appropriate way to inject dependencies.
Say I have this code where the FancyWrite and FancyRead functions have a dependency on the WriteToFile and ReadFromFile functions. Since these have side effects I'd like to be able to inject them so I can replace them in tests.
package main
func main() {
FancyWrite()
FancyRead()
}
////////////////
func FancyWrite() {
WriteToFile([]byte("content..."))
}
func FancyRead() {
ReadFromFile("/path/to/file")
}
////////////////
func WriteToFile(content []byte) (bool, error) {
return true, nil
}
func ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
One thing I tried is just put them as parameters into the functions:
package main
func main() {
FancyWrite(WriteToFile)
FancyRead(ReadFromFile)
}
////////////////
func FancyWrite(writeToFile func(content []byte) (bool, error)) {
writeToFile([]byte("content..."))
}
func FancyRead(readFromFile func(file string) ([]byte, error)) {
readFromFile("/path/to/file")
}
////////////////
func WriteToFile(content []byte) (bool, error) {
return true, nil
}
func ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
So, this actually works great, but I could see this becoming harder to maintain for more dependencies. I also tried a factory pattern like the following so that the main function doesn't have to concern itself with building the FancyWrite function. But, the syntax is getting a little hard to read and with even more functions would be hard to maintain.
func FancyWriteFactory(writeToFile func(content []byte) (bool, error)) func() {
return func() {
FancyWrite(writeToFile)
}
}
So next I tried housing the functions as methods in a struct:
package main
func main() {
dfu := DefaultFileUtil{}
ffm := FancyFileModule{
FileUtil: &dfu,
}
ffm.FancyWrite()
ffm.FancyRead()
}
////////////////
type FileUtil interface {
WriteToFile(content []byte) (bool, error)
ReadFromFile(file string) ([]byte, error)
}
type FancyFileModule struct {
FileUtil
}
func (fm *FancyFileModule) FancyWrite() {
fm.FileUtil.WriteToFile([]byte("content..."))
}
func (fm *FancyFileModule) FancyRead() {
fm.FileUtil.ReadFromFile("/path/to/file")
}
////////////////
type DefaultFileUtil struct{}
func (fu *DefaultFileUtil) WriteToFile(content []byte) (bool, error) {
return true, nil
}
func (fu *DefaultFileUtil) ReadFromFile(file string) ([]byte, error) {
return []byte{}, nil
}
Now, this actually works well and is cleaner. However, I'm worried I am just shoehorning my functions as methods now and something just felt odd about that. I guess I can reason about it because structs are good when you have some state, and I guess I can count the dependencies as state?
Those are the things I tried. So my question is, what is the proper way to do dependency injection in this case when the only reason to put functions as methods is to make them be a collection of dependencies elsewhere?
Thanks!
The simple answer is that you cannot cleanly use dependency injection with functions, only with methods. Technically, you could make the functions global vars instead (ex. var WriteToFile = func(content []byte) (bool, error) { [...] }), but this is rather brittle code.
The more proper solution, from an idiomatic perspective, is to make any behavior you want to replace, inject, or wrap into a method that is then wrapped in an interface.
For example:
type (
FancyReadWriter interface {
FancyWrite()
FancyRead()
}
fancyReadWriter struct {
w Writer
r Reader
}
Writer interface {
Write([]byte) (bool, error)
}
Reader interface {
Read() ([]byte, error)
}
fileWriter struct {
path string
// or f *os.File
}
fileReader struct {
path string
// or f *os.File
}
)
func (w fileWriter) Write([]byte) (bool, error) {
// Write to the file
return true, nil
}
func (r fileReader) Read() ([]byte, error) {
// Read from the file
return nil, nil
}
func (f fancyReadWriter) FancyWrite() {
// I like to be explicit when I'm ignoring return values,
// hence the underscores.
_, _ = f.w.Write([]byte("some content..."))
}
func (f fancyReadWriter) FancyRead() {
_, _ = f.r.Read()
}
func NewFancyReadWriter(w Writer, r Reader) FancyReadWriter {
// NOTE: Returning a pointer to the struct type, but it is actually
// returned as an interface instead, abstracting the underlying
// implementation.
return &fancyReadWriter{
w: w,
r: r,
}
}
func NewFileReader(path string) Reader {
// Same here, returning a pointer to the struct as the interface
return &fileReader {
path: path
}
}
func NewFileWriter(path string) Writer {
// Same here, returning a pointer to the struct as the interface
return &fileWriter {
path: path
}
}
func Main() {
w := NewFileWriter("/var/some/path")
r := NewFileReader("/var/some/other/path")
f := NewFancyReadWriter(w, r)
f.FancyWrite()
f.FancyRead()
}
And then in the test file (or wherever you want to do the dependency injection):
type MockReader struct {}
func (m MockReader) Read() ([]byte, error) {
return nil, fmt.Errorf("test error 1")
}
type MockWriter struct {}
func (m MockWriter) Write([]byte) (bool, error) {
return false, fmt.Errorf("test error 2")
}
func TestFancyReadWriter(t *testing.T) {
var w MockWriter
var r MockReader
f := NewFancyReadWriter(w, r)
// Now the methods on f will call the mock methods instead
f.FancyWrite()
f.FancyRead()
}
You could then go a step further and make the mock or injection framework functional and thus flexible. This is my preferred style for mocks for tests, actually, as it lets me define the behavior of the mocked dependency within the test using that behavior. Example:
type MockReader struct {
Readfunc func() ([]byte, error)
ReadCalled int
}
func (m *MockReader) Read() (ret1 []byte, ret2 error) {
m.ReadCalled++
if m.Readfunc != nil {
// Be *very* careful that you don't just call m.Read() here.
// That would result in an infinite recursion.
ret1, ret2 = m.Readfunc()
}
// if Readfunc == nil, this just returns the zero values
return
}
type MockWriter struct {
Writefunc func([]byte) (bool, error)
WriteCalled int
}
func (m MockWriter) Write(arg1 []byte) (ret1 bool, ret2 error) {
m.WriteCalled++
if m.Writefunc != nil {
ret1, ret2 = m.Writefunc(arg1)
}
// Same here, zero values if the func is nil
return
}
func TestFancyReadWriter(t *testing.T) {
var w MockWriter
var r MockReader
// Note that these definitions are optional. If you don't provide a
// definition, the mock will just return the zero values for the
// return types, so you only need to define these functions if you want
// custom behavior, like different returns or test assertions.
w.Writefunc = func(d []byte) (bool, error) {
// Whatever tests you want, like assertions on the input or w/e
// Then whatever returns you want to test how the caller handles it.
return false, nil
}
r.Readfunc = func() ([]byte, error) {
return nil, nil
}
// Since the mocks now define the methods as *pointer* receiver methods,
// so the mock can keep track of the number of calls, we have to pass in
// the address of the mocks rather than the mocks as struct values.
f := NewFancyReadWriter(&w, &r)
// Now the methods on f will call the mock methods instead
f.FancyWrite()
f.FancyRead()
// Now you have a simple way to assert that the calls happened:
if w.WriteCalled < 1 {
t.Fail("Missing expected call to Writer.Write().")
}
if r.ReadCalled < 1 {
t.Fail("Missing expected call to Reader.Read().")
}
}
Since all of the types involved here (the Reader, Writer, and the FancyReadWriter) are all handed around as interfaces rather than concrete types, it also becomes trivial to wrap them with middleware or similar (ex. logging, metrics/tracing, timeout aborts, etc).
This is hands down the most power strength of Go's interface system. Start thinking of types as bags of behavior, attach your behavior to types that can hold them, and pass all behavior types around as interfaces rather than concrete structs (data structs that are just used to organize specific bits of data are perfectly fine without interfaces, else you have to define Getters and Setters for everything and it's a real chore without much benefit). This lets you isolate, wrap, or entirely replace any particular bit of behavior you want at any time.
I am trying to write a simple custom marshaler and failing. Notice I have an interface that has three functions. Both Happy and Sad structs implement this interface by embedding the emotion struct which implements all the three required functions.
The problem is UnmarshalJSON does not get invoked when I call json.Unmarshal() on the pointer to either Happy or Sad and I can't understand why. You can reproduce the exact codebase in Go Playground or just look below. You will notice that while MarshalJSON is correctly called, UnmarshalJSON isn't.
type Emotion interface {
String() string
MarshalJSON() ([]byte, error)
UnmarshalJSON(data []byte) error
}
type emotion struct {
status string
}
func (s emotion) String() string {
return s.status
}
func (s emotion) MarshalJSON() ([]byte, error) {
fmt.Println("MarshalJSON is overriden: I am called fine")
x := struct {
Status string
}{
Status: s.String(),
}
return json.Marshal(x)
}
func (s *emotion) UnmarshalJSON(data []byte) error {
fmt.Println("MarshalJSON is overriden: I am never called")
y := struct {
Status string
}{
Status: "",
}
err := json.Unmarshal(data, &y)
if err != nil {
return err
}
s.status = y.Status
return nil
}
type Happy struct {
*emotion
}
// Job is not in any detention
type Sad struct {
*emotion
}
func main() {
x := Happy{&emotion{status: "happy"}}
jsonX, _ := json.Marshal(x)
var y Emotion
err := json.Unmarshal(jsonX, &y)
fmt.Printf("%v", err)
}
You cannot unmarshal into an abstract interface type.
An interface value is just a pointer to a type (associating that types methods) - it has no storage behind it - because an abstract type cannot know the exact size of any concrete value it may have in the future.
Using a concrete value type (that also implements that interface) will work:
y2 := emotion{}
err = json.Unmarshal(jsonX, &y2)
Playground: https://play.golang.org/p/8aCEjLgfKVQ
MarshalJSON is overriden: I am called fine
EXPECTED ERROR, Can't unmarshal into a non-concrete value: json: cannot unmarshal object into Go value of type main.Emotion
MarshalJSON is overriden: I am (fixed) and now called
SHOULD NOT ERROR: <nil>
VALUE: happy
Is it possible to have my function definition below accept any type of struct?
I've tried to refactor like so:
// This method should accept any type of struct
// Once I receive my response from the database,
// I scan the rows to create a slice of type struct.
func generateResponse(rows *sqlx.Rows, structSlice []struct{}, structBody struct{}) ([]struct{}, error) {
for rows.Next() {
err := rows.StructScan(&structBody)
if err != nil {
return nil, err
}
structSlice = append(structSlice, structBody)
}
err := rows.Err()
if err != nil {
return nil, err
}
return structSlice, nil
}
Assume my struct is of type OrderRevenue.
When I call the function above:
structSlice, err := generateResponse(rows, []OrderRevenue{}, OrderRevenue{})
The error I get is:
cannot use []OrderRevenue literal as type []struct{} in argument...
Am I going about this the wrong way?
This is considered the cornerstone (or more of a limitation) of Go's type system. struct{} is an unnamed type that is different from struct{ field1 int } and of course is not the same as OrderRevenue{}.
Go emphasizes abstraction through interfaces, and perhaps you should try that. Here is the first take:
type OrderRevenue interface {
MarshalMyself() ([]byte, error)
}
type Anonymous struct {}
func (a Anonymous) MarshalMyself() ([]byte, error) {
// implementation's up to you
return []byte{}, nil
}
// the function signature
generateResponse(rows *sqlx.Rows, structSlice []OrderRevenue, structBody Body) ([]Body, error) {
// ...
}
In this case you can also use empty interface interface{}, which all types implement, but you'll have to recursively go through the structure to do manual type assertion. The best approach in Go is to know the shape of your data in advance, at least partially.