If you create a variable of type bytes.Buffer (without initialization) and assign it to a field of type io.Reader, then after checking io.Reader for nil there will be an error: invalid memory address or nil pointer dereference. How to check this correctly to avoid such errors?
Playground
package main
import (
"bytes"
"io"
"io/ioutil"
)
type Request struct {
Body io.Reader
}
func main() {
var data *bytes.Buffer
request := &Request{
Body: data,
}
if request.Body != nil {
ioutil.ReadAll(request.Body) // panic: runtime error: invalid memory address or nil pointer dereference
}
}
To check if an io.Reader (or any other interface) value is nil, you simply compare it to nil.
Whether a non-nil io.Reader is a meaningful implementation, that's another question.
E.g. is this implementation meaningful?
type panicReader struct{}
func (panicReader) Read(p []byte) (int, error) {
panic("foo")
}
panicReader certainly implements io.Reader, but whenever you call its Read() method, it will always panic.
There is bytes.Buffer. A pointer to it implements io.Reader. But calling Buffer.Read() on a nil *bytes.Buffer pointer value will panic. But not because you can't call methods on nil pointer receivers, but because the implementation of bytes.Buffer.Read() tries to dereference the pointer receiver, and this dereference operation is what causes the panic:
// Excerpt from bytes.Buffer.Read implementation
func (b *Buffer) Read(p []byte) (n int, err error) {
b.lastRead = opInvalid
if b.empty() {
// ...
}
You can't make a general conclusion here (just yet). See this io.Reader implementation:
type myBuffer struct{}
var count int
func (*myBuffer) Read(p []byte) (int, error) {
if len(p) > 0 {
count++
if count >= 10 {
return 0, io.EOF
}
p[0] = 'a'
return 1, nil
}
return 0, nil
}
*myBuffer implements io.Reader, and its Read() method does not use the pointer receiver value. What does this mean? You can call Read() on a nil *myBuffer value:
var data *myBuffer
request := &Request{
Body: data,
}
if request.Body != nil {
data, err := ioutil.ReadAll(request.Body)
fmt.Println(string(data), err)
}
This will output (try it on the Go Playground):
aaaaaaaaa <nil>
So the conclusion is this: usually types that have methods with pointer receiver require a non-nil pointer because they use the pointed object (in case of bytes.Buffer they use the fields of the pointed struct). To use such types (to have a meaningful implementation of implemented interfaces), you often need a non-nil pointer value for the methods to "work". This–however–is not always a requirement as the above myBuffer implementation shows. It's your job to always read the documentation of the used types and methods to avoid such misuses (e.g. trying to use a nil *bytes.Buffer).
See related question:
Hiding nil values, understanding why Go fails here
Go reflection with interface embedded in struct - how to detect "real" functions?
Related
I have an io.ReadWriteCloser in which I want to peek into it without advancing the reader,
so I am using
bi := bufio.NewReader(i)
bi.Peek(1)
So far so good, but later when I want to reuse the original io.ReadWriteCloser (i) it has only EOF.
So my question is how to convert back from bufio.Reader back to io.ReadWriteCloser
Because the bufio.Reader buffers data from the underlying reader, the application must read from the bufio.Reader after the call to Peek.
To get an io.ReadWriteCloser that does this, wrap the bufio.Reader and the original io.ReadWriteCloser:
// BufferedReadWriteCloser has all of the methods
// from *bufio.Reader and io.ReadWriteCloser.
type BufferedReadWriteCloser struct {
*bufio.Reader
io.ReadWriteCloser
}
func (rw *BufferedReadWriteCloser) Read(p []byte) (int, error) {
return rw.Reader.Read(p)
}
Here's how to use it:
rw := &BufferedReadWriteCloser{bufio.NewReader(i), i}
p, err := rw.Peek(1)
The value of rw satisfies the io.ReadWriteCloser interface.
There is no requirement or assumption that the io.ReadWriteCloser has a Seek method.
As mentioned in my comment above, you need access to the original reader's Seek method. This means that passing the reader around as an io.ReadWriteCloser is insufficient. Having said that, the following helper function may be a workaround:
func peek(r io.Reader, n int) ([]byte, error) {
bi := bufio.NewReader(r)
peeked, err := bi.Peek(n)
if err != nil {
return nil, err
}
// Use type assertion to check if r implements the
// io.Seeker interface. If it does, then use it to
// reset the offset.
if seeker, ok := r.(io.Seeker); ok {
seeker.Seek(0, 0)
}
return peeked, nil
}
Now you can pass the io.ReadWriteCloser to this peek function. The peek function checks if the reader happens to implement the Seek method. If the Seek method is implemented, then peek will call it.
I have an interface:
type encoder interface {
encode() ([]byte, error)
}
Some implementations of encoder return an error:
type fooEncoder string
func (e fooEncoder) encode() ([]byte, error) {
if isSomeValidityCheck(e) {
return []byte(e), nil
}
return nil, fmt.Errorf("Invalid type!")
}
But for others, there will never be an error:
type boolEncoder bool
func (e boolEncoder) encode() ([]byte, error) {
if e {
return []byte{0xff}, nil
}
return []byte{0x00}, nil
}
Is it idiomatic/correct to say a method will return an error, even if it will always be nil, so that it conforms to an interface? I have boolEncoder.encode returning an error only so that it conforms to encoder and can be used as such.
This is completely OK / normal. Often it's more important to implement an interface than to reduce the code (of the method).
There are numerous examples in the standard lib too.
For example bytes/Buffer.Write() implements io.Writer with
func (b *Buffer) Write(p []byte) (n int, err error)
But writing to an in-memory buffer cannot fail, it documents that it never will return a non-nil error:
Write appends the contents of p to the buffer, growing the buffer as needed. The return value n is the length of p; err is always nil. If the buffer becomes too large, Write will panic with ErrTooLarge.
Buffer.Write() could have a signature that doesn't return anything because its return values carry no information (n is always len(p) and err is always nil), but then you couldn't use bytes.Buffer as an io.Writer, which is way more important.
See related: Is unnamed arguments a thing in Go? and Why does Go allow compilation of unused function parameters?
I am looking for a way to cast an interface into the concrete type to save a lot of source code.
The initial situation are two functions of a webserver handler. They differ only in that one function decodes an array of structs and the other decodes a single struct and stores it in the database. The functions which have to be called for saving are identical depending on the type.
To decide whether an array or a struct is passed, it is tried to cast the interface into the type and then to pass this appropriately as a parameter of the function. Similar as it is described in the documentation and in the stackoverflow post.
However, I do not get the concrete type as expected and the program always runs into the default section. What am I doing wrong or have I not taken into account?
These are the outputs of the default section:
# interface is a struct
... or a single repository struct: map[string]interface{}
# interface is an array of structs
... or a single repository struct: []interface{}
Below is the source code with the functions
func (rh *RouteHandler) AddOrUpdateRepository(rw http.ResponseWriter, req *http.Request) {
repository := new(types.Repository)
rh.addOrUpdateRepositories(rw, req, repository)
}
func (rh *RouteHandler) AddOrUpdateRepositories(rw http.ResponseWriter, req *http.Request) {
repositories := make([]*types.Repository, 0)
rh.addOrUpdateRepositories(rw, req, repositories)
}
func (rh *RouteHandler) addOrUpdateRepositories(rw http.ResponseWriter, req *http.Request, v interface{}) {
defer req.Body.Close()
switch req.Header.Get("Content-Type") {
case "application/xml":
xmlDecoder := xml.NewDecoder(req.Body)
err := xmlDecoder.Decode(&v)
if err != nil {
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to decode repositories or repository")
rh.ulogger.Error("Failed to decode repositories or repository: %v", err)
return
}
case "application/json":
fallthrough
default:
jsonDecoder := json.NewDecoder(req.Body)
err := jsonDecoder.Decode(&v)
if err != nil {
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to decode repositories or repository")
rh.ulogger.Error("Failed to decode repositories or repository: %v", err)
return
}
}
var err error
switch x := v.(type) {
case map[string]*types.Repository:
for _, repository := range x {
err = rh.manager.AddOrUpdateRepository(context.Background(), repository)
}
case *types.Repository:
err = rh.manager.AddOrUpdateRepository(context.Background(), x)
case map[string][]*types.Repository:
for i := range x {
for j := range x[i] {
err = rh.manager.AddOrUpdateRepository(context.Background(), x[i][j])
}
}
case []*types.Repository:
err = rh.manager.AddOrUpdateRepository(context.Background(), x...)
case nil:
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to cast interface")
rh.ulogger.Error("Failed to cast interface. Interface is a type of nil")
return
default:
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to cast interface")
rh.ulogger.Error("Failed to cast interface. Interface does not match onto an array of repositories or a single repository struct: %T", x)
return
}
if err != nil {
rw.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(rw, "Failed to add repositories or repository")
rh.ulogger.Error("Failed to add repositories or repository: %v", err)
return
}
rw.WriteHeader(http.StatusCreated)
}
(Simplified a bit.)
You have a function with the following signature:
func addOrUpdateRepositories(v interface{})
and then you call it like this:
repository := new(types.Repository)
addOrUpdateRepositories(repository)
and like this:
repositories := make([]*types.Repository, 0)
addOrUpdateRepositories(repositories)
In the first call, the concrete type of the value stored in v will be *types.Repository (as new returns a pointer to the allocated value) and in the second call the concrete type of the value stored in v will be []*types.Repository—because that's what make was told to create.
Now you do a type-switch on v, which reads:
switch x := v.(type) {
case map[string]*types.Repository:
case map[string][]*types.Repository:
case nil:
default:
}
Leaving aside the case that if you do not call addOrUpdateRepositories passing it a nil v which cannot happen in the snippet from your question, the switch will always pick the default branch because the type of the concrete value stored in v is never map[string]*types.Repository or map[string][]*types.Repository.
I am not sure why you fail to see this, so probably you should refine your question or may be try to clear up your confusion in a comment to my answer?
Another shot in the dark: type conversions (note that Go does not have type casts, as #Flimzy pointed out) and type switches in Go do not actually change the underlying representation of the value they operate on—except for the limited set of ("everyone expects this") cases such as type-converting a float64 to int64, which are precisely documented.
So you cannot take a []*types.Repository (a slice of pointers to values of type types.Repository) and somehow force it "to become" map[string][]*types.Repository: that'd be a nonsensical thing to do for a number of reasons, with the most glaring being this: if you were writing the Go compiler, how would you carry out such a "type cast"? Suppose you were to really allocate a map, but then which key in that map should have been assigned the original (source) slice? What about type-converting a []*types.Repository to struct {foo []*types.Repository; bar []*types.Repository}?
I have a struct that embeds an embedded pointer to another struct. When I use the default json.Unmarshal behavior, it works perfectly. But when I implement UnmarshalJSON for the embedded struct's type but not the outer struct, then go panics with null pointer dereference.
If I implement UnmarshalJSON for the outer struct type as well, then it works. However, the outer struct has many fields that I would rather not have to manually unmarshal.
Why does implementing UnmarshalJSON on one and not the other cause a panic?
Is there a way to get it work without implemented UnmarshalJSON for the outer type?
If not, is there a simpler/less manual way to implement UnmarshalJSON for the outer type?
Note: There is a question with a similar title, "json.Unmarshal fails when embedded type has UnmarshalJSON
", but the issue there is different from mine.
tl;dr: The rest of this question is just a lengthy example of the above.
Base Example
(play.golang.org version of example)
The two structs, one with an embedded field pointer to the other:
(Simplified for example -- this doesn't really need its own UnmarshalJSON but it demonstrates the problem.)
type Obj struct {
X int `json:"x"`
}
type Container struct {
*Obj
Y int `json:"y"`
}
Invoking unmarshal:
func main() {
b := []byte(`{"x": 5, "y": 3}`)
c := &Container{}
err := json.Unmarshal(b, c)
if err != nil {
fmt.Printf("error ummarshalling json: %+v\n", err)
return
}
fmt.Printf("unmarshalled: %+v --> %+v\n", c, c.Obj)
}
Without implementing any UnmarshalJSON funcs, this works fine:
unmarshalled: &{Obj:0x416080 Y:3} --> &{X:5}
Panic
But, if I add UnmarshalJSON to the embedded Obj type only, then the program panics, as the json.Unmarshal call passes a nil pointer when it tries to unmarshal *Obj.
func (o *Obj) UnmarshalJSON(b []byte) (err error) {
m := make(map[string]int)
err = json.Unmarshal(b, &m)
if err != nil {
return nil
}
o.X = m["x"] // the line indicated by panic
return nil
}
Output:
panic: runtime error: invalid memory address or nil pointer dereference
[...]
main.(*Obj).UnmarshalJSON(0x0, 0x416030, 0x10, 0x10, 0x0, 0x0)
/tmp/sandbox185809294/main.go:18 +0x130
[...]
Question: Why does it panic here but not with the default unmarshal behavior? I'd think that if a nil *Obj is being passed here, then the default behavior also passes around a nil pointer...
Fixing the panic
It no longer panics when I implement UnmarshalJSON for the outer Container type:
func (c *Container) UnmarshalJSON(b []byte) (err error) {
m := make(map[string]int)
err = json.Unmarshal(b, &m)
if err != nil {
return err
}
c.Obj = &Obj{X: m["x"]}
c.Y = m["y"]
return nil
}
But unmarshalling Container manually this way gets tedious if both the real Container and real Obj have more fields than this, each with different types.
Question: Is there a simpler way to prevent this panic?
Because the default behavior checks for nil and your custom unmarshaller does not. You need some logic in your UnmarshalJSON to check if o is nil and behave appropriately, instead of assuming o is not nil (by trying to access one of its fields), thereby triggering a panic.
func (o *Obj) UnmarshalJSON(b []byte) (err error) {
if o == nil {
return nil // maybe? What do you want to happen in this case?
}
m := make(map[string]int)
err = json.Unmarshal(b, &m)
if err != nil {
return nil
}
o.X = m["x"] // the line indicated by panic
return nil
}
Also just for future reference, your *Obj field is not an "anonymous field", it is an embedded field: https://golang.org/ref/spec#Struct_types
I am calling a function to do a http request, two pass by reference parameter is used for the function. I pass the []byte to v interface. I want the function to update the v interface reference value. The response body is a string, I want to pass the string value to v interface. However, tried many ways but not success.
Here is the code, you can see I declare byts as v.(*[]byte) in order to make v updated with the string value of response body. But it does not work. The v is always nil. Please suggest any way to make v can be updated with the string value.
func (s *BackendConfiguration) Do(req *http.Request, v interface{}) error {
res, err := s.HTTPClient.Do(req)
defer res.Body.Close()
resBody, err := ioutil.ReadAll(res.Body)
if v != nil {
byts, ok := v.(*[]byte)
if len(resBody) > 0 {
byts = append(byts, resBody...)
return nil
}
}
}
return nil
}
Well, the main reason this does not work is because you think of "call by reference", a concept completely unknown to Go. Absolutely everything is called by value in Go and once you spell out what is a byte slice, a pointer to a byte slice, a pointer to byte slice wrapped inside an interface, a copy of the pointer to a byte slice extracted from the interface, and so on you'll see how to update the value the pointer to byte slice points to:
package main
import "fmt"
func f(v interface{}) {
pbs := v.(*[]byte)
*pbs = append(*pbs, []byte{9,8,7}...)
}
func main() {
bs := []byte{1,2,3}
pbs := &bs
var v interface{} = pbs
f(v)
fmt.Printf("%v\n", *pbs)
}