This question already has answers here:
Hiding nil values, understanding why Go fails here
(3 answers)
Closed 5 years ago.
Can someone provide some explanation about this code behaviour:
https://play.golang.org/p/_TjQhthHl3
package main
import (
"fmt"
)
type MyError struct{}
func (e *MyError) Error() string {
return "some error"
}
func main() {
var err error
if err == nil {
fmt.Println("[OK] err is nil ...")
}else{
fmt.Println("[KO] err is NOT nil...")
}
isNil(err)
var err2 *MyError
if err2 == nil {
fmt.Println("[OK] err2 is nil ...")
}else{
fmt.Println("[KO] err2 is NOT nil...")
}
isNil(err2)
}
func isNil(err error){
if err == nil {
fmt.Println("[OK] ... still nil")
}else{
fmt.Println("[KO] .. why not nil?")
}
}
Output is:
[OK] err is nil ...
[OK] ... still nil
[OK] err2 is nil ...
[KO] .. why err2 not nil?
I found this post Check for nil and nil interface in Go but I still don't get it...
error is a built-in interface, and *MyError implements that interface. Even though the value of err2 is nil, when you pass it to isNil, the function gets a non-nil interface value. That value contains information about the type (*MyError) and the value itself, which is a nil pointer.
If you try printing err in isNil, you'll see that in the second case, you get "some error" even though err2 is nil. This demonstrates why err is not nil in that case (it has to contain the type information).
From my understanding your var err2 *MyError definition is generating a pointer to the struct definition and NOT an actual instantiated object.
In Go interface types are represented by a structure with 2 fields: one denotes its actual type and the other is a pointer to the value.
It means that an interface value is never equal to nil unless the value you passed is actually a nil value.
Relevant and good talk: GopherCon 2016: Francesc Campoy - Understanding nil
Related
Starting to play around with golang and was looking at a custom json.Unmarshal. In a blog post the had the following:
type FlexInt int
func (fi *FlexInt) UnmarshalJSON(data []byte) error {
if data[0] != '"' {
return json.Unmarshal(data, (*int)(fi))
}
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
i, err := strconv.Atoi(s)
if err != nil {
return err
}
*fi = FlexInt(i)
return nil
}
And I understand what it is doing - but I dont understand (*int)(fi) part. Looking at the value of the fi pointer it is the same as (*int)(fi) - (*int)(fi) == fi. Yet when I change that line to simply fi it then does an infinite loop
The expression converts fi to an *int. The result contains the same address, but has a different type.
If a *FlexInt is passed to json.Unmarshal, then json.Unmarshal will call the *FlexInt.UnmarshalJSON method which calls json.Unmarshal and so on.
It's the same pointer, but with a different type. When json.Unmarshal is called with an interface{} that contains a FlexInt*, it calls FlexInt's UnmarshalJSON method. When it's called with an interface{} that contains an int*, it uses the builtin behavior. Since FlexInt and int have the same underlying type, it's acceptable to convert a pointer to one into a pointer to the other, but it's the "actual" type that the interface will be marked with.
I have a struct that embeds an embedded pointer to another struct. When I use the default json.Unmarshal behavior, it works perfectly. But when I implement UnmarshalJSON for the embedded struct's type but not the outer struct, then go panics with null pointer dereference.
If I implement UnmarshalJSON for the outer struct type as well, then it works. However, the outer struct has many fields that I would rather not have to manually unmarshal.
Why does implementing UnmarshalJSON on one and not the other cause a panic?
Is there a way to get it work without implemented UnmarshalJSON for the outer type?
If not, is there a simpler/less manual way to implement UnmarshalJSON for the outer type?
Note: There is a question with a similar title, "json.Unmarshal fails when embedded type has UnmarshalJSON
", but the issue there is different from mine.
tl;dr: The rest of this question is just a lengthy example of the above.
Base Example
(play.golang.org version of example)
The two structs, one with an embedded field pointer to the other:
(Simplified for example -- this doesn't really need its own UnmarshalJSON but it demonstrates the problem.)
type Obj struct {
X int `json:"x"`
}
type Container struct {
*Obj
Y int `json:"y"`
}
Invoking unmarshal:
func main() {
b := []byte(`{"x": 5, "y": 3}`)
c := &Container{}
err := json.Unmarshal(b, c)
if err != nil {
fmt.Printf("error ummarshalling json: %+v\n", err)
return
}
fmt.Printf("unmarshalled: %+v --> %+v\n", c, c.Obj)
}
Without implementing any UnmarshalJSON funcs, this works fine:
unmarshalled: &{Obj:0x416080 Y:3} --> &{X:5}
Panic
But, if I add UnmarshalJSON to the embedded Obj type only, then the program panics, as the json.Unmarshal call passes a nil pointer when it tries to unmarshal *Obj.
func (o *Obj) UnmarshalJSON(b []byte) (err error) {
m := make(map[string]int)
err = json.Unmarshal(b, &m)
if err != nil {
return nil
}
o.X = m["x"] // the line indicated by panic
return nil
}
Output:
panic: runtime error: invalid memory address or nil pointer dereference
[...]
main.(*Obj).UnmarshalJSON(0x0, 0x416030, 0x10, 0x10, 0x0, 0x0)
/tmp/sandbox185809294/main.go:18 +0x130
[...]
Question: Why does it panic here but not with the default unmarshal behavior? I'd think that if a nil *Obj is being passed here, then the default behavior also passes around a nil pointer...
Fixing the panic
It no longer panics when I implement UnmarshalJSON for the outer Container type:
func (c *Container) UnmarshalJSON(b []byte) (err error) {
m := make(map[string]int)
err = json.Unmarshal(b, &m)
if err != nil {
return err
}
c.Obj = &Obj{X: m["x"]}
c.Y = m["y"]
return nil
}
But unmarshalling Container manually this way gets tedious if both the real Container and real Obj have more fields than this, each with different types.
Question: Is there a simpler way to prevent this panic?
Because the default behavior checks for nil and your custom unmarshaller does not. You need some logic in your UnmarshalJSON to check if o is nil and behave appropriately, instead of assuming o is not nil (by trying to access one of its fields), thereby triggering a panic.
func (o *Obj) UnmarshalJSON(b []byte) (err error) {
if o == nil {
return nil // maybe? What do you want to happen in this case?
}
m := make(map[string]int)
err = json.Unmarshal(b, &m)
if err != nil {
return nil
}
o.X = m["x"] // the line indicated by panic
return nil
}
Also just for future reference, your *Obj field is not an "anonymous field", it is an embedded field: https://golang.org/ref/spec#Struct_types
In Go, one often sees the following idiom:
func CopyFile(dstName, srcName string) (written int64, err error) {
src, err := os.Open(srcName)
if err != nil {
return
}
defer src.Close()
dst, err := os.Create(dstName)
if err != nil {
return
}
defer dst.Close()
return io.Copy(dst, src)
}
Is there any reason why the defer statement comes after the error check? My guess is that this is done in order to avoid dereferencing nil values in case err was not nil.
If the file Open or Create fails then you don't have a valid *File to close. The problem wouldn't be a nil value for *File as Close() will check for nil and simply return immediately in that case - the problem might be if the *File value is non-nil but invalid. Since documentation for os.Open() doesn't explicitly state that a failed call to Open() returns a nil value for *File you can't rely that all underlying implementations of it do in fact return a nil value or will always return a nil value..
I want to pass []map[string]interface{} by reference to a function. Here is my attempt.
package main
import "fmt"
func main() {
b := map[string]int{"foo": 1, "bar": 2}
a := [...]map[string]int{b}
fmt.Println(a)
editit(a)
fmt.Println(a)
}
func editit(a interface{}) {
fmt.Println(a)
b := map[string]int{"foo": 3, "bar": 4}
a = [...]map[string]int{b}
fmt.Println(a)
}
https://play.golang.org/p/9Bt15mvud1
Here is another attempt and what I want to do finally. This does not compile.
func (self BucketStats) GetSamples() {
buckets := []make(map[string]interface{})
self.GetAuthRequest(self.url, &buckets)
//ProcessBuckets()
}
func (self BucketStats) GetAuthRequest(rurl string, **data interface{}) (err error) {
client := &http.Client{}
req, err := http.NewRequest("GET", rurl, nil)
req.SetBasicAuth(self.un, self.pw)
resp, err := client.Do(req)
if err != nil {
return
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return
}
// it's all for this!!!
err = json.Unmarshal(body, data)
return
}
Several things wrong here.
First, [...]map[string]int{b} is not in fact a slice, but a fixed-length array. The [...] syntax means "make an array, and set the length at compile time based on what's being put into it". The slice syntax is simply []map[string]int{b}. As a result, your call to editit(a) is in fact passing a copy of the array, not a reference (slices are innately references, arrays are not). When a is reassigned in editit(), you're reassigning the copy, not the original, so nothing changes.
Second, it's almost never useful to use pointers to interfaces. In fact, the Go runtime was changed a few versions back to not automatically dereference pointers to interfaces (like it does for pointers to almost everything else) to discourage this habit. Interfaces are innately references already, so there's little reason to make a pointer to one.
Third, you don't actually need to pass a reference to an interface here. You're trying to unmarshal into the fundamental data structure contained within that interface. You don't actually care about the interface itself. GetAuthRequest(rurl string, data interface{}) works just fine here.
func (self BucketStats) GetSamples() {
var buckets []map[string]interface{}
self.GetAuthRequest(self.url, &buckets)
//ProcessBuckets()
}
func (self BucketStats) GetAuthRequest(rurl string, data interface{}) (err error) {
client := &http.Client{}
req, err := http.NewRequest("GET", rurl, nil)
req.SetBasicAuth(self.un, self.pw)
resp, err := client.Do(req)
if err != nil {
return
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return
}
// it's all for this!!!
err = json.Unmarshal(body, data)
return
}
Let me walk you through what exactly takes place, in order:
buckets := var buckets []map[string]interface{}
We don't need a make here, because json.Unmarshal() will fill it for us.
self.GetAuthRequest(self.url, &buckets)
This passes a reference into an interface field. Within GetAuthRequest, data is an interface with underlying type *[]map[string]interface{} and an underlying value equal to the address of the original buckets variable in GetSamples().
err = json.Unmarshal(body, data)
This passes the interface data, by value, to the interface argument to json.Unmarshal(). Inside json.Unmarshal(), it has new interface v with underlying type *[]map[string]interface{} and an underlying value equal to the address of the original buckets variable in GetSamples(). This interface is a different variable, with a different address in memory, from the interface that held that same data in GetAuthRequest, but the data was copied over, and the data contains a reference to your slice, so you're still good.
json.Unmarshal() will, by reflection, fill the slice pointed to by the interface you handed it with the data in your request. It has a reference to the slice header buckets that you handed it, even though it passed through two interfaces to get there, so any changes it make will affect the original buckets variable.
When you get all the way back up to ProcessBuckets(), the buckets variable will contain all of your data.
As an ancillary suggestion, don't use named returns if your function is more than a few lines long. It's better to explicitly return your variables. This is particularly important because of variable shadowing. For example, in your GetAuthRequest() function, it will never return a non-nil error. This is because you're declaring an error variable err in the function signature, but you're immediately shadowing it with a local variable err using the short declaration in req, err := http.NewRequest("GET", rurl, nil). For the remainder of the function, err now refers to this new error variable rather than the one defined as the return variable. As a result, when you return, the original err variable in the return is always nil. A much better style would be:
func (self BucketStats) GetAuthRequest(rurl string, **data interface{}) error {
client := &http.Client{}
req, err := http.NewRequest("GET", rurl, nil)
req.SetBasicAuth(self.un, self.pw)
resp, err := client.Do(req)
if err != nil {
return err
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
// it's all for this!!!
return json.Unmarshal(body, data)
}
The function is passing a **interface{} to Unmarshal. To pass the the *[]map[string]interface{} through to Unmarshal, change the function signature to:
func (self BucketStats) GetAuthRequest(rurl string, data interface{}) (err error) {
This question already has answers here:
Hiding nil values, understanding why Go fails here
(3 answers)
Closed 7 years ago.
I have a situation where the value of a variable "err error" value can only be "nil" but asserts "( err == nil ) == false" once it has been reassigned.
Example code below:
package main
import (
"fmt"
"log"
)
type TestError struct {
Message string
}
func (e *TestError) Error() string {
return e.Message
}
func NewTestError(err error) *TestError {
if err == nil {
return nil
}
log.Printf("NewTestError( ... ): creating new NewTestError err = %s", err)
return &TestError{Message: err.Error()}
}
func main() {
var err error
_, err = fmt.Printf("excuse.\n")
err = NewTestError(err)
log.Printf("main( ... ): err == nil. %v", (err == nil))
log.Printf("main( ... ): err = %#v", err)
}
I get the following output from the above code:
excuse.
2015/07/30 08:28:28 main( ... ): err == nil. false
2015/07/30 08:28:28 main( ... ): err = (*main.TestError)(nil)
How can those last two lines be output?
For an interface value (like an error value) to compare equal to nil, both the actual contained value and the actual contained type have to be nil. In your case you have an interface value, which holds a nil pointer, but a *TestError (i.e. non-nil) type.
PS. Just in case it's not clear what are these "interface values", you can check this http://research.swtch.com/interfaces
In the second case you're printing the interface instances information which is the type and the value, type being (*main.TestError) value being (nil). In the first case what you're actually comparing isn't nil because it's also the interface.