Create a struct that implements an interface.
Encapsulate it into another struct.
Fetch the struct / dereference it.
Fetching gets pointer to the implementation of the interface.
Dereferencing the pointer results in another point to the interface.
Why wouldn't dereferencing the pointer to the implementation gives the implementation instance?
package main
import (
"fmt"
"net/http"
"golang.org/x/net/http2"
)
func main() {
transport := &http2.Transport{}
client := &http.Client{Transport: transport}
tmp1 := client.Transport
tmp2 := &client.Transport
tmp3 := &tmp1
fmt.Printf("%T\n%T\n%T\n", tmp1, tmp2, tmp3)
//dialTLS := tmp1.DialTLS
}
This code outputs...
*http2.Transport
*http.RoundTripper
*http.RoundTripper
As well, trying to access a property of the tmp1 (uncommenting dialTLS := tmp1.DialTLS) results in compile error...
tmp1.DialTLS undefined (type http.RoundTripper has no field or method DialTLS)
...even though fmt.Printf("%+v", tmp1) outputs...
&{DialTLS:<nil> TLSClientConfig:0xc8203ec8c0 ConnPool:<nil> DisableCompression:false MaxHeaderListSize:0 t1:<nil> connPoolOnce:{m:{state:0 sema:0} done:0} connPoolOrDef:<nil>}
What I am attempting to do is access DialTLS in the Transport instance.
You have a type mismatch here. If you look at the documentation, you'll see that the field Transport of the Client struct returns an interface RoundTripper:
type Client struct {
// Transport specifies the mechanism by which individual
// HTTP requests are made.
// If nil, DefaultTransport is used.
Transport RoundTripper
// ...
}
So, the type of tmp1 is http.RoundTripper, though the underlying type is *http2.Transport which implements the RoundTripper interface.
As for tmp2 and tmp3, they are seen as pointers to a RoundTripper and not as **http2.Transport.
In order to retrieve the DialTLS field, you have to use type assertions to convert a RoundTripper back into a Transport:
dialTLS := client.Transport.(*http2.Transport).DialTLS
Related
This question already has answers here:
Strings encode/decode in gob
(4 answers)
Closed 12 months ago.
I am trying to decode an Inv struct, but decoding the same encoded value returns a different value.
// inv struct
type Inv struct {
AddrFrom string
Type int
data [][]byte
}
inv := Inv{
AddrFrom: nodeAddress,
Type: kind,
data: inventories,
}
data := GobEncode(inv)
var payload Inv
gob.NewDecoder(bytes.NewBuffer(data)).Decode(&payload)
Here payload and inv have different values. When decoded data field of inv struct is of length zero.
https://pkg.go.dev/encoding/gob
A struct field of chan or func type is treated exactly like an unexported field and is ignored.
https://go.dev/ref/spec#Exported_identifiers
An identifier may be exported to permit access to it from another package. An identifier is exported if both:
the first character of the identifier's name is a Unicode upper case letter (Unicode class "Lu"); and
the identifier is declared in the package block or it is a field name or method name.
All other identifiers are not exported.
https://pkg.go.dev/encoding/gob#hdr-Types_and_Values
Gob can encode a value of any type implementing the GobEncoder or encoding.BinaryMarshaler interfaces by calling the corresponding method, in that order of preference.
Internally, the gob package relies on the reflect package, which is designed to respect the visibility principle. Thus the gob package does not handle those fields automatically, it requires you to write dedicated implementation.
https://pkg.go.dev/encoding/gob#GobEncoder
GobEncoder is the interface describing data that provides its own representation for encoding values for transmission to a GobDecoder. A type that implements GobEncoder and GobDecoder has complete control over the representation of its data and may therefore contain things such as private fields, channels, and functions, which are not usually transmissible in gob streams.
Example
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
// The Vector type has unexported fields, which the package cannot access.
// We therefore write a BinaryMarshal/BinaryUnmarshal method pair to allow us
// to send and receive the type with the gob package. These interfaces are
// defined in the "encoding" package.
// We could equivalently use the locally defined GobEncode/GobDecoder
// interfaces.
type Vector struct {
x, y, z int
}
func (v Vector) MarshalBinary() ([]byte, error) {
// A simple encoding: plain text.
var b bytes.Buffer
fmt.Fprintln(&b, v.x, v.y, v.z)
return b.Bytes(), nil
}
// UnmarshalBinary modifies the receiver so it must take a pointer receiver.
func (v *Vector) UnmarshalBinary(data []byte) error {
// A simple encoding: plain text.
b := bytes.NewBuffer(data)
_, err := fmt.Fscanln(b, &v.x, &v.y, &v.z)
return err
}
// This example transmits a value that implements the custom encoding and decoding methods.
func main() {
var network bytes.Buffer // Stand-in for the network.
// Create an encoder and send a value.
enc := gob.NewEncoder(&network)
err := enc.Encode(Vector{3, 4, 5})
if err != nil {
log.Fatal("encode:", err)
}
// Create a decoder and receive a value.
dec := gob.NewDecoder(&network)
var v Vector
err = dec.Decode(&v)
if err != nil {
log.Fatal("decode:", err)
}
fmt.Println(v)
}
As the field type is already a byte slice, you really are just hitting a visibility access issue and the dedicated required marshalling implementation, while arguable because you could as well export that field, should be straightforward.
I have a struct type with a number of fields that all implement a Renderer interface. The field types implement the interface with pointer receivers. I would like to have a function that takes the field name and calls the Render method on that field. I am able to locate the field and get lots of information about it, but doing the type assertion seems to be biting me because of the pointer receivers. Here's some code that shows my problem:
package main
import (
"fmt"
"reflect"
)
type Renderer interface {
Render()
}
type First struct {
ExampleField Field
}
type Field []int
func (f *Field) Render() {
fmt.Println("Hello from first")
}
func main() {
f := First{
Field{1, 2, 3},
}
f.ExampleField.Render()
renderField("ExampleField", &f)
renderField2("ExampleField", &f)
}
func renderField(field string, f *First) {
structVal := reflect.ValueOf(*f)
renderType := reflect.TypeOf((*Renderer)(nil)).Elem()
fieldToRender := structVal.FieldByName(field)
fieldPtr := reflect.PtrTo(fieldToRender.Type())
fmt.Printf("Implements? %v\n", fieldPtr.Implements(renderType))
fmt.Printf("Addressable? %v\n", fieldToRender.CanAddr())
fieldInter := fieldToRender.Interface()
if renderer, ok := fieldInter.(Renderer); ok {
// Pointer receiver so this never gets called
fmt.Print("Able to cast")
renderer.Render()
}
}
func renderField2(field string, f *First) {
structVal := reflect.ValueOf(*f)
fieldToRender := structVal.FieldByName(field)
vp := reflect.New(reflect.TypeOf(fieldToRender))
vp.Elem().Set(reflect.ValueOf(fieldToRender))
vpAddr := vp.Elem().Addr()
typeVal := vpAddr.Interface()
fmt.Println(typeVal) // <main.Field Value>⏎
renderer := typeVal.(Renderer)
renderer.Render()
// interface conversion: *reflect.Value is not main.Renderer: missing method Render
}
renderField2 seems to get me close but Addr() gives me a *Reflect.Value and when I call Interface() that seems to be the underlying type instead. If I switch to a non-pointer receiver then the first function works. I found reflect value Interface and pointer receiver which seems to be almost exactly what I'm asking, and the question is answered but if I actually call the isZeroer method presented in the playground link it's always false so it doesn't actually seem to answer the question.
It seems like Addr is the key because it specifically mentions pointer receivers but I'm struggling to coerce it back into an interface.
Use this code:
func renderField(name string, f *First) {
structVal := reflect.ValueOf(f).Elem()
field := structVal.FieldByName(name).Addr().Interface()
if renderer, ok := field.(Renderer); ok {
renderer.Render()
}
}
The key point is to change:
structVal := reflect.ValueOf(*f)
to:
structVal := reflect.ValueOf(f).Elem()
The statement used in the question creates a non-addressable struct value. The fields in a non-addressable struct are also not addressable, therefore it's not possible to access the pointer receiver on the fields.
The statement used in this answer creates an addressable struct value.
I'm trying an example related to struct embedding of interfaces
// https://talks.golang.org/2014/go4java.slide#52
// Struct embedding of interfaces
// https://play.golang.org/p/SYiZ7M1OEhU
package main
import (
"bytes"
"fmt"
"net"
)
// net.Conn has Read and Write
type loopBack struct {
net.Conn
buf bytes.Buffer
}
func (c *loopBack) Read(b []byte) (int, error) {
fmt.Println("loopBack Read")
return 0, nil
}
func main() {
loop := loopBack{}
loop.Read(nil)
loop.Write(nil)
}
and the Write method is undefined, so I get this runtime error
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0xffffffff addr=0x0 pc=0xe28ca]
goroutine 1 [running]:
main.main()
/tmp/sandbox812386031/main.go:28 +0x6a
Exist some way to validate it at compile time?
link to code
https://play.golang.org/p/SYiZ7M1OEhU
What you are doing is not the same as saying "loopBack implements net.Conn".
To get compile-time error(s) about the missing method – and the mismatched Read(), too – declare loop's type:
(And don't embed net.Conn in loopBack)
func main() {
var loop net.Conn = loopBack{}
Golang doesn't require exlicit interface implementations. What you do here:
type loopBack struct {
net.Conn
buf bytes.Buffer
}
Is the similar to:
type loopBack struct{
Conn net.Conn
buf bytes.Buffer
}
net.Conn being an interface type, the first field of your loopBack type can be anything that implements the net.Conn interface, which is of course more than Read and Write alone (see here).
The advantage of embedding types is that the fields and receiver functions (name conflicts aside) can be accessed directly on the types that embeds them.
With an embedded net.Conn field, you can indeed write:
loop.Write(nil)
If the Conn field is initialised (otherwise, its value is nil). Changing the declaration to the second version, loop.Write won't work, you'll have to write:
loop.Conn.Write(nil)
Type embedding is very powerful, but there's a number of gotcha's when you first get started. Thankfully, there's an entire paragraph explaining embedding on the effective go doc
Anyway, as stated, you are able to call functions, and access fields (if you're embedding a struct type instead of an interface). There is one thing, though: the field must be initialised correctly!
And that's where you went wrong: You still have to initialise your Conn part of the loopBack variable, otherwise what you're doing is the same as:
net.Conn(nil).Write(nil)
Which, naturally, results in a panic (nil pointer dereference)...
For example:
conn, err := net.Dial("tcp", "localhost:80")
if err != nil {
log.Fatalf("failed to dial localhost: %+v", err)
}
loop := loopBack{
Conn: conn,
}
loop.Write(nil) // same as conn.Write(nil)
Failing to set the net.Conn embedded field is akin to doing something like this:
s := make([]*int, 10) // make slice of 10 pointers to int
fmt.Println(len(s)) // outputs 10
*s[0]++ // add 1 to first element PANICS
The 10 elements in the slice exist, but they've all been initialised to nil
I'm using gob to serialize structs to disk. The struct in question contains an interface field, so the concrete type needs to be registered using gob.Register(...).
The wrinkle here is that the library doing the gob-ing should be ignorant of the concrete type in use. I wanted the serialization to be possible even when callers have defined their own implementations of the interface.
I can successfully encode the data by registering the type on the fly (see trivial example below), but upon trying to re-read that data, gob refuses to accept the un-registered type. Its frustrating, because it feels like all the data is there - why isn't gob just unpacking that as a main.UpperCaseTransformation struct if it's labelled as such?
package main
import (
"encoding/gob"
"fmt"
"os"
"strings"
)
type Transformation interface {
Transform(s string) string
}
type TextTransformation struct {
BaseString string
Transformation Transformation
}
type UpperCaseTransformation struct{}
func (UpperCaseTransformation) Transform(s string) string {
return strings.ToUpper(s)
}
func panicOnError(err error) {
if err != nil {
panic(err)
}
}
// Execute this twice to see the problem (it will tidy up files)
func main() {
file := os.TempDir() + "/so-example"
if _, err := os.Stat(file); os.IsNotExist(err) {
tt := TextTransformation{"Hello, World!", UpperCaseTransformation{}}
// Note: didn't need to refer to concrete type explicitly
gob.Register(tt.Transformation)
f, err := os.Create(file)
panicOnError(err)
defer f.Close()
enc := gob.NewEncoder(f)
err = enc.Encode(tt)
panicOnError(err)
fmt.Println("Run complete, run again for error.")
} else {
f, err := os.Open(file)
panicOnError(err)
defer os.Remove(f.Name())
defer f.Close()
var newTT TextTransformation
dec := gob.NewDecoder(f)
// Errors with: `gob: name not registered for interface: "main.UpperCaseTransformation"'
err = dec.Decode(&newTT)
panicOnError(err)
}
}
My work-around would be to require implementers of the interface to register their type with gob. But I don't like how that reveals my serialization choices to the callers.
Is there any route forward that avoids this?
Philosophical argumentation
The encoding/gob package cannot (or rather should not) make that decision on its own. Since the gob package creates a serialized form independent of / detached from the app, there is no guarantee that values of interface types will exist in the decoder; and even if they do (matched by the concrete type name), there is no guarantee that they represent the same type (or the same implementation of a given type).
By calling gob.Register() (or gob.RegisterName()) you make that intent clear, you give green light to the gob package to use that type. This also ensures that the type does exist, else you would not be able to pass a value of it when registering.
Technical requirement
There's also a technical point of view that dictates this requirement (that you must register prior): you cannot obtain the reflect.Type type descriptor of a type given by its string name. Not just you, the encoding/gob package can't do it either.
So by requiring you to call gob.Register() prior, the gob package will receive a value of the type in question, and therefore it can (and it will) access and store its reflect.Type descriptor internally, and so when a value of this type is detected, it is capable of creating a new value of this type (e.g. using reflect.New()) in order to store the value being decoded into it.
The reason why you can't "lookup" types by name is that they may not end up in your binary (they may get "optimized out") unless you explicitly refer to them. For details see Call all functions with special prefix or suffix in Golang; and Splitting client/server code. When registering your custom types (by passing values of them), you are making an explicit reference to them and thus ensuring that they won't get excluded from the binaries.
package main
import (
"fmt"
"encoding/json"
"reflect"
)
type GeneralConfig map[string]interface{}
var data string = `
{
"key":"value",
"important_key":
{"foo":"bar"}
}`
func main() {
jsonData := &GeneralConfig{}
json.Unmarshal([]byte(data), jsonData)
fmt.Println(reflect.TypeOf(jsonData)) //main.GeneralConfig
jsonTemp := (*jsonData)["important_key"]
fmt.Println(reflect.TypeOf(jsonTemp)) //map[string]interface {}
//newGeneralConfig := GeneralConfig(jsonTemp)
//cannot convert jsonTemp (type interface {}) to type GeneralConfig:
//need type assertion
newGeneralConfig := jsonTemp.(GeneralConfig)
//fmt.Println(reflect.TypeOf(newGeneralConfig))
//panic: interface conversion: interface {} is map[string]interface {},
//not main.GeneralConfig
}
Available at the playground
I understand that I can use a nested struct in lieu of GeneralConfig, but that would require me knowing the exact structure of the payload, ie it wouldn't work for different keys (I would be locked into "important_key").
Is there a golang workaround for when I don't know what the value of "important_key" is? I say golang, because if possible, one could require all "important_keys" to have a constant parent key, which could resolve this issue.
To summarize, given an arbitrary json object, there must be a way that I can traverse its keys, and if a value is a custom type, convert the value to that type. Right now it seems that if I use type conversion, it tells me that the type is interface{} and I need to use type assertion; however, if I use type assertion, it tells me that interface{} is map[string]interface{} not main.GeneralConfig.
I agree the comments about trying to utilise the expected structure of the incoming JSON in order to write well-defined Structs, but I'll attempt to answer the question anyway.
The thing to take away from what you're seeing printed versus the error messages that you're seeing is that the compiler knows less about the type than the runtime because the runtime can look at the actual value. To bring the compiler up-to-speed we must (i) assert (*jsonData)["important_key"] is a map[string]interface{} -- the compiler only knows it to be an interface{} -- and then (ii) type-cast that to a GeneralConfig type. See:
package main
import (
"fmt"
"encoding/json"
)
type GeneralConfig map[string]interface{}
func main() {
jsonStruct := new(GeneralConfig)
json.Unmarshal([]byte(`{"parent_key": {"foo": "bar"}}`), jsonStruct)
fmt.Printf("%#v\n", jsonStruct)
// => &main.GeneralConfig{"parent_key":map[string]interface {}{"foo":"bar"}}
nestedStruct := (*jsonStruct)["parent_key"]
fmt.Printf("%#v\n", nestedStruct)
// => map[string]interface {}{"foo":"bar"}
// Whilst this shows the runtime knows its actual type is
// map[string]interface, the compiler only knows it to be an interface{}.
// First we assert for the compiler that it is indeed a
// map[string]interface{} we are working with. You can imagine the issues
// that might arrise if we has passed in `{"parent_key": 123}`.
mapConfig, ok := nestedStruct.(map[string]interface{})
if !ok {
// TODO: Error-handling.
}
// Now that the compiler can be sure mapConfig is a map[string]interface{}
// we can type-cast it to GeneralConfig:
config := GeneralConfig(mapConfig)
fmt.Printf("%#v\n", config)
// => main.GeneralConfig{"foo":"bar"}
}
You are looking for json.RawMessage.
You can delay unmarshalling based upon some other value and then force it to unmarshal to a specific type.
This is not a good idea, but might be closer to what you are looking for.
http://play.golang.org/p/PWwAUDySE0
This is a standard "workaround" if get what you're after. When handling unknown data you can implement this pattern (modified from your example) of switching on the type recursively to get to the concrete values in an unknown body of json data.
package main
import (
"encoding/json"
"fmt"
"reflect"
)
var data = `
{
"key":"value",
"important_key":
{"foo":"bar"}
}`
func main() {
var jsonData interface{}
json.Unmarshal([]byte(data), &jsonData)
fmt.Println(reflect.TypeOf(jsonData))
parseArbitraryJSON(jsonData.(map[string]interface{}))
}
func parseArbitraryJSON(data map[string]interface{}) {
for k, v := range data {
switch a := v.(type) {
case string:
fmt.Printf("%v:%v\n", k, a)
case map[string]interface{}:
fmt.Printf("%v:%v\n", k, a)
parseArbitraryJSON(a)
}
}
}
The resulting output is:
map[string]interface {}
key:value
important_key:map[foo:bar]
foo:bar
This example only accounts for the base data being a string type but you can switch on any type that you expect to receive, and like any switch you can group your cases, so you can treat all numbers similarly for example.