I'm new to golang and trying to figure out the correct way of casting a block of bytes to the correct struct. All structs start with two bytes that dictate the layout of the remaining bytes. In C I would point to the beginning of the block of memory and cast it as a simple struct that only contained those two bytes (X below) but here I get an invalid type assertion. I'm probably way off base here any help you be appreciated.
package main
import (
"fmt"
)
type A struct {
tag byte
ver byte
data1 int
data2 int
data3 int
}
type B struct {
tag byte
ver byte
data1 float32
}
type X struct {
tag byte
ver byte
}
func main() {
var a A
a.tag = 1
a.ver = 1
x := a.(X)
fmt.Printf("%d,%d", x.tag, x.ver)
}
Edit
In short I just want to create a custom method on type Foo that is the only reason why I want to perform the cast. If the solutions are very complex I will just create functions instead of methods I guess. I was just curios.
playground link
Here's my solution. It involves a few different tips:
Embed the shared struct in the individual structs.
Use encoding/binary package to load bytes into structs.
Fill header struct with first two bytes, then make a decision on which subtype to make and fill.
Always use fixed length int types for this kind of thing.
Your field names must be UpperCase to be fillable from encoding/binary
This is a pretty brittle way to manage marshalling.unmarshalling of data, but I'm sure you know that.
Here's my solution:
package main
import (
"bytes"
"encoding/binary"
"fmt"
"log"
)
type A struct {
X
Data1 int32
Data2 int32
Data3 int32
}
type B struct {
X
Data1 int32
}
type X struct {
Tag byte
Ver byte
}
func main() {
var err error
data := []byte{1, 1, 0, 0, 0, 42}
hdr := X{}
err = binary.Read(bytes.NewReader(data[:2]), binary.BigEndian, &hdr)
if err != nil {
log.Fatal(err)
}
fmt.Println(hdr.Tag, hdr.Ver)
if hdr.Tag == 1 {
b := B{}
err = binary.Read(bytes.NewReader(data), binary.BigEndian, &b)
if err != nil {
log.Fatal(err)
}
fmt.Println(b.Data1)
}
}
playground link
Go generally tries to discourage C-like memory fiddling as it leads to memory leaks, incorrect behavior, and security vulnerabilities unless extraordinary caution and testing are applied. That doesn't mean it's impossible though; in fact, the aptly-named unsafe.Pointer is exposed for exactly this purpose. Use it with caution.
Related
This question already has answers here:
generic function to get size of any structure in Go
(2 answers)
Closed 6 months ago.
In the case of a generic function that does byte serialization for a generic types, is there a way to proceed--other than reflection--if the different supported types have different sizes? For example:
package main
import (
"fmt"
)
type KeyType interface {
uint16 | uint32 | uint64
}
type Item[KT KeyType] struct {
Key KT
Data []byte
}
// set of generic types that hold collections of Item[T]
// sets of methods that operate on those generic types
func MarshalBinary[KT KeyType](i *Item[KT]) ([]byte, error) {
// How do I compute the size of the item and marshal it?
// It's 2 bytes for uint16, 4 for uint32, 8 for uint64,
// how do I distinguish here?
}
func main() {
i := new(Item[uint32])
i.Key = 42
fmt.Println(i)
}
Is there a way to access the size of the type within the serialization function without reflection?
I know I can proceed with reflection like this:
package main
import (
"fmt"
"reflect"
"strings"
)
type KeyType interface {
uint16 | uint32 | uint64
}
type Item[KT KeyType] struct {
Key KT
Data []byte
}
// set of generic types that hold collections of Item[T]
// sets of methods that operate on those generic types
func MarshalBinary[KT KeyType](i *Item[KT]) ([]byte, error) {
t := reflect.TypeOf(i)
var size int
if strings.Contains(t.String(), `uint32`) {
size = 4
}
fmt.Println(size)
// rest of function here
return nil, nil
}
func main() {
i := new(Item[uint32])
i.Key = 42
MarshalBinary(i)
fmt.Println(i)
}
Is there a better way? My main concern with using reflection here is the potential performance cost.
First off, I think your sample code may be incorrect, because you're using reflect.TypeOf(i), but i is of type Item[KT], and since Item includes both a KT and a []byte, it will be the size of KY plus the size of a pointer (the pointer to the byte slice). So, it will be 4 + pointer size if KT is a uint32, but you're setting it to 4.
So the question is, are you trying to get the size of i (Item[KT]), or the size of an instance of KT?
I assume it's the size of KT that you're actually looking for, since you're assigning a size of 4 if it's a uint32. You can do this without reflect by casting the value you want the size of as an interface{}, then using a standard type switch, as follows:
func MarshalBinary[KT KeyType](i *Item[KT]) ([]byte, error) {
var size int
switch (interface{})(i.Key).(type) {
case uint16:
size = 2
case uint32:
size = 4
case uint64:
size = 8
default:
panic("Unexpected type")
}
...
}
This is a bit problematic if you ever want to expand the different possible types that KeyType could be, though.
I'm looking for some solution to know whats the struct type of the hash. It is possible to do that without try an error method (casting to a specific type and see the cast is successfully)?
Please check the code:
import (
"bytes"
"encoding/binary"
"fmt"
"reflect"
)
type T struct {
A int64
B float64
}
type D struct {
A int64
B float64
C string
}
func main() {
// Create a struct and write it.
t := T{A: 0xEEFFEEFF, B: 3.14}
buf := &bytes.Buffer{}
err := binary.Write(buf, binary.BigEndian, t)
if err != nil {
panic(err)
}
fmt.Println(buf.Bytes())
out := getType(buf)
fmt.Println(out)
}
func getType(v interface{})(r string){
fmt.Println(reflect.TypeOf(v))
switch t := v.(type) {
case T:
return "Is type T"
case D:
return "Is type D"
default:
_ = t
return "unknown"
}
}
Since the encoding/binary package does not write out type information, it is not possible to tell what type was written / serialized.
And you're in a worse position that you might originally think: even trying to decode into a value of different type might succeed without errors, so there isn't even a reliable way to tell the type.
For example if you serialize a value of this type:
type T struct {
A int64
B float64
}
You can read it into a value of this type:
type T2 struct {
B float64
A int64
}
It will give no errors because the size of both structs is the same, but obviously you will get different numbers in the fields.
You are in a little better position if you use encoding/gob, as the gob package does transmit type information, and encoding a value of type T and then decoding it into a value of type T2 would work: order of fields does not matter, and extra or missing fields also do not cause trouble.
See this example:
// Create a struct and write it.
t := T{A: 0xEEFFEEFF, B: 3.14}
fmt.Println("Encoding:", t)
buf := &bytes.Buffer{}
fmt.Println(binary.Write(buf, binary.BigEndian, t))
fmt.Println(buf.Bytes())
fmt.Println(gob.NewEncoder(buf).Encode(t))
t2 := T2{}
fmt.Println(binary.Read(buf, binary.BigEndian, &t2))
fmt.Println(t2)
t2 = T2{}
fmt.Println(gob.NewDecoder(buf).Decode(&t2))
fmt.Println(t2)
Output (try it on the Go Playground):
Encoding: {4009750271 3.14}
<nil>
[0 0 0 0 238 255 238 255 64 9 30 184 81 235 133 31]
<nil>
<nil>
{1.9810798573e-314 4614253070214989087}
<nil>
{3.14 4009750271}
If you want to be able to detect the type before reading it, you have to take care of it yourself: you have to transmit type information (e.g. name of the type). Or even better, use a serialization method that already does this, for example Google's protocol buffers, and here is the Go implementation for it: github.com/golang/protobuf.
I have some different structs like Big with Small embedded at offset 0.
How can I access Small's structure fields from code, that doesn't know anything about Big type, but it is known that Small is at offset 0?
type Small struct {
val int
}
type Big struct {
Small
bigval int
}
var v interface{} = Big{}
// here i only know about 'Small' struct and i know that it is at the begining of variable
v.(Small).val // compile error
It seems that compiler is theoretically able to operate such expression, because it knows that Big type has Small type embedded at offset 0. Is there any way to do such things (maybe with unsafe.Pointer)?
While answer with reflection is working but it has performance penalties and is not idiomatic to Go.
I believe you should use interface. Like this
https://play.golang.org/p/OG1MPHjDlQ
package main
import (
"fmt"
)
type MySmall interface {
SmallVal() int
}
type Small struct {
val int
}
func (v Small) SmallVal() int {
return v.val
}
type Big struct {
Small
bigval int
}
func main() {
var v interface{} = Big{Small{val: 3}, 4}
fmt.Printf("Small val: %v", v.(MySmall).SmallVal())
}
Output:
Small val: 3
Avoid using unsafe whenever possible. The above task can be done using reflection (reflect package):
var v interface{} = Big{Small{1}, 2}
rf := reflect.ValueOf(v)
s := rf.FieldByName("Small").Interface()
fmt.Printf("%#v\n", s)
fmt.Printf("%#v\n", s.(Small).val)
Output (try it on the Go Playground):
main.Small{val:1}
1
Notes:
This works for any field, not just the first one (at "offset 0"). This also works for named fields too, not just for embedded fields. This doesn't work for unexported fields though.
type Small struct {
val int
}
type Big struct {
Small
bigval int
}
func main() {
var v = Big{Small{10},200}
print(v.val)
}
I'm trying to assign a value to a field, but my program panics with runtime error: invalid memory address or nil pointer dereference.
package main
type Node struct {
Value int
}
func (n *Node) SetValue(value int) {
n.Value = value
}
func main() {
var n *Node
n.SetValue(1)
}
This is reasonable since variable is nil.
But I've fount some Go internal structs are allowed to do this, e.g. bytes.Buffer
package main
import "bytes"
import "io"
import "os"
func main() {
var b bytes.Buffer
b.Write([]byte("Hello world"))
io.Copy(os.Stdout, &b)
}
Here is the `bytes.Buffer source code
func (b *Buffer) Write(p []byte) (n int, err error) {
b.lastRead = opInvalid
m := b.grow(len(p))
return copy(b.buf[m:], p), nil
}
Is it the thing only builtin structs can do or it's possible to accomplish this in my code?
EDIT
Here is the working example. Thanks #twotwotwo for suggestion.
package main
import "fmt"
type Node struct {
Value int
}
func (n *Node) SetValue(value int) {
n.Value = value
}
func main() {
var n Node
n.SetValue(1)
fmt.Println(n.Value)
}
The crucial thing is var b bytes.Buffer doesn't get you a nil pointer, it gets you a bytes.Buffer object with all its fields initialized with their zero values (in machine terms, with zero bytes). The spec says the zero value is "false for booleans, 0 for integers, 0.0 for floats, "" for strings, and nil for pointers, functions, interfaces, slices, channels, and maps"; follow that link for more detail.
It is possible to make your own structs whose zero values work and the Go team encourages it. struct Position { x, y int } is an easy example and Effective Go gives a more realistic one. But note that that doesn't make the nil pointer work; you would still need new(Node) or var n Node to allocate the zero Node. Same for bytes.Buffer.
Another common use of zero values: wherever your users create structs of your type directly (as folks do with, say, http.Server), the zero value is the default for any fields they don't specify. It's the default in a lot of other places: what you get for a not-found map key, if you receive from a closed channel, and probably others.
They seem to be the same:
package main
import "fmt"
type S struct {
i int
}
func main() {
var s1 *S = new(S)
fmt.Println(s1)
var s2 *S = &S{}
fmt.Println(s2) // Prints the same thing.
}
Update:
Hm. I just realized that there's no obvious way to initialize S.i using new. Is there a way to do that? new(S{i:1}) does not seem to work :/
From Effective Go:
As a limiting case, if a composite literal contains no fields at all, it creates a zero value for the type. The expressions new(File) and &File{} are equivalent.
Not only do they give the same resulting value, but if we allocate something both ways and look at their values...
// Adapted from http://tour.golang.org/#30
package main
import "fmt"
type Vertex struct {
X, Y int
}
func main() {
v := &Vertex{}
v2 := new(Vertex)
fmt.Printf("%p %p", v, v2)
}
...we'll see that they are in fact allocated in consecutive memory slots. Typical output: 0x10328100 0x10328108. I'm not sure if this is an implementation detail or part of the specification, but it does demonstrate that they're both being allocated from the same pool.
Play around with the code here.
As for initializing with new, according to the language spec: The built-in function new takes a type T and returns a value of type *T. The memory [pointed to] is initialized as described in the section on initial values. Because functions in go can't be overloaded, and this isn't a variadic function, there's no way to pass in any initialization data. Instead, go will initialize it with whatever version of 0 makes sense for the type and any member fields, as appropriate.
Case 1: package main
import (
"fmt"
)
type Drink struct {
Name string
Flavour string
}
func main() {
a := new(Drink)
a.Name = "Maaza"
a.Flavour = "Mango"
b := a
fmt.Println(&a)
fmt.Println(&b)
b.Name = "Frooti"
fmt.Println(a.Name)
}//This will output Frooti for a.Name, even though the addresses for a and b are different.
Case 2:
package main
import (
"fmt"
)
type Drink struct {
Name string
Flavour string
}
func main() {
a := Drink{
Name: "Maaza",
Flavour: "Mango",
}
b := a
fmt.Println(&a)
fmt.Println(&b)
b.Name = "Froti"
fmt.Println(a.Name)
}//This will output Maaza for a.Name. To get Frooti in this case assign b:=&a.