I have read the documentation of ( gob) and I have some problems :
Now I know how to encode structure and decode like that:
func main() {
s1 := &S{
Field1: "Hello Gob",
Field2: 999,
}
log.Println("Original value:", s1)
buf := new(bytes.Buffer)
err := gob.NewEncoder(buf).Encode(s1)
if err != nil {
log.Println("Encode:", err)
return
}
s2 := &S{}
err = gob.NewDecoder(buf).Decode(s2)
if err != nil {
log.Println("Decode:", err)
return
}
log.Println("Decoded value:", s2)
}
But I don't know the purpose of this method gob.Register() can someone explain to me when to use it and why?
If you're dealing with concrete types (structs) only, you don't really need it. Once you're dealing with interfaces you must register your concrete type first.
For example, let's assume we have these struct and interface (the struct implements the interface):
type Getter interface {
Get() string
}
type Foo struct {
Bar string
}
func (f Foo)Get() string {
return f.Bar
}
To send a Foo over gob as a Getter and decode it back, we must first call
gob.Register(Foo{})
So the flow would be:
// init and register
buf := bytes.NewBuffer(nil)
gob.Register(Foo{})
// create a getter of Foo
g := Getter(Foo{"wazzup"})
// encode
enc := gob.NewEncoder(buf)
enc.Encode(&g)
// decode
dec := gob.NewDecoder(buf)
var gg Getter
if err := dec.Decode(&gg); err != nil {
panic(err)
}
Now try removing the Register and this won't work because gob wouldn't know how to map things back to their appropriate type.
As http://golang.org/pkg/encoding/gob/#Register said:
Only types that will be transferred as implementations of interface
values need to be registered.
So it doesn't needed by your demo.
If you want to encode / decode a map[string]interface{}, since the field of the map is enclosed as interface type, then we need to register the specific type before.
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type SomeStruct struct {
Text string
}
func main() {
var bytes bytes.Buffer
// Remove one of these, then the decoding will produce error
gob.Register(SomeStruct{})
gob.Register([]interface{}{})
gob.Register([]SomeStruct{})
gob.Register(map[string]SomeStruct{})
writer := gob.NewEncoder(&bytes)
err := writer.Encode(map[string]interface{}{
"SomeStruct": SomeStruct{"Halo"},
"SomeSlice": []interface{}{},
"SomeSliceStruct": []SomeStruct{
{
Text: "SomeText",
},
},
"SomeMapStruct": map[string]SomeStruct{
"S": {"Test"},
},
})
if err != nil {
log.Fatalf("Error on encode process: %v\n", err)
return
}
reader := gob.NewDecoder(&bytes)
var aMap map[string]interface{}
err = reader.Decode(&aMap)
if err != nil {
log.Fatalf("Error on decode process: %v\n", err)
return
}
fmt.Printf("Decode is successful: %+v\n", aMap)
}
Related
I have an object of type foo containing an interface ActivationInterface ; this object is saved in MongoDB and I have trouble fetching it back as the underlying type of the inner object is not known.
I implemented UnmarshalBSON as follow without success, as it seems even after setting the concrete type of the interface, the unmarshaller still does now the underlying type, as I still get the error:
error decoding key act: no decoder found for main.ActivationInterface
Do you have any idea how I can achieve this ?
I found somethign close woking here, so I don't get why mine isn't: Unmarshal dynamic JSON based on a type key
I cannot see what I'm doing wrong and different...!
EDIT: I updated the code to compare with json. UnmarshalJSON is working great with exactly the same code while UnmarshalBSON still fails.
package main
import (
"fmt"
"log"
"go.mongodb.org/mongo-driver/bson"
)
type foo struct {
Type string `bson:"type"`
Act ActivationInterface
}
type ActivationInterface interface{}
type Activation1 struct {
Name string `bson:"name"`
}
type Activation2 struct {
Address string `bson:"adress"`
}
func (q *foo) UnmarshalBSON(data []byte) error {
// Unmarshall only the type
fooTemp := new(struct {
Type string `bson:"type"`
})
if err := bson.Unmarshal(data, fooTemp); err != nil {
return err
}
fmt.Println(fooTemp.Type)
// Set the type to the prop
switch fooTemp.Type {
case "act1":
// q.Act = &Activation1{}
q.Act = new(Activation1)
case "act2":
// q.Act = &Activation2{}
q.Act = new(Activation2)
default:
fmt.Println("DEFAULT")
}
// Call Unmarshal again
type Alias foo // avoids infinite recursion using a type alias
return bson.Unmarshal(data, (*Alias)(q))
}
func main() {
foo1 := foo{
Type: "act1",
Act: Activation1{
Name: "name: act1",
},
}
foo2 := foo{
Type: "act2",
Act: Activation2{
Address: "adress: act2",
},
}
// Marshal
m1, err := bson.Marshal(foo1)
if err != nil {
log.Fatal(err)
}
m2, err := bson.Marshal(foo2)
if err != nil {
log.Fatal(err)
}
//fmt.Println(m1, m2)
// Unmarshal
var u1, u2 foo
err = bson.Unmarshal(m1, &u1)
if err != nil {
fmt.Println("1 -> ", err) // error decoding key act: no decoder found for main.ActivationInterface
}
err = bson.Unmarshal(m2, &u2)
if err != nil {
fmt.Println("2 -> ", err) // error decoding key act: no decoder found for main.ActivationInterface
}
fmt.Println(foo1.Type, ":", u1.Act.(*Activation1).Name)
fmt.Println(foo2.Type, ":", u2.Act.(*Activation2).Address)
}
Go playground: https://go.dev/play/p/bHMy6-ZLsYQ
Almost the same code, but using JSON and working: https://go.dev/play/p/V5HLrQ_-ls3
Thanks !
When using an interface in your struct, unmarshall cannot figure out which "implementation" to choose... You have to do it manually, in your case based on the "type" field. Usually unmarshall methods put a map for interface{}, a key value store.
Anyway back on your problem, you have to store the interface data in bson.Raw (slice of bytes) and do the unmarshalling manually by choosing the right struct.
package main
import (
"fmt"
"log"
"go.mongodb.org/mongo-driver/bson"
)
type foo struct {
Type string `bson:"type"`
Act ActivationInterface
}
type ActivationInterface interface{}
type Activation1 struct {
Name string `bson:"name"`
}
type Activation2 struct {
Address string `bson:"adress"`
}
func (q *foo) UnmarshalBSON(data []byte) error {
// Unmarshall only the type
fooTemp := new(struct {
Type string `bson:"type"`
Act bson.Raw
})
if err := bson.Unmarshal(data, fooTemp); err != nil {
return err
}
fmt.Println(fooTemp.Type)
// Set the type to the prop
switch fooTemp.Type {
case "act1":
// q.Act = &Activation1{}
a := Activation1{}
err := bson.Unmarshal(fooTemp.Act, &a)
if err != nil {
return err
}
q.Act = a
case "act2":
// q.Act = &Activation2{}
a := Activation2{}
err := bson.Unmarshal(fooTemp.Act, &a)
if err != nil {
return err
}
q.Act = a
default:
fmt.Println("DEFAULT")
return fmt.Errorf("unknown type: %v", fooTemp.Type)
}
return nil
}
func main() {
foo1 := foo{
Type: "act1",
Act: Activation1{
Name: "name: act1",
},
}
foo2 := foo{
Type: "act2",
Act: Activation2{
Address: "adress: act2",
},
}
// Marshal
m1, err := bson.Marshal(foo1)
if err != nil {
log.Fatal(err)
}
m2, err := bson.Marshal(foo2)
if err != nil {
log.Fatal(err)
}
//fmt.Println(m1, m2)
// Unmarshal
var u1, u2 foo
err = bson.Unmarshal(m1, &u1)
if err != nil {
fmt.Println("1 -> ", err) // error decoding key act: no decoder found for main.ActivationInterface
}
err = bson.Unmarshal(m2, &u2)
if err != nil {
fmt.Println("2 -> ", err) // error decoding key act: no decoder found for main.ActivationInterface
}
fmt.Println(foo1.Type, ":", u1.Act.(Activation1).Name)
fmt.Println(foo2.Type, ":", u2.Act.(Activation2).Address)
}
https://go.dev/play/p/CG2SlEknNrO
I have a couple of example nested structs and need to serialize them. I am using the encoding/gob library, which should convert the struct data to bytes and the encoding/base64 library to convert the bytes to a readable base64 format. However, when I run my example code I get a serialization error error. I don't understand why this happens and how to fix the problem.
I followed this example: Golang serialize and deserialize back
Here is the code:
package main
import (
"bytes"
"encoding/base64"
"encoding/gob"
"errors"
"fmt"
)
type Hello struct {
greeting string
}
type Bye struct {
helloSaid Hello
byesaid Hello
}
func (b1 Bye) Serialize() (string, error) {
b := bytes.Buffer{}
e := gob.NewEncoder(&b)
err := e.Encode(b1)
if err != nil {
return string(b.Bytes()[:]), errors.New("serialization failed")
}
return base64.StdEncoding.EncodeToString(b.Bytes()), nil
}
func DeserializeBye(str string) (Bye, error) {
m := Bye{}
by, err := base64.StdEncoding.DecodeString(str)
if err != nil {
return m, errors.New("deserialization failed")
}
b := bytes.Buffer{}
b.Write(by)
d := gob.NewDecoder(&b)
err = d.Decode(&m)
if err != nil {
return m, errors.New("deserialization failed")
}
return m, nil
}
func main() {
h := Hello{greeting: "hello"}
b := Bye{helloSaid: h, byesaid: h}
serialized, err := b.Serialize()
if err != nil {
fmt.Println(err)
}
fmt.Println(serialized)
}
Please, make the fields of the Hello and Bye structures public. Please see the documentation for the gob package:
Structs encode and decode only exported fields.
I have gobs of unknown type. Is there way to print it to view inside?
There might be gob.Debug but it is not available for me
https://golang.org/src/encoding/gob/debug.go
Googling advices to use DecodeValue but it requires initialised reflect.Value
If I get unknown gob blob then I can't pass initialized value on unknown type
https://play.golang.org/p/OWxX1kPJ6Qa
package main
import (
"bytes"
"encoding/gob"
"fmt"
"reflect"
)
func encode1() []byte {
x := "123"
buf := &bytes.Buffer{}
enc := gob.NewEncoder(buf)
err := enc.Encode(x)
if err != nil {
panic(err)
}
return buf.Bytes()
}
func decode1(b1 []byte) {
var x string
dec := gob.NewDecoder(bytes.NewBuffer(b1))
err := dec.Decode(&x)
if err != nil {
panic(err)
}
fmt.Printf("%#v\n", x)
}
func decode2(b1 []byte) {
// var x reflect.Value
x := reflect.New(reflect.TypeOf(""))
dec := gob.NewDecoder(bytes.NewBuffer(b1))
err := dec.DecodeValue(x)
if err != nil {
panic(err)
}
fmt.Printf("%#v\n", x)
fmt.Printf("%#v\n", reflect.Indirect(x))
}
func main() {
b1 := encode1()
decode1(b1)
decode2(b1)
}
you need to Register your type before encode decoding
Register records a type, identified by a value for that type, under
its internal type name. That name will identify the concrete type of a
value sent or received as an interface variable. Only types that will
be transferred as implementations of interface values need to be
registered. Expecting to be used only during initialization, it panics
if the mapping between types and names is not a bijection.
I have a struct that I want to marshal to JSON. It has a defined field called Foo (exported as foo) and a data interface field to which I want to pass a dynamic struct with additional JSON fields.
However when the data field is an interface instead of the specific struct it never gets exported as JSON. How can I make this work?
package main
import (
"encoding/json"
"fmt"
)
type data interface{}
type foo struct {
Foo string `json:"foo,omitempty"`
data
}
type bar struct {
Bar string `json:"bar,omitempty"`
}
func main() {
b := bar{"bar"}
f := foo{"foo", b}
byt, err := json.Marshal(f)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(byt))
}
I need to output to look like this (it needs to be flat, not nested):
{"foo": "foo", "bar": "bar"}
You could do this with a custom json.Marshaler implementation and a little bit of byte slicing.
func (f foo) MarshalJSON() ([]byte, error) {
type goo foo
g := goo(f)
b1, err := json.Marshal(g)
if err != nil {
return nil, err
}
b2, err := json.Marshal(g.data)
if err != nil {
return nil, err
}
s1 := string(b1[:len(b1)-1])
s2 := string(b2[1:])
return []byte(s1 + ", " + s2), nil
}
https://play.golang.org/p/NYTNWIL-xu
Please note that this is not checking whether the bytes can actually be sliced and it does also not consider the possible case of the data field being a slice or an array, which i'm unsure how you would want that flattened anyway.
I would write a custom marshaller, like so:
func (f foo) MarshalJSON() ([]byte, error) {
type tmp foo
g := tmp(f)
first, err := json.Marshal(g)
if err != nil {
return nil, err
}
second, err := json.Marshal(f.data)
if err != nil {
return nil, err
}
data := make(map[string]interface{})
json.Unmarshal(first, &data)
json.Unmarshal(second, &data)
return json.Marshal(data)
//{"bar":"bar","foo":"foo"}
}
https://play.golang.org/p/TENiCe9nR0
2 options:
Set to it type json.RawMessage so it won’t be decoded automatically and left as an interface. https://golang.org/pkg/encoding/json/#RawMessage
Write custom unmarshaler on the structure.
I'm still new to Go and trying to use Beego's cache. I can put a []map[string]string into the cache but can't figure out how to convert the value back to a []map[string]string.
For instance, to put the item in the cache:
m:=make([]map[string]string)
// add items to the slice of maps
.......
// cache it
if err := c.Put("key", m, 100); err != nil {
fmt.Println(err)
}
// retrieve it
n := c.Get("key")
fmt.Println(reflect.TypeOf(n)) // ==>string
// failed attempt
a := n.([]map[string]string)
fmt.Println(a) // panic: interface conversion: interface is string, not []map[string]string
How do I convert n to a slice of maps?
well digging into the code seems like even if it says interface{} what it does it actually squash everything to []byte
https://github.com/astaxie/beego/blob/master/cache/memcache/memcache.go#L77
and when it Does the Get convert everything to string
https://github.com/astaxie/beego/blob/master/cache/memcache/memcache.go#L61
So you need to Marshal / Unmarshal the data structure yourself.
See this example and substitute your calls and error check with the dummies one provided:
http://play.golang.org/p/9z3KcOlgAx
package main
import (
"bytes"
"encoding/json"
"log"
)
var cache map[string]string = make(map[string]string)
func Put(key, value string) {
cache[key] = value
}
func Get(key string) string {
return cache[key]
}
func main() {
m := map[string]string{
"A": "1",
"B": "2",
}
if b, err := json.Marshal(m); err != nil {
log.Fatal(err)
} else {
Put("myKey", string(b))
}
b := bytes.NewBufferString(Get("myKey"))
var mm map[string]string
if err := json.Unmarshal(b.Bytes(), &mm); err != nil {
log.Fatal(err)
}
log.Printf("%#v", mm)
}