I have gobs of unknown type. Is there way to print it to view inside?
There might be gob.Debug but it is not available for me
https://golang.org/src/encoding/gob/debug.go
Googling advices to use DecodeValue but it requires initialised reflect.Value
If I get unknown gob blob then I can't pass initialized value on unknown type
https://play.golang.org/p/OWxX1kPJ6Qa
package main
import (
"bytes"
"encoding/gob"
"fmt"
"reflect"
)
func encode1() []byte {
x := "123"
buf := &bytes.Buffer{}
enc := gob.NewEncoder(buf)
err := enc.Encode(x)
if err != nil {
panic(err)
}
return buf.Bytes()
}
func decode1(b1 []byte) {
var x string
dec := gob.NewDecoder(bytes.NewBuffer(b1))
err := dec.Decode(&x)
if err != nil {
panic(err)
}
fmt.Printf("%#v\n", x)
}
func decode2(b1 []byte) {
// var x reflect.Value
x := reflect.New(reflect.TypeOf(""))
dec := gob.NewDecoder(bytes.NewBuffer(b1))
err := dec.DecodeValue(x)
if err != nil {
panic(err)
}
fmt.Printf("%#v\n", x)
fmt.Printf("%#v\n", reflect.Indirect(x))
}
func main() {
b1 := encode1()
decode1(b1)
decode2(b1)
}
you need to Register your type before encode decoding
Register records a type, identified by a value for that type, under
its internal type name. That name will identify the concrete type of a
value sent or received as an interface variable. Only types that will
be transferred as implementations of interface values need to be
registered. Expecting to be used only during initialization, it panics
if the mapping between types and names is not a bijection.
Related
I am attempting to unmarshal a custom/alias Hash of type [32]byte but the results are returning the zero value. The following code is just a snippet of what I am trying to do. In this snippet I am producing the json just to give an example but in my actual codebase it would be produced and read from another source. Here is the snippet:
package main
import (
"os"
"fmt"
"crypto/sha256"
"encoding/hex"
"encoding/json"
)
type Block struct {
Key Hash `json:"hash"`
}
type Hash [32]byte
func (h Hash) MarshalText() ([]byte, error) {
return []byte(hex.EncodeToString(h[:])), nil
}
func (h Hash) UnmarshalText(data []byte) error {
_, err := hex.Decode(h[:], data)
return err
}
func main() {
b := Block{sha256.Sum256([]byte("afasdfasfasfjiuoiuioupio"))}
j, _ := json.Marshal(b)
var unmarshalled Block
err := json.Unmarshal(j, &unmarshalled)
if err != nil {
os.Exit(1)
}
fmt.Printf("%x\n", unmarshalled.Key)
// prints out 0000000000000000000000000000000000000000000000000000000000000000
}
The UnmarshalText method modifies the value h, not the value in the caller. Use a pointer receiver to modify the caller's value.
func (h *Hash) UnmarshalText(data []byte) error {
_, err := hex.Decode(h[:], data)
return err
}
I have a couple of example nested structs and need to serialize them. I am using the encoding/gob library, which should convert the struct data to bytes and the encoding/base64 library to convert the bytes to a readable base64 format. However, when I run my example code I get a serialization error error. I don't understand why this happens and how to fix the problem.
I followed this example: Golang serialize and deserialize back
Here is the code:
package main
import (
"bytes"
"encoding/base64"
"encoding/gob"
"errors"
"fmt"
)
type Hello struct {
greeting string
}
type Bye struct {
helloSaid Hello
byesaid Hello
}
func (b1 Bye) Serialize() (string, error) {
b := bytes.Buffer{}
e := gob.NewEncoder(&b)
err := e.Encode(b1)
if err != nil {
return string(b.Bytes()[:]), errors.New("serialization failed")
}
return base64.StdEncoding.EncodeToString(b.Bytes()), nil
}
func DeserializeBye(str string) (Bye, error) {
m := Bye{}
by, err := base64.StdEncoding.DecodeString(str)
if err != nil {
return m, errors.New("deserialization failed")
}
b := bytes.Buffer{}
b.Write(by)
d := gob.NewDecoder(&b)
err = d.Decode(&m)
if err != nil {
return m, errors.New("deserialization failed")
}
return m, nil
}
func main() {
h := Hello{greeting: "hello"}
b := Bye{helloSaid: h, byesaid: h}
serialized, err := b.Serialize()
if err != nil {
fmt.Println(err)
}
fmt.Println(serialized)
}
Please, make the fields of the Hello and Bye structures public. Please see the documentation for the gob package:
Structs encode and decode only exported fields.
I have a struct that I want to marshal to JSON. It has a defined field called Foo (exported as foo) and a data interface field to which I want to pass a dynamic struct with additional JSON fields.
However when the data field is an interface instead of the specific struct it never gets exported as JSON. How can I make this work?
package main
import (
"encoding/json"
"fmt"
)
type data interface{}
type foo struct {
Foo string `json:"foo,omitempty"`
data
}
type bar struct {
Bar string `json:"bar,omitempty"`
}
func main() {
b := bar{"bar"}
f := foo{"foo", b}
byt, err := json.Marshal(f)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(byt))
}
I need to output to look like this (it needs to be flat, not nested):
{"foo": "foo", "bar": "bar"}
You could do this with a custom json.Marshaler implementation and a little bit of byte slicing.
func (f foo) MarshalJSON() ([]byte, error) {
type goo foo
g := goo(f)
b1, err := json.Marshal(g)
if err != nil {
return nil, err
}
b2, err := json.Marshal(g.data)
if err != nil {
return nil, err
}
s1 := string(b1[:len(b1)-1])
s2 := string(b2[1:])
return []byte(s1 + ", " + s2), nil
}
https://play.golang.org/p/NYTNWIL-xu
Please note that this is not checking whether the bytes can actually be sliced and it does also not consider the possible case of the data field being a slice or an array, which i'm unsure how you would want that flattened anyway.
I would write a custom marshaller, like so:
func (f foo) MarshalJSON() ([]byte, error) {
type tmp foo
g := tmp(f)
first, err := json.Marshal(g)
if err != nil {
return nil, err
}
second, err := json.Marshal(f.data)
if err != nil {
return nil, err
}
data := make(map[string]interface{})
json.Unmarshal(first, &data)
json.Unmarshal(second, &data)
return json.Marshal(data)
//{"bar":"bar","foo":"foo"}
}
https://play.golang.org/p/TENiCe9nR0
2 options:
Set to it type json.RawMessage so it won’t be decoded automatically and left as an interface. https://golang.org/pkg/encoding/json/#RawMessage
Write custom unmarshaler on the structure.
I have read the documentation of ( gob) and I have some problems :
Now I know how to encode structure and decode like that:
func main() {
s1 := &S{
Field1: "Hello Gob",
Field2: 999,
}
log.Println("Original value:", s1)
buf := new(bytes.Buffer)
err := gob.NewEncoder(buf).Encode(s1)
if err != nil {
log.Println("Encode:", err)
return
}
s2 := &S{}
err = gob.NewDecoder(buf).Decode(s2)
if err != nil {
log.Println("Decode:", err)
return
}
log.Println("Decoded value:", s2)
}
But I don't know the purpose of this method gob.Register() can someone explain to me when to use it and why?
If you're dealing with concrete types (structs) only, you don't really need it. Once you're dealing with interfaces you must register your concrete type first.
For example, let's assume we have these struct and interface (the struct implements the interface):
type Getter interface {
Get() string
}
type Foo struct {
Bar string
}
func (f Foo)Get() string {
return f.Bar
}
To send a Foo over gob as a Getter and decode it back, we must first call
gob.Register(Foo{})
So the flow would be:
// init and register
buf := bytes.NewBuffer(nil)
gob.Register(Foo{})
// create a getter of Foo
g := Getter(Foo{"wazzup"})
// encode
enc := gob.NewEncoder(buf)
enc.Encode(&g)
// decode
dec := gob.NewDecoder(buf)
var gg Getter
if err := dec.Decode(&gg); err != nil {
panic(err)
}
Now try removing the Register and this won't work because gob wouldn't know how to map things back to their appropriate type.
As http://golang.org/pkg/encoding/gob/#Register said:
Only types that will be transferred as implementations of interface
values need to be registered.
So it doesn't needed by your demo.
If you want to encode / decode a map[string]interface{}, since the field of the map is enclosed as interface type, then we need to register the specific type before.
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type SomeStruct struct {
Text string
}
func main() {
var bytes bytes.Buffer
// Remove one of these, then the decoding will produce error
gob.Register(SomeStruct{})
gob.Register([]interface{}{})
gob.Register([]SomeStruct{})
gob.Register(map[string]SomeStruct{})
writer := gob.NewEncoder(&bytes)
err := writer.Encode(map[string]interface{}{
"SomeStruct": SomeStruct{"Halo"},
"SomeSlice": []interface{}{},
"SomeSliceStruct": []SomeStruct{
{
Text: "SomeText",
},
},
"SomeMapStruct": map[string]SomeStruct{
"S": {"Test"},
},
})
if err != nil {
log.Fatalf("Error on encode process: %v\n", err)
return
}
reader := gob.NewDecoder(&bytes)
var aMap map[string]interface{}
err = reader.Decode(&aMap)
if err != nil {
log.Fatalf("Error on decode process: %v\n", err)
return
}
fmt.Printf("Decode is successful: %+v\n", aMap)
}
gob fails to encode map[string]interface{}
gob: type not registered for interface: map[string]interface {}
http://play.golang.org/p/Si4hd8I0JE
package main
import (
"bytes"
"encoding/gob"
"encoding/json"
"fmt"
"log"
)
func CloneObject(a, b interface{}) []byte {
buff := new(bytes.Buffer)
enc := gob.NewEncoder(buff)
dec := gob.NewDecoder(buff)
err := enc.Encode(a)
if err != nil {
log.Panic("e1: ", err)
}
b1 := buff.Bytes()
err = dec.Decode(b)
if err != nil {
log.Panic("e2: ", err)
}
return b1
}
func main() {
var a interface{}
a = map[string]interface{}{"X": 1}
b2, err := json.Marshal(&a)
fmt.Println(string(b2), err)
var b interface{}
b1 := CloneObject(&a, &b)
fmt.Println(string(b1))
}
Is it possible to encode map[string]interface{} in gob?
I am able to encode it with JSON
add
gob.Register(map[string]interface{}{})
http://play.golang.org/p/Dd3IzJgl0A
Probably yes, but you do have to Register your type beforehand. See http://golang.org/pkg/encoding/gob/#Register.
The details are documented in http://golang.org/pkg/encoding/gob/#hdr-Encoding_Details
(It really does help to look at the Go documentation :-)