I have a struct that I want to marshal to JSON. It has a defined field called Foo (exported as foo) and a data interface field to which I want to pass a dynamic struct with additional JSON fields.
However when the data field is an interface instead of the specific struct it never gets exported as JSON. How can I make this work?
package main
import (
"encoding/json"
"fmt"
)
type data interface{}
type foo struct {
Foo string `json:"foo,omitempty"`
data
}
type bar struct {
Bar string `json:"bar,omitempty"`
}
func main() {
b := bar{"bar"}
f := foo{"foo", b}
byt, err := json.Marshal(f)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(byt))
}
I need to output to look like this (it needs to be flat, not nested):
{"foo": "foo", "bar": "bar"}
You could do this with a custom json.Marshaler implementation and a little bit of byte slicing.
func (f foo) MarshalJSON() ([]byte, error) {
type goo foo
g := goo(f)
b1, err := json.Marshal(g)
if err != nil {
return nil, err
}
b2, err := json.Marshal(g.data)
if err != nil {
return nil, err
}
s1 := string(b1[:len(b1)-1])
s2 := string(b2[1:])
return []byte(s1 + ", " + s2), nil
}
https://play.golang.org/p/NYTNWIL-xu
Please note that this is not checking whether the bytes can actually be sliced and it does also not consider the possible case of the data field being a slice or an array, which i'm unsure how you would want that flattened anyway.
I would write a custom marshaller, like so:
func (f foo) MarshalJSON() ([]byte, error) {
type tmp foo
g := tmp(f)
first, err := json.Marshal(g)
if err != nil {
return nil, err
}
second, err := json.Marshal(f.data)
if err != nil {
return nil, err
}
data := make(map[string]interface{})
json.Unmarshal(first, &data)
json.Unmarshal(second, &data)
return json.Marshal(data)
//{"bar":"bar","foo":"foo"}
}
https://play.golang.org/p/TENiCe9nR0
2 options:
Set to it type json.RawMessage so it won’t be decoded automatically and left as an interface. https://golang.org/pkg/encoding/json/#RawMessage
Write custom unmarshaler on the structure.
Related
I have a very interesting case, let's say we have this struct
type Test struct {
Field1 string `json:"field1"`
Field2 ABC `json:"abc"`
}
type ABC interface {
Rest()
}
Unmarshalling this struct is not a problem, you could just point to the right struct which implements the interface, unless you have []Test
Is there a way to unmarshall slice of structs when one of the field is interface?
Thanks
You need to implement Unmarshaler interface to Test,
Then in UnmarshalJSON func you need to check it one by one (line 45-55 in the example)
Luckily there is now generic, this is the example :
package main
import (
"encoding/json"
"fmt"
"reflect"
)
type Test struct {
Field1 string `json:"field1"`
Field2 ABC `json:"abc"`
}
type ABCImplements interface {
A | B
}
func UnmarshalABC[T ABCImplements](b []byte) (T, error) {
var x T
err := json.Unmarshal(b, &x)
if err != nil {
return x, err
}
rv := reflect.ValueOf(x)
if rv.IsZero() {
return x, fmt.Errorf("T is zero value")
}
return x, nil
}
func (m *Test) UnmarshalJSON(data []byte) error {
temp := make(map[string]interface{}, 2)
err := json.Unmarshal(data, &temp)
if err != nil {
return err
}
m.Field1 = temp["field1"].(string)
b, err := json.Marshal(temp["abc"])
if err != nil {
return err
}
xB, err := UnmarshalABC[B](b)
if err == nil {
m.Field2 = xB
return nil
}
xA, err := UnmarshalABC[A](b)
if err == nil {
m.Field2 = &xA
return nil
}
return nil
}
type A struct {
B string `json:"b"`
}
func (a *A) Rest() {
fmt.Println(a.B)
}
type B struct {
C string `json:"c"`
}
func (b B) Rest() {
fmt.Println(b.C)
}
type ABC interface {
Rest()
}
func main() {
a := &A{"coba"}
t := Test{"oke", a}
arrT := []Test{t, t, t}
b, err := json.Marshal(arrT)
if err != nil {
panic(err)
}
var xT []Test
err = json.Unmarshal(b, &xT)
fmt.Printf("%#v\n", xT)
fmt.Println(xT[1].Field2)
}
playground
Use the following code to Unmarshal the interface field to a specific concrete type. See the commentary for more details:
// Example is the concrete type.
type Example struct{ Hello string }
func (e *Example) Rest() {}
// Implement the json.Unmarshaler interface to control how
// values of type Test are unmarsheled.
func (m *Test) UnmarshalJSON(data []byte) error {
// Setup fields as needed.
m.Field2 = &Example{}
// We cannot call json.Unmarshal(data, m) to do the work
// because the json.Unmarshal will recurse back to this
// function. To prevent the recursion, we declare a new
// type with the same field layout as Test, but no methods:
type t Test
// Convert receiver m to a t* and unmarshal using that pointer.
v := (*t)(m)
return json.Unmarshal(data, v)
}
Run the code on the playground.
I have a couple of example nested structs and need to serialize them. I am using the encoding/gob library, which should convert the struct data to bytes and the encoding/base64 library to convert the bytes to a readable base64 format. However, when I run my example code I get a serialization error error. I don't understand why this happens and how to fix the problem.
I followed this example: Golang serialize and deserialize back
Here is the code:
package main
import (
"bytes"
"encoding/base64"
"encoding/gob"
"errors"
"fmt"
)
type Hello struct {
greeting string
}
type Bye struct {
helloSaid Hello
byesaid Hello
}
func (b1 Bye) Serialize() (string, error) {
b := bytes.Buffer{}
e := gob.NewEncoder(&b)
err := e.Encode(b1)
if err != nil {
return string(b.Bytes()[:]), errors.New("serialization failed")
}
return base64.StdEncoding.EncodeToString(b.Bytes()), nil
}
func DeserializeBye(str string) (Bye, error) {
m := Bye{}
by, err := base64.StdEncoding.DecodeString(str)
if err != nil {
return m, errors.New("deserialization failed")
}
b := bytes.Buffer{}
b.Write(by)
d := gob.NewDecoder(&b)
err = d.Decode(&m)
if err != nil {
return m, errors.New("deserialization failed")
}
return m, nil
}
func main() {
h := Hello{greeting: "hello"}
b := Bye{helloSaid: h, byesaid: h}
serialized, err := b.Serialize()
if err != nil {
fmt.Println(err)
}
fmt.Println(serialized)
}
Please, make the fields of the Hello and Bye structures public. Please see the documentation for the gob package:
Structs encode and decode only exported fields.
I've implemented a custom JSON unmarshaler, but for some reason it won't return the proper value -all fields come back nil.
For example:
type test struct {
t string
}
func New(data string) (*test, error) {
return &test{t: data}, nil
}
func (t *test) UnmarshalJSON(b []byte) error {
tt, err := New(string(b))
if err != nil {
return err
}
t = tt
return nil
}
func main() {
str := `"hello"`
b := []byte(str)
t := &test{}
err := json.Unmarshal(b, t)
if err != nil {
fmt.Printf("unmarshal error occurred: %#v", err)
}
fmt.Printf("%#v", t)
}
https://play.golang.org/p/LuXkZQZHWz
The above code shows the output: &main.test{t:""}
Why doesn't it unmarshal the fields? i.e &main.test{t:"hello"}
Only when I dereference the pointers above, do I get the desired result.
i.e -
func (t *test) UnmarshalJSON(b []byte) error {
tt, err := New(string(b))
if err != nil {
return err
}
*t = *tt
return nil
}
You're assigning the local variable t, a pointer to test, to the value of the local variable tt, also a pointer to test. This has no effect on the value the original pointer t pointed to. You have to dereference the pointers to change the value it points to, rather than changing the local pointer itself:
*t = *tt
Ok, the title is a little bit misleading. What I'm after is as follows:
type MyStruct struct {
id int
name string
age int
}
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
var (
id int
name string
age int
ok bool
)
err := errors.New("Error!")
id, ok = m["id"].(int)
if !ok {
return nil, err
}
name, ok = m["name"].(string)
if !ok {
return nil, err
}
age, ok = m["age"].(int)
if !ok {
return nil, err
}
return MyStruct{id, name, age}, nil
}
Don't ask: Why I'm not using CreateFromMap(int, string, int). That object comes from somewhere else, out of my control.
It's already boring to map each key, value pair in the map to struct properties. But checking if everything is ok or not after each conversion is chaotic.
Is there an easier way of doing this other than reflection?
Let's say I assume you don't want to use reflection because you don't want to do it yourself. In this case, what about using an external package that does it for you ?
package main
import "fmt"
import "github.com/mitchellh/mapstructure"
type MyStruct struct {
Id int
Name string
Age int
}
func main() {
var m = make(map[string]interface{})
m["Id"] = 17
m["Name"] = "foo"
m["Age"] = 42
fmt.Printf("%+v\n", m)
res, err := CreateFromMap(m)
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%+v\n", res)
}
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
var result MyStruct
err := mapstructure.Decode(m, &result)
return result, err
}
Output:
map[Age:42 Name:foo Id:17]
{Id:17 Name:foo Age:42}
It has the advantage to work whatever your structure looks like, but it uses reflection internally. Actually, I don't see any "nice" way to do what you want without using reflection and/or repetitive code for each attribute of your structure. The downside, however, is that you will have to use capitalized attributes so that they would be exported to external packages.
Edit (to answer your question in the comments):
On my opinion, if you want to specify additional rules when "creating" the structure, it should be after the decoding operation.
For instance:
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
var result MyStruct
err := mapstructure.Decode(m, &result)
if err != nil {
return result, err
}
if result.Age <= 0 {
result.Age = 0
}
if result.Name == "" {
return result, errors.New("empty name is not allowed")
}
return result, err
}
This way, you will clearly separate the "conversion" part from your structure's specific rules processing.
You can just Marshal/Unmarshal, but property names should match
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
data, _ := json.Marshal(m)
var result MyStruct
err := json.Unmarshal(data, &result)
return result, err
}
I have read the documentation of ( gob) and I have some problems :
Now I know how to encode structure and decode like that:
func main() {
s1 := &S{
Field1: "Hello Gob",
Field2: 999,
}
log.Println("Original value:", s1)
buf := new(bytes.Buffer)
err := gob.NewEncoder(buf).Encode(s1)
if err != nil {
log.Println("Encode:", err)
return
}
s2 := &S{}
err = gob.NewDecoder(buf).Decode(s2)
if err != nil {
log.Println("Decode:", err)
return
}
log.Println("Decoded value:", s2)
}
But I don't know the purpose of this method gob.Register() can someone explain to me when to use it and why?
If you're dealing with concrete types (structs) only, you don't really need it. Once you're dealing with interfaces you must register your concrete type first.
For example, let's assume we have these struct and interface (the struct implements the interface):
type Getter interface {
Get() string
}
type Foo struct {
Bar string
}
func (f Foo)Get() string {
return f.Bar
}
To send a Foo over gob as a Getter and decode it back, we must first call
gob.Register(Foo{})
So the flow would be:
// init and register
buf := bytes.NewBuffer(nil)
gob.Register(Foo{})
// create a getter of Foo
g := Getter(Foo{"wazzup"})
// encode
enc := gob.NewEncoder(buf)
enc.Encode(&g)
// decode
dec := gob.NewDecoder(buf)
var gg Getter
if err := dec.Decode(&gg); err != nil {
panic(err)
}
Now try removing the Register and this won't work because gob wouldn't know how to map things back to their appropriate type.
As http://golang.org/pkg/encoding/gob/#Register said:
Only types that will be transferred as implementations of interface
values need to be registered.
So it doesn't needed by your demo.
If you want to encode / decode a map[string]interface{}, since the field of the map is enclosed as interface type, then we need to register the specific type before.
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type SomeStruct struct {
Text string
}
func main() {
var bytes bytes.Buffer
// Remove one of these, then the decoding will produce error
gob.Register(SomeStruct{})
gob.Register([]interface{}{})
gob.Register([]SomeStruct{})
gob.Register(map[string]SomeStruct{})
writer := gob.NewEncoder(&bytes)
err := writer.Encode(map[string]interface{}{
"SomeStruct": SomeStruct{"Halo"},
"SomeSlice": []interface{}{},
"SomeSliceStruct": []SomeStruct{
{
Text: "SomeText",
},
},
"SomeMapStruct": map[string]SomeStruct{
"S": {"Test"},
},
})
if err != nil {
log.Fatalf("Error on encode process: %v\n", err)
return
}
reader := gob.NewDecoder(&bytes)
var aMap map[string]interface{}
err = reader.Decode(&aMap)
if err != nil {
log.Fatalf("Error on decode process: %v\n", err)
return
}
fmt.Printf("Decode is successful: %+v\n", aMap)
}