Unmarshal slice of structs with interfaces - go

I have a very interesting case, let's say we have this struct
type Test struct {
Field1 string `json:"field1"`
Field2 ABC `json:"abc"`
}
type ABC interface {
Rest()
}
Unmarshalling this struct is not a problem, you could just point to the right struct which implements the interface, unless you have []Test
Is there a way to unmarshall slice of structs when one of the field is interface?
Thanks

You need to implement Unmarshaler interface to Test,
Then in UnmarshalJSON func you need to check it one by one (line 45-55 in the example)
Luckily there is now generic, this is the example :
package main
import (
"encoding/json"
"fmt"
"reflect"
)
type Test struct {
Field1 string `json:"field1"`
Field2 ABC `json:"abc"`
}
type ABCImplements interface {
A | B
}
func UnmarshalABC[T ABCImplements](b []byte) (T, error) {
var x T
err := json.Unmarshal(b, &x)
if err != nil {
return x, err
}
rv := reflect.ValueOf(x)
if rv.IsZero() {
return x, fmt.Errorf("T is zero value")
}
return x, nil
}
func (m *Test) UnmarshalJSON(data []byte) error {
temp := make(map[string]interface{}, 2)
err := json.Unmarshal(data, &temp)
if err != nil {
return err
}
m.Field1 = temp["field1"].(string)
b, err := json.Marshal(temp["abc"])
if err != nil {
return err
}
xB, err := UnmarshalABC[B](b)
if err == nil {
m.Field2 = xB
return nil
}
xA, err := UnmarshalABC[A](b)
if err == nil {
m.Field2 = &xA
return nil
}
return nil
}
type A struct {
B string `json:"b"`
}
func (a *A) Rest() {
fmt.Println(a.B)
}
type B struct {
C string `json:"c"`
}
func (b B) Rest() {
fmt.Println(b.C)
}
type ABC interface {
Rest()
}
func main() {
a := &A{"coba"}
t := Test{"oke", a}
arrT := []Test{t, t, t}
b, err := json.Marshal(arrT)
if err != nil {
panic(err)
}
var xT []Test
err = json.Unmarshal(b, &xT)
fmt.Printf("%#v\n", xT)
fmt.Println(xT[1].Field2)
}
playground

Use the following code to Unmarshal the interface field to a specific concrete type. See the commentary for more details:
// Example is the concrete type.
type Example struct{ Hello string }
func (e *Example) Rest() {}
// Implement the json.Unmarshaler interface to control how
// values of type Test are unmarsheled.
func (m *Test) UnmarshalJSON(data []byte) error {
// Setup fields as needed.
m.Field2 = &Example{}
// We cannot call json.Unmarshal(data, m) to do the work
// because the json.Unmarshal will recurse back to this
// function. To prevent the recursion, we declare a new
// type with the same field layout as Test, but no methods:
type t Test
// Convert receiver m to a t* and unmarshal using that pointer.
v := (*t)(m)
return json.Unmarshal(data, v)
}
Run the code on the playground.

Related

Go: Implementing a ManyDecode for a "set" of individual results

I have implemented a very simple Decode method (using gob.Decoder for now) - this works well for single responses - it would even work well for slices, but I need to implement a DecodeMany method where it is able to decode a set of individual responses (not a slice).
Working Decode method:
var v MyType
_ = Decode(&v)
...
func Decode(v interface{}) error {
buf, _ := DoSomething() // func DoSomething() ([]byte, error)
// error handling omitted for brevity
return gob.NewDecoder(bytes.NewReader(buf)).Decode(v)
}
What I'm trying to do for a DecodeMany method is to deal with a response that isn't necessarily a slice:
var vv []MyType
_ = DecodeMany(&vv)
...
func DecodeMany(vv []interface{}) error {
for _, g := range DoSomething() { // func DoSomething() []struct{Buf []bytes}
// Use g.Buf as an individual "interface{}"
// want something like:
var v interface{} /* Somehow create instance of single vv type? */
_ = gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(v)
vv = append(vv, v)
}
return
}
Besides not compiling the above also has the error of:
cannot use &vv (value of type *[]MyType) as type []interface{} in argument to DecodeMany
If you want to modify the passed slice, it must be a pointer, else you must return a new slice. Also if the function is declared to have a param of type []interface{}, you can only pass a value of type []interface{} and no other slice types... Unless you use generics...
This is a perfect example to start using generics introduced in Go 1.18.
Change DecodeMany() to be generic, having a T type parameter being the slice element type:
When taking a pointer
func DecodeMany[T any](vv *[]T) error {
for _, g := range DoSomething() {
var v T
if err := gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(&v); err != nil {
return err
}
*vv = append(*vv, v)
}
return nil
}
Here's a simple app to test it:
type MyType struct {
S int64
}
func main() {
var vv []MyType
if err := DecodeMany(&vv); err != nil {
panic(err)
}
fmt.Println(vv)
}
func DoSomething() (result []struct{ Buf []byte }) {
for i := 3; i < 6; i++ {
buf := &bytes.Buffer{}
v := MyType{S: int64(i)}
if err := gob.NewEncoder(buf).Encode(v); err != nil {
panic(err)
}
result = append(result, struct{ Buf []byte }{buf.Bytes()})
}
return
}
This outputs (try it on the Go Playground):
[{3} {4} {5}]
When returning a slice
If you choose to return the slice, you don't have to pass anything, but you need to assign the result:
func DecodeMany[T any]() ([]T, error) {
var result []T
for _, g := range DoSomething() {
var v T
if err := gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(&v); err != nil {
return result, err
}
result = append(result, v)
}
return result, nil
}
Using it:
vv, err := DecodeMany[MyType]()
if err != nil {
panic(err)
}
fmt.Println(vv)
Try this one on the Go Playground.

String to float64 receiving format ".01"

If I receive from an API a string obeying the format of ".01", and I have a struct like this:
type Mystruct struct {
Val float64 json:"val,string"
}
In this case, I receive trying to unmarshal val into float64. Is there a way I can accomplish this?
Add a string field to capture the string value:
type Mystruct struct {
Val float64 `json:"-"`
XVal string `json:"val"`
}
Unmarshal the JSON document. Convert the string value to a float value:
var v Mystruct
err := json.Unmarshal([]byte(data), &v)
if err != nil {
log.Fatal(err)
}
v.Val, err = strconv.ParseFloat(v.XVal, 64)
if err != nil {
log.Fatal(err)
}
I recommand defining a type alias which you can use it anywhere.
package main
import (
"encoding/json"
"fmt"
"strconv"
"strings"
)
type MyFloat64 float64
func (f *MyFloat64) UnmarshalJSON(data []byte) error {
raw := string(data)
raw = strings.TrimPrefix(raw, "\"")
raw = strings.TrimSuffix(raw, "\"")
if parsedFloat, err := strconv.ParseFloat(raw, 64); err != nil {
return err
} else {
*f = MyFloat64(parsedFloat)
return nil
}
}
type MyObj struct {
Val1 MyFloat64
Val2 string
}
func main() {
j := `{"Val1":"0.01", "Val2":"0.01"}`
o := MyObj{}
err := json.Unmarshal([]byte(j), &o)
if err != nil {
fmt.Println(err)
} else {
b, _ := json.Marshal(o)
fmt.Println("in:", j)
fmt.Println("out:", string(b))
}
}
output:
in: {"Val1":"0.01", "Val2":"0.01"}
out: {"Val1":0.01,"Val2":"0.01"}

Transform struct to slice struct

I'm trying to select a struct by string input and then depending on the return JSON Object or Array, unmarshall the JSON. Is it correct to think of a way to reflect the struct to slice struct? if so how to do that with reflection?
Regards,
Peter
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
)
type NameStruct struct {
Name string
}
func main() {
jsonData := []byte(`[{"name":"james"},{"name":"steven"}]`)
returnModel := InitializeModel("NameStruct", jsonData)
fmt.Println(returnModel)
jsonData = []byte(`{"name":"james"}`)
returnModel = InitializeModel("NameStruct", jsonData)
fmt.Println(returnModel)
}
func getModelByName(modelType string) interface{} {
modelMap := make(map[string]interface{})
modelMap["NameStruct"] = new(NameStruct)
//don't want to do this
modelMap["arrNameStruct"] = new([]NameStruct)
return modelMap[modelType]
}
func InitializeModel(modelName string, jsonData []byte) interface{} {
switch IsArray(jsonData) {
case true:
// some conversion here, how?
returnModel := getModelByName("NameStruct")
if err := json.Unmarshal(jsonData, &returnModel); err != nil {
log.Println(err)
}
return returnModel
case false:
returnModel := getModelByName("NameStruct")
if err := json.Unmarshal(jsonData, &returnModel); err != nil {
log.Println(err)
}
return returnModel
}
return nil
}
func IsArray(jsonData []byte) bool {
return (bytes.HasPrefix(jsonData, []byte("["))) && (bytes.HasSuffix(jsonData, []byte("]")))
}
Expanding on my comment, you can create a Factory where pre-defined types are registered:
type Factory struct {
m map[string]reflect.Type
}
func (f *Factory) Register(v interface{}) {
vt := reflect.TypeOf(v)
n := vt.Name()
f.m[n] = vt
f.m["[]"+n] = reflect.SliceOf(vt) // implicitly register a slice of type too
}
these types can be looked up by name at runtime and initialized with JSON data:
func (f *Factory) Make(k string, bs []byte) (interface{}, error) {
vt, ok := f.m[k]
if !ok {
return nil, fmt.Errorf("type %q not registered", k)
}
pv := reflect.New(vt).Interface()
err := json.Unmarshal(bs, pv)
if err != nil {
return nil, err
}
return pv, nil
}
To use:
type Place struct {
City string `json:"city"`
}
factory.Register(Place{})
p, err := factory.Make("Place", []byte(`{"city":"NYC"}`))
fmt.Printf("%#v\n", p) // &main.Place{City:"NYC"}
Slices also work:
ps, err := factory.Make("[]Place", []byte(`[{"city":"NYC"},{"city":"Dublin"}]`))
fmt.Printf("%#v\n", p, p) // &[]main.Place{main.Place{City:"NYC"}, main.Place{City:"Dublin"}}
Playground: https://play.golang.org/p/qWEdwk-YUug

Is there a reason why bb.person gets filled but a.Person does not?

package main
import (
"encoding/json"
"fmt"
)
type Person struct {
First string `json:"name"`
}
type person struct {
Last string `json:"name"`
}
type A struct {
*Person `json:"person"`
}
func (a *A) UnmarshalJSON(b []byte) error {
type alias A
bb := struct {
*person `json:"person"`
*alias
}{
alias: (*alias)(a),
}
if err := json.Unmarshal(b, &bb); err != nil {
return err
}
fmt.Printf("%+v\n", bb.person)
return nil
}
func main() {
b := []byte(`{"person": {"name": "bob"}}`)
a := &A{}
if err := json.Unmarshal(b, a); err != nil {
fmt.Println(err)
}
fmt.Printf("%+v\n", a.Person)
}
results in:
&{Last:bob}
<nil>
Why does bb.person and a.Person have the same struct tag but only bb.person gets filled in? I haven't been able to find the appropriate documentation but why does this happen and is it guaranteed to always happen?
https://play.golang.org/p/Fvo_hg3U6r

Using sqlx.Rows.StructScan for interface args

I need to use StructScan function for interface (pointer to struct).
But if I try to reflect value, I've got error, because reflect.New() returning reflect.Value type. How I can scan structure and store data into dest var?
// package 1
type Data struct {
id int `db:"id"`
caption string `db:"caption"`
}
func Func1 {
data := []Data{}
GetData(&data)
log.Println(data)
}
// package 2
func GetData(sql string, dest interface{}) {
rows, err := DBI.Queryx(sql)
if err == nil {
// reflect.Value
myData := reflect.New(reflect.TypeOf(dest).Elem().Elem())
for rows.Next() {
rows.StructScan(&myData) // Fail here
}
}
}
Solved
// package 2
func GetData(sql string, dest interface{}) {
arr := reflect.ValueOf(dest).Elem()
v := reflect.New(reflect.TypeOf(dest).Elem().Elem())
rows, err := DBI.Queryx(sql)
if err == nil {
if err = rows.StructScan(v.Interface()); err == nil {
arr.Set(reflect.Append(arr, v.Elem()))
}
}
}

Resources