Convert map to struct - go

Ok, the title is a little bit misleading. What I'm after is as follows:
type MyStruct struct {
id int
name string
age int
}
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
var (
id int
name string
age int
ok bool
)
err := errors.New("Error!")
id, ok = m["id"].(int)
if !ok {
return nil, err
}
name, ok = m["name"].(string)
if !ok {
return nil, err
}
age, ok = m["age"].(int)
if !ok {
return nil, err
}
return MyStruct{id, name, age}, nil
}
Don't ask: Why I'm not using CreateFromMap(int, string, int). That object comes from somewhere else, out of my control.
It's already boring to map each key, value pair in the map to struct properties. But checking if everything is ok or not after each conversion is chaotic.
Is there an easier way of doing this other than reflection?

Let's say I assume you don't want to use reflection because you don't want to do it yourself. In this case, what about using an external package that does it for you ?
package main
import "fmt"
import "github.com/mitchellh/mapstructure"
type MyStruct struct {
Id int
Name string
Age int
}
func main() {
var m = make(map[string]interface{})
m["Id"] = 17
m["Name"] = "foo"
m["Age"] = 42
fmt.Printf("%+v\n", m)
res, err := CreateFromMap(m)
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%+v\n", res)
}
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
var result MyStruct
err := mapstructure.Decode(m, &result)
return result, err
}
Output:
map[Age:42 Name:foo Id:17]
{Id:17 Name:foo Age:42}
It has the advantage to work whatever your structure looks like, but it uses reflection internally. Actually, I don't see any "nice" way to do what you want without using reflection and/or repetitive code for each attribute of your structure. The downside, however, is that you will have to use capitalized attributes so that they would be exported to external packages.
Edit (to answer your question in the comments):
On my opinion, if you want to specify additional rules when "creating" the structure, it should be after the decoding operation.
For instance:
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
var result MyStruct
err := mapstructure.Decode(m, &result)
if err != nil {
return result, err
}
if result.Age <= 0 {
result.Age = 0
}
if result.Name == "" {
return result, errors.New("empty name is not allowed")
}
return result, err
}
This way, you will clearly separate the "conversion" part from your structure's specific rules processing.

You can just Marshal/Unmarshal, but property names should match
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
data, _ := json.Marshal(m)
var result MyStruct
err := json.Unmarshal(data, &result)
return result, err
}

Related

Go: Implementing a ManyDecode for a "set" of individual results

I have implemented a very simple Decode method (using gob.Decoder for now) - this works well for single responses - it would even work well for slices, but I need to implement a DecodeMany method where it is able to decode a set of individual responses (not a slice).
Working Decode method:
var v MyType
_ = Decode(&v)
...
func Decode(v interface{}) error {
buf, _ := DoSomething() // func DoSomething() ([]byte, error)
// error handling omitted for brevity
return gob.NewDecoder(bytes.NewReader(buf)).Decode(v)
}
What I'm trying to do for a DecodeMany method is to deal with a response that isn't necessarily a slice:
var vv []MyType
_ = DecodeMany(&vv)
...
func DecodeMany(vv []interface{}) error {
for _, g := range DoSomething() { // func DoSomething() []struct{Buf []bytes}
// Use g.Buf as an individual "interface{}"
// want something like:
var v interface{} /* Somehow create instance of single vv type? */
_ = gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(v)
vv = append(vv, v)
}
return
}
Besides not compiling the above also has the error of:
cannot use &vv (value of type *[]MyType) as type []interface{} in argument to DecodeMany
If you want to modify the passed slice, it must be a pointer, else you must return a new slice. Also if the function is declared to have a param of type []interface{}, you can only pass a value of type []interface{} and no other slice types... Unless you use generics...
This is a perfect example to start using generics introduced in Go 1.18.
Change DecodeMany() to be generic, having a T type parameter being the slice element type:
When taking a pointer
func DecodeMany[T any](vv *[]T) error {
for _, g := range DoSomething() {
var v T
if err := gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(&v); err != nil {
return err
}
*vv = append(*vv, v)
}
return nil
}
Here's a simple app to test it:
type MyType struct {
S int64
}
func main() {
var vv []MyType
if err := DecodeMany(&vv); err != nil {
panic(err)
}
fmt.Println(vv)
}
func DoSomething() (result []struct{ Buf []byte }) {
for i := 3; i < 6; i++ {
buf := &bytes.Buffer{}
v := MyType{S: int64(i)}
if err := gob.NewEncoder(buf).Encode(v); err != nil {
panic(err)
}
result = append(result, struct{ Buf []byte }{buf.Bytes()})
}
return
}
This outputs (try it on the Go Playground):
[{3} {4} {5}]
When returning a slice
If you choose to return the slice, you don't have to pass anything, but you need to assign the result:
func DecodeMany[T any]() ([]T, error) {
var result []T
for _, g := range DoSomething() {
var v T
if err := gob.NewDecoder(bytes.NewReader(g.Buf)).Decode(&v); err != nil {
return result, err
}
result = append(result, v)
}
return result, nil
}
Using it:
vv, err := DecodeMany[MyType]()
if err != nil {
panic(err)
}
fmt.Println(vv)
Try this one on the Go Playground.

Go - How to deal with JSON response that has attribute that can be of different types

Let's say I have the following struct
type Response struct {
ID string `json:"id"`
Edited int `json:"edited"`
}
type Responses []Response
Then lets say I send a request to an API, but my issue is the API docs and from testing have told me that the edited value can come back as a bool or as a int. This obviously upsets Go and it throws an error when decoding the response body into the struct.
// ... http GET
// response [{"edited": true, id: 1}, {"edited": 1683248234.0, id: 2}]
r := Responses{}
err := json.NewDecoder(resp.Body).Decode(&r)
if err != nil {
return nil, err
}
defer resp.Body.Close()
How can I handle the above situation where I can't automatically load it into a struct? I'm assuming I'd need to do it into an interface first then filter the slice and handle the two different Response types in their own structs? But then I can't combine them!
So I'm thinking of conforming the field to one or the other, bool or int.
n.b. this relates to the Reddit API where some of the fields such as edited, created, created_utc don't have conforming types.
As told #mkopriva, the simplest way to handle different type of variable is use interface{} type:
const resp = `[{"edited": true, "id": 1}, {"edited": 1683248234.0, "id": 2}, {"id": 3}]`
type Response struct {
ID int `json:"id"`
Edited interface{} `json:"edited"`
}
type Responses []Response
r := Responses{}
err := json.Unmarshal([]byte(resp), &r)
if err != nil {
log.Fatal(err)
}
for _, response := range r {
switch response.Edited.(type) {
case float64:
fmt.Println("float64")
case bool:
fmt.Println("bool")
default:
fmt.Println("invalid")
}
}
PLAYGROUND
Also you can define new type with custom json.Unmarshaler implementation:
type Response struct {
ID int `json:"id"`
Edited Edited `json:"edited"`
}
type Edited struct {
BoolVal *bool
FloatVal *float64
}
func (e *Edited) UnmarshalJSON(data []byte) error {
boolVal, err := strconv.ParseBool(string(data))
if err == nil {
e.BoolVal = &boolVal
return nil
}
floatVal, err := strconv.ParseFloat(string(data), 64)
if err == nil {
e.FloatVal = &floatVal
return nil
}
return errors.New("undefined type")
}
r := Responses{}
err := json.Unmarshal([]byte(resp), &r)
if err != nil {
log.Fatal(err)
}
for _, response := range r {
edited := response.Edited
switch {
case edited.FloatVal != nil:
fmt.Println("float64")
case edited.BoolVal != nil:
fmt.Println("bool")
default:
fmt.Println("invalid")
}
}
PLAYGROUND

Transform struct to slice struct

I'm trying to select a struct by string input and then depending on the return JSON Object or Array, unmarshall the JSON. Is it correct to think of a way to reflect the struct to slice struct? if so how to do that with reflection?
Regards,
Peter
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
)
type NameStruct struct {
Name string
}
func main() {
jsonData := []byte(`[{"name":"james"},{"name":"steven"}]`)
returnModel := InitializeModel("NameStruct", jsonData)
fmt.Println(returnModel)
jsonData = []byte(`{"name":"james"}`)
returnModel = InitializeModel("NameStruct", jsonData)
fmt.Println(returnModel)
}
func getModelByName(modelType string) interface{} {
modelMap := make(map[string]interface{})
modelMap["NameStruct"] = new(NameStruct)
//don't want to do this
modelMap["arrNameStruct"] = new([]NameStruct)
return modelMap[modelType]
}
func InitializeModel(modelName string, jsonData []byte) interface{} {
switch IsArray(jsonData) {
case true:
// some conversion here, how?
returnModel := getModelByName("NameStruct")
if err := json.Unmarshal(jsonData, &returnModel); err != nil {
log.Println(err)
}
return returnModel
case false:
returnModel := getModelByName("NameStruct")
if err := json.Unmarshal(jsonData, &returnModel); err != nil {
log.Println(err)
}
return returnModel
}
return nil
}
func IsArray(jsonData []byte) bool {
return (bytes.HasPrefix(jsonData, []byte("["))) && (bytes.HasSuffix(jsonData, []byte("]")))
}
Expanding on my comment, you can create a Factory where pre-defined types are registered:
type Factory struct {
m map[string]reflect.Type
}
func (f *Factory) Register(v interface{}) {
vt := reflect.TypeOf(v)
n := vt.Name()
f.m[n] = vt
f.m["[]"+n] = reflect.SliceOf(vt) // implicitly register a slice of type too
}
these types can be looked up by name at runtime and initialized with JSON data:
func (f *Factory) Make(k string, bs []byte) (interface{}, error) {
vt, ok := f.m[k]
if !ok {
return nil, fmt.Errorf("type %q not registered", k)
}
pv := reflect.New(vt).Interface()
err := json.Unmarshal(bs, pv)
if err != nil {
return nil, err
}
return pv, nil
}
To use:
type Place struct {
City string `json:"city"`
}
factory.Register(Place{})
p, err := factory.Make("Place", []byte(`{"city":"NYC"}`))
fmt.Printf("%#v\n", p) // &main.Place{City:"NYC"}
Slices also work:
ps, err := factory.Make("[]Place", []byte(`[{"city":"NYC"},{"city":"Dublin"}]`))
fmt.Printf("%#v\n", p, p) // &[]main.Place{main.Place{City:"NYC"}, main.Place{City:"Dublin"}}
Playground: https://play.golang.org/p/qWEdwk-YUug

Go functions dot notation

I want to build this:
finalResult, err := Function1(whatever type)
.Function2(whatever type)
.Function3(whatever type)
Something similar to promises in javascript but not necessarily the same concepts. Or similar to nested methods in Java. I just pretend avoid more code for the same tasks.
I don't want to do this :
result, err := new(Function1(whatever type)) //or *Function1(whatever type)
if err != nil {...}
result1, err := result.Function2(whatever type)
if err != nil {...}
finalResult, err := result1.Function3(whatever type)
if err != nil {...}
I've been trying with several options with structs and interfaces but I can't get any result.
My apology if I have a mistake in my grammar. I'm not still so good with my skill English.
Thanks,
David
I think you mean the Fluent API design pattern. You return the same Object, or struct in Go, over and over.
This pattern does not allow you to return a tuple, or multiple return types though. You can only return one object.
https://play.golang.org/p/9PceZwi1a3
package main
import (
"fmt"
)
type Object struct {
Value string
Error error
}
func (o *Object) Before(s string) *Object {
o.Value = s + o.Value
// add an error if you like
// o.Error = error.New(...)
return o
}
func (o *Object) After(s string) *Object {
// could check for errors too
if o.Error != nil {
o.Value = o.Value + s
}
return o
}
func main() {
x := Object{}
x.
Before("123").
After("456").
After("789").
Before("0")
if x.Error != nil {
// handle error
}
fmt.Println(x.Value)
}
Edit: sberry's answer had a good idea. Add an Error state on the Object struct itself which could allow u to check for errors in each func call.
Since you are new, please remember to evaluate all answers and mark the best one you believe lead you the answer.
The only way to do this would be to have each FunctionX be a method on a receiver (most likely a pointer receiver if you are mutating state) and have them each return the receiver. Like so
type someNumber struct {
n int32
}
func (s *someNumber) Add(i int32) *someNumber {
s.n += i
return s
}
func (s *someNumber) Multiply(i int32) *someNumber {
s.n *= i
return s
}
func (s *someNumber) Subtract(i int32) *someNumber {
s.n -= i
return s
}
func main() {
num := &someNumber{n: 0}
result := num.Add(5).Multiply(4).Subtract(10)
fmt.Println(result.n)
}
You could set and error on the someNumber as well, and check it at the end.
If you are looking to actually carry through the error, then you would need to do something like this.
func Function1(a int, err error) (int, error) {
if err != nil {
return a, err
}
return a, nil
}
func Function2(a int, err error) (int, error) {
if err != nil {
return a, err
}
return a*2, nil
}
func Function3(a int, err error) (int, error) {
if err != nil {
return a, err
}
return a*3, nil
}
func main() {
a, err := Function3(Function2(Function1(1, nil)))
fmt.Println(a, err)
}
But, it seems like you are looking for the type of behavior an Error Monad would bring and that does not exist in Go.

Using sqlx.Rows.StructScan for interface args

I need to use StructScan function for interface (pointer to struct).
But if I try to reflect value, I've got error, because reflect.New() returning reflect.Value type. How I can scan structure and store data into dest var?
// package 1
type Data struct {
id int `db:"id"`
caption string `db:"caption"`
}
func Func1 {
data := []Data{}
GetData(&data)
log.Println(data)
}
// package 2
func GetData(sql string, dest interface{}) {
rows, err := DBI.Queryx(sql)
if err == nil {
// reflect.Value
myData := reflect.New(reflect.TypeOf(dest).Elem().Elem())
for rows.Next() {
rows.StructScan(&myData) // Fail here
}
}
}
Solved
// package 2
func GetData(sql string, dest interface{}) {
arr := reflect.ValueOf(dest).Elem()
v := reflect.New(reflect.TypeOf(dest).Elem().Elem())
rows, err := DBI.Queryx(sql)
if err == nil {
if err = rows.StructScan(v.Interface()); err == nil {
arr.Set(reflect.Append(arr, v.Elem()))
}
}
}

Resources