serializing golang structs in a unique unstructured datastore - go

I have a dilemma to store some data in an unstructured field.
Let's say I have these to store:
package main
type FoodType string
const (
BeefFood FoodType = "beef"
ChickenFood FoodType = "chicken"
)
type FoodI interface {
Prepare()
GetType() FoodType
}
type FoodContainer struct {
FoodI
}
type Chicken struct {
Way string
}
func (c *Chicken) Prepare() {}
func (c *Chicken) GetType() FoodType {
return ChickenFood
}
type Beef struct {
Way string
}
func (b *Beef) Prepare() {}
func (b *Beef) GetType() FoodType {
return BeefFood
}
Let's say I choose json as a serializer (I will probably go for msgpack, but that's another story).
Which way do you think is the most appropriate:
Solution 1
one container that contains all the message types, and simply use the default marshal/unmarshal:
package main
type FoodContainer struct {
Chicken *Chicken `json:"chicken"`
Beef *Beef `json:"beef"`
}
And then, I just need to check which of the field is nil to know what to do with it.
Solution 2
A more "typed" solution, with interfaces and FoodType
package main
import (
"encoding/json"
"fmt"
)
type FoodContainer struct {
FoodType
FoodI
}
func (fc *FoodContainer) MarshalJSON() ([]byte, error) {
return json.Marshal(&FoodContainerT{
FoodType: fc.FoodI.GetType(),
FoodI: fc.FoodI,
})
}
func (fc *FoodContainer) UnmarshalJSON(b []byte) error {
var fct struct {
FoodType
}
if err := json.Unmarshal(b, &fct); err != nil {
return fmt.Errorf("fc UnmarshalJSON: %v", err)
}
switch fct.FoodType {
case ChickenFood:
fmt.Printf("Yeah, Chickenfood: %v\n", fct.FoodType)
var fi struct {
FoodI Chicken
}
if err := json.Unmarshal(b, &fi); err != nil {
return fmt.Errorf("chicken UnmarshalJSON: %v", err)
}
fmt.Printf("%#v", fi.FoodI)
default:
return fmt.Errorf("no known FoodType found")
}
return nil
}
I benchmarked both, they have around the same processing time and same mem usage, but I'm wondering which of them could be more idiomatic?

Related

Unmarshal YAML into complex object which may be either struct or string

Trying to unmarshal YAML into complex object such as map[string]map[interface{}]string.
The problem is that I want to be able to differentiate an interface{} part between string and Source which is a struct.
type Source struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
LogoID string `yaml:"logoId"`
URL string `yaml:"url"`
}
type UNFT struct {
ItemMeta map[string]map[interface{}]string `yaml:"item_meta"`
// could be
// ItemMeta map[string]map[string]string `yaml:"item_meta"`
// or
// ItemMeta map[string]map[Source]string `yaml:"item_meta"`
}
Obviously YAML does not know how to unmarshal into Source struct so I have to implement Unmarshaler interface:
type Unmarshaler interface {
UnmarshalYAML(value *Node) error
}
But I don't quite understand the big picture of unmarshaling process. In general I assume that I have to manually traverse *yaml.Node and call func UnmarshalYAML(value *Node) error on every node.
package main
import (
"fmt"
"gopkg.in/yaml.v3"
)
type Source struct {
ID string `json:"id"`
Name string `json:"name"`
LogoID string `json:"logoId"`
URL string `json:"url"`
}
var data = `
unf:
item_meta:
source:
!struct
? id: "data-watch"
name: "DataWatch"
logoid: "data-watch"
url: "https"
: "product_any('SS')"
public_usage:
"": "source_any('SDF')"
"provider": "source_any('ANO')"`
type UNFT struct {
ItemMeta map[string]map[interface{}]string `yaml:"item_meta"`
}
type MetaConverterConfigT struct {
UNFT UNFT `yaml:"unf"`
}
func main() {
cfg := MetaConverterConfigT{}
err := yaml.Unmarshal([]byte(data), &cfg)
if err != nil {
fmt.Println("%w", err)
}
fmt.Println(cfg)
}
func (s *UNFT) UnmarshalYAML(n *yaml.Node) error {
var cfg map[string]map[interface{}]string
if err := n.Decode(&cfg); err != nil {
fmt.Println("%w", err)
}
return nil
}
Go playground
type MetaKey struct {
String string
Source Source
}
func (k *MetaKey) UnmarshalYAML(n *yaml.Node) error {
if n.Tag == "!!str" {
return n.Decode(&k.String)
}
if n.Tag == "!!map" {
return n.Decode(&k.Source)
}
return fmt.Errorf("unsupported MetaKey type")
}
// ...
type UNFT struct {
ItemMeta map[string]map[MetaKey]string `yaml:"item_meta"`
}
https://go.dev/play/p/Nhtab4l-ANT
If you need the map type to remain as is, i.e. without adding the custom key type, then you can implement the unmarshaler on UNFT as well and just do a re-mapping with any:
type UNFT struct {
ItemMeta map[string]map[any]string `yaml:"item_meta"`
}
func (u *UNFT) UnmarshalYAML(n *yaml.Node) error {
var obj struct {
ItemMeta map[string]map[MetaKey]string `yaml:"item_meta"`
}
if err := n.Decode(&obj); err != nil {
return err
}
u.ItemMeta = make(map[string]map[any]string, len(obj.ItemMeta))
for k, v := range obj.ItemMeta {
m := make(map[any]string, len(v))
for k, v := range v {
if k.Source != (Source{}) {
m[k.Source] = v
} else {
m[k.String] = v
}
}
u.ItemMeta[k] = m
}
return nil
}
https://go.dev/play/p/uwboGKf3qnD

How does Golang accept uncertain values

Back-end return values are not fixed, sometimes:
{"application": {"instance": [{"instanceId": "v1"}, {"instanceId": "v2"}]}}
or sometimes:
{"application": {"instance": {"instanceId": "v"}}}
how should I take out the corresponding instanceId value?
package main
import (
"encoding/json"
"fmt"
)
type Application struct {
Application struct {
Instance json.RawMessage `json:"instance"`
} `json:"application"`
}
func main() {
a := `{"application": {"instance": {"instanceId": "v"}}}`
//a := `{"application": {"instance": [{"instanceId": "v1"}, {"instanceId": "v2"}]}} `
var p Application
errJson := json.Unmarshal([]byte(a), &p)
if errJson != nil {
fmt.Printf("errJson")
}
fmt.Printf("type:%T", p.Application.Instance)
}
Since the 2 value types clash (one a struct another a slice of structs) it gets messy to encapsulate this into a single type even using catch-all solutions like interface{}.
The simplest solution is present two distinct types and marshal into either to see which "works":
func unmarsh(body []byte) (*type1, *type2, error) {
var (
t1 type1
t2 type2
)
err := json.Unmarshal(body, &t1)
if err == nil {
return &t1, nil, nil
}
err = json.Unmarshal(body, &t2)
if err == nil {
return nil, &t2, nil
}
return nil, nil, err
}
and in your example the two types would be:
type type1 struct {
Application struct {
Instance []struct {
InstanceID string `json:"instanceId"`
} `json:"instance"`
} `json:"application"`
}
type type2 struct {
Application struct {
Instance struct {
InstanceID string `json:"instanceId"`
} `json:"instance"`
} `json:"application"`
}
Working example:
https://play.golang.org/p/Kma32gWfghb
A cleaner solution would be a custom unmarshaler:
type Instances []Instance
func (i *Instances) UnmarshalJSON(in []byte) error {
if len(in)>0 && in[0]=='[' {
var a []Instance
if err:=json.Unmarshal(in,&a); err!=nil {
return err
}
*i=a
return nil
}
var s Instance
if err:=json.Unmarshal(in,&s) ; err!=nil {
return err
}
*i=[]Instance{s}
return nil
}
This would unmarshal an object into a slice of 1.
A more compact solution is provided by #mkopriva:
func (i *Instances) UnmarshalJSON(in []byte) error {
if len(in) > 0 && in[0] == '[' {
return json.Unmarshal(in, (*[]Instance)(i))
}
*i = Instances{{}}
return json.Unmarshal(in, &(*i)[0])
}

Can a variable be used as a placeholder for a function call?

I am writing a program which has several structs and functions to handle these structs differently. I am having a generic function which calls the required function based on the inputs. Is there a generic way to use the returned value from getStruct()?
package main
var X func(s []string) A
var Y func(s []string) B
type A struct {
Name string
Place string
}
type B struct {
Name string
Place string
Value string
}
func newA(s []string) A {
a := A{
Name: s[0],
Place: s[1],
}
return a
}
func newB(s []string) B {
a := B{
Name: s[0],
Place: s[1],
Value: s[2],
}
return a
}
func getStruct(t string) interface{} {
switch {
case t == "A":
return X
case t == "B":
return Y
default:
return //someStruct
}
}
func main() {
buildNewStruct := getStruct("A") //Lets assume "A" here is got as an argument
var strSlice = []string{"Bob", "US"}
buildNewStruct(strSlice) //How to do this operation?
//I am hoping to use buildNewStruct(strSlice) to dynamically call
//either of newA(strSlice) or newB(strSlice) function
}
I have tried looking at this and this the later is not exactly the same as my question.
Since I am new to go, I am not sure if something like this is possible.
you can use the reflect package to set the struct properties to the equivalent index positioned value from an []interface{} slice.
package main
import (
"fmt"
"log"
"reflect"
)
func main() {
var a A
err := decode(&a, []interface{}{"Name", "Place"})
log.Println(err)
log.Println(a)
}
func decode(dst interface{}, values []interface{}) error {
rvptr := reflect.ValueOf(dst)
if rvptr.Kind() != reflect.Ptr {
return fmt.Errorf("value must be ptr")
}
rv := rvptr.Elem()
if rv.NumField() < len(values) {
return fmt.Errorf("too many values")
}
if rv.NumField() > len(values) {
return fmt.Errorf("too few values")
}
rvalues := reflect.ValueOf(values)
for i := range values {
f := rv.FieldByIndex([]int{i})
f.Set(rvalues.Index(i).Elem())
}
return nil
}
type A struct {
Name string
Place string
}
type B struct {
Name string
Place string
Value string
}
prints
$ go run main.go
2019/11/21 17:00:17 <nil>
2019/11/21 17:00:17 {Name Place}
The problem is the return type for the function.
func newA(in []string) interface{} {...}
func newB(in []string) interface{} {...}
func getStruct(name string) func([]string) interface{} {
switch name {
case "A": return newA
case "B": return newB
}
return nil
}
func main() {
buildNewStruct := getStruct("A")
var strSlice = []string{"Bob", "US"}
str:=buildNewStruct(strSlice)
if a, ok:=str.(A); ok {
...
}
}
With this approach, even though you saved some code by calling a unified buildNewStruct(), you have to use type assertions to figure out what is returned from that function, so this may not make a lot of sense. It depends on your exact use case though.

how can i mock specific embedded method inside interface

I have this code and I wanna write a unit tests for update function.
how can i mock FindByUsername function ?
I try to overwrite u.FindByUsername but it's doesn't work.
also, I can write some function to give u *UserLogic and userName string as input parameters and execute u.FindByUsername() and mock this function but it's not a clean solution I need a better solution for mocking methods inside UserOperation interface.
package logic
import (
"errors"
"fmt"
)
var (
dataStore = map[string]*User{
"optic": &User{
Username: "bla",
Password: "ola",
},
}
)
//UserOperation interface
type UserOperation interface {
Update(info *User) error
FindByUsername(userName string) (*User, error)
}
//User struct
type User struct {
Username string
Password string
}
//UserLogic struct
type UserLogic struct {
UserOperation
}
//NewUser struct
func NewUser() UserOperation {
return &UserLogic{}
}
//Update method
func (u *UserLogic) Update(info *User) error {
userInfo, err := u.FindByUsername(info.Username)
if err != nil {
return err
}
fmt.Println(userInfo.Username, userInfo.Password)
fmt.Println("do some update logic !!!")
return nil
}
//FindByUsername method
func (u *UserLogic) FindByUsername(userName string) (*User, error) {
userInfo := &User{}
var exist bool
if userInfo, exist = dataStore[userName]; !exist {
return nil, errors.New("user not found")
}
return userInfo, nil
}
Update
I try to mock function with this code
func TestUpdate2(t *testing.T) {
var MockFunc = func(userName string) (*User, error) {
return &User{Username:"foo", Password:"bar"},nil
}
user := NewUser()
user.FindByUsername = MockFunc
user.Update(&User{Username:"optic", Password:"ola"})
}
You're mixing two levels of abstraction in your UserOperation interface: Update depends on FindByUsername. To make Update testable you need to inject the UserFinder functionality into your Update method. You can do this e.g. by defining a field in the UserLogic struct:
type UserOperation interface {
Update(info *User) error
}
type UserFinder func(userName string) (*User, error)
type UserLogic struct {
UserOperation
FindByUsername UserFinder
}
//NewUser struct
func NewUser() *UserLogic { // return structs, accept interfaces!
return &UserLogic{
findByUsername: FindByUsername
}
}
func (u *UserLogic) Update(info *User) error {
userInfo, err := u.findByUsername(info.Username)
if err != nil {
return err
}
fmt.Println(userInfo.Username, userInfo.Password)
fmt.Println("do some update logic !!!")
return nil
}
func FindByUsername(userName string) (*User, error) {
userInfo := &User{}
var exist bool
if userInfo, exist = dataStore[userName]; !exist {
return nil, errors.New("user not found")
}
return userInfo, nil
}

Can golang plugins be used for factory functions?

I have the following code in a golang plugin module:
plug.go
package main
import "fmt"
var (
Thing = New("first thing")
ThingFactory = thingFactory{}
)
type thing struct {
i int
s string
}
func New(s string) thing {
return thing{s: s}
}
func (t *thing) Say() string {
t.i++
return fmt.Sprintf("%s - %d", t.s, t.i)
}
type thingFactory struct{}
func (t thingFactory) Make(s string) thing {
return New(s)
}
it is compiled as a .so object and used in another program:
main.go
package main
import (
"fmt"
"plugin"
)
func main() {
p, err := plugin.Open("../plug/plug.so")
if err != nil {
panic(err)
}
symbol, err := p.Lookup("Thing")
if err != nil {
panic(err)
}
thing := symbol.(Sayer)
fmt.Println(thing.Say())
symbol, err = p.Lookup("ThingFactory") // <-problems start here
if err != nil {
panic(err)
}
factory := symbol.(GetSayer)
madeThing := factory.Make("how about me?")
fmt.Println(madeThing.Say())
fmt.Println(madeThing.Say())
}
type Sayer interface {
Say() string
}
type GetSayer interface {
Make(string) Sayer
}
I'm able to lookup the Thing, and call Say() on it, but the second interface conversion panics:
first thing - 1
panic: interface conversion: *main.thingFactory is not main.GetSayer: missing method Make
even though the runtime recognizes the first symbol as a Sayer it doesn't recognize that thingFactory obviously has a Make() method, which should return something that is also a Sayer.
Am I missing something obvious here?
The first problem is that in your plugin thingFactory (more precicely *thingfactory) does not have a method described in your main app's GetSayer interface:
Make(string) Sayer
You have:
Make(string) thing
So (first) you have to change thingFactory.Make() to this:
type Sayer interface {
Say() string
}
func (t thingFactory) Make(s string) Sayer {
th := New(s)
return &th
}
After this it still won't work. And the reason for this is because the plugin's Sayer type is not identical to your main app's Sayer type. But they must be the same in order to implement your main app's GetSayer interface.
One solution is to "outsource" the Sayer interface to its own package, and use this common, shared package both in the plugin and in the main app.
Let's create a new package, call it subplay:
package subplay
type Sayer interface {
Say() string
}
Import this package and use it in the plugin:
package main
import (
"fmt"
"path/to/subplay"
)
var (
Thing = New("first thing")
ThingFactory = thingFactory{}
)
type thing struct {
i int
s string
}
func New(s string) thing {
return thing{s: s}
}
func (t *thing) Say() string {
t.i++
return fmt.Sprintf("%s - %d", t.s, t.i)
}
type thingFactory struct{}
func (t thingFactory) Make(s string) subplay.Sayer {
th := New(s)
return &th
}
And also import and use it in the main app:
package main
import (
"fmt"
"path/to/subplay"
"plugin"
)
func main() {
p, err := plugin.Open("../plug/plug.so")
if err != nil {
panic(err)
}
symbol, err := p.Lookup("Thing")
if err != nil {
panic(err)
}
thing := symbol.(subplay.Sayer)
fmt.Println(thing.Say())
symbol, err = p.Lookup("ThingFactory")
if err != nil {
panic(err)
}
factory := symbol.(GetSayer)
madeThing := factory.Make("how about me?")
fmt.Println(madeThing.Say())
fmt.Println(madeThing.Say())
}
type GetSayer interface {
Make(string) subplay.Sayer
}
Now it will work, and output will be:
first thing - 1
how about me? - 1
how about me? - 2
See related questions:
go 1.8 plugin use custom interface
How do Go plugin dependencies work?
Your plugin Make method should return a Sayer object not thing
type Sayer interface {
Say() string
}
func (t *thingFactory) Make(s string) Sayer {
return New(s)
}

Resources