Back-end return values are not fixed, sometimes:
{"application": {"instance": [{"instanceId": "v1"}, {"instanceId": "v2"}]}}
or sometimes:
{"application": {"instance": {"instanceId": "v"}}}
how should I take out the corresponding instanceId value?
package main
import (
"encoding/json"
"fmt"
)
type Application struct {
Application struct {
Instance json.RawMessage `json:"instance"`
} `json:"application"`
}
func main() {
a := `{"application": {"instance": {"instanceId": "v"}}}`
//a := `{"application": {"instance": [{"instanceId": "v1"}, {"instanceId": "v2"}]}} `
var p Application
errJson := json.Unmarshal([]byte(a), &p)
if errJson != nil {
fmt.Printf("errJson")
}
fmt.Printf("type:%T", p.Application.Instance)
}
Since the 2 value types clash (one a struct another a slice of structs) it gets messy to encapsulate this into a single type even using catch-all solutions like interface{}.
The simplest solution is present two distinct types and marshal into either to see which "works":
func unmarsh(body []byte) (*type1, *type2, error) {
var (
t1 type1
t2 type2
)
err := json.Unmarshal(body, &t1)
if err == nil {
return &t1, nil, nil
}
err = json.Unmarshal(body, &t2)
if err == nil {
return nil, &t2, nil
}
return nil, nil, err
}
and in your example the two types would be:
type type1 struct {
Application struct {
Instance []struct {
InstanceID string `json:"instanceId"`
} `json:"instance"`
} `json:"application"`
}
type type2 struct {
Application struct {
Instance struct {
InstanceID string `json:"instanceId"`
} `json:"instance"`
} `json:"application"`
}
Working example:
https://play.golang.org/p/Kma32gWfghb
A cleaner solution would be a custom unmarshaler:
type Instances []Instance
func (i *Instances) UnmarshalJSON(in []byte) error {
if len(in)>0 && in[0]=='[' {
var a []Instance
if err:=json.Unmarshal(in,&a); err!=nil {
return err
}
*i=a
return nil
}
var s Instance
if err:=json.Unmarshal(in,&s) ; err!=nil {
return err
}
*i=[]Instance{s}
return nil
}
This would unmarshal an object into a slice of 1.
A more compact solution is provided by #mkopriva:
func (i *Instances) UnmarshalJSON(in []byte) error {
if len(in) > 0 && in[0] == '[' {
return json.Unmarshal(in, (*[]Instance)(i))
}
*i = Instances{{}}
return json.Unmarshal(in, &(*i)[0])
}
Related
Trying to unmarshal YAML into complex object such as map[string]map[interface{}]string.
The problem is that I want to be able to differentiate an interface{} part between string and Source which is a struct.
type Source struct {
ID string `yaml:"id"`
Name string `yaml:"name"`
LogoID string `yaml:"logoId"`
URL string `yaml:"url"`
}
type UNFT struct {
ItemMeta map[string]map[interface{}]string `yaml:"item_meta"`
// could be
// ItemMeta map[string]map[string]string `yaml:"item_meta"`
// or
// ItemMeta map[string]map[Source]string `yaml:"item_meta"`
}
Obviously YAML does not know how to unmarshal into Source struct so I have to implement Unmarshaler interface:
type Unmarshaler interface {
UnmarshalYAML(value *Node) error
}
But I don't quite understand the big picture of unmarshaling process. In general I assume that I have to manually traverse *yaml.Node and call func UnmarshalYAML(value *Node) error on every node.
package main
import (
"fmt"
"gopkg.in/yaml.v3"
)
type Source struct {
ID string `json:"id"`
Name string `json:"name"`
LogoID string `json:"logoId"`
URL string `json:"url"`
}
var data = `
unf:
item_meta:
source:
!struct
? id: "data-watch"
name: "DataWatch"
logoid: "data-watch"
url: "https"
: "product_any('SS')"
public_usage:
"": "source_any('SDF')"
"provider": "source_any('ANO')"`
type UNFT struct {
ItemMeta map[string]map[interface{}]string `yaml:"item_meta"`
}
type MetaConverterConfigT struct {
UNFT UNFT `yaml:"unf"`
}
func main() {
cfg := MetaConverterConfigT{}
err := yaml.Unmarshal([]byte(data), &cfg)
if err != nil {
fmt.Println("%w", err)
}
fmt.Println(cfg)
}
func (s *UNFT) UnmarshalYAML(n *yaml.Node) error {
var cfg map[string]map[interface{}]string
if err := n.Decode(&cfg); err != nil {
fmt.Println("%w", err)
}
return nil
}
Go playground
type MetaKey struct {
String string
Source Source
}
func (k *MetaKey) UnmarshalYAML(n *yaml.Node) error {
if n.Tag == "!!str" {
return n.Decode(&k.String)
}
if n.Tag == "!!map" {
return n.Decode(&k.Source)
}
return fmt.Errorf("unsupported MetaKey type")
}
// ...
type UNFT struct {
ItemMeta map[string]map[MetaKey]string `yaml:"item_meta"`
}
https://go.dev/play/p/Nhtab4l-ANT
If you need the map type to remain as is, i.e. without adding the custom key type, then you can implement the unmarshaler on UNFT as well and just do a re-mapping with any:
type UNFT struct {
ItemMeta map[string]map[any]string `yaml:"item_meta"`
}
func (u *UNFT) UnmarshalYAML(n *yaml.Node) error {
var obj struct {
ItemMeta map[string]map[MetaKey]string `yaml:"item_meta"`
}
if err := n.Decode(&obj); err != nil {
return err
}
u.ItemMeta = make(map[string]map[any]string, len(obj.ItemMeta))
for k, v := range obj.ItemMeta {
m := make(map[any]string, len(v))
for k, v := range v {
if k.Source != (Source{}) {
m[k.Source] = v
} else {
m[k.String] = v
}
}
u.ItemMeta[k] = m
}
return nil
}
https://go.dev/play/p/uwboGKf3qnD
I'm trying to seed my Postgres database as functionally. In my case, SeedSchema() function can take any type struct. So I define a interface and create functions to my structs which will seed. I tried with generics and without.
When I unmarshall any json array from file as byte array, json.Unmarshall method change my tempMember member of struct. Exp, models.Term to map[string]interface{}. I've used unmarshall before this function and I've not seen like this situation.
Here is my SeedSchema() function:
func (db *Database) SeedSchema(models ...globals.Seeder[any]) error {
var (
subjects []globals.Seeder[any]
fileByte []byte
err error
// tempMember any
)
if len(models) == 0 {
subjects = seederModelList
} else {
subjects = models
}
for _, model := range subjects {
fileName, tempMember := model.Seed()
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //1
if fileByte, err = os.ReadFile("db/seeds/" + fileName); err != nil {
fmt.Println(err)
return err
}
if err = json.Unmarshal(fileByte, &tempMember); err != nil {
fmt.Println(err)
return err
}
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //2
}
return nil
}
First print returns []models.AirportCodes and the second []interface {}.
Here is my interface and model:
func (AirportCodes) Seed() (string, any) {
return "airport_codes.json", []AirportCodes{}
}
type Seeder[T any] interface {
Seed() (string, T)
// Seed(*gorm.DB) error
TableName() string
}
seederModelList = []globals.Seeder[any]{
m.AirportCodes{},
m.Term{},
}
After a few weeks, I have looking for solve this problem and look unmarshaler interfaces and examples. Then Like what icza said, I started to look over the my code that convention between types and I solved like this. If you guys have better answer than mine, please add answer.
Data:
[
{
"id":1,
"name":"Term 1",
"content": [
"a1",
"a2",
"a3"
]
}
]
Result:
[{ID:1 Name:Term 1 Content:[a1 a2 a3]}]
UnmarshalJSON Function:
func (term *Term) UnmarshalJSON(data []byte) error {
tempMap := map[string]interface{}{}
if err := json.Unmarshal(data, &tempMap); err != nil {
return err
}
*term = Term{
Name: tempMap["name"].(string),
}
if tempMap["content"] != nil {
for _, v := range tempMap["content"].([]interface{}) {
(*term).Content = append((term).Content, v.(string))
}
}
return nil
}
Thank you for comments.
I have JSON like this that I need to parse into a golang type:
{
name: "something"
rules: [
{
"itemTypeBasedConditions": [["containsAny", ["first_match", "second_match"]]],
"validity": "INVALID"
}]
}
The problem is that each array of the array in itemTypeBasedConditions contains a mix of strings (always first element) and another array (second element), and I am not sure how to parse all of that into an object that I could then manipulate.
I got to:
type RulesFile struct {
Name string
Rules []RulesItem
}
type RulesItem struct {
itemTypeBasedConditions [][]interface{}
validity bool
}
And then I guess I have to convert elements one by one from interface{} to either string (containsAny) or an array of strings ("first_match", "second_match")
Is there a better way of approaching this JSON parsing?
I would do something like this, you can probably alter this to your needs.
package main
import (
"encoding/json"
"fmt"
"os"
"reflect"
)
type RulesFile struct {
Name string `json:"name"`
Rules []RulesItem `json:"rules"`
}
type RulesItem struct {
ItemTypeBasedConditions [][]Condition `json:"itemTypeBasedConditions"`
Validity bool `json:"validity"`
}
type Condition struct {
Value *string
Array *[]string
}
func (c Condition) String() string {
if c.Value != nil {
return *c.Value
}
return fmt.Sprintf("%v", *c.Array)
}
func (c *Condition) UnmarshalJSON(data []byte) error {
var y interface{}
err := json.Unmarshal(data, &y)
if err != nil {
return err
}
switch reflect.TypeOf(y).String() {
case "string":
val := fmt.Sprintf("%v", y)
c.Value = &val
return nil
case "[]interface {}":
temp := y.([]interface{})
a := make([]string, len(temp))
for i, v := range temp {
a[i] = fmt.Sprint(v)
}
c.Array = &a
return nil
}
return fmt.Errorf("cannot unmarshall into string or []string: %v", y)
}
var input string = `
{
"name": "something",
"rules": [
{
"itemTypeBasedConditions": [["containsAny",["first_match", "second_match"]]],
"validity": false
}
]
}`
func main() {
var ruleFile RulesFile
err := json.Unmarshal([]byte(input), &ruleFile)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Printf("%+v\n", ruleFile)
}
You can implement the json.Unmarshaler interface. Have that implementation first unmarshal the json into a slice of json.RawMessage, then, once you've done that, you can unmarshal the individual elements to their corresponding types.
type Cond struct {
Name string
Args []string
}
func (c *Cond) UnmarshalJSON(data []byte) error {
// unmarshal into a slice of raw json
var raw []json.RawMessage
if err := json.Unmarshal(data, &raw); err != nil {
return err
} else if len(raw) != 2 {
return errors.New("unsupported number of elements in condition")
}
// unmarshal the first raw json element into a string
if err := json.Unmarshal(raw[0], &c.Name); err != nil {
return err
}
// unmarshal the second raw json element into a slice of string
return json.Unmarshal(raw[1], &c.Args)
}
https://go.dev/play/p/-tbr73TvX0d
I have a case where I'm I have to support multiple versions. Each one has different data so I create 2 structs. Based on the version I will return 1 of the structs. Once I identify which struct, I then would request the data and Unmarshal into the struct. However, Since that struct satisfies an interface, I dont think the unmarshal is working correctly. I always get the zero value for the sctruct
package main
import (
"encoding/json"
"fmt"
)
// Ten300 ...
type Ten300 struct {
Map string `json:"map"`
Enabled string `json:"enabled"`
}
// Ten400 ...
type Ten400 struct {
Block1 int `json:"block_1"`
Block2 int `json:"block_2"`
}
// NET ...
type NET struct {
CMD Commands
S TenIft
}
// Commands ...
type Commands struct {
Get string
}
// TenIft ...
type TenIft interface {
Get(string) error
}
// Get ...
func (n *Ten300) Get(module string) error {
fmt.Println("Ten300", "Get()")
return nil
}
// Get ...
func (n *Ten400) Get(module string) error {
fmt.Println("Ten400", "Get()")
n.Block2 = 100
return nil
}
// TenGen easy to read fun type
type TenGen func() NET
// VersionSetup is a map of possible version
var VersionSetup = map[string]TenGen{
"3.0.0": func() NET {
return NET{
CMD: Commands{
Get: "getConfig",
},
S: &Ten300{},
}
},
"4.0.0": func() NET {
return NET{
CMD: Commands{
Get: "getSettings",
},
S: &Ten400{},
}
},
}
const input300 = `{
"map": "one",
"enabled": "two"
}`
const input400 = `{
"block_1": 1,
"block_2": 2
}`
// Setup ...
func (n *NET) Setup() error {
// This Switch is just to use hard coded data
var input string
switch n.S.(type) {
case *Ten400:
input = input400
case *Ten300:
input = input300
}
err := json.Unmarshal([]byte(input), n.S)
if err != nil {
fmt.Println("returning err:", err)
return err
}
fmt.Printf("n.s type: %T\nn.s value: %+v\n", n.S, n.S)
n.S.Get("xxx")
return nil
}
func main() {
version := "3.0.0"
if f, ok := VersionSetup[version]; ok {
net := f()
err := net.Setup()
if err != nil {
panic(err)
}
}
version = "4.0.0"
if f, ok := VersionSetup[version]; ok {
net := f()
err := net.Setup()
if err != nil {
panic(err)
}
}
}
Added go-playground ... this seems to work, but is this the best solution?
I am trying to dynamically set a field that is of type interface{}. In all of the cases below the json unmarshals into the correct structure but in the "problem" case the json unmarshals to []interface{}. For that case I am expecting []Person. Why do I get the wrong type for "problem"?
package main
import (
"encoding/json"
"fmt"
"reflect"
)
type Employees struct {
Indicator string `json:"indicator"`
Items interface{} `json:"items"`
}
type Person struct {
Name string `json:"name"`
}
func main() {
simple()
easy()
moreDifficult()
problem()
}
var j = []byte(`{"name": "bob"}`)
var jj = []byte(`[{"name": "bob"}, {"name": "jim"}, {"name": "fred"}]`)
func simple() {
p := Person{}
if err := json.Unmarshal(j, &p); err != nil {
fmt.Println(err)
}
fmt.Println("easy:", p, reflect.TypeOf(p))
}
func easy() {
p := []Person{}
if err := json.Unmarshal(jj, &p); err != nil {
fmt.Println(err)
}
fmt.Println("easy:", p, reflect.TypeOf(p))
}
func moreDifficult() {
var j = []byte(`{"indicator": "more difficult"}`)
e := Employees{}
if err := json.Unmarshal(j, &e); err != nil {
fmt.Println(err)
}
fmt.Println("moreDifficult", e.Items, reflect.TypeOf(e.Items))
}
func problem() {
var j = []byte(`{"indicator:": "problem"}`)
e := Employees{}
if err := json.Unmarshal(j, &e); err != nil {
fmt.Println(err)
}
fmt.Println("problem:", e.Items, reflect.TypeOf(e.Items)) // why not []Person???
}
func (e *Employees) UnmarshalJSON(b []byte) error {
type alias Employees
a := &alias{}
if err := json.Unmarshal(b, &a); err != nil {
return err
}
e.Indicator = a.Indicator
var k = jj
if e.Indicator == "more difficult" {
k = j
e.Items = &Person{}
} else {
e.Items = []Person{}
}
return json.Unmarshal(k, &e.Items)
}
https://play.golang.org/p/xQvjMyLTk5i
The problem is that the interface's underlying value is not a pointer:
e.Items = []Person{}
While it's true that you're passing a pointer to the interface itself:
json.Unmarshal(k, &e.Items)
That does not fix the problem of the underlying value being a non-pointer.
var a interface{} = &T{}
var b interface{} = T{}
b = &b
The types of a and b are different and will be handled differently by the unmarshaler, in b's case the unmarshaler will elect to replace the pointed-to value with a map[string]interface{}.
So to fix your immediate problem you can do something like this:
if e.Indicator == "more difficult" {
k = j
e.Items = &Person{}
} else {
e.Items = &[]Person{}
}
return json.Unmarshal(k, e.Items)
Your current problem has absolutely nothing to do with JSON unmarshaling.
You define Items as type interface{}, so obviously, when you inspect it, it will be of type interface{}. If you want it to be of type []Person, simply define it as such:
type Employees struct {
Indicator string `json:"indicator"`
Items []Person `json:"items"`
}
Once you've done that, your test case will yield the expected results. But your Items will still be empty, as your input JSON doesn't have the items field.