Unmarshaling json into a type - go

I get the following data:
{
"timestamp": "1526058949",
"bids": [
[
"7215.90",
"2.31930000"
],
[
"7215.77",
"1.00000000"
]
]
}
via websocket and I would like to unmarshall it into
type OrderBookItem struct {
Price string
Amount string
}
type OrderBookResult struct {
Timestamp string `json:"timestamp"`
Bids []OrderBookItem `json:"bids"`
Asks []OrderBookItem `json:"asks"`
}
Unmarshal it with:
s := e.Data.(string)
d := &OrderBookResult{}
err := json.Unmarshal([]byte(s), d)
if err == nil {
....
} else {
fmt.Println(err.Error())
}
But I keep getting the error:
json: cannot unmarshal string into Go struct field
OrderBookResult.bids of type feed.OrderBookItem
When I change the struct into
type OrderBookResult struct {
Timestamp string `json:"timestamp"`
Bids [][]string `json:"bids"`
Asks [][]string `json:"asks"`
}
it works. I would like them to be defined as float64 which is what they are. What do I have to change?

As the error says:
json: cannot unmarshal string into Go struct field
OrderBookResult.bids of type feed.OrderBookItem
We cannot convert OrderBookResult.bids which is a slice of string into OrderBookItem which is struct
Implement UnmarshalJSON interface to convert array into objects for price and amount of OrderBookItem struct. Like below
package main
import (
"fmt"
"encoding/json"
)
type OrderBookItem struct {
Price string
Amount string
}
func(item *OrderBookItem) UnmarshalJSON(data []byte)error{
var v []string
if err:= json.Unmarshal(data, &v);err!=nil{
fmt.Println(err)
return err
}
item.Price = v[0]
item.Amount = v[1]
return nil
}
type OrderBookResult struct {
Timestamp string `json:"timestamp"`
Bids []OrderBookItem `json:"bids"`
Asks []OrderBookItem `json:"asks"`
}
func main() {
var result OrderBookResult
jsonString := []byte(`{"timestamp": "1526058949", "bids": [["7215.90", "2.31930000"], ["7215.77", "1.00000000"]]}`)
if err := json.Unmarshal([]byte(jsonString), &result); err != nil{
fmt.Println(err)
}
fmt.Printf("%+v", result)
}
Playground working example
For more information read GoLang spec for Unmarshaler

You are treating your bids as a structure of two separate strings, when they are really a slice of strings in the JSON. If you change OrderBookItem to be
type OrderBookItem []string
which is how you have defined them in the second bit, which works.
To access the values you just have to do:
price := d.Bids[0]
amount := d.Bids[1]

Related

How to convert string to struct array

I have a json array which is converted into a string. Now I want to map the string to a struct array so that I can modify the string json. Below is my code base
type ProcessdetailsEntity struct {
Source []int64 `json:"source"`
Node string `json:"node"`
Targets []int64 `json:"targets"`
Issave bool `json:"isSave"`
Instate []int64 `json:"inState"`
OutState []int64 `json:"outState"`
}
func main() {
var stringJson = "[{\"source\":[-1],\"node\":\"1_1628008588902\",\"targets\":[],\"isSave\":true,\"inState\":[1],\"outState\":[2]},{\"source\":[\"1_1628008588902\",\"5_1628008613446\"],\"node\":\"2_1628008595757\",\"targets\":[],\"isSave\":true,\"inState\":[2,5],\"outState\":[3,6]}]"
in := []byte(stringJson)
detailsEntity := []ProcessdetailsEntity{}
err := json.Unmarshal(in, &detailsEntity)
if err != nil {
log.Print(err)
}
}
Now when I run this code base I got the error:
json: cannot unmarshal string into Go struct field ProcessdetailsEntity.source of type int64
How to properly map the string to struct so that I can modify the inState and outState value of the json ?
The error you get is already pretty much on the nose:
cannot unmarshal string into Go struct field ProcessdetailsEntity.source of type int64
That tells you that (at least one) of your source fields appears to have the wrong type: a string instead of something that can be represented by a int64.
So let's check your source fields in your stringJson:
"source":[-1]
"source":["1_1628008588902","5_1628008613446"]
As you can see the second source is an array of string. Hence the error.
To solve this you need to make sure that the source is an array of int. Unfortunately, 1_1628008588902 and 5_1628008613446 are not valid integers in Go.
I slightly modified your JSON and fixed your code an then it works:
package main
import (
"encoding/json"
"log"
)
type ProcessdetailsEntity struct {
Source []int64 `json:"source"`
Node string `json:"node"`
Targets []int64 `json:"targets"`
Issave bool `json:"isSave"`
Instate []int64 `json:"inState"`
OutState []int64 `json:"outState"`
}
func main() {
var stringJson = `[
{
"source":[-1],
"node":"1_1628008588902",
"targets":[],
"isSave":true,
"inState":[1],
"outState":[2]
},
{
"source":[11628008588902,51628008613446],
"node":"2_1628008595757",
"targets":[],
"isSave":true,
"inState":[2,5],
"outState":[3,6]
}
]`
in := []byte(stringJson)
detailsEntity := []ProcessdetailsEntity{}
err := json.Unmarshal(in, &detailsEntity)
if err != nil {
log.Print(err)
}
}
See: https://play.golang.org/p/kcrkfRliWJ5

Deserialize proto with oneof from BSON fails

Deserializing a BSON into a structure created by protobuf with a oneof property fails:
panic: no decoder found for test.isTest_Entry
This is probably because the oneof field is registered as a plain interface (from the generated code):
type Test struct {
// Types that are valid to be assigned to Entry:
// *Test_S1
// *Test_S2
Entry isTest_Entry `protobuf_oneof:"entry"`
XXX_NoUnkeyedLiteral struct{} `json:"-" bson:"-"`
XXX_unrecognized []byte `json:"-" bson:"-"`
XXX_sizecache int32 `json:"-" bson:"-"`
}
//...
type isTest_Entry interface {
isTest_Entry()
}
package main
import (
"fmt"
"go.mongodb.org/mongo-driver/bson"
"test"
)
func main() {
entry := test.Test{}
data, err := bson.Marshal(&entry)
if nil != err {
panic(err)
}
fmt.Println(data)
newMessage := &test.Test{}
err = bson.Unmarshal(data, newMessage)
if err != nil {
panic(err)
}
fmt.Println(newMessage.GetEntry())
}
Is there any tags I can add to make this work or is this a bug?

why the "switch t := policy.Parameter.(type)" does not work with viper.Unmarshal()

I want to unmarshal several types from JSON and use the interface to represent the actual struct that it is different. But when I send the struct as interface{} it converts it to a map. The animal.json is:
"{"type":"cat","policies":[{"name":"kitty","parameter":{"duration":600,"percent":90}}]}"
package main
import (
"reflect"
log "github.com/sirupsen/logrus"
"github.com/spf13/viper"
)
func main() {
var err error
animal := New()
viper.SetConfigType("json")
viper.SetConfigName("animal")
viper.AddConfigPath("~/Desktop/")
viper.AddConfigPath(".")
if err := viper.ReadInConfig(); err != nil {
return
}
if err = viper.Unmarshal(&animal); err != nil {
return
}
for _, policy := range animal.Policies {
log.Info(policy.Name)
log.Info(policy.Parameter)
//INFO[0000] map[duration:600 percent:90]
log.Info(reflect.TypeOf(policy.Parameter))
//INFO[0000] map[string]interface {}, Why is it not an interface{} and how do I get it?
switch t := policy.Parameter.(type) {
//why does the switch not work?
case *CatParameter:
log.Info("cat", t)
case *DogParameter:
log.Info("dog", t)
}
}
}
func New() *Animal {
var animal Animal
animal.Type = "cat"
return &animal
}
type Animal struct {
Type string `json:"type" form:"type"`
Policies []Policy `json:"policies" form:"policies"`
}
type CatParameter struct {
Duration int `json:"duration"`
Percent int `json:"percent"`
}
type DogParameter struct {
Percent int `json:"percent"`
Duration int `json:"duration"`
Operation string `json:"operation"`
}
type Policy struct {
Name string `json:"name"`
Parameter interface{} `json:"parameter"`
}
It's json unmarshal feature
If you use an interface{} as a decoder, the default json object for interface{} is map[string]interface{}
You can see it here:
https://godoc.org/encoding/json#Unmarshal
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
So in t := policy.Parameter.(type), the t is map[string]interface{}
For solving your problem, you can try to define another field to distinguish CatParameter or DogParameter
Maybe:
type Policy struct {
Name string `json:"name"`
Parameter Parameter `json:"parameter"`
}
type Parameter struct {
Name string `json:"name"` // cat or dog
Percent int `json:"percent,omitempty"`
Duration int `json:"duration,omitempty"`
Operation string `json:"operation,omitempty"`
}

Marshall/Unmarshal JSONPB

I am trying to Unmarshal some json data to a proto message.
JSON
{
"id": 1,
"first_name": "name",
"phone_numbers": []
}
Proto
message Item {
uint32 id=1;
string name=2;
repeated string numbers=3;
}
Proto.go
type Item struct {
Id uint32 `protobuf:"varint,1,opt,name=id" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name" json:"name,omitempty"`
Numbers []string `protobuf:"bytes,4,rep,name=numbers" json:"numbers,omitempty"`
}
How can I map the above JSON to my proto Message (from what I can see there is no way to specify tags in proto atm)?
Your JSON document doesn't match the proto definition; name != first_name and numbers != phone_numbers.
You can define another type that has the same fields as Item but different struct tags and then convert to Item:
var x struct {
Id uint32 `json:"id,omitempty"`
Name string `json:"first_name,omitempty"`
Numbers []string `json:"phone_numbers,omitempty"`
}
if err := json.Unmarshal(jsonDoc, &x); err != nil {
log.Fatal(err)
}
var i = Item(x)
If every JSON document you want to decode has this structure, it may be more convenient to let Item implement json.Unmarshaler:
package main
import (
"encoding/json"
"fmt"
"log"
)
var jsonDoc = []byte(`
{
"id": 1,
"first_name": "name",
"phone_numbers": [
"555"
]
}
`)
type Item struct {
Id uint32 `protobuf:"varint,1,opt,name=id" json:"id,omitempty"`
Name string `protobuf:"bytes,2,opt,name=name" json:"name,omitempty"`
Numbers []string `protobuf:"bytes,4,rep,name=numbers" json:"numbers,omitempty"`
}
// You can define this function is item_json.go or so, then it
// isn't removed if you re-generate your types.
func (i *Item) UnmarshalJSON(b []byte) error {
type item struct {
Id uint32 `json:"id,omitempty"`
Name string `json:"first_name,omitempty"`
Numbers []string `json:"phone_numbers,omitempty"`
}
var x item
if err := json.Unmarshal(jsonDoc, &x); err != nil {
return err
}
*i = Item(x)
return nil
}
func main() {
var i Item
if err := json.Unmarshal(jsonDoc, &i); err != nil {
log.Fatal(err)
}
fmt.Printf("%#v\n", i)
}
Try it on the playground: https://play.golang.org/p/0qibavRJbwi

Unmarshalling array into struct

I'm trying to figure out how I can (using gin) create a struct from an api call
"icon": [
"https://api.figo.me/assets/images/accounts/postbank.png",
{
"48x48": "https://api.figo.me/assets/images/accounts/postbank_48.png",
"60x60": "https://api.figo.me/assets/images/accounts/postbank_60.png",
"72x72": "https://api.figo.me/assets/images/accounts/postbank_72.png",
"84x84": "https://api.figo.me/assets/images/accounts/postbank_84.png",
"96x96": "https://api.figo.me/assets/images/accounts/postbank_96.png",
"120x120": "https://api.figo.me/assets/images/accounts/postbank_120.png",
"144x144": "https://api.figo.me/assets/images/accounts/postbank_144.png",
"192x192": "https://api.figo.me/assets/images/accounts/postbank_192.png",
"256x256": "https://api.figo.me/assets/images/accounts/postbank_256.png"
}
],
into
type CatalogBank struct {
Advice string `json:"advice"`
BankCode string `json:"bank_code"`
BankName string `json:"bank_name"`
BIC string `json:"bic"`
Credentials []struct {
Label string `json:"label"`
Masked bool `json:"masked"`
} `json:"credentials"`
Icon []struct {
} `json:"icon"`
Language []byte `json:"language"`
}
The icon part is just an extract from, but I always get an unmarshall error for this part. How would I have to definde the 'Icon' part in the struct?
This would work
package main
type CatalogBank struct {
Icon []interface{} `json:"icon"`
}
This is a little tricky in Golang because of the non-strict type in the JSON. If that is definitely the format you are going to receive the data in, you should unmarshal to an Interface{} and then parse the interface into a struct that you can use in your Golang
Direct Unmarshalling cannot be done, as the type of each field is not known
type Icon struct{
ImageLink string
ImageLink48 string
// ...
}
type CatalogBank struct {
Advice string `json:"advice"`
IconRaw []interface{} `json:"icon"`
Icon []Icon
//...
}
func UnmarshalIcon(c &CatalogBank, i interface{}):
// first convert it to the top level list
newIcon := Icon{}
listOfIcons := i.([]interface{})
for _, i := range listOfIcons:
switch iT := i.(type) {
case string:
newIcon.ImageLink = iT
case map[string]interface{}:
for smallIconsKey, smallIconLink := range iT {
if smallIconsKey == "48x48"{
newIcon.ImageLink48 = smallIconLink.(string)
}
// and so on
}
var c CatalogBank{}
_ := json.Unmarshal([]byte(your_json), &c)
for _, i := range c.IconRaw:
UnmarshalIcon(&c, i)
Caveat Emptor: I haven't checked above implementation but it should be something like this
You can not use []struct {} for icon, change it to []interface{} instead, or if you want operate on type safe type look at the second solution with cusom unmarshaler
Solution 1
package main
import (
"encoding/json"
"fmt"
"log"
)
type CatalogBank struct {
Advice string `json:"advice"`
BankCode string `json:"bank_code"`
BankName string `json:"bank_name"`
BIC string `json:"bic"`
Credentials []struct {
Label string `json:"label"`
Masked bool `json:"masked"`
} `json:"credentials"`
Icon []interface{} `json:"icon"`
Language []byte `json:"language"`
}
func main() {
data := `
{
"Advice":"abc",
"icon": [
"https://api.figo.me/assets/images/accounts/postbank.png",
{
"48x48": "https://api.figo.me/assets/images/accounts/postbank_48.png",
"60x60": "https://api.figo.me/assets/images/accounts/postbank_60.png",
"72x72": "https://api.figo.me/assets/images/accounts/postbank_72.png",
"84x84": "https://api.figo.me/assets/images/accounts/postbank_84.png",
"96x96": "https://api.figo.me/assets/images/accounts/postbank_96.png",
"120x120": "https://api.figo.me/assets/images/accounts/postbank_120.png",
"144x144": "https://api.figo.me/assets/images/accounts/postbank_144.png",
"192x192": "https://api.figo.me/assets/images/accounts/postbank_192.png",
"256x256": "https://api.figo.me/assets/images/accounts/postbank_256.png"
}
]
}
`
bank := &CatalogBank{}
err := json.Unmarshal([]byte(data), bank)
if err != nil {
log.Fatal(err)
}
for _, icon := range bank.Icon {
fmt.Printf(" %v\n ", icon)
}
}
Solution 2:
package main
import (
"encoding/json"
"fmt"
"log"
)
type Icons struct {
URL string
BySize map[string]string
}
type CatalogBank struct {
Advice string `json:"advice"`
BankCode string `json:"bank_code"`
BankName string `json:"bank_name"`
BIC string `json:"bic"`
Credentials []struct {
Label string `json:"label"`
Masked bool `json:"masked"`
} `json:"credentials"`
Icon *Icons `json:"-,"`
Language []byte `json:"language"`
}
func (p *CatalogBank) Unmarshal(data []byte) error {
type Transient struct {
*CatalogBank
Icon []interface{} `json:"icon"`
}
var transient = &Transient{CatalogBank:p}
err := json.Unmarshal([]byte(data), transient)
if err != nil {
return err
}
p.Icon = &Icons{
BySize: make(map[string]string),
}
if len(transient.Icon) > 0 {
if url, ok := transient.Icon[0].(string); ok {
p.Icon.URL = url
}
if aMap, ok := transient.Icon[1].(map[string]interface{}); ok {
for k, v := range aMap {
p.Icon.BySize[k] = v.(string)
}
}
}
return nil
}
func main() {
data := `
{
"Advice":"abc",
"icon": [
"https://api.figo.me/assets/images/accounts/postbank.png",
{
"48x48": "https://api.figo.me/assets/images/accounts/postbank_48.png",
"60x60": "https://api.figo.me/assets/images/accounts/postbank_60.png",
"72x72": "https://api.figo.me/assets/images/accounts/postbank_72.png",
"84x84": "https://api.figo.me/assets/images/accounts/postbank_84.png",
"96x96": "https://api.figo.me/assets/images/accounts/postbank_96.png",
"120x120": "https://api.figo.me/assets/images/accounts/postbank_120.png",
"144x144": "https://api.figo.me/assets/images/accounts/postbank_144.png",
"192x192": "https://api.figo.me/assets/images/accounts/postbank_192.png",
"256x256": "https://api.figo.me/assets/images/accounts/postbank_256.png"
}
]
}
`
bank := &CatalogBank{}
err := bank.Unmarshal([]byte(data))
if err != nil {
log.Fatal(err)
}
fmt.Printf("advice: %v\n", bank.Advice)
fmt.Printf("icon: %v\n", bank.Icon.URL)
for size, icon := range bank.Icon.BySize {
fmt.Printf("%v => %v\n ",size, icon)
}
}
You can define your icon like this:
package main
import (
"fmt"
"encoding/json"
)
var testIcon = []byte(`{"icon":[
"https://api.figo.me/assets/images/accounts/postbank.png",
{
"48x48":"https://api.figo.me/assets/images/accounts/postbank_48.png"
}]
}`)
func main() {
icon := make(map[string][]interface{})
err := json.Unmarshal(testIcon, &icon)
if err != nil {
panic(err)
}
fmt.Println(icon)
// map[icon:[https://api.figo.me/assets/images/accounts/postbank.png map[48x48:https://api.figo.me/assets/images/accounts/postbank_48.png]]]
}

Resources