I have a json array which is converted into a string. Now I want to map the string to a struct array so that I can modify the string json. Below is my code base
type ProcessdetailsEntity struct {
Source []int64 `json:"source"`
Node string `json:"node"`
Targets []int64 `json:"targets"`
Issave bool `json:"isSave"`
Instate []int64 `json:"inState"`
OutState []int64 `json:"outState"`
}
func main() {
var stringJson = "[{\"source\":[-1],\"node\":\"1_1628008588902\",\"targets\":[],\"isSave\":true,\"inState\":[1],\"outState\":[2]},{\"source\":[\"1_1628008588902\",\"5_1628008613446\"],\"node\":\"2_1628008595757\",\"targets\":[],\"isSave\":true,\"inState\":[2,5],\"outState\":[3,6]}]"
in := []byte(stringJson)
detailsEntity := []ProcessdetailsEntity{}
err := json.Unmarshal(in, &detailsEntity)
if err != nil {
log.Print(err)
}
}
Now when I run this code base I got the error:
json: cannot unmarshal string into Go struct field ProcessdetailsEntity.source of type int64
How to properly map the string to struct so that I can modify the inState and outState value of the json ?
The error you get is already pretty much on the nose:
cannot unmarshal string into Go struct field ProcessdetailsEntity.source of type int64
That tells you that (at least one) of your source fields appears to have the wrong type: a string instead of something that can be represented by a int64.
So let's check your source fields in your stringJson:
"source":[-1]
"source":["1_1628008588902","5_1628008613446"]
As you can see the second source is an array of string. Hence the error.
To solve this you need to make sure that the source is an array of int. Unfortunately, 1_1628008588902 and 5_1628008613446 are not valid integers in Go.
I slightly modified your JSON and fixed your code an then it works:
package main
import (
"encoding/json"
"log"
)
type ProcessdetailsEntity struct {
Source []int64 `json:"source"`
Node string `json:"node"`
Targets []int64 `json:"targets"`
Issave bool `json:"isSave"`
Instate []int64 `json:"inState"`
OutState []int64 `json:"outState"`
}
func main() {
var stringJson = `[
{
"source":[-1],
"node":"1_1628008588902",
"targets":[],
"isSave":true,
"inState":[1],
"outState":[2]
},
{
"source":[11628008588902,51628008613446],
"node":"2_1628008595757",
"targets":[],
"isSave":true,
"inState":[2,5],
"outState":[3,6]
}
]`
in := []byte(stringJson)
detailsEntity := []ProcessdetailsEntity{}
err := json.Unmarshal(in, &detailsEntity)
if err != nil {
log.Print(err)
}
}
See: https://play.golang.org/p/kcrkfRliWJ5
Related
I saw several questions asking on how to merge unique structs and how to merge identical structs.
But how would I merge structs that have some overlap? and which fields get taken & when?
e.g.:
type structOne struct {
id string `json:id`
date string `json:date`
desc string `json:desc`
}
and
type structTwo struct {
id string `json:id`
date string `json:date`
name string `json:name`
}
how would I merge it such that I get
{
id string `json:id`
date string `json:date`
desc string `json:desc`
name string `json:name`
}
also, what happens if in this case the two id's are the same (assuming a join over id's) but the names are different?
In javascript, doing something like Object.assign(structOne, structTwo).
Go is a strongly typed language, unlike javascript you can't merge two struct into one combined struct because all type are determined at compile-time. You have two solution here :
Using embedded struct:
One great solution is to use embedded struct because you don't have to merge anything anymore.
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
)
// Shared field
type common struct {
ID string `json:id`
Date string `json:date`
}
type merged struct {
// Common field is embedded
common
Name string `json:name`
Desc string `json:desc`
}
func main() {
buf := bytes.Buffer{}
buf.WriteString("{ \"id\": \"1\", \"date\": \"27/07/2020\", \"desc\": \"the decription...\" }")
merged := &merged{}
err := json.Unmarshal(buf.Bytes(), merged)
if err != nil {
log.Fatal(err)
}
// Look how you can easily access field from
// embedded struct
fmt.Println("ID:", merged.ID)
fmt.Println("Date:", merged.Date)
fmt.Println("Name:", merged.Name)
fmt.Println("Desc:", merged.Desc)
// Output:
// ID: 1
// Date: 27/07/2020
// Name:
// Desc: the decription...
}
If you want to read more about struct embedding:
golangbyexample.com
travix.io
Using Maps
Another solution is to use maps but you will loose the benefits of struct and methods. This example is not the simplest but there is some great example in the other responses.
In this example I'm using Mergo. Mergo is library that can merge structs and map. Here it is used for creating maps object in the Map methods but you can totally write your own methods.
package main
import (
"fmt"
"log"
"github.com/imdario/mergo"
)
type tOne struct {
ID string
Date string
Desc string
}
// Map build a map object from the struct tOne
func (t1 tOne) Map() map[string]interface{} {
m := make(map[string]interface{}, 3)
if err := mergo.Map(&m, t1); err != nil {
log.Fatal(err)
}
return m
}
type tTwo struct {
ID string
Date string
Name string
}
// Map build a map object from the struct tTwo
func (t2 tTwo) Map() map[string]interface{} {
m := make(map[string]interface{}, 3)
if err := mergo.Map(&m, t2); err != nil {
log.Fatal(err)
}
return m
}
func main() {
dst := tOne{
ID: "destination",
Date: "26/07/2020",
Desc: "destination object",
}.Map()
src := tTwo{
ID: "src",
Date: "26/07/1010",
Name: "source name",
}.Map()
if err := mergo.Merge(&dst, src); err != nil {
log.Fatal(err)
}
fmt.Printf("Destination:\n%+v", dst)
// Output:
// Destination:
// map[date:26/07/2020 desc:destination object iD:destination name:object name
}
Go structs and JavaScript objects are very different. Go structs do not have dynamic fields.
If you want dynamic key/value sets that you can easily iterate over and merge, and are very JSON friendly, why not a map[string]interface{}?
$ go run t.go
map[a:1 b:4]
map[a:1 b:4 c:3]
$ cat t.go
package main
import(
"fmt"
)
type MyObj map[string]interface{}
func (mo MyObj)Merge(omo MyObj){
for k, v := range omo {
mo[k] = v
}
}
func main() {
a := MyObj{"a": 1, "b": 4}
b := MyObj{"b": 2, "c": 3}
b.Merge(a)
fmt.Printf("%+v\n%+v\n", a, b)
}
you can use github.com/fatih/structs to convert your struct to map. Then iterate over that map and choose which fields need to copy over. I have a snippet of code which illustrates this solution.
func MergeStruct (a structOne,b structTwo) map[string]interface{}{
a1:=structs.Map(a)
b1:=structs.Map(b)
/* values of structTwo over writes values of structOne */
var myMap=make(map[string]interface{})
for val,key:=range(a1){
myMap[key]=val
}
for val,key:=range(b1){
myMap[key]=val
}
return myMap
}
You can use the reflect package to do this. Try iterating through the two structs and then you can either use another struct type to store the values or maybe use a map.
Check out this question to find out how you can iterate over a struct.
Check out this question to find out how to get the name of the fields.
Remember to use exported field names for the reflect package to work.
Here is an example which works.
I want to unmarshal several types from JSON and use the interface to represent the actual struct that it is different. But when I send the struct as interface{} it converts it to a map. The animal.json is:
"{"type":"cat","policies":[{"name":"kitty","parameter":{"duration":600,"percent":90}}]}"
package main
import (
"reflect"
log "github.com/sirupsen/logrus"
"github.com/spf13/viper"
)
func main() {
var err error
animal := New()
viper.SetConfigType("json")
viper.SetConfigName("animal")
viper.AddConfigPath("~/Desktop/")
viper.AddConfigPath(".")
if err := viper.ReadInConfig(); err != nil {
return
}
if err = viper.Unmarshal(&animal); err != nil {
return
}
for _, policy := range animal.Policies {
log.Info(policy.Name)
log.Info(policy.Parameter)
//INFO[0000] map[duration:600 percent:90]
log.Info(reflect.TypeOf(policy.Parameter))
//INFO[0000] map[string]interface {}, Why is it not an interface{} and how do I get it?
switch t := policy.Parameter.(type) {
//why does the switch not work?
case *CatParameter:
log.Info("cat", t)
case *DogParameter:
log.Info("dog", t)
}
}
}
func New() *Animal {
var animal Animal
animal.Type = "cat"
return &animal
}
type Animal struct {
Type string `json:"type" form:"type"`
Policies []Policy `json:"policies" form:"policies"`
}
type CatParameter struct {
Duration int `json:"duration"`
Percent int `json:"percent"`
}
type DogParameter struct {
Percent int `json:"percent"`
Duration int `json:"duration"`
Operation string `json:"operation"`
}
type Policy struct {
Name string `json:"name"`
Parameter interface{} `json:"parameter"`
}
It's json unmarshal feature
If you use an interface{} as a decoder, the default json object for interface{} is map[string]interface{}
You can see it here:
https://godoc.org/encoding/json#Unmarshal
bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null
So in t := policy.Parameter.(type), the t is map[string]interface{}
For solving your problem, you can try to define another field to distinguish CatParameter or DogParameter
Maybe:
type Policy struct {
Name string `json:"name"`
Parameter Parameter `json:"parameter"`
}
type Parameter struct {
Name string `json:"name"` // cat or dog
Percent int `json:"percent,omitempty"`
Duration int `json:"duration,omitempty"`
Operation string `json:"operation,omitempty"`
}
I get the following data:
{
"timestamp": "1526058949",
"bids": [
[
"7215.90",
"2.31930000"
],
[
"7215.77",
"1.00000000"
]
]
}
via websocket and I would like to unmarshall it into
type OrderBookItem struct {
Price string
Amount string
}
type OrderBookResult struct {
Timestamp string `json:"timestamp"`
Bids []OrderBookItem `json:"bids"`
Asks []OrderBookItem `json:"asks"`
}
Unmarshal it with:
s := e.Data.(string)
d := &OrderBookResult{}
err := json.Unmarshal([]byte(s), d)
if err == nil {
....
} else {
fmt.Println(err.Error())
}
But I keep getting the error:
json: cannot unmarshal string into Go struct field
OrderBookResult.bids of type feed.OrderBookItem
When I change the struct into
type OrderBookResult struct {
Timestamp string `json:"timestamp"`
Bids [][]string `json:"bids"`
Asks [][]string `json:"asks"`
}
it works. I would like them to be defined as float64 which is what they are. What do I have to change?
As the error says:
json: cannot unmarshal string into Go struct field
OrderBookResult.bids of type feed.OrderBookItem
We cannot convert OrderBookResult.bids which is a slice of string into OrderBookItem which is struct
Implement UnmarshalJSON interface to convert array into objects for price and amount of OrderBookItem struct. Like below
package main
import (
"fmt"
"encoding/json"
)
type OrderBookItem struct {
Price string
Amount string
}
func(item *OrderBookItem) UnmarshalJSON(data []byte)error{
var v []string
if err:= json.Unmarshal(data, &v);err!=nil{
fmt.Println(err)
return err
}
item.Price = v[0]
item.Amount = v[1]
return nil
}
type OrderBookResult struct {
Timestamp string `json:"timestamp"`
Bids []OrderBookItem `json:"bids"`
Asks []OrderBookItem `json:"asks"`
}
func main() {
var result OrderBookResult
jsonString := []byte(`{"timestamp": "1526058949", "bids": [["7215.90", "2.31930000"], ["7215.77", "1.00000000"]]}`)
if err := json.Unmarshal([]byte(jsonString), &result); err != nil{
fmt.Println(err)
}
fmt.Printf("%+v", result)
}
Playground working example
For more information read GoLang spec for Unmarshaler
You are treating your bids as a structure of two separate strings, when they are really a slice of strings in the JSON. If you change OrderBookItem to be
type OrderBookItem []string
which is how you have defined them in the second bit, which works.
To access the values you just have to do:
price := d.Bids[0]
amount := d.Bids[1]
Code below generates this error json: cannot unmarshal number into Go struct field TMP.a of type string
package main
import (
"encoding/json"
"fmt"
)
var b = []byte(`{"a": "str", "A": 123}`)
type TMP struct {
// A interface{} `json:"a"`
A string `json:"a"`
// A int `json:"A"`
}
func main() {
var tmp TMP
err := json.Unmarshal(b, &tmp)
if err != nil {
fmt.Println(err)
}
}
I've read through https://golang.org/pkg/encoding/json/#Marshal and cannot find anything that states this shouldn't work. What am I missing?
It's been asked that I clarify, and rightly so. What I'm really wondering is why when I use JSON with only 2 keys that differ in uppercase/lowercase why Unmarshal is not keeping the case I've provided in the struct json tag.
First of all change the names of your fields. Since they have same name only uppercase A and lowercase a. So when go try to marshal the fields it is unable to recognize between the fields.
package main
import (
"encoding/json"
"fmt"
)
var b = []byte(`{"a": "str", "B": 123}`)
type TMP struct {
// A interface{} `json:"a"`
A string `json:"A"`
B int `json:"B"`
}
func main() {
var tmp TMP
err := json.Unmarshal(b, &tmp)
if err != nil {
fmt.Println(err)
}
fmt.Println(tmp)
}
As the error says
json: cannot unmarshal string into Go struct field TMP.A of type int
even if you try to pass both fields like below it will give same error
var b = []byte(`{"a": "str", "A": 123}`)
type TMP struct {
// A interface{} `json:"a"`
a string `json:"a"`
A int `json:"A"`
}
The real problem is that you're trying to unmarshal int to a string field. Error will occur even if you remove the "a": "str", and will work fine if you double quote the 123
What is a little funny, however, is that the case is ignored with only one field. Changing your struct to:
type TMP struct {
// A interface{}json:"a"
A stringjson:"a"
B intjson:"A"
}
also works.
I'm writing a websocket client in Go. I'm receiving the following JSON from the server:
{"args":[{"time":"2013-05-21 16:57:17"}],"name":"send:time"}
I'm trying to access the time parameter, but just can't grasp how to reach deep into an interface type:
package main;
import "encoding/json"
import "log"
func main() {
msg := `{"args":[{"time":"2013-05-21 16:56:16", "tzs":[{"name":"GMT"}]}],"name":"send:time"}`
u := map[string]interface{}{}
err := json.Unmarshal([]byte(msg), &u)
if err != nil {
panic(err)
}
args := u["args"]
log.Println( args[0]["time"] ) // invalid notation...
}
Which obviously errors, since the notation is not right:
invalid operation: args[0] (index of type interface {})
I just can't find a way to dig into the map to grab deeply nested keys and values.
Once I can get over grabbing dynamic values, I'd like to declare these messages. How would I write a type struct to represent such complex data structs?
You may like to consider the package github.com/bitly/go-simplejson
See the doc： http://godoc.org/github.com/bitly/go-simplejson
Example:
time, err := json.Get("args").GetIndex(0).String("time")
if err != nil {
panic(err)
}
log.Println(time)
The interface{} part of the map[string]interface{} you decode into will match the type of that field. So in this case:
args.([]interface{})[0].(map[string]interface{})["time"].(string)
should return "2013-05-21 16:56:16"
However, if you know the structure of the JSON, you should try defining a struct that matches that structure and unmarshal into that. Ex:
type Time struct {
Time time.Time `json:"time"`
Timezone []TZStruct `json:"tzs"` // obv. you need to define TZStruct as well
Name string `json:"name"`
}
type TimeResponse struct {
Args []Time `json:"args"`
}
var t TimeResponse
json.Unmarshal(msg, &t)
That may not be perfect, but should give you the idea
I'm extremely new to Golang coming from Python, and have always struggled with encode/decoding json. I found gjson at https://github.com/tidwall/gjson, and it helped me immensely:
package main
import "github.com/tidwall/gjson"
func main() {
msg := (`{"args":[{"time":"2013-05-21 16:56:16", "tzs":[{"name":"GMT"}]}],"name":"send:time"}`)
value := gjson.Get(msg, "args.#.time")
println(value.String())
}
-----------------------
["2013-05-21 16:56:16"]
Additionally, I noticed the comment of how to convert into Struct
package main
import (
"encoding/json"
"fmt"
)
type msgFormat struct {
Time string `json:"time"`
Tzs msgFormatTzs `json:"tzs"`
Name string `json:"name"`
}
type msgFormatTzs struct {
TzsName string `json:"name"`
}
func main() {
msg := (`{"args":[{"time":"2013-05-21 16:56:16", "tzs":[{"name":"GMT"}]}],"name":"send:time"}`)
r, err := json.Marshal(msgFormatTzs{msg})
if err != nil {
panic(err)
}
fmt.Printf("%v", r)
}
Try on Go playground