get http.Response into struct using json.Unmarshal - go

I'm calling a remote API and getting a JSON response back. I'm trying to convert the *http.Response into a struct that I defined. Everything i've tried so far has resulted in an empty struct. Here is my attempt with json.Unmarshal
type Summary struct {
Created string `json:"created"`
High float64 `json:"high"`
Low float64 `json:"low"`
}
func getSummary() {
url := "http://myurl"
resp, err := http.Get(url)
if err != nil {
log.Fatalln(err)
}
body, err2 := ioutil.ReadAll(resp.Body)
if err2 != nil {
panic(err.Error())
}
log.Printf("body = %v", string(body))
//outputs: {"success":true,"message":"","result":["High":0.43600000,"Low":0.43003737],"Created":"2017-06-25T03:06:46.83"}]}
var summary = new(Summary)
err3 := json.Unmarshal(body, &summary)
if err3 != nil {
fmt.Println("whoops:", err3)
//outputs: whoops: <nil>
}
log.Printf("s = %v", summary)
//outputs: s = &{{0 0 0 0 0 0 0 0 0 0}}
}
What am I doing wrong? The JSON tags in my struct match the json keys from the response exactly...
edit: here is the JSON returned from the API
{
"success": true,
"message": "''",
"result": [
{
"High": 0.0135,
"Low": 0.012,
"Created": "2014-02-13T00:00:00"
}
]
}
edit
i changed the struct to this but still not working
type Summary struct {
Result struct {
Created string `json:"created"`
High float64 `json:"high"`
Low float64 `json:"low"`
}
}

Change your structure like this
type Summary struct {
Sucess bool `json:"success"`
Message string `json:"message"`
Result []Result `json:"result"`
}
type Result struct {
Created string `json:"Created"`
High float64 `json:"High"`
Low float64 `json:"Low"`
}
Try this link

Its because you're trying to unmarshal an array into struct,
Use array instead of the Result struct
type Summary struct {
Result []struct {
Created string `json:"created"`
High float64 `json:"high"`
Low float64 `json:"low"`
}
}
Use this weblink to convert your JSON objects to Go Struct >> https://mholt.github.io/json-to-go/

Related

Changing value of field in a go struct

I'm making an http request in golang to an external api. It gives a general response of {"error":[]string, "result":changing interface{}}. depending on the function that is making the request, the Result field changes. Since I know the structure of the Result field for each function I run, I want to be able to change the value of Result before unmarshalling to json. I've tried to do this with the following code:
func GetAssets(output *Resp, resultType interface{}) error {
return publicRequest("/Assets", output, resultType)
}
func publicRequest(endPoint string, output *Resp, resultType interface{}) error {
url := Rest_url + Pub_rest_url + endPoint //"https://api.kraken.com/0/public/Assets in this case
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
output.Result = resultType
return json.NewDecoder(resp.Body).Decode(&output)
}
Here is how it's being ran in main
type Resp struct {
Error []string `json:"error"`
Result interface{} `json:"result"`
}
type AssetInfo struct {
Aclass string `json:"aclass"`
Altname string `json:"altname"`
Decimals int `json:"decimals"`
Display int `json:"display_decimals"`
}
func main() {
var result map[string]AssetInfo
jsonData := Resp{}
rest_api_client.GetAssets(&jsonData, result)
fmt.Println(jsonData)
}
The issue is that it doesn't unmarshal correctly. A map is created for each asset, but the data contained inside of each asset is also being stored inside of a map. I'm not sure if I explained this well, but here is the current response after unmarshalling to understand what I mean.
Here is the data type of Resp.Result: map[string]interface {}
{[] map[1INCH:map[aclass:currency altname:1INCH decimals:10 display_decimals:5] AAVE:map[aclass:currency altname:AAVE decimals:10 display_decimals:5] ACA:map[aclass:currency altname:ACA decimals:10 display_decimals:5] ADA:map[aclass:currency altname:ADA decimals:8 display_decimals:6]...}
The response type I'm looking for is map[string]AssetInfo. Hopefully it could be unmarshalled like this:
{[] map[1INCH:{currency 1INCH 10 5} AAVE:{currency AAVE 10 5} ACA:{currency ACA 10 5} ADA:{currency ADA 8 6} ADA.S:{currency ADA.S 8 6}...}
Any help? I'd rather keep the Resp struct as generic as possible and just change the value of the Result field (if this is even possible to do correctly) since I plan to have multiple functions that call different endpoints of the api, and they'll all have the same underlying response type of the Resp struct with different Result types
You can view a working example in the following repo:
https://github.com/alessiosavi/GoArbitrage/blob/e107af466852b1ed30c2413eb4401595f7412b4f/markets/kraken/kraken.go
Basically, I've defined the following structure:
type Tickers struct {
Error []interface{} `json:"error"`
Result map[string]Ticker `json:"result"`
}
type Ticker struct {
Aclass string `json:"aclass"`
Altname string `json:"altname"`
Decimals int `json:"decimals"`
DisplayDecimals int `json:"display_decimals"`
}
Than I execute the request in the following way:
const KRAKEN_TICKERS_URL string = `https://api.kraken.com/0/public/Assets`
type Kraken struct {
PairsNames []string `json:"pairs_name"`
Pairs map[string]datastructure.KrakenPair `json:"pairs"`
OrderBook map[string]datastructure.KrakenOrderBook `json:"orderbook"`
MakerFee float64 `json:"maker_fee"`
TakerFees float64 `json:"taker_fee"`
// FeePercent is delegated to save if the fee is in percent or in coin
FeePercent bool `json:"fee_percent"`
Tickers []string
}
// Init is delegated to initialize the maps for the kraken
func (k *Kraken) Init() {
k.Pairs = make(map[string]datastructure.KrakenPair)
k.OrderBook = make(map[string]datastructure.KrakenOrderBook)
k.SetFees()
}
// SetFees is delegated to initialize the fee type/amount for the given market
func (k *Kraken) SetFees() {
k.MakerFee = 0.16
k.TakerFees = 0.26
k.FeePercent = true
}
func (k *Kraken) GetTickers() error {
res := datastructure.Tickers{}
var err error
var request req.Request
var data []byte
var tickers []string
resp := request.SendRequest(KRAKEN_TICKERS_URL, "GET", nil, nil, false, 10*time.Second)
if resp.Error != nil {
zap.S().Debugw("Error during http request. Err: " + resp.Error.Error())
return resp.Error
}
if resp.StatusCode != 200 {
zap.S().Warnw("Received a non 200 status code: " + strconv.Itoa(resp.StatusCode))
return errors.New("NON_200_STATUS_CODE")
}
data = resp.Body
if err = json.Unmarshal(data, &res); err != nil {
zap.S().Warn("ERROR! :" + err.Error())
return err
}
zap.S().Infof("Data: %v", res.Result)
tickers = make([]string, len(res.Result))
i := 0
for key := range res.Result {
tickers[i] = res.Result[key].Altname
i++
}
k.Tickers = tickers
return nil
}

Why go json.Unmarshal auto convert interface{} to map

The program will receive many msg, msg has different struct "Data", so I define the Msg struct:
type Msg struct {
MsgType int
Data interface{}
}
type Data1 struct {
//msg type 1 Data struct
}
type Data2 struct {
//msg type 2 Data struct
}
func (msgStr string) {
msg := Msg{}
if err := json.Unmarshal([]byte(msgStr), &msg); err != nil {
//log err
}
switch msg.MsgType{
case 1:
//convert msg.Data to a type 1 struct
case 2:
//convert msg.Data to a type 2 struct
}
}
But print out the msg.Data, it is a map, not interface{}, so when I convert it to Data1 by msg.Data.(Data1), got an err.
So,
1. Why interface{} auto convert to map?
2. How to convert it to Data1 struct I want?
3. What is the best practices in this scenes.
1. Because it sees a JSON object, and, as documented, a JSON object becomes a map[string]interface{} when stored into an interface{} (this is the only type that can hold whatever is in a JSON object in generality).
2. Given your current situation, you could assign each field of the map to the appropriate field of a new Data1 or Data2.
3. The ideal way to handle this is to use json.RawMessage to defer the decoding of Data until you know what it is. This can be handled like so:
type Msg struct {
MsgType int
Data interface{}
}
func (m *Msg) UnmarshalJSON(b []byte) (err error) {
var tmp struct {
MsgType int
Data json.RawMessage
}
err = json.Unmarshal(b, &tmp)
if err != nil {
return
}
m.MsgType = tmp.MsgType
switch (tmp.MsgType) {
case 1:
data := Data1{}
err = json.Unmarshal(tmp.Data, &data)
if err != nil {
return
}
m.Data = data
case 2:
data := Data2{}
err = json.Unmarshal(tmp.Data, &data)
if err != nil {
return
}
m.Data = data
default:
return errors.New("invalid DataType")
}
return
}
And then you can call json.Unmarshal or json.Decode directly on a *Msg and its Data will be decoded as you want.

Unmarshaling json into a type

I get the following data:
{
"timestamp": "1526058949",
"bids": [
[
"7215.90",
"2.31930000"
],
[
"7215.77",
"1.00000000"
]
]
}
via websocket and I would like to unmarshall it into
type OrderBookItem struct {
Price string
Amount string
}
type OrderBookResult struct {
Timestamp string `json:"timestamp"`
Bids []OrderBookItem `json:"bids"`
Asks []OrderBookItem `json:"asks"`
}
Unmarshal it with:
s := e.Data.(string)
d := &OrderBookResult{}
err := json.Unmarshal([]byte(s), d)
if err == nil {
....
} else {
fmt.Println(err.Error())
}
But I keep getting the error:
json: cannot unmarshal string into Go struct field
OrderBookResult.bids of type feed.OrderBookItem
When I change the struct into
type OrderBookResult struct {
Timestamp string `json:"timestamp"`
Bids [][]string `json:"bids"`
Asks [][]string `json:"asks"`
}
it works. I would like them to be defined as float64 which is what they are. What do I have to change?
As the error says:
json: cannot unmarshal string into Go struct field
OrderBookResult.bids of type feed.OrderBookItem
We cannot convert OrderBookResult.bids which is a slice of string into OrderBookItem which is struct
Implement UnmarshalJSON interface to convert array into objects for price and amount of OrderBookItem struct. Like below
package main
import (
"fmt"
"encoding/json"
)
type OrderBookItem struct {
Price string
Amount string
}
func(item *OrderBookItem) UnmarshalJSON(data []byte)error{
var v []string
if err:= json.Unmarshal(data, &v);err!=nil{
fmt.Println(err)
return err
}
item.Price = v[0]
item.Amount = v[1]
return nil
}
type OrderBookResult struct {
Timestamp string `json:"timestamp"`
Bids []OrderBookItem `json:"bids"`
Asks []OrderBookItem `json:"asks"`
}
func main() {
var result OrderBookResult
jsonString := []byte(`{"timestamp": "1526058949", "bids": [["7215.90", "2.31930000"], ["7215.77", "1.00000000"]]}`)
if err := json.Unmarshal([]byte(jsonString), &result); err != nil{
fmt.Println(err)
}
fmt.Printf("%+v", result)
}
Playground working example
For more information read GoLang spec for Unmarshaler
You are treating your bids as a structure of two separate strings, when they are really a slice of strings in the JSON. If you change OrderBookItem to be
type OrderBookItem []string
which is how you have defined them in the second bit, which works.
To access the values you just have to do:
price := d.Bids[0]
amount := d.Bids[1]

Golang unmarshal array without key/value to stuct

I'm trying to place a json array into a struct from Google Analytics API.
EG:
"rows": [
[
"female",
"18-24",
"1308"
],
[
"female",
"25-34",
"741"
]
]
Typically I'd have key/value so I can put json:"gender" but there are no keys to associate with, so the values that it would search for change.
The struct would be:
type Row struct {
Gender string `json:"gender"`
AgeRange string `json:"blah"`
Count string `json:"blah"`
}
If I do len(jResp.Rows) I can see that it's grabbing all twelve rows/arrays but the fields are empty.
I don't think it's possible, with encoding/json to directly decode that json into a slice of structs without first implementing a UnmarshalJSON method on your Row type.
func (r *Row) UnmarshalJSON(data []byte) error {
var s []string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
if len(s) >= 3 {
r.Gender = s[0]
r.AgeRange = s[1]
r.Count = s[2]
}
return nil
}
// make sure it's a slice of pointers to Row
type Resp struct {
Rows []*Row `json:"rows"`
}
Edit: fixed the code a little to make it actually compile. Heres a working example https://play.golang.org/p/eqVQj65xJv.
You could also just decode the data first into a slice of strings and then loop over the result to build you struct values.
type Resp struct {
Rows [][]string `json:"rows"`
}
type Row struct {
Gender string `json:"gender"`
AgeRange string `json:"blah"`
Count string `json:"blah"`
}
var resp jResp
if err := json.Unmarshal(data, &resp); err != nil {
panic(err)
}
var rows = make([]Row, len(resp.Rows))
for i, r := range resp.Rows {
rows[i] = Row{
Gender: r[0],
AgeRange: r[1],
Count: r[2],
}
}
Edit: fixed this one as well. https://play.golang.org/p/Otb7iULSh3

unexpected end of JSON input while unmarshal to struct

Could you help me to understand why I'm always getting the error unexpected end of JSON input while trying to unmarshal the following json to the LimitOrder struct?
P.S.: if I use map[string]json.RawMessage instead of LimitOrder struct I'm able to execute the unmarshal.
{
"response_data": {
"order": {
"order_id": 3,
"coin_pair": "BRLBTC",
"order_type": 1,
"status": 4,
"has_fills": true,
"quantity": "1.00000000",
"limit_price": "900.00000",
"executed_quantity": "1.00000000",
"executed_price_avg": "900.00000",
"fee": "0.00300000",
"created_timestamp": "1453835329",
"updated_timestamp": "1453835329",
"operations": [
{
"operation_id": 1,
"quantity": "1.00000000",
"price": "900.00000",
"fee_rate": "0.30",
"executed_timestamp": "1453835329"
}
]
}
},
"status_code": 100,
"server_unix_timestamp": "1453835329"
}
LimitOrder struct
type LimitOrder struct {
OrderId int `json:"order_id"`
CoinPair string `json:"coin_pair"`
OrderType int `json:"order_type"`
Status int `json:"status"`
HasFills bool `json:"has_fills"`
Quantity float64 `json:"quantity,string"`
LimitPrice float64 `json:"limit_price,string"`
ExecutedQuantity float64 `json:"executed_quantity,string"`
ExecutedPriceAvg float64 `json:"executed_price_avg,string"`
Fee float64 `json:"fee,string"`
Operations []*Operation `json:"operations"`
CreatedTimestamp string `json:"created_timestamp"`
UpdatedTimestamp string `json:"updated_timestamp"`
}
and this is how I'm trying to unmarshal it
func (limitOrder *LimitOrder) UnmarshalJSON(buf []byte) error {
tmp := make(map[string]json.RawMessage)
if err := json.Unmarshal(buf, &tmp); err != nil {
return err
}
tmp2 := make(map[string]json.RawMessage)
if err := json.Unmarshal(tmp["response_data"], &tmp2); err != nil {
return err
}
if err := json.Unmarshal(tmp2["order"], limitOrder); err != nil {
return err
}
return nil
}
I found some help on golang-nuts group. You can check the answer here https://groups.google.com/forum/#!topic/golang-nuts/SZXBcXgUoo0. To make a long story short, the problem was with my structures, I've built structures only for a small piece of the whole json, so I fixed it building structures for the whole json response.

Resources