Iterate over values of map of interfaces - go

How do I iterate over the map of interfaces below to get the values of the interface returned in the map?
I have read through the extensive list of questions regarding iteration in Go but they didn't help me.
// https://api.kraken.com/0/public/AssetPairs
pairsResult, err := api.Query("AssetPairs", map[string]string{
})
if err != nil {
log.Fatal(err)
}
ks := reflect.ValueOf(pairsResult).MapKeys()
fmt.Printf(" %+v ", pairsResult) // result A below
fmt.Printf(" %+v ", ks) // result B below
// the value here is the value of MapKeys, which is the key
for kix, key := range ks {
fmt.Printf(" %+v %+v \n ", kix, key) // result C below
}
Result A
map[AAVEETH:map[aclass_base:currency aclass_quote:currency altname:AAVEETH base:AAVE fee_volume_currency:ZUSD fees:[[0 0.26] [50000 0.24] [100000 0.22] [250000 0.2] [500000 0.18]...
Result B
[KEEPXBT LINKUSD LINKXBT NANOEUR ...]
Result C
0 KEEPXBT 1 LINKUSD 2 LINKXBT 3 NANOEUR 4 USDTAUD ...
This is the source of the API wrapper function that is being called above
// Query sends a query to Kraken api for given method and parameters
func (api *KrakenAPI) Query(method string, data map[string]string) (interface{}, error) {
values := url.Values{}
for key, value := range data {
values.Set(key, value)
}
// Check if method is public or private
if isStringInSlice(method, publicMethods) {
return api.queryPublic(method, values, nil)
} else if isStringInSlice(method, privateMethods) {
return api.queryPrivate(method, values, nil)
}
return nil, fmt.Errorf("Method '%s' is not valid", method)
}
This happens when I try to iterate over the value:
This happens when I try to iterate over the initial result:

Assuming you're using this and from looking a bit into its source it seems to me that the concrete type of the result will be map[string]interface{}, if that is the case then you can do this.
res, err := api.Query("AssetPairs", map[string]string{})
if err != nil {
log.Fatal(err)
}
pairs, ok := res.(map[string]interface{})
if !ok {
log.Fatal("unsupported type")
}
for k, v := range pairs {
fmt.Printf("key=%s value=%+v\n ", k, v)
}

As the previous response mentions, we see that the interface returned becomes a map[string]interface{}, the following code would do the trick to retrieve the types:
for _, v := range d.(map[string]interface{}) {
switch v.(type) {
case map[string]interface{}:
fmt.Println("Its another map of string interface")
case interface{}:
fmt.Println("its an interface")
case string:
fmt.Println("its a string")
case []string:
fmt.Println("its a string array")
case float32:
fmt.Println("its a float32")
case float64:
fmt.Println("its a float64")
default:
fmt.Printf("Different thing, %T\n", v)
}
}
Code here:
https://play.golang.org/p/81LLYSvJVf8
However, I would recommend use explicit type, that would make your life much easier:
// Generated by https://quicktype.io
type KrakenTypes struct {
Error []interface{} `json:"error"`
Result map[string]Result `json:"result"`
}
type Result struct {
Altname string `json:"altname"`
Wsname *string `json:"wsname,omitempty"`
AclassBase Aclass `json:"aclass_base"`
Base string `json:"base"`
AclassQuote Aclass `json:"aclass_quote"`
Quote FeeVolumeCurrency `json:"quote"`
Lot Lot `json:"lot"`
PairDecimals int64 `json:"pair_decimals"`
LotDecimals int64 `json:"lot_decimals"`
LotMultiplier int64 `json:"lot_multiplier"`
LeverageBuy []int64 `json:"leverage_buy"`
LeverageSell []int64 `json:"leverage_sell"`
Fees [][]float64 `json:"fees"`
FeesMaker [][]float64 `json:"fees_maker"`
FeeVolumeCurrency FeeVolumeCurrency `json:"fee_volume_currency"`
MarginCall int64 `json:"margin_call"`
MarginStop int64 `json:"margin_stop"`
Ordermin *string `json:"ordermin,omitempty"`
}
Here we can use json decoding once the response was read, so the iterations to find out the types of each level could be avoided and we can just access to each member directly.
Complete code here: https://play.golang.org/p/v3tlroyx1mW

Related

Given an interface of `string | int` how do I route correctly by type in Golang? [duplicate]

In other words, how do I implement type-specific solutions for different types in a union type set?
Given the following code...
type FieldType interface {
string | int
}
type Field[T FieldType] struct {
name string
defaultValue T
}
func NewField[T FieldType](name string, defaultValue T) *Field[T] {
return &Field[T]{
name: name,
defaultValue: defaultValue,
}
}
func (f *Field[T]) Name() string {
return f.name
}
func (f *Field[T]) Get() (T, error) {
value, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
return value, nil
}
the compiler shows the error:
field.go:37:9: cannot use value (variable of type string) as type T in return statement
Is there a way to provide implementations for all possible FieldTypes?
Like...
func (f *Field[string]) Get() (string, error) {
value, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
return value, nil
}
func (f *Field[int]) Get() (int, error) {
raw, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
value, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return *new(T), err
}
return int(value), nil
}
Any hint would be welcome.
The error occurs because operations that involve a type parameter (including assignments and returns) must be valid for all types in its type set.
In case of string | int, there isn't a common operation to initialize their value from a string.
However you still have a couple options:
Type-switch on T
You use the field with the generic type T in a type-switch, and temporarily set the values with concrete types into an interface{}/any. Then type-assert the interface back to T in order to return it. Beware that this assertion is unchecked, so it may panic if for some reason ret holds something that isn't in the type set of T. Of course you can check it with comma-ok but it's still a run-time assertion:
func (f *Field[T]) Get() (T, error) {
value, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
var ret any
switch any(f.defaultValue).(type) {
case string:
ret = value
case int:
// don't actually ignore errors
i, _ := strconv.ParseInt(value, 10, 64)
ret = int(i)
}
return ret.(T), nil
}
Type-switch on *T
You can further simplify the code above and get rid of the empty interface. In this case you take the address of the T-type variable and switch on the pointer types. This is fully type-checked at compile time:
func (f *Field[T]) Get() (T, error) {
value, ok := env[f.name]
if !ok {
return f.defaultValue, nil
}
var ret T
switch p := any(&ret).(type) {
case *string:
*p = value
case *int:
i, _ := strconv.ParseInt(value, 10, 64)
*p = int(i)
}
// ret has the zero value if no case matches
return ret, nil
}
Note that in both cases you must convert the T value to an interface{}/any in order to use it in a type switch. You can't type-switch directly on T.
Playground with map to simulate os.LookupEnv: https://go.dev/play/p/JVBEZwCXRMW
Ok, the type switch works if reflections are used.
func (f *Field[T]) Get() (T, error) {
raw, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
v := reflect.ValueOf(new(T))
switch v.Type().Elem().Kind() {
case reflect.String:
v.Elem().Set(reflect.ValueOf(raw))
case reflect.Int:
value, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return f.defaultValue, err
}
v.Elem().Set(reflect.ValueOf(int(value)))
}
return v.Elem().Interface().(T), nil
}
But better solutions are very welcome ;-)

Whats the best way to get content from a generic and somehow dynamic go map?

I have this json that I convert to:
var leerCHAT []interface{}
but I am going through crazy hoops to get to any point on that map inside map and inside map crazyness, specially because some results are different content.
this is the Json
[
null,
null,
"hub:zWXroom",
"presence_diff",
{
"joins":{
"f718a187-6e96-4d62-9c2d-67aedea00000":{
"metas":[
{
"context":{},
"permissions":{},
"phx_ref":"zNDwmfsome=",
"phx_ref_prev":"zDMbRTmsome=",
"presence":"lobby",
"profile":{},
"roles":{}
}
]
}
},
"leaves":{}
}
]
I need to get to profile then inside there is a "DisplayName" field.
so I been doing crazy hacks.. and even like this I got stuck half way...
First is an array so I can just do something[elementnumber]
then is when the tricky mapping starts...
SORRY about all the prints etc is to debug and see the number of elements I am getting back.
if leerCHAT[3] == "presence_diff" {
var id string
presence := leerCHAT[4].(map[string]interface{})
log.Printf("algo: %v", len(presence))
log.Printf("algo: %s", presence["joins"])
vamos := presence["joins"].(map[string]interface{})
for i := range vamos {
log.Println(i)
id = i
}
log.Println(len(vamos))
vamonos := vamos[id].(map[string]interface{})
log.Println(vamonos)
log.Println(len(vamonos))
metas := vamonos["profile"].(map[string]interface{}) \\\ I get error here..
log.Println(len(metas))
}
so far I can see all the way to the meta:{...} but can't continue with my hacky code into what I need.
NOTICE: that since the id after Joins: and before metas: is dynamic I have to get it somehow since is always just one element I did the for range loop to grab it.
The array element at index 3 describes the type of the variant JSON at index 4.
Here's how to decode the JSON to Go values. First, declare Go types for each of the variant parts of the JSON:
type PrescenceDiff struct {
Joins map[string]*Presence // declaration of Presence type to be supplied
Leaves map[string]*Presence
}
type Message struct {
Body string
}
Declare a map associating the type string to the Go type:
var messageTypes = map[string]reflect.Type{
"presence_diff": reflect.TypeOf(&PresenceDiff{}),
"message": reflect.TypeOf(&Message{}),
// add more types here as needed
}
Decode the variant part to a raw message. Use use the name in the element at index 3 to create a value of the appropriate Go type and decode to that value:
func decode(data []byte) (interface{}, error) {
var messageType string
var raw json.RawMessage
v := []interface{}{nil, nil, nil, &messageType, &raw}
err := json.Unmarshal(data, &v)
if err != nil {
return nil, err
}
if len(raw) == 0 {
return nil, errors.New("no message")
}
t := messageTypes[messageType]
if t == nil {
return nil, fmt.Errorf("unknown message type: %q", messageType)
}
result := reflect.New(t.Elem()).Interface()
err = json.Unmarshal(raw, result)
return result, err
}
Use type switches to access the variant part of the message:
defer ws.Close()
for {
_, data, err := ws.ReadMessage()
if err != nil {
log.Printf("Read error: %v", err)
break
}
v, err := decode(data)
if err != nil {
log.Printf("Decode error: %v", err)
continue
}
switch v := v.(type) {
case *PresenceDiff:
fmt.Println(v.Joins, v.Leaves)
case *Message:
fmt.Println(v.Body)
default:
fmt.Printf("type %T not handled\n", v)
}
}
Run it on the playground.

Cast a json in a properly struct instead of use an interface [duplicate]

This question already has answers here:
Unmarshal 2 different structs in a slice
(3 answers)
Closed 3 years ago.
i'm struggling to create a data structure for unmarshal the following json:
{
"asks": [
["2.049720", "183.556", 1576323009],
["2.049750", "555.125", 1576323009],
["2.049760", "393.580", 1576323008],
["2.049980", "206.514", 1576322995]
],
"bids": [
["2.043800", "20.691", 1576322350],
["2.039080", "755.396", 1576323007],
["2.036960", "214.621", 1576323006],
["2.036930", "700.792", 1576322987]
]
}
If I use the following struct with interfaces, there is no problem:
type OrderBook struct {
Asks [][]interface{} `json:"asks"`
Bids [][]interface{} `json:"bids"`
}
But i need a more strict typing, so i've tried with:
type BitfinexOrderBook struct {
Pair string `json:"pair"`
Asks [][]BitfinexOrder `json:"asks"`
Bids [][]BitfinexOrder `json:"bids"`
}
type BitfinexOrder struct {
Price string
Volume string
Timestamp time.Time
}
But unfortunately i had not success.
This is the code that I have used for parse the Kraken API for retrieve the order book:
// loadKrakenOrderBook is delegated to load the data related to pairs info
func loadKrakenOrderBook(data []byte) (datastructure.BitfinexOrderBook, error) {
var err error
// Creating the maps for the JSON data
m := map[string]interface{}{}
var orderbook datastructure.BitfinexOrderBook
// Parsing/Unmarshalling JSON
err = json.Unmarshal(data, &m)
if err != nil {
zap.S().Debugw("Error unmarshalling data: " + err.Error())
return orderbook, err
}
a := reflect.ValueOf(m["result"])
if a.Kind() == reflect.Map {
key := a.MapKeys()[0]
log.Println("KEY: ", key)
strct := a.MapIndex(key)
log.Println("MAP: ", strct)
m, _ := strct.Interface().(map[string]interface{})
log.Println("M: ", m)
data, err := json.Marshal(m)
if err != nil {
zap.S().Warnw("Panic on key: ", key.String(), " ERR: "+err.Error())
return orderbook, err
}
log.Println("DATA: ", string(data))
err = json.Unmarshal(data, &orderbook)
if err != nil {
zap.S().Warnw("Panic on key: ", key.String(), " during unmarshal. ERR: "+err.Error())
return orderbook, err
}
return orderbook, nil
}
return orderbook, errors.New("UNABLE_PARSE_VALUE")
}
The data that i use for test are the following:
{
"error": [],
"result": {
"LINKUSD": {
"asks": [
["2.049720", "183.556", 1576323009],
["2.049750", "555.125", 1576323009],
["2.049760", "393.580", 1576323008],
["2.049980", "206.514", 1576322995]
],
"bids": [
["2.043800", "20.691", 1576322350],
["2.039080", "755.396", 1576323007],
["2.036960", "214.621", 1576323006],
["2.036930", "700.792", 1576322987]
]
}
}
}
EDIT
NOTE: the data that i receive in input is the latest json that i've post, not the array of bids and asks.
I've tried to integrate the solution proposed by #chmike. Unfortunately there is a few preprocessing to be made, cause the data is the latest json that i've post.
So i've changed to code as following in order to extract the json data related to asks and bids.
func order(data []byte) (datastructure.BitfinexOrderBook, error) {
var err error
// Creating the maps for the JSON data
m := map[string]interface{}{}
var orderbook datastructure.BitfinexOrderBook
// var asks datastructure.BitfinexOrder
// var bids datastructure.BitfinexOrder
// Parsing/Unmarshalling JSON
err = json.Unmarshal(data, &m)
if err != nil {
zap.S().Warn("Error unmarshalling data: " + err.Error())
return orderbook, err
}
// Extract the "result" json
a := reflect.ValueOf(m["result"])
if a.Kind() == reflect.Map {
key := a.MapKeys()[0]
log.Println("KEY: ", key)
log.Println()
strct := a.MapIndex(key)
log.Println("MAP: ", strct)
m, _ := strct.Interface().(map[string]interface{})
log.Println("M: ", m)
log.Println("Asks: ", m["asks"])
log.Println("Bids: ", m["bids"])
// Here i retrieve the asks array
asks_data, err := json.Marshal(m["asks"])
log.Println("OK: ", err)
log.Println("ASKS: ", string(asks_data))
var asks datastructure.BitfinexOrder
// here i try to unmarshal the data into the struct
asks, err = UnmarshalJSON(asks_data)
log.Println(err)
log.Println(asks)
}
return orderbook, errors.New("UNABLE_PARSE_VALUE")
}
Unfortunately, i receive the following error:
json: cannot unmarshal array into Go value of type json.Number
As suggested by #Flimzy, you need a custom Unmarshaler. Here it is.
Note that the BitfinexOrderBook definition is slightly different from yours. There was an error in it.
// BitfinexOrderBook is a book of orders.
type BitfinexOrderBook struct {
Asks []BitfinexOrder `json:"asks"`
Bids []BitfinexOrder `json:"bids"`
}
// BitfinexOrder is a bitfinex order.
type BitfinexOrder struct {
Price string
Volume string
Timestamp time.Time
}
// UnmarshalJSON decode a BifinexOrder.
func (b *BitfinexOrder) UnmarshalJSON(data []byte) error {
var packedData []json.Number
err := json.Unmarshal(data, &packedData)
if err != nil {
return err
}
b.Price = packedData[0].String()
b.Volume = packedData[1].String()
t, err := packedData[2].Int64()
if err != nil {
return err
}
b.Timestamp = time.Unix(t, 0)
return nil
}
Note also that this custom unmarshaler function allows you to convert the price or volume to a float, which is probably what you want.
While you can hack your way by using reflex, or maybe even write your own parser, the most efficient way is to implement a json.Unmarshaler.
There are a few problem remaining, though.
You are transforming a json array to the struct, not just interface{} elements in it, so it should be: Asks []BitfinexOrder and Bids []BitfinexOrder.
You need to wrap the struct BitfinexOrderBook to get it work with its data. It is trivial and much simpler than using reflex.
By default, json.Unmarshal unmarshals a json number into a float64, which is not a good thing when parsing timestamp. You can use json.NewDecoder to get a decoder and then use Decoder.UseNumber to force use a string.
For example,
func (bo *BitfinexOrder) UnmarshalJSON(data []byte) error {
dec := json.NewDecoder(bytes.NewReader(data))
dec.UseNumber()
var x []interface{}
err := dec.Decode(&x)
if err != nil {
return errParse(err.Error())
}
if len(x) != 3 {
return errParse("length is not 3")
}
price, ok := x[0].(string)
if !ok {
return errParse("price is not string")
}
volume, ok := x[1].(string)
if !ok {
return errParse("volume is not string")
}
number, ok := x[2].(json.Number)
if !ok {
return errParse("timestamp is not number")
}
tint64, err := strconv.ParseInt(string(number), 10, 64)
if err != nil {
return errParse(fmt.Sprintf("parsing timestamp: %s", err))
}
*bo = BitfinexOrder{
Price: price,
Volume: volume,
Timestamp: time.Unix(tint64, 0),
}
return nil
}
and main func (wrapping the struct):
func main() {
x := struct {
Result struct{ LINKUSD BitfinexOrderBook }
}{}
err := json.Unmarshal(data, &x)
if err != nil {
log.Fatalln(err)
}
bob := x.Result.LINKUSD
fmt.Println(bob)
}
Playground link: https://play.golang.org/p/pC124F-3M_S .
Note: the playground link use a helper function to create errors. Some might argue it is best to name the helper function NewErrInvalidBitfinexOrder or rename the error. That is not the scope of this question and I think for the sake of typing, I will keep the short name for now.

Marshal sql.NullString with omitempty

How is one supposed to not render the null sql.NullString if it is not valid? - http://play.golang.org/p/pzSWS9vE0J
It doesn't seem to be working with omitempty struct tag and I can't quite figure out what to return from the MarshalJSON such that omitempty will be honored
type NS struct {
sql.NullString
}
func (ns *NS) MarshalJSON() ([]byte, error) {
fmt.Println("Marshaling json for NS")
if ns.String == "" && !ns.Valid {
return []byte("null"), nil
}
return json.Marshal(ns.String)
}
type A struct {
RStr string `json:rstr,omitempty"`
Str NS `json:"str,omitempty"`
}
func main() {
a := A{RStr: "rstr"}
s, _ := json.Marshal(&a)
fmt.Println(string(s))
}
As far as I know, there's no way to do that, however you can always use a pointer instead : http://play.golang.org/p/b4Q7YgpUa-
func main() {
a := A{"rstr", &sql.NullString{"Test", true}}
s, err := json.Marshal(&a)
fmt.Println(string(s), err)
a = A{}
fmt.Println(json.Unmarshal(s, &a))
fmt.Printf("%v %v\n", a.RStr, a.Str.String)
a = A{RStr: "rstr"}
s, err = json.Marshal(&a)
fmt.Println(string(s), err)
}
The reason why a pointer makes a difference from http://tip.golang.org/pkg/encoding/json/#Marshal:
-the field is empty and its tag specifies the "omitempty" option.
The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero.

Type conversion of string to int

Ok, so I've been having difficulties with the type conversion of a string to byte write. This is the compiler error:
cannot use row[5] (type uint8) as type string in function argument
cannot use &v (type *Field) as type int in function argument
This is an example of row[5]: $15,000.00
Ive declared a struct:
type Field struct {
Eia uint8
}
here is the main implementation:
for {
record, err := reader.Read()
if err == io.EOF {
break
} else if err != nil {
panic(err)
}
var v Field
for _, row := range record {
eia, err := strconv.ParseInt(row[5], 8, &v) // Estimated Incoming Amount
if err == nil {
fmt.Printf("%+v\n", v)
} else {
fmt.Println(err)
fmt.Printf("%+v\n", v)
}
Can anyone please explain to me how strconv can convert the row to a integer?
If you made a complete example on http://play.golang.org/ it'd be easier to give you a complete solution.
ParseInt() takes the string (you might have to do string(row[5])), the base (you probably meant 10) and the bitsize (that's where you should put 8).
It retuns an int (eia), it doesn't put it into the struct as it looks like you are trying.
Instead do if err == nil { v.Eia = eia }.

Resources