Changing value of field in a go struct - go

I'm making an http request in golang to an external api. It gives a general response of {"error":[]string, "result":changing interface{}}. depending on the function that is making the request, the Result field changes. Since I know the structure of the Result field for each function I run, I want to be able to change the value of Result before unmarshalling to json. I've tried to do this with the following code:
func GetAssets(output *Resp, resultType interface{}) error {
return publicRequest("/Assets", output, resultType)
}
func publicRequest(endPoint string, output *Resp, resultType interface{}) error {
url := Rest_url + Pub_rest_url + endPoint //"https://api.kraken.com/0/public/Assets in this case
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
output.Result = resultType
return json.NewDecoder(resp.Body).Decode(&output)
}
Here is how it's being ran in main
type Resp struct {
Error []string `json:"error"`
Result interface{} `json:"result"`
}
type AssetInfo struct {
Aclass string `json:"aclass"`
Altname string `json:"altname"`
Decimals int `json:"decimals"`
Display int `json:"display_decimals"`
}
func main() {
var result map[string]AssetInfo
jsonData := Resp{}
rest_api_client.GetAssets(&jsonData, result)
fmt.Println(jsonData)
}
The issue is that it doesn't unmarshal correctly. A map is created for each asset, but the data contained inside of each asset is also being stored inside of a map. I'm not sure if I explained this well, but here is the current response after unmarshalling to understand what I mean.
Here is the data type of Resp.Result: map[string]interface {}
{[] map[1INCH:map[aclass:currency altname:1INCH decimals:10 display_decimals:5] AAVE:map[aclass:currency altname:AAVE decimals:10 display_decimals:5] ACA:map[aclass:currency altname:ACA decimals:10 display_decimals:5] ADA:map[aclass:currency altname:ADA decimals:8 display_decimals:6]...}
The response type I'm looking for is map[string]AssetInfo. Hopefully it could be unmarshalled like this:
{[] map[1INCH:{currency 1INCH 10 5} AAVE:{currency AAVE 10 5} ACA:{currency ACA 10 5} ADA:{currency ADA 8 6} ADA.S:{currency ADA.S 8 6}...}
Any help? I'd rather keep the Resp struct as generic as possible and just change the value of the Result field (if this is even possible to do correctly) since I plan to have multiple functions that call different endpoints of the api, and they'll all have the same underlying response type of the Resp struct with different Result types

You can view a working example in the following repo:
https://github.com/alessiosavi/GoArbitrage/blob/e107af466852b1ed30c2413eb4401595f7412b4f/markets/kraken/kraken.go
Basically, I've defined the following structure:
type Tickers struct {
Error []interface{} `json:"error"`
Result map[string]Ticker `json:"result"`
}
type Ticker struct {
Aclass string `json:"aclass"`
Altname string `json:"altname"`
Decimals int `json:"decimals"`
DisplayDecimals int `json:"display_decimals"`
}
Than I execute the request in the following way:
const KRAKEN_TICKERS_URL string = `https://api.kraken.com/0/public/Assets`
type Kraken struct {
PairsNames []string `json:"pairs_name"`
Pairs map[string]datastructure.KrakenPair `json:"pairs"`
OrderBook map[string]datastructure.KrakenOrderBook `json:"orderbook"`
MakerFee float64 `json:"maker_fee"`
TakerFees float64 `json:"taker_fee"`
// FeePercent is delegated to save if the fee is in percent or in coin
FeePercent bool `json:"fee_percent"`
Tickers []string
}
// Init is delegated to initialize the maps for the kraken
func (k *Kraken) Init() {
k.Pairs = make(map[string]datastructure.KrakenPair)
k.OrderBook = make(map[string]datastructure.KrakenOrderBook)
k.SetFees()
}
// SetFees is delegated to initialize the fee type/amount for the given market
func (k *Kraken) SetFees() {
k.MakerFee = 0.16
k.TakerFees = 0.26
k.FeePercent = true
}
func (k *Kraken) GetTickers() error {
res := datastructure.Tickers{}
var err error
var request req.Request
var data []byte
var tickers []string
resp := request.SendRequest(KRAKEN_TICKERS_URL, "GET", nil, nil, false, 10*time.Second)
if resp.Error != nil {
zap.S().Debugw("Error during http request. Err: " + resp.Error.Error())
return resp.Error
}
if resp.StatusCode != 200 {
zap.S().Warnw("Received a non 200 status code: " + strconv.Itoa(resp.StatusCode))
return errors.New("NON_200_STATUS_CODE")
}
data = resp.Body
if err = json.Unmarshal(data, &res); err != nil {
zap.S().Warn("ERROR! :" + err.Error())
return err
}
zap.S().Infof("Data: %v", res.Result)
tickers = make([]string, len(res.Result))
i := 0
for key := range res.Result {
tickers[i] = res.Result[key].Altname
i++
}
k.Tickers = tickers
return nil
}

Related

Error while trying to fetch queryresult.KV object in JSON.Unmarshal

I am a little bit confused here and although I have searched a lot on this, something is clearly missing from my knowledge and I am asking your help.
I have created a Hyperledger Fabric Network and installed a chaincode in it. And I want to make a function that retrieves all the World State inputs about the Keys. I have done it already with the bytes.Buffer and it worked. But what I want to do is to do it with a struct.
So, I created the following struct that has only the key:
type WSKeys struct {
Key string `json: "key"`
Namespace string `json: "Namespace"`
}
And this is my code function:
func (s *SmartContract) getAllWsDataStruct(APIstub shim.ChaincodeStubInterface , args []string) sc.Response {
var keyArrayStr []WSKeys
resultsIterator, err := APIstub.GetQueryResult("{\"selector\":{\"_id\":{\"$ne\": null }} }")
if err != nil {
return shim.Error("Error occured when trying to fetch data: "+err.Error())
}
for resultsIterator.HasNext() {
// Get the next record
queryResponse, err := resultsIterator.Next()
if err != nil {
return shim.Error(err.Error())
}
fmt.Println(queryResponse)
var qry_key_json WSKeys
json.Unmarshal([]byte(queryResponse), &qry_key_json)
keyArray = append(keyArray, qry_key_json)
}
defer resultsIterator.Close()
all_bytes, _ := json.Marshal(keyArray)
fmt.Println(keyArray)
return shim.Success(all_bytes)
}
When executing the above I get the following error:
cannot convert queryResponse (type *queryresult.KV) to type []byte
I can get the results correctly if I, for example do this:
func (s *SmartContract) getAllWsDataStruct(APIstub shim.ChaincodeStubInterface , args []string) sc.Response {
var keyArray []string
resultsIterator, err := APIstub.GetQueryResult("{\"selector\":{\"_id\":{\"$ne\": null }} }")
if err != nil {
return shim.Error("Error occured when trying to fetch data: "+err.Error())
}
for resultsIterator.HasNext() {
// Get the next record
queryResponse, err := resultsIterator.Next()
if err != nil {
return shim.Error(err.Error())
}
fmt.Println(queryResponse)
keyArray = append(keyArray, queryResponse.Key)
}
defer resultsIterator.Close()
all_bytes, _ := json.Marshal(keyArray)
fmt.Println(keyArray)
return shim.Success(all_bytes)
}
But, why I get the above error when trying to add the queryResponse into a custom struct?
Do I need to add it to a struct that is only its type?
Please someone can explain what I am missing here?
The error statement is verbose enough to indicate, that your []byte conversion failed for the type queryResponse which, with a bit of lookup seems to be a struct type. In Go you cannot natively convert a struct instance to its constituent bytes without encoding using gob or other means.
Perhaps your intention was to use the Key record in the struct for un-marshalling
json.Unmarshal([]byte(queryResponse.Key), &qry_key_json)

How to get JSON response body from http.Get

I'm trying get body from response which is json and print this json or be able to put him to array. I find this post on stack: How to get JSON response from http.Get .
There is code:
var myClient = &http.Client{Timeout: 10 * time.Second}
func getJson(url string, target interface{}) error {
r, err := myClient.Get(url)
if err != nil {
return err
}
defer r.Body.Close()
return json.NewDecoder(r.Body).Decode(target)
}
but I don't undestand why there is "Decode(target)" and "target interface{}". What does it do? Why when I try just print json.NewDecoder(r.Body) there is nothing meaningful.
The function getJson in the question decodes the JSON response body for the specified URL to the value pointed to by target. The expression json.NewDecoder(r.Body).Decode(target) does the following:
create a decoder that reads from the response body
use the decoder to unmarshal the JSON to the value pointed to by target.
returns error or nil.
Here's an example use of the function. The program fetches and prints information about this answer.
func main() {
url := "https://api.stackexchange.com/2.2/answers/67655454?site=stackoverflow"
// Answers is a type the represents the JSON for a SO answer.
var response Answers
// Pass the address of the response to getJson. The getJson passes
// that address on to the JSON decoder.
err := getJson(url, &response)
// Check for errors. Adjust error handling to match the needs of your
// application.
if err != nil {
log.Fatal(err)
}
// Print the answer.
for _, item := range response.Items {
fmt.Printf("%#v", item)
}
}
type Owner struct {
Reputation int `json:"reputation"`
UserID int `json:"user_id"`
UserType string `json:"user_type"`
AcceptRate int `json:"accept_rate"`
ProfileImage string `json:"profile_image"`
DisplayName string `json:"display_name"`
Link string `json:"link"`
}
type Item struct {
Owner *Owner `json:"owner"`
IsAccepted bool `json:"is_accepted"`
Score int `json:"score"`
LastActivityDate int `json:"last_activity_date"`
LastEditDate int `json:"last_edit_date"`
CreationDate int `json:"creation_date"`
AnswerID int `json:"answer_id"`
QuestionID int `json:"question_id"`
ContentLicense string `json:"content_license"`
}
type Answers struct {
Items []*Item `json:"items"`
HasMore bool `json:"has_more"`
QuotaMax int `json:"quota_max"`
QuotaRemaining int `json:"quota_remaining"`
}
Link to complete code including code from question.
Here is an example:
package main
import (
"encoding/json"
"net/http"
)
func main() {
r, e := http.Get("https://github.com/manifest.json")
if e != nil {
panic(e)
}
defer r.Body.Close()
var s struct { Name string }
json.NewDecoder(r.Body).Decode(&s)
println(s.Name == "GitHub")
}
https://golang.org/pkg/encoding/json#Decoder.Decode

Managing multiple return types in Golang

I'm a bit new to Go and I'm having trouble converting a response object from an API call into different structures based on the request type
Essentially, I have one func which sends out a request
func (fpc *FPClient) request(path string, method string, params interface{}, Token string, response interface{}) *dto.AppError {
client := &http.Client{
Timeout: time.Second * 15,
}
requestBody, err := json.Marshal(params)
if err != nil {
//
}
req, _ := http.NewRequest(method, path, bytes.NewBuffer(requestBody))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Cookie", fmt.Sprintf("cookie=%s;", Token))
req.SetBasicAuth(fpc.username, fpc.password)
resp, err := client.Do(req)
if err != nil {
//
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
//
}
if FPErr := fpc.processErrors(resp, body); FPerr != nil {
return FPErr
}
responseData := FPApiSuccess{Head: response, Body: response}
if err := json.Unmarshal(body, &responseData); err != nil {
//
}
fmt.Printf("\n\n client Response : \n %+v \n", responseData.Body)
return nil
}
The struct for FPApiSuccess is:
type FPApiSuccess struct {
Body interface{} `json:"body"`
Head interface{} `json:"head"`
}
Right now, there are 2 calling functions and the API response expected is a bit different for both
Both API responses have the below structure
{
"head": {},
"body": {}
}
However, the nested details in each key is different based on the API used.
I want to capture the head and body keys in a struct argument I give and send it back to the calling function. The response argument in the request function is a different struct type based on the calling function.
I'm unable to get this to work - I'm only getting back a blank struct from the request function. This is the fmt.PrintF log
client Response :
&{Body:{BOrderID:0 CC: Exch: EOID: Type: Local:0 Message: } Head:{ResponseCode: Status: StatusDescription:}}
This is an empty struct - ideally, it should be populated with the values retrieved from the API.
For reference, heres the struct passed as an argument as response in the request function:
type FPApiResponse struct {
Body FPBodyResponse `json:"body"`
Head FPHeadResponse `json:"head"`
}
type FPHeadResponse struct {
ResponseCode string `json:"responseCode"`
Status string `json:"status"`
StatusDescription string `json:"statusDescription"`
}
type FPBodyResponse struct {
BOrderID int `json:"BOrderID"`
CC string `json:"CC"`
Exch string `json:"Exch"`
EOID string `json:"EOID"`
Type string `json:"Type"`
Local int `json:"Local"`
Message string `json:"Message"`
}
Update
So I did this; instead of
responseData := FPApiSuccess{Head: response, Body: response}
I did this
responseData := fivePaisaApiSuccess{}
So now, I get the below in console
Client Response :
{Body:map[BOrderID:0 CC:52715111 Type:D Local:0 Message:Invalid Session ] Head:map[responseCode:5POrdReq status:0 statusDescription:Success]}
So essentially, this works, but the calling function doesn't seem to get the proper response:
Here's the calling function
func (fpc *FPClient) PlaceOrder(orderParams dto.OrderBodyParams, variety string, Token string) (string, *dto.AppError) {
var result FPApiResponse
headParams := dto.FFPOrderHeadParams{
//
}
FPOrderParams := dto.FPOrderParams{
//
}
FPErr := fpc.request(FPURL+FPPlaceOrder, http.MethodPost, FPOrderParams, brokerAccessToken, &result)
if FPErr != nil {
return "", FPErr
}
fmt.Printf("\n\n Client result : \n %+v \n", result)
if result.Head.Status != "0" {
//
}
if result.Body.Status != 0 {
//
}
return strconv.Itoa(result.Body.Broker), nil
}
The result value is blank:
{Body:{BOId:0 CC: Exch: Type: Local:0 Message: Status:0} Head:{ResponseCode: Status: StatusDescription:}}
I don't understand, this pointer is getting populated in the request function
Here's the struct I'm passing to the request:
type FPApiResponse struct {
Body FPBodyResponse `json:"body"`
Head FPHeadResponse `json:"head"`
}

Golang RPC encode custom function

I am trying to use github.com/dullgiulio/pingo and send my custom struct
type LuaPlugin struct {
Name string
List []PluginTable
}
type PluginTable struct {
Name string
F lua.LGFunction
}
// LoadPlugins walks over the plugin directory loading all exported plugins
func LoadPlugins() {
//
p := pingo.NewPlugin("tcp", "plugins/test")
// Actually start the plugin
p.Start()
// Remember to stop the plugin when done using it
defer p.Stop()
gob.Register(&LuaPlugin{})
gob.Register(&PluginTable{})
var resp *LuaPlugin
// Call a function from the object we created previously
if err := p.Call("MyPlugin.SayHello", "Go developer", &resp); err != nil {
log.Print(err)
} else {
log.Print(resp.List[0])
}
}
However I am always getting nil for the F field of ym struct. This is what I am sending on the client
// Create an object to be exported
type MyPlugin struct{}
// Exported method, with a RPC signature
func (p *MyPlugin) SayHello(name string, msg *util.LuaPlugin) error {
//
//
*msg = util.LuaPlugin{
Name: "test",
List: []util.PluginTable{
{
Name: "hey",
F: func(L *lua.LState) int {
log.Println(L.ToString(2))
return 0
},
},
},
}
return nil
}
Is it not possible to send custom data types over RPC?
I'm not familiar with the library however, you could try converting the struct to a byte slice before transport. Late reply, might help others....
Simple conversion: returns a struct as bytes
func StructToBytes(s interface{}) (converted []byte, err error) {
var buff bytes.Buffer
encoder := gob.NewEncoder(&buff)
if err = encoder.Encode(s); err != nil {
return
}
converted = buff.Bytes()
return
}
Decoder: returns a wrapper to decode the bytes into
func Decoder(rawBytes []byte) (decoder *gob.Decoder) {
reader := bytes.NewReader(rawBytes)
decoder = gob.NewDecoder(reader)
return
}
Example:
type MyStruct struct {
Name string
}
toEncode := MyStruct{"John Doe"}
// convert the struct to bytes
structBytes, err := StructToBytes(toEncode)
if err != nil {
panic(err)
}
//-----------
// send over RPC and decode on other side
//-----------
// create a new struct to decode into
var DecodeInto MyStruct
// pass the bytes to the decoder
decoder := Decoder(structBytes)
// Decode into the struct
decoder.Decode(&DecodeInto)
fmt.Println(DecodeInto.Name) // John Doe
Due to the use of the gob package you can swap types and typecast to a certain extent.
For more see: https://golang.org/pkg/encoding/gob/

Iterate Over String Fields in Struct

I'm looking to iterate over the string fields of a struct so I can do some clean-up/validation (with strings.TrimSpace, strings.Trim, etc).
Right now I have a messy switch-case that's not really scalable, and as this isn't in a hot spot of my application (a web form) it seems leveraging reflect is a good choice here.
I'm at a bit of a roadblock for how to implement this however, and the reflect docs are a little confusing to me (I've been digging through some other validation packages, but they're way too heavyweight + I'm using gorilla/schema for the unmarshalling part already):
Iterate over the struct
For each field of type string, apply whatever I need to from the strings package i.e. field = strings.TrimSpace(field)
If there exists a field.Tag.Get("max"), we'll use that value (strconv.Atoi, then unicode.RuneCountInString)
Provide an error slice that's also compatible with the error interface type
type FormError []string
type Listing struct {
Title string `max:"50"`
Location string `max:"100"`
Description string `max:"10000"`
ExpiryDate time.Time
RenderedDesc template.HTML
Contact string `max:"255"`
}
// Iterate over our struct, fix whitespace/formatting where possible
// and return errors encountered
func (l *Listing) Validate() error {
typ := l.Elem().Type()
var invalid FormError
for i = 0; i < typ.NumField(); i++ {
// Iterate over fields
// For StructFields of type string, field = strings.TrimSpace(field)
// if field.Tag.Get("max") != "" {
// check max length/convert to int/utf8.RuneCountInString
if max length exceeded, invalid = append(invalid, "errormsg")
}
if len(invalid) > 0 {
return invalid
}
return nil
}
func (f FormError) Error() string {
var fullError string
for _, v := range f {
fullError =+ v + "\n"
}
return "Errors were encountered during form processing: " + fullError
}
Thanks in advance.
What you want is primarily the methods on reflect.Value called NumFields() int and Field(int). The only thing you're really missing is the string check and SetString method.
package main
import "fmt"
import "reflect"
import "strings"
type MyStruct struct {
A,B,C string
I int
D string
J int
}
func main() {
ms := MyStruct{"Green ", " Eggs", " and ", 2, " Ham ", 15}
// Print it out now so we can see the difference
fmt.Printf("%s%s%s%d%s%d\n", ms.A, ms.B, ms.C, ms.I, ms.D, ms.J)
// We need a pointer so that we can set the value via reflection
msValuePtr := reflect.ValueOf(&ms)
msValue := msValuePtr.Elem()
for i := 0; i < msValue.NumField(); i++ {
field := msValue.Field(i)
// Ignore fields that don't have the same type as a string
if field.Type() != reflect.TypeOf("") {
continue
}
str := field.Interface().(string)
str = strings.TrimSpace(str)
field.SetString(str)
}
fmt.Printf("%s%s%s%d%s%d\n", ms.A, ms.B, ms.C, ms.I, ms.D, ms.J)
}
(Playground link)
There are two caveats here:
You need a pointer to what you're going to change. If you have a value, you'll need to return the modified result.
Attempts to modify unexported fields generally will cause reflect to panic. If you plan on modifying unexported fields, make sure to do this trick inside the package.
This code is rather flexible, you can use switch statements or type switches (on the value returned by field.Interface()) if you need differing behavior depending on the type.
Edit: As for the tag behavior, you seem to already have that figured out. Once you have field and have checked that it's a string, you can just use field.Tag.Get("max") and parse it from there.
Edit2: I made a small error on the tag. Tags are part of the reflect.Type of a struct, so to get them you can use (this is a bit long-winded) msValue.Type().Field(i).Tag.Get("max")
(Playground version of the code you posted in the comments with a working Tag get).
I got beat to the punch, but since I went to the work, here's a solution:
type FormError []*string
type Listing struct {
Title string `max:"50"`
Location string `max:"100"`
Description string `max:"10000"`
ExpiryDate time.Time
RenderedDesc template.HTML
Contact string `max:"255"`
}
// Iterate over our struct, fix whitespace/formatting where possible
// and return errors encountered
func (l *Listing) Validate() error {
listingType := reflect.TypeOf(*l)
listingValue := reflect.ValueOf(l)
listingElem := listingValue.Elem()
var invalid FormError = []*string{}
// Iterate over fields
for i := 0; i < listingElem.NumField(); i++ {
fieldValue := listingElem.Field(i)
// For StructFields of type string, field = strings.TrimSpace(field)
if fieldValue.Type().Name() == "string" {
newFieldValue := strings.TrimSpace(fieldValue.Interface().(string))
fieldValue.SetString(newFieldValue)
fieldType := listingType.Field(i)
maxLengthStr := fieldType.Tag.Get("max")
if maxLengthStr != "" {
maxLength, err := strconv.Atoi(maxLengthStr)
if err != nil {
panic("Field 'max' must be an integer")
}
// check max length/convert to int/utf8.RuneCountInString
if utf8.RuneCountInString(newFieldValue) > maxLength {
// if max length exceeded, invalid = append(invalid, "errormsg")
invalidMessage := `"`+fieldType.Name+`" is too long (max allowed: `+maxLengthStr+`)`
invalid = append(invalid, &invalidMessage)
}
}
}
}
if len(invalid) > 0 {
return invalid
}
return nil
}
func (f FormError) Error() string {
var fullError string
for _, v := range f {
fullError = *v + "\n"
}
return "Errors were encountered during form processing: " + fullError
}
I see you asked about how to do the tags. Reflection has two components: a type and a value. The tag is associated with the type, so you have to get it separately than the field: listingType := reflect.TypeOf(*l). Then you can get the indexed field and the tag from that.
I don't know if it's a good way, but I use it like this.
https://play.golang.org/p/aQ_hG2BYmMD
You can send the address of a struct to this function.
Sorry for My English is not very good.
trimStruct(&someStruct)
func trimStruct(v interface{}) {
bytes, err := json.Marshal(v)
if err != nil {
fmt.Println("[trimStruct] Marshal Error :", err)
}
var mapSI map[string]interface{}
if err := json.Unmarshal(bytes, &mapSI); err != nil {
fmt.Println("[trimStruct] Unmarshal to byte Error :", err)
}
mapSI = trimMapStringInterface(mapSI).(map[string]interface{})
bytes2, err := json.Marshal(mapSI)
if err != nil {
fmt.Println("[trimStruct] Marshal Error :", err)
}
if err := json.Unmarshal(bytes2, v); err != nil {
fmt.Println("[trimStruct] Unmarshal to b Error :", err)
}
}
func trimMapStringInterface(data interface{}) interface{} {
if values, valid := data.([]interface{}); valid {
for i := range values {
data.([]interface{})[i] = trimMapStringInterface(values[i])
}
} else if values, valid := data.(map[string]interface{}); valid {
for k, v := range values {
data.(map[string]interface{})[k] = trimMapStringInterface(v)
}
} else if value, valid := data.(string); valid {
data = strings.TrimSpace(value)
}
return data
}

Resources