Could you help me to understand why I'm always getting the error unexpected end of JSON input while trying to unmarshal the following json to the LimitOrder struct?
P.S.: if I use map[string]json.RawMessage instead of LimitOrder struct I'm able to execute the unmarshal.
{
"response_data": {
"order": {
"order_id": 3,
"coin_pair": "BRLBTC",
"order_type": 1,
"status": 4,
"has_fills": true,
"quantity": "1.00000000",
"limit_price": "900.00000",
"executed_quantity": "1.00000000",
"executed_price_avg": "900.00000",
"fee": "0.00300000",
"created_timestamp": "1453835329",
"updated_timestamp": "1453835329",
"operations": [
{
"operation_id": 1,
"quantity": "1.00000000",
"price": "900.00000",
"fee_rate": "0.30",
"executed_timestamp": "1453835329"
}
]
}
},
"status_code": 100,
"server_unix_timestamp": "1453835329"
}
LimitOrder struct
type LimitOrder struct {
OrderId int `json:"order_id"`
CoinPair string `json:"coin_pair"`
OrderType int `json:"order_type"`
Status int `json:"status"`
HasFills bool `json:"has_fills"`
Quantity float64 `json:"quantity,string"`
LimitPrice float64 `json:"limit_price,string"`
ExecutedQuantity float64 `json:"executed_quantity,string"`
ExecutedPriceAvg float64 `json:"executed_price_avg,string"`
Fee float64 `json:"fee,string"`
Operations []*Operation `json:"operations"`
CreatedTimestamp string `json:"created_timestamp"`
UpdatedTimestamp string `json:"updated_timestamp"`
}
and this is how I'm trying to unmarshal it
func (limitOrder *LimitOrder) UnmarshalJSON(buf []byte) error {
tmp := make(map[string]json.RawMessage)
if err := json.Unmarshal(buf, &tmp); err != nil {
return err
}
tmp2 := make(map[string]json.RawMessage)
if err := json.Unmarshal(tmp["response_data"], &tmp2); err != nil {
return err
}
if err := json.Unmarshal(tmp2["order"], limitOrder); err != nil {
return err
}
return nil
}
I found some help on golang-nuts group. You can check the answer here https://groups.google.com/forum/#!topic/golang-nuts/SZXBcXgUoo0. To make a long story short, the problem was with my structures, I've built structures only for a small piece of the whole json, so I fixed it building structures for the whole json response.
Related
I'm trying to seed my Postgres database as functionally. In my case, SeedSchema() function can take any type struct. So I define a interface and create functions to my structs which will seed. I tried with generics and without.
When I unmarshall any json array from file as byte array, json.Unmarshall method change my tempMember member of struct. Exp, models.Term to map[string]interface{}. I've used unmarshall before this function and I've not seen like this situation.
Here is my SeedSchema() function:
func (db *Database) SeedSchema(models ...globals.Seeder[any]) error {
var (
subjects []globals.Seeder[any]
fileByte []byte
err error
// tempMember any
)
if len(models) == 0 {
subjects = seederModelList
} else {
subjects = models
}
for _, model := range subjects {
fileName, tempMember := model.Seed()
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //1
if fileByte, err = os.ReadFile("db/seeds/" + fileName); err != nil {
fmt.Println(err)
return err
}
if err = json.Unmarshal(fileByte, &tempMember); err != nil {
fmt.Println(err)
return err
}
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //2
}
return nil
}
First print returns []models.AirportCodes and the second []interface {}.
Here is my interface and model:
func (AirportCodes) Seed() (string, any) {
return "airport_codes.json", []AirportCodes{}
}
type Seeder[T any] interface {
Seed() (string, T)
// Seed(*gorm.DB) error
TableName() string
}
seederModelList = []globals.Seeder[any]{
m.AirportCodes{},
m.Term{},
}
After a few weeks, I have looking for solve this problem and look unmarshaler interfaces and examples. Then Like what icza said, I started to look over the my code that convention between types and I solved like this. If you guys have better answer than mine, please add answer.
Data:
[
{
"id":1,
"name":"Term 1",
"content": [
"a1",
"a2",
"a3"
]
}
]
Result:
[{ID:1 Name:Term 1 Content:[a1 a2 a3]}]
UnmarshalJSON Function:
func (term *Term) UnmarshalJSON(data []byte) error {
tempMap := map[string]interface{}{}
if err := json.Unmarshal(data, &tempMap); err != nil {
return err
}
*term = Term{
Name: tempMap["name"].(string),
}
if tempMap["content"] != nil {
for _, v := range tempMap["content"].([]interface{}) {
(*term).Content = append((term).Content, v.(string))
}
}
return nil
}
Thank you for comments.
I would like to bind a json array of objects like this one :
[
{
"id": "someid"
},
{
"id": "anotherid"
}
]
Here my model
type DeleteByID struct {
ID string `json:"id" binding:"required"`
}
I use gin to handle the object
var stock []DeleteByID
if err := ctx.ShouldBindJSON(&stock); err != nil {
return err
}
The problem is that it does not bind/check my object.
You can achieve this by using json.Unmarshal() like this:
var stock []DeleteByID
body, err := ioutil.ReadAll(c.Request.Body)
if err != nil {
c.AbortWithError(400, err)
return
}
err = json.Unmarshal(body, &stock)
if err != nil {
c.AbortWithError(400, err)
return
}
c.String(200, fmt.Sprintf("%#v", stock))
The alternative is to pass the array as a nested field. When marked with "dive", gin will bind and validate. These ones will cause an error:
{
"Deletions": [ {
"id": 13
},
{
}
]
}
This is acceptable input:
{
"Deletions": [
{
"id": "someid"
},
{
"id": "anotherid"
}
]
}
Here my model
type DeleteByID struct {
ID string `json:"id" binding:"required"`
}
type DeletePayload struct {
Deletions []DeleteByID `binding:"dive"`
}
The dive keyword will ensure that the JSON array is validated as it becomes a slice, map or array.
var stock DeletePayload
if err := ctx.ShouldBindJSON(&stock); err != nil {
return err
}
See this issue for some more details: https://github.com/gin-gonic/gin/issues/3238
Response:
{
"meta": {
"query_time": 0.039130201,
"pagination": {
"offset": 1345,
"limit": 5000,
"total": 1345
},
Structs:
type InLicense struct {
Total int16 json:"total,omitempty"
}
type OutLicense struct {
Pagination []InLicense json:"pagination,omitempty"
}
type MetaLicense struct {
Meta []OutLicense json:"meta,omitempty"
}
Code snip inside function:
req, err := http.NewRequest("GET", , nil)
if err != nil {
//handle error
}
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
log.Println("Error: ", err)
}
defer resp.Body.Close()
val := &MetaLicense{}
err = json.NewDecoder(resp.Body).Decode(&val)
if err != nil {
log.Fatal(err)
}
for _, s := range val.Meta {
for _, a := range s.Pagination {
fmt.Println(a.Total)
}
}
}
After I run this code I get the following error:
json: cannot unmarshal object into Go struct field MetaLicense.meta of type []OutLicense
Which type would []OutLicense need to be in order for this to be unmarshaled correctly? I cant print it another way, but it prints with the {} and Strings.Trim will not work.
You should use just a simple field declaration with actual type, not a [] of the type as it is done below:
type InLicense struct {
Total int16 json:"total,omitempty"
}
type OutLicense struct {
Pagination InLicense json:"pagination,omitempty"
}
type MetaLicense struct {
Meta OutLicense json:"meta,omitempty"
}
I simplified the parsing a bit and just used the json.Unmarshal() function instead.
raw := "{\n \"meta\": {\n \"query_time\": 0.039130201,\n \"pagination\": {\n \"offset\": 1345,\n \"limit\": 5000,\n \"total\": 1345\n }\n }\n}"
parsed := &MetaLicense{}
err := json.Unmarshal([]byte(raw), parsed)
if err != nil {
log.Fatal(err)
}
fmt.Println(parsed.Meta.Pagination.Total) // Prints: 1345
Here's the types I used
type InLicense struct {
Total int16 `json:"total,omitempty"`
}
type OutLicense struct {
Pagination InLicense `json:"pagination,omitempty"`
}
type MetaLicense struct {
Meta OutLicense `json:"meta,omitempty"`
}
As written your provided JSON has a extra , which makes your json unparsable (assuming you add the missing }'s too.
There are no lists in your JSON. Lists are denoted with the [] symbols. For your types to work it, your JSON would have to look like this:
{
"meta": [{
"query_time": 0.039130201,
"pagination": [{
"offset": 1345,
"limit": 5000,
"total": 1345
}]
}]
}
I'm trying to sort the result in golang chaincode, but the result is random, below is my chaincode sample:
package main
import (
"bytes"
"encoding/json"
"fmt"
"time"
"github.com/hyperledger/fabric/core/chaincode/shim"
"github.com/hyperledger/fabric/protos/peer"
)
type itemStruct struct {
ID string `json:"id"`
Status string `json:"status"`
CreatedAt time.Time `json:"created_at"`
}
func createItem(stub shim.ChaincodeStubInterface, args []string) peer.Response {
if len(args) != 3 {
return shim.Error(fmt.Sprintf("Expecting %v arguments {id, status, created_at}, but got %v", 3, len(args)))
}
itemID := args[0]
if len(itemID) == 0 {
return shim.Error("id field is required")
}
status := args[1]
if len(status) == 0 {
return shim.Error("status field is required")
}
createdAt, err := time.Parse(time.RFC3339, args[2])
if err != nil {
return shim.Error("created_at is not a valid datetime string")
}
item := itemStruct{
ID: itemID,
CreatedAt: createdAt,
}
itemAsJSONBytes, err := json.Marshal(item)
if err != nil {
return shim.Error(err.Error())
}
return shim.Success(itemAsJSONBytes)
}
func getPendingItems(stub shim.ChaincodeStubInterface, args []string) peer.Response {
var bookmark string
if len(args) > 0 && len(args[0]) > 0 {
bookmark = args[0]
}
queryString := `{
"selector": {
"status": "pending"
},
"sort": [
{"created_at": "desc"}
]
}`
result, pagination, err := queryWithPagination(stub, queryString, 20, bookmark)
if err != nil {
return shim.Error(err.Error())
}
return shim.Success(constructResponse(result, pagination).Bytes())
}
func queryWithPagination(stub shim.ChaincodeStubInterface, queryString string, pageSize int32, bookmark string) (map[string]string, string, error) {
var pagination string
iterator, meta, err := stub.GetQueryResultWithPagination(queryString, pageSize, bookmark)
if err != nil {
return nil, pagination, err
}
defer iterator.Close()
result, err := iterateResult(iterator)
if err != nil {
return nil, pagination, err
}
pagination = fmt.Sprintf(`{"count": %v, "next_page_token": "%v"}`, meta.FetchedRecordsCount, meta.Bookmark)
return result, pagination, nil
}
func constructResponse(items map[string]string, pagination string) *bytes.Buffer {
// buffer is a JSON array containing QueryResults
var buffer bytes.Buffer
if len(pagination) > 0 {
buffer.WriteString(`{"data":`)
}
buffer.WriteString(`[`)
bArrayMemberAlreadyWritten := false
for _, val := range items {
// Add a comma before array members, suppress it for the first array member
if bArrayMemberAlreadyWritten == true {
buffer.WriteString(",")
}
buffer.WriteString(val)
bArrayMemberAlreadyWritten = true
}
buffer.WriteString("]")
if len(pagination) > 0 {
buffer.WriteString(`,"pagination":`)
buffer.WriteString(pagination)
buffer.WriteString("}")
}
return &buffer
}
func iterateResult(iterator shim.StateQueryIteratorInterface) (map[string]string, error) {
result := map[string]string{}
for iterator.HasNext() {
queryResponse, err := iterator.Next()
if err != nil {
return nil, err
}
result[queryResponse.Key] = string(queryResponse.Value)
}
return result, nil
}
// SmartContract : Smart contract struct
type SmartContract struct {
}
// Init : This method is called when chaincode is initialized or updated.
func (s *SmartContract) Init(stub shim.ChaincodeStubInterface) peer.Response {
return shim.Success(nil)
}
// Invoke : This method is called when any transaction or query fired
func (s *SmartContract) Invoke(stub shim.ChaincodeStubInterface) peer.Response {
// Retrieve the requested Smart Contract function and arguments
function, args := stub.GetFunctionAndParameters()
// Route to the appropriate handler function to interact with the ledger appropriately
// Transactions
if function == "createItem" {
return createItem(stub, args)
}
// Queries
if function == "getItems" {
return getItems(stub, args)
}
return shim.Error("Invalid function")
}
func main() {
// Create a new Smart Contract
err := shim.Start(new(SmartContract))
if err != nil {
fmt.Printf("Error creating new Smart Contract: %s", err)
}
}
It creates and asset which can have different status based on what's passed, and I have defined one query function which fetches only pending items.
I have applied sort, but the result is still random, can anyone help me here and guide me where I'm going wrong in this?
the sort field has to be present in the selector !
something like:
queryString := `{
"selector": {
"created_at": "$gt": null
"status": "pending"
},
"sort": [
{"created_at": "desc"}
]
}`
or (range)
queryString := `{
"selector": {
"created_at": {
"$gt": "2015-01-01T00:00:00Z",
"$lt": "2019-01-01T00:00:00Z"
},
"status": "pending"
},
"sort": [
{"created_at": "desc"}
]
}`
Per the docs - to use sorting, ensure that:
At least one of the sort fields is included in the selector.
There is an index already defined, with all the sort fields in the same
order.
Each object in the sort array has a single key.
If an object in the sort array does not have a single key, the resulting sort order is implementation specific and might change.
Also, as an fyi Find does not support multiple fields with different sort orders, so the directions must be either all ascending or all descending.
See also CouchDB sort doesn't work
I'm calling a remote API and getting a JSON response back. I'm trying to convert the *http.Response into a struct that I defined. Everything i've tried so far has resulted in an empty struct. Here is my attempt with json.Unmarshal
type Summary struct {
Created string `json:"created"`
High float64 `json:"high"`
Low float64 `json:"low"`
}
func getSummary() {
url := "http://myurl"
resp, err := http.Get(url)
if err != nil {
log.Fatalln(err)
}
body, err2 := ioutil.ReadAll(resp.Body)
if err2 != nil {
panic(err.Error())
}
log.Printf("body = %v", string(body))
//outputs: {"success":true,"message":"","result":["High":0.43600000,"Low":0.43003737],"Created":"2017-06-25T03:06:46.83"}]}
var summary = new(Summary)
err3 := json.Unmarshal(body, &summary)
if err3 != nil {
fmt.Println("whoops:", err3)
//outputs: whoops: <nil>
}
log.Printf("s = %v", summary)
//outputs: s = &{{0 0 0 0 0 0 0 0 0 0}}
}
What am I doing wrong? The JSON tags in my struct match the json keys from the response exactly...
edit: here is the JSON returned from the API
{
"success": true,
"message": "''",
"result": [
{
"High": 0.0135,
"Low": 0.012,
"Created": "2014-02-13T00:00:00"
}
]
}
edit
i changed the struct to this but still not working
type Summary struct {
Result struct {
Created string `json:"created"`
High float64 `json:"high"`
Low float64 `json:"low"`
}
}
Change your structure like this
type Summary struct {
Sucess bool `json:"success"`
Message string `json:"message"`
Result []Result `json:"result"`
}
type Result struct {
Created string `json:"Created"`
High float64 `json:"High"`
Low float64 `json:"Low"`
}
Try this link
Its because you're trying to unmarshal an array into struct,
Use array instead of the Result struct
type Summary struct {
Result []struct {
Created string `json:"created"`
High float64 `json:"high"`
Low float64 `json:"low"`
}
}
Use this weblink to convert your JSON objects to Go Struct >> https://mholt.github.io/json-to-go/