I have data returned from Elasticsearch, using "github.com/olivere/elastic". That sort of works, when i add it to my struct and string it, like so,
data := Api {
Total: myTotal,
Data: string(result),
}
c.JSON(http.StatusOK, totalData)
the api is a struct like so,
type Api struct {
Total interface{}
Data interface{}
}
This returns data ok, from 1 to any number of results on request. How the results loaded into the data interface are not escaped or something, e.g.
"Data":"{\"CID\":\"XXXXXXXXXX\",\"Link\":\"XXXXXXXXX\",
So I have tried to unmarshal the data before adding it to the api struct.
var p DataApi
err := json.Unmarshal(result, &p)
if err != nil {
panic(err)
}
totalData := Api {
Total: myTotal,
Data: p,
}
c.JSON(http.StatusOK, totalData)
This sort of works fine, returns the data in the correct way, but only when loading one result. When 2 or more results are requested, I get this error from the unmarshal panic
invalid character '{' after top-level value
I have tried and google all over but can not find a solution to this? I am not sure what I am doing wrong? The DataApi is a nested set of structs, I was not sure if there was anything I should be being because of that?
This is being run within the Gin framework.
Thanks.
EDIT
So when I use fmt.Println on the string(result) I can print any number of results on the screen. How can I add this to the API struct and then I need the struct converted into JSON data. Is there some way of appending this string data on the JSON converted API struct?
Try to unmarshal multiple results into a slice:
var q []Api
err = json.Unmarshal(result, &q)
See on playground https://play.golang.org/p/D_bVAd4jBlI
package main
import (
"encoding/json"
"fmt"
)
type Api struct {
Total interface{}
Data interface{}
}
func main() {
data := Api{
Total: 1,
Data: "2",
}
result, err := json.Marshal(data)
if err != nil {
panic(err)
}
fmt.Printf("single data: %s\n", result)
var p Api
err = json.Unmarshal(result, &p)
if err != nil {
panic(err)
}
dataSlice := []Api{data}
result, err = json.Marshal(dataSlice)
if err != nil {
panic(err)
}
fmt.Printf("slice of data: %s\n", result)
var q []Api
err = json.Unmarshal(result, &q)
if err != nil {
panic(err)
}
}
Use json.RawMessage to store arbitrary JSON documents:
var p json.RawMessage
err := json.Unmarshal(result, &p)
if err != nil {
panic(err)
}
totalData := Api {
Total: myTotal,
Data: p,
}
c.JSON(http.StatusOK, totalData)
I have a working solution to my problem, I just use the Hits Hits From the data returned by elastic search, I would like just the source data but i think it does what I need it to do... for now.
Thanks.
Related
I have an API built in go fiber.
I'm tryng to parse request body data as key value pairs dynamically.
As we know, fiber has context.Body() and context.Bodyparser() methods to do this but I couldn't find any proper example to do this dynamically with these methods.
e.g:
func handler(c *fiber.Ctx) error {
req := c.Body()
fmt.Println(string(req))
return nil
}
output:
key=value&key2=value2&key3&value3
What I'm looking for is a dynamic json like:
{
key:"value",
key2:"value2",
key3:"value3",
}
The content's mime-type is application/x-www-form-urlencoded not application/json. To parse that you can use net/url.ParseQuery. The result of that is a map[string][]string which you can then easily convert to a map[string]string and then marshal that with the encoding/json package to get your desired JSON output.
func handler(c *fiber.Ctx) error {
values, err := url.ParseQuery(strings(c.Body()))
if err != nil {
return err
}
obj := map[string]string{}
for k, v := range values {
if len(v) > 0 {
obj[k] = v[0]
}
}
out, err := json.Marshal(obj)
if err != nil {
return err
}
fmt.Println(string(out))
return nil
}
How can I create a nested JSON response from a response I received from an API?
For example, I got a resp (*http.Response) from an API which I expect to be a list of objects ([{},{},{},...])
I want to create a response like so
{
total: 1234,
addresses: [{},{},{}]
}
I'm not quite sure how to deal with this. I got close as my code below returns a similar structure but my addresses section returns an escaped string like so
"[{\"access\":\"INTERNAL\",\"address\":\"1P9SnpTait5bS\"}]"
func (h *Handler) getAddresses(w http.ResponseWriter, r *http.Request) {
type Message struct {
Total int `json:total`
Addresses string `json:addresses`
}
resp, _ := h.Search(address, page, pageOffset) // *http.Response
body, _ := ioutil.ReadAll(resp.Body)
res := Message{
Total: total,
Addresses: string(body),
}
m, _ := json.Marshal(res)
w.Write(m)
}
You can use json.RawMessage if all you need is to pass the json along.
func (h *Handler) getAddresses(w http.ResponseWriter, r *http.Request) {
type Message struct {
Total int `json:"total"`
Addresses json.RawMessage `json:"addresses"`
}
resp, err := h.Search(address, page, pageOffset) // *http.Response
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
res := Message{
Total: total,
Addresses: json.RawMessage(body),
}
if err := json.NewEncoder(w).Encode(res); err != nil {
log.Println("failed to respond with json:", err)
}
}
According to your desired response format, Addresses should be defined as type []map[string]interface rather than of type string.
type Message struct {
Total int `json:total`
Addresses []map[string]interface{} `json:addresses`
}
Then you would json unmarshal the response bytes into a temporary variable of type []map[string]interface{} before assigning the temporary variable to Message.Address, and finally json marshal Message as you were doing already.
Your question currently doesn't demonstrate where total is coming from. I assume you know how to deal with total.
I have a file, 'test.txt', containing the following data. This file was created from the same structures from the code below using marshaling.
{"VLETXGJM":{"attrib1":"test1","attrib2":"test2"}}
I am trying to read it back from the file and unmarshal it into a map using the same structures. I can successfully read the data from the file. I receive no errors when I try to unmarshal it into the map. However, my map is empty.
The mutex is used to protect the map since my real implementation (this is an extracted test) needs to use a protected map for concurrency. I have tried this same code removing the sync library and received the same negative result.
The test code:
package main
import (
"encoding/json"
"fmt"
"sync"
"os"
)
type TestObject struct {
Attrib1 string `json:"attrib1"`
Attrib2 string `json:"attrib2"`
}
type TestMap map[string]TestObject
type TestList struct {
sync.RWMutex
list TestMap
}
func main() {
tl := TestList{ list: make(TestMap) }
// Read the list back out of the file
fi, err := os.Open("test.txt")
if os.IsNotExist(err) {
fmt.Println("data file does not exist")
panic(nil)
}
if err != nil {
panic(err)
}
defer func() {
if err := fi.Close(); err != nil {
panic(err)
}
}()
data := make([]byte, 1024 * 1024)
count, err := fi.Read(data)
if err != nil {
panic(err)
}
fmt.Printf("read from file: \"%s\"\n",data[:count])
tl.Lock()
err = json.Unmarshal(data[:count], &tl)
if err != nil {
panic(err)
}
tl.Unlock()
// List it out
tl.Lock()
if len(tl.list) == 0 {
fmt.Println("Empty list")
} else {
for key, _ := range tl.list {
fmt.Printf("%s: %s\n", tl.list[key].Attrib1, tl.list[key].Attrib2)
}
}
tl.Unlock()
}
The output of the run is:
read from file: "{"VLETXGJM":{"attrib1":"test1","attrib2":"test2"}}"
Empty list
Thank you for your help. I have searched for similar issues and not yet found an exact duplicate of this scenario.
I think you want to unmarshal into tl.list instead of tl:
err = json.Unmarshal(data[:count], &tl.list)
tl has no exported fields, so Unmarshal into tl won't do anything. tl.list (i.e., type TestMap) matches your data.
I am attempting to map a response from an API to a struct using Golang.
The JSON that comes back when I view the link in the browser is below:
{
"GBP": 657.54
}
And I just want to map it to a simple struct like so:
type Price struct {
Name string
Value float64
}
Here is my current code.
func FetchCoinPrice(fsym string, tsyms string) Price {
url := fmt.Sprintf("https://min-api.cryptocompare.com/data/price?fsym=" + fsym + "&tsyms=" + tsyms)
fmt.Println("Requesting data from " + url)
price := Price{}
// getting the data using http
request, err := http.Get(url)
if err != nil {
log.Fatal(err.Error())
}
// Read the response body using ioutil
body, err := ioutil.ReadAll(request.Body)
if err != nil {
log.Fatal(err.Error())
}
defer request.Body.Close()
if request.StatusCode == http.StatusOK {
json.Unmarshal(body, &price)
}
return price
}
At the moment all I receive is an empty struct, I know the link is bringing back the correct data and I've tested it in my browser.
The mapping doesn't work that way. Instead, you should use a map:
data := []byte(`{
"GBP": 657.54
}`)
priceMap := map[string]float64{}
err := json.Unmarshal(data, &priceMap)
// Check your errors!
if err != nil {
log.Fatal(err.Error())
}
fmt.Println(priceMap)
This will print:
map[GBP:657.54]
You can then iterate over the map and build the struct you mentioned above, or just access the entry directly if you know the currency. eg: priceMap["GBP"]
You should really check your errors, especially if you're not getting the output you expect from Unmarshal.
The problem is that the unmarshaler cannot guess that keys in a JSON object should correspond to some value in a struct. Golang JSON mapping simply doesn't work that way.
However, you can make your "Price" type implement json.Unmarshaler to deserialize a message into a map of floats (map[string]float64) then ensure the shape is right and populate the struct accordingly:
func (p *Price) UnmarshalJSON(bs []byte) error {
kvs := map[string]float64{}
err := json.Unmarshal(bs, &kvs)
if err != nil {
return err
}
if len(kvs) != 1 {
return fmt.Errorf("expected 1 key, got %d", len(kvs))
}
for name, value := range kvs {
p.Name, p.Value = name, value
}
return nil
}
func main() {
jsonstr := `[{"GBP":657.54},{"USD":123.45}]`
ps := []Price{}
err := json.Unmarshal([]byte(jsonstr), &ps)
if err != nil {
panic(err)
}
// ps=[]main.Price{
// main.Price{Name:"GBP", Value:657.54},
// main.Price{Name:"USD", Value:123.45}
// }
}
Using gopkg.in/mgo.v2/bson, I wonder how to marshal an interface{} value into a value of type bson.Raw.
The documentation for bson.Raw states:
Using this type it is possible to unmarshal or marshal values partially.
But I can't find a Marshal function that would return bson.Raw.
What am I missing?
Example of what I try to do:
package main
import (
"fmt"
"gopkg.in/mgo.v2/bson"
)
func main() {
// How to avoid a MarshalRaw help function?
raw, err := MarshalRaw("Hello world")
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", raw)
}
func MarshalRaw(v interface{}) (*bson.Raw, error) {
bin, err := bson.Marshal(struct{ Raw interface{} }{v})
if err != nil {
return nil, err
}
var raw struct{ Raw bson.Raw }
err = bson.Unmarshal(bin, &raw)
if err != nil {
return nil, err
}
return &raw.Raw, nil
}
Output:
&{Kind:2 Data:[12 0 0 0 72 101 108 108 111 32 119 111 114 108 100 0]}
bson.Raw is used as a value both when marshaling and unmarshaling.
To transform an interface{} into a bson.Raw, the first thing to do is to marshal it so that you get the plain document data that represents whatever is being marshaled:
var value interface{} = bson.M{"some": "value"}
data, err := bson.Marshal(value)
if err != nil {
log.Fatal(err)
}
and then it may have one or more fields unmarshaled into bson.Raw values:
var doc struct{ Some bson.Raw }
err = bson.Unmarshal(data, &doc)
if err != nil {
log.Fatal(err)
}
or even the entire document:
var doc bson.Raw
err = bson.Unmarshal(data, &doc)
if err != nil {
log.Fatal(err)
}
If you want the entire document rather than just a field, you can also use this shortcut:
doc := bson.Raw{3, data}
The 3 constant represents a document in the bson specification, and it must of course match the provided data. Since BSON only supports documents at the top level, we know this must be correct.
I believe that bson.Raw is intended to be used as a type for a variable.
for example: (in play)
type Bar struct {
AnInt int
AString bson.Raw
}
The "AString" field will be kept as the bson.Raw struct your link mentions.
This is super-useful if you want to partially decode the top level of a nested structure to figure out its actualy type so you can parse the rest in to the proper datatype.
Note, the above is untested, not in front of a machine I can actually run go on at the moment. This is based on the assumption that it works like the standard encoding/json package.
Solution I was looking for
m := bson.M{"ns": bson.M{"coll": "test1", "db": "testdb"}}
data, _ := bson.Marshal(m)
r := bson.Raw(data)
coll := r.Lookup("ns", "coll").StringValue()
println(coll)
Playground