I load a list of items from the database and would like to return it as Json.
it works well but the response is not a json object, its a string with is a problem on the receiver side.
var products []*database.Product
num, err := o.QueryTable("product").Filter("date", date).All(&products)
fmt.Printf("Returned Rows Num: %d, %s", num, err)
var jsonData []byte
jsonData, err2 := json.Marshal(products)
if err2 != nil {
fmt.Println(err)
}
this.Data["json"] = string(jsonData)
this.ServeJSON()
it returns
"[{\"Id\":\"68e7512f-ea50-45d3-a89e-845c7621b33d\",\"Producttypeid\":\"62c9ff0a-f599-4ac1-9442-ebae3bc049c1\",\"Producti...
but should be
[{"Id":"68e7512f-ea50-45d3-a89e-845c7621b33d","Producttypeid":"62c9ff0a-f599-4ac1-9442-ebae3bc049c1","Producti...
adding annotations like json:"id" does not change anything.
is it a problem because it is an array?
do I have to wrap that inside a struct?
as mentioned in the comments from Kosanovic the solution is very simple, just dont marshal yourself.
var products []*database.Product
num, err := o.QueryTable("product").Filter("date", date).All(&products)
fmt.Printf("Returned Rows Num: %d, %s", num, err)
this.Data["json"] = products
this.ServeJSON()
Related
Summary
I am trying to write data from several postgres tables into a nested Go struct for the purpose of returning a single json response to a GET request within my web app.
Questions
Is the way I'm declaring a nested struct reasonable from a Go best practices perspective, or is there a reason I should avoid this method and do it another way?
What am I doing wrong in Step 3 to prevent my code from working? (I fear the answer is 'everything')
What I've got so far
I've declared my struct of structs
type MainObject struct {
SubObjects []struct {
SpecificDetail string `json:"specific-detail"`
} `json:"sub-object"`
...(other []structs)...
}
I've retrieved rows from the tables
func getMainObjectHandler(w http.ResponseWriter, r *http.Request) {
...(database connnection)...
MainObjectID := r.URL.Query().Get("moid")
if MainObjectID != "null" {
NewMainObject := MainObject{}
SubObjectDetail_rows, err := db.Query("SELECT specific_detail from the_table WHERE moid= '" + MainObjectID + "'")
if err != nil {
log.Fatalf("could not execute query: %v", err)
}
...(other db.Query rows)...
I've tried (and failed) to build the row data into the struct.
for SubObjectDetail_rows.Next() {
SpecificDetail := NewMainObject.SubObject.SpecificDetail{}
SubObjectDetail_rows.Scan(&SpecificDetail)
SubObject = append(SubObject, SpecificDetail)
}
NewMainObject = append(MainObject, SubObject)
defer persona_rows.Close()
Finally, I've set up Marshal and write.
NMOListBytes, err := json.Marshal(NewMainObject)
if err != nil {
fmt.Println(fmt.Errorf("Error: %v", err))
w.WriteHeader(http.StatusInternalServerError)
return
}
w.Write(NMOListBytes)
Firstly, please use placeholders when creating your SQL query to avoid injections:
// db.Query("SELECT specific_detail from the_table WHERE moid= '" + MainObjectID + "'") // not this
db.Query("SELECT specific_detail from the_table WHERE moid=?", MainObjectID)
Unless you're using a framework like GORM you can't Scan() into a single struct value.
From the docs:
Scan copies the columns in the current row into the values pointed at
by dest. The number of values in dest must be the same as the number
of columns in Rows.
It looks like you're pulling JSON from a DB query, as you're only querying one column, so you probably want:
var bs []byte // get raw JSON bytes
err = SubObjectDetail_rows.Scan(&bs)
if err != nil { /* always check errors */ }
and then unmarshal them to your struct:
err = json.Unmarshal(bs, &SpecificDetail)
if err != nil { /* ... */ }
I have a use case where I have the code as below. I have a request coming in to hit the backend where I need to append data to a map. My question is how do I convert the below type to a []byte to unmarshal?
Any ideas would be appreciated.
type Example struct {
Category string `json:"category"`
Name string `json:"name"`
}
Incoming Postman request json looks like this:
[{"Category":"TestCategory", "Name":"Sample1"}]
but after doing
jsonString Type: []Example
if err := gc.ShouldBindJSON(&jsonString) it looks like [{TestCategory Sample1}] ; how do I convert this to a []byte?
for _, req := range blob{
var jsonString Example
if err := json.Unmarshal([]byte(jsonString), &blob); err != nil { //this does not work
logger.Fatal(err)
}
//I am checking if a key-value is present and appending it to the map
dict := make(map[string][]Example)
dict[req.Category] = append(dict[req.Category], req)
fmt.Println(dict)
if value, ok := dict["TestCategory"]; ok {
fmt.Printf("Found %d\n", value)
} else {
fmt.Println("not found")
}
}
//I was able to test the above logic by declaring the jsonString as a const and it works
There are two directions in which you can move the data:
from JSON to a Go data structure
// This is your payload coming from the request.
jsonStr := `[{"Category":"TestCategory", "Name":"Sample1"}]`
// This is the Go struct that will hold the unmarshalled data.
var examples []Example
err := json.Unmarshal([]byte(jsonStr), &examples)
if err != nil {
log.Fatal(err)
}
fmt.Println("Examples:", examples) // prints "Examples: [{TestCategory Sample1}]"
from a Go data structure to JSON (either string or []byte)
exampleBytes, err := json.Marshal(examples)
if err != nil {
log.Fatal(err)
}
fmt.Println("Example bytes:", string(exampleBytes)) // prints "Example bytes: [{"category":"TestCategory","name":"Sample1"}]"
You should check out "Go by Example" if you haven't already: https://gobyexample.com/json
Looking at your code:
You are looping on blob but instead of using the req you are trying to unmarshal onto the entire blob each time. I'm not sure what you are trying to achieve there but nothing good can come out of changing a struct you're looping over from within the loop.
The request JSON you are listing is an array of JSON objects. You are trying to unmarshal that into a single Example struct. That won't work, you need an array of those.
I have this json that I convert to:
var leerCHAT []interface{}
but I am going through crazy hoops to get to any point on that map inside map and inside map crazyness, specially because some results are different content.
this is the Json
[
null,
null,
"hub:zWXroom",
"presence_diff",
{
"joins":{
"f718a187-6e96-4d62-9c2d-67aedea00000":{
"metas":[
{
"context":{},
"permissions":{},
"phx_ref":"zNDwmfsome=",
"phx_ref_prev":"zDMbRTmsome=",
"presence":"lobby",
"profile":{},
"roles":{}
}
]
}
},
"leaves":{}
}
]
I need to get to profile then inside there is a "DisplayName" field.
so I been doing crazy hacks.. and even like this I got stuck half way...
First is an array so I can just do something[elementnumber]
then is when the tricky mapping starts...
SORRY about all the prints etc is to debug and see the number of elements I am getting back.
if leerCHAT[3] == "presence_diff" {
var id string
presence := leerCHAT[4].(map[string]interface{})
log.Printf("algo: %v", len(presence))
log.Printf("algo: %s", presence["joins"])
vamos := presence["joins"].(map[string]interface{})
for i := range vamos {
log.Println(i)
id = i
}
log.Println(len(vamos))
vamonos := vamos[id].(map[string]interface{})
log.Println(vamonos)
log.Println(len(vamonos))
metas := vamonos["profile"].(map[string]interface{}) \\\ I get error here..
log.Println(len(metas))
}
so far I can see all the way to the meta:{...} but can't continue with my hacky code into what I need.
NOTICE: that since the id after Joins: and before metas: is dynamic I have to get it somehow since is always just one element I did the for range loop to grab it.
The array element at index 3 describes the type of the variant JSON at index 4.
Here's how to decode the JSON to Go values. First, declare Go types for each of the variant parts of the JSON:
type PrescenceDiff struct {
Joins map[string]*Presence // declaration of Presence type to be supplied
Leaves map[string]*Presence
}
type Message struct {
Body string
}
Declare a map associating the type string to the Go type:
var messageTypes = map[string]reflect.Type{
"presence_diff": reflect.TypeOf(&PresenceDiff{}),
"message": reflect.TypeOf(&Message{}),
// add more types here as needed
}
Decode the variant part to a raw message. Use use the name in the element at index 3 to create a value of the appropriate Go type and decode to that value:
func decode(data []byte) (interface{}, error) {
var messageType string
var raw json.RawMessage
v := []interface{}{nil, nil, nil, &messageType, &raw}
err := json.Unmarshal(data, &v)
if err != nil {
return nil, err
}
if len(raw) == 0 {
return nil, errors.New("no message")
}
t := messageTypes[messageType]
if t == nil {
return nil, fmt.Errorf("unknown message type: %q", messageType)
}
result := reflect.New(t.Elem()).Interface()
err = json.Unmarshal(raw, result)
return result, err
}
Use type switches to access the variant part of the message:
defer ws.Close()
for {
_, data, err := ws.ReadMessage()
if err != nil {
log.Printf("Read error: %v", err)
break
}
v, err := decode(data)
if err != nil {
log.Printf("Decode error: %v", err)
continue
}
switch v := v.(type) {
case *PresenceDiff:
fmt.Println(v.Joins, v.Leaves)
case *Message:
fmt.Println(v.Body)
default:
fmt.Printf("type %T not handled\n", v)
}
}
Run it on the playground.
https://github.com/olivere/elastic
Version 5.x
The wiki documentation isn't really clear on how client.Update() works. It's needed to completely change a field and to modify arrays. i.e. in the example in the wiki documentation, how would one go about appending and removing tags to a tweet or changing a tweet's content? Also if a tweet was represented in go as a struct and I added a nested struct called "echo" which contains a foo of type int, content of type string and another type string array, how would one go about changing any of these fields using client.Update() if it's even possible?
In my personal example I have this function:
func UpdateEntryContent(eclient *elastic.Client, entryID string, newContent []rune) error{
ctx:=context.Background()
exists, err := eclient.IndexExists(ENTRY_INDEX).Do(ctx)
if err != nil {return err}
if !exists {return errors.New("Index does not exist")}
_, err = eclient.Update().Index(ENTRY_INDEX).Type(ENTRY_TYPE).Id(entryID).
Script("ctx._source.Content = newCont").
ScriptParams(map[string]interface{}{"newCont": newContent}).
Do(ctx)
if err != nil {return err}
return nil
}
But I get this following error when I try to compile:
cannot use "ctx._source.Content = newCont" (type string) as type *elastic.Script in argument to eclient.Update().Index(ENTRY_INDEX).Type(ENTRY_TYPE).Id(entryID).Script
eclient.Update().Index(ENTRY_INDEX).Type(ENTRY_TYPE).Id(entryID).Script("ctx._source.Content = newCont").ScriptParams undefined (type *elastic.UpdateService has no field or method ScriptParams)
The Script method accepts a *elastic.Script, not a string. The ScriptParams method is also found on *elastic.Script as Params instead of being on *elastic.UpdateService.
func UpdateEntryContent(eclient *elastic.Client, entryID string, newContent []rune) error{
ctx:=context.Background()
exists, err := eclient.IndexExists(ENTRY_INDEX).Do(ctx)
if err != nil {return err}
if !exists {return errors.New("Index does not exist")}
script := elastic.NewScript("ctx._source.Content = newCont").Params(map[string]interface{}{"newCont": newContent})
_, err = eclient.Update().Index(ENTRY_INDEX).Type(ENTRY_TYPE).Id(entryID).
Script(script).
Do(ctx)
if err != nil {return err}
return nil
}
You can see more information about the package with GoDoc or by looking through the source code.
The following code should resolve the issue
_, err = eclient.Update().Index(INDEX).
Type(TYPE).
Id(ID).
Doc(map[string]interface{}{field: message}).
Do(ctx)
Credit where it's due Gavin's answer put me on the right track. This is for another .Index but the full function that acts as a generic single field update is as follows:
func UpdateUser(eclient *elastic.Client, userID string, field string, newContent interface{})error {
//CHANGES A SINGLE FIELD OF ES USER DOCUMENT(requires an elastic client pointer,
// the user DocID, the feild you wish to modify as a string,
// and what you want to change that field to as any type necessary)
//RETURN AN error IF SUCESSFUL error = nil
ctx := context.Background()
exists, err := eclient.IndexExists(USER_INDEX).Do(ctx)
if err != nil {return err}
if !exists {return errors.New("Index does not exist")}
_, err = eclient.Update().
Index(USER_INDEX).
Type(USER_TYPE).
Id(userID).
Doc(map[string]interface{}{field: newContent}).
Do(ctx)
return nil
}
You can change the .Index, .Type, and .Id and it works with all fields and types as far as I can tell
I'm using the following code to get a response from a server after a post request:
type ResponseFromPost struct {
N_expediente string
Enviar string
}
func main(){
......
res, err := client.Do(req)
if err != nil {
return
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
var re ResponseFromPost
err = json.Unmarshal(body, &re)
fmt.Println(re.Enviar);
}
With this I get:
error: &{%!e(string=array) %!e(*reflect.rtype=&{32 2509985895 0 8 8 25 0x608170
[0x7703c0 <nil>] 0x730b80 0x69acb0 0x6116c0 0x7732c0})}
The value sent by the server is:
[{"n_expediente":"9","enviar":"2"}]
How can I use the json variables?
The json is an array of those objects having the strings n_expediente and enviar on each instance. In the Go you'll need an array of your type;
re := []ResponseFromPost{}
err := json.Unmarshal([]byte(`[{"n_expediente":"9","enviar":"2"}]`), &re)
fmt.Println(re[0].Enviar);
Here's an example to show what your model needs to be https://play.golang.org/p/d64Sict4AG
You'll probably want to make some other changes to your code based on that. Like a loop bound by the length of the slice ( for i := range re { print i } ) after the unmarshal and a different name for the type.