how to fetch data from another api with body raw json - go

i have default body raw json and want to paste it into a struct so it can fetch data automatically and save it into a struct
Body Raw Json
{
"jsonrpc": "2.0",
"params": {
}
}
Response from api
{
"jsonrpc": "2.0",
"id": null,
"result": {
"status": 200,
"response": [
{
"service_id": 1129,
"service_name": "Adobe Illustrator",
"service_category_id": 28,
"service_category_name": "License Software",
"service_type_id": 25,
"service_type_name": "Software",
"create_date": "2020-03-09 03:47:44"
},
],
"message": "Done All User Returned"
}
}
I want to put it in the repository file so I can get data automatically
Repo file
// Get request
resp, err := http.Get("look at API Response Example")
if err != nil {
fmt.Println("No response from request")
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body) // response body is []byte
if err != nil {
return err
}
// data that already fetch accomadate to struct
var result models.OdooRequest
if err := json.Unmarshal(body, &result); err != nil {
fmt.Println("Can not unmarshal JSON")
}
for _, rec := range result.Response {
fmt.Println(rec.ServiceName)
}
return err
after being fetched then accommodated into a struct
struct
type OdooRequest struct {
Response []UpsertFromOdooServices
}

Sure, here's a rough way to make that request and read the response:
package main
import (
"bytes"
"encoding/json"
"fmt"
"net/http"
)
type OdooRequest struct {
Result struct {
Status int `json:"status"`
Response []struct {
ServiceID int `json:"service_id"`
ServiceName string `json:"service_name"`
ServiceCategoryID int `json:"service_category_id"`
ServiceCategoryName string `json:"service_category_name"`
ServiceTypeID int `json:"service_type_id"`
ServiceTypeName string `json:"service_type_name"`
CreateDate string `json:"create_date"`
} `json:"response"`
Message string `json:"message"`
} `json:"result"`
}
func main() {
if err := run(); err != nil {
panic(err)
}
}
func run() error {
resp, err := http.Post(
"ADD_URL_HERE",
"application/json",
bytes.NewBufferString(`{"jsonrpc": "2.0","params": {}}`),
)
if err != nil {
return err
}
defer resp.Body.Close()
var odooResp OdooRequest
if err := json.NewDecoder(resp.Body).Decode(&odooResp); err != nil {
return err
}
for _, rec := range odooResp.Result.Response {
fmt.Println(rec.ServiceName)
}
return nil
}

Related

How to unmarshal/parse a request using Go correctly?

How to parse a curl request response in Golang correctly? I have tried the following where I send a request to an api and its response is:
{
"Certificates": [
{
"Name": "some-name.here",
.......
}
],
"DataRange": "Certificates 1 - 1",
"TotalCount": 1
}
Now I want to use the Name in the Certificates in a string variable. i.e match. Before even I get to the looping through of the response, I get the error: json: cannot unmarshal object into Go value of type []program.Item. This error is coming from json.Unmarshal function where I pass my []bytes and the struct to use the bytes. Am I doing this correctly?
type Item struct {
Certificates []CertificatesInfo
}
type CertificatesInfo struct {
Name string
}
func main() {
url := .....
req, err := http.NewRequest("GET", url, nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
// handle err
continue
}
defer resp.Body.Close()
bodyBytes, err := io.ReadAll(resp.Body)
if err != nil {
continue
}
var ItemInfo []Item
if err := json.Unmarshal(bodyBytes, &ItemInfo); err != nil {
return nil, fmt.Errorf("failed to parse %v", err)
}
for _, infos := range ItemInfo {
for _, names := range infos.Certificates {
infraId:= names.Name
match:= display_name
}
}
}
There's a mismatch between your sample JSON response data (a single object containing a list of certificates), and your code, which is expecting a list of objects, each of which contain a list of certificates).
Assuming that your JSON example is correct, this bit:
var ItemInfo []Item
if err := json.Unmarshal(bodyBytes, &ItemInfo); err != nil {
. . .
should probably be
var itemInfo Item
if err := json.Unmarshal(bodyBytes, &ItemInfo); err != nil {
. . .
Works on my machine:
https://go.dev/play/p/8FqIif1zzsQ
package main
import (
"encoding/json"
"fmt"
)
func main() {
resp := Item{}
err := json.Unmarshal([]byte(data), &resp)
if err != nil {
fmt.Print(err)
panic(err)
}
fmt.Printf("resp.DataRange: %s\n", resp.DataRange)
fmt.Printf("resp.TotalCount: %d\n", resp.TotalCount)
fmt.Printf("len(resp.Certificates: %d\n", len(resp.Certificates))
for i, c := range resp.Certificates {
fmt.Printf("resp.Certificates[%d].Name: %s\n", i, c.Name)
}
}
type Item struct {
Certificates []CertificatesInfo
DataRange string
TotalCount int
}
type CertificatesInfo struct {
Name string
}
const data = `
{
"Certificates": [
{ "Name": "alpha" },
{ "Name": "bravo" },
{ "Name": "charlie" }
],
"DataRange": "Certificates 1 - 1",
"TotalCount": 3
}
`

How to receive a json to insert

I am receiving some values per post and I have a json type field but it arrives empty and if I enter a normal text it works and I do not see the error in the field
the model was updated so that it receives the fields and allows inserting in mysql
POSTman
{
"Code":"1234",//it works
"Desc":"desc",//it works
"Config":{"link":"https://stackoverflow.com/" }, //not works
"Dev":[ {"item":1},{"item":2}]//not works
}
type User struct {
gorm.Model
Code string `gorm:"type:varchar(100);unique_index"`
Desc string `gorm:"type:varchar(255);"`
Config JSON `json:"currencies" gorm:"type:varchar(255);"`
Dev JSON `json:"currencies" gorm:"type:varchar(255);"`
}
func CreateUser(c *gin.Context) {
var usuario models.User
var bodyBytes []byte
if c.Request.Body != nil {
bodyBytes, _ = ioutil.ReadAll(c.Request.Body)
}
data := bytes.NewBuffer(bodyBytes)
fmt.Println(data.Config)
c.BindJSON(&usuario)
db.DB.Create(&usuario)
c.JSON(200, usuario)
}
Model update. receive post form with json fields and insert in mysql
package models
import (
"bytes"
"database/sql/driver"
"errors"
)
type JSON []byte
func (j JSON) Value() (driver.Value, error) {
if j.IsNull() {
return nil, nil
}
return string(j), nil
}
func (j *JSON) Scan(value interface{}) error {
if value == nil {
*j = nil
return nil
}
s, ok := value.([]byte)
if !ok {
errors.New("error")
}
*j = append((*j)[0:0], s...)
return nil
}
func (m JSON) MarshalJSON() ([]byte, error) {
if m == nil {
return []byte("null"), nil
}
return m, nil
}
func (m *JSON) UnmarshalJSON(data []byte) error {
if m == nil {
return errors.New("error")
}
*m = append((*m)[0:0], data...)
return nil
}
func (j JSON) IsNull() bool {
return len(j) == 0 || string(j) == "null"
}
func (j JSON) Equals(j1 JSON) bool {
return bytes.Equal([]byte(j), []byte(j1))
}
Thank you very much to everyone who helped me, I consider that the functionality of receiving a json and saving it in mysql is very common and this can be useful to many people
You can change the JSON like below or You can change the Struct like below (I prefer struct approach)
{
"Code": "1234",
"Desc": "desc",
"Config": {
"Link": "https://stackoverflow.com/"
},
"Dev": [
{
"Item": 1
},
{
"Item": 2
}
]
}
Struct:
type User struct {
gorm.Model
Code string `json:"Code" gorm:"type:varchar(100);unique_index"`
Desc string `json:"Desc" gorm:"type:varchar(255);"`
Config []struct {
Link string `json:"link" gorm:"type:varchar(255);"`
Title string `json:"title" gorm:"type:varchar(255);"`
}
Dev []struct {
Item string `json:"item" gorm:"type:varchar(255);"`
}
}
You have made two kind of mistakes
Your json decoding cannot work because your struct does not match your json. Config is defined as a array of something but in your json you have an object not array, and in Dev the property Item is a int not a string
Your model may not be well defined as you have not defined you joined table. Well I never seen a working example with this kind of definition. I suggest you to declare your nested struct as independent struct.
Here a full working example :
package main
import (
"database/sql"
"encoding/json"
"fmt"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
)
const data = `{
"Code":"1234",
"Desc":"desc",
"Config":{"link":"https://stackoverflow.com/" },
"Dev":[ {"item":1},{"item":2}]
}`
type Config struct {
Id int `gorm:"primaryKey"`
Link string `json:"link"`
Title string
UserId int
}
type Item struct {
Id int `gorm:"primaryKey"`
Item int `json:"item"`
UserId int
}
type User struct {
Id int `gorm:"primaryKey"`
Code string
Desc string
Config Config `gorm:"foreignkey:UserId"`
Dev []Item `gorm:"foreignkey:UserId"`
}
func initDb(url string) (*gorm.DB, *sql.DB, error) {
connexion := sqlite.Open(url)
db, err := gorm.Open(connexion, &gorm.Config{})
if err != nil {
return nil, nil, err
}
sql, err := db.DB()
if err != nil {
return nil, nil, err
}
err = db.AutoMigrate(&User{})
if err != nil {
return nil, nil, err
}
err = db.AutoMigrate(&Item{})
if err != nil {
return nil, nil, err
}
err = db.AutoMigrate(&Config{})
if err != nil {
return nil, nil, err
}
return db, sql, nil
}
func run() error {
db, sql, err := initDb("file::memory:?cache=shared")
if err != nil {
return err
}
defer sql.Close()
var user User
err = json.Unmarshal([]byte(data), &user)
fmt.Printf("%#v\n", user)
err = db.Create(&user).Error
if err != nil {
return err
}
var loaded User
db.Preload("Config").Preload("Dev").First(&loaded)
fmt.Printf("%#v\n", loaded)
return nil
}
func main() {
if err := run(); err != nil {
fmt.Println("failed", err)
}
}
try adding this JSON Field in your model
import (
"errors"
"database/sql/driver"
"encoding/json"
)
// JSON Interface for JSON Field of yourTableName Table
type JSON interface{}
// Value Marshal
func (a JSON) Value() (driver.Value, error) {
return json.Marshal(a)
}
// Scan Unmarshal
func (a *JSON) Scan(value interface{}) error {
b, ok := value.([]byte)
if !ok {
return errors.New("type assertion to byte failed")
}
return json.Unmarshal(b,&a)
}
All these answers didn't work for me, but this will work for everyone
Model
// This is the max Thing you need
import "gorm.io/datatypes"
import "encoding/json"
type CMSGenericModel struct {
gorm.Model
//... Other Posts
ExtraData datatypes.JSON `json:"data"`
}
In Handler Function
type CmsReqBody struct {
// ============= RAW ========
Data json.RawMessage `json:"data"`
// other props...
}
cmsBodyRecord := new(models.CMSGenericModel)
cmsBodyPayload := new(CmsReqBody)
if err := c.BodyParser(cmsBodyPayload); err != nil {
return c.Status(503).SendString(err.Error())
}
cmsBodyRecord.ExtraData = datatypes.JSON(cmsBodyPayload.Data)
My Sample Data
{
"title": "Blog Post 1",
"subtitle": "first",
"description": "Updated",
"type": "blog",
"isActive": true,
"uuid": "new",
"data": {
"complex1": ["kkkk", "yyyy"],
"complex2": [
{
"name": "sourav"
},
{
"name": "yahooo"
},
{
"yahoo": "name",
"kjk": ["abbsb", {"data": "abcd"}]
}
]
}
}

Golang Unmarshal an JSON response, then marshal with Struct field names

So I am hitting an API that returns a JSON response and I am unmarshalling it into a struct like so:
package main
type ProcessedRecords struct {
SLMIndividualID string `json:"individual_id"`
HouseholdPosition int `json:"Household Position"`
IndividualFirstName string `json:"individual_first_name"`
}
func main() {
req, _ := http.NewRequest(method, url, payload)
res, err := client.Do(req)
if err != nil {
fmt.Println(err)
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Println(err)
}
fmt.Println(body)
var responseObject Response
json.Unmarshal(body, &responseObject)
fmt.Println(responseObject)
which works great. However I need to marshal this struct again but I want to use the Struct Fields as keys instead of the json: ... fields. I am using the following code:
recordsInput := []*firehose.Record{}
for i := 0; i < len(records); i++ {
if len(recordsInput) == 500 {
* code to submit records, this part works fine *
}
b, err := json.Marshal(records[i])
if err != nil {
log.Printf("Error: %v", err)
}
record := &firehose.Record{Data: b}
recordsInput = append(recordsInput, record)
}
This does submit records successfully but it's in the format:
{"individual_id":"33c05b49-149b-480f-b1c2-3a3b30e0cb6f","Household Position":1...}
and I'd like it in the format:
{"SLMIndividualId":"33c05b49-149b-480f-b1c2-3a3b30e0cb6f","HouseholdPosition":1...}
How can I achieve this?
Those tags say how the struct should be marshalled, so if they are present, that is how the output will be. You'll need to convert it to a matching struct that does not have the json: tags:
type ProcessedRecords struct {
SLMIndividualID string `json:"individual_id"`
HouseholdPosition int `json:"Household Position"`
IndividualFirstName string `json:"individual_first_name"`
}
type ProcessedRecordsOut struct {
SLMIndividualID string
HouseholdPosition int
IndividualFirstName string
}
func process() {
var in ProcessedRecords
json.Unmarshal(data, &in)
// Convert to same type w/o tags
out := ProcessedRecordsOut(in)
payload, _ := json.Marshal(out)
// ...
}
See a working example here: https://play.golang.org/p/p0Fc8DJotYE
You can omit fields one-way by defining a custom type and implementing the correct interface, e.g.
package main
import (
"encoding/json"
"fmt"
)
type Animal struct {
Name ReadOnlyString
Order string
}
type ReadOnlyString string
func (ReadOnlyString) UnmarshalJSON([]byte) error { return nil }
func main() {
x := Animal{"Bob", "First"}
js, err := json.Marshal(&x)
if err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%s\n", js)
var jsonBlob = []byte(`{"Name": "Platypus", "Order": "Monotremata"}`)
if err := json.Unmarshal(jsonBlob, &x); err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%#v\n\n", x)
}
https://go.dev/play/p/-mwBL0kIqM
Found this answer here: https://github.com/golang/go/issues/19423#issuecomment-284607677

BindJSON does not return error when Input is wrong

Following is what I am trying
func TestCreteJob(t *testing.T) {
requestBody, err := json.Marshal(map[string]string{
"username": "u1",
"password": "p1",
})
if err != nil {
t.Errorf("Test Failed ")
}
res, err := http.Post("http://127.0.0.1:9090/job", "application/json", bytes.NewBuffer(requestBody))
if err != nil {
log.Fatal(err)
}
greeting, err := ioutil.ReadAll(res.Body)
res.Body.Close()
if err != nil {
log.Fatal(err)
}
fmt.Printf("%s", greeting)
}
and My Handler is like below
func Handlers() *gin.Engine {
router := gin.Default()
// Sample API
router.GET("/ping", func(c *gin.Context) {
c.JSON(200, gin.H{
"message": "pong",
})
})
router.POST("/job", func(c *gin.Context) {
if c.Request.Method == "POST" {
var u User
err := c.BindJSON(&u)
if err != nil {
c.AbortWithError(http.StatusBadRequest, err)
}
c.JSON(http.StatusOK, gin.H{
"user": u.Username,
"pass": u.Password,
})
}
})
return router
}
I have defined object as
type User struct {
Username string `json: abc`
Password string `json: password`
}
This should fail since input does not have 'abc'
Am I doing something wrong here.
Your struct tags are wrong for the json
type User struct {
Username string `json:"abc"`
Password string `json:"password"`
}
Take a look here
https://play.golang.org/p/oj_PwLoX5qs
package main
import (
"encoding/json"
"fmt"
)
func main() {
fmt.Println("Hello, playground")
dataz := []byte(`{"abc": "sberry", "password": "hunter2"}`)
var u User
err := json.Unmarshal(dataz, &u)
if err != nil {
fmt.Println(err.Error())
}
fmt.Printf("%+v\n", u)
}
type User struct {
Username string `json:"abc"`
Password string `json:"password"`
}

Index out of Range with array of structs in Go

I am new to Go so hopefully I'm making myself clear with this issue I'm having. My problem is that I am trying to iterate over an array of structs but I keep running into an index out of range issue. For the purposes of this problem, I have already verified that my array is not empty but that it in fact does contain at least one Services struct and file_content is the string that contains my valid JSON
Here is the snippet of code that represents the problem I'm having:
type service_config struct {
Services []struct {
Name string
Command string
Request map[string]interface{}
}
}
var ServiceConf = service_config{}
err_json := json.Unmarshal(file_content, &ServiceConf)
for _, s := range ServiceConf.Services {
log.Println(s)
}
So every time I run my code I get:
2014/03/14 18:19:53 http: panic serving [::1]:65448: runtime error: index out of range
{
"services" : [
{
"name": "translation",
"command": "to german",
"request": {
"key": "XXX",
"url": "https://www.googleapis.com/language/translate/v2?"
}
}
]
}
If you're interested in the complete source file:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"net/url"
"os"
)
type SlackResponse struct {
token string
team_id string
channel_id string
channel_name string
timestamp string
user_id string
user_name string
text string
}
type service_config struct {
Services []struct {
Name string
Command string
Request map[string]interface{}
}
}
var ServiceConf = service_config{}
func main() {
content, err_read := ioutil.ReadFile("config.ini")
if err_read != nil {
log.Println("Could not read config")
return
}
log.Println(string(content))
err_json := json.Unmarshal(content, &ServiceConf)
if err_json != nil {
log.Println(err_json)
}
http.HandleFunc("/", handler)
http.ListenAndServe(":"+os.Getenv("PORT"), nil)
}
func handler(w http.ResponseWriter, r *http.Request) {
slack_response := SlackResponse{
r.FormValue("token"),
r.FormValue("team_id"),
r.FormValue("channel_id"),
r.FormValue("channel_name"),
r.FormValue("timestamp"),
r.FormValue("user_id"),
r.FormValue("user_name"),
r.FormValue("text"),
}
// log.Println(ServiceConf.Services[0].Request["key"])
// loop through services to find command phrases
for _, s := range ServiceConf.Services {
log.Println(s)
}
if slack_response.user_name == "slackbot" {
return
}
// fmt.Fprintf(w, "{ \"text\": \"Master %s! You said: '%s'\" }", slack_response.user_name, slack_response.text)
content, err := getContent("https://www.googleapis.com/language/translate/v2?key=&source=en&target=de&q=" + url.QueryEscape(slack_response.text))
if err != nil {
fmt.Fprintf(w, "{ \"text\": \"Huh?!\" }")
} else {
type trans struct {
Data struct {
Translations []struct {
TranslatedText string `json:"translatedText"`
} `json:"translations"`
} `json:"data"`
}
f := trans{}
err := json.Unmarshal(content, &f)
if err != nil {
log.Println(err)
}
fmt.Fprintf(w, "{ \"text\": \"Translated to German you said: '%s'\" }", f.Data.Translations[0].TranslatedText)
}
}
// array of bytes if retrieved successfully.
func getContent(url string) ([]byte, error) {
// Build the request
req, err := http.NewRequest("GET", url, nil)
if err != nil {
return nil, err
}
// Send the request via a client
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
// Defer the closing of the body
defer resp.Body.Close()
// Read the content into a byte array
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
// At this point we're done - simply return the bytes
return body, nil
}
Here is the stack trace:
2014/03/21 23:21:29 http: panic serving [::1]:59508: runtime error: index out of range
goroutine 3 [running]:
net/http.funcĀ·009()
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1093 +0xae
runtime.panic(0x215f80, 0x4b6537)
/usr/local/Cellar/go/1.2/libexec/src/pkg/runtime/panic.c:248 +0x106
main.handler(0x5a85e8, 0xc21000f6e0, 0xc210037dd0)
/Users/et/src/go/src/github.com/etdebruin/gojacques/main.go:100 +0x81b
net/http.HandlerFunc.ServeHTTP(0x2cbc60, 0x5a85e8, 0xc21000f6e0, 0xc210037dd0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1220 +0x40
net/http.(*ServeMux).ServeHTTP(0xc21001e5d0, 0x5a85e8, 0xc21000f6e0, 0xc210037dd0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1496 +0x163
net/http.serverHandler.ServeHTTP(0xc21001f500, 0x5a85e8, 0xc21000f6e0, 0xc210037dd0)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1597 +0x16e
net/http.(*conn).serve(0xc210058300)
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1167 +0x7b7
created by net/http.(*Server).Serve
/usr/local/Cellar/go/1.2/libexec/src/pkg/net/http/server.go:1644 +0x28b
The error comes from this line
fmt.Fprintf(w, "{ \"text\": \"Translated to German you said: '%s'\" }",
f.Data.Translations[0].TranslatedText)
So you didn't get any Translations back - that array is empty.
You might want to check resp.Status to see if an error was returned. This isn't returned as an error - you need to check it yourself.

Resources