I am trying to write simple http request to get data from Zendesk API. The simple request works, but I am not able to find out any example to handle pagination. Unfortunatelly, I was not able able find any example on the net related to consuming REST API with pagination.
I am new to Go and I am not professional programmer. I have the code working in Python where I use while loop with NextPage variable. But I am trying to learn Go and trying to rewrite it.
Please can you give me an advise? Should I do it in the same way using a loop with NextPage and appending json data, or is there any other way?
Please assume I am learning. Thank you.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"strconv"
"time"
)
type Response struct {
Organizations []struct {
URL string `json:"url"`
ID int64 `json:"id"`
Name string `json:"name"`
SharedTickets bool `json:"shared_tickets"`
SharedComments bool `json:"shared_comments"`
ExternalID interface{} `json:"external_id"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DomainNames []string `json:"domain_names"`
Details interface{} `json:"details"`
Notes interface{} `json:"notes"`
GroupID interface{} `json:"group_id"`
Tags []interface{} `json:"tags"`
OrganizationFields struct {
SevenX24HourSupportPurchased interface{} `json:"7x24_hour_support_purchased_"`
K8SNodesLicensed interface{} `json:"k8s_nodes_licensed"`
LatestClosedOpportunityDate interface{} `json:"latest_closed_opportunity_date"`
PartnerOfRecordInSf string `json:"partner_of_record_in_sf"`
VcpusLicensed interface{} `json:"vcpus_licensed"`
} `json:"organization_fields"`
} `json:"organizations"`
NextPage string `json:"next_page"`
PreviousPage interface{} `json:"previous_page"`
Count int `json:"count"`
}
func organizationsGet() {
url := "https://zendesk.zendesk.com/api/v2/organizations.json"
req, _ := http.NewRequest("GET", url, nil)
req.Header.Add("Authorization", "Basic blabla")
res, _ := http.DefaultClient.Do(req)
defer res.Body.Close()
body, _ := ioutil.ReadAll(res.Body)
fmt.Println(res)
fmt.Println(string(body))
var responseObject Response
json.Unmarshal(body, &responseObject)
for i := 0; i < len(responseObject.Organizations); i++ {
fmt.Println(responseObject.Organizations[i].Name + " " + strconv.FormatInt(int64(responseObject.Organizations[i].ID), 10))
// fmt.Printf("%v,%v\n", responseObject.User[i].Name, responseObject.User[i].Email)
}
}
Related
I'm using the gin framework and the following code works fine for me
import (
"github.com/gin-gonic/gin"
"net/http"
)
type RequestBody struct {
MyRequiredField string `json:"myRequiredField" binding:"required"`
}
func Handle(context *gin.Context) {
var requestBody RequestBody
err := context.ShouldBindJSON(&requestBody)
if err != nil {
context.JSON(http.StatusBadRequest, err.Error())
return
}
// ...
}
I would like to know how I can tell gin that the given struct must be the request body. It must not search for fields in the route parameters or queries.
So is there a way to be more explicit, e.g.
err := context.BindStructToRequestBody(&requestBody)
I am sending the data to the API like following:
{"after": {"amount": 811,"id":123,"status":"Hi"}, "key": [70]}
and i am getting following while printing :
map[after:map[amount:811 id:123 status:Hi ] key:[70]]
Is there any way to print individual field like this??
amount::800
id:123
status:Hi
The code:
package main
import (
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
"strings"
)
var (
PORT = ":8080"
)
func main() {
fmt.Println("In Main")
http.HandleFunc("/", changedData)
http.ListenAndServe(PORT, nil)
}
type Data struct {
Id int64 `json:"id"`
Amount float64 `json:"amount"`
Status string `json:"status"`
}
type mark map[string]interface{}
func changedData(w http.ResponseWriter, r *http.Request) {
fmt.Println("Coming From API")
reqBody, _ := ioutil.ReadAll(r.Body)
fmt.Println("Data coming from API ", string(reqBody))
digit := json.NewDecoder(strings.NewReader(string(reqBody)))
for digit.More() {
var result mark
err := digit.Decode(&result)
if err != nil {
if err != io.EOF {
log.Fatal(err)
}
break
}
fmt.Println("final_data ", result)
}
}
Decode to a Go type that matches the structure of the JSON document. You declared a type for the "after" field. Wrap that type with a struct to match the document.
func changedData(w http.ResponseWriter, r *http.Request) {
var v struct{ After Data }
err := json.NewDecoder(r.Body).Decode(&v)
if err != nil {
http.Error(w, "bad request", 400)
return
}
fmt.Printf("final_data: %#v", v.After)
}
Playground example.
I think you can define a struct type if you know the JSON file format or if the JSON format is predefined. As far as I know that mostly using interface{} is a way when you don't know the JSON format or there is no predefined format of the JSON. If you define a struct type and use it while unmarshaling the JSON to struct, you can access the variables by typing like data.Id or data.Status.
Here's an example code:
package main
import (
"encoding/json"
"fmt"
)
type Data struct {
AfterData After `json:"after"`
Key []int `json:"key"`
}
type After struct {
Id int64 `json:"id"`
Amount float64 `json:"amount"`
Status string `json:"status"`
}
func main() {
j := []byte(`{"after": {"amount": 811,"id":123,"status":"Hi"}, "key": [70]}`)
var data *Data
err := json.Unmarshal(j, &data)
if err != nil {
fmt.Println(err.Error())
return
}
fmt.Println(data.AfterData)
fmt.Println(data.AfterData.Id)
fmt.Println(data.AfterData.Amount)
fmt.Println(data.AfterData.Status)
}
Output will be
{123 811 Hi}
123
811
Hi
Go Playground
In the below program I'm extracting some data from an API.
It outputs a rather complex data.
When I ioutil.ReadAll(resp.Body), the result is of type []uint8.
If I try to read the results, its just a random array of integers.
However, I'm able to read it if I convert it to string using string(diskinfo)
But I want to use this in a Struct and having trouble unmarshalling.
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"reflect"
)
type ApiResults struct {
results []struct {
statement_id int `json.statement_id`
series []struct {
name string `json.name`
tags struct {
host string `json.host`
}
columns []string `json.columns`
values []interface{} `json.values`
}
}
}
func main() {
my_url := "my_url"
my_qry := fmt.Sprintf("my_query")
resp, err := http.Get(my_url + url.QueryEscape(my_qry))
if err != nil {
fmt.Printf("ERROR: %v\n", err)
} else {
fmt.Println(reflect.TypeOf(resp))
diskinfo, _ := ioutil.ReadAll(resp.Body)
fmt.Println(reflect.TypeOf((diskinfo)))
fmt.Println(diskinfo)
fmt.Println(string(diskinfo))
diskinfo_string := string(diskinfo)
data := ApiResults{}
json.Unmarshal([]byte(diskinfo_string), &data)
//fmt.Printf("Values = %v\n", data.results.series.values)
//fmt.Printf("Server = %v\n", data.results.series.tags.host)
}
}
If I view the data as a string, I get this (formatted):
{"results":[
{"statement_id":0,
"series":[
{"name":"disk",
"tags":{"host":"myServer1"},
"columns":["time","disk_size"],
"values":[["2021-07-07T07:53:32.291490387Z",1044]]},
{"name":"disk",
"tags":{"host":"myServer2"},
"columns":["time","disk_size"],
"values":[["2021-07-07T07:53:32.291490387Z",1046]]}
]}
]}
I think my Apireturn struct is also structured incorrectly because the API results have info for multiple hosts.
But first, I doubt if the data has to be sent in a different format to the struct. Once I do this, I can probably try to figure out how to read from the Struct next.
The ioutil.ReadAll already provides you the data in the type byte[]. Therefore you can just call json.Unmarshal passing it as a parameter.
import (
"encoding/json"
"io/ioutil"
"net/http"
)
func toStruct(res *http.Response) (*ApiResults, error) {
body, err := ioutil.ReadAll(res.Body)
if err != nil {
return nil, err
}
defer res.Body.Close()
data := ApiResults{}
if err := json.Unmarshal(body, &data); err != nil {
return nil, err
}
return data, nil
}
There also seems to be an issue with your struct. The correct way to use struct tags is as follows. Plus, fields need to be exported for the json tag (used by json.Umarshal) to work – starting with uppercase will do it.
type ApiResults struct {
Results []struct {
StatementId int `json:"statement_id"`
Series []struct {
Name string `json:"name"`
Tags struct {
Host string `json:"host"`
} `json:"tags"`
Columns []string `json:"columns"`
Values []interface{} `json:"values"`
} `json:"series"`
} `json:"results"`
}
It would be awesome to have a straight forward mapping from the standard library URL.Query() to an struct.
Query() returns a map like:
map[a:[aaaa] b:[bbbb] c:[cccc]]
The struct looks like:
type Thing struct {
A string
B string
C string
}
I've no idea why URL.Query returns a map with array elements inside tough. (well.. I know why but a GET is not likely to have duplicated params)
Please find below the complete example of parsing get query params directly in a golang struct and then sending the struct back as response
package main
import (
"log"
"net/http"
"encoding/json"
"github.com/gorilla/schema"
)
var decoder = schema.NewDecoder()
type EmployeeStruct struct {
MemberId string `schema:"memberId"`
ActivityType string `schema:"activityType"`
BusinessUnitCode int `schema:"businessUnitCode"`
}
func GetEmployee(w http.ResponseWriter, r *http.Request) {
var employeeStruct EmployeeStruct
err := decoder.Decode(&employeeStruct, r.URL.Query())
if err != nil {
log.Println("Error in GET parameters : ", err)
} else {
log.Println("GET parameters : ", employeeStruct)
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(employeeStruct)
}
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/GetEmployee", GetEmployee)
log.Fatal(http.ListenAndServe(":8080", mux))
}
Steps to execute & Test (Assuming you are saving above code in employee.go) :
Step 1 : go run employee.go
Step 2 : Open in browser http://localhost:8080/GetEmployee?memberId=123&activityType=Call&businessUnitCode=56
Step 3 : You should get below response in browser window
{
"MemberId": "123",
"ActivityType": "Call",
"BusinessUnitCode": 56
}
Step 4 : On console you should see below
GET parameters : {123 Call 56}
example:
filters={"reference":["docker.io/library/alpine:latest"]}
need url encode to:
filters=%7B%22reference%22%3A%5B%22docker.io%2Flibrary%2Falpine%3Alatest%22%5D%7D
and could use "github.com/gorilla/schema"
query := struct {
All bool
Filters map[string][]string `schema:"filters"`
Digests bool
Filter string
}{}
decoder := schema.NewDecoder()
decoder.Decode(&query, r.URL.Query())
As pointed out by #mh-cbon gorilla schema is the ultimate solution here.
Instead for obtaining the queryParams from the URL attribute.
func handleRequest(w http.ResponseWriter, r *http.Request) {
queryString := r.URL.Query()
//...parsing the Values -> map[string][]string
}
The approach of gorilla schema is to ship r.PostForm to the decode function.
func handleRequest(w http.ResponseWriter, r *http.Request) {
err := decoder.Decode(person, r.PostForm)
//...using reflect each struct's property can be called using
// the PostForm(url string, data url.Values) signature
fmt.Print(person.GoodJobGorilla)
}
Just parse the string to URL and after you can use the lib github.com/gorilla/schema to parse it :)
// Example to parse querystring to struct
package main
import (
"log"
"net/url"
"github.com/gorilla/schema"
)
type URLParams struct {
Code string `schema:"code"`
State string `schema:"state"`
}
func main() {
var (
params URLParams
decoder = schema.NewDecoder()
)
p := "https://www.redirect-url.com?code=CODE&state=RANDOM_ID"
u, _ := url.Parse(p)
err := decoder.Decode(¶ms, u.Query())
if err != nil {
log.Println("Error in Decode parameters : ", err)
} else {
log.Printf("Decoded parameters : %#v\n", params)
}
}
https://go.dev/play/p/CmuPhdKh6Yg
Using ggicci/httpin
Disclaimer: I'm the creator and maintainer of this package.
httpin helps you easily decoding HTTP request data from
Query parameters, e.g. ?name=john&is_member=true
Headers, e.g. Authorization: xxx
Form data, e.g. username=john&password=******
JSON/XML Body, e.g. POST {"name":"john"}
Path variables, e.g. /users/{username}
File uploads
How to use?
type ListUsersInput struct {
Page int `in:"query=page"`
PerPage int `in:"query=per_page"`
IsMember bool `in:"query=is_member"`
}
func ListUsers(rw http.ResponseWriter, r *http.Request) {
input := r.Context().Value(httpin.Input).(*ListUsersInput)
if input.IsMember {
// Do sth.
}
// Do sth.
}
httpin is:
well documented: at https://ggicci.github.io/httpin/
well tested: coverage over 98%
open integrated: with net/http, go-chi/chi, gorilla/mux, gin-gonic/gin, etc.
extensible (advanced feature): by adding your custom directives. Read httpin - custom directives for more details.
awesome mentioned: https://github.com/avelino/awesome-go#forms
You can use the graceful package of Echo.
I write some codes as an example, with self-explanatory comments
package main
import (
"log"
"github.com/labstacks/echo"
)
// Declare your struct with form: "" tag
type Employee struct {
MemberId string `form:"memberId"`
ActivityType string `form:"activityType"`
BusinessUnitCode int `form:"businessUnitCode"`
}
// Your handlers should look like this method
// Which takes an echo.Context and returns an error
func GetEmployee(ctx echo.Context) error{
var employee Employee
// With Bind, you can get the Post Body or query params from http.Request
// that is wrapped by echo.Context here
if err := ctx.Bind(&employee);err != nil {
return err
}
// now you can use your struct , e.g
return ctx.json(200, employee.MemberId)
}
// now use the handler in your main function or anywhere you need
func main() {
e := echo.New()
e.Get("/employee", GetEmployee)
log.Fatal(e.Start(":8080"))
}
If anyone is using Echo, query struct tag will be useful for this case.
Example Struct
type struct Name {
FirstName string `query:"first_name"`
LastName string `query:"last_name"`
}
Example Query Param
?first_name="shahriar"&last_name="sazid"
Code
var name Name
err := c.Bind(&name); if err != nil {
return c.String(http.StatusBadRequest, "bad request")
}
I'm writing a websocket client in Go. I'm receiving the following JSON from the server:
{"args":[{"time":"2013-05-21 16:57:17"}],"name":"send:time"}
I'm trying to access the time parameter, but just can't grasp how to reach deep into an interface type:
package main;
import "encoding/json"
import "log"
func main() {
msg := `{"args":[{"time":"2013-05-21 16:56:16", "tzs":[{"name":"GMT"}]}],"name":"send:time"}`
u := map[string]interface{}{}
err := json.Unmarshal([]byte(msg), &u)
if err != nil {
panic(err)
}
args := u["args"]
log.Println( args[0]["time"] ) // invalid notation...
}
Which obviously errors, since the notation is not right:
invalid operation: args[0] (index of type interface {})
I just can't find a way to dig into the map to grab deeply nested keys and values.
Once I can get over grabbing dynamic values, I'd like to declare these messages. How would I write a type struct to represent such complex data structs?
You may like to consider the package github.com/bitly/go-simplejson
See the doc： http://godoc.org/github.com/bitly/go-simplejson
Example:
time, err := json.Get("args").GetIndex(0).String("time")
if err != nil {
panic(err)
}
log.Println(time)
The interface{} part of the map[string]interface{} you decode into will match the type of that field. So in this case:
args.([]interface{})[0].(map[string]interface{})["time"].(string)
should return "2013-05-21 16:56:16"
However, if you know the structure of the JSON, you should try defining a struct that matches that structure and unmarshal into that. Ex:
type Time struct {
Time time.Time `json:"time"`
Timezone []TZStruct `json:"tzs"` // obv. you need to define TZStruct as well
Name string `json:"name"`
}
type TimeResponse struct {
Args []Time `json:"args"`
}
var t TimeResponse
json.Unmarshal(msg, &t)
That may not be perfect, but should give you the idea
I'm extremely new to Golang coming from Python, and have always struggled with encode/decoding json. I found gjson at https://github.com/tidwall/gjson, and it helped me immensely:
package main
import "github.com/tidwall/gjson"
func main() {
msg := (`{"args":[{"time":"2013-05-21 16:56:16", "tzs":[{"name":"GMT"}]}],"name":"send:time"}`)
value := gjson.Get(msg, "args.#.time")
println(value.String())
}
-----------------------
["2013-05-21 16:56:16"]
Additionally, I noticed the comment of how to convert into Struct
package main
import (
"encoding/json"
"fmt"
)
type msgFormat struct {
Time string `json:"time"`
Tzs msgFormatTzs `json:"tzs"`
Name string `json:"name"`
}
type msgFormatTzs struct {
TzsName string `json:"name"`
}
func main() {
msg := (`{"args":[{"time":"2013-05-21 16:56:16", "tzs":[{"name":"GMT"}]}],"name":"send:time"}`)
r, err := json.Marshal(msgFormatTzs{msg})
if err != nil {
panic(err)
}
fmt.Printf("%v", r)
}
Try on Go playground