How to decode query parameter in golang - go

i have parameter
id_user
phone_number
I want to decode to my struct
type User struct{
IDUser int `json:"id_user"`
PhoneNumber string `json:"phone_number"`
}
is it possible to decode into struct? I use gorilla schema. My Code:
func User(w http.ResponseWriter, r *http.Request){
var decoder = schema.NewDecoder()
var user User
if err := r.ParseForm(); err != nil {
fmt.Println(err)
}
err := decoder.Decode(&user, r.PostForm)
if err != nil {
fmt.Println(err)
}
respBody, err := json.Marshal(user)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write(respBody)
}
i input id_user = 1 and phone_number = qwerty. But the result is id_user = 0 and phone_number = "".

If your mentioned two fields are query params you can directly read it like this way:
func User(w http.ResponseWriter, r *http.Request) {
idUser := r.URL.Query().Get("id_user")
phoneNumber := r.URL.Query().Get("phone_number")
var user User
id, err := strconv.Atoi(idUser)
if err != nil {
fmt.Println("error converting string to int")
return
}
user.IDUser = id
user.PhoneNumber = phoneNumber
respBody, _ := json.Marshal(user)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
w.Write(respBody)
}
Otherwise you can directly pass the User struct in api payload and directly do the payLoad decoding like this
if err := json.NewDecoder(r.Body).Decode(&user); err != nil {
fmt.Println("error decoding api payload")
return
}

I think you should fix your tags from this:
type User struct{
IDUser int `json:id_user`
PhoneNumber string `json:phone_number`
}
to this:
type User struct{
IDUser int `json:"id_user"`
PhoneNumber string `json:"phone_number"`
}
So, you should use quotes in tag names.

If you want to decode them directly into a structure without getting the values of the fields one by one then you can use github.com/gorilla/schema. Refer to this question.

This answer may not be suitable for the question since the op was seeking help of using gorilla/schema package.
However, if someone were looking for a package to decode HTTP query params into a struct in Go, I will recommend an awesome package here:
ggicci/httpin
Disclaimer: I'm the creator and maintainer of this package.
httpin helps you easily decoding HTTP request data from
Query parameters, e.g. ?name=john&is_member=true
Headers, e.g. Authorization: xxx
Form data, e.g. username=john&password=******
JSON/XML Body, e.g. POST {"name":"john"}
Path variables, e.g. /users/{username}
File uploads
How to use?
type ListUsersInput struct {
Page int `in:"query=page"`
PerPage int `in:"query=per_page"`
IsMember bool `in:"query=is_member"`
}
func ListUsers(rw http.ResponseWriter, r *http.Request) {
input := r.Context().Value(httpin.Input).(*ListUsersInput)
if input.IsMember {
// Do sth.
}
// Do sth.
}
httpin is:
well documented: at https://ggicci.github.io/httpin/
well tested: coverage over 98%
open integrated: with net/http, go-chi/chi, gorilla/mux, gin-gonic/gin, etc.
extensible (advanced feature): by adding your custom directives. Read httpin - custom directives for more details.
awesome mentioned: https://github.com/avelino/awesome-go#forms

Related

How to extract field from API response struct to another struct field while changing datatype?

I'm using range to loop through an array of structs to extract data which will be used as a URL parameter for my API calls. Within this loop, I'm trying to push response data from one struct to another.
I'm able to get everything working, except for moving data from one struct to another, but not entirely sure how to solve for the errors I keep getting. I've tried multiple methods and seem to be stuck in the mud here for something I don't consider to be too hard, until now... In my code I'm using the append method but I'm not so sure that might be the correct way to proceed.
Presenting my code:
models.go
//Here is my existing struct, with populated data that I get from a CSV
type TravelItenaries struct {
Origin string
Destination string
Flight_num string
Origin_latitude string
Origin_longitude string
Destination_latitude string
Destination_longitude string
Origin_weather string
Destination_weather string
Coordinates_ori string
Coordinates_dest string
Temp_c_ori string
Temp_f_ori string
Temp_c_dest string
Temp_f_dest string
}
//Here is the response data that I'm expected to get from my API calls.
//I'm trying to "push" Temp_c_dest and Temp_f_dest data into TravelItenaries.Temp_f_dest and TravelItenaries.Temp_c_dest
//While also changing the data types to fit above.
type Response struct {
Current struct {
LastUpdatedEpoch int `json:"last_updated_epoch"`
LastUpdated string `json:"last_updated"`
Temp_c_dest float64 `json:"temp_c"`
Temp_c_dest float64 `json:"temp_f"`
IsDay int `json:"is_day"`
} `json:"current"
}
weather.go
func (s *Server) getWeather(w http.ResponseWriter, r *http.Request) {
// open file
f, err := os.Open("challenge_dataset.csv")
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
// remember to close the file at the end of the program
defer f.Close()
// read csv values using csv.Reader
csvReader := csv.NewReader(f)
data, err := csvReader.ReadAll()
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
// convert records to array of structs
travelItenaries := createTravelItenaries(data)
// remove duplicate flight records
cleanTravelItenaries:= remDupKeys(travelItenaries)
// set up params for API get request
params := url.Values{
"key": []string{"xxx"},
"q": []string{""},
}
// Construct URL for API request
u := &url.URL{
Scheme: "https",
Host: "api.weatherapi.com",
Path: "/v1/current.json",
RawQuery: params.Encode(),
}
client := &http.Client{}
// Will need this to populate the params using a range over a struct
values := u.Query()
// loop through cleaned data set
for _, service := range cleanTravelItenaries {
// dynamically acquire data from struct to pass as parameter
values.Set("q", service.Coordinates_dest)
u.RawQuery = values.Encode()
req, err := http.NewRequest("GET", u.String(), nil)
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
resp, err := client.Do(req)
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
body, err := ioutil.ReadAll(resp.Body)
// create empty struct to parse response data with using Inmarshal
var responseData models.Response
json.Unmarshal(body, &responseData)
// Here is the issue, I don't think append might be the correct procedure here?
// I simply just need to pass this response data to my already existing struct
service.Temp_c_dest = append(responseData.Current.Temp_c_dest , cleanTravelItenaries )
service.Temp_f_dest = append(responseData.Current.Temp_f_dest , cleanTravelItenaries )
}
}
The errors I get are related to both append statements at the end of the range function.
first argument to append must be slice; have float64
first argument to append must be slice; have float64
for both append methods.
Also, note how type TravelItenaries struct uses string type for:
Temp_c_dest string
Temp_f_dest string
Hence why I also need to do some field type conversion from Float64 to string.
How can I extract the fields Temp_c_dest and Temp_f_dest from API response struct to TravelItenaries struct fields while changing datatypes?
EDIT:
I've managed to get this somewhat working, but only inside the for loop. The data is not being saved outside the function.
service.Temp_f_dest = strconv.FormatFloat(responseData.Current.Temp_f_dest, 'g', -1, 64)
service.Temp_c_dest = strconv.FormatFloat(responseData.Current.Temp_c_dest, 'g', -1, 64)

how do i validate the body structure of rest api request in golang

I am trying to ensure the body of a post request for example contains exact structure of the body and if not ahould throw an error
for example i have the following function
func UpdatePassword(c *fiber.Ctx) error {
type UpdatePasswordData struct {
Password string `json:"password" form:"password"`
NewPassword string `json:"new_password" form:"new_password"`
NewPasswordConfirm string `json:"new_password_confirm" form:"new_password_confirm"`
}
data := UpdatePasswordData{}
if err := c.BodyParser(&data); err != nil {
return err
}
var user models.User
if data.NewPassword != data.NewPasswordConfirm {
c.Status(400)
return c.JSON(fiber.Map{
"message": "passwords do not match",
})
}
email, _ := middlewares.GetUserEmail(c)
newPassword := models.HashPassword(data.NewPassword)
database.DB.Model(&user).Select("Password").Where("email = ?", email).Updates(map[string]interface{}{"Password": newPassword})
return c.JSON(user)
}
the POST request should be looking for body with this structure
{
"password": "oldpassword",
"new_password": "newpassword",
"new_password_confirm": "newpassword",
}
but currently this endpoint accepts body that does not have this exact structure. So how do i enforce the structure in the body of request, so that if structure does not match, i throw an error?
do not like gin, fiber has not builtin validate package
use go-playground/validator
go get github.com/go-playground/validator
example
type UpdatePasswordData struct {
Password string `json:"password" validate:"required,min=8,max=32"`
NewPassword string `json:"new_password" validate:"required,min=8,max=32"`
NewPasswordConfirm string `json:"new_password_confirm" validate:"eqfield=NewPassword"`
}
func UpdatePassword(c *fiber.Ctx) error {
var body UpdatePasswordData
if err := c.BodyParser(&body); err != nil {
return err
}
validate := validator.New()
if err := validate.Struct(body); err != nil {
return err
}
// do others
// get current user, check password == hash(body.password)
// save new passworld
}
or you can see fiber office docs https://docs.gofiber.io/guide/validation#validator-package
We can use struct tag
`validate:"required"`
to ensure that all the mandatory fields are there in the request payload.
Moreover we can validate the fields with the provided tags of the validator package and for additional validations we can implement custom validators and register them like this:
validate := validator.New()
validate.RegisterValidation("password-validator", PasswordValidator)

How to serialize `LastEvaluatedKey` from DynamoDB's Golang SDK?

When working with DynamoDB in Golang, if a call to query has more results, it will set LastEvaluatedKey on the QueryOutput, which you can then pass in to your next call to query as ExclusiveStartKey to pick up where you left off.
This works great when the values stay in Golang. However, I am writing a paginated API endpoint, so I would like to serialize this key so I can hand it back to the client as a pagination token. Something like this, where something is the magic package that does what I want:
type GetDomainObjectsResponse struct {
Items []MyDomainObject `json:"items"`
NextToken string `json:"next_token"`
}
func GetDomainObjects(w http.ResponseWriter, req *http.Request) {
// ... parse query params, set up dynamoIn ...
dynamoIn.ExclusiveStartKey = something.Decode(params.NextToken)
dynamoOut, _ := db.Query(dynamoIn)
response := GetDomainObjectsResponse{}
dynamodbattribute.UnmarshalListOfMaps(dynamoOut.Items, &response.Items)
response.NextToken := something.Encode(dynamoOut.LastEvaluatedKey)
// ... marshal and write the response ...
}
(please forgive any typos in the above, it's a toy version of the code I whipped up quickly to isolate the issue)
Because I'll need to support several endpoints with different search patterns, I would love a way to generate pagination tokens that doesn't depend on the specific search key.
The trouble is, I haven't found a clean and generic way to serialize the LastEvaluatedKey. You can marshal it directly to JSON (and then e.g. base64 encode it to get a token), but doing so is not reversible. LastEvaluatedKey is a map[string]types.AttributeValue, and types.AttributeValue is an interface, so while the json encoder can read it, it can't write it.
For example, the following code panics with panic: json: cannot unmarshal object into Go value of type types.AttributeValue.
lastEvaluatedKey := map[string]types.AttributeValue{
"year": &types.AttributeValueMemberN{Value: "1993"},
"title": &types.AttributeValueMemberS{Value: "Benny & Joon"},
}
bytes, err := json.Marshal(lastEvaluatedKey)
if err != nil {
panic(err)
}
decoded := map[string]types.AttributeValue{}
err = json.Unmarshal(bytes, &decoded)
if err != nil {
panic(err)
}
What I would love would be a way to use the DynamoDB-flavored JSON directly, like what you get when you run aws dynamodb query on the CLI. Unfortunately the golang SDK doesn't support this.
I suppose I could write my own serializer / deserializer for the AttributeValue types, but that's more effort than this project deserves.
Has anyone found a generic way to do this?
OK, I figured something out.
type GetDomainObjectsResponse struct {
Items []MyDomainObject `json:"items"`
NextToken string `json:"next_token"`
}
func GetDomainObjects(w http.ResponseWriter, req *http.Request) {
// ... parse query params, set up dynamoIn ...
eskMap := map[string]string{}
json.Unmarshal(params.NextToken, &eskMap)
esk, _ = dynamodbattribute.MarshalMap(eskMap)
dynamoIn.ExclusiveStartKey = esk
dynamoOut, _ := db.Query(dynamoIn)
response := GetDomainObjectsResponse{}
dynamodbattribute.UnmarshalListOfMaps(dynamoOut.Items, &response.Items)
lek := map[string]string{}
dynamodbattribute.UnmarshalMap(dynamoOut.LastEvaluatedKey, &lek)
response.NextToken := json.Marshal(lek)
// ... marshal and write the response ...
}
(again this is my real solution hastily transferred back to the toy problem, so please forgive any typos)
As #buraksurdar pointed out, attributevalue.Unmarshal takes an inteface{}. Turns out in addition to a concrete type, you can pass in a map[string]string, and it just works.
I believe this will NOT work if the AttributeValue is not flat, so this isn't a general solution [citation needed]. But my understanding is the LastEvaluatedKey returned from a call to Query will always be flat, so it works for this usecase.
Inspired by Dan, here is a solution to serialize and deserialize to/from base64
package dynamodb_helpers
import (
"encoding/base64"
"encoding/json"
"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
func Serialize(input map[string]types.AttributeValue) (*string, error) {
var inputMap map[string]interface{}
err := attributevalue.UnmarshalMap(input, &inputMap)
if err != nil {
return nil, err
}
bytesJSON, err := json.Marshal(inputMap)
if err != nil {
return nil, err
}
output := base64.StdEncoding.EncodeToString(bytesJSON)
return &output, nil
}
func Deserialize(input string) (map[string]types.AttributeValue, error) {
bytesJSON, err := base64.StdEncoding.DecodeString(input)
if err != nil {
return nil, err
}
outputJSON := map[string]interface{}{}
err = json.Unmarshal(bytesJSON, &outputJSON)
if err != nil {
return nil, err
}
return attributevalue.MarshalMap(outputJSON)
}

Bind request method POST

I have a problem with binding my request, because there are a lot of parameters, so I used struct containing param.
package api
import (
"github.com/labstack/echo/v4"
"net/http"
"trains-api/domain/models"
"trains-api/domain/services"
)
type reqCreate struct {
RequestNotifi models.ResquestCreateNotifi
}
func CreateNotification (c echo.Context) error {
req := reqCreate{}
if err := c.Bind(req); err != nil {
return c.JSON(http.StatusNotFound, err)
}
}
package models
type RequestCreateNotifi struct {
Name_param1 string `db:"Name_param1"`
Name_param2 string `db:"Name_param2"`
....
Name_param_n string `db:"Name_paramN"`
}
error at if err := c.Bind(req); err != nil
r = {interface {} | string } "reflect: Elem of invalid type"
You need to set the JSON equivalent of each field in the model like so:
package models
type RequestCreateNotifi struct {
Name_param1 string `json:"name_param1" db:"Name_param1"`
Name_param2 string `json:"name_param2" db:"Name_param2"`
....
Name_param_n string `json:"name_param_n" db:"Name_param n"`
}
This json field specifies how the field is represented in the request so it can bind it to the correct value.
You need to add the pointer
req := reqCreate{}
if err := c.Bind(&req); err != nil {
return c.JSON(http.StatusNotFound, err)
}
Unfortunately you can't bind automatically query parameter using Post methode for security reasons according to issue#1670, the way to do it is using echo.QueryParamsBinder
type Query struct {
Param1 string `query:"param1"`
Param2 string `query:"param2"`
}
...
query := new(Query)
err := echo.QueryParamsBinder(ctx).String("param1", &query.Param1).String("param2", &query.Param2).BindError()
...

Golang Rest api data not showing in postman

I am facing an issue where i have made an api in Go every thing work fine but i am not getting data in postman. When i print the data in logs i am getting the data properly but it is showing blank data in postman.
authorizeModel.go
func GetSkillList() map[string]interface{} {
db := GetDB()
var (
// id int
skillName string
)
type SkillList struct {
name string
}
skillList := SkillList{}
skillArr := []SkillList{}
rows, err := db.Query("select DISTINCT(name) as name from skills where company_id IN ('2') and name != 'Skill Needed' order by name")
if err != nil {
log.Fatal(err)
}
defer rows.Close()
for rows.Next() {
err := rows.Scan(&skillName)
if err != nil {
log.Fatal(err)
}
skillList.name = skillName
skillArr = append(skillArr, skillList)
}
response := u.Message(true, "Skill list retrieved successfully")
response["data"] = skillArr
log.Println(skillArr)
response["authorization"] = false
return response
}
authController.go
var SkillTagList = func(w http.ResponseWriter, r *http.Request) {
resp := models.GetSkillList()
u.Respond(w, resp)
}
routes.go
router.HandleFunc("/api/v1/authorize/skillTagList", controllers.SkillTagList).Methods("POST")
If you see authorizeModel.go i have printed my data in logs i am getting that data successfully in logs. But see the postman screenshot below.
You have to rename name to Name
I'm not sure what is u.Respond(), so I will assume it's a helper function of some framework that you are using, and I will assume u.Respond() is internally using json.Marshal.
If your struct has unexported fields(fields name starting with lowercase letter, in your case name), json.Marshal cannot access those field, and the result won't have name field. That is why you are getting empty objects in JSON.

Resources