Convert URL.Query (map of slices) to struct golang - go

It would be awesome to have a straight forward mapping from the standard library URL.Query() to an struct.
Query() returns a map like:
map[a:[aaaa] b:[bbbb] c:[cccc]]
The struct looks like:
type Thing struct {
A string
B string
C string
}
I've no idea why URL.Query returns a map with array elements inside tough. (well.. I know why but a GET is not likely to have duplicated params)

Please find below the complete example of parsing get query params directly in a golang struct and then sending the struct back as response
package main
import (
"log"
"net/http"
"encoding/json"
"github.com/gorilla/schema"
)
var decoder = schema.NewDecoder()
type EmployeeStruct struct {
MemberId string `schema:"memberId"`
ActivityType string `schema:"activityType"`
BusinessUnitCode int `schema:"businessUnitCode"`
}
func GetEmployee(w http.ResponseWriter, r *http.Request) {
var employeeStruct EmployeeStruct
err := decoder.Decode(&employeeStruct, r.URL.Query())
if err != nil {
log.Println("Error in GET parameters : ", err)
} else {
log.Println("GET parameters : ", employeeStruct)
}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(employeeStruct)
}
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/GetEmployee", GetEmployee)
log.Fatal(http.ListenAndServe(":8080", mux))
}
Steps to execute & Test (Assuming you are saving above code in employee.go) :
Step 1 : go run employee.go
Step 2 : Open in browser http://localhost:8080/GetEmployee?memberId=123&activityType=Call&businessUnitCode=56
Step 3 : You should get below response in browser window
{
"MemberId": "123",
"ActivityType": "Call",
"BusinessUnitCode": 56
}
Step 4 : On console you should see below
GET parameters : {123 Call 56}

example:
filters={"reference":["docker.io/library/alpine:latest"]}
need url encode to:
filters=%7B%22reference%22%3A%5B%22docker.io%2Flibrary%2Falpine%3Alatest%22%5D%7D
and could use "github.com/gorilla/schema"
query := struct {
All bool
Filters map[string][]string `schema:"filters"`
Digests bool
Filter string
}{}
decoder := schema.NewDecoder()
decoder.Decode(&query, r.URL.Query())

As pointed out by #mh-cbon gorilla schema is the ultimate solution here.
Instead for obtaining the queryParams from the URL attribute.
func handleRequest(w http.ResponseWriter, r *http.Request) {
queryString := r.URL.Query()
//...parsing the Values -> map[string][]string
}
The approach of gorilla schema is to ship r.PostForm to the decode function.
func handleRequest(w http.ResponseWriter, r *http.Request) {
err := decoder.Decode(person, r.PostForm)
//...using reflect each struct's property can be called using
// the PostForm(url string, data url.Values) signature
fmt.Print(person.GoodJobGorilla)
}

Just parse the string to URL and after you can use the lib github.com/gorilla/schema to parse it :)
// Example to parse querystring to struct
package main
import (
"log"
"net/url"
"github.com/gorilla/schema"
)
type URLParams struct {
Code string `schema:"code"`
State string `schema:"state"`
}
func main() {
var (
params URLParams
decoder = schema.NewDecoder()
)
p := "https://www.redirect-url.com?code=CODE&state=RANDOM_ID"
u, _ := url.Parse(p)
err := decoder.Decode(&params, u.Query())
if err != nil {
log.Println("Error in Decode parameters : ", err)
} else {
log.Printf("Decoded parameters : %#v\n", params)
}
}
https://go.dev/play/p/CmuPhdKh6Yg

Using ggicci/httpin
Disclaimer: I'm the creator and maintainer of this package.
httpin helps you easily decoding HTTP request data from
Query parameters, e.g. ?name=john&is_member=true
Headers, e.g. Authorization: xxx
Form data, e.g. username=john&password=******
JSON/XML Body, e.g. POST {"name":"john"}
Path variables, e.g. /users/{username}
File uploads
How to use?
type ListUsersInput struct {
Page int `in:"query=page"`
PerPage int `in:"query=per_page"`
IsMember bool `in:"query=is_member"`
}
func ListUsers(rw http.ResponseWriter, r *http.Request) {
input := r.Context().Value(httpin.Input).(*ListUsersInput)
if input.IsMember {
// Do sth.
}
// Do sth.
}
httpin is:
well documented: at https://ggicci.github.io/httpin/
well tested: coverage over 98%
open integrated: with net/http, go-chi/chi, gorilla/mux, gin-gonic/gin, etc.
extensible (advanced feature): by adding your custom directives. Read httpin - custom directives for more details.
awesome mentioned: https://github.com/avelino/awesome-go#forms

You can use the graceful package of Echo.
I write some codes as an example, with self-explanatory comments
package main
import (
"log"
"github.com/labstacks/echo"
)
// Declare your struct with form: "" tag
type Employee struct {
MemberId string `form:"memberId"`
ActivityType string `form:"activityType"`
BusinessUnitCode int `form:"businessUnitCode"`
}
// Your handlers should look like this method
// Which takes an echo.Context and returns an error
func GetEmployee(ctx echo.Context) error{
var employee Employee
// With Bind, you can get the Post Body or query params from http.Request
// that is wrapped by echo.Context here
if err := ctx.Bind(&employee);err != nil {
return err
}
// now you can use your struct , e.g
return ctx.json(200, employee.MemberId)
}
// now use the handler in your main function or anywhere you need
func main() {
e := echo.New()
e.Get("/employee", GetEmployee)
log.Fatal(e.Start(":8080"))
}

If anyone is using Echo, query struct tag will be useful for this case.
Example Struct
type struct Name {
FirstName string `query:"first_name"`
LastName string `query:"last_name"`
}
Example Query Param
?first_name="shahriar"&last_name="sazid"
Code
var name Name
err := c.Bind(&name); if err != nil {
return c.String(http.StatusBadRequest, "bad request")
}

Related

How to tell Gin explicitly to bind a struct to the request body and nothing else?

I'm using the gin framework and the following code works fine for me
import (
"github.com/gin-gonic/gin"
"net/http"
)
type RequestBody struct {
MyRequiredField string `json:"myRequiredField" binding:"required"`
}
func Handle(context *gin.Context) {
var requestBody RequestBody
err := context.ShouldBindJSON(&requestBody)
if err != nil {
context.JSON(http.StatusBadRequest, err.Error())
return
}
// ...
}
I would like to know how I can tell gin that the given struct must be the request body. It must not search for fields in the route parameters or queries.
So is there a way to be more explicit, e.g.
err := context.BindStructToRequestBody(&requestBody)

How do I bind a date string to a struct?

type TestModel struct {
Date time.Time `json:"date" form:"date" gorm:"index"`
gorm.Model
}
i'm using echo framwork, and
I have a struct like the one above, and I get string data like '2021-09-27' , how can I bind it to the struct?
func CreateDiary(c echo.Context) error {
var getData model.TestModel
if err := (&echo.DefaultBinder{}).BindBody(c, &getData); err != nil {
fmt.Print(err.Error())
}
return c.JSON(200, getData)
}
When I code like this, I get the following error:
code=400, message=parsing time "2021-09-27" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "T", internal=parsing time "2021-09-27" as "2006-01-02T15:04:05Z07:00": cannot parse "" as "T"
I'm a golang beginner, can you show me a simple example??, please.
i'm using echo framwork
Type CustomTime time.Time
func (ct *CustomTime) UnmarshalParam(param string) error {
t, err := time.Parse(`2006-01-02`, param)
if err != nil {
return err
}
*ct = CustomTime(t)
return nil
}
ref: https://github.com/labstack/echo/issues/1571
Here is list of available tags used in echo. If you want to parse from body, then use json
query - source is request query parameters.
param - source is route path parameter.
header - source is header parameter.
form - source is form. Values are taken from query and request body. Uses Go standard library form parsing.
json - source is request body. Uses Go json package for unmarshalling.
xml - source is request body. Uses Go xml package for unmarshalling.
You need to wrap time.Time into custom struct and then implement json.Marshaler and json.Unmarshaler interfaces
Example
package main
import (
"fmt"
"strings"
"time"
"github.com/labstack/echo/v4"
)
type CustomTime struct {
time.Time
}
type TestModel struct {
Date CustomTime `json:"date"`
}
func (t CustomTime) MarshalJSON() ([]byte, error) {
date := t.Time.Format("2006-01-02")
date = fmt.Sprintf(`"%s"`, date)
return []byte(date), nil
}
func (t *CustomTime) UnmarshalJSON(b []byte) (err error) {
s := strings.Trim(string(b), "\"")
date, err := time.Parse("2006-01-02", s)
if err != nil {
return err
}
t.Time = date
return
}
func main() {
e := echo.New()
e.POST("/test", CreateDiary)
e.Logger.Fatal(e.Start(":1323"))
}
func CreateDiary(c echo.Context) error {
var getData TestModel
if err := (&echo.DefaultBinder{}).BindBody(c, &getData); err != nil {
fmt.Print(err.Error())
}
return c.JSON(200, getData)
}
test
curl -X POST http://localhost:1323/test -H 'Content-Type: application/json' -d '{"date":"2021-09-27"}'

Validation in GO "ozzo-validation"

I'm nob in GO :) just try to create simple crud throw it using gin and plugin called ozzo-validation
My code:
package models
import (
validation "github.com/go-ozzo/ozzo-validation"
"gorm.io/gorm"
)
type Post struct {
gorm.Model
Name string `gorm:"type:varchar(255);"`
Desc string `gorm:"type:varchar(500);"`
}
type PostData struct {
Name string `json:"name"`
Desc string `json:"desc"`
}
func (p PostData) Validate() error {
return validation.ValidateStruct(&p,
validation.Field(&p.Name, validation.Required, validation.Length(5, 20)),
validation.Field(&p.Desc, validation.Required),
)
}
PostController:
package controllers
import (
"curd/app/models"
"fmt"
"github.com/gin-gonic/gin"
)
func Store(c *gin.Context) {
// Validate Input
var post models.PostData
err := post.Validate()
fmt.Println(err)
}
{
"name": "sdfsdfsfsdf"
}
The problem is once I submit the above JSON data from the postman the validation gives me this in terminal :
desc: cannot be blank; name: cannot be blank.
As noted in the comments, you need to decode the data from the HTTP request into the struct before you can perform validation. The validation errors you're seeing are a product of calling Validate() on a fresh instance (with zero values in every field) of your Post struct. Try this.
func Store(c *gin.Context) {
var post models.PostData
// This will infer what binder to use depending on the content-type header.
if err := c.ShouldBind(&post); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
// Validate Input
err := post.Validate()
fmt.Println(err)
}

How to dynamically register type in go lang

I'm newbie in go, actually I want to make an API with go those have dynamic route like I've done with Lumen, so the main func something like this :
func main() {
posts = append(posts, Post{ID: "1", Title: "My first post", Body: "This is the content of my first post"})
posts = append(posts, Post{ID: "2", Title: "My second post", Body: "This is the content of my second post"})
db, err = gorm.Open(dbclass.DbDriver, dbclass.DataSourceName)
if err != nil {
log.Println("Connection Failed to Open")
} else {
log.Println("Connection Established")
}
router := mux.NewRouter()
router.HandleFunc("/{model}", getAll).Methods("GET")
router.HandleFunc("/{model}", create).Methods("POST")
router.HandleFunc("/{model}/{id}", getById).Methods("GET")
router.HandleFunc("/{model}/{id}", update).Methods("PUT")
router.HandleFunc("/{model}/{id}", delete).Methods("DELETE")
http.ListenAndServe(":8000", router)
}
and I store the mcustm.go model file inside Models folder.
So when we want to get All of any model, we just call (example in postman) something like this :
localhost:8000/mcustm
that will call the getAll function then pass the mcustm parameter as a model name, and the getAll function something like this:
func getAll(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
vars := mux.Vars(r)
model = vars["model"] // the model slug
json.NewEncoder(w).Encode(model)
}
but that's not working at all, because the model are known as String instead of struct of Mcustm.
I've googling to solve that, and I found maybe some good solution using this approach:
package main
import (
"fmt"
"reflect"
)
var typeRegistry = make(map[string]reflect.Type)
func registerType(typedNil interface{}) {
t := reflect.TypeOf(typedNil).Elem()
typeRegistry[t.PkgPath() + "." + t.Name()] = t
}
type MyString string
type myString string
func init() {
registerType((*MyString)(nil))
registerType((*myString)(nil))
// ...
}
func makeInstance(name string) interface{} {
return reflect.New(typeRegistry[name]).Elem().Interface()
}
func main() {
for k := range typeRegistry {
fmt.Println(k)
}
fmt.Printf("%T\n", makeInstance("main.MyString"))
fmt.Printf("%T\n", makeInstance("main.myString"))
}
but still not solving my problem, because on init() function we have to manually type for registerType the struct. If I have an array string of struct name, how can I registerType dynamically using that array ?
sory for my bad English

jsonrpc server to accept requested lowercase method names (intended for uppercase registered services)

I'm trying to write a jsonrpc server that will accept requested method names in lower case, eg Arith.multiply, and correctly route them to the corresponding uppercase methed, e.g Arith.Multiply. Is this possible?
P.S. It's lightweight clone of production server for testing, the API is fixed, including the lowercase method names, so I can't change requested method names to uppercase.
package main
import (
"log"
"net/http"
"github.com/gorilla/mux"
"github.com/gorilla/rpc"
"github.com/gorilla/rpc/json"
)
type Args struct {
A, B int
}
type Arith int
type Result int
func (t *Arith) Multiply(r *http.Request, args *Args, result *Result) error {
log.Printf("Multiplying %d with %d\n", args.A, args.B)
*result = Result(args.A * args.B)
return nil
}
func main() {
s := rpc.NewServer()
s.RegisterCodec(json.NewCodec(), "application/json")
s.RegisterCodec(json.NewCodec(), "application/json;charset=UTF-8")
arith := new(Arith)
s.RegisterService(arith, "")
r := mux.NewRouter()
r.Handle("/rpc", s)
http.ListenAndServe(":1234", r)
}
It appears you can sneak something into a custom Codec to direct the lowercase method to the correct uppercase one. Compose a CodecRequest of the gorilla/rpc/json implementation and you can continue to use all of the gorilla machinery to process the request.
Working example below. It looks long but it's all comments.
package main
import (
"fmt"
"log"
"net/http"
"strings"
"unicode"
"unicode/utf8"
"github.com/gorilla/mux"
"github.com/gorilla/rpc"
"github.com/gorilla/rpc/json"
)
type Args struct {
A, B int
}
type Arith int
type Result int
func (t *Arith) Multiply(r *http.Request, args *Args, result *Result) error {
log.Printf("Multiplying %d with %d\n", args.A, args.B)
*result = Result(args.A * args.B)
return nil
}
// UpCodec creates a CodecRequest to process each request.
type UpCodec struct {
}
// NewUpCodec returns a new UpCodec.
func NewUpCodec() *UpCodec {
return &UpCodec{}
}
// NewRequest returns a new CodecRequest of type UpCodecRequest.
func (c *UpCodec) NewRequest(r *http.Request) rpc.CodecRequest {
outerCR := &UpCodecRequest{} // Our custom CR
jsonC := json.NewCodec() // json Codec to create json CR
innerCR := jsonC.NewRequest(r) // create the json CR, sort of.
// NOTE - innerCR is of the interface type rpc.CodecRequest.
// Because innerCR is of the rpc.CR interface type, we need a
// type assertion in order to assign it to our struct field's type.
// We defined the source of the interface implementation here, so
// we can be confident that innerCR will be of the correct underlying type
outerCR.CodecRequest = innerCR.(*json.CodecRequest)
return outerCR
}
// UpCodecRequest decodes and encodes a single request. UpCodecRequest
// implements gorilla/rpc.CodecRequest interface primarily by embedding
// the CodecRequest from gorilla/rpc/json. By selectively adding
// CodecRequest methods to UpCodecRequest, we can modify that behaviour
// while maintaining all the other remaining CodecRequest methods from
// gorilla's rpc/json implementation
type UpCodecRequest struct {
*json.CodecRequest
}
// Method returns the decoded method as a string of the form "Service.Method"
// after checking for, and correcting a lowercase method name
// By being of lower depth in the struct , Method will replace the implementation
// of Method() on the embedded CodecRequest. Because the request data is part
// of the embedded json.CodecRequest, and unexported, we have to get the
// requested method name via the embedded CR's own method Method().
// Essentially, this just intercepts the return value from the embedded
// gorilla/rpc/json.CodecRequest.Method(), checks/modifies it, and passes it
// on to the calling rpc server.
func (c *UpCodecRequest) Method() (string, error) {
m, err := c.CodecRequest.Method()
if len(m) > 1 && err == nil {
parts := strings.Split(m, ".")
service, method := parts[0], parts[1]
r, n := utf8.DecodeRuneInString(method) // get the first rune, and it's length
if unicode.IsLower(r) {
upMethod := service + "." + string(unicode.ToUpper(r)) + method[n:]
log.Printf("lowercase method %s requested: treated as %s\n", m, upMethod)
return upMethod, err
}
}
return m, err
}
func main() {
s := rpc.NewServer()
// Register our own Codec
s.RegisterCodec(NewUpCodec(), "application/json")
s.RegisterCodec(NewUpCodec(), "application/json;charset=UTF-8")
arith := new(Arith)
s.RegisterService(arith, "")
r := mux.NewRouter()
r.Handle("/rpc", s)
fmt.Println(http.ListenAndServe(":1234", r))
}
Call the method via:
curl -X POST -H "Content-Type: application/json" -d '{"id": 1, "method": "Arith.multiply", "params": [{"A": 10, "B": 30}]}' 127.0.0.1:1234/rpc

Resources