go version: 1.19
gin version (or commit ref): 1.8.1
operating system: Ubuntu
I have a saas project which is based upon Rest APIs. All apis are developed in GO using gin package. When the user logs in then I set current user details in the request context so that I can access these details furthere to display some data. However I had a case in which 2 requests hits in parallel & the context values for the 1st request are override with the context values in the 2nd request. Due to this, my data is displaying wrong.
package main
import (
"fmt"
"strings"
"github.com/gin-gonic/gin"
"github.com/golang-jwt/jwt"
)
func main() {
g := gin.Default()
g.Use(ParseJWTToken)
g.GET("/hello/:name", hello)
g.Run(":9000")
}
func hello(c *gin.Context) {
c.Keys = make(map[string]interface{})
c.Keys["current_user_id"] = 10
c.Keys["current_user_name"] = c.Param("name")
fmt.Println(c.Keys)
c.String(200, "Hello %s", c.Param("name"))
}
var role, userName string
var userId float64
func ParseJWTToken(c *gin.Context) {
merchantDatabase := make(map[string]interface{})
if values, _ := c.Request.Header["Authorization"]; len(values) > 0 {
bearer := strings.Split(c.Request.Header["Authorization"][0], "Bearer")
bearerToken := strings.TrimSpace(bearer[1])
var userAgent string
var userAgentCheck bool
if values, _ := c.Request.Header["User-Agent"]; len(values) > 0 {
userAgent = values[0]
}
_ = config.InitKeys()
token, err := jwt.Parse(bearerToken, func(token *jwt.Token) (interface{}, error) {
return config.SignKey, nil
})
if err != nil {
c.Abort()
return
}
if !token.Valid {
c.Abort()
return
}
if len(token.Claims.(jwt.MapClaims)) > 0 {
for key, claim := range token.Claims.(jwt.MapClaims) {
if key == "user_agent" {
if claim == userAgent {
userAgentCheck = true
}
}
if key == "role" {
role = claim.(string)
}
if key == "id" {
userId = claim.(float64)
}
if key == "name" {
userName = claim.(string)
}
}
}
merchantDatabase["userid"] = userId
merchantDatabase["role"] = role
merchantDatabase["username"] = userName
c.Keys = merchantDatabase
if userAgentCheck {
c.Next()
} else {
c.Abort()
return
}
} else {
c.Abort()
return
}
}
This issue is not produced every time for parallel requests.
How can I fix that ?
I have used global variables for the details that were overridden. Declaring these inside the middleware fixed the issue. Find complete thread here: https://github.com/gin-gonic/gin/issues/3437
Related
I am working on an existing application which is written in Go using framework such as gin, middleware. This application uses https://pkg.go.dev/log for logging.
I am trying to add a request id to the log for the API call trace.
main.go
// Creates a router without any middleware by default
r := gin.New()
r.Use(middleware.RequestID())
middleware.go
func (m *Middleware) CheckApiToken(allowWithoutAccessKey bool, validateTonce ...bool) gin.HandlerFunc {
return func(c *gin.Context) {
// Validate
.....
.....
logger.InitializeContext(c)
c.Next()
}
}
}
//RequestID is a middleware that injects a 'RequestID' into the context and header of each request.
func (m *Middleware) RequestID() gin.HandlerFunc {
return func(c *gin.Context) {
xRequestID := uuid.NewV4().String()
c.Request.Header.Set(logger.XRequestIDKey, xRequestID)
fmt.Printf("[GIN-debug] %s [%s] - \"%s %s\"\n", time.Now().Format(time.RFC3339), xRequestID, c.Request.Method, c.Request.URL.Path)
c.Next()
}
}
logger.go
const (
XRequestIDKey = "X-Request-ID"
)
var (
infoLogger *log.Logger
errorLogger *log.Logger
context *gin.Context
)
func init() {
infoLogger = log.New(os.Stdout, "", 0)
errorLogger = log.New(os.Stderr, "", 0)
}
// InitializeContext initialize golbal gin context to logger
func InitializeContext(c *gin.Context) {
context = c
}
//Check if the request id present in the context.
func getRequestID() interface{} {
if context != nil {
if context.Request != nil {
requestID := context.Request.Header.Get(XRequestIDKey)
if requestID != "" {
return requestID
}
}
}
return ""
}
func Info(entry Input) {
infoLogger.Println(getJSON(getRequestID(), msg))
}
This does not work in multi-threaded environment. How do I fix this solution to fix this in multi-threaded environment.
You cannot save the context in a global variable. Context is by definition local to that execution, and at any given moment, there will be multiple of them.
You can store the generated ID in the gin context:
func (m *Middleware) RequestID() gin.HandlerFunc {
return func(c *gin.Context) {
xRequestID := uuid.NewV4().String()
c.Set("requestId",xRequestID)
fmt.Printf("[GIN-debug] %s [%s] - \"%s %s\"\n", time.Now().Format(time.RFC3339), xRequestID, c.Request.Method, c.Request.URL.Path)
c.Next()
}
}
Then you can use the ID stored in the context with the custom log formatter:
router.Use(gin.LoggerWithFormatter(func(param gin.LogFormatterParams) string {
return fmt.Sprintf("%s ...",
param.Keys["requestId"],
...
)
}))
Or if you need to use a different logging library, you can write a wrapper:
func LogInfo(ctx *gin.Context,msg string) {
id:=ctx.Get("requestId")
// Log msg with ID
}
Many log libraries offer methods to get a logger with some parameters already set. For instance, if you use zerolog:
logger:=log.Info().With().Str("requestId",ctx.Get("requestId")).Logger()
logger.Info().Msg("This log msg will contain the request id")
I'm and using Go to setup my own API. I'm kind of stuck right now because of how I wrote the code to dynamically create/apply the query filter. It works but I'm wondering if there is a better way to do the scenario below.
For example, I have a search page with check boxes (1 for email and 1 for name) to narrow the search.
// If I checked the email, the query would be like this
query findOne() {
user(func: type(user)) #filter(eq(email, "john.doe#email.com")) {
name
email
age
home_address
}
}
// If name checkedbox is also checked, it would be like this
query findOne() {
user(func: type(user)) #filter(eq(email, "john") OR eq(name, "john")) {
name
email
age
home_address
}
}
This is what I got so far and I think there is a better way to do this:
func (s *Service) GetUser(email, name string) (*Users, error) {
c := db.NewClient()
defer db.Close()
var u Users
var filter string
if email != "" && mobileNumber != "" {
filter = fmt.Sprintf(`eq(email, "%s") OR eq(mobileNumber, "%s")`, email, mobileNumber)
} else if email != "" && mobileNumber == "" {
filter = fmt.Sprintf(`eq(email, "%s")`, email)
} else if email == "" && mobileNumber != "" {
filter = fmt.Sprintf(`eq(mobileNumber, "%s")`, mobileNumber)
}
q := fmt.Sprintf(`query findOne() {
users(func: type("user")) #filter(%s) {
name
email
home_address
contact_number
}
}`, filter)
ctx := context.Background()
res, err := c.NewTxn().Query(ctx, q)
if err != nil {
return nil, err
}
if err = json.Unmarshal(res.Json, &u); err != nil {
return nil, err
}
return &u, nil
}
Is there a better way to do this instead of creating long condition?
Here is the reflection version of it. Basically it enumerates fields, gets the value and field names to build an array of string based on them. Please not that i'm not well experienced it might also require some improvements.
import (
"fmt"
"reflect"
"strings"
)
type User struct {
Id int
FullName string
Phone string
Mail string
}
func main() {
u := &User{Id: 10, FullName: "John", Mail: "john#mail"}
u2 := struct {
id int
name string
}{10, "john"};
// inline struct
q := getQuery(&u2, "OR")
fmt.Println(q)
// typed struct
q = getQuery(u, "AND")
fmt.Println(q)
}
func getQuery(target interface{}, join string) string {
var filters []string
val := reflect.ValueOf(target).Elem()
for i := 0; i < val.NumField(); i++ {
value := val.Field(i)
s :=fmt.Sprintf("%v",value);
// this little trick is to check if it is an empty value
// so don't generate empty condition expressions
if s == "" {
continue
}
fieldType := val.Type().Field(i)
filters = append(filters, fmt.Sprintf(" eq(%s, %v) ", fieldType.Name, value))
}
return strings.Join(filters, join)
}
Here is the playground
I would suggest to refactor your filter logic as mentioned below:
package main
import (
"fmt"
"strings"
)
func getQuery(key, val string, filters *[]string) {
if val != "" {
*filters = append(*filters, fmt.Sprintf(`eq("%s", "%s")`, key, val))
}
}
func main() {
var filters []string
email := "demo#demo.com"
mobileNumber := "123456789"
getQuery("email", email, &filters)
getQuery("mobileNumber", mobileNumber, &filters)
filter := strings.Join(filters, " OR ")
fmt.Println(filter)
}
I need to validate that my http request has two parameters, Start and End. Currently, I set a default value that should not appear as either of the parameters and check for it along with other invalid values. However, this feels like a hack. What should be the proper way to do this?
Here is my code:
type Request struct {
Start int `json: "start"`
End int `json: "end"`
}
func HandlePost(w http.ResponseWriter, r *http.Request) {
body , _ := ioutil.ReadAll(r.Body)
reqData := Request{Start: -1, End: -1} // < whats the correct way to do this
json.Unmarshal(body, &reqData)
if reqData.Start < 0 && reqData.End < 0 {
w.WriteHeader(http.StatusBadRequest)
return
}
// rest of the logic
}
You can use https://github.com/asaskevich/govalidator for basic way of validating the request. But in case you want something more sophisticated, you need to write your own custom validator function. e.g.
type Request struct {
Start int `json: "start"`
End int `json: "end"`
}
func (a *Request) validate() url.Values {
err := url.Values{}
// Write your own validation rules
if a.Start < 0 {
err.Add("Start", "Start cannot be less than 0");
}
return err;
}
func handler(w http.ResponseWriter, r *http.Request) {
requestBody := &Request{}
defer r.Body.Close()
if err := json.NewDecoder(r.Body).Decode(requestBody); err != nil {
panic(err)
}
if errors := requestBody.validate(); len(errors) > 0 {
err := map[string]interface{}{"validationError": errors}
w.Header().Set("Content-type", "application/json")
w.WriteHeader(http.StatusBadRequest)
json.NewEncoder(w).Encode(err)
}
fmt.Fprint(w, "success request scenario!")
}
Here's another way to validate structures using struct tags and pointers. Note that if 0 is a valid thing to pass, then this solution will not work.
omitempty considers the 0 value to be empty. If you want this to work will considering 0 to be valid remove the pointers and modify the IsValid method
package main
import (
"encoding/json"
"fmt"
)
type Request struct {
Start *int `json: "start,omitempty"`
End *int `json: "end,omitempty"`
}
func (r Request) IsValid() (bool, error) {
if r.Start == nil {
return false, fmt.Errorf("start is missing")
}
if r.End == nil {
return false, fmt.Errorf("end is missing")
}
return true, nil
}
var (
invalidStartb = `{"end": 1}`
invalidEndb = `{"start": 1}`
valid = `{"start": 1, "end": 1}`
)
func main() {
var r Request
_ = json.Unmarshal([]byte(invalidStartb), &r)
fmt.Println(r.IsValid())
r = Request{}
_ = json.Unmarshal([]byte(invalidEndb), &r)
fmt.Println(r.IsValid())
r = Request{}
_ = json.Unmarshal([]byte(valid), &r)
fmt.Println(r.IsValid())
}
runnable version here https://goplay.space/#Z0eqLpEHO37
You can use https://github.com/buger/jsonparser getInt.
You'll get an error if the json is missing the expected key.
I recommend using benchmark and not decided by the code beauty or any other hunch
I was wondering if someone can explain this syntax to me. In the google maps go api, they have
type Client struct {
httpClient *http.Client
apiKey string
baseURL string
clientID string
signature []byte
requestsPerSecond int
rateLimiter chan int
}
// NewClient constructs a new Client which can make requests to the Google Maps WebService APIs.
func NewClient(options ...ClientOption) (*Client, error) {
c := &Client{requestsPerSecond: defaultRequestsPerSecond}
WithHTTPClient(&http.Client{})(c) //???????????
for _, option := range options {
err := option(c)
if err != nil {
return nil, err
}
}
if c.apiKey == "" && (c.clientID == "" || len(c.signature) == 0) {
return nil, errors.New("maps: API Key or Maps for Work credentials missing")
}
// Implement a bursty rate limiter.
// Allow up to 1 second worth of requests to be made at once.
c.rateLimiter = make(chan int, c.requestsPerSecond)
// Prefill rateLimiter with 1 seconds worth of requests.
for i := 0; i < c.requestsPerSecond; i++ {
c.rateLimiter <- 1
}
go func() {
// Wait a second for pre-filled quota to drain
time.Sleep(time.Second)
// Then, refill rateLimiter continuously
for _ = range time.Tick(time.Second / time.Duration(c.requestsPerSecond)) {
c.rateLimiter <- 1
}
}()
return c, nil
}
// WithHTTPClient configures a Maps API client with a http.Client to make requests over.
func WithHTTPClient(c *http.Client) ClientOption {
return func(client *Client) error {
if _, ok := c.Transport.(*transport); !ok {
t := c.Transport
if t != nil {
c.Transport = &transport{Base: t}
} else {
c.Transport = &transport{Base: http.DefaultTransport}
}
}
client.httpClient = c
return nil
}
}
And this is the line I don't understand in NewClient
WithHTTPClient(&http.Client{})(c)
Why are there two ()()?
I see that WithHTTPClient takes in a *http.Client which that line does, but then it also passes in a pointer to the client struct declared above it?
WithHTTPClient returns a function, ie:
func WithHTTPClient(c *http.Client) ClientOption {
return func(client *Client) error {
....
return nil
}
}
WithHTTPClient(&http.Client{})(c) is just calling that function with c (a pointer to a Client) as parameter. It could be written as:
f := WithHTTPClient(&http.Client{})
f(c)
Is there a native way for inplace url parameters in native Go?
For Example, if I have a URL: http://localhost:8080/blob/123/test I want to use this URL as /blob/{id}/test.
This is not a question about finding go libraries. I am starting with the basic question, does go itself provide a basic facility to do this natively.
There is no built in simple way to do this, however, it is not hard to do.
This is how I do it, without adding a particular library. It is placed in a function so that you can invoke a simple getCode() function within your request handler.
Basically you just split the r.URL.Path into parts, and then analyse the parts.
// Extract a code from a URL. Return the default code if code
// is missing or code is not a valid number.
func getCode(r *http.Request, defaultCode int) (int, string) {
p := strings.Split(r.URL.Path, "/")
if len(p) == 1 {
return defaultCode, p[0]
} else if len(p) > 1 {
code, err := strconv.Atoi(p[0])
if err == nil {
return code, p[1]
} else {
return defaultCode, p[1]
}
} else {
return defaultCode, ""
}
}
Well, without external libraries you can't, but may I recommend two excellent ones:
httprouter - https://github.com/julienschmidt/httprouter - is extremely fast and very lightweight. It's faster than the standard library's router, and it creates 0 allocations per call, which is great in a GCed language.
Gorilla Mux - http://www.gorillatoolkit.org/pkg/mux -
Very popular, nice interface, nice community.
Example usage of httprouter:
func Hello(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
fmt.Fprintf(w, "hello, %s!\n", ps.ByName("name"))
}
func main() {
router := httprouter.New()
router.GET("/hello/:name", Hello)
log.Fatal(http.ListenAndServe(":8080", router))
}
What about trying using regex, and find a named group in your url, like playground:
package main
import (
"fmt"
"net/url"
"regexp"
)
var myExp = regexp.MustCompile(`/blob/(?P<id>\d+)/test`) // use (?P<id>[a-zA-Z]+) if the id is alphapatic
func main() {
s := "http://localhost:8080/blob/123/test"
u, err := url.Parse(s)
if err != nil {
panic(err)
}
fmt.Println(u.Path)
match := myExp.FindStringSubmatch(s) // or match := myExp.FindStringSubmatch(u.Path)
result := make(map[string]string)
for i, name := range myExp.SubexpNames() {
if i != 0 && name != "" {
result[name] = match[i]
}
}
fmt.Printf("id: %s\n", result["id"])
}
output
/blob/123/test
id: 123
Below full code to use it with url, that is receiving http://localhost:8000/hello/John/58 and returning http://localhost:8000/hello/John/58:
package main
import (
"fmt"
"net/http"
"regexp"
"strconv"
)
var helloExp = regexp.MustCompile(`/hello/(?P<name>[a-zA-Z]+)/(?P<age>\d+)`)
func hello(w http.ResponseWriter, req *http.Request) {
match := helloExp.FindStringSubmatch(req.URL.Path)
if len(match) > 0 {
result := make(map[string]string)
for i, name := range helloExp.SubexpNames() {
if i != 0 && name != "" {
result[name] = match[i]
}
}
if _, err := strconv.Atoi(result["age"]); err == nil {
fmt.Fprintf(w, "Hello, %v year old named %s!", result["age"], result["name"])
} else {
fmt.Fprintf(w, "Sorry, not accepted age!")
}
} else {
fmt.Fprintf(w, "Wrong url\n")
}
}
func main() {
http.HandleFunc("/hello/", hello)
http.ListenAndServe(":8090", nil)
}
How about writing your own url generator (extend net/url a little bit) as below.
// --- This is how does it work like --- //
url, _ := rest.NewURLGen("http", "stack.over.flow", "1234").
Pattern(foo/:foo_id/bar/:bar_id).
ParamQuery("foo_id", "abc").
ParamQuery("bar_id", "xyz").
ParamQuery("page", "1").
ParamQuery("offset", "5").
Do()
log.Printf("url: %s", url)
// url: http://stack.over.flow:1234/foo/abc/bar/xyz?page=1&offset=5
// --- Your own url generator would be like below --- //
package rest
import (
"log"
"net/url"
"strings"
"straas.io/base/errors"
"github.com/jinzhu/copier"
)
// URLGen generates request URL
type URLGen struct {
url.URL
pattern string
paramPath map[string]string
paramQuery map[string]string
}
// NewURLGen new a URLGen
func NewURLGen(scheme, host, port string) *URLGen {
h := host
if port != "" {
h += ":" + port
}
ug := URLGen{}
ug.Scheme = scheme
ug.Host = h
ug.paramPath = make(map[string]string)
ug.paramQuery = make(map[string]string)
return &ug
}
// Clone return copied self
func (u *URLGen) Clone() *URLGen {
cloned := &URLGen{}
cloned.paramPath = make(map[string]string)
cloned.paramQuery = make(map[string]string)
err := copier.Copy(cloned, u)
if err != nil {
log.Panic(err)
}
return cloned
}
// Pattern sets path pattern with placeholder (format `:<holder_name>`)
func (u *URLGen) Pattern(pattern string) *URLGen {
u.pattern = pattern
return u
}
// ParamPath builds path part of URL
func (u *URLGen) ParamPath(key, value string) *URLGen {
u.paramPath[key] = value
return u
}
// ParamQuery builds query part of URL
func (u *URLGen) ParamQuery(key, value string) *URLGen {
u.paramQuery[key] = value
return u
}
// Do returns final URL result.
// The result URL string is possible not escaped correctly.
// This is input for `gorequest`, `gorequest` will handle URL escape.
func (u *URLGen) Do() (string, error) {
err := u.buildPath()
if err != nil {
return "", err
}
u.buildQuery()
return u.String(), nil
}
func (u *URLGen) buildPath() error {
r := []string{}
p := strings.Split(u.pattern, "/")
for i := range p {
part := p[i]
if strings.Contains(part, ":") {
key := strings.TrimPrefix(p[i], ":")
if val, ok := u.paramPath[key]; ok {
r = append(r, val)
} else {
if i != len(p)-1 {
// if placeholder at the end of pattern, it could be not provided
return errors.Errorf("placeholder[%s] not provided", key)
}
}
continue
}
r = append(r, part)
}
u.Path = strings.Join(r, "/")
return nil
}
func (u *URLGen) buildQuery() {
q := u.URL.Query()
for k, v := range u.paramQuery {
q.Set(k, v)
}
u.RawQuery = q.Encode()
}
With net/http the following would trigger when calling localhost:8080/blob/123/test
http.HandleFunc("/blob/", yourHandlerFunction)
Then inside yourHandlerFunction, manually parse r.URL.Path to find 123.
Note that if you don't add a trailing / it won't work. The following would only trigger when calling localhost:8080/blob:
http.HandleFunc("/blob", yourHandlerFunction)
As of 19-Sep-22, with go version 1.19, instance of http.request URL has a method called Query, which will return a map, which is a parsed query string.
func helloHandler(res http.ResponseWriter, req *http.Request) {
// when request URL is `http://localhost:3000/?first=hello&second=world`
fmt.Println(req.URL.Query()) // outputs , map[second:[world] first:[hello]]
res.Write([]byte("Hello World Web"))
}
No way without standard library. Why you don't want to try some library? I think its not so hard to use it, just go get bla bla bla
I use Beego. Its MVC style.
how about a simple utility function ?
func withURLParams(u url.URL, param, val string) url.URL{
u.Path = strings.ReplaceAll(u.Path, param, val)
return u
}
you can use it like this:
u, err := url.Parse("http://localhost:8080/blob/:id/test")
if err != nil {
return nil, err
}
u := withURLParams(u, ":id","123")
// now u.String() is http://localhost:8080/blob/123/test
If you need a framework and you think it will be slow because it's 'bigger' than a router or net/http, then you 're wrong.
Iris is the fastest go web framework that you will ever find, so far according to all benchmarks.
Install by
go get gopkg.in/kataras/iris.v6
Django templates goes easy with iris:
import (
"gopkg.in/kataras/iris.v6"
"gopkg.in/kataras/iris.v6/adaptors/httprouter"
"gopkg.in/kataras/iris.v6/adaptors/view" // <-----
)
func main() {
app := iris.New()
app.Adapt(iris.DevLogger())
app.Adapt(httprouter.New()) // you can choose gorillamux too
app.Adapt(view.Django("./templates", ".html")) // <-----
// RESOURCE: http://127.0.0.1:8080/hi
// METHOD: "GET"
app.Get("/hi", hi)
app.Listen(":8080")
}
func hi(ctx *iris.Context){
ctx.Render("hi.html", iris.Map{"Name": "iris"})
}