How to propagate value from child middleware to parent? - go

I am trying to customize request pipeline through middleware pattern, the code as follow:
func helloHandler(w http.ResponseWriter, r *http.Request) {
fmt.Println("Hello, middleware!")
}
func middleware1(next http.HandlerFunc) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
fmt.Println("[START] middleware1")
ctx := r.Context()
ctx = context.WithValue(ctx, middleware1Key, middleware1Value)
r = r.WithContext(ctx)
next(w, r)
fmt.Println("[END] middleware1")
ctx = r.Context()
if val, ok := ctx.Value(middleware2Key).(string); ok {
fmt.Printf("Value from middleware2 %s \n", val)
}
}
}
func middleware2(next http.HandlerFunc) func(w http.ResponseWriter, r *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
fmt.Println("[START] middleware2")
ctx := r.Context()
if val, ok := ctx.Value(middleware1Key).(string); ok {
fmt.Printf("Value from middleware1 %s \n", val)
}
ctx = context.WithValue(ctx, middleware2Key, middleware2Value)
r = r.WithContext(ctx)
next(w, r)
fmt.Println("[END] middleware2")
}
}
func main() {
mux := http.NewServeMux()
middlewares := newMws(middleware1, middleware2)
mux.HandleFunc("/hello", middlewares.then(helloHandler))
if err := http.ListenAndServe(":8080", mux); err != nil {
panic(err)
}
}
and the output is :
[START] middleware1
[START] middleware2
Value from middleware1 middleware1Value
Hello, middleware!
[END] middleware2
[END] middleware1
According to the output, the value could pass from parent to the child , while, if the child add something to the context, it is invisible to the parent
How can I propagate value from child middleware to parent?

What you're doing is creating a new pointer to the modified http.Request via WithContext method. So if you're passing it to next middleware in the chain everything works as expected since you're passing this new pointer as an argument.
If you want to modify the request and make it visible for those who hold the pointer to it, you need to dereference the pointer and set the modified value.
So in your 'child' middleware instead of:
r = r.WithContext(ctx)
Just do the following:
*r = *r.WithContext(ctx)
Good exercise to understand pointers in Go but you SHOULD NOT do similar operations in your production code. The docs are clear about it. See https://pkg.go.dev/net/http#Handler.
Another possible solution (without messing with the request itself) is to pass a map inside a context and read/write from/to map instead. So in your first middleware:
ctx := r.Context()
m := make(map[string]string)
m[middleware1Key] = middleware1Value
ctx = context.WithValue(ctx, dummyStoreKey, m)
r = r.WithContext(ctx)
...
if val, ok := m[middleware2Key]; ok {
fmt.Printf("Value from middleware2 %s \n", val)
}
And in the second one:
ctx := r.Context()
if store, ok := ctx.Value(dummyStoreKey).(map[string]string); ok {
if val, ok := store[middleware1Key]; ok {
fmt.Printf("Value from middleware1 %s \n", val)
}
store[middleware2Key] = middleware2Value
}
You could add a AddStoreMiddleware as the first one in the pipeline and then use it in each successor if needed. Remember, maps in Go are not concurrently safe so in some subtle cases you should serialize access.

Related

Lookup a string with a function

I'm designing a router API and I'd like to be able to lookup a path by its function. Something like:
createUser := func(w http.ResponseWriter, r *http.Request) {
// create a user
}
createPost := func(w http.ResponseWriter, r *http.Request) {
// create a post
}
router.Post("/users", createUser)
router.Post("/posts", createPost)
fmt.Println(router.Lookup(createPost))
Here's a playground link: https://play.golang.org/p/ec6U0jJUbfx
This is surprisingly hard to do because you can't test for equality on a function or stick it as a key in a map. Is this even possible?
Are there any other workarounds I'm not thinking of? A reflect solution would be just fine.
You can create a server struct with a ServerHTTP method that handles all the request. When there is a request you can look up for an specific method by the path and function
Here is an example:
type Server struct {
routes []route
}
func (s *Server) handlerServer(db mydb.IDB, ctx context.Context) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx := ServerContext{w, r, db, ctx}
for i := 0; i < len(s.routes); i++ {
currentRoute := s.routes[i]
if isValidMethod(currentRoute, r) {
err := currentRoute.h(&ctx)
if err != nil {
log.Fatal(err)
}
}
}
}
}
I hope this is helpful

VS Code error when running go application

I am new to go and I am following a tutorial online. I get this error from VS Code
"cannot use c.ReadConfig (type func(http.ResponseWriter, *http.Request)) as type http.Handler in argument to router.Get:
func(http.ResponseWriter, *http.Request) does not implement http.Handler (missing ServeHTTP method)".
I checked the Get and Redconfig functions and they look alright. The teacher on his end does not get the error and he is able to run the Go code fine. this is the snippet in the main
This the main function
func main() {
config := domain.Config{}
configService := service.ConfigService{
Config: &config,
Location: "config.yaml",
}
go configService.Watch(time.Second * 30)
c := controller.Controller{
Config: &config,
}
router := muxinator.NewRouter()
router.Get("/read/{serviceName}", c.ReadConfig)
log.Fatal(router.ListenAndServe(":8080"))
}
This is the Get function
// Get returns the config for a particular service
func (c *Config) Get(serviceName string) (map[string]interface{}, error) {
c.lock.RLock()
defer c.lock.RUnlock()
a, ok := c.config["base"].(map[string]interface{})
if !ok {
return nil, fmt.Errorf("base config is not a map")
}
// If no config is defined for the service
if _, ok = c.config[serviceName]; !ok {
// Return the base config
return a, nil
}
b, ok := c.config[serviceName].(map[string]interface{})
if !ok {
return nil, fmt.Errorf("service %q config is not a map", serviceName)
}
// Merge the maps with the service config taking precedence
config := make(map[string]interface{})
for k, v := range a {
config[k] = v
}
for k, v := range b {
config[k] = v
}
return config, nil
}
This is ReadConfig
// ReadConfig writes the config for the given service to the ResponseWriter
func (c *Controller) ReadConfig(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json; charset=UTF-8")
vars := mux.Vars(r)
serviceName, ok := vars["serviceName"]
if !ok {
w.WriteHeader(http.StatusBadRequest)
fmt.Fprintf(w, "error")
}
config, err := c.Config.Get(serviceName)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, "error")
}
rsp, err := json.Marshal(&config)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, "error")
}
w.WriteHeader(http.StatusOK)
fmt.Fprintf(w, string(rsp))
}
What should happen is that I should be able to run and I can go to http://localhost:8080/read/base
Use http.HandlerFunc:
router := muxinator.NewRouter()
router.Get("/read/{serviceName}", http.HandlerFunc(c.ReadConfig))
It's expecting a ServeHTTP method, but you gave it a direct function. http.HandlerFunc acts as a wrapper so you can use a plain function as your handler.

Go Routine: Shared Global variable in web server

I have go web server running on port and handling post request which internally calls different url to fetch response using goroutine and proceed.
I have divided the whole flow to different method. Draft of the code.
package main
import (
"bytes"
"fmt"
"github.com/gorilla/mux"
"log"
"net/http"
"time"
)
var status_codes string
func main() {
router := mux.NewRouter().StrictSlash(true)
/*router := NewRouter()*/
router.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprintf(w, "Hello!!!")
})
router.HandleFunc("/{name}", func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
prepare(w, r, vars["name"])
}).Methods("POST")
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", 8080), router))
}
func prepare(w http.ResponseWriter, r *http.Request, name string) {
//initializing for the current request, need to maintain this variable for each request coming
status_codes = ""
//other part of the code and call to goroutine
var urls []string
//lets say all the url loaded, call the go routine func and wait for channel to respond and then proceed with the response of all url
results := callUrls(urls)
process(w, results)
}
type Response struct {
status int
url string
body string
}
func callUrls(urls []string) []*Response {
ch := make(chan *Response, len(urls))
for _, url := range urls {
go func(url string) {
//http post on url,
//base on status code of url call, add to status code
//some thing like
req, err := http.NewRequest("POST", url, bytes.NewBuffer(somePostData))
req.Header.Set("Content-Type", "application/json")
req.Close = true
client := &http.Client{
Timeout: time.Duration(time.Duration(100) * time.Second),
}
response, err := client.Do(req)
if err != nil {
status_codes += "200,"
//do other thing with the response received
} else {
status_codes += "500,"
}
// return to channel accordingly
ch <- &Response{200, "url", "response body"}
}(url)
}
var results []*Response
for {
select {
case r := <-ch:
results = append(results, r)
if len(results) == len(urls) {
//Done
close(ch)
return results
}
}
}
}
func process(w http.ResponseWriter, results []*Response){
//read those status code received from all urls call for the given request
fmt.Println("status", status_codes)
//Now the above line keep getting status code from other request as well
//for eg. if I have called 5 urls then it should have
//200,500,204,404,200,
//but instead it is
//200,500,204,404,200,204,404,200,204,404,200, and some more keep growing with time
}
The above code does:
Variable declare globally, Initialized in prepare function.
append value in go routine callUrls function
read those variable in process function
Now should I pass those variable declared globally to each function call to make them local as it won't be shared then?(I would hate to do this.)
Or is there any other approach to achieve the same thing without adding more argument to function being called.
As I will have few other string and int value as well that will be used across the program and in go routine function as well.
What will be the correct way of making them thread safe and only 5 codes for each request coming on port simultaneously.
Don't use global variables, be explicit instead and use function arguments. Moreover, you have a race condition on status_codes because it is accessed by multiple goroutines without any mutex lock.
Take a look at my fix below.
func prepare(w http.ResponseWriter, r *http.Request, name string) {
var urls []string
//status_codes is populated by callUris(), so let it return the slice with values
results, status_codes := callUrls(urls)
//process() needs status_codes in order to work, so pass the variable explicitely
process(w, results, status_codes)
}
type Response struct {
status int
url string
body string
}
func callUrls(urls []string) []*Response {
ch := make(chan *Response, len(urls))
//In order to avoid race condition, let's use a channel
statusChan := make(chan string, len(urls))
for _, url := range urls {
go func(url string) {
//http post on url,
//base on status code of url call, add to status code
//some thing like
req, err := http.NewRequest("POST", url, bytes.NewBuffer(somePostData))
req.Header.Set("Content-Type", "application/json")
req.Close = true
client := &http.Client{
Timeout: time.Duration(time.Duration(100) * time.Second),
}
response, err := client.Do(req)
if err != nil {
statusChan <- "200"
//do other thing with the response received
} else {
statusChan <- "500"
}
// return to channel accordingly
ch <- &Response{200, "url", "response body"}
}(url)
}
var results []*Response
var status_codes []string
for !doneRes || !doneStatus { //continue until both slices are filled with values
select {
case r := <-ch:
results = append(results, r)
if len(results) == len(urls) {
//Done
close(ch) //Not really needed here
doneRes = true //we are done with results, set the corresponding flag
}
case status := <-statusChan:
status_codes = append(status_codes, status)
if len(status_codes) == len(urls) {
//Done
close(statusChan) //Not really needed here
doneStatus = true //we are done with statusChan, set the corresponding flag
}
}
}
return results, status_codes
}
func process(w http.ResponseWriter, results []*Response, status_codes []string) {
fmt.Println("status", status_codes)
}

Determining if delete actually removed an existing key in a map

I have a map called nearby
func Delete(w http.ResponseWriter, r *http.Request) {
params := mux.Vars(r)
delete(nearby, params["id"])
}
I want to find out if the delete() call actually found a key to delete, I tried reading the return value:
func Delete(w http.ResponseWriter, r *http.Request) {
params := mux.Vars(r)
result := delete(nearby, params["id"])
}
but the compiler didn't like that - how can I find out if a key/val was deleted?
Probe the map before deleting the value:
func Delete(w http.ResponseWriter, r *http.Request) {
params := mux.Vars(r)
_, deleted := nearby[params["id"]]
delete(nearby, params["id"])
fmt.Println(deleted)
}
This snippet and the code in the question have a data race because HTTP handlers can be called concurrently. Add a mutex to protect the map.
var (
nearby = make(map[string]string)
mu sync.Mutex
)
func Delete(w http.ResponseWriter, r *http.Request) {
params := mux.Vars(r)
mu.Lock()
_, deleted := nearby[params["id"]]
delete(nearby, params["id"])
mu.Unlock()
fmt.Println(deleted)
}
The Go builtin delete() doesn't return anything, so you can't tell whether it deleted anything.
But you can check if the map contains the key and delete() it if it is present.
if _, ok := nearby[params["id"]]; ok {
delete(nearby, params["id"])
} else {
// whatever
}

Is there 'middleware' for Go http client?

I would like to ask if we can create 'middleware' functions for Go http client? Example I want to add a log function, so every sent request will be logged, or add setAuthToken so the token will be added to each request's header.
You can use the Transport parameter in HTTP client to that effect, with a composition pattern, using the fact that:
http.Client.Transport defines the function that will handle all HTTP requests;
http.Client.Transport has interface type http.RoundTripper, and can thus be replaced with your own implementation;
For example:
package main
import (
"fmt"
"net/http"
)
// This type implements the http.RoundTripper interface
type LoggingRoundTripper struct {
Proxied http.RoundTripper
}
func (lrt LoggingRoundTripper) RoundTrip(req *http.Request) (res *http.Response, e error) {
// Do "before sending requests" actions here.
fmt.Printf("Sending request to %v\n", req.URL)
// Send the request, get the response (or the error)
res, e = lrt.Proxied.RoundTrip(req)
// Handle the result.
if (e != nil) {
fmt.Printf("Error: %v", e)
} else {
fmt.Printf("Received %v response\n", res.Status)
}
return
}
func main() {
httpClient := &http.Client{
Transport: LoggingRoundTripper{http.DefaultTransport},
}
httpClient.Get("https://example.com/")
}
Feel free to alter names as you wish, I did not think on them for very long.
I worked on a project that had similar requirement so I built a middleware pipeline library that allows setting multiple middleware to the http client. You can check it out here.
Using the library, you would solve this in the following way
type LoggingMiddleware struct{}
func (s LoggingMiddleware) Intercept(pipeline pipeline.Pipeline, req *http.Request) (*http.Response, error) {
body, _ := httputil.DumpRequest(req, true)
log.Println(fmt.Sprintf("%s", string(body)))
/*
If you want to perform an action based on the response, do the following
resp, err = pipeline.Next
// perform some action
return resp, err
*/
return pipeline.Next(req)
}
transport := pipeline.NewCustomTransport(&LoggingMiddleware{})
client := &http.Client{Transport: transport}
resp, err := client.Get("https://example.com")
if err != nil {
// handle err
}
fmt.Println(resp.Status)
I wrote a small tutorial/library to do just that https://github.com/HereMobilityDevelopers/mediary
Here is some basic usage example:
client := mediary.Init().AddInterceptors(dumpInterceptor).Build()
client.Get("https://golang.org")
func dumpInterceptor(req *http.Request, handler mediary.Handler) (*http.Response, error) {
if bytes, err := httputil.DumpRequestOut(req, true); err == nil {
fmt.Printf("%s", bytes)
//GET / HTTP/1.1
//Host: golang.org
//User-Agent: Go-http-client/1.1
//Accept-Encoding: gzip
}
return handler(req)
}
There is also an explanation here https://github.com/HereMobilityDevelopers/mediary/wiki/Reasoning
Good idea! Here is a simple implementation of HTTP service middleware in Go.
Usually a simple http service framework is to register a bunch of routes, and then call different logics to process them according to the routes.
But in fact, there may be some unified processing involving almost all routes, such as logs, permissions, and so on.
So it is a good idea to engage in intermediate preprocessing at this time.
Define a middleware unit:
package main
import (
"net/http"
)
// AdaptorHandle middleware func type
type AdaptorHandle func(w http.ResponseWriter, r *http.Request) (next bool, err error)
// MiddleWareAdaptor router middlewares mapped by url
type MiddleWareAdaptor struct {
URLs map[string][]AdaptorHandle
}
// MakeMiddleWareAdaptor make a middleware adaptor
func MakeMiddleWareAdaptor() *MiddleWareAdaptor {
mwa := &MiddleWareAdaptor{
URLs: make(map[string][]AdaptorHandle),
}
return mwa
}
// Regist regist a adaptor
func (mw *MiddleWareAdaptor) Regist(url string, Adaptor ...AdaptorHandle) {
for _, adp := range Adaptor {
mw.URLs[url] = append(mw.URLs[url], adp)
// mw.URLs[url] = adp
}
}
// Exec exec middleware adaptor funcs...
func (mw *MiddleWareAdaptor) Exec(url string, w http.ResponseWriter, r *http.Request) (bool, error) {
if adps, ok := mw.URLs[url]; ok {
for _, adp := range adps {
if next, err := adp(w, r); !next || (err != nil) {
return next, err
}
}
}
return true, nil
}
Then wrap the route processing function with a middleware entry:
func middlewareHandler(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// before call handler
start := time.Now()
do, _ := mwa.Exec(r.URL.Path, w, r) // exec middleware
// call next handler
if do {
log.Println("middleware done. next...")
next.ServeHTTP(w, r)
} else {
log.Println("middleware done.break...")
}
// after call handle
log.Printf("Comleted %s in %v", r.URL.Path, time.Since(start))
})
}
mux.Handle("/", middlewareHandler(&uPlusRouterHandler{}))
type uPlusRouterHandler struct {
}
func (rh *uPlusRouterHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
...
}
Finally, register the middleware you need:
mwa = MakeMiddleWareAdaptor() // init middleware
mwa.Regist("/", testMWAfunc, testMWAfunc2) // regist middleware
...
func testMWAfunc(w http.ResponseWriter, r *http.Request) (bool, error) {
log.Println("I am Alice Middleware...")
log.Printf("Started %s %s", r.Method, r.URL.Path)
return true, nil
}
func testMWAfunc2(w http.ResponseWriter, r *http.Request) (bool, error) {
log.Println("I am Ben Middleware...")
return false, nil // return false,break follow-up actions.
}
This can be achieved using closure functions. It's probably more clear with an example:
package main
import (
"fmt"
"net/http"
)
func main() {
http.HandleFunc("/hello", logged(hello))
http.ListenAndServe(":3000", nil)
}
func logged(f func(http.ResponseWriter, *http.Request)) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
fmt.Println("logging something")
f(w, r)
fmt.Println("finished handling request")
}
}
func hello(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "<h1>Hello!</h1>")
}
credit goes to: http://www.calhoun.io/5-useful-ways-to-use-closures-in-go/

Resources