Dynamically instantiating HandleFunc - go

I've been trying to wrap my head around implementing HandleFunc() in a dynamic way.
This yaml file I want to turn into a local http web server in Go.
# Example .mock.yaml config
Endpoints:
- Resource: /city/1
Method: GET
Response: '{ Id": 1, "Name": "Albuquerque", "Population": 559.374, "State": "New Mexico" }'
StatusCode: 200
- Resource: /city
Method: POST
Response: '{ "Name": "Albuquerque", "Population": 559.374, "State": "New Mexico" }'
statusCode: 200
- Resource: /city/1
Method: PUT
Response: '{ "Population": 601.255 }'
StatusCode: 204
- Resource: /city/1
Method: DELETE
StatusCode: 204
Now I managed to implement something as following:
package utils
import (
"io"
"net/http"
"github.com/gorilla/mux"
"github.com/bschaatsbergen/mock/model"
)
func StartServer(conf model.Config, port string) {
r := mux.NewRouter()
for _, endpoint := range conf.Endpoints {
r.HandleFunc(endpoint.Resource, func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(endpoint.StatusCOde)
io.WriteString(w, endpoint.Response)
}).Methods(endpoint.Method)
}
address := ":" + port
http.ListenAndServe(address, r)
}
^ https://play.golang.org/p/YoTTUKnQL_5
But this doesn't cut it as it overwrites an earlier created route ('/city/1' GET, DELETE and POST are conflicting).
If anyone could give me a hand on how to dynamically translate the yaml config into a local web server, it would be appreciated!

I quote from #mkopriva, thanks!
I'm fairly certain your problem is the same as the one here:
stackoverflow.com/questions/69595865/…. i.e. All of your anon handler
closures capture one and the same variable, the iteration variable,
which, at the end of the loop, will hold the last element of the
Endpoints slice. Correct? Can the question be closed as duplicate?
Working piece of code:
func StartServer(conf model.Config, port string) {
r := mux.NewRouter()
for _, endpoint := range conf.Endpoints {
route := endpoint
r.HandleFunc(route.Resource, func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(route.Statuscode)
io.WriteString(w, route.Response)
}).Methods(route.Method)
}
address := ":" + port
log.Fatal(http.ListenAndServe(address, r))
}

Related

How to exchange context value between servers?

I'm studying Golang and have question on context.
I want to send request from server1 to server2 with context and want to read context value set by server1.
But it seems context of sercer2 doesn't carry server1's context value.
When I send request to server1 by curl http://localhost:8080
server1's console.
Send request to http://localhost:8082
server2's console.
request coming in
<nil>
How can I retrieve context value set by server1?
And if it is possible, I also want to know whether it is a correct way of exchanging value's like authentication between servers.
Middleware pattern is more desirable?
Thank you.
Codes
package main
import (
"context"
"fmt"
"net/http"
"github.com/go-chi/chi/v5"
)
func main() {
r := chi.NewRouter()
r.HandleFunc("/", hello)
fmt.Println("Starting listening on port 8080...")
http.ListenAndServe(":8080", r)
}
// Context's key.
type Sample string
var sampleKey Sample = "sample"
func hello(w http.ResponseWriter, r *http.Request) {
fmt.Println("Send request to http://localhost:8082")
ctx := context.WithValue(context.Background(), sampleKey, "1234")
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://localhost:8082", nil)
if err != nil {
fmt.Println("Error while sending request: ", err)
}
// Send request.
var c http.Client
c.Do(req)
}
package main
import (
"fmt"
"net/http"
"github.com/go-chi/chi/v5"
)
func main() {
r := chi.NewRouter()
r.HandleFunc("/", receive)
fmt.Println("Start listening on port 8082...")
http.ListenAndServe(":8082", r)
}
type Sample string
var sampleKey Sample
func receive(w http.ResponseWriter, r *http.Request) {
fmt.Println("request coming in")
fmt.Println(r.Context().Value(sampleKey))
}
The context is supposed to be used in a function call stack to share information and cancellation point between these functions. There is no automatic mechanism to serialize the value bound in the context over http. If you want to send data over http between these two servers, you mostly have three solution, you can:
encode that as an url parameter in the path somewhere
GET /hello/1234
send that in the http body, for example in JSON
POST /hello
{
"sample": 1234
}
as url key/value encoded as url parameters
/hello?sample=1234

HTTP PUT request handler using go

Please consider this sample Go code snippet,
package main
import (
"fmt"
"log"
"net/http"
"time"
)
func main() {
listen_at := ":3114"
go http.Handle("/", http.FileServer(http.Dir(".")))
//go http.Handle("/max", http.FileServer(http.Dir("."))) <-- Fails
go http.HandleFunc("/ping", ping_handler)
log.Fatal(http.ListenAndServe(listen_at, nil))
}
func ping_handler(w http.ResponseWriter, r *http.Request) {
a := time.Now()
layout := "2006-01-02-03-04-05-000"
fmt.Println("Got root!")
fmt.Println("r is", r)
fmt.Println("RemoteAddr is", r.RemoteAddr)
send_this := "OK GOT ping! " + a.Format(layout)
w.Write([]byte(send_this))
}
I've two questions:
(1) How can I change the FileServer to serve /max instead of / - my attempts failed, I get 404 for http://localhost:3114/max/ and http://localhost:3114/max.
(2) I wish to accept PUT requests to /max - how can I achieve this?
Please point me the right direction, thanks!
Edit 1
package main
import (
"fmt"
"log"
"net/http"
)
func main() {
go http.HandleFunc("/ping", hello)
log.Fatal(http.ListenAndServe(":3114", nil))
}
func hello(w http.ResponseWriter, r *http.Request) {
fmt.Println("STarting hello!")
log.Println("Got connection")
if r.URL.Path != "/ping" {
http.Error(w, "404 not found", http.StatusNotFound)
return
}
log.Println("Method:", r.Method)
switch r.Method {
case "GET":
send_this := "OK GOT ping! "
w.Write([]byte(send_this))
case "PUT":
fmt.Println("We got put!")
err := r.ParseForm()
checkErr(err)
fmt.Println("r now", r)
fmt.Println("r.Form", r.Form)
fmt.Println("r.PostForm", r.PostForm)
default:
send_this := "Please dont!"
w.Write([]byte(send_this))
fmt.Fprintf(w, "Unknown request!")
}
}
func checkErr(err error) {
if err != nil {
fmt.Println("Error", err)
}
}
I send a PUT request as curl -k http://localhost:3114/ping -T /tmp/a.go -v, it shows:
STarting hello!
2019/06/07 15:05:10 Got connection
2019/06/07 15:05:10 Method: PUT
We got put!
r now &{PUT /ping HTTP/1.1 1 1 map[Content-Length:[10115] Expect:[100-continue] User-Agent:[curl/7.47.0] Accept:[*/*]] 0xc4200ec2e0 <nil> 10115 [] false localhost:3114 map[] map[] <nil> map[] 127.0.0.1:64612 /ping <nil> <nil> <nil> 0xc42005e340}
r.Form map[]
r.PostForm map[]
How can I find the actual data, and the filename that came in from PUT?
From https://golang.org/pkg/net/http/#ServeMux:
Patterns name fixed, rooted paths, like "/favicon.ico", or rooted
subtrees, like "/images/" (note the trailing slash).
That means that /max, which is a fixed rooted path, will match only /max and the pattern /max/, which is a rooted subtree, will match /max/, any other path that starts with /max/, and by default it will also match /max.
http.Handle("/max/", http.FileServer(http.Dir(".")))
Depending on what the layout of your . directory is, you may need to use https://golang.org/pkg/net/http/#StripPrefix.
Let's say your directory contains two files:
.
├── foo.txt
└── max
└── bar.txt
Given the handler above, a request to /max/bar.txt will return the bar.txt file, but a request to /max/foo.txt or /foo.txt will return 404, no file.
So if you want to serve the files from the /max/ path, but your . directory doesn't have a max sub-directory then you can use StripPrefix to remove the /max prefix from the request's url path before it is passed on to the FileServer.
http.Handle("/max/", http.StripPrefix("/max/", http.FileServer(http.Dir("."))))
To handle PUT requests at the same route you need a custom handler.
type myhandler struct {
fs http.Handler
}
func (h myhandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if r.Method != "PUT" {
// business as usual
h.fs.ServeHTTP(w, r)
return
}
// handle PUT
// ...
}
And then to register it, do:
fs := http.StripPrefix("/max/", http.FileServer(http.Dir(".")))
http.Handle("/max/", myhandler{fs: fs})

Proxying a REST API using golang

I m actually learning golang and try to implement a proxy on a an Rest API that I've made in another language
For now, I only want to query my golang API, extract they actual path parameters and query the other API based on it.
I want the result to be "exactly" the same (or at least, the body part), just like a simple JSON.
For now, I don't want to create a structure for my data, I just want to simply get and retrieve the content.
Here's what I have:
package main
import (
"fmt"
"net/http"
"github.com/gorilla/mux"
)
const API_URL string = "https://my-api-path/"
func setHeaders(w http.ResponseWriter) {
w.WriteHeader(http.StatusOK)
w.Header().Set("Content-Type", "application/json")
}
func extractParams(r *http.Request) map[string]string {
return mux.Vars(r)
}
func getHandler(w http.ResponseWriter, r *http.Request) {
setHeaders(w)
params := extractParams(r)
url := API_URL + params["everything"]
response, err := http.Get(url)
if err != nil {
fmt.Fprint(w, err)
}
fmt.Fprint(w, response)
}
func main() {
router := mux.NewRouter()
router.HandleFunc("/{everything}", getHandler)
http.ListenAndServe(":8080", router)
}
My problem
For now, I m not able to retrieve JSON information from my other API. I only have a text/plain Content-Type which is weird since I enforce application/json and I only have some header details in the response body, something like:
&{200 OK 200 HTTP/2.0 2 0 map[Allow:[GET, HEAD, OPTIONS] Expect-Ct:[max-age=86400, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"] Server:[cloudflare] Cf-Ray:[some-ray] Date:[Tue, 12 Jun 2018 14:38:57 GMT] Content-Type:[application/json] Set-Cookie:[__cfduid=lolol; expires=Wed, 12-Jun-19 14:38:56 GMT; path=/; domain=some-domain; HttpOnly; Secure] Vary:[Accept-Encoding Cookie] X-Frame-Options:[SAMEORIGIN] X-Xss-Protection:[1; mode=block]] 0xc4201926f0 -1 [] false true map[] 0xc420150800 0xc4200e8370}
Do you have any idea on how I can proxy this request (or the JSON result) ?
About the Content-Type header not being written to your response:
Seems to be expected due to the order in which you are performing these operations:
w.WriteHeader(http.StatusOK)
w.Header().Set("Content-Type", "application/json")
See here: https://golang.org/pkg/net/http/#ResponseWriter
Changing the header map after a call to WriteHeader (or Write) has no effect unless the modified headers are trailers.
Try inverting those to read:
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)

Logging http responses (in addition to requests)

I am using Go and the Gorilla web toolkit's mux and handler packages to build a complex application, part of which requires a http server. Gorilla's mux and handler packages work wonderfully and I am able to successfully get the http server up and running and it has been quite simple to log requests.
However, I am unable to determine how I may log responses. Ideally, I would like a mechanism, similar to Gorilla's LoggingHandler, that "wraps" the logging mechanism easily.
Is there a Go package that does easily wraps / logs responses? Is there a way to use Go or Gorilla's capabilities in this fashion that I have not considered?
Thanks for the great suggestions. I tried a few of the suggestions and landed on a rather simple solution that uses a minimalist wrapper. Here is the solution that worked for me (feel free to offer comments, or better yet, other solutions):
import (
"fmt"
"log"
"net/http"
"net/http/httptest"
"net/http/httputil"
"github.com/gorilla/mux"
)
:
func logHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
x, err := httputil.DumpRequest(r, true)
if err != nil {
http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
return
}
log.Println(fmt.Sprintf("%q", x))
rec := httptest.NewRecorder()
fn(rec, r)
log.Println(fmt.Sprintf("%q", rec.Body))
}
}
func MessageHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "A message was received")
}
And the following code will use the aforementioned handler:
:
router := mux.NewRouter()
router.HandleFunc("/", logHandler(MessageHandler))
:
Output from the above code will be something along the lines of:
:
2016/07/20 14:44:29 "GET ... HTTP/1.1\r\nHost: localhost:8088\r\nAccept: */*\r\nUser-Agent: curl/7.43.0\r\n\r\n"
2016/07/20 14:44:29 ...[response body]
:
The accepted answer by Eric Broda won't help much if you want to actually send your response to the client. I've made a modification to that code that will actually work:
func logHandler(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
x, err := httputil.DumpRequest(r, true)
if err != nil {
http.Error(w, fmt.Sprint(err), http.StatusInternalServerError)
return
}
log.Println(fmt.Sprintf("%q", x))
rec := httptest.NewRecorder()
fn(rec, r)
log.Println(fmt.Sprintf("%q", rec.Body))
// this copies the recorded response to the response writer
for k, v := range rec.HeaderMap {
w.Header()[k] = v
}
w.WriteHeader(rec.Code)
rec.Body.WriteTo(w)
}
}
edit sorry, I didn't notice your mention of gorilla-mux, I have only tried this with gin, but if it uses middlewares this should still work.
the trick is, c.Next() in a middleware blocks until all subsequent middlewares return. Here's a logrus solution. Put this as your first middleware:
func Logrus(logger *logrus.Logger) gin.HandlerFunc {
return func(c *gin.Context) {
start := time.Now().UTC()
path := c.Request.URL.Path
c.Next()
end := time.Now().UTC()
latency := end.Sub(start)
logger.WithFields(logrus.Fields{
"status": c.Writer.Status(),
"method": c.Request.Method,
"path": path,
"ip": c.ClientIP(),
"duration": latency,
"user_agent": c.Request.UserAgent(),
}).Info()
}
}
GinEngine.Use(Logrus(logrus.StandardLogger()))

Setting up Route Not Found in Gin

I've setup a default router and some routes in Gin:
router := gin.Default()
router.POST("/users", save)
router.GET("/users",getAll)
but how do I handle 404 Route Not Found in Gin?
Originally, I was using httprouter which I understand Gin uses so this was what I originally had...
router.NotFound = http.HandlerFunc(customNotFound)
and the function:
func customNotFound(w http.ResponseWriter, r *http.Request) {
//return JSON
return
}
but this won't work with Gin.
I need to be able to return JSON using the c *gin.Context so that I can use:
c.JSON(404, gin.H{"code": "PAGE_NOT_FOUND", "message": "Page not found"})
What you're looking for is the NoRoute handler.
More precisely:
r := gin.Default()
r.NoRoute(func(c *gin.Context) {
c.JSON(404, gin.H{"code": "PAGE_NOT_FOUND", "message": "Page not found"})
})
Adding to what Pablo Fernandez wrote I also seen that the same can be done with 405 MethodNotAllowed, so similarly if for 404 we've got NoRoute for 405 there is NoMethod method
So having this as result
import (
"github.com/gin-gonic/gin"
"log"
"net/http"
)
func main() {
app := gin.New()
app.NoRoute(func(c *gin.Context) {
c.JSON(http.StatusNotFound, gin.H{"code": "PAGE_NOT_FOUND", "message": "404 page not found"})
})
app.NoMethod(func(c *gin.Context) {
c.JSON(http.StatusMethodNotAllowed, gin.H{"code": "METHOD_NOT_ALLOWED", "message": "405 method not allowed"})
})
}
In order to add a 'catch all' method we can do:
import (
"github.com/gin-gonic/gin"
"log"
"net/http"
)
func main() {
app := gin.New()
app.NoRoute(func(c *gin.Context) {
c.JSON(http.StatusNotFound, gin.H{"code": "PAGE_NOT_FOUND", "message": "404 page not found"})
})
app.NoMethod(func(c *gin.Context) {
c.JSON(http.StatusMethodNotAllowed, gin.H{"code": "METHOD_NOT_ALLOWED", "message": "405 method not allowed"})
})
}
// Set-up Error-Handler Middleware
app.Use(func(c *gin.Context) {
log.Printf("Total Errors -> %d", len(c.Errors))
if len(c.Errors) <= 0 {
c.Next()
return
}
for _, err := range c.Errors {
log.Printf("Error -> %+v\n", err)
}
c.JSON(http.StatusInternalServerError, "")
})
Now, gin has these 2 method only for these 2 type of errors,my guess is because are the most common one while building an API and wanted to add some default handler when you first set up the application.
In fact, we can see the implementation here:
// NoRoute adds handlers for NoRoute. It returns a 404 code by default.
func NoRoute(handlers ...gin.HandlerFunc) {
engine().NoRoute(handlers...)
}
// NoMethod is a wrapper for Engine.NoMethod.
func NoMethod(handlers ...gin.HandlerFunc) {
engine().NoMethod(handlers...)
}
Now, the body that uses by default when these 2 handlers are not used by who uses the gin framework (so the default one are) are defined enter link description here
var (
default404Body = []byte("404 page not found")
default405Body = []byte("405 method not allowed")
)
And then later on used on the function handleHTTPRequest from line 632
if engine.HandleMethodNotAllowed {
for _, tree := range engine.trees {
if tree.method == httpMethod {
continue
}
if value := tree.root.getValue(rPath, nil, c.skippedNodes, unescape); value.handlers != nil {
c.handlers = engine.allNoMethod
serveError(c, http.StatusMethodNotAllowed, default405Body)
return
}
}
}
c.handlers = engine.allNoRoute
serveError(c, http.StatusNotFound, default404Body)
}

Resources