I am wondering if it would be possible to cancel a web request or send an internal response to the client inside the ReverseProxy.Director function.
Suppose we do something that throws an error, or we have other reason to not forward the request.
proxy := &httputil.ReverseProxy{
Director: func(r *http.Request) {
err := somethingThatThrows()
},
}
http.Handle("/", proxy)
A solution to this might be the below, but it's not as neat as the above way to use the proxy. I am also not sure to which degree the request should be modified that way. The director seems to be the place to do that.
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
err := somethingThatThrows()
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
proxy.ServeHTTP(w, r)
})
if it would be possible to cancel a web request [...]
You can cancel the request that is passed to the Director function, BUT there are some details to consider:
the correct way to cancel a request is to cancel its context
you can not cancel contexts where you didn't set a (deadline|timeout|cancelfunc) yourself → i.e. you must have access to the cancel function → i.e. you can't cancel parent contexts created by someone else.
the *http.Request passed to Director function is a clone of the original request
Based on the points above, you can replace the request in the Director with another one that has a cancellable context. It may look like the following:
proxy := &httputil.ReverseProxy{
Director: func(req *http.Request) {
// create a cancellable context, and re-set the request
ctx, cancel := context.WithCancel(req.Context())
*req = *req.WithContext(ctx)
err := somethingThatThrows()
if err != nil {
cancel()
return
}
},
}
Then the code above doesn't do anything else by itself. What should happen is that the httputil.ReverseProxy.Transport function, which implements http.RoundTripper checks whether the request context is cancelled, before actually send anything to the upstream service.
The documentation of Director states:
Director must be a function which modifies the request into a new request to be sent using Transport.
When the Transport is not provided, it will fall back to http.DefaultTransport, which aborts the request when the context is cancelled. The current code (Go 1.17.5) looks like:
select {
case <-ctx.Done():
req.closeBody()
return nil, ctx.Err()
default:
}
If you provide your own implementation of http.RoundTripper you may want to implement that behavior yourself. Remember also that the context done channel is nil if it's not cancellable, so you have to set a cancel func and call cancel() in order to have that select run the "done" case.
or send an internal response to the client inside the ReverseProxy.Director
Based on the same quote above docs, you should not write to the http.ResponseWriter from within the Director function — assuming you are even closing around it. As you can see the Director itself doesn't get the http.ResponseWriter as an argument, and this should already be a self-explanatory detail.
If you want to specify some other behavior in case the request can't be forwarded, and assuming that whatever implementation of http.RoundTripper is returning error when the req context is cancelled, you can provide your ReverseProxy.ErrorHandler function:
proxy.ErrorHandler = func(writer http.ResponseWriter, request *http.Request, err error) {
// inspect err
// write to writer
}
The ErrorHandler will be invoked when Transport returns error, including when the error comes from a cancelled request, and it does have http.ResponseWriter as an argument.
Related
I reuse the http client connection to make external calls to a single endpoint. An excerpt of the program is shown below:
var AppCon MyApp
func New(user, pass string, platformURL *url.URL, restContext string) (*MyApp, error) {
if AppCon == (MyApp{}) {
AppCon = MyApp{
user: user,
password: pass,
URL: platformURL,
Client: &http.Client{Timeout: 30 * time.Second},
RESTContext: restContext,
}
cj, err := cookiejar.New(nil)
if err != nil {
return &AppCon, err
}
AppCon.cookie = cj
}
return &AppCon, nil
}
// This is an example only. There are many more functions which accept *MyApp as a pointer.
func(ma *MyApp) GetUser(name string) (string, error){
// Return user
}
func main(){
for {
// Get messages from a queue
// The message returned from the queue provide info on which methods to call
// 'm' is a struct with message metadata
c, err := New(m.un, m.pass, m.url)
go func(){
// Do something i.e c.GetUser("123456")
}()
}
}
I now have the requirement to set up a client connections with different endpoints/credentials received via queue messages.
The problem I foresee is I can't just simply modify AppCon with the new endpoint details since a pointer to MyApp is returned, resulting in resetting c. This can impact a goroutine making a HTTP call to an unintended endpoint. To make matters non trivial, the program is not meant to have awareness of the endpoints (I was considering using a switch statement) but rather receive what it needs via queue messages.
Given the issues I've called out are correct, are there any recommendations on how to solve it?
EDIT 1
Based on the feedback provided, I am inclined to believe this will solve my problem:
Remove the use of a Singleton of MyApp
Decouple the http client from MyApp which will enable it for reuse
var httpClient *http.Client
func New(user, pass string, platformURL *url.URL, restContext string) (*MyApp, error) {
AppCon = MyApp{
user: user,
password: pass,
URL: platformURL,
Client: func() *http.Client {
if httpClient == nil {
httpClient = &http.Client{Timeout: 30 * time.Second}
}
return httpClient
}()
RESTContext: restContext,
}
return &AppCon, nil
}
// This is an example only. There are many more functions which accept *MyApp as a pointer.
func(ma *MyApp) GetUser(name string) (string, error){
// Return user
}
func main(){
for {
// Get messages from a queue
// The message returned from the queue provide info on which methods to call
// 'm' is a struct with message metadata
c, err := New(m.un, m.pass, m.url)
// Must pass a reference
go func(c *MyApp){
// Do something i.e c.GetUser("123456")
}(c)
}
}
Disclaimer: this is not a direct answer to your question but rather an attempt to direct you to a proper way of solving your problem.
Try to avoid a singleton pattern for you MyApp. In addition, New is misleading, it doesn't actually create a new object every time. Instead you could be creating a new instance every time, while preserving the http client connection.
Don't use constructions like this: AppCon == (MyApp{}), one day you will shoot in your leg doing this. Use instead a pointer and compare it to nil.
Avoid race conditions. In your code you start a goroutine and immediately proceed to the new iteration of the for loop. Considering you re-use the whole MyApp instance, you essentially introduce a race condition.
Using cookies, you make your connection kinda stateful, but your task seems to require stateless connections. There might be something wrong in such an approach.
I'm learning go and ran into some design issues while developing web app. The app has main route "/" where user can submit a simple form. With those form values I am calling external API and unmarshaling response into some struct. Now from here I want to make another call based on retrieved values to another external API and I'm not sure what's the proper way of doing this. Here is a snippet for better understandment:
func main() {
http.HandleFunc("/", mainHandler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
func mainHandler(w http.ResponseWriter, r *http.Request) {
//renders form template
//makes post and retrieves data from api
//here with retrieved data I want to make another call to different API,
// but mainHandler would get too big and complex. I'm not sure how should I pass this data to
// another handler or redirect to another handler with this data.
}
The handlers' semantics should be designed to match the desired HTTP behavior, regardless of the code complexity. If you want to handle a single client request by doing a bunch of stuff, that should be a single handler. If the handler becomes too complex, break it up. Handlers are just functions and can be broken up exactly like any other function - by extracting some part of it into another function and calling that new function. To take you pseudocode:
func mainHandler(w http.ResponseWriter, r *http.Request) {
err := renderTemplate(w)
if err != nil { ... }
err, data := postToApi()
if err != nil { ... }
err, data2 := postToApi2(data)
if err != nil { ... }
}
There's no reason for those functions to be handlers themselves or to get the client involved with a redirect. Just break up your logic the way you normally break up logic - it doesn't matter that it's an HTTP handler.
Hi golearner, in the mainHandler just render the form and make another handler kinda "/formaction" to handle the form, in that way you can easily organize your code.
So I made a few changes to this example to make it work with gin-gonic
https://github.com/utiq/go-in-5-minutes/tree/master/episode4
The websocket handshake between many clients is succesful. The problem is that when a client sends a message, the message is not propagated to the rest of the clients.
I had a look on your commit changes of episode4.
My observations as follows:
You're creating hub instance on every incoming request at stream handler. hub instance used to keeps track connections, etc. so you're losing it on every request.
You have removed index/home handler (may be you wanted to convert to gin handler or something, I don't know).
Now, let's bring episode4 into action. Please do following changes (as always improve it as you like). I have tested your episode4 with below changes, it's working fine.
Make /ws handler work on server.go:
h := newHub()
wsh := wsHandler{h: h}
r.GET("/ws", func(c *gin.Context) {
wsh.ServeHTTP(c.Writer, c.Request)
})
Remove the stream handler on connection.go:
func stream(c *gin.Context) {
h := newHub()
wsHandler{h: h}.ServeHTTP(c.Writer, c.Request)
}
Adding index HTML handler on server.go: (added it to test episode4 at my end)
r.SetHTMLTemplate(template.Must(template.ParseFiles("index.html")))
r.GET("/", func(c *gin.Context) {
c.HTML(200, "index.html", nil)
})
Go has come with a new package called context and with recent versions (Go 1.7 I think) we should be able to use it in the same way as gorilla/context package:
http://www.gorillatoolkit.org/pkg/context
With gorilla context you can very easily set and get variables that are relevant to a request, it's handlers, and middlewares.
To set a value in gorilla context is really easy:
func handleFunc(w http.ResponseWriter, r *http.Request) {
context.Set(r, "foo", "bar")
}
To get the value we can do:
func handleFunc(w http.ResponseWriter, r *http.Request) {
val := context.Get(r, "foo")
}
I understand that we can use this in middlewares so that the next middleware can use variables that were set in previous middleware. I would like to be able to do this with Go context package.
I understand that to get a value is quite simple like this:
func handleFunc(w http.ResponseWriter, r *http.Request) {
fmt.Println(r.Context().Value("foo"))
}
But I have no idea how to set the value. It was not very intuitive for me and I don't really understand how to do it.
See "Exploring the context package", using WithValue and the context associated to the Request:
Middleware
Middleware in Go refers to an http handler which wraps around a multiplexer. There are several 3rd party middleware solutions (such as negroni), but really the standard library supports a very similar pattern. The use of a Context in the request allows us to hold data in the request.
See the example code for invocation and definition.
func putClientIPIntoContext(r *http.Request) context.Context {
ci := r.RemoteAddr
fwd := r.Header.Get("X-Forwarded-For")
if fwd != "" {
ci = fwd
}
ctx := context.WithValue(r.Context(), ClientIPKey, ci)
return ctx
}
The Context can store request-scoped variables.
It’s useful when writing ‘middleware’, but it’s a little bit ‘anti-pattern’ — it’s a bit magical, because it’s not type-safe.
See more at "Pitfalls of context values and how to avoid or mitigate them in Go".
The example below only shows how you might use the authentication logic from above to verify that when a user is logged in when visiting any page with a path prefix of /dashboard/.
A similar approach could be used to verify that a user is an admin before allowing them access to any page with a path prefix of /admin/.
func requireUser(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
user := lookupUser(r)
if user == nil {
// No user so redirect to login
http.Redirect(w, r, "/login", http.StatusFound)
return
}
ctx := context.WithValue(r.Context(), "user", user)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
func main() {
dashboard := http.NewServeMux()
dashboard.HandleFunc("/dashboard/hi", printHi)
dashboard.HandleFunc("/dashboard/bye", printBye)
mux := http.NewServeMux()
// ALL routes that start with /dashboard/ require that a
// user is authenticated using the requireUser middleware
mux.Handle("/dashboard/", requireUser(dashboard))
mux.HandleFunc("/", home)
http.ListenAndServe(":3000", addRequestID(mux))
}
As kostix comments, use Context wisely, like Dave Cheney suggest in "Context is for cancelation"
I want to determine a simple and useful pattern for user authentication in a web app being written in golang.
I have come up with two patterns. First one is enabling the programmer to have his functions separate form the authentication logic, and has cleaner HandleFunc parts in main() that one can see only by loking main() to see what parts are under authentication control.
Second one is making programmer include a decision in every function deal with authentication required urls. An if statement checks by a authp() function defined else where.
Which one is better pattern for such necessity?
What are the better patterns for this job?
Is it even possible to pass a function to http.HandleFunc that has signature other than func urlFunc (ResponseWriter, *Request) bu like func urlFunc (successFunc, failFunc) or func urlFunc (ResponseWriter, *Request, successFunc, failFunc) as in authenticationGateKeeper function of First Way below, if not a suitable workaround for that?
//First Way
package main
func authGateKeeper(successFunc, failFunc) {
if (authp()) {
successFunc
} else {
failFunc
}
}
func authp() boolean {
//authentication logic, db query, or session check etc.
}
//usage in main
http.HandleFunc("/", authGateKeeper)
//Second Way; other alternative, in each function check pattern
func f(w, r) {
if (authp()) {
//function's processes
} else {
//the fail case function or processes
}
}
func authp() boolean {
//authentication logic, db query, or session check etc.
}
//usage in main
http.HandleFunc("/", f)
There are many ways to spin this, and it's arguable whether one is outright "better". I'd strongly suggest writing some middleware that wraps your routes and enforces the check, calling the wrapped handler only on success.
Note that I'm going to make a few assumptions here as you haven't told us how you're managing sessions (cookies? server-side?) and/or what kind of authorization you might need on top of authentication.
// Middleware - a function that sits in the 'middle' of your request processing.
func RequireAuth(h http.Handler) http.Handler) {
fn := func(w http.ResponseWriter, r *http.Request) {
// Assuming gorilla/sessions
session, err := store.Get("name", r)
if err != nil {
// Raise HTTP 500
return
}
// We'll assume you're storing the userID in the cookie|server session
// upon login elsewhere.
id := session.Values["userID"]
// Probably returns a *yourapp.User
user, err := db.GetUser(id)
if err != nil {
// Raise HTTP 500
return
}
if user == nil {
http.Error(w, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
// Don't forget these 'naked' returns - if you miss one, your
// handler will keep processing beyond the error and result in
// unintended side effects
return
}
// Further checks here - i.e. checking user.Active == true, etc.
// The userID matches one in the DB, so let's proceed
h.ServeHTTP(w, r)
}
return http.HandlerFunc(fn)
}
// And in your router - assuming just vanilla net/http
http.Handle("/", RequireAuth(yourHandlerFunc))
http.Handle("/", RequireAuth(someOtherHandler))
// Note that using gorilla/mux or goji can help give you "subrouters" so you
// don't have to wrap every single route with your middleware (messy, error prone)
I'd also suggest some reading on Go middleware1 composition2 which will help you in the future.
If you want to call a custom error page, just write a handler - e.g. UnauthorizedHandler that satisfies http.Handler and just call UnauthorizedHandler.ServeHTTP(w, r) instead of http.Error along the way.