I'm learning go and ran into some design issues while developing web app. The app has main route "/" where user can submit a simple form. With those form values I am calling external API and unmarshaling response into some struct. Now from here I want to make another call based on retrieved values to another external API and I'm not sure what's the proper way of doing this. Here is a snippet for better understandment:
func main() {
http.HandleFunc("/", mainHandler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
func mainHandler(w http.ResponseWriter, r *http.Request) {
//renders form template
//makes post and retrieves data from api
//here with retrieved data I want to make another call to different API,
// but mainHandler would get too big and complex. I'm not sure how should I pass this data to
// another handler or redirect to another handler with this data.
}
The handlers' semantics should be designed to match the desired HTTP behavior, regardless of the code complexity. If you want to handle a single client request by doing a bunch of stuff, that should be a single handler. If the handler becomes too complex, break it up. Handlers are just functions and can be broken up exactly like any other function - by extracting some part of it into another function and calling that new function. To take you pseudocode:
func mainHandler(w http.ResponseWriter, r *http.Request) {
err := renderTemplate(w)
if err != nil { ... }
err, data := postToApi()
if err != nil { ... }
err, data2 := postToApi2(data)
if err != nil { ... }
}
There's no reason for those functions to be handlers themselves or to get the client involved with a redirect. Just break up your logic the way you normally break up logic - it doesn't matter that it's an HTTP handler.
Hi golearner, in the mainHandler just render the form and make another handler kinda "/formaction" to handle the form, in that way you can easily organize your code.
Related
I am wondering if it would be possible to cancel a web request or send an internal response to the client inside the ReverseProxy.Director function.
Suppose we do something that throws an error, or we have other reason to not forward the request.
proxy := &httputil.ReverseProxy{
Director: func(r *http.Request) {
err := somethingThatThrows()
},
}
http.Handle("/", proxy)
A solution to this might be the below, but it's not as neat as the above way to use the proxy. I am also not sure to which degree the request should be modified that way. The director seems to be the place to do that.
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
err := somethingThatThrows()
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
return
}
proxy.ServeHTTP(w, r)
})
if it would be possible to cancel a web request [...]
You can cancel the request that is passed to the Director function, BUT there are some details to consider:
the correct way to cancel a request is to cancel its context
you can not cancel contexts where you didn't set a (deadline|timeout|cancelfunc) yourself → i.e. you must have access to the cancel function → i.e. you can't cancel parent contexts created by someone else.
the *http.Request passed to Director function is a clone of the original request
Based on the points above, you can replace the request in the Director with another one that has a cancellable context. It may look like the following:
proxy := &httputil.ReverseProxy{
Director: func(req *http.Request) {
// create a cancellable context, and re-set the request
ctx, cancel := context.WithCancel(req.Context())
*req = *req.WithContext(ctx)
err := somethingThatThrows()
if err != nil {
cancel()
return
}
},
}
Then the code above doesn't do anything else by itself. What should happen is that the httputil.ReverseProxy.Transport function, which implements http.RoundTripper checks whether the request context is cancelled, before actually send anything to the upstream service.
The documentation of Director states:
Director must be a function which modifies the request into a new request to be sent using Transport.
When the Transport is not provided, it will fall back to http.DefaultTransport, which aborts the request when the context is cancelled. The current code (Go 1.17.5) looks like:
select {
case <-ctx.Done():
req.closeBody()
return nil, ctx.Err()
default:
}
If you provide your own implementation of http.RoundTripper you may want to implement that behavior yourself. Remember also that the context done channel is nil if it's not cancellable, so you have to set a cancel func and call cancel() in order to have that select run the "done" case.
or send an internal response to the client inside the ReverseProxy.Director
Based on the same quote above docs, you should not write to the http.ResponseWriter from within the Director function — assuming you are even closing around it. As you can see the Director itself doesn't get the http.ResponseWriter as an argument, and this should already be a self-explanatory detail.
If you want to specify some other behavior in case the request can't be forwarded, and assuming that whatever implementation of http.RoundTripper is returning error when the req context is cancelled, you can provide your ReverseProxy.ErrorHandler function:
proxy.ErrorHandler = func(writer http.ResponseWriter, request *http.Request, err error) {
// inspect err
// write to writer
}
The ErrorHandler will be invoked when Transport returns error, including when the error comes from a cancelled request, and it does have http.ResponseWriter as an argument.
Say I have a html/template like the following:
<html>
<body>
<p>{{SomeFunc .SomeData}}</p>
</body>
and sometimes SomeFunc returns an error. Is there an idiomatic way to deal with this?
If I write directly to the ResponseWriter, then a status code 200 has already been written before I encounter the error.
var tmpl *template.Template
func Handler(w http.ResponseWriter, r *http.Request) {
err := tmpl.Execute(w, data)
// "<html><body><p>" has already been written...
// what to do with err?
}
Preferably I would return a status code 400 or some such, but I can't see a way to do this if I use template.Execute directly on the ResponseWriter. Is there something I'm missing?
Since the template engine generates the output on-the-fly, parts of the template preceding the SomeFunc call are already sent to the output. And if the output is not buffered, they (along with the HTTP 200 status) may already be sent.
You can't do anything about that.
What you can do is perform the check before you call template.Execute(). In trivial case it should be enough to call SomeFunc() and check its return value. If you choose this path and the return value of SomeFunc() is complex, you do not have to call it again from the template, you can simply pass its return value to the params you pass to the template and refer to this value in the template (so SomeFunc() won't have to be executed twice).
If this is not enough or you can't control it, you can create a bytes.Buffer or strings.Builder, execute your template directed into this buffer, and after the Execute() returns, check if there were errors. If there were errors, send back a proper error message / page. If everything went ok, you can just send the content of the buffer to the ResponseWriter.
This could look something like this:
buf := &bytes.Buffer{}
err := tmpl.Execute(buf, data)
if err != nil {
// Send back error message, for example:
http.Error(w, "Hey, Request was bad!", http.StatusBadRequest) // HTTP 400 status
} else {
// No error, send the content, HTTP 200 response status implied
buf.WriteTo(w)
}
I want to determine a simple and useful pattern for user authentication in a web app being written in golang.
I have come up with two patterns. First one is enabling the programmer to have his functions separate form the authentication logic, and has cleaner HandleFunc parts in main() that one can see only by loking main() to see what parts are under authentication control.
Second one is making programmer include a decision in every function deal with authentication required urls. An if statement checks by a authp() function defined else where.
Which one is better pattern for such necessity?
What are the better patterns for this job?
Is it even possible to pass a function to http.HandleFunc that has signature other than func urlFunc (ResponseWriter, *Request) bu like func urlFunc (successFunc, failFunc) or func urlFunc (ResponseWriter, *Request, successFunc, failFunc) as in authenticationGateKeeper function of First Way below, if not a suitable workaround for that?
//First Way
package main
func authGateKeeper(successFunc, failFunc) {
if (authp()) {
successFunc
} else {
failFunc
}
}
func authp() boolean {
//authentication logic, db query, or session check etc.
}
//usage in main
http.HandleFunc("/", authGateKeeper)
//Second Way; other alternative, in each function check pattern
func f(w, r) {
if (authp()) {
//function's processes
} else {
//the fail case function or processes
}
}
func authp() boolean {
//authentication logic, db query, or session check etc.
}
//usage in main
http.HandleFunc("/", f)
There are many ways to spin this, and it's arguable whether one is outright "better". I'd strongly suggest writing some middleware that wraps your routes and enforces the check, calling the wrapped handler only on success.
Note that I'm going to make a few assumptions here as you haven't told us how you're managing sessions (cookies? server-side?) and/or what kind of authorization you might need on top of authentication.
// Middleware - a function that sits in the 'middle' of your request processing.
func RequireAuth(h http.Handler) http.Handler) {
fn := func(w http.ResponseWriter, r *http.Request) {
// Assuming gorilla/sessions
session, err := store.Get("name", r)
if err != nil {
// Raise HTTP 500
return
}
// We'll assume you're storing the userID in the cookie|server session
// upon login elsewhere.
id := session.Values["userID"]
// Probably returns a *yourapp.User
user, err := db.GetUser(id)
if err != nil {
// Raise HTTP 500
return
}
if user == nil {
http.Error(w, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
// Don't forget these 'naked' returns - if you miss one, your
// handler will keep processing beyond the error and result in
// unintended side effects
return
}
// Further checks here - i.e. checking user.Active == true, etc.
// The userID matches one in the DB, so let's proceed
h.ServeHTTP(w, r)
}
return http.HandlerFunc(fn)
}
// And in your router - assuming just vanilla net/http
http.Handle("/", RequireAuth(yourHandlerFunc))
http.Handle("/", RequireAuth(someOtherHandler))
// Note that using gorilla/mux or goji can help give you "subrouters" so you
// don't have to wrap every single route with your middleware (messy, error prone)
I'd also suggest some reading on Go middleware1 composition2 which will help you in the future.
If you want to call a custom error page, just write a handler - e.g. UnauthorizedHandler that satisfies http.Handler and just call UnauthorizedHandler.ServeHTTP(w, r) instead of http.Error along the way.
I'm a little bit confused. Much of examples shows usage of both: http.ServeFile(..) and http.FileServer(..), but seems they have very close functionality. Also I have found no information about how to set custom NotFound handler.
// This works and strip "/static/" fragment from path
fs := http.FileServer(http.Dir("static"))
http.Handle("/static/", http.StripPrefix("/static/", fs))
// This works too, but "/static2/" fragment remains and need to be striped manually
http.HandleFunc("/static2/", func(w http.ResponseWriter, r *http.Request) {
http.ServeFile(w, r, r.URL.Path[1:])
})
http.ListenAndServe(":8080", nil)
I've tried to read source code and both of them use serveFile(ResponseWriter, *Request, FileSystem, string, bool) underlying function. However http.FileServer return fileHandler with its own ServeHTTP() method and make some preparation work before serving file (eg path.Clean()).
So why need this separation? Which method better to use? And how can I set custom NotFound handler, for example when requested file not found?
The main difference is that http.FileServer does effectively almost 1:1 mapping of an HTTP prefix with a filesystem. In plain english, it serves up an entire directory path. and all its children.
Say you had a directory called /home/bob/static and you had this setup:
fs := http.FileServer(http.Dir("/home/bob/static"))
http.Handle("/static/", http.StripPrefix("/static", fs))
Your server would take requests for e.g. /static/foo/bar and serve whatever is at /home/bob/static/foo/bar (or 404)
In contrast, the ServeFile is a lower level helper that can be used to implement something similar to FileServer, or implement your own path munging potentially, and any number of things. It simply takes the named local file and sends it over the HTTP connection. By itself, it won't serve a whole directory prefix (unless you wrote a handler that did some lookup similar to FileServer)
NOTE Serving up a filesystem naively is a potentially dangerous thing (there are potentially ways to break out of the rooted tree) hence I recommend that unless you really know what you're doing, use http.FileServer and http.Dir as they include checks to make sure people can't break out of the FS, which ServeFile doesn't.
Addendum
Your secondary question, how do you do a custom NotFound handler, unfortunately, is not easily answered. Because this is called from internal function serveFile as you noticed, there's no super easy place to break into that. There are potentially some sneaky things like intercepting the response with your own ResponseWriter which intercepts the 404 response code, but I'll leave that exercise to you.
Here a handler which sends a redirect to "/" if file is not found. This comes in handy when adding a fallback for an Angular application, as suggested here, which is served from within a golang service.
Note: This code is not production ready. Only illustrative (at best :-)
package main
import "net/http"
type (
// FallbackResponseWriter wraps an http.Requesthandler and surpresses
// a 404 status code. In such case a given local file will be served.
FallbackResponseWriter struct {
WrappedResponseWriter http.ResponseWriter
FileNotFound bool
}
)
// Header returns the header of the wrapped response writer
func (frw *FallbackResponseWriter) Header() http.Header {
return frw.WrappedResponseWriter.Header()
}
// Write sends bytes to wrapped response writer, in case of FileNotFound
// It surpresses further writes (concealing the fact though)
func (frw *FallbackResponseWriter) Write(b []byte) (int, error) {
if frw.FileNotFound {
return len(b), nil
}
return frw.WrappedResponseWriter.Write(b)
}
// WriteHeader sends statusCode to wrapped response writer
func (frw *FallbackResponseWriter) WriteHeader(statusCode int) {
Log.Printf("INFO: WriteHeader called with code %d\n", statusCode)
if statusCode == http.StatusNotFound {
Log.Printf("INFO: Setting FileNotFound flag\n")
frw.FileNotFound = true
return
}
frw.WrappedResponseWriter.WriteHeader(statusCode)
}
// AddFallbackHandler wraps the handler func in another handler func covering authentication
func AddFallbackHandler(handler http.HandlerFunc, filename string) http.HandlerFunc {
Log.Printf("INFO: Creating fallback handler")
return func(w http.ResponseWriter, r *http.Request) {
Log.Printf("INFO: Wrapping response writer in fallback response writer")
frw := FallbackResponseWriter{
WrappedResponseWriter: w,
FileNotFound: false,
}
handler(&frw, r)
if frw.FileNotFound {
Log.Printf("INFO: Serving fallback")
http.Redirect(w, r, "/", http.StatusSeeOther)
}
}
}
It can be added as in this example (using goji as mux):
mux.Handle(pat.Get("/*"),
AddFallbackHandler(http.FileServer(http.Dir("./html")).ServeHTTP, "/"))
I'm trying to chain HTTP handlers in go to provide some added functionality, like this:
package router
import (
// snip
"github.com/gorilla/mux"
"github.com/gorilla/handlers"
"net/http"
)
// snip
r := mux.NewRouter()
/* routing code */
var h http.Handler
h = r
if useGzip {
h = handlers.CompressHandler(h)
}
if useLogFile {
fn := pathToLog
accessLog, err := os.OpenFile(fn, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0666)
if err != nil {
panic(err)
}
h = handlers.CombinedLoggingHandler(accessLog, h)
}
// etc...
The problem is, if any HTTP headers are already set by one of the controllers that the gorilla/mux router points to (for example, w.WriteHeader(404) or w.Header().Set("Content-Type", "application/json")) - this silently breaks any "wrapper" handler trying to set or add its own headers, like the compress handler. I can't see any errors, unless I forgot to catch one somewhere, but the browser gets an invalid response.
Is there any graceful way to deal with this, short of just stashing the headers somewhere and then leaving the final handler to write them? It seems like that would mean rewriting the handlers' code, which I'd love to avoid if at all possible.
Once you call w.WriteHeader(404), the header goes on a wire. So you can't add to it anymore.
Best way you can do is to buffer status code and write it at the end of a chain.
For example, you can provide your own wrapper for http.ResponseWriter that would re-implement WriteHeader() to save status value. Then add method Commit() to actually write it.
Call Commit() in the last handler. You have to determine somehow which handler is last, of course.
I experienced the same silently-failing behaviour. But only in handlers where I did WritheHeader to set a status code other than StatusOK. I think things went wrong in this part of CompressHandler:
if h.Get("Content-Type") == "" {
h.Set("Content-Type", http.DetectContentType(b))
}
Which appears to be resolved when explicitly setting the content type in my own handler:
w.Header().Set("Content-Type", "text/html; charset=utf-8")
w.WriteHeader(code)