I am building a webpage. The page should be able to handle the different http methods (GET, POST...). My page technically works and handles each type of request, but in the case of GET requests (serving index.html at the root "/" path), my page is not displaying properly. None of the images or css are displaying correctly, presumably because those files cannot be found.
I have noticed that http.Handle provides better results than http.HandleFunc when substituted into my server.go code below, in that the images and css do display correctly using:
http.FileServer(http.Dir("static"))
http.Handle("/", http.StripPrefix("/", fs))
The following is my web server with images and css not being displayed correctly. In general, my intent is to use static files for everything, including html (ex. index.html) and to implement some solution using nothing but standard go.
server.go code
package main
import (
"net/http"
"fmt"
)
func indexHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Add("Content Type", "text/html")
switch r.Method {
case "GET":
http.ServeFile(w, r, "./static/index.html")
case "POST":
fmt.Pprintf(w, "POST!")
case "PUT":
fmt.Pprintf(w, "PUT!")
case "DELETE":
fmt.Pprintf(w, "DELETE!")
default:
fmt.Pprintf(w, "Default!")
}
}
func main() {
http.HandleFunc("/", indexHandler)
http.ListenAndServe(":3000", nil)
}
You've hard-coded your server to always return index.html for any GET request, no matter what is being requested. So if your index.html includes a reference to style.css, the browser is going to make a second request for style.css and you'll return index.html again.
I assume what you're trying to do is have all GET requests return static files, while other verbs will do something else. You just need to pass those along to the file server:
root := "static"
...
case "GET":
if r.URL.Path == "" || r.URL.Path == "/" {
http.ServeFile(w, r, path.Join(root, "index.html"))
} else {
http.ServeFile(w, r, path.Join(root, r.URL.Path))
}
Note that by the time your handler is called, all ".." references in the URL have been removed, attackers can't use this to escape your static tree. But ServeFile() will return directory listings, so you'll need to check for that if that's a problem.
Related
I'm developing a Go program based on Gin web framework.
I want to serve pictures from the local file system.
func main() {
r := gin.Default()
r.Static("/page2", "resources/pictures")
r.GET("/test", func(context *gin.Context) {
name := context.Query("name")
context.JSON(http.StatusOK, gin.H{
"name": name,
})
})
r.GET("/page", func(context *gin.Context) {
name := context.Query("picname")
context.File("resources/pictures/" + name + ".jpg")
context.JSON(http.StatusOK, "That is "+name)
})
r.Run(":9090")
}
I find it strange that when I use /page2 to get the pictures (url http://localhost:9090/page2/xiamei.jpg), it works fine.
But when I use /page to get the pictures (url http://localhost:9090/page/xiamei.jpg) an error happens.
http: wrote more than the declared Content-Length
What is the internal reason, and what is the root cause of the difference between these two access methods?
In the /page handler, you call both Context.File, and Context.JSON.
The first call to Context.File already writes the file to the response and sets Content-Length header.
By calling Context.JSON again with more payload, you are increasing the actual length of the content without updating the Content-Length header; this results in the error you see.
r.Static doesn't have this issue because it serves one file and doesn't add more payload than it should.
In a Gin handler you should call only one renderer function. If you are serving a file, you can remove the last c.JSON altogether.
i have a this code to open static folder
fs := http.FileServer(http.Dir("./static/"))
router.PathPrefix("/static/").Handler(http.StripPrefix("/static/", fs))
but when i'm going to /static path it returns a list of files in this folder
eg
license.txt
logo.png
but i want to return a blank page
You can add blank index.html in directory ./static/, it will be rendered as blank page.
Something like this
<html>
<head><title>Nothing here</title></head>
<body><h1>Nothing here</h1></body>
</html>
vodolaz095's answer is the simplest, but if you really must do it in Go, you could wrap the FileServer to catch the specific case of a request for the root:
func noIndex(fs http.Handler) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "" || strings.HasSuffix(r.URL.Path, "/") {
w.Write("")
return
}
fs.ServeHTTP(w, r)
}
}
Note that this doesn't catch requests for subdirectories without a trailing slash in the URL; to trap this you'd need a much more detailed custom implementation that was filesystem-aware to know what is a directory and what is a file.
It looks like you are using gorilla/mux - but if you don't mind using the standard library's http.ServeMux you can achieve this in Go code like so:
func main() {
fs := http.FileServer(http.Dir("./static/"))
h := http.NewServeMux()
blank := http.HandlerFunc(func(_ http.ResponseWriter, _ *http.Request) {})
h.Handle("/static/", http.StripPrefix("/static/", fs)) // will match "/static" with a redirect ...
h.Handle("/static", blank) // ... unless you create a hanlder explicitly
http.ListenAndServe(":8080", h)
}
from http.ServeMux docs:
...if a subtree has been registered and a request is received naming
the subtree root without its trailing slash, ServeMux redirects that
request to the subtree root (adding the trailing slash). This behavior
can be overridden with a separate registration for the path without
the trailing slash. For example, registering "/images/" causes
ServeMux to redirect a request for "/images" to "/images/", unless
"/images" has been registered separately.
I have a go web app which serves static HTML/JS/CSS files plus has some API endpoints. I noticed that my HTML/JS/CSS are not being cached on the browser. E.g., every time I reload a page, they are fully re-downloaded.
Is this a server side configuration change that I need to set? How can I accomplish this with Go and Gorilla Mux?
I'm using Google App Engine, so Nginx is not a possibility.
Here's my main.go code:
package main
import (
"associations"
"html/template"
"net/http"
"log"
"io/ioutil"
"github.com/gorilla/mux"
"github.com/rs/cors"
"google.golang.org/appengine"
"google.golang.org/appengine/mail"
)
var index = template.Must(template.ParseFiles(
"dist/index.html",
))
func init() {
r := mux.NewRouter()
r.HandleFunc("/", homeHandler)
r.HandleFunc("/api/{tenant}/certificates", associations.GetCertificate).Methods("GET")
r.HandleFunc("/api/{tenant}/requests", associations.PostRequest).Methods("POST")
// handle static files
r.PathPrefix("/static/").Handler(
http.StripPrefix("/static/", http.FileServer(http.Dir("dist/static/"))))
r.NotFoundHandler = http.HandlerFunc(homeHandler) // work around for SPA serving index.html
handler := cors.Default().Handler(r)
http.Handle("/", handler)
}
EDIT: Here's the solution with #Topo's suggestion:
// handle static files
r.PathPrefix("/static/").Handler(http.StripPrefix("/static/",
CacheControlWrapper(http.FileServer(http.Dir("dist/static/")))))
....
func CacheControlWrapper(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Cache-Control", "max-age=2592000") // 30 days
h.ServeHTTP(w, r)
})
}
To tell the browser to cache your files, you need to tell it for how long, if not the user would never see newer versions of the file.
To do this just set the Cache-Control header inside your handler function:
w.Header().Set("Cache-Control", "max-age=3600")
It's also a good idea to use the Etag header to let the browsers know when there is a new version of the file. That way you can cache the files for a really long time, and still serve the new content to the users as soon as it's available:
etag := "some unique value"
w.Header().Set("Etag", etag)
You need to use a different etag value every time the file changes. The browser stores it and only reloads the file when the etag is different to the stored value. You can use something like the file name + modified date for the etag:
var modTime time.Time
fi, err := fh.Stat()
if err == nil {
modTime = fi.ModTime()
} else {
modTime = time.Now()
}
etag := "\"" + file + modTime.String() + "\""
w.Header().Set("Etag", etag)
You can read the mozilla docs for the Cache-Control and Etag heders.
If you want to avoid writing your own handler function for static files and instead keep using http.FileServer, you will need to wrap it in a handler that sets the headers before writing the response. Here is a blog post on wrapping web handlers.
So I'm using a go server to serve up a single page web application.
This works for serving all the assets on the root route. All the CSS and HTML are served up correctly.
fs := http.FileServer(http.Dir("build"))
http.Handle("/", fs)
So when the URL is http://myserverurl/index.html or http://myserverurl/styles.css, it serves the corresponding file.
But for a URL like http://myserverurl/myCustompage, it throws 404 if myCustompage is not a file in the build folder.
How do I make all routes for which a file does not exist serve index.html?
It is a single page web application and it will render the appropriate screen once the html and js are served. But it needs index.html to be served on routes for which there is no file.
How can this be done?
The handler returned by http.FileServer() does not support customization, it does not support providing a custom 404 page or action.
What we may do is wrap the handler returned by http.FileServer(), and in our handler we may do whatever we want of course. In our wrapper handler we will call the file server handler, and if that would send a 404 not found response, we won't send it to the client but replace it with a redirect response.
To achieve that, in our wrapper we create a wrapper http.ResponseWriter which we will pass to the handler returned by http.FileServer(), and in this wrapper response writer we may inspect the status code, and if it's 404, we may act to not send the response to the client, but instead send a redirect to /index.html.
This is an example how this wrapper http.ResponseWriter may look like:
type NotFoundRedirectRespWr struct {
http.ResponseWriter // We embed http.ResponseWriter
status int
}
func (w *NotFoundRedirectRespWr) WriteHeader(status int) {
w.status = status // Store the status for our own use
if status != http.StatusNotFound {
w.ResponseWriter.WriteHeader(status)
}
}
func (w *NotFoundRedirectRespWr) Write(p []byte) (int, error) {
if w.status != http.StatusNotFound {
return w.ResponseWriter.Write(p)
}
return len(p), nil // Lie that we successfully written it
}
And wrapping the handler returned by http.FileServer() may look like this:
func wrapHandler(h http.Handler) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
nfrw := &NotFoundRedirectRespWr{ResponseWriter: w}
h.ServeHTTP(nfrw, r)
if nfrw.status == 404 {
log.Printf("Redirecting %s to index.html.", r.RequestURI)
http.Redirect(w, r, "/index.html", http.StatusFound)
}
}
}
Note that I used http.StatusFound redirect status code instead of http.StatusMovedPermanently as the latter may be cached by browsers, and so if a file with that name is created later, the browser would not request it but display index.html immediately.
And now put this in use, the main() function:
func main() {
fs := wrapHandler(http.FileServer(http.Dir(".")))
http.HandleFunc("/", fs)
panic(http.ListenAndServe(":8080", nil))
}
Attempting to query a non-existing file, we'll see this in the log:
2017/11/14 14:10:21 Redirecting /a.txt3 to /index.html.
2017/11/14 14:10:21 Redirecting /favicon.ico to /index.html.
Note that our custom handler (being well-behaviour) also redirected the request to /favico.ico to index.html because I do not have a favico.ico file in my file system. You may want to add this as an exception if you don't have it either.
The full example is available on the Go Playground. You can't run it there, save it to your local Go workspace and run it locally.
Also check this related question: Log 404 on http.FileServer
I'm a little bit confused. Much of examples shows usage of both: http.ServeFile(..) and http.FileServer(..), but seems they have very close functionality. Also I have found no information about how to set custom NotFound handler.
// This works and strip "/static/" fragment from path
fs := http.FileServer(http.Dir("static"))
http.Handle("/static/", http.StripPrefix("/static/", fs))
// This works too, but "/static2/" fragment remains and need to be striped manually
http.HandleFunc("/static2/", func(w http.ResponseWriter, r *http.Request) {
http.ServeFile(w, r, r.URL.Path[1:])
})
http.ListenAndServe(":8080", nil)
I've tried to read source code and both of them use serveFile(ResponseWriter, *Request, FileSystem, string, bool) underlying function. However http.FileServer return fileHandler with its own ServeHTTP() method and make some preparation work before serving file (eg path.Clean()).
So why need this separation? Which method better to use? And how can I set custom NotFound handler, for example when requested file not found?
The main difference is that http.FileServer does effectively almost 1:1 mapping of an HTTP prefix with a filesystem. In plain english, it serves up an entire directory path. and all its children.
Say you had a directory called /home/bob/static and you had this setup:
fs := http.FileServer(http.Dir("/home/bob/static"))
http.Handle("/static/", http.StripPrefix("/static", fs))
Your server would take requests for e.g. /static/foo/bar and serve whatever is at /home/bob/static/foo/bar (or 404)
In contrast, the ServeFile is a lower level helper that can be used to implement something similar to FileServer, or implement your own path munging potentially, and any number of things. It simply takes the named local file and sends it over the HTTP connection. By itself, it won't serve a whole directory prefix (unless you wrote a handler that did some lookup similar to FileServer)
NOTE Serving up a filesystem naively is a potentially dangerous thing (there are potentially ways to break out of the rooted tree) hence I recommend that unless you really know what you're doing, use http.FileServer and http.Dir as they include checks to make sure people can't break out of the FS, which ServeFile doesn't.
Addendum
Your secondary question, how do you do a custom NotFound handler, unfortunately, is not easily answered. Because this is called from internal function serveFile as you noticed, there's no super easy place to break into that. There are potentially some sneaky things like intercepting the response with your own ResponseWriter which intercepts the 404 response code, but I'll leave that exercise to you.
Here a handler which sends a redirect to "/" if file is not found. This comes in handy when adding a fallback for an Angular application, as suggested here, which is served from within a golang service.
Note: This code is not production ready. Only illustrative (at best :-)
package main
import "net/http"
type (
// FallbackResponseWriter wraps an http.Requesthandler and surpresses
// a 404 status code. In such case a given local file will be served.
FallbackResponseWriter struct {
WrappedResponseWriter http.ResponseWriter
FileNotFound bool
}
)
// Header returns the header of the wrapped response writer
func (frw *FallbackResponseWriter) Header() http.Header {
return frw.WrappedResponseWriter.Header()
}
// Write sends bytes to wrapped response writer, in case of FileNotFound
// It surpresses further writes (concealing the fact though)
func (frw *FallbackResponseWriter) Write(b []byte) (int, error) {
if frw.FileNotFound {
return len(b), nil
}
return frw.WrappedResponseWriter.Write(b)
}
// WriteHeader sends statusCode to wrapped response writer
func (frw *FallbackResponseWriter) WriteHeader(statusCode int) {
Log.Printf("INFO: WriteHeader called with code %d\n", statusCode)
if statusCode == http.StatusNotFound {
Log.Printf("INFO: Setting FileNotFound flag\n")
frw.FileNotFound = true
return
}
frw.WrappedResponseWriter.WriteHeader(statusCode)
}
// AddFallbackHandler wraps the handler func in another handler func covering authentication
func AddFallbackHandler(handler http.HandlerFunc, filename string) http.HandlerFunc {
Log.Printf("INFO: Creating fallback handler")
return func(w http.ResponseWriter, r *http.Request) {
Log.Printf("INFO: Wrapping response writer in fallback response writer")
frw := FallbackResponseWriter{
WrappedResponseWriter: w,
FileNotFound: false,
}
handler(&frw, r)
if frw.FileNotFound {
Log.Printf("INFO: Serving fallback")
http.Redirect(w, r, "/", http.StatusSeeOther)
}
}
}
It can be added as in this example (using goji as mux):
mux.Handle(pat.Get("/*"),
AddFallbackHandler(http.FileServer(http.Dir("./html")).ServeHTTP, "/"))