In trying to set up two different web.Request states for use in some test cases, one without any headers and one with, I run into issue:
Setup
I create fakeRequest,fakeRequestNoHeaders thus:
// create fake request
fakeRequest := new(web.Request)
fakeRequest.Request = httptest.NewRequest("GET",
fakeServer.URL,
nil)
fakeRequestNoHeaders := new(web.Request)
fakeRequestNoHeaders.Request = fakeRequest.Request
// give fakeRequest some headers
fakeRequest.Header.Add("Authorization", "Bearer ksjaf;oipyu7")
fakeRequest.Header.Add("Scope", "test")
Sanity Test
I expect, of course, that fakeRequest.Header != fakeRequestNoHeaders.Header.
I write that test:
t.Run("HeadersSanityTest", func(t *testing.T) {
assert.NotEqualf(t,
fakeRequest.Header,
fakeRequestNoHeaders.Header,
"fakeRequest,fakeRequestNoHeaders share the same header state")
Result of test
It fails.
Why is this and how can I achieve what I'm trying?
UPDATE: I found the culprit:
the underlying http.Request, returned by httptest.NewRequest, is actually a pointer. Header simply belongs to that Request. The problem now reduces down to "How to deep-copy that Request."
The issue was, indeed, not with the Header field, but instead the Request field, which was a pointer. (Oh no! I accidentally shallow-copied)
The Solution
I recalled, in one of my earlier tests, a method that I wrote specifically to get around this:
func makeBasicRequest() *web.Request {
baseReq := httptest.NewRequest(http.MethodPost,
"[some url here]",
nil)
req := new(web.Request)
req.Request = baseReq
return req
}
I basically just brought it into this test, and used it, hitting it once per fake request that I needed.
Related
I've been working on a Go project where gorilla/mux is used as the router.
I need to be able to have query values associated with a route, but these values should be optional.
That means that I'd like to catch both /articles/123 and /articles/123?key=456 in the same handler.
To accomplish so I tried using the r.Queries method that accepts key/value pairs:
router.
Path("/articles/{id:[0-9]+}").Queries("key", "{[0-9]*?}")
but this makes only the value (456) optional, but not the key.
So both /articles/123?key=456 and /articles/123?key= are valid, but not /articles/123.
Edit: another requirement is that, after registering the route, I'd like to build them programatically, and I can't seem to work out how to use r.Queries even though the docs specifically state that it's possible (https://github.com/gorilla/mux#registered-urls).
#jmaloney answer works, but doesn't allow to build URLs from names.
I would just register your handler twice.
router.Path("/articles/{id:[0-9]+}").
Queries("key", "{[0-9]*?}").
HandlerFunc(YourHandler).
Name("YourHandler")
router.Path("/articles/{id:[0-9]+}").HandlerFunc(YourHandler)
Here is a working program to demonstrate. Notice that I am using r.FormValue to get the query parameter.
Note: make sure you have an up to date version go get -u github.com/gorilla/mux since a bug of query params not getting added the built URLs was fixed recently.
package main
import (
"fmt"
"log"
"net/http"
"github.com/gorilla/mux"
)
var router = mux.NewRouter()
func main() {
router.Path("/articles/{id:[0-9]+}").Queries("key", "{key}").HandlerFunc(YourHandler).Name("YourHandler")
router.Path("/articles/{id:[0-9]+}").HandlerFunc(YourHandler)
if err := http.ListenAndServe(":9000", router); err != nil {
log.Fatal(err)
}
}
func YourHandler(w http.ResponseWriter, r *http.Request) {
id := mux.Vars(r)["id"]
key := r.FormValue("key")
u, err := router.Get("YourHandler").URL("id", id, "key", key)
if err != nil {
http.Error(w, err.Error(), 500)
return
}
// Output:
// /articles/10?key=[key]
w.Write([]byte(u.String()))
}
If you register query parameters they are required doc:
All variables defined in the route are required, and their values must conform to the corresponding patterns.
Because those parameters are optional you just need to check for them inside of a handler function: id, found := mux.Vars(r)["id"]. Where found will show if the parameter in the query or not.
Seems like the best way to handle optional URL parameters is to define your router as normal without them, then parse the optional params out like this:
urlParams := request.URL.Query()
This returns a map that contains the URL parameters as Key/Value pairs.
I have a handler which connects to a db and retrieves the records. I wrote a test case for that and it goes this way:
main_test.go
package main
import (
"os"
"fmt"
"testing"
"net/http"
"net/http/httptest"
)
var a App
func TestMain(m *testing.M) {
a = App{}
a.InitializeDB(fmt.Sprintf("postgres://****:****#localhost/db?sslmode=disable"))
code := m.Run()
os.Exit(code)
}
func TestRulesetGet(t *testing.T) {
req, err := http.NewRequest("GET", "/1/sig/", nil)
if err != nil {
t.Fatal(err)
}
// We create a ResponseRecorder (which satisfies http.ResponseWriter) to record the response.
rr := httptest.NewRecorder()
handler := http.HandlerFunc(a.Get)
handler.ServeHTTP(rr, req)
// Check the response body is what we expect.
if len(rr.Body.String()) != 0 {
fmt.Println("Status OK : ", http.StatusOK)
fmt.Println("handler returned body: got ",
rr.Body.String())
}
}
I feel like this is a very basic test case where I'm just checking the response length (This is because I expect a slice). I'm not really sure whether this is the right way to write a test case. Please point out some loop holes so that I could write a solid test cases for my remaining handlers. And also, I'm not using the actual Error, and Fatal methods for checking the errors.
If it works then it's not wrong, but it doesn't validate much, only that the response body is non-empty - not even that it's successful. I'm not sure why it prints out http.StatusOK, which is a constant, and doesn't tell you what the status code in the response was.
Personally when I do this level of test I check, at the very least, that the response code is as expected, the response body unmarshals correctly (if it's JSON or XML), the response data is basically sane, etc. For more complex payloads I might use a golden file test. For critical code I might use a fuzz (aka monte carlo) test. For performance-critical code I'll likely add benchmarks and load tests. There are practically infinite ways to test code. You'll have to figure out what your needs are and how to meet them.
Currently my web app is just a router and handlers.
What are some important things I am missing to make this production worthy?
I believe I have to set the # of procs to ensure this uses maximum goroutines?
Should I be using output buffering?
Anything else you see missing that is best-practise?
var (
templates = template.Must(template.ParseFiles("templates/home.html")
)
func main() {
r := mux.NewRouter()
r.HandleFunc("/", WelcomeHandler)
http.ListenAndServe(":9000", r)
}
func WelcomeHandler(w http.ResponseWriter, r *http.Request) {
homePage, err := api.LoadHomePage()
if err != nil {
}
tmpl := "home"
renderTemplate(w, tmpl, homePage)
}
func renderTemplate(w http.ResponseWriter, tmpl string, hp *HomePage) {
err := templates.ExecuteTemplate(w, tmpl+".html", hp)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
You don't need to set/change runtime.GOMAXPROCS() as since Go 1.5 it defaults to the number of available CPU cores.
Buffering output? From the performance point of view, you don't need to. But there may be other considerations for which you may.
For example, your renderTemplate() function may potentially panic. If executing the template starts writing to the output, it involves setting the HTTP response code and other headers prior to writing data. And if a template execution error occurs after that, it will return an error, and so your code attempts to send back an error response. At this point HTTP headers are already written, and this http.Error() function will try to set headers again => panic.
One way to avoid this is to first render the template into a buffer (e.g. bytes.Buffer), and if no error is returned by the template execution, then you can write the content of the buffer to the response writer. If error occurs, then of course you won't write the content of the buffer, but send back an error response just like you did.
To sum it up, your code is production ready performance-wise (excluding the way you handle template execution errors).
WelcomeHandler should return when err != nil is true.
Log the error when one is hit to help investigation.
Place templates = template.Must(template.ParseFiles("templates/home.html") in the init. Split it into separate lines. If template.ParseFiles returns an then error make a Fatal log. And if you have multiple templates to initialize then initialize them in goroutines with a common WaitGroup to speed up the startup.
Since you are using mux, HTTP Server is too clean with its URLs might also be good to know.
You might also want to reconsider the decision of letting the user's know why they got the http.StatusInternalServerError response.
Setting the GOMAXPROCS > 1 if you have more the one core would definitely be a good idea but I would keep it less than number of cores available.
I searched around and as far as I can tell, POST form requests are already limited to 10MB (http://golang.org/src/net/http/request.go#L721).
If I were to go about reducing this in my ServeHTTP method, I'm not sure how to properly do it. I would try something like this:
r.Body = http.MaxBytesReader(w, r.Body, MaxFileSize)
err := r.ParseForm()
if err != nil {
//redirect to some error page
return
}
But would returning upon error close the connection as well? How would I prevent having to read everything? I found this: https://stackoverflow.com/a/26393261/2202497, but what if content length is not set and in the middle of reading I realize that the file is too big.
I'm using this as a security measure to prevent someone from hogging my server's resources.
The correct way to limit the size of the request body is to do as you suggested:
r.Body = http.MaxBytesReader(w, r.Body, MaxFileSize)
err := r.ParseForm()
if err != nil {
// redirect or set error status code.
return
}
MaxBytesReader sets a flag on the response when the limit is reached. When this flag is set, the server does not read the remainder of the request body and the server closes the connection on return from the handler.
If you are concerned about malicious clients, then you should also set Server.ReadTimeout, Server.WriteTimeout and possibly Server.MaxHeaderBytes.
If you want to set the request body limit for all of your handlers, then wrap root handler with a handler that sets the limit before delegating to the root handler:
type maxBytesHandler struct {
h http.Handler
n int64
}
func (h *maxBytesHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
r.Body = http.MaxBytesReader(w, r.Body, h.n)
h.h.ServeHTTP(w, r)
}
Wrap the root handler when calling ListenAndServe:
log.Fatal(http.ListenAndServe(":8080", &maxBytesHandler{h:mux, n:4096))
or when configuring a server:
s := http.Server{
Addr: ":8080",
Handler: &maxBytesReader{h:mux, n:4096},
}
log.Fatal(s.ListenAndServe())
There's no need for a patch as suggested in another answer. MaxBytesReader is the official way to limit the size of the request body.
Edit: As others cited MaxByteReader is the supported way. It is interesting that the default reader is instead, limitreader after type asserting for max byte reader.
Submit a patch to the Go source code and make it configurable! You are working with an open source project after all. Adding a setter to http.Request and some unit tests for it is probably only 20 minutes worth of work. Having a hardcoded value here is a bit clunky, give back and fix it :).
You can of course implement your own ParseForm(r *http.Request) method if you really need to override this. Go is essentially BSD, so you can copy paste the library ParseForm and change the limit, but thats a bit ugly no?
I'm new to the Go language.
I'm making a small web application with Go, the Gorilla toolkit, and the Mustache template engine.
Everything works great so far.
I use hoisie/mustache and gorilla/sessions, but I'm struggling with passing variables from one to the other. I have a map[string]interface{} that I pass to the template engine. When a user is logged in, I want to take the user's session data and merge it with my map[string]interface{} so that the data becomes available for rendering.
The problem is that gorilla/sessions returns a map[interface{}]interface{} so the merge cannot be done (with the skills I have in this language).
I thought about extracting the string inside the interface{} variable (reflection?).
I also thought about making my session data a map[interface{}]interface{} just like what gorilla/sessions provides. But I'm new to Go and I don't know if that can be considered best practice. As a Java guy, I feel like working with variables of type Object.
I would like to know the best approach for this problem in your opinion.
Thanks in advance.
You'll need to perform type assertions: specifically this section of Effective Go.
str, ok := value.(string)
if ok {
fmt.Printf("string value is: %q\n", str)
} else {
fmt.Printf("value is not a string\n")
}
A more precise example given what you're trying to do:
if userID, ok := session.Values["userID"].(string); ok {
// User ID is set
} else {
// User ID is not set/wrong type; raise an error/HTTP 500/re-direct
}
type M map[string]interface{}
err := t.ExecuteTemplate(w, "user_form.tmpl", M{"current_user": userID})
if err != nil {
// handle it
}
What you're doing is ensuring that the userID you pull out of the interface{} container is actually a string. If it's not, you handle it (if you don't, you'll program will panic as per the docs).
If it is, you pass it to your template where you can access it as {{ .current_user }}. M is a quick shortcut that I use to avoid having to type out map[string]interface{} every time I call my template rendering function.