How to write test with httprouter - go

I am writing tests for a simple REST service in GoLang. But, because I am using julienschmidt/httprouter as the routing library. I am struggling on how to write test.
main.go
package main
func main() {
router := httprouter.New()
bookController := controllers.NewBookController()
router.GET("/book/:id", bookController.GetBook)
http.ListenAndServe(":8080", router)
}
controllers
package controllers
type BookController struct {}
func NewBookController *BookController {
return &BookController()
}
func (bc BookController) GetBook(w http.ResponseWriter, r *http.Request, p httprouter.Params) {
fmt.Fprintf(w,"%s", p)
}
My question is: How can test this while GetBook is neither HttpHandler nor HttpHandle
If I use a traditional handler, the test will be easy like this
func TestGetBook(t *testing.T) {
req, _ := http.NewRequest("GET", "/book/sampleid", nil)
rr := httptest.NewRecorder()
handler := controllers.NewBookController().GetBook
handler.ServeHTTP(rr,req)
if status := rr.code; status != http.StatusOK {
t.Errorf("Wrong status")
}
}
The problem is, httprouter is not handler, or handlefunc. So I am stuck now

Just spin up a new router for each test and then register the handler under test, then pass the test request to the router, not the handler, so that the router can parse the path parameters and pass them to the handler.
func TestGetBook(t *testing.T) {
handler := controllers.NewBookController()
router := httprouter.New()
router.GET("/book/:id", handler.GetBook)
req, _ := http.NewRequest("GET", "/book/sampleid", nil)
rr := httptest.NewRecorder()
router.ServeHTTP(rr, req)
if status := rr.Code; status != http.StatusOK {
t.Errorf("Wrong status")
}
}

You need to wrap your handler so that it can be accessed as an http.HandlerFunc:
func TestGetBook(t *testing.T) {
req, _ := http.NewRequest("GET", "/book/sampleid", nil)
rr := httptest.NewRecorder()
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
controllers.NewBookController().GetBook(w, r, httprouter.Params{})
})
handler.ServeHTTP(rr,req)
if status := rr.code; status != http.StatusOK {
t.Errorf("Wrong status")
}
}
If your handler requires parameters, you'll either have to parse them manually from the request, or just supply them as the third argument.

u can try this without ServeHTTP
func TestGetBook(t *testing.T) {
req := httptest.NewRequest("GET", "http://example.com/foo", nil)
w := httptest.NewRecorder()
controllers.NewBookController().GetBook(w, req, []httprouter.Param{{Key: "id", Value: "101"}})
resp := w.Result()
body, _ := ioutil.ReadAll(resp.Body)
t.Log(resp.StatusCode)
t.Log(resp.Header.Get("Content-Type"))
t.Log(string(body))
}

here is one more good blog and corresponding working code for this:
https://medium.com/#gauravsingharoy/build-your-first-api-server-with-httprouter-in-golang-732b7b01f6ab
https://github.com/gsingharoy/httprouter-tutorial/blob/master/part4/handlers_test.go

Related

How to rotate between multiple forwarding proxies for outgoing requests with golang

I will like to pass a list of forwarding proxy servers for POST request
Currently i am able to do it with just single forwarding proxy
serverProxy := "http://user:password#123.45.67.89:3128"
request, error := http.NewRequest("POST", httpposturl, bytes.NewBuffer(requestJSON))
request.Header.Set("Content-Type", "application/json; charset=UTF-8")
proxyURL, _ := url.Parse(serverProxy)
proxy := http.ProxyURL(proxyURL)
transport := &http.Transport{Proxy: proxy}
client := &http.Client{Transport: transport}
what i will like to do is pass a list to url.Parse and want it to use them using round robin balancing
so something like this
serverProxy := "http://user:password#123.45.67.89:3128, http://user:password#223.45.67.89:3128"
and then it will select which of the proxy servers to use and rotate them within requests
Is this possible?
UPDATE:
I want to be able to pass the rotated proxy server like this
proxyServer := roundRobin("http://round:robin#123.45.67.89:3128, http://robin:round#223.45.67.89:3128")
fmt.Println("proxy server used", proxyServer, "\n")
transport := &http.Transport{Proxy: proxyServer}
client := &http.Client{Transport: transport}
Create a proxy function that round-robins through your proxy URLs. Use that function in your transport:
func roundRobin(urls []*url.URL) func(*http.Request) (*url.URL, error) {
var mu sync.Mutex
var i int
return func(r *http.Request) (*url.URL, error) {
mu.Lock()
i = (i + 1) % len(urls)
u := urls[i]
mu.Unlock()
return u, nil
}
}
transport := &http.Transport{Proxy: roundRobin(yourProxyURLs)}
client := &http.Client{Transport: transport}
Here's the Montage's answer with explanation:
The requirement is to forward request through proxies in round-robin fashion
Since we are using http.Client to make request we can look at http.Client documentation to see if it provide any support for forwarding request through proxy when we look at the documentation we can see that it does support passing proxy which we can pass through http.Transport type which will be passed to http.Client. http.Transport takes proxy through Proxy field which takes in func that return *url.URL and error there are existing methods like http.ProxyURL and http.ProxyFromEnvironment provided within http package that we can use to pass proxies to http.Transport but the problem with these methods is that they only take a single proxy server which does not solve our problem at hand and hence we would require to create our own function which takes in multiple proxy servers urls and round-robin between them.
If we look at one of the existing method implemention as our base for creating our own method lets go with http.ProxyURL for our case the implementation can be found here. I have copied the implementation below
func ProxyURL(fixedURL *url.URL) func(*Request) (*url.URL, error) {
return func(*Request) (*url.URL, error) {
return fixedURL, nil
}
}
we can see that its a simple closure which takes in single url and return a closure function which then intern return the url passed in as parameter. so we can take it base and create our own round-robin clouse function
func roundRobin(proxies ...string) func(*http.Request) (*url.URL, error) {
var urls []*url.URL
for _, proxy := range proxies {
u, err := url.Parse(proxy)
if err != nil {
log.Fatal(err)
}
urls = append(urls, u)
}
var mu sync.Mutex
var i, lenUrls int = 0, len(urls)
return func(r *http.Request) (*url.URL, error) {
mu.Lock()
i = (i + 1) % lenUrls
u := urls[i]
mu.Unlock()
return u, nil
}
}
Lets go over the roundRobin function implementation it is a variadic function which takes in proxy url(s) in string format as argument, which internally gets converted to url.URL by parsing the string using url.Parse then using the parsed url.URL to create slice of urls []*url.URL which then being used to forward request in round-robin fashion
Complete working example can be found below:
package main
import (
"fmt"
"log"
"net/url"
"net/http"
"sync"
)
func roundRobin(proxies ...string) func(*http.Request) (*url.URL, error) {
var urls []*url.URL
for _, proxy := range proxies {
u, err := url.Parse(proxy)
if err != nil {
log.Fatal(err)
}
urls = append(urls, u)
}
var mu sync.Mutex
var i, lenUrls int = 0, len(urls)
return func(r *http.Request) (*url.URL, error) {
mu.Lock()
i = (i + 1) % lenUrls
u := urls[i]
mu.Unlock()
return u, nil
}
}
func main() {
proxyFn := roundRobin("http://user:password#123.45.67.89:3128", "http://user:password#223.45.67.89:3128")
transport := &http.Transport{Proxy: proxyFn}
client := &http.Client{Transport: transport}
req, err := http.NewRequest("POST", "http://example.com", nil)
req.Header.Set("Content-Type", "application/json; charset=UTF-8")
resp, err := client.Do(req)
if err != nil {
fmt.Println(resp)
}
fmt.Println(proxyFn(nil))
fmt.Println(proxyFn(nil))
}
Playground
Another version
package main
import (
"fmt"
"log"
"net/http"
"net/url"
"sync"
)
func praseUrls(proxies ...string) (urls []*url.URL) {
for _, proxy := range proxies {
u, err := url.Parse(proxy)
if err != nil {
log.Fatal(err)
}
urls = append(urls, u)
}
return
}
func roundRobin(max int) func() int {
var i int
return func() int {
i = (i + 1) % max
return i
}
}
func proxyFn(urls []*url.URL) func(*http.Request) (*url.URL, error) {
var m sync.Mutex
fn := roundRobin(len(urls))
return func(*http.Request) (*url.URL, error) {
m.Lock()
u := urls[fn()]
m.Unlock()
return u, nil
}
}
func main() {
proxies := []string{"http://user:password#123.45.67.89:3128", "http://user:password#223.45.67.89:3128"}
urls := praseUrls(proxies...)
transport := &http.Transport{Proxy: proxyFn(urls)}
client := &http.Client{Transport: transport}
req, err := http.NewRequest("POST", "http://example.com", nil)
req.Header.Set("Content-Type", "application/json; charset=UTF-8")
resp, err := client.Do(req)
if err != nil {
fmt.Println(resp)
}
}
Playground
Note: It would be better to pass proxy urls from env variable which would help in case any proxy server changes or new are added
Here's the Montage's answer with code to parse a string.
func roundRobin(serverProxy string) func(*http.Request) (*url.URL, error) {
parts := strings.Split(serverProxy, ",")
var urls []*url.URL
for _, part := range parts {
u, err := url.Parse(strings.TrimSpace(part))
if err != nil {
log.Fatal(err)
}
urls = append(urls, u)
}
var mu sync.Mutex
var i int
return func(r *http.Request) (*url.URL, error) {
mu.Lock()
i = (i + 1) % len(urls)
u := urls[i]
mu.Unlock()
return u, nil
}
}
serverProxy := "http://user:password#123.45.67.89:3128, http://user:password#223.45.67.89:3128"
transport := &http.Transport{Proxy: roundRobin(serverProxy)}
client := &http.Client{Transport: transport}

Gorila mux's subroute routing behaviour

I want to build a simple api server, and wanted to use gorila mux subrouter and faced the following issues
//Sample function
func Func1(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello"))
w.WriteHeader(200)
}
func Func2(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("World"))
w.WriteHeader(200)
}
//Issue 1
func TestRouter4(t *testing.T) {
router := mux.NewRouter()
subRouter := router.PathPrefix("/tenant").Subrouter()
subRouter.Methods("GET").HandlerFunc(Func1)
subRouter = subRouter.PathPrefix("/{id:[0-9]+}").Subrouter()
subRouter.Methods("GET").HandlerFunc(Func2)
req := httptest.NewRequest(http.MethodGet, "/tenant", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
blob, _ := ioutil.ReadAll(w.Body)
assert.Equal(t, string(blob), "Hello")
assert.Equal(t, w.Result().StatusCode, http.StatusOK)
req = httptest.NewRequest(http.MethodGet, "/tenant/31231", nil)
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
blob, _ = ioutil.ReadAll(w.Body)
assert.Equal(t, w.Result().StatusCode, http.StatusOK)
assert.Equal(t, string(blob), "World") // Test assertion fails because Func1 is called
}
The above testcase passes if the I change the registration to the following
router := mux.NewRouter()
tenantSubRouter := router.PathPrefix("/tenant").Subrouter()
searchSubRouter := tenantSubRouter.PathPrefix("/{search:[0-9]+}").Subrouter()
searchSubRouter.Methods("GET").HandlerFunc(Func2)
tenantSubRouter.Methods("GET").HandlerFunc(Func1)
//Issue 2
func TestRouter5(t *testing.T) {
router := mux.NewRouter()
tenantSubRouter := router.PathPrefix("/tenant").Subrouter()
searchSubRouter := tenantSubRouter.PathPrefix("/{search:[0-9]+}").Subrouter()
searchSubRouter.Methods("GET").HandlerFunc(Func2)
tenantSubRouter.Methods("GET").HandlerFunc(Func1)
req := httptest.NewRequest(http.MethodGet, "/tenant", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
blob, _ := ioutil.ReadAll(w.Body)
assert.Equal(t, string(blob), "Hello")
assert.Equal(t, w.Result().StatusCode, http.StatusOK)
req = httptest.NewRequest(http.MethodGet, "/tenant/31231", nil)
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
blob, _ = ioutil.ReadAll(w.Body)
assert.Equal(t, string(blob), "World")
assert.Equal(t, w.Result().StatusCode, http.StatusOK)
req = httptest.NewRequest(http.MethodGet, "/tenant/adffasd", nil)
w = httptest.NewRecorder()
router.ServeHTTP(w, req)
blob, _ = ioutil.ReadAll(w.Body)
assert.Equal(t, w.Result().StatusCode, http.StatusNotFound) // Assertion fails as the response is with status code 200 , calls Func1 mapped to route /tenant
}
The above testcase passes with the following code
router := mux.NewRouter()
router.HandleFunc("/tenant", Func1).Methods("GET")
router.HandleFunc("/tenant/{search:[0-9]+}", Func2).Methods("GET")
My inference from this that the PathPrefix().Subrouter() has some special behavior and use cases. Am I missing some config for PathPrefix().Subrouter() to work properly or does these have a special use case, if so what is that use case ?

How to test a Oauth2.0 resource of server

I want to code the test for validate the right document to reach to the Oauth2.0 of third party server, how should i complete the pseudocode?
import (
"net/http"
"net/http/httptest"
}
func testAuthServer(t *testing.T) {
form := url.Values{}
form.Set(...)
r := httptest.NewRequest(http.MethodPost, authUrl, strings.NewReader(form.Encode()))
r.Header.Set("Content-Type", "application/x-www-form-urlencoded")
w := httptest.NewRecorder()
// test the auth server
if w.Code != http.StatusOK {
...
}
}
You can rely on a third party library to mock the resource. You can take a look at gock.
func TestServer(t *testing.T) {
defer gock.Off()
authURL := "http://third-party-resource.com"
form := url.Values{}
form.Add("foo", "bar")
// Create the mock of the third-party resource. We assert that the code
// calls the resource with a POST request with the body set to "foo=bar"
gock.New(authURL).
Post("/").
BodyString("foo=bar").
Reply(200)
r, err := http.NewRequest(http.MethodPost, authURL, strings.NewReader(form.Encode()))
if err != nil {
t.Fatal(err)
}
r.Header.Set("Content-Type", "application/x-www-form-urlencoded")
c := http.Client{}
_, err = c.Do(r)
if err != nil {
t.Fatal(err)
}
if !gock.IsDone() {
// The mock has not been called.
t.Fatal(gock.GetUnmatchedRequests())
}
}
Finally i use the normal http client to resolve this problem.
import (
"net/http"
}
func testAuthServer(t *testing.T) {
form := url.Values{}
form.Set(...)
authReq := http.NewRequest(http.MethodPost, authUrl, strings.NewReader(form.Encode()))
authReq.Header.Set("Content-Type", "application/x-www-form-urlencoded")
authClient, _ := http.Client{}
authResp, _ := authClient.Do(authReq)
if authResp.Code != http.StatusOK {
...
}
}

How to correctlly create mock http.request in golang for my test case?

I want to write a test case to verify my parameter parser function.
The following is my sample code to mock a http.request
rawUrl := "http://localhost/search/content?query=test"
func createSearchRequest(rawUrl string) SearchRequest {
api := NewWebService()
req, err := http.NewRequest("POST", rawUrl, nil)
if err != nil {
logger.Fatal(err)
}
logger.Infof("%v", req)
return api.searchRequest(req)
}
My web server use github.com/gorilla/mux as route
router := mux.NewRouter()
router.HandleFunc("/search/{query_type}", searchApiService.Search).Methods("GET")
But in my test case I cannot get {query_type} from mock http.request
func (api WebService) searchRequest(req *http.Request){
// skip ....
vars := mux.Vars(req)
queryType := vars["query_type"]
logger.Infof("queryType:%v", queryType)
//skip ....
}
How can I get mux's path parameter in my test case?
func TestMaincaller(t *testing.T) {
r,_ := http.NewRequest("GET", "hello/1", nil)
w := httptest.NewRecorder()
//create a map of variable and set it into mux
vars := map[string]string{
"parameter_name": "parametervalue",
}
r = mux.SetURLVars(r, vars)
callfun(w,r)
}

How can I inject a specific IP address in the test server ? Golang

I'm trying to test an application which provides information based on ip address. However I can't find how to set the Ip address manually . Any idea ?
func TestClientData(t *testing.T) {
URL := "http://home.com/hotel/lmx=100"
req, err := http.NewRequest("GET", URL, nil)
if err != nil {
t.Fatal(err)
}
req.RemoveAddr := "0.0.0.0" ??
w := httptest.NewRecorder()
handler(w, req)
b := w.Body.String()
t.Log(b)
}
The correct line would be:
req.RemoteAddr = "0.0.0.0"
You don't need the :=. It won't work because you don't create a new variable.
Like this (on playground http://play.golang.org/p/_6Z8wTrJsE):
package main
import (
"io"
"log"
"net/http"
"net/http/httptest"
)
func handler(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "Got request from ")
io.WriteString(w, r.RemoteAddr)
}
func main() {
url := "http://home.com/hotel/lmx=100"
req, err := http.NewRequest("GET", url, nil)
if err != nil {
log.Fatal(err)
}
// can't use := here, because RemoteAddr is a field on a struct
// and not a variable
req.RemoteAddr = "127.0.0.1"
w := httptest.NewRecorder()
handler(w, req)
log.Print(w.Body.String())
}

Resources