How to debug httputil.NewSingleHostReverseProxy - go

Question

Problem:
I'm fowarding to a HTTPS address.
I want to see why removing
req.Host = req.URL.Host causes it to fail. Instead of returning {"Code":"OBRI.FR.Request.Invalid","Id":"c37baec213dd1227","Message":"An error happened when parsing the request arguments","Errors":[{"ErrorCode":"UK.OBIE.Header.Missing","Message":"Missing request header 'x-fapi-financial-id' for method parameter of type String","Url":"https://docs.ob.forgerock.financial/errors#UK.OBIE.Header.Missing"}]}
it returns a 404.
I want to trace the call the proxy returned from
httputil. NewSingleHostReverseProxy is making when I uncommment the line req.Host = req.URL.Host.
Given a request as such:
$ curl http://localhost:8989/open-banking/v2.0/accounts
And the code below (main.go):
package main
import (
"log"
"net/http"
"net/http/httputil"
"net/url"
)
func main() {
target, err := url.Parse("https://rs.aspsp.ob.forgerock.financial:443")
log.Printf("forwarding to -> %s%s\n", target.Scheme, target.Host)
if err != nil {
log.Fatal(err)
}
proxy := httputil.NewSingleHostReverseProxy(target)
http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
// https://stackoverflow.com/questions/38016477/reverse-proxy-does-not-work
// https://forum.golangbridge.org/t/explain-how-reverse-proxy-work/6492/7
// https://stackoverflow.com/questions/34745654/golang-reverseproxy-with-apache2-sni-hostname-error
req.Host = req.URL.Host // if you remove this line the request will fail... I want to debug why.
proxy.ServeHTTP(w, req)
})
err = http.ListenAndServe(":8989", nil)
if err != nil {
panic(err)
}
}

Answer 1

Set the proxy.Transport field to an implementation that dumps the request before delegating to the default transport:
package main
import (
"fmt"
"log"
"net/http"
"net/http/httputil"
"net/url"
)
type DebugTransport struct{}
func (DebugTransport) RoundTrip(r *http.Request) (*http.Response, error) {
b, err := httputil.DumpRequestOut(r, false)
if err != nil {
return nil, err
}
fmt.Println(string(b))
return http.DefaultTransport.RoundTrip(r)
}
func main() {
target, _ := url.Parse("https://example.com:443")
log.Printf("forwarding to -> %s\n", target)
proxy := httputil.NewSingleHostReverseProxy(target)
proxy.Transport = DebugTransport{}
http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
req.Host = req.URL.Host
proxy.ServeHTTP(w, req)
})
log.Fatal(http.ListenAndServe(":8989", nil))
}
The output from this program looks something like this:
2018/10/26 13:06:35 forwarding to -> https://example.com:443
GET / HTTP/1.1
Host: example.com:443
User-Agent: HTTPie/0.9.4
Accept: */*
Accept-Encoding: gzip, deflate
X-Forwarded-For: 127.0.0.1
Or, after removing the req.Host assignment:
2018/10/26 13:06:54 forwarding to -> https://example.com:443
GET / HTTP/1.1
Host: localhost:8989
User-Agent: HTTPie/0.9.4
Accept: */*
Accept-Encoding: gzip, deflate
X-Forwarded-For: 127.0.0.1
Since the Host header is very often used by webservers to route the request to the correct virtual host or backend server, it makes sense that an unexpected Host header ("localhost:8989" in the example above) causes the server to respond with 404.
Setting the Host header with httputil.ReverseProxy is typically done with the Director function:
target, err := url.Parse("https://example.com:443")
if err != nil {
log.Fatal(err)
}
log.Printf("forwarding to -> %s\n", target)
proxy := httputil.NewSingleHostReverseProxy(target)
d := proxy.Director
proxy.Director = func(r *http.Request) {
d(r) // call default director
r.Host = target.Host // set Host header as expected by target
}
log.Fatal(http.ListenAndServe(":8989", proxy))