I have written a simple http proxy in Go and need to read values of an HTTP POST sent parameters.
I have called request.ParseForm() in the application before create the reverse proxy and I got these parameteres but my reverse proxy stopped work. When I call request.ParseForm() after reverse proxy I get empty values.
error in go:
2018/10/31 12:26:45 http: panic serving [::1]:49967: runtime error: invalid
memory address or nil pointer dereference
goroutine 51 [running]:
net/http.(*conn).serve.func1(0xc0001c2000)
C:/Go/src/net/http/server.go:1746 +0xd7
panic(0x7ae340, 0xb00ba0)
C:/Go/src/runtime/panic.go:513 +0x1c7
main.(*myTransport).RoundTrip(0xb2bb10, 0xc000140400, 0xf, 0xc0000ec301, 0x3)
error in chrome: ERR_EMPTY_RESPONSE
func serveReverseProxy(target string, res http.ResponseWriter, req *http.Request) {
if req.Method == "POST" {
// req.ParseForm() here broke the reverse proxy but I get value of HTTP POST Param
req.ParseForm()
value := req.Form.Get("url")
fmt.Println(value)
// parse the url
url, _ := url.Parse(target)
// create the reverse proxy
proxy := httputil.NewSingleHostReverseProxy(url)
// Update the headers to allow for SSL redirection
req.URL.Path = url.Path
req.URL.Host = url.Host
req.URL.Scheme = url.Scheme
req.Header.Set("X-Forwarded-Host", req.Header.Get("Host"))
req.Host = url.Host
proxy.Transport = &myTransport{}
proxy.ServeHTTP(res, req)
// if I use req.ParseForm() here the reverse proxy works good but I get empty value of HTTP POST param
req.ParseForm()
value := req.Form.Get("url")
fmt.Println(value)
if(ableToSaveInDB){
handleInsert(res,req, value)
}
}
}
func (t *myTransport) RoundTrip(request *http.Request) (*http.Response, error) {
response, err := http.DefaultTransport.RoundTrip(request)
if(response.StatusCode == 200) {
ableToSaveInDB = true
}
return response, err
}
Related
In my Go API, I'm using gin, and I have one value set in my Access-Control-Allow-Origin header. If I have more than one, my react UI throws an error to the effect of The Access-Control-Allow-Origin header contains multiple values 'http://value1, http://value2', but only one is allowed.... I need to set multiple values. How do I do this?
The API is a reverse proxy, and here's the relevant code:
func proxy(c *gin.Context) {
var remote = "myUrl"
proxy := httputil.NewSingleHostReverseProxy(remote)
proxy.Director = func(req *http.Request) {
req.Header.Set("Authorization", "My Auth Values")
req.Host = remote.Host
req.URL.Scheme = remote.Scheme
req.URL.Host = remote.Host
}
proxy.ModifyResponse = addCustomHeader
proxy.ServeHTTP(c.Writer, c.Request)
}
func addCustomHeader(r *http.Response) error {
r.Header["Access-Control-Allow-Origin"] = []string{"value1"}
return nil
}
A CORS header can only contain a single value. If you want to implement your own CORS middleware, you need to work around that fact.
A simple CORS middleware will add the Access-Control-Allow-Origin header with the value of the specific address of the incoming request, usually taken from the Referer or Origin header. Typically, you match this against a list or map first, to see if it's in your allow list. If so, then the address of the request is added as allowed origin (as a single value).
A simple example could look like this
allowList := map[string]bool{
"https://www.google.com": true,
"https://www.yahoo.com": true,
}
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if origin := r.Header.Get("Origin"); allowList[origin] {
w.Header().Set("Access-Control-Allow-Origin", origin)
}
})
Since you are using the reverse proxy, you can access the request from the response.
mod := func(allowList map[string]bool) func(r *http.Response) error {
return func(r *http.Response) error {
if origin := r.Request.Header.Get("Origin"); allowList[origin] {
r.Header.Set("Access-Control-Allow-Origin", origin)
}
return nil
}
}
proxy := &httputil.ReverseProxy{
Director: func(r *http.Request) {
r.URL.Scheme = "https"
r.URL.Host = "go.dev"
r.Host = r.URL.Host
},
ModifyResponse: mod(allowList),
}
You only need a single value for each incoming request. The usual technique is to configure trusted origins on the server, eg:
trustedOrigins: [https://www.domain1.com, https://www.domain2.com]
Then check the runtime value of the origin header, which is sent by all modern browsers. If this is a trusted origin then add CORS headers:
Access-Control-Allow-Origin: https://www.domain2.com
A wildcard could be used but that is not recommended and also will not work as intended if you are also using credentialed requests (eg those with cookies).
Can you return json when http.Error is called?
myObj := MyObj{
MyVar: myVar}
data, err := json.Marshal(myObj)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Write(data)
w.Header().Set("Content-Type", "application/json")
http.Error(w, "some error happened", http.StatusInternalServerError)
I see that it returns 200 with no json but the json is embed in text
I've discovered that it's really easy to read the Go source. If you click on the function in the docs, you will be taken to the source for the Error function: https://golang.org/src/net/http/server.go?s=61907:61959#L2006
// Error replies to the request with the specified error message and HTTP code.
// It does not otherwise end the request; the caller should ensure no further
// writes are done to w.
// The error message should be plain text.
func Error(w ResponseWriter, error string, code int) {
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(code)
fmt.Fprintln(w, error)
}
So if you want to return JSON, it's easy enough to write your own Error function.
func JSONError(w http.ResponseWriter, err interface{}, code int) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(code)
json.NewEncoder(w).Encode(err)
}
It should be plain text only.
From docs
func Error(w ResponseWriter, error string, code int)
Error replies to the request with the specified error message and HTTP
code. It does not otherwise end the request; the caller should ensure
no further writes are done to w. The error message should be plain
text.
Also I think your usage of http.Error is not correct. When you call w.Write(data), the response is sent and response body will be closed. That is why you are getting 200 status instead of 500 from http.Error.
Instead of using http.Error, you can send your own error response with json just like how you would send any other response by setting the status code to an error code.
Like #ShashankV said, you are writing the response in a wrong way.
As an example, the following is what I did during learning about writing RESTful API serving in Golang:
type Response struct {
StatusCode int
Msg string
}
func respond(w http.ResponseWriter, r Response) {
// if r.StatusCode == http.StatusUnauthorized {
// w.Header().Add("WWW-Authenticate", `Basic realm="Authorization Required"`)
// }
data, err := json.Marshal(r)
if err != nil {
w.WriteHeader(http.StatusInternalServerError)
fmt.Fprintf(w, err.Error())
return
}
w.WriteHeader(r.StatusCode)
fmt.Fprintf(w, r.Msg)
}
func Hello(w http.ResponseWriter, r *http.Request) {
resp := Response{http.StatusOK, welcome}
respond(w, resp)
}
Ref: https://github.com/shudipta/Book-Server/blob/master/book_server/book_server.go
Hope, this will help.
My answer is a bit late and there are some good answers already. Here are my 2 cents.
If you want to return JSON in case of error there are multiple ways to do so. I can list two:
Write your own Error handler method
Use the go-boom library
1. Writing your own error handler method
One way is what #craigmj has suggested, i.e. create your own method, for eg.:
func JSONError(w http.ResponseWriter, err interface{}, code int) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.Header().Set("X-Content-Type-Options", "nosniff")
w.WriteHeader(code)
json.NewEncoder(w).Encode(err)
}
2. Use the go-boom library
Another approach is using the go-boom library. For eg., in case the err relates to resource not found, you can do:
err := errors.New("User doesn't exist")
...
boom.NotFound(w, err)
And the response will be:
{
"error": "Not Found",
"message": ",
"statusCode": 404
}
For more check the documentation of the go-boom.
Hope that helps.
I'm trying to log the response body of a request that has been redirected.
func main() {
r := gin.Default()
eanAPI := api.NewEanAPI()
v1 := r.Group("/v1")
v1.POST("/*action", eanAPI.Redirect, middleware.SaveRequest())
port := os.Getenv("PORT")
if len(port) == 0 {
port = "8000"
}
r.Run(":" + port)
}
func (api *eanAPI) Redirect(ctx *gin.Context) {
forwardToHost := "https://jsonplaceholder.typicode.com"
url := ctx.Request.URL.String()
ctx.Redirect(http.StatusTemporaryRedirect, forwardToHost)
}
type bodyLogWriter struct {
gin.ResponseWriter
body *bytes.Buffer
}
func (w bodyLogWriter) Write(b []byte) (int, error) {
w.body.Write(b)
return w.ResponseWriter.Write(b)
}
func SaveRequest() gin.HandlerFunc {
return func(c *gin.Context) {
c.Next()
blw := &bodyLogWriter{body: bytes.NewBufferString(""), ResponseWriter: c.Writer}
c.Writer = blw
c.Next()
statusCode := c.Writer.Status()
fmt.Println("status: ", statusCode)
if statusCode <= 400 {
//ok this is an request with error, let's make a record for it
// now print body (or log in your preferred way)
fmt.Println("Response body: " + blw.body.String())
}
}
Unfortunately, the body response is always empty. Maybe the redirection or the middleware is messing with my body response
When I tried with postman I can see the body response coming! You can try it by a POST on https://jsonplaceholder.typicode.com/posts. It should return a payload with an id in the body response
What am I doing wrong here?
Thanks in advance
The response body you're seeing in the browser/postman is from the page you're being redirected to. The page doing the actual redirecting likely has no response body, just a status and a Location header. This is the expected behavior of a redirect. If you want to try to capture the body of the page you're redirecting to, you can't actually use redirects for that (the redirect handler isn't involved in the final request); you'd have to fully proxy the request rather than redirecting it. Proxying is very different from redirecting though, so make sure that's really the behavior you're after.
I am trying to add context to Authorization middleware. The ContextHandler is a handler which will be passed to api handlers to take care of connections and config variables. A struct Method ServeHTTP also has been added to the ContextHandler so that it satisfies the net/Http interface for handling requests properly.
CheckAuth is the middle ware which takes in the request to check token validation etc, If token is valid, executes the ServeHTTP method and if not, Returns the appropriate error in the response.
Code compiles, but i am getting error in the ServeHTTP method.
type ContextHandler struct {
*AppContext
Handler func(*AppContext, http.ResponseWriter, *http.Request)(int, error)
}
type AppContext struct {
Db *mgo.Session
Config *simplejson.Json
}
func (ah *ContextedHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
status, err := ah.handler(ah.AppContext, w, r)
if err != nil {
switch status {
case http.StatusNotFound:
http.NotFound(w, r)
case http.StatusInternalServerError:
http.Error(w, http.StatusText(status), status)
default:
http.Error(w, http.StatusText(405), 405)
}}}
func CheckAuth(h http.Handler) http.Handler {
log.Println("Entered in CheckAuth")
f := func( w http.ResponseWriter, r *http.Request) {
authorizationToken := r.Header.Get("Authorization")
if authorizationToken != ""{
secret := []byte("somejunk")
var credentials authorization
token, err := jwt.ParseWithClaims(authorizationToken, &credentials, func(t *jwt.Token) (interface{}, error) {
return []byte(secret), nil
})
if err == nil && token.Valid {
//If everything is fine serve the Http request
h.ServeHTTP( w, r)
return
} else {
//Some response returned
json.NewEncoder(w).Encode(response)
return
}
//Check if user exists in the database
if dberr != nil {
//SOmeresponse to be returned
json.NewEncoder(w).Encode(response)
return
}
}else{
response := simplejson.New()
//response authorization header is missing
json.NewEncoder(w).Encode(response)
return
}
}
return http.HandlerFunc(f)
}
func Initdb(configfile *simplejson.Json) *mgo.Session {
//return mongodbsession, copy and close while using it
}
In main.go file in the parent package
func main() {
var FileUploadContextHandler *ContextedHandler = &ContextedHandler{&context, filesystem.FileUpload}
router.Methods("POST").Path("/decentralizefilesystem/fileupload").Name("FileUpload").Handler(CheckAuth(FileUploadContextHandler))
}
I am getting this error
2018/07/08 20:45:38 http: panic serving 127.0.0.1:52732: runtime error: invalid memory address or nil pointer dereference
goroutine 35 [running]:
net/http.(*conn).serve.func1(0xc4202ce140)
/usr/local/go/src/net/http/server.go:1726 +0xd0
panic(0x6fe680, 0x92cb10)
/usr/local/go/src/runtime/panic.go:502 +0x229
gitlab.com/mesha/Gofeynmen/vendor/gopkg.in/mgo%2ev2.(*Session).Copy(0x0, 0x7ff9485fb060)
/home/feynman/goworkspace/src/gitlab.com/mesha/Gofeynmen/vendor/gopkg.in/mgo.v2/session.go:1589 +0x22
gitlab.com/mesha/Gofeynmen/appsettings.CheckAuth.func1(0x7ff9485fb060, 0xc420276300, 0xc4202e4200)
/home/feynman/goworkspace/src/gitlab.com/mesha/Gofeynmen/appsettings/appsettings.go:115 +0x361
net/http.HandlerFunc.ServeHTTP(0xc420290180, 0x7ff9485fb060, 0xc420276300, 0xc4202e4200)
/usr/local/go/src/net/http/server.go:1947 +0x44
github.com/gorilla/mux.(*Router).ServeHTTP(0xc42024a310, 0x7ff9485fb060, 0xc420276300, 0xc4202e4200)
/home/feynman/goworkspace/src/github.com/gorilla/mux/mux.go:162 +0xed
github.com/gorilla/handlers.loggingHandler.ServeHTTP(0x7a8120, 0xc42000e018, 0x7a7b20, 0xc42024a310, 0x7aad60, 0xc4202f0000, 0xc4202e4000)
/home/feynman/goworkspace/src/github.com/gorilla/handlers/handlers.go:69 +0x123
github.com/gorilla/handlers.(*cors).ServeHTTP(0xc4202c4090, 0x7aad60, 0xc4202f0000, 0xc4202e4000)
/home/feynman/goworkspace/src/github.com/gorilla/handlers/cors.go:52 +0xa3b
net/http.serverHandler.ServeHTTP(0xc4202da0d0, 0x7aad60, 0xc4202f0000, 0xc4202e4000)
/usr/local/go/src/net/http/server.go:2694 +0xbc
net/http.(*conn).serve(0xc4202ce140, 0x7ab120, 0xc42025e100)
/usr/local/go/src/net/http/server.go:1830 +0x651
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2795 +0x27b
It's likely an attempt to dereference ah from (ah *ContextedHandler), when ah is not a pointer to a ContextedHandler.
The types in this assignment don't match up:
var FileUploadContextHandler *ContextedHandler =
ContextedHandler{&context, filesystem.FileUpload}
On the left side you have type *ContextedHandler. On the right side you have type ContextedHandler.
Did you mean
var FileUploadContextHandler *ContextedHandler =
&ContextedHandler{&context, filesystem.FileUpload}
Or did you mean
var FileUploadContextHandler ContextedHandler =
ContextedHandler{&context, filesystem.FileUpload}
?
The argument passed to the CheckAuth function appears to not match the function signature either:
CheckAuth(FileUploadContextHandler)
FileUploadContextHandler is type *ContextedHandler. The function signature is:
func CheckAuth(h contextHandlerFunc) contextHandlerFunc
The type definition of contextHandlerFunc does not appear to be part of the code you shared.
A problem with this line:
router.Methods("POST").Path("/decentralizefilesystem/fileupload").Name("FileUpload").Handler(CheckAuth(FileUploadContextHandler))
...would be easier to track down if you broke it up into variable assignments on several lines and then figured out which line the panic pointed to.
Hopefully, this is an easy way to earn some rep. This seems very simple, so I must be doing something wrong and just cant see it.
I have a simple middleware which a transaction id and adds it to the request and response headers.
func HandleTransactionID(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
tid := uuid.NewV4()
req.Header.Set(TransIDHeader, TransIDPrefix + tid.String())
w.Header().Set(TransIDHeader, TransIDPrefix + tid.String())
fn(w, req)
}
}
In my unit tests, I've confirmed the response header is successfully set, but it doesn't appear the the request header is being set. I would assume that it is possible to modify the request headers, so ?
const (
WriteTestHeader = "WriterTransHeader"
RequestTestHeader = "ReqTransHeader"
)
func recorderFunc(w http.ResponseWriter, req *http.Request){
w.Header().Set(WriteTestHeader, w.Header().Get(TransIDHeader))
w.Header().Set(RequestTestHeader, req.Header.Get(TransIDHeader))
}
func TestHandleTransactionID(t *testing.T) {
recorder := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/foo", nil)
middleware.HandleTransactionID(recorderFunc)(recorder, req)
if req.Header.Get(RequestTestHeader) == "" {
t.Error("request header is nil")
}
if recorder.Header().Get(WriteTestHeader) == "" {
t.Error("response header is nil")
}
if req.Header.Get(RequestTestHeader) != recorder.Header().Get(WriteTestHeader) {
t.Errorf("header value mismatch: %s != %s",
req.Header.Get(RequestTestHeader),
recorder.Header().Get(WriteTestHeader))
}
}
In your test, req.Header.Get(RequestTestHeader) will always remain an empty string because you are not setting the Key as 'RequestTestHeader' in the request header but in the ResponseWriter w.Header().Set(RequestTestHeader, req.Header.Get(TransIDHeader))
On an unrelated note, It would be considered idomatic Go, to have your middleware function signature using the http.Handler interface, func HandleTransactionID(fn http.Handler) http.Handler.