func forwarderHandlerFunc(w http.ResponseWriter, r *http.Request) {
client := &http.Client{}
u, _ := url.Parse(r.RequestURI)
req, _ := http.NewRequest(r.Method, fmt.Sprintf("%s%s", apiUrl, u.Path), r.Body)
fmt.Printf(fmt.Sprintf("%s\n", nutils.ReaderToString(req.Body)))
resp, _ := client.Do(req)
resp.Write(w)
}
I am trying to forward an incoming HTTP request to another endpoint, while copying the body, including POST/PUT form data into the new request.
However, it doesn't seem to work, even if the Body seems to print out correct with data.
Print output is:
email=meh%!g(MISSING)mail.com
How can I fix it?
Edit: Added more debug info, this time, printing out the output of resp
func forwarderHandlerFunc(w http.ResponseWriter, r *http.Request) {
client := &http.Client{}
u, _ := url.Parse(r.RequestURI)
req, _ := http.NewRequest(r.Method, fmt.Sprintf("%s%s", apiUrl, u.Path), r.Body)
fmt.Printf(fmt.Sprintf("%s\n", nutils.ReaderToString(req.Body)))
resp, _ := client.Do(req)
b,_ := ioutil.ReadAll(resp.Body)
fmt.Printf(fmt.Sprintf("%s\n", nutils.BytesToString(b)))
resp.Write(w)
}
$ go install && gom-proxy-forwarder run --listen localhost:5002 --api-url http://localhost:5001
email=meh2%!g(MISSING)mail.com
{
"email": null
}
It should not be null. It should be meh#gmail.com
Got it. The problem was my endpoint in Python's Flask server does not support chunked encoding, which Go's Request insists on.
When I manually specified the ContentLength like req.ContentLength = 25, it worked.
Lesson learnt: It might not always be your Go code be the problem.
You may want to set the request content type
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
reference http://golang.org/src/net/http/client.go?s=14234:14316#L450
To fix your print output you need to change this:
fmt.Printf(fmt.Sprintf("%s\n", nutils.ReaderToString(req.Body)))
Into this:
fmt.Printf("%s", fmt.Sprintf("%s\n", nutils.ReaderToString(req.Body)))
Or this:
fmt.Println(fmt.Sprintf("%s\n", nutils.ReaderToString(req.Body)))
By printing out the request body you are consuming it. Use a TeeReader:
req, _ := http.NewRequest(r.Method, fmt.Sprintf("%s%s", apiUrl, u.Path), io.TeeReader(r.Body, os.Stdout))
And get rid of the nutils.ReaderToString call. You can only read once from a Reader (unless it's also a Seeker but then you still need to Seek it before reusing it)
Related
I just started to use Golang and I want to remake my already working NodeJS/TypeScript app in Go.
One endpoint of my API simply adds server-side generated authorization headers and sends a request to a remote API. Basically filling those headers for me by calling my API instead of the remote API.
This is what I am currently writing
func Endpoint(ctx *fiber.Ctx) error {
url := "https://api.twitch.tv" + ctx.OriginalURL()
req, _ := http.NewRequest(http.MethodGet, url, nil)
req.Header.Set("Authorization", "Bearer ---------")
req.Header.Set("Client-Id", "---------")
client := &http.Client{}
res, err := client.Do(req)
// temporary error handling
if err != nil {
log.Fatalln(err)
}
body, err := ioutil.ReadAll(res.Body)
// temporary error handling
if err != nil {
log.Fatalln(err)
}
var forwardedBody interface{}
json.Unmarshal(body, &forwardedBody)
return ctx.Status(fiber.StatusOK).JSON(forwardedBody)
}
I'd like to know if I am on the right steps, because making a request, parsing the JSON response with ioutil then unmarshall it to send it back seems kind of overboard for the simplicity of what I am trying to achieve ?
Edit: Thank you for the help, this is what I will be going for
func Endpoint(ctx *fiber.Ctx) error {
url := "https://api.twitch.tv" + ctx.OriginalURL()
req, _ := http.NewRequest(http.MethodGet, url, nil)
req.Header.Set("Authorization", "Bearer ---------")
req.Header.Set("Client-ID", "---------")
client := &http.Client{}
res, err := client.Do(req)
if err != nil {
return ctx.SendStatus(fiber.StatusBadRequest)
}
ctx.Set("Content-Type", "application/json; charset=utf-8")
return ctx.Status(res.StatusCode).SendStream(res.Body)
}
You can use httputil.ReverseProxy. Which takes a base URL and forwards requests to the base URL, concatenating the path.
ReverseProxy is an HTTP Handler that takes an incoming request and sends it to another server, proxying the response back to the client.
http.Handle("/", &httputil.ReverseProxy{
Director: func(r *http.Request) {
r.URL.Scheme = "https"
r.URL.Host = "go.dev"
r.Host = r.URL.Host
r.Header.Set("X-Foo", "Bar")
},
})
If you are not serving this from the root path / you can use StripPrefix.
http.HandleFunc("/foo/", http.StripPrefix("/foo/", proxy)
There is also a helper function NewSingleHostReverseProxy, which possibly removes the need to configure the proxy struct yourself. But I think it will be better to set the Host header along with your custom header.
You don't need to attempt to parse the data as JSON. This will be problematic if any of your endpoints don't return JSON, anyway, so just inject the body directly into the response:
body, err := ioutil.ReadAll(res.Body)
// temporary error handling
if err != nil {
log.Fatalln(err)
}
// Inject the body from the inner response into the actual response so it can be returned
ctx.Response().SetBody(body)
return cx.Status(fiber.StatusOK)
I'm currently attempting to make a POST request using the HTTP package in Go. In the body of the request, it needs a 'code' pulled from the query of the API call to complete the request.
However, I first need to declare the req to add the query URL values in to do this. So I'm stuck with declaring q as url.Values{}, passing that in to the body of my post, and then having to add the values after the initial HTTP declaration.
But because I'm passing q in to the request before adding these values, the request URL doesn't include them when I'm sending the request. So I'm essentially just sending a blank query (I think).
So how can I get around this and pass in the query details to my http request but access the query value?
Hopefully that makes sense - it's confusing!
Here's my code:
func Fetch(w http.ResponseWriter, r *http.Request) {
client := &http.Client{}
q := url.Values{}
req, err := http.NewRequest("POST", "https://auth.truelayer-sandbox.com/connect/token", strings.NewReader(q.Encode()))
if err != nil {
log.Print(err)
fmt.Println("Error was not equal to nil at first stage.")
os.Exit(1)
}
q.Add("grant_type", "authorization_code")
q.Add("id", os.Getenv("ID"))
q.Add("secret", os.Getenv("SECRET"))
q.Add("redirect_uri", "https://callback.com")
q.Add("query-param", req.URL.Query().Get("query-param"))
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
resp, err := client.Do(req)
if err != nil {
fmt.Println("Error sending request to server")
os.Exit(1)
}
respBody, _ := ioutil.ReadAll(resp.Body)
w.WriteHeader(resp.StatusCode)
w.Write(respBody)
}
Creating a request, then writing it
var w CustomWriter
req, _ := http.NewRequest("POST", "/Foo", bytes.NewReader([]byte(<very long string>)))
req.Write(w)
func (w *CustomWriter) Write(request []byte) (int, error) {
fmt.Println(len(request))
return 0, nil
}
Output: 4096
But the body of my request is noticeably longer, but it just gets cut off when trying to write it. How do I write and send this HTTP Request that has a very long body without the body being lost or cut off?
This is not how you execute HTTP requests in Go. You need to do
req, _ := http.NewRequest("POST", "/Foo", bytes.NewReader([]byte(<very long string>)))
client := &http.Client{}
resp, err := client.Do(req)
If you are ok with the request being turned into a huge array, this is already part of the Go stdlib:
import "net/http/httputil"
...
rData, err := httputil.DumpRequest(r, true)
...
...
func debugResponse(resp *http.Response) {
dump, _ := httputil.DumpResponse(resp, true)
fmt.Println("DEBUG:")
fmt.Printf("%q", dump)
}
Once you consume the response or request, you have to assume that it has been held in memory. Use the flag false if you don't want to print the body because it's large or binary.
I have two go programs - one running as a server daemon, the other being executed manually. I want to be able to send a request to the server from the other program, sending some binary data there via post request. How can I do this?
I know I can send a string like this:
data := url.Values{}
data.Set("req", buf)
u, _ := url.ParseRequestURI(domain)
u.Path = path
urlStr := fmt.Sprintf("%v", u)
client := &http.Client{}
r, _ := http.NewRequest("POST", urlStr, bytes.NewBufferString(data.Encode()))
r.Header.Add("Authorization", "auth_token=\"XXXXXXX\"")
r.Header.Add("Content-Type", "application/x-www-form-urlencoded")
r.Header.Add("Content-Length", strconv.Itoa(len(data.Encode())))
resp, _ := client.Do(r)
return resp.Status
But I want to send an octet-stream which I can then read from ioutil.ReadAll(r.Body).
The following show how to send data inside the request body to a server, and read it on the server side. The client part is as follow:
c := http.Client{}
data := []byte("This is a content that will be sent in the body")
r, err := http.NewRequest("POST", "http://localhost:8080", bytes.NewBuffer(data))
// You should never ignore the error returned by a call.
if err != nil {
panic(err)
}
c.Do(r)
And in your http.Handler function:
d, err := ioutil.ReadAll(r.Body)
if err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
}
fmt.Println("Request content : ", string(d))
This is the easiest way.
I am trying to write a web server using Go-bootstrap library and I have written my own handler in handler/users which is called on a post request on
localhost/app/signup. I am trying to print the json data to the terminal but fmt.Printf() prints nothing. The files are as follows:
In handler/users
func AppPostSignup(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
fmt.Println("In AppPostSignup")
data := map[string]interface{}{}
body, _ := ioutil.ReadAll(r.Body)
json.Unmarshal(body, &data)
db := context.Get(r, "db").(*sqlx.DB)
email := data["Email"]
password := data["Password"]
passwordAgain := data["PasswordAgain"]
fmt.Printf("\ntype : %T\nData: %v", email, email)
_, err := dal.NewUser(db).Signup(nil, email.(string), password.(string), passwordAgain.(string))
if err != nil {
libhttp.HandleErrorJson(w, err)
return
}
}
In main.go
router.Handle("/", MustLogin(http.HandlerFunc(handlers.GetHome))).Methods("GET")
router.HandleFunc("/signup", handlers.GetSignup).Methods("GET")
router.HandleFunc("/signup", handlers.PostSignup).Methods("POST")
router.HandleFunc("/login", handlers.GetLogin).Methods("GET")
router.HandleFunc("/login", handlers.PostLogin).Methods("POST")
router.HandleFunc("/app/signup", handlers.AppPostSignup).Methods("POST")
router.HandleFunc("/authenticate", handlers.Authenticate).Methods("POST")
router.HandleFunc("/logout", handlers.GetLogout).Methods("GET")
What did I do wrong?
I just tested your code and I saw the following printed to the console:
In AppPostSignup
My guess is that you tried to test your /app/signup endpoint using a GET request, when you have it marked only as a POST request.
If you really want it to be a POST request, then you can test it with curl like so:
curl -X POST localhost:8888/app/signup
Note, that request is missing a valid body, but you will be able to see your message printed.