I want to build a golang service which will listen for GET request, do some URL manipulation and then proxy a new request (to the manipulated URL) back to the browser:
(from browser -> server) GET http://www.example.com/7fbsjfhfh93hdkwhfbf398fhkef93..
(server manipulates URL - decrypts "7fbsjfhfh93hdkwhfbf398fhkef93.." -> "my-super-resource")
(server -> URL resource) GET http://www.somewhereelse.com/my-super-resource
(server -> browser) Response from http://www.somewhereelse.com/my-super-resource passed on to browser (using cors)
The whole chain will need to be synchronous which is ok. Is there a decent proxy library which allows for this sort of thing?
You can do something like this in less than 10 lines of code with the Sling package:
type Foo struct {
Bar string `json:"bar"`
}
func Handler(w http.ResponseWriter, r *http.Request) {
// Get the URL and decrypt it
url := getUrl(r)
decryptedUrl := decryptUrl(url)
// Use the decrypted URL to make a request to another service
var data *Foo
req, err := sling.New().Get(decryptedUrl).Receive(data)
if err != nil {
// Handle error...
}
// Respond to the original request using the data from the other service
respond(w, http.StatusOK, data)
}
Related
TLDR: I am looking for a way to update headers on an open stream for each call to stream.Send(msg) without closing the stream and opening a new one.
Summary
I have a GRPC client and server built to handle bidirectional streams. To authenticate with the server the client must send a JWT in the request headers, set as "authorization". The token is valid for 30 minutes. After the token has expired, the server will terminate the connection.
I am looking for a way to refresh my authorization token from the client, and keep the stream open. The client should run in a loop executing a new request every 30 minutes with the updated token, and the updated payload. I have not seen a way to update a header from the client side for an already opened stream.
Let's look at some code to get an idea of what the client side looks like. The code below has a function to create a new instance of the client, and another function to establish the connection to the GRPC server.
func NewWatchClient(config *Config, logger *logrus.Logger) (*WatchClient, error) {
cc, err := newConnection(config, logger)
if err != nil {
return nil, err
}
service := proto.NewWatchServiceClient(cc)
return &WatchClient{
config: config,
conn: cc,
logger: entry,
service: service,
}, nil
}
func newConnection(config *Config, logger *logrus.Logger) (*grpc.ClientConn, error) {
address := fmt.Sprintf("%s:%d", config.Host, config.Port)
// rpcCredential implements credentials.PerRPCCredentials
rpcCredential := newTokenAuth(config.Auth, config.TenantID)
return grpc.Dial(
address,
grpc.WithPerRPCCredentials(rpcCredential),
)
}
Looking at the newConnection function above I can see that there is a call to another function, newTokenAuth, to create an auth token. This func returns a struct that implements the PerRPCCredentials interface.
There are two ways to set the authorization for a request.
Use grpc.WithPerRPCCredentials to add the authorization at the time of creating the connection to the server.
Use grpc.PerRPCCredentials to add the authorization to each stream opened on the connection to the server.
In this case, I am using grpc.WithPerRPCCredentials to attach the token at the time of creating the connection to the server.
Now, let's take a look at the definition of PerRPCCredentials.
type PerRPCCredentials interface {
// GetRequestMetadata gets the current request metadata, refreshing
// tokens if required. This should be called by the transport layer on
// each request, and the data should be populated in headers or other
// context. If a status code is returned, it will be used as the status
// for the RPC. uri is the URI of the entry point for the request.
// When supported by the underlying implementation, ctx can be used for
// timeout and cancellation. Additionally, RequestInfo data will be
// available via ctx to this call.
// TODO(zhaoq): Define the set of the qualified keys instead of leaving
// it as an arbitrary string.
GetRequestMetadata(ctx context.Context, uri ...string) (map[string]string, error)
// RequireTransportSecurity indicates whether the credentials requires
// transport security.
RequireTransportSecurity() bool
}
The interface requires that you define two methods. The documentation of GetRequestMetadata says
GetRequestMetadata gets the current request metadata, refreshing tokens if required
So, it looks like my implementation of PerRPCCredentials should be able to handle a token refresh for my stream or connection. Let's take a look at my implementation of PerRPCCredentials.
// tokenAuth implements the PerRPCCredentials interface
type tokenAuth struct {
tenantID string
tokenRequester auth.PlatformTokenGetter
token string
}
// RequireTransportSecurity leave as false for now
func (tokenAuth) RequireTransportSecurity() bool {
return false
}
// GetRequestMetadata sets the http header prior to transport
func (t tokenAuth) GetRequestMetadata(_ context.Context, _ ...string) (map[string]string, error) {
token, err := t.tokenRequester.GetToken()
if err != nil {
return nil, err
}
t.token = token
go func() {
time.Sleep(25 * time.Minute)
token, _ := t.tokenRequester.GetToken()
t.token = token
}()
return map[string]string{
"tenant-id": t.tenantID,
"authorization": "Bearer " + t.token,
}, nil
}
As you can see, the call to GetRequestMetadata will establish a go routine that will attempt to refresh a token every 25 minutes. Adding a go routine right here is probably not the right way to do it. It was an attempt to get the auth header to refresh, which doesn't work.
Let's take a look at the stream.
func (w WatchClient) CreateWatch() error {
topic := &proto.Request{SelfLink: w.config.TopicSelfLink}
stream, err := w.service.CreateWatch(context.Background())
if err != nil {
return err
}
for {
err = stream.Send(topic)
if err != nil {
return err
}
time.Sleep(25 * time.Minute)
}
}
The client sends a message on the stream every 25 minutes. All I'm looking to get here is that when stream.Send is called, the updated token is also sent.
This function, GetRequestMetadata only gets called once, regardless if I am setting the auth through grpc.WithPerRPCCredentials or grpc.PerRPCCredsCallOption so there appears to be no way to update the authorization header.
If you have any idea what I have missed in my attempt to utilize the PerRPCCredentials for token refresh then please let me know.
Thank you.
Headers are sent at the beginning of an RPC, and cannot be updated during the RPC. If you need to send data during the life of a stream, it needs to be part of the request message in your proto definition.
I'm trying to make a HTTP request using http.Get(url) in Go and I want to open the response in a browser. I'm using browser.OpenURL() to launch the system browser, but I cannot figure out how to obtain the response url.
In Python, using the requests library, it is an attribute of the response object.
I can obtain and open it in a browser (using the browser library) like so:
response = requests.get(endpoint)
browser.open(response.url)
How can I accomplish this using http/net library in Go? The response object is a struct that doesn't contain that attribute.
I am trying to call the Spotify API to authenticate an app, and this requires opening a browser window for user input. So far I've got this:
func getAuth(endpoint *url.Url) {
request, _ := http.NewRequest("GET", endpoint.string(), nil)
client := &http.Client{}
resp, err := client.Do(request)
if err != nil {
panic(err)
}
headers := resp.Header
page, _ := ioutil.ReadAll(resp.Body)
Where can I obtain the response URL or how can I handle the response so that it opens it in a browser?
Go will update the Request struct on the response if there is a redirect.
resp.Request.URL is what you are looking for.
// Request is the request that was sent to obtain this Response.
// Request's Body is nil (having already been consumed).
// This is only populated for Client requests.
Request *Request
Just get the redirect URL from response header.
redirectURL := resp.Header.Get("Location")
I'm using LabStack's Golang Echo Framework to build out a service.
One of the routes, needs to proxy requests and responses to and from a backend service.
But I also need CORS to work on this service as well.
So I'm using middleware.CORSWithConfig along w/ a middleware.ProxyWithConfig in my request/response stack.
I'm seeing some oddness w/ the Access-Control-Allow-Origins header where the value for that header on the response from the proxied service to my Echo server *, but once it passes through the proxy, it changes to *, * by the time it gets back into the client.
Upon which I start seeing the following browser errors related to CORS violations:
VM1627:362 Access to XMLHttpRequest at 'http://localhost:6273/' from origin 'http://localhost:8002' has been blocked by CORS policy: The 'Access-Control-Allow-Origin' header contains multiple values '*, *', but only one is allowed.
Has anyone come across this? Anyone have any idea why this might be happening and maybe a way around it?
Here's some example code:
package main
func singleTargetBalancer(url *url.URL) middleware.ProxyBalancer {
targetURL := []*middleware.ProxyTarget{
{
URL: url,
},
}
return middleware.NewRoundRobinBalancer(targetURL)
}
func Noop(ctx echo.Context) (err error) {
ctx.String(
http.StatusNotImplemented,
"No op handler should never be reached!",
)
return err
}
func main() {
e := echo.New()
e.HideBanner = true
e.Use(
middleware.CORSWithConfig(middlewares.CustomCorsConfig),
middlewares.ThriftMetrics(),
)
// Have to use a Noop handler since we're not trying to set up a full-on proxy for the backend service. We only want this one route to be proxied.
e.POST(
"/",
handlers.Noop,
middleware.ProxyWithConfig(middleware.ProxyConfig{
Balancer: singleTargetBalancer("[backend service URL]"),
})
)
}
I ultimately solved this by writing a custom Echo middleware to hook into the response before Echo's proxy middleware could send the headers back to the client.
func setResponseACAOHeaderFromRequest (req http.Request, resp echo.Response) {
resp.Header().Set(echo.HeaderAccessControlAllowOrigin,
req.Header.Get(echo.HeaderOrigin))
}
func ACAOHeaderOverwriteMiddleware(next echo.HandlerFunc) echo.HandlerFunc {
return func(ctx echo.Context) error {
ctx.Response().Before(func() {
setResponseACAOHeaderFromRequest(*ctx.Request(), *ctx.Response())
})
return next(ctx)
}
}
Then just put this middleware in e.Use() right before your proxy middleware:
e.POST(
"/",
handlers.Noop,
ACAOHeaderOverwriteMiddleware,
middleware.ProxyWithConfig(middleware.ProxyConfig{
Balancer: singleTargetBalancer("[backend service URL]"),
})
)
Docs for Echo's Request::Before() hook: https://echo.labstack.com/guide/response#before-response
When a user hits a certain url with a GET request I'd like to redirect them to a POST request at another location.
package main
import (
"bytes"
"fmt"
"io/ioutil"
"log"
"net/http"
"net/url"
)
func old(w http.ResponseWriter, r *http.Request) {
newURL := "/new"
var bdy = []byte(`title=Buy cheese and bread for breakfast.`)
r.Method = "POST"
r.URL, _ = url.Parse(newURL)
r.RequestURI = newURL
r.Body = ioutil.NopCloser(bytes.NewReader(bdy))
r.Header.Set("Content-Type", "application/x-www-form-urlencoded")
http.Redirect(w, r, newURL, 302)
}
func new(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
fmt.Printf("Method:%v\n", r.Method)
fmt.Printf("Title:%v\n", r.Form.Get("title"))
}
func main() {
http.HandleFunc("/", old)
http.HandleFunc("/new", new)
port := 8000
fmt.Printf("listening on %v\n", port)
if err := http.ListenAndServe(fmt.Sprintf(":%v", port), nil); err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
When I hit "/" I end up getting redirected to "/new" but with a GET request and no form data:
Method:GET
Title:
If I curl "/new" directly I get :
curl -XPOST localhost:8000/new -d "title=Buy cheese and bread for breakfast."
Method:POST
Title:Buy cheese and bread for breakfast.
A HTTP redirect (i.e. reply with status code 301, 302, 307,308 and Location header) can only redirect the existing request to another location and not change the payload of the request. It can add some cookies in the response header though.
In order to automatically change a GET request into a POST request with a specific payload you might try to send the client a HTML page with a <form method=POST... and the payload with hidden input fields, i.e. <input name=... value=... type=hidden> and then add some JavaScript to the page which automatically submits the form. But this kind of hack will only work in browsers and only if JavaScript is enabled and will not work with all kind of payloads either.
To keep compatibility with a broader range of clients it is probably better to design it differently, i.e. keep the GET request in the redirect but give the necessary payload as a parameter to the new target, i.e. http://new.target/foo?payload=..... But the details depend on what the target of the request can deal with.
Unfortunately I don't believe a redirect can change the verb (e.g., GET, POST) or add data to the request. It can only change the URL.
See Redirect () for more information.
I've never heard about changing verb from GET to POST. I guess it's impossible because POST supposes body of body (however may be empty) and GET doesn't. So in general case browser would not be able to take the body from nothing.
Otherwise is possible: you may send 302 redirect after post to make browser perform get. Also verb can be kept with 307 reply code.
Try to rethink browser-server interaction. May be you can redirect POST to another location to solve a task?
I'm starting to play around with websockets + go and well I think I'm misunderstanding something quite basic with websockets in Go.
I'd like to simply listen for a websocket connection and process accordingly. However all examples I see in Go using websocket is serving the web page that then connects to the websocket, is this a requirement?
The following is a basic echo server I have setup:
package main
import (
"fmt"
"code.google.com/p/go.net/websocket"
"net/http"
)
func webHandler(ws *websocket.Conn) {
var s string
fmt.Fscan(ws, &s)
fmt.Println("Received: ", s)
}
func main() {
fmt.Println("Starting websock server: ")
http.Handle("/echo", websocket.Handler(webHandler))
err := http.ListenAndServe(":8080", nil)
if err != nil {
panic("ListenAndServe: " + err.Error())
}
}
This is the javascript used to connect:
ws = new WebSocket("ws://localhost:8080/echo");
ws.onmessage = function(e) {
console.log("websock: " + e.data);
};
However this results in:
WebSocket connection to 'ws://localhost:8080/echo' failed: Unexpected response code: 403
When working with websockets from Javascript, you will seldom have to read the frames directly. To be honest, I am not even sure how to do that.
Fortunately, the websocket package already has a type, Codec that does this for you. My suggestion is to use the predefined websocket.Message codec to Recieve and Send messages instead.
Message is a codec to send/receive text/binary data in a frame on WebSocket connection. To send/receive text frame, use string type. To send/receive binary frame, use []byte type.
Using websocket.Message, your webHandler would look something like this:
func webHandler(ws *websocket.Conn) {
var in []byte
if err := websocket.Message.Receive(ws, &in); err != nil {
return
}
fmt.Printf("Received: %s\n", string(in))
websocket.Message.Send(ws, in)
}
And, no, it is not a requirement that Go serves the webpage. The 403 error you received does not have to do with Go or the websocket package.
I had a similar issue and the 403 error problem is related to how Go treats the http Origin header:
Handler is a simple interface to a WebSocket browser client. It checks if Origin header is valid URL by default. You might want to verify websocket.Conn.Config().Origin in the func. If you use Server instead of Handler, you could call websocket.Origin and check the origin in your Handshake func. So, if you want to accept non-browser client, which doesn't send Origin header, you could use Server . that doesn't check origin in its Handshake.
In order to disable the Origin check, you must use something like:
http.HandleFunc("/echo",
func (w http.ResponseWriter, req *http.Request) {
s := websocket.Server{Handler: websocket.Handler(webHandler)}
s.ServeHTTP(w, req)
});
At least that solved the issue for me (server to server WebSocket communication) and I think it could solve the problem as well if the origin header does not match.