I have a specific use case, where I have to use HTTP2 protocol with a proxy configured. This proxy function uses the request to determine, which proxy URL to use.
Since http2.transport{} struct does not have a proxy field, I was using:
httpTransport := &http.Transport{
Proxy: *ProxyFunction, // var ProxyFunction *func(*http.Request) (*url.URL, error)
ForceAttemptHTTP2: true,
}
http2.ConfigureTransport(httpTransport)
But, this does not allow me to send "http://" requests as HTTP2. Only "https://" were sent as HTTP2, "http://" requests were using HTTP1.1
This leads me to
httpTransport := &http.Transport{
Proxy: *config.ProxyFunction,
ForceAttemptHTTP2: true,
}
h2, _ := http2.ConfigureTransports(httpTransport)
h2.AllowHTTP = true
But using this config, on request, I receive
error getting: Get http://google.com: http2: no cached connection was available, similar to https://github.com/golang/go/issues/26479
Since there is no official support, what would be the right approach to achieve this?
Can I use a dialer to achieve this?
Note:
proxy URL is obtained from ProxyFunction *func(*http.Request) (*url.URL, error) and changes based on the request
https is not a requirement and need not be supported.
go version go1.13.7
Related
I have an application that is using the OAuth 2.0 Authorization Code flow and as part of that we regularly check whether a token has expired and refresh if it has. This is all done using golang's built in oauth2 library. The code that performs the refresh looks something like this:
if time.Now().Before(currentToken.Expiry) {
return false, nil
}
// if the expiry has passed, generate a new token
refreshConf := oauth2.Config{
ClientID: config.OAuthConfig.ClientID,
ClientSecret: config.OAuthConfig.ClientSecret,
Endpoint: oauth2.Endpoint{
TokenURL: config.TokenURL,
},
}
tokenRefreshSource := refreshConf.TokenSource(ctx, ¤tToken.Token)
newToken, err := tokenRefreshSource.Token()
if err != nil {
return false, err
}
currentToken.Token = *newToken
Recently my work has cracked down on unregulated outbound traffic in an effort to make things more secure. Part of these changes has required that all outbound traffic go through a specific proxy. I've been able to do this by setting http.Client.Transport.Proxy for the requests that I'm manually sending, but I have not found a way to have the underlying request that is fired from tokenRefreshSource.Token() use a proxy.
Does anyone know if this is possible? If so - suggestions on how I can do it?
I am trying to route my requests through a proxy and also sending cert.pem in TLS config. Below code is throwing this error - proxyconnect tcp: tls: first record does not look like a TLS handshake. When I change the proxy URL from https to HTTP, the same code works. However proxy URL with https works in python. Below is my code so far
certs := x509.NewCertPool()
pemFile, err := ioutil.ReadFile("cert.pem")
if err != nil {
return
}
certs.AppendCertsFromPEM(pemFile)
tlsConfig := &tls.Config{
RootCAs: certs,
}
proxyUrl, err := url.Parse("https://someproxyurlhere.com:8080")
if err != nil {
return
}
t := &http.Transport{
TLSClientConfig: tlsConfig,
Proxy: http.ProxyURL(proxyUrl),
}
client := http.Client{
Transport: t,
}
reqBody := "some JSON body here"
buff, err := json.Marshal(reqBody)
if err != nil {
return
}
req, err := http.NewRequest(http.MethodPost, "https://someurlhere.com", bytes.NewBuffer(buff))
if err != nil {
return
}
res, err := client.Do(req)
if err != nil {
// Error here - proxyconnect tcp: tls: first record does not look like a TLS handshake
return
}
defer res.Body.Close()
Python code
import requests
os.environ['HTTPS_PROXY'] = 'https://someproxyurlhere.com:8080'
response = requests.post("https://someurlhere.com",
json={'key': 'value'},
verify='cert.pem')
print(str(response.content))
When I change the proxy URL from https to HTTP, the same code works.
This is because you are using a HTTP proxy which need to be accessed by HTTP and not HTTPS even for https://.. URLs. This is how HTTPS proxying usually works. Note that while in theory a proxy could sniff if it gets a plain or a TLS connection in practice proxies (and servers) use different ports for HTTP and HTTPS - thus if it works on one port with HTTP it will very likely not work on the same port with HTTPS.
proxyconnect tcp: tls: first record does not look like a TLS handshake.
This is because the proxy answers with an plain HTTP error to the strange HTTP request (which is actually the start of the TLS handshake).
However proxy URL with https works in python.
While it looks that it works it actually doesn't. Python requests will still use plain HTTP to the proxy even if https:// was given as URL.
You can try to do a simple TLS connection to the proxy itself, for example with Python or with openssl s_client. It will very likely fail with some handshake error because the proxy isn't actually expecting TLS.
In looking at the Go docs for http it looks like the Authorization header is removed when a response is a 307. Obviously it makes sense for almost every case but is there a way not to remove the Authorization header?
You can modify your http.Client to add the header again after it has been removed using CheckRedirect:
CheckRedirect func(req *Request, via []*Request) error
Since req is the upcoming request, it can be modified before it is sent. After making the changes, return nil to indicate that the request should still be sent.
Since this is a change to the http client instead of the request, you should check that this redirect is only used for the one URL where you need it (in case you use that client to do other requests).
You client definition could look like this:
http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
// you can check old responses for a status code
if len(via) != 0 && via[0].Response.StatusCode == http.StatusTemporaryRedirect {
req.Header.Add("Authorization", "some-value")
}
return nil
},
}
I’m using gorilla web socket and I want to run it locally , I mean with the following chrome client
or other recommended tool …when I run into debug mode I got error
I use
"github.com/gorilla/websocket"
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
}
upgrader.CheckOrigin = func(r *http.Request) bool { return true }
c, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Print("upgrade:", err)
return
}
When I run the following url in chrome or web socket client I got error
websocket: not a websocket handshake: 'upgrade' token not found in 'Connection' header
localhost:8081/mypath
and I want to run it
ws://localhost:8081/mypath
and provide token for local simulation, how I can do it ?
To check it I use Simple WebSocket Client of chrome. any other client will be helpful
EDIT:
when I try it in the chrome console I got the following error:
VM42:164 Refused to connect to 'ws://localhost:8081/mypath' because it
violates the following Content Security Policy directive: "connect-src
'self' uploads.github.com status.github.com collector.githubapp.com
api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com
github-production-repository-file-5c1aeb.s3.amazonaws.com
github-production-upload-manifest-file-7fdce7.s3.amazonaws.com
github-production-user-asset-6210df.s3.amazonaws.com
wss://live.github.com
Browsers do not use the WebSocket protocol when fetching a web page for display. Code is required to use a WebSocket endpoint from a browser.
The Gorilla package includes examples showing how to connect from a browser (the chat and command examples are good places to start).
You can can connect to a WebSocket endpoint using the browser console:
> ws = new WebSocket("ws://localhost:8080/mypath")
> ws.onmessage = function(ev) { console.log(ev.data) }
> ws.send("hello")
I'm starting to play around with websockets + go and well I think I'm misunderstanding something quite basic with websockets in Go.
I'd like to simply listen for a websocket connection and process accordingly. However all examples I see in Go using websocket is serving the web page that then connects to the websocket, is this a requirement?
The following is a basic echo server I have setup:
package main
import (
"fmt"
"code.google.com/p/go.net/websocket"
"net/http"
)
func webHandler(ws *websocket.Conn) {
var s string
fmt.Fscan(ws, &s)
fmt.Println("Received: ", s)
}
func main() {
fmt.Println("Starting websock server: ")
http.Handle("/echo", websocket.Handler(webHandler))
err := http.ListenAndServe(":8080", nil)
if err != nil {
panic("ListenAndServe: " + err.Error())
}
}
This is the javascript used to connect:
ws = new WebSocket("ws://localhost:8080/echo");
ws.onmessage = function(e) {
console.log("websock: " + e.data);
};
However this results in:
WebSocket connection to 'ws://localhost:8080/echo' failed: Unexpected response code: 403
When working with websockets from Javascript, you will seldom have to read the frames directly. To be honest, I am not even sure how to do that.
Fortunately, the websocket package already has a type, Codec that does this for you. My suggestion is to use the predefined websocket.Message codec to Recieve and Send messages instead.
Message is a codec to send/receive text/binary data in a frame on WebSocket connection. To send/receive text frame, use string type. To send/receive binary frame, use []byte type.
Using websocket.Message, your webHandler would look something like this:
func webHandler(ws *websocket.Conn) {
var in []byte
if err := websocket.Message.Receive(ws, &in); err != nil {
return
}
fmt.Printf("Received: %s\n", string(in))
websocket.Message.Send(ws, in)
}
And, no, it is not a requirement that Go serves the webpage. The 403 error you received does not have to do with Go or the websocket package.
I had a similar issue and the 403 error problem is related to how Go treats the http Origin header:
Handler is a simple interface to a WebSocket browser client. It checks if Origin header is valid URL by default. You might want to verify websocket.Conn.Config().Origin in the func. If you use Server instead of Handler, you could call websocket.Origin and check the origin in your Handshake func. So, if you want to accept non-browser client, which doesn't send Origin header, you could use Server . that doesn't check origin in its Handshake.
In order to disable the Origin check, you must use something like:
http.HandleFunc("/echo",
func (w http.ResponseWriter, req *http.Request) {
s := websocket.Server{Handler: websocket.Handler(webHandler)}
s.ServeHTTP(w, req)
});
At least that solved the issue for me (server to server WebSocket communication) and I think it could solve the problem as well if the origin header does not match.