I have proxyString 123.44.333.42:51244
and want to request url through the proxy
proxyString := "123.44.333.42:51244"
proxyURL, err := url.Parse("http://"+proxyStr)
if err != nil {
panic(err)
}
////adding the proxy settings to the Transport object
transport := &http.Transport{
Proxy: http.ProxyURL(proxyURL),
}
client := &http.Client{
Transport: transport,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}}
resp, contents := getContent(client, person.personHref)
get errror Proxy Authentication Required
Related
When I try to resend requests from simple proxy
http.HandleFunc("/",func(w http.ResponseWriter, r *http.Request) {
log.Printf("proxy rq: %v", r)
client := &http.Client{}
proxyRes, err := client.Do(r) // 👈🏻 Get "http://localhost:8097/tmp": http: Request.RequestURI can't be set in client requests
if err != nil {
log.Fatalf("err proxy request: %v",err)
}
resBody := proxyRes.Body
defer resBody.Close()
if _, err := io.Copy(w, resBody); err != nil {
log.Printf("copy error:%v\n", err)
}
})
http.ListenAndServe(":8099", nil)
, with set http_proxy ENV (to send requests thought my proxy)
% export http_proxy=http://localhost:8099
% curl -v http://localhost:8097/tmp
I get an error like
Get "http://localhost:8097/tmp": http: Request.RequestURI can't be set in client requests
What did I miss?
You can't use the client request as a parameter of Do. Create a new request with the same parameter as r, then perform Do on this request
The error is defined in the ../src/net/http/client.go:217 (https://go.dev/src/net/http/client.go) as:
if req.RequestURI != "" {
req.closeBody()
return nil, alwaysFalse, errors.New("http: Request.RequestURI can't be set in client requests")
}
You should be able to reuse existing r *http.Request if you set r.RequestURI = "" before sending a HTTP request:
r.RequestURI = ""
proxyRes, err := client.Do(r)
I want to make https calls via a third party proxy using Go.
The third party proxy accepts a username and password.
My code works fine when I call the proxy using http. But I get a runtime error when I change the proxy to https.
The error message is:
proxyconnect tcp: tls: first record does not look like a TLS handshake
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1248521]
package main
import (
"log"
"net/http"
"net/url"
)
func main() {
// proxyURL - change to https get runtime error
proxyStr := "http://username:password#ip:port"
proxyURL, err := url.Parse(proxyStr)
if err != nil { log.Println(err) }
// targetURL
targetStr := "https://httpbin.org/get"
targetURL, err := url.Parse(targetStr)
if err != nil { log.Println(err) }
//config Transport w Proxy
transport := &http.Transport{
Proxy: http.ProxyURL(proxyURL),
}
//config Client w Transport
client := &http.Client{
Transport: transport,
}
//config Request
request, err := http.NewRequest("GET", targetURL.String(), nil)
if err != nil { log.Println(err) }
log.Println("Requesting", targetStr)
//try it
response, err := client.Do(request)
if err != nil { log.Println(err) }
log.Println(response.Status)
}
I'm going to send http request via ssl proxy in Go.
Unfortunately, it always fails. Here is my code I have tried.
proxyUrl, err := url.Parse("https://" + proxyURLStr)
transport := &http.Transport{
Proxy: http.ProxyURL(proxyUrl),
}
//adding the Transport object to the http Client
client := &http.Client{
Transport: transport,
Timeout: 60 * time.Second,
}
request, err := http.NewRequest("GET", "https://test.com", nil)
if err != nil {
log.Print("Create Request Error - test.com ")
return -2, invalid_data
}
random_agent := browser.Random()
request.Header.Set("User-Agent", random_agent)
// Make request
response, err := client.Do(request)
if err != nil {
log.Println(err)
current := time.Now()
elapsed := current.Sub(start)
log.Printf("test.com is not available. %02d, %02d ", int(elapsed.Minutes()), int(elapsed.Seconds()))
return -2, invalid_data
}
I get this error:
net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I coded the same function in Python and tested it — works good.
But the Go code does not work.
I have a go service that makes REST requests to an HTTP server that I don't control. A customer asked my to "confirm" that my service is connecting via TLS 1.2. Is that something that I can do in code?
Current code looks something like this:
request, _ := http.NewRequest("PUT",
"https://example.com/path/to/endpoint",
bytes.NewReader(json))
client := &http.Client{}
response, _ := client.Do(request)
defer response.Body.Close()
str, err := ioutil.ReadAll(response.Body)
Based on a quick read of the docs I believe I need to use a Transport and build my client using that transport. Something like this:
tr := &http.Transport{
... some options here ...
}
client := &http.Client{Transport: tr}
But I'm not sure what options I should set.
At the time of writing, Go will speak TLS 1.2 automatically if the server supports it.
tls.ConnectionState reports various negotiated TLS parameters of a connection, including the protocol version.
To get the underlying TLS connection for an HTTP client it is easiest to set the DialTLS field of the Transport to a function that establishes and remembers the connection. Once the response arrived (but before you close the response body!), call tls.Conn.ConnectionState:
package main
import (
"crypto/tls"
"fmt"
"log"
"net"
"net/http"
)
func main() {
var (
conn *tls.Conn
err error
)
tlsConfig := http.DefaultTransport.(*http.Transport).TLSClientConfig
c := &http.Client{
Transport: &http.Transport{
DialTLS: func(network, addr string) (net.Conn, error) {
conn, err = tls.Dial(network, addr, tlsConfig)
return conn, err
},
},
}
res, err := c.Get("https://example.com")
if err != nil {
log.Fatal(err)
}
versions := map[uint16]string{
tls.VersionSSL30: "SSL",
tls.VersionTLS10: "TLS 1.0",
tls.VersionTLS11: "TLS 1.1",
tls.VersionTLS12: "TLS 1.2",
}
fmt.Println(res.Request.URL)
fmt.Println(res.Status)
v := conn.ConnectionState().Version
fmt.Println(versions[v])
res.Body.Close()
}
// Output:
// https://example.com
// 200 OK
// TLS 1.2
From the docs
Package tls partially implements TLS 1.2, as specified in RFC 5246.
That beeing said I keep this function as a snippet to create the necessary configuration:
func NewTLSConfig(clientCertFile, clientKeyFile, caCertFile string) (*tls.Config, error) {
tlsConfig := tls.Config{}
// Load client cert
cert, err := tls.LoadX509KeyPair(clientCertFile, clientKeyFile)
if err != nil {
return &tlsConfig, err
}
tlsConfig.Certificates = []tls.Certificate{cert}
// Load CA cert
caCert, err := ioutil.ReadFile(caCertFile)
if err != nil {
return &tlsConfig, err
}
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(caCert)
tlsConfig.RootCAs = caCertPool
tlsConfig.BuildNameToCertificate()
return &tlsConfig, err
}
After that you just need to initialize the transport:
transport := &http.Transport{TLSClientConfig: tlsConfig}
client := &http.Client{Transport: transport}
Golang's package net/http/transport can automatic setup Proxy-Authorization header in
func (t *Transport) dialConn(ctx context.Context, cm connectMethod) (*persistConn, error)
like
proxyURL, _ := url.Parse("http://username:password#example.com")
client := http.Client{Transport: &http.Transport{Proxy:http.ProxyURL(proxyURL)}}
But I need submit X-Header to proxy server. How Can I custom transport CONNECT method request header?
net/http/transport
how about this:
// ...
request, err := http.NewRequest("GET", "https://www.google.com", nil)
if err != nil {
// do something
}
// add header here.
request.Header.Add("X-Header", "xxx")
response, err := client.Do(request)
if err != nil {
// do something
}
// ...
http.Transport has a function which allows you to set some additional headers which will be sent during CONNECT.
Example:
var client http.Client
client.Transport = &http.Transport{
Proxy: http.ProxyURL(myProxy),
GetProxyConnectHeader: func(ctx context.Context, proxyURL *url.URL, target string) (http.Header, error) {
return http.Header{"My-Custom-Header": []string{"My-Custom-Value"}}, nil
},
}