I've been over the docs at https://golang.org/pkg/ but can't make this connection.
I am creating a client and request like so (error handling removed):
client := http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
req, reqErr := http.NewRequest(requestMethod, requestUrl, nil)
resp, clientErr := client.Do(req)
I need to get a x509.Certificate to read details of the cert returned from the server, but still need the http.Repsonse as well.
How can I get a x509.Certificate instance and an http.Response while only making a single request?
The response has a TLS *tls.ConnectionState field, which in turn has:
type ConnectionState struct {
// other fields
PeerCertificates []*x509.Certificate // certificate chain presented by remote peer
}
so you can just do:
resp, clientErr := client.Do(req)
if clientErr != nil {
panic(clientErr)
}
if resp.TLS != nil {
certificates := resp.TLS.PeerCertificates
if len(certificates) > 0 {
// you probably want certificates[0]
cert := certificates[0]
}
}
Related
I just started to use Golang and I want to remake my already working NodeJS/TypeScript app in Go.
One endpoint of my API simply adds server-side generated authorization headers and sends a request to a remote API. Basically filling those headers for me by calling my API instead of the remote API.
This is what I am currently writing
func Endpoint(ctx *fiber.Ctx) error {
url := "https://api.twitch.tv" + ctx.OriginalURL()
req, _ := http.NewRequest(http.MethodGet, url, nil)
req.Header.Set("Authorization", "Bearer ---------")
req.Header.Set("Client-Id", "---------")
client := &http.Client{}
res, err := client.Do(req)
// temporary error handling
if err != nil {
log.Fatalln(err)
}
body, err := ioutil.ReadAll(res.Body)
// temporary error handling
if err != nil {
log.Fatalln(err)
}
var forwardedBody interface{}
json.Unmarshal(body, &forwardedBody)
return ctx.Status(fiber.StatusOK).JSON(forwardedBody)
}
I'd like to know if I am on the right steps, because making a request, parsing the JSON response with ioutil then unmarshall it to send it back seems kind of overboard for the simplicity of what I am trying to achieve ?
Edit: Thank you for the help, this is what I will be going for
func Endpoint(ctx *fiber.Ctx) error {
url := "https://api.twitch.tv" + ctx.OriginalURL()
req, _ := http.NewRequest(http.MethodGet, url, nil)
req.Header.Set("Authorization", "Bearer ---------")
req.Header.Set("Client-ID", "---------")
client := &http.Client{}
res, err := client.Do(req)
if err != nil {
return ctx.SendStatus(fiber.StatusBadRequest)
}
ctx.Set("Content-Type", "application/json; charset=utf-8")
return ctx.Status(res.StatusCode).SendStream(res.Body)
}
You can use httputil.ReverseProxy. Which takes a base URL and forwards requests to the base URL, concatenating the path.
ReverseProxy is an HTTP Handler that takes an incoming request and sends it to another server, proxying the response back to the client.
http.Handle("/", &httputil.ReverseProxy{
Director: func(r *http.Request) {
r.URL.Scheme = "https"
r.URL.Host = "go.dev"
r.Host = r.URL.Host
r.Header.Set("X-Foo", "Bar")
},
})
If you are not serving this from the root path / you can use StripPrefix.
http.HandleFunc("/foo/", http.StripPrefix("/foo/", proxy)
There is also a helper function NewSingleHostReverseProxy, which possibly removes the need to configure the proxy struct yourself. But I think it will be better to set the Host header along with your custom header.
You don't need to attempt to parse the data as JSON. This will be problematic if any of your endpoints don't return JSON, anyway, so just inject the body directly into the response:
body, err := ioutil.ReadAll(res.Body)
// temporary error handling
if err != nil {
log.Fatalln(err)
}
// Inject the body from the inner response into the actual response so it can be returned
ctx.Response().SetBody(body)
return cx.Status(fiber.StatusOK)
Context: I want to establish a TLS Connection to a server, which has a certificate with a different (but known!) domain.
So I want to use tls.Dial('tcp', 'real-domain', conf), but verify the domain of the certificate as if it would be the other domain (lets call it wrong-domain), of which I know the server should return it.
So I think the way to do this is override VerifyPeerCertificate in the clients tls.Config.
VerifyPeerCertificate gets rawCerts [][]byte as parameter, but to use x509.Verify I need the certificate as a x509.Certificate type.
The question is: How do I cenvert the rawCerts [][]byte, which are passed as a parameter to VerifyPeerCertificate converted to x509.Certificate?
This is how I want to use it:
conf := &tls.Config{
VerifyPeerCertificate: func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
verifyOptions := x509.VerifyOptions{
DNSName: "wrong-domain",
Roots: serverCertPool,
}
cert := &x509.Certificate{???} // How do I get the x509.Certificate out of the rawCerts [][]byte parameter?
_, err := cert.Verify(verifyOptions)
return err
},
InsecureSkipVerify: true, // We verify in VerifyPeerCertificate
}
Use x509.ParseCertificate.
ParseCertificate parses a single certificate from the given ASN.1 DER data.
Example: cert, err := x509.ParseCertificate(rawCerts[0])
By the way this is also how the function that calls VerifyPeerCertificate does:
// crypto/tls/handshake/handshake_client.go
func (c *Conn) verifyServerCertificate(certificates [][]byte) error {
for i, asn1Data := range certificates {
cert, err := x509.ParseCertificate(asn1Data)
if err != nil {
c.sendAlert(alertBadCertificate)
return errors.New("tls: failed to parse certificate from server: " + err.Error())
}
certs[i] = cert
}
// later on...
if c.config.VerifyPeerCertificate != nil {
if err := c.config.VerifyPeerCertificate(certificates, c.verifiedChains); err != nil {
c.sendAlert(alertBadCertificate)
return err
}
}
Is there an easy way to determine if the server I communicate with from the http.Client accepted and validated my client certificate in mTLS?
In my code I want to know if the remote server accepted the client certificate I put in the transport for the client when I perform a http.Client.Get call.
The http.Response struct has a TLS field that contains the ConnectionState but this is also set when I do not provide a client cert in the transport. I do see that I get more elements in VerifiedChains when I perform a Get with client certificate than without the client certificate.
Thanks for all the suggestions and comments. I will comment on these and share the solution I came up with to get the information I sought.
I am in the process of testing the server and needed to see if the server asked for a client certificate and also which CAs it presents to the client for signing. I want a report similar to what this command produces:
openssl s_client -state -connect example.org:443
Thing is that our server only presents a client certificate for a specific path and not for the servername alone.
The code I came up with looks like this:
...
// Client wraps http.Client and has a callback for flagging a certificate request.
type client struct {
http.Client
clientAutenticated bool
}
// SetClientAuthenticated
func (c *client) SetClientAutenticated(auth bool) {
c.clientAutenticated = auth
}
...
// This prepares a tls config with customized GetClientCertificate
func prepareTLSConfig(setAuth func(bool)) *tls.Config {
certPool := x509.NewCertPool()
err := loadCertFiles(certPool, config.cacerts)
if err != nil {
log.Fatalf("%s", err)
}
cert, err := tls.LoadX509KeyPair(config.cert, config.key)
if err != nil {
log.Fatalf("error loading key pair: %s", err)
}
tlsConfig := &tls.Config{
// Certificates: []tls.Certificate{cert},
RootCAs: certPool,
MinVersion: config.tlsver,
Renegotiation: tls.RenegotiateOnceAsClient,
GetClientCertificate: buildGetClientCertificate([]tls.Certificate{cert}, setAuth),
}
return tlsConfig
}
// buildGetClientCertificate returns a closure that returns an verified client cert
// or an error
func buildGetClientCertificate(certs []tls.Certificate, setAuth func(bool))
func(*tls.CertificateRequestInfo) (*tls.Certificate, error) {
// return as closure
return func(requestInfo *tls.CertificateRequestInfo) (*tls.Certificate, error) {
logCertificates(requestInfo.AcceptableCAs)
setAuth(false)
log.Printf("Client cert requested by server")
var err error
for _, c := range certs {
if err = requestInfo.SupportsCertificate(&c); err == nil {
var cert *x509.Certificate
if c.Leaf != nil {
cert = c.Leaf
} else {
cert, _ = x509.ParseCertificate(c.Certificate[0])
}
subject := cert.Subject
issuer := cert.Issuer
log.Printf("Client cert accepted")
log.Printf(" s:/C=%v/ST=%v/L=%v/O=%v/OU=%v/CN=%s", subject.Country,
subject.Province, subject.Locality, subject.Organization,
subject.OrganizationalUnit, subject.CommonName)
log.Printf(" i:/C=%v/ST=%v/L=%v/O=%v/OU=%v/CN=%s", issuer.Country,
issuer.Province, issuer.Locality, issuer.Organization,
issuer.OrganizationalUnit, issuer.CommonName)
// Signal that a suitable CA has been found and therefore
// the client has been authenticated.
setAuth(true)
return &c, nil
}
err = fmt.Errorf("cert not supported: %w", err)
}
log.Print("Could not find suitable client cert for authentication")
return nil, err
}
}
...
// logCertificates logs the acceptableCAs for client certification
func logCertificates(acceptableCAs [][]byte) {
log.Printf("CA Names offered by server")
for _, ca := range acceptableCAs {
var name pkix.RDNSequence
if _, err := asn1.Unmarshal(ca, &name); err == nil {
log.Printf(" %s", name.String() )
}else {
log.Printf("error unmarshalling name: %s", err)
}
}
}
I'm trying to connect to a server over SSL/TLS using golang http/tsl client which is resulting in 'Handshake Faliure(40)' error, but for some reason, this same endpoint works with CURL command. After some debugging, I have collected the following data.
func PrepCerts(certMap map[string]string) (*http.Transport, bool) {
ok := false
tlsConfig := &tls.Config{}
if len(certMap["ca"]) > 0 {
caCert, err := ioutil.ReadFile(certMap["ca"])
fmt.Println("caCert : ", caCert)
if err != nil {
log.Fatal(err)
} else {
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(caCert)
(*tlsConfig).RootCAs = caCertPool
ok = true
}
}
if len(certMap["cert"]) > 0 && len(certMap["key"]) > 0 {
cert, err := tls.LoadX509KeyPair(certMap["cert"], certMap["key"])
fmt.Println("cert : ", cert)
if err != nil {
log.Fatal(err)
} else {
(*tlsConfig).Certificates = []tls.Certificate{cert}
ok = true
}
}
tlsConfig.BuildNameToCertificate()
return &http.Transport{TLSClientConfig: tlsConfig}, ok
}
Code that uses above function
client := &http.Client{
Timeout: timeout,
}
//certMap = map[string]string{
// ca : "filelocation",
// cert : "filelocation",
// key " "filelocation",
//}
if transport, ok := PrepCerts(certMap); ok {
(*client).Transport = transport
}
resp, err := client.Do(req)
From the captured packets it can be seen that the server is requesting a certificate from the client (Certificate Request). From the the detailed images included in the question it can also be seen that no certificates are sent by the client (Certificate record with Certificate Length 0).
What can also be seen is that the server complains with an Alert after the client has send the (possible empty) certificate so it likely does not like what the client has sent. So it is for sure not a problem of agreeing to a cipher (server agreed on one already) and not a problem that the client does not like the servers certificate (alert is send by server not client).
Based on your code you are trying to do something with client certificates but based on the pcap it looks like you don't succeed in using one. So somewhere there is the problem.
As stated by FiloSottile on Github.
What I think is happening here is that the Certificate Request applies constraints (RSA vs ECDSA, or a specific issuer) which are not satisfied by your certificate.
Using his suggestions you can override the client transports tls.Config.GetClientCertificate()) method.
After following this advice I came to the conclusion that Go will not present tls.Config.RootCAs along with tls.Config.Certificates in response to a certificate request packet.
To solve this issue combine the client certificate and its CA bundle into a single file before calling x509.LoadX509KeyPair(). Taking note that the order of certificates in the file matters. If the client certificate isn't the first one in the bundle you will get a tls: private key does not match public key error.
Once you have combined the client certificate with its CA bundle you can load them into your client like so.
package main
import (
"crypto/tls"
"io/ioutil"
"net/http"
"time"
log "github.com/sirupsen/logrus"
)
const (
certFile = "/location/of/client_cert.bundle.pem"
keyFile = "/location/of/client_cert.key.pem"
testURL = "https://mtls-site"
)
func main() {
clientCert, err := tls.LoadX509KeyPair(certFile, keyFile)
if err != nil {
panic(err)
}
client := http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
Certificates: []tls.Certificate{clientCert},
},
},
}
resp, err := client.Get(testURL)
if err != nil {
panic(err)
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}
log.Infof("%s %s", resp.Status, body)
}
Golang's package net/http/transport can automatic setup Proxy-Authorization header in
func (t *Transport) dialConn(ctx context.Context, cm connectMethod) (*persistConn, error)
like
proxyURL, _ := url.Parse("http://username:password#example.com")
client := http.Client{Transport: &http.Transport{Proxy:http.ProxyURL(proxyURL)}}
But I need submit X-Header to proxy server. How Can I custom transport CONNECT method request header?
net/http/transport
how about this:
// ...
request, err := http.NewRequest("GET", "https://www.google.com", nil)
if err != nil {
// do something
}
// add header here.
request.Header.Add("X-Header", "xxx")
response, err := client.Do(request)
if err != nil {
// do something
}
// ...
http.Transport has a function which allows you to set some additional headers which will be sent during CONNECT.
Example:
var client http.Client
client.Transport = &http.Transport{
Proxy: http.ProxyURL(myProxy),
GetProxyConnectHeader: func(ctx context.Context, proxyURL *url.URL, target string) (http.Header, error) {
return http.Header{"My-Custom-Header": []string{"My-Custom-Value"}}, nil
},
}