How to test if https server my client certficate in Golang - go

Is there an easy way to determine if the server I communicate with from the http.Client accepted and validated my client certificate in mTLS?
In my code I want to know if the remote server accepted the client certificate I put in the transport for the client when I perform a http.Client.Get call.
The http.Response struct has a TLS field that contains the ConnectionState but this is also set when I do not provide a client cert in the transport. I do see that I get more elements in VerifiedChains when I perform a Get with client certificate than without the client certificate.

Thanks for all the suggestions and comments. I will comment on these and share the solution I came up with to get the information I sought.
I am in the process of testing the server and needed to see if the server asked for a client certificate and also which CAs it presents to the client for signing. I want a report similar to what this command produces:
openssl s_client -state -connect example.org:443
Thing is that our server only presents a client certificate for a specific path and not for the servername alone.
The code I came up with looks like this:
...
// Client wraps http.Client and has a callback for flagging a certificate request.
type client struct {
http.Client
clientAutenticated bool
}
// SetClientAuthenticated
func (c *client) SetClientAutenticated(auth bool) {
c.clientAutenticated = auth
}
...
// This prepares a tls config with customized GetClientCertificate
func prepareTLSConfig(setAuth func(bool)) *tls.Config {
certPool := x509.NewCertPool()
err := loadCertFiles(certPool, config.cacerts)
if err != nil {
log.Fatalf("%s", err)
}
cert, err := tls.LoadX509KeyPair(config.cert, config.key)
if err != nil {
log.Fatalf("error loading key pair: %s", err)
}
tlsConfig := &tls.Config{
// Certificates: []tls.Certificate{cert},
RootCAs: certPool,
MinVersion: config.tlsver,
Renegotiation: tls.RenegotiateOnceAsClient,
GetClientCertificate: buildGetClientCertificate([]tls.Certificate{cert}, setAuth),
}
return tlsConfig
}
// buildGetClientCertificate returns a closure that returns an verified client cert
// or an error
func buildGetClientCertificate(certs []tls.Certificate, setAuth func(bool))
func(*tls.CertificateRequestInfo) (*tls.Certificate, error) {
// return as closure
return func(requestInfo *tls.CertificateRequestInfo) (*tls.Certificate, error) {
logCertificates(requestInfo.AcceptableCAs)
setAuth(false)
log.Printf("Client cert requested by server")
var err error
for _, c := range certs {
if err = requestInfo.SupportsCertificate(&c); err == nil {
var cert *x509.Certificate
if c.Leaf != nil {
cert = c.Leaf
} else {
cert, _ = x509.ParseCertificate(c.Certificate[0])
}
subject := cert.Subject
issuer := cert.Issuer
log.Printf("Client cert accepted")
log.Printf(" s:/C=%v/ST=%v/L=%v/O=%v/OU=%v/CN=%s", subject.Country,
subject.Province, subject.Locality, subject.Organization,
subject.OrganizationalUnit, subject.CommonName)
log.Printf(" i:/C=%v/ST=%v/L=%v/O=%v/OU=%v/CN=%s", issuer.Country,
issuer.Province, issuer.Locality, issuer.Organization,
issuer.OrganizationalUnit, issuer.CommonName)
// Signal that a suitable CA has been found and therefore
// the client has been authenticated.
setAuth(true)
return &c, nil
}
err = fmt.Errorf("cert not supported: %w", err)
}
log.Print("Could not find suitable client cert for authentication")
return nil, err
}
}
...
// logCertificates logs the acceptableCAs for client certification
func logCertificates(acceptableCAs [][]byte) {
log.Printf("CA Names offered by server")
for _, ca := range acceptableCAs {
var name pkix.RDNSequence
if _, err := asn1.Unmarshal(ca, &name); err == nil {
log.Printf(" %s", name.String() )
}else {
log.Printf("error unmarshalling name: %s", err)
}
}
}

Related

Golang x509: Certificate is valid for *.<DNS> not <DB_hostname>

I am trying to connect to MySQL DB using SSL via Golang and I am getting this error message
x509: certificate is valid for *., not <DB_hostname>
I have created certificates using this tutorial
https://www.devdungeon.com/content/creating-self-signed-ssl-certificates-openssl
Function to create TLSConf:
func createTLSConf() tls.Config {
rootCertPool := x509.NewCertPool()
pem, err := ioutil.ReadFile("certificate.pem")
if err != nil {
log.Fatal(err)
}
if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
log.Fatal("Failed to append PEM.")
}
clientCert := make([]tls.Certificate, 0, 1)
certs, err := tls.LoadX509KeyPair("certificate.pem", "privkey.pem")
if err != nil {
log.Fatal(err)
}
clientCert = append(clientCert, certs)
return tls.Config{
RootCAs: rootCertPool,
Certificates: clientCert,
InsecureSkipVerify: false,
}
}
Connection to DB
tlsConf := createTLSConf()
err := mysql.RegisterTLSConfig("custom", &tlsConf)
if err != nil {
log.Printf("Error %s when RegisterTLSConfig\n", err)
return
}
dsn := fmt.Sprintf("%s:%s#tcp(%s)/%s?tls=custom", "user", "pass", "db_host", "db_name")
db, err := sql.Open("mysql", dsn)
if err != nil {
log.Printf("Error %s when opening DB\n", err)
log.Printf("%s", dsn)
return
}
Using connection
defer db.Close()
e := db.Ping()
fmt.Println(dsn, e)
db.Close()
Output:
<user>:<pass>#tcp(<db_host>)/<db_name>?tls=custom x509: certificate is valid for *.abc-xy-pxc, not <db_host>
abc-xy is a part of db_host and also abc_xy is the db_name, nut sure if it is important
When changing InsecureSkipVerify to true, the output looks like this:
[mysql] 2022/10/21 22:31:27 packets.go:37: remote error: tls: unknown certificate authority
Any idea how can I get this to work?
EDIT:
Sooo, I don't know why, but removing part of the code with Client certificates and changing InsecureSkipVerify to true solved the issue:
func createTLSConf() tls.Config {
rootCertPool := x509.NewCertPool()
pem, err := ioutil.ReadFile("certificate.pem")
if err != nil {
log.Fatal(err)
}
if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
log.Fatal("Failed to append PEM.")
}
return tls.Config{
RootCAs: rootCertPool,
InsecureSkipVerify: true, // needed for self signed certs
}
}
Hey I am also new to Go but you can give this a try. Consider my answer might not be the best but I am trying to be more active by helping and asking. You can generate your certificates in terminal and then use them.
run this in terminal at your project directory. (where your main.go located) and the /usr/local/go/src/crypto/tls/generate_cert.go should be the directory where Go standard library is installed.
go run /usr/local/go/src/crypto/tls/generate_cert.go --rsa-bits=2048 --host=localhost
open a DB connection return errors if any:
func openDB(dsn string) (*sql.DB, error){
db, err := sql.Open("mysql",dsn)
if err!= nil{
return nil,err
}
if err = db.Ping();err!=nil{
return nil,err
}
return db,nil
}
Then you have to configure your server settings. You will have a http.Server with your pem files.
declare a variable of crypto/tls.Config where only elliptic curves with assembly implementation are used.
tlsConfig := &tls.Config{
CurvePreferences:[]tls.CurveID{tls.X25519,tls.CurveP256}
}
then declare your server.
yourServer := &http.Server{
//Your configs
TLSConfig : tlsConfig,
//Your other configs if any
}
Now we have created our server with specific TLS configuration. All you have to do is running your server now. However, you have to update your server's method with ListenAndServeTLS(first arg = cert,second arg = key)
err = srv.ListenAndServeTLS(<Location of your cert.pem>, <Location of your key.pem>)

With golang and tls, how to convert rawCerts [][]byte (passed to VerifyPeerCertificate) to an x509.Certificate?

Context: I want to establish a TLS Connection to a server, which has a certificate with a different (but known!) domain.
So I want to use tls.Dial('tcp', 'real-domain', conf), but verify the domain of the certificate as if it would be the other domain (lets call it wrong-domain), of which I know the server should return it.
So I think the way to do this is override VerifyPeerCertificate in the clients tls.Config.
VerifyPeerCertificate gets rawCerts [][]byte as parameter, but to use x509.Verify I need the certificate as a x509.Certificate type.
The question is: How do I cenvert the rawCerts [][]byte, which are passed as a parameter to VerifyPeerCertificate converted to x509.Certificate?
This is how I want to use it:
conf := &tls.Config{
VerifyPeerCertificate: func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
verifyOptions := x509.VerifyOptions{
DNSName: "wrong-domain",
Roots: serverCertPool,
}
cert := &x509.Certificate{???} // How do I get the x509.Certificate out of the rawCerts [][]byte parameter?
_, err := cert.Verify(verifyOptions)
return err
},
InsecureSkipVerify: true, // We verify in VerifyPeerCertificate
}
Use x509.ParseCertificate.
ParseCertificate parses a single certificate from the given ASN.1 DER data.
Example: cert, err := x509.ParseCertificate(rawCerts[0])
By the way this is also how the function that calls VerifyPeerCertificate does:
// crypto/tls/handshake/handshake_client.go
func (c *Conn) verifyServerCertificate(certificates [][]byte) error {
for i, asn1Data := range certificates {
cert, err := x509.ParseCertificate(asn1Data)
if err != nil {
c.sendAlert(alertBadCertificate)
return errors.New("tls: failed to parse certificate from server: " + err.Error())
}
certs[i] = cert
}
// later on...
if c.config.VerifyPeerCertificate != nil {
if err := c.config.VerifyPeerCertificate(certificates, c.verifiedChains); err != nil {
c.sendAlert(alertBadCertificate)
return err
}
}

TLS : Handshake Failure Using GoLang tls client

I'm trying to connect to a server over SSL/TLS using golang http/tsl client which is resulting in 'Handshake Faliure(40)' error, but for some reason, this same endpoint works with CURL command. After some debugging, I have collected the following data.
func PrepCerts(certMap map[string]string) (*http.Transport, bool) {
ok := false
tlsConfig := &tls.Config{}
if len(certMap["ca"]) > 0 {
caCert, err := ioutil.ReadFile(certMap["ca"])
fmt.Println("caCert : ", caCert)
if err != nil {
log.Fatal(err)
} else {
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(caCert)
(*tlsConfig).RootCAs = caCertPool
ok = true
}
}
if len(certMap["cert"]) > 0 && len(certMap["key"]) > 0 {
cert, err := tls.LoadX509KeyPair(certMap["cert"], certMap["key"])
fmt.Println("cert : ", cert)
if err != nil {
log.Fatal(err)
} else {
(*tlsConfig).Certificates = []tls.Certificate{cert}
ok = true
}
}
tlsConfig.BuildNameToCertificate()
return &http.Transport{TLSClientConfig: tlsConfig}, ok
}
Code that uses above function
client := &http.Client{
Timeout: timeout,
}
//certMap = map[string]string{
// ca : "filelocation",
// cert : "filelocation",
// key " "filelocation",
//}
if transport, ok := PrepCerts(certMap); ok {
(*client).Transport = transport
}
resp, err := client.Do(req)
From the captured packets it can be seen that the server is requesting a certificate from the client (Certificate Request). From the the detailed images included in the question it can also be seen that no certificates are sent by the client (Certificate record with Certificate Length 0).
What can also be seen is that the server complains with an Alert after the client has send the (possible empty) certificate so it likely does not like what the client has sent. So it is for sure not a problem of agreeing to a cipher (server agreed on one already) and not a problem that the client does not like the servers certificate (alert is send by server not client).
Based on your code you are trying to do something with client certificates but based on the pcap it looks like you don't succeed in using one. So somewhere there is the problem.
As stated by FiloSottile on Github.
What I think is happening here is that the Certificate Request applies constraints (RSA vs ECDSA, or a specific issuer) which are not satisfied by your certificate.
Using his suggestions you can override the client transports tls.Config.GetClientCertificate()) method.
After following this advice I came to the conclusion that Go will not present tls.Config.RootCAs along with tls.Config.Certificates in response to a certificate request packet.
To solve this issue combine the client certificate and its CA bundle into a single file before calling x509.LoadX509KeyPair(). Taking note that the order of certificates in the file matters. If the client certificate isn't the first one in the bundle you will get a tls: private key does not match public key error.
Once you have combined the client certificate with its CA bundle you can load them into your client like so.
package main
import (
"crypto/tls"
"io/ioutil"
"net/http"
"time"
log "github.com/sirupsen/logrus"
)
const (
certFile = "/location/of/client_cert.bundle.pem"
keyFile = "/location/of/client_cert.key.pem"
testURL = "https://mtls-site"
)
func main() {
clientCert, err := tls.LoadX509KeyPair(certFile, keyFile)
if err != nil {
panic(err)
}
client := http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
Certificates: []tls.Certificate{clientCert},
},
},
}
resp, err := client.Get(testURL)
if err != nil {
panic(err)
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}
log.Infof("%s %s", resp.Status, body)
}

How to get x509 Certificate from http Client in Go

I've been over the docs at https://golang.org/pkg/ but can't make this connection.
I am creating a client and request like so (error handling removed):
client := http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
req, reqErr := http.NewRequest(requestMethod, requestUrl, nil)
resp, clientErr := client.Do(req)
I need to get a x509.Certificate to read details of the cert returned from the server, but still need the http.Repsonse as well.
How can I get a x509.Certificate instance and an http.Response while only making a single request?
The response has a TLS *tls.ConnectionState field, which in turn has:
type ConnectionState struct {
// other fields
PeerCertificates []*x509.Certificate // certificate chain presented by remote peer
}
so you can just do:
resp, clientErr := client.Do(req)
if clientErr != nil {
panic(clientErr)
}
if resp.TLS != nil {
certificates := resp.TLS.PeerCertificates
if len(certificates) > 0 {
// you probably want certificates[0]
cert := certificates[0]
}
}

failed to complete security handshake on grpc?

I've already changed many times the port number on the server and client, but the server always get the incorrect port number.
when I execute the client the server will log this:
2017/05/07 15:06:07 grpc: Server.Serve failed to complete security handshake from "127.0.0.1:32763": remote error: tls: bad certificate
and on the client side, i got this:
2017/05/07 15:06:07 Failed to dial localhost:8070: connection error: desc = "transport: x509: certificate is not valid for any names, but wanted to match localhost:8070"; please retry.
rpc error: code = Internal desc = connection error: desc = "transport: x509: certificate is not valid for any names, but wanted to match localhost:8070"
I have this code for the server.go
func serve() {
addr := "localhost:8070"
crt, key := certificate.CreatePemKey()
certificate, err := tls.X509KeyPair(crt, key)
if err != nil {
fmt.Println(err)
}
certPool := x509.NewCertPool()
ca, err := ioutil.ReadFile("F:/GIAG3.crt")
if err != nil {
fmt.Println(err)
}
if ok := certPool.AppendCertsFromPEM(ca); !ok {
fmt.Println("unable to append certificate")
}
lis, err := net.Listen("tcp", addr)
if err != nil {
fmt.Println("could not list on %s: %s", addr, err)
}
// Create the TLS credentials
creds := credentials.NewTLS(&tls.Config{
ClientAuth: tls.RequireAndVerifyClientCert,
Certificates: []tls.Certificate{certificate},
ClientCAs: certPool,
})
srv := grpc.NewServer(grpc.Creds(creds))
pb.RegisterPingerServer(srv, &server{})
if err := srv.Serve(lis); err != nil {
fmt.Println("grpc serve error: %s", err)
}
}
and this is for the client.go
func testDial2() {
addr := "localhost:8070"
crt, key := certificate.CreatePemKey()
certificate, err := tls.X509KeyPair(crt, key)
if err != nil {
fmt.Println(err)
}
certPool := x509.NewCertPool()
ca, err := ioutil.ReadFile("F:/GIAG3.crt")
if err != nil {
fmt.Println(err)
}
if ok := certPool.AppendCertsFromPEM(ca); !ok {
fmt.Println("unable to append certificate")
}
creds := credentials.NewTLS(&tls.Config{
ServerName: addr,
Certificates: []tls.Certificate{certificate},
RootCAs: certPool,
})
conn, err := grpc.Dial(addr, grpc.WithTransportCredentials(creds))
if err != nil {
fmt.Println(err)
}
defer conn.Close()
c := pb.NewPingerClient(conn)
r, err := c.Ping(context.Background(), &pb.Payload{Message: "Ping"})
if err != nil {
fmt.Println(err)
}
log.Printf("%s", r.Message)
}
this is for the CreatePemKey, it is based on this example https://golang.org/src/crypto/tls/generate_cert.go
func publicKey(priv interface{}) interface{} {
switch k := priv.(type) {
case *rsa.PrivateKey:
return &k.PublicKey
case *ecdsa.PrivateKey:
return &k.PublicKey
default:
return nil
}
}
func pemBlockForKey(priv interface{}) *pem.Block {
switch k := priv.(type) {
case *rsa.PrivateKey:
return &pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(k)}
case *ecdsa.PrivateKey:
b, err := x509.MarshalECPrivateKey(k)
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to marshal ECDSA private key: %v", err)
os.Exit(2)
}
return &pem.Block{Type: "EC PRIVATE KEY", Bytes: b}
default:
return nil
}
}
func CreatePemKey() (certpem, keypem []byte) {
priv, _ := rsa.GenerateKey(rand.Reader, 2048)
notBefore := time.Now()
notAfter := notBefore.AddDate(1, 0, 0)
serialNumberLimit := new(big.Int).Lsh(big.NewInt(1), 128)
serialNumber, _ := rand.Int(rand.Reader, serialNumberLimit)
template := x509.Certificate{
SerialNumber: serialNumber,
Subject: pkix.Name{
Organization: []string{"Acme Co"},
},
NotBefore: notBefore,
NotAfter: notAfter,
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
BasicConstraintsValid: true,
}
// template.IPAddresses = append(template.IPAddresses, net.ParseIP("localhost"))
template.IsCA = true
derbytes, _ := x509.CreateCertificate(rand.Reader, &template, &template, publicKey(priv), priv)
certpem = pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: derbytes})
keypem = pem.EncodeToMemory(pemBlockForKey(priv))
return certpem, keypem
}
BTW the GIAG3.crt is came from here https://pki.goog/
Please help me, thanks
If your server certificate doesn't have the domain definition and it's not signed by GIAG3 (like your example), you should add InsecureSkipVerify (this allow you skip the validation of the server name and the server certificate) in the client configuration, that will fix the issue with the invalid name.
creds := credentials.NewTLS(&tls.Config{
ServerName: addr,
Certificates: []tls.Certificate{certificate},
RootCAs: certPool,
InsecureSkipVerify: true,
})
But you'll have another problem, because the client is using a self-signed certificate, and the server required a certificate signed by GIAG3 for the authentication (tls.RequireAndVerifyClientCert), so you have some options with this,
you use a certificate signed by GIAG3 for the client.
reduce the authentication type to tls.RequireAnyClientCert, this allow you use any certificate at the moment of the auth (it could be or not signed by GIAG3), the client just need to use any certificate when it's connected.
remove the client authentication with certificate.
If you found this log in journalctl -u docker:
mai 29 10:33:04 ca275nt dockerd[1523]: time="2019-05-29T10:33:04.454362399-03:00" level=warning msg="grpc: Server.Serve failed to complete security handshake from \"192.168.0.45:58392\": remote error: tls: bad certificate" module=grpc
It can be related with docker swarm requests from another swarm cluster node host trying to connect to docker swarm master node with a invalid token.
You can discover the invalid cluster node hostname with:
nslookup 192.168.0.45
45.0.168.192.in-addr.arpa name = hostabc.domain.com.
45.0.168.192.in-addr.arpa name = hostabc.
Authoritative answers can be found from:
By running a docker swarm leave on the reported host, resolve the issue.
I know it's not related with the main question, but I was searching for:
docker "grpc: Server.Serve failed to complete security handshake from" and this was the first question. then I think it's useful to put the troubleshooting here to save some time to others.

Resources