For first I am newbie in golang.
I try to send https request. I create http.Client like this:
func httpClient(c *Config) (httpClient *http.Client) {
cert, _ := tls.LoadX509KeyPair(c.CertFile, c.KeyFile)
ssl := &tls.Config{
Certificates: []tls.Certificate{cert},
InsecureSkipVerify: true,
}
ssl.Rand = rand.Reader
return &http.Client{
Transport: &http.Transport{
TLSClientConfig: ssl,
},
}
}
But as result I get local error: no renegotiation.
Thanks for any help!
This is likely a problem with the remote server you're accessing, but it is a known problem (with Microsoft Azure services for one).
There may be a workaround on the way for go1.4, but until then the go client still doesn't support TLS renegotiation.
Relevant issue: https://code.google.com/p/go/issues/detail?id=5742
It looks as though renegotiation (and client certificate authentication) was previously unsupported. This looks to have been fixed by commit https://github.com/golang/go/commit/af125a5193c75dd59307fcf1b26d885010ce8bfd
Related
Before I know which certificate files to use, I need to check the request Host, how to do that ?
I can't call
http.ListenAndServeTLS(":443", "cerfile", "certkey", mux)
because the cert files and "mux" to use depends on the request host address, which is only available in the handler!
and of course I should only use the 443 port!
Can this be done with http package alone ?
As Burak Serdar comment, you could custom GetCertificate of tls.Config to return the specific certificate by different host.
Sample Code
mux := http.NewServeMux()
cfg := &tls.Config{
GetCertificate: func(info *tls.ClientHelloInfo) (*tls.Certificate, error) {
// get certificate by info.ServerName
},
}
srv := &http.Server{
Addr: ":443",
Handler: mux,
TLSConfig: cfg,
}
fmt.Println(srv.ListenAndServeTLS("defaulttls.crt", "defaulttls.key"))
About how to get a certificate by ClientHelloInfo, you could refer to autocert sample codes
I'm trying to use testcontainers-go with HTTPS mode enabled in tests:
req := testcontainers.ContainerRequest{
Image: "wiremock/wiremock",
ExposedPorts: []string{"8080/tcp", "8443/tcp"},
Cmd: []string{"--https-port", "8443", "--verbose"},
}
uri := fmt.Sprintf("https://%s:%s", hostIP, mappedPort.Port())
# see
# https://github.com/testcontainers/testcontainers-go/blob/main/docs/examples/cockroachdb.md
# https://github.com/wiremock/wiremock-docker#start-a-wiremock-container-with-wiremock-arguments
together with walkerus/go-wiremock (WireMock go client) and I'm running into
Post "https://localhost:59279/foo": x509: “Tom Akehurst” certificate is not trusted
I think the reason is go-wiremockconverts wiremock.Post(wiremock.URLPathEqualTo into a direct http call (i.e., it doesn't "expose" http client):
// A Client implements requests to the wiremock server.
type Client struct {
url string
}
so I can't override it to:
tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
Is there other workaround?
I am using Colly to scrape a website and I am trying to also get the TLS certificate that the site is presenting during the TLS handshake. I looked through the documentation and the response object but did not find what I was looking for.
According to the docs, I can customize some http options by changing the default HTTP roundtripper. I tried setting custom GetCertificate and GetClientCertificate functions, assuming that these functions would be used during the TLS handshake, but the print statements are never called.
// Instantiate default collector
c := colly.NewCollector(
// Visit only domains: hackerspaces.org, wiki.hackerspaces.org
colly.AllowedDomains("pkg.go.dev"),
)
c.WithTransport(&http.Transport{
TLSClientConfig: &tls.Config{
GetCertificate: func(ch *tls.ClientHelloInfo) (*tls.Certificate, error) {
fmt.Println("~~~GETCERT CALLED~~")
return nil, nil
},
GetClientCertificate: func(cri *tls.CertificateRequestInfo) (*tls.Certificate, error) {
fmt.Println("~~~GETCLIENTCERT CALLED~~")
return nil, nil
},
},
})
Please help me scrape TLS certificates using Colly.
This is a snippet to get leaf certificate from raw http.Response in case you give up getting certificate using Colly.
tls := ""
if res.TLS != nil && len(res.TLS.PeerCertificates) > 0 {
cert := res.TLS.PeerCertificates[0]
tls = base64.StdEncoding.EncodeToString(cert.Raw)
}
I am currently having an API running on :443 as you can see below:
// RunAsRESTAPI runs the API as REST API
func (api *API) RunAsRESTAPI(restAddr string) error {
// Generate a `Certificate` struct
cert, err := tls.LoadX509KeyPair( ".certificates/my-domain.crt", ".certificates/my-domain.key" )
if err != nil {
return errors.New(fmt.Sprintf("couldn't load the X509 certificates: %v\n", err))
}
// create a custom server with `TLSConfig`
restAPI := &http.Server{
Addr: restAddr,
Handler: nil, // use `http.DefaultServeMux`
TLSConfig: &tls.Config{
Certificates: []tls.Certificate{ cert },
},
}
// Defining the routes
routes := map[string]func(http.ResponseWriter, *http.Request){
"": api.handleIndex,
}
// Initialize mux
mux := http.NewServeMux()
// Register endpoints handlers
for route, function := range routes {
endpoint := "/" + route
mux.HandleFunc(endpoint, function)
log.Printf("[%s] endpoint registered.\n", endpoint)
}
// cors.Default() setup the middleware with default options being
// all origins accepted with simple methods (GET, POST). See
// documentation below for more options.
restAPI.Handler = cors.Default().Handler(mux)
log.Printf("REST TLS Listening on %s\n", restAddr)
return restAPI.ListenAndServeTLS("", "")
}
I created my certificates like so:
$ openssl req -new -newkey rsa:2048 -nodes -keyout my-domain.key -out my-domain.csr
$ openssl x509 -req -days 365 -in my-domain.csr -signkey my-domain.key -out my-domain.crt
I then dockerized, then deployed to Google Compute Engine, but, I am still getting this net::ERR_CERT_INVALID while requesting my API from a ReactJs App (Google Chrome)
I have no issues on Postman.. I don't understand, it even says that this certificate has not been verified by a third party
I am a bit lost, to be honest, how can I solve this? So my app can request my HTTPS backend
Thanks
Options
Instead of using a self-signed cert, use LetsEncrypt to generate a free cert that is considered valid because it has a backing Certificate Authority.
If your cert is secure enough for your purposes, add it to your client (browser) as a root CA. No one else will be able to connect to the API without doing the same.
This line should have error checking:
cert, err := tls.LoadX509KeyPair( ".certificates/my-domain.crt", ".certificates/my-domain.key" )
if err != nil {
log.Fatalf("key-pair error: %v", err)
}
While it may work locally, you can't be sure it also works in your Docker container.
Are those key/cert files available? If not - and without error checking - you are effectively passing a nil cert to your TLS config which will have no effect and no other discernable error.
You're using a self-signed certificate, so your browser doesn't trust it -- by design. Postman is telling you this with that message, too. This isn't a go issue, which is doing precisely what you told it to do: serving your application with an insecure certificate.
Thanks for your answers. I actually found a way to make it work in a SUPER EASY WAY:
func (api *API) RunAsRESTAPI() error {
// Defining the routes
routes := map[string]func(http.ResponseWriter, *http.Request){
"": api.handleIndex,
}
// Initialize mux
mux := http.NewServeMux()
// Register endpoints handlers
for route, function := range routes {
endpoint := "/" + route
mux.HandleFunc(endpoint, function)
log.Printf("[%s] endpoint registered.\n", endpoint)
}
// cors.Default() setup the middleware with default options being
// all origins accepted with simple methods (GET, POST). See
// documentation below for more options.
handler := cors.Default().Handler(mux)
log.Printf("REST TLS Listening on %s\n", "api.my-domain.com")
return http.Serve(autocert.NewListener("api.my-domain.com"), handler)
}
autocert.NewListener("api.my-domain.com") directly fixed my problem :)
I'm attempting to connect to RabbitMQ with amqps:// in Go using streadway/amqp. I can connect successfully with amqp://. When enabling TLS and using amqps:// I get the following error:
panic: remote error: tls: handshake failure
RabbitMQ is running in docker with the following environment variables and settings:
environment:
RABBITMQ_SSL_CACERTFILE: /ca_certificate.pem
RABBITMQ_SSL_CERTFILE: /server_certificate.pem
RABBITMQ_SSL_KEYFILE: /server_key.pem
ports:
- 5671:5671 # Note that 5671 is for tls and 5672 is non-tls
volumes:
- ./ca_certificate.pem:/ca_certificate.pem:ro
- ./server_certificate.pem:/server_certificate.pem:ro
- ./server_key.pem:/server_key.pem:ro
I've tried the following with amqp/streadway:
err := amqp.DialTLS(amqps://guest:guest#localhost:5671", nil)
if err != nil {
panic(err)
}
I've also tried reading the cert files, creating a key pair, and appending the certificate authority to the cert pool and using it that way in a tls.Config{} with the following functions:
tls.LoadX509KeyPair()
x509.NewCertPool().AppendCertsFromPEM()
I generate the certs with mkcert for 127.0.0.1, localhost, rabbitmq.
According to some answers that aren't related to RabbitMQ, some people suggest the ciphers could be wrong. So I took a look at what ciphers rabbitmq is using:
$ openssl s_client -connect localhost:5671 -tls1
Protocol : TLSv1
Cipher : ECDHE-RSA-AES256-SHA
<etc etc...>
Verify return code: 0 (ok)
There are also one or two errors when I run the above command, but I'm guessing it's because I'm not providing the CA certificate in this command (I'm using MacOS). Maybe related, maybe not, as I don't have this issue with postgres, for example:
verify error:num=19:self signed certificate in certificate chain
verify return:0
4644699756:error:1401E410:SSL routines:CONNECT_CR_FINISHED:sslv3 alert handshake failure:/AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-47.100.4/libressl-2.8/ssl/ssl_pkt.c:1200:SSL alert number 40
Then I use the following tls.Config settings in golang:
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{cert}, // from tls.LoadX509KeyPair
RootCAs: caCertPool,
CipherSuites: []uint16{
tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, // these look like they match the Cipher above
tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
},
CurvePreferences: []tls.CurveID{tls.CurveP521, tls.CurveP384, tls.CurveP256},
PreferServerCipherSuites: true,
InsecureSkipVerify: true,
MinVersion: tls.VersionTLS10,
}
I still have the same issue. I highly doubt it's the library, it must be something I'm doing wrong, but what is it?
I reproduced your setup. It doesn't work because you need to configure the AMQP connection with the client certs.
Using mkcert: mkcert -client rabbitmq.test localhost 127.0.0.1 ::1 (note the -client flag).
After this, you just need to pass the client certs into your AMQP tlsConfig with tls.LoadX509KeyPair, and it should just work:
cert, err := tls.LoadX509KeyPair("./rabbitmq.test+3-client.pem", "./rabbitmq.test+3-client-key.pem")
// Load CA cert
caCert, err := ioutil.ReadFile("./rootCA.pem") // The same you configured in your MQ server
if err != nil {
log.Fatal(err)
}
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(caCert)
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{cert}, // from tls.LoadX509KeyPair
RootCAs: caCertPool,
// ...other options are just the same as yours
}
conn, err := amqp.DialTLS("amqps://test:secret#127.0.0.1:5671", tlsConfig)
if err != nil {
panic(err) // does not panic!
}
// ... application code
PS: in my setup I used some different names (user/password/container) than yours, but those should be irrelevant