The server created by NewTLSServer can validate calls for a client that is explicitly created from it:
ts := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "Hello, client")
}))
defer ts.Close()
client := ts.Client()
res, err := client.Get(ts.URL)
in the line client := ts.Client().
However, I have a production program that I want to set to use ts.URL as its host. I am getting
x509: certificate signed by unknown authority
errors when I call it.
How can I set ts up to authenticate with the client like a normal HTTPS server?
As of Go 1.11, this simply isn't possible to accomplish fully within a _test.go program, due to the mechanics of HTTPS.
However, you can do a single certificate signing and generation of server.crt and server.key files, then reference them in your _test.go programs from a local directory indefinitely.
One-time .crt and .key generation
This is an abridged, slightly streamlined version of the steps specified in Daksh Shah's Medium article, How to get HTTPS working on your local development environment in 5 minutes, which will work on a Mac.
In the directory where you want your server.crt and server.key files, create the two configuration files
server.csr.cnf
[req]
default_bits = 2048
prompt = no
default_md = sha256
distinguished_name = dn
[dn]
C=US
ST=RandomState
L=RandomCity
O=RandomOrganization
OU=RandomOrganizationUnit
emailAddress=hello#example.com
CN = localhost
and
v3.ext
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
subjectAltName = #alt_names
[alt_names]
DNS.1 = localhost
IP.1 = 127.0.0.1
Then enter the following commands in that directory
openssl genrsa -des3 -out rootCA.key 2048
# create a passphrase
openssl req -x509 -new -nodes -key rootCA.key -sha256 -days 1024 -out rootCA.pem -config server.csr.cnf
# enter passphrase
openssl req -new -sha256 -nodes -out server.csr -newkey rsa:2048 -keyout server.key -config server.csr.cnf
openssl x509 -req -in server.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out server.crt -days 500 -sha256 -extfile v3.ext
# enter passphrase
Finally, make your system trust the certificate you used to sign the files by running
open rootCA.pem
This should open the certificate in the Keychain Acces app, where it will be found in the section Certificates and named localhost. Then to Always Trust it
Press enter to open its window
Press space to twirl down Trust
Change "When using this certificate:" to Always Trust
Close the window and authenticate your decision
Note: I have tried many permutations of security add-trusted-cert from the command line and, despite the fact that it adds the cert to the keychain and marks it as "Always Trust", my Go programs just won't trust it. Only the GUI method puts the system in a state that my Go programs will trust the cert.
Any Go programs you run locally using HTTPS will now trust servers you run using server.crt and server.key.
Running the server
You can create *httptest.Server instances that use these credentials with
func NewLocalHTTPSTestServer(handler http.Handler) (*httptest.Server, error) {
ts := httptest.NewUnstartedServer(handler)
cert, err := tls.LoadX509KeyPair("server.crt", "server.key")
if err != nil {
return nil, err
}
ts.TLS = &tls.Config{Certificates: []tls.Certificate{cert}}
ts.StartTLS()
return ts, nil
}
Here is an example usage:
func TestLocalHTTPSserver(t *testing.T) {
ts, err := NewLocalHTTPSTestServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprint(w, "Hello, client")
}))
assert.Nil(t, err)
defer ts.Close()
res, err := http.Get(ts.URL)
assert.Nil(t, err)
greeting, err := ioutil.ReadAll(res.Body)
res.Body.Close()
assert.Nil(t, err)
assert.Equal(t, "Hello, client", string(greeting))
}
Related
Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 4 months ago.
Improve this question
I'm trying to generate an RSA PSS signature with openssl in a bash script and verify it in a go microservice. Verification fails across openssl/go.
Generate the keys and signature (signature.bin) in openssl/bash...
openssl genrsa -out key.pem 4096
openssl rsa -in key.pem -pubout > key.pub
truncate -size=16 zeroes.bin
openssl dgst -sign key.pem -keyform PEM -sha512 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:64 -out signature.bin -binary zeroes.bin
Use openssl to verify signature (ok)...
openssl dgst -verify key.pub -keyform PEM -sha512 -sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:64 -signature signature.bin -binary zeroes.bin
Verified OK
Go can create and verify its own RSA-PSS, but fails on the signature from openssl...
go run main.go
Verified go signature.
panic: crypto/rsa: verification error
goroutine 1 [running]:
main.main()
/root/taas/applications.security.amber.core-services/pss/x/main.go:51 +0x339
exit status 2
openssl also fails to verify the go signature...
openssl dgst -verify key.pub -keyform PEM -sha512 -signature signature.**go**.bin -binary zeroes.bin
Verification Failure
main.go
package main
import (
"crypto"
"crypto/rand"
"crypto/rsa"
"crypto/sha512"
"crypto/x509"
"encoding/pem"
"fmt"
"io/ioutil"
)
func main() {
privatePem, _ := ioutil.ReadFile("key.pem")
privateBlock, _ := pem.Decode(privatePem)
if privateBlock == nil {
panic("Failed to decode private pem")
}
privateKey, err := x509.ParsePKCS1PrivateKey(privateBlock.Bytes)
if err != nil {
panic(err)
}
h := sha512.New()
h.Write(make([]byte, 16))
zeroes := h.Sum(nil)
signature, err := rsa.SignPSS(rand.Reader, privateKey, crypto.SHA512, zeroes, &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthEqualsHash, // should be 64 for sha512
})
if err != nil {
panic(err)
}
ioutil.WriteFile("signature.go.bin", signature, 0600)
err = rsa.VerifyPSS(&privateKey.PublicKey, crypto.SHA512, zeroes, signature, &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthEqualsHash,
})
if err != nil {
panic(err)
}
fmt.Println("Verified go signature.")
opensslSignature, _ := ioutil.ReadFile("signature.bin")
err = rsa.VerifyPSS(&privateKey.PublicKey, crypto.SHA512, zeroes, opensslSignature, &rsa.PSSOptions{
SaltLength: rsa.PSSSaltLengthEqualsHash,
})
if err != nil {
panic(err)
}
fmt.Println("Not getting here -- crypto/rsa: verification error")
}
I have encrypted a file using this code.
block, err := aes.NewCipher([]byte("TESTPASSWORD1234TESTPASSWORD1234"))
if err != nil {
panic(err)
}
bReader, err := os.Open("doc.docx")
if err != nil {
panic(err)
}
var iv [aes.BlockSize]byte
stream := cipher.NewOFB(block, iv[:])
var out bytes.Buffer
writer := &cipher.StreamWriter{S: stream, W: &out}
if _, err := io.Copy(writer, bReader); err != nil {
panic(err)
}
if os.WriteFile("doc-encrypted.docx", out.Bytes(), 0644) != nil {
panic(err)
}
and when I try to decrypt it using this command
openssl enc -in doc-encrypted.docx -out doc-decryted.docx -d -aes-256-ofb
it gives the error bad magic number
Your OpenSSL statement is missing the specification of key and IV. For decryption, the following OpenSSL statement is required:
openssl enc -in doc-encrypted.docx -out doc-decryted.docx -d -aes-256-ofb -K 5445535450415353574f5244313233345445535450415353574f524431323334 -iv 00000000000000000000000000000000
The -K option specifies the hex encoded key, and -iv specifies the hex encoded IV, s. enc.
With this change, the ciphertext generated with the Go code can be decrypted with the OpenSSL statement.
Keep in mind that the use of a static IV is insecure. Typically, a random IV is generated for each encryption. This is not secret and is usually concatenated with the ciphertext: iv|ciphertext so that it is available during decryption. See the documentation for NewOFB for an example (without file I/O).
Recently, I encountered RSA signature in a project. I hope an experienced brother could help.
The OpenSSL (version 1.0.2u) command I used:
echo -n -e "test\n"|openssl dgst -sha256 -sign ./apiclient_key.pem| openssl base64 -A
The result is as follows
pyoBMuN8UqRGLVR7YcQ11yn+dQ9rSU/fB7obQhs27eotvd51q+E8BqxB6AYQDTnlqAQnOiR1rnuxPjlGkAOaPxpCqfhS5VGblh3HuNNHiycdKKa5mM1XyaWROiL7YpyYHRUcblkICW4XEN8v5wyFHxQ+TZfBN8fdqmlLdSczZ66YnIUgaWjBkdC1UH9GqMOQkySaQbgxjh4WhWvVE4umlKz+lAj9OLBhqI/ZXcs6gPFIpyNl8hlMPi6QOxFDSPZmQZl9G7mzx4E0lBoCY5XJtm5VwG3IYTryKZvSF0/GjVyR6QA3/sY25WCPL6f/y0biovk+mJ2KvPvPX26hK4DYug==
My signature related golang Code:
// get private key
bPrivateKey, err := ioutil.ReadFile("./apiclient_key.pem")
if err != nil || bPrivateKey == nil {
log.Println(err)
return
}
block, _ := pem.Decode(bPrivateKey)
if block == nil {
return
}
pKeyInterface, err := x509.ParsePKCS8PrivateKey(block.Bytes)
if err != nil {
log.Println(err)
return
}
pKey, ok := pKeyInterface.(*rsa.PrivateKey)
if !ok {
return
}
// sign
strForSign = "test\n"
shaForSign := sha256.New()
shaForSign.Write([]byte(strForSign))
hashedForSign := shaForSign.Sum(nil)
rawSignature, err := rsa.SignPKCS1v15(rand.Reader, pKey, crypto.SHA256, hashedForSign)
if err != nil {
log.Println(err)
return
}
signature := base64.StdEncoding.EncodeToString(rawSignature)
log.Println(signature)
The result is as follows
NcW5pBmUfHqVNus1PTDjGOilazWkcyxquGc/Ldu5IAjg/gAIQOKBGp7rs8thec/THhWKjZOJtZ1Xvv85vc+bG5bB4IuCZp+wkUMgDC3kFuTPjtLEBBnlhshZ1nS0Haq5BuS6aWAF9sIz6Ulq9dLMjaOAACijwEltdOdkRo8Z5V01CZMOPM3FI0dVvTGOvXxsMvLjw3XPAxNpajXhxTTGZB5jElDkTb61U/cZ9tM+iVpd7Oo+vxvgCsrx2VhV4gURrxndj3V6Nc2iNV2bByrgw8XK1htzkqLqZpWo8JF8i5LdMrfaMukn6aikhWKFOo2icatjowiSQAAXrj9EnnrbQA==
apicclient_ key.pem Document content follows:
-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC+YVIbsE+b5Rsg
oSrcV7H21SBICjTcCKDiqEv94ghL3MfBjW5g/m9o9dcTdBpBED4NaWX+N9+aXi9o
aGNUU5Eghg1eZeFcw0utasuQnMwIc64K4L60FQxFVHjEoV0uEML7TkxMIPWh2Vdm
qyOAmBZldPBMRTXyQoqohlS0Z8z1NAwz9mVy9rnNaIgQT5FFjvfPW2dvCuvLjgo8
wqWfgJttB/8ptcUZoJU7EtPqLheEibuSDQtjGBVxttq9O6Eo3zyS2I6/0JZCRGqF
RRIqPmD2ivpI7ZwXE/cEk9R6Dv9lGZS4XZxUFpDhJnQircnH3N41nYgnFaWDm7sJ
BP9TZnkBAgMBAAECggEATumOHDYHuYnMpcjIOE8NUE/lMpcwc+gLsAesgA7Z+4AH
OL5D+2ZbjbkhLkoYu9Fctzhx8177i0PxCKkESybcaAvIrNE0CbxVuQskV8v2cqZo
hM/FnGWBEZfC+1YbhoWiHILG82NSDYVGSiMz8xCXb/R5ylOB2eolCVes8p54AUab
KU58GGFEYFMGnccy9SZ7QkG00fBLU7vJ9MINzpujujDTOS+f+MTkPpushjfUqWyj
2pfj235xajVTeHPag4Gig5wwmblu+fUxEsXtIkvM91/5GzsRVh6VIqSABhrsUXr4
zhFuiqw/ZhtYa32HnU6cpMOKIbDaHu31C0bE/CEcwQKBgQDtnEnEd8+QAuLfSsV6
wGBXV5cfnmmlkWKFxhDzW0solk9eBDKfIVw7Zw1NdanBlIrK187sOETArzeVEVFK
J4r4CnPuguTMoik5AXNAOxnA5OrYP4Tn8xk3AV9Kuh16f+CAt6zzVQQo2FXzoteK
NOlLM+oLe+XrY8kQFJkv4GXDmQKBgQDNHUVPmP2FeHbpU79mwun143HGDSPDslek
nOEv+/1Cm9+L1sla1Yglb6OZrk9wxjnw0c4HnQV5+l1K6c72ujI840BobjUqMqCW
mvb14lr/p1pVyAsSbMBlGR6EgR/QkMD/CL7KTKa/RwiM393oslvxvdGxCPuKEISN
rCt/zxLBqQKBgEHXPNmkGW/eXN9i6LXK+Oc67jVkwAGQ96v55liIw5rLi9KRdzWB
GP2c4KLGJ/PsJUlv43axYNvDsbMsyUmzC18QBm/g9WV+yc1yCW2S8t/wjuK9NybO
xf3n82/RgrmfFaaVPudud2iaCbcpOHCjmMGt8PydqGhBjkVWb9xEYe6BAoGBAIjx
/hDC6KFwaJQoIiaOBH7zOEcvBq3mKLXzODY9XD5Yq8xFv3IyFxBHlUdKJif5cT3k
Flbm830ZN6iIqnH4pwoYACy5SC3AV4+2LyTD2FXQenPDeAD5MwtM1h9JxuANhY3B
XQDcZIhY+NB1t0dhrhmczqAQi0EG/jlItlJDAxbhAoGAV5l05wHqqwEZroN0gXq2
86JTftIRzmstgs6O/yfjuInqHjMGp1fTOLyqbwA6+ZrHmzNyyoD8jiGW068LC1ue
7xucV/dYuUXUXOcX0HN/trIdMkl5jQWkrXSKP5o5ekw9gsMcicWFUBXdnJMxCcYb
i8CRH78crAdaIqZ+2buIozo=
-----END PRIVATE KEY-----
$ echo -n -e "test\n" | openssl dgst -sha256 -sign ./apiclient_key.pem | openssl base64 -A
NcW5pBmUfHqVNus1PTDjGOilazWkcyxquGc/Ldu5IAjg/gAIQOKBGp7rs8thec/THhWKjZOJtZ1Xvv85vc+bG5bB4IuCZp+wkUMgDC3kFuTPjtLEBBnlhshZ1nS0Haq5BuS6aWAF9sIz6Ulq9dLMjaOAACijwEltdOdkRo8Z5V01CZMOPM3FI0dVvTGOvXxsMvLjw3XPAxNpajXhxTTGZB5jElDkTb61U/cZ9tM+iVpd7Oo+vxvgCsrx2VhV4gURrxndj3V6Nc2iNV2bByrgw8XK1htzkqLqZpWo8JF8i5LdMrfaMukn6aikhWKFOo2icatjowiSQAAXrj9EnnrbQA==%
I would double check how you are running the openssl command in the shell. Any extraneous output to stdout is going to change the signature.
I get the "NcW.." signature from both 1.0.2n-fips and openssl-3.0.0-alpha6, and from running your Go code locally.
I'd like to verify a PEM certificate against an issuing chain which is also a .pem file with several certificates separated by newline characters as shown in this gist, https://gist.github.com/kurtpeek/8bf3282e344c781a20c5deadac75059f. I've tried this with Certpool.AppendCertsFromPEM as follows:
package main
import (
"crypto/x509"
"encoding/pem"
"io/ioutil"
"github.com/sirupsen/logrus"
)
func main() {
caCertPEM, err := ioutil.ReadFile("issuing_chain.pem")
if err != nil {
logrus.WithError(err).Fatal("read CA PEM file")
}
certPEM, err := ioutil.ReadFile("3007e750-e769-440b-9075-41dc2b5b1787.pem")
if err != nil {
logrus.WithError(err).Fatal("read cert PEM file")
}
block, rest := pem.Decode(certPEM)
if block == nil {
logrus.WithField("rest", rest).Fatal("Decode CA PEM")
}
cert, err := x509.ParseCertificate(block.Bytes)
if err != nil {
logrus.WithError(err).Fatal("parse certificate")
}
roots := x509.NewCertPool()
roots.AppendCertsFromPEM(caCertPEM)
chain, err := cert.Verify(x509.VerifyOptions{Roots: roots})
if err != nil {
logrus.WithError(err).Fatal("failed to verify cert")
}
logrus.Infof("issuing chain: %+v", chain)
}
However, if I run this I get the following error:
FATA[0000] failed to verify cert error="x509: certificate specifies an incompatible key usage"
exit status 1
I believe this error is returned on line 790 of https://golang.org/src/crypto/x509/verify.go:
if len(chains) == 0 {
return nil, CertificateInvalidError{c, IncompatibleUsage, ""}
}
In other words, the Verify() method is unable to build any chains from the options provided. I've tried splitting out the intermediates (the top two in the issuing_chain.pem shown in the gist) into a separate PEM file and adding those as Intermediates to the x509.VerifyOptions, but I still get the same error.
What is the correct way to verify a certificate against an issuing chain in Go?
Your leaf certificate is for client authentication only.
$ openssl x509 -noout -text -in leaf.pem | grep -A1 'Key Usage'
X509v3 Key Usage: critical
Digital Signature, Key Encipherment
X509v3 Extended Key Usage:
TLS Web Client Authentication
If this is intentional, you must specify the KeyUsages option because "an empty list means ExtKeyUsageServerAuth". You also have to go back to your version of the code that provides the intermediate certs separately:
chain, err := cert.Verify(x509.VerifyOptions{
Roots: roots,
Intermediates: inters,
KeyUsages: []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth},
})
Try it on the playground: https://play.golang.org/p/1BNLthzu5Tz. Note that the playground requires the CurrentTime option to verify correctly. Remove this when copying elsewhere!
I have a cert file and a key file where the key file is encrypted with a password. I am trying to decrypt the key file programmatically before loading the key pair.
// Key file and cert file path
cf := filepath.Join(certPath, certFile)
kf := filepath.Join(certPath, keyFile)
//Read & decode the encrypted key file with the pass to make tls work
keyIn, err := ioutil.ReadFile(kf)
if err != nil {
log.Error("Read key error", err)
return nil, err
}
// Decode and decrypt our PEM block
decodedPEM, _ := pem.Decode([]byte(keyIn))
decrypedPemBlock, err := x509.DecryptPEMBlock(decodedPEM, []byte("somepassword"))
if err != nil {
log.Error("decrypt key error", err)
return nil, err
}
// Load our decrypted key pair
crt, err := tls.LoadX509KeyPair(cf, string(decrypedPemBlock))
if err != nil {
log.Error("load key pair error", err)
return nil, err
}
The original certs and key were generated using the following openssl parameters
openssl req -new -newkey rsa:2048 -x509 -keyout $CA_CERT.key -out $CA_CERT -days $validity -passin "pass:$password" -passout "pass:$password" -subj "/C=$C/ST=$ST/L=$L/O=$O/CN=$CN/emailAddress=$EMAIL"
The variables are substituted accordingly with $password being "somepassword"
I have tried decrypting the key using openssl rsa from the command line and it works fine with the the password above.
However in Go I get an error in tls.LoadX509KeyPair saying invalid argument.
time="2018-01-17T18:57:40Z" level=error msg="load key pair error: open
My best guess is that the key encoding might be messed up and I was wondering if anything is wrong with my code.
Update: Added error message and it seems like tls.LoadX509KeyPair can't understand the format as comments have pointed out below.