My use case looks like I know the public certificates of my clients and only want to allow them. I have a go server based on gin and a TLS configuration in which I have assigned a method to the property "VerifyPeerCertificate".
The function looks like
func customVerifyPeerCertificate(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
if len(verifiedChains) < 1 {
return errors.New("Verified certificate chains is empty.")
}
if len(verifiedChains[0]) < 1 {
return errors.New("No certificates in certificate chains.")
}
if len(verifiedChains[0][0].Subject.CommonName) < 1 {
return errors.New("Common name can not be empty.")
}
fmt.Println(verifiedChains[0][0].Raw)
publicKeyDer, _ := x509.MarshalPKIXPublicKey(verifiedChains[0][0].PublicKey)
publicKeyBlock := pem.Block{
Type: "CERTIFICATE",
Bytes: publicKeyDer,
}
publicKeyPem := string(pem.EncodeToMemory(&publicKeyBlock))
}
The problem is, however, that the string in the variable "publicKeyPem" does not look like the client public certificate I used to send the request to the server, it's also shorter in length.
A certificate is more than its public key. The entire x509.Certificate object represents the certificate presented by the client, the public key field is only the actual value of the public key.
If you want to compare certificates for strict equality, you should use the rawCerts [][]byte argument passed to your callback. This is mentioned in the tls.Config comments for VerifyPeerCertificate:
VerifyPeerCertificate, if not nil, is called after normal
certificate verification by either a TLS client or server. It
receives the raw ASN.1 certificates provided by the peer and also
any verified chains that normal processing found. If it returns a
non-nil error, the handshake is aborted and that error results.
Thanks to Marc, I know that I used the wrong variable. To convert the certificate as a string, as used by the client, use the following code
publicKeyBlock := pem.Block{
Type: "CERTIFICATE",
Bytes: rawCerts[0],
}
publicKeyPem := string(pem.EncodeToMemory(&publicKeyBlock))
Related
I'm trying to use GitHub App and I need to generate a JWT for authenticating (https://docs.github.com/en/developers/apps/building-github-apps/authenticating-with-github-apps#generating-a-private-key)
I'm trying to do that using Goland.
How can I generate a JWT from PEM private key in Go??
The jwt-go library has all the tools you need, and is fairly well documented. You can find it at https://github.com/golang-jwt/jwt.
Assuming you understand what JWTs are and how they're structured, and that you can get that PEM key as a []byte, the process is roughly:
Add "github.com/golang-jwt/jwt/v4" to your imports.
Create a set of claims, which can include the RegisteredClaims type and any custom claims you may need.
Create the token with jwt.NewWithClaims() - you'll need to provide the appropriate signing method. I've primarily used RS256.
Create the JWT string from the token with token.SignedString().
In practice, it will look something like this:
imports "github.com/golang-jwt/jwt/v4"
type MyCustomClaims struct {
*jwt.RegisteredClaims
FooClaim int
BarClaim string
}
func CreateJWT(pemKey []byte) string {
// expires in 60 minutes
expiration := time.Now().Add(time.Second * 3600)
claims := MyCustomClaims{
RegisteredClaims: &jwt.RegisteredClaims{
Issuer: "Example Code Inc.",
ExpiresAt: jwt.NewNumericDate(expiration),
Subject: "JWT Creation",
},
FooClaim: 123,
BarClaim: "bar",
}
token := jwt.NewWithClaims(jwt.SigningMethodRS256, claims)
privateKey, _ := jwt.ParseRSAPrivateKeyFromPEM(pemKey)
myJWT, _ := jwt.SignedString(privateKey)
return myJWT
}
I suggest reading code from this repository:
https://github.com/bradleyfalzon/ghinstallation
I don't know why, but the code in the answer from #JesseB above didn't work for me - it always throws: 401 Unauthorized. Although this repository does use golang-jwt package internally
I am using Colly to scrape a website and I am trying to also get the TLS certificate that the site is presenting during the TLS handshake. I looked through the documentation and the response object but did not find what I was looking for.
According to the docs, I can customize some http options by changing the default HTTP roundtripper. I tried setting custom GetCertificate and GetClientCertificate functions, assuming that these functions would be used during the TLS handshake, but the print statements are never called.
// Instantiate default collector
c := colly.NewCollector(
// Visit only domains: hackerspaces.org, wiki.hackerspaces.org
colly.AllowedDomains("pkg.go.dev"),
)
c.WithTransport(&http.Transport{
TLSClientConfig: &tls.Config{
GetCertificate: func(ch *tls.ClientHelloInfo) (*tls.Certificate, error) {
fmt.Println("~~~GETCERT CALLED~~")
return nil, nil
},
GetClientCertificate: func(cri *tls.CertificateRequestInfo) (*tls.Certificate, error) {
fmt.Println("~~~GETCLIENTCERT CALLED~~")
return nil, nil
},
},
})
Please help me scrape TLS certificates using Colly.
This is a snippet to get leaf certificate from raw http.Response in case you give up getting certificate using Colly.
tls := ""
if res.TLS != nil && len(res.TLS.PeerCertificates) > 0 {
cert := res.TLS.PeerCertificates[0]
tls = base64.StdEncoding.EncodeToString(cert.Raw)
}
I am currently having an API running on :443 as you can see below:
// RunAsRESTAPI runs the API as REST API
func (api *API) RunAsRESTAPI(restAddr string) error {
// Generate a `Certificate` struct
cert, err := tls.LoadX509KeyPair( ".certificates/my-domain.crt", ".certificates/my-domain.key" )
if err != nil {
return errors.New(fmt.Sprintf("couldn't load the X509 certificates: %v\n", err))
}
// create a custom server with `TLSConfig`
restAPI := &http.Server{
Addr: restAddr,
Handler: nil, // use `http.DefaultServeMux`
TLSConfig: &tls.Config{
Certificates: []tls.Certificate{ cert },
},
}
// Defining the routes
routes := map[string]func(http.ResponseWriter, *http.Request){
"": api.handleIndex,
}
// Initialize mux
mux := http.NewServeMux()
// Register endpoints handlers
for route, function := range routes {
endpoint := "/" + route
mux.HandleFunc(endpoint, function)
log.Printf("[%s] endpoint registered.\n", endpoint)
}
// cors.Default() setup the middleware with default options being
// all origins accepted with simple methods (GET, POST). See
// documentation below for more options.
restAPI.Handler = cors.Default().Handler(mux)
log.Printf("REST TLS Listening on %s\n", restAddr)
return restAPI.ListenAndServeTLS("", "")
}
I created my certificates like so:
$ openssl req -new -newkey rsa:2048 -nodes -keyout my-domain.key -out my-domain.csr
$ openssl x509 -req -days 365 -in my-domain.csr -signkey my-domain.key -out my-domain.crt
I then dockerized, then deployed to Google Compute Engine, but, I am still getting this net::ERR_CERT_INVALID while requesting my API from a ReactJs App (Google Chrome)
I have no issues on Postman.. I don't understand, it even says that this certificate has not been verified by a third party
I am a bit lost, to be honest, how can I solve this? So my app can request my HTTPS backend
Thanks
Options
Instead of using a self-signed cert, use LetsEncrypt to generate a free cert that is considered valid because it has a backing Certificate Authority.
If your cert is secure enough for your purposes, add it to your client (browser) as a root CA. No one else will be able to connect to the API without doing the same.
This line should have error checking:
cert, err := tls.LoadX509KeyPair( ".certificates/my-domain.crt", ".certificates/my-domain.key" )
if err != nil {
log.Fatalf("key-pair error: %v", err)
}
While it may work locally, you can't be sure it also works in your Docker container.
Are those key/cert files available? If not - and without error checking - you are effectively passing a nil cert to your TLS config which will have no effect and no other discernable error.
You're using a self-signed certificate, so your browser doesn't trust it -- by design. Postman is telling you this with that message, too. This isn't a go issue, which is doing precisely what you told it to do: serving your application with an insecure certificate.
Thanks for your answers. I actually found a way to make it work in a SUPER EASY WAY:
func (api *API) RunAsRESTAPI() error {
// Defining the routes
routes := map[string]func(http.ResponseWriter, *http.Request){
"": api.handleIndex,
}
// Initialize mux
mux := http.NewServeMux()
// Register endpoints handlers
for route, function := range routes {
endpoint := "/" + route
mux.HandleFunc(endpoint, function)
log.Printf("[%s] endpoint registered.\n", endpoint)
}
// cors.Default() setup the middleware with default options being
// all origins accepted with simple methods (GET, POST). See
// documentation below for more options.
handler := cors.Default().Handler(mux)
log.Printf("REST TLS Listening on %s\n", "api.my-domain.com")
return http.Serve(autocert.NewListener("api.my-domain.com"), handler)
}
autocert.NewListener("api.my-domain.com") directly fixed my problem :)
I'm making a little API using Go. I would like to be able to handle Github webhooks with secret token. I set up the secret on my webhook Github which is "azerty".
Now I try to verify that the incoming webhook has the correct secret token. I've read Github documentation which say the algorithm use HMAC with SHA1. But I can't verify the secret from the incoming Github webhook ...
func IsValidSignature(r *http.Request, key string) bool {
// KEY => azerty
gotHash := strings.SplitN(r.Header.Get("X-Hub-Signature"), "=", 2)
if gotHash[0] != "sha1" {
return false
}
defer r.Body.Close()
b, err := ioutil.ReadAll(r.Body)
if err != nil {
log.Printf("Cannot read the request body: %s\n", err)
return false
}
hash := hmac.New(sha1.New, []byte(key))
if _, err := hash.Write(b); err != nil {
log.Printf("Cannot compute the HMAC for request: %s\n", err)
return false
}
expectedHash := hex.EncodeToString(hash.Sum(nil))
log.Println("EXPECTED HASH:", expectedHash)
log.Println("GOT HASH:", gotHash[1])
return gotHash[1] == expectedHash
}
EXPECTED HASH: 10972179a3b0efc337f79ec41847062bc598bb04
GOT HASH: 36de72e0d386e36e2c7b034c85cd3b3889594992
To test, I copy the payload of the Github webhook in Postman with the right headers. I don't know why I get two different hash ... I've checked my key is non-empty with the correct value and my body is non-empty too.
Do I miss something?
I copy the payload of the Github webhook in Postman with the right header.
I've checked my key is non-empty with the correct value and my body is non-empty too.
The crypto is correct, except few minor issues. Obviously your body does not match the same body you have got from Github. Could be formatting, trailing newlines, etc. It must exactly byte-to-byte match the original body.
If this code works with Github and does not work with copy in Postman just replace X-Hub-Signature in your fixture with that "wrong" hash.
Some extras:
Use hmac.Equal for secure comparison
hash.Write never returns errors
I'm trying to use the new Firebase Realtime Database for a simple logging application. All interraction with the database will be from my server, so I only need one account that can read/write anything.
As far as I can tell (the documentation is awful - there's plenty of it but it contradicts itself and half of it is for the 'old' Firebase, and often it is in some random language that you aren't using), I need to create a service account and then create a JWT token using OAuth. Fortunately Go has some nice built in libraries for this. Here is my code:
const firebasePostUrl = "https://my-product-logging.firebaseio.com/tests.json"
// Obtained from the Google Cloud API Console
var firebaseServiceAccount map[string]string = map[string]string{
"type": "service_account",
"project_id": "my-product-logging",
"private_key_id": "1c35ac0c501617b8f1610113c492a5d3321f4318",
"private_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoblahblahhwWlteuRDrsxmRq+8\ncDGMKcXyDHl3nWdIrWqJcDw=\n-----END PRIVATE KEY-----\n",
"client_email": "log-user#my-product-logging.iam.gserviceaccount.com",
"client_id": "101403085113430683797",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/log-user%40my-product-logging.iam.gserviceaccount.com",
}
func firebaseClient() *http.Client {
jwtConfig := jwt.Config{
// Email is the OAuth client identifier used when communicating with
// the configured OAuth provider.
Email: firebaseServiceAccount["client_email"],
// PrivateKey contains the contents of an RSA private key or the
// contents of a PEM file that contains a private key. The provided
// private key is used to sign JWT payloads.
// PEM containers with a passphrase are not supported.
// Use the following command to convert a PKCS 12 file into a PEM.
//
// $ openssl pkcs12 -in key.p12 -out key.pem -nodes
//
PrivateKey: []byte(firebaseServiceAccount["private_key"]),
// PrivateKeyID contains an optional hint indicating which key is being
// used.
PrivateKeyID: firebaseServiceAccount["private_key_id"],
// Subject is the optional user to impersonate.
Subject: "",
// Scopes optionally specifies a list of requested permission scopes.
Scopes: []string{
"https://www.googleapis.com/auth/devstorage.readonly",
},
// TokenURL is the endpoint required to complete the 2-legged JWT flow.
TokenURL: firebaseServiceAccount["token_uri"],
// Expires optionally specifies how long the token is valid for.
Expires: 0,
}
ctx := context.Background()
return jwtConfig.Client(ctx)
}
func firebaseFunc() {
authedClient := firebaseClient()
msg := map[string]string{
"hello": "there",
"every": "one",
}
data, err := json.Marshal(msg)
if err != nil {
log.Fatal("JSON Marshall Error: ", err)
continue
}
resp, err := authedClient.Post(firebasePostUrl, "application/json", bytes.NewReader(data))
if err != nil {
log.Fatal("Firebase Error: ", err)
continue
}
log.Print("Firebase Response Code: ", resp.StatusCode)
}
The problem is, I always get this error:
{
"error" : "invalid_scope",
"error_description" : "https://www.googleapis.com/auth/devstorage.readonly is not a valid audience string."
}
I assume it is a type that the error is invalid_scope, and the description says it is an invalid audience (I assume the JWT aud parameter).
What do I use as my scope to allow me to read/write the Firebase Database (using the default "auth != null" rules)?
Edit: I found the answer here finally:
https://www.googleapis.com/auth/firebase
However now it gives me a 403 response when actually doing the post.
{
"error" : "Permission denied."
}
Ugh, I found the undocumented answer here. Currently you also need this scope:
https://www.googleapis.com/auth/userinfo.email