How does wget on macOS handle certificates? - macos

Question

OS: macOS Big Sur
Applications: wget/openssl1.1 both via brew
I want to download some files from a webserver with wget via https, but I get the error message: Unable to locally verify the issuer's authority. However, I can connect to this server with curl which comes with macOS. Of course I could use --no-check-certificate, but I'd like to understand the background and do it the proper way.
As I understand it curl uses macOS' libressl and wget uses openssl as backend. For wget I've tried to export all system root certificates as a pem file from keychain and passed this pem file via --ca-certificate to wget. Still the same error.
Why can curl verify the certificate without doing anything while wget can not?
╰─$ brew info openssl
openssl#1.1: stable 1.1.1i (bottled) [keg-only]
Cryptography and SSL/TLS Toolkit
https://openssl.org/
/usr/local/Cellar/openssl#1.1/1.1.1i (8,067 files, 18.5MB)
Poured from bottle on 2021-01-28 at 07:11:37
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/openssl#1.1.rb
License: OpenSSL
==> Caveats
A CA file has been bootstrapped using certificates from the system
keychain. To add additional certificates, place .pem files in
/usr/local/etc/openssl#1.1/certs
and run
/usr/local/opt/openssl#1.1/bin/c_rehash
openssl#1.1 is keg-only, which means it was not symlinked into /usr/local,
because macOS provides LibreSSL.
Edit: This is what I've tried so far:
download via system curl -> no problem
installed (and reinstalled) curl and wget via brew (openssl as dependency) -> both can not verify the certificate, although it is a normal root CA certificate and OpenSSL via brew claims to export all these certificates to its own storage.
system curl with verbose option says it uses /etc/ssl/cert.pem for lookup, so I've tried to pass this file via --cacert resp. --ca-certificate to curl/wget -> same error.
I also tried to export the root CA explicitly in pem format and pass it on, still not working
Download the site's certificate manually via openssl: openssl s_client -showcerts -servername domain -connect domain, save the certificate as .pem and use it with wget -> works
I also checked that the certificate key for the root ca 'Digicert Global Root CA' is present in both /etc/ssl/cert.pem as well as in openssl's /usr/local/etc/openssl#1.1/cert.pem
I still don't understand why it won't work when OpenSSL via brew is supposed to export all the Root CAs to its own storage during install.

Answer 1

There is not enough information in your post to nail down the exact issue, but here are some suggestions that may help you forward:
Download the server's certificate bundle and locally analyse its verification using the openssl verify command.
Leverage the openssl s_client command to set up a TLS connection to the server directly and inspect the logged information, or dig deeper using its debug option(s). It has an undocumented debug option -security_debug_verbose that may help. If your OpenSSL is built with the the enable-ssl-trace option (which is not the case for the brew formula), you could use the -trace option for more output.
Run curl with -v or some --trace flags to get more insight in its verification behavior.
You could do all of this with both the OpenSSL and LibreSSL versions of openssl and curl and observe the differences. The openssl version of curl can be installed with brew.
You could also use another (non-OpenSSL) client to set up a connection and see if it fails with more information. For example gnutls-cli, which you can install via brew install gnutls, provides quite a bit of debug information.