I've been struggling recently with using the standalone DartVM and SSL as a client. I'm of the understanding that Dart uses Mozilla NSS to manage the certificates. What I'm having a problem wit, is that on Windows, for example, there exists no binaries that I can find (other than third parties compiling the Mozilla source and uploading to mega or similar, which is pretty alarming if you ask me) released for the Windows platform. Compiling this C++ code is not a trivial task. I've not the resources to do so on my own under the Windows platform. This is why I write Dart (or other high level languages) in the first place.
Despite that, the error message I get when attempting to connect securely and being presented with a self-signed (or rather more technically correct, untrusted authority) certificate, is that the OS itself doesn't trust the certificate. On Windows, this is not the case. The certificate in question I'm using is a CA root certificate of my generating, with proper authority/signing chain, installed into Windows trusted roots manually. Both Chrome and Internet Explorer (of which use the Windows underlying certificate store) trust my certificate(s) without any warnings after having done this. So if the DartVM is not using the "OS" to validate a certificate upon handshake, then that message is very uninformative/misleading.
What can be done to overcome this outside of compiling NSS and trying to figure out just how to import my certificates by way of over-complicated and under-documented steps? Is there not a parameter that one could specify when initiating a secure connection to ignore SSL errors of this nature?
My web server forces the use of HTTPS so dropping back to plain HTTP would
not be an option for me. I also don't want to trust and much less want to pay a third party for my certificates of which are pretty much only used internally, which is why I generated a wildcard certificate under my own root CA in the first place. Paying for a wildcard certificate, for multiple domains, that aren't always necessarily exposed to the public or meant for public use is a bit astronomically priced and completely out of the question.
Related
How do browsers like Chrome check SSL Certificate?
IS there any online databases or websites that they use?
What steps are taken by browsers to validate a SSL certificate?
Am I able to do it manually without using browser?
How do browsers like Chrome check SSL Certificate?
The certificate and chain is send by the server during the SSL handshake. The browser will create the trust chain based on the certificate issuer, provided chain certificates and the local root certificates. It will check expiration and purpose of the certificate and also check the subject alternative names (and maybe subject too) to make sure that the certificate is actually issued for the domain in the URL. It might also do some checks for certificate revocation.
For details see SSL Certificate framework 101: How does the browser actually verify the validity of a given server certificate? and How Do Browsers Handle Revoked SSL/TLS Certificates?.
I there any online database or websites that they use?
Not really. The necessary trust store is local. They might check revocation though against some online resource. See Is Certificate validation done completely local?.
Am I able to do it manually without using browser?
Sure, what the browser does could in theory be replicated manually. You can for example access the site and get the leaf and intermediate certificates with openssl s_client -showcerts .... You can then use openssl verify to verify the certificate chain, see also Verify a certificate chain using openssl verify. Then you need to look at the leaf certificate with openssl x509 -text ... to check purpose, expiration and the subject. Revocation is tricky but could be done with the help of openssl crl and openssl ocsp, although this does not really reflect what browsers do.
The official algorithm for validating any SSL/TLS certificate is defined by PKIX as modified by OCSP. For TLS nowadays the OCSP token is often transported by 'stapling' in the TLS handshake instead of by a separate connection, which requires several other RFCs, but that only affects transport, not the actual validation by the relier. For HTTPS specifically, the client must also check server identity aka 'hostname' as defined by rfc2818.
In practice, browsers may vary some. Chrome mostly uses a google-determined scheme to 'push' revocation data they select, but this has changed from time to time. Firefox, last I heard, used their own 'one-CRL' scheme. Also, although the standard and traditional practice was to check hostname against SAN if present and otherwise fall back to Subject.CN, Chrome since a few years ago requires SAN and never uses CN; you can find dozens of Qs on several stacks about "my selfsigned or otherwise DIY cert not from a real CA stopped working on Chrome".
If by 'do it manually' you really mean manually, that will be a lot of work. If you mean with tools other than a browser offline, somewhat easier; OpenSSL (if installed) can do most of this, although you need more options than shown in Steffen's link to get it right.
If you mean with tools other than a browser online, absolutely. The WWW has become extremely popular in recent decades, and there are zillions of programs and libraries for accessing it, nearly all of them including HTTPS (although two decades ago that was less common), which includes validating the certificate -- at least by default; many have options to disable, bypass, or override validation. There are standalone tools like curl and wget -- or openssl s_client can do the SSL/TLS and certificate part without doing HTTP on top. There are innumerable libraries and middlewares like ssl in python (or requests using it), tls in nodejs, (older) HttpsURLConnection and (newer) java.net.http in Java as well as third-parties like Apache HttpComponents; I'm sure there are counterparts for perl and dotnet although I'm not familiar with them. As well as powershell, which is fuzzy on the program/library distinction.
Recently I found myself working with Guzzle while making requests to another server to post and fetch some data, in some cases, tokens. But I was getting certificate invalid error and I even tried to get a new .pem certificate, but Guzzle was still not accepting and kept throwing that error. So finally, I did what the "Internet" said:
$guzzleClient = new Client([
'verify' => false
]);
Now although this solution works, I am not sure how insecure it can get. Do I need to worry? If yes, in what scenarios?
well this is a big problem if you are for example
having login system on the request you are sending using guzzle
having payment/checkout on the request
basically any sensitive data being passed to the other server
because when you pass data without SSL certificate then your requests might get caught by malicious programs like
BurbSuite / WireShark , cain and abel / EtterCap
as these programs are Sniffing programs and anyone can get a version from the internet as they are open sourced and every thing going without SSL can be intercepted by the hacker using the tools mentioned above and the hacker can look to the entire request in plaintext! so its highly recommended to use SSL connection when passing sensitive data
Worth Mentioning : now a days even SSL isn't very secure because hackers can remove it using SSLStrip tool but believe me SSL will make it much harder for them to get to your request because if they used it your website sometimes will make non-completed requests and it will notify the user that the network isn't secure this will make it very hard for the hacker to get the user's data,
TLS/SSL in common configurations is meant to give you three things:
confidentiality - no third party is able to read the messages sent and received,
integrity - no third party is able to modify the messages sent and received,
server authentication - you know who are you talking to.
What you do with setting verify to false is disabling the certificate verification. It immediately disables the server authentication feature and enables loosing confidentiality and integrity too when facing an active attacker that has access to your data stream.
How is that?
First of all TLS/SSL relieas on Public Key Infrastructure. Without going into too much details: you hold on your machine a set of certificates of so called Certification Authorities (CA) whom you trust. When you open a new communication to a service, you get the services certificates and in the process of verification you validate amongst other things if the certificate belongs to a CA you trust. If yes, then the communication may proceed. If no, then the communication channel is closed.
Attack patterns
Disabling certificate verification allows for Man-in-the-Middle (MitM) attacks than can easily be performed in your local network (e.g. via ARP poisoning attacks), in the local network of the service you are calling or in the network between. As we usually do not trust the network completely, we tend to verify.
Imagine me performing an attack on you. I have performed ARP poisoning, now I can see all your traffic. It's encrypted, isn't it? Well, not obviously. The TCP handshake and TLS handshake you believe you have performed with the target service - you have performed with me. I have presented you not the certificate of the target service, as I am unable to fake it, but my own. But you did not validate it to reject it. I have opened a connection between me and the target service on your behalf too so I can look into the decrypted traffic, modify if necessary and reply to you to make you believe everything is ok.
First of all - all your secrets are belong to me. Second of all - I am able to perform attacks on both you and the target service (which might have been secured by authentication mechanisms, but now is not).
How to fix this?
In XXI century there should be little reason to disable TLS verification anywhere. Configuring it to work properly might be a pain though, even more when you are doing it for the first time. From my experience the most common issues in the micro service world are:
the target certificate is self-signed,
you are missing a CA root certificate in your trust store,
the microservice does provide his certificate, but does not provide an intermediate CA certificate.
It's hard to guess what your issue is. We would need to dig deeper.
While the other answers points out some really good point about how important SSL/TLS is, your connection is still encrypted and the remote endpoint you're using has https:// in it as well. So you're not entirely disabling SSL when you set verify to false if I'm not mistken. It's just less secure since that you're not verifying the certificate of the remote server if they are signed by a Certificate Authority (CA) using the CA bundle.
Do you need to worry?
If this is something on your production, ideally you'd want things to be secure and configured correctly, so yes.
By not verifying the certificate, like Marek Puchalski mentioned, there's possibility of the server might not be the one you think it is and allows mitm (man in the middle) attack as well. More about mitm here, and peer verification here.
Why is it happening & how do you fix it?
Most common issue is misconfigured server, especially PHP configuration. You can fix your PHP configuration following this guide, where you'll be using adding the CA root certificates bundle to your configuration. Alternatively you can add this to Guzzle.
Another common issue is, the remote server is using a self-signed certificate. Even if you configured your CA bundle in your trustedstore, this certificate can't be trusted since it's not signed by a trusted CA. So the server needs to configure a SSL certificated signed by a CA. If that's not possible, you can manually trust this CA root, however this comes with some security concerns as well.
Hope this helped :)
I want my site to be secure using HTTPS protocols. I managed to make a self-signed key to be trustedCertEntry as I made my own CA certificate, with different CN, which I used to sign my own private certificate.
It works smooth testing it with openssl with something like:
openssl s_client -connect www.mydomain.com:80 -tls1 -state
Thus, browser doesn't report a certificate self-signed error, as it sees a different CA.
But I get a SEC_ERROR_UNKNOWN_ISSUER error. Still it seems logical to me as nobody knows me as a CA. It is supposed to work if user adds exception for me.
I thought this trick was acceptable and it was like many https compliant sites were working, as you may visit a unknown site and you want to encrypt communications from 3rd party watchers but trust that page.
After trying to get a clear response for it, beyond coding that I will find resources, my question is:
If I want to have a site, for which the users don't have to add an exception in the first visit, do I have to get a certificate from a "world-known" CA? Or am I missing a solution for self-signing my certificate with my own CA certificate?
Technically speaking, the answer is: Yes, you will have to get a certificate from a CA that is trusted by your users' browsers via a chain of intermediary CA's that ends at an inherently trusted root CA. The accepted answer to this question explains how it works: SSL Certificate framework 101: How does the browser actually verify the validity of a given server certificate?
Having said that, if your "only" concern is to provide encrypted connections, you might be able to leverage the Let's Encrypt CA, which provides free certificates for that purpose. Those certificates will be only domain-validated, which provides a weaker kind of assurance of identity than, for example, an Extended Validation Certificate.
Depending on the browser used, there will be minimal difference in user experience between DV and EV certificates. For Safari, the user will see a grey padlock in the address bar for the lower assurance DV-backed sites, like this:
and a green padlock when higher identity assurance is provided, like this:
Whether the former is good enough for you (or your customers) depends on your situation.
In case you want to understand what "inherently trusted" actually means for web browsers, see this blog post: Who your browser trusts, and how to control it.
I'm writing a utility Mac OS X app that basically acts as a web server accepting incoming HTTP requests (think of it as a mock REST API server). I want to be able to support HTTPS, but ideally I'd like to remove the requirement for my users to have to purchase their own SSL certificates.
I've been thinking a little on how I might achieve this. Let's say I register a domain called myapp.com. I then purchase an SSL cert for myserver.myapp.com that is signed by a registered CA. I ship my app with those SSL cert details embedded within it. All my users have to do is update their /etc/hosts file to point myserver.myapp.com to whatever IP address my app is installed and running on.
In fact, by far, the most common scenario would be my app running on the same machine as the client, so I'm considering updating the main DNS entry for myserver.myapp.com to point to 127.0.0.1, and most users wouldn't have to change anything.
So, that's the basic theory. What have I missed that would make this an unworkable plan? A couple of things that have crossed my mind:
I could use a self-signed cert. However, many SSL clients barf (or throw up warnings) if the cert doesn't have a valid CA chain. I'm happy to pay the money for a real cert to alleviate this inconvenience for my users.
I would be embedding the private key for my SSL cert into my app. In theory, someone could extract that and use it to impersonate my app. I guess my reaction is "so what?" My app is a small productivity app, it isn't an e-commerce site. It doesn't collect sensitive info. It literally just simulates web server responses so devs can test their apps.
Any advice/feedback would be greatly appreciated. Thanks.
This won't work - but for nontechnical reasons.
Distributing an SSL certificate to your users along with its associated key will violate the issuance terms of your SSL certificate provider, and they will revoke the certificate when they discover what you have done. (This happened, for example, when Pivotal tried to offer SSL service for developers through sslip.io.) This will, of course, cause your application to stop working.
If your users have administrative access to their machines, they can create and trust their own self-signed CA using Keychain Access. Once they have done so, they could create a certificate (again, using Keychain Access) and insert that into your application to enable SSL.
As said in the other answer you can't ship the same certificate for everybody. What you could do is generate different for everybody:
The application ask them the domain name they want to use (a domain they must own, like myapp.example.com)
The application use the ACME protocol to get automatically a trusted certificate from let's encrypt
Note: you can provide them subdomains of a domain you control (like [clientid].yourappname.yourdomain.com) ONLY of you can register yourappname.yourdomain.com in the public suffix list (because let's encrypt have rate limits)
Is it different from any other certificate I can generate via makecert or buy from some authority?
As mentioned by Mile L and Boot to the Head the Extended Key Usage is what determines the purpose that the key can be used for.
Most commercial certificate authorities (Verisign et al) issue certificates for single purposes, or for as few as possible.
They use this narrowing of the puropse to carve out different markets for the certificates and then price them accordingly.
You see them selling different Object Signing certs for Windows Assemblies / Java / Office / Adobe Air etc when (in most cases) the resulting certificate is the same.
For example the Comodo codesigning cert issued here can sign Java applets, WebStart applications, Firefox extensions and even Windows assemblies.
The certificate that's used to sign software is the same certificate that would be used to sign any document. What's different about signing software is where the signature finally resides. In a typical document signing, the signature just gets appended to the original document. You cannot append a signature to most types of software for obvious reasons (some interpreted languages would allow this, but I don't know if it's done in practice).
The solutions to the signature problem vary based on the execution environment. For an executable binary, the signature is often stored in a separate file. In Java you can have a signature embedded in an executable JAR file.
Microsoft has a good reference for an introduction to the signing process.
It depends on what you are doing with it. If you want the certificate to be accepted by a browser in an SSL communication, then it must have a root certificate installed in the browser. The certificates generated by authorities already have their root certs installed in browsers.
If you are using the cert just to sign an assembly, then you don't need it. It depends on who is checking the cert and whether they care if the root is a known authority.
More here:
http://en.wikipedia.org/wiki/Root_certificate
To my knowledge, certificates have a "key usage" attribute that describes what uses the cert is intended for: SSL server, code signing, e-mail signing, etc. So I think it's up to the OS, or web browser, or e-mail client, to check these bits.
When a cert is called into action, the role it purports to perform is as important as identification. It's not just about identity, but also about role authorization. An email protection cert should not be able to perform server authentication. Security concerns dictate a necessary restriction in the power given through a single certificate. The underlying API should enforce the correct usage, be it through the OS or an abstraction such as the .NET Framework.
There are different certificate types because there are very different roles in authentication and authorization that would need them. Allowing different certificate types and hierarchies allow for a model of certificate chains, as found in the "Certification Path" on a certificate. A Server Authentication cert will need to have a top-level CA cert somewhere in the trusted root certificates... or be a part of a family tree of certs which ultimately does. 3rd party Certificate Authorities, I'm sure, price them on a scale of functionality and trust.
Boot To The Head is right... there is an Enhanced Key Usage attribute which provides a description of what the cert claims the role to be (e.g. Server Authentication; or in the case of my CA's cert: Digital Signature, Certificate Signing, Off-line CRL Signing, CRL Signing). Look at the details in a certificate's properties and you'll find it.
I'd also add that a .NET assembly has to be strongly named (which requires it to be signed) in order to be added to the GAC.
There are different types of certs... from the CA that is bundled in Win 2003 server, you can request:
Client authentication
Email protection
Server authentication
Code signing
Time stamp signing
IPSec
Other