We had an issue with our automated build machine yesterday. We are using a TFS Build server, and when it tried to automatically download NuGet packages, we got the infamous "The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel" error.
There are a lot of threads around the 'net regarding why this happens. That isn't my question. It can be fixed easily enough by changing your NuGet repository from
https://nuget.org/api/v2/
to
http://nuget.org/api/v2/
or
http://packages.nuget.org/v1/FeedService.svc/
What I'd like to know is why the repository is using SSL in the first place? I assume it is there for a reason, but I can't figure out what. There is no login that would require security. I can't think of any information being sent that would need to be secure. I just want to make sure that by using an unsecured connection (which works just fine) we aren't somehow compromising our build machine.
Can anyone explain what is gained from connecting to NuGet using a secured connection?
I can't think of any information being sent that would need to be
secure.
It is not necessarily because the information you exchange with nuget.org contains anything secret and thus needs to be secure. By using using SSL you will be certain that it actually is nuget.org you're talking with. Without SSL, somebody might in theory be feeding you bogus packages, and that might be a security problem.
As for the issue you're experiencing with "Could not establish trust relationship for the SSL/TLS secure channel", we've had a similar problem when we started using a new build server:
If you look at the SSL certificate presented by https://nuget.org/, the certification path is: GeoTrust Global CA > RapidSSL CA > *.nuget.org
GeoTrust Global CA was missing as a trusted CA on our new build server, so the problem was easily solved by adding them to the build servers list of trusted root CAs (using the MMC console with the Certificates snap-in).
Update:
On a later service, I've experienced the same SSL issue, and adding GeoTrust as a trusted CA alone didn't solve the problem. In addition, the server was also missing to root CA for https://go.microsoft.com/, which is Baltimore CyberTrust Root (go to https://microsoft.com, and you'll be able to view and download the certificate). Adding this to the servers list of trusted root CAs solved the issue.
Related
Recently I found myself working with Guzzle while making requests to another server to post and fetch some data, in some cases, tokens. But I was getting certificate invalid error and I even tried to get a new .pem certificate, but Guzzle was still not accepting and kept throwing that error. So finally, I did what the "Internet" said:
$guzzleClient = new Client([
'verify' => false
]);
Now although this solution works, I am not sure how insecure it can get. Do I need to worry? If yes, in what scenarios?
well this is a big problem if you are for example
having login system on the request you are sending using guzzle
having payment/checkout on the request
basically any sensitive data being passed to the other server
because when you pass data without SSL certificate then your requests might get caught by malicious programs like
BurbSuite / WireShark , cain and abel / EtterCap
as these programs are Sniffing programs and anyone can get a version from the internet as they are open sourced and every thing going without SSL can be intercepted by the hacker using the tools mentioned above and the hacker can look to the entire request in plaintext! so its highly recommended to use SSL connection when passing sensitive data
Worth Mentioning : now a days even SSL isn't very secure because hackers can remove it using SSLStrip tool but believe me SSL will make it much harder for them to get to your request because if they used it your website sometimes will make non-completed requests and it will notify the user that the network isn't secure this will make it very hard for the hacker to get the user's data,
TLS/SSL in common configurations is meant to give you three things:
confidentiality - no third party is able to read the messages sent and received,
integrity - no third party is able to modify the messages sent and received,
server authentication - you know who are you talking to.
What you do with setting verify to false is disabling the certificate verification. It immediately disables the server authentication feature and enables loosing confidentiality and integrity too when facing an active attacker that has access to your data stream.
How is that?
First of all TLS/SSL relieas on Public Key Infrastructure. Without going into too much details: you hold on your machine a set of certificates of so called Certification Authorities (CA) whom you trust. When you open a new communication to a service, you get the services certificates and in the process of verification you validate amongst other things if the certificate belongs to a CA you trust. If yes, then the communication may proceed. If no, then the communication channel is closed.
Attack patterns
Disabling certificate verification allows for Man-in-the-Middle (MitM) attacks than can easily be performed in your local network (e.g. via ARP poisoning attacks), in the local network of the service you are calling or in the network between. As we usually do not trust the network completely, we tend to verify.
Imagine me performing an attack on you. I have performed ARP poisoning, now I can see all your traffic. It's encrypted, isn't it? Well, not obviously. The TCP handshake and TLS handshake you believe you have performed with the target service - you have performed with me. I have presented you not the certificate of the target service, as I am unable to fake it, but my own. But you did not validate it to reject it. I have opened a connection between me and the target service on your behalf too so I can look into the decrypted traffic, modify if necessary and reply to you to make you believe everything is ok.
First of all - all your secrets are belong to me. Second of all - I am able to perform attacks on both you and the target service (which might have been secured by authentication mechanisms, but now is not).
How to fix this?
In XXI century there should be little reason to disable TLS verification anywhere. Configuring it to work properly might be a pain though, even more when you are doing it for the first time. From my experience the most common issues in the micro service world are:
the target certificate is self-signed,
you are missing a CA root certificate in your trust store,
the microservice does provide his certificate, but does not provide an intermediate CA certificate.
It's hard to guess what your issue is. We would need to dig deeper.
While the other answers points out some really good point about how important SSL/TLS is, your connection is still encrypted and the remote endpoint you're using has https:// in it as well. So you're not entirely disabling SSL when you set verify to false if I'm not mistken. It's just less secure since that you're not verifying the certificate of the remote server if they are signed by a Certificate Authority (CA) using the CA bundle.
Do you need to worry?
If this is something on your production, ideally you'd want things to be secure and configured correctly, so yes.
By not verifying the certificate, like Marek Puchalski mentioned, there's possibility of the server might not be the one you think it is and allows mitm (man in the middle) attack as well. More about mitm here, and peer verification here.
Why is it happening & how do you fix it?
Most common issue is misconfigured server, especially PHP configuration. You can fix your PHP configuration following this guide, where you'll be using adding the CA root certificates bundle to your configuration. Alternatively you can add this to Guzzle.
Another common issue is, the remote server is using a self-signed certificate. Even if you configured your CA bundle in your trustedstore, this certificate can't be trusted since it's not signed by a trusted CA. So the server needs to configure a SSL certificated signed by a CA. If that's not possible, you can manually trust this CA root, however this comes with some security concerns as well.
Hope this helped :)
I have been managing Let's Encrypt's SSL certificates for a domain.
Now I am moving to Amazon API gateway. I will be using the AWS Certificate Manager to generate HTTPS certificates for the root domain and a bunch of subdomains.
If I make the transfer, what happens to my current HTTPS certificate which is associated with my domain. If browsers suddenly start seeing a new HTTPS certificate for a domain, for which they had been getting a different HTTPS certificate until now, would this be a problem?
Also, once I make the shift, what do I do with my current (manually managed) Let's Encrypt certificate? Is there a way to permanently void it?
Szabolcs Dombi says
You can have multiple valid certificates for the same domain at the
same time. Moving from one certificate issuer to another should not
cause a problem.
Toby Osbourn says
SSL certificates don’t last forever, most of them need to be renewed
on a yearly cycle and occasionally you will want to change the type of
the SSL certificate mid-cycle.
Since you are replacing certificates, I suggest you to back up the ones you have.
Once you have backed up the old certificates, just overwrite the .crt and .key files with your new ones. Then, reload your web server so it knows to look at these new certificates, and you should be good to go.
If it's within your interest to know more about how to Generate SSL certificate using Amazon Certificate Manager (ACM), I suggest Barguzar, A. (July 2018). Building Serverless Python Web Services with Zappa. where one can read a good step by step guide. See an excerpt of it below:
ACM is a service that manages and creates SSL/TSL certificates for
AWS-based services and applications. An ACM certificate works with
multiple domain names and subdomains. You can also use ACM to create a
wildcard SSL.
ACM is strictly linked with AWS Certificate Manager Private
Certificate Authority (ACM PCA). ACM PCA is responsible for validating
the domain authority and issuing the certificate.
You can have multiple valid certificates for the same domain at the same time. Moving from one certificate issuer to another should not cause a problem.
This also means that if you create a new certificate the old one still can be used unless it already expired.
I have an application that receives items from a high-speed scanner device. As the items are received, they are written to disk using SQL Compact. The following digitally signed Microsoft DLLs are used:
sqlceca40.dll
sqlcecompact40.dll
sqlceer40EN.dll
sqlceme40.dll
sqlceoledb40.dll
sqlceqp40.dll
sqlcese40.dll
I recieved a performance complain from a customer, and traced the issue using Microsoft Procmon to a TCP Reconnect failure when attempting to contact the site for certificate validation when we make calls to methods in these dlls. At first, I could not recreate the issue locally. After talking to their infrastructure people and developers, I learned that they must use a proxy for internet connectivity. Some of the customer's users (in the test environment) had valid proxy settings, and they got good performance from our application. Naturally when they turned their proxy settings off, the validation could not be done and the performance issue arose.
I attempted to recreate the issue by setting our machine up with false proxy settings to a non-existent machine. On my initial attempt, I still got good performacne from our application, and no attempt was made to contact the internet for cert validation. After looking at the cert's validation chain, I noticed that it derived from the certificate "Microsoft Root Certificate Authority". I then exported and deleted that Cert, and was able to reproduce the issue as determined by a comparison of logs.
I did the following tests:
Test 1:
1. Opened the proxy settings, and enabled them pointing to a non-existent address.
2. Ran a test.
Results: No performance issue.
Test 2:
1. Exported the “Microsoft Root Certificate Authority” cert and moved it to the untrusted folder.
2. Ran a test.
Results: The performance issue occurred.
Test 3:
1. Deleted the “Microsoft Root Certificate Authority” cert.
2. Started a test.
Results: The performance issue began occuring.
3. While the test was in progress and device was hesitating I removed the false proxy settings.
Results: The performance issue disappeared and the application recovered.
Tentative Conclusions:
1. That I can simulate the no internet access condition by providing false proxy settings.
2. If the “Microsoft Root Certificate Authority” cert is installed properly, the .Net infrastructure does not need to access the network to verify the necessary cert.
3. If not, it will attempt to validate via the internet connection.
Nevertheless, when the customer checked the certificates in the "Trusted Roots Certificates" folder of mmc->certificates-local computer. The "Microsoft Root Certificate Authority" certificate does appear there, and it seems to be identical to mine. Yet for some reason the use of the dll's causes certificate validation to attempt to access the internet resulting in a performance issue.
In the customer's situation, eventually devices will be used in production with no internet access.
My question is, is there a setting (registry, or GPO) that might cause certificate validation to always attempt to use the internet, regardless of whether the root certificate of the validation chain is installed in the local computer?
Can a setting be enabled that causes a certificate validation to access the internet to check to see if the root certificate has been revoked, for example?
Please feel free to ask questions if you need more information.
This appears to occur for SQL Server Compact 4.0 on any system with an invalid proxy configuration, as a Certificate Revocation List check is run each time the engine is loaded (which happens on the first call to .Open()).
Solution: To avoid this delay, which probably affects any signed app on the system in question, you must fix the configuration or disable the check. The check can be disabled via UI or via registry settings, as described here: http://digital.ni.com/public.nsf/allkb/18E25101F0839C6286256F960061B282
For additionla issues see my blog post here: http://erikej.blogspot.com/2013/08/faq-why-is-opening-my-sql-server.html
I have been working with web services connecting to URLs provided by different clients and so far it has all been done using one-way authentication. Now I'm asked to enable 2-way (mutual) authentication for one of the clients. I did a lot of research and reading but still confused about a lot of things.
I could test successfully on my local machine following instructions from various different articles. But the problem is now to deploy it in production.
Here's what I did for testing: I created a test Web service Host and assigned it a self-signed certificate and created a client to test this. After this I created a client certificate using makecert and verified that this is installed via MMC. I then modified my Host app to only allow clients with certificate and tested from client to see the connection refused due to not providing the client certificate. Then I modified the bindings in the client application to include the certificate name and I was able to connect to the Host successfully. So this completes local hosting.
Now the real problem. The tech team is going to create a certificate in "cert store" on the server. And I need to test again to make sure everything works as expected. We have a few different developers who all want to test on their machines on their local code. Can we all use the same certificate somehow? I don't think we would be allowed to import the certificate but what suggestions could I give them so all of us can use the same certificate?
I'm also confused about issues like difference between windows certificate and IIS certificate. What advantages would the IIS certificate provide?
Thanks for help!
Edit: Could one of the differences between installing on IIS be so that the hosted sites be accessed via SSL connection? This would mean we don't really need to install on IIS if it's just a client certificate. Is this correct?
I'm running mitmdump (from mitmproxy) on my Macbook Pro, and I'm connecting to the proxy through my Windows desktop PC.
However, Chrome (running on the PC) refuses to connect to so many sites because of the invalid certificates which mitmproxy provides.
Chrome throws the error: ERR::NET_CERT_AUTHORITY_INVALID
Here's what mitmdump shows:
But why? What's wrong with mitmproxy's certificates, why can't it just send back google's as if nothing happened?
I'd like to know how I can fix this and make (force) my desktop PC to connect to any website through my Macbook's mitmproxy.
Answering this question for people who may find this important now. To get the proxy working, you have to add the certificate as trusted in your browser.
For windows follow this: https://www.nullalo.com/en/chrome-how-to-install-self-signed-ssl-certificates/2/
For linux follow this: https://dev.to/suntong/using-squid-to-proxy-ssl-sites-nj3
For Mac-os follow this: https://www.andrewconnell.com/blog/updated-creating-and-trusting-self-signed-certs-on-macos-and-chrome/#add-certificate-to-trusted-root-authority
There are some additional details in the above links; tldr; import the certificate in your chrome://settings url and add the certificate as trusted. That shall do.
This will make your browser trust your self-signed certificate(mitm auto generated certificates too.)
The default certificates of mitmproxy is at ~/.mitmproxy/ directory.
Per the Getting Started page of the docs you add the CA by going to http://mitm.it while mitmproxy is running and selecting the operating system that you are using. This should solve your problem and will allow https sites to work with mitmproxy.
This is the expected behavior.
mitmproxy performes a Man-In-The-Middle attack to https connections by providing on-the-fly generated fake certificates to the client while it keeps communicating to the server over fully encrypted connection using the real certificates.
This way the communication between client and proxy can be decrypted. But the client has to actively approve using those fake certificates.
If that wasn't the case then SSL would be broken - which it isn't.
The whole story is very well explained here:
http://docs.mitmproxy.org/en/stable/howmitmproxy.html