Disable SNI Extension for Ruby net/http - Using IP address with SSL/TLS - ruby

I have a Ruby polling script that runs on a set of servers in an IP range. I very strongly prefer to do this polling by IP address, not by hostname, because:
1) I have defined IP address ranges to poll, and hostnames are arbitrary/change a lot
2) Because they change a lot, most of the hostnames do not have a reverse DNS lookup, so I can't engineer a list of hostnames from IPs
Before our web servers had no problem with this polling, but on a new server that does not accept SSLv3 communication, this is the error I get when I run my poll:
/home/dashboard/.rvm/rubies/ruby-2.1.6/lib/ruby/2.1.0/net/http.rb:923:in `connect': SSL_connect returned=1 errno=0 state=unknown state: tlsv1 unrecognized name (OpenSSL::SSL::SSLError)
On the server side, this is the error:
nss_engine_init.c(1802): start function ownSSLSNISocketConfig for SNI
nss_engine_init.c(1827): Search [val = 172.16.99.18] failed, unrecognized name
When I run the poll with hostname, everything works fine.
Here is the crux of the HTTP Client code in Ruby:
def init_http(url)
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
http.read_timeout = 10
http.use_ssl = true
#http.ssl_version = 'TLSv1'
return [http, uri]
end
As you can tell, I've been playing around with TLS and SSL version, because I figured that might be the issue. My next thought (that Google only has evidence of for Java) is, "How easy is it to just disable the SNI extension on my client?" The more general question is, "Can I keep using IP addresses with Ruby net/http while taking advantage of newer, more secure communication protocols?"

... tlsv1 unrecognized name (OpenSSL::SSL::SSLError)
This is in most cases not a problem you can solve by disabling SNI on the client side. SNI is required when you have multiple certificates on the same IP address and if you just connect by IP address and don't send the requested hostname (i.e. disabling SNI) the server will not know which certificate it should provide - which then results in the above error.
I very strongly prefer to do this polling by IP address, not by hostname, ...
If you have to deal with server which require SNI then you have to use SNI and you have to use SNI with the proper hostname, which is not necessarily the same name as you get from reverse lookup.

The easiest way to solve this is to add the patch that #steffenullrich mentioned for bug 10613.
All I did was look at the diff, and edit the file myself, but you can use the Linux patch tool if you're familiar with it.
For those who are unsure of where /net/http.rb is located, it is in the same location as the rest of your Ruby sources. For example, mine was here:
/home/myuser/.rvm/rubies/ruby-2.1.6/lib/net/http.rb
Once you patch the file, set the .disable_sni property of your HTTP object to true, and SNI will not be required, allowing the use of IP addresses in mixed-TLS communication.

Related

Hostname does not match the server certificate - using old expired certificate

I am trying to connect via TCP socket to an old, unmaintained game server. When I try to do so using OpenSSL and Socketry, I get following error.
hostname "xxx" does not match the server certificate
This is how my code works.
ssl_context = OpenSSL::SSL::SSLContext.new
ssl_context.verify_mode = OpenSSL::SSL::VERIFY_NONE
ssl_context.ciphers = "AES128-SHA"
ssl_context.cert = OpenSSL::X509::Certificate.new(File.open("certificate.crt"))
ssl_context.key = OpenSSL::PKey::RSA.new(File.open("certificate.key"))
socket = Socketry::SSL::Socket.new(ssl_context: ssl_context)
socket.connect(address, port)
I've had to force cipher to AES128-SHA, as otherwise I would get a different error.
SSL_connect returned=1 errno=0 state=error: dh key too small (OpenSSL::SSL::SSLError)
How can I bypass these errors and just force connection to said server? The certificate itself has expired 10 years ago.
Socketry doesn't include much documentation about this, so you need to turn to the source. Depending on exactly what you're trying to do, you might try one of the following:
Passing verify_hostname: false to Socket#connect.
Rescuing Socketry::SSL::HostnameError yourself.
Rescuing Socketry::SSL::CertificateVerifyError.
At least one of these ought to address your specific issue. If not, the source offers other places you might monkey-patch or refine for your use case.

Using winhttp for client with non default SSL server name indication (sni)

I'm using winhttp in order to establish https connection on port 443 with my remote. However, the server running this service also contains more services on the same https port (443), so it uses SNI in order to resolve the requested session.
However, the server doesn't expect to get the hostname as SNI, since it uses single URL for all services. instead, the SNI address is chosen not according to the URL but according to some other string notation (i.e. service_api or service_web_if ...)
In my client connection flow, I set the URL in method WinHttpConnect which also set the SNI accordingly, and the actual SSL/TLS handshake is made when calling WinHttpSentRequest.
I wonder how can I change the SNI value from the default URL value after calling WinHttpConnect.
So far, while investigating possible solutions, I've learned about HTTP_SERVICE_CONFIG_SSL_SNI_KEY structure which is set by method HttpSetServiceConfiguration along with the matching certificate for this SNI, but this seems to be related to the server side configuration. Besides that, I haven't found any references for such action unfortunately.
Perhaps anybody ever used non-default SNI using winhttp API and can tell me how to do so ? is the only option to do so is doing the SSL handshake using some lower level API such as schannel, and than switching back to winhttp ?
if it's not possible, perhaps there's an option to use extended hostname with directory tree in order to get multiple sni on a single url...

http2: The 421 Misdirected Request Status Code example

I'm reading the spec and trying to understand exactly when 421 might be returned. An example is given but I don't completely understand it.
Background
The spec establishes two conditions that allow for connection reuse:
For TCP connections without TLS, this depends on the host having resolved to the same IP address.
and
For https resources, connection reuse additionally depends on having a
certificate that is valid for the host in the URI.
If the certificate used in the connection has multiple subjectAltName or any of the subjectAltName is a wildcard, then the connection can be reused for any request that has a hostname that is in the list of subjectAltNames or matches any of the wildcards.
Specific Example In the spec
In some deployments, reusing a connection for multiple origins can
result in requests being directed to the wrong origin server. For
example, TLS termination might be performed by a middlebox that uses
the TLS Server Name Indication (SNI) [TLS-EXT] extension to select an
origin server. This means that it is possible for clients to send
confidential information to servers that might not be the intended
target for the request, even though the server is otherwise
authoritative.
Please explain where my understanding of this example is wrong:
An https connection is established to a middlebox with a request that has domain x.com. The middlebox has IP address 1.2.3.4 and x.com resolves to that address. Using SNI, the TLS handshake has x.com and the middlebox returns a certificate valid for that domain. All messages on this connection go from the client to the middlebox or from middlebox to client. Applicaiton level messages from client to middlebox are forwarded by middlebox to an origin on a different connection. Messages from origin to middlebox are forwarded to the client. If the connection is to be reused, meeting the two conditions discussed above is not enough. Specifically, for a request with domain y.com: if y.com resolves to 1.2.3.4 and the middlebox has a certificate valid for y.com, there can still be a problem. Because the original connection did its TLS handshake using x.com and because handshakes are only done at the beginning of new connections, there is no way establishing an https connection that would get the certificate for y.com. So the client incorrectly sends a request on the same connection to y.com. The middlebox rejects the request because the certificate associated with the connection is valid for x.com - not y.com. (The x.com certificate is only valid for x.com and the y.com certificate is only valid for y.com).
None of your examples will trigger a 421 as far as I can see.
Yes you are correct that a connection needs both the IP address and the SAN field in the certificate to be valid - without those a connection should not be reused.
So what would trigger a 421? As far as I can tell it will be mostly due to different SSL/TLS setups.
For example:
Assume website A (siteA.example.com) and website B (www.example.com) are both on same IP address. Assume website A has a wildcard cert for *.example.com and website B has a specific one. Could be a few reasons for this: for example it serves an EV cert for the main website which can't be a wildcard cert.
So cert A covers website A and website B. As does the IP address. So if you are connected to website siteA.example.com, and then try to connect to www.example.com then technically, by HTTP/2 standards, you could reuse the connection. But we wouldn't want that to happen, as we want to use our EV cert. So the server should reject with a 421. Now in this example the webserver is able to distinguish the correct host and has a valid cert for that host so could, in theory, serve the correct content under the wildcard cert, instead of sending a 421 - but since that wildcard cert is not defined for that virtualhost it should not do this.
Other examples include if you have different ciphers set up on different hosts. For example site A has super lax HTTPS config, because it's not really secure content and they want to reach even legacy browsers, but site B has super secure config and only accepts the latest TLS version and strong ciphers. Here you obviously wouldn't want them to reuse the same connection details. See here for a real would example of this.
Also this is only an issue for certain browsers, depending on how they decide to connection share. This page shows how different each of them do this (at least at the time of this blog post not not aware of anything changing since then): https://daniel.haxx.se/blog/2016/08/18/http2-connection-coalescing/
Also note that some bugs will exist with this (for example: https://bugs.chromium.org/p/chromium/issues/detail?id=546991). Best advice is: if you do not want connection sharing to happen, have a different IP address and/or ensure no overlaps in certificates.

Chrome show : HTTP hostname and TLS SNI hostname mismatch

I have problem, when I started my chrome, and try visit https page, i see this error:
Error 1013 Ray ID: 2cb7c5989ac13dd8 • 2016-08-01 08:03:08 UTC
HTTP hostname and TLS SNI hostname mismatch
You've requested an IP address that is part of the CloudFlare network. The request's Host header does not match the request's TLS SNI Host header.
What is it? where place cloudfare and how can I this remove from my browser?
SNI stands for Server Name Indication, essentially it allows you to serve more than one SSL certificate from more than one IP Address. Most modern browsers submit this as part of a web request. CloudFlare Free Universal SSL makes use of SNI.
Similarly, the Host header allows a server to know which hostname it should serve. The Host header fundamentally allows virtualhosting, serving more than one website per IP Address.
So these two clearly need to match, the SNI hostname and the hostname in the Host header must be identical. When this fails to occur CloudFlare will present a Error 1013 HTTP hostname and TLS SNI hostname mismatch.
There may be a proxy in your local network that is stripping out SNI headers or deforming them thus causing them to mismatch. This can happen as a result of a firewall or intermediary web server.
There is a tool here to check if your SNI header and Host header match. Simply visit the linked site and check the "SNI information:" field states "cc.dcsec.uni-hannover.de".
If you require support for browsers which do not support SNI, CloudFlare's Pro, Business or Enterprise support have legacy support for non-SNI browsers; but it would make sense to contact your network administrator to see why SNI requests are being malformed.
The tags on this question suggest you are using Firefox on Windows 10 which should support SNI; therefore you should reach out to your network team to see why they are malforming SNI requests.

How to work with HTTPS for multiple domains and and sub-domains on localhost?

I am using
Apache
Ruby and Ruby on Rails 3
Mac Os running "Snow Leopard"
and I would like to use HTTPS on localhost for my domains and sub-domains.
I have already set everything (I think correctly):
I generated a wildcard certificate for my domains and sub-domains (example: *.sitename.com)
I have set base-named virtualhosts in the http.conf file listening on port :433 and :80
My browser accept certificates also if it alerts me that those aren't safe and I can have access to pages using HTTPS
From the official Apache guide I read that it is not possible to do that using name-based virtualhost, but I also read someone that made that in some way (what?! I don't understand...).
So, is it possible or not to use HTTPS in localhost for multiple domains and sub-domains? If so, what I must "to do"\"to check" for working with that?
UPDATE for #sarnold
typhoeus appears to use libcurl, and
libcurl appears to support SNI -- is
your version of libcurl new enough to
support SNI? Does typhoeous know how
to enable it? (Do clients of libcurl
need to "enable" it SNI themselves?)
I think so because I can access all sub_domains over HTTPS and libcurl should be updated:
curl -V--version
curl 7.21.2 (x86_64-apple-darwin10.5.0) libcurl/7.21.2 OpenSSL/1.0.0c zlib/1.2.5 libidn/1.19
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: IDN IPv6 Largefile NTLM SSL libz
# Typhoeus request
Typhoeus::Request.get("https://<sub_domain_name>.<domain_name>.com/")
How can I check if "Do clients of libcurl need to "enable" it SNI themselves?"?
The techniques for doing name-based virtual servers with SSL/TLS aren't great choices, but the Server Name Indication extension allows browsers to request a specific site by name, allowing different certificates to be used with different sites. Not all browsers support SNI yet.
Though one might ask what value there is is in having multiple certificates if they are all served out of the same process with the same privileges, anything to improve the user's TLS experience has to be worth the hassle. :)

Resources