I'm reading the spec and trying to understand exactly when 421 might be returned. An example is given but I don't completely understand it.
Background
The spec establishes two conditions that allow for connection reuse:
For TCP connections without TLS, this depends on the host having resolved to the same IP address.
and
For https resources, connection reuse additionally depends on having a
certificate that is valid for the host in the URI.
If the certificate used in the connection has multiple subjectAltName or any of the subjectAltName is a wildcard, then the connection can be reused for any request that has a hostname that is in the list of subjectAltNames or matches any of the wildcards.
Specific Example In the spec
In some deployments, reusing a connection for multiple origins can
result in requests being directed to the wrong origin server. For
example, TLS termination might be performed by a middlebox that uses
the TLS Server Name Indication (SNI) [TLS-EXT] extension to select an
origin server. This means that it is possible for clients to send
confidential information to servers that might not be the intended
target for the request, even though the server is otherwise
authoritative.
Please explain where my understanding of this example is wrong:
An https connection is established to a middlebox with a request that has domain x.com. The middlebox has IP address 1.2.3.4 and x.com resolves to that address. Using SNI, the TLS handshake has x.com and the middlebox returns a certificate valid for that domain. All messages on this connection go from the client to the middlebox or from middlebox to client. Applicaiton level messages from client to middlebox are forwarded by middlebox to an origin on a different connection. Messages from origin to middlebox are forwarded to the client. If the connection is to be reused, meeting the two conditions discussed above is not enough. Specifically, for a request with domain y.com: if y.com resolves to 1.2.3.4 and the middlebox has a certificate valid for y.com, there can still be a problem. Because the original connection did its TLS handshake using x.com and because handshakes are only done at the beginning of new connections, there is no way establishing an https connection that would get the certificate for y.com. So the client incorrectly sends a request on the same connection to y.com. The middlebox rejects the request because the certificate associated with the connection is valid for x.com - not y.com. (The x.com certificate is only valid for x.com and the y.com certificate is only valid for y.com).
None of your examples will trigger a 421 as far as I can see.
Yes you are correct that a connection needs both the IP address and the SAN field in the certificate to be valid - without those a connection should not be reused.
So what would trigger a 421? As far as I can tell it will be mostly due to different SSL/TLS setups.
For example:
Assume website A (siteA.example.com) and website B (www.example.com) are both on same IP address. Assume website A has a wildcard cert for *.example.com and website B has a specific one. Could be a few reasons for this: for example it serves an EV cert for the main website which can't be a wildcard cert.
So cert A covers website A and website B. As does the IP address. So if you are connected to website siteA.example.com, and then try to connect to www.example.com then technically, by HTTP/2 standards, you could reuse the connection. But we wouldn't want that to happen, as we want to use our EV cert. So the server should reject with a 421. Now in this example the webserver is able to distinguish the correct host and has a valid cert for that host so could, in theory, serve the correct content under the wildcard cert, instead of sending a 421 - but since that wildcard cert is not defined for that virtualhost it should not do this.
Other examples include if you have different ciphers set up on different hosts. For example site A has super lax HTTPS config, because it's not really secure content and they want to reach even legacy browsers, but site B has super secure config and only accepts the latest TLS version and strong ciphers. Here you obviously wouldn't want them to reuse the same connection details. See here for a real would example of this.
Also this is only an issue for certain browsers, depending on how they decide to connection share. This page shows how different each of them do this (at least at the time of this blog post not not aware of anything changing since then): https://daniel.haxx.se/blog/2016/08/18/http2-connection-coalescing/
Also note that some bugs will exist with this (for example: https://bugs.chromium.org/p/chromium/issues/detail?id=546991). Best advice is: if you do not want connection sharing to happen, have a different IP address and/or ensure no overlaps in certificates.
Related
I'm using winhttp in order to establish https connection on port 443 with my remote. However, the server running this service also contains more services on the same https port (443), so it uses SNI in order to resolve the requested session.
However, the server doesn't expect to get the hostname as SNI, since it uses single URL for all services. instead, the SNI address is chosen not according to the URL but according to some other string notation (i.e. service_api or service_web_if ...)
In my client connection flow, I set the URL in method WinHttpConnect which also set the SNI accordingly, and the actual SSL/TLS handshake is made when calling WinHttpSentRequest.
I wonder how can I change the SNI value from the default URL value after calling WinHttpConnect.
So far, while investigating possible solutions, I've learned about HTTP_SERVICE_CONFIG_SSL_SNI_KEY structure which is set by method HttpSetServiceConfiguration along with the matching certificate for this SNI, but this seems to be related to the server side configuration. Besides that, I haven't found any references for such action unfortunately.
Perhaps anybody ever used non-default SNI using winhttp API and can tell me how to do so ? is the only option to do so is doing the SSL handshake using some lower level API such as schannel, and than switching back to winhttp ?
if it's not possible, perhaps there's an option to use extended hostname with directory tree in order to get multiple sni on a single url...
I have problem, when I started my chrome, and try visit https page, i see this error:
Error 1013 Ray ID: 2cb7c5989ac13dd8 • 2016-08-01 08:03:08 UTC
HTTP hostname and TLS SNI hostname mismatch
You've requested an IP address that is part of the CloudFlare network. The request's Host header does not match the request's TLS SNI Host header.
What is it? where place cloudfare and how can I this remove from my browser?
SNI stands for Server Name Indication, essentially it allows you to serve more than one SSL certificate from more than one IP Address. Most modern browsers submit this as part of a web request. CloudFlare Free Universal SSL makes use of SNI.
Similarly, the Host header allows a server to know which hostname it should serve. The Host header fundamentally allows virtualhosting, serving more than one website per IP Address.
So these two clearly need to match, the SNI hostname and the hostname in the Host header must be identical. When this fails to occur CloudFlare will present a Error 1013 HTTP hostname and TLS SNI hostname mismatch.
There may be a proxy in your local network that is stripping out SNI headers or deforming them thus causing them to mismatch. This can happen as a result of a firewall or intermediary web server.
There is a tool here to check if your SNI header and Host header match. Simply visit the linked site and check the "SNI information:" field states "cc.dcsec.uni-hannover.de".
If you require support for browsers which do not support SNI, CloudFlare's Pro, Business or Enterprise support have legacy support for non-SNI browsers; but it would make sense to contact your network administrator to see why SNI requests are being malformed.
The tags on this question suggest you are using Firefox on Windows 10 which should support SNI; therefore you should reach out to your network team to see why they are malforming SNI requests.
Is it possible to route dns to different servers based on the protocol of the request without using a proxy server?
For example wss://example.com goes to 1 server and https://example.com goes to a different one.
In principle no, this is not possible. Although there are exceptions.
When you an application and request a network address, e.g. example.com the application will (typically) pass on a request to the OS to open a connection to that address.
Because the OS can only make a connection to an IP Address, the first thing it does is to make a DNS request to find out the address it can connect to. There is no mechanism at all, to tell that DNS request what protocol is being requested. At the point the OS makes the DNS request it is simply, 'What is the address for example.com' there is no space in the body of the message for more information.
In that sense it is not possible.
However there are a few services that use SRV records to find the server they need to connect to.
In these cases the application will say to the OS 'get me the SRV record for _somefancyservice.example.com' When it has that record, it will then send another request to the OS to open a connection the whatever it found in the SRV record.
So you could, in theory, write an application that performed the SRV record query first, and then opened a connection to whatever was returned.
I wrote a more detailed answer specifically about wss, which you may find interesting.
I have a Ruby polling script that runs on a set of servers in an IP range. I very strongly prefer to do this polling by IP address, not by hostname, because:
1) I have defined IP address ranges to poll, and hostnames are arbitrary/change a lot
2) Because they change a lot, most of the hostnames do not have a reverse DNS lookup, so I can't engineer a list of hostnames from IPs
Before our web servers had no problem with this polling, but on a new server that does not accept SSLv3 communication, this is the error I get when I run my poll:
/home/dashboard/.rvm/rubies/ruby-2.1.6/lib/ruby/2.1.0/net/http.rb:923:in `connect': SSL_connect returned=1 errno=0 state=unknown state: tlsv1 unrecognized name (OpenSSL::SSL::SSLError)
On the server side, this is the error:
nss_engine_init.c(1802): start function ownSSLSNISocketConfig for SNI
nss_engine_init.c(1827): Search [val = 172.16.99.18] failed, unrecognized name
When I run the poll with hostname, everything works fine.
Here is the crux of the HTTP Client code in Ruby:
def init_http(url)
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
http.read_timeout = 10
http.use_ssl = true
#http.ssl_version = 'TLSv1'
return [http, uri]
end
As you can tell, I've been playing around with TLS and SSL version, because I figured that might be the issue. My next thought (that Google only has evidence of for Java) is, "How easy is it to just disable the SNI extension on my client?" The more general question is, "Can I keep using IP addresses with Ruby net/http while taking advantage of newer, more secure communication protocols?"
... tlsv1 unrecognized name (OpenSSL::SSL::SSLError)
This is in most cases not a problem you can solve by disabling SNI on the client side. SNI is required when you have multiple certificates on the same IP address and if you just connect by IP address and don't send the requested hostname (i.e. disabling SNI) the server will not know which certificate it should provide - which then results in the above error.
I very strongly prefer to do this polling by IP address, not by hostname, ...
If you have to deal with server which require SNI then you have to use SNI and you have to use SNI with the proper hostname, which is not necessarily the same name as you get from reverse lookup.
The easiest way to solve this is to add the patch that #steffenullrich mentioned for bug 10613.
All I did was look at the diff, and edit the file myself, but you can use the Linux patch tool if you're familiar with it.
For those who are unsure of where /net/http.rb is located, it is in the same location as the rest of your Ruby sources. For example, mine was here:
/home/myuser/.rvm/rubies/ruby-2.1.6/lib/net/http.rb
Once you patch the file, set the .disable_sni property of your HTTP object to true, and SNI will not be required, allowing the use of IP addresses in mixed-TLS communication.
I have a ruby server based on TCPSocket (non-HTTP).
I have 2 different domains, both pointing with an A-Record to my servers IP Address (the same one). So, there are clients connecting to one of those domains.
Is it possible to tell which domain a client was connecting to?
I saw that this is possible in other protocols, but I'm not sure if this is based on manually added headers or really extracted from the basic tcp/ip connection. E.g. in PHP there is $_SERVER["HTTP_HOST"] which shows to which domain a client was connecting.
At the TCP socket level, the only things that are known are the source and destination IP addresses (and ports) of the connection. How the IP address was resolved via DNS is not possible to know at this layer. Even though HTTP works on top of TCP, HTTP servers have to look at the HTTP headers from the client to know which domain they are making a request to. (That's how the HTTP_HOST value gets filled in.)
One possible solution is to configure your server to have an additional IP address. This can be by assigning an additional IP address to the NIC or adding an additional NIC. Then have each domain use a different IP address. Otherwise, this is not possible and you may want to consider your application protocol on top of TCP to convey this information.