Even after lots of googling, I'm pretty naive about quic.
I'm specifically looking into lucasclemente/go-quic.
Should quic servers use 443? If so, will it fallback to http2 & tls 1.2?
I'm speaking generally. Not for that specific go package. But if you have specifics for that go package, that would be great.
Yes, I realize quic is still young and not necessary yet. I'm simply curious and couldn't find that specific answer. Thanks!
Quic servers can use any port they want; 443 is often used, but this is not necessary at all. Note we're talking about UDP ports, not TCP ports.
A QUIC server cannot fallback to HTTP2 or TLS 1.2, because if the client sends a QUIC request, the server can only respond with a QUIC response. If the browser doesn't support QUIC, it will not send a QUIC request in the first place, so there is no issue at all.
I think you are assuming that a QUIC server runs (or can run) on the same port as HTTPS and therefore has to generate either a QUIC or an HTTPS response; this is not the case, as QUIC is running over UDP, not over TCP (like HTTP and HTTPS do). TCP ports and UDP ports are different address spaces, so one can run an HTTPS server on tcp port 443 and simultaneously run another QUIC server on udp port 443.
Related
I'm currently considering the option of implementing websockets in my application. But before doing so, I want to make sure I understand correctly how it works and if it's gonna be worth it.
I understand the basics: Via WebSockets the handshake will be made only once via HTTP and then will talk to the server in order to switch to a lower level TCP layer, at that point, we have a full-duplex channel between the server and the client.
Currently I'm measuring the ajax requests made to my server (which are many), I've got this information:
The "DNS Lookup", "Initial connection" and "SSL" times are what I'm trying to eliminate (if possible)
For my understanding these times are part of the handshaking process and I'm assuming that using websockets it will happen only on the beginning (the handshake), but I'm not sure.
So my question is: Am I correct? Implementing WebSockets will ensure that the "DNS lookup" and the "Initial connection" steps happen only on the handshake?
Thanks in advance for your help, and sorry if my understanding is wrong.
I understand the basics: Via WebSockets the handshake will be made only once via HTTP and then will talk to the server in order to switch to a lower level TCP layer, at that point, we have a full-duplex channel between the server and the client.
It will not switch to a lower level TCP layer. Instead it will switch the protocol from plain HTTP (request, response) to a message based protocol - which like HTTP is on top of TCP at the application layer and not at a lower level. It's just a different protocol on the same level. It behaves a bit like TCP in that you can send and receive messages at any time without being restricted to request/response scheme of HTTP. But for example TCP is a data stream while WebSockets implements a message oriented protocol.
And, DNS is outside of WebSockets. DNS is only needed to lookup the IP address to establish the TCP connection which then gets used to do the initial HTTP handshake which is needed before the protocol switch to WebSockets.
The situation is similar with TLS. After the DNS lookup to get the IP address the TCP connection is established, then the TLS session on top of the TCP connection is established and then the initial HTTP handshake preceding the switch to WebSockets is done: i.e. HTTP inside a TLS tunnel inside a TCP connection - in other words HTTPS. The WebSocket protocol is then also spoken inside this TLS tunnel, similar how it is done with HTTP.
So my question is: Am I correct? Implementing WebSockets will ensure that the "DNS lookup" and the "Initial connection" steps happen only on the handshake?
Correct. At the beginning of each ws:// connection you might have a DNS lookup if the IP for the name is not already cached. You will have a TCP handshake and you will have a HTTP handshake which then leads to the protocol switch. This is true for every ws:// connection. And for wss:// there is additionally the creation of the TLS tunnel after the TCP connection was established and before the HTTP handshake gets started.
I am using Ruby's net/http library to send RESTful requests to my dev server. I've been having issues correctly parsing the data I'm sending in the request and so I'm not looking into ways to inspect the request I'm sending out through Ruby.
On another question I posted here, asking how JSON data was sent in an http request, I received a comment saying that I could possibly inspect the request by using TCPMon. In order to use TCPMon though to check the request, I need to know which port to look for the request on.
And that brings me here. What port does Ruby (and net/http) use to send out HTTP requests?
Unless otherwise specified, it will make requests to port 80 for http requests, and port 443 for https requests (the standard http/https ports).
However, you'll set up TCPMon to listen to requests on a certain port, and then forward them on to their destination. So if TCPMon is listening on 8080, you'll make the requests to port 8080.
The standard port for HTTP is port 80. If you don't specify a port in the URL, that's what it will use.
There's nothing special about port 80, it's just the standard for HTTP. You can run your server on any numeric port you want, test servers often run on something like 8080 or 8000, but you'll have to ensure your clients are connecting to that port.
Here's a list of standard TCP and UDP ports.
Shall I use WebSocket on non-80 ports? Does it ruin the whole purpose of using existing web/HTTP infrastructures? And I think it no longer fits the name WebSocket on non-80 ports.
If I use WebSocket over other ports, why not just use TCP directly? Or is there any special benefits in the WebSocket protocol itself?
And since current WebSocket handshake is in the form of a HTTP UPGRADE request, does it mean I have to enable HTTP protocol on the port so that WebSocket handshake can be accomplished?
Shall I use WebSocket on non-80 ports? Does it ruin the whole purpose
of using existing web/HTTP infrastructures? And I think it no longer
fits the name WebSocket on non-80 ports.
You can run a webSocket server on any port that your host OS allows and that your client will be allowed to connect to.
However, there are a number of advantages to running it on port 80 (or 443).
Networking infrastructure is generally already deployed and open on port 80 for outbound connections from the places that clients live (like desktop computers, mobile devices, etc...) to the places that servers live (like data centers). So, new holes in the firewall or router configurations, etc... are usually not required in order to deploy a webSocket app on port 80. Configuration changes may be required to run on different ports. For example, many large corporate networks are very picky about what ports outbound connections can be made on and are configured only for certain standard and expected behaviors. Picking a non-standard port for a webSocket connection may not be allowed from some corporate networks. This is the BIG reason to use port 80 (maximum interoperability from private networks that have locked down configurations).
Many webSocket apps running from the browser wish to leverage existing security/login/auth infrastructure already being used on port 80 for the host web page. Using that exact same infrastructure to check authentication of a webSocket connection may be simpler if everything is on the same port.
Some server infrastructures for webSockets (such as socket.io in node.js) use a combined server infrastructure (single process, one listener) to support both HTTP requests and webSockets. This is simpler if both are on the same port.
If I use WebSocket over other ports, why not just use TCP directly? Or
is there any special benefits in the WebSocket protocol itself?
The webSocket protocol was originally defined to work from a browser to a server. There is no generic TCP access from a browser so if you want a persistent socket without custom browser add-ons, then a webSocket is what is offered. As compared to a plain TCP connection, the webSocket protocol offers the ability to leverage HTTP authentication and cookies, a standard way of doing app-level and end-to-end keep-alive ping/pong (TCP offers hop-level keep-alive, but not end-to-end), a built in framing protocol (you'd have to design your own packet formats in TCP) and a lot of libraries that support these higher level features. Basically, webSocket works at a higher level than TCP (using TCP under the covers) and offers more built-in features that most people find useful. For example, if using TCP, one of the first things you have to do is get or design a protocol (a means of expressing your data). This is already built-in with webSocket.
And since current WebSocket handshake is in the form of a HTTP UPGRADE
request, does it mean I have to enable HTTP protocol on the port so
that WebSocket handshake can be accomplished?
You MUST have an HTTP server running on the port that you wish to use webSocket on because all webSocket requests start with an HTTP request. It wouldn't have to be heavily featured HTTP server, but it does have to handle the initial HTTP request.
Yes - Use 443 (ie, the HTTPS port) instead.
There's little reason these days to use port 80 (HTTP) for anything other than a redirection to port 443 (HTTPS), as certification (via services like LetsEncrypt) are easy and free to set up.
The only possible exceptions to this rule are local development, and non-internet facing services.
Should I use a non-standard port?
I suspect this is the intent of your question. To this, I'd argue that doing so adds an unnecessary layer of complication with no obvious benefits. It doesn't add security, and it doesn't make anything easier.
But it does mean that specific firewall exceptions need to be made to host and connect to your websocket server. This means that people accessing your services from a corporate/school/locked down environment are probably not going to be able to use it, unless they can somehow convince management that it is mandatory. I doubt there are many good reasons to exclude your userbase in this way.
But there's nothing stopping you from doing it either...
In my opinion, yes you can. 80 is the default port, but you can change it to any as you like.
We need to have a persistent connection between servers on WAN and LAN. Basically multiple LAN servers have to register with a server on Internet and always keep an open connection, because we can not ask clients who own serves on LAN to set up port forwarding, and because there might be firewall issues.
What are my options here? I am thinking, either Websockets, or servers on LAN long polling our server on the Internet. Is there a better solution?
Your options are - considering you want to avoid problems with proxies and firewalls - probably:
Use websockets, either port 80 or 443
Use long polling (that's not really persistent, if you need bidirectional communication!) port 80 or 443
Encapsulate whatever data into a http request, using port 80
Use port 443 with ssl encryption, and do whatever you want on the protocol level. Normally, you won't have any problems using even some strange custom protocol as long as you use this port with ssl encryption, as neither firewalls nor proxies normally (try to) inspect what you're doing when they sense what they think is https.
Everything else is probably out of scope for you as far as I get the scenario, but you should be able to do what you need using port 443 with some protocol made for persistent connections (it's possible with websockets, but you'll run into other problems really fast (like missing ping/pong implementations) then if you don't custom build your code, which would probably make it pointless to use websockets in the first place)
A Web Socket detects the presence of a proxy server and automatically sets up a tunnel to pass through the proxy. The tunnel is established by issuing an HTTP CONNECT statement to the proxy server, which requests for the proxy server to open a TCP/IP connection to a specific host and port. Once the tunnel is set up, communication can flow unimpeded through the proxy. Since HTTP/S works in a similar fashion, secure Web Sockets over SSL can leverage the same HTTP CONNECT technique. [1]
OK, sounds useful! But, in the client implementations I've seen thus far (Go [2], Java [3]) I do not see anything related to proxy detection.
Am I missing something or are these implementations just young? I know WebSockets is extremely new and client implementations may be equally young and immature. I just want to know if I'm missing something about proxy detection and handling.
[1] http://www.kaazing.org/confluence/display/KAAZING/What+is+an+HTML+5+WebSocket
[2] http://golang.org/src/pkg/websocket/client.go
[3] http://github.com/adamac/Java-WebSocket-client/raw/master/src/com/sixfire/websocket/WebSocket.java
Let me try to explain the different success rates you may have encountered. While the HTML5 Web Socket protocol itself is unaware of proxy servers and firewalls, it features an HTTP-compatible handshake so that HTTP servers can share their default HTTP and HTTPS ports (80 and 443) with a Web Sockets gateway or server.
The Web Socket protocol defines a ws:// and wss:// prefix to indicate a WebSocket and a WebSocket Secure connection, respectively. Both schemes use an HTTP upgrade mechanism to upgrade to the Web Socket protocol. Some proxy servers are harmless and work fine with Web Sockets; others will prevent Web Sockets from working correctly, causing the connection to fail. In some cases additional proxy server configuration may be required, and certain proxy servers may need to be upgraded to support Web Sockets.
If unencrypted WebSocket traffic flows through an explicit or a transparent proxy server on its way the WebSocket server, then, whether or not the proxy server behaves as it should, the connection is almost certainly bound to fail today (in the future, proxy servers may become Web Socket aware). Therefore, unencrypted WebSocket connections should be used only in the simplest topologies.
If encrypted WebSocket connection is used, then the use of Transport Layer Security (TLS) in the Web Sockets Secure connection ensures that an HTTP CONNECT command is issued when the browser is configured to use an explicit proxy server. This sets up a tunnel, which provides low-level end-to-end TCP communication through the HTTP proxy, between the Web Sockets Secure client and the WebSocket server. In the case of transparent proxy servers, the browser is unaware of the proxy server, so no HTTP CONNECT is sent. However, since the wire traffic is encrypted, intermediate transparent proxy servers may simply allow the encrypted traffic through, so there is a much better chance that the WebSocket connection will succeed if Web Sockets Secure is used. Using encryption, of course, is not free, but often provides the highest success rate.
One way to see it in action is to download and install the Kaazing WebSocket Gateway--a highly optimized, proxy-aware WebSocket gateway, which provides native WebSocket support as well as a full emulation of the standard for older browsers.
The answer is that these clients simply do not support proxies.
-Occam
The communication channel is already established by the time the WebSocket protocol enters the scene. The WebSocket is built on top of TCP and HTTP so you don't have to care about the things already done by these protocols, including proxies.
When a WebSocket connection is established it always starts with a HTTP/TCP connection which is later "upgraded" during the "handshake" phase of WebSocket. At this time the tunnel is established so the proxies are transparent, there's no need to care about them.
Regarding websocket clients and transparent proxies,
I think websocket client connections will fail most of the time for the following reasons (not tested):
If the connection is in clear, since the client does not know it is communicating with a http proxy server, it won't send the "CONNECT TO" instruction that turns the http proxy into a tcp proxy (needed for the client after the websocket handshake). It could work if the proxy supports natively websocket and handles the URL with the ws scheme differently than http.
If the connection is in SSL, the transparent proxy cannot know to which server it should connect to since it has decrypt the host name in the https request. It could by either generating a self-signed certificate on the fly (like for SSLStrip) or providing its own static certificate and decrypt the communication but if the client validates the server certificate it will fail (see https://serverfault.com/questions/369829/setting-up-a-transparent-ssl-proxy).
You mentioned Java proxies, and to respond to that I wanted to mention that Java-Websocket now supports proxies.
You can see the information about that here: http://github.com/TooTallNate/Java-WebSocket/issues/88
websocket-client, a Python package, supports proxies, at the very least over secure scheme wss:// as in that case proxy need no be aware of the traffic it forwards.
https://github.com/liris/websocket-client/commit/9f4cdb9ec982bfedb9270e883adab2e028bbd8e9