I'm currently considering the option of implementing websockets in my application. But before doing so, I want to make sure I understand correctly how it works and if it's gonna be worth it.
I understand the basics: Via WebSockets the handshake will be made only once via HTTP and then will talk to the server in order to switch to a lower level TCP layer, at that point, we have a full-duplex channel between the server and the client.
Currently I'm measuring the ajax requests made to my server (which are many), I've got this information:
The "DNS Lookup", "Initial connection" and "SSL" times are what I'm trying to eliminate (if possible)
For my understanding these times are part of the handshaking process and I'm assuming that using websockets it will happen only on the beginning (the handshake), but I'm not sure.
So my question is: Am I correct? Implementing WebSockets will ensure that the "DNS lookup" and the "Initial connection" steps happen only on the handshake?
Thanks in advance for your help, and sorry if my understanding is wrong.
I understand the basics: Via WebSockets the handshake will be made only once via HTTP and then will talk to the server in order to switch to a lower level TCP layer, at that point, we have a full-duplex channel between the server and the client.
It will not switch to a lower level TCP layer. Instead it will switch the protocol from plain HTTP (request, response) to a message based protocol - which like HTTP is on top of TCP at the application layer and not at a lower level. It's just a different protocol on the same level. It behaves a bit like TCP in that you can send and receive messages at any time without being restricted to request/response scheme of HTTP. But for example TCP is a data stream while WebSockets implements a message oriented protocol.
And, DNS is outside of WebSockets. DNS is only needed to lookup the IP address to establish the TCP connection which then gets used to do the initial HTTP handshake which is needed before the protocol switch to WebSockets.
The situation is similar with TLS. After the DNS lookup to get the IP address the TCP connection is established, then the TLS session on top of the TCP connection is established and then the initial HTTP handshake preceding the switch to WebSockets is done: i.e. HTTP inside a TLS tunnel inside a TCP connection - in other words HTTPS. The WebSocket protocol is then also spoken inside this TLS tunnel, similar how it is done with HTTP.
So my question is: Am I correct? Implementing WebSockets will ensure that the "DNS lookup" and the "Initial connection" steps happen only on the handshake?
Correct. At the beginning of each ws:// connection you might have a DNS lookup if the IP for the name is not already cached. You will have a TCP handshake and you will have a HTTP handshake which then leads to the protocol switch. This is true for every ws:// connection. And for wss:// there is additionally the creation of the TLS tunnel after the TCP connection was established and before the HTTP handshake gets started.
Related
If it matters, I am asking for the HMR (hot module replacement) use case.
I have already read this. My understanding is that WebSocket uses the same connection that was used during the handshake. My question is after that:
Can the client (browser) still send requests to the same server and same port after the WebSocket protocol starts?
If it can, will it be a different connection?
The short answer is YES and YES.
TL;DR
After reading this and this, I think I can answer my own question now.
A client can build as many connections to a server (to the limit that they can afford). There are always two ports involved in a connection - one at the server, and another at the client.
The server-side port is specified (and is known to the client). Clients open a temporary or dynamic port that lasts only as long as the connection is made.
I am trying to clarify/understand whether websockets over HTTP/2 will also be multiplexed over a TCP connection using streams. Section 5 of RFC8441 seems to suggest it
After successfully processing the opening handshake, the peers should proceed with the WebSocket Protocol [RFC6455] using the HTTP/2 stream from the CONNECT transaction as if it were the TCP connection referred to in [RFC6455]. The state of the WebSocket connection at this point is OPEN, as defined by [RFC6455], Section 4.1.
The HTTP/2 stream closure is also analogous to the TCP connection closure of [RFC6455]. Orderly TCP-level closures are represented as END_STREAM flags ([RFC7540], Section 6.1). RST exceptions are represented with the RST_STREAM frame ([RFC7540], Section 6.4) with the CANCEL error code ([RFC7540], Section 7).
But my confusion arises from the fact that even with HTTP/1.1, while tabs in a browser share the underlying TCP connections (e.g. chrome makes 6 TCP connections) to the same host, creating a websocket to the same host in different tabs leads to distinct TCP connection in each tab.
I am not sure why the difference between the two & if it is likely to be the same for websockets over HTTP/2 as well.
Any experts out here who can clarify. Thanks.
But my confusion arises from the fact that even with HTTP/1.1, while tabs in a browser share the underlying TCP connections (e.g. chrome makes 6 TCP connections) to the same host, creating a websocket to the same host in different tabs leads to distinct TCP connection in each tab.
You are right that this is the current state of affairs, unfortunately, for HTTP/1.1.
RFC 8441, as you point out, has been specified to solve this problem and piggy back WebSocket "connections" over HTTP/2 streams, so it would be possible to open just one TCP connection to an origin server and use that connection for both HTTP/2 communication and for WebSocket communication.
The difference between HTTP/1.1 and HTTP/2 stems from the fact that HTTP/1.1 WebSocket connections cannot be (efficiently) pooled.
Every WebSocket connection is tied to a specific URI (e.g. ws://host/path1) and it's more typical for an application to open different WebSocket connections for different URIs (rather than many WebSocket connections for the same URI).
Because they cannot be pooled, browsers basically have to allow an unlimited number of them, a new one every time you call new WebSocket(...) from JavaScript.
With HTTP/2 instead, you will be able to open a new HTTP/2 stream inside the same HTTP/2 connection.
The number of concurrent streams depends on browser implementations, but it's typically around 100 if not more, which leaves plenty of concurrency for both HTTP/2 and WebSocket (unless the client application is really abusing WebSocket).
Fortunately, client applications won't need to be changed to leverage this feature.
When browsers and server will support it, your application will use less resources (just one TCP connection) rather than the many it's using now.
[Disclaimer, I'm the implementer of such feature in Jetty]
We have seen a few browsers implementing this feature and we are finalizing the implementation of this feature in the Jetty 10.0.x server, see https://github.com/eclipse/jetty.project/issues/3537.
creating a websocket to the same host in different tabs leads to distinct TCP connection in each tab
A WebSocket connection is always a new TCP connection, since it has to perform an HTTP/S request that Upgrades to a WebSocket connection and is therefore no longer an HTTP/S connection if successful. WebSocket connections are distinct and can't be shared or reused, unlike HTTP/S connections (assuming keep-alives are used).
There's a fantastic answer which goes into detail as to how REST apis work.
How do websockets work in the similar detail?
Websockets create and represent a standard for bi-directional communication between a server and client. This communication channel creates a TCP connection which is outside of HTTP and is run on a seperate server.
To start this process a handshake is performed between the server and client.
Here is the work flow
1) The user makes an HTTP request to the server with an upgrade header, indicating that the client wishes to establish a WebSocket connection.
2) If the server uses the WebSocket protocol, then it will accept the upgrade and send a response back.
3) With the handshake finished, the WebSocket protocol is used from now on. All communications will use the same underlying TCP port. The new returning status code, 101, signifies Switching Protocols.
As part of HTML5 it should work with most modern browsers.
my question is simple
When you send data through TCP/IP protocol with EX:firefox you can receive reply on some random port that the browser listen on, while when i try to use a port for another task like CS Gaming or anything else it don't work unless i use kind of VPN ?
PS: there r no firewall blocking connection and port forwarding from my router didn't work as well.
Browsers are client apps that make outbound connections to web servers. When connecting to a server through a router’s NAT, the NAT takes note of the source and destination IP/port pairs so messages sent back from the server on the same connection are automatically routed to the correct client IP/port.
Browsers also support the websocket protocol. This feature makes it seem like the browser is listening on a specific port. However, in reality, it is initiated on a new connection to the server, a connection which remains open all throughout the websocket communication.
What matters is which peer is behind the NAT — the server or the client. For an outbound connection from a client, it can usually use any random port that is available at the time. For an inbound connection to a server, the server's IP/port must be known ahead of time and be routable. If the server is behind a NAT, the router(s) must be configured to make the server reachable from the other side of the NAT.
The server software can make a UPnP request to ask a router to forward inbound packets to the correct IP/Port. The router, depending on its configuration, may or may not honor such a request. If not, the router has to be configured manually by a network administrator.
A Web Socket detects the presence of a proxy server and automatically sets up a tunnel to pass through the proxy. The tunnel is established by issuing an HTTP CONNECT statement to the proxy server, which requests for the proxy server to open a TCP/IP connection to a specific host and port. Once the tunnel is set up, communication can flow unimpeded through the proxy. Since HTTP/S works in a similar fashion, secure Web Sockets over SSL can leverage the same HTTP CONNECT technique. [1]
OK, sounds useful! But, in the client implementations I've seen thus far (Go [2], Java [3]) I do not see anything related to proxy detection.
Am I missing something or are these implementations just young? I know WebSockets is extremely new and client implementations may be equally young and immature. I just want to know if I'm missing something about proxy detection and handling.
[1] http://www.kaazing.org/confluence/display/KAAZING/What+is+an+HTML+5+WebSocket
[2] http://golang.org/src/pkg/websocket/client.go
[3] http://github.com/adamac/Java-WebSocket-client/raw/master/src/com/sixfire/websocket/WebSocket.java
Let me try to explain the different success rates you may have encountered. While the HTML5 Web Socket protocol itself is unaware of proxy servers and firewalls, it features an HTTP-compatible handshake so that HTTP servers can share their default HTTP and HTTPS ports (80 and 443) with a Web Sockets gateway or server.
The Web Socket protocol defines a ws:// and wss:// prefix to indicate a WebSocket and a WebSocket Secure connection, respectively. Both schemes use an HTTP upgrade mechanism to upgrade to the Web Socket protocol. Some proxy servers are harmless and work fine with Web Sockets; others will prevent Web Sockets from working correctly, causing the connection to fail. In some cases additional proxy server configuration may be required, and certain proxy servers may need to be upgraded to support Web Sockets.
If unencrypted WebSocket traffic flows through an explicit or a transparent proxy server on its way the WebSocket server, then, whether or not the proxy server behaves as it should, the connection is almost certainly bound to fail today (in the future, proxy servers may become Web Socket aware). Therefore, unencrypted WebSocket connections should be used only in the simplest topologies.
If encrypted WebSocket connection is used, then the use of Transport Layer Security (TLS) in the Web Sockets Secure connection ensures that an HTTP CONNECT command is issued when the browser is configured to use an explicit proxy server. This sets up a tunnel, which provides low-level end-to-end TCP communication through the HTTP proxy, between the Web Sockets Secure client and the WebSocket server. In the case of transparent proxy servers, the browser is unaware of the proxy server, so no HTTP CONNECT is sent. However, since the wire traffic is encrypted, intermediate transparent proxy servers may simply allow the encrypted traffic through, so there is a much better chance that the WebSocket connection will succeed if Web Sockets Secure is used. Using encryption, of course, is not free, but often provides the highest success rate.
One way to see it in action is to download and install the Kaazing WebSocket Gateway--a highly optimized, proxy-aware WebSocket gateway, which provides native WebSocket support as well as a full emulation of the standard for older browsers.
The answer is that these clients simply do not support proxies.
-Occam
The communication channel is already established by the time the WebSocket protocol enters the scene. The WebSocket is built on top of TCP and HTTP so you don't have to care about the things already done by these protocols, including proxies.
When a WebSocket connection is established it always starts with a HTTP/TCP connection which is later "upgraded" during the "handshake" phase of WebSocket. At this time the tunnel is established so the proxies are transparent, there's no need to care about them.
Regarding websocket clients and transparent proxies,
I think websocket client connections will fail most of the time for the following reasons (not tested):
If the connection is in clear, since the client does not know it is communicating with a http proxy server, it won't send the "CONNECT TO" instruction that turns the http proxy into a tcp proxy (needed for the client after the websocket handshake). It could work if the proxy supports natively websocket and handles the URL with the ws scheme differently than http.
If the connection is in SSL, the transparent proxy cannot know to which server it should connect to since it has decrypt the host name in the https request. It could by either generating a self-signed certificate on the fly (like for SSLStrip) or providing its own static certificate and decrypt the communication but if the client validates the server certificate it will fail (see https://serverfault.com/questions/369829/setting-up-a-transparent-ssl-proxy).
You mentioned Java proxies, and to respond to that I wanted to mention that Java-Websocket now supports proxies.
You can see the information about that here: http://github.com/TooTallNate/Java-WebSocket/issues/88
websocket-client, a Python package, supports proxies, at the very least over secure scheme wss:// as in that case proxy need no be aware of the traffic it forwards.
https://github.com/liris/websocket-client/commit/9f4cdb9ec982bfedb9270e883adab2e028bbd8e9