I am reading the documentation of Alexa Voice Service capabilities and came across the part on managing HTTP2 connection. I don't really understand how this down channel works behind the scenes. Is it using server push? Well, could server push be used to keep a long connection? Or is it just using some tricks to keep the connection alive for a very long time?
As stated on the documentation, the client needs to establish a down channel stream with the server.
Based on what I read here https://www.rfc-editor.org/rfc/rfc7540, From this state diagram:
once the stream sends a HEADER frame, followed by an END STREAM flag, the state will be half-closed(local) on the point of view of the client. So, this is how half-closed state for the device happened, as stated in above image. Correct me that if I am wrong.
For managing the HTTP connection, this is what it says.
Based on my understanding: the client sets a timeout of 60minutes for the GET request. After the request is sent, the server will not send any response. Then the connection will remain open for 60minutes. But once a response is sent from the server, the connection should be closed. Isn't that supposed to happen? Or, is it because when the server sends response through the down channel stream, it did not send an END STREAM flag so the stream will not be closed?
But once a response is sent from the server, the connection should be closed.
HTTP/1.1 and HTTP/2 use persistent connections, which means that a single connection can be used not just for one request/response, but for several request/response cycles.
Only HTTP/1.0 was closing the connection after the response, and so for HTTP/2 this is not the case, the connection will remain open until either peer decides to explicitly close it.
The recommendations about the idle timeouts are exactly to prevent the client to explicitly close the connection too early when it sees no network traffic, independently from requests or responses.
As HTTP2 supports multiplexing, do we need still a pool of connections for microservice communication?
If yes, what are the benefits of having such a pool?
Example:
Service A => Service B
Both the above services have only one instance available.
Multiple connections may help overcome OS buffer size limitation for each Connection(Socket)? What else?
Yes, you still need connection pool in a client contacting a microservice.
First, in general it's the server that controls the amount of multiplexing. A particular microservice server may decide that it cannot allow beyond a very small multiplexing.
If a client wants to use that microservice with a higher load, it needs to be prepared to open multiple connections and this is where the connection pool comes handy.
This is also useful to handle load spikes.
Second, HTTP/2 has flow control and that may severely limit the data throughput on a single connection. If the flow control window are small (the default defined by the HTTP/2 specification is 65535 bytes, which is typically very small for microservices) then client and server will spend a considerable amount of time exchanging WINDOW_UPDATE frames to enlarge the flow control windows, and this is detrimental to throughput.
To overcome this, you either need more connections (and again a client should be prepared for that), or you need larger flow control windows.
Third, in case of large HTTP/2 flow control windows, you may hit TCP congestion (and this is different from socket buffer size) because the consumer is slower than the producer. It may be a slow server for a client upload (REST request with a large payload), or a slow client for a server download (REST response with a large payload).
Again to overcome TCP congestion the solution is to open multiple connections.
Comparing HTTP/1.1 with HTTP/2 for the microservice use case, it's typical that the HTTP/1.1 connection pools are way larger (e.g. 10x-50x) than HTTP/2 connection pools, but you still want connection pools in HTTP/2 for the reasons above.
[Disclaimer I'm the HTTP/2 implementer in Jetty].
We had an initial implementation where the Jetty HttpClient was using the HTTP/2 transport with an hardcoded single connection per domain because that's what HTTP/2 preached for browsers.
When exposed to real world use cases - especially microservices - we quickly realized how bad of an idea that was, and switched back to use connection pooling for HTTP/2 (like HttpClient always did for HTTP/1.1).
I have an API running on a server, which handle users connection and a messaging system.
Beside that, I launched a websocket on that same server, waiting for connections and stuff.
And let's say we can get access to this by an Android app.
I'm having troubles to figure out what I should do now, here are my thoughts:
1 - When a user connect to the app, the API connect to the websocket. We allow the Android app only to listen on this socket to get new messages. When the user want to answer, the Android app send a message to the API. The API writes itself the received message to the socket, which will be read back by the Android app used by another user.
This way, the API can store the message in database before writing it in the socket.
2- The API does not connect to the websocket in any way. The Android app listen and write to the websocket when needed, and should, when writing to the websocket, also send a request to the API so it can store the message in DB.
May be none of the above is correct, please let me know
EDIT
I already understood why I should use a websocket, seems like it's the best way to have this "real time" system (when getting a new message for example) instead of forcing the client to make an HTTP request every x seconds to check if there are new messages.
What I still don't understand, is how it is suppose to communicate with my database. Sorry if my example is not clear, but I'll try to keep going with it :
My messaging system need to store all messages in my API database, to have some kind of historic of the conversation.
But it seems like a websocket must be running separately from the API, I mean it's another program right? Because it's not for HTTP requests
So should the API also listen to this websocket to catch new messages and store them?
You really have not described what the requirements are for your application so it's hard for us to directly advise what your app should do. You really shouldn't start out your analysis by saying that you have a webSocket and you're trying to figure out what to do with it. Instead, lay out the requirements of your app and figure out what technology will best meet those requirements.
Since your requirements are not clear, I'll talk about what a webSocket is best used for and what more traditional http requests are best used for.
Here are some characteristics of a webSocket:
It's designed to be continuously connected over some longer duration of time (much longer than the duration of one exchange between client and server).
The connection is typically made from a client to a server.
Once the connection is established, then data can be sent in either direction from client to server or from server to client at any time. This is a huge difference from a typical http request where data can only be requested by the client - with an http request the server can not initiate the sending of data to the client.
A webSocket is not a request/response architecture by default. In fact to make it work like request/response requires building a layer on top of the webSocket protocol so you can tell which response goes with which request. http is natively request/response.
Because a webSocket is designed to be continuously connected (or at least connected for some duration of time), it works very well (and with lower overhead) for situations where there is frequent communication between the two endpoints. The connection is already established and data can just be sent without any connection establishment overhead. In addition, the overhead per message is typically smaller with a webSocket than with http.
So, here are a couple typical reasons why you might choose one over the other.
If you need to be able to send data from server to client without having the client regular poll for new data, then a webSocket is very well designed for that and http cannot do that.
If you are frequently sending lots of small bits of data (for example, a temperature probe sending the current temperature every 10 seconds), then a webSocket will incur less network and server overhead than initiating a new http request for every new piece of data.
If you don't have either of the above situations, then you may not have any real need for a webSocket and an http request/response model may just be simpler.
If you really need request/response where a specific response is tied to a specific request, then that is built into http and is not a built-in feature of webSockets.
You may also find these other posts useful:
What are the pitfalls of using Websockets in place of RESTful HTTP?
What's the difference between WebSocket and plain socket communication?
Push notification | is websocket mandatory?
How does WebSockets server architecture work?
Response to Your Edit
But it seems like a websocket must be running separately from the API,
I mean it's another program right? Because it's not for HTTP requests
The same process that supports your API can also be serving the webSocket connections. Thus, when you get incoming data on the webSocket, you can just write it directly to the database the same way the API would access the database. So, NO the webSocket server does not have to be a separate program or process.
So should the API also listen to this websocket to catch new messages
and store them?
No, I don't think so. Only one process can be listening to a set of incoming webSocket connections.
I'm learning about HTTP/2 protocol. It's a binary protocol with small message frames. It allows stream multiplexing over single TCP connection. Conceptually it seems very similar to WebSockets.
Are there plans to obsolete websockets and replace them with some kind of headerless HTTP/2 requests and server-initiated push messages? Or will WebSockets complement HTTP/2?
After just getting finished reading RFC 7540, HTTP/2 does obsolete websockets for all use cases except for pushing binary data from the server to a JS webclient. HTTP/2 fully supports binary bidi streaming (read on), but browser JS doesn't have an API for consuming binary data frames and AFAIK such an API is not planned.
For every other application of bidi streaming, HTTP/2 is as good or better than websockets, because (1) the spec does more work for you, and (2) in many cases it allows fewer TCP connections to be opened to an origin.
PUSH_PROMISE (colloquially known as server push) is not the issue here. That's just a performance optimization.
The main use case for Websockets in a browser is to enable bidirectional streaming of data. So, I think the OP's question becomes whether HTTP/2 does a better job of enabling bidirectional streaming in the browser, and I think that yes, it does.
First of all, it is bi-di. Just read the introduction to the streams section:
A "stream" is an independent, bidirectional sequence of frames
exchanged between the client and server within an HTTP/2 connection.
Streams have several important characteristics:
A single HTTP/2 connection can contain multiple concurrently open
streams, with either endpoint interleaving frames from multiple
streams.
Streams can be established and used unilaterally or shared by
either the client or server.
Streams can be closed by either endpoint.
Articles like this (linked in another answer) are wrong about this aspect of HTTP/2. They say it's not bidi. Look, there is one thing that can't happen with HTTP/2: After the connection is opened, the server can't initiate a regular stream, only a push stream. But once the client opens a stream by sending a request, both sides can send DATA frames across a persistent socket at any time - full bidi.
That's not much different from websockets: the client has to initiate a websocket upgrade request before the server can send data across, too.
The biggest difference is that, unlike websockets, HTTP/2 defines its own multiplexing semantics: how streams get identifiers and how frames carry the id of the stream they're on. HTTP/2 also defines flow control semantics for prioritizing streams. This is important in most real-world applications of bidi.
(That wrong article also says that the Websocket standard has multiplexing. No, it doesn't. It's not really hard to find that out, just open the Websocket RFC 6455 and press ⌘-F, and type "multiplex". After you read
The protocol is intended to be extensible; future versions will likely introduce additional concepts such as multiplexing.
You will find that there is 2013 draft extension for Websocket multiplexing. But I don't know which browsers, if any, support that. I wouldn't try to build my SPA webapp on the back of that extension, especially with HTTP/2 coming, the support may never arrive).
Multiplexing is exactly the kind of thing that you normally have to do yourself whenever you open up a websocket for bidi, say, to power a reactively updating single page app. I'm glad it's in the HTTP/2 spec, taken care of once and for all.
If you want to know what HTTP/2 can do, just look at gRPC. gRPC is implemented across HTTP/2. Look specifically at the half and full duplex streaming options that gRPC offers. (Note that gRPC doesn't currently work in browsers, but that is actually because browsers (1) don't expose the HTTP/2 frame to the client javascript, and (2) don't generally support Trailers, which are used in the gRPC spec.)
Where might websockets still have a place? The big one is server->browser pushed binary data. HTTP/2 does allow server->browser pushed binary data, but it isn't exposed in browser JS. For applications like pushing audio and video frames, this is a reason to use websockets.
Edit: Jan 17 2020
Over time this answer has gradually risen up to the top (which is good, because this answer is more-or-less correct). However there are still occasional comments saying that it is not correct for various reasons, usually related to some confusion about PUSH_PROMISE or how to actually consume message-oriented server -> client push in a single page app.
If you need to build a real-time chat app, let's say, where you need to broadcast new chat messages to all the clients in the chat room that have open connections, you can (and probably should) do this without websockets.
You would use Server-Sent Events to push messages down and the Fetch api to send requests up. Server-Sent Events (SSE) is a little-known but well supported API that exposes a message-oriented server-to-client stream. Although it doesn't look like it to the client JavaScript, under the hood your browser (if it supports HTTP/2) will reuse a single TCP connection to multiplex all of those messages. There is no efficiency loss and in fact it's a gain over websockets because all the other requests on your page are also sharing that same TCP connection. Need multiple streams? Open multiple EventSources! They'll be automatically multiplexed for you.
Besides being more resource efficient and having less initial latency than a websocket handshake, Server-Sent Events have the nice property that they automatically fall back and work over HTTP/1.1. But when you have an HTTP/2 connection they work incredibly well.
Here's a good article with a real-world example of accomplishing the reactively-updating SPA.
From what I understood HTTP/2 is not a replacement for websocket but aims to standardize SPDY protocol.
In HTTP/2, server-push is used behind the scene to improve resource loading by the client from the browser. As a developer, you don't really care about it during your development. However, with Websocket, the developer is allowed to use API which is able to consume and push message with an unique full-duplex connection.
These are not the same things, and they should complement each other.
I say Nay (Websockets aren't obsolete).
The first and most often ignored issue is that HTTP/2 push isn't enforceable and might be ignored by proxies, routers, other intermediaries or even the browser.
i.e. (from the HTTP2 draft):
An intermediary can receive pushes from the server and choose not to forward them on to the client. In other words, how to make use of the pushed information is up to that intermediary. Equally, the intermediary might choose to make additional pushes to the client, without any action taken by the server.
Hence, HTTP/2 Push can't replace WebSockets.
Also, HTTP/2 connections do close after a while.
It's true that the standard states that:
HTTP/2 connections are persistent. For best performance, it is expected that clients will not close connections until it is determined that no further communication with a server is necessary (for example, when a user navigates away from a particular web page) or until the server closes the connection.
But...
Servers are encouraged to maintain open connections for as long as possible but are permitted to terminate idle connections if necessary. When either endpoint chooses to close the transport-layer TCP connection, the terminating endpoint SHOULD first send a GOAWAY (Section 6.8) frame so that both endpoints can reliably determine whether previously sent frames have been processed and gracefully complete or terminate any necessary remaining tasks.
Even if the same connection allows for pushing content while it is open and even if HTTP/2 resolves some of the performance issues introduced by HTTP/1.1's 'keep-alive'... HTTP/2 connections aren't kept open indefinitely.
Nor can a webpage re-initiate an HTTP/2 connection once closed (unless we're back to long-pulling, that is).
EDIT (2017, two years later)
Implementations of HTTP/2 show that multiple browser tabs/windows share a single HTTP/2 connection, meaning that push will never know which tab / window it belongs to, eliminating the use of push as a replacement for Websockets.
EDIT (2020)
I'm not sure why people started downvoting the answer. If anything, the years since the answer was initially posted proved that HTTP/2 can't replace WebSockets and wasn't designed to do so.
Granted, HTTP/2 might be used to tunnel WebSocket connections, but these tunneled connections will still require the WebSocket protocol and they will effect the way the HTTP/2 container behaves.
The answer is no. The goal between the two are very different. There is even an RFC for WebSocket over HTTP/2 which allows you to make multiple WebSocket connections over a single HTTP/2 TCP pipe.
WS over HTTP/2 will be a resource conservation play by decreasing the time to open new connections and allowing for more communication channels without the added expense of more sockets, soft IRQs, and buffers.
https://datatracker.ietf.org/doc/html/draft-hirano-httpbis-websocket-over-http2-01
Well, to quote from this InfoQ article:
Well, the answer is clearly no, for a simple reason: As we have seen above, HTTP/2 introduces Server Push which enables the server to proactively send resources to the client cache. It does not, however, allow for pushing data down to the client application itself. Server pushes are only processed by the browser and do not pop up to the application code, meaning there is no API for the application to get notifications for those events.
And so HTTP2 push is really something between your browser and server, while Websockets really expose the APIs that can be used by both client (javascript, if its running on browser) and application code (running on server) for transferring real time data.
As of today, no.
HTTP/2, compared to HTTP, allows you to maintain a connection with a server. From there, you can have multiple streams of data at the same time. The intent is that you can push multiple things at the same time even without the client requesting it. For example, when a browser asks for a index.html, the server might want to also push index.css and index.js. The browser didn't ask for it, but the server might provide it without being asked because it can assume you're going to want in a few seconds.
This is faster than the HTTP/1 alternative of getting index.html, parsing it, discovering it needs index.js and index.css and then building 2 other requests for those files. HTTP/2 lets the server push data the client hasn't even asked for.
In that context, it's similar to WebSocket, but not really by design. WebSocket is supposed to allow a bi-directional communication similar to a TCP connection, or a serial connection. It's a socket where both communicate with each other. Also, the major difference is that you can send any arbitrary data packets in raw bytes, not encapsulated in HTTP protocol. The concepts of headers, paths, query strings only happen during the handshake, but WebSocket opens up a data stream.
The other difference is you get a lot more fine-tuned access to WebSocket in Javascript, whereas with HTTP, it's handled by the browser. All you get with HTTP is whatever you can fit in XHR/fetch(). That also means the browser will get to intercept and modify HTTP headers without you being able to control it (eg: Origin, Cookies, etc). Also, what HTTP/2 is able to push is sent to the browser. That means JS doesn't always (if ever) know things are being pushed. Again, it makes sense for index.css and index.js because the browser will cache it, but not so much for data packets.
It's really all in the name. HTTP stands for HyperText Transfer Protocol. We're geared around the concept of transferring assets. WebSocket is about building a socket connection where binary data gets passed around bidirectionally.
The one we're not really discussing is SSE (Server-Sent Events). Pushing data to the application (JS) isn't HTTP/2's intent, but it is for SSE. SSE gets really strengthened with HTTP/2. But it's a not a real replacement for WebSockets when what's important is the data itself, not the variable endpoints being reached. For each endpoint in with WebSocket a new data stream is created, but with SSE it's shared between the already existing HTTP/2 session.
Summarized here are the objectives for each:
HTTP - Respond to a request with one asset
HTTP/2 - Respond to a request with multiple assets
SSE - Respond with a unidirectional text (UTF-8) event stream
WebSocket - Create a bidirectional binary data stream
Message exchange and simple streaming(not audio, video streaming) can be done via both Http/2 multiplexing and WebSockets. So there is some overlap, but WebSockets have well established protocol, a lot of frameworks/APIs and less headers overhead.
Here is nice article about the topic.
No, WebSockets are not obsolete. However, HTTP/2 breaks websockets as defined for HTTP/1.1 (mostly by forbidding protocol updates using the Upgrade header). Which is why this rfc:
https://datatracker.ietf.org/doc/html/rfc8441
defines a websocket bootstrapping procedure for HTTP/2.
For the time being April 2020, HTTP/2 is not making WebSockets obsolete. The greatest advantage of WebSockets over HTTP2 is that
HTTP/2 works only on Browser Level not Application Level
Means that HTTP/2 does not offer any JS API like WebSockets to allow communication and transfer some kind of JSON or other data to server directly from Application (e.g. Website). So, as far as I believe, HTTP/2 will only make WebSockets obsolete if it starts offering API like WebSockets to talk to server. Till that it is just updated and faster version of HTTP 1.1.
No HTTP/2 does not make websockets obsolete, but SSE over HTTP/2 offers a viable alternative. The minor caveat is that SSE does not support unsolicited events from server to client (and neither does HTTP/2): i.e. the client has to explicitly subscribe by creating an EventSource instance specifying the event source endpoint. So you may have to slightly reorganise how the client arranges for events to be delivered - I can't think of a scenario where this is actually a technical barrier.
SSE works with HTTP/1.1. But HTTP/2 makes using SSE generally viable and competitive with websockets in terms of efficiency, instead of practically unusable in the case of HTTP/1.1. Firstly, HTTP/2 multiplexes many event source connections (or rather "streams" in HTTP/2 terms) onto a single TCP connection where as in HTTP/1.1 you'd need one connection for each. According to the HTTP/2 spec, millions of streams can be created per connection by default with the recommended (configurable) minimum being 100, where as browsers maybe severly limited in the number of TCP connections they can make. Second reason is efficiency: many streams in HTTP/2 is requires much less overhead than the many connections required in HTTP/1.1.
One final thing is, if you want to replace websockets with SSE your forgoing some of the tools / middlewares built on top of websockets. In particular I'm thinking of socket.io (which is how a lot of people actually use websockets), but I'm sure there is a ton more.
According to the Wikipedia article: http://en.wikipedia.org/wiki/WebSocket,
The server sends back this response to the client during handshake:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk=
Sec-WebSocket-Protocol: chat
Does this close the connection (as HTTP responses usually do) or it is kept open throughout the entire handshake and it can start sending WebSocket frames straight away (assuming that it succeeds)?
An HTTP socket going through the handshake process to be upgraded to the webSocket protocol is not closed during that process. The same open socket goes through the whole process and then becomes the socket used for the webSocket protocol. As soon as the upgrade is complete, that very socket is ready for messages to be sent per the webSocket protocol.
It is this use of the exact same socket that enables a webSocket connection to run on the same port as an HTTP request (no extra port is needed) because it literally starts out as an HTTP request (with some extra headers attached) and then when those headers are recognized and both sides agree, the socket from that original HTTP request on the original web port (often port 80) is then switched to use the webSocket protocol. No additional connection on some new port is needed.
I actually find it a relatively elegant design because it makes for easy coexistence with a web server which was an important design parameter. And, a slight extra bit of connection overhead (protocol upgrade negotiation) is generally not an issue because webSocket connections by their very nature are designed to be long running sockets which you open once and use over an extended period of time so a little extra overhead to open them doesn't generally bother their use.
If, for any reason, the upgrade is not completed (both sides don't agree on the upgrade to webSocket), then the socket would remain an HTTP socket and would behave as HTTP sockets normally do (likely getting closed right away, but subject to normal HTTP interactions).
You can see this answer for more details on the back and forth during an upgrade to webSocket: SocketIO tries to connect using same port as the browser used to get web page