The point of my question is about "TCP connections flow" between my application server and Google Firebase Cloud Messaging (FCM) server.
I plan that my application disconnect a TCP connection every time for each
HTTP request
and response. (This behavior is like HTTP/1.0).
However, I can't find related mentions about it on FCM's web pages.
(FCM web page (relating legacy HTTP Protocol) has a illustration about communication flow, but I want one about HTTP Protocol).
This is outside of the scope of FCM specifications, for an example,
Apple Notification Service (APNs) specification request that a tcp
connection
must not disconnect while the connection is fine. (If I eagerly want to disconnect, I have been requested once a day by ANPs specification).
Can I disconnect the connection for every HTTP communication with FCM ?
I am worried about that FCM will guess that this behavior is DDoS attack.
However, my application does not repeat connection fast like DDoS attack.
Please excuse my poor English.
Best regards,
The Firebase Cloud Messaging legacy HTTP API is a connectionless protocol. You can either establish a new connection for each request, or reuse an existing connection, as you see fit.
That said, I'd recommend re-using the connection where possible, especially if you expect a high number of requests. This both optimizes throughput, and prevents current or future misclassification as malware.
Related
I'm new user of envoy proxy, maybe someone can help me
I have 2 upstream hosts (UH1 and UH2), and configured envoy proxy to proxy (round robin) websocket connections from a client (frontend) to them.
Let's say UH1 has an active websocket connection (WC1) and UH1 becomes down (closes all its connections) I would like to keep the connection WC1 alive with a client and move the connection to UH2. Could anyone say if it is possible?
Thank you in advance!
I asked this question in the official google group of envoy-dev team and got an answer from Stephan:
This isn’t currently supported.
My sense is that one might be able to write an HTTP filter that takes
the place of the router filter to support this behavior in conjunction
with upgrade configs[1], but it’s probably a lot of work. That filter
would probably need to buffer the original HTTP request to be able
form a new websocket to a different upstream, and it would need to
decode websocket frames to avoid forwarding partial frames. Further if
the messages on the websocket are request/response oriented the filter
would have buffer websocket frames and have knowledge of their content
to avoid dropping requests.
In my experience, optimizing reconnect from the client and just
forming a new websocket connection is simpler to implement and also
provides benefits when the connection between the client and Envoy is
severed.
Stephan
https://groups.google.com/g/envoy-dev/c/3yCTqLx5ePE
I have an API running on a server, which handle users connection and a messaging system.
Beside that, I launched a websocket on that same server, waiting for connections and stuff.
And let's say we can get access to this by an Android app.
I'm having troubles to figure out what I should do now, here are my thoughts:
1 - When a user connect to the app, the API connect to the websocket. We allow the Android app only to listen on this socket to get new messages. When the user want to answer, the Android app send a message to the API. The API writes itself the received message to the socket, which will be read back by the Android app used by another user.
This way, the API can store the message in database before writing it in the socket.
2- The API does not connect to the websocket in any way. The Android app listen and write to the websocket when needed, and should, when writing to the websocket, also send a request to the API so it can store the message in DB.
May be none of the above is correct, please let me know
EDIT
I already understood why I should use a websocket, seems like it's the best way to have this "real time" system (when getting a new message for example) instead of forcing the client to make an HTTP request every x seconds to check if there are new messages.
What I still don't understand, is how it is suppose to communicate with my database. Sorry if my example is not clear, but I'll try to keep going with it :
My messaging system need to store all messages in my API database, to have some kind of historic of the conversation.
But it seems like a websocket must be running separately from the API, I mean it's another program right? Because it's not for HTTP requests
So should the API also listen to this websocket to catch new messages and store them?
You really have not described what the requirements are for your application so it's hard for us to directly advise what your app should do. You really shouldn't start out your analysis by saying that you have a webSocket and you're trying to figure out what to do with it. Instead, lay out the requirements of your app and figure out what technology will best meet those requirements.
Since your requirements are not clear, I'll talk about what a webSocket is best used for and what more traditional http requests are best used for.
Here are some characteristics of a webSocket:
It's designed to be continuously connected over some longer duration of time (much longer than the duration of one exchange between client and server).
The connection is typically made from a client to a server.
Once the connection is established, then data can be sent in either direction from client to server or from server to client at any time. This is a huge difference from a typical http request where data can only be requested by the client - with an http request the server can not initiate the sending of data to the client.
A webSocket is not a request/response architecture by default. In fact to make it work like request/response requires building a layer on top of the webSocket protocol so you can tell which response goes with which request. http is natively request/response.
Because a webSocket is designed to be continuously connected (or at least connected for some duration of time), it works very well (and with lower overhead) for situations where there is frequent communication between the two endpoints. The connection is already established and data can just be sent without any connection establishment overhead. In addition, the overhead per message is typically smaller with a webSocket than with http.
So, here are a couple typical reasons why you might choose one over the other.
If you need to be able to send data from server to client without having the client regular poll for new data, then a webSocket is very well designed for that and http cannot do that.
If you are frequently sending lots of small bits of data (for example, a temperature probe sending the current temperature every 10 seconds), then a webSocket will incur less network and server overhead than initiating a new http request for every new piece of data.
If you don't have either of the above situations, then you may not have any real need for a webSocket and an http request/response model may just be simpler.
If you really need request/response where a specific response is tied to a specific request, then that is built into http and is not a built-in feature of webSockets.
You may also find these other posts useful:
What are the pitfalls of using Websockets in place of RESTful HTTP?
What's the difference between WebSocket and plain socket communication?
Push notification | is websocket mandatory?
How does WebSockets server architecture work?
Response to Your Edit
But it seems like a websocket must be running separately from the API,
I mean it's another program right? Because it's not for HTTP requests
The same process that supports your API can also be serving the webSocket connections. Thus, when you get incoming data on the webSocket, you can just write it directly to the database the same way the API would access the database. So, NO the webSocket server does not have to be a separate program or process.
So should the API also listen to this websocket to catch new messages
and store them?
No, I don't think so. Only one process can be listening to a set of incoming webSocket connections.
I've read that HTTP is not suitable as a messaging protocol in several places such as here in reference to RabbitMQ.
I assume that there's a technical reason for this and that it's not a mere opinion. I've looked through the AMQP spec for example and can't see any reason why HTTP + Web Sockets can't work. In fact, something seems to be in the works for AMQP over Web Sockets. Furthermore, I've looked at the STOMP protocol which does use HTTP + Web Sockets and can't see any significant limitations (other than a small performance hit).
What technical characteristic does HTTP + Web Sockets lack that makes it unsuitable as a messaging protocol?
UPDATE:
This is what I was looking for: Crossbar.IO - a WAMP message broker. I needed a message broker that I can easily connect to from a browser and have not been satisfied with RabbitMQ (over STOMP) or HiveMQ (MQTT).
HTTP is request/response based, what makes it difficult to work in a publisher/subscriber fashion. Basically, you can either poll the source of messages for new ones, or create another local endpoint where the other end push messages to you.
WebSocket is different. Despite of starting as a HTTP request, it switches straightaway to a persistent, full-duplex connection, where both end can push data. Basically, in this case HTTP is only used as protocol to negotiate the connection, once negotiated WebSocket uses its own protocol to transfer data.
UPDATE: We are clear that HTTP is not a messaging protocol, since it is request/response. WebSockets, although it allows pushing data from both ends, it is not a messaging protocol neither. It defines a way of framing data, but there are not defined semantic or grammar to subscribe to topics or any operation about messaging. For example WAMP is an actual messaging protocol for websockets.
I'm trying to get a better understanding of how the server-side architecture works for WebSockets with the goal of implementing it in an embedded application. It seems that there are 3 different server-side software components in play here: 1) the web server to serve static HTTP pages and handle upgrade request, 2) a WebSockets library such as libwebsockets to handle the "nuts and bolts" of WebSockets communications, and 3) my custom application to actually figure out what to do with incoming data. How do all these fit together? Is it common to have a separate web server and WebSocket handling piece, aka a WebSocket server/daemon?
How does my application communicate with the web server and/or WebSockets library to send/receive data? For example, with CGI, the web server uses environmental variables to send info to the custom application, and stdout to receive responses. What is the equivalent communication system here? Or do you typically link in a WebSocket library into the customer application? But then how would communication with the web server to the WebSocket library + custom application work? Or all 3 combined into a single component?
Here's why I am asking. I'm using the boa web server on a uClinux/no MMU platform on a Blackfin processor with limited memory. There is no native WebSocket support in boa, only CGI. I'm trying to figure out how I can add WebSockets support to that. I would prefer to use a compiled solution as opposed to something interpreted such as JavaScript, Python or PHP. My current application using long polling over CGI, which does not provide adequate performance for planned enhancements.
First off, it's important to understand how a webSocket connection is established because that plays into an important relationship between webSocket connections and your web server.
Every webSocket connection starts with an HTTP request. The browser sends an HTTP request to the host/port that the webSocket connection is requested on. That request might look something like this:
GET /chat HTTP/1.1
Host: example.com:8000
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
What distinguishes this request from any other HTTP request to that server is the Upgrade: websocket header in the request. This tells the HTTP server that this particular request is actually a request to initiate a webSocket connection. This header also allows the web server to tell the difference between a regular HTTP request and a request to open a webSocket connection. This allows something very important in the architecture and it was done this way entirely on purpose. This allows the exact same server and port to be used for both serving your web requests and for webSocket connections. All that is needed is a component on your web server that looks for this Upgrade header on all incoming HTTP connections and, if found, it takes over the connection and turns it into a webSocket connection.
Once the server recognizes this upgrade header, it responds with a legal HTTP response, but one that signals the client that the upgrade to the webSocket protocol has been accepted that looks like this:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
At that point, both client and server keep that socket from the original HTTP request open and both switch to the webSocket protocol.
Now, to your specific questions:
How does my application communicate with the web server and/or
WebSockets library to send/receive data?
Your application may use the built-in webSocket support in modern browsers and can initiate a webSocket connection like this:
var socket = new WebSocket("ws://www.example.com");
This will instruct the browser to initiate a webSocket connection to www.example.com use the same port that the current web page was connected with. Because of the built-in webSocket support in the browser, the above HTTP request and upgrade protocol is handled for you automatically from the client.
On the server-side of things, you need to make sure you are using a web server that has incoming webSocket support and that the support is enabled and configured. Because a webSocket connection is a continuous connection once established, it does not really follow the CGI model at all. There must be at least one long-running process handling live webSocket connections. In server models (like CGI), you would need some sort of webServer add-on that supports this long-running process for your webSocket connections. In a server environment like node.js which is already a long running process, the addition of webSockets is no change at all architecturally - but rather just an additional library to support the webSocket protocol.
I'd suggest you may find this article interesting as it discussions this transition from CGI-style single request handling to the continuous socket connections of webSocket:
Web Evolution: from CGI to Websockets (and how it will help you better monitor your cloud infrastructure)
If you really want to stick with the stdin/stdout model, there are libraries that model that for your for webSockets. Here's one such library. Their tagline is "It's like CGI, twenty years later, for WebSockets".
I'm trying to figure out how I can add WebSockets support to that. I
would prefer to use a compiled solution as opposed to something
interpreted such as JavaScript, Python or PHP.
Sorry, but I'm not familiar with that particular server environment. It will likely take some in-depth searching to find out what your options are. Since a webSocket connection is a continuous connection, then you will need a process that is running continuously that can be the server-side part of the webSocket connection. This can either be something built into your webServer or it can be an additional process that the webServer starts up and forwards incoming connections to.
FYI, I have a custom application at home here built on a Raspberry Pi that uses webSockets for real-time communication with browser web pages and it works just fine. I happen to be using node.js for the server environment and the socket.io library that runs on top of webSockets to give me a higher level interface on top of webSockets. My server code checks several hardware sensors on a regular interval and then whenever there is new/changed data to report, it sends messages down any open webSockets so the connected browsers get real-time updates on the sensor readings.
You would likely need some long-running application that incoming webSocket connections were passed from the web server to your long running process or you'd need to make the webSocket connections on a different port than your web server (so they could be fielded by a completely different server process) in which case you'd have a whole separate server to handle your webSocket requests and sockets (this server would also have to support CORS to enable browsers to connect to it since it would be a different port than your web pages).
I want a bit of clarity on whether HTTPS is stateful or stateless? This is with regards to a RESTful API I built. We were initially using HTTP. Since HTTP essentially works over TCP/IP which is stateless hence HTTP is stateless, but when I switched to HTTPS my API became stateful. I wanted to know whether my conclusion that HTTPS is stateful. is correct or not?
I created my API using a middleware tool called webMethods.
Thanks
TLS/SSL is stateful. The web server and the client (browser) cache the session including the cryptographic keys to improve performance and do not perform key exchange for every request.
HTTP 1 is not stateful. HTTP/2 however defines many stateful components, but the "application layer" still remains stateless.
TL;DR: The transport pipe (TLS) is stateful, original HTTP is not.
Additional note: Cookies and other stateful mechanisms are later additions defined in separate RFC's. They are not part of the original HTTP/1.0 specification, although other stateful mechanisms like caching and HTTP auth are defined HTTP 1.1 RFC and RFC 2617. HTTP 1 is said to be stateless although in practice we use standardized stateful mechanisms. HTTP/2 defines stateful components in its standard and is therefore stateful. A particular HTTP/2 application can use a subset of HTTP/2 features to maintain statelessness.
Theory aside, in practice you use HTTP statefully in your everyday life.
The S in HTTPS is concerned with the transport, not the protocol. The semantics of the HTTP protocol remain the same for HTTPS. As the article about HTTPS on Wikipedia states,
Strictly speaking, HTTPS is not a separate protocol, but refers to use of ordinary HTTP over an encrypted SSL/TLS connection.
And the HTTP protocol is stateless by design, not because it is used most frequently over TCP/IP (nothing stops you to use HTTP over UDP for example).
HTTPS is HTTP over a secure connection.
HTTP is a higher level than a connection.
When connecting to a web server, your connection is (maybe always?) of type TCP/IP. So, in case you are visiting a website via HTTPS, your TCP/IP connection is encrypted.
The data the server and/or client send has not been encrypted by the server and/or client. It is just sent, as it is usually via HTTP, but this time using a connection via TCP/IP that is secured via encryption.
If data were vehicles, and the connexion the highway, then:
- using HTTP would be like the vehicles going on the highway, and everyone can see them;
- using HTTPS would be like the same, but the vehicles go through a tunnel or anything that prevents people not on the highway from seeing them. You can determine there is trafic, but you cannot identify the vehicles, except on both ends of the tunnel.
I believe this is an image close to what happens behind the scene. But I'm no expert. I just hope it helps.
HTTP and HTTPS both are stateless protocols. The S in HTTPS stands for Secure and it refers to use of ordinary HTTP over an encrypted SSL/TLS connection.
Use of JWT tokens or the traditional way of establishing sessions using cookies help us to overcome the problem of HTTP being a stateless protocol, as it enables the server to authenticate the identity of the client, so that you don't need to login every time you click a link to navigate on the web-page.
So For example, when you log in to the website of your bank, it only asks you to enter your login details once. Once you are signed in, you don't need to re-enter them when you navigate to the account settings page, this is because the bank site is able to authenticate your identity using JWT tokens.
JWT tokens are only used on HTTPS and not in HTTP, because the connection is encrypted in HTTPS, so it cannot be intercepted by anyone.
Thus, HTTP and HTTPS both are stateless protocols, but JWT Tokens provides a workaround for it.
I believe HTTPS is a stateful protocol as it contains Session identifier field.This generated by server initially to identify a session with the chosen client.