Just starting to use Spring Webflux Webclient,Just wanted to know what is the default KeepAlive time for the Http Connection ? Is there a way to increase the keepAlive Time? In our Rest Service we get a request probably every five minutes,The request takes long time to process .It takes time between 500 seconds-- 10 second. However in load test if I send frequent requests the processing time is less than 250ms.
Spring WebFlux WebClient is an HTTP client API that wraps actual HTTP libraries - so configuration like connection management, timeouts, etc. are configured at the library level directly and behavior might change depending on the chosen library.
The default library with WebClient is Reactor Netty.
Many HTTP clients (and this is the case with Reactor Netty) are maintaining HTTP connections in a connection pool to reuse them. Clients usually acquire a new connection to a remote host, use it to send/receive information and then put it back in the connection pool. This is very useful since sometimes acquiring a new connection can be costly. This seems to be really costly in your case.
HTTP clients leave those unused connections in the pool, but what about keepAlive time?
Most clients leave those connections in the pool as long as possible and test them before acquiring them to see if they're still valid or listen to server events asynchronously to remove them from the pool (I believe Reactor Netty does that). So ultimately, the server is in control and decides when to close connections if they're inactive.
Now your problem description might suggest that connecting to that remote host is very costly, but it could be also the remote host taking a long time to respond to your requests (for example, it might be operating on an empty cache and needs to calculate a lot of things).
Related
I am struggling to find information on how to gauge the scalability of websockets. A scenario -
Let's say from client wants to establish socket connection from a browser, and the client application and service layer (Micronaut) both have two instances behind an elb - service layer will sit us-east region and can expect anyone from around the world can access the frontend app from browser and can expect an open connection for an avg of 2-5 min, no longer than 30 minutes.
Is there a ballpark number on how many concurrent websocket connections a couple servers can handle? Or if there are certain factors that I didn't mention that are vital to handling websocket connections in general?
Thank you in advance.
I'm assuming you want to know the scalability of the implementation of WS in Micronaut and not WS in general. Of course, the scalability of WS is dependent on the specific implementation and WS itself. You probably already know this, but wanted to state it for the record. You may also want to be sure you increase your file descriptors for your server process to the max number (you may have to adjust your kernel to increase the FDs).
Btw, don't forget to handle retries and reconnects as you would for a low-level TCP connection
As HTTP2 supports multiplexing, do we need still a pool of connections for microservice communication?
If yes, what are the benefits of having such a pool?
Example:
Service A => Service B
Both the above services have only one instance available.
Multiple connections may help overcome OS buffer size limitation for each Connection(Socket)? What else?
Yes, you still need connection pool in a client contacting a microservice.
First, in general it's the server that controls the amount of multiplexing. A particular microservice server may decide that it cannot allow beyond a very small multiplexing.
If a client wants to use that microservice with a higher load, it needs to be prepared to open multiple connections and this is where the connection pool comes handy.
This is also useful to handle load spikes.
Second, HTTP/2 has flow control and that may severely limit the data throughput on a single connection. If the flow control window are small (the default defined by the HTTP/2 specification is 65535 bytes, which is typically very small for microservices) then client and server will spend a considerable amount of time exchanging WINDOW_UPDATE frames to enlarge the flow control windows, and this is detrimental to throughput.
To overcome this, you either need more connections (and again a client should be prepared for that), or you need larger flow control windows.
Third, in case of large HTTP/2 flow control windows, you may hit TCP congestion (and this is different from socket buffer size) because the consumer is slower than the producer. It may be a slow server for a client upload (REST request with a large payload), or a slow client for a server download (REST response with a large payload).
Again to overcome TCP congestion the solution is to open multiple connections.
Comparing HTTP/1.1 with HTTP/2 for the microservice use case, it's typical that the HTTP/1.1 connection pools are way larger (e.g. 10x-50x) than HTTP/2 connection pools, but you still want connection pools in HTTP/2 for the reasons above.
[Disclaimer I'm the HTTP/2 implementer in Jetty].
We had an initial implementation where the Jetty HttpClient was using the HTTP/2 transport with an hardcoded single connection per domain because that's what HTTP/2 preached for browsers.
When exposed to real world use cases - especially microservices - we quickly realized how bad of an idea that was, and switched back to use connection pooling for HTTP/2 (like HttpClient always did for HTTP/1.1).
How connect to https://api.push.apple.com using http2 with persistent connection ?
Persistent connection is to avoid rapid connection and disconnection:
APNs treats rapid connection and disconnection as a denial-of-service attack
https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/RemoteNotificationsPG/Chapters/APNsProviderAPI.html
Is writing a client in c using https://nghttp2.org the only solution?
(If that question should be ask in another StackExchange website, please do tell me)
Non-persistent connections are a relic of the past. They were used in HTTP/1.0, but HTTP/1.1 already moved to a model where the connections were persistent by default, and HTTP/2 (also being multiplexed) continues on that model of connections being persistent by default.
Independently on the language you are using to develop your applications, any HTTP/2 compliant client will, by default, use persistent connections.
You only need to use the HTTP/2 client library in a way that you don't explicitly close the connection after every request you make.
Typically these libraries employ a connection pool that keeps the connections open, typically until an idle timeout fires.
When your application makes HTTP requests, the library will pick an open connection and send the request. When the response arrives the library will not close the connection but instead put it back into the pool for the next usage.
Just study how the library you want to use allows you to make multiple requests without closing the connection.
I also met this question!
If the connection be idle for a long time (about 1 hour), then function poll catches no socket status changed. It always returns 0 even as on_frame_send_callback was invoked.
Is there anyone can figure out the problem?
I'm using org.eclipse.jetty.websocketclient and I want to open multiple web sockets to different URLs.
I'm working with Java.
How do I need to do that?
I want to open the web sockets in multiple threads.
1. Do I need to create websocketclient for each connection?
2. Can I use any websocketclient factory? Is there any?
3. Do I need to open only one websocketclient, keep it opened and open somehow web sockets with it?
4. What is wrong with creating multiple websocket clients?
This answer talks about Jetty 9 WebSockets.
you have 1 WebSocketClient, think of it as a Browser, with each call to connect() establishing a new connection.
Each call to connect() should have a new WebSocket instance, each instance will be managed by the WebSocketClient's Executor causing in essence each websocket instance to be on its own thread.
Followup Answers
Ideally, have only 1 WebSocketClient, and start it only once. leave it started for the time period where you have active websocket connections.
Stop the WebSocketClient when there are no more connections.
Generally speaking, avoid reusing objects for multiple requests, unless you know what you are doing. Example: the ClientUpgradeRequest and URI, are associated with the WebSocket Session, which if reused across multiple connections, will have a state change on close of the first connection, making the data invalid for the other connections, then there is also the Garbage collection references that make cleaning up the old connections difficult until all connections are closed.
You can call connect() concurrently, go for it. Each connection attempt is processed based on the Executor behavior (eg: if you have a single threaded Executor, then only 1 connect occurs at a time)
Creating a new WebSocketClient for every connect is excessively wasteful of resources. It would be like starting an entire WebServer for each incoming request. A WebSocketClient manages the selectors, threading, session tracking, etc. I realize where you are coming from, with older http client libraries having this behavior, but even those http clients are updating themselves to this new browser-ish model thanks to spdy and http/2.
I've not been able to find a clear answer as to whether or not CometD's long polling mechanism uses a persistent connection, or disconnects and then reconnects after a message is pushed to it.
The reason this is important to me is that I am currently using a long polling push client which disconnects and reconnects after every message (or batch of messages) is sent from the server, and the reconnect time introduces random latency which I am looking to get rid of. I am assuming it does this for compatibility's sake, as it makes every "push" just look like a really long request/response, which should work on any and every browser.
So, does CometD's long polling use a persistent, long-lived http connection? If the answer is yes, is it conditional? That is, are there cases/browsers where it falls back to a "request/response/reconnect" per message sent?
CometD long polling is using HTTP 1.1, and therefore persistent connections.
When CometD is used from a browser, the browser manages the connection pool and the HTTP protocol version, and CometD does not add any Connection header to close the connection after every message: all it is left to the browser, and my experience is that the long poll always stays on the same connection.
When the CometD Java client library is used, the same applies: Jetty's HTTP client manages the connection pool, defaults to HTTP 1.1 and keeps the connections open.
The main difference with browsers is that Jetty HTTP client allows more than few (usually 6) connections per domain, so it is appropriate for load testing simulations.
Check out the CometD performance report.
The updated CometD documentation can be found at http://docs.cometd.org.
It is wrong to say that "Long polling by definition does not use a persistent connection but reconnects". HTTP 1.1 is perfectly capable to send multiple long pollings over the same connection, and CometD does exactly that.
I am not aware of cases where clients like browsers fallback to open/request/response/close behaviour when using HTTP 1.1, unless this is explicitly requested by the application adding a Connection: close header to HTTP requests or responses (CometD does not do this).
With WebSocket, CometD opens 1 connection only, persistent, and all the messages are exchanged over that connection, until the application decides to close the connection by disconnecting the CometD client.