Redis is single threaded so do we need to setup connection pool for our redis client apps? I see several libraries use connection pool for redis such as sidekiq, but does it really needed?
Redis server is single threaded. It makes a lot of sense to keep it busy with clients sending multiple requests instead of blocking by making a single request at a time.
Related
Service1 -> Our Wrapper API service -> Redis
Service2 -> Our Wrapper API service -> Redis
Client side caching with Redis is very useful. Redis provides server push-based cache invalidation for client side cache in 2 modes:
Default mode: Redis keeps a mapping of all clients(services) and the key they requested on the server side and sends requests for invalidation when data changes for keys.
Broadcast mode: Sends invalidation to all clients whenever any key changes and expects clients to update their local cache.
Problem
We have a use case in which we are building a platform service over Redis for some use cases and exposing APIs for clients to consume. We want the clients should not directly talk to Redis and talk via our API.
The problem here is how would Redis server push based cache invalidation work?
How would cache invalidation requests reach the clients(services) when we have an API layer in between. We do not wish the clients to directly talk to Redis and talk only via an API, do not want to leak the database to the clients, want to keep it internal.
Question
Do I need to implement my own push based cache invalidation layer then? Maintain clients who connect to my API and garbage collect the dead ones etc...? Is there anything better I could do and use the Redis cache invalidation or extend it for this use case?
Either the Redis client moves to those wrapper services ( becoming the actual Redis clients),
OR move the wrapper services into Redis as an extension that the clients connect to,
OR
move the wrappers into the consumers and the wrappers wrap the Redis client ( thereby continuing to enjoy the Redis client benefits)
Just starting to use Spring Webflux Webclient,Just wanted to know what is the default KeepAlive time for the Http Connection ? Is there a way to increase the keepAlive Time? In our Rest Service we get a request probably every five minutes,The request takes long time to process .It takes time between 500 seconds-- 10 second. However in load test if I send frequent requests the processing time is less than 250ms.
Spring WebFlux WebClient is an HTTP client API that wraps actual HTTP libraries - so configuration like connection management, timeouts, etc. are configured at the library level directly and behavior might change depending on the chosen library.
The default library with WebClient is Reactor Netty.
Many HTTP clients (and this is the case with Reactor Netty) are maintaining HTTP connections in a connection pool to reuse them. Clients usually acquire a new connection to a remote host, use it to send/receive information and then put it back in the connection pool. This is very useful since sometimes acquiring a new connection can be costly. This seems to be really costly in your case.
HTTP clients leave those unused connections in the pool, but what about keepAlive time?
Most clients leave those connections in the pool as long as possible and test them before acquiring them to see if they're still valid or listen to server events asynchronously to remove them from the pool (I believe Reactor Netty does that). So ultimately, the server is in control and decides when to close connections if they're inactive.
Now your problem description might suggest that connecting to that remote host is very costly, but it could be also the remote host taking a long time to respond to your requests (for example, it might be operating on an empty cache and needs to calculate a lot of things).
I am a beginner with websockets.
I have a need in my application where server needs to notify clients when something changes and am planning to use websockets.
Single server instance and single client ==> How many websockets will be created and how many connections to websockets?
Single server instance and 10 clients ==> How many websockets will be created and how many connections to websockets?
Single server instance and 1000 clients ==> How many websockets will be created and how many connections to websockets?
How do you scale with websockets when your application has a 1000’s of user base?
Thanks much for your feedback.
1) Single server instance and single client ==> How many websockets will be created and how many connections to websockets?
If your client creates one webSocket connection, then that's what there will be one webSocket connection on the client and one on the server. It's the client that creates webSocket connections to the server so it is the client that determines how many there will be. If it creates 3, then there will be 3. If it creates 1, then there will be 1. Usually, the client would just create 1.
2) Single server instance and 10 clients ==> How many websockets will be created and how many connections to websockets?
As described above, it depends upon what the client does. If each client creates 1 webSocket connection and there are 10 clients connected to the server, then the server will see a total of 10 webSocket connections.
3) Single server instance and 1000 clients ==> How many websockets will be created and how many connections to websockets?
Same as point #2.
How do you scale with webscokets when your application has a 1000’s of user base?
A single server, configured appropriately can handle hundreds of thousands of simultaneous webSocket connections that are mostly idle since an idle webSocket uses pretty much no server CPU. For even larger scale deployments, one can cluster the server (run multiple server processes) and use sticky load balancing to spread the load.
There are many other articles like these on Google worth reading if you're pursuing large scale webSocket or socket.io deployments:
The Road to 2 Million Websocket Connections in Phoenix
600k concurrent websocket connections on AWS using Node.js
10 million concurrent webSockets
Ultimately, the achievable scale per a properly configured server will likely have more to do with how much activity there is per connection and how much computation is needed to deliver that.
I'm using org.eclipse.jetty.websocketclient and I want to open multiple web sockets to different URLs.
I'm working with Java.
How do I need to do that?
I want to open the web sockets in multiple threads.
1. Do I need to create websocketclient for each connection?
2. Can I use any websocketclient factory? Is there any?
3. Do I need to open only one websocketclient, keep it opened and open somehow web sockets with it?
4. What is wrong with creating multiple websocket clients?
This answer talks about Jetty 9 WebSockets.
you have 1 WebSocketClient, think of it as a Browser, with each call to connect() establishing a new connection.
Each call to connect() should have a new WebSocket instance, each instance will be managed by the WebSocketClient's Executor causing in essence each websocket instance to be on its own thread.
Followup Answers
Ideally, have only 1 WebSocketClient, and start it only once. leave it started for the time period where you have active websocket connections.
Stop the WebSocketClient when there are no more connections.
Generally speaking, avoid reusing objects for multiple requests, unless you know what you are doing. Example: the ClientUpgradeRequest and URI, are associated with the WebSocket Session, which if reused across multiple connections, will have a state change on close of the first connection, making the data invalid for the other connections, then there is also the Garbage collection references that make cleaning up the old connections difficult until all connections are closed.
You can call connect() concurrently, go for it. Each connection attempt is processed based on the Executor behavior (eg: if you have a single threaded Executor, then only 1 connect occurs at a time)
Creating a new WebSocketClient for every connect is excessively wasteful of resources. It would be like starting an entire WebServer for each incoming request. A WebSocketClient manages the selectors, threading, session tracking, etc. I realize where you are coming from, with older http client libraries having this behavior, but even those http clients are updating themselves to this new browser-ish model thanks to spdy and http/2.
Let's say a server gets 10,000 concurrent connections (via socket.io). That's a lot, and if it can't handle any more, I need to spin up another server.
How can I sync the two servers together with their socket.io?
I wouldn't use Cluster to scale Socket.IO. Socket.IO 0.6 is designed as a single process server, and it uses long living connections or polling connections to achieve a real time connection between the server and client.
If you put Cluster infront of your socket.io client you will basically distribute the polling transports between different servers, who are not aware of the client. This will result in broken connections. But also broadcasting to all your clients will be a pain as they are all distributed on different servers and you don't have IPC between them.
So I would only advice to use Cluster if you only use Web Socket & Flash Socket connections and don't need to use the broadcast functionality.
So what should you do?
You could wait until socket.io 0.7 is released which is designed from the ground up to be used on multiple processes.
Or you can use pub/sub to send messages between different servers.
You can try to use for example cluster module and distribute the load to multiple cores (in case you have a multi-core CPU). In case this is not enough you can try to use reverse proxy for distributing requests across multiple servers and redis as a central session data store (if it's possible for your scenario).