Service1 -> Our Wrapper API service -> Redis
Service2 -> Our Wrapper API service -> Redis
Client side caching with Redis is very useful. Redis provides server push-based cache invalidation for client side cache in 2 modes:
Default mode: Redis keeps a mapping of all clients(services) and the key they requested on the server side and sends requests for invalidation when data changes for keys.
Broadcast mode: Sends invalidation to all clients whenever any key changes and expects clients to update their local cache.
Problem
We have a use case in which we are building a platform service over Redis for some use cases and exposing APIs for clients to consume. We want the clients should not directly talk to Redis and talk via our API.
The problem here is how would Redis server push based cache invalidation work?
How would cache invalidation requests reach the clients(services) when we have an API layer in between. We do not wish the clients to directly talk to Redis and talk only via an API, do not want to leak the database to the clients, want to keep it internal.
Question
Do I need to implement my own push based cache invalidation layer then? Maintain clients who connect to my API and garbage collect the dead ones etc...? Is there anything better I could do and use the Redis cache invalidation or extend it for this use case?
Either the Redis client moves to those wrapper services ( becoming the actual Redis clients),
OR move the wrapper services into Redis as an extension that the clients connect to,
OR
move the wrappers into the consumers and the wrappers wrap the Redis client ( thereby continuing to enjoy the Redis client benefits)
Related
I'm building a system where client IoT devices will be making persistent websocket connections to a single instance of a microservice. We'll call it the "hardware gateway". End devices will be connecting to one of these service instances and may migrate between services at anytime (perhaps due to a reboot or network interruption).
Other services will be pushing notifications to these hardware clients via some hardware gateway instance. I need a way to route these requests to the specific instance that is maintaining a connection to a specific IoT device. At the moment, my solution is to maintain an external KV store where I can map an IoT device's UUID to a service instance, but that puts an extra dependency on all other services to know about this KV store. Not to mention the additional latency introduced by this query.
Maybe there's some reverse proxy that allows me to dynamically update its matching criteria? I've also looked into using a message broker like RabbitMQ, but it doesn't seem to support this use case.
There's a reasonable solution in JVM land for this: Akka.
The instances form an Akka cluster. When a device makes a websocket connection, an actor is spawned to handle the interactions over the websocket. The actor registers that it is the actor interacting with the device with a cluster sharded actor keyed by the device's ID (and likely periodically reregisters with the sharded actor). As instances are deployed, etc. the cluster rebalances. An important feature of this is that the service is stateful, but the instances deploy in a way that looks to the outside world like it's stateless: requests can go to any node.
For pushing notifications to the devices, the HTTP endpoint or message-bus consumer in the service looks up the cluster sharded actor which forwards the notification to the websocket actor (you'll want to think about whether you want at-least-once or at-most-once delivery, which will govern whether there's some portion of the cluster sharded actor which should be persistent).
Is there any CDN provider able to offer some sort of WebSocket caching either via "Edge Workers" or other mechanism?
I would like that some WebSocket requests that we make to be able to serve them as close to the user as possible without us setting up our entire infrastructure in multiple datacenters.
No, there is no way to cache WebSocket messages - e.g. the protocol does not have any "addressable" resources as e.g. HTTP. HTTP has more built in design for caching in the protocol, e.g. GET requests are typically cacheable but not POST requests and you can add cache related headers to the requests, e.g. how long it can be cached and so on.
For this reason, I would recommend to use HTTP instead of WebSocket unless you have a use case where you really have to use WebSockets. With HTTP you have access to all CDN and proxy infrastructure that is available globally.
It is getting more and more common to use e.g. Server Sent Events, HTTP/2 streams or possibly GRPC for use cases where WebSockets was used before. The upcoming WebTransport protocol will probably be the replacement(?) - at least when most other traffic is HTTP/3.
Seconded, caching WebSocket messages is not possible/not recommended. I would encourage you to look into transforming your WebSocket messaging into a form of Edge Computing.
For instance:
Akamai offers EdgeWorkers.
CloudFlare offers Workers.
AWS offers Lambda.
This might offer you the ability to handle your workflow instead of relying on WebSockets.
/Mike (and yes, I work at Akamai)
So I have a bit of a question since I'm having a hard time wrapping my head around it. Currently I have a GraphQL API Server created using Apollo-Server and persisted using a local sqlite database. I have the queries and mutations working correctly.
I also have an external WebSocket server that constantly has messages (that match my GraphQL/Database schema) produced to it at say ws://localhost:8000/websocket. Is it possible to have my GraphQL Server subscribe to that websocket address and constantly parse those messages and use the appropriate mutation to insert into the backend database?
I would then have a Vue frontend that would constantly display the results (via Vue Apollo Clients WS subscription maybe?)
to have my GraphQL Server subscribe to that websocket address and constantly parse those messages
Typically no, its the other way around - you can make ws server to call graphql server to do the mutations. That is if you want to use WS as the primary transport layer for everything - queries, mutations & subscriptions.
But usually architecture separates queries & mutations because they are more stateless and critical from subscriptions which are more stateful (persisted connection)
client -> queries & mutations -> graphql server --> redis pubsub -+
|
client <--> subscriptions <-- graphql subscription server <-------+
(in simpler cases when you don't need high load, you can combine both servers to use in-memory pubsub)
BUT, if you very much want to, ofc you can write custom code to connect graphql server -> listen ws server in the background. See https://github.com/enisdenjo/graphql-ws#node-client for example
The problem can appear if you have some user context. You would need to either have custom connection where changes of all users happen. Or have a dedicated connection for every user
Yes , it can be done quite easily , just write a service worker or a worker thread that constantly checks for new messages
Can be done using worker_threads in node js
And if you need to implement it realtime
Make sure your worker thread starts a socket connection and is constantly connected to the port where you are publishing your messages
You can do it using socket.io library
I have exposed a websocket enabled service endpoint through Azure Application Gateway and the service is hosted on azure service fabric. Client initiates a websocket connection with my endpoint and is able to exchange data. During certain message flows, my Web Socket enabled service calls other services hosted on the service fabric using azure service bus. These are handled in a completely async manner. Once the other services finish processing, they post a message to the service bus which my WebSocket service reads back.
The problem I am having is to route the messages back to the right service fabric node so that it can be pushed back to the client at the other end of the WebSocket connection
In the picture below, you can imagine each node containing multiple services including the web socket enabled service. Once the Websocket service posts a message to the service bus, the downstream services start processing and finally they post a message back to the service bus which the websocket service reads back. Here a random node will pick up the message and it might not have the relevent websocket connection to push the processed data back
Sample Design
I have looked at redis pubsub model and it looks like I have to maintain last message processed on the nodes. It also means, every node on the cluster will need to read the message and discard it if they don't have the websocket connection with the client. I am looking for any suggested design models for this kind of problem
I ran into a similar scenario and didn't like the idea of using a new external service (Redis/SQL Server) as a backplane that would simply duplicate each message/event across all nodes.
The solution I settled on was to lean on a property of actor proxies, using actor events to call-back to a specific instance of a stateless service. Creating an actor service to act as a pub/sub backplane.
The solution is summarised in this blog post and this GitHub repo. It's worth pointing out that the documentation states actor events are best effort. This hasn't really been an issue when the application is running as normal, I presume that during a deployment or failover, some events may get lost, however this could be mitigated with additional work.
It's also worth noting that your load balancing rules should maintain sticky connections between clients and back-end instances. You could create separate rules for websockets if you only wanted this to apply to them and not your regular HTTP traffic.
In server-side load balancing, the clients call an intermediate server, which then decides which instance of the actual server (or microservice) to call.
In client-side load balancing also, the clients call an intermediate server (the API gateway - Zuul for instance, configured with a load-balancer - Ribbon for instance and a naming server - Eureka for instance), which then decides which instance of the microservice to call.
Unless we include the API gateway as part of the client, the client still doesn't know the IP address of the exact server to which it should send the request. Seems to me, to be a lot like server-side load-balancing. Is there something I'm missing?
(Including the API gateway as part of client seems weird, since its usually deployed on a different server from the client)
In Client Side load balancing, the Client is doing the heavy lifting of discovery and connection to the origin server. The client may reference a lookup (Eureka, Consul, maybe DDNS), to discover what the end destination is and the registry will dole out a valid origin. The communication is direct, client to server without a middle man.
In Server Side load balancing, the client is dumb, and makes a call to a predetermined address (usually DNS or static IP). That device then either proxies (TCP or protocol level) the connection to the origin server based on either a lookup, heartbeat, etc.
I've seen benefits in client side routing in that as long as you have IP connectivity between client and server, the work of the infrastructure is trivial to add new services, locations, products, apps, etc. As long as the new server can "register" with the registry, and the client has IP access to the server, it just works and IT does not have to be involved in rolling out your new service.
The drawback is it makes the client a little more heavy, it does require IP access direct from client to server, and may be confusing for traditional IT folks and auditors. Each client needs to be aware of the registry and have code to make calls (or use a sidecar/sidekick).
I've seen it in practice where a group started to transition their apps to a Docker environment, and they were able to run their Docker based apps along side their non-docker versions at the same time w/o having to get IT involved and do a lot of experimentation and testing quickly and autonomously.
If you have autonomous teams, are highly advanced on the devops spectrum, and have a lot of trust with your teams, Client Side routing and load balancing may be a good experience for you.