I have a react application and I would like to setup a websocket connection to my backend for some realtime updates. I was going to deploy an EC2 or ECS-cluster to host websocket connections. Then I stumbled into some articles showing how websocket connection can be setup in a serverless manner.
One example: https://medium.com/#likhita507/real-time-chat-application-using-webscockets-in-apigateway-e3ed759c4740
However, I can't seem to figure out how this works for a few reasons.
Lambda has a max runtime of 15 min
How does the backend establish a connection when no lambda is running and the backend wants to invoke a message to the frontend
Does this entail that I have to keep a lambda alive all the time, if so, it no longer feels like a good idea. In the above example, what I can't grasp is that when creating that chat application, can each chat room only exist for 15 min? And if a user disconnects from the room, how will that user be updated on new messages.
Does anyone have any experience with this kind of solution?
It's the API Gateway that keeps the websocket connection alive. The browser (or whatever your client is) is connecting to the Gateway, not the lambda function.
The gateway triggers the Lambda function. You hook this up by selecting LAMBDA_PROXY from Integration Request. You can connect each route to a separate function, or have them all dealt with by one, whichever you prefer. Unless you're doing something very complicated in the function, it should only be executing for a few ms.
Communicating from the function to the original client is done through the gateway too - with APIGatewayManagementAPI.postToConnection (or you could roll your own http version using the connection URL I guess).
Related
I am trying to implement a GraphQL WebSocket-based #subscription on a server (using NestJS #subscription). The server is hosted on an AWS ECS and is behind an ALB.
We currently have an AWS API GW connection via VPC-link to our ALB.
I tried to build a dedicated Websocket API GW with the same VPC link we use in the HTTP API GW.
I also tried to spin up a new NLB (Network Load Balancer) over our ECS and a new REST VPC link to be used in the dedicated Websocket API GW.
The client and server are communicating over a graphql-transport-ws sub-protocol using graphql-ws library and the communication is working fine on a localhost setup.
When running the following command on our local host I am able to establish a web socket connection:
wscat -c ws://localhost:3000/graphql -s graphql-transport-ws
When running the same against the WebSocket API GW URL
wscat -c wss://*****.execute-api.*****.amazonaws.com/**** -s graphql-transport-ws
I’m getting this:
error: Server sent no subprotocol
The error indicates a problem with the sub-protocol so when removing the sub-protocol a connection is established and I am getting a prompt:
Connected (press CTRL+C to quit)
>
However, there’s no indication of reaching the server and it seems like the connection is only made with the WebSocket API GW itself.
When I circumvent the gateway and directly connect an internet-facing NLB I'm able to establish a WebSocket connection.
I am not a super Websocket expert, but I understand WebSocket connections will be terminated by the API Gateway and cannot be used as a connection pass-through. You can forward web socket events using AWS_PROXY integration to a graphQL server backend, BUT it's not a maintained direct connection - API Gateway terminates and events towards the backend integration and will not return the integration response to the WebSocket since it is event-driven and not a connection-oriented service - hence the “error: Server sent no subprotocol” you are seeing.
So to use API GW as the WebSocket layer, you would need to build out connection management functionality somewhere to manage the event-based nature of the APIGW and send out data to the APIGW connections or adjust the integration mechanism within the graphql server to utilise the #connection functionality to send responses/notifications to WebSocket consumers.
Integrating Backend Service documentation:
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-websocket-api-routes-integrations.html
Sending responses to a connected client:
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-connections.html
API GW Websockets are great for building custom solutions but take some effort since you will be configuring the setup for the events.
For a GraphQL API on AWS - I would recommend taking a look at AppSync, which is an AWS Managed GraphQL service - it handles GraphQL subscriptions via WebSockets natively and with zero additional code and its highly scalable out of the box and would simplify the GraphQL hosting burden of an ECS based solution.
I suspect there may be a lot of other reasons for the need to build out using existing GraphQL on ECS, so understand it's not always possible to pivot to something like AppSync. I feel the NLB solution you tried is okay within the existing ECS backend landscape and, as you have noted, is connection-oriented (via NLB), so will achieve the outcome you are after.
I have exposed a websocket enabled service endpoint through Azure Application Gateway and the service is hosted on azure service fabric. Client initiates a websocket connection with my endpoint and is able to exchange data. During certain message flows, my Web Socket enabled service calls other services hosted on the service fabric using azure service bus. These are handled in a completely async manner. Once the other services finish processing, they post a message to the service bus which my WebSocket service reads back.
The problem I am having is to route the messages back to the right service fabric node so that it can be pushed back to the client at the other end of the WebSocket connection
In the picture below, you can imagine each node containing multiple services including the web socket enabled service. Once the Websocket service posts a message to the service bus, the downstream services start processing and finally they post a message back to the service bus which the websocket service reads back. Here a random node will pick up the message and it might not have the relevent websocket connection to push the processed data back
Sample Design
I have looked at redis pubsub model and it looks like I have to maintain last message processed on the nodes. It also means, every node on the cluster will need to read the message and discard it if they don't have the websocket connection with the client. I am looking for any suggested design models for this kind of problem
I ran into a similar scenario and didn't like the idea of using a new external service (Redis/SQL Server) as a backplane that would simply duplicate each message/event across all nodes.
The solution I settled on was to lean on a property of actor proxies, using actor events to call-back to a specific instance of a stateless service. Creating an actor service to act as a pub/sub backplane.
The solution is summarised in this blog post and this GitHub repo. It's worth pointing out that the documentation states actor events are best effort. This hasn't really been an issue when the application is running as normal, I presume that during a deployment or failover, some events may get lost, however this could be mitigated with additional work.
It's also worth noting that your load balancing rules should maintain sticky connections between clients and back-end instances. You could create separate rules for websockets if you only wanted this to apply to them and not your regular HTTP traffic.
Whereas the corporate environment I am working in accepts the use of http(s) based request response patterns, which is OK for GraphQL Query and Mutation, they have issues with the use of websockets as needed for GraphQL Subscription and would prefer that the subscription is routed via IBM MQ.
Does anyone have any experience with this? I am thinking of using Apollo Server to serve up the GraphQL interface. Perhaps there is a front-end subscription solution that can be plugged in using IBM MQ? The back end data sources are Oracle databases.
Message queues are usually used to communicate between services while web sockets are how browsers can communicate with the server over a constant socket. This allows the server to send data to the client when a new event of a subscription arrived (classically browsers only supported "pull" and could only receive data when they asked for it). Browsers don't implement the MQ protocols you would need to directly subscribe to the MQ itself. I am not an expert on MQs but what is usually done is there is a subscription server that connects to the client via web socket. The subscription service then itself subscribes to the message queue and notifies relevant clients about their subscribed events. You can easily scale the subscription servers horizontally when you need additional resources.
I am thinking about using a gRPC service to facilitate notifications between two services. (as an aside, I will be using protobuf-net/ protobuf-net.Grpc) The intent is that the client service would establish and maintain a connection to the server service, and react to notifications over time. In an perfect technology world where there are no network blips, no server restarts, etc the idea would be to establish this connection once and have that server streaming call live for the lifetime of the application. Obviously in the real world we need to deal with retries, reconnects, fail-overs etc.
My question is: Is calling a server streaming call in grpc and keeping the call open for long periods of time an appropriate use of server streaming calls, or is it an abuse of that feature?
This is a perfectly fine use case for gRPC. gRPC is designed for this kind of use.
Yes, you have to deal with reconnections or more exactly reestablishment of streams when the connection to the server dies.
I have an API running on a server, which handle users connection and a messaging system.
Beside that, I launched a websocket on that same server, waiting for connections and stuff.
And let's say we can get access to this by an Android app.
I'm having troubles to figure out what I should do now, here are my thoughts:
1 - When a user connect to the app, the API connect to the websocket. We allow the Android app only to listen on this socket to get new messages. When the user want to answer, the Android app send a message to the API. The API writes itself the received message to the socket, which will be read back by the Android app used by another user.
This way, the API can store the message in database before writing it in the socket.
2- The API does not connect to the websocket in any way. The Android app listen and write to the websocket when needed, and should, when writing to the websocket, also send a request to the API so it can store the message in DB.
May be none of the above is correct, please let me know
EDIT
I already understood why I should use a websocket, seems like it's the best way to have this "real time" system (when getting a new message for example) instead of forcing the client to make an HTTP request every x seconds to check if there are new messages.
What I still don't understand, is how it is suppose to communicate with my database. Sorry if my example is not clear, but I'll try to keep going with it :
My messaging system need to store all messages in my API database, to have some kind of historic of the conversation.
But it seems like a websocket must be running separately from the API, I mean it's another program right? Because it's not for HTTP requests
So should the API also listen to this websocket to catch new messages and store them?
You really have not described what the requirements are for your application so it's hard for us to directly advise what your app should do. You really shouldn't start out your analysis by saying that you have a webSocket and you're trying to figure out what to do with it. Instead, lay out the requirements of your app and figure out what technology will best meet those requirements.
Since your requirements are not clear, I'll talk about what a webSocket is best used for and what more traditional http requests are best used for.
Here are some characteristics of a webSocket:
It's designed to be continuously connected over some longer duration of time (much longer than the duration of one exchange between client and server).
The connection is typically made from a client to a server.
Once the connection is established, then data can be sent in either direction from client to server or from server to client at any time. This is a huge difference from a typical http request where data can only be requested by the client - with an http request the server can not initiate the sending of data to the client.
A webSocket is not a request/response architecture by default. In fact to make it work like request/response requires building a layer on top of the webSocket protocol so you can tell which response goes with which request. http is natively request/response.
Because a webSocket is designed to be continuously connected (or at least connected for some duration of time), it works very well (and with lower overhead) for situations where there is frequent communication between the two endpoints. The connection is already established and data can just be sent without any connection establishment overhead. In addition, the overhead per message is typically smaller with a webSocket than with http.
So, here are a couple typical reasons why you might choose one over the other.
If you need to be able to send data from server to client without having the client regular poll for new data, then a webSocket is very well designed for that and http cannot do that.
If you are frequently sending lots of small bits of data (for example, a temperature probe sending the current temperature every 10 seconds), then a webSocket will incur less network and server overhead than initiating a new http request for every new piece of data.
If you don't have either of the above situations, then you may not have any real need for a webSocket and an http request/response model may just be simpler.
If you really need request/response where a specific response is tied to a specific request, then that is built into http and is not a built-in feature of webSockets.
You may also find these other posts useful:
What are the pitfalls of using Websockets in place of RESTful HTTP?
What's the difference between WebSocket and plain socket communication?
Push notification | is websocket mandatory?
How does WebSockets server architecture work?
Response to Your Edit
But it seems like a websocket must be running separately from the API,
I mean it's another program right? Because it's not for HTTP requests
The same process that supports your API can also be serving the webSocket connections. Thus, when you get incoming data on the webSocket, you can just write it directly to the database the same way the API would access the database. So, NO the webSocket server does not have to be a separate program or process.
So should the API also listen to this websocket to catch new messages
and store them?
No, I don't think so. Only one process can be listening to a set of incoming webSocket connections.