I am aware that Azure application gateway supports websockets. However, I cant figure out from the samples and documentation how websocket access is reflected in the Access Logs.
I have been going over Azure gateway documentation for Access logs over here
https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-diagnostics#for-application-gateway-and-waf-v2-sku
There is no protocol field - which usually carry ws or wss to indicate websocket access.
Thanks for help in advance
There is no user-configurable setting to selectively enable or disable WebSocket support in Application gateway. WebSocket protocols are designed to work over traditional HTTP ports of 80 and 443. You can continue using a standard HTTP listener on port 80 or 443 to receive WebSocket traffic. WebSocket traffic is then directed to the WebSocket enabled backend server using the appropriate backend pool as specified in application gateway rules.
Here is a clear documentation explaining about Websocket workflow in Application Gateway.
Related
I am trying to implement a GraphQL WebSocket-based #subscription on a server (using NestJS #subscription). The server is hosted on an AWS ECS and is behind an ALB.
We currently have an AWS API GW connection via VPC-link to our ALB.
I tried to build a dedicated Websocket API GW with the same VPC link we use in the HTTP API GW.
I also tried to spin up a new NLB (Network Load Balancer) over our ECS and a new REST VPC link to be used in the dedicated Websocket API GW.
The client and server are communicating over a graphql-transport-ws sub-protocol using graphql-ws library and the communication is working fine on a localhost setup.
When running the following command on our local host I am able to establish a web socket connection:
wscat -c ws://localhost:3000/graphql -s graphql-transport-ws
When running the same against the WebSocket API GW URL
wscat -c wss://*****.execute-api.*****.amazonaws.com/**** -s graphql-transport-ws
I’m getting this:
error: Server sent no subprotocol
The error indicates a problem with the sub-protocol so when removing the sub-protocol a connection is established and I am getting a prompt:
Connected (press CTRL+C to quit)
>
However, there’s no indication of reaching the server and it seems like the connection is only made with the WebSocket API GW itself.
When I circumvent the gateway and directly connect an internet-facing NLB I'm able to establish a WebSocket connection.
I am not a super Websocket expert, but I understand WebSocket connections will be terminated by the API Gateway and cannot be used as a connection pass-through. You can forward web socket events using AWS_PROXY integration to a graphQL server backend, BUT it's not a maintained direct connection - API Gateway terminates and events towards the backend integration and will not return the integration response to the WebSocket since it is event-driven and not a connection-oriented service - hence the “error: Server sent no subprotocol” you are seeing.
So to use API GW as the WebSocket layer, you would need to build out connection management functionality somewhere to manage the event-based nature of the APIGW and send out data to the APIGW connections or adjust the integration mechanism within the graphql server to utilise the #connection functionality to send responses/notifications to WebSocket consumers.
Integrating Backend Service documentation:
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-websocket-api-routes-integrations.html
Sending responses to a connected client:
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-connections.html
API GW Websockets are great for building custom solutions but take some effort since you will be configuring the setup for the events.
For a GraphQL API on AWS - I would recommend taking a look at AppSync, which is an AWS Managed GraphQL service - it handles GraphQL subscriptions via WebSockets natively and with zero additional code and its highly scalable out of the box and would simplify the GraphQL hosting burden of an ECS based solution.
I suspect there may be a lot of other reasons for the need to build out using existing GraphQL on ECS, so understand it's not always possible to pivot to something like AppSync. I feel the NLB solution you tried is okay within the existing ECS backend landscape and, as you have noted, is connection-oriented (via NLB), so will achieve the outcome you are after.
I am trying to add an API Gateway to a client-server system, which communicates through WebSockets.
The Server is an audio recognition engine that sits in a remote machine, and clients are connecting to it through WebSockets ('wss://...'), sending audio files and receiving their text translate representation.
My main goal is to add a free open-source API Gateway in the middle in order to authenticate\authorize\rate limit\etc (Tried Kong\Tyk\krakenD, but they are not free).
After searching the web, I focused on Apache APISIX gateway.
As a test, I managed to connect the APISIX to a client-server which communicates by http/https and it worked fine.
Moreover, I managed to connect the client with the server on regular websocket connection (ws://) and the data transactions were successful, but no matter what I do I cant connect to the secured connection. Are there any special configurations I should edit in the dashboard?
I'm wondering if anyone knows from experience if it's possible to use
the secured WebSockets with Apache APISIX and if yes, how exactly it should be done - because the docs are missing any informative info\examples, or if there are better solutions known to my problem.
The only WebSocket reference in APISIX dashboard -
What you should do is create an SSL object (you can find it in the sidebar on the left side). See https://apisix.apache.org/docs/apisix/admin-api/#ssl to learn the APISIX SSL object.
You can see an example at https://apisix.apache.org/docs/apisix/certificate
The SSL object provides the required TLS Certificate and Private Key so that this key pair can be used in a TLS handshake according to the TLS SNI sent from clients. After that, your client established a secured connection with APISIX, and now you can send the WebSocket traffic securely.
I've seen a lot of documentation and tutorials how to setup HTTPS and Websockets on heroku, but it's possible to setup another protocol like TLS or NATS?
If it's possible how can I do that?
Unfortunately, no.
Inbound requests are received by a load balancer that offers SSL
termination. From here they are passed directly to a set of routers.
The routers are responsible for determining the location of your
application’s web dynos and forwarding the HTTP request to one of
these dynos.
https://devcenter.heroku.com/articles/http-routing#routing
Not supported
TCP Routing
https://devcenter.heroku.com/articles/http-routing#not-supported
Heroku offers only http/https routing for applications hosted on it.
We are hosting an application in the preprod azure PCF environment which exposes websocket endpoints for client devices to connect to. Is there a prescribed methodology to secure the said websocket endpoint using TLS/SSL when hosted on PCF and running behind the PCF HAProxy?
I am having trouble interpreting this information, as in, are we supposed to expose port 4443 on the server and PCF shall by default pick it up to be a secure port that ensures unsecured connections cannot be established? Or does it require some configuration to be done on HAProxy?
Is there a prescribed methodology to secure the said websocket endpoint using TLS/SSL when hosted on PCF and running behind the PCF HAProxy?
A few things:
You don't need to configure certs or anything like that when deploying your app to PCF. The platform takes care of all that. In your case, it'll likely be handled by HAProxy, but it could be some other load balancer or even Gorouter depending on your platform operations team installed PCF. The net result is that TLS is first terminated before it hits your app, so you don't need to worry about it.
Your app should always force users to HTTPS. How you do this depends on the language/framework you're using, but most have some functionality for this.
This process generally works by checking to see if the incoming request was over HTTP or HTTPS. If it's HTTP, then you issue a redirect to the same URL, but over HTTPS. This is important for all apps, not just ones using WebSockets. Encrypt all the things.
Do keep in mind that you are behind one or more reverse proxies, so if you are doing this manually, you'll need to consider what's in x-forwarded-proto or x-forwarded-port, not just the upstream connection which would be Gorouter, not your client's browser.
https://docs.pivotal.io/platform/application-service/2-7/concepts/http-routing.html#http-headers
If you are forcing your user's to HTTPS (#1 above), then your users will be unable to initiate an insecure WebSocket connection to your app. Browsers like Chrome & Firefox have restrictions to prevent an insecure WebSocket connection from being made when the site was loaded over HTTPS.
You'll get a message like The operation is insecure in Firefox or Cannot connect: SecurityError: Failed to construct 'WebSocket': An insecure WebSocket connection may not be initiated from a page loaded over HTTPS. in Chrome.
I am having trouble interpreting this information, as in, are we supposed to expose port 4443 on the server and PCF shall by default pick it up to be a secure port that ensures unsecured connections cannot be established? Or does it require some configuration to be done on HAProxy?
From the application perspective, you don't do anything different. Your app is supposed to start and listen on the assigned port, i.e. what's in $PORT. This is the same for HTTP, HTTP, WS & WSS traffic. In short, as an app developer you don't need to think about this when deploying to PCF.
The only exception would be if your platform operations team uses a load balancer that does not natively support WebSockets. In this case, to work around the issue they need to separate traffic. HTTP and HTTPS go on the traditional ports 80 and 443, and they will route WebSockets on a different port. The PCF docs recommend 4443, which is where you're probably seeing that port. I can't tell you if your platform is set up this way, but if you know that you're using HAproxy, it is probably not.
https://docs.pivotal.io/platform/application-service/2-8/adminguide/supporting-websockets.html
At any rate, if you don't know just push an app and try to initiate a secure WebSocket connection over port 443 and see if it works. If it fails, try 4443 and see if that works. That or ask your platform operations team.
For what it's worth, even if your need to use port 4443 there is no difference to your application that runs on PCF. The only difference would be in your JavaScript code that initiates the WebSocket connection. It would need to know to use port 4443 instead of the default 443.
Hasura GraphQL Engine is deployed on a Cloudfoundry instance backed by AWS, it is exposed at a subdomain via an AWS ELB. The console is exposed at https://hasura.cloud.domain.com/console and the GraphQL API accepts queries at https://hasura.cloud.domain.com/v1alpha1/graphql.
But when a subscription is executed from console, an error happens with the following log on JS Console:
vendor.js:1 WebSocket connection to 'wss://hasura.cloud.domain.com/v1alpha1/graphql' failed: Error during WebSocket handshake: Unexpected response code: 200
Analyzing the websocket frames on Chrome indicates an error with (Opcode -1).
Basically, the client is unable to open a websocket connection.
Some load balancers do not support passing WebSocket handshake requests containing the Upgrade header to the CF router. For instance, the Amazon Web Services (AWS) Elastic Load Balancer (ELB) does not support this behavior. In this scenario, you must configure your load balancer to forward TCP traffic to your CF router to support WebSockets.
ref: https://docs.cloudfoundry.org/adminguide/supporting-websockets.html#config
Basically, there is some configuration required with AWS ELB and CF Router to get websockets working. This is typically done by setting up a non-standard port to forward all TCP connections to the CF Router. We have learned from our clients that this port is typically 4443.
So, to get websocket connections to work, choose the endpoint as wss://hasura.cloud.domain.com:4443/v1alpha1/graphql for websocket connections and thus subscriptions.
The console can be opened at https://hasura.cloud.domain.com:4443 as well.