I have an use case where I have services which require interaction with DynamoDB (Programming env is in golang). But assume these service doesn't have AWS credentials and I have custom AuthN/AuthZ mechanism to validate the services internally and set credentials. So, I want to write a AuthN proxy service which intercepts requests to DynamoDB, check what type of operation (Get/Set/Delete), validate them, set DDB credentials to that request, query dynamodb and send response back to clients. I tried using proxy as mentioned here in DDB documentation, but it is HTTP Connect tunnelling and I couldn't intercept traffic in between as it is HTTPS traffic to DynamoDB. Can someone tell me how I can achieve this using AWS Go sdk library?
Thanks in advance.
Related
Does anyone know if it is possible to have a Websocket Api Gateway support mTLS authentication? According to the documentation it is not supported (https://aws.amazon.com/blogs/compute/evaluating-access-control-methods-to-secure-amazon-api-gateway-apis/). Does anyone know if there is a feature request for this or there is a workaround? We've debated using a lambda authorizer on the $connect route on the websocket API to authenticate client certificates manually, but it seems like an artificial implementation for mTLS since we're performing the client authentication after the TLS handshake. Not sure if this is our best option or if there is something better. Thanks.
I am trying to implement a GraphQL WebSocket-based #subscription on a server (using NestJS #subscription). The server is hosted on an AWS ECS and is behind an ALB.
We currently have an AWS API GW connection via VPC-link to our ALB.
I tried to build a dedicated Websocket API GW with the same VPC link we use in the HTTP API GW.
I also tried to spin up a new NLB (Network Load Balancer) over our ECS and a new REST VPC link to be used in the dedicated Websocket API GW.
The client and server are communicating over a graphql-transport-ws sub-protocol using graphql-ws library and the communication is working fine on a localhost setup.
When running the following command on our local host I am able to establish a web socket connection:
wscat -c ws://localhost:3000/graphql -s graphql-transport-ws
When running the same against the WebSocket API GW URL
wscat -c wss://*****.execute-api.*****.amazonaws.com/**** -s graphql-transport-ws
I’m getting this:
error: Server sent no subprotocol
The error indicates a problem with the sub-protocol so when removing the sub-protocol a connection is established and I am getting a prompt:
Connected (press CTRL+C to quit)
>
However, there’s no indication of reaching the server and it seems like the connection is only made with the WebSocket API GW itself.
When I circumvent the gateway and directly connect an internet-facing NLB I'm able to establish a WebSocket connection.
I am not a super Websocket expert, but I understand WebSocket connections will be terminated by the API Gateway and cannot be used as a connection pass-through. You can forward web socket events using AWS_PROXY integration to a graphQL server backend, BUT it's not a maintained direct connection - API Gateway terminates and events towards the backend integration and will not return the integration response to the WebSocket since it is event-driven and not a connection-oriented service - hence the “error: Server sent no subprotocol” you are seeing.
So to use API GW as the WebSocket layer, you would need to build out connection management functionality somewhere to manage the event-based nature of the APIGW and send out data to the APIGW connections or adjust the integration mechanism within the graphql server to utilise the #connection functionality to send responses/notifications to WebSocket consumers.
Integrating Backend Service documentation:
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-websocket-api-routes-integrations.html
Sending responses to a connected client:
https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-how-to-call-websocket-api-connections.html
API GW Websockets are great for building custom solutions but take some effort since you will be configuring the setup for the events.
For a GraphQL API on AWS - I would recommend taking a look at AppSync, which is an AWS Managed GraphQL service - it handles GraphQL subscriptions via WebSockets natively and with zero additional code and its highly scalable out of the box and would simplify the GraphQL hosting burden of an ECS based solution.
I suspect there may be a lot of other reasons for the need to build out using existing GraphQL on ECS, so understand it's not always possible to pivot to something like AppSync. I feel the NLB solution you tried is okay within the existing ECS backend landscape and, as you have noted, is connection-oriented (via NLB), so will achieve the outcome you are after.
I have a react application and I would like to setup a websocket connection to my backend for some realtime updates. I was going to deploy an EC2 or ECS-cluster to host websocket connections. Then I stumbled into some articles showing how websocket connection can be setup in a serverless manner.
One example: https://medium.com/#likhita507/real-time-chat-application-using-webscockets-in-apigateway-e3ed759c4740
However, I can't seem to figure out how this works for a few reasons.
Lambda has a max runtime of 15 min
How does the backend establish a connection when no lambda is running and the backend wants to invoke a message to the frontend
Does this entail that I have to keep a lambda alive all the time, if so, it no longer feels like a good idea. In the above example, what I can't grasp is that when creating that chat application, can each chat room only exist for 15 min? And if a user disconnects from the room, how will that user be updated on new messages.
Does anyone have any experience with this kind of solution?
It's the API Gateway that keeps the websocket connection alive. The browser (or whatever your client is) is connecting to the Gateway, not the lambda function.
The gateway triggers the Lambda function. You hook this up by selecting LAMBDA_PROXY from Integration Request. You can connect each route to a separate function, or have them all dealt with by one, whichever you prefer. Unless you're doing something very complicated in the function, it should only be executing for a few ms.
Communicating from the function to the original client is done through the gateway too - with APIGatewayManagementAPI.postToConnection (or you could roll your own http version using the connection URL I guess).
I have a micro service architecture which is working with Spring Zuul Gateway like below image.
My authentication service returns x-auth-token which is generated by spring authentication resolver and my token repository is redis. So users should use this service to authenticate and then use other services.
All my other services connect to same redis instance, so when they receive x-auth-token they can get user session details. I normally do the authorization by using #PreAuthorize annotation and then specifying the roles that can have access to controller or method.
Everything was so far working fine. Then I have been asked to add rate-limit functionality to this architecture. So for example a single user should not be able to make more than 1 POST request to specific api in books service. Also, if there were two book service instances, I would want to both be counted as single service when its about rate limiting.
I found tons of documents that referred me to this project called spring-cloud-zuul-ratelimit. Looking at the document I realized it does support redis as storage (good for me because I already have redis there) and it also supports handling rate limits per users.
The problem is that my zuul gateway knows nothing about the users! It has no access to redis storage. If I give it an access to redis, the problem might be solved but another one would rise: I'm gonna need to authorize user twice which takes more time and more redis traffic! once at gateway, once at each service (to check the roles and session details).
I'm looking for solutions that are most close to this list of needs:
Does not change my authentication method (I cant just switch to JWT or OAuth)
Does not duplicate authorization or redis queries
Balancing the requests between my services should not effect the rate limit. If each instance of service X is requested once for single user, then user has sent two requests.
Hopefully there is a good spring support for the answer.
I would prefer to be able to change the limits dynamically.
Zuul gateway rate limiter plugin basically tracks counter of user request based
on specific key (could be user's IP, some ID, request path or custom
combinations using custom key generator) given user requests during time interval. You can add it to the
existing zuul gateway application.
Let's say the ratelimiter-gateway is using "[clientIP][userID][method][path]" as request counter key stored in redis, e.g. "10.8.14.58:some#mail.com:POST:/books".
Here's some options I can think of:
If the client send some ID, you can use it directly as rate limiter
combination key.
If the user only send JWT token, you can verify it's claim to get the user
ID, assuming it's embedded in the token, using the same secret key to generate the JWT token in authn service
as Zuul gateway app properties (using OS env credentials, vault etc.). Or
you can just use the token as user ID.
Move the authorization logic to Spring zuul+ratelimiter service. It will
validate incoming request to author & books service, get the user ID from token. And then pass it as
another header, ex: "x-app-user-id", to the upstream services using
SpringBoot Filter. This way, the upstream services won't do any authn logic, it's just read the user id from
header. Communication between author & books service might be using the same header. This, of course, assuming the upstream servers won't be accessed
directly from the outside network.
It might also be good idea to use different redis instance as the ratelimit key storage.
As for dynamic config, based on it's documentation, you can adjust the rate limit config via
properties. I don't know if it can be adjusted dynamically at runtime via
Spring Cloud Config or other remote config implementations without the gateway
app needs to be restarted.
I'm trying to integrate an openFire XMPP server to my current company Spring server but have two major questions I cannot find the answer to -
I'll start with my current architecture first -
1. The xmpp server have a DB-server of it's own seperated from the Spring server DB, This is a dedicated machine to keep the users char history etc
2. The spring server have a DB of it's own where it keeps the user credentials (md5 encrypted) and also client applications data
3. The spring server is dedicated to serve HTTP requests (a dedicated REST server)
All in all I have 2 DB servers once chat server and one Rest server
Now for the questions -
1. Can I forbid registration to the xmpp server (i.e. whitelist the rest server ip and let it be the only one who can create users after a user registers on it)?
2.For security reasons the Rest server switch the session for a logged in user every 2 days the iOS and Android clients deal with session managment locally - How can I use those session with the XMPP server?
To clarify - I want the users to be able use the xmpp server only for chat purposes but only after they logged in to the application itself since the user session may expire the chat client will also have to re-authenticate against the REST server, how can I achieve this?
3. Won't it create an overload on the REST server? (i.e. the Rest server will now have to handle client requests and also XMPP server requests)
4. What is the best architecture to achieve this kind of a system (chat server, db server for chat server, rest server, db server for rest server) so that the system can scale horizontally?
I searched google for an article or something related to describe the general architecture but couldn't find nothing relevant, since I'm not "inveneting the wheel" here I would love to hear a good advice or be directed to an article that explains the How-To's
Thanks in advance.
The standard way in XMPP world for user authentication is SASL.
SASL have a very simple model: server sends to client some "challenge" string to client, and client sends "response" string to server, and they repeat this until server decides client send all required data. What data to send is defined in SASL "mechanism". There are number of well-known SASL mechanisms, e.g. SCRAM, and they are provided by most XMPP servers and clients "out of the box".
Your problem is - you already have authentication system and user database and want to reuse it for chat purposes. There are two ways:
Add your custom REST authentication as SASL module to your server. Google say it is already possible to write and add Openfire SASL plugin. Your SASL REST mechanism will do the same things as for browser, but required urls, tokens, etc. will be wrapped as "challenges" and "responses", e.g. server will send REST auth url as "challenge" for client, and client will open url, post credentials, get a token and send them as "response" back to server. Of course you need to add this SASL REST mechanism in client too.
Adopt your XMPP server to use your authentication database directly. In this case you only need to modify Openfire code to link it with your users/passwords tables (maybe there is already an admin tool for this). In this case clients will continue to use standard SASL mechanisms without modification. When this way may be easier than first one, remember your XMPP server should have access to plain-text passwords, which may be insecure.
You questions in order:
Yes, you can disable registration from XMPP client and point users to registration website.
You will see chat sessions in Openfire administration console and able to stop them, also you can write a module for do this by your schedule
If you will write SASL REST mechanism, there will no any difference between requests from chat clients and web clients for your REST backend, they will look the same.
As I described first, you no need separate DB for chat server and you able to setup multiple chat servers connected to your REST backend.