I am learning AWS lambda and have a basic question regarding architecture with respect to managing https calls from multiple lambda functions to a single external service.
The external service will only process 3 requests per second from any IP address. Since I have multiple asynchronous lambdas I cannot be sure I will be below this threshold. I also don't know what IPs my lambdas use or even if they are the same or not.
How should this be managed?
I was thinking of using an SQS FIFO queue, but I would need to setup a bidirectional system to get the call responses back to the appropriate lambda. I think there must be a simple solution to this, but I'm just not familiar enough yet.
What would you experts suggest?
If I am understanding your question correctly then
You can create and API Endpoint by build an API Gateway with Lambda integrations(preferred Lambda proxy integration) and then use throttling option to decide the throughput this can be done in different ways aws docs account level, method level etc.
You can perform some load testing using gatling or any other tool and then generate a report for eg. which can show that even if you have say 6tps on your site you can throttle at method level and see that the external service is hit only at say 3tps.
It would depend upon your architecture how do you want to throttle I had done method level to protect the external service at 8tps.
Related
I have a use case where an API backed by a lambda has to be latency critical for a few clients but there are clients how call the API with high volume in bursts and the latency restrictions are liberal .
We are using provisioned concurrency for the latency critical calls and do not want to use it for non latency critical calls as the cost is high.
Since provisioned concurrency can only be used with alias/version, is it possible to choose the lambda version at runtime based on the API Key?
Determine the client based on the API Key and point to the appropriate version. I am trying to avoid creating 2 API endpoints one for latency critical clients and the other for non-latency critical clients.
It is not possible for API Gateway to invoke a Lambda function alias based on the API key passed in the request. What you can do is set up 2 API Gateway stages, one for latency critical calls and the other for non-critical ones. Now, the Lambda function integration would need to be set up to use API GW stage variables so the appropriate Lambda function alias can be invoked based on the stage. You can refer to this blog post on how to configure that: https://docs.aws.amazon.com/apigateway/latest/developerguide/stages.html
So, using this method, you would be creating two endpoints, but the API configuration for both would be similar.
I didn't find any posts related to this topic. It seems natural to use Lambda as a getstream backend, but I'm not sure if it heavily depends on persistent connections or other architectural choices that would rule it out. Is it a sensible approach? Has anyone made it work? Any advice?
While you can build an entire website only in Lambda, you have to consider the followings:
Lambda behind API Gateway has a timeout limit of 30 seconds and a Payload size limit (both received and sended) of 6MB. While for most of the cases this is fine, if you have some really big operations or you need to send some really big datas (like a high resolution image), you can't do it with this approach, but you need to think about something else (for instance you can send an SNS to another Lambda function with higher timeout that can do all this asynchronously and then send the result to the client when it's done, supposing the client is capable of receiving events)
Lambda has cold starts, which in terms slow down your APIs when a client calls them for the first time in a while. The cold start time depends on the language you are doing your Lambdas, so you might consider this too. If you are using C# or Java for your Lambdas, than this is probably not the best choice. From this point of view, Node.JS and Python seems to be the best choices, with Golang rising. You can find more about it here. And by the way, you can now specify a Provisioned Throughput for your Lambda, which aims to fix the cold start issue, but I haven't used it yet so I can't tell if there is any difference (but I'm sure there is)
If done correctly you'll end up managing hundreds of Lambda functions, while with a standard Docker Container under ECS you'll manage few APIs with multiple endpoints. This point should not be underestimated, as on one side it will make changes easier in the future, since lambda will be small and you'll easily find the bug and fix it, but on the other side you have to move across these functions, which if you don't know exactly which lambda is responsible of what can be a long process
Lambda can't handle sessions as far as I know. Because after some time the Lambda container gets dropped, you can't store any session inside the Lambda itself. You'll always need a structure to store the session so it can be shared across multiple Lambda invocations, such as some records in a DynamoDB table or something else, but this mean that you have to write the code for this, while in a classic API (like a .NET Core one) all of this is handled by the language itself and you only need to store or retrieve items from the session (most of the times)
So... yeah! A backed written entirely in Lambda is possible. The company I work in does it and I must say is a lot better, both in terms of speed and development time. But those benefits comes later, since you need to face all of the reasons I listed above before, and is not as easy as it could seem
Yes, you can use AWS Lambda as backend and integrate with Stream API there.
Building an entire application on Lambda directly is going to be very complex and requires writing lot of boiler plate code just to enforce some basic organization and structure to your project.
My recommendation is use a serverless framework to do this that takes care of keeping your application well organized and to deploy new versions (and environments).
Serverless is a good option for that: https://serverless.com/framework/docs/providers/aws/guide/intro/
I am considering to use the AWS lambda serverless architecture for my next project. This is my understanding of the technology and I would very much appreciate it if somebody can correct me.
You can deploy function that acts as the event handlers.
The event handlers are configured to respond to any events that are provided
In the case of writing the lambda functions in Javascript, you can require any other Javascript modules you write and use them.
All your lambda and its required modules are written stateless. Your app's states are ultimately kept in the database.
If you ever want to write some stateful logic such as keeping the results from one HTTP request and temporarily store it somewhere and look it up in the subsequent request, is this not possible in Lambda?
About your question, lambdas can use a temporal directory /tmp to storage files. This has a limitation of 500MB. Since the lambda container COULD be reused for performance, there is a chance that the file is still there for the next lambda invocation. This is discouraged but in some particular cases could be helpful. Anyway, if you really need it, the better approach would be to use a cache system.
In addition to your considerations, AWS Lambdas are not good for:
To keep state, like files that are downloaded and could be reused later.
Handle OS
Long running tasks
Hard latency requirements apps.
Depending on the database client, multiple concurrent lambdas can lead to an overhead in the database connections since a client is instantiated for each lambda.
I am building out a microservice architecture and kinda confused on one part. I am using Kafka as a message broker system to communicate within my services. A perfect example would be Uber's API for request estimation. It returns duration, distance, price, etc. I would assume they have a microservice for each of those, i.e. service for pricing, service for duration/distance, service for drivers, etc. My question is when hitting the endpoint /requests/estimate does the requests microservice make rest calls to the other microservices to retrieve data for the duration, distance, etc? or does the API Gateway take care of that?
I say it depends on the use case. If service A needs to know what service B knows, then it is perfectly sane for service A to make a REST call to service B. But if the combined knowledge for A and B is only needed in your gateway then the gateway can combine the results.
Both are perfectly valid ways of doing it, but I would go the Estimate microservice way to avoid putting too much logic in the API Gateway.
Maybe in the future your estimation calculation will change, and it wouldn't make much sense to me to update the gateway every time.
In practice, not all gateway APIs support multiple calls and aggregation. In the micro service architecture, there is a common pattern ("API Composition", "Composition Patterns" in particular "Aggregator Pattern"), the idea of which is that you need to make a separate service that will contain the business logic of multiple calls and aggregation.
Say we have following taxi-hailing application that is composed of loosely coupled microservices:
The example is taken from https://www.nginx.com/blog/introduction-to-microservices/
Each services has its own rest api and all services are combined in a single api gateway. The client does not talk to a single service but to the gateway. The gateway requests information from several services and combines them to a single response. For the client it looks like it is talking to a monolithic application.
I am trying to understand: where could we incorporate falcor into this application?
One Model Everywhere from http://netflix.github.io/falcor/
Falcor lets you represent all your remote data sources as a single
domain model via a virtual JSON graph. You code the same way no matter
where the data is, whether in memory on the client or over the network
on the server.
In this taxi-hailing application each microservice represents a single domain model already. Can you think of any benefit we could thrive by wrapping each microservice with falcor? I cannot.
However I think it is very convenient to incorporate falcor into the api gateway because we can abstract away the different domain models created by the microservices into one single or at least a few models.
What is your opinion?
You are right. This is how Netflix uses Falcor and what the Falcor router is designed for.
From the documentation:
The Router is appropriate as an abstraction over a service layer or REST API. Using a Router over these types of APIs provides just enough flexibility to avoid client round-trips without introducing heavy-weight abstractions. Service-oriented architectures are common in systems that are designed for scalability. These systems typically store data in different data sources and expose them through a variety of different services. For example, Netflix uses a Router in front of its Microservice architecture.
It is rarely ideal to use a Router to directly access a single SQL Database. Applications that use a single SQL store often attempt to build one SQL Query for every server request. Routers work by splitting up requests for different sections of the JSON Graph into separate handlers and sending individual requests to services to retrieve the requested data. As a consequence, individual Router handlers rarely have sufficient context to produce a single optimized SQL query. We are currently exploring different options for supporting this type of data access pattern with Falcor in future.
Falcor is really a great api if it is used in the correct way for very relevant use cases, like :
If your page has to make multiple REST end point calls
These calls don't depend on each other
All the REST calls happens on initial page load
Performance : If you want to cache the REST responses (for example, the microservice uses gemfire caching, you may not need falcor cache. You could still use falcor caching if you want to reduce the network latency)
Server requests batching : When running Falcor in node environment, you may want to cut down the amount of calls to node server from the client side.
Easier response parsing : If you don't want the client code to worry about extracting the data-points from REST response (Including error handling)
and so on ..
However, there are plenty of situations where falcor does not serve the purpose as much and feel that it is better off calling the end point directly :
If REST calls are dependent on one another
If you want to pass lot of parameters for calling the end point
If you don't intend to cache the response(s)
If you want to share some secure cookies (ex:XSRF tokens) with the REST web service