I created a new lambda function but do not see cloudfront as an option in the Triggers. Does anybody know why that might be? Thanks
As per AWS current documentation:
Make sure that you’re in the US-East-1 (N. Virginia) Region. You must
be in this Region to create Lambda#Edge functions.
See: AWS Tutorial: Creating a Simple Lambda#Edge Function
You cannot add from Lambda console. For adding trigger for cache behavior, you need to do it from CloudFront console.
Its is explained in detail here - https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-add-triggers-cf-console.html
CloudFront's Lambda#Edge integration feature requires that the functions be written in Node.js. It isn't possible to trigger a function in another language directly from CloudFront.
You must create functions with the nodejs6.10 or nodejs8.10 runtime property.
https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-requirements-limits.html#lambda-requirements-lambda-function-configuration
Of course, in the Node.js runtime environment, you have the AWS Javascript SDK available, so if you had a really compelling case, you could use that from a Javascript function to invoke another, different Lambda function written in a different language... but it's difficult to imagine a common case where that would make sense, because of the added latency and cost, but I have, for example, used this solution to allow Lambda#Edge to reach inside a VPC -- which can only be done by invoking a second Lambda function (which could be configured to have VPC access) from inside the first (which can't, because Lambda#Edge functions run in the region nearest to the viewer, rather than in a single region, so they will not run inside a VPC).
Related
We are moving our infrastructure to cloud formation since it's much easier to describe the infrastructure in a nice manner. This works fantastically well for things like security groups, routing, VPCs, transit gateways.
However, we have two issues which we are struggling with and I don't think fit the declarative, infrastructure-as-code paradigm which things like terrafrom and cloud formation are.
(1) We have a business requirement where we run a scheduled batch at specific times in the day. These are very computationally intensive. To save costs, we run these on an EC2 which is brought up at that time, then torn down when the batch is finished. However, this seems to require a temporary change to the terraform/CF files, then a change back. Is there a more native way of doing this?
(2) We dynamically store and allow to be edited by clients their firewalling rules on their load balancer (ALB). This information cannot be stored in the terraform/CF files since it can be changed by clients on demand.
Is there a way of properly doing these things in CF/Terraform?
(1) If you have to use EC2, you could create a Lambda that would start your EC2 instances. Then, create a CloudWatch Event that triggers the Lambda at your specified date / time. For more details you can see https://aws.amazon.com/premiumsupport/knowledge-center/start-stop-lambda-cloudwatch/. Once the job is done, have your EC2 shut itself down using the awssdk or awscli.
Alternatively, you could use AWS Lambda to run your batch job. You only get charged when the Lambda runs. Likewise, create a CloudWatch Event rule that schedules the Lambda.
(2) You could store the firewall rules in your own DB and modify the actual ALB SG rules using the awssdk. I don't think it's a good idea to store these things in Terraform/CF. IMHO Terraform/CF are great for declaring infrastructure but won't be a good solution for resources that are dynamically changing, especially by third parties like your clients.
I would like to know which is the better option amongst these two techniques:
Writing a function in EC2 instance using AWS CLI
Writing a function using AWS Lambda
I prefer writing code in AWS Lambda, but I would like to know if there are any specific advantages on using Lambda.
P.S: Those functions that I have to execute are almost the same(they utilize the same algorithm), so there is no difference in the functionality.
Regards
I understand that you would like to know if there is any particular advantage of using AWS Lambda instead of using EC2 Instances.
Here are some advantages of AWS Lambda:
Reduced cost. Lambda follows a pay-as-you-invoke pricing model, unlike AWS EC2, and the first million invokes fall under the free tier category[1]. Depending on your use case, you might be able to save a lot using AWS Lambda on your production environment.
No system administration payload. AWS Lambda follows serverless computing paradigms, and there's no need to start servers, configure them as per your need, and maintain them.
An AWS Lambda function can be pretty convenient for automation tasks and can be triggered by a number of services[2]. Eg: If you upload a file to an AWS S3 bucket, you could choose to trigger a Lambda function that compresses the file and stores it in another S3 bucket.
However, Lambda has some disadvantages as well, compared to EC2/ECS:
Lambda functions are prone to the Cold Start issue. A Cold Start issue usually occurs when a Lambda function hasn't been invoked for quite some time. AWS deploys a new container to the Lambda function in the backend, and there might be delayed invocations at times[3].
It can get arduous to debug AWS Lambda function logs and metrics in Amazon CloudWatch.
A Lambda function has a supported maximum execution time of 15 minutes, and there is a time period limitation. Therefore, it might not be possible to use a Lambda function for time-consuming operations(eg: Processing large flat files).
Amazon EC2 has a system administration payload and it might cost a bit higher, but there are no Lambda Cold Start issues, and it can even work for long running tasks.
Therefore, you could choose to use EC2 or Lambda depending on your exact use-case.
I hope this answer helps you out.
References
[1]. https://aws.amazon.com/lambda/pricing/
[2]. https://docs.aws.amazon.com/lambda/latest/dg/invoking-lambda-function.html
[3]. https://docs.aws.amazon.com/lambda/latest/dg/running-lambda-code.html
AWS Lambda run as the stateless service, means, we can't store files inside the function. We built the whole application with the 60 lambda functions. Out of 60, 54 lambda functions are triggered by API gateway remaining are act as the service modules (means, called by another lambda function).
If you use lambda function as the microservice, you can gain more in the performance and price aspects.
My suggestion: Don't create a single lambda function to run the whole system. Go with microservices method.
Looking for some advice on the best approach to launch a Lambda immediately after a cloudformation template stack gets run.
The use case is: I run my cloudformation template that setups up An API Gateway with Authorizers, and lambda functions. I then need to get reach out to another SAS/Cloud Service and run some configurations on their side to get everything dialed in.
I was looking into if there is a way to create a Lambda Function with an CloudWatch equivalent of Now, so it will launch immediately and do any finishing work that might need to be done.
I don't imagine I am the only person who might need to do this type of configuration. I could use multiple Cloudformation Scripts, but I'm trying to simply this for others who might not have much AWS experience, and Automate as much as possible. The more I can do in Amazon, the less scripting someone else might have to do.
If anyone has any ideas, or I missed something basic stuff, please let me know.
You can take a look at AWS CloudFormation Custom Resources. From the docs:
Custom resources provide a way for you to write custom provisioning logic in AWS CloudFormation template and have AWS CloudFormation run it during a stack operation, such as when you create, update or delete a stack.
With this custom resource you can call a Lambda function by handing over its ARN to the custom resource. See also this doc article for more background information.
Note: I haven't tried it by myself, but it seems to be the right thing for you.
I am considering to use the AWS lambda serverless architecture for my next project. This is my understanding of the technology and I would very much appreciate it if somebody can correct me.
You can deploy function that acts as the event handlers.
The event handlers are configured to respond to any events that are provided
In the case of writing the lambda functions in Javascript, you can require any other Javascript modules you write and use them.
All your lambda and its required modules are written stateless. Your app's states are ultimately kept in the database.
If you ever want to write some stateful logic such as keeping the results from one HTTP request and temporarily store it somewhere and look it up in the subsequent request, is this not possible in Lambda?
About your question, lambdas can use a temporal directory /tmp to storage files. This has a limitation of 500MB. Since the lambda container COULD be reused for performance, there is a chance that the file is still there for the next lambda invocation. This is discouraged but in some particular cases could be helpful. Anyway, if you really need it, the better approach would be to use a cache system.
In addition to your considerations, AWS Lambdas are not good for:
To keep state, like files that are downloaded and could be reused later.
Handle OS
Long running tasks
Hard latency requirements apps.
Depending on the database client, multiple concurrent lambdas can lead to an overhead in the database connections since a client is instantiated for each lambda.
Can someone explain how AWS lambda would know about any custom classes or methods I have written? Do you need to copy them into the zip file?
Basically I am outsourcing part of my server to Lambda where I have custom classes and methods.
Thanks
Just include the python files with the additional classes in your zip file and then you can do a regular import foo from your lambda. This is handy if you have a custom library that you need to use in several lambdas. See Creating a Deployment Package (Python).
To do this in C# and create a not so microservice Lambda, yes I've done this, you can reference the other projects. The trick is that these other projects also have to be AWS "Lambdas". That way they have the same underlying dotnetcore references. I don't think it is possible to take a standard framework dll and reference it.
Can you do this via reflection without a reference? Don't know. If you needed to do that then you might as well just create another AWS Lambda and call it directly from another Lambda. AWS Lambda to Lambda communication is very fast. AWS Lambda to API Gateway to AWS Lambda not so much because of the added latency. One could also do Step Functions which is really AWS Lambda to AWS Lambda with state.