I am running AWS Lambda functions in a VPC.
And during the course of the project I have hit problems because:
no access to my database - had to solve this somehow
no access to AWS SES - had to find workaround
no access to AWS SQS -removed all queuing functionality from Lambda functions
no access to external Internet - still don't know how to implement ReCapthca
without Internet access
no access to AWS Cognito - cannot get
information about logged in users
I COULD implement a NAT gateway in the VPC but what is the point of serverless if I have to run a NAT server instance? That's not serverless.
So finally AWS has worn me down and I have decided to give up on running my AWS Lambda functions in a VPC - without endpoints for Internet proxying and the various AWS services its just too hard.
SO my question is - what is the downside/disadvantage of running my AWS Lambda functions with no VPC?
If you need access to resources within a VPC, then run your AWS Lambda function within a VPC. If you do not require this access, then do not run it within a VPC.
If you require Internet access, then you should connect your Lambda functions to a Private Subnet and use a NAT Gateway, which is a fully-managed NAT so you can remain serverless. It will solve the problems you listed.
AWS has provided a reference document for Lambda deployments: Serverless Application Lens, AWS Well-Architected Framework. In it they provide the following decision tree:
The only major downside noted is that a Lambda outside of a VPC cannot directly access private resources within a VPC.
One reason to create a Lambda in a VPC would be that you have a specific IP or IP range for it. This could be the case if a system just accepts calls from a specific IP which would need to be whitlistet for it.
Fix IP for Lambda function is discussed here: Is there a way to assign a Static IP to a AWS Lambda without VPC?
Downside of not having Lambda in VPC: Not having specific IP / IP-range for your Lambda function.
In the end I stayed with the VPC but I added an EC2 instance into the VPC and ran TinyProxy on it. I then configured my AWS Lambda functions with the environment variable:
HTTPS_PROXY https://ip-10-0-1-53.eu-west-1.compute.internal:8888
boto3 picked up the environment variable and sent all requests to the proxy. This seems to work fine without the complexity of a NAT gateway.
Related
I am trying to trigger code pipeline from lambda using below link got the lambda python code as well.
https://aws.amazon.com/blogs/devops/adding-custom-logic-to-aws-codepipeline-with-aws-lambda-and-amazon-cloudwatch-events/
But somehow while running i am getting exception as
Connect timeout on endpoint URL "https://codepipeline.ap-southeast-2.amazonaws.com/"
I have opened all traffic using security group attached to lambda.
Please suggest what else to check here.
Thanks
Sharad
You are running your Lambda function in a VPC (as evidenced by the fact that you said it has a security group attached). A Lambda function in a VPC cannot access anything outside the VPC without a route to a NAT Gateway. A Lambda function in a VPC never gets a public IP assigned to it, so it can never use a VPC Internet Gateway directly. Thus to access anything outside your VPC, such as the AWS API to trigger a CodePipeline run, the Lambda function needs to be deployed only in subnets of your VPC that have a route to a NAT Gateway.
The alternative would be to add a VPC Endpoint for the specific AWS Service you want to access.
I have a couple of Lambda functions that require access to other services I have in AWS inside a VPC. So I have added these Lambda functions to the same VPC which makes them lose internet access.
One of the service I need access to is a SQS queue, which is not in the VPC as it doesn't support it.
I found two solutions to my problem:
Create a NAT gateway and allow my function to access the internet
Create a VPC endpoint for SQS
I wanted to go with the latter. When I did that, the Lambda worked fine, but I couldn't deploy new code to my Beanstalk instances. That should have nothing to do with it! I spent hours to realize Beanstalk uses SQS to send messages to my instances to perform the deploy.
Questions:
Is it a better solution to use a VPC endpoint rather than a NAT gateway? (it seems to be cheaper)
If so, how can I keep my Beanstalk instances working correctly?
When I'm moving my AWS Lambda function inside an RDS MySQL VPC, then the API Gateway cannot call it and timeouts. Is it possible to call a VPC Lambda from API Gateway?
VPC Links cannot be created with Application Load Balancers that is the only choice for a Lambda function, they need Network Load Balancers.
Are you sure that it's not your lambda that is timing out. API Gateway has no problem calling Lambda functions that are configured to run inside a VPC.
Consider using a VPC service endpoint for KMS and/or other services you need if you need your lambda to run in a VPC. Depending on the services this might be cheaper than a NAT gateway.
Even after going through the AWS documentation and various blogs, I still don't understand how AWS lambda would behave when it is configured with VPC.
When AWS lambda configured with VPC, does that mean all instances of lambda would get the IP address from the specified subnet of that VPC?
How the ENI plays the role in AWS Lambda-VPC configuration?
The formula for ENI capacity from AWS doc -
Projected peak concurrent executions * (Memory in GB / 3GB)
Does it means AWS lambda’s running instance used to have 3 GB memory? And when that exceeds another ENI would get attached?
Most the AWS Lambda-VPC configuration related architecture diagrams shows Lambda inside VPC. Does that means Lambda would run inside VPC?
Here, I’m sure I’m missing a few pieces of information. Any pointers would be helpful.
When you configure a Lambda function to run in the VPC it uses an ENI that is created with and IP address in one of the subnets you select. Based on the formula of expected ENIs needed it seems that ENIs can be shared between lambdas.
There are only two reasons that I know of for running your lambda in a VPC.
It needs to access resources inside your VPC that do not have a public endpoint, e.g. Redis/Memcached caching clusters (Elasticache) or an RDS/Redshift cluster that doesn't have a public ip (good idea to not have public ip's on databases). When you lambda runs inside the VPC it uses a private ip and can connect to the private resources in your VPC
If you need to have your lambda's have a consistent IP address (perhaps a service that only allows whitelisting of IPs for authentication). This is achieved by using a NAT gateway.
Lambda functions cannot received inbound connections in any case.
Disadvantages of putting your lambda in a VPC are
Slower cold start times since a ENI might need to be provisioned.
You need a NAT gateway (or VPC endpoint) to access external resources
Needing to manage concurrency and available ip addresses more closely.
I launched one RDS instance,s3 and EC2 in AWS and its is triggered properly using lambda. Now I wish to change the change the RDS and EC2 from AWS to local machine. My lambda is triggered from s3.
How do I connect the local database through lambda in AWS?
It appears that your requirement is:
You wish to run an AWS Lambda function
Within the function, you wish to connect to a database running on your own computer (outside of AWS)
Firstly, I would not recommend this strategy. To maintain good performance, you should always have an application as close as possible to the database. This means on the same network, in the same location and not going across remote network connections or the Internet.
However, if you wish to do this, then here's some things you would need to do:
Your database will need to be accessible on the Internet, so that you can connect to it remotely. To test this, try accessing it from an Amazon EC2 instance.
The AWS Lambda function should either be configured without VPC connectivity (which means that it is connected to the Internet) or, if you have configured it for VPC connectivity, it needs to be in a Private Subnet with a NAT Gateway enabling Internet access.
(Optional) For added security, you could lock-down your database to only accept connections from a known IP address. To achieve this, you would need to use the VPC + NAT Gateway so that all traffic is coming from the Elastic IP address assigned to the NAT Gateway.
I agree with John Rotenstein that connecting your local machine to a Lambda running on AWS is probably a bad idea.
If your intention is to develop or test locally, I recommend the serverless framework, and the serverless-offline plugin. It will allow you to simulate Lambda locally, and you can pass database config values through as environment variables.
See: Running AWS Lambda and API Gateway locally: serverless-offline