I have a Lambda function that needs to read messages from an SQS queue using it's URL. Then it needs to insert that data to Cassandra running on a server inside a VPC.
I am able to access the Cassandra server from my Lambda function, using it's private IP and configuring the security groups correctly.
However, I am not able to read messages from the SQS Queue. When I change the configuration of Lambda function to No VPC, then I am able to read the messages from the SQS Queue. However, with VPC settings, it just times out.
How can I overcome this ? I have checked the security group of my Lambda function has full outbound access to all IP addresses.
At the end of 2018, AWS announced support for SQS endpoints which provide
connectivity to Amazon SQS without requiring an internet gateway, network address translation (NAT) instance, or VPN connection.
There is a tutorial for Sending a Message to an Amazon SQS Queue from Amazon Virtual Private Cloud
See also the SQS VPC Endpoints Documentation for more information.
Its important to note that if you want to access SQS within the Lambda VPC there are a couple other things you need to do:
Make sure to specify the SQS region in your code. For example, I had to set my endpoint_url to "https://sqs.us-west-2.amazonaws.com"
Make sure that you have attached a "wide open" security group to the SQS VPC Interface, otherwise SQS will not work.
Make sure that your subnets in your Lambda VPC match what you have set up for your SQS VPC Interface.
Some services (e.g. S3) are offering VPC endpoints to solve this particular problem but SQS is not one of them. I think the only real solution to this problem is to run a NAT inside your VPC so the network traffic from the Lambda function can be routed to the outside world.
I ran into the same kind of problem when I was running lambda function with access to elasticache on the VPC. While the function was configured to run in the VPC, I wasnt able to talk to any other service (specifically codedeploy for me).
As #garnaat pointed out NAT seems to be the only way to go about solving this problem for services without VPC endpoints.
And like you pointed out, I also ran into the same trouble where I could'nt SSH into the machine(s) once I replaced the entry with the IGW in the route table. Seems like detaching the IGW starves the VPC of either the incoming traffic (mostly) or the outgoing traffic from or to the internet respectively. So here's what I did and it worked for me:
Create a new Subnet within the VPC
Now, when lambda runs, make sure lambda operates from this subnet.
You can do this by using aws-cli like so:
aws lambda update-function-configuration --function-name your-function-name --vpc-config SubnetIds="subnet-id-of-created-subnet",SecurityGroupIds="sg-1","sg-2"
Make sure you add all the security groups whose inbound and outbound traffic rules apply for your lambda function.
Next, go to Route Tables in the VPC console and create a new route table.
Here is where you add the NAT gateway to the target.
finally go to the Subnet Associations tab in the new route table and add the newly created subnet there.
Thats all this should get it working . Mind you, please treat this as only a workaround. I haven't done much digging and I have a very limited idea on how things get resolved internally while doing this. This might not be an ideal solution.
The ideal solution seems to be to design the VPC before hand. Use subnets to isolate resources/instances that need internet access and that dont(private and public subnets) and place appropriate gateways where needed.( so that you may not have to create a seperate subnet for this purpose later). Thanks
I was unable to get either of the other two answers to this question to work. Perhaps this is due to one or more mistakes on my part. Regardless, I did find a workaround that I wanted to share, in case I'm not alone with this problem.
Solution: I created two Lambda functions. The first Lambda function runs inside my VPC and performs the desired work (in mandeep_m91's case, that's a data insertion to Cassandra; in my case is was accessing an RDS instance). The second Lambda function lives outside the VPC, so I could hook it up to the SQS queue. I then had the second Lambda function call the first, using the information found this this StackOverflow Q&A answer. Note, the linked question has both node.js and Python examples in the answers.
This will effectively double the cost of making a function call, since each call results in two function executions. However, for my situation, the volume is so low it won't make a real difference.
To clarify a point above about a "wide open" security group, the group set on the endpoint needs to allow inbound access to SQS from your lambda function.
I created a security group for my endpoint that only opened 443 to my lambda's security group.
Related
I am trying to write AWS lambda function which gets user from AWS Cognito using AWS.CognitoIdentityServiceProvider.adminGetUser and then stores some data into AWS Elasticache (Redis). The parts work fine separately, but not together, as it seems that when I enable VPC for Lambda (to be able to access Elasticache), I loose access to Cognito (Lambda times out).
Is there a way to access both services from single lambda function, or my approach is totally wrong?
Yes, you can for sure. You need to create a NAT Gateway or NAT instance in a public subnet of your VPC, then add a rule to the route table that enables NAT from lambda's security group. I don't think there are public Cognito IP addresses, therefore your security outbound group rule CIDR would be 0.0.0.0/0.
More details are available on official docs here and also there are tons of tutorials online, because it's a common problem. Good luck!
P.S. Don't forget to add permissions for adminGetUser action to lambda's IAM role.
I have a couple of Lambda functions that require access to other services I have in AWS inside a VPC. So I have added these Lambda functions to the same VPC which makes them lose internet access.
One of the service I need access to is a SQS queue, which is not in the VPC as it doesn't support it.
I found two solutions to my problem:
Create a NAT gateway and allow my function to access the internet
Create a VPC endpoint for SQS
I wanted to go with the latter. When I did that, the Lambda worked fine, but I couldn't deploy new code to my Beanstalk instances. That should have nothing to do with it! I spent hours to realize Beanstalk uses SQS to send messages to my instances to perform the deploy.
Questions:
Is it a better solution to use a VPC endpoint rather than a NAT gateway? (it seems to be cheaper)
If so, how can I keep my Beanstalk instances working correctly?
hello I want to call a api which is hoisted in Apache server on AWS EC2 server from my lambda function.
I have configured VPC for my lambda function according to my EC2 server instance with same security groups and same zone ,but still no luck.
Thanks in advance.
It appears that your situation is:
You want to make an HTTP/S API call from an AWS Lambda function to an Amazon EC2 instance
The Lambda function has been configured to connect to the same VPC as the EC2 instance
The Lambda function is reporting that it is unable to connect to the EC2 instance
I would recommend:
Create a Security Group (let's call it Lambda-SG) in the same VPC. It does not need any rules.
Configured the Lambda function to use the Security Group (in the Network section)
Create another Security Group (let's call it EC2-SG) and associate it with the EC2 instance. Add a rule that permits inbound access on the desired port, with the source set to ID of Lambda-SG.
That is the EC2-SG is saying that it will accept inbound connects from Lambda-SG on the desired port.
Merely putting resources in the same security group does not necessarily mean they can communicate. There needs to be a specific rule that permits the inbound access. The above method is the best way to do so.
I have a beautiful Lambda setup with Cloudwatch Events triggering the functions on a schedule.
I am starting to move my Lambda functions into VPC, but I am having a dickens of a time getting Cloudwatch Events to properly trigger Lambda functions. They continually time out.
Is there a way to use CW Events to trigger VPC Lambda functions? If not, is there a canonical solution to triggering Lambda functions on schedule?
This issue isn't related to Cloudwatch Events. The source of the function trigger doesn't modify how Lambda actually invokes the function.
The problem is almost certainly that your functions don't currently work inside your VPC, and this in turn because you have not configured the VPC as needed for Internet access -- and most of the various AWS service APIs are accessed via an Internet connection. A NAT device is required, because your Lambda containers have only private IP addresses.
Important
If your Lambda function needs Internet access, do not attach it to a public subnet or to a private subnet without Internet access. Instead, attach it only to private subnets with Internet access through a NAT instance or an Amazon VPC NAT gateway.
http://docs.aws.amazon.com/lambda/latest/dg/vpc.html
The functions will otherwise time out the first time and any time your code attempts to access any resource outside the VPC.
I am running into a security related issue with AWS lambda and not sure what is the right way to resolve this.
Consider an EC2 instance A accessing the database on another EC2 instance B. If I want to restrict the accessibility of the DB on instance B to instance A only, I would modify the security group and add a custom TCP rule to allow access to only the public IP of instance A. So, this way, AWS will take care of everything and the DB server will not be accessible from any other IP address.
Now let us replace instance A by a lambda function. Since it is no longer an instance, there is no definite IP address. So, how do I restrict access to only the lambda function and block any other traffic ?
Have the Lambda job determine its IP, and dynamically update the instance B security group, then reset the security group when done.
Until there is support for Lambda running within a VPC this is the only option. Support for that has been announced for later this year. The following quote is from the referenced link above.
Many AWS customers host microservices within a Amazon Virtual Private
Cloud and would like to be able to access them from their Lambda
functions. Perhaps they run a MongoDB cluster with lookup data, or
want to use Amazon ElastiCache as a stateful store for Lambda
functions, but don’t want to expose these resources to the Internet.
You will soon be able to access resources of this type by setting up
one or more security groups within the target VPC, configure them to
accept inbound traffic from Lambda, and attach them to the target VPC
subnets. Then you will need to specify the VPC, the subnets, and the
security groups when your create your Lambda function (you can also
add them to an existing function). You’ll also need to give your
function permission (via its IAM role) to access a couple of EC2
functions related to Elastic Networking.
This feature will be available later this year. I’ll have more info
(and a walk-through) when we launch it.
I believe the below link will explain lambda permission model for you.
http://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html