I'm trying to access a external MyQSL database (Not AWS RDS), and I need to have a static IP in order to open up the firewall for accepting connections. Is it possible to set a static IP with a Lambda instance? If not what are some other options?
In order to do that, you need to deploy your Lambda function into a VPC and within the VPC, provide NAT Gateway. Then assign an Elastic IP (static IP) to the NAT Gateway. These two links describe it step-by-step:
AWS: How to Create a Static IP Address Using a NAT Gateway (Medium)
How do I give internet access to my Lambda function in a VPC? (AWS Knowledge Center
I have to do this every year or two and always forget how to do it :) Fortunately, I've discovered the AWS now has a wizard that steps you through this process: https://ap-southeast-2.console.aws.amazon.com/vpc/home?region=ap-southeast-2#wizardFullpagePublicAndPrivate:
The wizard didn't pick up my Elastic IP Allocation ID so I had to manually paste it in from the Elastic IP section of the VPC console but after that everything works. https://ap-southeast-2.console.aws.amazon.com/vpc/home?region=ap-southeast-2#Addresses:sort=PublicIp
Then you just set up your lambda function to use that VPC. The only remaining gotcha is to select the Private Subnet that the wizard created rather than the public subnet (of course).
If you are deploying your Lambda functions using SAM rather than the console you can direct your function to use the VPC by including Policy and VpcConfig sections in your SAM template as shown below.
In another year or two when I have to do this again, I'll hopefully find this answer :)
No, this is not possible.
What you should do instead is:
deploy the Lambda function into the private subnet of a VPC
deploy a NAT Gateway (or NAT instance) into a public subnet of the VPC
deploy an Internet Gateway into the VPC
give the NAT an Elastic IP
make the NAT be the default route for the Lambda subnet
whitelist the NAT's Elastic IP at the remote firewall
Related
I am creating a status page using LambStatus https://lambstatus.github.io/ and would like to have the status of components update automatically via cloudwatch alarms. I have to limit traffic to the status page. I currently only have traffic whitelisted that originates from a VPN or from our VPC IP addresses. In a perfect world the cloudwatch alarm triggers SNS ----> Lambda function ---> curls the API endpoint to update the component.
I need to either place the Lambda in the VPC so that the HTTPS Patch comes from a set of IP addresses or discover some other way of whitelisting the VPC.
Lambda functions can have access to VPC resources but still exist in the default VPC. Is it possible for me to control the IP addresses of the lambda function or have some way to whitelist the lambda request?
Since Lambda uses IP address from EC2 range if you're planning to use WAF, You need to add an explicit header in CURL and allow the request in WAF only if the header is present.
Since you mentioned about WAF, I assume the API endpoint is a public endpoint and accessible over a Public IP address.
In this case, the good option is to use Lambda in VPC.
Lambda to communication to a Public IP in a VPC environment , it needs to be launched a private subnet , you can choose a subnet which has default route to a NAT gateway, NAT gateway requires a Elastic IP address so all the traffic from Lambda would go by the NAT gateway and you can whitelist the NAT gateway IP.
I would not recommend using a NAT instance because I don't know the instance type and the number of requests.
I set up an Aurora Database (provisioned) in a newly created VPC and no public accessibility. As I want to run a Lambda function in the VPC which is able to both, access the RDS instances as well as the Internet, I changed the routing tables of the RDS instances to allowing traffic from a NAT gateway which I placed in a public subnet in the same VPC.
For the Lambda function itself, I created a separate private subnet, also just allowing traffic from the NAT gateway in the routing table. I assigned this subnet and VPC to the Lambda function in the Lambda settings. The internet connection works fine with this configuration but I can not access the database. That's why I followed this post (https://serverfault.com/questions/941886/connect-an-aws-lambda-function-triggered-by-api-gateway-to-aurora-serverless-mys) and added the IP CIDR of the Lambda subnet to the Security Group of the RDS instances (called rds-launch-wizard).
Still, the Lambda function is able to interact with the public internet but can not connect to the RDS instances (timeout). I'm running out of ideas, what is wrong here?
The configuration should be:
A Public subnet with a NAT Gateway (and, by definition, an Internet Gateway)
A Private subnet with the Amazon RDS instance
The same, or a different, Private Subnet associated with the Lambda function
The Private Subnet(s) configured with a Route Table with a destination of 0.0.0.0/0 to the NAT Gateway
Then consider the Security Groups:
A security group for the Lambda function (Lambda-SG) that permits all outbound access
A security group for the RDS instance (RDS-SG) that should permit inbound access from Lambda-SG on the appropriate database port
That is, RDS-SG is allowing incoming traffic from Lambda-SG (by name). There is no need to use CIDRs in the security group.
The Lambda function will connect to a private subnet via an Elastic Network Interface (ENI) and will be able to communicate both with the RDS instance (directly) and with the Internet (via the NAT Gateway).
Please note that you are not directing "traffic from the NAT Gateway". Rather, you are directing Internet-bound traffic to the NAT Gateway. Nor is there such a thing as "routing tables of the RDS instances" because the Route Tables are associated with subnets, not RDS.
My Redshift cluster is in a private VPC. I've written the following AWS Lamba in Node.js which should connect to Redshift (dressed down for this question):
'use strict';
console.log('Loading function');
const pg = require('pg');
exports.handler = (event, context, callback) => {
var client = new pg.Client({
user: 'myuser',
database: 'mydatabase',
password: 'mypassword',
port: 5439,
host: 'myhost.eu-west-1.redshift.amazonaws.com'
});
// connect to our database
console.log('Connecting...');
client.connect(function (err) {
if (err) throw err;
console.log('CONNECTED!!!');
});
};
I keep getting Task timed out after 60.00 seconds unfortunately. I see in the logs "Connecting...", but never "CONNECTED!!!".
Steps I've taken so far to get this to work:
As per Connect Lambda to Redshift in Different Availability Zones I have the Redshift cluster and the Lamba function in the same VPC
Also Redshift cluster and the Lamba function are on the same subnet
The Redshift cluster and the Lamba function share the same security group
Added an inbound rule at the security group of the Redshift cluster as per the suggestion here (https://github.com/awslabs/aws-lambda-redshift-loader/issues/86)
The IAM role associated with the Lamba Function has the following policies: AmazonDMSRedshiftS3Role, AmazonRedshiftFullAccess, AWSLambdaBasicExecutionRole, AWSLambdaVPCAccessExecutionRole, AWSLambdaENIManagementAccess scrambled together from this source: http://docs.aws.amazon.com/lambda/latest/dg/vpc.html (I realize I have some overlap here, but figured that it shouldn't matter)
Added Elastic IP to the Inbound rules of the Security Group as per an answer from a question listed prior (even if I don't even have a NAT gateway configured in the subnet)
I don't have Enhanced VPC Routing enabled because I figured that I don't need it.
Even tried it by adding the Inbound rule 0.0.0.0/0 ALL types, ALL protocols, ALL ports in the Security Group (following this question: Accessing Redshift from Lambda - Avoiding the 0.0.0.0/0 Security Group). But same issue!
So, does anyone have any suggestions as to what I should check?
*I should add that I am not a network expert, so perhaps I've made a mistake somewhere.
The timeout is probably because your lambda in VPC cannot access Internet in order to connect to your cluster(you seem to be using the public hostname to connect). Your connection options depend on your cluster configuration. Since both your lambda function and cluster are in the same VPC, you should use the private IP of your cluster to connect to it. In your case, I think simply using the private IP should solve your problem.
Depending on whether your cluster is publicly accessible, there are some points to keep in mind.
If your cluster is configured to NOT be publicly accessible, you can use the private IP to connect to the cluster from your lambda running in a VPC and it should work.
If you have a publicly accessible cluster in a VPC, and you want to
connect to it by using the private IP address from within the VPC, make sure the following VPC parameters to true/yes:
DNS resolution
DNS hostnames
The steps to verify/change these settings are given here.
If you do not set these parameters to true, connections from within VPC will resolve to the EIP instead of the private IP and your lambda won't be able to connect without having Internet access(which will need a NAT gateway or a NAT instance).
Also, an important note from the documentation here.
If you have an existing publicly accessible cluster in a VPC,
connections from within the VPC will continue to use the EIP to
connect to the cluster even with those parameters set until you resize
the cluster. Any new clusters will follow the new behavior of using
the private IP address when connecting to the publicly accessible
cluster from within the same VPC.
My issues got resolved after adding the CIDR range of the VPC to the Redshift Inbound rules.
For the ones that are trying to move to redshift serverless due to it's recent release to the public... this may be a commom issue but at least for me the answer from #pcothenet worked:
For what it's worth, I had a similar issue. My problem was that I had
set the lambda to have access to my public subnets only. My public
subnet is routing all outbound traffic to an internet gateway, while
my private subnets are routing outbound traffic via an NAT Gateway.
But according to the doc "You cannot use an Internet gateway attached
to your VPC, since that requires the ENI to have public IP addresses."
Switching the lambda to the private subnets (and therefore using the
NAT Gateway) solved the problem. – pcothenet
You must use the Endpoint to connect.
Best.
I had this same issue and followed the steps above and I found that in my case the issue was that the lambda was in a subnet that did not have a route to the NAT gateway. So I moved the lambda into a subnet with route to the NAT gateway.
I have some instances with an EC2 VPC (using only ip addresses from RFC 1918) that need to use some services of EC2 via CLI interface (ec2-describe-instances, ec2-run-instances, etc)
I can't get it to work : my understanding is that the service point of the CLI interface is located somewhere in AWS cloud and my requests originating from an RFC1918 address are not routable in the AWS cloud between EC2 service point and my instance.
Is that correct ?
Is my only solution to install a NAT instance within my VPC (I would like to avoid it) ? Or could I get a way to remap this Ec2 service point within my VPC on a RFC1918 address
Any help welcome !
Thanks in advance
didier
You can give the instance an elastic IP address and get outbound access to other publicIPs, like the EC2 API endpoint. Make sure your security group doesn't allow any inbound traffic from the Internet.
Alternatively, if you don't want to use an EIP, you can launch an instance in a VPC with a publicIP address. more here: http://aws.typepad.com/aws/2013/08/additional-ip-address-flexibility-in-the-virtual-private-cloud.html
I have for a long time a VPC (with 1 subnet) on Amazon Web Services (AWS) with several instances each having an Elastic IP address.
For new needs, I have defined a second VPC (with 1 subnet also) on my same account: for some reasons, I can't associate EIP (which is allocated with no problem) to instances launched in VPC #2: the interactive wizard of the console only presents me the instances of the first VPC.
Is it a known limitation or am I doing something wrong?
Two questions:
How many EIP's do you have on your account?
Is the 2nd VPC using a NAT instance to access the Internet?
EIP addresses should only be used on instances in subnets configured to route their traffic directly to the Internet Gateway. EIPs cannot be used on instances in subnets configured to use a NAT instance to access the Internet. (aws.amazon.com)