Since April 2020, AWS supports VPC endpoints for SES (Simple Email Service.) Despite this, I cannot get a Lambda to communicate with SES using the endpoint. It always times out. Perhaps this is because the SES VPC Endpoints are only supported for EC2 instances in the VPC and not for Lambdas?
I have my infrastructure describe using Terraform, as described in this gist. ses_lambda.tf defines the infrastructure and basically implements this guide. sesTest.js is my Lambda function. terraform-show.out shows the result of terraform show limited just to the resources defined by ses_lambda.tf.
As you can see in ses_lambda.tf, the SES Endpoint's security group allows all traffic on any SMTP port (25, 465, 587, 2465, or 2587) from any IP in the VPC's CIDR block. So I'm confused about why this doesn't work...
It looks like SES VPC endpoints are SMTP endpoints not SES API endpoints. The guide is not explicit about it but it does mention SMTP ports and endpoints.
I created an endpoint and the private DNS name it produced was email-smtp.us-east-1.amazonaws.com (SMTP). SDK tries to connect to email.us-east-1.amazonaws.com (SES API).
It appears that the solution is to either use SMTP or setup NAT Gateway.
Update: Confirming that SMTP works over SES VPC endpoint in Lambda (as exptected).
I was having difficulties with the same setup. The reality is that nzhuk98's answer is the solution but for someone who didn't have experience with this - it did not make sense to me too.
You cannot use AWS SDK's SES methods, even if you adjust the endpoint setting to the Private DNS that was given to you after the VPC endpoint was created.
AWS SDK's SES method uses https only, and the VPC endpoint exposes only the SMTP port of the SES Service. Therefore, you need to opt-out from using AWS SDK completely and switch to something like Nodemailer.
This is what I did https://nodemailer.com/smtp/#1-single-connection
I hope this is useful for the next person that is looking for this.
When I hear AWS timeout issues, the first thing I check is always IAM. Does the Lambda function have an IAM Policy and Execution Role that grants the appropriate permissions to access SES within your account?
https://aws.amazon.com/premiumsupport/knowledge-center/lambda-send-email-ses/
Related
I am trying to trigger code pipeline from lambda using below link got the lambda python code as well.
https://aws.amazon.com/blogs/devops/adding-custom-logic-to-aws-codepipeline-with-aws-lambda-and-amazon-cloudwatch-events/
But somehow while running i am getting exception as
Connect timeout on endpoint URL "https://codepipeline.ap-southeast-2.amazonaws.com/"
I have opened all traffic using security group attached to lambda.
Please suggest what else to check here.
Thanks
Sharad
You are running your Lambda function in a VPC (as evidenced by the fact that you said it has a security group attached). A Lambda function in a VPC cannot access anything outside the VPC without a route to a NAT Gateway. A Lambda function in a VPC never gets a public IP assigned to it, so it can never use a VPC Internet Gateway directly. Thus to access anything outside your VPC, such as the AWS API to trigger a CodePipeline run, the Lambda function needs to be deployed only in subnets of your VPC that have a route to a NAT Gateway.
The alternative would be to add a VPC Endpoint for the specific AWS Service you want to access.
I thought I had an understanding of AWS security groups, but this doesn't make any sense.
I have a Lambda Function that is inside of my VPC.
It is assigned a security group (TestLambdaSG).
TestLambdaSG has inbound HTTP/HTTPS rules from IP 1.2.3.4/32
I can connect to my Lambda Function from my IP. Why?
AWS says that security groups are restrictive by default, so I shouldn't be able to connect. My Lambda function is an API that I created a test route that returns a "Success" message.
https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-security-groups.html
The security group assigned to lambda only used to validate outgoing traffic , it is impossible to access the lambda directly through the socket connection. Therefore I dont think lambda security group's inbound rules are any useful.
But other services such as API gateway can invoke a lambda. An API gateway is publicly available by default when it is deployed.
That's why it's accessible from postman for you.
I am trying to write AWS lambda function which gets user from AWS Cognito using AWS.CognitoIdentityServiceProvider.adminGetUser and then stores some data into AWS Elasticache (Redis). The parts work fine separately, but not together, as it seems that when I enable VPC for Lambda (to be able to access Elasticache), I loose access to Cognito (Lambda times out).
Is there a way to access both services from single lambda function, or my approach is totally wrong?
Yes, you can for sure. You need to create a NAT Gateway or NAT instance in a public subnet of your VPC, then add a rule to the route table that enables NAT from lambda's security group. I don't think there are public Cognito IP addresses, therefore your security outbound group rule CIDR would be 0.0.0.0/0.
More details are available on official docs here and also there are tons of tutorials online, because it's a common problem. Good luck!
P.S. Don't forget to add permissions for adminGetUser action to lambda's IAM role.
I have a couple of Lambda functions that require access to other services I have in AWS inside a VPC. So I have added these Lambda functions to the same VPC which makes them lose internet access.
One of the service I need access to is a SQS queue, which is not in the VPC as it doesn't support it.
I found two solutions to my problem:
Create a NAT gateway and allow my function to access the internet
Create a VPC endpoint for SQS
I wanted to go with the latter. When I did that, the Lambda worked fine, but I couldn't deploy new code to my Beanstalk instances. That should have nothing to do with it! I spent hours to realize Beanstalk uses SQS to send messages to my instances to perform the deploy.
Questions:
Is it a better solution to use a VPC endpoint rather than a NAT gateway? (it seems to be cheaper)
If so, how can I keep my Beanstalk instances working correctly?
I am running AWS Lambda functions in a VPC.
And during the course of the project I have hit problems because:
no access to my database - had to solve this somehow
no access to AWS SES - had to find workaround
no access to AWS SQS -removed all queuing functionality from Lambda functions
no access to external Internet - still don't know how to implement ReCapthca
without Internet access
no access to AWS Cognito - cannot get
information about logged in users
I COULD implement a NAT gateway in the VPC but what is the point of serverless if I have to run a NAT server instance? That's not serverless.
So finally AWS has worn me down and I have decided to give up on running my AWS Lambda functions in a VPC - without endpoints for Internet proxying and the various AWS services its just too hard.
SO my question is - what is the downside/disadvantage of running my AWS Lambda functions with no VPC?
If you need access to resources within a VPC, then run your AWS Lambda function within a VPC. If you do not require this access, then do not run it within a VPC.
If you require Internet access, then you should connect your Lambda functions to a Private Subnet and use a NAT Gateway, which is a fully-managed NAT so you can remain serverless. It will solve the problems you listed.
AWS has provided a reference document for Lambda deployments: Serverless Application Lens, AWS Well-Architected Framework. In it they provide the following decision tree:
The only major downside noted is that a Lambda outside of a VPC cannot directly access private resources within a VPC.
One reason to create a Lambda in a VPC would be that you have a specific IP or IP range for it. This could be the case if a system just accepts calls from a specific IP which would need to be whitlistet for it.
Fix IP for Lambda function is discussed here: Is there a way to assign a Static IP to a AWS Lambda without VPC?
Downside of not having Lambda in VPC: Not having specific IP / IP-range for your Lambda function.
In the end I stayed with the VPC but I added an EC2 instance into the VPC and ran TinyProxy on it. I then configured my AWS Lambda functions with the environment variable:
HTTPS_PROXY https://ip-10-0-1-53.eu-west-1.compute.internal:8888
boto3 picked up the environment variable and sent all requests to the proxy. This seems to work fine without the complexity of a NAT gateway.