How to get the Arn of a lambda function's execution role in AWS CDK - aws-lambda

My use case is:
I want to execute a lambda function for the PUT event of an existing S3 bucket.
The problem is that you in CloudFormation or CDK you cannot add notifications for an existing bucket, only for buckets that are created.
To get around this I am trying to use a custom resource that adds the Lambda function to the PutNotification. I have this working fine in CloudFormation but I am trying to use CDK now to do something similar.
To simulate what I have in CloudFormation I need to add a bucket policy to the existing bucket granting permission to the action s3:PutBucketNotification to the lambda execution role principal.
In CloudFormation I do this like this:
NotificationBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref BucketName
PolicyDocument:
Statement:
- Effect: "Allow"
Action:
- 's3:PutBucketNotification'
Resource: !Sub "arn:aws:s3:::${BucketName}"
Principal:
AWS: !GetAtt LambdaExecutionRole.Arn
I am attempting to create the bucket policy and add the statement to it in CDK but I need the Arn of the Lambda Function's Arn
const bucket = Bucket.fromBucketName(this, "Bucket", "my-bucket-name");
const bucketConfigurationFunction = new lambda.SingletonFunction(this, "bucketConfigurationFunction ", {
runtime: lambda.Runtime.NODEJS_8_10,
code: lambda.Code.asset('lambda/bucket-configuration'),
handler: 'lambda_function.handler',
timeout: cdk.Duration.seconds(300),
uuid: '72561a5f-e772-4365-b3d1-f59e8ddc60b1'
})
const bucketPolicy = new BucketPolicy(this, "TargetBucketPolicy", {
bucket: bucket
})
const bucketPolicyStatement = new PolicyStatement()
bucketPolicyStatement.addActions("s3:PutBucketNotification");
//Need to put the execution role arn here but role is undefined
bucketPolicyStatement.addArnPrincipal(bucketConfigurationFunction.role.roleArn)
I have read the CDK creates a lambda function execution role automatically however when I try to access the role Arn to add it as the principal in the policy statement, it is undefined.
Am I doing this totally the wrong way?

Because TypeScript is quite strict on checking optional variables and the role is generated at runtime, you need to box it using and if but that's still fine. For example this works:
const bucketPolicyStatement = new iam.PolicyStatement()
bucketPolicyStatement.addActions("s3:PutBucketNotification");
if (bucketConfigurationFunction.role) {
bucketPolicyStatement.addArnPrincipal(bucketConfigurationFunction.role.roleArn)
}
const bucketPolicy = new s3.BucketPolicy(this, "TargetBucketPolicy", {
bucket: bucket,
})
bucketPolicy.document.addStatements(bucketPolicyStatement);

Related

SAM Template - define HttpApi with Lambda Authorizer and Simple Response

Description of the problem
I have created a Lambda function with API Gateway in SAM, then deployed it and it was working as expected. In API Gateway I used HttpApi not REST API.
Then, I wanted to add a Lambda authorizer with Simple Response. So, I followed the SAM and API Gateway docs and I came up with the code below.
When I call the route items-list it now returns 401 Unauthorized, which is expected.
However, when I add the header myappauth with the value "test-token-abc", I get a 500 Internal Server Error.
I checked this page but it seems all of the steps listed there are OK https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-http-lambda-integrations/
I enabled logging for the API Gateway, following these instructions: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-logging.html
But all I get is something like this (redacted my IP and request ID):
[MY-IP] - - [07/Jul/2021:08:24:06 +0000] "GET GET /items-list/{userNumber} HTTP/1.1" 500 35 [REQUEST-ID]
(Perhaps I can configure the logger in such a way that it prints a more meaningful error message? EDIT: I've tried adding $context.authorizer.error to the logs, but it doesn't print any specific error message, just prints a dash: -)
I also checked the logs for the Lambda functions, there is nothing there (all logs where from the time before I added the authorizer).
So, what am I doing wrong?
What I tried:
This is my Lambda Authorizer function which I have deployed using sam deploy, when I test it in isolation using an event with the myappauth header, it works:
exports.authorizer = async (event) => {
let response = {
"isAuthorized": false,
};
if (event.headers.myappauth === "test-token-abc") {
response = {
"isAuthorized": true,
};
}
return response;
};
and this is the SAM template.yml which I deployed using sam deploy:
AWSTemplateFormatVersion: 2010-09-09
Description: >-
myapp-v1
Transform:
- AWS::Serverless-2016-10-31
Globals:
Function:
Runtime: nodejs14.x
MemorySize: 128
Timeout: 100
Environment:
Variables:
MYAPP_TOKEN: "test-token-abc"
Resources:
MyAppAPi:
Type: AWS::Serverless::HttpApi
Properties:
FailOnWarnings: true
Auth:
Authorizers:
MyAppLambdaAuthorizer:
AuthorizerPayloadFormatVersion: "2.0"
EnableSimpleResponses: true
FunctionArn: !GetAtt authorizerFunction.Arn
FunctionInvokeRole: !GetAtt authorizerFunctionRole.Arn
Identity:
Headers:
- myappauth
DefaultAuthorizer: MyAppLambdaAuthorizer
itemsListFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src/handlers/v1-handlers.itemsList
Description: A Lambda function that returns a list of items.
Policies:
- AWSLambdaBasicExecutionRole
Events:
Api:
Type: HttpApi
Properties:
Path: /items-list/{userNumber}
Method: get
ApiId: MyAppAPi
authorizerFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src/handlers/v1-handlers.authorizer
Description: A Lambda function that authorizes requests.
Policies:
- AWSLambdaBasicExecutionRole
Edit:
User #petey suggested that I tried returning an IAM policy in my authorizer function, so I changed EnableSimpleResponses to false in the template.yml, then I changed my function as below, but got the same result:
exports.authorizer = async (event) => {
let response = {
"principalId": "my-user",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Action": "execute-api:Invoke",
"Effect": "Deny",
"Resource": event.routeArn
}]
}
};
if (event.headers.myappauth == "test-token-abc") {
response = {
"principalId": "my-user",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Action": "execute-api:Invoke",
"Effect": "Allow",
"Resource": event.routeArn
}]
}
};
}
return response;
};
I am going to answer my own question because I have resolved the issue, and I hope this will help people who are going to use the new "HTTP API" format in API Gateway, since there is not a lot of tutorials out there yet; most examples you will find online are for the older API Gateway standard, which Amazon calls "REST API". (If you want to know the difference between the two, see here).
The main problem lies in the example that is presented in the official documentation. They have:
MyLambdaRequestAuthorizer:
FunctionArn: !GetAtt MyAuthFunction.Arn
FunctionInvokeRole: !GetAtt MyAuthFunctionRole.Arn
The problem with this, is that this template will create a new Role called MyAuthFunctionRole but that role will not have all the necessary policies attached to it!
The crucial part that I missed in the official docs is this paragraph:
You must grant API Gateway permission to invoke the Lambda function by using either the function's resource policy or an IAM role. For this example, we update the resource policy for the function so that it grants API Gateway permission to invoke our Lambda function.
The following command grants API Gateway permission to invoke your Lambda function. If API Gateway doesn't have permission to invoke your function, clients receive a 500 Internal Server Error.
The best way to solve this, is to actually include the Role definition in the SAM template.yml, under Resources:
MyAuthFunctionRole
Type: AWS::IAM::Role
Properties:
# [... other properties...]
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action:
- 'sts:AssumeRole'
Policies:
# here you will put the InvokeFunction policy, for example:
- PolicyName: MyPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action: 'lambda:InvokeFunction'
Resource: !GetAtt MyAuthFunction.Arn
You can see here a description about the various Properties for a role: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html
Another way to solve this, is to separately create a new policy in AWS Console, which has InvokeFunction permission, and then after deployment, attach that policy to the MyAuthFunctionRole that SAM created. Now the Authorizer will be working as expected.
Another strategy would be to create a new role beforehand, that has a policy with InvokeFunction permission, then copy and paste the arn of that role in the SAM template.yml:
MyLambdaRequestAuthorizer:
FunctionArn: !GetAtt MyAuthFunction.Arn
FunctionInvokeRole: arn:aws:iam::[...]
Your lambda authorizer is not returning what is expected to be an actual lambda authorizer (an IAM policy). This could explain that internal error 500.
To fix, replace is with something like this that returns an IAM policy (or rejects):
// A simple token-based authorizer example to demonstrate how to use an authorization token
// to allow or deny a request. In this example, the caller named 'user' is allowed to invoke
// a request if the client-supplied token value is 'allow'. The caller is not allowed to invoke
// the request if the token value is 'deny'. If the token value is 'unauthorized' or an empty
// string, the authorizer function returns an HTTP 401 status code. For any other token value,
// the authorizer returns an HTTP 500 status code.
// Note that token values are case-sensitive.
exports.handler = function(event, context, callback) {
var token = event.authorizationToken;
// modify switch statement here to your needs
switch (token) {
case 'allow':
callback(null, generatePolicy('user', 'Allow', event.methodArn));
break;
case 'deny':
callback(null, generatePolicy('user', 'Deny', event.methodArn));
break;
case 'unauthorized':
callback("Unauthorized"); // Return a 401 Unauthorized response
break;
default:
callback("Error: Invalid token"); // Return a 500 Invalid token response
}
};
// Help function to generate an IAM policy
var generatePolicy = function(principalId, effect, resource) {
var authResponse = {};
authResponse.principalId = principalId;
if (effect && resource) {
var policyDocument = {};
policyDocument.Version = '2012-10-17';
policyDocument.Statement = [];
var statementOne = {};
statementOne.Action = 'execute-api:Invoke';
statementOne.Effect = effect;
statementOne.Resource = resource;
policyDocument.Statement[0] = statementOne;
authResponse.policyDocument = policyDocument;
}
// Optional output with custom properties of the String, Number or Boolean type.
authResponse.context = {
"stringKey": "stringval",
"numberKey": 123,
"booleanKey": true
};
return authResponse;
}
Lots more information here : https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html#api-gateway-lambda-authorizer-lambda-function-create
Just to complete the answer. You have to add an AssumeRolePolicyDocument under Properties.
The role will then state
MyAuthFunctionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action:
- 'sts:AssumeRole'
Policies:
# see answer above

CloudFormation: The runtime parameter of nodejs6.10 is no longer supported for creating or updating AWS Lambda functions

I'm trying to update a cloud formation template with a few lambda functions in it. The last version of the template was deployed a few years ago, and all the lambda functions currently have a runtime of node6.10.
I have updated the runtime for all functions to node10.x, but when I deploy the template, I get the following message:
The runtime parameter of nodejs6.10 is no longer supported for creating or updating AWS Lambda functions
I've created a change set, and reviewed it, and it includes an update to the runtime property for each lambda function, however Cloud Formation seems to be ignoring it.
Is there something I'm missing?
Context:
I assume that you encountered this issue - (you got "nodejs version not supported error" message after you tried to amplify push followed by amplify add auth)
Go to amplify -> backend -> auth -> cognito -> click cognito cloudformation ->
search for "Runtime: node"
change it to "Runtime: nodejs8.10" - whatever latest recommended in error message
re-run
$ amplify push
Unfortunately, I found I had to update the runtime of all functions in a template outside of Cloud Formation, to get the stacks to deploy. I used this script:
const AWS = require('aws-sdk')
const lambda = new AWS.Lambda(...)
main().catch(err => {
console.error(err)
process.exit(1)
})
async function main() {
const functions = await getFunctions()
await Promise.all(
functions
// filter only functions you want to update
.filter(...)
.filter(x => x.Runtime !== 'nodejs10.x')
.map(updateFunction)
)
}
async function updateFunction(func) {
await lambda
.updateFunctionConfiguration({
FunctionName: func.FunctionName,
Runtime: 'nodejs10.x'
})
.promise()
console.log(`function updated: ${func.FunctionName}`)
}
async function getFunctions() {
let marker
let functions = []
do {
const result = await lambda
.listFunctions({
Marker: marker
})
.promise()
functions = [...functions, ...result.Functions]
marker = result.NextMarker
} while (marker)
return functions
}

Is it possible to trigger a lambda on creation from CloudFormation template

I tried creating a set of lambdas using cloudformation. I want the lambdas to get triggered once they are created. I saw at various blogs to create a trigger to s3 or sns but none seems to be a option to trigger lambda once it has been created. Any options?
Yes, it is possible. Here are a few options:
Manually create an SNS Topic. Add an AWS::SNS::Subscription to your stack with the lambda function as the Endpoint and the SNS topic as the TopicArn. On stack creation/update, configure Stack Event Notifications to be sent to this SNS topic.
(See Setting AWS CloudFormation Stack Options for documentation on how to do this when using the AWS Console to create your stack, or use the equivalent option like --notification-arns if creating/updating your stack using the AWS CLI or other AWS SDK.)
Add a Custom Resource referencing a Lambda function to be called on creation.
If you need the Lambda function to be called after some specific Resource is created, add a DependsOn attribute on the Custom Resource referencing the Resource you want to make sure is created first before the function is called.
In order for the Custom Resource to create successfully (and not cause a failure/rollback in your stack), you will need to adapt your Lambda function to support the CloudFormation request/response format (see Custom Resource Reference).
This option will call the Lambda function while the stack status is still CREATE_IN_PROGRESS, because the Custom Resource is part of the stack itself.
The Lambda function will also be called again when the stack (and associated Custom Resource) is deleted. This will need to be handled by your Lambda function correctly, or your stack could get stuck in the DELETE_FAILED state.
Add the Lambda function reference to a Stack Output, then write a simple script that performs the stack creation and then manually invokes the Lambda function afterwards.
by yl.
The following just works great !
It invokes a lambda as a part of deployment:
LambdaFunction2:
Type: AWS::Lambda::Function
Properties:
FunctionName: caller
Code:
ZipFile: |
import boto3, json
import cfnresponse
def handler(event, context):
print('EVENT:[{}]'.format(event))
lambda_client = boto3.client('lambda')
test_event = '{"name":"test1"}'
lambda_client.invoke(
FunctionName='target1',
InvocationType='Event',
Payload=test_event,
)
responseValue = 120
responseData = {}
responseData['Data'] = responseValue
cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)
Handler: index.handler
Role:
arn:aws:iam::11111111111:role/mylambda-role
Runtime: python3.7
Timeout: 60
Primerinvoke:
Type: AWS::CloudFormation::CustomResource
DependsOn: LambdaFunction2
Version: "1.0"
Properties:
ServiceToken: !GetAtt LambdaFunction2.Arn
For who looking the similar workaround.
CloudWatch is able to capture API calls of CloudFormation, which is "CreateStack", "UpdateStack" and "DeleteStack", stack states like "Create_complete" or "Complete_Rollback" are uncapturable, which means such state changes not be able to trigger lambda.
The workaround is SNS, stacks are able to send notifications to SNS (In advance settings when you creating stack) and SNS can choose to trigger lambda, however, you can't choose for specific states. So, lambda function takes the job to find out what state in "Message" content of an event. Everyone, just coding.
I know this is a bit old- but a solution could also be too use CommandRunner as a resource type in your template.
https://aws.amazon.com/blogs/mt/running-bash-commands-in-aws-cloudformation-templates/.
You can run virtually any shell command. Add a DependsOn attribute to your CommandRunner type and run a shell script:
aws lambda invoke --function-name my-function --invocation-type RequestRespone --payload '{ "name": "Bob" }'
Improving on Kyr's answer, because it lacks two important things:
how to pass paramaters to the Lambda you invoke
how to treat UPDATE and DELETE on your Stack (his solution would cause CloudFormation to crash on delete)
Here is the revised and improved code:
LambdaInvoker:
DependsOn: ## important, add stuff here you need to existe BEFORE the lambda is called
Type: AWS::Lambda::Function
Properties:
FunctionName: YourLambdaName
Description: 'Lambda invoke wrapper for Custom CFN actions'
Code:
ZipFile: !Sub |
import boto3, json
import cfnresponse
def handler(event, context):
print('EVENT:')
print(event)
if event['RequestType'] == "Create":
lambda_client = boto3.client('lambda')
cfn_event = {
"param1" : "${Param1}",
"param2" : "${Param2}"
}
lambda_client.invoke(
FunctionName='scm-custom-cfn-actions',
InvocationType='Event',
Payload=json.dumps(cfn_event)
)
responseValue = 120
responseData = {}
responseData['Data'] = responseValue
cfnresponse.send(event, context, cfnresponse.SUCCESS,
responseData, 'scm-cfn-customresource-id')
Handler: index.handler
Role: YourLambdaRoleARN
Runtime: python3.7
Timeout: 5
You have the option to notify to a SNS topic, and you may build a lambda that listens to the topic, so the workflow would be: Cloudformation launch -> SNS Topic -> Lambda.
The following template should invoke the lambda :
"InvokeLambda" : {
"Type": "Custom::InvokeLambda",
"Version" : "1.0",
"Properties" : {
"ServiceToken": {
"Fn::GetAtt": ["InitFunction","Arn"]
}
}
},

AWS Lambda:The provided execution role does not have permissions to call DescribeNetworkInterfaces on EC2

Today I have a new AWS Lambda question, and can't find anywhere in Google.
I new a Lambda function, there is no question.
But when I input any code in this function[eg. console.log();] and click "Save", error is occured:
"The provided execution role does not have permissions to call DescribeNetworkInterfaces on EC2"
exports.handler = (event, context, callback) => {
callback(null, 'Hello from Lambda');
console.log(); // here is my code
};
I bound the function with Role: lambda_excute_execution(Policy:AmazonElasticTranscoderFullAccess)
And this function is not bound with any triggers now.
And then, I give the role "AdministratorAccess" Policy, I can save my source code correctly.
This role can run Functions successfully before today.
Is anyone know this error?
Thanks Very much!
This error is common if you try to deploy a Lambda in a VPC without giving it the required network interface related permissions ec2:DescribeNetworkInterfaces, ec2:CreateNetworkInterface, and ec2:DeleteNetworkInterface (see AWS Forum).
For example, this a policy that allows to deploy a Lambda into a VPC:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeNetworkInterfaces",
"ec2:CreateNetworkInterface",
"ec2:DeleteNetworkInterface",
"ec2:DescribeInstances",
"ec2:AttachNetworkInterface"
],
"Resource": "*"
}
]
}
If you are using terraform, just add:
resource "aws_iam_role_policy_attachment" "AWSLambdaVPCAccessExecutionRole" {
role = aws_iam_role.lambda.name
policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
}
via Managed Policy
To grant Lambda necessary permissions to dig in to a VPC where a production RDS db resides in a private subnet.
As mentioned by #portatlas above, the AWSLambdaVPCAccessExecutionRole managed policy fits like a glove (and we all know use of IAM Managed Policies is an AWS-recommended best-practice).
This is for Lambdas with a service role already attached.
AWS CLI
1. Get Lambda Service Role
Ask Lambda API for function configuration, query the role from that, output to text for an unquoted return.
aws lambda get-function-configuration \
--function-name <<your function name or ARN here>> \
--query Role \
--output text
return, take your-service-role-name to #2
your-service-role-name
2. Attach Managed Policy AWSLambdaVPCAccessExecutionRole to Service Role
aws iam attach-role-policy \
--role-name your-service-role-name \
--policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
CDK 2 TypeScript
const lambdaVPCExecutionRole:iam.Role = new iam.Role(this, `createLambdaVPCExecutionRole`, {
roleName : `lambdaVPCExecutionRole`,
assumedBy : new iam.ServicePrincipal(`lambda.amazonaws.com`),
description : `Lambda service role to operate within a VPC`,
managedPolicies : [
iam.ManagedPolicy.fromAwsManagedPolicyName(`service-role/AWSLambdaVPCAccessExecutionRole`),
],
});
const lambdaFunction:lambda.Function = new lambda.Function(this, `createLambdaFunction`, {
runtime : lambda.Runtime.NODEJS_14_X,
handler : `lambda.handler`,
code : lambda.AssetCode.fromAsset(`./src`),
vpc : vpc,
role : lambdaVPCExecutionRole,
});
This is actually such a common issue.
You can resolve this by adding a custom Inline Policy to the Lambda execution role under the Permissions tab.
Just add this:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeNetworkInterfaces",
"ec2:CreateNetworkInterface",
"ec2:DeleteNetworkInterface",
"ec2:DescribeInstances",
"ec2:AttachNetworkInterface"
],
"Resource": "*"
}
]
}
There's a full tutorial with pictures here if you need more information (Terraform, CloudFormation, and AWS Console) or are confused: https://ao.ms/the-provided-execution-role-does-not-have-permissions-to-call-createnetworkinterface-on-ec2/
Additionally, a more recent sequence of steps follows:
Under your Lambda Function, select "Configuration"
Select "Permissions"
Select the execution role:
Select "Add Permissions"
Create Inline Policy
Select "JSON"
Paste the JSON above and select Review.
It seems like this has been answered many different ways already but as of this posting, AWS has a managed policy. If you just search for the AWSLambdaVPCAccessExecutionRole you will be able to attached that, and this method worked for me.
Here is the arn:
arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole
Just go to execution role -> Attach policy -> Search for 'AWSLambdaVPCAccessExecutionRole' and add it.
An example for Cloudformation and AWS SAM users.
This example lambda role definition adds the managed AWSLambdaVPCAccessExecutionRole and solves the issue:
Type: "AWS::IAM::Role"
Properties:
RoleName: "lambda-with-vpc-access"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action:
- sts:AssumeRole
Principal:
Service:
- lambda.amazonaws.com
Just cause there aren't enough answers already ;) I think this is the easiest way. If you're using the web admin console, when you're creating your Lambda function in the first place, down the bottom just expand 'Advanced Settings' and check 'Enable VPC' & choose your vpc... Simple! Before doing this, my connection to my RDS proxy was timing out. After doing this (and nothing else) - works great!
After a bit of experimentation, here is a solution using "least privilege". It's written in Python, for the AWS CDK. However the same could be applied to normal JSON
iam.PolicyDocument(
statements=[
iam.PolicyStatement(
effect=iam.Effect.ALLOW,
actions=["ec2:DescribeNetworkInterfaces"],
resources=["*"],
),
iam.PolicyStatement(
effect=iam.Effect.ALLOW,
actions=["ec2:CreateNetworkInterface"],
resources=[
f"arn:aws:ec2:{region}:{account_id}:subnet/{subnet_id}"
f"arn:aws:ec2:{region}:{account_id}:security-group/{security_group_id}",
f"arn:aws:ec2:{region}:{account_id}:network-interface/*",
],
),
iam.PolicyStatement(
effect=iam.Effect.ALLOW,
actions=["ec2:DeleteNetworkInterface"],
resources=[f"arn:aws:ec2:{region}:{account_id}:*/*"],
),
],
),
Here's a quick and dirty way of resolving the error.
Open IAM on AWS console, select the role that's attached to the Lambda function and give it the EC2FullAccess permission.
This will let you update the Lambda VPC by granting EC2 control access. Be sure to remove the permission from the role, the function still runs.
Is it more or less secure than leaving some permissions attached permanently? Debatable.
If you are using SAM you just need to add to the Globals in the Template, like this:
Globals:
Function:
VpcConfig:
SecurityGroupIds:
- sg-01eeb769XX2d6cc9b
SubnetIds:
- subnet-1a0XX614
- subnet-c6dXXb8b
- subnet-757XX92a
- subnet-8afXX9ab
- subnet-caeXX7ac
- subnet-b09XXd81
(of course, you can put all in variables, or parameters!)
and then, to the Lambda Function, add Policies to the Properties, like this:
BasicFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- AWSLambdaVPCAccessExecutionRole
- AWSLambdaBasicExecutionRole
It is definitely a strange error, but are you sure the example code you added is the one you're using in your lambda?
Because in your code, you are trying to log something in your lambda after returning control via the callback. In other words, first you told your lambda that you're done. Next, while it is busy shutting down and returning your results, you try to do some logging...
So first, I'd try this:
exports.handler = (event, context, callback) => {
console.log('this is a test');
// do stuff
callback(null, 'Hello from Lambda'); // only do a callback *after* you've run all your code
};
And see if that fixes the problem.

Create AMI image as part of a cloudformation stack

I want to create an EC2 cloudformation stack which basically can be described in the following steps:
1.- Launch instance
2.- Provision the instance
3.- Stop the instance and create an AMI image out of it
4.- Create an autoscaling group with the created AMI image as source to launch new instances.
Basically I can do 1 and 2 in one cloudformation template and 4 in a second template. What I don't seem able to do is to create an AMI image from an instance inside a cloudformation template, which basically generates the problem of having to manually remove the AMI if I want to remove the stack.
That being said, my questions are:
1.- Is there a way to create an AMI image from an instance INSIDE the cloudformation template?
2.- If the answer to 1 is no, is there a way to add an AMI image (or any other resource for that matter) to make it part of a completed stack?
EDIT:
Just to clarify, I've already solved the problem of creating the AMI and using it in a cloudformation template, I just can't create the AMI INSIDE the cloudformation template or add it somehow to the created stack.
As I commented on Rico's answer, what I do now is use an ansible playbook which basically has 3 steps:
1.- Create a base instance with a cloudformation template
2.- Create, using ansible, an AMI of the instance created on step 1
3.- Create the rest of the stack (ELB, autoscaling groups, etc) with a second cloudformation template that updates the one created on step 1, and that uses the AMI created on step 2 to launch instances.
This is how I manage it now, but I wanted to know if there's any way to create an AMI INSIDE a cloudformation template or if it's possible to add the created AMI to the stack (something like telling the stack, "Hey, this belongs to you as well, so handle it").
Yes, you can create an AMI from an EC2 instance within a CloudFormation template by implementing a Custom Resource that calls the CreateImage API on create (and calls the DeregisterImage and DeleteSnapshot APIs on delete).
Since AMIs can sometimes take a long time to create, a Lambda-backed Custom Resource will need to re-invoke itself if the wait has not completed before the Lambda function times out.
Here's a complete example:
Description: Create an AMI from an EC2 instance.
Parameters:
ImageId:
Description: Image ID for base EC2 instance.
Type: AWS::EC2::Image::Id
# amzn-ami-hvm-2016.09.1.20161221-x86_64-gp2
Default: ami-9be6f38c
InstanceType:
Description: Instance type to launch EC2 instances.
Type: String
Default: m3.medium
AllowedValues: [ m3.medium, m3.large, m3.xlarge, m3.2xlarge ]
Resources:
# Completes when the instance is fully provisioned and ready for AMI creation.
AMICreate:
Type: AWS::CloudFormation::WaitCondition
CreationPolicy:
ResourceSignal:
Timeout: PT10M
Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
UserData:
"Fn::Base64": !Sub |
#!/bin/bash -x
yum -y install mysql # provisioning example
/opt/aws/bin/cfn-signal \
-e $? \
--stack ${AWS::StackName} \
--region ${AWS::Region} \
--resource AMICreate
shutdown -h now
AMI:
Type: Custom::AMI
DependsOn: AMICreate
Properties:
ServiceToken: !GetAtt AMIFunction.Arn
InstanceId: !Ref Instance
AMIFunction:
Type: AWS::Lambda::Function
Properties:
Handler: index.handler
Role: !GetAtt LambdaExecutionRole.Arn
Code:
ZipFile: !Sub |
var response = require('cfn-response');
var AWS = require('aws-sdk');
exports.handler = function(event, context) {
console.log("Request received:\n", JSON.stringify(event));
var physicalId = event.PhysicalResourceId;
function success(data) {
return response.send(event, context, response.SUCCESS, data, physicalId);
}
function failed(e) {
return response.send(event, context, response.FAILED, e, physicalId);
}
// Call ec2.waitFor, continuing if not finished before Lambda function timeout.
function wait(waiter) {
console.log("Waiting: ", JSON.stringify(waiter));
event.waiter = waiter;
event.PhysicalResourceId = physicalId;
var request = ec2.waitFor(waiter.state, waiter.params);
setTimeout(()=>{
request.abort();
console.log("Timeout reached, continuing function. Params:\n", JSON.stringify(event));
var lambda = new AWS.Lambda();
lambda.invoke({
FunctionName: context.invokedFunctionArn,
InvocationType: 'Event',
Payload: JSON.stringify(event)
}).promise().then((data)=>context.done()).catch((err)=>context.fail(err));
}, context.getRemainingTimeInMillis() - 5000);
return request.promise().catch((err)=>
(err.code == 'RequestAbortedError') ?
new Promise(()=>context.done()) :
Promise.reject(err)
);
}
var ec2 = new AWS.EC2(),
instanceId = event.ResourceProperties.InstanceId;
if (event.waiter) {
wait(event.waiter).then((data)=>success({})).catch((err)=>failed(err));
} else if (event.RequestType == 'Create' || event.RequestType == 'Update') {
if (!instanceId) { failed('InstanceID required'); }
ec2.waitFor('instanceStopped', {InstanceIds: [instanceId]}).promise()
.then((data)=>
ec2.createImage({
InstanceId: instanceId,
Name: event.RequestId
}).promise()
).then((data)=>
wait({
state: 'imageAvailable',
params: {ImageIds: [physicalId = data.ImageId]}
})
).then((data)=>success({})).catch((err)=>failed(err));
} else if (event.RequestType == 'Delete') {
if (physicalId.indexOf('ami-') !== 0) { return success({});}
ec2.describeImages({ImageIds: [physicalId]}).promise()
.then((data)=>
(data.Images.length == 0) ? success({}) :
ec2.deregisterImage({ImageId: physicalId}).promise()
).then((data)=>
ec2.describeSnapshots({Filters: [{
Name: 'description',
Values: ["*" + physicalId + "*"]
}]}).promise()
).then((data)=>
(data.Snapshots.length === 0) ? success({}) :
ec2.deleteSnapshot({SnapshotId: data.Snapshots[0].SnapshotId}).promise()
).then((data)=>success({})).catch((err)=>failed(err));
}
};
Runtime: nodejs4.3
Timeout: 300
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal: {Service: [lambda.amazonaws.com]}
Action: ['sts:AssumeRole']
Path: /
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
- arn:aws:iam::aws:policy/service-role/AWSLambdaRole
Policies:
- PolicyName: EC2Policy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- 'ec2:DescribeInstances'
- 'ec2:DescribeImages'
- 'ec2:CreateImage'
- 'ec2:DeregisterImage'
- 'ec2:DescribeSnapshots'
- 'ec2:DeleteSnapshot'
Resource: ['*']
Outputs:
AMI:
Value: !Ref AMI
For what it's worth, here's Python variant of wjordan's AMIFunction definition in the original answer. All other resources in the original yaml remain unchanged:
AMIFunction:
Type: AWS::Lambda::Function
Properties:
Handler: index.handler
Role: !GetAtt LambdaExecutionRole.Arn
Code:
ZipFile: !Sub |
import logging
import cfnresponse
import json
import boto3
from threading import Timer
from botocore.exceptions import WaiterError
logger = logging.getLogger()
logger.setLevel(logging.INFO)
def handler(event, context):
ec2 = boto3.resource('ec2')
physicalId = event['PhysicalResourceId'] if 'PhysicalResourceId' in event else None
def success(data={}):
cfnresponse.send(event, context, cfnresponse.SUCCESS, data, physicalId)
def failed(e):
cfnresponse.send(event, context, cfnresponse.FAILED, str(e), physicalId)
logger.info('Request received: %s\n' % json.dumps(event))
try:
instanceId = event['ResourceProperties']['InstanceId']
if (not instanceId):
raise 'InstanceID required'
if not 'RequestType' in event:
success({'Data': 'Unhandled request type'})
return
if event['RequestType'] == 'Delete':
if (not physicalId.startswith('ami-')):
raise 'Unknown PhysicalId: %s' % physicalId
ec2client = boto3.client('ec2')
images = ec2client.describe_images(ImageIds=[physicalId])
for image in images['Images']:
ec2.Image(image['ImageId']).deregister()
snapshots = ([bdm['Ebs']['SnapshotId']
for bdm in image['BlockDeviceMappings']
if 'Ebs' in bdm and 'SnapshotId' in bdm['Ebs']])
for snapshot in snapshots:
ec2.Snapshot(snapshot).delete()
success({'Data': 'OK'})
elif event['RequestType'] in set(['Create', 'Update']):
if not physicalId: # AMI creation has not been requested yet
instance = ec2.Instance(instanceId)
instance.wait_until_stopped()
image = instance.create_image(Name="Automatic from CloudFormation stack ${AWS::StackName}")
physicalId = image.image_id
else:
logger.info('Continuing in awaiting image available: %s\n' % physicalId)
ec2client = boto3.client('ec2')
waiter = ec2client.get_waiter('image_available')
try:
waiter.wait(ImageIds=[physicalId], WaiterConfig={'Delay': 30, 'MaxAttempts': 6})
except WaiterError as e:
# Request the same event but set PhysicalResourceId so that the AMI is not created again
event['PhysicalResourceId'] = physicalId
logger.info('Timeout reached, continuing function: %s\n' % json.dumps(event))
lambda_client = boto3.client('lambda')
lambda_client.invoke(FunctionName=context.invoked_function_arn,
InvocationType='Event',
Payload=json.dumps(event))
return
success({'Data': 'OK'})
else:
success({'Data': 'OK'})
except Exception as e:
failed(e)
Runtime: python2.7
Timeout: 300
No.
I suppose Yes. Once the stack you can use the "Update Stack" operation. You need to provide the full JSON template of the initial stack + your changes in that same file (Changed AMI) I would run this in a test environment first (not production), as I'm not really sure what the operation does to the existing instances.
Why not create an AMI initially outside cloudformation and then use that AMI in your final cloudformation template ?
Another option is to write some automation to create two cloudformation stacks and you can delete the first one once the AMI that you've created is finalized.
While #wjdordan's solution is good for simple use cases, updating the User Data will not update the AMI.
(DISCLAIMER: I am the original author) cloudformation-ami aims at allowing you to declare AMIs in CloudFormation that can be reliably created, updated and deleted. Using cloudformation-ami You can declare custom AMIs like this:
MyAMI:
Type: Custom::AMI
Properties:
ServiceToken: !ImportValue AMILambdaFunctionArn
Image:
Name: my-image
Description: some description for the image
TemplateInstance:
ImageId: ami-467ca739
IamInstanceProfile:
Arn: arn:aws:iam::1234567890:instance-profile/MyProfile-ASDNSDLKJ
UserData:
Fn::Base64: !Sub |
#!/bin/bash -x
yum -y install mysql # provisioning example
# Signal that the instance is ready
INSTANCE_ID=`wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`
aws ec2 create-tags --resources $INSTANCE_ID --tags Key=UserDataFinished,Value=true --region ${AWS::Region}
KeyName: my-key
InstanceType: t2.nano
SecurityGroupIds:
- sg-d7bf78b0
SubnetId: subnet-ba03aa91
BlockDeviceMappings:
- DeviceName: "/dev/xvda"
Ebs:
VolumeSize: '10'
VolumeType: gp2

Resources