Lambda template fails for SubnetIds and SecurityGroupIds - aws-lambda

I have the following template in my sam function:
Resources:
TagChangedFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: tag_changed_function
Handler: tag_changed/app.lambda_handler
Runtime: python3.8
Policies:
- VPCAccessPolicy: {}
- Statement:
- Sid: EC2DescribeInstancesPolicy
Effect: "Allow"
Action:
- ec2:DescribeInstances
Resource: '*'
VpcConfig:
SubnetIds:
- sg-061328bxxxxx
SecurityGroupIds:
- subnet-03afd77xxxxx
Events:
TagChanged:
Type: CloudWatchEvent
Properties:
Pattern:
source:
- aws.tag
detail-type:
- Tag Change on Resource
(I masked the SubnetIds and SecurityGroupIds in the template with xxxxx).
But when I build and try to upload my code to aws, I get the following error message:
2 validation errors detected: Value
'[subnet-061328bxxxxx]' at
'vpcConfig.securityGroupIds' failed to satisfy
constraint: Member must satisfy constraint: [Member must
have length less than or equal to 1024, Member must have
length greater than or equal to 0, Member must satisfy
regular expression pattern: ^sg-[0-9a-z]*$]; Value
'[sg-03afd77xxxxx]' at 'vpcConfig.subnetIds' failed
to satisfy constraint: Member must satisfy constraint:
[Member must have length less than or equal to 1024,
Member must have length greater than or equal to 0,
Member must satisfy regular expression pattern:
^subnet-[0-9a-z]*$] (Service: AWSLambdaInternal; Status
Code: 400; Error Code: ValidationException; Request ID:
641be279-a48f-4249-b0a1-3e221f8bbdf
(again masking with xxxxxx)
As far as I can see, the regex constraints are satisfied. Do anyone see what is wrong in the template?
If I remove the VpcConfig section, it uploads fine.

You're giving sg ID in subnet section and Subnet ID in SG section. Kindly try the below
Resources:
TagChangedFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: tag_changed_function
Handler: tag_changed/app.lambda_handler
Runtime: python3.8
Policies:
- VPCAccessPolicy: {}
- Statement:
- Sid: EC2DescribeInstancesPolicy
Effect: "Allow"
Action:
- ec2:DescribeInstances
Resource: '*'
VpcConfig:
SubnetIds:
- subnet-03afd77xxxxx
SecurityGroupIds:
- sg-061328bxxxxx
Events:
TagChanged:
Type: CloudWatchEvent
Properties:
Pattern:
source:
- aws.tag
detail-type:
- Tag Change on Resource

Related

AWS lambda SAM deploy error - Template format error: Unresolved resource dependencies

I have am trying to deploy an aws lambda function using the SAM cli. I have some layers defined in the sam template. Testing locally using sam local start-api works quite well. The but deploying using the sam deploy --guided command throws the following error
Error: Failed to create changeset for the stack: sam-app, ex: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state: For expression "Status" we matched expected path: "FAILED" Status: FAILED. Reason: Template format error: Unresolved resource dependencies [arn:aws:lambda:us-west-1:338231645678:layer:ffmpeg:1] in the Resources block of the template
The SAM template is as follows
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
video-processor-functions
Functions to generate gif and thumbnail from uploaded videos
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Tracing: Active
Resources:
VideoProcessorFunctions:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: src/
Handler: app.lambdaHandler
Runtime: nodejs14.x
# timeout in seconds - 2 minutes
Timeout: 120
Layers:
- !Ref VideoProcessorDepLayer
- !Ref arn:aws:lambda:us-west-1:338231645678:layer:ffmpeg:1
Architectures:
- x86_64
Events:
HelloWorld:
Type: Api # More info about API Event Source: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#api
Properties:
Path: /hello
Method: get
VideoProcessorDepLayer:
Type: AWS::Serverless::LayerVersion
Properties:
LayerName: mh-video-processor-dependencies
Description: Dependencies for sam app [video-processor-functions]
ContentUri: dependencies/
CompatibleRuntimes:
- nodejs14.17
LicenseInfo: 'MIT'
RetentionPolicy: Retain
Outputs:
# ServerlessRestApi is an implicit API created out of Events key under Serverless::Function
# Find out more about other implicit resources you can reference within SAM
# https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
HelloWorldApi:
Description: "API Gateway endpoint URL for Prod stage for Hello World function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"
VideoProcessorFunctions:
Description: "Generate GIF and Thumnail from Video"
Value: !GetAtt VideoProcessorFunctions.Arn
VideoProcessorFunctionsIamRole:
Description: "Implicit IAM Role created for MH Video Processor function"
Value: !GetAtt VideoProcessorFunctionsRole.Arn
Any ideas what i'm doing wrong?
You are trying to reference a logical id that doesn’t exist.
Just put the layer's ARN of 'ffmpeg':
Layers:
- !Ref VideoProcessorDepLayer
- arn:aws:lambda:us-west-1:338231645678:layer:ffmpeg:1

An error occurred: IamRoleLambdaExecution - Maximum policy size of 10240 bytes exceeded for role

using serverless-plugin-split-stacks in serverless.yml and getting this error
An error occurred: IamRoleLambdaExecution - Maximum policy size of 10240 bytes exceeded for role Vkonnect-dev-ap-south-1-lambdaRole (Service: AmazonIdentityManagement; Status Code: 409; Error Code: LimitExceeded; Request ID: 51920d55-4b81-4b6c-99f1-d9f0ba087cc2; Proxy: null).
when i use serverless-plugin-custom-roles i get this error
The CloudFormation template is invalid: Circular dependency between resources: [GenerateOtpDocLambdaPermissionApiGateway, DoctorUnderscorelistLambdaPermissionApiGateway .......]
serverless.yml
service: Vkonnect #Name of your App
provider:
name: aws
runtime: nodejs14.x # Node JS version
memorySize: 128
timeout: 10
stage: dev
region: ap-south-1 # AWS region
deploymentBucket:
name: vkonnectlayers
# iamRoleStatements:
# - Effect: "Allow"
# Action:
# - "s3:*"
# Resource:
# NOTE you can't refer to the LogicalID of S3Bucket, otherwise
# there will be a circular reference in CloudFormation
iamRoleStatements:
- Effect: "Allow"
Action:
- "cloudformation:*"
- "codecommit:*"
- "apigateway:*"
- "execute-api:Invoke"
- "execute-api:ManageConnections"
- "cloudformation:DescribeStacks"
- "cloudformation:ListStackResources"
- "cloudwatch:ListMetrics"
- "cloudwatch:GetMetricData"
- "ec2:DescribeSecurityGroups"
- "ec2:DescribeSubnets"
- "ec2:DescribeVpcs"
- "kms:ListAliases"
- "iam:GetPolicy"
- "iam:GetPolicyVersion"
- "iam:GetRole"
- "iam:GetRolePolicy"
- "iam:ListAttachedRolePolicies"
- "iam:ListRolePolicies"
- "iam:ListRoles"
- "lambda:*"
- "logs:DescribeLogGroups"
- "states:DescribeStateMachine"
- "states:ListStateMachines"
- "tag:GetResources"
- "xray:GetTraceSummaries"
- "xray:BatchGetTraces"
Resource:
- "*"
- "arn:aws:apigateway:*::/*"
- "arn:aws:events:*:*:rule/codecommit*"
- "arn:aws:logs:*:*:log-group:/aws/lambda/*"
plugins:
- serverless-offline
- serverless-layers
- serverless-plugin-split-stacks
- serverless-plugin-custom-roles
# - serverless-nested-stack
package:
individually: true
exclude:
- ./**
custom:
splitStacks:
perFunction: false
perType: false
perGroupFunction: true
nestedStackCount: 5
serverless-layers: # All Layers
- moment:
name: moment
excludeDevDependencies: false
individually: true
dependenciesPath: ./layers/moment-layer/package.json
package:
patterns:
- /**
- "!node_modules/**"
first create
IAM role
in your aws account with full access to the service that u want then do following
serverless.yaml
provider:
name: aws
runtime: nodejs14.x
memorySize: 128
timeout: 5
stage: prod
region: us-east-1 # AWS region
versionFunctions: false
deploymentBucket:
name: XXXXXX
iam:
role: arn:aws:iam::XXXXXX:role/full //your role arn
plugins:
- serverless-offline
- serverless-layers
- serverless-plugin-split-stacks
package:
individually: true
exclude:
- ./**
For policy size limit error:
AWS has a limit on policy size. Check this article for reference: https://aws.amazon.com/premiumsupport/knowledge-center/iam-increase-policy-size/
For circular dependency error:
Check this AWS blog: https://aws.amazon.com/blogs/infrastructure-and-automation/handling-circular-dependency-errors-in-aws-cloudformation/
AWS is setting limit on few of the resources like IAM, S3 etc. Resources should not exceed whatever the limit is set. You can submit a request to AWS Support to increase the limit.
Before that, you can go to service quota in AWS to know the limit for AWS resources. Based on that you can take a call to submit a request to AWS or follow the above document to reduce the size.

AWS SAM - Apply policy template for a resource created conditionally

I create a DynamoDb table conditionally:
MyDynamoTable:
Type: AWS::DynamoDB::Table
Condition: IsDevAccount
and this is how IsDevAccount is defined using an input parameter:
Conditions:
IsDevAccount: !Equals [ !Ref Stage, dev ]
Now I'm creating a Lambda function that accepts the table's name (amongst other things) as input through environment variables. This is done conditionally, too. Within the function's code, I'd check if the table name is passed (pass empty if condition isn't met). If so, I'd put some items into it.
However, I'm not sure how to apply policy templates to the function's role conditionally. Normally I do it like this:
MyFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- DynamoDBWritePolicy:
TableName: !Ref MyDynamoTable
What happens to the function's execution role if the table isn't created because the condition isn't met (e.g.: in another account)? Can I apply this policy template conditionally, as well?
What I don't want to do is to blindly give write permission to all DynamoDB tables within the account.
Yes, you could add the condition to the DB write policy so that only when the condition is met, it will allow the write policy.
You're creating the table only if the environment is staging or development, you could apply a condition on the policy to check for your table name then apply the write policy. Example below
MyDynamoTable:
Type: AWS::DynamoDB::Table
Condition: IsDevAccount
Conditions:
IsDevAccount: !Equals [ !Ref Stage, dev ]
MyFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- DynamoDBWritePolicy:
Condition: !Equals [ !Ref MyDynamoTable, "myTableName" ],
TableName: !Ref MyDynamoTable
Update in response to comments:
!Ref returns the value of the specified parameter or resource. We need parameters with allowed values for the environment and DBtable for the condition.
Parameters:
Environment:
Type: String
Default: dev
AllowedValues:
- dev
- stage
- prod
MyDynamoTable:
Description: table name for the db
Type: String
AllowedValues:
- tableOne
- tableTwo
- myTableName
Conditions:
IsDevAccount: !Equals [ !Ref Environment, "dev" ]
TableExists: !Equals [ !Ref MyDynamoTable, "myTableName" ]
MyFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- DynamoDBWritePolicy:
Condition: !And [IsDevAccount, TableExists] // Only with TableExists condition, it'll work fine with the added parameters
TableName: !Ref MyDynamoTable
Ref:- https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/parameters-section-structure.html
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/conditions-section-structure.html
Update 2:
Agreed, I researched and confirmed, there is no way to check for resources created in the same stack template (That's why suggested parameter). Conditions are all parameter based.
However if the resource was created already in other stack, you could do this through Resource import. I don't think, resource import will be of help in your requirement.
However, a workaround would be to have Boolean parameters for TableExists condition and can pass the value through AWS CLI on the run like below,
MyDynamoTable:
Description: dynamo db table
Type: String
AllowedValues:
- true
- false
Conditions:
TableExists: !Equals [ !Ref MyDynamoTable, "true" ]
MyFunction:
Type: AWS::Serverless::Function
Properties:
Policies:
- DynamoDBWritePolicy:
Condition: !Ref TableExists
TableName: !Ref MyDynamoTable
AWS CLI on deploy pass required parameters
aws cloudformation deploy --template templateName.yml --parameter-overrides MyDynamoTable="true" dynamoDBtableName="myTableName" (any parameter required)

In CloudFormation, how do I target a Lambda alias in Events::Rule

I'm trying to trigger a Lambda:alias (the alias is key here) on a schedule. The following code errors out with
"SampleLambdaLiveAlias is not valid. Reason: Provided Arn is not in
correct format. (Service: AmazonCloudWatchEvents; Status Code: 400;
Error Code: ValidationException;"
How do I properly target the lambda:alias in CloudFormation? I've tried !Ref, !Sub and just the logical name.
My custom-resource approach to retrieving the latest lambda version appears to be a necessary evil of setting up the "live" alias because AWS maintains old lambda versions, even after you delete the lambda and stack AND a valid version is required for a new alias. If anyone knows a more elegant approach to that problem, please see: how-to-use-sam-deploy-to-get-a-lambda-with-autopublishalias-and-additional-alises
SampleLambdaFunction:
Type: AWS::Serverless::Function
Properties:
FunctionName: SampleLambda
AutoPublishAlias: staging
CodeUri: src/
Handler: SampleLambda.handler
MemorySize: 512
Runtime: nodejs12.x
Role: !GetAtt SampleLambdaRole.Arn
SampleLambdaLiveAlias:
Type: AWS::Lambda::Alias
Properties:
FunctionName: !Ref SampleLambdaFunction
FunctionVersion: !GetAtt SampleLambdaGetMaxVersionFunction.version
Name: live
SampleLambdaFunctionScheduledEvent:
Type: AWS::Events::Rule
Properties:
State: ENABLED
ScheduleExpression: rate(1 minute) # same as cron(0/1 * * * ? *)
Description: Run SampleLambdaFunction once every 5 minutes.
Targets:
- Id: EventSampleLambda
Arn: SampleLambdaLiveAlias
Your error is in the last line of the piece of configuration you shared. In order to get the resource ARN you need to use Ref intrinsic function such as, !Ref SampleLambdaLiveAlias:
SampleLambdaFunctionScheduledEvent:
Type: AWS::Events::Rule
Properties:
State: ENABLED
ScheduleExpression: rate(1 minute) # same as cron(0/1 * * * ? *)
Description: Run SampleLambdaFunction once every 5 minutes.
Targets:
- Id: EventSampleLambda
Arn: !Ref SampleLambdaLiveAlias
Be aware that Ref intrinsic function may return different things for different types of resources. For Lambda alias it returns the ARN, just what you need.
You can check the official documentation for more detail.

AWS Cloud Formation !Sub & !Ref functions inside AWS::Serverless::Function Policies

I have been using the !Sub function in my CloudFormation Yaml templates just fine. And when used it as an object property value it works for me
Object:
Property1: !Sub some-value-with-a-${variable}-in-it
The value of variable gets replaced as expected.
However, I can't figure out how to use the !Sub function in an element of a string array
Array:
- !Sub some-value-with-a-${variable}-in-it
That array element just gets ignored.
I am trying this in the context of a SAM template creating a AWS::Serverless::Function type resource. The Policies property can take an array of strings:
lambda:
Type: AWS::Serverless::Function
Properties:
CodeUri: api
FunctionName: !Sub api-${MyStageName}
Handler: Lambda:Api.Function::HandleAsync
Runtime: dotnetcore1.0
Policies:
- AWSLambdaBasicExecutionRole
- !Sub arn:aws:iam::${AWS::AccountId}:policy/some-policy
- arn:aws:iam::123456789:policy/another-policy
The !Sub function works in the FunctionName property in this example. But I only end up with 2 Policies attached to my generated role - AWSLambdaBasicExecutionRole and arn:aws:iam::123456789:policy/another-policy. The one including the !Sub function gets ignored.
I have tried options like putting the function on a new line:
Array:
-
!Sub some value with a ${variable} in it
Can anyone help?
Update
#Tom Melo pointed out that this is not a array problem so I have adjusted my question.
Further investigation has revealed it is not a Cloud Formation issue exactly, but very specific to the AWS::Serverless::Function resource type, and the Policies property within in. I suspect it has something to do with the fact that the Policies property is so flexible in what it can accept. It can accept strings referring to policy names or Arns, and it can also accept policy documents describing new policy. I suspect this means it is not able to support the functions.
Apparently, there's nothing wrong with !Sub function in a array of elements.
I have tried to create the following stack on Cloudformation and it worked:
AWSTemplateFormatVersion: '2010-09-09'
Description: 'IAM Roles Template'
Parameters:
ArnBase:
Type: String
Default: arn:aws:iam::aws:policy/
AWSLambdaFullAccess:
Type: String
Default: AWSLambdaFullAccess
AmazonSESFullAccess:
Type: String
Default: AmazonSESFullAccess
Resources:
LambdaRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
-
PolicyName: CloudFormationFullAccess
PolicyDocument:
Version: "2012-10-17"
Statement:
-
Effect: "Allow"
Action: "cloudformation:*"
Resource: "*"
ManagedPolicyArns:
- !Sub ${ArnBase}${AWSLambdaFullAccess}
- !Sub ${ArnBase}${AmazonSESFullAccess}
- !Sub arn:aws:iam::${AWS::AccountId}:policy/CustomAmazonGlacierReadOnlyAccess
That should work using SAM either...
Workaround
The AWS::Serverless::Function resource type supports several ways of configuring access.
Refer to policies directly via the Policies property and have the framework create a role that has those policies.
The Role property can refer to a role which already contains policies.
I was using option 1, but option 2 proves to be a way around the issues with !Sub function.
Creating a role explicitly with the policies we want using the AWS::IAM::Role means we can use the !Sub function within the ManagedPolicyArns property. For example
role:
Type: AWS::IAM::Role
Properties:
...
ManagedPolicyArns:
- !Sub arn:aws:iam::${AWS::AccountId}:policy/some-policy
...
lambda:
Type: AWS::Serverless::Function
Properties:
...
Role: !GetAtt role.Arn
...

Resources