I am trying to add an iot rule with error action using following cloudformation yaml file
extract from yaml:
DaIoTRule:
Type: AWS::IoT::TopicRule
Properties:
RuleName: sda
TopicRulePayload:
RuleDisabled: false
Sql: SELECT *, topic(2) AS source FROM 'topic/sensors/+'
Actions:
- Lambda:
FunctionArn: !GetAtt LambdaFunction.Arn
ErrorAction:
S3:
RoleArn: !GetAtt DAIoTRuleErrorActionIamRole.Arn
Bucket: iot-message-dump
Key: "errors/${topic()}/${timestamp()}"
But I keep getting this error:
{
"StackId": "arn:aws:cloudformation:us-east-1:961234632786:stack/wx-da-lambda/91423s00-4e97-11ea-aedd-0ee829hbc650",
"EventId": "DAIoTRule-CREATE_FAILED-2020-02-17T06:51:38.299Z",
"StackName": "da-lambda",
"LogicalResourceId": "DaIoTRule",
"PhysicalResourceId": "",
"ResourceType": "AWS::IoT::TopicRule",
"Timestamp": "2020-02-17T06:51:38.299Z",
"ResourceStatus": "CREATE_FAILED",
"ResourceStatusReason": "Encountered unsupported property bucket"
}
I created this yaml using https://docs.aws.amazon.com/iot/latest/developerguide/rule-error-handling.html as reference.
Can someone point me to what I am doing wrong ?
CloudFormation uses the BucketName property instead of Bucket for S3 actions (including error actions).
See https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-iot-topicrule-s3action.html#cfn-iot-topicrule-s3action-bucketname
Related
Description of the problem
I have created a Lambda function with API Gateway in SAM, then deployed it and it was working as expected. In API Gateway I used HttpApi not REST API.
Then, I wanted to add a Lambda authorizer with Simple Response. So, I followed the SAM and API Gateway docs and I came up with the code below.
When I call the route items-list it now returns 401 Unauthorized, which is expected.
However, when I add the header myappauth with the value "test-token-abc", I get a 500 Internal Server Error.
I checked this page but it seems all of the steps listed there are OK https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-http-lambda-integrations/
I enabled logging for the API Gateway, following these instructions: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-logging.html
But all I get is something like this (redacted my IP and request ID):
[MY-IP] - - [07/Jul/2021:08:24:06 +0000] "GET GET /items-list/{userNumber} HTTP/1.1" 500 35 [REQUEST-ID]
(Perhaps I can configure the logger in such a way that it prints a more meaningful error message? EDIT: I've tried adding $context.authorizer.error to the logs, but it doesn't print any specific error message, just prints a dash: -)
I also checked the logs for the Lambda functions, there is nothing there (all logs where from the time before I added the authorizer).
So, what am I doing wrong?
What I tried:
This is my Lambda Authorizer function which I have deployed using sam deploy, when I test it in isolation using an event with the myappauth header, it works:
exports.authorizer = async (event) => {
let response = {
"isAuthorized": false,
};
if (event.headers.myappauth === "test-token-abc") {
response = {
"isAuthorized": true,
};
}
return response;
};
and this is the SAM template.yml which I deployed using sam deploy:
AWSTemplateFormatVersion: 2010-09-09
Description: >-
myapp-v1
Transform:
- AWS::Serverless-2016-10-31
Globals:
Function:
Runtime: nodejs14.x
MemorySize: 128
Timeout: 100
Environment:
Variables:
MYAPP_TOKEN: "test-token-abc"
Resources:
MyAppAPi:
Type: AWS::Serverless::HttpApi
Properties:
FailOnWarnings: true
Auth:
Authorizers:
MyAppLambdaAuthorizer:
AuthorizerPayloadFormatVersion: "2.0"
EnableSimpleResponses: true
FunctionArn: !GetAtt authorizerFunction.Arn
FunctionInvokeRole: !GetAtt authorizerFunctionRole.Arn
Identity:
Headers:
- myappauth
DefaultAuthorizer: MyAppLambdaAuthorizer
itemsListFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src/handlers/v1-handlers.itemsList
Description: A Lambda function that returns a list of items.
Policies:
- AWSLambdaBasicExecutionRole
Events:
Api:
Type: HttpApi
Properties:
Path: /items-list/{userNumber}
Method: get
ApiId: MyAppAPi
authorizerFunction:
Type: AWS::Serverless::Function
Properties:
Handler: src/handlers/v1-handlers.authorizer
Description: A Lambda function that authorizes requests.
Policies:
- AWSLambdaBasicExecutionRole
Edit:
User #petey suggested that I tried returning an IAM policy in my authorizer function, so I changed EnableSimpleResponses to false in the template.yml, then I changed my function as below, but got the same result:
exports.authorizer = async (event) => {
let response = {
"principalId": "my-user",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Action": "execute-api:Invoke",
"Effect": "Deny",
"Resource": event.routeArn
}]
}
};
if (event.headers.myappauth == "test-token-abc") {
response = {
"principalId": "my-user",
"policyDocument": {
"Version": "2012-10-17",
"Statement": [{
"Action": "execute-api:Invoke",
"Effect": "Allow",
"Resource": event.routeArn
}]
}
};
}
return response;
};
I am going to answer my own question because I have resolved the issue, and I hope this will help people who are going to use the new "HTTP API" format in API Gateway, since there is not a lot of tutorials out there yet; most examples you will find online are for the older API Gateway standard, which Amazon calls "REST API". (If you want to know the difference between the two, see here).
The main problem lies in the example that is presented in the official documentation. They have:
MyLambdaRequestAuthorizer:
FunctionArn: !GetAtt MyAuthFunction.Arn
FunctionInvokeRole: !GetAtt MyAuthFunctionRole.Arn
The problem with this, is that this template will create a new Role called MyAuthFunctionRole but that role will not have all the necessary policies attached to it!
The crucial part that I missed in the official docs is this paragraph:
You must grant API Gateway permission to invoke the Lambda function by using either the function's resource policy or an IAM role. For this example, we update the resource policy for the function so that it grants API Gateway permission to invoke our Lambda function.
The following command grants API Gateway permission to invoke your Lambda function. If API Gateway doesn't have permission to invoke your function, clients receive a 500 Internal Server Error.
The best way to solve this, is to actually include the Role definition in the SAM template.yml, under Resources:
MyAuthFunctionRole
Type: AWS::IAM::Role
Properties:
# [... other properties...]
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action:
- 'sts:AssumeRole'
Policies:
# here you will put the InvokeFunction policy, for example:
- PolicyName: MyPolicy
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action: 'lambda:InvokeFunction'
Resource: !GetAtt MyAuthFunction.Arn
You can see here a description about the various Properties for a role: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-iam-role.html
Another way to solve this, is to separately create a new policy in AWS Console, which has InvokeFunction permission, and then after deployment, attach that policy to the MyAuthFunctionRole that SAM created. Now the Authorizer will be working as expected.
Another strategy would be to create a new role beforehand, that has a policy with InvokeFunction permission, then copy and paste the arn of that role in the SAM template.yml:
MyLambdaRequestAuthorizer:
FunctionArn: !GetAtt MyAuthFunction.Arn
FunctionInvokeRole: arn:aws:iam::[...]
Your lambda authorizer is not returning what is expected to be an actual lambda authorizer (an IAM policy). This could explain that internal error 500.
To fix, replace is with something like this that returns an IAM policy (or rejects):
// A simple token-based authorizer example to demonstrate how to use an authorization token
// to allow or deny a request. In this example, the caller named 'user' is allowed to invoke
// a request if the client-supplied token value is 'allow'. The caller is not allowed to invoke
// the request if the token value is 'deny'. If the token value is 'unauthorized' or an empty
// string, the authorizer function returns an HTTP 401 status code. For any other token value,
// the authorizer returns an HTTP 500 status code.
// Note that token values are case-sensitive.
exports.handler = function(event, context, callback) {
var token = event.authorizationToken;
// modify switch statement here to your needs
switch (token) {
case 'allow':
callback(null, generatePolicy('user', 'Allow', event.methodArn));
break;
case 'deny':
callback(null, generatePolicy('user', 'Deny', event.methodArn));
break;
case 'unauthorized':
callback("Unauthorized"); // Return a 401 Unauthorized response
break;
default:
callback("Error: Invalid token"); // Return a 500 Invalid token response
}
};
// Help function to generate an IAM policy
var generatePolicy = function(principalId, effect, resource) {
var authResponse = {};
authResponse.principalId = principalId;
if (effect && resource) {
var policyDocument = {};
policyDocument.Version = '2012-10-17';
policyDocument.Statement = [];
var statementOne = {};
statementOne.Action = 'execute-api:Invoke';
statementOne.Effect = effect;
statementOne.Resource = resource;
policyDocument.Statement[0] = statementOne;
authResponse.policyDocument = policyDocument;
}
// Optional output with custom properties of the String, Number or Boolean type.
authResponse.context = {
"stringKey": "stringval",
"numberKey": 123,
"booleanKey": true
};
return authResponse;
}
Lots more information here : https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html#api-gateway-lambda-authorizer-lambda-function-create
Just to complete the answer. You have to add an AssumeRolePolicyDocument under Properties.
The role will then state
MyAuthFunctionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service:
- apigateway.amazonaws.com
Action:
- 'sts:AssumeRole'
Policies:
# see answer above
My use case is:
I want to execute a lambda function for the PUT event of an existing S3 bucket.
The problem is that you in CloudFormation or CDK you cannot add notifications for an existing bucket, only for buckets that are created.
To get around this I am trying to use a custom resource that adds the Lambda function to the PutNotification. I have this working fine in CloudFormation but I am trying to use CDK now to do something similar.
To simulate what I have in CloudFormation I need to add a bucket policy to the existing bucket granting permission to the action s3:PutBucketNotification to the lambda execution role principal.
In CloudFormation I do this like this:
NotificationBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref BucketName
PolicyDocument:
Statement:
- Effect: "Allow"
Action:
- 's3:PutBucketNotification'
Resource: !Sub "arn:aws:s3:::${BucketName}"
Principal:
AWS: !GetAtt LambdaExecutionRole.Arn
I am attempting to create the bucket policy and add the statement to it in CDK but I need the Arn of the Lambda Function's Arn
const bucket = Bucket.fromBucketName(this, "Bucket", "my-bucket-name");
const bucketConfigurationFunction = new lambda.SingletonFunction(this, "bucketConfigurationFunction ", {
runtime: lambda.Runtime.NODEJS_8_10,
code: lambda.Code.asset('lambda/bucket-configuration'),
handler: 'lambda_function.handler',
timeout: cdk.Duration.seconds(300),
uuid: '72561a5f-e772-4365-b3d1-f59e8ddc60b1'
})
const bucketPolicy = new BucketPolicy(this, "TargetBucketPolicy", {
bucket: bucket
})
const bucketPolicyStatement = new PolicyStatement()
bucketPolicyStatement.addActions("s3:PutBucketNotification");
//Need to put the execution role arn here but role is undefined
bucketPolicyStatement.addArnPrincipal(bucketConfigurationFunction.role.roleArn)
I have read the CDK creates a lambda function execution role automatically however when I try to access the role Arn to add it as the principal in the policy statement, it is undefined.
Am I doing this totally the wrong way?
Because TypeScript is quite strict on checking optional variables and the role is generated at runtime, you need to box it using and if but that's still fine. For example this works:
const bucketPolicyStatement = new iam.PolicyStatement()
bucketPolicyStatement.addActions("s3:PutBucketNotification");
if (bucketConfigurationFunction.role) {
bucketPolicyStatement.addArnPrincipal(bucketConfigurationFunction.role.roleArn)
}
const bucketPolicy = new s3.BucketPolicy(this, "TargetBucketPolicy", {
bucket: bucket,
})
bucketPolicy.document.addStatements(bucketPolicyStatement);
In Serverless, I am creating an AWS::Lambda::Permission CFN resource that allows Cognito to invoke a Custom Message Lambda Handler.
AWS::Lambda::Permission depends on the lambda. How do I ensure that the lambda is created first?
I have already tried adding a DependsOn property to the AWS::Lambda::Permission CFN resource with no luck.
Below is my CFN resource that is trying to add permissions to Cognito to invoke a lambda:
UserPoolLambdaInvokePermission:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:invokeFunction
Principal: cognito-idp.amazonaws.com
FunctionName: arn:aws:lambda:${self:provider.region}:#{AWS::AccountId}:function:${self:service}-${self:provider.stage}-cognitoCustomMessage
SourceArn: arn:aws:cognito-idp:${self:provider.region}:#{AWS::AccountId}:userpool/${self:provider.environment.USER_POOL_ID}
Here is what my lambda looks like in my serverless.yml:
cognitoCustomMessage:
handler: src/main/lambdas/users_handler.cognitoCustomMessage
Here is what my lambda is doing on a very basic level:
cognitoCustomMessage(event, next) {
if (event.triggerSource === 'CustomMessage_ForgotPassword') {
// do stuff
}
return next(null, event);
}
The error I get from the above is:
An error occurred: CognitoCustomMessageLambdaFunction - Function not
found:
arn:aws:lambda:us-west-2:1234567890:my-service-dev-cognitoCustomMessage
(Service: AWSLambdaInternal; Status Code: 404; Error Code:
ResourceNotFoundException; Request ID:
e2a98525-5090-4d0f-a1f5-20610474f93b).
If I add a DependsOn:
UserPoolLambdaInvokePermission:
Type: AWS::Lambda::Permission
DependsOn: arn:aws:lambda:${self:provider.region}:#{AWS::AccountId}:function:${self:service}-${self:provider.stage}-cognitoCustomMessage
...
....
The error I get from above is:
The CloudFormation template is invalid: Template format error:
DependsOn must be a string or list of strings.
I have also tried:
UserPoolLambdaInvokePermission:
Type: AWS::Lambda::Permission
DependsOn: CognitoCustomMessageLambdaFunction
...
....
The error I get from above is:
An error occurred: CognitoCustomMessageLambdaFunction - Function not
found:
arn:aws:lambda:us-west-2:1234567890:my-service-dev-cognitoCustomMessage
(Service: AWSLambdaInternal; Status Code: 404; Error Code:
ResourceNotFoundException; Request ID:
b888ae82-a0d7-4d69-888e-9e63027925c1).
I expect that there should be some method to create a lambda function first prior to the CFN resource needing to use it, but this does not seem to be the case with DependsOn.
The DependsOn attribute should have the logical name of the Lambda Function in the Cloud formation template not the ARN of Lambda Function. For example, if your Lambda function's logical name in the Cloud Formation template is MyLambda then DependsOn should be like this:
UserPoolLambdaInvokePermission:
Type: AWS::Lambda::Permission
DependsOn: MyLambda
NOTE: open cloud formation template .serverless/cloudformation-template-update-stack.json and look for logical lambda function name.
Example:
{
"MonitorLambdaFunction": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"S3Bucket": "deploy-dev",
"S3Key": "serverless/dev/1641551717730-2022-01-07T10:35:17.730Z/zip"
},
"Handler": "src/monitors/handler.devMonitor",
"Runtime": "nodejs14.x",
"FunctionName": "dev-monitor",
"MemorySize": 1024,
"Timeout": 6,
"Environment": {
"Variables": {
"STAGE": "dev",
}
},
"Role": {
"Fn::GetAtt": [
"monitorIamRoleLambdaExecution",
"Arn"
]
}
}
}
MonitorLambdaFunction is the name you are looking for.
Your function defined in serverless.yml is converted into a cloudformation resource under the hood. The resource is called XLambdaFunction where X = The name of your function with the first letter capitalized.
So if you have:
functions:
hello:
handler: handler.hello
...other function stuff...
You can reference:
DependsOn: HelloLambdaFunction
I can only assume that if your function is already named CognitoCustomMessageLambdaFunction then you would have to reference:
DependsOn: CognitoCustomMessageLambdaFunctionLambdaFunction
I faced similar issue, adding dependsOn key worked for me.
trying to install the approuter currently, following this tutorial:
https://blogs.sap.com/2017/07/18/step-7-with-sap-s4hana-cloud-sdk-secure-your-application-on-sap-cloud-platform-cloudfoundry/
When pushing the approuter to CF, I get an destination error:
xs-app.json/routes/0: Format validation failed (Route references unknown destination "service-destination")
This is my manifest.yml:
---
applications:
- name: xyz
command: 'node approuter/approuter.js'
host: xyz-93deb1cd-7b72-4060-94e7-21342342
path: approuter
memory: 128M
buildpack: https://github.com/cloudfoundry/nodejs-buildpack
env:
TENANT_HOST_PATTERN: 'xyz(.*).cfapps.eu10.hana.ondemand.com'
destinations: "[{"name":"service-destination", "url": "https://gfuowbasdatq19agtuthorizations-srv.cfapps.eu10.hana.ondemand.com", "forwardAuthToken": true}]"
SAP_JWT_TRUST_ACL: '[{"clientid" : "*", "identityzone" : "*"}]'
services:
- my-xsuaa
- service-destination
This is my xs-app.json:
{
"routes": [{
"source": "/",
"target": "/",
"destination": "service-destination"
}]
}
Where is the application actually searching for this destination? I created it in my CF-account as well, pointing to my service-url.
In YAML (as well as many other markup languages, e.g. JSON) double quotes inside double quotes need to be escaped. (Cf. the YAML spec section 7.3.1 and this blog post)
So for you there are two options for your destinations variable:
1. Replacing all " inside with \" (relatively cumbersome, especially if you want to change the destinations in the future)
2. Use single quotes as boundaries. So the value of your destinations variable would look like this:
'[{"name":"service-destination", "url": "https://gfuowbasdatq19agtuthorizations-srv.cfapps.eu10.hana.ondemand.com", "forwardAuthToken": true}]'
I want to use AWS macro Transform::Include with some dynamic parameters for my file.
Resources:
'Fn::Transform':
Name: 'AWS::Include'
Parameters:
TestMacroVariable:
Default: 2
Type: Number
Location: !Sub "s3://${InstallBucketName}/test.yaml"
test.yaml:
DataAutoScalingGroup:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
LaunchConfigurationName:
Ref: DataLaunchConfiguration
MinSize: '1'
MaxSize: '100'
DesiredCapacity:
Ref: TestMacroVariable
...
After calling: aws cloudformation describe-stack-events --stack-name $stack
I get:
"ResourceStatusReason": "The value of parameter TestMacroVariable
under transform Include must resolve to a string, number, boolean or a
list of any of these.. Rollback requested by user."
When I try to do it this way:
Resources:
'Fn::Transform':
Name: 'AWS::Include'
Parameters:
TestMacroVariable: 2
Location: !Sub "s3://${InstallBucketName}/test.yaml"
I get:
"ResourceStatusReason": "Template format error: Unresolved resource
dependencies [TestMacroVariable] in the Resources block of the
template"
Error is the same when I don't provide TestMacroVariable at all.
Tried with different types: String, Number, Boolean, List - none of them work.
As i know you cannot have anything other than Location key in the Parameters section of the AWS::Include. Check here AWS DOC
As an alternative, you can pass in the whole S3 path as a parameter and reference it in Location:
Parameters:
MyS3Path:
Type: String
Default: 's3://my-cf-templates/my-include.yaml'
...
'Fn::Transform':
Name: 'AWS::Include'
Parameters:
Location: !Ref MyS3Path
Building on what #BatteryAcid Said you can refer the parameters in your Cloudformation template directly from your file using Sub function:
In your CF template :
Parameters:
TableName:
Type: String
Description: Table Name of the Dynamo DB Users table
Default: 'Users'
In the file you are including:
"Resource": [
{
"Fn::Sub": [
"arn:${AWS::Partition}:dynamodb:${AWS::Region}:${AWS::AccountId}:table/${tableName}",
{
"tableName": {
"Ref": "TableName"
}
}
]
}
Alternatively doesn't have to be a parameter but any resource from your template :
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${QueryTelemtryFunction.Arn}/invocations