Execution failed due to configuration error: Invalid permissions on Lambda function - aws-lambda

I am building a serverless application using AWS Lambda and API Gateway via Visual Studio. I am working in C#, and using the serverless application model (SAM) in order to deploy my API. I build the code in Visual Studio, then deploy via publish to Lambda. This is working, except every time I do a new build, and try to execute an API call, I get this error:
Execution failed due to configuration error: Invalid permissions on Lambda function
Doing some research, I found this fix mentioned elsewhere (to be done via the AWS Console):
Fix: went to API Gateway > API name > Resources > Resource name > Method > Integration Request > Lambda Function and reselected my existing function, before "saving" it with the little checkmark.
Now this works for me, but it breaks the automation of using the serverless.template (JSON) to build out my API. Does anyone know how to fix this within the serverless.template file? So that I don't need to take action in the console to resolve? Here's a sample of one of my methods from the serverless.template file
{
"AWSTemplateFormatVersion" : "2010-09-09",
"Transform" : "AWS::Serverless-2016-10-31",
"Description" : "An AWS Serverless Application.",
"Resources" : {
"Get" : {
"Type" : "AWS::Serverless::Function",
"Properties": {
"VpcConfig":{
"SecurityGroupIds" : ["sg-111a1476"],
"SubnetIds" : [ "subnet-3029a769","subnet-5ec0b928"]
},
"Handler": "AWSServerlessInSiteDataGw::AWSServerlessInSiteDataGw.Functions::Get",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [ "AWSLambdaBasicExecutionRole","AWSLambdaVPCAccessExecutionRole","AmazonSSMFullAccess"],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/",
"Method": "GET"
}
}
}
}
},

You may have an issue in permission config, that's why API couldn't call your lambda. try to explicitly add to template.yaml file invoke permission to your lambda from apigateway as a principal here's a sample below:
ConfigLambdaPermission:
Type: "AWS::Lambda::Permission"
DependsOn:
- MyApiName
- MyLambdaFunctionName
Properties:
Action: lambda:InvokeFunction
FunctionName: !Ref MyLambdaFunctionName
Principal: apigateway.amazonaws.com
Here's the issue that was reported in SAM github repo for complete reference and here is an example of hello SAM project
If you would like to add permission by AWS CLI for testing things out, you may want to use aws lambda add-permission. please visit official documentation website for more details.

I had a similar issue - I deleted then re-installed a lambda function. My API Gateway was still pointing at the old one, so I had to go into the API Gateway and change my Resource Methods to alter the Integration Request setting to point to the new one (it may look like it's pointing to the correct one but wasn't in my case)

I was having the same issue but I was deploying through Terraform. After a suggestion from another user, I reselected my Lambda function in the Integration part of the API Gateway, and then checked what changed in my Lambda permissions. Turns out I needed to add a "*" where I was putting the stage name in the source_arn section of the API Gateway trigger in my Lambda resource. Not sure how SAM compares to Terraform but perhaps you can change the stage name or just try this troubleshooting technique that I tried.
My SO posting: AWS API Gateway and Lambda function deployed through terraform -- Execution failed due to configuration error: Invalid permissions on Lambda function

Same error, and the solution was simple: clearing and applying the "Lambda Function" mapping again in the integration setting of the API Gateway.
My mapping looks like this: MyFunction-894AR653OJX:test where "test" is the alias to point to the right version of my lambda
The problem was caused by removing the ALIAS "test" on the lambda, and recreating it on another version (after publishing). It seems that the API gateway internally still links to the `old' ALIAS instance.
You would expect that the match is purely done on name...
Bonus: so, via the AWS console you cannot move that ALIAS, but you can do this via the AWS CLI, using the following command:
aws lambda --profile <YOUR_PROFILE> update-alias --function-name <FUNCTION_NAME> --name <ALIAS_NAME> --function-version <VERSION_NUMBER>

I had the same issue. I changed the integration to mock first, i.e unsetting the integration type to Lambda, and then after one deployment, set the integration type to lambda again. It worked flawlessly thereafter.
I hope it helps.

Facing the same issue, I figured out the problem is : API Gateway is not able to invoke the Lambda function as I couldn't see any CloudWatch logs for the lambda Function.
So firstly I went through API Gateway console and under the Integration Request - gave the full ARN for the Lambda Function. and it is started working.
Secondly, through the CloudFormation
x-amazon-apigateway-integration:
credentials:
Fn::Sub: "${ApiGatewayLambdaRole.Arn}"
type: "aws"
uri:
Fn::Sub: "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${lambda_function.Arn}/invocations"

I had the same problem so I deleted then created the stack and it worked.

Looks like "Execution failed due to configuration error: Invalid permissions on Lambda function" is a catch all for multiple things :D
I deployed a stack with CloudFormation templates and hit this issue.
I was using the stage name in the SourceArn for the AWS::Lambda::Permission segment.
when i changed that to a * AWS was a bit more explicit about the cause, which in my case happened to be an invalid Handler reference (I was using Java, the handler had moved package) in the AWS::Lambda::Function section.
Also, when i hit my API GW i got this message
{
"message": "Internal server error"
}
It was only when I was at the console and sent through the payload as a test for the resource that I got the permissions error.
If I check the Cloudwatch logs for the API GW when I configured that, it does indeed mention the true cause even when the Stage Name is explicit.
Lambda execution failed with status 200 due to customer function error: No public method named ...

In my case, I got the error because the Lambda function had been renamed. Double-check your configuration just in case.
Technically, the error message was correct—there was no function, and therefore no permissions. A helpful message would, of course, have been useful.

I had a similar problem and was using Terraform. It needed the policy with the "POST" in it. For some reason the /*/ (wildcard) policy didn't work?
Here's the policy and the example terraform I used to solve the issue.
Many thanks to all above.
Here is what my Lambda function policy JSON looked like and the terraform:
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "AllowAPIGatewayInvoke",
"Effect": "Allow",
"Principal": {
"Service": "apigateway.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:999999999999:function:MY-APP",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:execute-api:us-east-1:999999999999:d85kyq3jx3/test/*/MY-APP"
}
}
},
{
"Sid": "e841fc76-c755-43b5-bd2c-53edf052cb3e",
"Effect": "Allow",
"Principal": {
"Service": "apigateway.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:999999999999:function:MY-APP",
"Condition": {
"ArnLike": {
"AWS:SourceArn": "arn:aws:execute-api:us-east-1:999999999999:d85kyq3jx3/*/POST/MY-APP"
}
}
}
]
}
add in a terraform like this:
//************************************************
// allows you to read in the ARN and parse out needed info, like region, and account
//************************************************
data "aws_arn" "api_gw_deployment_arn" {
arn = aws_api_gateway_deployment.MY-APP_deployment.execution_arn
}
//************************************************
// Add in this to support API GW testing in AWS Console.
//************************************************
resource "aws_lambda_permission" "apigw-post" {
statement_id = "AllowAPIGatewayInvokePOST"
action = "lambda:InvokeFunction"
//function_name = aws_lambda_function.lambda-MY-APP.arn
function_name = module.lambda.function_name
principal = "apigateway.amazonaws.com"
// "arn:aws:execute-api:us-east-1:473097069755:708lig5xuc/dev/POST1/cloudability-church-ws"
source_arn = "arn:aws:execute-api:${data.aws_arn.api_gw_deployment_arn.region}:${data.aws_arn.api_gw_deployment_arn.account}:${aws_api_gateway_deployment.MY-APP_deployment.rest_api_id}/*/POST/${var.api_gateway_root_path}"
}

The documentation for AWS lambda resource permissions shows 3 levels of access you can filter or wildcard, /*/*/*, which is documented as $stage/$method/$path. However, their example and most examples online only use 2 levels and I was bashing my head against the wall using 3 only to get Access Denied. I changed down to 2 levels and lambda then created the trigger. Hopefully, this will save someone from throwing their computer against the wall.

In my case I used Lambda path, that doesn't starts with a '/', like Path: "example/path" in my template.yaml.
As a result AWS generate incorrect permission for this lambda:
{
"ArnLike": {
"AWS:SourceArn": "arn:aws:execute-api:{Region}:{AccountId}:{ApiId}/*/GETexample/path/*"
}
}
So I fixed it by adding '/' to my lambda path in the template.

Related

Unable to reference public Lambda ARN layer when publishing via boto3

I have a public ARN layer that I wish to use in deploying a Lambda function.
The relevant snippet of my code is below:
LAMBDA_ARN_LAYER = "arn:aws:lambda:eu-north-1:012345678901:layer:canedge-influxdb-writer-2:5"
lambda_client = session.client("lambda")
lambda_client.create_function(
FunctionName=LAMBDA_FUNCTION_NAME,
Runtime=PYTHON_BUILD,
Role=LAMBDA_ROLE_ARN,
Handler=LAMBDA_HANDLER,
Code={"ZipFile": open(f"{LAMBDA_ZIP_FILE}.zip", "rb").read()},
Timeout=180,
MemorySize=1024,
Layers=[LAMBDA_ARN_LAYER],
)
When I run this with my own account (which I also used to create and publish the ARN layers), it works as it should. However, if I try to deploy the Lambda function with a different account, I get the following error:
An error occurred (AccessDeniedException) when calling the CreateFunction operation: User: arn:aws:iam::XXX:user/XXX is not authorized to perform: lambda:GetLayerVersion on resource: arn:aws:lambda:eu-north-1:012345678901:layer:canedge-influxdb-writer-2:5 because no resource-based policy allows the lambda:GetLayerVersion action
What is strange is that I can "manually" go into my Lambda function with the same user and add the ARN layer and it works as it should, so it appears to be public.
When I created the Lambda ARN layer originally, I used below code (in a Python script referencing the AWS CLI via subprocess:
aws lambda publish-layer-version --region {region} --layer-name {layer_name} --description "{layer_description}" --cli-connect-timeout 6000 --license-info "MIT" --zip-file "fileb://canedge-influxdb-writer.zip" --compatible-runtimes python3.9
aws lambda add-layer-version-permission --layer-name {layer_name} --version-number {version} --statement-id allAccountsExample --principal * --action lambda:GetLayerVersion --region {region}
The user I am trying to deploy with has the default admin access rights:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "*",
"Resource": "*"
}
]
}
What am I missing in the above?
The problem was that I did not explicitly define the region_name when setting up the boto3 client initially. After fixing this, the problem was resolved.

Debugging lambda locally using cdk not sam

AWS CDK provides great features for developers. Using CDK deveolper can manage not only total infrastructure but also security, codepipeline, ...
However I recently struggling something. I used to debug lambda using SAM for local debugging. I know how to set up CDK environment, and debug CDK application itself. But I can't figure out how to debug lambda application inside CDK.
Can anyone help me?
As of 4/29/2021, there's an additional option for debugging CDK apps via SAM. It's in preview but this blog post covers it : https://aws.amazon.com/blogs/compute/better-together-aws-sam-and-aws-cdk/.
Basically, install the AWS CLI and AWS CDK. The install the SAM CLI - beta, available here : https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-cdk-getting-started.html.
Then you can run command's like:
sam-beta-cdk build sam-beta-cdk local invoke sam-beta-cdk local invoke start-api and even emulate the Lambda service with sam-beta-cdk local start-lambda
You can use SAM and CDK together as described here. In particular:
Run your AWS CDK app and create a AWS CloudFormation template
cdk synth --no-staging > template.yaml
Find the logical ID for your
Lambda function in template.yaml. It will look like
MyFunction12345678, where 12345678 represents an 8-character unique
ID that the AWS CDK generates for all resources. The line right
after it should look like: Type: AWS::Lambda::Function
Run the function by executing:
sam local invoke MyFunction12345678 --no-event
if you are using VSCode, you can set up a launch action to run the current file in node to test it locally. All you need to do is hit F5 on the file you want to test.
You will need to add the following at the end of your handler files so that when executed in node the handler gets executed:
if (process.env.NODE_ENV === "development" && process.argv.includes(__filename)) {
// Exercise the Lambda handler with a mock API Gateway event object.
handler(({
pathParameters: {
param1: "test",
param2: "code",
},
} as unknown) as APIGatewayProxyEvent)
.then((response) => {
console.log(JSON.stringify(response, null, 2));
return response;
})
.catch((err: any) => console.error(err));
}
Add this to your launch configurations in your .vscode/launch.json:
"configurations": [
{
"name": "Current TS File",
"type": "node",
"request": "launch",
"args": ["${relativeFile}", "-p", "${workspaceFolder}/tsconfig.json"],
"runtimeArgs": ["-r", "ts-node/register", "-r", "tsconfig-paths/register", "--nolazy"],
"cwd": "${workspaceRoot}",
"internalConsoleOptions": "openOnSessionStart",
"envFile": "${workspaceFolder}/.env",
"smartStep": true,
"skipFiles": ["<node_internals>/**", "node_modules/**"]
},
The ts-node and tsconfig-paths are only needed if using Typescript. You must add those with npm i -D ts-node tsconfig-paths if you dont already have them.
Before you run any of the sam local commands with a AWS CDK application, you must run cdk synth.
When running sam local invoke you need the function construct identifier that you want to invoke, and the path to your synthesized AWS CloudFormation template. If your application uses nested stacks, to resolve naming conflicts, you also need the stack name where the function is defined.
# Invoke the function FUNCTION_IDENTIFIER declared in the stack STACK_NAME
sam local invoke [OPTIONS] [STACK_NAME/FUNCTION_IDENTIFIER]
# Start all APIs declared in the AWS CDK application
sam local start-api -t ./cdk.out/CdkSamExampleStack.template.json [OPTIONS]
# Start a local endpoint that emulates AWS Lambda
sam local start-lambda -t ./cdk.out/CdkSamExampleStack.template.json [OPTIONS]

Serverless Framework - What permissions do I need to use AWS SSM Parameter Store?

I'm opening this question because there seems to be no documentation on this, so I would like to provide the answer after much time wasted in trial and error.
As background, the Serverless framework [allows loading both plaintext & SecureString values from AWS SSM Parameter Store].1
What permissions are needed to access & load these SSM Parameter Store values when performing serverless deploy?
In general, accessing & decrypting AWS SSM parameter store values requires these 3 permissions:
ssm:DescribeParameters
ssm:GetParameters
kms:Decrypt
-
Here's a real world example that only allows access to SSM parameters relating to my lambda functions (distinguished by following a common naming convention/pattern) - it works under the following circumstances:
SecureString values are encrypted with the default AWS SSM encryption key.
All parameters use the following naming convention
a. /${app-name-or-app-namespace}/serverless/${lambda-function-name/then/whatever/else/you/want
b.${lambda-function-name} must begin with sls-
So let's say I have an app called myCoolApp, and a Lambda function called sls-myCoolLambdaFunction. Perhaps I want to save database config values such as username and password.
I would have two SSM parameters created:
/myCoolApp/serverless/sls-myCoolLambdaFunction/dev/database/username (plaintext)
/myCoolApp/serverless/sls-myCoolLambdaFunction/dev/database/password (SecureString)
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ssm:DescribeParameters"
],
"Resource": [
"arn:aws:ssm:${region-or-wildcard}:${aws-account-id-or-wildcard}:*"
]
},
{
"Effect": "Allow",
"Action": [
"ssm:GetParameter"
],
"Resource": [
"arn:aws:ssm:${region-or-wildcard}:${aws-account-id-or-wildcard}:parameter/myCoolApp/serverless/sls-*"
]
},
{
"Effect": "Allow",
"Action": [
"kms:Decrypt"
],
"Resource": [
"arn:aws:kms:*:${aws-account-id}:key/alias/aws/ssm"
]
}
]
}
Then in my serverless.yml file, I might reference these two SSM values as function level environment variables like so
environment:
DATABASE_USERNAME: ${ssm:/myCoolApp/serverless/sls-myCoolLambdaFunction/dev/database/username}
DATABASE_PASSWORD: ${ssm:/myCoolApp/serverless/sls-myCoolLambdaFunction/dev/database/password~true}
Or, even better, if I want to be super dynamic for situations where I have different config values depending on the stage, I can set the environment variables like so
environment:
DATABASE_USERNAME: ${ssm:/myCoolApp/serverless/sls-myCoolLambdaFunction/${self:provider.stage}/database/username}
DATABASE_PASSWORD: ${ssm:/myCoolApp/serverless/sls-myCoolLambdaFunction/${self:provider.stage}/database/password~true}
With this above example, if I had two stages - dev & prod, perhaps I would create the following SSM parameters:
/myCoolApp/serverless/sls-myCoolLambdaFunction/dev/database/username (plaintext)
/myCoolApp/serverless/sls-myCoolLambdaFunction/dev/database/password (SecureString)
/myCoolApp/serverless/sls-myCoolLambdaFunction/prod/database/username (plaintext)
/myCoolApp/serverless/sls-myCoolLambdaFunction/prod/database/password (SecureString)
I suggest to use AWS SDK to get SSM parameters in code instead of saving in environment file (i.e. .env). It's more secure that way. You need to assign permission to the role you use with action=ssm:GetParameter and resource point to the parameter in the SSM Parameter store. I use serverless framework for deployment. Below is what I have in serverless.yml assumming parameter names with pattern "{stage}-myproject-*" (e.g. dev-myproject-username, qa-myproject-password):
custom:
myStage: ${opt:stage}
provider:
name: aws
runtime: nodejs10.x
stage: ${self:custom.myStage}
region: us-east-1
myAccountId: <aws account id>
iamRoleStatements:
- Effect: Allow
Action:
- ssm:GetParameter
Resource: "arn:aws:ssm:${self:provider.region}:${self:provider.myAccountId}:parameter/${self:provider.stage}-myproject-*"
two useful resources are listed below:
where to save credentials?
wireless framework IAM doc
In the case you are using codebuild in a ci/cd pipeline, dont forget to add the ssm authorization policies to the codebuild service role. (when we are talking about ssm we have to differenciate between secretsmanager and parametstore)

CloudFormation Permission Denied for Config Attempting to Invoke Lambda

I am attempting to create a CloudFormation Template which creates an AWS Config rule which will invoke a lambda function but I am having trouble with giving the Config rule permissions invoke the lambda. The name of the config rule is testConfigRule and the name of the lambda it is trying to invoke is testLambda. My AWs::Lambda::Permission looks like this...
"lambdaInvokePermission": {
"Type": "AWS::Lambda::Permission",
"Properties": {
"FunctionName": {
"Fn::GetAtt": [
"testLambda",
"Arn"
]
},
"SourceArn": {
"Fn::GetAtt": [
"testConfigRule",
"Arn"
]
},
"Action": "lambda:InvokeFunction",
"Principal": "config.amazonaws.com"
},
"DependsOn": [
"testConfigRule"
]
},
but I keep get the following error...
The AWS Lambda function arn:aws:lambda:us-east-2:1234567:function:testLambda cannot be invoked. Check the specified function ARN, and check the function's permissions.
If I take out the SourceArn it works but I want to limit the Permission to the one config rule.
Currently, the CFT is...
Creating the lambda
Creating the Config rule
Creating the Permission for the Config rule to invoke the lambda
It seems to fail when it tries to test the Config rule's invoking of the lambda after the Config is created and before the Permission is created so it inevitably fails with the permissions error.
Is there anyway to prevent the testing of the Config rule until after the Permission is created?
You must leave out the SourceArn as you said as it's not meant for all sources, actually only 2 at time of writing (S3 and SES).
As it says here
This property is not supported by all event sources. For more
information, see the SourceAccount parameter for the AddPermission
action in the AWS Lambda Developer Guide.
Checking the api:
This parameter is used for S3 and SES.

Mantaining OAuth keys between elastic beanstalk deployment

I have a laravel application run in AWS Elastic Beanstalk environment. I use Laravel Passport to handle the authentication.
Every time I run eb deploy the keys will be deleted, since it is not part of the version control files (included in .gitignore). Thus, I have to manually run php artisan passport:keys in the EC2 instance to generate the keys. But this will make all users need to login again because the old token is now invalid, since it's a new key pair.
What is the best practice to provide a consistent oauth-public and oauth-private key for my configuration?
I am thinking of including the key into the repository, but I believe this is not recommended.
Another way is that I generate the key once, then upload it to S3. Then have a post-deployment script to retrieve the S3.
Is there any better way?
I managed to solve this yesterday, with S3.
Create a private S3 Repository, where you store your sensitive files (oauth-private.key, etc.)
In your .ebextensions directory, you have to create a .config file where you define a Resource(see https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#linux-files - authentication section) - looking like this:
Resources:
AWSEBAutoScalingGroup:
Metadata:
AWS::CloudFormation::Authentication:
S3Auth:
type: "s3"
buckets: ["<BUCKET-NAME>"]
roleName:
"Fn::GetOptionSetting":
Namespace: "aws:autoscaling:launchconfiguration"
OptionName: "IamInstanceProfile"
DefaultValue: "aws-elasticbeanstalk-ec2-role"
Assuming A) Your S3 Bucket is called <BUCKET-NAME> and B) The IAM instance profile in your ElasticBeanstalk environment is called aws-elasticbeanstalk-ec2-role
Now you have to add the files to a location on the instance, where you can access it, you're free too choose where. In your .config file insert following:
files:
"/etc/keys/oauth-private.key":
mode: "000755"
owner: webapp
group: webapp
authentication: "S3Auth" # Notice, this is the same as specified in the Resources section
source: "https://<BUCKET-NAME>.s3-<REGION>.amazonaws.com/<PATH-TO-THE-FILE-IN-THE-BUCKET>"
Now for this to work, you still need to grant access to the IAM instance profile (aws-elasticbeanstalk-ec2-role), therefore you need to edit the ACL of your Bucket, like this:
{
"Version": "2012-10-17",
"Id": "BeanstalkS3Copy",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::<ID>:role/aws-elasticbeanstalk-ec2-role"
},
"Action": [
"s3:ListBucketVersions",
"s3:ListBucket",
"s3:GetObjectVersion",
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::<BUCKET-NAME>/*"
]
}
]
}
You can find the ARN of the IAM instance profile by going to the IAM Dashboard > Roles > aws-elasticbeanstalk-ec2-role and the copy the Role ARN
In your Laravel application you have to use Passport::loadKeysFrom('/etc/keys')

Resources