I want to create a CloudFront distribution that could invoke a lambda (#edge). I was able to do that using AWS console. I am now trying to achieve the same using Terraform. My configurations are as follows.
First, I created a role for lambda.
resource "aws_iam_role" "my_edge_lambda_iam_role" {
name = "my_edge_lambda_iam_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
Next, I created a the lambda.
resource "aws_lambda_function" "redirect_lambda" {
filename = "myscript.js.zip"
function_name = "my_js_script"
role = "arn:aws:iam::123456789:role/my_edge_lambda_iam_role"
handler = "index.handler"
runtime = "nodejs4.3-edge"
}
Finally, I (attempted to) created the CloudFront distribution using the ARN of the lambda function above. The definition is as follows.
resource "aws_cloudfront_distribution" "s3_distribution" {
origin {
domain_name = "my-bucket.s3.amazonaws.com"
origin_id = "<my S3 path>"
}
enabled = true
is_ipv6_enabled = true
default_cache_behavior {
allowed_methods = ["GET", "HEAD"]
cached_methods = ["GET", "HEAD"]
target_origin_id = "<my target origin>"
forwarded_values {
query_string = false
cookies {
forward = "none"
}
}
lambda_function_association {
event_type = "viewer-request"
lambda_arn = "<arn of the lambda function denerated above>"
}
viewer_protocol_policy = "allow-all"
min_ttl = 0
default_ttl = 86400
max_ttl = 31536000
}
price_class = "PriceClass_All"
viewer_certificate {
cloudfront_default_certificate = true
}
restrictions {
geo_restriction {
restriction_type = "none"
}
}
}
When trying to create the distribution, I get the following exception.
Error applying plan:
1 error(s) occurred:
* aws_cloudfront_distribution.s3_distribution: 1 error(s) occurred:
* aws_cloudfront_distribution.s3_distribution: InvalidLambdaFunctionAssociation: Failed to retrieve the function from Lambda. ErrorCode: AccessDeniedException Function: arn:aws:lambda:us-east-2:12345678:function:my_js_script
status code: 400, request id: 65579sd33-3f2d5-181e7-9140-79c1ff79fbdd
Could it be a problem with how I have defined the role?
As Michael mentioned in the comments, you need to add an aws_lambda_permission resource to allow an AWS service to invoke a Lambda function.
resource "aws_lambda_permission" "allow_cloudfront" {
statement_id = "AllowExecutionFromCloudFront"
action = "lambda:GetFunction"
function_name = "${aws_lambda_function.redirect_lambda.function_name}"
principal = "edgelambda.amazonaws.com"
}
The AWS docs have a little more information about how to allow this outside of Terraform, using the CLI in this case:
aws lambda add-permission \
--function-name arn \
--statement-id statement-id \
--action lambda:GetFunction \
--principal edgelambda.amazonaws.com
Related
I would like to trigger a Lambda once an RDS Replication Task has successfully completed. I have the following Terraform code, which successfully creates all the assets, but my Lambda is not being triggered.
resource "aws_dms_event_subscription" "my_event_subscription" {
enabled = true
event_categories = ["state change"]
name = "my-event-subscription"
sns_topic_arn = aws_sns_topic.my_event_subscription_topic.arn
source_ids = ["my-replication-task"]
source_type = "replication-task"
}
resource "aws_sns_topic" "my_event_subscription_topic" {
name = "my-event-subscription-topic"
}
resource "aws_sns_topic_subscription" "my_event_subscription_topic_subscription" {
topic_arn = aws_sns_topic.my_event_subscription_topic.arn
protocol = "lambda"
endpoint = aws_lambda_function.my_lambda_function.arn
}
resource "aws_sns_topic_policy" "allow_publish" {
arn = aws_sns_topic.my_event_subscription_topic.arn
policy = data.aws_iam_policy_document.allow_dms_and_events_document.json
}
resource "aws_lambda_permission" "allow_sns_invoke" {
statement_id = "AllowExecutionFromSNS"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.my_lambda_function.function_name
principal = "sns.amazonaws.com"
source_arn = aws_sns_topic.my_event_subscription_topic.arn
}
data "aws_iam_policy_document" "allow_dms_and_events_document" {
statement {
actions = ["SNS:Publish"]
principals {
identifiers = [
"dms.amazonaws.com",
"events.amazonaws.com"
]
type = "Service"
}
resources = [aws_sns_topic.my_event_subscription_topic.arn]
}
}
Am I missing something?
Is event_categories = ["state change"] correct? (This suggests state change is correct.
I'm less concerned right now if the Lambda is triggered for every state change, and not just DMS-EVENT-0079.)
Is there something I can add to get CloudWatch logs from the event subscription, to tell me what's wrong?
You can try giving it a JSON as shared on AWS Documentation.
{
"version":"0",
"id":"11a11b11-222b-333a-44d4-01234a5b67890",
"detail-type":"DMS Replication Task State Change",
"source":"aws.dms",
"account":"0123456789012",
"time":"1970-01-01T00:00:00Z",
"region":"us-east-1",
"resources":[
"arn:aws:dms:us-east-1:012345678901:task:AAAABBBB0CCCCDDDDEEEEE1FFFF2GGG3FFFFFF3"
],
"detail":{
"type":"ReplicationTask",
"category":"StateChange",
"eventType":"REPLICATION_TASK_STARTED",
"eventName":"DMS-EVENT-0069",
"resourceLink":"https://console.aws.amazon.com/dms/v2/home?region=us-east-1#taskDetails/taskName",
"detailMessage":"Replication task started, with flag = fresh start"
}
}
You can check how to give this as JSON in Terraform here
I'm trying to deploy a basic API consisting in a lambda function as main endpoint and API gateway as proxy to this function. With the following configuration i'm able to build up the infrastructure, but I cannot set to trigger the lambda function through IaC, I have to go to the AWS console in order to manually set the trigger.
resource "aws_lambda_function" "main_endpoint_function" {
function_name = "main_endpoint_function"
s3_bucket = module.s3.function_bucket_name
s3_key = "index.zip"
handler = "index.handler"
runtime = var.runtime_handler
role = aws_iam_role.lambda_role.arn
}
resource "aws_iam_role" "lambda_role" {
name = "role_lambda_test"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
//lambda has to be manually triggered from api gateway
resource "aws_api_gateway_rest_api" "apiLambda" {
name = "myAPI"
description = "terraform test"
}
resource "aws_api_gateway_resource" "proxy" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
parent_id = aws_api_gateway_rest_api.apiLambda.root_resource_id
path_part = "{proxy+}"
}
resource "aws_api_gateway_method" "proxyMethod" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "lambda" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_method.proxyMethod.resource_id
http_method = aws_api_gateway_method.proxyMethod.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.main_endpoint_function.invoke_arn
}
resource "aws_api_gateway_method" "proxy_root" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_rest_api.apiLambda.root_resource_id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "lambda_root" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_method.proxy_root.resource_id
http_method = aws_api_gateway_method.proxy_root.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.main_endpoint_function.invoke_arn
}
resource "aws_api_gateway_deployment" "apideploy" {
depends_on = [
aws_api_gateway_integration.lambda,
aws_api_gateway_integration.lambda_root,
]
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
stage_name = "test"
}
Without the manually set trigger through the console I get Internal Server Error. With triggering functions correctly. Perhaps there is something wrong with my configuration ?
Update::
After adding the permission resources for Lambda and API Gateway I also created aws_api_gateway_method_response and aws_api_gateway_integration_response. On first run errors, on second run completes. I tried adding explicit implications but they didn't solve the issue..
resource "aws_api_gateway_method_response" "response_200" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = aws_api_gateway_method.proxy_root.http_method
status_code = "200"
depends_on = [
aws_api_gateway_rest_api.apiLambda,
aws_api_gateway_resource.proxy,
aws_api_gateway_method.proxy_root
]
}
resource "aws_api_gateway_integration_response" "MyDemoIntegrationResponse" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = aws_api_gateway_method.proxy_root.http_method
status_code = aws_api_gateway_method_response.response_200.status_code
depends_on = [
aws_api_gateway_rest_api.apiLambda,
aws_api_gateway_resource.proxy,
aws_api_gateway_method.proxy_root,
aws_api_gateway_method_response.response_200
]
# Transforms the backend JSON response to XML
response_templates = {
"application/xml" = <<EOF
#set($inputRoot = $input.path('$'))
<?xml version="1.0" encoding="UTF-8"?>
<message>
$inputRoot.body
</message>
EOF
}
}
the Error is:
│ Error: Error creating API Gateway Method Response: NotFoundException: Invalid Method identifier specified
│
│ with aws_api_gateway_method_response.response_200,
│ on main.tf line 130, in resource "aws_api_gateway_method_response" "response_200":
│ 130: resource "aws_api_gateway_method_response" "response_200" {
It seems you are missing the aws_lambda_permission resource [1]. In your case, you would need to add the following (similar to example from the reference):
resource "aws_lambda_permission" "apigw_lambda" {
statement_id = "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.main_endpoint_function.function_name
principal = "apigateway.amazonaws.com"
source_arn = "arn:aws:execute-api:${var.myregion}:${var.accountId}:${aws_api_gateway_rest_api.apiLambda.id}/*/${aws_api_gateway_method.proxyMethod.http_method}${aws_api_gateway_resource.proxy.path}"
}
Since I don't know which region and account you are using (I don't need to know that), you just have to replace the var.myregion with the API Gateway region and var.accountId with the AWS account where the API Gateway is created. You can achieve that by using data sources. In theory, you might as well leave out the method reference from the source_arn and use something like:
source_arn = "arn:aws:execute-api:${var.myregion}:${var.accountId}:${aws_api_gateway_rest_api.apiLambda.id}/*/*/*"
I have removed the second reference from the code and it is now referenced in [2].
[1] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_integration#lambda-integration
[2] http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-control-access-using-iam-policies-to-invoke-api.html
This was a previous question I asked: How to get AWS Lambda ARN using Terraform?
This question was answered but turns out didn't actually solve my problem so this is a follow up.
The terraform code I have written provisions a Lambda function:
Root module:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
}
provider "aws" {
region = var.region
profile = var.aws_profile
}
module "aws_lambda_function" {
source = "./modules/lambda_function"
}
Child module:
resource "aws_lambda_function" "lambda_function" {
function_name = "lambda_function"
handler = "lambda_function.lambda_handler"
runtime = "python3.8"
filename = "./task/dist/package.zip"
role = aws_iam_role.lambda_exec.arn
}
resource "aws_iam_role" "lambda_exec" {
name = "aws_iam_lambda"
assume_role_policy = file("policy.json")
}
What I want the user to be able to do to get the Lambda ARN:
terraform output
The problem: I cannot seem to include the following code anywhere in my terraform code as it causes a "ResourceNotFOundException: Function not found..." error.
data "aws_lambda_function" "existing" {
function_name = "lambda_function"
}
output "arn" {
value = data.aws_lambda_function.existing.arn
}
Where or how do I need to include this to be able to get the ARN or is this possible?
You can't lookup the data for a resource you are creating at the same time. You need to output the ARN from the module, and then output it again from the main terraform template.
In your Lambda module:
output "arn" {
value = aws_lambda_function.lambda_function.arn
}
Then in your main file:
output "arn" {
value = module.aws_lambda_function.arn
}
I'm learning TF and trying to apply an infrastructure that creates:
a simple lambda function
an SNS topic
get that lambda to subscribe the SNS topic
a Cloud Watch Event that publishes a message to the topic at some interval
a Cloud Watch Log Group to check if the lambda gets notified by the SNS
The lambda permission to allow calls from SNS
I'm able to apply that successfully. The infrastructure seems perfectly fine (it has the same aspect when I create that myself through the visual aws console)
But the cloud watch Event doesn't get triggered (when built from TF), so no message is published to SNS and lambda doesn't get called. I don't know why
Anyone know how can I accomplish that? Bellow my .tf script:
provider "aws" {
region = "us-east-1"
}
//lambda function handler & code file
resource "aws_lambda_function" "lambda-function" {
function_name = "Function01"
handler = "com.rafael.lambda.Function01"
role = "arn:aws:iam::12345:role/LambdaRoleTest"
runtime = "java8"
s3_bucket = aws_s3_bucket.sns-test.id
s3_key = aws_s3_bucket_object.file_upload.id
source_code_hash = filebase64sha256("../target/sns-cw-lambda-poc.jar")
}
//allow sns to call lambda
resource "aws_lambda_permission" "allow-sns-to-lambda" {
function_name = aws_lambda_function.lambda-function.function_name
action = "lambda:InvokeFunction"
principal = "sns.amazonaws.com"
source_arn = aws_sns_topic.call-lambdas-topic.arn
statement_id = "AllowExecutionFromSNS"
}
//app s3 repository
resource "aws_s3_bucket" "sns-test" {
bucket = "app-bucket-12345"
region = "us-east-1"
}
//app jar file
resource "aws_s3_bucket_object" "file_upload" {
depends_on = [
aws_s3_bucket.sns-test
]
bucket = aws_s3_bucket.sns-test.id
key = "sns-cw-lambda-poc.jar"
source = "../target/sns-cw-lambda-poc.jar"
server_side_encryption = "AES256"
etag = filebase64sha256("../target/sns-cw-lambda-poc.jar")
}
//to check lambda exec logs
resource "aws_cloudwatch_log_group" "lambda-cloudwatch-logs" {
name = "/aws/lambda/${aws_lambda_function.lambda-function.function_name}"
retention_in_days = 1
}
//rule to trigger SNS
resource "aws_cloudwatch_event_rule" "publish-sns-rule" {
name = "publish-sns-rule"
schedule_expression = "rate(1 minute)"
}
//cloud watch event targets SNS
resource "aws_cloudwatch_event_target" "sns-publish" {
count = "1"
rule = aws_cloudwatch_event_rule.publish-sns-rule.name
target_id = aws_sns_topic.call-lambdas-topic.name
arn = aws_sns_topic.call-lambdas-topic.arn
}
//SNS topic to subscribe
resource "aws_sns_topic" "call-lambdas-topic" {
name = "call-lambdas-topic"
}
//lambda subscribes the topic, so it should be nofied when other resource publishes to the topic
resource "aws_sns_topic_subscription" "sns-lambda-subscritption" {
topic_arn = aws_sns_topic.call-lambdas-topic.arn
protocol = "lambda"
endpoint = aws_lambda_function.lambda-function.arn
}
I figured it out, I forgot to add the SNS policies that allow CloudWatch to publish to SNS topic. To get the above script to work, just add this:
resource "aws_sns_topic_policy" "default" {
count = 1
arn = aws_sns_topic.call-lambdas-topic.arn
policy = "${data.aws_iam_policy_document.sns_topic_policy.0.json}"
}
data "aws_iam_policy_document" "sns_topic_policy" {
count = "1"
statement {
sid = "Allow CloudwatchEvents"
actions = ["sns:Publish"]
resources = [aws_sns_topic.call-lambdas-topic.arn]
principals {
type = "Service"
identifiers = ["events.amazonaws.com"]
}
}
}
I am trying to invoke one AWS lambda from another and perform lambda chaining. The rationale behind doing this is AWS does not provide multiple trigger from same S3 bucket.
I have created one lambda, with an s3 trigger. The java code of first lambda will listen to S3 event and contains the invocation of another lambda. The second lambda will be invoked from first lambda. Both the lambda creation is done by terraform.
Lambda A has S3 trigger. This will be invoked on S3 event on a particular bucket. Lambda A will do the processing and will invoke Lambda B using invoke request. Lambda B invocation from Lambda A code in java is :
public class EventHandler implements RequestHandler<S3Event, String> {
#Override
public String handleRequest(S3Event event, Context context) throws RuntimeException {
InvokeRequest req = new InvokeRequest()
.withFunctionName("LambdaFunctionB")
.withPayload(json);
return "Lambda B invoked"
}
}
Both the lambdas are created using terraform. Scripts below:
Lambda A terraform:
module "lambda_function" {
source = "Git Path"
absolute_artifact_path = "../lambda.jar"
lambda_function_name = "LambdaFunctionA"
lambda_function_description = ""
lambda_function_runtime = "java8"
lambda_handler_name = "EventHandler"
lambda_execution_role_name = "lambda-iam-role"
lambda_memory = "512"
dead_letter_target_arn = "error-handling-arn"
}
resource "aws_lambda_permission" "allow_bucket" {
statement_id = "statementId"
action = "lambda:InvokeFunction"
function_name = "${module.lambda_function.lambda_arn}"
principal = "s3.amazonaws.com"
source_arn = "s3.bucket.arn"
}
resource "aws_s3_bucket_notification" "bucket_notification" {
bucket = "bucketName"
lambda_function {
lambda_function_arn = "${module.lambda_function.lambda_arn}"
events = ["s3:ObjectCreated:*"]
filter_prefix = "path/subPath"
}
}
Lambda B terraform:
module "lambda_function" {
source = "git path"
absolute_artifact_path = "../lambda.jar"
lambda_function_name = "LambdaFunctionB"
lambda_function_description = ""
lambda_function_runtime = "java8"
lambda_handler_name = "LambdaBEventHandler"
lambda_execution_role_name = "lambda-iam-role"
lambda_memory = "512"
dead_letter_target_arn = "error-handling-arn"
}
resource "aws_lambda_permission" "allow_lambda" {
statement_id = "AllowExecutionFromLambda"
action = "lambda:InvokeFunction"
function_name = "${module.lambda_function.lambda_arn}"
principal = "s3.amazonaws.com"
source_arn = "arn:aws:lambda:eu-west-1:xxxxxxxxxx:function:LambdaFunctionA"
}
lambda-iam-role has below policies attached
AmazonS3FullAccess
AWSLambdaBasicExecutionRole
AWSLambdaVPCAccessExecutionRole
AmazonSNSFullAccess
CloudWatchEventsFullAccess
Expectation was that Lambda A should successfully invoke Lambda B. But I am getting AccessDeniedException in Lambda A logs and it is not able to invoke Lambda B. Error is
com.amazonaws.services.lambda.model.AWSLambdaException: User: arn:aws:sts::xxxxxxxxx:assumed-role/lambda-iam-role/LambdaFunctionA is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:eu-west-1:xxxxxxxxx:function:LambdaFunctionB (Service: AWSLambda; Status Code: 403; Error Code: AccessDeniedException; Request ID: f495ede3-b3cb-47a1-b884-16996545233d)
Hope this helps you, not exactly similar but its invoking one lambda from another lambda Github
I think the lambda needs this policy as well "lambda:InvokeFunction"
I found an answer online, using the aws-sdk.
var aws = require('aws-sdk');
var lambda = new aws.Lambda({
region: 'default'
});
lambda.invoke({
FunctionName: 'name_of_your_lambda_function',
Payload: JSON.stringify(event, null, 2) // pass params
}, function(error, data) {
if (error) {
context.done('error', error);
}
if(data.Payload){
context.succeed(data.Payload)
}
});
You can find the doc here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html
Hope it helps
:)