Lambda chaining - Invoke lambda from another lambda using terraform - aws-lambda

I am trying to invoke one AWS lambda from another and perform lambda chaining. The rationale behind doing this is AWS does not provide multiple trigger from same S3 bucket.
I have created one lambda, with an s3 trigger. The java code of first lambda will listen to S3 event and contains the invocation of another lambda. The second lambda will be invoked from first lambda. Both the lambda creation is done by terraform.
Lambda A has S3 trigger. This will be invoked on S3 event on a particular bucket. Lambda A will do the processing and will invoke Lambda B using invoke request. Lambda B invocation from Lambda A code in java is :
public class EventHandler implements RequestHandler<S3Event, String> {
#Override
public String handleRequest(S3Event event, Context context) throws RuntimeException {
InvokeRequest req = new InvokeRequest()
.withFunctionName("LambdaFunctionB")
.withPayload(json);
return "Lambda B invoked"
}
}
Both the lambdas are created using terraform. Scripts below:
Lambda A terraform:
module "lambda_function" {
source = "Git Path"
absolute_artifact_path = "../lambda.jar"
lambda_function_name = "LambdaFunctionA"
lambda_function_description = ""
lambda_function_runtime = "java8"
lambda_handler_name = "EventHandler"
lambda_execution_role_name = "lambda-iam-role"
lambda_memory = "512"
dead_letter_target_arn = "error-handling-arn"
}
resource "aws_lambda_permission" "allow_bucket" {
statement_id = "statementId"
action = "lambda:InvokeFunction"
function_name = "${module.lambda_function.lambda_arn}"
principal = "s3.amazonaws.com"
source_arn = "s3.bucket.arn"
}
resource "aws_s3_bucket_notification" "bucket_notification" {
bucket = "bucketName"
lambda_function {
lambda_function_arn = "${module.lambda_function.lambda_arn}"
events = ["s3:ObjectCreated:*"]
filter_prefix = "path/subPath"
}
}
Lambda B terraform:
module "lambda_function" {
source = "git path"
absolute_artifact_path = "../lambda.jar"
lambda_function_name = "LambdaFunctionB"
lambda_function_description = ""
lambda_function_runtime = "java8"
lambda_handler_name = "LambdaBEventHandler"
lambda_execution_role_name = "lambda-iam-role"
lambda_memory = "512"
dead_letter_target_arn = "error-handling-arn"
}
resource "aws_lambda_permission" "allow_lambda" {
statement_id = "AllowExecutionFromLambda"
action = "lambda:InvokeFunction"
function_name = "${module.lambda_function.lambda_arn}"
principal = "s3.amazonaws.com"
source_arn = "arn:aws:lambda:eu-west-1:xxxxxxxxxx:function:LambdaFunctionA"
}
lambda-iam-role has below policies attached
AmazonS3FullAccess
AWSLambdaBasicExecutionRole
AWSLambdaVPCAccessExecutionRole
AmazonSNSFullAccess
CloudWatchEventsFullAccess
Expectation was that Lambda A should successfully invoke Lambda B. But I am getting AccessDeniedException in Lambda A logs and it is not able to invoke Lambda B. Error is
com.amazonaws.services.lambda.model.AWSLambdaException: User: arn:aws:sts::xxxxxxxxx:assumed-role/lambda-iam-role/LambdaFunctionA is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:eu-west-1:xxxxxxxxx:function:LambdaFunctionB (Service: AWSLambda; Status Code: 403; Error Code: AccessDeniedException; Request ID: f495ede3-b3cb-47a1-b884-16996545233d)

Hope this helps you, not exactly similar but its invoking one lambda from another lambda Github
I think the lambda needs this policy as well "lambda:InvokeFunction"

I found an answer online, using the aws-sdk.
var aws = require('aws-sdk');
var lambda = new aws.Lambda({
region: 'default'
});
lambda.invoke({
FunctionName: 'name_of_your_lambda_function',
Payload: JSON.stringify(event, null, 2) // pass params
}, function(error, data) {
if (error) {
context.done('error', error);
}
if(data.Payload){
context.succeed(data.Payload)
}
});
You can find the doc here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html
Hope it helps
:)

Related

Triggering a Lambda once a DMS Replication Task has completed in Terraform

I would like to trigger a Lambda once an RDS Replication Task has successfully completed. I have the following Terraform code, which successfully creates all the assets, but my Lambda is not being triggered.
resource "aws_dms_event_subscription" "my_event_subscription" {
enabled = true
event_categories = ["state change"]
name = "my-event-subscription"
sns_topic_arn = aws_sns_topic.my_event_subscription_topic.arn
source_ids = ["my-replication-task"]
source_type = "replication-task"
}
resource "aws_sns_topic" "my_event_subscription_topic" {
name = "my-event-subscription-topic"
}
resource "aws_sns_topic_subscription" "my_event_subscription_topic_subscription" {
topic_arn = aws_sns_topic.my_event_subscription_topic.arn
protocol = "lambda"
endpoint = aws_lambda_function.my_lambda_function.arn
}
resource "aws_sns_topic_policy" "allow_publish" {
arn = aws_sns_topic.my_event_subscription_topic.arn
policy = data.aws_iam_policy_document.allow_dms_and_events_document.json
}
resource "aws_lambda_permission" "allow_sns_invoke" {
statement_id = "AllowExecutionFromSNS"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.my_lambda_function.function_name
principal = "sns.amazonaws.com"
source_arn = aws_sns_topic.my_event_subscription_topic.arn
}
data "aws_iam_policy_document" "allow_dms_and_events_document" {
statement {
actions = ["SNS:Publish"]
principals {
identifiers = [
"dms.amazonaws.com",
"events.amazonaws.com"
]
type = "Service"
}
resources = [aws_sns_topic.my_event_subscription_topic.arn]
}
}
Am I missing something?
Is event_categories = ["state change"] correct? (This suggests state change is correct.
I'm less concerned right now if the Lambda is triggered for every state change, and not just DMS-EVENT-0079.)
Is there something I can add to get CloudWatch logs from the event subscription, to tell me what's wrong?
You can try giving it a JSON as shared on AWS Documentation.
{
"version":"0",
"id":"11a11b11-222b-333a-44d4-01234a5b67890",
"detail-type":"DMS Replication Task State Change",
"source":"aws.dms",
"account":"0123456789012",
"time":"1970-01-01T00:00:00Z",
"region":"us-east-1",
"resources":[
"arn:aws:dms:us-east-1:012345678901:task:AAAABBBB0CCCCDDDDEEEEE1FFFF2GGG3FFFFFF3"
],
"detail":{
"type":"ReplicationTask",
"category":"StateChange",
"eventType":"REPLICATION_TASK_STARTED",
"eventName":"DMS-EVENT-0069",
"resourceLink":"https://console.aws.amazon.com/dms/v2/home?region=us-east-1#taskDetails/taskName",
"detailMessage":"Replication task started, with flag = fresh start"
}
}
You can check how to give this as JSON in Terraform here

How to make integrations in AWS Api Gateway v2 (http) with lambda alias and stage variables in terraform

I have several lambdas with aliases (develop, production, staging..) and when I try to make an integration with his route like this
resource "aws_apigatewayv2_route" "http_routes" {
for_each = local.lambda_integrations
api_id = aws_apigatewayv2_api.api_http.id
route_key = each.key
target = "integrations/${aws_apigatewayv2_integration.http_integrations[each.key].id}"
}
resource "aws_apigatewayv2_integration" "http_integrations" {
for_each = local.lambda_integrations
api_id = aws_apigatewayv2_api.api_http.id
integration_type = "AWS_PROXY"
integration_method = "POST"
integration_uri = "arn:aws:apigateway:${var.auth.region}:lambda:path/2015-03-31/functions/${module.lambdas_functions[index(module.lambdas_functions.*.function_name,each.value.lambda)].arn}:$${stageVariables.alias}/invocations"
}
and
# Main Permission
resource "aws_lambda_permission" "permission_lambda" {
for_each = local.lambda_integrations
statement_id = can(each.value.statement) ? each.value.statement : "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = module.lambdas_functions[
index(module.lambdas_functions.*.function_name,each.value.lambda)
].function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.api_http.execution_arn}/*/*"
}
# Stage develop permission
resource "aws_lambda_permission" "permission_lambda_alias_develop" {
for_each = local.lambda_integrations
statement_id = can(each.value.statement) ? each.value.statement : "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = module.lambdas_functions[
index(module.lambdas_functions.*.function_name,each.value.lambda)
].function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.api_http.execution_arn}/*/*"
qualifier = "develop"
}
this works with a [number] version in CloudWatch, but in the lambda trigger I get this:
and although the code works, it does not seem to me to be correct.
what could be the best and correct approach to this situation?.
Regards
What worked for me is this:
Destroyed the API Gateway components (used target)
terraform destroy --var-file VAR_FILES --target API_AND_LAMBDA_MODULES
Renamed the stage to "$default"
Made the value of the perm_source_arn parameter as
"arn:aws:execute-api:${REGION}:${ACCOUNT_ID}:${API_ID}/*/*${ROUTE_KEY_PATH}"
After these, just re-apply.

How can I add a AWS cloudwatch event to a aws_lambda_function which is based on a container image with terraform?

I want to achieve that my lambda function which is based on a docker image in ecr is triggered by a scheduled cloudwatch event.
The problem is that I can not attach the function_name myFunction from the module "lambda_function_container_image" to the aws_lambda_permission.
It works when I have a normal lambda function like, but not with the lambda function from a image URI:
resource "aws_lambda_function" "myFunction" {
function_name = "myFunction"
role = aws_iam_role.lambda_execution_role.arn
handler = "exports.handler"
runtime = "python3.8"
}
I have the following code:
AWS CloudWatch event:
resource "aws_cloudwatch_event_rule" "every_five_minutes" {
name = "every-five-minutes"
description = "Fires every five minutes"
schedule_expression = "rate(5 minutes)"
}
Lambda function based on container image:
module "lambda_function_container_image" {
source = "terraform-aws-modules/lambda/aws"
function_name = "myFunction"
description = "awesome function"
create_package = false
image_uri = "${data.aws_caller_identity.current.account_id}.dkr.ecr${var.aws_region}.amazonaws.com/container_name"
package_type = "Image"
}
Lambda permission:
resource "aws_lambda_permission" "allow_cloudwatch_to_call_myFunction" {
statement_id = "AllowExecutionFromCloudWatch"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.myFunction.function_name
principal = "events.amazonaws.com"
source_arn = aws_cloudwatch_event_rule.every_five_minutes.arn
}
I get the following error with the current aws_lambda_permission:
Error message:
Error: Reference to undeclared resource
-> points to function_name in aws_lambda_permission
You need to reference function_name through the module you are using. According to the documentation of terraform-aws-modules/lambda/aws the module has an output called lambda_function_name.
That means, that the following should work for you:
resource "aws_lambda_permission" "allow_cloudwatch_to_call_myFunction" {
[...]
function_name = module.lambda_function_container_image.lambda_function_name
[...]
}

How to structure terraform code to get Lambda ARN after creation?

This was a previous question I asked: How to get AWS Lambda ARN using Terraform?
This question was answered but turns out didn't actually solve my problem so this is a follow up.
The terraform code I have written provisions a Lambda function:
Root module:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
}
provider "aws" {
region = var.region
profile = var.aws_profile
}
module "aws_lambda_function" {
source = "./modules/lambda_function"
}
Child module:
resource "aws_lambda_function" "lambda_function" {
function_name = "lambda_function"
handler = "lambda_function.lambda_handler"
runtime = "python3.8"
filename = "./task/dist/package.zip"
role = aws_iam_role.lambda_exec.arn
}
resource "aws_iam_role" "lambda_exec" {
name = "aws_iam_lambda"
assume_role_policy = file("policy.json")
}
What I want the user to be able to do to get the Lambda ARN:
terraform output
The problem: I cannot seem to include the following code anywhere in my terraform code as it causes a "ResourceNotFOundException: Function not found..." error.
data "aws_lambda_function" "existing" {
function_name = "lambda_function"
}
output "arn" {
value = data.aws_lambda_function.existing.arn
}
Where or how do I need to include this to be able to get the ARN or is this possible?
You can't lookup the data for a resource you are creating at the same time. You need to output the ARN from the module, and then output it again from the main terraform template.
In your Lambda module:
output "arn" {
value = aws_lambda_function.lambda_function.arn
}
Then in your main file:
output "arn" {
value = module.aws_lambda_function.arn
}

Terraform: CloudWatch Event that notifies SNS

I'm learning TF and trying to apply an infrastructure that creates:
a simple lambda function
an SNS topic
get that lambda to subscribe the SNS topic
a Cloud Watch Event that publishes a message to the topic at some interval
a Cloud Watch Log Group to check if the lambda gets notified by the SNS
The lambda permission to allow calls from SNS
I'm able to apply that successfully. The infrastructure seems perfectly fine (it has the same aspect when I create that myself through the visual aws console)
But the cloud watch Event doesn't get triggered (when built from TF), so no message is published to SNS and lambda doesn't get called. I don't know why
Anyone know how can I accomplish that? Bellow my .tf script:
provider "aws" {
region = "us-east-1"
}
//lambda function handler & code file
resource "aws_lambda_function" "lambda-function" {
function_name = "Function01"
handler = "com.rafael.lambda.Function01"
role = "arn:aws:iam::12345:role/LambdaRoleTest"
runtime = "java8"
s3_bucket = aws_s3_bucket.sns-test.id
s3_key = aws_s3_bucket_object.file_upload.id
source_code_hash = filebase64sha256("../target/sns-cw-lambda-poc.jar")
}
//allow sns to call lambda
resource "aws_lambda_permission" "allow-sns-to-lambda" {
function_name = aws_lambda_function.lambda-function.function_name
action = "lambda:InvokeFunction"
principal = "sns.amazonaws.com"
source_arn = aws_sns_topic.call-lambdas-topic.arn
statement_id = "AllowExecutionFromSNS"
}
//app s3 repository
resource "aws_s3_bucket" "sns-test" {
bucket = "app-bucket-12345"
region = "us-east-1"
}
//app jar file
resource "aws_s3_bucket_object" "file_upload" {
depends_on = [
aws_s3_bucket.sns-test
]
bucket = aws_s3_bucket.sns-test.id
key = "sns-cw-lambda-poc.jar"
source = "../target/sns-cw-lambda-poc.jar"
server_side_encryption = "AES256"
etag = filebase64sha256("../target/sns-cw-lambda-poc.jar")
}
//to check lambda exec logs
resource "aws_cloudwatch_log_group" "lambda-cloudwatch-logs" {
name = "/aws/lambda/${aws_lambda_function.lambda-function.function_name}"
retention_in_days = 1
}
//rule to trigger SNS
resource "aws_cloudwatch_event_rule" "publish-sns-rule" {
name = "publish-sns-rule"
schedule_expression = "rate(1 minute)"
}
//cloud watch event targets SNS
resource "aws_cloudwatch_event_target" "sns-publish" {
count = "1"
rule = aws_cloudwatch_event_rule.publish-sns-rule.name
target_id = aws_sns_topic.call-lambdas-topic.name
arn = aws_sns_topic.call-lambdas-topic.arn
}
//SNS topic to subscribe
resource "aws_sns_topic" "call-lambdas-topic" {
name = "call-lambdas-topic"
}
//lambda subscribes the topic, so it should be nofied when other resource publishes to the topic
resource "aws_sns_topic_subscription" "sns-lambda-subscritption" {
topic_arn = aws_sns_topic.call-lambdas-topic.arn
protocol = "lambda"
endpoint = aws_lambda_function.lambda-function.arn
}
I figured it out, I forgot to add the SNS policies that allow CloudWatch to publish to SNS topic. To get the above script to work, just add this:
resource "aws_sns_topic_policy" "default" {
count = 1
arn = aws_sns_topic.call-lambdas-topic.arn
policy = "${data.aws_iam_policy_document.sns_topic_policy.0.json}"
}
data "aws_iam_policy_document" "sns_topic_policy" {
count = "1"
statement {
sid = "Allow CloudwatchEvents"
actions = ["sns:Publish"]
resources = [aws_sns_topic.call-lambdas-topic.arn]
principals {
type = "Service"
identifiers = ["events.amazonaws.com"]
}
}
}

Resources