Trigger Lambda from API Gateway in Terraform - aws-lambda

I'm trying to deploy a basic API consisting in a lambda function as main endpoint and API gateway as proxy to this function. With the following configuration i'm able to build up the infrastructure, but I cannot set to trigger the lambda function through IaC, I have to go to the AWS console in order to manually set the trigger.
resource "aws_lambda_function" "main_endpoint_function" {
function_name = "main_endpoint_function"
s3_bucket = module.s3.function_bucket_name
s3_key = "index.zip"
handler = "index.handler"
runtime = var.runtime_handler
role = aws_iam_role.lambda_role.arn
}
resource "aws_iam_role" "lambda_role" {
name = "role_lambda_test"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
//lambda has to be manually triggered from api gateway
resource "aws_api_gateway_rest_api" "apiLambda" {
name = "myAPI"
description = "terraform test"
}
resource "aws_api_gateway_resource" "proxy" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
parent_id = aws_api_gateway_rest_api.apiLambda.root_resource_id
path_part = "{proxy+}"
}
resource "aws_api_gateway_method" "proxyMethod" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "lambda" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_method.proxyMethod.resource_id
http_method = aws_api_gateway_method.proxyMethod.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.main_endpoint_function.invoke_arn
}
resource "aws_api_gateway_method" "proxy_root" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_rest_api.apiLambda.root_resource_id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "lambda_root" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_method.proxy_root.resource_id
http_method = aws_api_gateway_method.proxy_root.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.main_endpoint_function.invoke_arn
}
resource "aws_api_gateway_deployment" "apideploy" {
depends_on = [
aws_api_gateway_integration.lambda,
aws_api_gateway_integration.lambda_root,
]
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
stage_name = "test"
}
Without the manually set trigger through the console I get Internal Server Error. With triggering functions correctly. Perhaps there is something wrong with my configuration ?
Update::
After adding the permission resources for Lambda and API Gateway I also created aws_api_gateway_method_response and aws_api_gateway_integration_response. On first run errors, on second run completes. I tried adding explicit implications but they didn't solve the issue..
resource "aws_api_gateway_method_response" "response_200" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = aws_api_gateway_method.proxy_root.http_method
status_code = "200"
depends_on = [
aws_api_gateway_rest_api.apiLambda,
aws_api_gateway_resource.proxy,
aws_api_gateway_method.proxy_root
]
}
resource "aws_api_gateway_integration_response" "MyDemoIntegrationResponse" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = aws_api_gateway_method.proxy_root.http_method
status_code = aws_api_gateway_method_response.response_200.status_code
depends_on = [
aws_api_gateway_rest_api.apiLambda,
aws_api_gateway_resource.proxy,
aws_api_gateway_method.proxy_root,
aws_api_gateway_method_response.response_200
]
# Transforms the backend JSON response to XML
response_templates = {
"application/xml" = <<EOF
#set($inputRoot = $input.path('$'))
<?xml version="1.0" encoding="UTF-8"?>
<message>
$inputRoot.body
</message>
EOF
}
}
the Error is:
│ Error: Error creating API Gateway Method Response: NotFoundException: Invalid Method identifier specified
│
│ with aws_api_gateway_method_response.response_200,
│ on main.tf line 130, in resource "aws_api_gateway_method_response" "response_200":
│ 130: resource "aws_api_gateway_method_response" "response_200" {

It seems you are missing the aws_lambda_permission resource [1]. In your case, you would need to add the following (similar to example from the reference):
resource "aws_lambda_permission" "apigw_lambda" {
statement_id = "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.main_endpoint_function.function_name
principal = "apigateway.amazonaws.com"
source_arn = "arn:aws:execute-api:${var.myregion}:${var.accountId}:${aws_api_gateway_rest_api.apiLambda.id}/*/${aws_api_gateway_method.proxyMethod.http_method}${aws_api_gateway_resource.proxy.path}"
}
Since I don't know which region and account you are using (I don't need to know that), you just have to replace the var.myregion with the API Gateway region and var.accountId with the AWS account where the API Gateway is created. You can achieve that by using data sources. In theory, you might as well leave out the method reference from the source_arn and use something like:
source_arn = "arn:aws:execute-api:${var.myregion}:${var.accountId}:${aws_api_gateway_rest_api.apiLambda.id}/*/*/*"
I have removed the second reference from the code and it is now referenced in [2].
[1] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_integration#lambda-integration
[2] http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-control-access-using-iam-policies-to-invoke-api.html

Related

Triggering a Lambda once a DMS Replication Task has completed in Terraform

I would like to trigger a Lambda once an RDS Replication Task has successfully completed. I have the following Terraform code, which successfully creates all the assets, but my Lambda is not being triggered.
resource "aws_dms_event_subscription" "my_event_subscription" {
enabled = true
event_categories = ["state change"]
name = "my-event-subscription"
sns_topic_arn = aws_sns_topic.my_event_subscription_topic.arn
source_ids = ["my-replication-task"]
source_type = "replication-task"
}
resource "aws_sns_topic" "my_event_subscription_topic" {
name = "my-event-subscription-topic"
}
resource "aws_sns_topic_subscription" "my_event_subscription_topic_subscription" {
topic_arn = aws_sns_topic.my_event_subscription_topic.arn
protocol = "lambda"
endpoint = aws_lambda_function.my_lambda_function.arn
}
resource "aws_sns_topic_policy" "allow_publish" {
arn = aws_sns_topic.my_event_subscription_topic.arn
policy = data.aws_iam_policy_document.allow_dms_and_events_document.json
}
resource "aws_lambda_permission" "allow_sns_invoke" {
statement_id = "AllowExecutionFromSNS"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.my_lambda_function.function_name
principal = "sns.amazonaws.com"
source_arn = aws_sns_topic.my_event_subscription_topic.arn
}
data "aws_iam_policy_document" "allow_dms_and_events_document" {
statement {
actions = ["SNS:Publish"]
principals {
identifiers = [
"dms.amazonaws.com",
"events.amazonaws.com"
]
type = "Service"
}
resources = [aws_sns_topic.my_event_subscription_topic.arn]
}
}
Am I missing something?
Is event_categories = ["state change"] correct? (This suggests state change is correct.
I'm less concerned right now if the Lambda is triggered for every state change, and not just DMS-EVENT-0079.)
Is there something I can add to get CloudWatch logs from the event subscription, to tell me what's wrong?
You can try giving it a JSON as shared on AWS Documentation.
{
"version":"0",
"id":"11a11b11-222b-333a-44d4-01234a5b67890",
"detail-type":"DMS Replication Task State Change",
"source":"aws.dms",
"account":"0123456789012",
"time":"1970-01-01T00:00:00Z",
"region":"us-east-1",
"resources":[
"arn:aws:dms:us-east-1:012345678901:task:AAAABBBB0CCCCDDDDEEEEE1FFFF2GGG3FFFFFF3"
],
"detail":{
"type":"ReplicationTask",
"category":"StateChange",
"eventType":"REPLICATION_TASK_STARTED",
"eventName":"DMS-EVENT-0069",
"resourceLink":"https://console.aws.amazon.com/dms/v2/home?region=us-east-1#taskDetails/taskName",
"detailMessage":"Replication task started, with flag = fresh start"
}
}
You can check how to give this as JSON in Terraform here

How to make integrations in AWS Api Gateway v2 (http) with lambda alias and stage variables in terraform

I have several lambdas with aliases (develop, production, staging..) and when I try to make an integration with his route like this
resource "aws_apigatewayv2_route" "http_routes" {
for_each = local.lambda_integrations
api_id = aws_apigatewayv2_api.api_http.id
route_key = each.key
target = "integrations/${aws_apigatewayv2_integration.http_integrations[each.key].id}"
}
resource "aws_apigatewayv2_integration" "http_integrations" {
for_each = local.lambda_integrations
api_id = aws_apigatewayv2_api.api_http.id
integration_type = "AWS_PROXY"
integration_method = "POST"
integration_uri = "arn:aws:apigateway:${var.auth.region}:lambda:path/2015-03-31/functions/${module.lambdas_functions[index(module.lambdas_functions.*.function_name,each.value.lambda)].arn}:$${stageVariables.alias}/invocations"
}
and
# Main Permission
resource "aws_lambda_permission" "permission_lambda" {
for_each = local.lambda_integrations
statement_id = can(each.value.statement) ? each.value.statement : "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = module.lambdas_functions[
index(module.lambdas_functions.*.function_name,each.value.lambda)
].function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.api_http.execution_arn}/*/*"
}
# Stage develop permission
resource "aws_lambda_permission" "permission_lambda_alias_develop" {
for_each = local.lambda_integrations
statement_id = can(each.value.statement) ? each.value.statement : "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = module.lambdas_functions[
index(module.lambdas_functions.*.function_name,each.value.lambda)
].function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.api_http.execution_arn}/*/*"
qualifier = "develop"
}
this works with a [number] version in CloudWatch, but in the lambda trigger I get this:
and although the code works, it does not seem to me to be correct.
what could be the best and correct approach to this situation?.
Regards
What worked for me is this:
Destroyed the API Gateway components (used target)
terraform destroy --var-file VAR_FILES --target API_AND_LAMBDA_MODULES
Renamed the stage to "$default"
Made the value of the perm_source_arn parameter as
"arn:aws:execute-api:${REGION}:${ACCOUNT_ID}:${API_ID}/*/*${ROUTE_KEY_PATH}"
After these, just re-apply.

Lambda Web API returning 403 for verbs other than POST

I have created a Serverless Lambda Web API using a built-in template. The web api is deployed to AWS using CloudFormation and everything is working fine (i.e. All GET, POST etc).
I have then checked the resources created by CloudFormation and created all of the resources using Terraform. I have double checked and REST API in the API Gateway, Lambda, Lambda permissions and triggers are exactly the same when created from both serverless template and terraform scripts.
The issue is the when using the resources created from Terraform, I am not able to call the Web API using HTTP verb other then POST. Below is my terraform script for the API Gateway.
resource "aws_api_gateway_rest_api" "SimulationServer_api_gateway_rest" {
name = "SimulationServer"
description = "Terraform Serverless Application Example"
}
resource "aws_api_gateway_resource" "api_gw_proxy_resource" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
parent_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.root_resource_id
path_part = "{proxy+}"
}
resource "aws_api_gateway_method" "api_gw_proxy_method" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
resource_id = aws_api_gateway_resource.api_gw_proxy_resource.id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "api_gw_proxy_integration" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
resource_id = aws_api_gateway_method.api_gw_proxy_method.resource_id
http_method = aws_api_gateway_method.api_gw_proxy_method.http_method
integration_http_method = "ANY"
type = "AWS_PROXY"
uri = aws_lambda_function.SimulationServer.invoke_arn
}
resource "aws_api_gateway_method" "api_gw_proxy_root_method" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
resource_id =
aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.root_resource_id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "api_gw_proxy_root_integration" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
resource_id = aws_api_gateway_method.api_gw_proxy_root_method.resource_id
http_method = aws_api_gateway_method.api_gw_proxy_root_method.http_method
integration_http_method = "ANY"
type = "AWS_PROXY"
uri = aws_lambda_function.SimulationServer.invoke_arn
}
resource "aws_api_gateway_deployment" "api_gw_deployment" {
depends_on = [
aws_api_gateway_integration.api_gw_proxy_integration,
aws_api_gateway_integration.api_gw_proxy_root_integration,
]
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
stage_name = var.env
}
output "base_url" {
value = aws_api_gateway_deployment.api_gw_deployment.invoke_url
}
It turned out to be very strange but simple solution.
Instead of using,
integration_http_method = "ANY"
When I used,
integration_http_method = "POST"
Then it started working for all HTTP Verbs (GET, POST, PUT etc)

Terraform API Gateway for use with Lambda Proxy Integration

I've spent the day fighting with API Gateway and AWS Serverless Express to no avail. My goal is to deploy an API Gateway, via Terraform (v0.12), that proxies all requests to an AWS Serverless Express based lambda. The connection between API Gateway and Lambda seems to exist, but tenuous, as any invocation (from API Gateway console or Postman) respond with 502 Bad Gateway, apparently due to timeout (so states the lambda CloudWatch logs). It does not appear that the lambda code actually runs, only that it spins up ineffectually.
The API Gateway and Lambda should support path parameters and query strings:
GET /some/path/:id get by id
GET /some/path?query=param get
collection
POST /some/path create resource
PATCH /some/path/:id update resource
DELETE /some/path/:id remove resource
After several false starts, I've tried to make the API Gateway Terraform module as flexible as possible:
resource "aws_api_gateway_rest_api" "rest_api" {
name = "${var.application_name} API"
description = var.description
}
resource "aws_api_gateway_resource" "proxy" {
rest_api_id = aws_api_gateway_rest_api.rest_api.id
parent_id = aws_api_gateway_rest_api.rest_api.root_resource_id # aws_api_gateway_resource.version.id
path_part = "{proxy+}"
}
resource "aws_api_gateway_method" "method" {
rest_api_id = aws_api_gateway_rest_api.rest_api.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = "ANY"
authorization = "NONE"
request_parameters = {
"method.request.path.proxy" = true
}
}
resource "aws_api_gateway_integration" "integration" {
rest_api_id = aws_api_gateway_rest_api.rest_api.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = aws_api_gateway_method.method.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = "arn:aws:apigateway:${local.region}:lambda:path/2015-03-31/functions/${var.lambda_arn}/invocations"
}
resource "aws_api_gateway_deployment" "apig_deployment" {
depends_on = [
"aws_api_gateway_resource.proxy",
"aws_api_gateway_method.method",
"aws_api_gateway_integration.integration"
]
rest_api_id = aws_api_gateway_rest_api.rest_api.id
stage_name = var.api_stage_name
lifecycle {
create_before_destroy = true
}
}
resource "aws_lambda_permission" "apig_to_lambda" {
statement_id = "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = var.function_name
principal = "apigateway.amazonaws.com"
source_arn = "arn:aws:execute-api:${local.region}:${local.account_id}:${aws_api_gateway_rest_api.rest_api.id}/*/*/*" # TODO: lock this down
}
You are getting 502 error that indicates that the response received by api gateway is not proper.
Having said that your application seems to return json response. Can you try below/confirm the settings?
Add the correct binary mime types in your lambda configuration. See here for more details.
Allow the same mime types or wildcard on api gateway side as below.
resource "aws_api_gateway_rest_api" "rest_api" {
name = "${var.application_name} API"
description = var.description
// Wildcard mimes, accept any
binary_media_types = ["*/*"]
}

InvalidLambdaFunctionAssociation when creating CloudFront distribution via Terraform

I want to create a CloudFront distribution that could invoke a lambda (#edge). I was able to do that using AWS console. I am now trying to achieve the same using Terraform. My configurations are as follows.
First, I created a role for lambda.
resource "aws_iam_role" "my_edge_lambda_iam_role" {
name = "my_edge_lambda_iam_role"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
Next, I created a the lambda.
resource "aws_lambda_function" "redirect_lambda" {
filename = "myscript.js.zip"
function_name = "my_js_script"
role = "arn:aws:iam::123456789:role/my_edge_lambda_iam_role"
handler = "index.handler"
runtime = "nodejs4.3-edge"
}
Finally, I (attempted to) created the CloudFront distribution using the ARN of the lambda function above. The definition is as follows.
resource "aws_cloudfront_distribution" "s3_distribution" {
origin {
domain_name = "my-bucket.s3.amazonaws.com"
origin_id = "<my S3 path>"
}
enabled = true
is_ipv6_enabled = true
default_cache_behavior {
allowed_methods = ["GET", "HEAD"]
cached_methods = ["GET", "HEAD"]
target_origin_id = "<my target origin>"
forwarded_values {
query_string = false
cookies {
forward = "none"
}
}
lambda_function_association {
event_type = "viewer-request"
lambda_arn = "<arn of the lambda function denerated above>"
}
viewer_protocol_policy = "allow-all"
min_ttl = 0
default_ttl = 86400
max_ttl = 31536000
}
price_class = "PriceClass_All"
viewer_certificate {
cloudfront_default_certificate = true
}
restrictions {
geo_restriction {
restriction_type = "none"
}
}
}
When trying to create the distribution, I get the following exception.
Error applying plan:
1 error(s) occurred:
* aws_cloudfront_distribution.s3_distribution: 1 error(s) occurred:
* aws_cloudfront_distribution.s3_distribution: InvalidLambdaFunctionAssociation: Failed to retrieve the function from Lambda. ErrorCode: AccessDeniedException Function: arn:aws:lambda:us-east-2:12345678:function:my_js_script
status code: 400, request id: 65579sd33-3f2d5-181e7-9140-79c1ff79fbdd
Could it be a problem with how I have defined the role?
As Michael mentioned in the comments, you need to add an aws_lambda_permission resource to allow an AWS service to invoke a Lambda function.
resource "aws_lambda_permission" "allow_cloudfront" {
statement_id = "AllowExecutionFromCloudFront"
action = "lambda:GetFunction"
function_name = "${aws_lambda_function.redirect_lambda.function_name}"
principal = "edgelambda.amazonaws.com"
}
The AWS docs have a little more information about how to allow this outside of Terraform, using the CLI in this case:
aws lambda add-permission \
--function-name arn \
--statement-id statement-id \
--action lambda:GetFunction \
--principal edgelambda.amazonaws.com

Resources