Terraform API Gateway for use with Lambda Proxy Integration - aws-lambda

I've spent the day fighting with API Gateway and AWS Serverless Express to no avail. My goal is to deploy an API Gateway, via Terraform (v0.12), that proxies all requests to an AWS Serverless Express based lambda. The connection between API Gateway and Lambda seems to exist, but tenuous, as any invocation (from API Gateway console or Postman) respond with 502 Bad Gateway, apparently due to timeout (so states the lambda CloudWatch logs). It does not appear that the lambda code actually runs, only that it spins up ineffectually.
The API Gateway and Lambda should support path parameters and query strings:
GET /some/path/:id get by id
GET /some/path?query=param get
collection
POST /some/path create resource
PATCH /some/path/:id update resource
DELETE /some/path/:id remove resource
After several false starts, I've tried to make the API Gateway Terraform module as flexible as possible:
resource "aws_api_gateway_rest_api" "rest_api" {
name = "${var.application_name} API"
description = var.description
}
resource "aws_api_gateway_resource" "proxy" {
rest_api_id = aws_api_gateway_rest_api.rest_api.id
parent_id = aws_api_gateway_rest_api.rest_api.root_resource_id # aws_api_gateway_resource.version.id
path_part = "{proxy+}"
}
resource "aws_api_gateway_method" "method" {
rest_api_id = aws_api_gateway_rest_api.rest_api.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = "ANY"
authorization = "NONE"
request_parameters = {
"method.request.path.proxy" = true
}
}
resource "aws_api_gateway_integration" "integration" {
rest_api_id = aws_api_gateway_rest_api.rest_api.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = aws_api_gateway_method.method.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = "arn:aws:apigateway:${local.region}:lambda:path/2015-03-31/functions/${var.lambda_arn}/invocations"
}
resource "aws_api_gateway_deployment" "apig_deployment" {
depends_on = [
"aws_api_gateway_resource.proxy",
"aws_api_gateway_method.method",
"aws_api_gateway_integration.integration"
]
rest_api_id = aws_api_gateway_rest_api.rest_api.id
stage_name = var.api_stage_name
lifecycle {
create_before_destroy = true
}
}
resource "aws_lambda_permission" "apig_to_lambda" {
statement_id = "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = var.function_name
principal = "apigateway.amazonaws.com"
source_arn = "arn:aws:execute-api:${local.region}:${local.account_id}:${aws_api_gateway_rest_api.rest_api.id}/*/*/*" # TODO: lock this down
}

You are getting 502 error that indicates that the response received by api gateway is not proper.
Having said that your application seems to return json response. Can you try below/confirm the settings?
Add the correct binary mime types in your lambda configuration. See here for more details.
Allow the same mime types or wildcard on api gateway side as below.
resource "aws_api_gateway_rest_api" "rest_api" {
name = "${var.application_name} API"
description = var.description
// Wildcard mimes, accept any
binary_media_types = ["*/*"]
}

Related

How to make integrations in AWS Api Gateway v2 (http) with lambda alias and stage variables in terraform

I have several lambdas with aliases (develop, production, staging..) and when I try to make an integration with his route like this
resource "aws_apigatewayv2_route" "http_routes" {
for_each = local.lambda_integrations
api_id = aws_apigatewayv2_api.api_http.id
route_key = each.key
target = "integrations/${aws_apigatewayv2_integration.http_integrations[each.key].id}"
}
resource "aws_apigatewayv2_integration" "http_integrations" {
for_each = local.lambda_integrations
api_id = aws_apigatewayv2_api.api_http.id
integration_type = "AWS_PROXY"
integration_method = "POST"
integration_uri = "arn:aws:apigateway:${var.auth.region}:lambda:path/2015-03-31/functions/${module.lambdas_functions[index(module.lambdas_functions.*.function_name,each.value.lambda)].arn}:$${stageVariables.alias}/invocations"
}
and
# Main Permission
resource "aws_lambda_permission" "permission_lambda" {
for_each = local.lambda_integrations
statement_id = can(each.value.statement) ? each.value.statement : "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = module.lambdas_functions[
index(module.lambdas_functions.*.function_name,each.value.lambda)
].function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.api_http.execution_arn}/*/*"
}
# Stage develop permission
resource "aws_lambda_permission" "permission_lambda_alias_develop" {
for_each = local.lambda_integrations
statement_id = can(each.value.statement) ? each.value.statement : "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = module.lambdas_functions[
index(module.lambdas_functions.*.function_name,each.value.lambda)
].function_name
principal = "apigateway.amazonaws.com"
source_arn = "${aws_apigatewayv2_api.api_http.execution_arn}/*/*"
qualifier = "develop"
}
this works with a [number] version in CloudWatch, but in the lambda trigger I get this:
and although the code works, it does not seem to me to be correct.
what could be the best and correct approach to this situation?.
Regards
What worked for me is this:
Destroyed the API Gateway components (used target)
terraform destroy --var-file VAR_FILES --target API_AND_LAMBDA_MODULES
Renamed the stage to "$default"
Made the value of the perm_source_arn parameter as
"arn:aws:execute-api:${REGION}:${ACCOUNT_ID}:${API_ID}/*/*${ROUTE_KEY_PATH}"
After these, just re-apply.

Trigger Lambda from API Gateway in Terraform

I'm trying to deploy a basic API consisting in a lambda function as main endpoint and API gateway as proxy to this function. With the following configuration i'm able to build up the infrastructure, but I cannot set to trigger the lambda function through IaC, I have to go to the AWS console in order to manually set the trigger.
resource "aws_lambda_function" "main_endpoint_function" {
function_name = "main_endpoint_function"
s3_bucket = module.s3.function_bucket_name
s3_key = "index.zip"
handler = "index.handler"
runtime = var.runtime_handler
role = aws_iam_role.lambda_role.arn
}
resource "aws_iam_role" "lambda_role" {
name = "role_lambda_test"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Principal": {
"Service": "lambda.amazonaws.com"
},
"Effect": "Allow",
"Sid": ""
}
]
}
EOF
}
//lambda has to be manually triggered from api gateway
resource "aws_api_gateway_rest_api" "apiLambda" {
name = "myAPI"
description = "terraform test"
}
resource "aws_api_gateway_resource" "proxy" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
parent_id = aws_api_gateway_rest_api.apiLambda.root_resource_id
path_part = "{proxy+}"
}
resource "aws_api_gateway_method" "proxyMethod" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "lambda" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_method.proxyMethod.resource_id
http_method = aws_api_gateway_method.proxyMethod.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.main_endpoint_function.invoke_arn
}
resource "aws_api_gateway_method" "proxy_root" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_rest_api.apiLambda.root_resource_id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "lambda_root" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_method.proxy_root.resource_id
http_method = aws_api_gateway_method.proxy_root.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_function.main_endpoint_function.invoke_arn
}
resource "aws_api_gateway_deployment" "apideploy" {
depends_on = [
aws_api_gateway_integration.lambda,
aws_api_gateway_integration.lambda_root,
]
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
stage_name = "test"
}
Without the manually set trigger through the console I get Internal Server Error. With triggering functions correctly. Perhaps there is something wrong with my configuration ?
Update::
After adding the permission resources for Lambda and API Gateway I also created aws_api_gateway_method_response and aws_api_gateway_integration_response. On first run errors, on second run completes. I tried adding explicit implications but they didn't solve the issue..
resource "aws_api_gateway_method_response" "response_200" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = aws_api_gateway_method.proxy_root.http_method
status_code = "200"
depends_on = [
aws_api_gateway_rest_api.apiLambda,
aws_api_gateway_resource.proxy,
aws_api_gateway_method.proxy_root
]
}
resource "aws_api_gateway_integration_response" "MyDemoIntegrationResponse" {
rest_api_id = aws_api_gateway_rest_api.apiLambda.id
resource_id = aws_api_gateway_resource.proxy.id
http_method = aws_api_gateway_method.proxy_root.http_method
status_code = aws_api_gateway_method_response.response_200.status_code
depends_on = [
aws_api_gateway_rest_api.apiLambda,
aws_api_gateway_resource.proxy,
aws_api_gateway_method.proxy_root,
aws_api_gateway_method_response.response_200
]
# Transforms the backend JSON response to XML
response_templates = {
"application/xml" = <<EOF
#set($inputRoot = $input.path('$'))
<?xml version="1.0" encoding="UTF-8"?>
<message>
$inputRoot.body
</message>
EOF
}
}
the Error is:
│ Error: Error creating API Gateway Method Response: NotFoundException: Invalid Method identifier specified
│
│ with aws_api_gateway_method_response.response_200,
│ on main.tf line 130, in resource "aws_api_gateway_method_response" "response_200":
│ 130: resource "aws_api_gateway_method_response" "response_200" {
It seems you are missing the aws_lambda_permission resource [1]. In your case, you would need to add the following (similar to example from the reference):
resource "aws_lambda_permission" "apigw_lambda" {
statement_id = "AllowExecutionFromAPIGateway"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.main_endpoint_function.function_name
principal = "apigateway.amazonaws.com"
source_arn = "arn:aws:execute-api:${var.myregion}:${var.accountId}:${aws_api_gateway_rest_api.apiLambda.id}/*/${aws_api_gateway_method.proxyMethod.http_method}${aws_api_gateway_resource.proxy.path}"
}
Since I don't know which region and account you are using (I don't need to know that), you just have to replace the var.myregion with the API Gateway region and var.accountId with the AWS account where the API Gateway is created. You can achieve that by using data sources. In theory, you might as well leave out the method reference from the source_arn and use something like:
source_arn = "arn:aws:execute-api:${var.myregion}:${var.accountId}:${aws_api_gateway_rest_api.apiLambda.id}/*/*/*"
I have removed the second reference from the code and it is now referenced in [2].
[1] https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_integration#lambda-integration
[2] http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-control-access-using-iam-policies-to-invoke-api.html

Lambda Web API returning 403 for verbs other than POST

I have created a Serverless Lambda Web API using a built-in template. The web api is deployed to AWS using CloudFormation and everything is working fine (i.e. All GET, POST etc).
I have then checked the resources created by CloudFormation and created all of the resources using Terraform. I have double checked and REST API in the API Gateway, Lambda, Lambda permissions and triggers are exactly the same when created from both serverless template and terraform scripts.
The issue is the when using the resources created from Terraform, I am not able to call the Web API using HTTP verb other then POST. Below is my terraform script for the API Gateway.
resource "aws_api_gateway_rest_api" "SimulationServer_api_gateway_rest" {
name = "SimulationServer"
description = "Terraform Serverless Application Example"
}
resource "aws_api_gateway_resource" "api_gw_proxy_resource" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
parent_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.root_resource_id
path_part = "{proxy+}"
}
resource "aws_api_gateway_method" "api_gw_proxy_method" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
resource_id = aws_api_gateway_resource.api_gw_proxy_resource.id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "api_gw_proxy_integration" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
resource_id = aws_api_gateway_method.api_gw_proxy_method.resource_id
http_method = aws_api_gateway_method.api_gw_proxy_method.http_method
integration_http_method = "ANY"
type = "AWS_PROXY"
uri = aws_lambda_function.SimulationServer.invoke_arn
}
resource "aws_api_gateway_method" "api_gw_proxy_root_method" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
resource_id =
aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.root_resource_id
http_method = "ANY"
authorization = "NONE"
}
resource "aws_api_gateway_integration" "api_gw_proxy_root_integration" {
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
resource_id = aws_api_gateway_method.api_gw_proxy_root_method.resource_id
http_method = aws_api_gateway_method.api_gw_proxy_root_method.http_method
integration_http_method = "ANY"
type = "AWS_PROXY"
uri = aws_lambda_function.SimulationServer.invoke_arn
}
resource "aws_api_gateway_deployment" "api_gw_deployment" {
depends_on = [
aws_api_gateway_integration.api_gw_proxy_integration,
aws_api_gateway_integration.api_gw_proxy_root_integration,
]
rest_api_id = aws_api_gateway_rest_api.SimulationServer_api_gateway_rest.id
stage_name = var.env
}
output "base_url" {
value = aws_api_gateway_deployment.api_gw_deployment.invoke_url
}
It turned out to be very strange but simple solution.
Instead of using,
integration_http_method = "ANY"
When I used,
integration_http_method = "POST"
Then it started working for all HTTP Verbs (GET, POST, PUT etc)

Terraform: CloudWatch Event that notifies SNS

I'm learning TF and trying to apply an infrastructure that creates:
a simple lambda function
an SNS topic
get that lambda to subscribe the SNS topic
a Cloud Watch Event that publishes a message to the topic at some interval
a Cloud Watch Log Group to check if the lambda gets notified by the SNS
The lambda permission to allow calls from SNS
I'm able to apply that successfully. The infrastructure seems perfectly fine (it has the same aspect when I create that myself through the visual aws console)
But the cloud watch Event doesn't get triggered (when built from TF), so no message is published to SNS and lambda doesn't get called. I don't know why
Anyone know how can I accomplish that? Bellow my .tf script:
provider "aws" {
region = "us-east-1"
}
//lambda function handler & code file
resource "aws_lambda_function" "lambda-function" {
function_name = "Function01"
handler = "com.rafael.lambda.Function01"
role = "arn:aws:iam::12345:role/LambdaRoleTest"
runtime = "java8"
s3_bucket = aws_s3_bucket.sns-test.id
s3_key = aws_s3_bucket_object.file_upload.id
source_code_hash = filebase64sha256("../target/sns-cw-lambda-poc.jar")
}
//allow sns to call lambda
resource "aws_lambda_permission" "allow-sns-to-lambda" {
function_name = aws_lambda_function.lambda-function.function_name
action = "lambda:InvokeFunction"
principal = "sns.amazonaws.com"
source_arn = aws_sns_topic.call-lambdas-topic.arn
statement_id = "AllowExecutionFromSNS"
}
//app s3 repository
resource "aws_s3_bucket" "sns-test" {
bucket = "app-bucket-12345"
region = "us-east-1"
}
//app jar file
resource "aws_s3_bucket_object" "file_upload" {
depends_on = [
aws_s3_bucket.sns-test
]
bucket = aws_s3_bucket.sns-test.id
key = "sns-cw-lambda-poc.jar"
source = "../target/sns-cw-lambda-poc.jar"
server_side_encryption = "AES256"
etag = filebase64sha256("../target/sns-cw-lambda-poc.jar")
}
//to check lambda exec logs
resource "aws_cloudwatch_log_group" "lambda-cloudwatch-logs" {
name = "/aws/lambda/${aws_lambda_function.lambda-function.function_name}"
retention_in_days = 1
}
//rule to trigger SNS
resource "aws_cloudwatch_event_rule" "publish-sns-rule" {
name = "publish-sns-rule"
schedule_expression = "rate(1 minute)"
}
//cloud watch event targets SNS
resource "aws_cloudwatch_event_target" "sns-publish" {
count = "1"
rule = aws_cloudwatch_event_rule.publish-sns-rule.name
target_id = aws_sns_topic.call-lambdas-topic.name
arn = aws_sns_topic.call-lambdas-topic.arn
}
//SNS topic to subscribe
resource "aws_sns_topic" "call-lambdas-topic" {
name = "call-lambdas-topic"
}
//lambda subscribes the topic, so it should be nofied when other resource publishes to the topic
resource "aws_sns_topic_subscription" "sns-lambda-subscritption" {
topic_arn = aws_sns_topic.call-lambdas-topic.arn
protocol = "lambda"
endpoint = aws_lambda_function.lambda-function.arn
}
I figured it out, I forgot to add the SNS policies that allow CloudWatch to publish to SNS topic. To get the above script to work, just add this:
resource "aws_sns_topic_policy" "default" {
count = 1
arn = aws_sns_topic.call-lambdas-topic.arn
policy = "${data.aws_iam_policy_document.sns_topic_policy.0.json}"
}
data "aws_iam_policy_document" "sns_topic_policy" {
count = "1"
statement {
sid = "Allow CloudwatchEvents"
actions = ["sns:Publish"]
resources = [aws_sns_topic.call-lambdas-topic.arn]
principals {
type = "Service"
identifiers = ["events.amazonaws.com"]
}
}
}

Terraform - what is the URI to invoke lambda via alias?

Question
To invoke a Lambda from API GW, invoke_arn can be used for aws_lambda_function resources.
invoke_arn - The ARN to be used for invoking Lambda Function from API Gateway.
resource "aws_api_gateway_integration" "videos" {
rest_api_id = "${aws_api_gateway_rest_api.24_hour_video.id}"
resource_id = "${aws_api_gateway_method.videos_get.resource_id}"
http_method = "${aws_api_gateway_method.videos_get.http_method}"
integration_http_method = "GET"
type = "AWS_PROXY" # Lambda Proxy
uri = "${aws_lambda_function.list_videos.invoke_arn}"
}
What is to set in uri to invoke the same lambda via an alias?
resource "aws_lambda_alias" "lambda_alias_list_videos" {
name = "get_video_list"
description = "Alias to lambda_list_videos"
function_name = "${aws_lambda_function.list_videos.arn}"
function_version = "$LATEST"
}
The aws_lambda_alias resource creates an alias which points to a specific version of a Lambda function. The alias itself is not invocable.
Instead you should create an aws_lambda_function Data Source that points to the aliased version, and use its invoke_arn property. You can use the qualifier argument in an aws_lambda_function to specify either a version or an alias name (see AWS Lambda Invoke Docs for more info).
Your example shows you have already created an alias named get_video_list, which points to the $LATEST version. You need to create a new Data Source that points to this alias:
data "aws_lambda_function" "my_function_get_video_list" {
function_name = "your-function-name"
qualifier = "get_video_list"
}
You can now get the invocation ARN of the aliased function:
${aws_lambda_function.my_function.get_video_list.invoke_arn}
aws_lambda_alias resource has invoke_arn attribute (see docs) which is designed for API Gateway. For example:
resource "aws_lambda_alias" "api_function_alias_live" {
name = "live"
function_name = aws_lambda_function.api_function.function_name
function_version = "1"
}
resource "aws_api_gateway_integration" "proxy_integration" {
rest_api_id = aws_api_gateway_rest_api.api_gateway.id
resource_id = aws_api_gateway_resource.proxy_resource.id
http_method = aws_api_gateway_method.proxy_method.http_method
integration_http_method = "POST"
type = "AWS_PROXY"
uri = aws_lambda_alias.api_function_alias_live.invoke_arn
}

Resources