Terraform API gateway integration with Swagger (Localstack) - aws-lambda

i created a simple crud API lambda using mongoDb and nodeJs and it's working fine. so i passed to another step
I'm currently creating an AWS API Gateway with terraform by using an open api spec from a yaml file.
this is my swagger file:
info:
description: "API Ankan-v2"
version: "V0.0.1"
title: "ANKAN V2 API"
host: "localhost:4567"
basePath: "/restapis/api/{apiId}/test/_user_request_"
tags:
- name: "user"
description: "Operations about user"
schemes:
- "http"
# - "https"
paths:
/documents/{documentId}:
get:
tags:
- "document"
summary: "Find document by ID"
description: "Returns a single document"
operationId: "readOneDocument"
produces:
- "application/json"
parameters:
- name: "documentId"
in: "path"
description: "Id of document"
required: true
type: "string"
responses:
200:
description: "successful operation"
schema:
$ref: "#/definitions/Document"
404:
description: "document not found"
x-amazon-apigateway-integration:
uri: ${get_lambda_arn}
passthroughBehavior: "WHEN_NO_MATCH"
httpMethod: "POST"
type: "AWS_PROXY"
options:
consumes:
- "application/json"
produces:
- "application/json"
parameters:
- name: "documentId"
in: "path"
description: "Id of document"
required: true
type: "string"
responses:
200:
description: "200 response"
schema:
$ref: "#/definitions/Document"
x-amazon-apigateway-integration:
responses:
default:
statusCode: "200"
responseParameters:
method.response.header.Access-Control-Allow-Methods: "'DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT'"
method.response.header.Access-Control-Allow-Headers: "'Content-Type,Authorization,X-Amz-Date,X-Api-Key,X-Amz-Security-Token'"
method.response.header.Access-Control-Allow-Origin: "'*'"
requestTemplates:
application/json: "{\"statusCode\": 200}"
passthroughBehavior: "when_no_match"
type: "mock"
x-amazon-apigateway-any-method:
produces:
- "application/json"
parameters:
- name: "documentId"
in: "path"
description: "Id of document"
required: true
type: "string"
responses: {}
x-amazon-apigateway-integration:
responses:
default:
statusCode: "200"
uri: ${get_lambda_arn}
passthroughBehavior: "WHEN_NO_MATCH"
httpMethod: "POST"
cacheNamespace: "1rnijn"
cacheKeyParameters:
- "method.request.path.documentId"
type: "AWS"
definitions:
Document:
type: "object"
required:
- "documentName"
- "documentPath"
- "miniature"
- "status"
- "date"
properties:
_id:
type: "string"
readOnly: true
documentName:
type: "string"
documentPath:
type: "string"
miniature:
type: "string"
status:
type: "string"
enum:
- "VALID"
- "ERROR"
- "ONLOAD"
date:
type: "string"
format: "date"
xml:
name: "Document"
in my terraform i removed the integration and the method because i added them in my swagger .
so i need to import the swagger file in aws_api_gateway_rest_api with body = data.template_file.swagger.rendered
resource "aws_api_gateway_rest_api" "api" {
name = "api-gateway"
description = "document api"
body = data.template_file.swagger.rendered
}
data "template_file" "swagger" {
template = jsonencode("${path.module}/../../../../shared/swagger.yaml")
vars = {
get_lambda_arn = local.get_lambda_arn
}
}
output "json" {
value = data.template_file.swagger.rendered
}
locals {
get_lambda_arn = aws_lambda_function.lambda_document.invoke_arn
}
resource "aws_api_gateway_resource" "documents" {
rest_api_id = aws_api_gateway_rest_api.api.id
parent_id = aws_api_gateway_rest_api.api.root_resource_id
path_part = "documents"
depends_on = [aws_api_gateway_rest_api.api]
}
resource "aws_api_gateway_resource" "document_by_id" {
parent_id = aws_api_gateway_resource.documents.id
path_part = "{documentId}"
rest_api_id = aws_api_gateway_rest_api.api.id
}
resource "aws_api_gateway_deployment" "ApiDeloyment" {
rest_api_id = aws_api_gateway_rest_api.api.id
stage_name = "INT"
depends_on = [aws_api_gateway_rest_api.api]
}
resource "aws_lambda_function" "lambda_document" {
function_name = "lambda_document"
handler = "lambda.handler"
role = "arn:aws::iam::123456:role/irrelevant"
runtime = "nodejs10.x"
filename = "${path.module}/lambda.zip"
source_code_hash = filebase64sha256("${path.module}/lambda.zip")
}
After terraform apply i got this message in terraform logs
terraform apply --auto-approve
module.backend_api.module.document.aws_api_gateway_rest_api.api: Creating...
module.backend_api.module.document.aws_lambda_function.lambda_document: Creating...
module.backend_api.module.document.aws_lambda_function.lambda_document: Creation complete after 5s [id=lambda_document]
module.backend_api.module.document.data.template_file.swagger: Refreshing state...
module.backend_api.module.document.aws_api_gateway_rest_api.api: Still creating... [10s elapsed]
module.backend_api.module.document.aws_api_gateway_rest_api.api: Still creating... [20s elapsed]
module.backend_api.module.document.aws_api_gateway_rest_api.api: Still creating... [30s elapsed]
module.backend_api.module.document.aws_api_gateway_rest_api.api: Still creating... [40s elapsed]
module.backend_api.module.document.aws_api_gateway_rest_api.api: Still creating... [50s elapsed]
module.backend_api.module.document.aws_api_gateway_rest_api.api: Still creating... [1m0s elapsed]
module.backend_api.module.document.aws_api_gateway_rest_api.api: Still creating... [1m10s elapsed]
and my docker-compose logs show this
localstack | 2020-01-13T07:37:56:ERROR:localstack.services.generic_proxy: Error forwarding request: Expecting value: line 1 column 1 (char 0) Traceback (most recent call last):
localstack | File "/opt/code/localstack/localstack/services/generic_proxy.py", line 242, in forward
localstack | path=path, data=data, headers=forward_headers)
localstack | File "/opt/code/localstack/localstack/services/apigateway/apigateway_listener.py", line 53, in forward_request
localstack | data = data and json.loads(to_str(data))
localstack | File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
localstack | return _default_decoder.decode(s)
localstack | File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
localstack | obj, end = self.raw_decode(s, idx=_w(s, 0).end())
localstack | File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
localstack | raise JSONDecodeError("Expecting value", s, err.value) from None
localstack | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
PS : i tried with JSON but same problem
is there any solution for this? i want to integrate swagger with api gateway using terraform in localstack

This is an issue with the moto library. Under the hood, local stack uses moto to create its resources, including mock api gateway. Unfortunately moto does not currently support the body parameter for putting a swagger file definition as part of api gateway deployment. So even if you give a valid swagger file and api gateway configuration (ie one that works in aws) localstack will not be able to build it.

Related

attach authorizer to api gateway V2 route in aws cloudformation

How to attach authorizer to api gateway V2 route in aws cloudformation?
I am using Api Gateway v2 and cloudformation.
In AWS console it is just one click of one button "Attach Authorization" in "Routes" section
I am using simple authorizer:
My cloudformation looks like this:
Authorizer:
Type: 'AWS::ApiGatewayV2::Authorizer'
Properties:
ApiId: !Ref ApiGateway
AuthorizerPayloadFormatVersion: 2.0
AuthorizerResultTtlInSeconds: 5
AuthorizerType: REQUEST
AuthorizerUri: !Join
- ''
- - 'arn:'
- !Ref 'AWS::Partition'
- ':apigateway:'
- !Ref 'AWS::Region'
- ':lambda:path/2015-03-31/functions/'
- 'arn:aws:lambda:'
- !Ref 'AWS::Region'
- ':'
- !Ref 'AWS::AccountId'
- :function:${stageVariables.AuthorizerFunctionName}
- /invocations
EnableSimpleResponses: true
IdentitySource:
- '$request.header.Authorization'
Name: !Sub ${ProjectName}-gateway-authorizer
MyRoute:
Type: AWS::ApiGatewayV2::Route
Properties:
ApiId: !Ref ApiGateway
AuthorizationType: CUSTOM
AuthorizerId: !Ref Authorizer
RouteKey: 'POST /posts/all'
Target: !Join
- /
- - integrations
- !Ref PostsLambdaIntegrationGet
Authorizer lambda body:
import json
# import jwt
def lambda_handler(event, context):
print('*********** The event is: ***************')
print(event)
print('headers is:')
print(event['headers'])
print('headers Authorization is:')
# !!!!! DONWCASE by postam or api !!!!! "A" -> "a"
print(event['headers']['authorization'])
if event['headers']['authorization'] == 'abc123':
response = {
"isAuthorized": True,
"context": {
"anyotherparam": "values"
}
}
else:
response = {
"isAuthorized": False,
"context": {
"anyotherparam": "values"
}
}
print('response is:')
print(response)
return response
BTW I do not see this option in cli apigatewayv2 cli documentation too:

Running a Cloudformation project on localhost with DynamoDB, Lambda, and API Gateway resources using Localstack

I have a CloudFormation project with the following configs.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
team-up-interactions
Resources:
handler:
Type: AWS::Serverless::Function
Properties:
Description: Team Up interaction handler logic
CodeUri: ./handler/
Handler: handler.handler
Runtime: nodejs14.x
EventInvokeConfig:
MaximumRetryAttempts: 0
Timeout: 60
Environment:
Variables:
dbStorage: !Ref dbStorage
dbSearch: !Ref dbSearch
Architectures:
- x86_64
Policies:
- AWSLambdaBasicExecutionRole
Events:
HttpPost:
Type: Api
Properties:
Path: '/interaction-endpoint'
Method: post
dbStorage:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
-
AttributeName: "id"
AttributeType: "S"
KeySchema:
-
AttributeName: "id"
KeyType: "HASH"
BillingMode: "PAY_PER_REQUEST"
dbSearch:
Type: AWS::OpenSearchService::Domain
Properties:
EngineVersion: 'OpenSearch_1.1'
ClusterConfig:
InstanceCount: '1'
InstanceType: 't3.small.search'
EBSOptions:
EBSEnabled: true
VolumeSize: '20'
VolumeType: 'gp2'
These configs should create a Lambda, a DynamoDB table, and an OpenSearch Domain.
I am using LocalStack's samlocal command to run the project.
samlocal local start-api --host 0.0.0.0 --warm-containers EAGER
I have the following code that attempts to connect to the DynamoDB and is failing.
try {
const client = new DynamoDBClient({
endpoint: "http://localhost:4566"
});
await client.send(new UpdateItemCommand({
TableName: process.env.dbStorage,
Key: marshall({id: "test"}),
ExpressionAttributeValues: marshall({":test": "hello world"}),
UpdateExpression: "SET test = :test",
}));
//
const x = await client.send(new GetItemCommand({
TableName: process.env.dbStorage,
Key: marshall({id: "test"}),
}))
console.log("JSON.stringify(x)");
console.log(JSON.stringify(x));
} catch (ex) {
console.log("JSON.stringify(ex)")
console.log(JSON.stringify(ex))
}
This is the error.
{
"errno": -111,
"code": "ECONNREFUSED",
"syscall": "connect",
"address": "127.0.0.1",
"port": 4566,
"$metadata": {
"attempts": 1,
"totalRetryDelay": 0
}
}
I also tried creating the client without any parameter and that failed too.
const client = new DynamoDBClient();
This is the error.
{
"name": "UnrecognizedClientException",
"$fault": "client",
"$metadata": {
"httpStatusCode": 400,
"requestId": "VAHE0KG1D1UTE9AQNAREF9RJHFVV4KQNSO5AEMVJF66Q9ASUAAJG",
"attempts": 1,
"totalRetryDelay": 0
},
"__type": "com.amazon.coral.service#UnrecognizedClientException",
"message": "The security token included in the request is invalid."
}
How can I get LocalStack to create the DynamoDB resource from the configurations and/or how do I access those resources?
Update
I am using this endpoint and it seems to be able to connect the client to localstack.
const client = new DynamoDBClient({
endpoint: "http://host.docker.internal:4566",
});
I get the following error when doing a GetItemCommand
ResourceNotFoundException: Cannot do operations on a non-existent table
at deserializeAws_json1_0ResourceNotFoundExceptionResponse (/var/task/node_modules/#aws-sdk/client-dynamodb/dist-cjs/protocols/Aws_json1_0.js:3036:23)
at deserializeAws_json1_0GetItemCommandError (/var/task/node_modules/#aws-sdk/client-dynamodb/dist-cjs/protocols/Aws_json1_0.js:1729:25)
at processTicksAndRejections (internal/process/task_queues.js:95:5)
at async /var/task/node_modules/#aws-sdk/client-dynamodb/node_modules/#aws-sdk/middleware-serde/dist-cjs/deserializerMiddleware.js:7:24
at async /var/task/node_modules/#aws-sdk/client-dynamodb/node_modules/#aws-sdk/middleware-signing/dist-cjs/middleware.js:11:20
at async StandardRetryStrategy.retry (/var/task/node_modules/#aws-sdk/client-dynamodb/node_modules/#aws-sdk/middleware-retry/dist-cjs/StandardRetryStrategy.js:51:46)
at async /var/task/node_modules/#aws-sdk/client-dynamodb/node_modules/#aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:6:22
at async module.exports (/var/task/interaction/instant/help/help.js:31:19)
at async Runtime.exports.handler (/var/task/handler.js:94:16) {
'$fault': 'client',
'$metadata': {
httpStatusCode: 400,
requestId: '3ad8e2c9-be76-449d-bc86-4b529f3ca45b',
extendedRequestId: undefined,
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
},
__type: 'com.amazonaws.dynamodb.v20120810#ResourceNotFoundException'
}

Getting "internal server error" on passing binary data to AWS Lambda function deployed using serverless framework and apigw-binary plugin

what I'm trying
Passing binary data via Lambda integration in API gateway. Lambda returns back text.
issue
The function returns desired output when API gateway is configured from console. To implement it using serverless framework I installed serverless-apigw-binary plugin. The required binary types show up in API gateway>settings>binary media types. However on calling API I get "internal server error". The function works properly on application/json type input. After enabling-disabling lambda proxy integration and adding mappings via console, I get correct output.
serverless.yml file
org: ------
app: ---------
service: ---------
frameworkVersion: ">=1.34.0 <2.0.0"
plugins:
- serverless-python-requirements
- serverless-offline
- serverless-apigw-binary
provider:
name: aws
runtime: python3.7 #fixed with pipenv
region: us-east-1
memorySize: 128
timeout: 60
profile: ----
custom:
pythonRequirements:
usePipenv: true
useDownloadCache: true
useStaticCache: true
apigwBinary:
types: #list of mime-types
- 'application/octet-stream'
- 'application/zip'
functions:
main:
handler: handler.main
events:
- http:
path: ocr
method: post
integration: lambda
request:
passThrough: WHEN_NO_TEMPLATES
template:
application/zip: '
{
"type": "zip",
"zip": "$input.body",
"lang": "$input.params(''lang'')",
"config": "$input.params(''config'')",
"output_type": "$input.params(''output_type'')"
}'
application/json: '
{
"type": "json",
"image": $input.json(''$.image''),
"lang": "$input.params(''lang'')",
"config": "$input.params(''config'')",
"output_type": "$input.params(''output_type'')"
}'
application/octet-stream: '
{
"type": "img_file",
"image": "$input.body",
"lang": "$input.params(''lang'')",
"config": "$input.params(''config'')",
"output_type": "$input.params(''output_type'')"
}'
handler.py
def main(event, context):
# do something on event and get txt
return txt
edit
I compared swagger definitions and found this
1. API generated from console(working)
paths:
/ocr:
post:
consumes:
- "application/octet-stream"
produces:
- "application/json"
responses:
API generated from serverless framework
paths:
/ocr:
post:
consumes:
- "application/x-www-form-urlencoded"
- "application/zip"
- "application/octet-stream"
- "application/json"
responses:
produces: - "application/json" is missing. How do I add it in serverless?

AWS SAM template/cloudformation No integration defined for method (Service: AmazonApiGateway

I am trying to deploy a lambda function and API gateway . I create a .net core web API project with AWS CLI . Deploying only the function and creating the API gateway and resource manually on aws web console does work.
If I do include the API gateway in the template, after doing SAM package deploying through web console or CLI I get the following error:
"No integration defined for method (Service: AmazonApiGateway; Status Code: 400; Error Code: BadRequestException; Request ID: ....)"
Is anything wrong or missing here?
SAM package command:
sam package --template-file sam-profile.yaml --output-template-file serverless-output.yaml --s3-bucket testapp-fewtfvdy-lambda-deployments
SAM Template:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
ProfileFunction:
Type: AWS::Serverless::Function
Properties:
Handler: testapp.Profile.NetCoreVS::testapp.Profile.NetCoreVS.LambdaEntryPoint::FunctionHandlerAsync
Runtime: dotnetcore2.0
MemorySize : 128
Timeout : 5
CodeUri: bin/Release/netcoreapp2.0/publish
Events:
ProfileAny:
Type: Api
Properties:
RestApiId: !Ref ProfileApiGateway
Path: /profile/v1
Method: GET
ProfileApiGateway:
DependsOn: ProfileFunction
Type: 'AWS::Serverless::Api'
Properties:
StageName: Prod
DefinitionUri: './swagger.yaml'
swagger.yaml:
---
swagger: '2.0'
info:
version: v1
title: ProfileAPI
paths:
"/profile/v1":
get:
tags:
- Values
operationId: ProfileV1Get
consumes: []
produces:
- text/plain
- application/json
- text/json
parameters: []
responses:
'200':
description: Success
schema:
type: array
items:
type: string
definitions: {}
.net core method:
[Route("profile/v1")]
public class ValuesController : Controller
{
// GET api/values
[HttpGet]
public IEnumerable<string> Get()
{
return new string[] { "value1", "value2","value_new3" };
}
}
Your swagger definition is missing x-amazon-apigateway-integration.
This should provide that integration layer for you:
---
swagger: '2.0'
info:
version: v1
title: ProfileAPI
paths:
"/profile/v1":
get:
tags:
- Values
operationId: ProfileV1Get
consumes: []
produces:
- text/plain
- application/json
- text/json
parameters: []
responses:
'200':
description: Success
schema:
type: array
items:
type: string
x-amazon-apigateway-integration:
httpMethod: post
type: aws_proxy
uri:
Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${ProfileFunction.Arn}/invocations
definitions: {}
Note that the httpMethod for x-amazon-apigateway-integration is always POST, since API Gateway always makes POST requests to Lambda regardless of what method your API route is using.

swagger editor UI sends a OPTIONS request method instead of PUT and DELETE

http://editor.swagger.io sends a OPTIONS request method instead of PUT and DELETE but for POST and GET it's work perfectly
even I have installed a Chrome extension that adds Access-Control-Allow-Origin to outgoing requests. and same added in the c# code like below
[EnableCors(origins: "", headers: "", methods: "*", exposedHeaders: "x-docname,x-docid")]
put:
tags:
- sources
summary: Updated sources Groups
description: This Will Create new Group under sources
operationId: PUT
consumes:
- application/x-www-form-urlencoded
produces:
- application/json
parameters:
- name: sourceId
in: path
description: name that need to be updated
required: true
type: string
- name: text
in: formData
description: Updated name of the text
required: true
type: string
responses:
200:
description: "Ok : Successful requests"
201:
description: "Created : Successful creation"
400:
description: "Bad Request : The path info doesn't have the right format"
403:
description: "Forbidden : The invoker is not authorized to invoke the operation"
404:
description: "Not Found : The object referenced by the path does not exist"
500:
description: "Internal Server Error : The execution of the service failed in some way"

Resources