Serverless with localstack-serverless plugin not querying template locally - aws-lambda

Question

Based on the official example on GitHub demonstrating a serverless REST API I enabled the localstack-serverless plugin so I could develop my services locally. I adjusted the serverless.yml file accordingly:
service: serverless-rest-api-with-dynamodb
frameworkVersion: ">=1.1.0 <2.0.0"
provider:
name: aws
runtime: python2.7
deploymentBucket:
name: ${self:service}-${opt:stage}-deployment-bucket
environment:
DYNAMODB_TABLE: ${self:service}-${opt:stage, self:provider.stage}
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:Query
- dynamodb:Scan
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
Resource: "arn:aws:dynamodb:${opt:region, self:provider.region}:*:table/${self:provider.environment.DYNAMODB_TABLE}"
custom:
localstack:
stages:
- local
- dev
endpoints:
S3: http://localhost:4572
DynamoDB: http://localhost:4570
CloudFormation: http://localhost:4581
Elasticsearch: http://localhost:4571
ES: http://localhost:4578
SNS: http://localhost:4575
SQS: http://localhost:4576
Lambda: http://localhost:4574
Kinesis: http://localhost:4568
plugins:
- serverless-localstack
functions:
create:
handler: todos/create.create
events:
- http:
path: todos
method: post
cors: true
list:
handler: todos/list.list
events:
- http:
path: todos
method: get
cors: true
get:
handler: todos/get.get
events:
- http:
path: todos/{id}
method: get
cors: true
update:
handler: todos/update.update
events:
- http:
path: todos/{id}
method: put
cors: true
delete:
handler: todos/delete.delete
events:
- http:
path: todos/{id}
method: delete
cors: true
resources:
Resources:
TodosDynamoDbTable:
Type: 'AWS::DynamoDB::Table'
DeletionPolicy: Retain
Properties:
AttributeDefinitions:
-
AttributeName: id
AttributeType: S
KeySchema:
-
AttributeName: id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:provider.environment.DYNAMODB_TABLE}
Deploying on localstack running over docker:
SLS_DEBUG=3 serverless deploy --stage local --region us-east-1
it fails with the following message on the localstack side:
localstack_1 | 2019-12-04 10:41:33,692:API: 127.0.0.1 - - [04/Dec/2019 10:41:33] "PUT /serverless-rest-api-with-dynamodb-local-deployment-bucket HTTP/1.1" 200 -
localstack_1 | 2019-12-04 10:41:41,402:API: 127.0.0.1 - - [04/Dec/2019 10:41:41] "GET /serverless-rest-api-with-dynamodb-local-deployment-bucket?location HTTP/1.1" 200 -
localstack_1 | 2019-12-04 10:41:41,470:API: 127.0.0.1 - - [04/Dec/2019 10:41:41] "GET /serverless-rest-api-with-dynamodb-local-deployment-bucket?list-type=2&prefix=serverless%2Fserverless-rest-api-with-dynamodb%2Flocal HTTP/1.1" 200 -
localstack_1 | 2019-12-04 10:41:41,544:API: 127.0.0.1 - - [04/Dec/2019 10:41:41] "PUT /serverless-rest-api-with-dynamodb-local-deployment-bucket/serverless/serverless-rest-api-with-dynamodb/local/1575456101319-2019-12-04T10%3A41%3A41.319Z/compiled-cloudformation-template.json HTTP/1.1" 200 -
localstack_1 | 2019-12-04 10:41:41,569:API: 127.0.0.1 - - [04/Dec/2019 10:41:41] "PUT /serverless-rest-api-with-dynamodb-local-deployment-bucket/serverless/serverless-rest-api-with-dynamodb/local/1575456101319-2019-12-04T10%3A41%3A41.319Z/serverless-rest-api-with-dynamodb.zip HTTP/1.1" 200 -
localstack_1 | 2019-12-04T10:41:41:DEBUG:localstack.services.cloudformation.cloudformation_listener: Error response from CloudFormation (400) POST /: b'<ErrorResponse xmlns="http://cloudformation.amazonaws.com/doc/2010-05-15/">\n <Error>\n <Type>Sender</Type>\n <Code>ValidationError</Code>\n <Message>Stack with id serverless-rest-api-with-dynamodb-local does not exist</Message>\n </Error>\n <RequestId>cf4c737e-5ae2-11e4-a7c9-ad44eEXAMPLE</RequestId>\n</ErrorResponse>'
localstack_1 | 2019-12-04T10:41:41:WARNING:localstack.services.awslambda.lambda_api: Function not found: arn:aws:lambda:us-east-1:000000000000:function:serverless-rest-api-with-dynamodb-local-list
localstack_1 | 2019-12-04T10:41:41:WARNING:localstack.services.awslambda.lambda_api: Function not found: arn:aws:lambda:us-east-1:000000000000:function:serverless-rest-api-with-dynamodb-local-create
localstack_1 | 2019-12-04T10:41:41:WARNING:localstack.services.awslambda.lambda_api: Function not found: arn:aws:lambda:us-east-1:000000000000:function:serverless-rest-api-with-dynamodb-local-update
localstack_1 | 2019-12-04T10:41:41:WARNING:localstack.services.awslambda.lambda_api: Function not found: arn:aws:lambda:us-east-1:000000000000:function:serverless-rest-api-with-dynamodb-local-get
localstack_1 | 2019-12-04T10:41:41:WARNING:localstack.services.awslambda.lambda_api: Function not found: arn:aws:lambda:us-east-1:000000000000:function:serverless-rest-api-with-dynamodb-local-delete
localstack_1 | 2019-12-04T10:41:42:ERROR:localstack.services.generic_proxy: Error forwarding request: Unable to fetch template body (code 404) from URL https://s3.amazonaws.com/serverless-rest-api-with-dynamodb-local-deployment-bucket/serverless/serverless-rest-api-with-dynamodb/local/1575456101319-2019-12-04T10:41:41.319Z/compiled-cloudformation-template.json Traceback (most recent call last):
localstack_1 | File "/opt/code/localstack/localstack/services/generic_proxy.py", line 240, in forward
localstack_1 | path=path, data=data, headers=forward_headers)
localstack_1 | File "/opt/code/localstack/localstack/services/cloudformation/cloudformation_listener.py", line 151, in forward_request
localstack_1 | modified_request = transform_template(req_data)
localstack_1 | File "/opt/code/localstack/localstack/services/cloudformation/cloudformation_listener.py", line 70, in transform_template
localstack_1 | template_body = get_template_body(req_data)
localstack_1 | File "/opt/code/localstack/localstack/services/cloudformation/cloudformation_listener.py", line 105, in get_template_body
localstack_1 | raise Exception('Unable to fetch template body (code %s) from URL %s' % (response.status_code, url[0]))
localstack_1 | Exception: Unable to fetch template body (code 404) from URL https://s3.amazonaws.com/serverless-rest-api-with-dynamodb-local-deployment-bucket/serverless/serverless-rest-api-with-dynamodb/local/1575456101319-2019-12-04T10:41:41.319Z/compiled-cloudformation-template.json
Where we can see that the serverless deployment is trying to reach Amazon S3 directly, while I would expect it to use the local endpoint defined in the serverless.yml config (http://localhost:4572).
Is there a missing entry in the serverless.yml config file in order to force using the localstack S3 storage ?

Answer 1

There is a regression with the localstack image I was using. Using the docker image localstack/localstack:0.10.5 the deployment is successful.
Explicitely setting the region both in the config files AND the code also helps.