I am creating a AWS serverless application with SAM. Basically what I would like to achieve is to use API Gateway's different stages (dev/test/prod) to invoke various Lambda functions alias (dev/test/prod).
I am totally stucked, I would like to know what are the strategies people have taken to shift lambda traffics, eg. from LambdaA:dev to LambdaA:prod?
I have tried to use "AutoPublishAlias", but in SAM AutoPublishAlias you can't have more then one alias in a single cloudformation stack, so that makes traffic shifting impossible.
Before using a single stack, I have also used Canary Deployment, it works ok when I separate lambda into multiple envrs (ie. dev-lambaA, test-lambdaA, prod-lambdaA) managed by different cloudformation stack. But I would like to reduce the number of lambda functions by only have lambdas reside in a single stack.
What you can do is add the following to your template.yaml file:
Resources:
ProductionAPI:
Type: AWS::Serverless::Api
Properties:
StageName: PRD
DefinitionUri: ./prdswagger.yaml
DevelopmentAPI:
Type: AWS::Serverless::Api
Properties:
StageName: DEV
DefinitionUri: ./devswagger.yaml
And use the swagger files to create your endpoints. At every endpoint add an x-amazon-apigateway-integration to the correct lambda version that you are targeting.
x-amazon-apigateway-integration:
httpMethod: "POST"
type: aws_proxy
uri: "arn:aws:apigateway:eu-central-1:lambda:path/2015-03-31/functions/arn:aws:lambda:eu-central-1:[account_nr]:function:[myfunctionname]:PRD/invocations"
passthroughBehavior: "when_no_match"
Related
Note: Please read my question before flagging it as it is different from many other Provisioned Concurrency questions I've seen on SO.
I need to configure provisioned concurrency in one of my existing applications that uses CloudFormation templates with Lambda functions (AWS::Lambda::Function resource, NOT SAM with AWS::Serverless::Function resource).
I did some tests but here's where I am stuck right now:
Provisioned concurrency can only be configured for Alias or Version however...
It can't be configured for Alias that points to the Live function, it must point to a Version
It can't be configured for Version that is the $LATEST
So what's the "right" way to setup Provisioned concurrency?
When deploying CloudFormation template, I can create a Version resource which can have provisioned concurrency configured (shown below). The API Gateway endpoint can directly point to this specific Version instead of the $LATEST version.
However, there is no way to update Version resource. Once it's created, it can only be deleted.
So each time I update my lambda function code, I would have to manually remove the current Version resource from CloudFormation and add a new one so it can create a new Version. This defeats the purpose of having template to deploy.
What are my other options? How can I have a Lambda function ($LATEST, Version or Alias) that has
provisioned concurrency configured
I can make changes to Lambda code without having to modify CloudFormation template each time.
######## LambdaTest Function ########
LambdaTest:
Type: "AWS::Lambda::Function"
DependsOn:
- LambdaRole
- LambdaPolicy
Properties:
FunctionName: "LambdaTest"
Role: !GetAtt LambdaRole.Arn
Code:
S3Bucket: !Ref JarFilesBucketName
S3Key: LambdaTest.jar
Handler: com.example.RnD.LambdaTest::handleRequest
Runtime: "java11"
Timeout: 30
MemorySize: 512
####### LambdaTest Function Version ########
LambdaTestVersion:
Type: "AWS::Lambda::Version"
Properties:
FunctionName: !GetAtt LambdaTest.Arn
Description: "v1"
ProvisionedConcurrencyConfig:
ProvisionedConcurrentExecutions: 5
While you are correct, we cannot use $LATEST alias per AWS. I also think you are missing some key pieces of information, that SAM usually generates the Lambda, let me try and share what I did.
First, just FYI - SAM generates the Lambda resources that you are seeking, as part of its process. Now if you want to go directly/manually/create the lambda resources, sure you can, then you will have to wire up the AutoPublishAlias: live.
Second part, in the picture (AWS SDK) is the actual solution on deploying live parts.
My solution/workaround is to perform an AutoPublishAlias: live in the function preference
you can just add that per the ref, or follow the steps below and compare/copy/paste what the SAM file gave you vs the AWS Lambda file has
Optional for your help -->
Select Add configuration, with provisioned Concurrency enabled for a specific Lambda function version or alias (but you can’t use $LATEST).
Since you can have different settings for each version of a function. Using an alias, it is easier to enable these settings to the correct version of your function.
Select the alias live ,
note: that
you will have to keep that updated to the latest version using the AWS SAM AutoPublishAlias function preference.
Then go to the Provisioned Concurrency, use something like 500 and Save.
Now -> Provisioned Concurrency configuration is in progress and all execution environments are ready to handle the inbound concurrent requests.
I'm fairly new to the Serverless Framework but I was able to achieve a lot of the bits and pieces of our aspired infrastructure using the framework and the serverless-lift plugin.
The serverless-lift plugin allows me to easily create worker queues with workers and DLQs for each of them. That's exactly what we need as it allows us to redrive the failed jobs. See this great article https://medium.com/serverless-transformation/serverless-queues-and-workers-designing-lift-d870afdba867 (under "A production-ready approach")
The Serverless Framework can create Lambda functions triggered by the EventBridge - that's also amazing.
However, as seen in the diagram we'd like the EventBridge to put the events - according to rules - into the respective queues. That bit seems to be missing or I haven't found it yet.
My question now:
Can I, and if so how can I, have the EventBridge be the central piece in our infrastructure, serving events to the worker queues?
From what I read I'd be able to define resources via CloudFormation notation (see below). Since Serverless is so much simpler and can easily be handled by our developers, I'd like to avoid using CF if possible.
Another approach could be to have intermediate Lambda functions putting the event into the queues but it's adding unnecessary complexity.
Thanks in advance!
For those interested, here's the CF code:
# serverless-list constructs
constructs:
my_queue:
type: queue
worker:
name: my-dev-my_queue_worker
handler: src/functions/hello-dlq/handler.main
# CloudFormation
resources:
Resources:
EventBridge:
Type: AWS::Events::EventBus
Properties:
Name: my-dev-event-bus
EventBridgeSQSRule:
Type: AWS::Events::Rule
DependsOn:
- EventBridge
Properties:
Name: my-dev-event-bus-rule1
Description: Test Event Bus
EventBusName: my-dev-event-bus
EventPattern:
source:
- test.v1
State: ENABLED
Targets:
- Arn:
Fn::GetAtt:
- ${construct:my_queue.queueArn}
- Arn
Id: SQSQueueTestV1
EventBridgeSQSSendPolicy:
Type: AWS::SQS::QueuePolicy
Properties:
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: events.amazonaws.com
Action: sqs:SendMessage
Resource: arn:aws:sqs:${aws:region}:${aws:accountId}:*
Queues:
- ${construct:my_queue.queueUrl}
I currently have a stack deployed to AWS which has a lot of REST endpoints (Lambda functions), some other Lambdas for maintenance operations, and a DynamoDB, Cognito User Pool, Elastic Search Domain, IAM roles etc. It's all deployed with the serverless framework, using serverless.yml for defining the stack.
In order to avoid the 200 resources limit (and get a better structure), I'm trying to split the current stack into multiple stacks.
The plan is to keep the current stack for all the resources with persisted data (DynamoDB, Elastic Search, Cognito, IAM etc), and then define new stacks for the lambda functions. One for the maintenance functions, and a couple of other stacks for different types for functions invoked by HTTP through API Gateway.
Now to the problem: I have commented out the entire functions: section of serverless.yml.
I have a section containing the resources, which looks like this:
resources:
- ${file(resources/dynamoDb.yml)}
- ${file(resources/cognito.yml)}
- ${file(resources/iam.yml)}
- ${file(resources/elasticsearch.yml)}
When I try do deploy the stack now (with all functions commented out), I get this error:
Error: The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [ApiGatewayRestApi] in the Resources block of the template
The reason that I get this error is probably because I have a reference to ApiGatewayRestApi in resources/iam.yml:
GetVehicleByLicensePlatePolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
ManagedPolicyName: GetVehicleByLicensePlate
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Action: execute-api:Invoke
Resource:
Fn::Join:
- ""
- - "arn:aws:execute-api"
- ":"
- Ref: AWS::Region
- ":"
- Ref: AWS::AccountId
- ":"
- Ref: ApiGatewayRestApi
- "/*/GET/vehicles/licenseplate/*"
I understand that the ApiGatewayRestApi reference does not resolve when I have removed all functions triggered by HTTP, since there's no API Gateway being deployed in this stack.
I'll have the HTTP lambda functions in a couple of other stacks, but these stacks will depend on this one. (And I sure don't want circular dependencies.)
So how do I make it possible for my "main" stack to have a reference to the API Gateway used by the sub-stacks?
What is the common/best practice way to solve this problem?
I think I solved it by moving the API gateway specific IAM roles to the stack that contains the relevant endpoints. That actually makes sense to me, since the root stack does not need to know about api endpoint specific roles.
i have the following lambda configuration:
MyFunc:
handler: my_handler
timeout: 60
role: myrole
events:
- stream:
type: dynamodb
arn: <<dynamo_db_stream_arn>
startingPosition: LATEST
maximumRetryAttempts: 3
destinations:
onFailure: <sqs_queue_arn>
enabled: True
Yet, when deploying, i don't see that the onFailure is even rendered in the cloudformation template.
i've set it up as said in this documentation:
https://serverless.com/framework/docs/providers/aws/events/streams/
Any idea what i'm missing?
==========================
So, completing Snickers3192's answer - I actually am not sure what's wrong with the configuration above, as serverless should support it, but eventually what i did is created the stream handler in another resource, so basically my serverless looks like that:
functions:
MyFunc:
handler: my_handler
timeout: 60
role: myrole
resources:
Resources:
MySourceMapping:
Type: AWS::Lambda::EventSourceMapping
DependsOn: MyFuncLambdaFunction
Properties:
EventSourceArn: <dynamo_db_stream_arn>
FunctionName: MyFunc
MaximumRetryAttempts: 3
StartingPosition: LATEST
DestinationConfig:
OnFailure:
Destination: <sqs_queue_qrn>
I think you're just missing "arn:"
Here's what worked for us.
maximumRetryAttempts: 10
maximumRecordAgeInSeconds: 300
bisectBatchOnFunctionError: true
destinations:
onFailure:
arn:
Fn::GetAtt:
- fileReducerDeadLetterQueue
- Arn
type: sqs
Even though I like serverless framework, I don't recommend using it for anything other than developing Lambda functions, I wouldn't even use the http event for creating an API gateway. Stick to the unix philosophy do one thing good, that's how I feel serverless should stick to that, not try and become another terraform or something, it isn't.
So create your Lambda functions in serverless and that's it. Do the other stuff else where. If the resource can be managed effectively in Cloudformation AWS::Lambda::EventSourceMapping, then you can use that. If it makes sense to put it at the bottom of the serverless.yml in resources: you can do that, but if not then let it have it's own template.
There is quite a number of permissions needed for setting up your lambda for DynamoDB streams, I wouldn't trust serverless to do that for you. A proper AWS prod setup as well might not let some external tool create iam roles as well.
As soon as you differ the slightest little bit from the serverless default cloudformation template, you'll have problems, probably you are spending hours right now, on a tool which was supposed to save you time, therefore defeating it's purpose. I suggest making more stacks than less, and using conventions when one stack needs a Lambda in another, this is actually more manegable as when one thing fails you can still update other stacks, and swap stacks as you change, you can't do that if you stick it all in a serverless.yml.
I'm using Serverless Framework and have multiple services which are attempting to use the same SQS queue. I can successfully make the resource in the first service but the second one is missing the lambda trigger when deployed to AWS. Hardcoding the ARN ID will successfully make the trigger so I can only assume I have something wrong with my syntax/indentation, but it's very similar to how I'm exporting/importing my API Gateway details and I'm just not seeing it.
I have an SQS Queue set up and exported from my first service like this:
resources:
- Resources:
InitializeAuthenticationQueue:
Type: "AWS::SQS::Queue"
Properties:
QueueName: "InitializeAuthenticationQueue"
- Outputs:
InitializeAuthenticationQueueArnId:
Value:
Fn::GetAtt:
- InitializeAuthenticationQueue
- Arn
Export:
Name: ${self:provider.stage}-InitializeAuthenticationQueueQueueArnId
In my second service I am attempting to use the SQS ARN ID as a trigger for a function, like this:
functions:
authenticationIntialize:
handler: myHandlerFile.myHandler
events:
- sqs:
arn:
'Fn::ImportValue': ${self:provider.stage}-InitializeAuthenticationQueueArnId
I've also tried this to see if I have my indentation wrong:
functions:
authenticationIntialize:
handler: myHandlerFile.myHandler
events:
- sqs:
arn:
'Fn::ImportValue': ${self:provider.stage}-InitializeAuthenticationQueueArnId
Feel like I'm missing something obvious on this one, but I've been stuck on it way too long. Anyone able to help me spot the obvious?
What errors do you get? What does the generated .serverless/cloudformation-template-update-stack.json have for the export and import values?
I usually find it easier to use the internal Serverless CloudFormation property reference. So where you are trying to import the SQS ARN do this:
${cf:STACK_NAME.InitializeAuthenticationQueueArnId}
where STACK_NAME is the name of the CloudFormation stack generated by the Serverless deployment this has the SQS ARN output. Using this method the way you reference the value to import is via the CloudFormation key and not the export name (which is admittedly confusing)