Cross-stack reference within a lambda function - aws-lambda

Going round in circles and tearing my hair out on this one, any pointers are greatly appreciated.
I have a CloudFormation stack which creates an EC2 instance. I am outputting its ID as follows:
Outputs:
DevBoxInstanceId:
Description: The instance ID of the EC2 Dev_Box.
Value: !Ref TestInstance
Export:
Name: DevBoxId
Now, in the console I can see that this outputs the ID as I'd like. My problem is that I can't work out how to reference this in my second stack. I haven't used Fn::ImportValues much but for obvious reasons I'd like to. My second stack creates a lambda function which will stop an instance. I want to reference the DevBoxId within the function - have I misunderstood something here? I've tried several variations on the following:
LambdaFunctionStop:
Type: AWS::Lambda::Function
Properties:
Runtime: python3.7
Timeout: 30
Code:
ZipFile: |
import boto3
region = 'eu-west-2'
instances = ['!ImportValue DevBoxId']
ec2 = boto3.client('ec2', region_name=region)
def handler(event, context):
ec2.stop_instances(InstanceIds=instances)
print('stopped your instances: ' + str(instances))
Description: Automatically stop Dev_Env instances based on specified schedule.
Handler: index.handler
Role: !GetAtt 'IAMRole.Arn'
The relevant part which I'm seeking help on is:
instances = ['!ImportValue DevBoxId']
How do I write this correctly?

Your ZipFile component is meant to be a string. So you will need to refer to Export Values outside that string and then substitute the value within that string. Try the below
LambdaFunctionStop:
Type: AWS::Lambda::Function
Properties:
Runtime: python3.7
Timeout: 30
Code:
ZipFile:
!Sub
- |
import boto3
region = 'eu-west-2'
instances = '${devboxvalue}'
ec2 = boto3.client('ec2', region_name=region)
def handler(event, context):
ec2.stop_instances(InstanceIds=instances)
print('stopped your instances: ' + str(instances))
- { devboxvalue : !ImportValue "DevBoxId" }
Description: Automatically stop Dev_Env instances based on specified schedule.
Handler: index.handler
Role: !GetAtt 'IAMRole.Arn'

You can pass the instance id to tha lambda's environment variables this way:
and you can then access if from you code depending on the way you read env variables

Related

Is it possible to set EventBridge ScheduleExpression value from SSM in Serverless

I want to schedule one lambda via AWS EventBridge. The issue is I want to read the number value used in ScheduledExpression from SSM variable GCHeartbeatInterval
Code I used is below
heartbeat-check:
handler: groupconsultation/heartbeatcheck.handler
description: ${self:custom.gitVersion}
timeout: 15
memorySize: 1536
package:
include:
- groupconsultation/heartbeatcheck.js
- shared/*
- newrelic-lambda-wrapper.js
events:
- eventBridge:
enabled: true
schedule: rate(2 minutes)
resources:
Resources:
GCHeartbeatInterval:
Type: AWS::SSM::Parameter
Properties:
Name: /${file(vars.js):values.environmentName}/lambda/HeartbeatInterval
Type: String
Value: 1
Description: value in minute. need to convert it to seconds/milliseconds
Is this possible to achieve in serverless.yml ?
Reason for reading it from SSM is, it's a heartbeat service and the same value will be used by FE to send a heartbeat in set interval. BE lambda needs to be triggerred after 2x heartbeat interval
It turns out it's not possible. Only solution to it was to pass the variable as a command line argument. something like below.
custom:
mySchedule: ${opt:mySchedule, 1} # Allow overrides from CLI
...
schedule: ${self:custom.mySchedule}
...
resources:
Resources:
GCHeartbeatInterval:
Type: AWS::SSM::Parameter
Properties:
Name: /${file(vars.js):values.environmentName}/lambda/HeartbeatInterval
Type: String
Value: ${self:custom.mySchedule}
With the other approach, if we make it work we still have to redeploy the application as we do need to redeploy in this case also.

In CloudFormation, how do I target a Lambda alias in Events::Rule

I'm trying to trigger a Lambda:alias (the alias is key here) on a schedule. The following code errors out with
"SampleLambdaLiveAlias is not valid. Reason: Provided Arn is not in
correct format. (Service: AmazonCloudWatchEvents; Status Code: 400;
Error Code: ValidationException;"
How do I properly target the lambda:alias in CloudFormation? I've tried !Ref, !Sub and just the logical name.
My custom-resource approach to retrieving the latest lambda version appears to be a necessary evil of setting up the "live" alias because AWS maintains old lambda versions, even after you delete the lambda and stack AND a valid version is required for a new alias. If anyone knows a more elegant approach to that problem, please see: how-to-use-sam-deploy-to-get-a-lambda-with-autopublishalias-and-additional-alises
SampleLambdaFunction:
Type: AWS::Serverless::Function
Properties:
FunctionName: SampleLambda
AutoPublishAlias: staging
CodeUri: src/
Handler: SampleLambda.handler
MemorySize: 512
Runtime: nodejs12.x
Role: !GetAtt SampleLambdaRole.Arn
SampleLambdaLiveAlias:
Type: AWS::Lambda::Alias
Properties:
FunctionName: !Ref SampleLambdaFunction
FunctionVersion: !GetAtt SampleLambdaGetMaxVersionFunction.version
Name: live
SampleLambdaFunctionScheduledEvent:
Type: AWS::Events::Rule
Properties:
State: ENABLED
ScheduleExpression: rate(1 minute) # same as cron(0/1 * * * ? *)
Description: Run SampleLambdaFunction once every 5 minutes.
Targets:
- Id: EventSampleLambda
Arn: SampleLambdaLiveAlias
Your error is in the last line of the piece of configuration you shared. In order to get the resource ARN you need to use Ref intrinsic function such as, !Ref SampleLambdaLiveAlias:
SampleLambdaFunctionScheduledEvent:
Type: AWS::Events::Rule
Properties:
State: ENABLED
ScheduleExpression: rate(1 minute) # same as cron(0/1 * * * ? *)
Description: Run SampleLambdaFunction once every 5 minutes.
Targets:
- Id: EventSampleLambda
Arn: !Ref SampleLambdaLiveAlias
Be aware that Ref intrinsic function may return different things for different types of resources. For Lambda alias it returns the ARN, just what you need.
You can check the official documentation for more detail.

How to specify AutoPublishAlias at runtime of 'sam package/deploy'?

I am wrapping our AWS SAM deployment in Jenkins as part of our CI/CD pipeline. I only want to add the "live" alias to the lambdas when we are merging for example, yet I want "branch builds" to be without an alias. This allows developers to test the code in AWS without it being "live". Other than sed replacing part of the template.yaml before I run "sam package/deploy" is there some other way to accomplish this?
--UPDATE--
It looks like I can use Parameters to create environments in my lambda, but I don't know how to toggle between them. This would look like:
Parameters:
MyEnv:
Description: Environment of this stack of resources
Type: String
Default: testing
AllowedValues:
- testing
- prod
Then I can reference this w/:
Environment:
Variables:
ENV: !Ref: MyEnv
If someone knows how to toggle this parameter at runtime that solves my problem.
I got this working. My template.yaml:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
sams-app
Globals:
Function:
Timeout: 3
Parameters:
Stage:
Type: String
Description: Which stage the code is in
Default: test
AllowedValues:
- test
- prod
Resources:
HelloWorldSQSFunction:
Type: AWS::Serverless::Function
Properties:
Role: arn:aws:iam::xxxxxxxxxxxx:role/service_lambda_default1
CodeUri: hello_world/
Handler: app.lambda_handler
Runtime: python3.7
AutoPublishAlias: !Ref Stage
DeploymentPreference:
Type: AllAtOnce
Environment:
Variables:
STAGE: !Ref Stage
Outputs:
HelloWorldSQSFunction:
Description: "Hello World SQS Lambda Function ARN"
Value: !GetAtt HelloWorldSQSFunction.Arn
My lambda code:
import json
import os
def lambda_handler(event, context):
stage = os.environ['STAGE']
print(f"My stage is: {stage}")
return {
"statusCode": 200,
}
And to run it locally (I'm using Cloud9):
DEVELOPER:~/environment/sams-app $ sam local invoke --parameter-overrides Stage=prod
Invoking app.lambda_handler (python3.7)
Fetching lambci/lambda:python3.7 Docker container image......
Mounting /home/ec2-user/environment/sams-app/hello_world as /var/task:ro,delegated inside runtime container
START RequestId: 85da81b1-ef74-1b7d-6ad0-a356f4aa8b76 Version: $LATEST
My stage is: prod
END RequestId: 85da81b1-ef74-1b7d-6ad0-a356f4aa8b76
REPORT RequestId: 85da81b1-ef74-1b7d-6ad0-a356f4aa8b76 Init Duration: 127.56 ms Duration: 3.69 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 22 MB
{"statusCode":200}
One thing to note is that this will cause your "sam validate" to fail. For info on that, see: https://github.com/awslabs/serverless-application-model/issues/778
Special thanks to JLarky for the comment on this thread: aws-sam-local environment variables

How to show lambda function return values in Outputs of Cloudformation

I am looking for an option to list results of AWS::Lambda::Function in Outputs of Cloudformation.
Following is the snippet of cloudformation template for AWS:Lambda::Function
Resources:
AthenaLambdaFunction:
Type: 'AWS::Lambda::Function'
DeletionPolicy: Delete
DependsOn:
- IamRoleLambdaForAthena
Properties:
Code:
ZipFile: |
import boto3
import botocore
import os
ath = boto3.client('athena')
def handler(event, context):
outputBucket = os.environ.get("outputBucket")
QSTRING = 'select * from tableName limit 10'
response = ath.start_query_execution(QueryString=str(QSTRING), ResultConfiguration={'OutputLocation': outputBucket})
s3BucketOut = output_bucket + response['ResponseMetadata']['RequestId']
return s3BucketOut
Handler: index.handler
Runtime: python3.6
MemorySize: 128
Role: !GetAtt IamRoleLambdaForAthena.Arn
Timeout: 30
Environment:
Variables:
outputBucket: !Ref OutputS3Bucket
I want to show the value retuned by lambda function s3BucketOut in Outputs of Cloudformation. Something like below (off course, the code below doesn't work).
Outputs:
LambdaFunctionOutput:
Value: !Ref AthenaLambdaFunction.s3BucketOut
Description: Return Value of Lambda Function
Any suggestions please. TIA
You are half-way through it. With your code, you created the AWS Lambda function that you want to run. Now you need to make this function run on CloudFormation and capture its value. Note that you need to make small changes on your code to allow the value to be captured by CloudFormation.
The full code will be similar to this:
Resources:
AthenaLambdaFunction:
Type: 'AWS::Lambda::Function'
DeletionPolicy: Delete
DependsOn:
- IamRoleLambdaForAthena
Properties:
Code:
ZipFile: |
import boto3
import botocore
import os
import cfnresponse # this needs to be imported for replying to CloudFormation
ath = boto3.client('athena')
def handler(event, context):
outputBucket = os.environ.get("outputBucket")
QSTRING = 'select * from tableName limit 10'
response = ath.start_query_execution(QueryString=str(QSTRING), ResultConfiguration={'OutputLocation': outputBucket})
s3BucketOut = output_bucket + response['ResponseMetadata']['RequestId']
responseData = {} # added
responseData['S3BucketOut'] = s3BucketOut # added
cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData) # return modified
Handler: index.handler
Runtime: python3.6
MemorySize: 128
Role: !GetAtt IamRoleLambdaForAthena.Arn
Timeout: 30
Environment:
Variables:
outputBucket: !Ref OutputS3Bucket
S3BucketOutInvocation:
Type: Custom::S3BucketOut
Properties:
ServiceToken: !GetAtt AthenaLambdaFunction.Arn
Region: !Ref "AWS::Region"
Outputs:
LambdaFunctionOutput:
Value: !GetAtt S3BucketOutInvocation.S3BucketOut
Description: Return Value of Lambda Function
References:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-code.html
What you can do is create what is called a "Lambda-backed Custom Resource" You would use this during Stack creation to get bits of information at creation type.
Further information can be found here
AWS Lambda-backed Custom Resources

Auto-assign IPv6 address via AWS and CloudFormation

Is there any way to have IPv6 addresses auto-assigned to EC2 instances within an autoscaling group+launch configuration?
VPC and subnets are all set up for IPv6. Manually created instances are ok.
I can also manually assign them, but I can't seem to find a way to do it in CloudFormation.
The current status is that CloudFormation support for IPv6 is workable. Not fun or complete, but you can build a stack with it - I had to use 2 custom resources:
The first is a generic resource that I use for other things and also reused here, to work around the missing feature to construct a subnet /64 CIDR block from a VPC's /56 auto-provided network
The other I had to add specifically to work around a bug in the EC2 API that CloudFormation uses correctly.
Here is my setup:
1. Add IPv6 CIDR block to your VPC:
VPCipv6:
Type: "AWS::EC2::VPCCidrBlock"
Properties:
VpcId: !Ref VPC
AmazonProvidedIpv6CidrBlock: true
2. Extract the network prefix for creating /64 subnets:
As explained in this answer.
VPCipv6Prefix:
Type: Custom::Variable
Properties:
ServiceToken: !GetAtt [ IdentityFunc, Arn ]
Value: !Select [ 0, !Split [ "00::/", !Select [ 0, !GetAtt VPC.Ipv6CidrBlocks ] ] ]
IdentityFunc is an "identity function" implemented in Lambda for "custom variables", as described in this answer. Unlike this linked answer, I implement the function directly in the same stack so it is easier to maintain. See here for the gist.
3. Add an IPv6 default route to your internet gateway:
RouteInternet6:
Type: "AWS::EC2::Route"
Properties:
RouteTableId: !Ref RouteTableMain
DestinationIpv6CidrBlock: "::/0"
GatewayId: !Ref IGWPublicNet
DependsOn:
- IGWNetAttachment
IGWNetAttachment is a reference to the AWS::EC2::VPCGatewayAttachment defined in the stack. If you don't wait for it, the route may fail to be set properly
4. Add an IPv6 CIDR block to your subnets:
SubnetA:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: !Select [ 0, !GetAZs { Ref: "AWS::Region" } ]
CidrBlock: 172.20.0.0/24
MapPublicIpOnLaunch: true
# The following does not work if MapPublicIpOnLaunch because of EC2 bug
## AssignIpv6AddressOnCreation: true
Ipv6CidrBlock: !Sub "${VPCipv6Prefix.Value}00::/64"
VpcId:
Ref: VPC
Regarding the AssignIpv6AddressOnCreation being commented out - this is normally what you want to do, but apparently, there's a bug in the EC2 API that prevents this from working - through no fault of CloudFormation. This is documented in this AWS forums thread, as well as the solution which I'll present here next.
5. Fix the AssignIpv6AddressOnCreation problem with another lambda:
This is the lambda setup:
IPv6WorkaroundRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: !Sub "ipv6-fix-logs-${AWS::StackName}"
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Resource: arn:aws:logs:*:*:*
- PolicyName: !Sub "ipv6-fix-modify-${AWS::StackName}"
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- ec2:ModifySubnetAttribute
Resource: "*"
IPv6WorkaroundLambda:
Type: AWS::Lambda::Function
Properties:
Handler: "index.lambda_handler"
Code: #import cfnresponse below required to send respose back to CFN
ZipFile:
Fn::Sub: |
import cfnresponse
import boto3
def lambda_handler(event, context):
if event['RequestType'] is 'Delete':
cfnresponse.send(event, context, cfnresponse.SUCCESS)
return
responseValue = event['ResourceProperties']['SubnetId']
ec2 = boto3.client('ec2', region_name='${AWS::Region}')
ec2.modify_subnet_attribute(AssignIpv6AddressOnCreation={
'Value': True
},
SubnetId=responseValue)
responseData = {}
responseData['SubnetId'] = responseValue
cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, "CustomResourcePhysicalID")
Runtime: python2.7
Role: !GetAtt IPv6WorkaroundRole.Arn
Timeout: 30
And this is how you use it:
IPv6WorkaroundSubnetA:
Type: Custom::SubnetModify
Properties:
ServiceToken: !GetAtt IPv6WorkaroundLambda.Arn
SubnetId: !Ref SubnetA
This call races with the autoscaling group to complete the setup, but it is very unlikely to lose - I ran this a few dozen times and it never had a problem to set the field correctly before the first instance boots.
I had a very similar issue and had a chat with AWS Support concerning this. The current state is that IPv6 support in CloudFormation is very limited.
We ended up creating Custom Resources for lots of IPv6-specific things. We have a Custom Resource that:
Enables IPv6-allocation on a subnet
Creates an Egress-Only Internet Gateway
Adds a route to the Egress-Only Internet Gateway (the built-in Route resource says it "fails to stabilize" when pointing to an EIGW)
The Custom Resources are just Lambda functions that do the "raw" API call, and a IAM Role that grants the Lambda enough permissions to do that API call.

Resources