how to refer sns arn from terraform code in a lambda python py file? - aws-lambda

my lambda python uses SNS topic arn. But this sns arn id is generated from terraform code. Is there way to refer it somehow in python lambda code?
def lambda_handler(event, context):
try:
#some code
publish_vote(vote, voter)
except:
#some code
return {'statusCode': 200, 'body': '{"status": "success"}'}
def publish_vote(vote, voter):
sns = boto3.client('sns', region_name='us-east-1')
sns.publish(
TopicArn='arn:aws:sns:us-east-1:025416187662:erjan',
Message='""',
MessageAttributes={
"vote": {
"DataType": "String",
"StringValue": vote,
},
"voter": {
"DataType": "String",
"StringValue": voter,
}
}
)
SNS terraform code:
resource "aws_sns_topic" "vote_sns" {
name = "erjan-sns"
}
resource "aws_sns_topic_policy" "vote_sns_access_policy" {
arn = aws_sns_topic.vote_sns.arn
policy = data.aws_iam_policy_document.vote_sns_access_policy.json
}
data "aws_iam_policy_document" "vote_sns_access_policy" {
policy_id = "__default_policy_ID"
statement {
#some stuff code
}
}
output "sns_arn_erjan" {
value = aws_sns_topic.vote_sns.arn
description = "aws full sns topic"
}

For your information:
I see you have already solved this problem, but I have one suggestion.
The lambda function can refer to the topic ARN by putting the ARN as a parameter into Parameter Store with Terraform.
resource "aws_ssm_parameter" "vote_sns" {
name = "sns_arn_erjan"
type = "String"
value = aws_sns_topic.vote_sns.arn
}
aws_ssm_parameter | Resources | hashicorp/aws | Terraform Registry
The lambda function can refer to the parameter stored in Parameter Store using boto3.
get_parameter - SSM — Boto3 Docs 1.26.54 documentation

Your terraform code does not have code for creating the lambda function itself. Are you creating it manually? If yes, then first create that as well using terraform. A basic example is mentioned here
Within the definition, there is an argument for environment. Use that to define your env variables as:
environment {
variables = {
SNS_ARN = aws_sns_topic.vote_sns.arn # Arn from the defined sns resource.
}
}
Then refer the same in your python code as:
import os
SNS_ARN = os.environ.get("SNS_ARN")
...
Alternatively, you could also consider using AWS SAM

Related

How to structure terraform code to get Lambda ARN after creation?

This was a previous question I asked: How to get AWS Lambda ARN using Terraform?
This question was answered but turns out didn't actually solve my problem so this is a follow up.
The terraform code I have written provisions a Lambda function:
Root module:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
}
}
}
provider "aws" {
region = var.region
profile = var.aws_profile
}
module "aws_lambda_function" {
source = "./modules/lambda_function"
}
Child module:
resource "aws_lambda_function" "lambda_function" {
function_name = "lambda_function"
handler = "lambda_function.lambda_handler"
runtime = "python3.8"
filename = "./task/dist/package.zip"
role = aws_iam_role.lambda_exec.arn
}
resource "aws_iam_role" "lambda_exec" {
name = "aws_iam_lambda"
assume_role_policy = file("policy.json")
}
What I want the user to be able to do to get the Lambda ARN:
terraform output
The problem: I cannot seem to include the following code anywhere in my terraform code as it causes a "ResourceNotFOundException: Function not found..." error.
data "aws_lambda_function" "existing" {
function_name = "lambda_function"
}
output "arn" {
value = data.aws_lambda_function.existing.arn
}
Where or how do I need to include this to be able to get the ARN or is this possible?
You can't lookup the data for a resource you are creating at the same time. You need to output the ARN from the module, and then output it again from the main terraform template.
In your Lambda module:
output "arn" {
value = aws_lambda_function.lambda_function.arn
}
Then in your main file:
output "arn" {
value = module.aws_lambda_function.arn
}

How to get AWS Lambda ARN using Terraform?

I am trying to define a terraform output block that returns the ARN of a Lambda function. The Lambda is defined in a sub-module. According to the documentation it seems like the lambda should just have an ARN attribute already: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/lambda_function#arn
Using that as a source I thought I should be able to do the following:
output "lambda_arn" {
value = module.aws_lambda_function.arn
}
This generates the following error:
Error: Unsupported attribute
on main.tf line 19, in output "lambda_arn":
19: value = module.aws_lambda_function.arn
This object does not have an attribute named "arn".
I would appreciate any input, thanks.
Documentation is correct. Data source data.aws_lambda_function has arn attribute. However, you are trying to access the arn from a custom module module.aws_lambda_function. To do this you have to define output arn in your module.
So in your module you should have something like this:
data "aws_lambda_function" "existing" {
function_name = "function-to-get"
}
output "arn" {
value = data.aws_lambda_function.existing.arn
}
Then if you have your module called aws_lambda_function:
module "aws_lambda_function" {
source = "path-to-module"
}
you will be able to access the arn:
module.aws_lambda_function.arn

Lambda chaining - Invoke lambda from another lambda using terraform

I am trying to invoke one AWS lambda from another and perform lambda chaining. The rationale behind doing this is AWS does not provide multiple trigger from same S3 bucket.
I have created one lambda, with an s3 trigger. The java code of first lambda will listen to S3 event and contains the invocation of another lambda. The second lambda will be invoked from first lambda. Both the lambda creation is done by terraform.
Lambda A has S3 trigger. This will be invoked on S3 event on a particular bucket. Lambda A will do the processing and will invoke Lambda B using invoke request. Lambda B invocation from Lambda A code in java is :
public class EventHandler implements RequestHandler<S3Event, String> {
#Override
public String handleRequest(S3Event event, Context context) throws RuntimeException {
InvokeRequest req = new InvokeRequest()
.withFunctionName("LambdaFunctionB")
.withPayload(json);
return "Lambda B invoked"
}
}
Both the lambdas are created using terraform. Scripts below:
Lambda A terraform:
module "lambda_function" {
source = "Git Path"
absolute_artifact_path = "../lambda.jar"
lambda_function_name = "LambdaFunctionA"
lambda_function_description = ""
lambda_function_runtime = "java8"
lambda_handler_name = "EventHandler"
lambda_execution_role_name = "lambda-iam-role"
lambda_memory = "512"
dead_letter_target_arn = "error-handling-arn"
}
resource "aws_lambda_permission" "allow_bucket" {
statement_id = "statementId"
action = "lambda:InvokeFunction"
function_name = "${module.lambda_function.lambda_arn}"
principal = "s3.amazonaws.com"
source_arn = "s3.bucket.arn"
}
resource "aws_s3_bucket_notification" "bucket_notification" {
bucket = "bucketName"
lambda_function {
lambda_function_arn = "${module.lambda_function.lambda_arn}"
events = ["s3:ObjectCreated:*"]
filter_prefix = "path/subPath"
}
}
Lambda B terraform:
module "lambda_function" {
source = "git path"
absolute_artifact_path = "../lambda.jar"
lambda_function_name = "LambdaFunctionB"
lambda_function_description = ""
lambda_function_runtime = "java8"
lambda_handler_name = "LambdaBEventHandler"
lambda_execution_role_name = "lambda-iam-role"
lambda_memory = "512"
dead_letter_target_arn = "error-handling-arn"
}
resource "aws_lambda_permission" "allow_lambda" {
statement_id = "AllowExecutionFromLambda"
action = "lambda:InvokeFunction"
function_name = "${module.lambda_function.lambda_arn}"
principal = "s3.amazonaws.com"
source_arn = "arn:aws:lambda:eu-west-1:xxxxxxxxxx:function:LambdaFunctionA"
}
lambda-iam-role has below policies attached
AmazonS3FullAccess
AWSLambdaBasicExecutionRole
AWSLambdaVPCAccessExecutionRole
AmazonSNSFullAccess
CloudWatchEventsFullAccess
Expectation was that Lambda A should successfully invoke Lambda B. But I am getting AccessDeniedException in Lambda A logs and it is not able to invoke Lambda B. Error is
com.amazonaws.services.lambda.model.AWSLambdaException: User: arn:aws:sts::xxxxxxxxx:assumed-role/lambda-iam-role/LambdaFunctionA is not authorized to perform: lambda:InvokeFunction on resource: arn:aws:lambda:eu-west-1:xxxxxxxxx:function:LambdaFunctionB (Service: AWSLambda; Status Code: 403; Error Code: AccessDeniedException; Request ID: f495ede3-b3cb-47a1-b884-16996545233d)
Hope this helps you, not exactly similar but its invoking one lambda from another lambda Github
I think the lambda needs this policy as well "lambda:InvokeFunction"
I found an answer online, using the aws-sdk.
var aws = require('aws-sdk');
var lambda = new aws.Lambda({
region: 'default'
});
lambda.invoke({
FunctionName: 'name_of_your_lambda_function',
Payload: JSON.stringify(event, null, 2) // pass params
}, function(error, data) {
if (error) {
context.done('error', error);
}
if(data.Payload){
context.succeed(data.Payload)
}
});
You can find the doc here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html
Hope it helps
:)

How to access Terraform Lambda Variables in Ruby Lambda Function?

I have terraform code which creates a Lambda function. I then have some ruby code that is the lambda function. I can not figure out, or find any information in how to actually use the variables which are being passed in from the terraform into the lambda. I ultimately just need to know how to use the terraform variables in the ruby lambda function
I have found examples in python and JS. There is little similarities.
Here is my terraform code
resource "aws_lambda_function" "send_sns_lambda" {
filename = "statuslambda.zip"
function_name = "status-page-send-sns"
source_code_hash = "${data.archive_file.status_lambdas.output_base64sha256}"
role = "${aws_iam_role.status_lambda.arn}"
handler = "statusLambda.send_sns"
runtime = "ruby2.5"
vpc_config = {
subnet_ids = ["subnet-xxxx", "subnet-xxxxx"]
security_group_ids = ["sg-xxxxxx"]
}
environment = {
variables = {
status = "Major Outage"
}
}
}
And my Lambda function
def send_sns(event:,context:)
sns = Aws::SNS::Resource.new(region: 'us-xxx-xxx')
topic = sns.topic('arn:aws:sns:us-east-1:xxxxxxxx')
topic.publish({
message: '#{status}'
})
end
The idea is that the status variable in terraform gets passed into the status variable in the ruby code
Here is the python example I have found
import os
def lambda_handler(event, context):
return "{} from Lambda!".format(os.environ['greeting'])
So your question is "how to access environment variables in Ruby"? That would be ENV['status'].

Is it possible to trigger a lambda on creation from CloudFormation template

I tried creating a set of lambdas using cloudformation. I want the lambdas to get triggered once they are created. I saw at various blogs to create a trigger to s3 or sns but none seems to be a option to trigger lambda once it has been created. Any options?
Yes, it is possible. Here are a few options:
Manually create an SNS Topic. Add an AWS::SNS::Subscription to your stack with the lambda function as the Endpoint and the SNS topic as the TopicArn. On stack creation/update, configure Stack Event Notifications to be sent to this SNS topic.
(See Setting AWS CloudFormation Stack Options for documentation on how to do this when using the AWS Console to create your stack, or use the equivalent option like --notification-arns if creating/updating your stack using the AWS CLI or other AWS SDK.)
Add a Custom Resource referencing a Lambda function to be called on creation.
If you need the Lambda function to be called after some specific Resource is created, add a DependsOn attribute on the Custom Resource referencing the Resource you want to make sure is created first before the function is called.
In order for the Custom Resource to create successfully (and not cause a failure/rollback in your stack), you will need to adapt your Lambda function to support the CloudFormation request/response format (see Custom Resource Reference).
This option will call the Lambda function while the stack status is still CREATE_IN_PROGRESS, because the Custom Resource is part of the stack itself.
The Lambda function will also be called again when the stack (and associated Custom Resource) is deleted. This will need to be handled by your Lambda function correctly, or your stack could get stuck in the DELETE_FAILED state.
Add the Lambda function reference to a Stack Output, then write a simple script that performs the stack creation and then manually invokes the Lambda function afterwards.
by yl.
The following just works great !
It invokes a lambda as a part of deployment:
LambdaFunction2:
Type: AWS::Lambda::Function
Properties:
FunctionName: caller
Code:
ZipFile: |
import boto3, json
import cfnresponse
def handler(event, context):
print('EVENT:[{}]'.format(event))
lambda_client = boto3.client('lambda')
test_event = '{"name":"test1"}'
lambda_client.invoke(
FunctionName='target1',
InvocationType='Event',
Payload=test_event,
)
responseValue = 120
responseData = {}
responseData['Data'] = responseValue
cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)
Handler: index.handler
Role:
arn:aws:iam::11111111111:role/mylambda-role
Runtime: python3.7
Timeout: 60
Primerinvoke:
Type: AWS::CloudFormation::CustomResource
DependsOn: LambdaFunction2
Version: "1.0"
Properties:
ServiceToken: !GetAtt LambdaFunction2.Arn
For who looking the similar workaround.
CloudWatch is able to capture API calls of CloudFormation, which is "CreateStack", "UpdateStack" and "DeleteStack", stack states like "Create_complete" or "Complete_Rollback" are uncapturable, which means such state changes not be able to trigger lambda.
The workaround is SNS, stacks are able to send notifications to SNS (In advance settings when you creating stack) and SNS can choose to trigger lambda, however, you can't choose for specific states. So, lambda function takes the job to find out what state in "Message" content of an event. Everyone, just coding.
I know this is a bit old- but a solution could also be too use CommandRunner as a resource type in your template.
https://aws.amazon.com/blogs/mt/running-bash-commands-in-aws-cloudformation-templates/.
You can run virtually any shell command. Add a DependsOn attribute to your CommandRunner type and run a shell script:
aws lambda invoke --function-name my-function --invocation-type RequestRespone --payload '{ "name": "Bob" }'
Improving on Kyr's answer, because it lacks two important things:
how to pass paramaters to the Lambda you invoke
how to treat UPDATE and DELETE on your Stack (his solution would cause CloudFormation to crash on delete)
Here is the revised and improved code:
LambdaInvoker:
DependsOn: ## important, add stuff here you need to existe BEFORE the lambda is called
Type: AWS::Lambda::Function
Properties:
FunctionName: YourLambdaName
Description: 'Lambda invoke wrapper for Custom CFN actions'
Code:
ZipFile: !Sub |
import boto3, json
import cfnresponse
def handler(event, context):
print('EVENT:')
print(event)
if event['RequestType'] == "Create":
lambda_client = boto3.client('lambda')
cfn_event = {
"param1" : "${Param1}",
"param2" : "${Param2}"
}
lambda_client.invoke(
FunctionName='scm-custom-cfn-actions',
InvocationType='Event',
Payload=json.dumps(cfn_event)
)
responseValue = 120
responseData = {}
responseData['Data'] = responseValue
cfnresponse.send(event, context, cfnresponse.SUCCESS,
responseData, 'scm-cfn-customresource-id')
Handler: index.handler
Role: YourLambdaRoleARN
Runtime: python3.7
Timeout: 5
You have the option to notify to a SNS topic, and you may build a lambda that listens to the topic, so the workflow would be: Cloudformation launch -> SNS Topic -> Lambda.
The following template should invoke the lambda :
"InvokeLambda" : {
"Type": "Custom::InvokeLambda",
"Version" : "1.0",
"Properties" : {
"ServiceToken": {
"Fn::GetAtt": ["InitFunction","Arn"]
}
}
},

Resources