I am attempting to create an Eventbridge that will get notifications from Datadog, and trigger a lambda function to store the notifications to an S3 bucket. This is all going to be done through Terraform.
The following is the code I have written:
###################################
# Eventbridge Integration to Lambda
###################################
data "aws_cloudwatch_event_source" "datadog_event_source" {
name_prefix = var.spog_event_bus_name # aws.partner/datadog.com/my_eventbus
}
resource "aws_cloudwatch_event_bus" "datadog_event_bus" {
name = data.aws_cloudwatch_event_source.datadog_event_source.name
event_source_name = data.aws_cloudwatch_event_source.datadog_event_source.name
}
resource "aws_cloudwatch_event_rule" "spog_cloudwatch_rule" {
name = "spog_cloudwatch_rule"
event_bus_name = aws_cloudwatch_event_bus.datadog_event_bus.name
event_pattern = <<EOF
{
"account": [
"${var.aws_account_id}"
]
}
EOF
}
resource "aws_cloudwatch_event_target" "spog_cloudwatch_event_target" {
rule = aws_cloudwatch_event_rule.spog_cloudwatch_rule.name
target_id = aws_lambda_function.write_datadog_events.function_name
arn = aws_lambda_function.write_datadog_events.arn
}
resource "aws_lambda_permission" "spog_allow_cloudwatch" {
statement_id = "AllowExecutionFromCloudWatch"
action = "lambda:InvokeFunction"
function_name = aws_lambda_function.write_datadog_events.function_name
principal = "events.amazonaws.com"
source_arn = aws_cloudwatch_event_rule.spog_cloudwatch_rule.arn
}
Here, the write_datadog_events is the lambda function to store the notifications.
The build succeeds, but when I try to apply the plan, I get an error saying that "validationException: EventBus name starting with 'aws.' is not valid". From inspecting the aws console, it seems that the actual eventbridge and rule are created successfully, but the event bus name is not registered properly on the eventbridge. The event bus name only says default, and I thought by changing the event_bus_name value within aws_cloudwatch_event_rule, it would not be default, but I was wrong.
Can anyone help me out with this. The lambda function itself is not wrong (since I ran a test case on it), it seems that the core issue is eventbridge not registering my event bus name. Also, although the rule is generated, the event bus is never generated.
Thanks for your help.
I am consistently missing lambda triggers from a dynamodb stream. I am mapping the dynamodb stream to the lambda function with event_source_mapping resource.
resource "aws_lambda_event_source_mapping" "history" {
event_source_arn = "${data.aws_dynamodb_table.auditlogDynamoDB.stream_arn}"
function_name = "${module.cx-clientcomm-history-lambda.lambda_function_arn}"
starting_position = "LATEST"
maximum_retry_attempts = 0
maximum_record_age_in_seconds = 604800 // adding to explicitly set. QA plan was providing invalid value
#(Optional) An Amazon SQS queue or Amazon SNS topic destination for failed records.
destination_config {
on_failure {
destination_arn = "${data.aws_sns_topic.cx-clientcomm-sns-slack.arn}"
}
}
}
It appears when there are entries in dynamodb are coming it very close to each other, some are not picked up by the event_source_mapping and not sent to the lambda.
Is there a way to make sure this doesn't happen? This needs to be very reliable
I'm working on a small serverless offline assignment and I got error Unsupported Media Type when tried to invoke one lambda function in another.
I found a solution but when I tried to applied to my project was not working:
here in the link all the details. cloud anyone help me with that
https://github.com/dherault/serverless-offline/issues/1005#issue-632401297
there are three possible solutions.
Make sure that the lambda_A have the same port and host where the lambda_B is running.
Lambda_A:
const { Lambda } = require('aws-sdk');
const lambda = new Lambda({
region: 'us-east-1',
endpoint: 'http://localhost:3000',
});
module.exports.main = async (event, context) => {
// invoke
}
Lambda_B: Is running on http://localhost:3000
You have configured out serverless-offline in twice functions.
https://www.serverless.com/plugins/serverless-offline#usage-with-invoke
Lambda_A or Lambda_B have correctly stage?. Remember to use sls offline --stage local in both functions.
I'm trying to update a cloud formation template with a few lambda functions in it. The last version of the template was deployed a few years ago, and all the lambda functions currently have a runtime of node6.10.
I have updated the runtime for all functions to node10.x, but when I deploy the template, I get the following message:
The runtime parameter of nodejs6.10 is no longer supported for creating or updating AWS Lambda functions
I've created a change set, and reviewed it, and it includes an update to the runtime property for each lambda function, however Cloud Formation seems to be ignoring it.
Is there something I'm missing?
Context:
I assume that you encountered this issue - (you got "nodejs version not supported error" message after you tried to amplify push followed by amplify add auth)
Go to amplify -> backend -> auth -> cognito -> click cognito cloudformation ->
search for "Runtime: node"
change it to "Runtime: nodejs8.10" - whatever latest recommended in error message
re-run
$ amplify push
Unfortunately, I found I had to update the runtime of all functions in a template outside of Cloud Formation, to get the stacks to deploy. I used this script:
const AWS = require('aws-sdk')
const lambda = new AWS.Lambda(...)
main().catch(err => {
console.error(err)
process.exit(1)
})
async function main() {
const functions = await getFunctions()
await Promise.all(
functions
// filter only functions you want to update
.filter(...)
.filter(x => x.Runtime !== 'nodejs10.x')
.map(updateFunction)
)
}
async function updateFunction(func) {
await lambda
.updateFunctionConfiguration({
FunctionName: func.FunctionName,
Runtime: 'nodejs10.x'
})
.promise()
console.log(`function updated: ${func.FunctionName}`)
}
async function getFunctions() {
let marker
let functions = []
do {
const result = await lambda
.listFunctions({
Marker: marker
})
.promise()
functions = [...functions, ...result.Functions]
marker = result.NextMarker
} while (marker)
return functions
}
I tried creating a set of lambdas using cloudformation. I want the lambdas to get triggered once they are created. I saw at various blogs to create a trigger to s3 or sns but none seems to be a option to trigger lambda once it has been created. Any options?
Yes, it is possible. Here are a few options:
Manually create an SNS Topic. Add an AWS::SNS::Subscription to your stack with the lambda function as the Endpoint and the SNS topic as the TopicArn. On stack creation/update, configure Stack Event Notifications to be sent to this SNS topic.
(See Setting AWS CloudFormation Stack Options for documentation on how to do this when using the AWS Console to create your stack, or use the equivalent option like --notification-arns if creating/updating your stack using the AWS CLI or other AWS SDK.)
Add a Custom Resource referencing a Lambda function to be called on creation.
If you need the Lambda function to be called after some specific Resource is created, add a DependsOn attribute on the Custom Resource referencing the Resource you want to make sure is created first before the function is called.
In order for the Custom Resource to create successfully (and not cause a failure/rollback in your stack), you will need to adapt your Lambda function to support the CloudFormation request/response format (see Custom Resource Reference).
This option will call the Lambda function while the stack status is still CREATE_IN_PROGRESS, because the Custom Resource is part of the stack itself.
The Lambda function will also be called again when the stack (and associated Custom Resource) is deleted. This will need to be handled by your Lambda function correctly, or your stack could get stuck in the DELETE_FAILED state.
Add the Lambda function reference to a Stack Output, then write a simple script that performs the stack creation and then manually invokes the Lambda function afterwards.
by yl.
The following just works great !
It invokes a lambda as a part of deployment:
LambdaFunction2:
Type: AWS::Lambda::Function
Properties:
FunctionName: caller
Code:
ZipFile: |
import boto3, json
import cfnresponse
def handler(event, context):
print('EVENT:[{}]'.format(event))
lambda_client = boto3.client('lambda')
test_event = '{"name":"test1"}'
lambda_client.invoke(
FunctionName='target1',
InvocationType='Event',
Payload=test_event,
)
responseValue = 120
responseData = {}
responseData['Data'] = responseValue
cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData)
Handler: index.handler
Role:
arn:aws:iam::11111111111:role/mylambda-role
Runtime: python3.7
Timeout: 60
Primerinvoke:
Type: AWS::CloudFormation::CustomResource
DependsOn: LambdaFunction2
Version: "1.0"
Properties:
ServiceToken: !GetAtt LambdaFunction2.Arn
For who looking the similar workaround.
CloudWatch is able to capture API calls of CloudFormation, which is "CreateStack", "UpdateStack" and "DeleteStack", stack states like "Create_complete" or "Complete_Rollback" are uncapturable, which means such state changes not be able to trigger lambda.
The workaround is SNS, stacks are able to send notifications to SNS (In advance settings when you creating stack) and SNS can choose to trigger lambda, however, you can't choose for specific states. So, lambda function takes the job to find out what state in "Message" content of an event. Everyone, just coding.
I know this is a bit old- but a solution could also be too use CommandRunner as a resource type in your template.
https://aws.amazon.com/blogs/mt/running-bash-commands-in-aws-cloudformation-templates/.
You can run virtually any shell command. Add a DependsOn attribute to your CommandRunner type and run a shell script:
aws lambda invoke --function-name my-function --invocation-type RequestRespone --payload '{ "name": "Bob" }'
Improving on Kyr's answer, because it lacks two important things:
how to pass paramaters to the Lambda you invoke
how to treat UPDATE and DELETE on your Stack (his solution would cause CloudFormation to crash on delete)
Here is the revised and improved code:
LambdaInvoker:
DependsOn: ## important, add stuff here you need to existe BEFORE the lambda is called
Type: AWS::Lambda::Function
Properties:
FunctionName: YourLambdaName
Description: 'Lambda invoke wrapper for Custom CFN actions'
Code:
ZipFile: !Sub |
import boto3, json
import cfnresponse
def handler(event, context):
print('EVENT:')
print(event)
if event['RequestType'] == "Create":
lambda_client = boto3.client('lambda')
cfn_event = {
"param1" : "${Param1}",
"param2" : "${Param2}"
}
lambda_client.invoke(
FunctionName='scm-custom-cfn-actions',
InvocationType='Event',
Payload=json.dumps(cfn_event)
)
responseValue = 120
responseData = {}
responseData['Data'] = responseValue
cfnresponse.send(event, context, cfnresponse.SUCCESS,
responseData, 'scm-cfn-customresource-id')
Handler: index.handler
Role: YourLambdaRoleARN
Runtime: python3.7
Timeout: 5
You have the option to notify to a SNS topic, and you may build a lambda that listens to the topic, so the workflow would be: Cloudformation launch -> SNS Topic -> Lambda.
The following template should invoke the lambda :
"InvokeLambda" : {
"Type": "Custom::InvokeLambda",
"Version" : "1.0",
"Properties" : {
"ServiceToken": {
"Fn::GetAtt": ["InitFunction","Arn"]
}
}
},