I had successfully deployed an AWS Lambda function to receive Image Scan events from AWS ECR. The region I was using was ap-southeast-1. However, I had noticed that the Lambda function could not receive events from AWS ECR from another region (i.e eu-central-1).
Is there a way to make my Lambda to receive event from AWS ECR in another region without having to deploy it in multiple regions?
Thanks!
Genzer
This depends on how ECR sends events to EventBridge. I'm not certain but most AWS services send events within the same region only. So eu-central-1 events are in eu-central-1 only. The easiest workaround would be to deploy the same function in all regions.
You can also leverage API-Gateway's multi-region abilities. This blog shows a slightly different use-case but may be helpful in understanding how to call a cross-region lambda https://aws.amazon.com/blogs/compute/building-a-multi-region-serverless-application-with-amazon-api-gateway-and-aws-lambda/
In all cases, you need to create a rule in each region you need to reach ECR events and send them to a same-region target.
My understanding of EventBridge events is similar to #blr's, but I've gotten around this by having the events go to a queue first, which the lambda can then be subscribed to across regions. It seems to be a bit lower overhead than deploying the lambda function in every region.
Related
I have a lambda that is triggered whenever an event is dropped in the eventbus to which my lambda is connected and is triggered automatically. How can I performance test it to test how it performs of 500 events are dropped at a time?
Also I know aws has some inbuilt metrics like lambda execution time, xray tracing etc. Can anyone let me know how to use them for my use case?
If by "eventbus" you mean AWS EventBus which is a part of Amazon EventBrigde my expectation is that the easiest would be using PutEvents API endpoint, you can come up with a JSON payload having 500 events or make 500 separate calls with 1 event or any combination you can think of.
Be aware that the AWS API requests need to be signed to the load testing tool you choose must have the possibility to calculate this signature. A guide for Apache JMeter: How to Handle Dynamic AWS SigV4 in JMeter
With regards to metrics - check out AWS CloudWatch
My use case is the following. I have 5 lambdas. They need to talk to each other. I've heard that it can be done with SNS but also SNS and SQS. What is the difference, why not call lambdas only from one another directly?
It's possible to design durable and scalable applications using SNS-SQS AWS pattern. You can do this by having an SNS topic to which lambda A posts then the SNS triggers directly SQS which is a queue. In that way if you have high volume messages they will be processed sequentially.
Take care that the SNS and SQS can trigger more than once.
For more info check the article here:
https://aws.amazon.com/blogs/compute/designing-durable-serverless-apps-with-dlqs-for-amazon-sns-amazon-sqs-aws-lambda/
You can also use AWS Step Function which is a serverless function orchestrator that makes it easy to sequence AWS Lambda functions and multiple AWS services.
You can check out getting started guide here - https://docs.aws.amazon.com/step-functions/latest/dg/getting-started.html
Example use case
Send the user a notification 2 hours after signup.
Options considered
setTimeout(() => { /* send notification */ }, 2*60*60*1000); is not an option in serverless environments since the function terminates after execution (so it has to be stateless).
CloudWatch events can schedule lambda invocations using cron expressions - but this was designed for repetitive invocations (there's a limit of 100 rules/region).
I have not seen scheduling options in AWS SNS/SQS or GCP Pub/Sub. Are there alternatives with scheduling?
I want to avoid (if possible) setting up a dedicated message broker (overkill) or stateful/non-serverless instance - is there a serverless way to do this?
I can queue the events in a database and invoke a lambda function every minute to poll the database for events to execute in that minute... is there a more elegant solution?
Use AWS Step functions, they are like serverless functions that don't have the 15 minute limit like AWS Lambda does. You can design a workflow in AWS step that integrates with API Gateway, Lambda and SNS to send email and text notifications as follows:
Create a REST API via API gateway that will invoke a Lambda function passing in for example, the destination address (email, phone #) of the SNS notification, when it should be sent, notification method (e.g. email, text, etc.).
The Lambda function on invocation will invoke the Step function passing in the data (Lambda is needed because API Gateway currently can't invoke Step functions directly).
The Step function is basically a workflow, you can define states for waiting (like waiting for the specified time to send the notification e.g. 30 seconds), and states for invoking other Lambda functions that can use SNS to send out an email and/or text notifications.
A rudimentary example is provided by AWS w/ their Task Timer example.
Things are coming on GCP for doing this, but not very soon. Thereby, today, the solution is to poll a database.
You can to that with Datastore/firestore with the execution datetime indexed (to prevent to read all the documents each minute). But be careful of traffic spike, you could create hotspot.
You can use Cloud Scheduler on Google Cloud Platform. As is is stated in the official documentation :
Cloud Scheduler is a fully managed enterprise-grade cron job scheduler. It allows you to schedule virtually any job, including batch, big data jobs, cloud infrastructure operations, and more. You can automate everything, including retries in case of failure to reduce manual toil and intervention. Cloud Scheduler even acts as a single pane of glass, allowing you to manage all your automation tasks from one place.
Here you can check a quickstart for using it with Pub/Sub and Cloud Functions.
My desired flow is:
ask my iot device to do something using AVS sdk
aws lambda triggered and update the device shadow
iot device triggered based on the shadow topic: delta and do something locally. Publish the status to shadow when done doing something
aws lambda sends voice feedback to my iot device to tell users the reported status
I am stuck in point 4 since I dont know how to trigger AVS(ASK)'s speech response only after the topic is updated "within the same lambda triggered by AVS (as mentioned in point 1 and 2).
You don't want lambda to wait.
I heard from a wise man onetime,
A long lived Lambda = EC2 Instance
Either create an iot rule to trigger a lambda on specific topics,
or,
create an api endpoint to update the topic and trigger it from the client.
I have an architecture which looks like as follows:-
Multiple SNS -> (AWS Lambda or SQS with Poller)??? -> Dynamo Db
So, basically multiple SNS have subscribed to AWS Lambda or SQS with Poller and that thing pushes data to Dynamo Db.
But this ? thing do lot of transformation of message in between. So, now for such case, I can either use AWS Lambda or SQS with Poller. With AWS Lambda, I can do transformation in Lambda function and with SQS with Poller, I can do transformation in Poller. With AWS Lambda, I see one problem that code would become quite large as transformation is quite complex(has lot of rules), so I am thinking to use SQS. But before finalising on SQS, I wanted to know of the drawbacks of SQS which AWS Lambda removes?
Please help. Let me know if you need further information.
Your question does not contain much detail, so I shall attempt to interpret your needs.
Option 1: SQS Polling
Information is sent to an Amazon SNS topic
An SQS queue is subscribed to the SNS topic
An application running on Amazon EC2 instance(s) regularly poll the SQS queue to ask for a message
If a message is available, the data in the message is transformed and saved to an Amazon DynamoDB table
This approach is good if the transformation takes a long time to process. The number of EC2 instances can be scaled based upon the amount of work in the queue. Multiple messages can be received at the same time. It is a traditional message-based approach.
Option 2: Using Lambda
Information is sent to an Amazon SNS topic
An AWS Lambda function is subscribed to the SNS topic
A Lambda function is invoked when a message is sent to the SNS topic
The Lambda function transforms the data in the message and saves it to an Amazon DynamoDB table
AWS Lambda functions are limited to five minutes of execution time, so this approach will only work if the transformation process can be completed within that timeframe.
No servers are required because Lambda will automatically run multiple functions in parallel. When no work is to be performed, no Lambda functions execute and there is no compute charge.
Between the two options, using AWS Lambda is much more efficient and scalable but it might vary depending upon your specific workload.
We can now use SQS messages to trigger AWS Lambda Functions.
28 JUN 2018: AWS Lambda Adds Amazon Simple Queue Service to Supported
Event Sources
Moreover, no longer required to run a message polling service or create an SQS to SNS mapping.
AWS Serverless Model supports a new event source as following:
Type: SQS
PropertiesProperties:
QueueQueue: arn:aws:sqs:us-west-2:213455678901:test-queue arn:aws:sqs:us-west-2:123791293
BatchSize: 10
AWS Console also support:
Further details:
https://aws.amazon.com/blogs/aws/aws-lambda-adds-amazon-simple-queue-service-to-supported-event-sources/
https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html