I have an aws instances EC2. But lately it's stopped working for reasons I don't know about.
Does AWS have a way that if it shuts down, it automatically turns on the instance?.
Same way I have an instance of staging. Same thing happened to him.
Yes, follow this tutorial but make the cloudwatch event fire when an EC2 instance is stopped instead. https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/LogEC2InstanceState.html
One general strategy for reacting to events in AWS is:
Define a CloudWatch Event that "listens" for a certain event to happen
Define a Lambda function that gets triggered when the event happens
Write some code in the Lambda function that does something
In your case this would be
Your cloudwatch event listens for the "EC2 instance stopped" event
Your lambda function gets triggered, and cloudwatch passes in your stopped EC2 instance's details as a parameter
You write some lambda code that starts the EC2 instance
Related
I'm looking for a method to programmatically identify the triggers associated with an SQS queue. Looking through the SQS sdk docs, it doesn't seem this is possible. I had thought instead to try from the other end, and it appears the Lambda ListEventSourceMappings function would likely do what I want, since I'm able to provide it with the queue ARN. However, this requires the ListSourceMappings permission on all lambdas (*), which isn't really ideal - though it shouldn't really hurt, just not what I want. Is there another mechanism for this that I'm missing, or another approach?
Lambda polls SQS queues. It doesn't appear that way in the console, because they hide some of the details from you, but behind the scenes there is a process running within the AWS Lambda system that is polling your SQS queue and invoking your Lambda function when a message is available.
SQS doesn't push messages to Lambda (or anywhere else). SQS just holds messages and hands them out to anything that asks for them. So from an SQS perspective, there is no knowledge of who the message consumers are.
Given the above, the only way to find what you want is to use the Lambda ListEventSourceMappings API.
I have configured a lambda function as EventBridge rule target and I have configured a Dead Letter Queue on the EventBridge rule to capture exceptions.
Now, if the lambda function fails, Event Bridge does not recognize that failure as an error.
Since the EventBridge invocation to the Lambda is asynchronous, for EventBridge it is enough to reach the lambda to consider the event as successfull,but in this way I am not able to track and retry events once lambda fails.
Anyone know a way to make the EventBridge to the Lamdba request synchronous or another way to be able to Retry the events if the Lambda code fails after the invocation?
one option is to make the SQS to be rule target and use SQS event to trigger the lambda. when failure occurs, the lambda won't flag the event done so as to keep the event in the SQS. retry will auto happen after a configured period (SQS configuration). also you can configure dead letter queue after the retention time expires
EventBridge guarantees the delivery of the event to the lambda function but is not aware of what happens post that. It's lambda invocation vs lambda execution. Eventbridge successfully delivered the message to the lambda service, so it's a successful invocation.
For Lambda, EventBridge calls invokeAsync API. So, if this API sends a success response, EventBridge will assume the delivery was successful. Any failures within Lambda from async to sync is not visible to EventBridge. We should configure retries and create DLQs in our Lambda functions to make sure the events are not lost in case the Lambda function fails to execute. We could in fact configure the same DLQ used by the EventBridge to be used by the Lambda as well so that all eventual failures land in a single place.
AWS has a dedicated documentation page for this, which states the following for asynchronous invocation:
Lambda retries function errors twice. If the function doesn't have enough capacity to handle all incoming requests, events might wait in the queue for hours or days to be sent to the function. You can configure a dead-letter queue on the function to capture events that weren't successfully processed. For more information, see Asynchronous invocation.
So that means that as long as your Lambda functions handler function does return an error, the AWS Lambda service should retry to run your Lambda again.
Therefore, you might not need EventBridge to retry your event.
See: Error handling and automatic retries in AWS Lambda
I had successfully deployed an AWS Lambda function to receive Image Scan events from AWS ECR. The region I was using was ap-southeast-1. However, I had noticed that the Lambda function could not receive events from AWS ECR from another region (i.e eu-central-1).
Is there a way to make my Lambda to receive event from AWS ECR in another region without having to deploy it in multiple regions?
Thanks!
Genzer
This depends on how ECR sends events to EventBridge. I'm not certain but most AWS services send events within the same region only. So eu-central-1 events are in eu-central-1 only. The easiest workaround would be to deploy the same function in all regions.
You can also leverage API-Gateway's multi-region abilities. This blog shows a slightly different use-case but may be helpful in understanding how to call a cross-region lambda https://aws.amazon.com/blogs/compute/building-a-multi-region-serverless-application-with-amazon-api-gateway-and-aws-lambda/
In all cases, you need to create a rule in each region you need to reach ECR events and send them to a same-region target.
My understanding of EventBridge events is similar to #blr's, but I've gotten around this by having the events go to a queue first, which the lambda can then be subscribed to across regions. It seems to be a bit lower overhead than deploying the lambda function in every region.
How can I inform to my sqs polling that my ec2 in going to terminated in next two minutes ?
I have used sqs and autos calling group, my sqs simple queue pooling request for ce2. ec2 takes task request from sqs and excute.
problem scenario : as i used spot ece aws terminate ec2 to manage capacity. now i want to stop new task execution over ec2 once it got termination request from autos calling or from aws.
How can I deal with it?
Normally you would setup CloudWatch Event (CWE) rule:
{
"source": [
"aws.ec2"
],
"detail-type": [
"EC2 Spot Instance Interruption Warning"
]
}
The CWE rule would trigger a lambda function, and the function could perform a number of actions on your instance, depending on what you want to do. This is use-case and application specific, and depends how your app gets notified about such events and what it does.
It could, for example:
use SSM Run Command to execute a bash/powershell commands on your instance to do cleaning before termination occurs.
it could call http endpoint on your instance which is exposed by your application. This way your application can directly get notified that it is going to be terminated soon.
copy some logs or data files to s3 before it gets terminated
and more
I have an architecture which looks like as follows:-
Multiple SNS -> (AWS Lambda or SQS with Poller)??? -> Dynamo Db
So, basically multiple SNS have subscribed to AWS Lambda or SQS with Poller and that thing pushes data to Dynamo Db.
But this ? thing do lot of transformation of message in between. So, now for such case, I can either use AWS Lambda or SQS with Poller. With AWS Lambda, I can do transformation in Lambda function and with SQS with Poller, I can do transformation in Poller. With AWS Lambda, I see one problem that code would become quite large as transformation is quite complex(has lot of rules), so I am thinking to use SQS. But before finalising on SQS, I wanted to know of the drawbacks of SQS which AWS Lambda removes?
Please help. Let me know if you need further information.
Your question does not contain much detail, so I shall attempt to interpret your needs.
Option 1: SQS Polling
Information is sent to an Amazon SNS topic
An SQS queue is subscribed to the SNS topic
An application running on Amazon EC2 instance(s) regularly poll the SQS queue to ask for a message
If a message is available, the data in the message is transformed and saved to an Amazon DynamoDB table
This approach is good if the transformation takes a long time to process. The number of EC2 instances can be scaled based upon the amount of work in the queue. Multiple messages can be received at the same time. It is a traditional message-based approach.
Option 2: Using Lambda
Information is sent to an Amazon SNS topic
An AWS Lambda function is subscribed to the SNS topic
A Lambda function is invoked when a message is sent to the SNS topic
The Lambda function transforms the data in the message and saves it to an Amazon DynamoDB table
AWS Lambda functions are limited to five minutes of execution time, so this approach will only work if the transformation process can be completed within that timeframe.
No servers are required because Lambda will automatically run multiple functions in parallel. When no work is to be performed, no Lambda functions execute and there is no compute charge.
Between the two options, using AWS Lambda is much more efficient and scalable but it might vary depending upon your specific workload.
We can now use SQS messages to trigger AWS Lambda Functions.
28 JUN 2018: AWS Lambda Adds Amazon Simple Queue Service to Supported
Event Sources
Moreover, no longer required to run a message polling service or create an SQS to SNS mapping.
AWS Serverless Model supports a new event source as following:
Type: SQS
PropertiesProperties:
QueueQueue: arn:aws:sqs:us-west-2:213455678901:test-queue arn:aws:sqs:us-west-2:123791293
BatchSize: 10
AWS Console also support:
Further details:
https://aws.amazon.com/blogs/aws/aws-lambda-adds-amazon-simple-queue-service-to-supported-event-sources/
https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html