How do I get an instance's termination time? I'm using the .net SDK - I can get the LaunchTime through the property by that name from a RunningInstance object. But how do I get the termination time?
The API does not expose this in a structured fashion, however it does record the reason for the last state transition which usually has a date: for example User initiated (2011-06-16 13:56:08 GMT)
Related
For over two days, I've been trying to deploy a CloudFormation stack using serverless framework. The thing is, as part of the stack, I have an RDS cluster as well as a custom resource which relies on a Lambda function (written in Python) for initializing some database tables.
The details of this custom resource in the serverless.yml file are the following:
rdsMigration:
Type: Custom::DatabaseMigration
DependsOn: rdsCluster
Properties:
ServiceToken: !GetAtt MigrateDatabaseLambdaFunction.Arn
Version: 1.0
When deploying using sls deploy, the cluster and the lambda functions are created correctly, but the process is stuck on creating the rdsMigration resource.
In the Lambda code, I've been careful to generate the response in all possible scenarios, including exceptions. However, that does not seem to be the problem.
Apparently, the function is not being invoked... kind of, because even the charts look weird:
You can see how there are no invocations, but there is a red dot in "Error count and success rate" about 5:15 PM, which is the time at which the resource creation started. Also, there are no green dots, and you can see the warning down in the legend, which claims that "One or more data-points have been dropped due to non-numeric values (NaN, -Infinite, +Infinite)". How is this possible? I assume it is no standard behavior, since other Lambda functions (which must be called using an API Gateway endpoint) do not show this strange chart.
Also, there are no log streams in CloudWatch. It is completely empty, as if the function was never invoked (which seems the case, except for the strange "red dot" at the moment of resource creation).
Finally, if I run a test case using the "AWS CloudFormation Create Request" template, the function runs properly, it creates the initial tables I expected for the DB (not always, but that is a different matter) and returns the response.
Do you have any idea of what is going on here? The worst about this is that I need to wait two hours between tests, since the CFN stack gets stuck during the creation and destruction steps until the timeout occurs.
Thanks!
The issue is with your lambda function. You have to send back the SUCCESS or FAILURE signals back to the CFN. Since your lambda function is nots sending any signals, its waiting for Timeout (2 hours) and the Cloudformation gets failed
1.The custom resource provider processes the AWS CloudFormation request and
returns a response of SUCCESS or FAILED to the pre-signed URL. AWS
CloudFormation waits and listens for a response in the pre-signed URL location.
2.After getting a SUCCESS response, AWS CloudFormation proceeds with the stack
operation. If a FAILURE or no response is returned, the operation fails.
Please use cfnresponse module in your lambda function to send the SUCCESS/FAILURE signals back to your Cloudformation
For more details:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-lambda-function-code-cfnresponsemodule.html
I finally managed to find a solution to the issue, albeit it is not explaining the strange behavior with the charts that I explained in the question.
My problem was similar to what Abhinaya suggested in her response. The Lambda function was not sending the signal properly because of a programming error. Essentially, I took the code from the documentation (the one for Python 3, second fragment starting by the end) and apparently I mistakenly removed the line for retrieving the ResponseURL. Of course, that was failing.
A side-comment about this: be careful when using Python's cfnresponse library or even the code snippet I linked in the documentation. It relies on botocore.vendored which was deprecated and no longer exist in latest botocore releases. Therefore, it will fail if your code relies on new versions of this library (as in my case). A simple solution is to replace botocore.vendored.requests with the requests library.
Still, there is some strange behavior that I cannot understand. On creation, the Lambda function is not recording anything to CloudWatch and there is this strange behavior in the charts that I explained in my question. However, this only happens on creation. If the function is manually invoked, or is invoked as part of the delete process (when removing the CFN stack), then it does write to CloudWatch. Therefore, the problem only occurs in the first invokation, apparently.
Best.
I have a handler function on AWS Lambda that is connecting to a Redis instance to store a single key in the cache. The function has completed successfully but the key in Redis shows up minutes (or more) after the fact.
This behavior is observable on both Heroku Redis and Redis Cloud, they're both hosted solutions.
I can't for the life of me figure out what's causing this lag. My Redis knowledge is practically zero, I know how to store a list using LPUSH and how to trim that list using LTRIM.
The writer to Redis uses this Node client while I observe the lag using redis-cli on my local machine.
Is it common to experience this kind of lack in the setup I describe? What can I do to debug this?
I'm purposefully ignoring most of the information in the question and would like to refer only to the alleged symptom, namely that
key show up only minutes after being stored
This behavior is impossible with Redis - any change to the data is immediately visible given Redis' design. That said, the only scenario what you're describing could be remotely possible is when you're writing to a Redis master server and reading from a very-badly-lagged slave. I can ensure you that this is not the case with Redis Cloud however.
The main reason is due to the fact that the Lambda container starts to sleep as soon as your function terminates, and the Redis client you are using is all asynchronous APIs.
Note that the API is entire asynchronous. To get data back from the server, you'll need to use a callback.
I'm assuming that the asynchronous SET is the last action performed in your Lambda function. Once that is called, the underlying Lambda container goes to sleep, and most likely, the actual SET action hasn't finished its job yet. Therefore, the record will not show in Redis until the exact same Lambda container was called to execute your function again, and finished the job that it was supposed to finish on the last execution. This is probably the lag that you are experiencing.
To test whether or not this is true, do a sleep action for a couple of seconds at the end of your function to delay the Lambda container going to sleep immediately, and see if the lag is still there.
I would also recommend not to use asynchronous behaviour APIs inside Lambda functions. They'll add state to your Lambda computation, and this is actually not recommended by AWS themselves within the Lambda documentations too.
Using AmazonSimpleEmailServiceClient.sendEmail() ; as per java docs
All service calls made using this client are blocking, and will not return until the service call completes.
Quick question is, is it thread safe?. Can multiple threads be using same instance of AmazonSimpleEmailServiceClient to call sendEmail() in parallel? Like, suppose, multiple users of my website trying to register on the website concurrently.
This is the answer I got from AWS:
Hi,
You should be able to use the same AmazonSimpleEmailServiceClient() object to send using multiple threads.
Regards,
Rohan
I am adding an updated answer since the original one is from 2012 and not very reassuring about the thread safety.
I am not sure in which version it was introduced, but in latest Java AWS SDK version available today (October 2016, version 1.11.43) the class is annotated as #ThreadSafe, so the sendEmail method can be assumed thread safe
I am writing java code for EC2 that will consume messages from SQS. My code will run for the hour and then review the queue to determine whether it should shut itself down (due to not enough items to warrant processing for another hour) or to keep running. In order for this to work, I need to know the precise time stamp of when billing for the instance began so I can add one hour to it and shutdown a few minutes before the hour is up if necessary.
How can I get the launch time stamp (the time AWS begins billing for the instance hour) of the EC2 instance my java code is currently running on using the java SDK preferably?
If shelling to uptime or dropping date +%s > /etc/started-at into your bootscript are too platform-dependent for your tastes, you could perform the DescribeInstances SOAP call and pluck out the launchTime field.
To discover the running machine's instance id, you can (from within the machine) make an http GET such as
curl http://169.254.169.254/latest/meta-data/instance-id
In a simple windows setup we have a COM singleton that runs as an out of process server.
Clients connect by calling cocreate and each receives an interface to the same instance of the server.
If clients shutdown normally they release their references.
The server has a bit of logic that keeps it alive for a short time after the last release to allow for new connections.
I'm interested in one special case - the server is running with only one client which crashes (consider this to be any random unknown crash) and the client exits without having released its references.
I observe that after a undefined period of time say 8 minutes the server receives release calls on the stubs of any objects that the server had returned interfaces from to the client. This appears to be an automatic cleanup that I assuming is started by the LRPC layer.
Is this documented anywhere and is the timeout configurable?
Note: Multithreaded apartment model used throughout.
This https://web.archive.org/web/20171228092925/http://www.microsoft.com/msj/0398/dcom.aspx under DCOM Garbage Collection seems to indicate that DCOM uses a 120 second timeout which needs to be missed three times, so about 6 minutes the client will be considered disconnected. Unfortunately it also indicates that it isn't user configurable and I cannot find any stuff to the contrary.