I'm looking for a unique ID associated with an Amazon EC2 instance. This ID must not change across reboots/poweroff-on/etc, and the ID must not be able to be duplicated (by cloning the instance) etc.
Can someone confirm if the instance id (i-XXXXXX) is such a number? If not, I suspect I could use the MAC address (if the instance has ENI option which makes MAC permanent).
Is there another number to use?
Your instance IDs are globally unique, for as long as you have access to the resource.
source: https://forums.aws.amazon.com/message.jspa?messageID=319825
By access, I mean so long as the instance is in your console / assigned to your account.
If you can't see the instance in your console, you can assume you no longer have access to that resource (and as such, can't expect you'll ever get another instance with the same ID).
If you see the instance in your console, regardless of state (other than terminated, which takes some minutes to be cleaned up from the console), the instance is "yours" and the unique ID will not change. This includes reboots.
If your instance needs to be "re-provisioned" or some other terminology in a notice from AWS, you can expect the instance will come up with a different unique ID. This sometimes happens if the underlying host that's servicing what you see as your "instance" needs maintenance work done that can not avoid a restart / tearing down of the instances that machine hosts, etc. Typically, this will not occur without ample notice from AWS.
Related
I want to maintain a pool of stopped amazon ec2 instances. Whenever the amount is below the threshold, I would like to be able to create new instances and then immediately stop them once they are running. Is this possible within the amazon infrastructure alone?
You can certainly create Amazon EC2 instances and then Stop them, making the available to Start later. As you point out, this has the benefit that instances will Start faster than they take to Launch a new instance.
There is no automated method to assist with this. You could have to code a solution that does the following:
Monitor the number of Stopped instances
If the quantity is below the threshold, launch a new instance
The new instance could automatically stop itself via User Data (either via a Shutdown command to the Operating System, or via a StopInstances call to EC2)
Some things you would have to consider:
What triggers the monitoring? Would it be on a schedule?
The task that launches a new instance would need to wait for the new instance to Launch & Stop before launching any more instances
What Starts the instances when they are needed?
Do instances ever get Stopped when they are no longer required?
The much better choice would be to use Auto Scaling, with a scale-out alarm based on some metric that says your fleet is busy, and a scale-in alarm to remove instances when the fleet is not busy. The scale-out alarm could be set to launch instances once a threshold is passed (eg 80% CPU) that should allow the new instance(s) to launch before things are 100% busy. The time difference between launching a new instance and starting an existing instance is quite small (at least for Linux).
If you're using Windows, the biggest time delay when launching a new instance is due to Sysprep, which makes a "clean" machine with new Unique IDs. You could cheat by creating an AMI without Sysprep, which would boot faster.
Perhaps I am misunderstanding your objective... you can't "ensure availability" of instances without paying for them.
Instances in the stopped state are only logical entities that don't physically exist anywhere -- hardware is allocated on launch, deallocated on stop, reallocated on the next start. In the unlikely condition where an availability zone is exhausted of capacity for given instance class, stopped instances of that class won't start, because there is no hardware available for them to be deployed onto.
To ensure that instances are always available, you have to reserve them, and you have to specify the reservations in a specific availability zone:
Amazon EC2 Reserved Instances provide a significant discount (up to 75%) compared to On-Demand pricing and provide a capacity reservation when used in a specific Availability Zone. [emphasis added]
https://aws.amazon.com/ec2/pricing/reserved-instances/
Under most plans, reserved instances are billed the same rate whether they are running or not, so there would be little point in stopping them.
Just upgraded and have no idea how to migrate my existing m1.small instance to a reserved m3.medium instance.
I upgraded because my server was running slow. Would it be possible to give me a step by step on how to migrate my original instance over to the reserved one?
I'm using the ec2 dashboard. From what I've read I create a snapshot, but that's how far I've got. Thanks in advance
A Reserved Instance on Amazon EC2 is a means of reducing the hourly cost of using EC2. In exchange for an up-front payment, the hourly charge for EC2 is reduced.
A Reserved Instance has several components:
Instance type (the size of EC2 instance, consisting of an family, CPU count and RAM size)
Operating System (eg costs vary for Windows vs Linux)
Availability Zone (used to ensure that capacity has been allocated)
Each hour, the AWS billing system looks for any running instances in an account that owns Reserved Instances. If any of the running instances match the above three attributes, then the instance will receive the pricing discount. (Only one instance will receive a discount per Reserved Instance per 'running' hour.)
It is not possible to select a specific Amazon EC2 instance and designate it as a Reserved Instance. The billing system does this automatically. Thus, an instance could be discounted one hour and not another hour, depending upon what other instances are running. Think of it as a billing discount rather than a particular instance being "the" Reserved Instance.
Reserved Instances are also shared across AWS Accounts that are connected via consolidated billing. If there is no matching instance to receive a discount in the AWS Account that owns the Reserved Instance, the billing system will attempt to allocate it to a matching running instance in a linked AWS Account for that hour.
Therefore, to make use of your new m3.medium Reserved Instance via the AWS management console:
Stop your existing m1.small instance
From the Actions menu, select Instance Settings / Change Instance Type
Select m3.medium
Start your instance
Make sure that your instance matches the Instance Type, Availability Zone and Operating System listed in your Reserved Instance and you will automatically receive the billing discount. If multiple running EC2 instances match the profile of the Reserved Instance, only one will receive the discount.
There is no need to create an EBS snapshot or create a new instance. Simply changing the instance attributes to match the Reserved Instance is sufficient.
Note: The Reserved Instance itself will not provide any speed improvement. It is just a billing discount. It is your choice of a larger instance type will provide the benefit.
See also:
Reserved Instance documentation
Resizing your instance
Does amazon have the ability to ever offer a feature to allow users to change their ec2 instance types while the server is running? So like a t1.micro to a m1.large and not shut anything down. I know nothing about VMs or what would be involved, so I'm not sure if this is even possible, the level of difficulty (I'd assume difficult enough if they haven't rolled it out), and if there are any plans to do so.
No, instance type can not be changed while the instance is running. To change the instance type you must stop and, change the instance type and then start it.
Is there a way to identify the heroku dyno name (e.g. web.1, web.2) from within the application? I'd like to be able to generate a unique request id (e.g. to track requests between web and worker dynos for consolidated logging of the entire request stack) and it seems to me that the dyno identifier would make a decent starting point.
If this can't be done, does anyone have a fallback recommendation?
Recently that issue has been addressed by Heroku team.
The Dyno Manager adds DYNO environment variables that holds identifier of your dyno e.g. web.1, web.2, foo.1 etc. However, the variable is still experimental and subject to change or removal.
I needed that value (actually instance index like 1, 2 etc) to initialize flake id generator at instance startup and this variable was working perfectly fine for me.
You can read more about the variables on Local environment variables.
I asked this question of Heroku support, and since there are others here who have asked similar questions to mine I figured I should share it. Heroku staff member JD replied with the following:
No, it's not possible to see this information from inside the dyno.
We've reviewed this feature request before and have chosen not to
implement it, as this would introduce a Heroku-specific variable which
we aim to avoid in our stack. As such, we don't have plans to
implement this feature.
You can generate / add to your environment a unique identifier (e.g. a
UUID) on dyno boot to accomplish a similar result, and you can
correlate this to your app's dynos by printing it to your logs at that
time. If you ever need to find it later, you can check your logs for
that line (of course, you'll need to drain your logs using Papertrail,
Loggly, etc, or to your own server).
Unfortunately for my scenario, a UUID is too long (if I wanted such a large piece of data, I would just use a UUID to track things in the first place). But it's still good to have an official answer.
Heroku has a $DYNO environment variable, however there are some big caveats attached to it:
"The $DYNO variable is experimental and subject to change or removal." So they may take it away at any point.
"$DYNO is not guaranteed to be unique within an app." This is the more problematic one, especially if you're looking to implement something like Snowflake IDs.
For the problem you're attempting to solve, the router request ID may be more appropriate. Heroku passes a unique ID to every web request via the X-Request-ID header. You can pass that to the worker and have both the web and worker instance log the request ID anytime they log information for a particular request/bit of work. That will allow you to correlate incidents in the logs.
This may not exactly answer the question, but you could have a different line in your Procfile for each worker process (using a ps:scale of 1 for each). You could then pass in the worker number as an environment variable from the Procfile.
Two lines from an example procfile might look like:
worker_1: env WORKER_NUMBER=1 node worker
worker_2: env WORKER_NUMBER=2 node worker
The foreman package which heroku local uses seems to have changed the ENV variable name again (heroku/7.54.0). You can now get the worker name via $FOREMAN_WORKER_NAME when running locally. It has the same value $DYNO will have when running on Heroku (web.1, web.2, etc)
The foreman gem still uses $PS, so to access the dyno name and have it work both on heroku and in development (when using foreman) you can check $PS first and then $DYNO. To handle the case of a local console, check for Rails.console
dyno_name = ENV['PS'] || ENV['DYNO'] || (defined?(Rails::Console) ? "console" : "")
It's dangerous to use the DYNO environment variable because its value is not guaranteed to be unique. That means you can have two dynos running at the same time that briefly have the same DYNO variable value. The safe way to do this is to enable dyno metadata and then use the HEROKU_DYNO_ID environment variable. That will better let you generate unique request ids. See: https://devcenter.heroku.com/articles/dyno-metadata
In my C++/winapi program I use an event object to detect if an instance of my application is already running, in order to not let the second instance start.
But in the same time I want to allow multiple instances of the program to run if they are launched using a different user account. E.g. using "runas" command.
(That is: for each user account only 1 instance can run simultaneously)
I use the code that looks like this:
HANDLE hSingleInstance=OpenEvent(EVENT_MODIFY_STATE,FALSE,
"Local\\SingleInstanceEventName");
if(hSingleInstance!=NULL) {
// there is an instance already running
SetEvent(hSingleInstance);// let know the 1st instance that we are trying to start
CloseHandle(hSingleInstance);
return 0; // exit the program
}
else {
// this is the 1st instance
hSingleInstance=CreateEvent(NULL,FALSE,FALSE, "Local\\SingleInstanceEventName");
}
It works as expected in XP - I can run only 1 instance using the same user account, and I can run multiple instances using multiple user accounts.
But in Windows 7, the OpenEvent() function always "finds" my event object, even if the event object is created using another user account. Because of this I cannot run multiple instances using different user accounts (as desired).
What should I change in my code to allow multiple instances be run using diffent user accounts in Windows 7?
Or maybe my approach is completely wrong, then what one is correct?
thank you
All you have to do is include the username in the name of the event somehow. I'm quite frankly a bit surprised that your scenario worked in XP.
I think you're running into security issues here. Generally speaking, one app cannot modify objects created by another app that's running as another user, so if one app creates the event object, an app running as another user should not be able to open it - even it already exits.
There are some subtle changes to this on Vista onwards; the concept of running something 'elevated' introduces an extra wrinkle; it's similar, but not exactly the same, as running as another user. Some resources are allowed through for compat reasons - I can't remember offhand if events/mutexes are in this category.
Either way, as currently written, your code likely is incorrect: part of the problem is that OpenEvent can fail for multiple reasons: your code assumes it's failing because the event is not created, but it could be failing because the event is created but that the calling code does not have permission to modify the object.
As mentioned in one of the other replies, if you want to do this on a per-user basis (but still within the same session), you need to give your events per-user names: add the username to the event's base name, and you should be done.