I'm stuck in the storage stage. It says: To use hibernation, encrypt the root volume.. But I cannot click the encrypt checkbox, nor did I find any way to encrypt the root volume.
Update 2020-03
A lot has changed since my post below and Hibernation is now a piece of cake. Encryption can be done in the normal ec2 creation flow in AWS console, Amazon Linux 2 is also supported.
All you need to be aware of:
not all instance types are supported, see here for an up to date list
RAM > 150GB is not supported
your disk space needs to be bigger than your RAM (as RAM will be written on disk when hibernating)
Original post:
In the end I was stuck at many stages, the official documentation is somehow lacking so I thought I'd document my findings:
In order to hibernate an instance it needs to be of type C3, C4, C5, M3, M4, M5, R3, R4 or R5. Plus it needs to be a Amazon Linux 1 AMI (this may be outdated, see here for an up to date list). I started with a Linux 2 AMI and didn't get any warning until it just didn't work in the end (only showing Suspend key pressed., Requested operation not supported, ignoring. in /var/log/messages), even their suggested steps to enable hibernation didn't work, as ec2-hibinit-agent is not an available packet on Amazon Linux 2.
Hibernation requires the boot volume to be encrypted. In order to achieve that, you need to encrypt the AWS Linux 1 AMI prior to starting the launch wizard:
create a KMS key using AWS console
find the ami id if AWS Linux 1 (not 2!) of your region e.g. using the ec2 launch wizard. In my case (eu-central-1) this was ami-0cfbf4f6db41068ac
create your own, encrypted ami (replace eu-central-1, the ami and the kms key with your own
aws ec2 copy-image --source-region eu-central-1 --source-image-id ami-0cfbf4f6db41068ac \
--region eu-central-1 --name "LinuxAMIEncrypted" --encrypted \
--kms-key-id arn:aws:kms:eu-central-1:123412341234:key/aaaaaaaa-1234-abce-abcd-1234abcdef01
Wait a few minutes until you see the new AMI showing up in EC2 -> AMIs
From AMIs (in EC2 AWS console) launch your new AMI
choose C3, C4, C5, M3, M4, M5, R3, R4 or R5
click Enable hibernation as an additional stop behavior
To test it:
run tail -f /var/log/messages on the ec2 instance to check for errors
run aws ec2 stop-instances --instance-ids "i-…" --hibernate to hibernate via terminal, or alternatively over aws console
Additionally I saw this error in /var/log/messages:
Agent hibernate - AccessDeniedException: User: arn:… is not authorized to perform: ssm:UpdateInstanceInformation on resource: ssm:UpdateInstanceInformation. I needed to attach AmazonEC2RoleforSSM and AmazonSSMAutomationRole to make these errors go away.
A visual guide as of 2022
Prerequisites
First, check the AWS Hibernation prerequisites about the requirement of Hibernation, pay attention to OS, RAM Cap (less than 150GB), and the supported instance families.
To enable hibernation, space is allocated on the root volume to store the instance memory (RAM). Make sure that the root volume is large enough to store the RAM contents and accommodate your expected usage, e.g. OS, applications. To use hibernation, the root volume must be an encrypted EBS volume
EC2 Instance Creation Steps
Within the Launch an instance wizard (the wizard when you create the new EC2 instance), expand the Storage (volumes) and EBS Volumes section
Pay attention to the following attributes:
Size (GiB): Make sure it's greater than the memory size and your expected usage
Encrypted: Yes
KMS Key: You can pick the default one for simplicity
Then scroll to the Advanced Details section, select Enable for the Stop - Hibernate behavior option
Then proceed to create your instance
Usage
To hibernate the created instance, select it from the dashboard, choose Instance state -> Hibernate instance
Related
I ran into some issues with my EC2 micro instance and had to terminate it and create a new one in its place. But it seems even though the old instance is no longer visible in the list, it is still using up some space on my disk. My df -h is listed below:
Filesystem Size Used Avail Use%
/dev/xvda1 7.8G 7.0G 719M 91% /
When I go to the EC22 console I see there are 3 volumes each 8gb in the list. One of them is attached (/dev/xvda) and this one is showing as "in-use". The other 2 are simply showing as "Available"
Is the terminated instance really using up my disk space? If yes, how to free it up?
I have just solved my problem by running this command:
sudo apt autoremove
and a lot of old packages are going to be removed, for instance many files like this linux-aws-headers-4.4.0-1028
Amazon Elastic Block Storage (EBS) is a service that provides virtual disks for use with Amazon EC2. It is network-attached storage that persists even when an EC2 instance is stopped or terminated.
When launching an Amazon EC2 instance, a boot volume is automatically attached to the instance. The contents of the boot volume is copied from an Amazon Machine Image (AMI), which can be chosen from a pre-populated list (including the ability to create your own AMI).
When an Amazon EC2 instance is Stopped, all EBS volumes remain attached to the instance. This allows the instance to be Started with the same configuration as when it was stopped.
When an Amazon EC2 instance is Terminated, EBS volumes might or might not be deleted, based upon the Delete on Termination setting of each volume:
By default, boot volumes are deleted when an instance is terminated. This is because the volume was originally just a copy of an AMI, so there is unlikely to be any important data on the volume. (Hint: Don't store data on a boot volume.)
Additional volumes default to "do not delete on termination", on the assumption that they contain data that should be retained. When the instance is terminated, these volumes will remain in an Available state, ready to be attached to another instance.
So, if you do not require any content on your remaining EBS volumes, simply delete them. In future, when launching instances, keep an eye on the Delete on Termination setting to make the clean-up process simpler.
Please note that the df -h command is only showing currently-attached volumes. It is not showing the volumes in Available state, since they are not visible to that instance. The concept of "Disk Space" typical refers to the space within an EBS volume, while "EBS Storage" refers to the volumes themselves. So, the 7GB of the volume that is used is related to that specific (boot) volume.
If you are running out of space on an EBS volume, see: Expanding the Storage Space of an EBS Volume on Linux. Expanding the volume involves:
Creating a snapshot
Creating a new (bigger) volume from the snapshot
Swapping the disks (requiring a Stop/Start if you are swapping a boot volume)
These 2 steps add an extra hard drive to your EC2 and format it for use:
Attach an extra hard drive (EBS: Elastic Block Storage) to an EC2
Format an EBS drive attached to an EC2
Here's pricing info. Free Tier includes 30GB. Afterward it's $1.25/month for 10GB on a General Purpose SSD (gp2).
To see how much space you are using/need:
Check your current disk use/available in Linux with df -h.
Check the size of a directory in Linux with du -sh [path].
As I understood, for an EBS backed EC2 instance, it's root device will be an EBS volume. Now if I want to have the content of the EBS volume to be a snapshot that I took earlier (for the root device of another EBS backed EC2 instance), how can I do that?
The short version is that you find the snapshot in the AWS management console, click the Launch button, and follow the steps in the wizard (to e.g. select availability zone).
There is a detailed walk through here:
http://www.techrepublic.com/blog/datacenter/how-to-create-a-new-ami-from-a-snapshot-and-launch-a-new-vm/5349
This can also be done a number of other ways, including
From the command line / a script
Programmatically through the API
Automatically e.g. using Auto Scaling
I have a Windows 2008 EC2 instance to which I have done some customizing on the EBS boot drive.
I started the instance as m1.small (or m1.large) and the instance storage does not appear as an additional drive.
I've read that the -b switch in the ec2-run-instances command allows you to create mappings for the ephymeral instance storage. The ec2-run-instances command creates a new instance, however, in my case, the instance already exists and therefore I start it as ec2-start-instances, which does not have a -b switch for ephymeral instance storage.
Is there any way I can get to the ephymeral instance storage that comes with an m1.small instance for my existing EBS-booted instance?
UPDATE: It seems that nowadays (Feb 2015) Windows machines mount ephymeral instance storage in the Z: drive.
I'm afraid this functionality isn't available (yet) for Amazon EC2, but it's a very good question in fact - the common answer used to refer to the explicated launch time requirement, see e.g. ec2-modify-instance-attribute:
Note
If you want to add ephemeral storage to an Amazon EBS-backed instance,
you must add the ephemeral storage at the time you launch the
instance. For more information, go to Overriding the AMI's Block
Device Mapping in the Amazon Elastic Compute Cloud User Guide, or to
Adding A Default Instance Store in the Amazon Elastic Compute Cloud
User Guide. [emphasis mine]
That hasn't been that much of an issue in the past, but given the recent introduction of 64-bit ubiquity implies a significant improvement of vertical scaling versatility (see EC2 Updates: New Medium Instance, 64-bit Ubiquity, SSH Client), this is suddenly a topic indeed - your question yields even more questions in turn:
What happens for the converse case, i.e. when I start a sufficiently large instance with lots of ephemeral storage and scale it down (and possibly up again) thereafter?
In case the initial block device mapping is retained somehow, should we always start with a large instance therefore? (I actually doubt that this is the case though.)
This question can only be addressed by the AWS team I guess, so you may want to file a support request or relay the question to the Amazon Elastic Compute Cloud forum at least.
I think what you're asking (but correct me if I'm wrong) is "how do I add additional storage to an EC2 instance?".
In which case, the answer is:
Select the Volumes panel in the AWS console and create a new volume of the size you want, making sure it's in the same Availability Zone as the instance you want to attach it to. Then select that new Volume, and click 'Attach' - select the instance you want to attach it to, and click OK.
Now log-on to the instance, and in Computer Management select the Disk Management plugin, format the new unassigned partition, and give it whatever drive letter you wish. It will then show up in Explorer as a standard Windows drive.
I know it is a newbie question but the documentation is really not clear about that.
I have successfully launched a Cluster Instances Amazon Linux AMI 2011.02.1 cc1.4xlarge instance. I need to upload and process a 15 GB text file and I don't know how to activate the "additional" storage.
By default, I have just 8 GB of storage. Can someone help me writing simple and clear instruction on how to activate all the storage (1690 GB) I should have?
Thanks a lot in advance.
Regards,
Eugenio
You should be able to mount your ephemeral drives by following this guide.
Cheers.
It's a very common question and there's a large confusion that everyone has around this when starting to work with AWS EC2.
There are 2 things to have in mind here :
Instance Type
AMI (Amazon Machine Image, basically a disk snapshot with some operating system)
Each AMI has some requirements that will dictate parts of the configuration of your created instance.
Here we're concerned with the Root Device type of an AMI :
EBS-backed AMI
These AMI's only work on instances that have an EBS volume attached. And therefore if you choose an EBS-backed AMI, the wizard will by default add an EBS volume (usually of size 8 GiB) without you noticing, and you'll be charged for this EBS separately in addition to the EC2 hourly charge. In this case when you ssh to your instance you will only find 8 GiBs of storage and not the large storage you're promised when you view the EC2 Pricing table at https://aws.amazon.com/ec2/pricing/ . And that is confusing.
If you also want to use your -usually large- instance store with these AMIs, then when you create that instance you have to explicitly add a volume and select Instance Store 0 from Volume Type (in Step 4 - Add Storage). You may have more than 1 drives assigned to your instance type, you need to add all of them in this case (Instance Store 0,1,..etc). These volumes can only be added at creation time. If the volumes were automatically mounted then you'll find them at /media/ephemral*, if not, you'll have to mount them manually.
Instance-Store-backed AMI
Those images use the Instance Storage as their root device (the OS will be installed on the instance store at / ). I think the confusion comes from the fact that these AMI's are not instantly viewable in the Quick Start tab of Step 1 at which you choose the AMI. All AMIs in the Quick Start tab are EBS-backed, and Instance Store AMIs are very uncommon these days, everyone uses EBS-backed AMIs because of their various speed and persistence benefits, and that's recommended by Amazon as well.
If you want an instance-store-backed AMI to avoid dealing with any EBS whatsoever, then in Step 1, choose the tab Community AMIs, and filter them by Root Device Type = Instance Store, then choose your AMI. Whether or not you add Instance Store 0 in Step 4 - Add Storage, the instance store will be available.
Check what Amazon Linux AMI type you're launching the image as. In my experience, if you launch with the "EBS-backed" AMI type, it won't mount the ephemeral storage. If you switch to the "Instance Store" type, it will mount it for you at /media/ephemeral0
http://aws.amazon.com/amazon-linux-ami/
Or is it absolutely necessary to have an EBS backed storage? Does anyone have manuals/Howtos?
I am following the manual given on this site.
You can create an AMI from either an EBS or S3-backed running instance. The simplest way is to use the AWS Management Console to select the instance and click 'Create Image' from the Instance Actions menu.
This will create either an EBS or S3-backed AMI, depending on the type of instance.
Be aware that creating an AMI from a Running instance is inherently potentially unsafe - the storage is not imaged as a hard 'point-in-time' snapshot, so changes to the filesystem whilst the image is being created may not be recorded, and may even result in a compromised image. You should quiesce the OS as far as possible before starting.
Note also that your instance will reboot during the image-creation process, so make sure you're prepared for any temporary loss of service from the VM.
If you bundle a Windows EBS instance while it is running, its Administrator password will be reset by Amazon's rebundling tools - so you'll have to use the EC2 "get Admin password" function with instances launched from the new AMI.
If you stop your Windows instance before rebundling, its Administrator password will remain intact through the rebundling process.
I don't know if this is documented anywhere, but it's certainly been my experience with Windows 2003 instances.
you will to refer the manual for the three commands:
ec2-bundle-vol, ec2-upload-bundle, ec2-register.
Keep at hand your private and certificate key (eg. pk.pem, cert.pem files) and your access and shared key. Download the ec2 ami tools and setup the environment variables for the ec2 ami tools. This site might give you the details on how to bundle the volume, create the images, upload them to S3 and finally register it as your own ami.
http://alestic.com/2009/06/ec2-ami-bundle
It is possible to create an AMI from an EBS running instance using the AWS management console. as mentioned by Eight-Bit Guru by clicking 'Create Image' from the Instance Actions menu.
For the S3 backed instance it is currently not possible without bundling the instance.
If you are using a linux ami then you have two ways as explained in this guide:
Creating an AMI from S3 backed instance - For Linux
For windows based S3 backed instance you can follow other guide
Creating an AMI from S3 backed instance - For Windows
Hope this will help for creating an AMI for S3 backed intances.