I'm looking to setup a demo environment in Amazon that consists of a pre-configured EC2 image that resets itself back to a snapshot configuration every hour, this is would be a Linux VM.
What would be the best way to go about doing this in EC2? Does Amazon offer any tools for scheduling and reverting to the snapshot or would this need to be done from a third party VM or software?
There is no VMWare-like 'snapshot' functionality in Amazon EC2 (where you can roll-back to a point-in-time).
The network-attached disk storage system used with Amazon EC2 is called Amazon Elastic Block Store (EBS). While EBS does have a 'snapshot' function, this actually takes a backup of an EBS Volume and stores it in Amazon S3. The snapshot can then be used to create a new EBS volume, which will contain the same contents as the original disk at the time the snapshot was created.
One option would be to launch a new Amazon EC2 instance, which will automatically create a new boot disk from the indicated Amazon Machine Image (AMI). This is the way to launch new machines with the same disk content. However, this might not lend itself well to your "revert every half hour" since it requires a new machine to be started, which will also trigger a new hourly billing cycle.
You might be able to script the deletion of files or the reload of some database tables, but this will depend upon your particular system and applications.
Related
I create an instance but do not add additional storage / a volume, will a snapshot backup the instance and the changes I have made to the config or does it only backup data from the volumes?
I am making some training material and would like to do the following:
Create an instance
Create users
Make a few more config/program installation changes
At this point, I would have a "clean copy".
I then would like to take a Snapshot of the state of the instance.
fool around, possibly break stuff
Then restore the instance to the "clean copy" after the instance has been created but before I started messing around.
Is this possible with snapshots?
Let's clarify some terminology:
A Snapshot can be made of an Amazon EBS volume. This makes a backup of the specific disk volume. The snapshot can be used later to create a new EBS Volume.
Or, you can create an Amazon Machine Image (AMI) of an Amazon EC2 instance. This creates an EBS Snapshot of all associated volumes and also stores some metadata about the AMI.
You can then Launch a new Amazon EC2 instance from the AMI. This will include a copy of the disks.
If you wish to restore the instance to the "clean copy", then it is best to launch a new instance from the AMI rather than trying to 'reset' the instance to an earlier state. The instance will have a different Instance ID and a different IP address, but will otherwise be the same as the instance from which the AMI was created.
As I understood, for an EBS backed EC2 instance, it's root device will be an EBS volume. Now if I want to have the content of the EBS volume to be a snapshot that I took earlier (for the root device of another EBS backed EC2 instance), how can I do that?
The short version is that you find the snapshot in the AWS management console, click the Launch button, and follow the steps in the wizard (to e.g. select availability zone).
There is a detailed walk through here:
http://www.techrepublic.com/blog/datacenter/how-to-create-a-new-ami-from-a-snapshot-and-launch-a-new-vm/5349
This can also be done a number of other ways, including
From the command line / a script
Programmatically through the API
Automatically e.g. using Auto Scaling
I know it is a newbie question but the documentation is really not clear about that.
I have successfully launched a Cluster Instances Amazon Linux AMI 2011.02.1 cc1.4xlarge instance. I need to upload and process a 15 GB text file and I don't know how to activate the "additional" storage.
By default, I have just 8 GB of storage. Can someone help me writing simple and clear instruction on how to activate all the storage (1690 GB) I should have?
Thanks a lot in advance.
Regards,
Eugenio
You should be able to mount your ephemeral drives by following this guide.
Cheers.
It's a very common question and there's a large confusion that everyone has around this when starting to work with AWS EC2.
There are 2 things to have in mind here :
Instance Type
AMI (Amazon Machine Image, basically a disk snapshot with some operating system)
Each AMI has some requirements that will dictate parts of the configuration of your created instance.
Here we're concerned with the Root Device type of an AMI :
EBS-backed AMI
These AMI's only work on instances that have an EBS volume attached. And therefore if you choose an EBS-backed AMI, the wizard will by default add an EBS volume (usually of size 8 GiB) without you noticing, and you'll be charged for this EBS separately in addition to the EC2 hourly charge. In this case when you ssh to your instance you will only find 8 GiBs of storage and not the large storage you're promised when you view the EC2 Pricing table at https://aws.amazon.com/ec2/pricing/ . And that is confusing.
If you also want to use your -usually large- instance store with these AMIs, then when you create that instance you have to explicitly add a volume and select Instance Store 0 from Volume Type (in Step 4 - Add Storage). You may have more than 1 drives assigned to your instance type, you need to add all of them in this case (Instance Store 0,1,..etc). These volumes can only be added at creation time. If the volumes were automatically mounted then you'll find them at /media/ephemral*, if not, you'll have to mount them manually.
Instance-Store-backed AMI
Those images use the Instance Storage as their root device (the OS will be installed on the instance store at / ). I think the confusion comes from the fact that these AMI's are not instantly viewable in the Quick Start tab of Step 1 at which you choose the AMI. All AMIs in the Quick Start tab are EBS-backed, and Instance Store AMIs are very uncommon these days, everyone uses EBS-backed AMIs because of their various speed and persistence benefits, and that's recommended by Amazon as well.
If you want an instance-store-backed AMI to avoid dealing with any EBS whatsoever, then in Step 1, choose the tab Community AMIs, and filter them by Root Device Type = Instance Store, then choose your AMI. Whether or not you add Instance Store 0 in Step 4 - Add Storage, the instance store will be available.
Check what Amazon Linux AMI type you're launching the image as. In my experience, if you launch with the "EBS-backed" AMI type, it won't mount the ephemeral storage. If you switch to the "Instance Store" type, it will mount it for you at /media/ephemeral0
http://aws.amazon.com/amazon-linux-ami/
I am new to Amazon web services. I need to create a server image with a few software packages pre installed on the EC2 instances.
One option I am considering is to create an EBS volume with these packages and then use them to launch EC2 instances.
The other option I am thinking of is to create a private AMI and then use them to launch the EC2 instances.
I am not sure which option is better.
One other slightly related question I have is can I create a private EBS volume and share it with some other account.
You should create your own AMI, as it's probably more efficient. In order to create an instance, you must specify an AMI. Specifying an EBS snapshot as the root device in addition to specifying the AMI (if this even works, I haven't tested it) would just result in the EC2 cloud launching your instance with the AMI, followed by overwriting it with the EBS snapshot for the root volume.
It's easy enough to keep the AMIs laying around as prototypes.
I am running a vanilla Windows install on Amazon EBS volume. The computer takes 10 minutes to boot, which may be understandable as 2 reboots are required. However, taking a snapshot is also a 10-15 minute process. Can anyone explain this? Any way to speed it up? I am a bit surprised, because I thought that snapshots are immediate replicas of the running EBS volume, in which case shouldn't they take just a couple of seconds to complete?
I will add that the console shows that "snapshot" is completed very quickly. But the "AMI" section is what seems to take 10-20 minutes. What's the difference? Is the snapshot available for use immediately, or do I need to wait for the AMI?
From the EBS product page:
Amazon EBS snapshots are incremental
backups, meaning that only the blocks
on the device that have changed since
your last snapshot will be saved. If
you have a device with 100 GBs of
data, but only 5 GBs of data has
changed since your last snapshot, only
the 5 additional GBs of snapshot data
will be stored back to Amazon S3.
Subsequent snapshots are fast because only the changed blocks need to be saved. So the time it takes scales with the amount of changes since the last snapshot.
Is the snapshot available for use
immediately, or do I need to wait for
the AMI?
Also from the product page:
New volumes created from existing
Amazon S3 snapshots load lazily in the
background. This means that once a
volume is created from a snapshot,
there is no need to wait for all of
the data to transfer from Amazon S3 to
your Amazon EBS volume before your
attached instance can start accessing
the volume and all of its data. If
your instance accesses a piece of data
which hasn’t yet been loaded, the
volume will immediately download the
requested data from Amazon S3, and
then will continue loading the rest of
the volume’s data in the background.
The creation of an AMI is a multiple-step process.
The Snapshot of the current machine is started (that's darn near instantaneous)
The snapshot copies the "changed blocks" from the base AMI to the snapshot lazily (that's pretty quick too)
The underlying Windows image is then prepared to be an AMI base image this starts with booting a "ghost" instance from the image with the snapshot as the disk image.
A SYSPREP is started to "reseal" the machine so it gets new machine SIDs.
The new image is then re-snapshotted
The AMI is marked "complete"