So I create an instance using one of the Public AMI EBS Ubuntu flavors. I create an EBS volume and attach it to the instance. I format the volume and add an entry to /etc/fstab to mount it on /vol. I add mysql to the AMI and move the data files to the EBS volume I formatted and mounted at /vol. I then create an AMI from the running instance. Then I terminate the running instance.
I start a new instance using the freshly created AMI (with mysql on it). The /vol is mounted has the mysql data files - good, I expect that. Here's where I am confused. When I create any directory or files on the EBS volume /vol they aren't there any more after I terminate the instance and create a new one. The mysql stuff is there but no new stuff I created. Aren't those files and directory supposed to be there? Or am I misunderstanding how this works?
When you create an AMI, "Amazon EC2 powers down the instance, takes images of any volumes that were attached, creates and registers the AMI, and then reboots the instance." -Amazon. When the AMI is used to launch an instance, the images (snapshots) of the attached drives are used to create new volumes. It is these new volumes that are attached to the new instance, not your original EBS. (This generates lots of orphan volumes and snapshots with ongoing use.)
There is no automatic attaching of the EBS volume you created. What is automatically attached is the volume it creates at the time of launching the instance from your AMI! It creates this volume from the snapshot it made of your EBS at the time of the AMI creation!
The way to avoid clone volumes from being created and attached to new instances is simple: detach your volumes before making AMIs. You need to attach your EBS volumes manually with the EC2 Web Control Panel, or programmatically with .net or Java programming, scripting or command line tools.
EBS volumes are not tied to an AMI, only to the literal instance you attach them to. When you created your AMI and a new instance from that, the EBS is not cloned, nor does it follow you to the new instance.
You could move the EBS drive to the new instance manually. Alternately you could snapshot the EBS volume & clone a new drive from that.
Related
I create an instance but do not add additional storage / a volume, will a snapshot backup the instance and the changes I have made to the config or does it only backup data from the volumes?
I am making some training material and would like to do the following:
Create an instance
Create users
Make a few more config/program installation changes
At this point, I would have a "clean copy".
I then would like to take a Snapshot of the state of the instance.
fool around, possibly break stuff
Then restore the instance to the "clean copy" after the instance has been created but before I started messing around.
Is this possible with snapshots?
Let's clarify some terminology:
A Snapshot can be made of an Amazon EBS volume. This makes a backup of the specific disk volume. The snapshot can be used later to create a new EBS Volume.
Or, you can create an Amazon Machine Image (AMI) of an Amazon EC2 instance. This creates an EBS Snapshot of all associated volumes and also stores some metadata about the AMI.
You can then Launch a new Amazon EC2 instance from the AMI. This will include a copy of the disks.
If you wish to restore the instance to the "clean copy", then it is best to launch a new instance from the AMI rather than trying to 'reset' the instance to an earlier state. The instance will have a different Instance ID and a different IP address, but will otherwise be the same as the instance from which the AMI was created.
I'm using a community AMI, it's great but some of the stuff in it is outdated. Every time I spin up a new machine based on it I have to update all the libraries. I want to instead update them once and save the modified image. It's and EBS backed AMI. I tried creating a snapshot off of the volume of the running instance and then creating an AMI from the snapshot. The resulting AMI does indeed have all the modifications I made but the operating system is different! The original AMI has ubuntu while the thing that comes out is "other linux" - and some things are not working (CUDA). Both "RAM disk ID" and "kernel ID" in the original AMI details are blank so I leave them as "default" when creating the new AMI.
The preferred way to save a modified EC2 instance is to burn an AMI directly from the running instance, rather than taking a snapshot of its root volume.
If for any reason all you had was a snapshot of the root volume of a previously running instance, to create a bootable AMI you have to follow the following process: launch one of the stock EC2 AMIs, one that has the same OS as your EBS snapshot. Create an EBS volume from that snapshot. Stop the newly launched instance. Detach the root volume, and attach the new volume you created form the EBS snapshot as the root volume and start the instance. See Launching a Linux Instance from a Backup. NOTE: Although you can create a Windows AMI from a snapshot, you cannott successfully launch an instance from the AMI.
The easiest way to save an AMI with new modificationa, is to create the AMI image directly from the running instance, and not simply take a snapshot of the running volume.
From the AWS Management Console, click on the instance, then right-click Image -> Create Image.
From that dialog, set the Name, Description etc. Make sure to leave No Reboot unchecked. From the Instance Volumes section adjust the volume settings.
Note that your instance will reboot during the image creation process. Make sure you are prepared for the temporary loss of service of the instance during this time.
Let's say there is a single disk Disk 0 at an Amazon EC2 instance Instance1 with the corresponding EBS volume having the device-name /dev/sd1. Let's say this disk maps to C-drive on the VM.
Now can we create a new EBS volume attach it to Instance1 with the same device-name /dev/sd1?
If this is possible, does it mean, after the volume is attached, we can go ahead and extend the existing C-drive at Instance1, without having to go through the process of detaching it first, creating a snapshot etc.?
If an EBS volume is already mounted at /dev/sd1, it is not possible to mount another volume as that device.
First, detach the existing volume, then attach a new volume.
To extend the size of an EBS volume:
Detach the EBS volume from the EC2 instance
Create Snapshot of the EBS volume
Create Volume from the snapshot, making the volume larger
Attach the new EBS volume to the EC2 instance
The disk partition will also need to be extended. This can be done with the Disk Management console (Windows) or with the resize2fs command (Linux).
Had some data on ephemeral storage of a ec2 medium instance. They are lost between server reboot. I had a ami created just before the reboot. Is it supposed to contain that data? I had created a new instance from that ami but can't find the data on new ephemeral drive.
ok. I got it. Create Image only creates ebs backed ami. Creating instance storage backed ami is different. It is described here creating an instance store ami
Ephemeral drives on EC2 are, by definition, ephemeral. This means that their data in not kept between reboots, and not saved as part of the AMI image.
You can read more on what you can an cannot do with these kinds of disks here.
I have an ebs-backed instance running on EC2. I'm using it to do some computationally intensive text processing on around 16Gb of data which is stored on sdb (i.e. the larger ebs volume associated with the instance).
I'd like to parallelized the processing by creating replicas of this instance, each with its own copy of the data. I can create an AMI from the instance but I need the image to include BOTH sda (the root ebs volume) AND ALSO sdb, which is the volume where all the data is. How can I make a replica of the whole package?
Creating an image in the AWS Management Console just copies sda (i.e. the root volume, which is too small to hold my data).
Is this even possible?
(PS: I don't even see the sdb volume in the AWS Management Console Elastic Block Store->Volumes panel)
Thanks!
I once needed this sort of setting where I had to setup a MySQL on a EBS backed machine with data store in a separate EBS Volume. The AMI had to be such that every time you instanciate it, it should have the data volume (with static data in it) attached. This is how I did:
Created an EBS backed instance from any existing image
Attached a EBS volume, performed mkfs, mounted on /database
Copied data to the volume, e.g. under /database/mysql
Created image of this setup from AMI web console.
Now, every time I launch this image, I see the volume with all the data is there. I just mount it on /database and things get going.
I am not sure, if this is helpful to you but your problem seemed to close to this.
Update after #NAD's comment
Yeah, AMI creation process excludes stuffs that are under
/sys
/proc
/dev
/media
/mnt
So, the trick is to not have stuffs that you want to bundle up with your AMI under these directories.
Also, if you have volume that you want to auto-mount at boot, register it in fstab