Had some data on ephemeral storage of a ec2 medium instance. They are lost between server reboot. I had a ami created just before the reboot. Is it supposed to contain that data? I had created a new instance from that ami but can't find the data on new ephemeral drive.
ok. I got it. Create Image only creates ebs backed ami. Creating instance storage backed ami is different. It is described here creating an instance store ami
Ephemeral drives on EC2 are, by definition, ephemeral. This means that their data in not kept between reboots, and not saved as part of the AMI image.
You can read more on what you can an cannot do with these kinds of disks here.
Related
I create an instance but do not add additional storage / a volume, will a snapshot backup the instance and the changes I have made to the config or does it only backup data from the volumes?
I am making some training material and would like to do the following:
Create an instance
Create users
Make a few more config/program installation changes
At this point, I would have a "clean copy".
I then would like to take a Snapshot of the state of the instance.
fool around, possibly break stuff
Then restore the instance to the "clean copy" after the instance has been created but before I started messing around.
Is this possible with snapshots?
Let's clarify some terminology:
A Snapshot can be made of an Amazon EBS volume. This makes a backup of the specific disk volume. The snapshot can be used later to create a new EBS Volume.
Or, you can create an Amazon Machine Image (AMI) of an Amazon EC2 instance. This creates an EBS Snapshot of all associated volumes and also stores some metadata about the AMI.
You can then Launch a new Amazon EC2 instance from the AMI. This will include a copy of the disks.
If you wish to restore the instance to the "clean copy", then it is best to launch a new instance from the AMI rather than trying to 'reset' the instance to an earlier state. The instance will have a different Instance ID and a different IP address, but will otherwise be the same as the instance from which the AMI was created.
We are trying to use the d2.xlarge instance type for worker nodes in Hadoop cluster.
When i look at the AWS Site
https://aws.amazon.com/ec2/instance-types/
It says HDD based local storage.
Is it EBS Storage? or attached to the EC2 instance?
You can attach Elastic Block Store (EBS) volumes to any Amazon EC2 instance. EBS is persistent disk storage.
Some EC2 instances also have Instance Store, which is locally-attached disk storage. If the instance is stopped or terminated, the contents of Instance Store is lost.
Instance Store is popular for use with Amazon EMR because it provides very large amounts of storage for HDFS. However, please be aware that the data is lost if the cluster is terminated.
The d2.xlarge instance type has 3 x 2000 GB instance store drives, which are stored on magnetic disk. This is in addition to any EBS volumes you attach.
Local means attached storage. HDD means spinning disks (not SSD's).
I have an m3 large. Although I can find the other EBS volumes associated with that instance in the Volumes Section.
But I am not able to find my 32GB SSD disk.
How can we take backup of this SSD?
It appears that you are referring to the Instance Store SSD volume that is provided as part of an m3.large Amazon EC2 instance.
Instance Store volumes are temporary (aka "ephemeral") and the content is lost when the instance is Stopped, Terminated or fails. Therefore, it is recommended only for temporary files and swap files. Be sure to copy off any data you wish to keep before the instance is Stopped.
Instance Store volumes are not the same as Elastic Block Store (EBS) volumes. While EBS provides a snapshot capability, this is not available for Instance Store volumes.
Instead, you must copy off any data you wish to keep via normal filesystem commands, or run traditional backup software. There is no snapshot-like capability available for Instance Store volumes.
So I create an instance using one of the Public AMI EBS Ubuntu flavors. I create an EBS volume and attach it to the instance. I format the volume and add an entry to /etc/fstab to mount it on /vol. I add mysql to the AMI and move the data files to the EBS volume I formatted and mounted at /vol. I then create an AMI from the running instance. Then I terminate the running instance.
I start a new instance using the freshly created AMI (with mysql on it). The /vol is mounted has the mysql data files - good, I expect that. Here's where I am confused. When I create any directory or files on the EBS volume /vol they aren't there any more after I terminate the instance and create a new one. The mysql stuff is there but no new stuff I created. Aren't those files and directory supposed to be there? Or am I misunderstanding how this works?
When you create an AMI, "Amazon EC2 powers down the instance, takes images of any volumes that were attached, creates and registers the AMI, and then reboots the instance." -Amazon. When the AMI is used to launch an instance, the images (snapshots) of the attached drives are used to create new volumes. It is these new volumes that are attached to the new instance, not your original EBS. (This generates lots of orphan volumes and snapshots with ongoing use.)
There is no automatic attaching of the EBS volume you created. What is automatically attached is the volume it creates at the time of launching the instance from your AMI! It creates this volume from the snapshot it made of your EBS at the time of the AMI creation!
The way to avoid clone volumes from being created and attached to new instances is simple: detach your volumes before making AMIs. You need to attach your EBS volumes manually with the EC2 Web Control Panel, or programmatically with .net or Java programming, scripting or command line tools.
EBS volumes are not tied to an AMI, only to the literal instance you attach them to. When you created your AMI and a new instance from that, the EBS is not cloned, nor does it follow you to the new instance.
You could move the EBS drive to the new instance manually. Alternately you could snapshot the EBS volume & clone a new drive from that.
I know it is a newbie question but the documentation is really not clear about that.
I have successfully launched a Cluster Instances Amazon Linux AMI 2011.02.1 cc1.4xlarge instance. I need to upload and process a 15 GB text file and I don't know how to activate the "additional" storage.
By default, I have just 8 GB of storage. Can someone help me writing simple and clear instruction on how to activate all the storage (1690 GB) I should have?
Thanks a lot in advance.
Regards,
Eugenio
You should be able to mount your ephemeral drives by following this guide.
Cheers.
It's a very common question and there's a large confusion that everyone has around this when starting to work with AWS EC2.
There are 2 things to have in mind here :
Instance Type
AMI (Amazon Machine Image, basically a disk snapshot with some operating system)
Each AMI has some requirements that will dictate parts of the configuration of your created instance.
Here we're concerned with the Root Device type of an AMI :
EBS-backed AMI
These AMI's only work on instances that have an EBS volume attached. And therefore if you choose an EBS-backed AMI, the wizard will by default add an EBS volume (usually of size 8 GiB) without you noticing, and you'll be charged for this EBS separately in addition to the EC2 hourly charge. In this case when you ssh to your instance you will only find 8 GiBs of storage and not the large storage you're promised when you view the EC2 Pricing table at https://aws.amazon.com/ec2/pricing/ . And that is confusing.
If you also want to use your -usually large- instance store with these AMIs, then when you create that instance you have to explicitly add a volume and select Instance Store 0 from Volume Type (in Step 4 - Add Storage). You may have more than 1 drives assigned to your instance type, you need to add all of them in this case (Instance Store 0,1,..etc). These volumes can only be added at creation time. If the volumes were automatically mounted then you'll find them at /media/ephemral*, if not, you'll have to mount them manually.
Instance-Store-backed AMI
Those images use the Instance Storage as their root device (the OS will be installed on the instance store at / ). I think the confusion comes from the fact that these AMI's are not instantly viewable in the Quick Start tab of Step 1 at which you choose the AMI. All AMIs in the Quick Start tab are EBS-backed, and Instance Store AMIs are very uncommon these days, everyone uses EBS-backed AMIs because of their various speed and persistence benefits, and that's recommended by Amazon as well.
If you want an instance-store-backed AMI to avoid dealing with any EBS whatsoever, then in Step 1, choose the tab Community AMIs, and filter them by Root Device Type = Instance Store, then choose your AMI. Whether or not you add Instance Store 0 in Step 4 - Add Storage, the instance store will be available.
Check what Amazon Linux AMI type you're launching the image as. In my experience, if you launch with the "EBS-backed" AMI type, it won't mount the ephemeral storage. If you switch to the "Instance Store" type, it will mount it for you at /media/ephemeral0
http://aws.amazon.com/amazon-linux-ami/