We have recently been experimenting with a micro instance to undertake a specific task. Last night, for no apparent reason, the EBS backed micro instance failed with no warning. We can no longer access the server through the Terminal (MacBook) and Monitoring show 100% utilisation (flat). We took a snapshot and created a new volume, but cannot attach this to the new micro instance, as its EBS backed. We are now in the process of setting up a new micro instance from scratch which is taking a lot of effort due to having to re-install software and configure them for our purpose
What are the best practices for running EBS backed micro instances? How can we avoid the same happening again?
You can certainly attach a new EBS boot volume to an EBS-backed instance:
Stop the instance (NOT terminate).
Snapshot its' boot volume.
Create a new volume from the snapshot.
Detach the original boot volume, and attach your new one.
Start the instance.
However, in your case, you don't know why the instance failed, so how do you know that a new boot volume created from a snapshot of the failed one will work? Did you try rebooting the instance? It sounds as though the OS might have panicked/bluescreened (you don't mention which OS is installed) or possibly just the SSH session daemon died, in which case a simple reboot might be all that is needed.
Related
I accidentally took snapshot from volume (100GB) that was attached to running instance (as a ROOT device). The snapshot creation took over one hour! After that I created bigger (400GB) volume from that snapshot and everything went fine. Then I:
stopped the instance
detached the old 100GB device from my instance
attached the new 400GB volume to that instance
started the instance
Everything seemed OK in AWS control panel (server running and disk status was fine) but I could not connect to that instance via SSH nor HTTP. SSH error: connection refused. So I changed back to old 100GB disk and instance worked well.
My suggestion was that something went wrong because of taking snapshot from running instance. So I deleted the snapshot and took new snapshot from stopped instance (volume). This time the creation of snapshot took only 5 minutes. Then I just created new volume from that snapshot, attached it to instance and started.. still not working.
So my questions are:
Can it be possible that even when I delete the old snapshot the data of that snapshot was partially used when making new snapshot?
How can I backup my current volume if every time I take a snapshot it uses old corrupted data
All these things were done via AWS control panel.
THANKS!!
Can it be possible that even when I delete the old snapshot the data
of that snapshot was partially used when making new snapshot?
I don't think so. It could be that when you created the first snapshot there were some resources in contention when trying to create it. Thus it took longer. Amazon is not clear about all the resources that they are using under the hood. In a way is good and bad. Good because it takes away most complexities and bad well because you can't see what the hell is going on.
How can I backup my current volume if every time I take a snapshot it
uses old corrupted data
You can either stop your instance before taking the volume snapshot or you can just create an AMI (Image) with making sure that the no reboot option is not checked.
See below:
How to replace a Amazon EC2 instance with a new instance without removing tomcat7, mysql and other setting done on the instance?
Simple:
Stop the instance.
Detach the volume(s) - if you just have one volume (the boot volume) detach that.
Take a snapshot of the volume(s) for safekeeping (not strictly necessary in this case, but no-one ever got fired for taking backups of critical systems).
Terminate the instance.
Spin-up a new instance of the desired type (must be the same platform, i.e. x86 or x64, as the original).
Wait until it's running, then stop it.
Detach its' volume and delete it.
Attach your original volumes to it (the boot volume must be /dev/sda1).
Restart the new instance and re-assign Elastic IP if appropriate.
What is the different between starting an AWS Image and Instances?
Example:
I do notice when I am running AWS image using boto, I can only stop the Image while running AWS instance using boto, I can only terminate.
Think of an EC2 instance as a single running server with CPU, memory, hard disk, networking, etc. Any changes you make to that instance affect only that instance.
Think of an AMI (Amazon Machine Image) as an exact copy of the root file system that gets copied to the hard disk when you start a new instance. The AMI is a hard disk sitting on a shelf. You make an exact copy of the hard disk on the shelf, install the new hard disk in a server, and turn the server on. You can do this for as many servers as you'd like to start without affecting the master copy.
The AMI defines the initial state of each instance. Each instance changes as it runs, but you can never change the original AMI once it has been created (other than to delete it).
There are more details that refine this conceptual model, but that's the basics.
Specific to the wording in your question:
Sometimes we say we're "starting an AMI" sometimes we say we're "starting an instance". We mean the same thing. We're really starting an instance using an AMI as the template.
We never say we're "stopping/terminating an image" or "stopping an ami" as, once started, it's really the instance that's running.
You can have one or more instances running that are derived from an image (AMI). Here is a good little tutorial, that's a bit old mind you, talking about how you can convert an Instance to an AMI ... which you can then redeploy one or many times:
http://webkist.wordpress.com/2010/03/16/creating-an-amazon-ec2-ebs-ami-from-a-running-instance/
What is an AMI: Amazon Machine Image
http://en.wikipedia.org/wiki/Amazon_Machine_Image
Technically, you can't start an AMI. You can start an instance that is derived from an AMI.
If I stop an EC2 instance, rather than terminate, will this image be saved in my account and be available to be used at a later stage? ..as I noticed terminated instances eventually dissappear, would a stopped instance be available to boot up and start when ever?
Also regarding charging, I assume the cost for having a stopped instance would be charged per GB similar to a custom AMI?
Also another possibly simple question, if I shutdown a machine over SSH or via a script, does this initiate a termination or just a stop an instance (I assume it terminates the instance).
Thanks
If you terminate the EBS backed instance, it will remove it from the list of running instance, including it's allocated EBS volume. Unless you set the instance attribute not to delete the volume.
If you only stop, it will changed to stopped status and you can start it again later.
If you shutdown a machine, it default's to stop.
A good read to protect your instance see: http://alestic.com/2010/01/ec2-instance-locking
If you stop an instance based on EBS, then the instance will terminate automatically but you'll be charged for EBS storage until you delete the EBS.
I have an EC2 instance with "instance store" device as a root device (I did not know the difference between it and EBS before launching it). I would like to stop it but I can not do it with the command ec2-stop-instances, the output is:
Client.UnsupportedOperation: The instance 'i-XXXXXXXX' does not have an 'ebs' root device type and cannot be stopped.
Does anybody know how to stop it with Windows Console (I am not the owner of the Amazon account and I won't be able to contact with him for weeks)
Thanks in advance.
EC2 instance with "instance store" cannot be stopped. They can only be terminated.
If you would like the ability to stop an instance, I suggest recreating the instance with an EBS root device. These types of instances support the ec2-stop-instances command. You can consider this command as a "pause", since it can be resumed at any time, and you will not be charged for the time the instance is stopped.
Amazon announced the ability to boot instances from an EBS root volume only in December 2009, so you will find that older documentation and tutorials ignore the extended command-set that came with the EBS root volumes.
Further reading: Amazon EC2 Instances Now Can Boot from Amazon EBS
I am not the owner of the Amazon
account and I won't be able to contact
with him for weeks.
If you really want to "pause" this instance for a few weeks, you can create a machine image (AMI) of your instance and terminate it.
You will then be able to launch a new instance with your private AMI when you want to "resume" this instance. It will not be the same instance, as in it will have a different instance ID and a different IP, but you will be cloning the setup of your instance.
The methods to build an AMI differ if your instance is running Windows or Linux, but you should be able to find adequate information on the web about both scenarios.