I have 3 VMs in my Azure autoscale set. My question is if I stop one of the VMs using the Azure Portal manually (i.e. deallocate it) will the AutoScale Set be able to start it again?
I have been through this but it still doesn't answer my question.
Nope. Autoscale will not start a deallocated VM in a scale set :)
Instead of starting the deallocated VM, if usage was high enough to trigger scale-out, Autoscale would create a new VM.
The deallocated VM would remain deallocated until you started it, issued a start to the scale set, or until it was deleted by autoscale scaling in.
Related
I currently have a try-out environment with ~16 services divided over 4 micro-instances. Instances are managed by an autoscaling group (ASG). When I need to update the AMI of my cluster instances, currently I do:
Create new launch config, edit ASG with new launch config.
Detach all instances with replacement option from the ASG and wait until the new ones are listed in the cluster instance list.
MANUALLY find and deregister the old instances from the ECS cluster (very tricky)
Now the services are killed by ECS due to deregistering the instances :(
Wait 3 minutes until the services are restarted on the new instances
MANUALLY find the EC2 instances in the EC2 instance list and terminate them (be very very careful not to terminate the new ones).
With this approach I have about 3 minutes of downtime and I shiver from the idea to do this in production envs.. Is there a way to do this without downtime but keeping the overall amount of instance the same (so without 200% scaling settings etc.).
You can update the Launch Configuration with the new AMI and then assign it to the ASG. Make sure to include the following in the user-data section:
echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config
Then terminate one instance at a time, and wait until the new one is up and automatically registered before terminating the next.
This could be scriptable to be automated too.
I'm creating an AMI for a server with 100G files. It's been like an hour and it's still not finished. (The AMI still says pending) Is there something wrong with it? What should I do?
Just to let other people know, this process could take very, very long. My 100 GB AMI takes like 2.5 hours to create and the progress bar jumps from 0 to 100 directly after that. So don't worry.
This answer pertains to a retirement situation where there is potentially a hardware failure for an instance with a EBS root volume. In this situation, AWS recommends creating an AMI of the instance to be retired, and starting a new instance from this AMI. In my situation, the instance was not responding before starting the AMI creation process.
The AMI creation step failed to complete after >8 hours.
I next tried stopping the instance. This also failed to complete in >10 minutes, so I tried force stopping the instance (by issuing the stop command again). The force stop did complete after a few minutes. After the instance was stopped, the AMI creation succeeded in <10 minutes.
This AWS forum message seemed relevant to my case:
https://forums.aws.amazon.com/thread.jspa?messageID=372982񛃶
It fully depends on the disk size that is attached to the es2 instance.
But one can check the progress of the AMI backup job by checking the EBS snapshot, as it will also take a snapshot of the attached EBS. Once taking a snapshot is completed, it will show Available(100%), before that it will show Unavailable(X%). Once it's become Available(100%), AMI backup will complete.
Here are some image of the snapshot progress.
Snapshot progress
I am trying to understand how Amazon implements the auto scaling feature. I can understand how it is triggered but I don't know what exactly happens during the auto scaling. How does it expand. For instance,
If I set the triggering condition as cpu>90. Once the vm's cpu usage increases above 90:
Does it have a template image which will be copied to the new machine and started?
How long will it take to start servicing the new requests ?
Will the old vm have any downtime ?
I understand that it has the capability to provide load balancing between the VMs. But, I cannot find any links/paper which explains how Amazon auto scaling works. It will be great if you can provide me some information regarding the same. Thank you.
Essentially, in the set up you register an AMI, and a set of EC2 start parameters - a launch configuration (Instance size, userdata, security group, region, availability zone etc) You also set up scaling policies.
Your scaling trigger fires
Policies are examined to determine which launch configuration pplies
ec2 run instance is called with the registered AMI and the launch configuration parameters.
At this point, an instance is started which is a combination of the AMI and the launch configuration. It registers itself with an IP address into the AWS environment.
As part of the initial startup (done by ec2config or ec2run - going from memory here) - the newly starting instance can connect to instance meta data and run the script stored in "userdata". This script can bootstrap software installation, operating system configuration, settings, anything really that you can do with a script.
Once it's completed, you've got a newly created instance.
Now - if this process was kicked off by auto-scale and elastic-load-balancing, at the point that the instance is "Windows is ready" (Check ec2config.log), the load balancer will add the instance to itself. Once it's responding to requests, it will be marked healthy, and the ELB will start routing traffic.
The gold standard is to have a generic AMI, and use your bootstrap script to install all the packages / msi's / gems or whatever you need onto the server. But what often happens is that people build a golden image, and register that AMI for scaling.
The downside to the latter method is that every release requires a new AMI to be created, and the launch configurations to be updated.
Hope that gives you a bit more info.
may be this can help you
http://www.cardinalpath.com/autoscaling-your-website-with-amazon-web-services-part-2/
http://www.cardinalpath.com/autoscaling-your-website-with-amazon-web-services-part-1/
this post helped me achiving this
Have a read of this chaps blog, it helped me when I doing some research on the subject.
http://www.codebelay.com/blog/2009/08/02/how-to-load-balance-and-auto-scale-with-amazons-ec2/
We have recently been experimenting with a micro instance to undertake a specific task. Last night, for no apparent reason, the EBS backed micro instance failed with no warning. We can no longer access the server through the Terminal (MacBook) and Monitoring show 100% utilisation (flat). We took a snapshot and created a new volume, but cannot attach this to the new micro instance, as its EBS backed. We are now in the process of setting up a new micro instance from scratch which is taking a lot of effort due to having to re-install software and configure them for our purpose
What are the best practices for running EBS backed micro instances? How can we avoid the same happening again?
You can certainly attach a new EBS boot volume to an EBS-backed instance:
Stop the instance (NOT terminate).
Snapshot its' boot volume.
Create a new volume from the snapshot.
Detach the original boot volume, and attach your new one.
Start the instance.
However, in your case, you don't know why the instance failed, so how do you know that a new boot volume created from a snapshot of the failed one will work? Did you try rebooting the instance? It sounds as though the OS might have panicked/bluescreened (you don't mention which OS is installed) or possibly just the SSH session daemon died, in which case a simple reboot might be all that is needed.
If I stop an EC2 instance, rather than terminate, will this image be saved in my account and be available to be used at a later stage? ..as I noticed terminated instances eventually dissappear, would a stopped instance be available to boot up and start when ever?
Also regarding charging, I assume the cost for having a stopped instance would be charged per GB similar to a custom AMI?
Also another possibly simple question, if I shutdown a machine over SSH or via a script, does this initiate a termination or just a stop an instance (I assume it terminates the instance).
Thanks
If you terminate the EBS backed instance, it will remove it from the list of running instance, including it's allocated EBS volume. Unless you set the instance attribute not to delete the volume.
If you only stop, it will changed to stopped status and you can start it again later.
If you shutdown a machine, it default's to stop.
A good read to protect your instance see: http://alestic.com/2010/01/ec2-instance-locking
If you stop an instance based on EBS, then the instance will terminate automatically but you'll be charged for EBS storage until you delete the EBS.