I've been playing around with EC2 Autoscaling but was wondering how do I make changes to these instances once launched? Do I need to create a new AMI then re-launch everything?
Use Puppet or Chef to rapidly change production settings and then rotate the AMIs from time to time to keep AMIs updated so your configuration management app does not need to apply too much changes on startup.
It depends what types of updates you are trying to do.
Setting up a new AMI is certainly one way to go. I do this when i need to make changes to the system configuration.
Use some type of automated release process or tools. There are a lot of ways to do do this and it will likely depend on your application. But you can set up a process which would update your application or other system configuration as needed without relaunching a new AMI.
Related
My organization's website is a Django app running on front end webservers + a few background processing servers in AWS.
We're currently using Ansible for both :
system configuration (from a bare OS image)
frequent manually-triggered code deployments.
The same Ansible playbook is able to provision either a local Vagrant dev VM, or a production EC2 instance from scratch.
We now want to implement autoscaling in EC2, and that requires some changes towards a "treat servers as cattle, not pets" philosophy.
The first prerequisite was to move from a statically managed Ansible inventory to a dynamic, EC2 API-based one, done.
The next big question is how to deploy in this new world where throwaway instances come up & down in the middle of the night. The options I can think of are :
Bake a new fully-deployed AMI for each deploy, create a new AS Launch config and update the AS group with that. Sounds very, very cumbersome, but also very reliable because of the clean slate approach, and will ensure that any system changes the code requires will be here. Also, no additional steps needed on instance bootup, so up & running more quickly.
Use a base AMI that doesn't change very often, automatically get the latest app code from git upon bootup, start webserver. Once it's up just do manual deploys as needed, like before. But what if the new code depends on a change in the system config (new package, permissions, etc) ? Looks like you have to start taking care of dependencies between code versions and system/AMI versions, whereas the "just do a full ansible run" approach was more integrated and more reliable. Is it more than just a potential headache in practice ?
Use Docker ? I have a strong hunch it can be useful, but I'm not sure yet how it would fit our picture. We're a relatively self-contained Django front-end app with just RabbitMQ + memcache as services, which we're never going to run on the same host anyway. So what benefits are there in building a Docker image using Ansible that contains system packages + latest code, rather than having Ansible just do it directly on an EC2 instance ?
How do you do it ? Any insights / best practices ?
Thanks !
This question is very opinion based. But just to give you my take, I would just go with prebaking the AMIs with Ansible and then use CloudFormation to deploy your stacks with Autoscaling, Monitoring and your pre-baked AMIs. The advantage of this is that if you have most of the application stack pre-baked into the AMI autoscaling UP will happen faster.
Docker is another approach but in my opinion it adds an extra layer in your application that you may not need if you are already using EC2. Docker can be really useful if you say want to containerize in a single server. Maybe you have some extra capacity in a server and Docker will allow you to run that extra application on the same server without interfering with existing ones.
Having said that some people find Docker useful not in the sort of way to optimize the resources in a single server but rather in a sort of way that it allows you to pre-bake your applications in containers. So when you do deploy a new version or new code all you have to do is copy/replicate these docker containers across your servers, then stop the old container versions and start the new container versions.
My two cents.
A hybrid solution may give you the desired result. Store the head docker image in S3, prebake the AMI with a simple fetch and run script on start (or pass it into a stock AMI with user-data). Version control by moving the head image to your latest stable version, you could probably also implement test stacks of new versions by making the fetch script smart enough to identify which docker version to fetch based on instance tags which are configurable at instance launch.
You can also use AWS CodeDeploy with AutoScaling and your build server. We use CodeDeploy plugin for Jenkins.
This setup allows you to:
perform your build in Jenkins
upload to S3 bucket
deploy to all the EC2s one by one which are part of the assigned AWS Auto-Scaling group.
All that with a push of a button!
Here is the AWS tutorial: Deploy an Application to an Auto Scaling Group Using AWS CodeDeploy
I have TeamCity (7.0.2) successfully spinning up an EC2 VM from a custom AMI, running our build, and sending back the build artifacts.
However, even when I used to do this with older TeamCity versions, I was always unhappy with the notion that it simply terminates the instances after they are done, and then creates new instances using the configured AMI next time a build agent is needed.
Can I get TeamCity to issue "stop" commands instead, followed by "start" commands? This has a tonne of advantages - quicker spin-up time, allowing for named instances in the agent stats, and saving the Mercurial clone to EBS for the next build are just three.
p.s. I guess I could use chained builds to call the EC2 API directly rather than use the in-built cloud support, but that sounds like a lot of work and feels flaky
We plan to provide support of EBS instances start stop in TeamCity 7.1
Please vote for TW-16419
TeamCity 7.0 may leak EBS volumes TW-12517
I am trying to understand how Amazon implements the auto scaling feature. I can understand how it is triggered but I don't know what exactly happens during the auto scaling. How does it expand. For instance,
If I set the triggering condition as cpu>90. Once the vm's cpu usage increases above 90:
Does it have a template image which will be copied to the new machine and started?
How long will it take to start servicing the new requests ?
Will the old vm have any downtime ?
I understand that it has the capability to provide load balancing between the VMs. But, I cannot find any links/paper which explains how Amazon auto scaling works. It will be great if you can provide me some information regarding the same. Thank you.
Essentially, in the set up you register an AMI, and a set of EC2 start parameters - a launch configuration (Instance size, userdata, security group, region, availability zone etc) You also set up scaling policies.
Your scaling trigger fires
Policies are examined to determine which launch configuration pplies
ec2 run instance is called with the registered AMI and the launch configuration parameters.
At this point, an instance is started which is a combination of the AMI and the launch configuration. It registers itself with an IP address into the AWS environment.
As part of the initial startup (done by ec2config or ec2run - going from memory here) - the newly starting instance can connect to instance meta data and run the script stored in "userdata". This script can bootstrap software installation, operating system configuration, settings, anything really that you can do with a script.
Once it's completed, you've got a newly created instance.
Now - if this process was kicked off by auto-scale and elastic-load-balancing, at the point that the instance is "Windows is ready" (Check ec2config.log), the load balancer will add the instance to itself. Once it's responding to requests, it will be marked healthy, and the ELB will start routing traffic.
The gold standard is to have a generic AMI, and use your bootstrap script to install all the packages / msi's / gems or whatever you need onto the server. But what often happens is that people build a golden image, and register that AMI for scaling.
The downside to the latter method is that every release requires a new AMI to be created, and the launch configurations to be updated.
Hope that gives you a bit more info.
may be this can help you
http://www.cardinalpath.com/autoscaling-your-website-with-amazon-web-services-part-2/
http://www.cardinalpath.com/autoscaling-your-website-with-amazon-web-services-part-1/
this post helped me achiving this
Have a read of this chaps blog, it helped me when I doing some research on the subject.
http://www.codebelay.com/blog/2009/08/02/how-to-load-balance-and-auto-scale-with-amazons-ec2/
I was searching for EC2 EBS storage Centos 5.4 AMI in the community AMI and eventually I found Rightscale AMI (I think they called it RightImage).
Now I have created instance using that AMI, but I found out there is some Rightscale stuff inside which is worrying me about the safety on using it. I found out there are the following files in that AMI:
/etc/init.d/rightimage
/etc/init.d/rightlink
/etc/init.d/rightscale
/home/ec2
/home/s3sync
(may be more other files I haven't found out yet)
I know I can look into the script and folder and see what they do, but since a lot of user here recommended using Rightscale Centos AMI in EC2, I hope may be there is already some gurus here know what those mentioned script and folder doing and could advice me
i)whether is it safe to delete them. (I'm more concern on whether my data in the server will be safe by using this AMI)
ii)any installed apps in RightScale AMI that should be deleted
And if you think there is other free EC2 Centos AMI that is secure and solid, do suggest as well, thanks !
In order for RightScale to properly manage instances in ec2 they use a ruby based daemon called RightLink as a communication device between their core platform and each instance that is launched. The init scripts that you saw are required for the instance to self configure itself to the point where it can be managed by RightScale properly.
/etc/init.d/rightimage is the first script that is run. Essentially it just determines the OS, arch version, and installs the correct RightLink package from the S3 bucket. Afterwards it kicks off the /opt/rightscale/bin/post_install.sh script which uses the OS init control tools to register the startup scripts to be invoked on future boots of the OS; this ensures that RightLink will always be started.
/etc/init.d/rightscale is the next script that is run. It initializes RightScale-specific (but not RightLink-specific) system state. It is responsible for caching launch settings (aka userdata) and metadata in /var/spool and installing any available patches to the RightLink agent.
/etc/init.d/rightlink is the final script that is run. It configures and enrolls the RightLink agent idempotently. If configuration and enrollment succeed, rightlink starts the sandboxed monit which starts the persistent agent process. If you're not launching the AMI using the RightScale platform this will never properly enroll because they aren't expecting it to, as such RightScale will have no communication with the instance at all.
Removing all three of these from the image shouldn't in any way harm the overall stability of the image, but from a security standpoint they shouldn't cause any problems if they are present.
If you have any further specific questions about it I'd suggest hopping on their forums at https://forums.rightscale.com/
You could also try #rightscale on freenode.
When autoscaling my EC2 instances for application, what is the best way to keep every instances in sync?
For example, there are custom settings and application files like below...
Apache httpd.conf
php.ini
PHP source for my application
To get my autoscaling working, all of these must be configured same in each EC2 instances, and I want to know the best practice to sync these elements.
You could use a private AMI which contains scripts that install software or checkout the code from SVN, etc.. The second possibility to use a deployment framework like chef or puppet.
The way this works with Amazon EC2 is that you can pass user-data to each instance -- generally a script of some sort to run commands, e.g. for bootstrapping. As far as I can see CreateLaunchConfiguration allows you to define that as well.
If running this yourself is too much of an obstacle, I'd recommend a service like:
scalarium
rightscale
scalr (also opensource)
They all offer some form of scaling.
HTH