I am exploring the possibility of modifying https://github.com/riptano/ComboAMI to support Ec2MultiRegionSnitch.
In that:
Add option --snitch Ec2MultiRegionSnitch -> modify cassandra.yaml to write snitch as multi region
Add option --broadcast_address_as_public_ip yes -> modify cassandra.yaml to write broadcast_address: public_ip
Add option --seeds 100.222.111.222, so as the newly created instances can join an existing cassandra, e.g. 100.222.111.222.
Tested the settings and worked.
The restrictions
I can't copy the datastax ami to be my own ami.
I can't snapshot an existing datastax cassandra instance into an AMI, such that I modify the script locally to get it launched.
The question:
How to modify the script and test it out.
Should I use AutoScalingGroup with a Launchconfiguration to point to this AMI, then use sed to modify the cassandra.yaml, service restart cassandra instead? It is not obvious to me how to run a script after the AWS launch configuration has completed launching the instance, especially I can't get the AWS::Instances::GetAtt PublicIP for the broadcast address. Ideally speaking the changes should have been done during cassandra.yaml construction in the script, not after.
Thanks!
That's correct, the AMI has to be rebuilt on a clean image under your account. We have instructions here on how to do so:
https://github.com/riptano/ComboAMI/blob/2.5/presetup/setup.md
As far as the the AutoScalingGroup question, I'm not sure how beneficial that would be. If you create your own image, off your own repo, feel free to create a pull request and I'll look them over to merge them into the official AMI.
Related
I've been playing around with EC2 Autoscaling but was wondering how do I make changes to these instances once launched? Do I need to create a new AMI then re-launch everything?
Use Puppet or Chef to rapidly change production settings and then rotate the AMIs from time to time to keep AMIs updated so your configuration management app does not need to apply too much changes on startup.
It depends what types of updates you are trying to do.
Setting up a new AMI is certainly one way to go. I do this when i need to make changes to the system configuration.
Use some type of automated release process or tools. There are a lot of ways to do do this and it will likely depend on your application. But you can set up a process which would update your application or other system configuration as needed without relaunching a new AMI.
My organization's website is a Django app running on front end webservers + a few background processing servers in AWS.
We're currently using Ansible for both :
system configuration (from a bare OS image)
frequent manually-triggered code deployments.
The same Ansible playbook is able to provision either a local Vagrant dev VM, or a production EC2 instance from scratch.
We now want to implement autoscaling in EC2, and that requires some changes towards a "treat servers as cattle, not pets" philosophy.
The first prerequisite was to move from a statically managed Ansible inventory to a dynamic, EC2 API-based one, done.
The next big question is how to deploy in this new world where throwaway instances come up & down in the middle of the night. The options I can think of are :
Bake a new fully-deployed AMI for each deploy, create a new AS Launch config and update the AS group with that. Sounds very, very cumbersome, but also very reliable because of the clean slate approach, and will ensure that any system changes the code requires will be here. Also, no additional steps needed on instance bootup, so up & running more quickly.
Use a base AMI that doesn't change very often, automatically get the latest app code from git upon bootup, start webserver. Once it's up just do manual deploys as needed, like before. But what if the new code depends on a change in the system config (new package, permissions, etc) ? Looks like you have to start taking care of dependencies between code versions and system/AMI versions, whereas the "just do a full ansible run" approach was more integrated and more reliable. Is it more than just a potential headache in practice ?
Use Docker ? I have a strong hunch it can be useful, but I'm not sure yet how it would fit our picture. We're a relatively self-contained Django front-end app with just RabbitMQ + memcache as services, which we're never going to run on the same host anyway. So what benefits are there in building a Docker image using Ansible that contains system packages + latest code, rather than having Ansible just do it directly on an EC2 instance ?
How do you do it ? Any insights / best practices ?
Thanks !
This question is very opinion based. But just to give you my take, I would just go with prebaking the AMIs with Ansible and then use CloudFormation to deploy your stacks with Autoscaling, Monitoring and your pre-baked AMIs. The advantage of this is that if you have most of the application stack pre-baked into the AMI autoscaling UP will happen faster.
Docker is another approach but in my opinion it adds an extra layer in your application that you may not need if you are already using EC2. Docker can be really useful if you say want to containerize in a single server. Maybe you have some extra capacity in a server and Docker will allow you to run that extra application on the same server without interfering with existing ones.
Having said that some people find Docker useful not in the sort of way to optimize the resources in a single server but rather in a sort of way that it allows you to pre-bake your applications in containers. So when you do deploy a new version or new code all you have to do is copy/replicate these docker containers across your servers, then stop the old container versions and start the new container versions.
My two cents.
A hybrid solution may give you the desired result. Store the head docker image in S3, prebake the AMI with a simple fetch and run script on start (or pass it into a stock AMI with user-data). Version control by moving the head image to your latest stable version, you could probably also implement test stacks of new versions by making the fetch script smart enough to identify which docker version to fetch based on instance tags which are configurable at instance launch.
You can also use AWS CodeDeploy with AutoScaling and your build server. We use CodeDeploy plugin for Jenkins.
This setup allows you to:
perform your build in Jenkins
upload to S3 bucket
deploy to all the EC2s one by one which are part of the assigned AWS Auto-Scaling group.
All that with a push of a button!
Here is the AWS tutorial: Deploy an Application to an Auto Scaling Group Using AWS CodeDeploy
I have ec2 instance set up where mysql DB is mounted on separate volume.
(as detailed in http://aws.amazon.com/articles/1663 )
I want to duplicate this instance set up where my application servers on duplicated instances share the DB volume which is attached to the already running ec2 instance.(I can specify mysql ip through configuration file)
Since almost every set up except the mysql ip is identical, i'd like to create an ami from the first instance and slightly modify to create 2nd,3rd instances.
The question is, the mount information stored in the first instance will take effect when I launch the 2nd instance.
I can elaborate the question,
1. I read that a volume can not be attached to more than one ec2 instance at the same time.
2. the running instance attaches/mount an volume to itself on start up.(so it seems)
3. if I were to create an ami from first instance and use that to initiate other instances, how would auto attach/mount information(which I assume, will be stored in the ami) will affect the other instances.
Eugene,
Mounting the same device to several servers is not possible, so you better forget about this option.
The best solution is to:
Create a copy of your master instance.
Detach the created mount volume. We are going to create an image from this new instance, and you don't want the useless drive copy to be re-created every time.
Change the settings that you need to change, in order to make this server rely on the remote (master) mysql server.
Once you are satisfied with the outcome, create an image from this instance.
Good luck!
Dotan
As I want to install Jenkins (ex-Hudson) to operate my continuous integration processes on AWS Beanstalk, I need a custom AMI because some parameters in Tomcat & Linux have to be changed for Jenkins
I run the process of installing and customizing the instance started initially by Beanstalk until the end and Jenkins works like a charm on it.
But, what I can't do is reuse the AMI that I generated at the end of my customization: the health check done by BeansTalk doesn't see the EC2 instance although Beanstalk started it and it works fine.
In order to understand my issue, I reduced my failing process to the following:
a) I create a new BT application / environment based on sample provided by Amazon (only parameter that I had is a keypair to SSH my EC2 instance)
b) when the EC2 instance is started, I use the EC2 to flash the AMI
c) I modify the BT env config by changing the original AWS Ami (id: 100fff79 - Tomcat 6 64 bits) by the 1 that I genrated in (b)
d) the BT rebuilds when I change the ami id
e) the rebuild restarts the EC2 instance.
f) It starts fine (can ssh to it) but the health checking fails and my env turns to red status.
Can somebody replicate this process and tell me what I am doing wrong ?
(I would like to use the AMI of (b) as starting point for my Jenkins customization.?
Additional info that I can provide:
when ssh-ing to the EC2 instance, a grep for apache, java, thin & bluepilld as described at bottom of https://forums.aws.amazon.com/thread.jspa?threadID=59027&tstart=25 shows that the 4 expected processes disappeared. Hence, the failure.
Please, help !
regards
didier
will answer my own question: the right way to obtain a working customized ami for Beanstalk is not to try to flash a running instance launched by Beanstalk but rather start the template ami for Beanstalk (ami-100fff79 for Tomcat 6 64 bits in my case) from EC2 console and customize it from there, flash it and you're done.
You can then "edit configuration" for your BT environment by changing the ami to the new one and it works fine.
regards
didier
If you give more details, this is a feature I'm planning for version 0.3.0 of Beanstalker, my set of Maven plugins for automating maven deployments to Elastic Beanstalk and Elastic MapReduce. It is available at http://beanstalker.ingenieux.com.br/
Actually, the placeholders are there, but I haven't still done full testing of that. Are you willing to try and give help and advice?
You should be able to create a customized AMI from a running instance as long as you delete /opt/elasticbeanstalk/srv/hostmanager/db/hostmanager.db on the instance before building the new AMI. I keep seeing people say "it can't be done, you need to start a clean instance outside of Elastic Beanstalk" and that's bunk. I've done it.
A full write-up of what I've done to customize my install is here: http://stormerider.com/blog/2012/08/16/building-an-ubuntu-ami-with-elastic-beanstalk-support/ -- some of it may not apply to you, some of it may.
When autoscaling my EC2 instances for application, what is the best way to keep every instances in sync?
For example, there are custom settings and application files like below...
Apache httpd.conf
php.ini
PHP source for my application
To get my autoscaling working, all of these must be configured same in each EC2 instances, and I want to know the best practice to sync these elements.
You could use a private AMI which contains scripts that install software or checkout the code from SVN, etc.. The second possibility to use a deployment framework like chef or puppet.
The way this works with Amazon EC2 is that you can pass user-data to each instance -- generally a script of some sort to run commands, e.g. for bootstrapping. As far as I can see CreateLaunchConfiguration allows you to define that as well.
If running this yourself is too much of an obstacle, I'd recommend a service like:
scalarium
rightscale
scalr (also opensource)
They all offer some form of scaling.
HTH