I am trying to set up a project and my BeforeInstall scripts are failing. I am working on this problem, but it creates a side effect that every time I attempt to deploy I get new instances. I'm up to 22. When I try to delete instances they reappear later. Something is forcing them to stay alive.
I'm not fully familiar with what is going on here, but my theory is that CodeDeploy creates a new AutoScaling group for each deploy, and those are configured to have 3 instances.
So my question is:
Is the above what is going on?
Why does CodeDeploy create a new autoscaling group each time?
If (2) is the correct behavior, why doesn't it delete the old group OR why doesn't it delete the new group when the deploy fails.
Related to (3) do I have something configured in correctly?
Thanks for any help.
"Why does CodeDeploy create a new autoscaling group?" - CodeDeploy does not create a new autoscaling group unless you are creating a blue green deployment with Automatically copy Auto Scaling group option selected. The behavior you are getting into is because if you use CodeDeploy once to deploy to your instances in an Autoscaling group, there is a hook that gets established between CodeDeploy and Auto Scaling group and whenever your deployment to any of the instances fail Austoscaling marks that instance as a bad instance and spins up a new instance and notifies CodeDeploy to deploy the last successful revision to that instance. For more details refer http://docs.aws.amazon.com/codedeploy/latest/userguide/integrations-aws-auto-scaling.html.
To resolve this issue, you can either set the autoscaling group desired value to 0 to terminate all instances or suspend the scaling policy.
Related
I'm trying to run the S3->DynamoDB example and having some problems running the EMR cluster that is created for the MyImportJob activity.
We configured our IAM accounts such that every user can create EC2 machines with a specific 'team_id' tag (of his team). That helps us control the resources, prevent mistakes and monitor usage.
When Pipeline tries to launch the EMR cluster, it (probably) does it without the tags and therefore it fails with Terminated with errors: User account is not authorized to call EC2. I tried to find a configuration in the EMRCluster resource but couldn't find anything that will help me set that. I'm pretty sure that it fails because of the tags policy.
Any idea how I can overcome this?
Does it help if If create a CloudFormation template for that? Do I have more control there? (I'm going to create the pipeline as a part of the application template anyway, just wanted to experience the product before).
Thanks!
I could not find a solution for how to add tags to EMR(and how to set it to be visible to all users) so I have created a python script to run as bootstrap action. If its still relevant you can find it here
I have an AMI which has configured with production code setup.I am using Nginx + unicorn as server setup.
The problem I am facing is, whenever traffic goes up I need to boot the instance log in to instance and do a git pull,bundle update and also precompile the assets.Which is time consuming.So I want to avoid all this process.
Now I want to go with a script/process where I can automate whole deployment process, like git pull, bundle update and precompile as soon as I boot a new instance from this AMI.
Is there any best way process to get this done ? Any help would be appreciated.
You can place your code in /etc/rc.local (commands in this file will be executed when server will be loaded).
But the best way is using (capistrano). You need to add require "capistrano/bundler" to your deploy.rb file, and bundle update will be runned automatically. For more information you can read this article: https://semaphoreapp.com/blog/2013/11/26/capistrano-3-upgrade-guide.html
An alternative approach is to deploy your app to a separate EBS volume (you can still mount this inside /var/www/application or wherever it currently is)
After deploying you create an EBS snapshot of this volume. When you create a new instance, you tell ec2 to create a new volume for your instance from the snapshot, so the instance will start with the latest gems/code already installed (I find bundle install can take several minutes). All your startup script needs to do is to mount the volume (or if you have added it to the fstab when you make the ami then you don't even need to do that). I much prefer scaling operations like this to have no dependencies (eg what would you do if github or rubygems have an outage just when you need to deploy)
You can even take this a step further by using amazon's autoscaling service. In a nutshell you create a launch configuration where you specify the ami, instance type, volume snapshots etc. Then you control the group size either manually (through the web console or the api) according to a fixed schedule or based on cloudwatch metrics. Amazon will create or destroy instances as needed, using the information in your launch configuration.
I have an AWS account with 14 instances and using scalr. I added the Api reference details and it showed up, at that time instances were pretty low. As and when I keep adding new instances it accepted few and reject the rest. Now I have an instance newly made on AWS which is not getting loaded in scalr.
Any ideas?
Instances that you create using AWS will not show up in Scalr.
Instead, you create Farms (in Scalr) through the use of custom and/or pre-configured Scalr Roles. When you launch those farms/roles, it will launch the required instances in AWS. It's like a wrapper around AWS that provides extra features, but it will only ever know about instances that have been launched from a Scalr role.
It is possible to import an existing server into Scalr although it involves installing the scalarizr software onto that server and opening some ports. Full details can be found here. Once complete, you'll have a new role that you can add to a farm and then launch.
Say I've got a PHP site on an ec2 instance in an autoscale group. The group has 10 running instances. I want to add some new php code to the site. Is there a sane way to do this and have it populate to all other instances in the autoscaling group?
The only way I can see is to manually bundle an image from the instance, upload it to S3, register the bundle with aws and attach the bundle to the launch config of the autoscale group. I then have to terminate all my instances and launch new ones with the new image.
I would have to do all this every time I want to make a change to the codebase. Surely this is madness.
The best way is NOT to bundle your page into the image. Make the ami download them from s3 on start. In this way you can upload to only one place and call the download script.
Is it possible to clone a EC2 instance data and all?
You can make an AMI of an existing instance, and then launch other instances using that AMI.
The easier way is through the web management console:
go to the instance
select the instance and click on instance action
create image
Once you have an image you can launch another cloned instance, data and all. :)
There is no explicit Clone button. Basically what you do is create an image, or snapshot of an existing EC2 instance, and then spin up a new instance using that snapshot.
First create an image from an existing EC2 instance.
Check your snapshots list to see if the process is completed. This usually takes around 20 minutes depending on how large your instance drive is.
Then, you need to create a new instance and use that image as the AMI.
Nowadays it is even easier to clone the machine with EBS-backed instances released a while ago. This is how we do it in BitNami Cloud Hosting.
Basically you just take a snapshot of the instance which can be used
later to launch a new server. You can do it either using AWS console
(saving the EBS-backed instance as AWS AMI) or using the EC2 API
tools:
create a snapshot with ec2-create-snapshot
and then launch an instance from a snapshot
Cloning the instance is nothing else but creating the backup and then
launching a new server based on that. You can find bunch of articles
out there describing this problem, try to find the info about "how to
..." backup or resize the whole EC2 instance, for example this blog is
a really good place to start: alestic.com
To Answer your question: now AWS make cloning real easy see Launch instance from your Existing Instance
On the EC2 Instances page, select the instance you want to use
Choose Actions --> Image and Templates, and then Launch More Like This.
Review & Launch
This will take the existing instance as a Template for the new once.
or you can also take a snapshot of the existing volume and use the snapshot with the AMI (existing one) which you ping during your instance launch
You can use AWS API or console UI to create an AMI(Amazon Machine Image) of your running instance. You can specify to reboot the instance when create your AMI. Then you can use AWS API or console UI to launch more instances with the AMI you created.
You can do it very easily with a Cloud Management software -like enStratus, RightScale or Scalr (disclaimer: I work there). With the cloned farm you can:
Create a snapshot or a pre-made image to launch another day
Duplicate your configuration to test it before production