I am using aws sdk for auto scaling command line tool. I want to know that how to use cloud formation with this command line tool? i.e. can we give template id instead of instance/ ami id?
Example:
as-create-launch-config MyLC --image-id <CF tamplete id> --instance-type m1.small
can we give template id instead of instance/ ami id?
No, using Auto Scaling with AWS CloudFormation currently works the other way round only:
AWS CloudFormation fully supports all Auto Scaling resources like AWS::AutoScaling::AutoScalingGroup, AWS::AutoScaling::LaunchConfiguration etc., see the AWS Resource Types Reference for details on all available resources; there are a couple of Auto Scaling related AWS CloudFormation Sample Templates to get you started as well:
AutoScalingKeepAtNSample.template - An example of using Auto Scaling groups to manage a set of EC2 instances.
AutoScalingMultiAZSample.template - An example of using Load Balanced, Auto Scaling groups spanning multiple EC2 availability zones. This template has been updated to use Auto Scaling Policies.
AutoScalingMultiAZWithNotifications.template - An example of using Load Balanced Auto Scaling groups spanning multiple EC2 availability zones. This template uses Auto Scaling Policies and configures Auto Scaling notifications so that you can get emails when scaling events occur.
You can then use the AWS CloudFormation Command Line Tools to use your template(s) with said Auto Scaling resources, e.g. via cfn-create-stack, cfn-update-stack etc., see the Command Line Tools Reference for details on all available commands.
Related
I use the module, https://github.com/terraform-aws-modules/terraform-aws-ec2-instance to provision the ec2 instances on AWS. I would like to have AWS auto scaling group feature implemented, i.e. in case one of the ec2 instances is not healthy, AWS auto replaces it. But, I do not see the code, resource "aws_autoscaling_group" inside the above ec2 instance module.
Question:
Is it reasonable to implement AWS auto scaling group with the ec2 module? I understand we can implement AWS auto scaling group with eks. But, I am not sure about ec2 instances.
If it is possible, how to do it?
Yes, it is not only reasonable, but actually considered a best practice to use an auto scaling group for EC2 instances.
you can do so using the terraform autoscaling module: https://registry.terraform.io/modules/terraform-aws-modules/autoscaling/aws/latest
I'm trying to set up a way to monitor disk usage for instances belonging to an AutoScaling Group, and add an alarm when the volumes associated to the instances are almost full.
Since it seems there are no metrics normally offered by Amazon to do that, I resorted using the CloudWatch Agent to get what I wanted. So far so good, I can create graphs and alarms for the metrics I want using the CloudWatch console.
My issue is how to automate everything with CDK. How can I automate the creation of the metric for each instance, without knowing the instance id beforehand? Is there a solution for this issue?
You can install and config CloudWatch agent via EC2 user data and the auto scaling group uses launch template to launch EC2 instance. All of those things can be done by AWS CDK.
There is an example from this open source project for your reference.
Another approach you could take is using AWS Systems Manager. Essentially, you install an SSM agent for your instances, and create an SSM Document (think Shell/Python script) that will run your setup script/automation.
You then create a State Manager Association, tying the SSM Document with your instances based on EC2 tags e.g. Application=MyApp or Team=MyTeam. This way, you don't have to provide any resource ids, just the tag key value pair which could extend multiple instances and future instance replacements. You can schedule it to run at specific times (cron) or at a certain frequency (rate) to enforce state.
I have a requirement to launch multiple EC2 instances in the Tokyo region, based on the number of AMIs owned by our account in that same region. The AMIs are backed-up daily from another region.
What this CloudFormation needs to achieve is:
Retrieve a list of AMIs created today
Attempt to launch each of them in the same region
For example, if today there are 10 different AMIs created in the Tokyo region, then CloudFormation will then create 10 EC2 instances based on these 10 AMIs.
I have looked at some examples at Walkthrough: Looking Up Amazon Machine Image IDs - AWS CloudFormation but found the code does not suit the requirement.
I already have the Lambda function retrieve-today-ami.py, the challenge is to include them in the CF template found in Walkthrough: Looking Up Amazon Machine Image IDs - AWS CloudFormation
Normally, CloudFormation is used to launch pre-defined infrastructure. Your requirement to launch a variable number of instances with information that changes for each instance every day, does not match the model for using CloudFormation.
Based on your use-case, I would recommend writing a script to perform the operation you want.
For example, a Python scripts that lists the AMIs, identifies the ones you want to use, then launches EC2 instances using those AMIs.
You might be able to achieve this by using a Lambda-backed custom resource to fetch the names of the AMIs. Then, the outputs of your custom resource could be used in the EC2 stanzas in the template. You could have the one template defining the Lambda export the values and import them on your EC2 templates.
In all tutorials for ECS you need to create a cluster and after that an autoscaling group, that will spawn instances. Somehow in all these tutorials the instances magically show up in the cluster, but noone gives a hint what's connecting the autoscaling group and the cluster.
my autoscaling group spawns instances as expected, but they just dont show up on my ecs cluster, who holds my docker definitions.
Where is the connection I'm missing?
I was struggling with this for a while. The key to getting the instances in the autoscaling group associated with your ECS cluster is in the user data. When you are creating your launch config when you get to step 3 "Configure Details" hit the advanced tab and enter a simple bash script like the following for your user data.
#!/usr/bin/env bash
echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config
All the available parameters for agent configuration can be found here http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html
An autoscaling group is not strictly associated to a cluster. However, an autoscaling group can be configured such that each instance launched registers itself into a particular cluster.
Registering an instance into a cluster is the responsibility of the ECS Agent running on the instance. If you're using the Amazon ECS-optimized AMI, the ECS Agent will launch when the instance boots and register itself into the configured cluster. However, you can also use the ECS Agent on other Linux AMIs by following the installation instructions.
Well, i found out.
Its all about the ecs-agent and its config file /etc/ecs/ecs.config
(This file will be created through the Userdata field, when creating EC2 instances, even from an autoscaling configuration.)
Read about its configuration options here: http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html
But you can even copy a ecs.config stored on Amazon S3, do it like this (following lines go into Userdata field):
#!/bin/bash
yum install -y aws-cli
aws configure set default.s3.signature_version s3v4
aws configure set default.s3.addressing_style path
aws configure set default.region eu-central-1
aws s3 cp s3://<bucketname>/ecs.config /etc/ecs/ecs.config
note: Signature_version v4 is specific for some regions, like eu-central-1.
This ofc only works, if your IAM role for the instance (in my case its ecsInstanceRole) has the right AmazonS3ReadOnlyAccess
The AWS GUI console way for that would be:
Use the cluster wizard at https://console.aws.amazon.com/ecs/home#/firstRun .
It will create an autoscaling grou for your cluster, a loadbalancer in front of it, and connect it all nicely.
This question is old but the answer is not complete. There are 2 parts to getting your own auto-scaling group to show up in your cluster (as of Jan 2022).
You need to ensure your cluster name is set for ECS_CLUSTER variable in /etc/ecs/ecs.config as mentioned in this answer: https://stackoverflow.com/a/35324937/583875
You need to create a new capacity provider for the cluster and attach this auto scaling group. To do this, go to Cluster -> Capacity Provider -> Create -> Select your auto scaling group under Auto Scaling group.
Another tricky part is getting your service to use the instances (if you have a service running). You need to edit the Service, and change the Capacity provider strategy. Click on Add another provider and choose the new capacity provider you created in (2) above.
That's all! To ensure things are working properly: you should see your capacity provider under Graph -> Capacity Providers and you should see instances from your auto scaling group under Graph -> ECS Instances.
I just have created Amazon EC2 image and I'm new to this environment.
I'm interested in "auto scale" part of Amazon EC2.
But I could not find clear guide to find whether I'm using "auto scale" or not and how to auto scale my instance.
How can I setup "auto scale" properly and easily?
Here are some links that might help you setup auto-scaling:
http://kkpradeeban.blogspot.com/2011/01/auto-scaling-with-amazon-ec2.html
http://www.codebelay.com/blog/2009/08/02/how-to-load-balance-and-auto-scale-with-amazons-ec2/
You will need to download, unzip and setup the Auto Scaling Command Line Tool
You will need an AMI e.g. ami-xxxxxx
and a security group e.g. my-securitygroup-sg
and a key e.g. myKey
Now create a Launch Configuration, in this case called: my-launch-config-1
as-create-launch-config my-launch-config-1 --image-id ami-xxxxxx --region eu-west-1 --instance-type m1.small --group my-securitygroup-sg --key myKey
Then you can create the Auto Scaling Group
as-create-auto-scaling-group my-auto-scaling-group --region eu-west-1 --launch-configuration my-launch-config-1 --availability-zones eu-west-1a eu-west-1b eu-west-1c --min-size 3 --max-size 3 --desired-capacity 3 --default-cooldown 5 --grace-period 5 --tag "k=Name, v=my-servers, p=true" --tag "k=enabled, v=true, p=true"
This will create 3 instances base on the AMI, one in each zone
You can check on the progress of the creation of the Auto Scaling group using this command
as-describe-scaling-activities --auto-scaling-group my-auto-scaling-group --region eu-west-1
You can find more useful commands for things like deleting or updating Auto Scaling Group in my blog post:
How to use Amazon’s Auto Scaling Groups
Or there is the Amazon Web Services getting started documentation
You also have a look into this blog,
http://geekospace.com/installing-aws-command-line-tools-from-amazon-downloads/
Auto Scaling is a tool that uses the results from Amazon CloudWatch to define the scaling policies on various instances.
Auto Scaling gives you power to decide the scaling, Schedule it and also define the resource to be scaled. These configurations are stored under an Auto Scaling Group and can be used to track applications working over various instances.