Looking at the documentation for creating a service in Fargate (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_AWSCLI_Fargate.html#ECS_AWSCLI_Fargate_create_service), it does not advise you to create a target group for the service. A target group is necessary for attaching a load balancer so I have no idea why they don't include it in the documentation. Considering you can't add a load balancer to the service after creation, this is inevitably causing me a headache. So how do I go about making sure my Fargate service creates the target group(s) necessary so I can add an ELB to it?
If I go about creating a Target Group, https://aws.amazon.com/premiumsupport/knowledge-center/create-alb-auto-register/, there are no targets from my VPC to attach to, so this is basically useless as I'm doing something wrong on creation.
aws ecs create-service --cluster fargate-cluster --service-name fargate-service --task-definition sample-fargate:1 --desired-count 1 --launch-type "FARGATE" --network-configuration "awsvpcConfiguration={subnets=[subnet-abcd1234],securityGroups=[sg-abcd1234]}"
Can anyone guide me here?
I ended up deleting the created service which didn't have a load balancer, creating the target group for the same VPC as my load balancer (IP type), and then grabbing the Arn of the target group for when I create the service:
aws ecs create-service --cluster evenflo-cms-dev --service-name django --task-definition evenflo-cms:4 --desired-count 1 --launch-type "FARGATE" --network-configuration "awsvpcConfiguration={subnets=[subnet-0950e9d7ffc3fc8e3, subnet-036375b8ae487f2c9],securityGroups=[sg-077d0e3ba1cfb4145],assignPublicIp=ENABLED}" --load-balancers='[{"targetGroupArn": "arn:aws:elasticloadbalancing:us-east-1:898591449577:targetgroup/evenflo-cms-api/f5501d5733c6d91c", "containerName": "django", "containerPort": 7000 }]'
Is there a cleaner way to approach this with Terraform? Probably. But I just wanted a straight way to accomplish this with the AWS CLI and to be able to understand the order in which things should be done. Hopefully this helps someone.
Related
Currently, I have a CFN template to creates an auto-scaling group, which launches instances based on a launch template. I would like to onboard with Systems Manager to automate patching, but I can only find the UI (Quick Setup) and not CFN for performing all those actions; I want to make sure any new instances spun up in the auto-scaling group will be onboarded with SSM as well and become managed instances. How do I go about doing that?
There are few prerequisites.
You can refer this : https://aws.amazon.com/premiumsupport/knowledge-center/systems-manager-ec2-instance-not-appear/
In short, ssm agent must be running and must be able to reach ssm endpoints (public or vpc endpoints) and must have necessary permission in instance profile to make API calls to SSM service, in order to register the instance.
I am confused about how Service AutoScaling automately works. Will it create EC2 instance automately?
I create it and add it to a Cluster's service, but it does no create EC2 for placing my required number of tasks. Is any thing wrong with my settings? I check the [Events] and see "service s2 was unable to place a task because no container instance met all of its requirements. ", but shouldn't it create a EC2 if no instance met? Please give me some advice, thanks in advance.
but shouldn't it create a EC2 if no instance met
Not really. There are two types of scaling policies: scaling policies on an ECS service and scaling policies on the ECS cluster. Instances are added based on cluster scaling policies, and that's what you should set up in addition to your service scaling policy.
AWS has a couple of detailed tutorials on scaling ECS clusters:
https://aws.amazon.com/blogs/compute/automatic-scaling-with-amazon-ecs/
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cloudwatch_alarm_autoscaling.html
AWS Elastic Container Services has two methods to deploy containers over aws environment
Where you no need to worry about orchestration of containers (task in aws)
Fargate (Available in few regions like N.Virginia)
Using EC2 in ECS
I guess you are using 2nd option to deploy application over ECS where you can provide details of scaling tasks/containers not ec2 instances.
For Auto-scaling of ec2 instances you should look into ASG of AWS.
As far as AWS ECS is concerned you need some building blocks which are as follows-
Cluster
Task definition (Memory, Network and Storage configs of tasks)
Service contains EC2 instance configuration
Auto scaling policies if you want to auto-scale tasks
I have created my own EC2 instance in AWS. That AMI is AWS ECS optimized AMI for launching ecs service from my EC2 instance. I previously discussed the same thing. And tried with that approach. The link is below,
Microservice Deployment Using AWS ECS Service
I created my cluster and configured that cluster name when I am creating optimized AMI by following code snippet in advanced userdata section,
#!/bin/bash
echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config
I followed the documentation of cluster creation from following link,
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create_cluster.htmlecs
But, no result - when creating cluster and ECS task definitions it creates and launches into one EC2. And again creating another EC2 by specifying above code. So total 2 Ec2. I already created my own ECS optimized.
I am finding for launching ECS service from my own AMI (that I created). Actually I need to launch my ECS service from my Ec2 (I had created my machine Amazon optimized AMI).
The reason behind this requirement is I don't want to launch my services in machine that owned by others. I need to launch from my machine. And also I need to host my angular application in the same my machine. So I need control of my machine. How can I do this?
Sounds like you just need to create a Launch Configuration. With this you can specify the User Data settings that should be applied when a host is setup.
After you create your Launch Configuration, create a new Auto Scaling Group based off of it (there's a drop-down to select the launch configuration you want to use).
From here, any new instances launched under that ASG will apply the settings you've configured in the associated Launch Configuration.
In all tutorials for ECS you need to create a cluster and after that an autoscaling group, that will spawn instances. Somehow in all these tutorials the instances magically show up in the cluster, but noone gives a hint what's connecting the autoscaling group and the cluster.
my autoscaling group spawns instances as expected, but they just dont show up on my ecs cluster, who holds my docker definitions.
Where is the connection I'm missing?
I was struggling with this for a while. The key to getting the instances in the autoscaling group associated with your ECS cluster is in the user data. When you are creating your launch config when you get to step 3 "Configure Details" hit the advanced tab and enter a simple bash script like the following for your user data.
#!/usr/bin/env bash
echo ECS_CLUSTER=your_cluster_name >> /etc/ecs/ecs.config
All the available parameters for agent configuration can be found here http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html
An autoscaling group is not strictly associated to a cluster. However, an autoscaling group can be configured such that each instance launched registers itself into a particular cluster.
Registering an instance into a cluster is the responsibility of the ECS Agent running on the instance. If you're using the Amazon ECS-optimized AMI, the ECS Agent will launch when the instance boots and register itself into the configured cluster. However, you can also use the ECS Agent on other Linux AMIs by following the installation instructions.
Well, i found out.
Its all about the ecs-agent and its config file /etc/ecs/ecs.config
(This file will be created through the Userdata field, when creating EC2 instances, even from an autoscaling configuration.)
Read about its configuration options here: http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html
But you can even copy a ecs.config stored on Amazon S3, do it like this (following lines go into Userdata field):
#!/bin/bash
yum install -y aws-cli
aws configure set default.s3.signature_version s3v4
aws configure set default.s3.addressing_style path
aws configure set default.region eu-central-1
aws s3 cp s3://<bucketname>/ecs.config /etc/ecs/ecs.config
note: Signature_version v4 is specific for some regions, like eu-central-1.
This ofc only works, if your IAM role for the instance (in my case its ecsInstanceRole) has the right AmazonS3ReadOnlyAccess
The AWS GUI console way for that would be:
Use the cluster wizard at https://console.aws.amazon.com/ecs/home#/firstRun .
It will create an autoscaling grou for your cluster, a loadbalancer in front of it, and connect it all nicely.
This question is old but the answer is not complete. There are 2 parts to getting your own auto-scaling group to show up in your cluster (as of Jan 2022).
You need to ensure your cluster name is set for ECS_CLUSTER variable in /etc/ecs/ecs.config as mentioned in this answer: https://stackoverflow.com/a/35324937/583875
You need to create a new capacity provider for the cluster and attach this auto scaling group. To do this, go to Cluster -> Capacity Provider -> Create -> Select your auto scaling group under Auto Scaling group.
Another tricky part is getting your service to use the instances (if you have a service running). You need to edit the Service, and change the Capacity provider strategy. Click on Add another provider and choose the new capacity provider you created in (2) above.
That's all! To ensure things are working properly: you should see your capacity provider under Graph -> Capacity Providers and you should see instances from your auto scaling group under Graph -> ECS Instances.
I want to perform autoscaling without using CLI tools. I want to do it from the console itself.
The instance is in vpc ? how can i apply the autoscale policy on instance
Any lead is appriciated.
Thanks in advance.
Documentation:
Follow the instructions on how to Set Up an Auto-Scaled and Load-Balanced Application
Notes:
The instance, created outside of AutoScaling Group can be added to Elastic Load Balancer, but will not be monitored or managed by AutoScaling group.
Instance, created outside of AutoScaling Group can be marked as unhealthy by Elastic Load Balancer if the health check fails, but it will not cause AutoScaling Group to spawn a new instance.