We run an ECS cluster behind an ELB (ALB, to be specific).
I have a process that allows me to find out which ECS cluster is associated with the ALB by querying the ALB and tracing the results back through the target group and then instances:
Here is the bash script:
ELB_NAME=$(aws route53 list-resource-record-sets --hosted-zone-id <Zone-ID> | jq -r --arg URL "$URL"'.ResourceRecordSets[]|select(.Name==$URL)|.AliasTarget.DNSName')
ELB_NAME=$(echo $ELB_NAME | cut -f 2- -d "." | rev | cut -f 2- -d "." | rev)
ELB_ARN=$(aws elbv2 describe-load-balancers | jq -r --arg ELB_NAME "$ELB_NAME" '.LoadBalancers[]|select((.DNSName|ascii_downcase)==$ELB_NAME)|.LoadBalancerArn')
TG_ARNS=$(aws elbv2 describe-target-groups | jq -r --arg ELB_ARN "$ELB_ARN" '.TargetGroups[]|select(.LoadBalancerArns[]==$ELB_ARN)|.TG_ARN=$(echo $TG_ARNS | cut -f 1 -d " ")
INSTANCE_ID=$(aws elbv2 describe-target-health --target-group-arn $TG_ARN | jq -r '.TargetHealthDescriptions[].Target.Id' | head -n 1)
CLUSTER=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID | jq -r '.Reservations[].Instances[].Tags[]|select(.Key=="aws:cloudformation:stack-name")|.Value' | cut -f 2 -d "-")
The problem I have is that when there are no running instances associated with the ECS cluster, I can no longer query them for the the tag that returns the Cloudformation stack name, the request for the targets from the target group is empty.
How can I use the AWS API so that I can determine which ECS cluster the ALB would target if it had running instances?
It's not really clear what you're asking for, or indeed the purpose you are trying to achieve, but the following should set you on the right track.
An ECS "cluster" is really just an Amazon service, when you create a cluster nothing is really provisioned. You can think of an empty cluster as a record or a placeholder in the ECS service.
In order to do anything with a cluster, it needs instances. When you boot an EC2 machine from a supported AMI, appropriate IAM role and the cluster name written to a config file, the instance will join the cluster. (If you create a cluster via the AWS console, a CloudFormation template is created that handles the provisioning and orchestration of these steps.) The ECS cluster management can then schedule tasks and services onto that instance as you have defined in the ECS service.
Without any instances, there can be no listening containers, therefore there can be no target groups in your ALB that route to anything. So it is not possible to get from the ELB to the cluster... as you have asked when there are no running instances.
You might find the following commands are a better way of determining whether or not you have a running cluster.
First, use the list-clusters command to show which clusters are available:
aws ecs list-clusters
{
"clusterArns": [
"arn:aws:ecs:eu-west-1:XXXXXXXXX:cluster/your_cluster"
]
}
Then use the output from that to show if there are any EC2 instances registered to the cluster:
aws ecs describe-clusters --clusters your_cluster
{
"clusters": [
{
"status": "ACTIVE",
"statistics": [],
"clusterName": "your_cluster",
"registeredContainerInstancesCount": 1,
"pendingTasksCount": 0,
"runningTasksCount": 0,
"activeServicesCount": 0,
"clusterArn": "arn:aws:ecs:eu-west-1:XXXXXXXXX:cluster/your_cluster"
}
],
"failures": []
}
Note the registeredContainerInstancesCount property shows the number of running instances. I assume you have your ECS services set to register tasks (containers) with the ALB, so when the count is greater than 0, this will be possible.
So, querying that property should tell you if your cluster is "on" or not:
if [[ $(aws ecs describe-clusters --clusters your_cluster | jq -r '.clusters[].registeredContainerInstancesCount') -gt 0 ]] ; then
echo "cluster is on"
else
echo "cluster is off"
fi
Related
I have a monotonous work of creating an ec2 instance from an AMI and then ssh into it and run a crypto bot to get a sub domain SSL certificates for it and then run some frontend and backend onto it.
I would like to automate this...
I am able to create the EC2 instance
aws ec2 run-instances --image-id ami-0bef9cfcxxxxxx --count 1 --instance-type t2.medium --key-name MyKey --security-group-ids SG-1 --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=SubDomain}]'
but how do I wait till the instance is UP and get the public IP for the next steps?
Any suggestions would help,
Thanks!
Assuming you know the target instance id, you can query your instance and get the current state, instanceId, name (assuming you tagged them with a name) and public IP via something like:
aws ec2 describe-instances \
--instance-ids YOUR_TARGET_INSTANCE_ID \
--query "Reservations[*].Instances[*].[State.Name, InstanceId, Tags[?Key==\`Name\`]|[0].Value, PublicIpAddress]" \
--output text
You could run this query in a loop until the status comes back as running and then capture the public ip address:
while :
do
output=$(aws ec2 describe-instances \
--instance-ids YOUR_TARGET_INSTANCE_ID \
--query "Reservations[*].Instances[*].[State.Name, InstanceId, \
Tags[?Key==\`Name\`]|[0].Value, PublicIpAddress]" \
--output text)
if [[ "${output}" == running* ]] ; then
cut -d $'\t' -f4- <<<"${output}"
break
fi
done
Note:
The aws ec2 queries are all using the default AWS profile. You can specify a different profile via --profile your_target_profile_name if needed.
If you do not know your target instance id, you can remove the --instance-id portion of the aws ec2 query and list all of your instances (in your default region). You could choose the required instance id from that list.
I seem to keep getting an error whenever I try to use bash to automate getting the status of a job.
My current bash script currently looks like this:
#!/bin/bash
aws ec2 start-instances --instance-ids=$1;
start=$(aws ec2 describe-instance-status --instance-id $1)
status=$(echo $start | jq '.InstanceStatuses[0].InstanceState.Name')
#wait for ec2 instance to start running before launching command
while [ "$status" != "\"running\"" ]
do
start=$(aws ec2 describe-instance-status --instance-id $1)
status=$(echo $star | jq '.InstanceStatueses[0].InstanceState.Name')
done
sh_command_id=$(aws ssm send-command --instance-ids=$1 --document-name "AWS-RunShellScript" --parameters 'commands=["echo Helloworld","sleep 60"]');
command_id=$(echo $sh_command_id | jq '.Command.CommandId')
full_status=$(aws ssm list-commands --command-id $command_id)
echo $command_id;
aws ec2 stop-instances --instance-ids=$1;
When the script gets to aws ssm list-commands --command-id $command_id I get this error.
An error occurred (ValidationException) when calling the ListCommands operation: 2
validation errors detected: Value '"67fb9aed-00bf-4741-ae1a-736ddbfba498"' at 'commandId'
failed to satisfy constraint: Member must satisfy regular expression pattern: ^[A-Fa-
f0-9]{8}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{12}$.; Value
'"67fb9aed-00bf-4741-ae1a-736ddbfba498"' at 'commandId' failed to satisfy constraint:
Member must have length less than or equal to 36.
When running everything individually in terminal I get the same error. However, I do not get an error when I mannually type in the commandId as so: full_status=$(aws ssm list-commands --command-id 67fb9aed-00bf-4741-ae1a-736ddbfba498).
Is there some aws formatting I am missing here?
You might be able to avoid the use of jq by using the aws cli built in --query 'your.json.query' to specify your JSON query and then the --output text to return plain text. It has been a while since I checked so your mileage may vary.
I was able to verify that the following works for checking an ec2 is running:
check_instance() {
local instance_id="${1}"
local status="_"
while [ "${status}" != "running" ] ; do
status=$(aws ec2 describe-instance-status \
--instance-ids ${instance_id} \
--query "InstanceStatuses[*].InstanceState.Name" \
--output text)
done
echo "${instance_id} is running"
}
I have 2 instances of the same deployment/project on AWS Elastic Beanstalk.
Both contain a Laravel project which contains scheduling code which runs various commands which can be found in the schedule method/function of the Kernel.php class within 'app/Console' - the problem I have is that if a command runs from one instance then it will also run the command from the second instance which is not what I want to happen.
What I would like to happen is that the commands get run from only one instance and not the other. How do I achieve this in the easiest way possible?
Is there a Laravel package which could help me achieve this?
From Laravel 5.6:
Laravel provides a onOneServer method which you can use if your applications share a single cache server. You could use something like ElastiCache to host Redis or Memcached and use it as your cache server for both of your application instances. Then you would be able to use the onOneServer method like this:
$schedule->command('report:generate')
->fridays()
->at('17:00')
->onOneServer();
For older versions of Laravel:
You could use the jdavidbakr/multi-server-event package. Once you have it set up you should be able to use it like:
$schedule->command('inspire')
->daily()
->withoutOverlappingMultiServer();
I had the same issue to run some cronjobs (nothing related to Laravel) and I found a nice solution (don't remember where I found it)
What I do is check if the instance running the code is the first instance on the Auto Scaling Group, if it's the first then I execute the command otherwise just exit.
This is the way it's implemented:
#!/bin/bash
INSTANCE_ID=`curl http://169.254.169.254/latest/meta-data/instance-id 2>/dev/null`
REGION=`curl -s http://169.254.169.254/latest/dynamic/instance-identity/document 2>/dev/null | jq -r .region`
# Find the Auto Scaling Group name from the Elastic Beanstalk environment
ASG=`aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" \
--region $REGION --output json | jq -r '.[][] | select(.Key=="aws:autoscaling:groupName") | .Value'`
# Find the first instance in the Auto Scaling Group
FIRST=`aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names $ASG \
--region $REGION --output json | \
jq -r '.AutoScalingGroups[].Instances[] | select(.LifecycleState=="InService") | .InstanceId' | sort | head -1`
# If the instance ids are the same exit 0
[ "$FIRST" = "$INSTANCE_ID" ]
Try implementing those calls using PHP and it should work.
This results instances listed with ELB:
aws elb describe-load-balancers --load-balancer-name XXXXXXX --region us-east-1 | jq -r '.LoadBalancerDescriptions[].Instances[].InstanceId'
I need some help in writing a script, which checks for new aws instances attached to the ELB.
- Need to compare the present result and previous results, which helps us to get the new instance info.
I'm using the AWS CLI and I want to get the ID of security group whose name I know (kingkajou_sg). How can I do it?
When I ask it to list all the security groups, it does so happily:
$ aws ec2 describe-security-groups | wc -l
430
When I grep through this information, I see that the SG in question is listed:
$ aws ec2 describe-security-groups | grep -i kingkajou_sg
"GroupName": "kingkajou_sg",
However, when I try to get the information about only that security group, it won't let me. Why?
$ aws ec2 describe-security-groups --group-names kingkajou_sg
An error occurred (InvalidGroup.NotFound) when calling the
DescribeSecurityGroups operation: The security group 'kingkajou_sg' does not exist in default VPC 'vpc-XXXXXXXX'
Can someone please provide me the one line command that I can use to extract the Security group's ID given its name? You can assume that the command will be run from within an EC2 which is in the same VPC as the Security group.
From the API Documentation:
--group-names (list)
[EC2-Classic and default VPC only] One or more security group names. You can specify either the security group name or the security group ID. For security groups in a nondefault VPC, use the group-name filter to describe security groups by name.
If you are using a non-default VPC, use the Filter
aws ec2 describe-security-groups --filter Name=vpc-id,Values=<my-vpc-id> Name=group-name,Values=kingkajou_sg --query 'SecurityGroups[*].[GroupId]' --output text
If it's in a VPC and you know the name & region and vpc id, you can try it like below:
aws ec2 describe-security-groups --region eu-west-1 --filter Name=vpc-id,Values=vpc-xxxxx Name=group-name,Values=<your sg name> --query 'SecurityGroups[*].[GroupId]' --output text
You just need to add --query 'SecurityGroups[*].[GroupId]' option with aws cli command.
aws ec2 describe-security-groups --group-names kingkajou_sg --query 'SecurityGroups[*].[GroupId]' --output text
To get the IDs of all security groups with a name matching exactly a specified string (default in this example) without specifying a VPC ID, use the following:
aws ec2 describe-security-groups --filter Name=group-name,Values=default --output json | jq -r .SecurityGroups[].GroupId
Note: this works for security groups even if they are not in the default VPC.
Small shell script to list security with search string as a variable. and we can tag the security groups.
https://ideasofpraveen.blogspot.com/2022/09/aws-cli-get-security-group-id-with-name.html.
If you want boto3 script to integrate with lambda for automations .
https://ideasofpraveen.blogspot.com/2022/09/aws-cli-get-security-group-id-with-name_15.html