I am using the Amazon AWS ELB command line tools. Is there a way of finding out the instances attached to a particular Elastic Load Balancer (ELB)?
2013/12/18: To update this and since the links are dead!
I installed the new AWS cli tools:
$ pip install awscli
Then ran:
$ aws configure
AWS Access Key ID [None]: my-key
AWS Secret Access Key [None]: my-secret
Default region name [None]: us-east-1
Default output format [None]:
This data is saved into ~/.aws/config.
Then I can find instances connected to a loadbalancer like so:
$ aws elb describe-load-balancers --load-balancer-name "my-name"
{
"LoadBalancerDescriptions": [
{
"Subnets": [],
"CanonicalHostedZoneNameID": "ID",
"CanonicalHostedZoneName": "my-name-foo.us-east-1.elb.amazonaws.com",
"ListenerDescriptions": [
{
"Listener": {
"InstancePort": 80,
"LoadBalancerPort": 80,
"Protocol": "HTTP",
"InstanceProtocol": "HTTP"
},
"PolicyNames": []
},
{
"Listener": {
"InstancePort": 80,
"SSLCertificateId": "arn:aws:iam::x:server-certificate/x-ssl-prod",
"LoadBalancerPort": 443,
"Protocol": "HTTPS",
"InstanceProtocol": "HTTP"
},
"PolicyNames": [
"AWSConsole-SSLNegotiationPolicy-api-production"
]
}
],
"HealthCheck": {
"HealthyThreshold": 10,
"Interval": 30,
"Target": "HTTP:80/healthy.php",
"Timeout": 5,
"UnhealthyThreshold": 2
},
"BackendServerDescriptions": [],
"Instances": [
{
"InstanceId": "i-FIRST-INSTANCEID"
},
{
"InstanceId": "i-SECOND-INSTANCEID"
}
],
"DNSName": "my-name-foo.us-east-1.elb.amazonaws.com",
"SecurityGroups": [],
"Policies": {
"LBCookieStickinessPolicies": [],
"AppCookieStickinessPolicies": [],
"OtherPolicies": [
"AWSConsole-SSLNegotiationPolicy-my-name"
]
},
"LoadBalancerName": "my-name",
"CreatedTime": "2013-08-05T16:55:22.630Z",
"AvailabilityZones": [
"us-east-1d"
],
"Scheme": "internet-facing",
"SourceSecurityGroup": {
"OwnerAlias": "amazon-elb",
"GroupName": "amazon-elb-sg"
}
}
]
}
The data is in LoadBalancerDescriptions.Instances.
My loadbalancer is called my-name — this is the name you selected when you created it.
Old answer below!
I'm not familiar with the cli tool, but I used the API.
I'd check these two requests:
DescribeLoadBalancers
DescribeInstanceHealth
The cli tool probably has something to resemble these?
HTH!
Assuming you have aws-cli and jq installed, you can use the following command to get associated ec2 instance ids:
aws elb describe-load-balancers --load-balancer-name my-elb \
| jq -r '.LoadBalancerDescriptions[].Instances[].InstanceId'
This will return the ec2 ids associated with that ELB.
Side note: I recommend you setup aws cli profiles so you don't have to fiddle with environment variables and region params (as much).
Because I love answers that can be used with a minimum of search/replace and copy paste
Prerequisites : aws-cli configured
pip install awscli
aws configure
Configure : your ELB name
$ELB_NAME = "Your-elb-name"
Copy-n-Paste in terminal
for ID in $(aws elb describe-load-balancers --load-balancer-name $ELB_NAME \
--query LoadBalancerDescriptions[].Instances[].InstanceId \
--output=text);
do
aws ec2 describe-instances --instance-ids $ID \
--query Reservations[].Instances[].PublicIpAddress \
--output text
done
Will output a list of Public IPs. You could also just execute the query inside the parenthesis of the for ID in $(...) to just get the instance IDs
Want something different ?
Feel free to have a look at the structure of
aws elb describe-load-balancers --load-balancer-name $ELB_NAME
aws ec2 describe-instances --instance-ids $INSTANCE_ID
and change the query accordingly!
If anyone arrives here from a search as to why the elb-describe-lbs command returns nothing when they have ELBs up and running, what I realised was I needed to add EC2_REGION=eu-west-1 to my environment variables (or use elb-describe-lbs --region command)
If you want to see all your ELB's and the instances attached use JMESPath like this:
aws elb describe-load-balancers --query "LoadBalancerDescriptions[*].{ID:LoadBalancerName,InstanceId:Instances[*].InstanceId}[*]. {ELB:ID,InstanceId:InstanceId[*]}" --output=json
Result
[
{
"ELB": "my_name",
"InstanceId": [
"i-0cc72"
]
},
{
"ELB": "my_name2",
"InstanceId": [
"i-02ff5f",
"i-09e467"
]
}
]
If you know the name of the ELB and want to see what is attached use JMESPath like this:
aws elb describe-load-balancers --load-balancer-name "my_name" --query "LoadBalancerDescriptions[].{ID:LoadBalancerName,InstanceId:Instances[].InstanceId}[].{ELB:ID,InstanceId:InstanceId[]}" --output=json
Result:
[
{
"ELB": "my_name",
"InstanceId": [
"i-02ff5f72",
"i-09e46743"
]
}
]
replace INSTANCEID with actual instance id
aws elb describe-load-balancers --query "LoadBalancerDescriptions[*].{ID:LoadBalancerName,InstanceId:Instances[?InstanceId=='INSTANCEID'].InstanceId}[*].{ID:ID,InstanceId:InstanceId[0]}" --output=text | grep INSTANCEID | awk '{print $1}'
In node.js you can do this by using aws-sdk.
var AWS = require('aws-sdk')
var options = {
accessKeyId: 'accessKeyId',
secretAccessKey: 'secretAccessKey',
region: 'region'
}
var elb = new AWS.ELB(options)
elb.describeLoadBalancers({LoadBalancerNames: ['elbName']}, function(err, data) {
if (err) {
console.log('err: ', err)
}
else {
console.log('data: ', data.LoadBalancerDescriptions)
}
})
data.LoadBalancerDescriptions is an array and each element in the array is an object with the property Instances that has the instance id.
You can loop trough all you load balancer instance ids as follows:
while read -r lb ; do echo -e "\n\n start lb: $lb " ; \
echo run cmd on lb: $lb ; echo " stop lb: $lb" ; \
done < <(aws elb describe-load-balancers --query \
'LoadBalancerDescriptions[].Instances[].InstanceId' \
--profile dev|perl -nle 's/\s+/\n/g;print')
You can loop trough your load balancers names as follows :
# how-to loop trough all your load balancer names
while read -r lb ; do \
echo -e "\n\n start lb: $lb " ; \
echo run cmd on lb: $lb ; \
echo " stop lb: $lb" ; \
done < <(aws elb describe-load-balancers --query \
'LoadBalancerDescriptions[].LoadBalancerName' \
--profile rnd|perl -nle 's/\s+/\n/g;print')
Provided that you have configured your aws cli :
src: http://docs.aws.amazon.com/cli/latest/topic/config-vars.html
cat << "EOF" > ~/.aws/config
[profile dev]
output = text
region = us-east-1
[profile dev]
output = text
region = us-east-1
[default]
output = text
region = Global
EOF
And configured your security credentials:
# in aws admin console :
# Services => iam => users => <<your_username>> => Security Credentials => Access Keys
# configure the aws cli
cat << "EOF" > ~/.aws/credentials
[dev]
aws_access_key_id = <<your_aws_access_key_id_in_the_dev_environment>>
aws_secret_access_key = <<your_aws_secret_access_key_in_dev_env>>
[dev]
aws_access_key_id = <<your_aws_access_key_id_in_the_dev_environment>>
aws_secret_access_key = <<your_aws_secret_access_key_in_dev_env>>
[default]
aws_access_key_id = <<your_aws_access_key_id_in_the_dev_environment>>
aws_secret_access_key = <<your_aws_secret_access_key_in_dev_env>>
EOF
aws elb describe-load-balancers --load-balancer-name "LB_NAME" | grep "InstanceId" | awk '{print $2}' | sed 's/\"//g'
First do elb-describe-lbs to get a list of your load balancers and their names.
Then do elb-describe-instance-health <LB_NAME> to get a list of instances behind that load balancer. LB_NAME is the value of the 2nd column in the output of elb-describe-lbs.
You can use AWS command line tools with some bash piping:
elb-describe-instance-health loadbalancer_name --region eu-west-1 | awk '{ print $2 }' | xargs ec2-describe-instances --region eu-west-1 | grep ^INSTANCE | awk '{ print $4 }'
This will give you the public DNS name for every instance attached to the ELB, you can change the awk columns respectively to get other details.
Related
I am trying to extract a few properties of AWS EC2 instance using JMESPath. I just started learning JMESPath using this useful guide. But because my knowledge is limited, I was only able to get the output like below.
% aws ec2 describe-instances --query "Reservations[].Instances[].[InstanceId,InstanceType,KeyName,LaunchTime,State.Name,Tags[].[Key,Value]]" --instance-ids i-123456789101123
[
[
"i-123456789101123",
"r5.8xlarge",
"My_EMR",
"2022-12-19T16:11:42.000Z",
"running",
[
[
"tag-key-1",
"tag-value-1"
],
[
"tag-key-2",
"tag-value-2"
],
[
"tag-key-3",
"tag-value-3"
],
[
"project",
"my-project"
]
]
]
]
Ideally, I would like to format the output using JMESPath like below.
[
[
"i-123456789101123",
"r5.8xlarge",
"My_EMR",
"2022-12-19T16:11:42.000Z",
"running",
{
"tag-key-1": "tag-value-1",
"tag-key-2": "tag-value-2",
"tag-key-3": "tag-value-3",
"project": "my-project"
}
]
]
I have tried a bunch of different JMESPath expressions like below, but none of them seem to work. I'd greatly appreciate any suggestion. Thank you in advance!
# JMESPath statements that I've tried so far (actually there are more, but these are the 'promising' ones that I tried, but failed to work)
% aws ec2 describe-instances --query "Reservations[].Instances[].[InstanceId,InstanceType,KeyName,LaunchTime,State.Name,map(Tags[].Key,Tags[].Value)]"
% aws ec2 describe-instances --query "Reservations[].Instances[].[InstanceId,InstanceType,KeyName,LaunchTime,State.Name,map(Tags[].Key,Tags[].Value)]"
% aws ec2 describe-instances --query "Reservations[].Instances[].{InstanceId: InstanceId, InstanceType: InstanceType, KeyName: KeyName, LaunchTime: LaunchTime, StateName: State.Name, Tags[].Key: Tags[].Value}"
% aws ec2 describe-instances --query "Reservations[].Instances[].{InstanceId: InstanceId, InstanceType: InstanceType, KeyName: KeyName, LaunchTime: LaunchTime, StateName: State.Name, Tags[*].Key: Tags[*].Value}"
I have the following bash script that I execute in order to create new Glue Job via CLI:
#!/usr/bin/env bash
set -e
NAME=$1
PROFILE=$2
SCRIPT_LOCATION='s3://bucket/scripts/'$1'.py'
echo [*]--- Creating new job on AWS
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json | jq '.Command.ScriptLocation = '\"$SCRIPT_LOCATION\"'' ./resources/config.json
I'm using jq as i need one of the values to be replaced on runtime before i pass the .json as --cli-input-json argument. How can i pass json with replaced value to this command? As of now, it prints out the json content (although with value already replaced).
Running the command above causes the following error:
[*]--- Creating new job on AWS
{
"Description": "Template for Glue Job",
"LogUri": "",
"Role": "arn:aws:iam::11111111111:role/role",
"ExecutionProperty": {
"MaxConcurrentRuns": 1
},
"Command": {
"Name": "glueetl",
"ScriptLocation": "s3://bucket/scripts/script.py",
"PythonVersion": "3"
},
"DefaultArguments": {
"--TempDir": "s3://temp/admin/",
"--job-bookmark-option": "job-bookmark-disable",
"--enable-metrics": "",
"--enable-glue-datacatalog": "",
"--enable-continuous-cloudwatch-log": "",
"--enable-spark-ui": "true",
"--spark-event-logs-path": "s3://assets/sparkHistoryLogs/"
},
"NonOverridableArguments": {
"KeyName": ""
},
"MaxRetries": 0,
"AllocatedCapacity": 0,
"Timeout": 2880,
"MaxCapacity": 0,
"Tags": {
"KeyName": ""
},
"NotificationProperty": {
"NotifyDelayAfter": 60
},
"GlueVersion": "3.0",
"NumberOfWorkers": 2,
"WorkerType": "G.1X"
}
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws.exe: error: argument --cli-input-json: expected one argument
The command line
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json | jq '.Command.ScriptLocation = '\"$SCRIPT_LOCATION\"'' ./resources/config.json
executes the command
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json,
takes its standard output and uses it as input to
jq '.Command.ScriptLocation = '\"$SCRIPT_LOCATION\"'' ./resources/config.json
(which will ignore the input and read from the file given as argument). Please also note that blanks or spaces in $SCRIPT_LOCATION will break your script, because it is not quoted (your quotes are off).
To use the output of one command in the argument list of another command, you must use Command Substitution: outer_command --some-arg "$(inner_command)".
So your command should become:
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json "$(jq '.Command.ScriptLocation = "'"$SCRIPT_LOCATION"'"' ./resources/config.json)"
# or simplified with only double quotes:
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json "$(jq ".Command.ScriptLocation = \"$SCRIPT_LOCATION\"" ./resources/config.json)"
See https://superuser.com/questions/1306071/aws-cli-using-cli-input-json-in-a-pipeline for additional examples.
Although, I have to admit I am not 100% certain that the JSON content can be passed directly on the command line. From looking at the docs and some official examples, it looks like this parameter expects a file name, not a JSON document's content. So it could be possible that your command in fact needs to be:
# if "-" filename is specially handled:
jq ".Command.ScriptLocation = \"$SCRIPT_LOCATION\"" ./resources/config.json | aws glue create-job --profile $PROFILE --name $NAME --cli-input-json -
# "-" filename not recognized:
jq ".Command.ScriptLocation = \"$SCRIPT_LOCATION\"" ./resources/config.json > ./resources/config.replaced.json && aws glue create-job --profile $PROFILE --name $NAME --cli-input-json file://./resources/config.replaced.json
Let us know which one worked.
I'm trying to check that all instances attached to an AWS ELB are in a state of "InService",
For that, I created an AWS CLI command to check the status of the instances.
problem is that the JSON output returns the status of both instances.
So it is not that trivial to examine the output as I wish.
When I run the command:
aws elb describe-instance-health --load-balancer-name ELB-NAME | jq -r '.[] | .[] | .State'
The output is:
InService
InService
The complete JSON is:
{
"InstanceStates": [
{
"InstanceId": "i-0cc1e6d50ccbXXXXX",
"State": "InService",
"ReasonCode": "N/A",
"Description": "N/A"
},
{
"InstanceId": "i-0fc21ddf457eXXXXX",
"State": "InService",
"ReasonCode": "N/A",
"Description": "N/A"
}
]
}
What I've done so far is creating that one liner shell command:
export STR=$'InService\nInService'
if aws elb describe-instance-health --load-balancer-name ELB-NAME | jq -r '.[] | .[] | .State' | grep -q "$STR"; then echo 'yes'; fi
But I get "yes" as long as there is "InService" at the first command output
Is there a way I can get TRUE/YES only if I get twice "InService" as an output?
or any other way to determine that this is indeed what I got in return?
Without seeing an informative sample of the JSON it's not clear what the best solution would be, but the following meets the functional requirements as I understand them, without requiring any further post-processing:
jq -r '
def count(stream): reduce stream as $s (0; .+1);
if count(.[][] | select(.State == "InService")) > 1 then "yes" else empty end
'
I try to sort this output from AWS CLI by ImageId, and I executed command below.
aws ec2 describe-images --profile xxxxxxxxxx \
--filter Name=tag:Name,Values=Backup*some-string* \
--query "Images[*].[Tags[?Key=='Name'].Value[]|[0],ImageId]"
output is:
[
[
"Backup-20191215T174530Z-utc-some-string",
"ami-004"
],
[
"Backup-20191219T174631Z-utc-some-string",
"ami-002"
],
[
"Backup-20191208T174534Z-utc-some-string",
"ami-001"
],
[
"Backup-20191222T174530Z-utc-some-string",
"ami-003"
],
[
"Backup-20191221T174530Z-utc-some-string",
"ami-005"
]
]
I found sort_by functions of JMESPath could be a solution but that is too hard to understand.
aws ec2 describe-images --profile xxxxxxxxxx \
--filter "Name=tag:Name,Values=Backup*some-string*" \
--query "sort_by(Images[*].[Tags[?Key=='Name'].Value[]|[0],ImageId], &[0])"
or
aws ec2 describe-images --profile xxxxxxxxxx \
--filter "Name=tag:Name,Values=Backup*some-string*" \
--query "Images[*].[Tags[?Key=='Name'].Value[]|[0],ImageId] | sort_by(#, &[0])"
is working fine for me. & (expression type operator) is needed.
The idea is, In my solution below, I am sorting the output first by the ImageId and then applying the projections.
aws ec2 describe-images --filter Name=tag:Environment,Values=Staging --output json --query "(sort_by(Images[], &ImageId))[*].[ImageId, Tags[?Key=='Environment'].Value | [0]]"
i need to run an instance and access with my ip address..but the problem is that myISP changes my IP adress every day.plz help me how do i create a security group so that my instance remains accessible even if my ip changes....
thanks in advance..
Here's a way to restrict AWS security group to your dynamic IP address for SSH.
You can write a cronjob to regularly repeat the steps below:
Fetch external IP address (e.g. http://checkip.amazonaws.com/)
Fetch security group details using AWS SDK
Loop through all security rules, check for port 22
If IP address mismatches that from step 1, update it.
Create a security group with port 22 ingress access from your current ip.
Add a Tag to the security group with the key ssh-from-my-ip and the value true
Whenever your IP changes run this script (or run it periodically via cron)
#! /bin/bash
# This script makes it easier to maintain security groups that allow SSH access
# from a computer with a dynamic IP, such as a computer on a home network or ISP.
#
# Using the script will allow you to SSH to an EC2 without having to allow
# access to the whole world (0.0.0.0/0). If you run this script whenever your IP
# changes then the security groups in your account specified by your AWS profile
# will be updated.
#
# The script will find any security groups for your current profile that are
# tagged with a Tag with a Key of "ssh-from-my-ip" and a case insensitive value
# of "true" or "yes".
#
# For each security group found it will revoke any existing tcp ingress on
# port 22 and authorize ingress on port 22 for your current IP.
#
# Dependencies - AWS CLI and jq
# need my current ip
MY_IP=$(curl --silent https://checkip.amazonaws.com)
echo "Your IP is ${MY_IP}"
# need security group id(s) and existing CIDR for the SG
pairs=$(aws ec2 describe-security-groups | aws ec2 describe-security-groups | jq -c '.SecurityGroups[]? | select( (.Tags[]? | select(.Key == "ssh-from-my-ip") | .Value | test("true|yes"; "i"))) | if .IpPermissions | length == 0 then {sg: .GroupId, cidr: null } else {sg: .GroupId, cidr: .IpPermissions[].IpRanges[].CidrIp} end')
for p in $pairs
do
SG=$(echo "$p" | jq -r '.sg')
OLD_CIDR=$(echo "$p" | jq -r '.cidr')
echo "Updating security group ${SG}"
if [[ $OLD_CIDR != 'null' ]]
then
echo "Revoking ingress permission for ${OLD_CIDR} in security group ${SG}"
# remove the existing ingress permission
aws ec2 revoke-security-group-ingress \
--group-id "${SG}" \
--protocol tcp \
--port 22 \
--cidr "${OLD_CIDR}"
fi
# authorize my new IP CIDR
NEW_CIDR="${MY_IP}"/32
echo "Authorizing ingress permission for ${NEW_CIDR} in security group ${SG}"
aws ec2 authorize-security-group-ingress --group-id "${SG}" --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "'"${NEW_CIDR}"'", "Description": "Rule0"}]}]'
done
#!/bin/bash
# User specific data:
SECURITY_GROUP_NAME="" # Setup here your group name
REGION=${1:-"us-east-1"} # Change default region if needed
USER=`aws iam get-user --query "User.UserName" | tr -d '"'`
RULE_DESCRIPTION='DynamicIP'$USER
echo 'User: '$RULE_DESCRIPTION', Region: '$REGION', Security group: '$SECURITY_GROUP_NAME
checkip () {
OLD_CIDR_IP=`aws ec2 describe-security-groups --region $REGION --query "SecurityGroups[?GroupName=='$SECURITY_GROUP_NAME'].IpPermissions[*].IpRanges[?Description=='$RULE_DESCRIPTION'].CidrIp" --output text`
NEW_IP=`curl -s http://checkip.amazonaws.com`
NEW_CIDR_IP=$NEW_IP'/32'
if [[ $OLD_CIDR_IP != "" ]] && [[ $OLD_CIDR_IP != $NEW_CIDR_IP ]]; then
echo "Revoking $OLD_CIDR_IP"
aws ec2 revoke-security-group-ingress --region $REGION --group-name $SECURITY_GROUP_NAME --protocol tcp --port 22 --cidr $OLD_CIDR_IP --output text >> /dev/null
fi
if [[ $NEW_IP != "" ]] && [[ $OLD_CIDR_IP != $NEW_CIDR_IP ]]; then
echo "Setting up new ip $NEW_IP"
aws ec2 authorize-security-group-ingress --region $REGION --group-name $SECURITY_GROUP_NAME --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 22, "ToPort": 22, "IpRanges": [{"CidrIp": "'$NEW_CIDR_IP'", "Description": "'$RULE_DESCRIPTION'"}]}]'
fi
sleep 30
checkip
}
checkip
You can use a Source CIDR of 0.0.0.0/0 to allow universal access.
You could restrict it to your ISP's address space by looking up their allocations or just monitoring what IP addresses you end up with.
To do it properly and restrict access to a single IP dynamic address you could write an application that monitors your public IP address and when it changes call the EC2 API AuthorizeSecurityGroupIngress method and delete the old address with RevokeSecurityGroupIngress.