I have an application which needs to know how many EC2 instances I have in both the pending and running states. I could run the two following commands and sum them to get the result, but that means my awscli will be making two requests, which is both slow and probably bad practice.
aws ec2 describe-instances \
--filters Name=instance-state-name,Values="running" \
--query 'Reservations[*].Instances[*].[InstanceId]' \
--output text \
| wc -l
aws ec2 describe-instances \
--filters Name=instance-state-name,Values="pending" \
--query 'Reservations[*].Instances[*].[InstanceId]' \
--output text \
| wc -l
Is there a way of combining these two queries into a single one, or another way of getting the total pending + running instances using a single query?
Cheers!
Shorthand syntax with commas:
Values=running,pending
You can add several filter values as follows:
aws ec2 describe-instances \
--filters "Name=instance-state-name,Values=running,pending" \
--query 'Reservations[*].Instances[*].[InstanceId]' \
--output text \
| wc -l
Related
Goal: find specific AMI's and copy them to another AWS region.
using describe-images and its filter i get a list of ImageId and Name,
AMI_LIST=$(aws ec2 describe-images --filters "Name=tag:Name,Values=*one*,*two*,*three*,*four*" \
"Name=state,Values=available" "Name=tag:Name,Values=${CUSTOMER_NAME}*" \
--query 'Images[*].{ID:ImageId,NAME:Name}' --output text)
echo $AMI_LIST
result:
ami-036ba4ef9fa1d148d big394_one_1 ami-06d13684f11138f1f big394_two_3 ami-0706803a11e21946d big394_two_1 ami-094043f896db39243 big394_two_2 ami-0c11ff60c981c2273 big394_three_1 ami-0d0b30fcc69f30af8 big394_four_1
then i want to copy the images to another AWS region using a loop:
for ami in $AMI_LIST; do
aws ec2 copy-image --source-image-id ${ami[0]} --source-region us-east-1 --region us-west-2 --name ${ami[2]}
done
ofc it does not work because ${ami[0]} and ${ami[1]} has no meaning, but they represent what i would like to achieve.
i did try to play with converting the list to array but without success.
Thanks.
This should achieve what you expected :
aws ec2 describe-images --filters "Name=tag:Name,Values=*one*,*two*,*three*,*four*" \
"Name=state,Values=available" "Name=tag:Name,Values=${CUSTOMER_NAME}*" \
--query 'Images[*].{ID:ImageId,NAME:Name}' --output text \
| while read ami name; do
aws ec2 copy-image --source-image-id $ami --source-region us-east-1\
--region us-west-2 --name $name
done
I am using AWS EC2 CLI to perform a filter on stopped instances, then create an AMI out of these with the AMI name taken from the instance tag.
aws ec2 describe-instances --output text --profile proj --query 'Reservations[*].[Instances[*].[InstanceId, InstanceType, State.Name, Platform, Placement.AvailabilityZone, PublicIpAddress, PrivateIpAddress,[Tags[?Key==`Name`].Value][0][0]]]' --filter --filters Name=instance-state-name,Values=stopped | awk '{print $1, $8}' | xargs -n2 aws ec2 create-image --profile proj --instance-id {} --name {} --no-reboot
how to let args differentiate the two different parameters from AWK (instnaceid, instance name tag), thereby it can be correctly pumped into the ec2 create-image on the instance-id and --name parameter accordingly
You do not need awk.Using AWS CLI, you are extracting 8 values first and then using awk to extract 2 values from that 8 values. Why? Just extract the 2 values from AWS CLI without using awk.
--query 'Reservations[*].[Instances[*].[InstanceId, [Tags[?Key==`Name`].Value][0][0]]]'
will return only the values you are interested in. Then use xargs to pass the arguments to your next command.
xargs -n2 command --arg1 $1 --arg2 $2
Your entire command becomes:
aws ec2 describe-instances --output text --profile proj --query 'Reservations[*].[Instances[*].[InstanceId, [Tags[?Key==`Name`].Value][0][0]]]' --filter --filters Name=instance-state-name,Values=stopped | xargs -n2 aws ec2 create-image --profile proj --instance-id $1 --name $2 --no-reboot
I've got a long list of commands strung together (aws cli, jq, cut, sed) that works for me with which I'm trying to now run within a bash script. I'm having problems with the single and double quotes.
If I ran this manually it would be the following:
aws \
--profile app-prod \
--region us-east-1 ec2 describe-instances \
--filter 'Name=tag:ChefRole,Values=foo_bar' \
'Name=instance-state-name,Values=running' \
| jq -r '.Reservations[].Instances[] | [.InstanceId, .Tags[].Value] | #csv' \
| cut -d ',' -f1 \
| sort > sort.out;sed 's/"//g' sort.out
In my script I'm trying to pass in parameters the region and Values
I've tried this:
awsCLI(){
cmd=$(aws --profile app-prod --region ${region} ec2 describe-instances --filter 'Name=tag:ChefRole,Values='${component}'| jq -r '.Reservations[].Instances[] | [.InstanceId, .Tags[].Value] | #csv' | cut -d ',' -f1 | sort > "${_sorted}";sed 's/"//g' "${_sorted}")
}
Placing in the variable - cmd=$(commands here) has worked for me before in other scripts, but not this time.
I also tried just running as is:
aws --profile app-prod --region ${region} ec2 describe-instances --filter 'Name=tag:ChefRole,Values='${component}'| jq -r '.Reservations[].Instances[] | [.InstanceId, .Tags[].Value] | #csv' | cut -d ',' -f1 | sort > "${_sorted}";sed 's/"//g' ${_sorted}"
Either of these didn't work for me. Please advise on best practice for this. Should I break this out? How would you build this long command?
Thanks you.
(Editor's note: first code block split by \ and indented; A.T.)
I was actually trying to eliminate the eternally long horizontal scroll bars from your question, but then decided to convert it into an answer, maybe it actually solves your problem (if not, just tell, will delete):
awsCLI(){
cmd=$(aws \
--profile tealium-prod \
--region "${region}" ec2 describe-instances \
--filter 'Name=tag:ChefRole,Values='"${component}" \
| jq -r '.Reservations[].Instances[] | [.InstanceId, .Tags[].Value] | #csv' \
| cut -d ',' -f1 \
| sort > "${_sorted}" \
; sed 's/"//g' "${_sorted}" \
)
}
I think the main problem was superfluous single tick around Values='"${component}". In general you can concatenate strings by juxtaposition, all you have to remember is that "" expand the variables, whereas '' do not.
The root cause was non-existent indentation. Look up what the backslash in the end of the line does, it will make your life much easier.
If you have the ability to use Python, the boto3 library is very easy to use. The issue with multiple types of quotation marks becomes moot.
I want to list the public IP addresses of my EC2 instances using Bash, separated by a delimiter (space or a new-line).
I tried to pipe the output to jq with aws ec2 describe-instances | jq, but can't seem to isolate just the IP addresses.
Can this be done by aws alone, specifying arguments to jq, or something else entirely?
Directly from the aws cli:
aws ec2 describe-instances \
--query "Reservations[*].Instances[*].PublicIpAddress" \
--output=text
Filter on running instances (you can drop that part if you don't need it)
Query for each PublicIPaddress and the Name Tag, handling when Name isn't set
aws ec2 describe-instances \
--filter "Name=instance-state-name,Values=running" \
--query "Reservations[*].Instances[*].[PublicIpAddress, Tags[?Key=='Name'].Value|[0]]" \
--output text
The below command would list the IP addresses of all your running EC2 instances
aws ec2 describe-instances | grep PublicIpAddress | grep -o -P "\d+\.\d+\.\d+\.\d+" | grep -v '^10\.'
Hope that answers your query...
But this works without all the errors about access:
wget -qO- http://instance-data/latest/meta-data/public-ipv4/|grep .
You can use instance metadata so you can run the following command from the ec2 instance:
curl http://169.254.169.254/latest/meta-data/public-ipv4
and it will give you the public IP of the instance. If you want the private IP, you will run
curl http://169.254.169.254/latest/meta-data/local-ipv4
aws ec2 describe-instances --query "Reservations[].Instances[][PublicIpAddress]"
Refer:
http://docs.aws.amazon.com/cli/latest/userguide/controlling-output.html
I'm unable to bootstrap my server because "knife ec2 server create" keeps expanding my runlist to "roles".
knife ec2 server create \
-V \
--run-list 'role[pgs]' \
--environment $1 \
--image $AMI \
--region $REGION \
--flavor $PGS_INSTANCE_TYPE \
--identity-file $SSH_KEY \
--security-group-ids $PGS_SECURITY_GROUP \
--subnet $PRIVATE_SUBNET \
--ssh-user ubuntu \
--server-connect-attribute private_ip_address \
--availability-zone $AZ \
--node-name pgs \
--tags VPC=$VPC
Consistently fails because 'roles[pgs]' is expanded to 'roles'. Why is this? Is there some escaping or alternative method I can use?
I'm currently working around this by bootstrapping with an empty run-list and then overriding the runlist by running chef-client once the node is registered.
This is a feature of bash. [] is a wildcard expander. You should can escape the brackets using "\".