I want to list all ec2 instances for each account we have on ~/.aws/credentials, but the aws ec2 describe-instances doesn't return anything.
ACC=$(cat ~/.aws/credentials | egrep "\[.*\]" | cut -d '[' -f2 | cut -d ']' -f1 | grep -v default | sort)
FILE="${HOME}/ec2.csv"
echo -e "ACCOUNT;EC2" > ${FILE}
for account in ${ACC}
do
for region in $(aws ec2 describe-regions --profile default --region sa-east-1 --output text | cut -f3)
do
EC2=$(aws ec2 describe-instances --profile ${account} --region ${region} --query "Reservations[*].Instances[*].{PrivateIP:PrivateDnsName,Name:Tags[?Key=='Name']|[0].Value,Status:State.Name}" --output text)
echo -e "${account};${EC2}" >> ${FILE}
done
done
i solved this issue replacing the default profile on the second for.
for region in $(aws ec2 describe-regions --profile ${account} --region sa-east-1 --output text | cut -f3)
This script works, i hope this script can help's.
Related
I am using AWS EC2 CLI to perform a filter on stopped instances, then create an AMI out of these with the AMI name taken from the instance tag.
aws ec2 describe-instances --output text --profile proj --query 'Reservations[*].[Instances[*].[InstanceId, InstanceType, State.Name, Platform, Placement.AvailabilityZone, PublicIpAddress, PrivateIpAddress,[Tags[?Key==`Name`].Value][0][0]]]' --filter --filters Name=instance-state-name,Values=stopped | awk '{print $1, $8}' | xargs -n2 aws ec2 create-image --profile proj --instance-id {} --name {} --no-reboot
how to let args differentiate the two different parameters from AWK (instnaceid, instance name tag), thereby it can be correctly pumped into the ec2 create-image on the instance-id and --name parameter accordingly
You do not need awk.Using AWS CLI, you are extracting 8 values first and then using awk to extract 2 values from that 8 values. Why? Just extract the 2 values from AWS CLI without using awk.
--query 'Reservations[*].[Instances[*].[InstanceId, [Tags[?Key==`Name`].Value][0][0]]]'
will return only the values you are interested in. Then use xargs to pass the arguments to your next command.
xargs -n2 command --arg1 $1 --arg2 $2
Your entire command becomes:
aws ec2 describe-instances --output text --profile proj --query 'Reservations[*].[Instances[*].[InstanceId, [Tags[?Key==`Name`].Value][0][0]]]' --filter --filters Name=instance-state-name,Values=stopped | xargs -n2 aws ec2 create-image --profile proj --instance-id $1 --name $2 --no-reboot
I have a shell script as given below. This script actually add AWS instance in autoscalling scale in protection group. When I run individual commands that went fine. But when I created a shell file and tried to execute same there are error. See below script
set -x
INSTANCE_ID=$(wget -q -O - http://169.254.169.254/latest/meta-data/instance-id)
ASG_NAME=$(aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" --region us-east-2 | jq '.Tags[] | select(.["Key"] | contains("a:autoscaling:groupName")) | .Value')
ASG_NAME=$(echo $ASG_NAME | tr -d '"')
aws autoscaling set-instance-protection --instance-ids $INSTANCE_ID --auto-scaling-group-name $ASG_NAME --protected-from-scale-in --region us-east-2
error is as given below. I think issue is with second line. It is not able to get ASG_NAME, I tried some of escape character but nothing is working.
+++ wget -q -O - http://169.254.169.254/latest/meta-data/instance-id
++ INSTANCE_ID=i-----
+++ aws ec2 describe-tags --filters Name=resource-id,Values=i------ --region us-east-2
+++ jq '.Tags[] | select(.["Key"] | contains("a:autoscaling:groupName")) | .Value'
++ ASG_NAME=
+++ echo
+++ tr -d '"'
++ ASG_NAME=
++ aws autoscaling set-instance-protection --instance-ids i---- --auto-scaling-group-name --protected-from-scale-in --region us-east-2
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: argument --auto-scaling-group-name: expected one argument
> Blockquote
Solved issue by recommendation of #chepner. Modified second line by
ASG_NAME=$(aws ec2 describe-tags --filters "Name=resource-id,Values=$INSTANCE_ID" --region us-east-2 --query 'Tags[1].Value')
I've got a long list of commands strung together (aws cli, jq, cut, sed) that works for me with which I'm trying to now run within a bash script. I'm having problems with the single and double quotes.
If I ran this manually it would be the following:
aws \
--profile app-prod \
--region us-east-1 ec2 describe-instances \
--filter 'Name=tag:ChefRole,Values=foo_bar' \
'Name=instance-state-name,Values=running' \
| jq -r '.Reservations[].Instances[] | [.InstanceId, .Tags[].Value] | #csv' \
| cut -d ',' -f1 \
| sort > sort.out;sed 's/"//g' sort.out
In my script I'm trying to pass in parameters the region and Values
I've tried this:
awsCLI(){
cmd=$(aws --profile app-prod --region ${region} ec2 describe-instances --filter 'Name=tag:ChefRole,Values='${component}'| jq -r '.Reservations[].Instances[] | [.InstanceId, .Tags[].Value] | #csv' | cut -d ',' -f1 | sort > "${_sorted}";sed 's/"//g' "${_sorted}")
}
Placing in the variable - cmd=$(commands here) has worked for me before in other scripts, but not this time.
I also tried just running as is:
aws --profile app-prod --region ${region} ec2 describe-instances --filter 'Name=tag:ChefRole,Values='${component}'| jq -r '.Reservations[].Instances[] | [.InstanceId, .Tags[].Value] | #csv' | cut -d ',' -f1 | sort > "${_sorted}";sed 's/"//g' ${_sorted}"
Either of these didn't work for me. Please advise on best practice for this. Should I break this out? How would you build this long command?
Thanks you.
(Editor's note: first code block split by \ and indented; A.T.)
I was actually trying to eliminate the eternally long horizontal scroll bars from your question, but then decided to convert it into an answer, maybe it actually solves your problem (if not, just tell, will delete):
awsCLI(){
cmd=$(aws \
--profile tealium-prod \
--region "${region}" ec2 describe-instances \
--filter 'Name=tag:ChefRole,Values='"${component}" \
| jq -r '.Reservations[].Instances[] | [.InstanceId, .Tags[].Value] | #csv' \
| cut -d ',' -f1 \
| sort > "${_sorted}" \
; sed 's/"//g' "${_sorted}" \
)
}
I think the main problem was superfluous single tick around Values='"${component}". In general you can concatenate strings by juxtaposition, all you have to remember is that "" expand the variables, whereas '' do not.
The root cause was non-existent indentation. Look up what the backslash in the end of the line does, it will make your life much easier.
If you have the ability to use Python, the boto3 library is very easy to use. The issue with multiple types of quotation marks becomes moot.
I wrote a simple bash script to download my RDS postgres files.
But the kicker is that is all works fine tine terminal, but when I try the same thing in the script I get an error:
An error occurred (DBLogFileNotFoundFault) when calling the DownloadDBLogFilePortion operation: DBLog File: "error/postgresql.log.2017-11-05-23", is not found on the DB instance
The command in question is this:
aws rds download-db-log-file-portion --db-instance-identifier foobar --starting-token 0 --output text --log-file error/postgresql.log.2017-11-05-23 >> test.log
It all works fine, but when I put the exact same line in the bash script I get the error message that there are no db log files - which is nonsense, they are there.
This is the bash script:
download_generate_report() {
for filename in $( aws rds describe-db-log-files --db-instance-identifier $1 | awk {'print $2'} | grep $2 )
do
echo $filename
echo $1
aws rds download-db-log-file-portion --db-instance-identifier $1 --starting-token 0 --output text --log-file $filename >> /home/ubuntu/pgbadger_script/postgres_logs/postgres_$1.log.$2
done
}
Tnx,
Tom
I re-wrote your script a little and it seems to work for me. It barked about grep. This uses jq.
for filename in $( aws rds describe-db-log-files --db-instance-identifier $1 | jq -r '.DescribeDBLogFiles[] | .LogFileName' )
do
aws rds download-db-log-file-portion --db-instance-identifier $1 --output text --no-paginate --log-file $filename >> /tmp/postgres_$1.log.$2
done
Thanks you Ian, I have an issue, with aws cli 2.4, because the log files download truncated.
to solve this I changed --no-paginate with --starting-token 0 more info in the RDS Reference
finally in bash:
#/bin/bash
set -x
for filename in $( aws rds describe-db-log-files --db-instance-identifier $1 | jq -r '.DescribeDBLogFiles[] | .LogFileName' )
do
aws rds download-db-log-file-portion --db-instance-identifier $1 --output text --starting-token 0 --log-file $filename >> $filename
done
When I inside of instance that have Autoscaling Group. I'm trying to select some specific tag of Autoscaling Group. i know that I can use AWS CLI command "aws autoscaling describe-tags".
So I wrote some .sh file. Than contain:
#!/bin/sh
#Instance ID
InstanceID=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/instance-id`
#Allocation ID from Autoscaling tags
Autoscaling=`aws ec2 describe-tags --filters "Name=resource-id,Values=$InstanceID" "Name=key,Values=aws:autoscaling:groupName"`
AllocationID=`aws autoscaling describe-tags --filter "Name=key,Values=$Autoscaling"| select(["Value"] | contains("eipalloc"))`
#Assigning VPC Elastic IP to Instance via Allocation ID
aws ec2 associate-address --instance-id $InstanceID --allocation-id $AllocationID
It doesn't work only because I can't select value in this way. Who can tell me how I can properly select some specific tag?
Thanks!
Looks terrible, but it works:
#!/bin/sh
#Instance ID
InstanceID=`/usr/bin/curl -s http://169.254.169.254/latest/meta-data/instance-id`
#Elastic IP captured through the AllocationID Autoscaling tag
ScalingGroup=`aws ec2 describe-tags --filters "Name=resource-id,Values=$InstanceID" "Name=key,Values=aws:autoscaling:groupName" | jq '.Tags[] | .Value'`
AllocationID=`aws autoscaling describe-tags --filters "Name=auto-scaling-group,Values=$ScalingGroup" | jq '.Tags[] | select(.["Key"] | contains("AllocationID")) | .Value'`
AllocationID=`echo $AllocationID | tr -d '"'`
#Assigning Elastic IP to Instance
aws ec2 associate-address --instance-id $InstanceID --allocation-id $AllocationID