This question already has answers here:
How can I loop over the output of a shell command?
(4 answers)
Closed 1 year ago.
I am trying to execute this script as bash test.sh
#!/bin/bash
for region in "aws ec2 describe-regions --output text | cut -f4";
do
echo -e "\nListing Instances in region:'$region'...";
aws ec2 describe-instances --region $region;
done
But the only output that I get for $region is the print of the command. so it says
"Listing Instances aws ec2 describe-regions .. "
How can I fix this ?
You aren't executing a command; you are iterating over a sequence containing exactly one element, the strings aws ec2 ...
You need a command substitution to actually execute the command:
for region in $(aws ec2 describe-regions --output text | cut -f4); do
echo -e ...
aws ec2 describe-instances --region "$region"
done
Related
I'm creating ec2 instances and would like to get the user_data for each instance using the command:
aws ec2 describe-instance-attribute --instance-id i-xxxx --attribute userData --output text --query "UserData.Value" | base64 --decode > file.txt
When running this directly via terminal it works, I'm able to get the userData printed into file.txt. However, I need this to run in a shell script I have, which gets the instance id as a parameter.
The lines in test.sh are the following:
#!/bin/bash
echo "$(aws ec2 describe-instance-attribute --instance-id $1 --attribute userData --output text --query "UserData.Value" | base64 --decode)" > file.txt
Where $1 is the instance-id. When running:
./test.sh i-xxxxxxx
It creates an empty file.txt. I have changed the line in the script to:
echo "$(aws ec2 describe-instance-attribute --instance-id $1 --attribute userData --output text --query "UserData.Value" | base64 --decode)"
and it prints the userData to stdout. So why it is not working for file redirection?
Thank you,
sorry i am still new to bash scripting. I have around 10000 EC2 instance, i have created this bash script to change my EC2 instance type, all instance name and type are stored in a file. the code is working but it is taking so long to run through instance by instance.
does any have knows if i can run AWS Cli command on all EC2 instance in one go ? Thanks :)
#!/bin/bash
my_file='test.txt'
declare -a instanceID
declare -a fmo #Future Instance Size
while IFS=, read -r COL1 COL2; do
instanceID+=("$COL1")
fmo+=("$COL2")
done <"$my_file"
len=${#instanceID[#]}
for (( i=0; i < $len; i++)); do
vm_instance_id="${instanceID[$i]}"
vm_type="${fmo[$i]}"
echo Stoping $vm_instance_id
aws ec2 stop-instances --instance-ids $vm_instance_id
echo " Waiting for $vm_instance_id state to be STOP "
aws ec2 wait instance-stopped --instance-ids $vm_instance_id
echo Resizing $vm_instance_id to $vm_type
aws ec2 modify-instance-attribute --instance-id $vm_instance_id --instance-type $vm_type
echo Starting $vm_instance_id
aws ec2 start-instances --instance-ids $vm_instance_id
done
Refactor your code to a function that is passed a line from the file.
work() {
IFS=, read -r instanceID fmo <<<"$1"
stuff "$instanceID" "$fmo"
}
Run GNU xargs or GNU parallel for each line of file that calls the exported function. Use -P option run the function in paralell, see documentation.
export -f work
xargs -P0 -t bash -c 'work "$#"' -- <"$my_file"
As #KamilCuk pointed here, you can easily make this run in parallel. However, If you run this script in parallel, you might end up getting throttled by EC2, so make sure you include some backoff + retry logic / respect the limits specified here https://docs.aws.amazon.com/AWSEC2/latest/APIReference/throttling.html
i am trying to get the subnet ids within a particular VPC and store them in variables
so I can use them in a bash script
aws ec2 describe-subnets --filter "Name=vpc-id,Values=VPCid" --region $REGION --query "Subnets[*].SubnetId" --output text
and this gives something like this
subnet-12345 subnet-78910
(END)
I wonder how I can store them into a variable.
I tried with
SBnet=$(aws ec2 describe-subnets --filter "Name=vpc-id,Values=VPCid" --region $REGION --query "Subnets[*].SubnetId" --output text)
but then I do not know I can access the array/list created.
I tried with
echo $(SBnet[0])
but does not work
I am on MACos usin zsh
You can do this as follows (add your VPC and the region):
#!/bin/bash
SUBNET_IDS=$(aws ec2 describe-subnets --filter "Name=vpc-id,Values=vpc-1234" --query "Subnets[*].SubnetId" --output text)
for SUBNET_ID in $SUBNET_IDS;
do
echo $SUBNET_ID
done
To split the list of subnet IDs into variables, you can do this:
#!/bin/bash
SUBNET_IDS=$(aws ec2 describe-subnets --filter "Name=vpc-id,Values=vpc-1234" --query "Subnets[*].SubnetId" --output text)
IFS=$'\t ' read -r -a subnet_ids <<< $SUBNET_IDS
echo "${subnet_ids[0]}"
echo "${subnet_ids[1]}"
And the individual subnet IDs will be in the subnet_ids array.
you can do as #jarmod suggested and you could also write a query to extract all the subnets tied to all the VPC's in your system in a comma separated output and use it further like this
aws ec2 describe-subnets --query "Subnets[].[SubnetId,VpcId,CidrBlock,AvailabilityZone]" --output text|sed 's/\t/,/g'
I am using AWS EC2 CLI to perform a filter on stopped instances, then create an AMI out of these with the AMI name taken from the instance tag.
aws ec2 describe-instances --output text --profile proj --query 'Reservations[*].[Instances[*].[InstanceId, InstanceType, State.Name, Platform, Placement.AvailabilityZone, PublicIpAddress, PrivateIpAddress,[Tags[?Key==`Name`].Value][0][0]]]' --filter --filters Name=instance-state-name,Values=stopped | awk '{print $1, $8}' | xargs -n2 aws ec2 create-image --profile proj --instance-id {} --name {} --no-reboot
how to let args differentiate the two different parameters from AWK (instnaceid, instance name tag), thereby it can be correctly pumped into the ec2 create-image on the instance-id and --name parameter accordingly
You do not need awk.Using AWS CLI, you are extracting 8 values first and then using awk to extract 2 values from that 8 values. Why? Just extract the 2 values from AWS CLI without using awk.
--query 'Reservations[*].[Instances[*].[InstanceId, [Tags[?Key==`Name`].Value][0][0]]]'
will return only the values you are interested in. Then use xargs to pass the arguments to your next command.
xargs -n2 command --arg1 $1 --arg2 $2
Your entire command becomes:
aws ec2 describe-instances --output text --profile proj --query 'Reservations[*].[Instances[*].[InstanceId, [Tags[?Key==`Name`].Value][0][0]]]' --filter --filters Name=instance-state-name,Values=stopped | xargs -n2 aws ec2 create-image --profile proj --instance-id $1 --name $2 --no-reboot
I'm getting a list of EC2 instances and then trying to loop through them but for some reason I'm not able to get the loop to work.
output="$(aws ec2 describe-instances --filters 'Name=tag:Environment,Values=development' --query '[Reservations[*].Instances[*].PublicDnsName]' --output text)"
echo $output displays something like:
ec2-55-55-555-555.eu-west-1.compute.amazonaws.com
ec2-66-66-666-666.eu-west-1.compute.amazonaws.com
Then I create an array like this:
instances=(${output//'\n'/ })
echo ${instances[0]} and echo ${instances[1]} gives the correct output.
And then try to iterate through the array:
for i in $instances; do echo instance: "$i"; done
But I get:
instance: ec2-55-55-555-555.eu-west-1.compute.amazonaws.com
ec2-66-66-666-666.eu-west-1.compute.amazonaws.com
Instead of:
instance: ec2-55-55-555-555.eu-west-1.compute.amazonaws.com
instance: ec2-66-66-666-666.eu-west-1.compute.amazonaws.com
What am I doing wrong? And is there a better way to loop through the results, maybe rather using the json output format?
I am not sure if you got an answer for this question. Will this help?
for dns in $(aws ec2 describe-instances --region ap-northeast-1 --query 'Reservations[*].Instances[*].PublicDnsName' --output text) ; do echo $dns ; done
For windows cli:
aws ec2 describe-instances --query "Reservations[].Instances[].InstanceId" > instances
FOR /f %i IN (instances) DO aws ec2 terminate-instances --instance-ids %i
This also works for me, add it to a bash array:
for instance in $(aws ec2 describe-instances --filters "Name=tag:Application,Values=yourValue" "Name=tag:Environment,Values=development" --query 'Reservations[*].Instances[*].InstanceId' --output text); do envInstances+=(${instance}); done
for i in ${envInstances[#]}; do echo "hello $i"; done
hello i-instance1
hello i-instance12
hello i-instance16
Not sure if you just gave up, but in BASH, better to just do this:
OUTPUT=($(aws ec2 describe-instances --filters
'Name=tag:Environment,Values=development' --query
'[Reservations[].Instances[].PublicDnsName]' --output text))
then (if you want a count)
echo "${#OUTPUT[#]} instances found."
Good luck!