I'm trying to associate the Elastic IP address with Auto scaling group, so whenever the autoscaling triggers it will automatically associate with the EIP.
For this I'm trying to add the script in user data.
My intention is to we have 2 servers so its associated with 2 EIP's, whenever the autoscaling triggers it has to check whether the EIP is free or not if its free it has to associate with that instance using the instance id.
So when i run the script im getting line 19: syntax error: unexpected end of file.
I have checked the indentations but i think its correct.
#!/bin/bash
INSTANCE_ID=$(ec2-metadata --instance-id | cut -d " " -f 2);
EIP_LIST=(eipalloc-07da69856432f7cef eipalloc-0355263fcb50412ed)
for EIP in $${EIP_LIST}; do
echo"Checkin if EIP is free"
ISFREE=$(aws ec2 describe-addresses --allocation-ids $EIP --query Addresses[].InstanceID --output text --region ap-south-1)
STARTWAIT=$(date +%s)
while [ ! -z "$ISFREE" ]; do
if [ "$(($(date +%s) - $STARTWAIT))" -gt $MAXWAIT ]; then
echo "WARNING: We waited for 30 seconds, we are forcing it now."
ISFREE=""
else
echo "checking the other EIP [$EIP]"
ISFREE=$(aws ec2 describe-adresses --allocation-ids $EIP --query Addresses[].InstanceID --output text --region ap-south-1)
fi
done
echo "Running: aws ec2 associate-address --instance-id $INSTANCE_ID --allocation-id $EIP --allow-reassociation --region ap-south-1"
aws ec2 association-address --instance-id $INSTANCE_ID --allocation-id $EIP --allow-reassociation --region ap-south-1
You are missing the final done for the for-loop.
Related
sorry i am still new to bash scripting. I have around 10000 EC2 instance, i have created this bash script to change my EC2 instance type, all instance name and type are stored in a file. the code is working but it is taking so long to run through instance by instance.
does any have knows if i can run AWS Cli command on all EC2 instance in one go ? Thanks :)
#!/bin/bash
my_file='test.txt'
declare -a instanceID
declare -a fmo #Future Instance Size
while IFS=, read -r COL1 COL2; do
instanceID+=("$COL1")
fmo+=("$COL2")
done <"$my_file"
len=${#instanceID[#]}
for (( i=0; i < $len; i++)); do
vm_instance_id="${instanceID[$i]}"
vm_type="${fmo[$i]}"
echo Stoping $vm_instance_id
aws ec2 stop-instances --instance-ids $vm_instance_id
echo " Waiting for $vm_instance_id state to be STOP "
aws ec2 wait instance-stopped --instance-ids $vm_instance_id
echo Resizing $vm_instance_id to $vm_type
aws ec2 modify-instance-attribute --instance-id $vm_instance_id --instance-type $vm_type
echo Starting $vm_instance_id
aws ec2 start-instances --instance-ids $vm_instance_id
done
Refactor your code to a function that is passed a line from the file.
work() {
IFS=, read -r instanceID fmo <<<"$1"
stuff "$instanceID" "$fmo"
}
Run GNU xargs or GNU parallel for each line of file that calls the exported function. Use -P option run the function in paralell, see documentation.
export -f work
xargs -P0 -t bash -c 'work "$#"' -- <"$my_file"
As #KamilCuk pointed here, you can easily make this run in parallel. However, If you run this script in parallel, you might end up getting throttled by EC2, so make sure you include some backoff + retry logic / respect the limits specified here https://docs.aws.amazon.com/AWSEC2/latest/APIReference/throttling.html
This question already has answers here:
How can I loop over the output of a shell command?
(4 answers)
Closed 1 year ago.
I am trying to execute this script as bash test.sh
#!/bin/bash
for region in "aws ec2 describe-regions --output text | cut -f4";
do
echo -e "\nListing Instances in region:'$region'...";
aws ec2 describe-instances --region $region;
done
But the only output that I get for $region is the print of the command. so it says
"Listing Instances aws ec2 describe-regions .. "
How can I fix this ?
You aren't executing a command; you are iterating over a sequence containing exactly one element, the strings aws ec2 ...
You need a command substitution to actually execute the command:
for region in $(aws ec2 describe-regions --output text | cut -f4); do
echo -e ...
aws ec2 describe-instances --region "$region"
done
we have to check for the status of instance and iam trying to capture if any error to logfile. logfile has instance inforamtion but the error is not being writen to logfile below is code let me know what needs to be corrected
function wait-for-status {
instance=$1
target_status=$2
status=unknown
while [[ "$status" != "$target_status" ]]; do
status=`aws rds describe-db-instances \
--db-instance-identifier $instance | head -n 1 \
| awk -F \ '{print $10}'` >> ${log_file} 2>&1
sleep 30
echo $status >> ${log_file}
done
Rather than using all that head/awk stuff, if you want a value out of the CLI, you should use the --query parameter. For example:
aws rds describe-db-instances --db-instance-identifier xxx --query 'DBInstances[*].DBInstanceStatus'
See: Controlling Command Output from the AWS Command Line Interface - AWS Command Line Interface
Also, if your goal is to wait until an Amazon RDS instance is available, then you should use db-instance-available — AWS CLI Command Reference:
aws rds wait db-instance-available --db-instance-identifier xxx
Our application requires the use of HAProxy to load balance and route traffic (one per AZ), ALBs and ELBs are not configurable enough for our purposes. When deploying new code via AWS CodeDeploy, we would like the instances being patched to be placed into Maintenance Mode (removed from load balancing, connections drained). We have modified the default CodeDeploy lifecycle bash scripts to remove the instances from their respective HAProxy instances by sending an SSM Run Command to HAProxy from the instances in question. Currently this modification doesn't work, and the reason for failure is unknown. The script works when executed manually step by step (at least to the current point of failure). The part that fails is either the test that returns "$INSTANCE_ID doesn't seem to be in an AZ with a HAProxy instance, skipping deregistration.", or the setting of $HAPROXY_ID which aforementioned test depends on. The script runs just fine up until that point, but at that point, exits because it can't find the HAProxy instance ID.
I have checked IAM role permissions/credentials, environment variables, and file permissions which all appear to be correct. Normally I would place more logging into the script to debug, but deployments are too few and far between for us to make that practical.
My question: Is there a better way to do this? I can only guess we're not the only ones to use HAProxy with CodeDeploy, and there has to be a reliable method of doing this. Below is the current code being used that is not working.
#!/bin/bash
#
# Copyright 2014 Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License").
# You may not use this file except in compliance with the License.
# A copy of the License is located at
#
# http://aws.amazon.com/apache2.0
#
# or in the "license" file accompanying this file. This file is distributed
# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
# express or implied. See the License for the specific language governing
# permissions and limitations under the License.
. $(dirname $0)/common_functions.sh
if [[ "$DEPLOYMENT_GROUP_NAME" != "redacted" ]]; then
msg "ELB Deregistration doesn't need to happen when not on redacted."
exit
fi
msg "Running AWS CLI with region: $(get_instance_region)"
# get this instance's ID
INSTANCE_ID=$(get_instance_id)
if [ $? != 0 -o -z "$INSTANCE_ID" ]; then
error_exit "Unable to get this instance's ID; cannot continue."
fi
# Get current time
msg "Started $(basename $0) at $(/bin/date "+%F %T")"
start_sec=$(/bin/date +%s.%N)
msg "Checking if instance $INSTANCE_ID is part of an AutoScaling group"
asg=$(autoscaling_group_name $INSTANCE_ID)
if [ $? == 0 -a -n "${asg}" ]; then
msg "Found AutoScaling group for instance $INSTANCE_ID: ${asg}"
msg "Checking that installed CLI version is at least at version required for AutoScaling Standby"
check_cli_version
if [ $? != 0 ]; then
error_exit "CLI must be at least version ${MIN_CLI_X}.${MIN_CLI_Y}.${MIN_CLI_Z} to work with AutoScaling Standby"
fi
msg "Attempting to put instance into Standby"
autoscaling_enter_standby $INSTANCE_ID "${asg}"
if [ $? != 0 ]; then
error_exit "Failed to move instance into standby"
else
msg "Instance is in standby"
fi
fi
msg "Instance is not part of an ASG, continuing..."
## Get the instanceID of the HAProxy instance in this AZ and ENVIRONMENT - Will there ever be more than one???
HAPROXY_ID=$(/usr/local/bin/aws ec2 describe-instances --region us-east-1 --filters "Name=availability-zone,Values=$(/usr/bin/curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)" "Name=tag:deployment_group,Values=haproxy.$ENVIRONMENT" --output text | \
grep INSTANCES | \
awk '{print $8}' )
HAPROXY_IP=$(/usr/local/bin/aws ec2 describe-instances --region us-east-1 --filters "Name=availability-zone,Values=$(/usr/bin/curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)" "Name=tag:deployment_group,Values=haproxy.$ENVIRONMENT" --output text | \
grep INSTANCES | \
awk '{print $13}' )
if test -z "$HAPROXY_ID"; then
msg "$INSTANCE_ID doesn't seem to be in an AZ with a HAProxy instance, skipping deregistration."
exit
fi
## Put the current instance into MAINT mode with the HAProxy instance via SSM
msg "Deregistering $INSTANCE_ID from HAProxy $HAPROXY_ID"
DEREGCMD="{\"commands\":[\"haproxyctl disable server bk_app_servers/$INSTANCEID\"],\"executionTimeout\":[\"3600\"]}"
/usr/local/bin/aws ssm send-command \
--document-name "AWS-RunShellScript" \
--instance-ids "$HAPROXY_ID" \
--parameters "$DEREGCMD" \
--timeout-seconds 600 \
--output-s3-bucket-name "redacted" \
--output-s3-key-prefix "haproxy-codedeploy/deregister" \
--region us-east-1
if [ $? != 0 ]; then
error_exit "Failed to send SSM command to deregister instance $INSTANCE_ID from HAProxy $HAPROXY_ID"
fi
## Wait for all connections to drain from instance
SESS_COUNT=$(/usr/bin/curl -s "http://$HAPROXY_IP:<portredacted>/<urlredacted>" | grep $INSTANCEID | awk -F "," '{print $5}')
DRAIN_TIME=60
msg "Initial session count: $SESS_COUNT"
while [[ "$SESS_COUNT" -gt 0 ]]; do
if [[ "$COUNTER" -gt "$DRAIN_TIME" ]]; then
msg "Instance failed to drain all connections within $DRAIN_TIME seconds. Continuing to deploy anyway."
break
fi
msg $SESS_COUNT
sleep 1
COUNTER=$(($COUNTER + 1))
SESS_COUNT=$(/usr/bin/curl -s "http://$HAPROXY_IP:<portredacted>/<urlredacted>" | grep $INSTANCEID | awk -F "," '{print $5}')
done
msg "Finished $(basename $0) at $(/bin/date "+%F %T")"
end_sec=$(/bin/date +%s.%N)
elapsed_seconds=$(echo "$end_sec - $start_sec" | /usr/bin/bc)
msg "Elapsed time: $elapsed_seconds"
At the moment the only option for you is to add more logging and issue a deployment to test out this script and then look at your deployment logs. It sounds like you don't know why it's failing and only the logs can tell you that.
Try adding logging and seeing what happens. We should be just executing your script as is so it shouldn't act any differently but it's hard to tell without seeing the logs.
Good luck,
-Asaf
I'm getting a list of EC2 instances and then trying to loop through them but for some reason I'm not able to get the loop to work.
output="$(aws ec2 describe-instances --filters 'Name=tag:Environment,Values=development' --query '[Reservations[*].Instances[*].PublicDnsName]' --output text)"
echo $output displays something like:
ec2-55-55-555-555.eu-west-1.compute.amazonaws.com
ec2-66-66-666-666.eu-west-1.compute.amazonaws.com
Then I create an array like this:
instances=(${output//'\n'/ })
echo ${instances[0]} and echo ${instances[1]} gives the correct output.
And then try to iterate through the array:
for i in $instances; do echo instance: "$i"; done
But I get:
instance: ec2-55-55-555-555.eu-west-1.compute.amazonaws.com
ec2-66-66-666-666.eu-west-1.compute.amazonaws.com
Instead of:
instance: ec2-55-55-555-555.eu-west-1.compute.amazonaws.com
instance: ec2-66-66-666-666.eu-west-1.compute.amazonaws.com
What am I doing wrong? And is there a better way to loop through the results, maybe rather using the json output format?
I am not sure if you got an answer for this question. Will this help?
for dns in $(aws ec2 describe-instances --region ap-northeast-1 --query 'Reservations[*].Instances[*].PublicDnsName' --output text) ; do echo $dns ; done
For windows cli:
aws ec2 describe-instances --query "Reservations[].Instances[].InstanceId" > instances
FOR /f %i IN (instances) DO aws ec2 terminate-instances --instance-ids %i
This also works for me, add it to a bash array:
for instance in $(aws ec2 describe-instances --filters "Name=tag:Application,Values=yourValue" "Name=tag:Environment,Values=development" --query 'Reservations[*].Instances[*].InstanceId' --output text); do envInstances+=(${instance}); done
for i in ${envInstances[#]}; do echo "hello $i"; done
hello i-instance1
hello i-instance12
hello i-instance16
Not sure if you just gave up, but in BASH, better to just do this:
OUTPUT=($(aws ec2 describe-instances --filters
'Name=tag:Environment,Values=development' --query
'[Reservations[].Instances[].PublicDnsName]' --output text))
then (if you want a count)
echo "${#OUTPUT[#]} instances found."
Good luck!