I have written cloudformation which consists of an EC2 instance and a RDS database. I wish to get the endpoint address from the RDS, which i can via the intrinsic function GetAtt, and pass it into the EC2 instance in a text file. Below is the code i've tried however its not working and giving me what i want.
Resources:
MyEC2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-74e6b80d
InstanceType: t2.nano
SecurityGroups:
- Ref: MySecurityGroup
KeyName:
- Ref: MyKeyPair
UserData:
Fn::Base64:
Fn::Sub:
- |
#!/bin/bash
touch /home/ubuntu/touch1.txt
- DatabaseAddress: !GetAtt MyDatabase.Endpoint.Address
echo "My address is ${DatabaseAddress}" >> touch1.txt
DependsOn: MyDatabase
You can fetch attributes from a resource without the use of GetAtt while in a Sub function.
Example:
UserData:
Fn::Base64:
Fn::Sub:
- |
#!/bin/bash
touch /home/ubuntu/touch1.txt
echo "My address is ${MyDatabase.Endpoint.Address}" >> /home/ubuntu/touch1.txt
Related
I have fully automation to setup my infrastructure needs on aws with cloudformation. But I'm stuck to change hostname from default hostname ip-192.123.2.1 to something meaningful string name in the template.
When I do this manually its working but when I try to do with userdata on cloudformation template. It does not work. What am I missing in the template, wrong syntax, wrong usage?
First Attempt:
Resources:
EC2Instance:
Type: 'AWS::EC2::Instance'
Properties:
[....]
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
echo "preserve_hostname: true" >> /etc/cloud/cloud.cfg
hostnamectl set-hostname myserver.localdomain
echo "HOSTNAME=myserver.localdomain" >> /etc/sysconfig/network
echo "127.0.0.1 myserver.localdomain" >> /etc/hosts
Second Attempt:
Resources:
EC2Instance:
Type: 'AWS::EC2::Instance'
Properties:
[....]
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
sed -i '15s/false/true/g' /etc/cloud/cloud.cfg
hostnamectl set-hostname --static myserver.localdomain
echo "127.0.0.1 myserver.localdomain" >> /etc/hosts
Third Attempt:
Resources:
EC2Instance:
Type: 'AWS::EC2::Instance'
Properties:
[....]
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
fqdn: myserver.localdomain
runcmd:
- echo "preserve_hostname: true" >> /etc/cloud/cloud.cfg
I am trying to fetch the dns of internal classic load balancer and pass it in the launch configuration of ec2 instance as a user data. Creating a db in ec2 instance and in the config file trying to get the dns of the internal classic load balancer but I am unable to get it. !GetAtt InternalLB.DNSName I have used this in the user data but the db is not connected but when I manually pass the dns it works.I need to fetch the dns on its on using this userdata script.
ec2instance:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
# AvailabilityZone: us-east-2a
UserData:
Fn::Base64: !Sub |
#!/bin/bash
cd /var/www
mkdir inc
cd inc
sudo echo "<?php
define('DB_SERVER', '!GetAtt InternalLB.DNSName');
define('DB_USERNAME', 'db');
define('DB_PASSWORD', 'db');
define('DB_DATABASE', 'db');
?>" > dbinfo.inc
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
DeleteOnTermination: "true"
VolumeSize: "8"
VolumeType: gp2
ImageId: ami-0bdcc6c05dec346bf
InstanceType: t2.micro
KeyName: wahaj(webserver)
SecurityGroups:
- Ref: wahajwebserver
myASG:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
AvailabilityZones:
- "us-east-2a"
- "us-east-2b"
AutoScalingGroupName: myASG
LoadBalancerNames:
- Fn::ImportValue: !Sub "${elb}-MyLoadBalancer"
MinSize: "2"
MaxSize: "2"
DesiredCapacity: "2"
HealthCheckGracePeriod: 300
LaunchConfigurationName:
Ref: ec2instance
VPCZoneIdentifier:
- Fn::ImportValue: !Sub "${SourceStackName}-SubnetC"
- Fn::ImportValue: !Sub "${SourceStackName}-SubnetD"
internalelbsg:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: internal-elb
VpcId:
Fn::ImportValue:
Fn::Sub: "${SourceStackName}-VpcID"
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 3306
ToPort: 3306
SourceSecurityGroupId: !GetAtt wahajwebserver.GroupId
Description: For traffic from Internet
GroupDescription: Security Group for demo server
InternalLB:
Type: AWS::ElasticLoadBalancing::LoadBalancer
Properties:
Scheme: internal
Listeners:
- LoadBalancerPort: "80"
InstancePort: "80"
Protocol: HTTP
- LoadBalancerPort: "3306"
InstancePort: "3306"
Protocol: TCP
SecurityGroups:
- !Ref internalelbsg
LoadBalancerName: internalelbsg
Subnets:
- Fn::ImportValue: !Sub "${SourceStackName}-SubnetC"
- Fn::ImportValue: !Sub "${SourceStackName}-SubnetD"
HealthCheck:
Target: HTTP:80/index.html
HealthyThreshold: "3"
UnhealthyThreshold: "5"
Interval: "30"
Timeout: "5"
The main issue is in define('DB_SERVER', '!GetAtt InternalLB.DNSName'); this line I am not fetching the dns in the right way. Please help
To use GetAtt inside of Sub, you have to use the VarName: VarValue syntax as described in the Fn::Sub documentation page. Additionally, you have to wrap VarName with ${}, i.e. ${VarName}.
I believe this syntax should work:
UserData:
Fn::Base64: !Sub
- |
#!/bin/bash
cd /var/www
mkdir inc
cd inc
sudo echo "<?php
define('DB_SERVER', '${InternalLBDNSName}');
define('DB_USERNAME', 'db');
define('DB_PASSWORD', 'db');
define('DB_DATABASE', 'db');
?>" > dbinfo.inc
- InternalLBDNSName: !GetAtt InternalLB.DNSName
I have a YAML Cloudformation script which launches a single EC2 instance and runs some UserData upon startup.
I am using ami-0727f3c2d4b0226d5, a standard Ubuntu 18:04 LTS server.
Everything works fine provided the UserData is simple, eg -
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -ex
echo "Hello World EC2!"
which gives me the following in the EC2 system log -
[[0;32m OK [0m] Started Apply the settings specified in cloud-config.
Starting Execute cloud user/final scripts...
[ 21.827930] cloud-init[1307]: + echo 'Hello World EC2!'
[ 21.832906] cloud-init[1307]: Hello World EC2!
but if I extend the UserData for some fairly normal- looking Ubuntu commands -
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -ex
apt-get update
apt-get install -y ruby
echo "Hello World EC2!"
then (having torn down the original machine and restarted a new instance from scratch) the UserData process seems to hang with the following messages in the system log -
[ 29.606055] cloud-init[1304]: + apt-get install -y ruby
[ 29.675005] cloud-init[1304]: Reading package lists...
[ 29.828430] cloud-init[1304]: Building dependency tree...
[ 29.836236] cloud-init[1304]: Reading state information...
[ ... ]
[ ... ]
[ ... ]
[ 34.233706] cloud-init[1304]: Checking for services that may need to be restarted...done.
[ 34.254767] cloud-init[1304]: Checking for services that may need to be restarted...done.
[ 34.262182] cloud-init[1304]: Checking init scripts...
ie Checking init scripts ... never returns. Any thoughts on how to debug this situation / find out what is going wrong ?
TIA
[full YAML CF included]
---
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
AppName:
Type: String
InstanceType:
Type: String
Default: t2.micro
ImageId:
Type: String
Default: ami-0727f3c2d4b0226d5 # 18.04 LTS eu-west-1
KeyName:
Type: String
Outputs:
MyDNSName:
Value:
Fn::GetAtt:
- AppEC2Instance
- PublicDnsName
Description: "EC2 public DNS name"
MyIPAddress:
Value:
Fn::GetAtt:
- AppEC2Instance
- PublicIp
Description: "EC2 public IP address"
MyInstanceId:
Value:
Ref: AppEC2Instance
Description: "EC2 instance id"
Resources:
AppEC2Instance:
Properties:
IamInstanceProfile:
Ref: AppInstanceProfile
ImageId:
Ref: ImageId
InstanceType:
Ref: InstanceType
KeyName:
Ref: KeyName
SecurityGroupIds:
- Fn::GetAtt:
- AppSecurityGroup
- GroupId
SubnetId:
Ref: AppSubnet
Tags:
- Key: Name
Value:
Ref: AppName
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -ex
apt-get update
apt-get install -y ruby
echo "Hello World EC2!"
Type: AWS::EC2::Instance
AppInstanceProfile:
Properties:
Path: /
Roles:
- Ref: AppInstanceRole
Type: AWS::IAM::InstanceProfile
AppInstanceRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Version: '2012-10-17'
Path: /
Policies:
- PolicyDocument:
Statement:
- Action:
- ec2:DescribeTags # allow codedeploy to find machine
Effect: Allow
Resource: '*'
- Action: s3:* # allow machine to access deployables
Effect: Allow
Resource: '*'
- Action: logs:*
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName: # required
Fn::Join:
- '-'
- - Ref: AppName
- ec2
Type: AWS::IAM::Role
AppSecurityGroup:
Properties:
GroupDescription:
Ref: AppName
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
FromPort: '3000'
IpProtocol: tcp
ToPort: '3000'
VpcId:
Ref: AppVPC
Type: AWS::EC2::SecurityGroup
AppInternetGateway:
Type: AWS::EC2::InternetGateway
AppRoute:
DependsOn: AppInternetGateway
Properties:
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: AppInternetGateway
RouteTableId:
Ref: AppRouteTable
Type: AWS::EC2::Route
AppRouteTable:
Properties:
VpcId:
Ref: AppVPC
Type: AWS::EC2::RouteTable
AppSubnet:
Properties:
CidrBlock: 172.31.0.0/20
MapPublicIpOnLaunch: true
VpcId:
Ref: AppVPC
Type: AWS::EC2::Subnet
AppSubnetRouteTableAssociation:
Properties:
RouteTableId:
Ref: AppRouteTable
SubnetId:
Ref: AppSubnet
Type: AWS::EC2::SubnetRouteTableAssociation
AppVPC:
Properties:
CidrBlock: 172.31.0.0/16
EnableDnsHostnames: true
EnableDnsSupport: true
InstanceTenancy: default
Type: AWS::EC2::VPC
AppVPCGatewayAttachment:
Properties:
InternetGatewayId:
Ref: AppInternetGateway
VpcId:
Ref: AppVPC
Type: AWS::EC2::VPCGatewayAttachment
AppCodeDeployApp:
Properties:
ApplicationName:
Ref: AppName
Type: AWS::CodeDeploy::Application
AppCodeDeployGroup:
Properties:
ApplicationName:
Ref: AppCodeDeployApp
DeploymentConfigName: CodeDeployDefault.AllAtOnce
DeploymentGroupName:
Ref: AppName
Ec2TagFilters: # lookup ec2 machine for deployment
- Key: Name
Type: KEY_AND_VALUE
Value:
Ref: AppName
ServiceRoleArn:
Fn::GetAtt:
- AppCodeDeployRole
- Arn
Type: AWS::CodeDeploy::DeploymentGroup
AppCodeDeployRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- codedeploy.amazonaws.com
Version: '2012-10-17'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSCodeDeployRole
Path: /
Type: AWS::IAM::Role
So i was trying to over ride the lambda environment, i used string interpolation, but there is a small thing i am unable to understand, so basically following is my Lambda, if you see the function name it has a place holder for Environment. But when i deploy it like this
aws cloudformation deploy --template-file build/output.yaml --stack-name test-stack --capabilities CAPABILITY_IAM --parameter-overrides Environment=de
v
The placeholder doesn't update the following code
Parameters:
Environment:
Type: String
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: src
Handler: index.lambda_handler
Runtime: python3.6
FunctionName: HelloLambda-${Environment}
MemorySize: 128
Timeout: 30
Policies:
- AWSLambdaBasicExecutionRole
But if i do same like this
Parameters:
Environment:
Type: String
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: src
Handler: index.lambda_handler
Runtime: python3.6
FunctionName: !Sub HelloLambda-${Environment}
MemorySize: 128
Timeout: 30
Policies:
- AWSLambdaBasicExecutionRole
The above execution works, so whats the difference between FunctionName: !Sub HelloLambda-${Environment} and FunctionName: HelloLambda-${Environment}
By having !Sub in the front, you are invoking Sub-function with cloud formation. It takes the template parameters and applies the replacement where needed.
More documentation at,
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-sub.html
!Sub
HelloLambda-${Environment}
takes the Environment variable and replace with the value specified, thus you get different functions based on the environment variable.
So I have this CloudFormation resource in my networking template:
Resources:
...
PubSubnetAz2:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref Vpc
CidrBlock: !FindInMap [VpcCidrs, !Ref "AWS::Region", pubsubnet2]
AvailabilityZone: !Select
- 1
- Fn::GetAZs: !Ref "AWS::Region"
I'm getting this error when I try to create this stack:
17:40:06 UTC-0700 CREATE_FAILED AWS::EC2::Subnet PubSubnetAz2 Template error: Fn::Select cannot select nonexistent value at index 1
The template validates, I have a PubSubnetAz1 block which is identical and passes, (it selects index=0 though).
Am I using Fn::GetAZs wrong?
PS. I am using us-west-2 region, as far as I can tell that has > 1 AZs.
The following template successfully deployed a VPC across multiple subnets in us-west-2:
---
AWSTemplateFormatVersion: '2010-09-09'
Resources:
vpc1:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
InstanceTenancy: default
EnableDnsSupport: true
EnableDnsHostnames: false
Tags:
- Key: Name
Value: My-VPC
subnet1:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ''
CidrBlock: 10.0.0.0/24
VpcId:
Ref: vpc1
Tags:
- Key: Name
Value: Subnet-A
subnet2:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ''
CidrBlock: 10.0.1.0/24
VpcId:
Ref: vpc1
Tags:
- Key: Name
Value: Subnet-B
I created the VPC manually, used Hava to convert it to a JSON CloudFormation template, manually inserted the Select statements, then converted to YAML using json2yaml.com.
I was getting a similar error while doing AWS Mysfits sample demo, for a second time. I found I needed to list the AZ I wished to use and the stack created successfully.
PrivateSubnetTwo:
Type: AWS::EC2::Subnet
Properties:
AvailabilityZone: us-east-2b
VpcId: !Ref 'VPC'
CidrBlock: !FindInMap ['SubnetConfig', 'PrivateTwo', 'CIDR']
The Fn::GetAZs documentation shows examples like:
mySubnet:
Type: "AWS::EC2::Subnet"
Properties:
VpcId:
!Ref VPC
CidrBlock: 10.0.0.0/24
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
It has an empty Region, which apparently is equivalent to specifying AWS::Region.
It also shows an example using the full format as:
"Fn::GetAZs": ""
"Fn::GetAZs": { Ref: "AWS::Region" }
"Fn::GetAZs": "us-east-1"
Try some of those formats to see whether they work.