I have fully automation to setup my infrastructure needs on aws with cloudformation. But I'm stuck to change hostname from default hostname ip-192.123.2.1 to something meaningful string name in the template.
When I do this manually its working but when I try to do with userdata on cloudformation template. It does not work. What am I missing in the template, wrong syntax, wrong usage?
First Attempt:
Resources:
EC2Instance:
Type: 'AWS::EC2::Instance'
Properties:
[....]
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
echo "preserve_hostname: true" >> /etc/cloud/cloud.cfg
hostnamectl set-hostname myserver.localdomain
echo "HOSTNAME=myserver.localdomain" >> /etc/sysconfig/network
echo "127.0.0.1 myserver.localdomain" >> /etc/hosts
Second Attempt:
Resources:
EC2Instance:
Type: 'AWS::EC2::Instance'
Properties:
[....]
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
sed -i '15s/false/true/g' /etc/cloud/cloud.cfg
hostnamectl set-hostname --static myserver.localdomain
echo "127.0.0.1 myserver.localdomain" >> /etc/hosts
Third Attempt:
Resources:
EC2Instance:
Type: 'AWS::EC2::Instance'
Properties:
[....]
UserData:
Fn::Base64:
!Sub |
#!/bin/bash
fqdn: myserver.localdomain
runcmd:
- echo "preserve_hostname: true" >> /etc/cloud/cloud.cfg
Related
Resources:
MyFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
FunctionName: myFunction
Policies:
- !Ref Policy1
I have an example yaml as such, and I'd like to append Policy2 into the Policies block for all resources that has Type: AWS::Serverless::Function
Following is the command I have:
yq '(.Resources.[] | select(.Type=="AWS::Serverless::Function") | .Properties.Policies) += ["!Ref Policy2"]' example.yaml
And this ends up with:
Resources:
MyFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
FunctionName: myFunction
Policies:
- !Ref Policy1
- '!Ref Policy2'
Where !Ref Policy2 is single quoted. Any guidance on how I can append the policy without it being single quoted?
Thank you
!Ref is a tag, and not part of the string value. Set it explicitly:
yq '
(.Resources.[] | select(.Type=="AWS::Serverless::Function").Properties.Policies
) += ["Policy2" | . tag = "!Ref"]
' example.yaml
Resources:
MyFunction:
Type: AWS::Serverless::Function
Properties:
PackageType: Image
FunctionName: myFunction
Policies:
- !Ref Policy1
- !Ref Policy2
I run this Job:
kubectl apply -f - <<EOF
apiVersion: batch/v1
kind: Job
metadata:
name: sample
spec:
template:
spec:
containers:
- command:
- /bin/bash
- -c
- |
env
echo "MY_VAR : ${MY_VAR}"
sleep 800000
env:
- name: MY_VAR
value: MY_VALUE
image: mcr.microsoft.com/azure-cli:2.0.80
imagePullPolicy: IfNotPresent
name: sample
restartPolicy: Never
backoffLimit: 4
EOF
But when I look at the log the value MY_VALUE its empty even though env prints it:
$ kubectl logs -f sample-7p6bp
...
MY_VAR=MY_VALUE
...
MY_VAR :
Why does this line contain an empty value for ${MY_VAR}:
echo "MY_VAR : ${MY_VAR}"
?
UPDATE: Tried the same with a simple pod:
kubectl -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
name: sample
spec:
containers:
- name: sample
imagePullPolicy: Always
command: ["/bin/sh", "-c", "echo BEGIN ${MY_VAR} END"]
image: radial/busyboxplus:curl
env:
- name: MY_VAR
value: MY_VALUE
EOF
Same/empty result:
$ kubectl logs -f sample
BEGIN END
The reason this happens is because your shell expands the variable ${MY_VAR} before it's ever sent to the kubernetes. You can disable parameter expansion inside of a heredoc by quoting the terminator:
kubectl apply -f - <<'EOF'
Adding these quotes should resolve your issue.
I am trying to fetch the dns of internal classic load balancer and pass it in the launch configuration of ec2 instance as a user data. Creating a db in ec2 instance and in the config file trying to get the dns of the internal classic load balancer but I am unable to get it. !GetAtt InternalLB.DNSName I have used this in the user data but the db is not connected but when I manually pass the dns it works.I need to fetch the dns on its on using this userdata script.
ec2instance:
Type: AWS::AutoScaling::LaunchConfiguration
Properties:
# AvailabilityZone: us-east-2a
UserData:
Fn::Base64: !Sub |
#!/bin/bash
cd /var/www
mkdir inc
cd inc
sudo echo "<?php
define('DB_SERVER', '!GetAtt InternalLB.DNSName');
define('DB_USERNAME', 'db');
define('DB_PASSWORD', 'db');
define('DB_DATABASE', 'db');
?>" > dbinfo.inc
BlockDeviceMappings:
- DeviceName: /dev/xvda
Ebs:
DeleteOnTermination: "true"
VolumeSize: "8"
VolumeType: gp2
ImageId: ami-0bdcc6c05dec346bf
InstanceType: t2.micro
KeyName: wahaj(webserver)
SecurityGroups:
- Ref: wahajwebserver
myASG:
Type: AWS::AutoScaling::AutoScalingGroup
Properties:
AvailabilityZones:
- "us-east-2a"
- "us-east-2b"
AutoScalingGroupName: myASG
LoadBalancerNames:
- Fn::ImportValue: !Sub "${elb}-MyLoadBalancer"
MinSize: "2"
MaxSize: "2"
DesiredCapacity: "2"
HealthCheckGracePeriod: 300
LaunchConfigurationName:
Ref: ec2instance
VPCZoneIdentifier:
- Fn::ImportValue: !Sub "${SourceStackName}-SubnetC"
- Fn::ImportValue: !Sub "${SourceStackName}-SubnetD"
internalelbsg:
Type: AWS::EC2::SecurityGroup
Properties:
GroupName: internal-elb
VpcId:
Fn::ImportValue:
Fn::Sub: "${SourceStackName}-VpcID"
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 3306
ToPort: 3306
SourceSecurityGroupId: !GetAtt wahajwebserver.GroupId
Description: For traffic from Internet
GroupDescription: Security Group for demo server
InternalLB:
Type: AWS::ElasticLoadBalancing::LoadBalancer
Properties:
Scheme: internal
Listeners:
- LoadBalancerPort: "80"
InstancePort: "80"
Protocol: HTTP
- LoadBalancerPort: "3306"
InstancePort: "3306"
Protocol: TCP
SecurityGroups:
- !Ref internalelbsg
LoadBalancerName: internalelbsg
Subnets:
- Fn::ImportValue: !Sub "${SourceStackName}-SubnetC"
- Fn::ImportValue: !Sub "${SourceStackName}-SubnetD"
HealthCheck:
Target: HTTP:80/index.html
HealthyThreshold: "3"
UnhealthyThreshold: "5"
Interval: "30"
Timeout: "5"
The main issue is in define('DB_SERVER', '!GetAtt InternalLB.DNSName'); this line I am not fetching the dns in the right way. Please help
To use GetAtt inside of Sub, you have to use the VarName: VarValue syntax as described in the Fn::Sub documentation page. Additionally, you have to wrap VarName with ${}, i.e. ${VarName}.
I believe this syntax should work:
UserData:
Fn::Base64: !Sub
- |
#!/bin/bash
cd /var/www
mkdir inc
cd inc
sudo echo "<?php
define('DB_SERVER', '${InternalLBDNSName}');
define('DB_USERNAME', 'db');
define('DB_PASSWORD', 'db');
define('DB_DATABASE', 'db');
?>" > dbinfo.inc
- InternalLBDNSName: !GetAtt InternalLB.DNSName
I have a YAML Cloudformation script which launches a single EC2 instance and runs some UserData upon startup.
I am using ami-0727f3c2d4b0226d5, a standard Ubuntu 18:04 LTS server.
Everything works fine provided the UserData is simple, eg -
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -ex
echo "Hello World EC2!"
which gives me the following in the EC2 system log -
[[0;32m OK [0m] Started Apply the settings specified in cloud-config.
Starting Execute cloud user/final scripts...
[ 21.827930] cloud-init[1307]: + echo 'Hello World EC2!'
[ 21.832906] cloud-init[1307]: Hello World EC2!
but if I extend the UserData for some fairly normal- looking Ubuntu commands -
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -ex
apt-get update
apt-get install -y ruby
echo "Hello World EC2!"
then (having torn down the original machine and restarted a new instance from scratch) the UserData process seems to hang with the following messages in the system log -
[ 29.606055] cloud-init[1304]: + apt-get install -y ruby
[ 29.675005] cloud-init[1304]: Reading package lists...
[ 29.828430] cloud-init[1304]: Building dependency tree...
[ 29.836236] cloud-init[1304]: Reading state information...
[ ... ]
[ ... ]
[ ... ]
[ 34.233706] cloud-init[1304]: Checking for services that may need to be restarted...done.
[ 34.254767] cloud-init[1304]: Checking for services that may need to be restarted...done.
[ 34.262182] cloud-init[1304]: Checking init scripts...
ie Checking init scripts ... never returns. Any thoughts on how to debug this situation / find out what is going wrong ?
TIA
[full YAML CF included]
---
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
AppName:
Type: String
InstanceType:
Type: String
Default: t2.micro
ImageId:
Type: String
Default: ami-0727f3c2d4b0226d5 # 18.04 LTS eu-west-1
KeyName:
Type: String
Outputs:
MyDNSName:
Value:
Fn::GetAtt:
- AppEC2Instance
- PublicDnsName
Description: "EC2 public DNS name"
MyIPAddress:
Value:
Fn::GetAtt:
- AppEC2Instance
- PublicIp
Description: "EC2 public IP address"
MyInstanceId:
Value:
Ref: AppEC2Instance
Description: "EC2 instance id"
Resources:
AppEC2Instance:
Properties:
IamInstanceProfile:
Ref: AppInstanceProfile
ImageId:
Ref: ImageId
InstanceType:
Ref: InstanceType
KeyName:
Ref: KeyName
SecurityGroupIds:
- Fn::GetAtt:
- AppSecurityGroup
- GroupId
SubnetId:
Ref: AppSubnet
Tags:
- Key: Name
Value:
Ref: AppName
UserData:
Fn::Base64:
!Sub |
#!/bin/bash -ex
apt-get update
apt-get install -y ruby
echo "Hello World EC2!"
Type: AWS::EC2::Instance
AppInstanceProfile:
Properties:
Path: /
Roles:
- Ref: AppInstanceRole
Type: AWS::IAM::InstanceProfile
AppInstanceRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Version: '2012-10-17'
Path: /
Policies:
- PolicyDocument:
Statement:
- Action:
- ec2:DescribeTags # allow codedeploy to find machine
Effect: Allow
Resource: '*'
- Action: s3:* # allow machine to access deployables
Effect: Allow
Resource: '*'
- Action: logs:*
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName: # required
Fn::Join:
- '-'
- - Ref: AppName
- ec2
Type: AWS::IAM::Role
AppSecurityGroup:
Properties:
GroupDescription:
Ref: AppName
SecurityGroupIngress:
- CidrIp: 0.0.0.0/0
FromPort: '3000'
IpProtocol: tcp
ToPort: '3000'
VpcId:
Ref: AppVPC
Type: AWS::EC2::SecurityGroup
AppInternetGateway:
Type: AWS::EC2::InternetGateway
AppRoute:
DependsOn: AppInternetGateway
Properties:
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: AppInternetGateway
RouteTableId:
Ref: AppRouteTable
Type: AWS::EC2::Route
AppRouteTable:
Properties:
VpcId:
Ref: AppVPC
Type: AWS::EC2::RouteTable
AppSubnet:
Properties:
CidrBlock: 172.31.0.0/20
MapPublicIpOnLaunch: true
VpcId:
Ref: AppVPC
Type: AWS::EC2::Subnet
AppSubnetRouteTableAssociation:
Properties:
RouteTableId:
Ref: AppRouteTable
SubnetId:
Ref: AppSubnet
Type: AWS::EC2::SubnetRouteTableAssociation
AppVPC:
Properties:
CidrBlock: 172.31.0.0/16
EnableDnsHostnames: true
EnableDnsSupport: true
InstanceTenancy: default
Type: AWS::EC2::VPC
AppVPCGatewayAttachment:
Properties:
InternetGatewayId:
Ref: AppInternetGateway
VpcId:
Ref: AppVPC
Type: AWS::EC2::VPCGatewayAttachment
AppCodeDeployApp:
Properties:
ApplicationName:
Ref: AppName
Type: AWS::CodeDeploy::Application
AppCodeDeployGroup:
Properties:
ApplicationName:
Ref: AppCodeDeployApp
DeploymentConfigName: CodeDeployDefault.AllAtOnce
DeploymentGroupName:
Ref: AppName
Ec2TagFilters: # lookup ec2 machine for deployment
- Key: Name
Type: KEY_AND_VALUE
Value:
Ref: AppName
ServiceRoleArn:
Fn::GetAtt:
- AppCodeDeployRole
- Arn
Type: AWS::CodeDeploy::DeploymentGroup
AppCodeDeployRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action:
- sts:AssumeRole
Effect: Allow
Principal:
Service:
- codedeploy.amazonaws.com
Version: '2012-10-17'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/service-role/AWSCodeDeployRole
Path: /
Type: AWS::IAM::Role
I have written cloudformation which consists of an EC2 instance and a RDS database. I wish to get the endpoint address from the RDS, which i can via the intrinsic function GetAtt, and pass it into the EC2 instance in a text file. Below is the code i've tried however its not working and giving me what i want.
Resources:
MyEC2Instance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-74e6b80d
InstanceType: t2.nano
SecurityGroups:
- Ref: MySecurityGroup
KeyName:
- Ref: MyKeyPair
UserData:
Fn::Base64:
Fn::Sub:
- |
#!/bin/bash
touch /home/ubuntu/touch1.txt
- DatabaseAddress: !GetAtt MyDatabase.Endpoint.Address
echo "My address is ${DatabaseAddress}" >> touch1.txt
DependsOn: MyDatabase
You can fetch attributes from a resource without the use of GetAtt while in a Sub function.
Example:
UserData:
Fn::Base64:
Fn::Sub:
- |
#!/bin/bash
touch /home/ubuntu/touch1.txt
echo "My address is ${MyDatabase.Endpoint.Address}" >> /home/ubuntu/touch1.txt