Having CloudFormation wait for the user data - bash

Question

I have a cloudformation stack which creates a EC2 instance and install something in it using UserData. Cloudformation immediately reports CREATE_COMPLETE upon creation of the EC2 instance based on RedHat. But at this point, the instance is not really usable since the userdata takes about 40 min to finish. I read through documentation and even tried cfn-signal but I could not successfully execute it.
Can someone tell me how exactly it has to be done?
EC2Instance:
Type: AWS::EC2::Instance
Properties:
CreditSpecification:
CPUCredits: standard
IamInstanceProfile:
Fn::ImportValue:
!Sub ${InstanceProfileStackName}-instanceProfile
ImageId: !Ref ImageId
InstanceInitiatedShutdownBehavior: stop
InstanceType: !Ref InstanceType
SubnetId: !Ref SubnetId
SecurityGroupIds:
- !Ref DefaultSecurityGroup
- !Ref WebSecurityGroup
UserData:
Fn::Base64: !Sub |
#!/bin/bash
set -e
yum update -y
The above is the truncated part of my Cloudformation template.
UPDATE
I have the script which has the following line
source scl_source enable rh-python36
The default of my instance is python2.7 but I had to install my pip packages with python3.6. I am not sure if that was making the cfn-signal fail.
The script is going till the final step and seems to fail there. I am creating a recordset from the EC2 IP but Cloudformation still thinks the EC2 instance is not done and waiting till the timeout.
Screenshot of the instance snapshot
Log file end is as follows
Also my log file is named /var/log/cloud-init.log. There was no cloud-init-output.log in that directory.

Answer 1

You need two components:
CreationPolicy so that CFN waits for a SUCCESS signal from the instance.
cfn-signal helper script to perform the signalling action.
Thus your template could be modified as follows for Redhat 8:
EC2Instance:
Type: AWS::EC2::Instance
CreationPolicy: # <--- creation policy with timeout of 5 minutes
ResourceSignal:
Timeout: PT5M
Properties:
CreditSpecification:
CPUCredits: standard
IamInstanceProfile:
Fn::ImportValue:
!Sub ${InstanceProfileStackName}-instanceProfile
ImageId: !Ref ImageId
InstanceInitiatedShutdownBehavior: stop
InstanceType: !Ref InstanceType
SubnetId: !Ref SubnetId
SecurityGroupIds:
- !Ref DefaultSecurityGroup
- !Ref WebSecurityGroup
UserData:
Fn::Base64: !Sub |
#!/bin/bash -x
yum update -y
yum -y install python2-pip
pip2 install https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz
python2 /usr/bin/cfn-signal -e $? \
--stack ${AWS::StackName} \
--resource EC2Instance \
--region ${AWS::Region}
For debugging, as the user data may error out, have to login to the instance and check /var/log/cloud-init-output.log file

Answer 2

I could recreate your error and fixed here. Here is the corrected template. I added to the answer from Marcin
EC2Instance:
Type: AWS::EC2::Instance
CreationPolicy:
ResourceSignal:
Timeout: PT5M # Specify the time here
Properties:
CreditSpecification:
CPUCredits: standard
IamInstanceProfile:
Fn::ImportValue:
!Sub ${InstanceProfileStackName}-instanceProfile
ImageId: !Ref ImageId
InstanceInitiatedShutdownBehavior: stop
InstanceType: !Ref InstanceType
SubnetId: !Ref SubnetId
SecurityGroupIds:
- !Ref DefaultSecurityGroup
- !Ref WebSecurityGroup
UserData:
Fn::Base64: !Sub |
#!/bin/bash -ex
yum update -y
source scl_source enable rh-python36
<Your additional commands>
cfn-signal -e $? --stack ${AWS::StackName} --resource EC2Instance --region ${AWS::Region}
You might want to counter check the indentation before trying.