Why does terraform aws code fail to render? - bash

Terraform version = 0.12
resource "aws_instance" "bespin-ec2-web" {
ami = "ami-0bea7fd38fabe821a"
instance_type = "t2.micro"
vpc_security_group_ids = [aws_security_group.bespin-sg.id]
subnet_id = aws_subnet.bespin-subnet-public-a.id
associate_public_ip_address = true
tags = {
Name = "bespin-ec2-web-a"
}
user_data = data.template_file.user_data.rendered
}
data "template_file" "user_data" {
template = file("${path.module}/userdata.sh")
}
userdata.sh file
#!/bin/bash
USERS="bespin"
GROUP="bespin"
for i in $USERS; do
/usr/sbin/adduser ${i};
/bin/echo ${i}:${i}1! | chpasswd;
done
cp -a /etc/ssh/sshd_config /etc/ssh/sshd_config_old
sed -i 's/PasswordAuthentication no/#PasswordAuthentication no/' /etc/ssh/sshd_config
sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config
systemctl restart sshd
terraform plan result
Error: failed to render : <template_file>:5,24-25: Unknown variable; There is no variable named "i"., and 2 other di
agnostic(s)
on instance.tf line 13, in data "template_file" "user_data":
13: data "template_file" "user_data" {
Why am I getting an error?

The template argument in the template_file data source is processed as Terraform template syntax.
In this syntax, using ${...} has a special meaning, that the ... part will be injected by some var that is passed into the template.
Bash also allows this syntax, for getting the values of variables as you're intending to use it.
To reconcile this, you'll need to escape the $ character so that the terraform template compiler will leave it be, which you can do by doubling up the character: $${i} in all cases.
https://www.terraform.io/docs/configuration/expressions.html#string-templates

Related

How can I source Terraform HCL variables in bash?

I have Terraform variables defined like
variable "location" {
type = string
default = "eastus"
description = "Desired Azure Region"
}
variable "resource_group" {
type = string
default = "my-rg"
description = "Desired Azure Resource Group Name"
}
and potentially / partially overwritten in terraform.tfvars file
location = "westeurope"
and then defined variables as outputs e.g. a file outputs.tf:
output "resource_group" {
value = var.resource_group
}
output "location" {
value = var.location
}
How can I "source" the effective variable values in a bash script to work with these values?
One way is to use Terraform output values as JSON and then an utility like jq to convert and source as variables:
source <(terraform output --json | jq -r 'keys[] as $k | "\($k|ascii_upcase)=\(.[$k] | .value)"')
note that output is only available after executing terraform plan, terraform apply or even a terraform refresh
If jq is not available or not desired, sed can be used to convert Terraform HCL output into variables, even with upper case variable names:
source <(terraform output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
or using -chdir argument if Terraform templates / modules are in another folder:
source <(terraform -chdir=$TARGET_INFRA_FOLDER output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
Then these variables are available in bash script:
LOCATION="westeurope"
RESOURCE_GROUP="my-rg"
and can be addressed as $LOCATION and $RESOURCE_GROUP.

Templatefile and Bash script

I need to be able to run bash script as userdata for launchtemplate and this is how I try to do it :
resource "aws_launch_template" "ec2_launch_template" {
name = "ec2_launch_template"
image_id = data.aws_ami.latest_airbus_ami.id
instance_type = var.instance_type[terraform.workspace]
iam_instance_profile {
name = aws_iam_instance_profile.ec2_profile.name
}
vpc_security_group_ids = [data.aws_security_group.default-sg.id, aws_security_group.allow-local.id] # the second parameter should be according to the user
monitoring {
enabled = true
}
block_device_mappings {
device_name = "/dev/sda1"
ebs {
volume_size = 30
encrypted = true
volume_type = "standard"
}
}
tags = {
Name = "${var.app_name}-${terraform.workspace}-ec2-launch-template"
}
#user_data = base64encode(file("${path.module}/${terraform.workspace}-script.sh")) # change the base encoder as well
user_data = base64encode(templatefile("${path.module}/script.sh", {app_name = var.app_name, env = terraform.workspace, high_threshold = var.high_threshold, low_threshold = var.low_threshold})) # change the base encoder as well
}
as you can see, I pass parameters as map in the "templatefile" function, I managed to retrieve them doing this :
#!/bin/bash -xe
# Activate logs for everything
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
# Retrieve variables from Terraform
app_name = ${app_name}
environment = ${env}
max_memory_perc= ${high_threshold}
min_memory_perc= ${low_threshold}
instance_id=$(wget -q -O - http://169.254.169.254/latest/meta-data/instance-id)
ami_id=$(wget -q -O - http://169.254.169.254/latest/meta-data/ami-id)
instance_type=$(wget -q -O - http://169.254.169.254/latest/meta-data/instance-type)
scale_up_name=$${app_name}"-"$${environment}"-scale-up"
scale_down_name=$${app_name}"-"$${environment}"-scale-down"
Then, when I look at launchtemplate in AWS console, I can see that the values used in parameters are filled in :
app_name = test-app
environment = prod
max_memory_perc= 80
min_memory_perc= 40
the problem that I have is, when I run that, I get this error :
+ app_name = test-app
/var/lib/cloud/instances/scripts/part-001: line 7: app_name: command not found
I assume there is a problem with interpretation or something like that but cannot put the finger on it
any ideas ?
Thanks
As they said, it was a problem with spaces, it's fixed now
thanks

How to pick variables in vars.tf from BASH script

I am provisiong an EC2 instance using Terraform. It also has a startup script. I have vars.tf where I have specified all the variables in it. In my bash.sh script it should pickup one variable from vars.tf
Is it possible to refer the variable in vars.tf from bash script? Below is my use case.
bash.sh
#!/bin/bash
docker login -u username -p token docker.io
vars.tf
variable "username" {
default = "myuser"
}
variable "token" {
default = "mytoken"
}
My bash script should pick the variable from vars.tf
If this is not possible any workaround?
In order to provide Terraform variables to a script, we can use templatefile function. This function reads to content of a template file and injects Terraform variables in places marked by the templating syntax (${ ... }).
First we want to create a template file with the bash script and save it as init.tftpl:
#!/bin/bash
docker login -u ${username} -p ${token} docker.io
When creating the instance, we can use templatefile to provide the rendered script as user data:
resource "aws_instance" "web" {
ami = "ami-xxxxxxxxxxxxxxxxx"
instance_type = "t2.micro"
user_data = templatefile("init.tftpl", {
username = var.username
token = var.token
})
}

How can I make Terraform perform the virtual_machine_extension containing a sed command?

I want to disable the user password when a VM is creating using this command:
sudo sed -i -E '/^username/s/^([^:]*):([^:]*):([^:]*):([^:]*):([^:]*)(:.*)$/\1:!!:'$(($(date +%s)/86400))':\4:99998\6/' /etc/shadow
in Terraform virtual_machine_extension. I´ve tested the command on a freshly created VM and it works, but when using it with Terraform I get this Error:
Error: Code="VMExtensionProvisioningError" Message="VM has reported a failure when processing extension 'vm-02'. Error message: \"Command returned an error.\n---stdout---\n\n---errout---\nsed: -e expression #1, char 71: unterminated `s' command\n\n\"."
on ../modules/vm.tf line 100, in resource "azurerm_virtual_machine_extension" "disable_pw":
100: resource "azurerm_virtual_machine_extension" "disable_pw" {
I already tried to escape \ with \\. Running this exact vm_extension with other bash commands works fine.
Did someone has experience in it?
resource "azurerm_virtual_machine_extension" "disable_pw" {
name = "${var.vm_hostname}
location = "${module.global_variables.location}"
resource_group_name = "${module.global_variables.resource_group_name}"
virtual_machine_name = "${azurerm_virtual_machine.vm-linux.name}"
publisher = "Microsoft.OSTCExtensions"
type = "CustomScriptForLinux"
type_handler_version = "1.5"
auto_upgrade_minor_version = false
protected_settings = <<SETTINGS
{
"commandToExecute": "sudo sed -i -E '/^username/s/^([^:]*):([^:]*):([^:]*):([^:]*):([^:]*)(:.*)$/\1:!!:'$(($(date +%s)/86400))':\4:99998\6/' /etc/shadow"
}
SETTINGS
settings = <<SETTINGS
{
}
SETTINGS
}
This will work
sed -E '/^username/s/^([^:]*):([^:]*):([^:]*):([^:]*):([^:]*)(:.*)$/1:!!:'$(($(date +%s)/86400))':4:999986/g' /etc/shadow
Explanation:
s/ is used after checking the required username.
use '\' escaping only for special characters like regex.

Issue using Terraform EC2 Userdata

I am deploying a bunch of EC2 instances that require a mount called /data, this is a seperate disk that I am attaching using volume attach in AWS.
Now when I did the following manually it works fine, so the script I use works however when adding it via userdata I am seeing issues and the mkfs command is not happening.
If you see my terraform config:
resource "aws_instance" "riak" {
count = 5
ami = "${var.aws_ami}"
vpc_security_group_ids = ["${aws_security_group.bastion01_sg.id}","${aws_security_group.riak_sg.id}","${aws_security_group.outbound_access_sg.id}"]
subnet_id = "${element(module.vpc.database_subnets, 0)}"
instance_type = "m4.xlarge"
tags {
Name = "x_riak_${count.index}"
Role = "riak"
}
root_block_device {
volume_size = 20
}
user_data = "${file("datapartition.sh")}"
}
resource "aws_volume_attachment" "riak_data" {
count = 5
device_name = "/dev/sdh"
volume_id = "${element(aws_ebs_volume.riak_data.*.id, count.index)}"
instance_id = "${element(aws_instance.riak.*.id, count.index)}"
}
And then the partition script is as follows:
#!/bin/bash
if [ ! -d /data ];
then mkdir /data
fi
/sbin/mkfs -t ext4 /dev/xvdh;
while [ -e /dev/xvdh ] ; do sleep 1 ; done
mount /dev/xvdh /data
echo "/dev/xvdh /data ext4 defaults 0 2" >> /etc/fstab
Now when I do this via terraform the mkfs doesn't appear to happen and I see no obvious errors in the syslog. If I copy the script manually and just bash script.sh the mount is created and works as expected.
Has anyone got any suggestions here?
Edit: It's wort noting adding this in AWS gui under userdata also works fine.
You could try with remote_exec instead of user_data.
User_data relates on cloud-init which can act differently depending on images of your cloud provider.
And also i'm not sure it's a good idea to exec a script that would wait for some time before executing in the cloud-init section => this may lead to VM considering launch has failed because of a timeout (depending on your cloud provider).
Remote_exec may be better here because you will be able to wait until your /dev/xvdh is attached
See here
resource "aws_instance" "web" {
# ...
provisioner "file" {
source = "script.sh"
destination = "/tmp/script.sh"
}
provisioner "remote-exec" {
inline = [
"chmod +x /tmp/script.sh",
"/tmp/script.sh args",
]
}
}

Resources