How can I source Terraform HCL variables in bash? - bash

I have Terraform variables defined like
variable "location" {
type = string
default = "eastus"
description = "Desired Azure Region"
}
variable "resource_group" {
type = string
default = "my-rg"
description = "Desired Azure Resource Group Name"
}
and potentially / partially overwritten in terraform.tfvars file
location = "westeurope"
and then defined variables as outputs e.g. a file outputs.tf:
output "resource_group" {
value = var.resource_group
}
output "location" {
value = var.location
}
How can I "source" the effective variable values in a bash script to work with these values?

One way is to use Terraform output values as JSON and then an utility like jq to convert and source as variables:
source <(terraform output --json | jq -r 'keys[] as $k | "\($k|ascii_upcase)=\(.[$k] | .value)"')
note that output is only available after executing terraform plan, terraform apply or even a terraform refresh
If jq is not available or not desired, sed can be used to convert Terraform HCL output into variables, even with upper case variable names:
source <(terraform output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
or using -chdir argument if Terraform templates / modules are in another folder:
source <(terraform -chdir=$TARGET_INFRA_FOLDER output | sed -r 's/^([a-z_]+)\s+=\s+(.*)$/\U\1=\L\2/')
Then these variables are available in bash script:
LOCATION="westeurope"
RESOURCE_GROUP="my-rg"
and can be addressed as $LOCATION and $RESOURCE_GROUP.

Related

How to extract command output from the multi lines shell in Jenkins

How to get the output of kubectl describe deployment nginx | grep Image in an environment variable?
My code:
stage('Deployment'){
script {
sh """
export KUBECONFIG=/tmp/kubeconfig
kubectl describe deployment nginx | grep Image"""
}
}
In this situation, you can access the environment variables in the pipeline scope within the env object, and assign values to its members to initialize new environment variables. You can also utilize the optional returnStdout parameter to the sh step method to return the stdout of the method, and therefore assign it to a Groovy variable (because it is within the script block in the pipeline).
script {
env.IMAGE = sh(script: 'export KUBECONFIG=/tmp/kubeconfig && kubectl describe deployment nginx | grep Image', returnStdout: true).trim()
}
Note you would also want to place the KUBECONFIG environment variable within the environment directive at the pipeline scope instead (unless the kubeconfig will be different in different scopes):
pipeline {
environment { KUBECONFIG = '/tmp/kubeconfig' }
}
You can use the syntax:
someVariable = sh(returnStdout: true, script: some_script).trim()

How to do a bash `for` loop in terraform termplatefile?

I'm trying to include a bash script in an AWS SSM Document, via the Terraform templatefile function. In the aws:runShellScript section of the SSM document, I have a Bash for loop with an # sign that seems to be creating an error during terraform validate.
Version of terraform: 0.13.5
Inside main.tf file:
resource "aws_ssm_document" "magical_document" {
name = "magical_ssm_doc"
document_type = "Command"
document_format = "YAML"
target_type = "/AWS::EC2::Instance"
content = templatefile(
"${path.module}/ssm-doc.yml",
{
Foo: var.foo
}
)
}
Inside my ssm-doc.yaml file, I loop through an array:
for i in "$\{arr[#]\}"; do
if test -f "$i" ; then
echo "[monitor://$i]" >> $f
echo "disabled=0" >> $f
echo "index=$INDEX" >> $f
fi
done
Error:
Error: Error in function call
Call to function "templatefile" failed:
./ssm-doc.yml:1,18-19: Invalid character;
This character is not used within the language., and 1 other diagnostic(s).
I tried escaping the # symbol, like \#, but it didn't help. How do I
Although the error is pointing to the # symbol as being the cause of the error, it's the ${ } that's causing the problem, because this is Terraform interpolation syntax, and it applies to templatefiles too. As the docs say:
The template syntax is the same as for string templates in the main Terraform language, including interpolation sequences delimited with ${ ... }.
And the way to escape interpolation syntax in Terraform is with a double dollar sign.
for i in "$${arr[#]}"; do
if test -f "$i" ; then
echo "[monitor://$i]" >> $f
echo "disabled=0" >> $f
echo "index=$INDEX" >> $f
fi
done
The interpolation syntax is useful with templatefile if you're trying to pass in an argument, such as, in the question Foo. This argument could be accessed within the yaml file as ${Foo}.
By the way, although this article didn't give the answer to this exact issue, it helped me get a deeper appreciation for all the work Terraform is doing to handle different languages via the templatefile function. It had some cool tricks for doing replacements to escape for different scenarios.

Why does terraform aws code fail to render?

Terraform version = 0.12
resource "aws_instance" "bespin-ec2-web" {
ami = "ami-0bea7fd38fabe821a"
instance_type = "t2.micro"
vpc_security_group_ids = [aws_security_group.bespin-sg.id]
subnet_id = aws_subnet.bespin-subnet-public-a.id
associate_public_ip_address = true
tags = {
Name = "bespin-ec2-web-a"
}
user_data = data.template_file.user_data.rendered
}
data "template_file" "user_data" {
template = file("${path.module}/userdata.sh")
}
userdata.sh file
#!/bin/bash
USERS="bespin"
GROUP="bespin"
for i in $USERS; do
/usr/sbin/adduser ${i};
/bin/echo ${i}:${i}1! | chpasswd;
done
cp -a /etc/ssh/sshd_config /etc/ssh/sshd_config_old
sed -i 's/PasswordAuthentication no/#PasswordAuthentication no/' /etc/ssh/sshd_config
sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config
systemctl restart sshd
terraform plan result
Error: failed to render : <template_file>:5,24-25: Unknown variable; There is no variable named "i"., and 2 other di
agnostic(s)
on instance.tf line 13, in data "template_file" "user_data":
13: data "template_file" "user_data" {
Why am I getting an error?
The template argument in the template_file data source is processed as Terraform template syntax.
In this syntax, using ${...} has a special meaning, that the ... part will be injected by some var that is passed into the template.
Bash also allows this syntax, for getting the values of variables as you're intending to use it.
To reconcile this, you'll need to escape the $ character so that the terraform template compiler will leave it be, which you can do by doubling up the character: $${i} in all cases.
https://www.terraform.io/docs/configuration/expressions.html#string-templates

Jenkins Pipeline Environment Variable in Shell script creates a new line

I am trying to access an env variable in Jenkins pipeline and want to use it in a Shell Script executing in the same pipeline but a differnt step,
pipeline {
agent any
tools {
maven 'M2'
}
environment {
stable_revision = sh(script: 'curl -H "Authorization: Basic $base64encoded" "https://xyz.sad" | jq -r "name"', returnStdout: true)
}
stages {
stage('Initial-Checks') {
steps {
echo "Stable Revision: ${env.stable_revision}" //displays the value
bat "sh && sh undeploy.sh"
}}
...
}}
This is a sample Shell script, it has many lines, but I have an issue in only accessing the above stable_revision variable,
#!/bin/bash
echo xyz = ${stable_revision} #### this gives the right value xyz = 22
echo xyz2 = ${stable_revision}/d ### here after the value the /d is written in new line
For example, let's say the stable_revision value is 22, then in the SH script echo I am getting the value as,
xyz2 = 22
/d
I want the value to be xyz2 = 22/d
You can use .trim() to strip off a trailing newline.
environment {
stable_revision = sh(script: 'curl -H "Authorization: Basic $base64encoded" "https://xyz.sad" | jq -r "name"', returnStdout: true).trim()
}
returnStdout (optional):
If checked, standard output from the task is returned as the step value as a String, rather than being printed
to the build log. (Standard error, if any, will still be printed to
the log.) You will often want to call .trim() on the result to strip
off a trailing newline.
https://jenkins.io/doc/pipeline/steps/workflow-durable-task-step/#sh-shell-script
If you use bash instead of sh for your commands, you can benefit from Bash's built-in string transformations
Here it trims all trailing characters from the [:space:] class witch includes actual spaces and newlines.
echo "xyz2 = ${stable_revision%%[[:space:]]}/d"
If $stable_revision is always an integer, you can force the shell to use it like an integer with:
echo "xyz2 = $((stable_revision))/d"
If you are sure that $stable_revision contains no space, you can force the shell to trim all spaces by using it like a table element:
sr=($stable_revision); echo "xyz2 = ${sr[0]}/d"
You can also use the automatic trimming of a sub-shell returned value, that would trim any leading, trailing and duplicate spaces in-between:
echo "xyz2 = $(echo ${stable_revision})/d"`

How to use environment variable parameter in jq parsing

In the following bash code, I'm getting null as value instead of actual value. I'm using jq 1.5 in Linux OS.
getProperty(){
export PROP=$1
export value_found=`cat my.json | jq '."env.PROP"'`
echo "$1 = $value_found"
}
getProperty "first.name"
getProperty "last.name"
Contents of my.json
{"first.name":"Adam", "last.name":"Eve"}
My program requires to get value for properties selected randomly, so the property name is not known before runtime (hence the reason for trying to use parameters)
Your idea is right, but you don't need to export the variables/use the environment variables here. Just pass in the arguments and use the internal jq logic to import variables from shell.
The key here is to access arguments containing ., wrap them over []
getProperty(){
jq --arg namearg "$1" '.[$namearg]' < json
}
and now call with the args as needed
getProperty "first.name"
"Adam"
getProperty "last.name"
"Eve"

Resources