import databases via data source dynamically(created separately via ansible).
command output is form of list which should be accepted for terraform for_each
data "azurerm_sql_server" "sql_server" {
name = "sql-server-mon-test"
resource_group_name = "ex-net-rg"
}
data "external" "databases_ids" {
program = ["sh", "${path.module}/db_id.sh"]
query = {
db_rg = data.azurerm_sql_server.sql_server.resource_group_name
server_name = data.azurerm_sql_server.sql_server.name
}
}
data "azurerm_sql_database" "database" {
for_each = toset(data.external.databases_ids.result["name_of_db"])
name = each.key
server_name = data.azurerm_sql_server.sql_server.name
resource_group_name = data.azurerm_sql_server.sql_server.resource_group_name
}
Bash code:
eval "$(jq -r '#sh "export DB_RG=\(.db_rg) SERVER_NAME=\(.server_name)"')"
if [[ -z $DB_RG || -z $SERVER_NAME ]]; then
echo "Required variables DB_RG & SERVER_NAME not set" 1>&2
exit 1
fi
name_of_db=$(az sql db list --resource-group $DB_RG --server $SERVER_NAME --query [*].name 2>/dev/null)
jq -n --arg name_of_db"$name_of_db" '{"name_of_db":$name_of_db}'
unset DB_RG SERVER_NAME name_of_db
exit 0
ERROR :
Error: Invalid function argument
on main.tf line 37, in data "azurerm_sql_database" "database":
37: for_each = toset(data.external.databases_ids.result["name_of_db"])
data.external.databases_ids.result["name_of_db"] is "\"db1\""
Invalid value for "v" parameter: cannot convert string to set of any single
type.*
Sample az command output :
az sql db list --resource-group ex-net-rg --server sql-server-mon-test --query [*].name
[
"db1",
"db2",
"master"
]
Edit 1: More debugging :
Bash script output :
{
"name_of_db": "[\n \"db1\",\n \"db2\",\n \"master\"\n]"
}
added local block in terraform code to populate state file :
locals {
database_name =(jsondecode(data.external.databases_ids.result["name_of_db"]))
}
tfstate file generated :
"query": {
"db_rg": "ex-net-rg",
"server_name": "sql-server-mon-test"
},
"result": {
"name_of_db": "[\n \"db1\",\n \"db2\",\n \"master\"\n]"
},
You should add probably jsondecode there:
for_each = toset(jsondecode(data.external.databases_ids.result["name_of_db"]))
Related
I need to be able to run bash script as userdata for launchtemplate and this is how I try to do it :
resource "aws_launch_template" "ec2_launch_template" {
name = "ec2_launch_template"
image_id = data.aws_ami.latest_airbus_ami.id
instance_type = var.instance_type[terraform.workspace]
iam_instance_profile {
name = aws_iam_instance_profile.ec2_profile.name
}
vpc_security_group_ids = [data.aws_security_group.default-sg.id, aws_security_group.allow-local.id] # the second parameter should be according to the user
monitoring {
enabled = true
}
block_device_mappings {
device_name = "/dev/sda1"
ebs {
volume_size = 30
encrypted = true
volume_type = "standard"
}
}
tags = {
Name = "${var.app_name}-${terraform.workspace}-ec2-launch-template"
}
#user_data = base64encode(file("${path.module}/${terraform.workspace}-script.sh")) # change the base encoder as well
user_data = base64encode(templatefile("${path.module}/script.sh", {app_name = var.app_name, env = terraform.workspace, high_threshold = var.high_threshold, low_threshold = var.low_threshold})) # change the base encoder as well
}
as you can see, I pass parameters as map in the "templatefile" function, I managed to retrieve them doing this :
#!/bin/bash -xe
# Activate logs for everything
exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
# Retrieve variables from Terraform
app_name = ${app_name}
environment = ${env}
max_memory_perc= ${high_threshold}
min_memory_perc= ${low_threshold}
instance_id=$(wget -q -O - http://169.254.169.254/latest/meta-data/instance-id)
ami_id=$(wget -q -O - http://169.254.169.254/latest/meta-data/ami-id)
instance_type=$(wget -q -O - http://169.254.169.254/latest/meta-data/instance-type)
scale_up_name=$${app_name}"-"$${environment}"-scale-up"
scale_down_name=$${app_name}"-"$${environment}"-scale-down"
Then, when I look at launchtemplate in AWS console, I can see that the values used in parameters are filled in :
app_name = test-app
environment = prod
max_memory_perc= 80
min_memory_perc= 40
the problem that I have is, when I run that, I get this error :
+ app_name = test-app
/var/lib/cloud/instances/scripts/part-001: line 7: app_name: command not found
I assume there is a problem with interpretation or something like that but cannot put the finger on it
any ideas ?
Thanks
As they said, it was a problem with spaces, it's fixed now
thanks
Here is my script to get telnet status
#!/bin/bash
IP=$1;
PORT=$2;
exec 3> /dev/tcp/$IP/$PORT
if [ $? -eq 0 ];then echo "PortOpen";else echo "PortClosed";fi
I am calling the func in my pipeline stage
def telnetTest (namespace, release, port) {
script {
// Getting Service IP
def serviceIP = sh (
returnStdout: true,
script: "kubectl get svc -n ${namespace} | grep ${release} | awk '{print \$4}'"
)
echo "ServiceIP: ${serviceIP}"
// Checking Service IP is Exsisting ?
if (serviceIP.equals('')) {
echo "ERROR: Getting service IP failed"
sh 'exit 1'
}
// Telnet Testing
sh "chmod +x telnetPort.sh"
def telnetTesting = sh (
returnStdout: true,
script: "./telnetPort.sh ${serviceIP} ${port}"
)
echo "${telnetTesting}"
}
}
Pipeline Stage
Pipeline {
environment {
NAMESPACE = default
RELEASE = test
PORT = 9040
}
stages {
stage ('Telnet Test') {
steps {
script {
telnetTest ("${NAMESPACE}", "${RELEASE}", "${PORT}")
}
}
}
}
}
Now its taking only first arg passing to the script
Any one let me know why & where i am going wrong
In your function, you write port, and in the defining line, you wrote PORT.
I have a terraform script that deploys a linux VM into azure, snippet below
data "template_file" "setup_script" {
template = file("myscript.sh")
}
resource "azurerm_linux_virtual_machine" "myterraformvm" {
name = var.vmname
location = var.zone
resource_group_name = azurerm_resource_group.myresourcegroup.name
network_interface_ids = [azurerm_network_interface.myterraformnic.id]
size = "Standard_DS1_v2"
os_disk {
name = "myOsDisk"
caching = "ReadWrite"
storage_account_type = "Premium_LRS"
}
source_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04-LTS"
version = "latest"
}
computer_name = var.vmname
admin_username = "myusername"
disable_password_authentication = true
custom_data = base64encode(data.template_file.setup_script.rendered)
tags = {
environment = var.envname
}
}
On boot I want to run the script myscript.sh which derives from template_file. It looks like this
#!/bin/bash
REMOTEHOST=8.8.8.8
REMOTEPORT=(22 80 443)
TIMEOUT=1
for i in "${REMOTEPORT[#]}"; do
if nc -w 1 -z 8.8.8.8 $i; then
echo "I was able to connect via $i" >> /tmp/output.txt
else
echo "Connection failed on $i. Exit code from Netcat was ($?)." >> /tmp/output.txt
fi
done
When i run terraform apply it get the following error
fatal: [localhost]: FAILED! => changed=false
msg: |-
Terraform plan could not be created
STDOUT:
STDERR:
Error: failed to render : <template_file>:6,24-25: Invalid character; This character is not used within the language., and 1 other diagnostic(s)
on main.tf line 134, in data "template_file" "setup_script":
134: data "template_file" "setup_script" {
The bash script works fine if i run it locally and works in the terraform deployment if I remove the for loop/'#' character and just do a static run. Is there a way to loop over an array in a bash file and deploy it on azurerm_linux_virtual_machine?
I have the following JSON file:
{
"1":
{
"media_content":"test3.xspf"
},
"2":
{
"media_content":"test3.xspf"
}
}
In the terminal, using bash as shell, I can execute the following commands:
export schedules="1"
echo $(jq '.[env.schedules]["media_content"]' json_file.json)
Which results in outputing this:
test3.xspf
So it works as expected, but when I place that jq command in a script and run it, it just returns null.
I did echo the values of schedules to make sure the value is non-null inside the script, and it is ok:
echo $schedules
But I did not manage to find the reason, why this command works when run directly in shell and does not work when run in script.
I run the script in the following ways:
bash script.sh
./script.sh
PS: yes, I did offer execute permission: chmod +x script.sh
HINT: env.schedules represents the environment variable 'schedules', and I did make sure that it is assigned in the script before calling jq.
EDIT: I am posting now a whole script, specifying the files tree.
There is one directory containing:
script.sh
json_file.json
static.json
script.sh:
export zone=$(cat static.json | jq '.["1"]');
echo "json block: "$zone
export schedules="$(echo $zone | jq '.schedules')"
echo "environment variable: "$schedules
export media_content=$(jq '.[env.schedules]["media_content"]' json_file.json)
echo "What I want to get: \"test3.xspf\""
echo "What I get: "$media_content
json_file.json:
{
"1":
{
"media_content":"test3.xspf"
},
"2":
{
"media_content":"test3.xspf"
}
}
static.json:
{
"1":
{
"x": "0",
"y": "0",
"width": "960",
"height": "540",
"schedules":"1"
}
}
If I run the script, it displays:
json block: { "x": "0", "y": "0", "width": "960", "height": "540", "schedules": "1" }
environment variable: "1"
What I want to get: "test3.xspf"
What I get: null
If I hardcode the variable:
export schedules="1"
The problem no longer occurs
The problem is simple.
It's not jq's fault.
It the unproper way the schedule's value is piped to the next command.
You have to remove the "s that surround the variable's value, add the second command that uses sed to do that:
export schedules="$(echo $zone | jq '.schedules')"
schedules=$( echo $schedules | sed s/\"//g )
Long answer
Let's see:
here schedules is a string and echo shows its value as being 1:
export schedules="1" ; echo $schedules
here even though double quotes are not mentioned:
export schedules=1 ; echo $schedules
But the result from this also generates additional "s:
export schedules=$(echo $zone | jq '.schedules')
If you print it now you will see additional "s:
echo $schedules # "1"
So just remove the "s from the value:
schedules=$( echo $schedules | sed s/\"//g )
Terraform version = 0.12
resource "aws_instance" "bespin-ec2-web" {
ami = "ami-0bea7fd38fabe821a"
instance_type = "t2.micro"
vpc_security_group_ids = [aws_security_group.bespin-sg.id]
subnet_id = aws_subnet.bespin-subnet-public-a.id
associate_public_ip_address = true
tags = {
Name = "bespin-ec2-web-a"
}
user_data = data.template_file.user_data.rendered
}
data "template_file" "user_data" {
template = file("${path.module}/userdata.sh")
}
userdata.sh file
#!/bin/bash
USERS="bespin"
GROUP="bespin"
for i in $USERS; do
/usr/sbin/adduser ${i};
/bin/echo ${i}:${i}1! | chpasswd;
done
cp -a /etc/ssh/sshd_config /etc/ssh/sshd_config_old
sed -i 's/PasswordAuthentication no/#PasswordAuthentication no/' /etc/ssh/sshd_config
sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config
systemctl restart sshd
terraform plan result
Error: failed to render : <template_file>:5,24-25: Unknown variable; There is no variable named "i"., and 2 other di
agnostic(s)
on instance.tf line 13, in data "template_file" "user_data":
13: data "template_file" "user_data" {
Why am I getting an error?
The template argument in the template_file data source is processed as Terraform template syntax.
In this syntax, using ${...} has a special meaning, that the ... part will be injected by some var that is passed into the template.
Bash also allows this syntax, for getting the values of variables as you're intending to use it.
To reconcile this, you'll need to escape the $ character so that the terraform template compiler will leave it be, which you can do by doubling up the character: $${i} in all cases.
https://www.terraform.io/docs/configuration/expressions.html#string-templates