Terraform output object with multiple attributes for each of `for` resources? - for-loop

I have terraform with a resource being created with for. As is typical, each instance of this resource has several attributes. At the moment I have a series of map outputs for this resource group but each consists of only a single key-value pair. I would like my terraform output to include a list or map of maps or objects with all of the attributes grouped by resource instance. How do I do this without using flatten; zipmap etc to construct them from my current outputs? This example is with aws_route53_record but this is a generic query:
Current code
output "r53record_zonal_fqdn" {
value = {
for entry in aws_route53_record.zonal :
entry.name => entry.fqdn
}
}
output "r53record_zonal_records" {
value = {
for entry in aws_route53_record.zonal :
entry.name => entry.records
}
}
output "r53record_zonal_zone_id" {
value = {
for entry in aws_route53_record.zonal :
entry.name => entry.zone_id
}
}
As you would expect, this renders three maps with aws_route53_record.zonal.name as the key and the other attribute(s) as the value.
What I would like is to have these outputs grouped by resource with a predefined key for each value, e.g. (pseudocode):
output "r53record_zonal_zone_id" {
value = {
for entry in aws_route53_record.zonal : {
instance[count.index] {
"name" = entry.name
"fqdn" = entry.fqdn
"records" = entry.records
"zone_id" = entry.zone_id
}
}
}
}
Producing a map or list of maps for each instance.
How can this or something like it be done?

I created a random route53_record resource block with two "name" arguments in for_each loop and tried to output something close to what you were looking for.
Assuming "mydomain.com" is the domain in Route53 as example....
resource "aws_route53_record" "zonal" {
for_each=toset(["site1","site2"])
name = each.key
zone_id = "ABCDZONEIDSTRING"
type = "A"
ttl = "300"
records = ["192.168.1.10"]
}
output "test" {
value = {
for dns, details in aws_route53_record.zonal:
dns => ({"fqdn" = details.fqdn , "zone_id" = details.zone_id , "records" = details.records})
}
}
this will create output in this fashion..
test = {
"site1" = {
"fqdn" = "site1.mydomain.com"
"records" = [
"192.168.1.10",
]
"zone_id" = "Z0630117NTQNSYTXYQ4Z"
}
"site2" = {
"fqdn" = "site2.mydomain.com"
"records" = [
"192.168.1.10",
]
"zone_id" = "Z0630117NTQNSYTXYQ4Z"
}
}
Suppose you passed the name values with domain name, like below...
for_each=toset(["site1.mydomain.com","site2.mydomain.com"])
the output would look like
test = {
"site1.mydomain.com" = {
"fqdn" = "site1.mydomain.com"
"records" = [
"192.168.1.10",
]
"zone_id" = "ABCDMYZONEIDSTRING"
}
"site2.mydomain.com" = {
"fqdn" = "site2.mydomain.com"
"records" = [
"192.168.1.10",
]
"zone_id" = "ABCDMYZONEIDSTRING"
}
}

Related

protobuf map value is object or array of object

I have
message {
message leadsEntity{
optional string name = 1;
}
optional string description = 1;
map <string, leadsEntity> entity = 2;
}
Now I can have 2 types of responses-
// 1st case
{
description : "this is test",
entity : {
key1 : { name : "name1" },
key2 : { name : "name2" }
}
}
Or
// 2nd case
{
description : "this is test",
entity : {
key1 : { name : "name1" },
key2 : [ { name : "name2" } ] //the value in array now
}
}
Now the above given proto message works perfectly for 1st case.
But how to handle 2nd case? where the value is an object array?
I assume by the use of optional that you prefer proto2
Your existing proto is invalid.
Your response types are going to make your life difficult because the entity values are a union of types.
Perhaps something like this:
syntax = "proto2";
message X {
message LeadsEntity{
optional string name = 1;
}
message LeadsEntities {
repeated LeadsEntity leads_entity = 1;
}
message Value {
oneof These {
LeadsEntity leads_entity = 1;
LeadsEntities leads_entities = 2;
}
}
optional string description = 1;
map <string, Value> entity = 2;
}
I gave the root message the name X
I added LeadsEntities to hold the repeated LeadsEntity
I added Value to hold the oneof
from google.protobuf.json_format import MessageToJson
import x_pb2
x = x_pb2.X()
x.description = "this is a test"
x.entity["key1"].leads_entity.name = "name1"
le = x.entity["key2"].leads_entities.leads_entity.add()
le.name = "name2"
print(MessageToJson(x))
Yields:
{
"description": "this is a test",
"entity": {
"key1": {
"leadsEntity": {
"name": "name1"
}
},
"key2": {
"leadsEntities": {
"leadsEntity": [
{
"name": "name2"
}
]
}
}
}
}

how to describe stop / start AWS EC2 instances process by schedule in Terraform

I trying figure out how to simply start / stop EC2 instances by schedule and EventBridge
This behavior may be easily set via AWS WEB console (EventBridge → Rules → Create Rule → AWS service → EC2 StopInstances API call):
But I can't figure out how to describe this rule in Terraform
Only possible way that I found is to create Lambda. But it looks like a huge overhead for this simple action. Is here any way to add EC2 StopInstances API call Rule with Terraform?
Okay, looks like it is possible to control instance running time with SSM Automation:
variables.tf
variable "start_cron_representation" {
type = string
}
variable "stop_cron_representation" {
type = string
}
variable "instance_id" {
type = string
}
variable "instance_type" {
description = "ec2 or rds"
type = string
}
locals.tf
locals {
stop_task_name = var.instance_type == "rds" ? "AWS-StopRdsInstance" : "AWS-StopEC2Instance"
start_task_name = var.instance_type == "rds" ? "AWS-StartRdsInstance" : "AWS-StartEC2Instance"
permissions = var.instance_type == "rds" ? [
"rds:StopDBInstance",
"rds:StartDBInstance",
"rds:DescribeDBInstances",
"ssm:StartAutomationExecution"
] : [
"ec2:StopInstances",
"ec2:StartInstances",
"ec2:DescribeInstances",
"ssm:StartAutomationExecution"
]
}
main.tf
data "aws_iam_policy_document" "ssm_lifecycle_trust" {
statement {
actions = ["sts:AssumeRole"]
principals {
type = "Service"
identifiers = [
"ssm.amazonaws.com",
"events.amazonaws.com"
]
}
}
}
data "aws_iam_policy_document" "ssm_lifecycle" {
statement {
effect = "Allow"
actions = local.permissions
resources = ["*"]
}
statement {
effect = "Allow"
actions = [
"iam:PassRole"
]
resources = [aws_iam_role.ssm_lifecycle.arn]
}
}
resource "aws_iam_role" "ssm_lifecycle" {
name = "${var.instance_type}-power-control-role"
assume_role_policy = data.aws_iam_policy_document.ssm_lifecycle_trust.json
}
resource "aws_iam_policy" "ssm_lifecycle" {
name = "${var.instance_type}-power-control-policy"
policy = data.aws_iam_policy_document.ssm_lifecycle.json
depends_on = [
aws_iam_role.ssm_lifecycle
]
}
resource "aws_iam_role_policy_attachment" "ssm_lifecycle" {
policy_arn = aws_iam_policy.ssm_lifecycle.arn
role = aws_iam_role.ssm_lifecycle.name
}
resource "aws_cloudwatch_event_rule" "stop_instance" {
name = "stop-${var.instance_type}"
description = "Stop ${var.instance_type} instance"
schedule_expression = var.stop_cron_representation
}
resource "aws_cloudwatch_event_target" "stop_instance" {
target_id = "stop-${var.instance_type}"
arn = "arn:aws:ssm:ap-northeast-1::automation-definition/${local.stop_task_name}"
rule = aws_cloudwatch_event_rule.stop_instance.name
role_arn = aws_iam_role.ssm_lifecycle.arn
input = <<DOC
{
"InstanceId": ["${var.instance_id}"],
"AutomationAssumeRole": ["${aws_iam_role.ssm_lifecycle.arn}"]
}
DOC
}
resource "aws_cloudwatch_event_rule" "start_instance" {
name = "start-${var.instance_type}"
description = "Start ${var.instance_type} instance"
schedule_expression = var.start_cron_representation
}
resource "aws_cloudwatch_event_target" "start_instance" {
target_id = "start-${var.instance_type}"
arn = "arn:aws:ssm:ap-northeast-1::automation-definition/${local.start_task_name}"
rule = aws_cloudwatch_event_rule.start_instance.name
role_arn = aws_iam_role.ssm_lifecycle.arn
input = <<DOC
{
"InstanceId": ["${var.instance_id}"],
"AutomationAssumeRole": ["${aws_iam_role.ssm_lifecycle.arn}"]
}
DOC
}
This module may be called like:
module "ec2_start_and_stop" {
source = "./module_folder"
start_cron_representation = "cron(0 0 * * ? *)"
stop_cron_representation = "cron(0 1 * * ? *)"
instance_id = aws_instance.instance_name.id # or aws_db_instance.db.id for RDS
instance_type = "ec2" # or "rds" for RDS
depends_on = [
aws_instance.instance_name
]
}

Run Elasticsearch processor on all the fields of a document

I am trying to trim and lowercase all the values of the document that is getting indexed into Elasticsearch
The processors available has the field key is mandatory. This means one can use a processor on only one field
Is there a way to run a processor on all the fields of a document?
There sure is. Use a script processor but beware of reserved keys like _type, _id etc:
PUT _ingest/pipeline/my_string_trimmer
{
"description": "Trims and lowercases all string values",
"processors": [
{
"script": {
"source": """
def forbidden_keys = [
'_type',
'_id',
'_version_type',
'_index',
'_version'
];
def corrected_source = [:];
for (pair in ctx.entrySet()) {
def key = pair.getKey();
if (forbidden_keys.contains(key)) {
continue;
}
def value = pair.getValue();
if (value instanceof String) {
corrected_source[key] = value.trim().toLowerCase();
} else {
corrected_source[key] = value;
}
}
// overwrite the original
ctx.putAll(corrected_source);
"""
}
}
]
}
Test with a sample doc:
POST my-index/_doc?pipeline=my_string_trimmer
{
"abc": " DEF ",
"def": 123,
"xyz": false
}

How to use environment variables in terraform using TF_VAR_name?

I'm trying to export list variables and use them via TF_VAR_name and getting error while combining them with toset function.
Success scenario:
terraform apply -auto-approve
# Variables
variable "sg_name" { default = ["SG1", "SG2", "SG3", "SG4", "SG5"] }
variable "Project" { default = "POC" }
variable "Owner" { default = "Me" }
variable "Environment" { default = "Testing" }
locals {
common_tags = {
Project = var.Project
Owner = var.Owner
Environment = var.Environment
}
}
# Create Security Group
resource "aws_security_group" "application_sg" {
for_each = toset(var.sg_name)
name = each.value
description = "${each.value} security group"
tags = merge(local.common_tags, { "Name" = each.value })
}
# Output the SG IDs
output "sg_id" {
value = values(aws_security_group.application_sg)[*].id
}
Failure scenario:
TF_VAR_sg_name='["SG1", "SG2", "SG3", "SG4", "SG5"]' terraform apply -auto-approve
# Variables
variable "sg_name" { }
variable "Project" { default = "POC" }
variable "Owner" { default = "Me" }
variable "Environment" { default = "Testing" }
locals {
common_tags = {
Project = var.Project
Owner = var.Owner
Environment = var.Environment
}
}
# Create Security Group
resource "aws_security_group" "application_sg" {
for_each = toset(var.sg_name)
name = each.value
description = "${each.value} security group"
tags = merge(local.common_tags, { "Name" = each.value })
}
# Output the SG IDs
output "sg_id" {
value = values(aws_security_group.application_sg)[*].id
}
Error
Error: Invalid function argument
on main.tf line 16, in resource "aws_security_group" "application_sg":
16: for_each = toset(var.sg_name)
|----------------
| var.sg_name is "[\"SG1\", \"SG2\", \"SG3\", \"SG4\", \"SG5\"]"
Invalid value for "v" parameter: cannot convert string to set of any single
type.
You'll need to specify the type (i.e. type = list(string) in your case) of your variable then it should work.
I tested it with the following configuration:
variable "sg_name" {
type = list(string)
}
resource "null_resource" "application_sg" {
for_each = toset(var.sg_name)
triggers = {
name = each.key
}
}
Then a TF_VAR_sg_name='["SG1", "SG2", "SG3", "SG4", "SG5"]' terraform apply works.
If I remove the type = list(string) it errors out as you say.

Terraform - Iterate through map of maps depending on value of internal map

i'm trying to create a resource depending on a value of the internal map of a map.
If that specific value is true then create the resource
Please find below the code:
variable "ip_restrictions" {
type = map(map(string))
default = {
test01 = {
name = "test01"
start_ip_address = "0.0.0.0"
end_ip_address = "0.0.0.0"
is_firewall = false
},
test02 = {
name = "test02"
start_ip_address = "0.0.0.0"
end_ip_address = "0.0.0.0"
is_firewall = true
}
}
}
resource "azurerm_sql_firewall_rule" "sql_firewall_rules" {
for_each = {
for restr in var.ip_restrictions :
restr => restr
if restr.is_firewall == true
}
name = each.value.name
resource_group_name = azurerm_resource_group.rg.name
server_name = azurerm_sql_server.sqls.name
start_ip_address = each.value.start_ip_address
end_ip_address = each.value.end_ip_address
}
The correct form of your for_each should be:
for_each = {
for key, restr in var.ip_restrictions :
key => restr if restr.is_firewall == "true"
}
This will filter out all records with is_firewall of false.
Since your var.ip_restrictions is a map and not a list, you need to iterate over it like a map (returning a key and value). The shortest possible way in your situation:
for_each = {
for key, restr in var.ip_restrictions :
key => restr if restr.is_firewall
}
Gives:
"test02" = {
"end_ip_address" = "0.0.0.0"
"is_firewall" = "true"
"name" = "test02"
"start_ip_address" = "0.0.0.0"
}

Resources