Iterate through a yaml file that key name changes - for-loop

I have tried a couple of ways to iterate through a map in yaml but each results i keep getting errors ex: local.settings is object with # attributes .
to start with here is my yaml and locals setup
config:
params:
server:
host: 127.0.0.1
port: 8080
media:
name: foobar
type: html
s3:
bucket: foobarbucket
uploadFolder: folderfoobar
type:
public: false
email:
to: noreply#foobar.com
locals {
settings = yamldecode(file("test.yaml"))
}
module "store_write" {
source = "cloudposse/ssm-parameter-store/aws"
for_each = local.settings.variables
parameter_write = [
{
name = "/settings/${each.key}(i need each param key name ex: server)/${each.key.NEXTKET}(then the following key name after ex:host)"
value = "${each.key.value}(that keys value, ex: 127.0.0.1)"
type = "String"
overwrite = "true"
description = "Terraform deployed param: ${local.app_name} ${each.key}"
}
]
}
now of course i get an output of whats decoded from the yaml file, but every for loop i try wont work. at this point i cant even find 1 that is clean enough to paste here. i dont think i can achieve what i need IF sometimes the key names ex: s3 or email change over time right?
also notice sometimes i might get 3 levels,
config:
params:
s3:
bucket: foobarbucket
uploadFolder: folderfoobar
type:
public: false
I got a bit further in trying the following and had to edit the yaml file a bit. BUT now im getting duplicates
locals {
params = yamldecode(file("./test.yaml"))["app"]
folder_project_apis = flatten([
for fk, param in local.params : [
for pk, config in param.params : [
for lk, name in config : [
for vk, value in config : {
key1 = fk
key2 = pk
key3 = lk
value = value
}
]
]
]
])
}
using new yaml structure
app:
testing: #fk
display_name: "Folder A"
parent:
params:
server: #pk
host:
- 127.0.0.1
port:
- 3000
s3:
layout:
- "uploads"
image:
- "total"
outputs
> local.folder_project_apis
[
{
"key1" = "testing"
"key2" = "s3"
"key3" = "image"
"value" = [
"total",
]
},
{
"key1" = "testing"
"key2" = "s3"
"key3" = "image"
"value" = [
"uploads",
]
},
{
"key1" = "testing"
"key2" = "s3"
"key3" = "layout"
"value" = [
"total",
]
},
{
"key1" = "testing"
"key2" = "s3"
"key3" = "layout"
"value" = [
"uploads",
]
},
{
"key1" = "testing"
"key2" = "server"
"key3" = "host"
"value" = [
"127.0.0.1",
]
},
{
"key1" = "testing"
"key2" = "server"
"key3" = "host"
"value" = [
3000,
]
},
{
"key1" = "testing"
"key2" = "server"
"key3" = "port"
"value" = [
"127.0.0.1",
]
},
{
"key1" = "testing"
"key2" = "server"
"key3" = "port"
"value" = [
3000,
]
},
]

If i understand correctly, you want to iterate through a map in the file 'test.yaml' and access objects ? You can do below.
outputs.tf
output host{
value = local.settings.params.server.host
}
output email {
value = local.settings.params.email
}
terraform output
email = {
"to" = "noreply#foobar.com"
}
host = "127.0.0.1"
And you have concern that if key value is changed then you cannot access above parsing of yaml objects , yes, that is correct.
For example, if I change 's3' to 's4' in the given file and terraform apply.
entire key-value of s3 is removed and inserted of s4
~ params = {
- s3 = {
- bucket = "foobarbucket"
- uploadFolder = "folderfoobar"
} -> null
+ s4 = {
+ bucket = "foobarbucket"
+ uploadFolder = "folderfoobar"
}
# (3 unchanged elements hidden)
}
}

Related

Query trougth yaml with terraform

Hello i am trying to get a specific value out of my yaml file.
gitlab_project_id: 1222
name: testing-all
hostnames:
- name: testing-a
external: true
internal: true
internal_stages: ["dev","qs"]
- name: testing-b
external: true
internal: true
internal_stages: ["dev","qs"]
I am using terraform locals to get the yaml data:
applications = [for filename in fileset(path.module, "apps/*.yml") : yamldecode(file(filename))]
In my query I am referring just to the first element inside hostnames, but if I delete the index after hostnames, the list is empty.
my_query = { for app_name, hostnames in { for app in local.applications : replace(lower(app.name), "/[\\s-]+/", "-") => app.hostnames if can(app.hostnames) } : app_name => hostnames.0 if try(hostnames.0.internal, false) }
My goal is to get a output something like this:
testing-all = {
external = true
internal = true
internal_stages = [
"dev",
"qs",
]
name = "testing-a"
},{
external = true
internal = true
internal_stages = [
"dev",
"qs",
]
name = "testing-b"
}
While it is slightly unclear what you're after, it is clear you're probably over-complicating this a fair bit. First, I'm not sure if there are multiple yaml files, and you need to group them, or if there is one and you are just trying to matching it without giving the name explicitly. So, I've assumed there could be multiple and created a second. They are as follows.
apps/testing-all.yml
gitlab_project_id: 1222
name: testing-all
hostnames:
- name: testing-a
external: true
internal: true
internal_stages: ["dev","qs"]
- name: testing-b
external: true
internal: true
internal_stages: ["dev","qs"]
apps/testing-other.yml
gitlab_project_id: 5678
name: testing-other
hostnames:
- name: testing-y
external: true
internal: true
internal_stages: ["dev","qs"]
- name: testing-z
external: true
internal: true
internal_stages: ["dev","qs"]
The next issue is that your sample data, simply isn't valid data. You can't just connect objects with a comma, so I assume what you are after is a list of objects, one each per file / application, containing a list of objects housing the data within hostnames as shown.
This is done with:
main.tf
locals {
applications = [
for filename in fileset(path.module, "apps/*.yml")
: yamldecode(file(filename))
]
}
output "application_map" {
value = {
for app in local.applications
: app.name => app.hostnames
}
}
Which yields:
Changes to Outputs:
+ application_map = {
+ testing-all = [
+ {
+ external = true
+ internal = true
+ internal_stages = [
+ "dev",
+ "qs",
]
+ name = "testing-a"
},
+ {
+ external = true
+ internal = true
+ internal_stages = [
+ "dev",
+ "qs",
]
+ name = "testing-b"
},
]
+ testing-other = [
+ {
+ external = true
+ internal = true
+ internal_stages = [
+ "dev",
+ "qs",
]
+ name = "testing-y"
},
+ {
+ external = true
+ internal = true
+ internal_stages = [
+ "dev",
+ "qs",
]
+ name = "testing-z"
},
]
}

Return multiple results with index terraform

I currently have the below lists,
local.security_list =
[
{
ocid = security_list_id_1
name = subnet1
},
{
ocid = security_list_id_2
name = subnet1
},
{
ocid = security_list_id_3
name = subnet2
},
{
name = subnet2
ocid = security_list_id_4
},
]
var.vcn_security_lists = [
{
security_list_name = security_list_1
subnet_name = subnet1
},
{
security_list_name = security_list_2
subnet_name = subnet1
},
{
security_list_name = security_list_3
subnet_name = subnet2
}
]
I want to create the security lists using the var.vcn_security_lists list and then assign them to the subnet later on by using the subnet_name field. I have done this previously using the index() function as below, which loops through a subnet and pulls out the security list with the same subnet name. The problem is that this only returns one item - how can I return multiple from a list ?
subnet_security_list_ocid = [local.security_list[index(local.security_list.*.name,var.vcn_security_lists[index(var.vcn_security_lists.*.subnet_name,var.vcn_subnets[count.index].subnet_name)].security_list_name)].id]

How to use ConcatArrays in Spring Data MongoDB

I have the following JS code:
$concatArrays: [
"$some.path",
[
{
"foo": "baz",
"x": 5
}
]
]
How do I do the same with ArrayOperators.ConcatArrays?
Toy can use like following. You can pass the aggregation expresion or reference field name in concat(). So what you can do is, you can add the new array before concat() like
db.collection.aggregate([
{
$addFields: {
secondArray: [
{ "foo": "baz", "x": 5 }
]
}
},
{
"$project": {
combined: {
"$concatArrays": [ "$path", "$secondArray" ]
}
}
}
])
Working Mongo playground
ArrayOperators.ConcatArrays concatArrays = ArrayOperators.ConcatArrays
.arrayOf("path")
.concat("secondArray");
Assuming the array field is defined as follows in a document:
{
"_id": ObjectId("60bdc6b228acee4a5a6e1736"),
"some" : {
"path" : [
{
"foo" : "bar",
"x" : 99
}
]
},
}
The following Spring Data MongoDB code returns the result- an array with two elements. The array field "some.path" and the input array myArr are concatenated using the ArrayOperators.ConcatArrays API.
String json = "{ 'foo': 'baz', 'x': 5 }";
Document doc = Document.parse(json);
List<Document> myArr = Arrays.asList(doc);
MongoOperations mongoOps = new MongoTemplate(MongoClients.create(), "testDB");
Aggregation agg = newAggregation(
addFields()
.addFieldWithValue("myArr", myArr)
.build(),
addFields()
.addField("result").withValue(ConcatArrays.arrayOf("$some.path").concat("$myArr"))
.build()
);
AggregationResults<Document> results = mongoOps.aggregate(agg, "testColl", Document.class);

How to Convert a string to lua code to read value from nested table

I have a configurable value of a table fetch function which i get as an input string. I need that string to be executed as a code and fetch a value from the nested table.
I tried using load(string), its not working
local function main()
local t = {
["name1"] = "value1",
["name2"] = {["name1"] = "value1",
["name2"] = { 1, false, true, 23.54, "a \021 string" },
name3 = nil
},
name3 = nil
}
local string = 't.name2.name1'
print(type(string))
print(load(string))
end
print(load(string)) should print value1.
Below code works
local function main()
t = {
["name1"] = "value1",
["name2"] = {["name1"] = "value1",
["name2"] = { 1, false, true, 23.54, "a \021 string" },
name3 = nil
},
name3 = nil
}
string = "t.name2.name1"
print(type(string))
val = load("return "..string)()
print(val)
end

How do I format this json correctly?

Having a problem translating my json code form this dataObject...
var dataObject = {
"timeline":
{
"headline":"The Main Timeline Headline Goes here",
"type":"default",
"text":"<p>Intro body text goes here, some HTML is ok</p>",
"asset": {
"media":"http://yourdomain_or_socialmedialink_goes_here.jpg",
"credit":"Credit Name Goes Here",
"caption":"Caption text goes here"
},
"date": [
{
"startDate":"2011,12,10",
"endDate":"2011,12,11",
"headline":"Headline Goes Here",
"text":"<p>Body text goes here, some HTML is OK</p>",
"asset": {
"media":"http://twitter.com/ArjunaSoriano/status/164181156147900416",
"thumbnail":"optional-32x32px.jpg",
"credit":"Credit Name Goes Here",
"caption":"Caption text goes here"
}
}
],
"era": [
{
"startDate":"2011,12,10",
"endDate":"2011,12,11",
"headline":"Headline Goes Here",
"text":"<p>Body text goes here, some HTML is OK</p>",
}
]
}
}
into this method...
def timeline
t = {}
t['timeline'] = {}
t['timeline']['headline'] = "Lorem"
t['timeline']['text'] = "default"
t['timeline']['asset'] = {}
t['timeline']['asset']['media'] = ""
t['timeline']['asset']['credit'] = ""
t['timeline']['asset']['caption'] = ""
t['timeline']['date'] = [{}]
t['timeline']['date']['startDate'] = "2011,12,10"
t['timeline']['date']['endDate'] = "2011,12,11"
t['timeline']['date']['headline'] = ""
t['timeline']['date']['text'] = ""
t['timeline']['date']['asset'] = {}
t['timeline']['date']['asset']['media'] = ""
t['timeline']['date']['asset']['thumbnail'] = ""
t['timeline']['date']['asset']['credit'] = ""
t['timeline']['date']['asset']['caption'] = ""
t['timeline']['era'] = [{}]
t['timeline']['era']['startDate'] = "2011,12,10"
t['timeline']['era']['endDate'] = "2011,12,11"
t['timeline']['era']['headline'] = ""
t['timeline']['era']['text'] = ""
return t
end
specifically I'm uncertain about
t['timeline']['date'] = [{}]
and
t['timeline']['era'] = [{}]
How should I properly write those lines?
t['timeline']['date'] = [{}]
that should work just fine. Only you have to add attributes slightly different. Like this:
t['timeline']['date'][0]['startDate'] = "2011,12,10"
^^^
first element in the array
Just build a hash directly:
def timeline
{
"timeline" = {
"headline" = "Lorem",
"text" = "default",
"asset" = {}
},
"date" = [{
"startDate" = "2011,12,10",
"asset" = {
"media" = ""
}
}]
}
end
Saves a lot of typing, and will mentally map much more naturally to JSON.

Resources