terrraform get yaml values - yaml

I need to get the values from the yaml snippet into 1. the Route53 Zones where I need the apex_nme to be the zone names and 2. the records to be added as Route53 records into the specific zones. I have no clue how to do this. Any help is highly appreciated.
resource "aws_route53_zone" "this" {
for_each = {
for apex in var.source_domains : apex => {
name = apex.name
}
}
}
resource "aws_route53_record" "this" {
for_each = {
for records in var.source_domains : records => {
zone_id = aws_route53_zone.this[each.key].zone_id
name = subdomains.records
type = "A"
records = ["192.168.0.1"]
}
}
}
source_domains:
- apax_name: elastic2ls.com
records:
- elastic2ls.com
- www.elastic2ls.com
- apax_name: elastic2ls.ch
records:
- elastic2ls.ch
- www.elastic2ls.ch
- image.elastic2ls.ch
- m.elastic2ls.ch
- static.elastic2ls.ch

Your question is presented as if it's about YAML parsing, but I suspect you're really asking about how to write the for_each expression for aws_route53_record.this to create all of the records across all of the domains.
For completeness, I'll note that you could get var.source_domains to be populated from that YAML by making the calling module define that variable with an expression like this:
module "example" {
source = "../modules/example"
source_domains = yamldecode(file("${path.module}/example.yaml")).source_domains
# ...
}
Inside the module itself then, I'd first declare that module with a suitable type constraint to make it clear what data structure we're expecting, like this:
variable "source_domains" {
type = set(object({
apax_name = string
records = set(string)
}))
}
Here I defined both the top-level structure and the nested record as set types, because the way we're going to use them means that the ordering isn't important and we're expecting them all to be unique.
With all of that in place we can start to write out the resource definitions. Let's start with the zones, which are the simpler case because var.source_domains already meets the main requirement of having one element per resource instance we want to declare:
resource "aws_route53_zone" "example" {
for_each = {
for d in var.source_domains : d.apax_name => d
}
name = each.value.apax_name
}
With the example YAML input you shared, this block will declare two instances of this resource:
aws_route53_zone.example["elastic2ls.com"]
aws_route53_zone.example["elastic2ls.ch"]
The aws_route53_record declaration is a little trickier because we need to project the input data structure into a new structure where there's one element per record we want to declare, rather than one element per zone. Flattening nested data structures for for_each is a common use for the flatten function, and we can adapt the networks and subnets example from the documentation to work with zones and records instead:
locals {
zone_records = flatten([
for d in var.source_domains : [
for r in d.records : {
zone_name = d.apax_name
zone_id = aws_route53_zone.example[d.apax_name].id
record = r
}
]
])
}
This local value is constructing a list of objects where each object represents one valid pairing of zone and record. That means that the number of elements matches the number of aws_route53_record instances we need to declare, and so we can use this data structure in for_each:
resource "aws_route53_record" "example" {
for_each = {
for zr in local.zone_records : zr.record => zr
}
zone_id = each.value.zone_id
name = each.value.record
# ...
}
This example diverges a little from the typical flatten/for_each pattern because all of your record names already have the zone name embedded in them anyway, and so we don't need the usual expression to generate a compound unique key with multiple parts, like "${subnet.network_key}.${subnet.subnet_key}" in the documentation's example. The record name alone is sufficient for a unique key across all pairs in this case.
This then, again based on your example YAML, will declare the following instances:
aws_route53_record.example["elastic2ls.com"]
aws_route53_record.example["www.elastic2ls.com"]
aws_route53_record.example["elastic2ls.ch"]
aws_route53_record.example["www.elastic2ls.ch"]
aws_route53_record.example["image.elastic2ls.ch"]
aws_route53_record.example["m.elastic2ls.ch"]
aws_route53_record.example["static.elastic2ls.ch"]

Related

Filebeat Script Processor Event.Get All Fields In Log

I am looking to get all of the fields in a record in filebeat using the Script processor and perform an action on them. Using the event.Get() from the script processor, it says, "Get a value from the event (either a scalar or an object). If the key does not exist null is returned. If no key is provided then an object containing all fields is returned."
https://www.elastic.co/guide/en/beats/filebeat/current/processor-script.html
Therefore, my question is, what would I do to ensure that no key is provided to get an object that contains all of the fields are returned?
The event.Get() field will provide the top level fields. To look through these top level fields, use a for loop like:
- script:
lang: javascript
id: get_fields
source: >
function process(event) {
var a = event.Get();
for (var key in a) {
if(event.Get(key) == ""){
event.Delete(key);
}
}
}
I am unsure how to do this for nested fields in this way nor have I tried to extend it to nested fields, but this is how it works for now.

Use cases - for vs for_each in terraform - struggling to visualise

Is the for expression in Terraform basically just to provide more granular control on a for_each meta argument, so maybe it just iterates on a subset of a list, for example?
The difference between for_each and count is clear enough. I am just a bit lost trying to visualise the point of the for
Many thanks for any pointers, including articles. Google isn't proving that helpful
In Terraform for and for_each are used for different purposes.
for_each is a meta-argument. It is applied to resources and to modules (Terraform 0.13 and above). for_each meta-arguments accepts a set or a map, for every item from the set/map a new resource is created.
For example:
resource "aws_instance" "example" {
for_each = {
"t2.micro" = "ami-b374d5a5"
"t3.nano" = "ami-4b32be2b"
}
instance_type = each.key
ami = each.value
}
This will create 2 EC2 instances with instance types and AMIs from the map. Please note, for_each provides each.key and each.value objects which can be used when iterating through the map to reference its key-value items. If the input of a for_each is a set, than each.key and each.value are pointing to the same value.
for, in the other hand, is an expression used to manipulate lists, sets, tuples and maps. It can be used to apply some transformations to the elements of those complex types.
For example:
variable "names" {
default = ["alice", "bob"]
}
locals {
upper = [for n in names: upper(n)] # result will be ["ALICE", "BOB"]
}
The ouput of a for expression can be used as input for a for_each.
Example:
var ids {
default = ["b374d5a5", "4b32be2b"]
}
resource "aws_instance" "example" {
for_each = toset([for id in ids: "ami-${id}"])
instance_type = "t2.micro"
ami = each.value
}
Hmm, answering my own Q..
Seems to be helpful to apply a function to a list,etc that is being iterated in a resource e.g. changing case, converting variable type, decoding from base64, etc

JMeter populate user defined variables for forEach Controller

I have a service API to be tested which returns some default values of various parameters for different countries. country code being a template parameter of the service. i.e.
http://${hostname}:${port}/country/${countryCode}
One of the country code I can use is "ALL" (the others being "IN", "US" , "UK" , "MX" ...) which will return all the countries supported by the API and the response will contain the country code as well.
Thus I am able to populate all the country codes I need to test. The groovy scipt
import groovy.json.JsonSlurper
def slurper = new JsonSlurper()
def result = slurper.parseText(prev.getResponseDataAsString())
assert prev.isResponseCodeOK()
def countries = result.country
assert countries instanceof List // Should get as [IN, US, UK, MX] for e.g.
def numOfCountries = countries.size()
I am trying to use the variable "countries" which is a list of all the countries I need to test the service in a ForEachController. For this I will be needing the UDV Names in the format
country_1 , country_2 , country_3 ...
How do I save the elements of list to the UDVs with that name format. The problem is I do not know or rather cannot assume how many countries are supported, so the UDVs cannot be named in advance.
Can that be done ? Am I going the correct way for the problem ? Any different approach is welcome.
Thanks
I'm not familiar with groovy, but here goes:
You have the list of countries- you get that in your script. If you create a for loop (in your groovy script), you can dynamically create your variable name ("country_" + [loop iterator]) and plug that into your vars.put().
So:
for(int x=0; x < v.size(); x++)
{
String country_name = "country_" + x;
vars.put(country_name, v[x]);
}

Apache Cayenne Get Collection Expression

Using Apache Cayenne I am trying to figure out how to avoid doing iterative calls to the DB when I have a Collection of attribute values.
Say we have a persistent object Person with an attribute name of type String. If I have a List containing names of the Person I would like to generate an expression that allows a single DB call rather than iterating over the list and getting each Person in turn.
This doesn't actually exist but I would like something like this:
List<String> names = ...;
ExpressionFactory.orLikeExp(Person.NAME_PROPERTY, names);
You can use ExpressionFactory.join(..):
List<Expression> pairs = new ArrayList<>(names.size());
for(String name : names) {
// use an expression appropriate for comparison...
// "like" in this example, but can be "equals", etc.
pairs.add(ExpressionFactory.likeExp(Person.NAME_PROPERTY, name));
}
Expression e = ExpressionFactory.joinExp(Expression.OR, pairs);

Attribute routing not working with dictionaries

Being new to attribute routing, I'd like to ask for help getting this to work.
This test is a simple dynamic DB table viewer: Given a table name (or stored query name or whatever) and optionally some WHERE parameters, return query results.
Table COMPANIES (one of any number of tables which has an associated SELECT query stored somewhere, keyed by table name):
ID NAME HQ INDUSTRY
1 Apple USA Consumer electronics
2 Bose USA Low-quality, expensive audio equipment
3 Nokia FIN Mobile Phones
Controller:
[Route("view/{table}/{parameters}")]
public object Get(string table, Dictionary<string, string> parameters) {
var sql = GetSql(table);
var dbArgs = new DynamicParameters(parameters);
return Database.Query(sql, dbArgs); // Return stuff/unrelated to problem
}
SQL stored in some resource or table. Obviously the parameters must match exactly:
SELECT * FROM companies
WHERE name = :name
-- OR hq = :hq
-- OR ...etc. Doesn't matter since it never gets this far.
Request (Should look clean, but the exact URL format isn't important):
www.website.com/view/companies?hq=fin --> 404: No matching controller
www.website.com/view/companies/hq=fin --> parameters is null
www.website.com/view/companies/hq=fin&name=nokia --> Exception: A potentially dangerous Request.Path value was detected from the client (&).
When I use: [Route("view/{table}{parameters}")] I get:
A path segment cannot contain two consecutive parameters. They must be separated by a '/' or by a literal string. Parameter name: routeTemplate. Makes sense.
My question is: How do I accept a table name and any number of unknown parameters in the usual key1=val1&key2=val2 form (not some awkward indexed format like the one mentioned here) which will be later bound to SQL parameters, preferably using a vanilla data structure rather than something like FormCollection.
I don't think that binding URL parameters to a Dictionary is built-in to the framework. I'm sure there's a way to extend it if you wanted to.
I think quickest (but still acceptable) option is to get the query string parameters using Request.GetQueryNameValuePairs() like this:
[Route("view/{table}")]
public object Get(string table) {
Dictionary<string, string> parameters = Request.GetQueryNameValuePairs()
.ToDictionary(x => x.Key, x => x.Value);
var sql = GetSql(table);
var dbArgs = new DynamicParameters(parameters);
return Database.Query(sql, dbArgs); // Return stuff/unrelated to problem
}

Resources