Helm ConfigMap - Multiline variable in a multiline key - yaml

I am trying to leverage a multiline variable injected into a multiline key in a helm chart configmap, but the formatting is all wonky.
For instance, the variable I'm trying to set looks like this:
plugin: |
Foo "example" {
plugin_data {
foo = "bar"
foz = "baz"
}
}
And the existing configmap I'm trying to pass this to looks like this:
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "chart.fullname" . }}
labels:
{{ include "chart.labels" . | indent 4 }}
data:
server.conf: |
server {
bind_address = "{{ .Values.config.bindAddress }}"
bind_port = "{{ .Values.config.bindPort }}"
log_level = "{{ .Values.config.logLevel }}"
plugins {
DataStore "sql" {
plugin_data {
database_type = "sqlite3"
}
}
{{- if .Values.config.plugin -}}
{{- toYaml .Values.config.plugin | nindent 4 -}}
{{- end -}}
}
The templated configmap comes out looking like this, which is obviously not valid YAML. Note the | before the templated plugin var:
apiVersion: v1
kind: ConfigMap
metadata:
name: example
labels:
data:
server.conf: |
server {
bind_address = "0.0.0.0"
bind_port = "8081"
log_level = "DEBUG"
plugins {
DataStore "sql" {
plugin_data {
database_type = "sqlite3"
}
}
|
Foo "example" {
plugin_data {
foo = "bar"
foz = "baz"
}
}
But what I'm looking for is this, without the additional |
apiVersion: v1
kind: ConfigMap
metadata:
name: example
labels:
data:
server.conf: |
server {
bind_address = "0.0.0.0"
bind_port = "8081"
log_level = "DEBUG"
plugins {
DataStore "sql" {
plugin_data {
database_type = "sqlite3"
}
}
Foo "example" {
plugin_data {
foo = "bar"
foz = "baz"
}
Is this possible? These are not my helm charts, I am merely trying to add functionality to an open-source repo, so changing up the server.config isn't possible.

ToYaml is redundant
values.yaml
config:
bindAddress: 127.0.0.1
bindPort: 8080
logLevel: info
plugin: |
Foo "example" {
plugin_data {
foo = "bar"
foz = "baz"
}
}
cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
server.conf: |
server {
bind_address = "{{ .Values.config.bindAddress }}"
bind_port = "{{ .Values.config.bindPort }}"
log_level = "{{ .Values.config.logLevel }}"
plugins {
DataStore "sql" {
plugin_data {
database_type = "sqlite3"
}
}
{{- if .Values.config.plugin }}
{{- .Values.config.plugin | nindent 6 }}
{{- end }}
}
output
apiVersion: v1
kind: ConfigMap
metadata:
name: test
data:
server.conf: |
server {
bind_address = "127.0.0.1"
bind_port = "8080"
log_level = "info"
plugins {
DataStore "sql" {
plugin_data {
database_type = "sqlite3"
}
}
Foo "example" {
plugin_data {
foo = "bar"
foz = "baz"
}
}
}

Related

Query trougth yaml with terraform

Hello i am trying to get a specific value out of my yaml file.
gitlab_project_id: 1222
name: testing-all
hostnames:
- name: testing-a
external: true
internal: true
internal_stages: ["dev","qs"]
- name: testing-b
external: true
internal: true
internal_stages: ["dev","qs"]
I am using terraform locals to get the yaml data:
applications = [for filename in fileset(path.module, "apps/*.yml") : yamldecode(file(filename))]
In my query I am referring just to the first element inside hostnames, but if I delete the index after hostnames, the list is empty.
my_query = { for app_name, hostnames in { for app in local.applications : replace(lower(app.name), "/[\\s-]+/", "-") => app.hostnames if can(app.hostnames) } : app_name => hostnames.0 if try(hostnames.0.internal, false) }
My goal is to get a output something like this:
testing-all = {
external = true
internal = true
internal_stages = [
"dev",
"qs",
]
name = "testing-a"
},{
external = true
internal = true
internal_stages = [
"dev",
"qs",
]
name = "testing-b"
}
While it is slightly unclear what you're after, it is clear you're probably over-complicating this a fair bit. First, I'm not sure if there are multiple yaml files, and you need to group them, or if there is one and you are just trying to matching it without giving the name explicitly. So, I've assumed there could be multiple and created a second. They are as follows.
apps/testing-all.yml
gitlab_project_id: 1222
name: testing-all
hostnames:
- name: testing-a
external: true
internal: true
internal_stages: ["dev","qs"]
- name: testing-b
external: true
internal: true
internal_stages: ["dev","qs"]
apps/testing-other.yml
gitlab_project_id: 5678
name: testing-other
hostnames:
- name: testing-y
external: true
internal: true
internal_stages: ["dev","qs"]
- name: testing-z
external: true
internal: true
internal_stages: ["dev","qs"]
The next issue is that your sample data, simply isn't valid data. You can't just connect objects with a comma, so I assume what you are after is a list of objects, one each per file / application, containing a list of objects housing the data within hostnames as shown.
This is done with:
main.tf
locals {
applications = [
for filename in fileset(path.module, "apps/*.yml")
: yamldecode(file(filename))
]
}
output "application_map" {
value = {
for app in local.applications
: app.name => app.hostnames
}
}
Which yields:
Changes to Outputs:
+ application_map = {
+ testing-all = [
+ {
+ external = true
+ internal = true
+ internal_stages = [
+ "dev",
+ "qs",
]
+ name = "testing-a"
},
+ {
+ external = true
+ internal = true
+ internal_stages = [
+ "dev",
+ "qs",
]
+ name = "testing-b"
},
]
+ testing-other = [
+ {
+ external = true
+ internal = true
+ internal_stages = [
+ "dev",
+ "qs",
]
+ name = "testing-y"
},
+ {
+ external = true
+ internal = true
+ internal_stages = [
+ "dev",
+ "qs",
]
+ name = "testing-z"
},
]
}

jenkins pipeline fails without some explanatory exception

I have a pipeline where among other parameters like machine name, client it is supposed to get latest ami from aws for that branch for example and then put it the clien.json which terraform would use to create the machine but als I want to enable the user to be able to provide a value for the parameter and when there is no value for that parameter to have the value picked from latest ami in develop for example:
#!/usr/bin/env groovy
pipeline {
agent { label 'new' }
parameters {
string(name: 'AMI_ID', defaultValue: '', description: '[Mandatory]')
}
stages {
stage('Retrieve latest AMI.') {
when {
expression { ${AMI_ID} == '' }
}
steps {
script {
AMI_ID = sh(script: "aws ec2 describe-images --region region1 --owners 123456 --filters \"Name=tag:type,Values=develop\" --query 'sort_by(Images, &CreationDate)[-1].ImageId' | jq -r '.'", returnStdout: true).trim()
echo "AMI retrieved: " + $ { AMI_ID }
}
}
}
stage("Updating client data") {
environment {
TERRAHELP_KEY = credentials('some-key')
}
steps {
dir("data/clients/") {
clientJson = readJSON file: "${CLIENT}.json"
clientJson.put("client_ec2_eda_ami_id", ${ AMI_ID })
writeJSON file: "${CLIENT}.json", json: clientJson, pretty: 4
echo "Following data will be applied:"
sh "cat ${CLIENT}.json"
}
}
}
}
}
Found the root cause was that I was passing a shell script variable to groovy where I say:
clientJson.put("client_ec2_eda_ami_id", ${ AMI_ID })
Instead I should have passed the AMI_ID to a groovy variable above and say like:
clientJson.put("client_ec2_eda_ami_id", currentAmi)
Peace of the code with the aws query dimmed:
stage('Retrieve latest AMI.') {
}
steps {
script {
currentAmi = params.AMI_ID
if (currentAmi.isEmpty())
currentAmi = sh(script: "aws ec2 query blah blah")
echo "Ami retrieved is: ${currentAmi}"
}
}
}
stage("Updating client data") {
environment {
TERRAHELP_KEY = credentials('some-key')
}
steps {
dir("data/clients/") {
clientJson = readJSON file: "${CLIENT}.json"
clientJson.put("client_ec2_eda_ami_id", currentAmi)
writeJSON file: "${CLIENT}.json", json: clientJson, pretty: 4
echo "Following data will be applied:"
sh "cat ${CLIENT}.json"
}
}
}
}
}

How to embed a literal json within a YAML using jsonnet?

Here's what I'm trying to achieve using jsonnet:
version: "v1"
data:
j.json: |-
{
"foo": "bar"
}
Here's my failed attempt:
local j = {
foo: "bar"
};
local wrapper = {
version: "v1",
data: {
'j.json': |||
j
|||
}
};
std.manifestYamlDoc(wrapper)
In my attempt I'm getting the following result:
"data":
"j.json": |
j
"version": "v1"
How can achieve a desired result?
Couple things there:
the multiline string you build with the ||| expression is a literal, would need %<blah> format operator as any other string
looks like you want std.manifestJson() there
I'd rather take advantage of JSON being YAML and use jsonnet output, fwiw more legible:
foo.jsonnet:
local j = {
foo: "bar"
};
local wrapper = {
version: "v1",
data: {
'j.json': std.manifestJson(j)
}
};
wrapper
jsonnet output:
$ jsonnet foo.jsonnet
{
"data": {
"j.json": "{\n \"foo\": \"bar\"\n}"
},
"version": "v1"
}
verifying j.json field with jq
$ jsonnet foo.jsonnet | jq -r '.data["j.json"]' | jq
{
"foo": "bar"
}

How can I manipulate key value of a dictionary

I have the following dict in ansible:
finder_expand_infopane:
- { name: General , value: yes }
- { name: MetaData , value: yes }
- { name: OpenWith , value: yes }
- { name: Comments , value: no }
- { name: Preview , value: no }
- { name: Privileges , value: yes }
How can I use set_fact to create a second dict where the value key gets modified with the following filters?
| bool | string() | lower
So the dict I need should look like this:
finder_expand_infopane_parsed:
- { name: General , value: true }
- { name: MetaData , value: true }
- { name: OpenWith , value: true }
- { name: Comments , value: false }
- { name: Preview , value: false }
- { name: Privileges , value: true }
Any tips would be greatly appreciated! Thanks.
You can generate required elements as temporary facts and then extract them as a single list from registered variable.
Here's an example:
---
- hosts: localhost
gather_facts: no
vars:
finder_expand_infopane:
- { name: General , value: yes }
- { name: MetaData , value: yes }
- { name: OpenWith , value: yes }
- { name: Comments , value: no }
- { name: Preview , value: no }
- { name: Privileges , value: yes }
tasks:
- set_fact:
tmp_item:
name: "{{ item.name }}"
value: "{{ item.value | bool | string() | lower }}"
register: tmp_items
with_items: "{{ finder_expand_infopane }}"
- set_fact:
res: "{{ tmp_items.results | map(attribute='ansible_facts.tmp_item') | list }}"
- debug:
msg: "{{ res }}"
In a more general way you can generate only additional values and merge original items with additional generated properties. See this post for details.

Ruby transform hash based on mapping

Let's say I have a mapping of how I want a hash to turn out, along with new key names like this:
JSON_MAP = {
image: {
id: :id,
media_url: :url,
time: :duration,
timestamp: :time_posted,
text_caption: :caption,
metadata: {
camera: :camera_type,
flash: :camera_flash
}
},
viewers: {
views: :view_count,
likes: :likes_count
}
}
and I have a hash like this:
{
image: {
id: 1,
media_url: 'http://placekitten.com',
nsfw: false,
time: 4,
timestamp: 14149292,
text_caption: "I'm a kitten",
metadata: {
camera: 'iPhone',
flash: true
}
},
viewers: {
views: 50,
likes: 15
},
extras: {
features: {
enabled: true
}
}
}
I only want it to transform the data so it ends up like:
{
image: {
id: 1,
url: 'http://placekitten.com',
duration: 4,
time_posted: 14149292,
caption: "I'm a kitten",
metadata: {
camera: 'iPhone',
flash: true
}
},
viewers: {
view_count: 50,
likes_count: 15
}
}
Basically, renaming all the keys based on the source map, and deleting any keys that don't match the source map...
You can obtain your desired result using recursion.
def convert(mapper, hsh)
mapper.each_with_object({}) do |(k,o),h|
next unless hsh.key?(k)
if o.is_a? Hash
h[k] = convert(o, hsh[k])
else
h[o] = hsh[k]
end
end
end
Assuming h equals your second hash,
convert(JSON_MAP, h)
#=> { :image=>{
# :id=>1,
# :url=>"http://placekitten.com",
# :duration=>4,
# :time_posted=>14149292,
# :caption=>"I'm a kitten",
# :metadata=>{
# :camera_type=>"iPhone",
# :camera_flash=>true
# }
# },
# :viewers=>{
# :view_count=>50,
# :likes_count=>15
# }
# }

Resources