Convert YAML string to dict in Helm template - go

I am creating a chart for a project that has a binary that when executed generates a configuration file in YAML format that looks like this:
---
PARAM_1: value1
PARAM_2: value2
My chart needs to read this file and and load all of its values into environment variables in a container, so I created a variable config in my values.yaml file and when the chart is installed I am passing the file content using --set-file:
helm install <CHART> --set-file config=/path/to/yaml/config/file
Next I create a ConfigMap withe the value of .Values.config:
apiVersion: v1
kind: ConfigMap
metadata:
...
data:
{{ .Values.config }}
The problem I am having is that I need to do two things with values of config:
prefix all keys with a predefined value (so in the example above I would MY_APP_PARAM_1 as key)
make sure the values are all string, otherwise the ConfigMap will fail
How can I parse the value of .Values.config in my template as a dict so that I can use a range loop do these changes?

In the end I was able to do something like this:
{{ $lines := splitList "\n" .Values.config -}}
{{- range $lines }}
{{- if not (. | trim | empty) -}}
{{- $kv := . | splitn ":" 2 -}}
{{ printf "MY_APP_%s: %s" $kv._0 ($kv._1 | trim | quote) | indent 2 }}
{{ end -}}
{{- end -}}
I had a hard time getting the {{- vs {{ right, and helm install --debug --dry-run . help a lot in this part.
It's kind of messy, so I would be very interested in seeing if anyone has a better solution.

If You need to work with more tricky YAML (map in a map) You can use this:
List Handler (list to comma separated string)
{{- define "helm-toolkit.utils.joinListWithComma" -}}
{{- $local := dict "first" true -}}
{{- range $k, $v := . -}}{{- if not $local.first -}}, {{ end -}}{{ $v -}}{{- $_ := set $local "first" false -}}{{- end -}}
{{- end -}}
Convert Yaml dict to the properties-like look
{{- define "utils.yaml2properties" }}
{{- $yaml := . -}}
{{- range $key, $value := $yaml }}
{{- if kindIs "map" $value -}}
{{ $top:=$key }}
{{- range $key, $value := $value }}
{{- if kindIs "map" $value }}
{{- $newTop := printf "%s.%s" $top $key }}
{{- include "utils.yaml2properties" (dict $newTop $value) }}
{{- else if kindIs "slice" $value }}
{{ $top }}.{{ $key }}={{ include "helm-toolkit.utils.joinListWithComma" $value }}
{{- else }}
{{ $top }}.{{ $key }}={{ $value }}
{{- end }}
{{- end }}
{{- else if kindIs "slice" $value }}
{{ $key }}={{ include "helm-toolkit.utils.joinListWithComma" $value }}
{{- else }}
{{ $key }}={{ $value }}
{{- end }}
{{- end }}
{{- end }}
Use (example from K8s configMap)
{{- define "config.yaml" }}
vault.pki:
enabled: {{.Values.vault.pki.enabled}}
role: idverify
common-name: idverify
role:
map:
list:
- one
- two
{{- end }}
data:
bootstrap.properties: |
{{ (include "utils.yaml2properties" (include "config.yaml" . | fromYaml )) | indent 4 }}

Related

Helm template for Vault annotation

I'm trying to nest one template into another, namely I have a k8s deployment.yaml which should be templated by helm. However, it is already using Vault template in annotation field responsible for injecting secrets into the pod.
Without Helm template it looks like this:
vault.hashicorp.com/agent-inject-template-.env: |
{{- with secret (print "envs/data/test") -}}{{- range $k, $v := .Data.data -}}
{{ $k }}={{ $v }}
{{ end }}{{- end -}}
Now, I would like to make a template for Helm and replace path envs/data/test with value coming from Helm values. So I've tried to use it like that:
vault.hashicorp.com/agent-inject-template-.env: |
{{- with secret (print {{ .Values.path }}) -}}{{- range $k, $v := .Data.data -}}
{{ $k }}={{ $v }}
{{ end }}{{- end -}}
This however does not work meaning {{ .Values.path }} is outputed as is, as a string.
How can I template over another template and resolve {{ .Values.path }} as a variable?

Check if a key/value exists in helm chart yaml and add a value to it if exists, otherwise create a new key-pair value

Let's say that I have the following structure in one helm template file
{{- define "deployAnnotations" -}}
"key1" : "value1",
"key2" : "value2"
"ip_range": "20.20.20.20/32"
{{- end -}}
and the above are included in the deployment.yaml file of my helm chart
template:
metadata:
annotations:
{{ include "podAnnotations" . | nindent 8 }}
Now I want to add one more annotation that also define in a template file the following one:
{{- define "ipdeployAnnotations" -}}
ip_range: 10.10.10.10/32
{{- end -}}
My question is how can I search if ip_range exists in the podAnnotations structure. If exists I just want to add the 10.10.10.10/32 to the existing line , otherwise I would like to create a new one.
I imagine something like below but in operator is not working for me , and also I do not know how to add the additional value to ip_range
template:
metadata:
annotations:
{{ include "podAnnotations" . | nindent 8 }}
{{ - if "ip_range" in "podAnnotations" }}
// do something in order to have the following result
"ip_range": "20.20.20.20/32", "10.10.10.10/32"
{{- else }}
{{ include "ipdeployAnnotations" . | nindent 8 }}
{{- end }}
You may convert it to a struct first, and then use it in the way of manipulating dictionary.
templates/_helper.tpl
{{- define "deployAnnotations" -}}
"key1" : "value1"
"key2" : "value2"
"ip_range": "20.20.20.20/32"
{{- end -}}
{{- define "ipdeployAnnotations" -}}
ip_range: "10.10.10.10/32"
{{- end -}}
templates/xxx.yaml
...
{{- $da := (include "deployAnnotations" . | fromYaml) }}
{{- $ida := (include "ipdeployAnnotations" . | fromYaml) }}
{{- if (hasKey $da "ip_range") }}
{{- $_ := set $da "ip_range" (printf "%s, %s" $da.ip_range $ida.ip_range) }}
{{- else }}
{{- $_ := set $da "ip_range" $ida.ip_range }}
{{- end }}
{{- range $k, $v := $da }}
{{ $k }}: {{ $v }}
{{- end }}
...
output
ip_range: 20.20.20.20/32, 10.10.10.10/32
key1: value1
key2: value2

Helm configmap error. expected string; got bool

I have a Helm chart with has multiple templates. One is the configmap which was working fine.
But when I want to add the enabled part I´m getting the error message.
executing "base-helm-chart/templates/configmap.yaml" at <$config>: wrong type for value; expected string; got bool
This are the files I´m using:
{{- if .Values.configMap.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
namespace: {{ .Release.Namespace }}
name: {{include "chart.fullname" .}}
labels: {{ include "chart.labels" . | nindent 4 }}
data:
{{- range $name, $config := .Values.configMap }}
{{ $name }}: |
{{ tpl $config $ | indent 4 }}
{{- end }}
{{- end -}}
values.yaml
configMap:
enabled: true
config.json: |
food = pizza
drink = soda
I want user to enable/disable if he wants to add configmap or not from the values.yaml
You can add condition to skip value of another type then string to be passed in tpl function
{{- range $name, $config := .Values.configMap -}}
{{ if typeOf $config | eq "string" }}
{{ $name }}: |
{{- tpl $config $ | nindent 12 }}
{{ end }}
{{ end }}
{{ end }}
If you want to also print another key value in output then you can use print, printf, println or any other print option.
{{- range $name, $config := .Values.configMap -}}
{{ if typeOf $config | eq "string" }}
{{ $name }}: |
{{- tpl $config $ | nindent 12 }}
{{- else -}}
{{ printf "%v: %v" $name $config }}
{{ end }}
{{ end }}
{{ end }}
I want user to enable/disable if he wants to add configmap or not from the values.yaml
Does this mean that by default the configMap is empty?
If that's the case, you can check for the empty value
{{- if .Values.configMap }}
apiVersion: v1
kind: ConfigMap
metadata:
namespace: {{ .Release.Namespace }}
name: {{include "chart.fullname" .}}
labels: {{ include "chart.labels" . | nindent 4 }}
data:
{{- range $name, $config := .Values.configMap }}
{{ $name }}: |
{{ tpl $config $ | indent 4 }}
{{- end }}
{{- end -}}
and in values.yaml the default is an empty dictionary:
configMap: {}
In this way, only when a user fills in configMap, the manifest will be generated.
Aside from this optional activation, you seem to have a problem in the iteration over the values, because they differ types.
You can use the much simpler toYaml filter (see here)
The end result could be something like this:
{{- if .Values.configMap }}
apiVersion: v1
kind: ConfigMap
metadata:
namespace: {{ .Release.Namespace }}
name: {{include "chart.fullname" .}}
labels: {{ include "chart.labels" . | nindent 4 }}
data:
{{- toYaml .Values.configMap | nindent 2 -}}
{{- end -}}

default key value when reading values from values file in helm chart

I want to pass the default key value while reading values file.
{{- range $key, $val := .Values.nodeLabel | default map[string]string{"a":"b"}}}
Values.yaml nodeLabel - a: b
But i am trying to pass default value too.
{{- range $key, $val := .Values.nodeLabel | default "b:c"
error- range cannot iterate over b:c
Please try this:
{{- if .Values.nodeLabel -}}
{{- range $key, $val := .Values.nodeLabel }}
{{ $key }}: {{ $val }}
# {{- end }}
{{ else }}
{{ default "b: c" }}
{{- end -}}
Additional resources: The Chart Best Practices Guide
Hope this help
{{- range $key, $val := .Values.nodeLabel | default (dict "a" "b")}}

Using shell script to parse lines from a consul template file

As you can see the in the template file below , I have few variables declared (like MYSQL_ACCOUNT_PASSWORD , MYSQL_ACCOUNT_USERNAME etc) between "if" and "end" and I am looking forward to write a shell script to get all the variables in such file.
With the format of the consul template using "Golang's text/template package", I am afraid that basic tools like cut,sed or awk wont do my job. How can I do this.
{{- template "findService" "mysql-master-acc" }}
{{- if scratch.Key "mysql-master-acc-service" }}
{{- $mysql_account_db_name := printf "service/%s/mysql-master-acc/%s/config/acc_mysql_db_name" $dc ( scratch.Get "mysql-master-acc-tag" ) | key }}
{{ range scratch.Get "mysql-master-acc-service"}}
MYSQL_ACCOUNT_URL=jdbc:mysql://{{- .Address }}:{{- .Port }}/{{ $mysql_account_db_name }}?useSSL=false&autoReconnect=true&serverTimezone=America/Chicago&rewriteBatchedStatements=true
{{- end }}
{{- if eq (env "SECRETS_BACKEND") "vault" }}
{{ $vault_secret_path := printf "secret/service/%s/kos-mysql-acc/%s" $dc ( scratch.Get "mysql-master-acc-tag" ) -}}
{{ with printf "%s/%s" $vault_secret_path $mysql_account_db_name | secret -}}
MYSQL_ACCOUNT_PASSWORD={{ .Data.password }}
MYSQL_ACCOUNT_USERNAME={{ .Data.username }}
{{- end }}
{{- else if eq ( env "SECRETS_BACKEND" ) "consul" }}
{{- $secretPath := printf "secret/%s/mysql-master-acc/%s/acc_mysql_db_user_password" $dc ( scratch.Get "mysql-master-acc-tag" ) }}
{{- $keyPath := printf "service/%s/mysql-master-acc/%s/config/acc_mysql_db_user" $dc ( scratch.Get "mysql-master-acc-tag" ) }}
MYSQL_ACCOUNT_PASSWORD={{ key $secretPath }}
MYSQL_ACCOUNT_USERNAME={{ key $keyPath }}
{{- end }}
{{- end }}
Could you just use grep?
To get all lines that don't start with {{ you could use
grep -v "^{{" inputfile
If you want all lines starting with MYSQL..., use
grep "^MYSQL" inputfile
In this case, these both return:
MYSQL_ACCOUNT_URL=jdbc:mysql://{{- .Address }}:{{- .Port }}/{{ $mysql_account_db_name }}?useSSL=false&autoReconnect=true&serverTimezone=America/Chicago&rewriteBatchedStatements=true
MYSQL_ACCOUNT_PASSWORD={{ .Data.password }}
MYSQL_ACCOUNT_USERNAME={{ .Data.username }}
MYSQL_ACCOUNT_PASSWORD={{ key $secretPath }}
MYSQL_ACCOUNT_USERNAME={{ key $keyPath }}

Resources