Merge annotations in Helm - yaml

Assuming I have the following values.yaml for a subchart:
global:
ingress:
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
add_header yyy "yyy";
tag: 123
port: 1234
ingress:
enabled: true
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
add_header xxx "xxx";
...
how can I merge both annotation blocks in the ingress.yaml template together so that it results in:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations: |
add_header xxx "xxx";
add_header yyy "yyy";
...
Has anyone a hint where to start here?

Both values are just strings, and you can either write them one after another or concatenate them at the template layer.
So, for example, a minimal thing that should come close to working, for this specific example, could be
{{- $key := "nginx.ingress.kubernetes.io/configuration-snippet" -}}
metadata:
annotations: |
{{ index .Values.ingress.annotations $key | trim | indent 4 }}
{{ index .Values.global.ingress.annotations $key | trim | indent 4 }}
This takes advantage of the fact that Helm doesn't really "understand" YAML at the templating layer; instead, a template writes out an arbitrary string and them Helm tries to parse it afterwards. So you can use the YAML | block scalar marker, and then write out arbitrary content under it, and so long as you get the indentation right it will work.
The question sounds like it's reaching for a more general question of how to merge the two annotation lists, combining individual values by concatenating the strings. Helm includes a set of dictionary template functions, which somewhat unusually work by mutating a dictionary in-place. You can combine this with the underdocumented toYaml function to write the dictionary in valid YAML syntax.
In pseudocode, I might write:
Create a new dictionary that's a copy of the local annotations.
Loop through the global annotations. For each, if the key does not exist, save its value, but if it does, append the global value.
Write the result as YAML.
You could translate this into Helm template code:
{{- $annotations := deepCopy .Values.ingress.annotations -}}
{{- range $key, $value := .Values.global.ingress.annotations -}}
{{- $existing := get $annotations $key -}}
{{- $new := cat $existing $value -}}
{{- $_ := set $annotations $key $new -}}
{{- end -}}
metadata:
annotations:
{{ $annotations | toYaml | indent 4 }}
In particular here I've taken advantage of the property that get returns an empty string if the key doesn't exist, which happens to be what you want in this particular case. For other "reduce values" type operations you might need to check if the value exists using hasKey, or use a default value.

Related

Helm: It is possible to use nindent function with template function?

I'm a bit new to helm syntax and I'm trying to include a template in my deployments file. In some of my deployments the indent must be different. Is there a way to include my template and use indent function as well?
{{ template "my-chart.nodeaffinity" include "my-chart.fullname" . | toYaml | nindent 8 }}
It is possible to use the template and toYaml function in the same statement?
Thanks!
I have tried to add the bellow code but is not working and when I use helm template the nindent function is not working.
| toYaml | nindent 8
You can't really use template and toYaml together as you describe. What you can do is use the similar but Helm-specific include function in place of template:
{{ include "my-chart.nodeaffinity" (include "my-chart.fullname" .) | toYaml | nindent 8 }}
{{/*^^^^^^ not template ^--- need parens here ---^ */}}
template is a built-in Go text/template action. It consumes the pipeline in the rest of its block, and it always sends its output to the rendered template output. So {{ template "name" "arg" | toYaml }} reads "arg" | toYaml as the parameter to the template, and there's no way to process its output.
include is an extension function in the template syntax. It renders a template in the same way template does, but it returns the rendered string, which can then participate in pipelines. So {{ include "name" "arg" | toYaml }} calls the template with exactly the specified argument, and then processes the return value.
One corollary to this is that include always returns a string. Even if it looks like the helper template is producing a YAML object, it's really a string. This might mean you don't want toYaml here, since it could cause extra escaping you don't need.
You can see a more concrete example of this in the default chart scaffolding. A standard set of labels is defined as a template
{{- define "<CHARTNAME>.labels" -}}
helm.sh/chart: {{ include "<CHARTNAME>.chart" . }}
...
{{- end }}
When this is used, it's used via include (not template), and its output is (n)indented
metadata:
labels:
{{- include "<CHARTNAME>.labels" . | nindent 4 }}

helm filter over range

Is it possible to filter the results of a range in helm?
For example the values file contains a list of maps like:
clients:
- name: clientA
id: id001
user: usernameA
pass: passwordA
- name: clientB
id: id002
user: usernameB
pass: passwordB
- name: clientA
id: id003
user: usernameA
pass: passwordA
In my template I need to extract only the unique values of user and pass.
I have something like this:
{{- range .Values.clients }}
- name: {{ .name | printf "user_%s" }}
valueFrom:
secretKeyRef:
name: somesecret
key: {{ .user | quote }}
- name: {{ .name | printf "pass_%s" }}
valueFrom:
secretKeyRef:
name: somesecret
key: {{ .pass | quote }}
{{- end }}
The end result should be something like:
- name: user_clientA
valueFrom:
secretKeyRef:
name: somesecret
key: usernameA
- name: pass_clientA
valueFrom:
secretKeyRef:
name: somesecret
key: passwordA
- name: user_clientB
valueFrom:
secretKeyRef:
name: somesecret
key: usernameB
- name: pass_clientB
valueFrom:
secretKeyRef:
name: somesecret
key: passwordB
I tried {{- range .Values.clients | pick "user" "pass" | uniq }} but doesn't seem to work. I don't know how to write this so i can filter just "user" and "pass" keys and discard the duplicates.
Even with its extensions, Helm doesn't support deep filtering of lists like you propose. There's not a great way to say "filter this list to the unique values of this map item", or "given an item, is it the first one in the list with some property". In a more functional language I could imagine using constructs like map or filter here, and Helm has no equivalents to these.
In practice the thing I'd be most likely to do is to declare this an invalid configuration. It would be lazy, but not unreasonable, to just execute the template as you've shown it. (If this is in a pod spec's env: block, I believe this would produce a valid object and the last setting of each environment variable takes effect.)
It's possible to do this checking in Helm templates; but it's a lot of code, in an unfamiliar language, that's hard to test. Also consider whether something like an operator could be a better implementation choice for you, since this can be written in a more standard language.
If you believe the length of the list will be reasonably short, you can write a recursive template to process this. It would receive the list of remaining items and the list of names it's already seen. If the list of items is empty, stop; if the current item has been seen, call ourselves with the rest of the list and the same seen list; otherwise emit the item, add it to the "seen" list, and call ourselves. In Python pseudocode this could look like:
def helper(remaining, seen):
if len(remaining) == 0:
# do nothing if the list is empty
return
else:
first = remaining[0]
name = first['name']
rest = remaining[1:]
if name in seen:
# we have already seen this item; skip it and go to the next one
return helper(rest, seen)
else:
# this is a new item; emit it and remember its name
emit(first)
new_seen = seen + [name]
return helper(rest, new_seen)
def process(the_list):
helper(the_list, [])
You can translate that logic to Helm templates, with the additional trick of packing the multiple parameters into a single list, since the Go text/template templates only take a single parameter. (If you need the top-level Helm object for .Values, .Release, .Files, etc. you need to pass that explicitly as well.)
{{- define "helper" -}}
{{- $remaining = index . 0 -}}
{{- $seen := index . 1 -}}
{{- if empty $remaining -}}
{{-/* do nothing */-}}
{{- else -}}
{{- $first := first $remaining -}}
{{- $name := $first.name -}}
{{- $rest := rest $remaining -}}
{{- if has $name $seen -}}
{{-/* recurse without emitting anything */-}}
{{- template "helper" (list $rest $seen) -}}
{{- else -}}
{{-/* emit the item */-}}
- name: user_{{ $first.name }}
valueFrom:
secretKeyRef:
name: somesecret
key: {{ quote $first.user }}
- name: pass_{{ $first.name }}
valueFrom:
secretKeyRef:
name: somesecret
key: {{ quote $first.pass }}
{{/* ...then move on to the next item */-}}
{{- $new_seen := append $seen $first.name -}}
{{- template "helper" (list $first $new_seen) -}}
{{- end -}}
{{- end -}}
{{- end -}}
env:
{{ include "helper" (list .Values.clients list) | indent 2 }}

Merge two Value files in helm

I'm looking to merge two values files in helm.
secrets.yaml:
serviceMonitor:
endpoints:
- module: oracledb
port: http
scheme: http
url: "http://user:password#ip:port/xxx"
I have another values.yaml file which has multiple endpoints. I want to merge both the values files. I'm tried using append function to do that: {{$endpoints := (append .Values.serviceMonitor.endpoints .Values.oracle_db.serviceMonitor.endpoints) }} When I do a dry-run, I see its picking up both the values but won't merge. Any one come across this?
In the current Helm version (3) merging values is not supported.
This feature was discussed in this Github issue: Helm should preform deep merge on multiple values files.
One important quote from there
If this is implemented, it should be optional, i. e. the merge mode should be specified as a command-line flag. Anything else would be a breaking change and, I think, in most cases not desirable. The problem is that you would not be able to override defaults for lists.
See: https://github.com/helm/helm/issues/3486#issuecomment-364534501
You can use a python script to merge the values files before passing them. Below is a code snippet of what I am using.
import yaml
from deepmerge import always_merger
fileA = “tmp.yaml"
fileB = “feature.yaml"
with open(fileA,'r+') as f:
fileAdictionary= yaml.load(f)
with open(fileB,'r+') as f:
fileBdictionary = yaml.load(f)
result = always_merger.merge(fileAdictionary, fileBdictionary)
with open(‘newFile.yaml’,'w+') as f:
yaml.dump(result,f)
another option:
{{- define "template.valueOrDefault" -}}
{{- $value := dict -}}
{{- range (rest .) -}}
{{- $value = merge $value . -}}
{{- end -}}
{{- if $value -}}
{{- printf "%s:" (first .) }}
{{- toYaml $value | nindent 2 }}
{{- end }}
{{- end -}}
usage:
containers:
- name: db-migrations-job
{{- include "template.valueOrDefault" (list "resources" .Values.migrationJob.resources .Values.resources) | nindent 8 }}

Kubernetes Helm chart: Use _helpers.tpl to concat values and feed back into Values.yaml

I am new to Kubernetes, Helm, and Golang. Basically, I'm trying to concat strings in my values.yaml file to populate another value in my values.yaml file. I'm using _helpers.tpl to do this (in an attempt to understand Helm and Golang better). I could just write "users: "datafeed:password:::incoming" " in my values.yaml file and be done with it; but would like to avoid that.
I have the following in my values.yaml file:
sftp:
users: ""
username: "datafeed"
password: "password"
incoming: "incoming"
And want the final values.yaml file to read:
sftp:
users: "datafeed:password:::incoming"
username: "datafeed"
password: "password"
incoming: "incoming"
To do this, I am trying to edit the _helpers.tpl file. I have tried
{{- define "sftp.users" -}}
{{- .Values.sftp.users: .Values.sftp.username+":"+.Values.sftp.password+":::"+.Values.sftp.incoming -}}
{{- end -}}
and
{{- define "sftp.users" -}}
{{- .Values.sftp.users:= .Values.sftp.username+":"+.Values.sftp.password+":::"+.Values.sftp.incoming -}}
{{- end -}}
Then I tried making each segment a variable (and deleted the explicit values in the values.yaml file):
{{- define "sftp.users" -}}
{{ $username:= "datafeed" }}
{{ $password:= "password" }}
{{ $incoming:= "incoming" }}
{{- .Values.sftp.users= {{$username}}+":"+{{$password}}":::"+{{$incoming}} -}}
and then setting the fields/keys explicitly:
username: {{ .Values.sftp.username | default "datafeed" }}
password: {{ .Values.sftp.password | default "password" }}
incoming: {{ .Values.sftp.incoming | default "incoming" }}
{{- .Values.sftp.users:= username+":"+password+":::"+incoming -}}
and:
{{define "username"}}datafeed{{end}}
{{define "password"}}password{{end}}
{{define "incoming"}}incoming{{end}}
{{define "users"}}{{template "username"}}:{{template "password"}}:::{{template "incoming"}}{{end}}
{{- printf "users" -}}
{{- .Values.sftp.users: users -}}
I have also looked at previous posts:
Helm _helpers.tpl: Calling defined templates in other template definitions
Kubernetes Helm, combine two variables with a string in the middle
How to get values from values.yaml to _helpers.tpl in helm charts
None of this seems to work. I can't tell if it's my approach or my syntax. Probably both.

Helm yaml keys from values.yaml

I want to make a yaml KEY (not the value) dynamically.
In my values.yaml
failoverip1: 0.0.0.0` (<- this is only a demo IP)
In my templates/configmap.yaml I have this:
apiVersion: v1
kind: ConfigMap
metadata:
name: vip-configmap
data:
{{- .Values.failoverip1 -}}: {{ .Release.Namespace -}}/{{- .Values.target -}}
^^^^^^^^^^^^^^^^^^^^^----> here should be an IP address from values.yaml
{{ .Release.Namespace -}}/{{- .Values.target -}} renders successfully.
But if I add {{- .Values.failoverip1 -}} to the key part, it renders nothing.
(Nothing means, the whole data: block, does not get rendered.
This is the error message when I run helm install --name hetzner-failover .
Error: YAML parse error on hetzner-failover/templates/configmap-ip.yaml: error converting YAML to JSON: yaml: line 4: mapping values are not allowed in this context
Is it not allowed to make a
key dynamic?
If not, how to drive around that?
Here is the repo I am talking of:
https://github.com/exocode/helm-charts/blob/master/hetzner-failover/templates/configmap-ip.yaml
The error seems to be, that the leading - got cut.
So the correct way is to remove that minus:
Before:
{{- .Values.failoverip1 | indent 2 -}}
After:
{{ .Values.failoverip1 | indent 2 -}}
The yaml is now:
apiVersion: v1
kind: ConfigMap
metadata:
name: vip-configmap
data:
{{ .Values.failoverip1 | indent 2 -}}: {{ .Release.Namespace -}}/{{- .Values.target -}} # add your config map here. must map the base64 encoded IP in secrets.yaml
And the rendered result is:
kubectl get configmap -o yaml
apiVersion: v1
items:
- apiVersion: v1
data:
0.0.0.0: default/nginx# add your config map here. must map the base64 encoded
IP in secrets.yaml
kind: ConfigMap

Resources