I'm looking to merge two values files in helm.
secrets.yaml:
serviceMonitor:
endpoints:
- module: oracledb
port: http
scheme: http
url: "http://user:password#ip:port/xxx"
I have another values.yaml file which has multiple endpoints. I want to merge both the values files. I'm tried using append function to do that: {{$endpoints := (append .Values.serviceMonitor.endpoints .Values.oracle_db.serviceMonitor.endpoints) }} When I do a dry-run, I see its picking up both the values but won't merge. Any one come across this?
In the current Helm version (3) merging values is not supported.
This feature was discussed in this Github issue: Helm should preform deep merge on multiple values files.
One important quote from there
If this is implemented, it should be optional, i. e. the merge mode should be specified as a command-line flag. Anything else would be a breaking change and, I think, in most cases not desirable. The problem is that you would not be able to override defaults for lists.
See: https://github.com/helm/helm/issues/3486#issuecomment-364534501
You can use a python script to merge the values files before passing them. Below is a code snippet of what I am using.
import yaml
from deepmerge import always_merger
fileA = “tmp.yaml"
fileB = “feature.yaml"
with open(fileA,'r+') as f:
fileAdictionary= yaml.load(f)
with open(fileB,'r+') as f:
fileBdictionary = yaml.load(f)
result = always_merger.merge(fileAdictionary, fileBdictionary)
with open(‘newFile.yaml’,'w+') as f:
yaml.dump(result,f)
another option:
{{- define "template.valueOrDefault" -}}
{{- $value := dict -}}
{{- range (rest .) -}}
{{- $value = merge $value . -}}
{{- end -}}
{{- if $value -}}
{{- printf "%s:" (first .) }}
{{- toYaml $value | nindent 2 }}
{{- end }}
{{- end -}}
usage:
containers:
- name: db-migrations-job
{{- include "template.valueOrDefault" (list "resources" .Values.migrationJob.resources .Values.resources) | nindent 8 }}
Related
Assuming I have the following values.yaml for a subchart:
global:
ingress:
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
add_header yyy "yyy";
tag: 123
port: 1234
ingress:
enabled: true
annotations:
nginx.ingress.kubernetes.io/configuration-snippet: |
add_header xxx "xxx";
...
how can I merge both annotation blocks in the ingress.yaml template together so that it results in:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations: |
add_header xxx "xxx";
add_header yyy "yyy";
...
Has anyone a hint where to start here?
Both values are just strings, and you can either write them one after another or concatenate them at the template layer.
So, for example, a minimal thing that should come close to working, for this specific example, could be
{{- $key := "nginx.ingress.kubernetes.io/configuration-snippet" -}}
metadata:
annotations: |
{{ index .Values.ingress.annotations $key | trim | indent 4 }}
{{ index .Values.global.ingress.annotations $key | trim | indent 4 }}
This takes advantage of the fact that Helm doesn't really "understand" YAML at the templating layer; instead, a template writes out an arbitrary string and them Helm tries to parse it afterwards. So you can use the YAML | block scalar marker, and then write out arbitrary content under it, and so long as you get the indentation right it will work.
The question sounds like it's reaching for a more general question of how to merge the two annotation lists, combining individual values by concatenating the strings. Helm includes a set of dictionary template functions, which somewhat unusually work by mutating a dictionary in-place. You can combine this with the underdocumented toYaml function to write the dictionary in valid YAML syntax.
In pseudocode, I might write:
Create a new dictionary that's a copy of the local annotations.
Loop through the global annotations. For each, if the key does not exist, save its value, but if it does, append the global value.
Write the result as YAML.
You could translate this into Helm template code:
{{- $annotations := deepCopy .Values.ingress.annotations -}}
{{- range $key, $value := .Values.global.ingress.annotations -}}
{{- $existing := get $annotations $key -}}
{{- $new := cat $existing $value -}}
{{- $_ := set $annotations $key $new -}}
{{- end -}}
metadata:
annotations:
{{ $annotations | toYaml | indent 4 }}
In particular here I've taken advantage of the property that get returns an empty string if the key doesn't exist, which happens to be what you want in this particular case. For other "reduce values" type operations you might need to check if the value exists using hasKey, or use a default value.
I have config like this:
{{- with .Files.Glob "files/my-files/*.json" }}
{{ .AsConfig | indent 2 }}
{{- end }}
In the end of each file I want to add "FIHISHED!"
How can I achieve it in helm ?
The .AsConfig method renders and returns all files as a single YAML text. So you can't format the result.
If you want to list all files (with content), separated with an arbitrary text, I suggest to do this "yourself". Files is a map of files, mapping from string name to []byte content.
{{- with .Files.Glob "files/my-files/*.json" }}
{{ range $name, $content := . -}}
{{ printf "-%s:\n%s\nFINISHED!" $name $content | indent 2 }}:
{{- end }}
{{- end }}
I'd like my Helm chart to use different image repository depending on cloud provider (AWS vs. Aliyun) unfortunately I am getting the following error when trying to run helm package command:
Error: cannot load values.yaml: error converting YAML to JSON: yaml: invalid map key: map[interface {}]interface {}{"tpl (.Files.Get \"config/repository.config\") . | quote":interface {}(nil)}
In my values.yaml
I have:
configuration:
system:
mode:
cloud_provider: aws
image:
repository: {{ tpl (.Files.Get "config/repository.config") . | quote }}
in my config/repository.config file I have:
{{- if eq ( include "cloud_provider" . | trim ) "aliyun" -}}
registry.cn-qingdao.aliyuncs.com/x/kube-state-metrics
{{- end -}}
{{- if eq ( include "cloud_provider" . | trim ) "aws" -}}
k8s.gcr.io/kube-state-metrics/kube-state-metrics
{{- end -}}
in templates/_helpers.tpl file I've added:
{{- define "cloud_provider" -}}
{{ required "Cloud provider required at .Values.configuration.system.mode.cloud_provider due to Google to unreachable in China" .Values.configuration.system.mode.cloud_provider }}
{{ $valid_cloud_provider := list "aws" "aliyun" }}
{{- if not (has .Values.configuration.system.mode.cloud_provider $valid_cloud_provider ) -}}
{{ fail "Invalid cloud provider set. Should be aws or aliyun" }}
{{end}}
{{- else -}}
{{ .Values.configuration.system.mode.cloud_provider | default "disabled" }}
{{- end -}}
{{- end -}}
This is not a direct solution for the error but an alternative approach. I hope this also could help:
I won't put such expressions in the values file values.yaml
repository: {{ tpl (.Files.Get "config/repository.config") . | quote }}
I would use instead different yaml files for each environment:
Add a aliyun.yaml values file
registry: registry.cn-qingdao.aliyuncs.com/x/kube-state-metrics
And call helm with this values file
helm ... -f aliyun.yaml
I have a function that I want to call in another function. I want it to return a map, but instead it returns the toString representation of the map. It is an actual map inside the function, but not outside it.
{{- define "app.getSubKey" -}}
{{- $name := .source }}
{{- range $key, $value := .keys }}
{{- if kindIs "int" $value }}
{{- $name = index $name (int $value) }}
{{- else }}
{{- $name = index $name $value }}
{{- end }}
{{- end }}
{{- if kindIs "string" $name }}
{{- trim $name }}
{{- else }}
{{ $name }}
{{- end }}
{{- end }}
When I call this function on the below YAML with {{- include "app.getSubKey" (dict "source" .Values.vars "keys" (list 0)) }}, inside the function $name is a map (I used kindOf to check this) but outside it's of kind string. How can I get it returned as a map?
vars:
- name: something
value: blah
To be precise: your app.getSubKey is not a function, it is a named template and as such, it does not return anything. On the other hand, include is a function. It takes as arguments a template name and its context, and returns rendered template as a string. You cannot change the type of the include function return value.
Said that, I see two options:
if your goal is to get a properly formatted YAML from the include "app.getSubKey" call (still as a string) use toYaml function inside the named template, i.e.:
{{- define "app.getSubKey" -}}
...
{{- else }}
{{- $name | toYaml | nindent 0 }}
{{- end }}
{{- end }}
nindent 0 is necessary to get a properly left align YAML
if you want to get an actual dictionary (for further processing), you need to convert the string returned from the include call back to an object (the toYaml in the named template is still required):
{{- $d := dict "source" .Values.vars "keys" (list 0) | include "app.getSubKey" | fromYaml -}}
The function fromYaml is undocumented, so using it might be a bit risky. The only trace of it I found in the official docs is in the examples on the library charts page.
BTW, the app.getSubKey logic doesn't seem correct: it will fail for the keys list with more then one element.
If app.getSubKey returns an array, you must use fromYamlArray to "decode" the data.
I am new to Kubernetes, Helm, and Golang. Basically, I'm trying to concat strings in my values.yaml file to populate another value in my values.yaml file. I'm using _helpers.tpl to do this (in an attempt to understand Helm and Golang better). I could just write "users: "datafeed:password:::incoming" " in my values.yaml file and be done with it; but would like to avoid that.
I have the following in my values.yaml file:
sftp:
users: ""
username: "datafeed"
password: "password"
incoming: "incoming"
And want the final values.yaml file to read:
sftp:
users: "datafeed:password:::incoming"
username: "datafeed"
password: "password"
incoming: "incoming"
To do this, I am trying to edit the _helpers.tpl file. I have tried
{{- define "sftp.users" -}}
{{- .Values.sftp.users: .Values.sftp.username+":"+.Values.sftp.password+":::"+.Values.sftp.incoming -}}
{{- end -}}
and
{{- define "sftp.users" -}}
{{- .Values.sftp.users:= .Values.sftp.username+":"+.Values.sftp.password+":::"+.Values.sftp.incoming -}}
{{- end -}}
Then I tried making each segment a variable (and deleted the explicit values in the values.yaml file):
{{- define "sftp.users" -}}
{{ $username:= "datafeed" }}
{{ $password:= "password" }}
{{ $incoming:= "incoming" }}
{{- .Values.sftp.users= {{$username}}+":"+{{$password}}":::"+{{$incoming}} -}}
and then setting the fields/keys explicitly:
username: {{ .Values.sftp.username | default "datafeed" }}
password: {{ .Values.sftp.password | default "password" }}
incoming: {{ .Values.sftp.incoming | default "incoming" }}
{{- .Values.sftp.users:= username+":"+password+":::"+incoming -}}
and:
{{define "username"}}datafeed{{end}}
{{define "password"}}password{{end}}
{{define "incoming"}}incoming{{end}}
{{define "users"}}{{template "username"}}:{{template "password"}}:::{{template "incoming"}}{{end}}
{{- printf "users" -}}
{{- .Values.sftp.users: users -}}
I have also looked at previous posts:
Helm _helpers.tpl: Calling defined templates in other template definitions
Kubernetes Helm, combine two variables with a string in the middle
How to get values from values.yaml to _helpers.tpl in helm charts
None of this seems to work. I can't tell if it's my approach or my syntax. Probably both.