Using Ansible YAML format, I would like to add some metadata to tasks definition (similar to kubernetes format):
tasks:
- name: Boostrap
block:
- shell: "helm repo add {{ item.name }} {{ item.url }}"
with_items:
- {name: "elastic", url: "https://helm.elastic.co"}
- {name: "stable", url: "https://kubernetes-charts.storage.googleapis.com/"}
metadata:
kind: utils
tags:
- always
- name: Stack Djobi > Metrics
include_tasks: "./tasks/metrics.yaml"
tags:
- service_metrics_all
metadata:
kind: stack
stack: metrics
- name: Stack Djobi > Tintin
include_tasks: "./tintin/tasks.yaml"
tags:
- tintin
metadata:
kind: stack
stack: tintin
Use case:
Add semantic information to YAML
Better debuging
Could be used to filter tasks like with tag (ansible-playbook ./playbook.yaml --filter kind=stack)
Is that possible? (currently ERROR! 'metadata' is not a valid attribute for a Block)
Related
Note: I have next to zero experience with Ansible.
I need to be able to conditionally modify the configuration of a Kubernetes cluster control plane service. To do this I need to be able to find a specific piece of information in the file and if its value matches a specific pattern, change the value to something else.
To illustrate, consider the following YAML file:
apiVersion: v1
kind: Pod
metadata:
labels:
component: kube-controller-manager
tier: control-plane
name: kube-controller-manager
namespace: kube-system
spec:
containers:
- command:
- kube-controller-manager
- --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
- --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
- --bind-address=127.0.0.1
...
In this scenario, the line I'm interested in is the line containing --bind-address. If that field's value is "127.0.0.1", it needs to be changed to "0.0.0.0". If it's already "0.0.0.0", nothing needs to be done. (I could also approach it from the point of view of: if its not "0.0.0.0" then it needs to change to that.)
The initial thought that comes to mind is: just search for "--bind-address=127.0.0.1" and replace it with "--bind-address=0.0.0.0". Simple enough, eh? No, not that simple. What if, for some reason, there is another piece of configuration in this file that also matches that pattern? Which one is the right one?
The only way I can think of to ensure I find the right text to change, is a multiple expression RegEx match. Something along the lines of:
find spec:
if found, find containers: "within" or "under" spec:
if found, find - command: "within" or "under" containers: (Note: there can be more than one "command")
if found, find - kube-controller-manager "within" or "under" - command:
if found, find - --bind-address "within" or "under" - kube-controller-manager
if found, get the value after the =
if 127.0.0.1 change it to 0.0.0.0, otherwise do nothing
How could I write an Ansible playbook to perform these steps, in sequence and only if each step returns true?
Read the data from the file into a dictionary
- include_vars:
file: conf.yml
name: conf
gives
conf:
apiVersion: v1
kind: Pod
metadata:
labels:
component: kube-controller-manager
tier: control-plane
name: kube-controller-manager
namespace: kube-system
spec:
containers:
- command:
- kube-controller-manager
- --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
- --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
- --bind-address=127.0.0.1
Update the containers
- set_fact:
containers: []
- set_fact:
containers: "{{ containers +
(update_candidate is all)|ternary([_item], [item]) }}"
loop: "{{ conf.spec.containers|d([]) }}"
vars:
update_candidate:
- item is contains 'command'
- item.command is contains 'kube-controller-manager'
- item.command|select('match', '--bind-address')|length > 0
update: "{{ item.command|map('regex_replace',
'--bind-address=127.0.0.1',
'--bind-address=0.0.0.0') }}"
_item: "{{ item|combine({'command': update}) }}"
gives
containers:
- command:
- kube-controller-manager
- --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
- --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
- --bind-address=0.0.0.0
Update conf
conf_update: "{{ conf|combine({'spec': spec}) }}"
spec: "{{ conf.spec|combine({'containers': containers}) }}"
give
spec:
containers:
- command:
- kube-controller-manager
- --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
- --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
- --bind-address=0.0.0.0
conf_update:
apiVersion: v1
kind: Pod
metadata:
labels:
component: kube-controller-manager
tier: control-plane
name: kube-controller-manager
namespace: kube-system
spec:
containers:
- command:
- kube-controller-manager
- --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
- --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
- --bind-address=0.0.0.0
Write the update to the file
- copy:
dest: /tmp/conf.yml
content: |
{{ conf_update|to_nice_yaml(indent=2) }}
gives
shell> cat /tmp/conf.yml
apiVersion: v1
kind: Pod
metadata:
labels:
component: kube-controller-manager
tier: control-plane
name: kube-controller-manager
namespace: kube-system
spec:
containers:
- command:
- kube-controller-manager
- --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
- --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
- --bind-address=0.0.0.0
Example of a complete playbook for testing
- hosts: localhost
vars:
conf_update: "{{ conf|combine({'spec': spec}) }}"
spec: "{{ conf.spec|combine({'containers': containers}) }}"
tasks:
- include_vars:
file: conf.yml
name: conf
- debug:
var: conf
- set_fact:
containers: []
- set_fact:
containers: "{{ containers +
(update_candidate is all)|ternary([_item], [item]) }}"
loop: "{{ conf.spec.containers|d([]) }}"
vars:
update_candidate:
- item is contains 'command'
- item.command is contains 'kube-controller-manager'
- item.command|select('match', '--bind-address')|length > 0
update: "{{ item.command|map('regex_replace',
'--bind-address=127.0.0.1',
'--bind-address=0.0.0.0') }}"
_item: "{{ item|combine({'command': update}) }}"
- debug:
var: containers
- debug:
var: spec
- debug:
var: conf_update
- copy:
dest: /tmp/conf.yml
content: |
{{ conf_update|to_nice_yaml(indent=2) }}
Currently using yq (mikefarah/yq/ - version 4.27.2) and having trouble modifying inline an existing yaml file.
What I'm trying to do:
select the labels field
if labels contains a field named monitored_item, create a key named affectedCi and use the value of monitored_item
if labels does not contain a field named monitored_item, create a key named affectedCi with the value {{ $labels.affected_ci }}
return the whole yaml object with changes made inline
jobs-prometheusRule.yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
labels:
app: jobs
prometheus: k8s
role: alert-rules
name: jobs-rules
namespace: ops
spec:
groups:
- name: job-detailed
rules:
- alert: JobInstanceFailed
annotations:
description: Please check the status of the {{ $labels.app_name }} job {{ $labels.job_name }} as it has failed.
summary: Job has failed
expr: (process_failed{context="job_failed"} + on(app_name, job_name) group_left(severity)(topk by(app_name, job_name) (1, property{context="job_max_allowed_failures"}))) == 1
for: 1m
labels:
monitored_item: '{{ $labels.app_name }} job {{ $labels.job_name }}'
severity: '{{ $labels.severity }}'
I've scoured through the docs and stack overflow with no luck - below is as far as i've been able to get.
yq command:
yq '(.spec.groups[].rules[] | select(.labels | has("monitored_item")) | .labels.affectedCi) |= .labels.monitored_item ' jobs-prometheusRule.yaml
The output returns the whole yaml object with the field affectedCi: null instead of the specified values
Anyone able to help?
You could use with to update, and // for fall-back:
yq 'with(
.spec.groups[].rules[] | select(.labels).labels;
.affectedCi = .monitored_item // "{{ $labels.affected_ci }}"
)' jobs-prometheusRule.yaml
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
labels:
app: jobs
prometheus: k8s
role: alert-rules
name: jobs-rules
namespace: ops
spec:
groups:
- name: job-detailed
rules:
- alert: JobInstanceFailed
annotations:
description: Please check the status of the {{ $labels.app_name }} job {{ $labels.job_name }} as it has failed.
summary: Job has failed
expr: (process_failed{context="job_failed"} + on(app_name, job_name) group_left(severity)(topk by(app_name, job_name) (1, property{context="job_max_allowed_failures"}))) == 1
for: 1m
labels:
monitored_item: '{{ $labels.app_name }} job {{ $labels.job_name }}'
severity: '{{ $labels.severity }}'
affectedCi: '{{ $labels.app_name }} job {{ $labels.job_name }}'
Apologies for this simple question, but I tried various approach without success.
This is my vars file
---
preprod:
name: nginx
prod:
name: apache
I am trying to pass the value of name based on the environment name user provides (preprod, prod etc).
This is my template
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: {{ env.name }}
name: {{ env.name }}
namespace: default
spec:
selector:
matchLabels:
app: {{ env.name }}
template:
metadata:
labels:
app: {{ env.name }}
spec:
containers:
- image: {{ env.name }}
imagePullPolicy: Always
name: {{ env.name }}
resources: {}
However, when I try with this using following command:
ansible-playbook playbook.yaml -e env=preprod
I am getting the following error.
fatal: [localhost]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: 'str object' has no attribute 'name'"}
My expectation is the {{ env.name }} should have been replaced with the value of preprod.name as in nginx in this case.
I want users to provide the value for env via -e on the command line, it seems if I do like preprod.name directly on the template, it seems to work, but I don't want that.
I hope this clarifies what I am trying to do, but it didn't work.
May I know what I am missing?
This error message indicates that the extra var passed on command line as -e is a string, and not the key (we expect) of the dict we are loading from the vars file.
I'm making up an example playbook as you have not shown how you load your vars file. I'm using include_vars as we can name the variable to load dict into.
# include vars with a name, so that we can access testvars[env]
- include_vars:
file: testvars.yml
name: testvars
- template:
src: test/deployment.yaml.j2
dest: /tmp/deployment.yaml
vars:
_name: "{{ testvars[env]['name'] }}"
With this approach, the prod and preprod keys will be available under testvars, and can be referenced with a variable such as env.
Then the template should use _name variable, like:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: {{ _name }}
Given the variables in a place where the play can find it, e.g. group_vars/all. Optionally add default_env
shell> cat group_vars/all
preprod:
name: nginx
prod:
name: apache
default_env:
name: lighttpd
Use vars lookup plugin to "Retrieve the value of an Ansible variable". See
shell> ansible-doc -t lookup vars
For example, the playbook
shell> cat playbook.yml
- hosts: test_11
vars:
env: "{{ lookup('vars', my_env|default('default_env')) }}"
app: "{{ env.name }}"
tasks:
- debug:
var: app
by default displays
shell> ansible-playbook playbook.yml
...
app: lighttpd
Now you can select the environment by declaring the variable my_env, .e.g
shell> ansible-playbook playbook.yml -e my_env=prod
...
app: apache
and
shell> ansible-playbook playbook.yml -e my_env=preprod
...
app: nginx
I am trying to load an ansible vault file into an k8 configmap YAML file using Ansible Jinja template but facing an issue with a trailing whitespace getting added at the end of the contents of the YAML file. This is causing errors as below:
Vault format unhexlify error: Odd-length string
Sample of ansible template am using is :
Playbook main.yml -
- name: display multiple files
shell: cat /tmp/test.yml
register: test
Ansible Jinja Template
apiVersion: v1
data:
test.yml: |-
{{ test.stdout.splitlines()|indent(4, false)|trim|replace(' ','') }}
kind: ConfigMap
metadata:
name: test
namespace: test-namespace
test.yml example:
$ANSIBLE_VAULT;1.1;AES256
62313365396662343061393464336163383764373764613633653634306231386433626436623361
6134333665353966363534333632666535333761666131620a663537646436643839616531643561
63396265333966386166373632626539326166353965363262633030333630313338646335303630
3438626666666137650a353638643435666633633964366338633066623234616432373231333331
6564
Output YAML created from Jinja Template is below
apiVersion: v1
data:
test.yml:
$ANSIBLE_VAULT;1.1;AES256
62313365396662343061393464336163383764373764613633653634306231386433626436623361
6134333665353966363534333632666535333761666131620a663537646436643839616531643561
63396265333966386166373632626539326166353965363262633030333630313338646335303630
3438626666666137650a353638643435666633633964366338633066623234616432373231333331
6564
kind: ConfigMap
metadata:
name: test
namespace: test-namespace
Can you please let me know what i may be missing in my ansible template file to fix the above trailing whitespace issues.
I am trying to load a Ansible Vault encrypted file into a configmap using jinja2 templating
Then you are solving the wrong problem; let the to_yaml filter do all that escaping for you, rather than trying to jinja your way through it.
- command: cat /tmp/test.yml
register: tmp_test
- set_fact:
cm_skeleton:
apiVersion: v1
data:
kind: ConfigMap
metadata:
name: test
namespace: test-namespace
- copy:
content: >-
{{ cm_skeleton | combine({"data":{"test.yml": tmp_test.stdout}}) | to_yaml }}
dest: /tmp/test.configmap.yml
If you have other things you are trying to template into that ConfigMap, fine, you can still do so, but deserialize in into a dict so you can insert the literal contents of test.yml into the dict and then re-serialize using the to_yaml filter:
- set_fact:
cm_skeleton: '{{ lookup("template", "cm.j2") | from_yaml }}'
- copy:
contents: '{{ cm_sketeton | combine({"data"...}) | to_yaml }}'
I am currently using a k8s lookup to search for resources with a certain tag attached to them (in this case branch). This branch is a variable that changes regularly. The problem is that I can't seem to find the correct syntax for adding a variable into the lookup since it is itself using the Jinja syntax.
This works:
- name: delete the replicaset
k8s:
state: absent
api_version: v1
kind: ReplicaSet
namespace: default
name: "{{ replicaset.metadata.name }}"
kubeconfig: /var/lib/awx/.kube/config
vars:
replicaset: "{{ lookup('k8s', kind='ReplicaSet', namespace='default', label_selector='branch=testing' ) }}"
However, when trying to use the branch variable, nothing I try seems to work. Here is one example of not working:
- name: delete the replicaset
k8s:
state: absent
api_version: v1
kind: ReplicaSet
namespace: default
name: "{{ replicaset.metadata.name }}"
kubeconfig: /var/lib/awx/.kube/config
vars:
replicaset: "{{ lookup('k8s', kind='ReplicaSet', namespace='default', label_selector='branch={{ branch }}' ) }}"
You can either add a helper variable:
- name: delete the replicaset
k8s:
state: absent
api_version: v1
kind: ReplicaSet
namespace: default
name: "{{ replicaset.metadata.name }}"
kubeconfig: /var/lib/awx/.kube/config
vars:
replicaset: "{{ lookup('k8s', kind='ReplicaSet', namespace='default', label_selector=my_selector ) }}"
my_selector: branch={{ branch }}
or use a Jinja2 string concatenation:
replicaset: "{{ lookup('k8s', kind='ReplicaSet', namespace='default', label_selector='branch='+branch ) }}"