ansible set_fact to the list with jinja2 condition - ansible

I'd like to set a common_apt_packages list based on OS distribution, so I've used jinja2 if condition as the script below, but the return common_apt_packages type is AnsibleUnsafeText
- hosts: localhost
vars:
common_apt_packages_ubuntu_22_04:
- ack-grep
- acl
- apt-transport-https
- build-essential
- dstat
- git-core
- htop
- iftop
- iotop
tasks:
- name: Set common_apt_packages for ubuntu {{ ansible_distribution_version }}
set_fact:
common_apt_packages: "{% if ansible_distribution_version =='22.04' %} {{ common_apt_packages_ubuntu_22_04 }} {% else %} {{ common_apt_packages_ubuntu_18_04 }} {% endif %}"
How can I improve the script to return common_apt_packages as a List variable?

Just remove the white spaces between %} {{ and }} {%, because Ansible will then handle it as a string, not as a list.
With type_debug you can get the output type.
- hosts: localhost
vars:
common_apt_packages_ubuntu_22_04:
- ack-grep
- acl
common_apt_packages_ubuntu_18_04:
- vim
- nano
tasks:
- name: Set common_apt_packages for ubuntu {{ ansible_distribution_version }}
set_fact:
common_apt_packages_without_spaces: "{% if ansible_distribution_version =='22.04' %}{{ common_apt_packages_ubuntu_22_04 }}{% else %}{{ common_apt_packages_ubuntu_18_04 }}{% endif %}"
- name: Set common_apt_packages for ubuntu {{ ansible_distribution_version }}
set_fact:
common_apt_packages_with_spaces: "{% if ansible_distribution_version =='22.04' %} {{ common_apt_packages_ubuntu_22_04 }} {% else %} {{ common_apt_packages_ubuntu_18_04 }} {% endif %}"
- debug:
msg: "{{ common_apt_packages_without_spaces }} ==> {{ common_apt_packages_without_spaces | type_debug }}"
- debug:
msg: "{{ common_apt_packages_with_spaces }} ==> {{ common_apt_packages_with_spaces | type_debug }}"
TASK [Set common_apt_packages for ubuntu 20.04] ************
ok: [localhost]
TASK [debug] ***********************************************
ok: [localhost] => {
"msg": "['vim', 'nano'] ==> list"
}
TASK [debug] ***********************************************
ok: [localhost] => {
"msg": " ['vim', 'nano'] ==> AnsibleUnsafeText"
}
When you try to install them using the apt module:
# without white spaces
"package": [
"vim",
"nano"
]
# with white spaces
"package": [
" ['vim'",
" 'nano'] "
]
"msg": "No package(s) matching '['vim'' available"

Create the lists. For example in group_vars
shell> cat group_vars/all/packages.yml
list_of_packages_for_18_04: [pkg1_18_04, pkg2_18_04, pkg3_18_04]
list_of_packages_for_20_04: [pkg1_20_04, pkg2_20_04, pkg3_20_04]
list_of_packages_for_22_04: [pkg1_22_04, pkg2_22_04, pkg3_22_04]
default_list_of_packages: [pkg1, pkg2, pkg3]
and put the lists of the packages into a dictionary. For example,
- hosts: localhost
vars:
packages:
'18.04': "{{ list_of_packages_for_18_04 }}"
'20.04': "{{ list_of_packages_for_20_04 }}"
'22.04': "{{ list_of_packages_for_22_04 }}"
'default': "{{ default_list_of_packages }}"
my_packages: "{{ packages[ansible_distribution_version]|
default(packages.default) }}"
tasks:
- debug:
var: ansible_distribution_version
- debug:
var: my_packages
gives (abridged)
TASK [debug] ******************************************************
ok: [localhost] =>
ansible_distribution_version: '20.04'
TASK [debug] ******************************************************
ok: [localhost] =>
my_packages:
- pkg1_20_04
- pkg2_20_04
- pkg3_20_04
Your problem is that output of Jinja is always a string. Ansible should convert it automatically if it is a valid YAML. If you for whatever reason have to use Jinja create the string first and convert it to YAML explicitly. For example,
- set_fact:
packages_str: |
{% if ansible_distribution_version == '22.04' %}
{{ common_apt_packages_ubuntu_22_04 }}
{% else %}
{{ common_apt_packages_ubuntu_18_04 }}
{% endif %}"
- set_fact:
packages: "{{ packages_str|from_yaml }}"
You can't put the declarations into a single set_fact because the second declaration knows nothing about the first one. But, you can put them into any vars, of course.
Test of the conversion in Ansible 2.12.9 Python 3.8.5, and Jinja 3.0.1
- hosts: localhost
vars:
packages_str: |
{% if ansible_distribution_version == '20.04' %}
{{ list_of_packages_for_20_04 }}
{% else %}
{{ list_of_packages_for_18_04 }}
{% endif %}
packages: "{{ packages_str|from_yaml }}"
tasks:
- debug:
var: ansible_distribution_version
- debug:
var: packages_str|type_debug
- debug:
var: packages|type_debug
- debug:
var: packages_str
- debug:
var: packages
gives (abridged)
TASK [debug] **************************************************
ok: [localhost] =>
ansible_distribution_version: '20.04'
TASK [debug] **************************************************
ok: [localhost] =>
packages_str|type_debug: list
TASK [debug] **************************************************
ok: [localhost] =>
packages|type_debug: list
TASK [debug] **************************************************
ok: [localhost] =>
packages_str:
- pkg1_20_04
- pkg2_20_04
- pkg3_20_04
TASK [debug] **************************************************
ok: [localhost] =>
packages:
- pkg1_20_04
- pkg2_20_04
- pkg3_20_04

Related

How to prevent Jinja2 substitution in Ansible

I goal is to replace all the {{ and }} with {% raw %}{{ and }}{% endraw %} in all my markdown notes so that my hexo plugin won't attempt to template when rendering.
My code is:
- name: Get file list of post files
find:
paths: "{{ hexo_data_post_file_dir }}"
register: hexo_data_post_file_list
- name: Replace special char
replace:
path: "{{ item[0]['path'] }}"
regexp: "{{ item[1]['pattern'] }}"
replace: "{{ item[1]['string'] }}"
with_nested:
- "{{ hexo_data_post_file_list['files'] }}"
- - pattern: "{{ '{{' }}"
string: "{{ '{% raw %}{{' }}"
- pattern: "{{ '}}' }}"
string: "{{ '}}{% endraw %}' }}"
Despite I used "{{ '{{' }}" to ask jinja not to render {{, I still get error:
TASK [setup_docker_hexo : Replace special char] ********************************************************************************************************************************************
Tuesday 27 December 2022 11:58:56 +0800 (0:00:00.491) 0:00:13.572 ******
fatal: [uranus-debian]: FAILED! => {"msg": "template error while templating string: unexpected 'end of template'. String: {{. unexpected 'end of template'"}
Given the file
shell> cat test.txt.j2
{{ test_var }}
Declare the strings unsafe to block templating. For example,
my_files:
- "{{ playbook_dir }}/test.txt.j2"
my_regex:
- pattern: !unsafe '{{ '
string: !unsafe '{% raw %}{{ {% endraw %}'
- pattern: !unsafe ' }}'
string: !unsafe '{% raw %} }}{% endraw %}'
Test the expansion
- debug:
msg: "{{ item }}"
with_nested:
- "{{ my_files }}"
- "{{ my_regex }}"
gives
TASK [debug] *********************************************************************************
ok: [localhost] => (item=['/export/scratch/tmp7/test-117/test.txt.j2', {'pattern': '{{ ', 'string': '{% raw %}{{ {% endraw %}'}]) =>
msg:
- /export/scratch/tmp7/test-117/test.txt.j2
- pattern: '{{ '
string: '{% raw %}{{ {% endraw %}'
ok: [localhost] => (item=['/export/scratch/tmp7/test-117/test.txt.j2', {'pattern': ' }}', 'string': '{% raw %} }}{% endraw %}'}]) =>
msg:
- /export/scratch/tmp7/test-117/test.txt.j2
- pattern: ' }}'
string: '{% raw %} }}{% endraw %}'
Replace the patterns
- replace:
path: "{{ item.0 }}"
regexp: "{{ item.1.pattern }}"
replace: "{{ item.1.string }}"
with_nested:
- "{{ my_files }}"
- "{{ my_regex }}"
- debug:
msg: "{{ lookup('file', my_files.0) }}"
- debug:
msg: "{{ lookup('template', my_files.0) }}"
gives
TASK [debug] *********************************************************************************
ok: [localhost] =>
msg: '{% raw %}{{ {% endraw %}test_var{% raw %} }}{% endraw %}'
TASK [debug] *********************************************************************************
ok: [localhost] =>
msg: |-
{{ test_var }}
Declare the variable
my_file: "{{ playbook_dir }}/test.txt"
test_var: foo bar
and use the template
- template:
src: "{{ my_files.0 }}"
dest: "{{ my_file }}"
- debug:
msg: "{{ lookup('file', my_file) }}"
- debug:
msg: "{{ lookup('template', my_file) }}"
gives
TASK [debug] *********************************************************************************
ok: [localhost] =>
msg: '{{ test_var }}'
TASK [debug] *********************************************************************************
ok: [localhost] =>
msg: |-
foo bar
Notes
Example of a complete playbook for testing
- hosts: localhost
vars:
my_files:
- "{{ playbook_dir }}/test.txt.j2"
my_regex:
- pattern: !unsafe '{{'
string: !unsafe '{% raw %}{{{% endraw %}'
- pattern: !unsafe '}}'
string: !unsafe '{% raw %}}}{% endraw %}'
my_file: "{{ playbook_dir }}/test.txt"
test_var: foo bar
tasks:
- debug:
msg: "{{ item }}"
with_nested:
- "{{ my_files }}"
- "{{ my_regex }}"
- replace:
path: "{{ item.0 }}"
regexp: "{{ item.1.pattern }}"
replace: "{{ item.1.string }}"
with_nested:
- "{{ my_files }}"
- "{{ my_regex }}"
- debug:
msg: "{{ lookup('file', my_files.0) }}"
- debug:
msg: "{{ lookup('template', my_files.0) }}"
- template:
src: "{{ my_files.0 }}"
dest: "{{ my_file }}"
- debug:
msg: "{{ lookup('file', my_file) }}"
- debug:
msg: "{{ lookup('template', my_file) }}"
The playbook is not idempotent!

Is there any way to loop through Ansible list of dictionary registered variable in combination with Jinja2?

In my inventory I have 3 servers in a group. I want to be able to increase that size in the future so I can add more nodes into the network and generate the template with Jinja2.
- name: Gathering API results
shell:
cmd: "curl {{ groups['nodes'][node_index] }}/whatever/api/result "
loop: "{{ groups['nodes'] }}"
loop_control:
index_var: node_index
register: api_value
If I run some debug tasks hardcoding which list I want to use everyhing works fine
- debug: "msg={{ api_value.results.0.stdout }}"
- debug: "msg={{ api_value.results.1.stdout }}"
- debug: "msg={{ api_value.results.2.stdout }}"
output:
ok: [server-1] => {
"msg": "random-value-a"
ok: [server-2] => {
"msg": "random-value-b"
ok: [server-3] => {
"msg": "random-value-c"
The problem is when I try to increase the list number in Jinja template. I tried several for loops combination, nested for loops and many other things but nothing seems to be working.
For example I want my Jinja template look similar like this:
{% for vm in groups['nodes'] %}
NODE_{{ loop.index }}={{ api_value.results.{loop.index}.stdout }}
{% endfor %}
This way I want to achieve this output:
NODE_0=random-value-a
NODE_1=random-value-b
NODE_2=random-value-c
Is there any other way to workaround this? Or maybe is something I could do better in the "Gathering API results" task?
Given the inventory
shell> cat hosts
[nodes]
server-1
server-2
server-3
Either run it in the loop at a single host, e.g.
- hosts: localhost
gather_facts: false
vars:
whatever_api_result:
server-1: random-value-a
server-2: random-value-b
server-3: random-value-c
tasks:
- command: "echo {{ whatever_api_result[item] }}"
register: api_value
loop: "{{ groups.nodes }}"
- debug:
msg: "{{ api_value.results|json_query('[].[item, stdout]') }}"
gives
msg:
- - server-1
- random-value-a
- - server-2
- random-value-b
- - server-3
- random-value-c
Then, in the Jinja template, fix the index variable
- debug:
msg: |-
{% for vm in groups.nodes %}
NODE_{{ loop.index0 }}={{ api_value.results[loop.index0].stdout }}
{% endfor %}
gives what you want
msg: |-
NODE_0=random-value-a
NODE_1=random-value-b
NODE_2=random-value-c
Optionally, iterate api_value.results. This gives the same result
- debug:
msg: |-
{% for v in api_value.results %}
NODE_{{ loop.index0 }}={{ v.stdout }}
{% endfor %}
Or run it in the group, e.g.
- hosts: nodes
gather_facts: false
vars:
whatever_api_result:
server-1: random-value-a
server-2: random-value-b
server-3: random-value-c
tasks:
- command: "echo {{ whatever_api_result[inventory_hostname] }}"
register: api_value
delegate_to: localhost
- debug:
msg: "{{ api_value.stdout }}"
(delegate to localhost for testing)
gives
ok: [server-1] =>
msg: random-value-a
ok: [server-2] =>
msg: random-value-b
ok: [server-3] =>
msg: random-value-c
Then, in the Jinja template, use hostvars
- debug:
msg: |-
{% for vm in groups.nodes %}
NODE_{{ loop.index0 }}={{ hostvars[vm].api_value.stdout }}
{% endfor %}
run_once: true
gives also what you want
msg: |-
NODE_0=random-value-a
NODE_1=random-value-b
NODE_2=random-value-c
Optionally, iterate hostvars. This gives the same result
- debug:
msg: |-
{% for k,v in hostvars.items() %}
NODE_{{ loop.index0 }}={{ v.api_value.stdout }}
{% endfor %}
run_once: true

How to use the jinja2 output as an ansible directive instead of a chain of characters?

I am currently displaying some vars with 'extract':
- name: Display hostvars over conditional
hosts: all
tasks:
- debug:
msg: "{{
ansible_play_hosts
| map('extract', hostvars, inventory_hostname)
|selectattr('ansible_distribution', 'regex', 'Rocky|CentOS' )
|selectattr('ansible_distribution_major_version', '==', '8' )
|flatten
}}"
The result is a list of name that match to those conditions:
TASK [debug]
ok: [ancolie-lba] => {
"msg": [
"vm703-dev"
]
}
and now I'd like to template this display with jinja2 ansible_facts.j2
{{'{{'}}
ansible_play_hosts
|map('extract', hostvars)
{% for condition in conditions %}
|selectattr('{{condition.attribute}}', '{{condition.verb}}', '{{condition.text}}' )
{% endfor %}
|flatten
{{'}}'}}
So as to use the jinja2 output file like follow:
- block:
- name: generate a conf
template:
src: ansible_facts.j2
dest: /tmp/ansible_facts
- debug:
msg: "{{lookup('file', '/tmp/ansible_facts')}}"
tags: template
delegate_to: localhost
vars:
conditions:
- attribute: 'ansible_distribution'
verb: 'regex'
text: 'Rocky|CentOS'
- attribute: 'ansible_distribution_major_version'
verb: '=='
text: '8'
But it seems that ansible interprets the lookup output as a chain character instead of excuting the filters as initially:
What is returned:
"msg": "{{ansible_play_hosts|map('extract', hostvars, os)|selectattr('ansible_distribution', 'regex', 'Rocky|CentOS' )|selectattr('ansible_distribution_major_version', '==', '8' )|flatten}}"
Expected result:
TASK [debug] *********************************************************************************************************************************************************************************
ok: [ancolie-lba] => {
"msg": [
"vm703-dev"
]
}
How to use the jinja2 output as an ansible directive instead of a chain of characters?
You appear to be trying to produce a Jinja template as the result of your template, then somehow induce Ansible to re-do the templating that it thinks is already complete. This might be technically possible to achieve, but is fighting against Ansible's Jinja integration instead of working with it.
As long as you have at least Jinja 2.10 (for namespace() support) you can do this in a fairly straightforward way with no double-templating. It may also be doable on older versions, but would be more of a pain.
test.j2:
{% set ns = namespace(result=(ansible_play_hosts | map('extract', hostvars))) %}
{% for condition in conditions %}
{% set ns.result = ns.result | selectattr(condition.attribute, condition.verb, condition.text) %}
{% endfor %}
{{ ns.result | map(attribute='inventory_hostname') | to_nice_yaml }}
test.yml:
- hosts: all
tasks:
- debug:
msg: "{{ lookup('template', 'test.j2') | from_yaml }}"
run_once: true
vars:
conditions:
- attribute: ansible_facts.hostname
verb: match
text: pe
Output:
PLAY [all] ************************************************************************
TASK [Gathering Facts] ************************************************************ok: [rotten-rusalka.authz-relay.x.mail.umich.edu]
ok: [persistent-servitor.relay-egress.x.mail.umich.edu]
ok: [bounteous-abumiguchi.mx.x.mail.umich.edu]
ok: [worthy-yithian.vdc-relay.x.mail.umich.edu]
ok: [commutual-chthonian.syslog.x.mail.umich.edu]
ok: [yern-elderthing.authz-static.x.mail.umich.edu]
ok: [diligent-griffin.authn-relay.x.mail.umich.edu]
ok: [dauntless-ratthing.dnsbl.x.mail.umich.edu]
ok: [powerful-yithian.covid-relay.x.mail.umich.edu]
ok: [legible-sansei.egress.x.mail.umich.edu]
ok: [dominant-shantak.dnsbl.x.mail.umich.edu]
ok: [trustworthy-dhole.jail.x.mail.umich.edu]
ok: [rose-dobyoshi.mx.x.mail.umich.edu]
ok: [pear-kaichigo.mx.x.mail.umich.edu]
ok: [pandora.x.mail.umich.edu]
ok: [queenly-kuzunoha.egress.x.mail.umich.edu]
TASK [debug] *********************************************************************$
ok: [worthy-yithian.vdc-relay.x.mail.umich.edu] => {
"msg": [
"persistent-servitor.relay-egress.x.mail.umich.edu",
"pear-kaichigo.mx.x.mail.umich.edu"
]
}

How to get the sum of two defined variables in Jinja2 ansible

I have 04 server packs. Those are downloaded and extracted in a loop. I want to change the port numbers as 4000,4001,4002,4003 and 40004.
I have defined the variables in roles/myrole/vars/main.yml as follow
port: 4000
node:4
item:4
In my roles/myrole/tasks/main.yml, I have defined the tasks
- name: Change axis2 configs
template:
src: ~/myproject/roles/myrole/templates/axis2.xml.j2
dest: ~/myproject/{{ item }}/sever/axis2/axis2.xml
with_sequence: start=0 end={{ node }}
In the axis2 template I have added the variable as;
<parameter name="localMemberPort">{{ port }}</parameter>
{% set port = port + 1 %}
But I when I run the playbook, still the ports are replaced with 4000. How to do this task or is there another way to do this?
The {% set ... %} command only sets variables within the context of the template. It has no effect on subsequent iterations of the task. Since you're iterating using with_sequence, you can just add your loop variable to port like this:
<parameter name="localMemberPort">{{ port|int + item|int }}</parameter>
For example, the following playbook:
- hosts: localhost
gather_facts: false
vars:
node: 4
port: 4000
tasks:
- debug:
msg: >-
<parameter name="localMemberPort">{{ port|int + item|int }}</parameter>
with_sequence: start=0 end={{ node }}
Produces as output:
TASK [debug] *************************************************************************
ok: [localhost] => (item=0) => {
"msg": "<parameter name=\"localMemberPort\">4000</parameter>"
}
ok: [localhost] => (item=1) => {
"msg": "<parameter name=\"localMemberPort\">4001</parameter>"
}
ok: [localhost] => (item=2) => {
"msg": "<parameter name=\"localMemberPort\">4002</parameter>"
}
ok: [localhost] => (item=3) => {
"msg": "<parameter name=\"localMemberPort\">4003</parameter>"
}
ok: [localhost] => (item=4) => {
"msg": "<parameter name=\"localMemberPort\">4004</parameter>"
}
Incidentally, I hope you're not actually setting item in roles/myrole/vars/main.yml: this is the default loop variable name, and attempting to set it in a vars file like this is just going to cause confusion.
Change your roles/myrole/vars/main.yml as follows
port: 4000
node: 4
You don't need to specify item, because item is the current sequence number
And if you would like you can write a direct count in your task
- name: Change axis2 configs
template:
src: ~/myproject/roles/myrole/templates/axis2.xml.j2
dest: ~/myproject/{{ item }}/sever/axis2/axis2.xml
with_sequence: count={{ node }}
Just change your axis2 template to
<parameter name="localMemberPort">{{ port|int + item|int }}</parameter>
You don't need to set a variable in your logic for this purpose.
Hope this helps.
If anyone is interested in looping through a list and using the index value of each list item this what I did:
- name: "Install supervisord template for {{ role }} and notify supervisor of the change"
template:
src: "supervisord.conf.j2"
dest: "{{ supervisor_conf_dir }}/{{ role }}_{{ item.1 }}.conf"
owner: "{{ deploy_user }}"
group: "{{ deploy_user }}"
with_indexed_items:
- "{{ the_endpoints }}"
notify:
- "add_{{ role }}"
- "update_{{ role }}"
tags:
- "additional_templates"
- "supervisor_configs"
debug of this gives output where index.0 is the index value and index.1 is the item from the list:
item=(1, u'm19')
and use it in a template such as:
[program:{{ role }}_{{ item.1 }}]
autorestart = true
autostart = true
command = {{ opskit_dir }}/{{ role }}_{{ item.1 }}/bin/prometheus --web.external-url='https://{{ inventory_hostname }}:4434/{{ deploy_env }}-{{ role }}_{{ item.1 }}' --config.file='{{ opskit_dir }}/{{ role }}_{{ item.1 }}/conf/{{ role }}_{{ item.1 }}.yml' --storage.tsdb.path='{{ deploy_dir }}/data/{{ role }}
_{{ item.1 }}/data' --storage.tsdb.retention='365d' --log.level='debug' --web.listen-address=':{{ prometheus_internal_port|int + item.0|int }}'
directory = {{ opskit_dir }}/{{ role }}_{{ item.1 }}
redirect_stderr = true
stdout_logfile = {{ opskit_dir }}/log/{{ role }}_{{ item.1 }}.log
stdout_logfile_backups = 5
stdout_logfile_maxbytes = 10MB
stopwaitsecs = 300

Ansible template with list of hosts excluding current

I'm super fresh to ansible and creating a playbook that in one of the tasks should copy templated file and replace values in 2 lines. First line should have current hostname, and in second semicolon separated list of all other hosts (used in the play) - it will be different group
First line is super easy, as it's just:
localnode={{ inventory_hostname }}
but I'm having problem with exclusion in the second line. I'd like something similar to:
{% for host in groups.nodes -%} # but without inventory_hostname
othernodes={{ host }}{% if not loop.last %};{% endif %}
{%- endfor %}
Given the inventory of:
nodes:
hosts:
hosta:
hostb:
hostc:
hostd:
I'd like to get following output (example for hostd):
localnode=hostd
othernodes=hosta,hostb,hostc
I'll be very grateful for all hints on possible solution
Create the list of hosts without inventory_hostname and use it in the template
- set_fact:
list_other_hosts: "{{ groups.nodes|difference([inventory_hostname]) }}"
Simplify the template
othernodes={{ list_other_hosts|join(';') }}
As an example, the inventory
shell> cat hosts
test_jails:
hosts:
test_01:
test_02:
test_03:
and the play
- hosts: test_jails
tasks:
- set_fact:
list_other_hosts: "{{ groups.test_jails|
difference([inventory_hostname]) }}"
- debug:
msg: "{{ msg.split('\n') }}"
vars:
msg: |-
localnode={{ inventory_hostname }}
othernodes={{ list_other_hosts|join(';') }}
give
TASK [debug] ********************************************************
ok: [test_01] => {
"msg": [
"localnode=test_01",
"othernodes=test_02;test_03"
]
}
ok: [test_02] => {
"msg": [
"localnode=test_02",
"othernodes=test_01;test_03"
]
}
ok: [test_03] => {
"msg": [
"localnode=test_03",
"othernodes=test_01;test_02"
]
}

Resources