Generate dictionary by template in inventory - ansible

At the moment I have the following inventory (simplified):
children:
child-1111:
param: some-data-1111
child-1112:
param: some-data-1112
child-1113:
param: some-data-1113
Of course, it's not good when you need a thousand hosts instead of just three and many parameters with substitutions. I would like to write something like this (which doesn't work):
children:
"child-{{item}}":
param: "some-data-{{item}}"
loop:
- 1111
- 1112
- 1113
All the examples I see always produce lists, not dictionaries.
Also, loop seems to work only for tasks, not inventories.
How do I achieve this?

Those kind of requirements tend to be managed with the add_host module.
Given the playbook:
- hosts: localhost
gather_facts: no
tasks:
- add_host:
name: "child-{{ item.name }}"
param: "some-data-{{ item.param }}"
group: dynamic_hosts
loop: "{{ _hosts }}"
vars:
_hosts:
- name: 1111
param: 1111-bis
- name: 1112
param: 1112-bis
- name: 1113
param: 1113-bis
- hosts: dynamic_hosts
gather_facts: no
tasks:
- debug:
msg: "On host `{{ item }}` the value of `param` is `{{ hostvars[item].param }}`"
loop: "{{ ansible_play_hosts }}"
run_once: true
We are constructing a dynamic inventory group named dynamic_hosts, which we can then use in the same playbook.
This would actually yields:
PLAY [localhost] ****************************************************************
TASK [add_host] *****************************************************************
ok: [localhost] => (item={'name': 1111, 'param': '1111-bis'})
ok: [localhost] => (item={'name': 1112, 'param': '1112-bis'})
ok: [localhost] => (item={'name': 1113, 'param': '1113-bis'})
PLAY [dynamic_hosts] ************************************************************
TASK [debug] ********************************************************************
ok: [child-1111] => (item=child-1111) =>
msg: On host `child-1111` the value of `param` is `some-data-1111-bis`
ok: [child-1111] => (item=child-1112) =>
msg: On host `child-1112` the value of `param` is `some-data-1112-bis`
ok: [child-1111] => (item=child-1113) =>
msg: On host `child-1113` the value of `param` is `some-data-1113-bis`

Some forenotes:
Your question is not very precise. My example will only give a general approach which is a bit more scalable than yours but will hit limits very soon.
Your example is not a valid yaml inventory. Moreover it is not clear if you use the name children to declare children groups for the default all or if you want to use this as a group name (which I strongly suggest you don't do).
Having to declare host specific variables depending on the host name inside a static inventory for loads of hosts is generally a sign of either a bad design or that you should switch to a fully dynamic inventory or by managing the hosts in your playbook dynamically through add_host (see #β.εηοιτ.βε's answer)
My example below declares a range of hosts in a group named my_children and takes advantage of the host naming convention to capture their number and reuse it in a variable declared for the group
inv.yml
---
my_children:
vars:
param: "some-data-{{ inventory_hostname.split('-')[1] }}"
hosts:
child-[1111:1113]:
You can see the hosts are parsed correctly:
$ ansible-inventory -i inv.yml --list
{
"_meta": {
"hostvars": {
"child-1111": {
"param": "some-data-{{ inventory_hostname.split('-')[1] }}"
},
"child-1112": {
"param": "some-data-{{ inventory_hostname.split('-')[1] }}"
},
"child-1113": {
"param": "some-data-{{ inventory_hostname.split('-')[1] }}"
}
}
},
"all": {
"children": [
"my_children",
"ungrouped"
]
},
"my_children": {
"hosts": [
"child-1111",
"child-1112",
"child-1113"
]
}
}
And the vars are interpreted correctly:
$ ansible my_children -i inv.yml -m debug -a "msg={{ param }}"
child-1111 | SUCCESS => {
"msg": "some-data-1111"
}
child-1112 | SUCCESS => {
"msg": "some-data-1112"
}
child-1113 | SUCCESS => {
"msg": "some-data-1113"
}

Q: "Need a thousand hosts (.e.g. 'child-1111' param: 'some-data-1111')"
A: Use ranges and constructed plugin. For example
shell> cat inventory/01-hosts
child-[1111:1113]
shell> cat inventory/02-constructed.yml
plugin: constructed
strict: True
groups:
group_children: inventory_hostname.startswith('child-')
compose:
param: "'some-data-' ~ inventory_hostname|split('-')|last"
Test it
shell> ansible-inventory -i inventory --list --yaml
all:
children:
group_children:
hosts:
child-1111:
param: some-data-1111
child-1112:
param: some-data-1112
child-1113:
param: some-data-1113
ungrouped: {}
and use it in a playbook
- hosts: all
tasks:
- debug:
var: param
gives
ok: [child-1111] =>
param: some-data-1111
ok: [child-1112] =>
param: some-data-1112
ok: [child-1113] =>
param: some-data-1113

Related

Unique values from ansible output dict's

I have some servers with a lot of wordpress instances, who I ask them what versions they have.
- name: CONTADOR WP VERSIONES
shell: mycommand
register: wp_versions
- debug: msg: "{{ wp_versions.stdout_lines }}"
For example:
TASK [debug] *********************************************************************
ok: [server1] => {
"msg": [
"5.1.13"
]
}
ok: [server2] => {
"msg": [
"5.1.12",
"5.1.13"
]
}
ok: [server3] => {
"msg": [
"5.1.10",
"5.1.13",
]
}
I need to list a unique values like this:
"msg": [
"5.1.10",
"5.1.12",
"5.1.13",
]
I have tried all that i found but nothing works as I want.
Thanks
Use special variable ansible_play_hosts and extract the variables from the hostvars
- set_fact:
all_vers: "{{ ansible_play_hosts|
map('extract', hostvars, ['wp_versions', 'stdout_lines'])|
flatten|unique }}"
run_once: true
gives
all_vers:
- 5.1.13
- 5.1.12
- 5.1.10
You could do something like this:
- hosts: all
gather_facts: false
tasks:
- name: CONTADOR WP VERSIONES
shell: mycommand
register: wp_versions
- hosts: localhost
gather_facts: false
tasks:
# This tasks builds a flattened list of all the
# wp_versions.stdout_lines values collected from your hosts.
- name: Collect wp_versions information
set_fact:
all_wp_versions_pre: "{{ all_wp_versions_pre + hostvars[item].wp_versions.stdout_lines }}"
loop: "{{ groups.all }}"
vars:
all_wp_versions_pre: []
# Here we use the `unique` filter to produce a list of
# unique versions.
- name: Set all_wp_versions fact
set_fact:
all_wp_versions: "{{ all_wp_versions_pre|unique }}"
- debug:
var: all_wp_versions
Given you examples, this would produce the following output:
TASK [debug] ********************************************************************************************
ok: [localhost] => {
"all_wp_versions": [
"5.1.13",
"5.1.12",
"5.1.10"
]
}

ansible groups.variable in a tasks

I am trying to use a variable I have defined in a group_vars/file.yml and use it in a taks of my role like groups.my_variable (my_variable is the name of the group define in my inventory)
Here is my role tasks:
---
- name: "Create strusted storage pool"
gluster_peer:
state: "{{ CURRENT_NODES.state }}"
nodes: "{{groups.group_name}}"
tags: STORAGE_POOL
my inventory
[gluster]
glusterfs01
glusterfs02
ly-lab-elk
lpic
when I use it like this it works:
---
- name: "Create strusted storage pool"
gluster_peer:
state: "{{ CURRENT_NODES.state }}"
nodes: "{{ groups.gluster }}"
tags: STORAGE_POOL
When I use groups.gluster directly in my tasks/pool_storage.yml as above I have expected result
TASK [debug] ******************************************************************************************************************************************************************************************************
ok: [glusterfs01] => {
"groups.gluster": [
"glusterfs01",
"glusterfs02",
"ly-lab-elk",
"lpic"
]
}
ok: [glusterfs02] => {
"groups.gluster": [
"glusterfs01",
"glusterfs02",
"ly-lab-elk",
"lpic"
]
}
ok: [ly-lab-elk] => {
"groups.gluster": [
"glusterfs01",
"glusterfs02",
"ly-lab-elk",
"lpic"
]
}
ok: [lpic] => {
"groups.gluster": [
"glusterfs01",
"glusterfs02",
"ly-lab-elk",
"lpic"
]
}
But when I use the variable set in group_vars/gluster.yml
group_names: gluster
in my tasks tasks/pool_storage.yml
nodes: "{{groups.group_name}}"
I have this result which is not good
TASK [debug] ******************************************************************************************************************************************************************************************************
ok: [glusterfs01] => {
"group_names": [
"gluster"
]
}
ok: [glusterfs02] => {
"group_names": [
"gluster"
]
}
ok: [ly-lab-elk] => {
"group_names": [
"gluster"
]
}
ok: [lpic] => {
"group_names": [
"gluster"
]
}
As I have different groups in my inventory file, I want to use a variable in which I will define my group name and then append it to groups.group_name or other way but to have the expected result.
Can someone help ?
Thank you in advance.
So, to reference a group_name variable in groups you can try using the groups[group_name] syntax.
An example inventory:
[servers]
host-101.xxx.com
host-202.xxx.com
[group1]
g1-server1
g1-server2
A group_vars/servers.yaml:
group_name: servers
A playbook:
- hosts: servers
connection: local
tasks:
- debug:
var: groups[group_name]
Gives the servers under group1 defined as group_name:
ok: [host-101.xxx.com] => {
"groups[group_name]": [
"host-101.xxx.com",
"host-202.xxx.com"
]
}
ok: [host-202.xxx.com] => {
"groups[group_name]": [
"host-101.xxx.com",
"host-202.xxx.com"
]
}
One more thing to note is that you don't need to establish the peers on each host of GlusterFS cluster. So, either:
target any 1 host of that group
or
add run_once: true to the task
Example:
- name: "Create strusted storage pool"
gluster_peer:
state: "{{ CURRENT_NODES.state }}"
nodes: "{{ groups[group_name] }}"
tags: STORAGE_POOL
run_once: true
Put the variable into the file group_vars/gluster.yml if you want the hosts of the group gluster to use it. The name of the file must be identical to the name of the group. This way Ansible will recognize to which groups the files in the directory group_vars belong.
If you can't change the name of group_vars/file.yml link it to group_vars/gluster.yml.
For example, given the simplified inventory for testing
shell> cat hosts
[gluster]
glusterfs01
glusterfs02
The group_vars
shell> cat group_vars/gluster.yml
my_variable: foo
and the playbook
shell> cat playbook.yml
- hosts: gluster
gather_facts: false
tasks:
- debug:
var: my_variable
give (abridged)
shell> ansible-playbook -i hosts playbook.yml
TASK [debug] *******************************************************
ok: [glusterfs01] =>
my_variable: foo
ok: [glusterfs02] =>
my_variable: foo

ansible flipping inventory still reads in the same order

I am working on a project aimed at populating the IP's of some routers based on East/West locations. The first host will always be the primary and the second will always be the secondary.
Based on the location passed, I flip the inventory. I see the inventory being flipped, but Ansible get the value from the list in the same order.
It doesn't matter what order the inventory list is read. I need for the first host to read the first element e.g. 20.21.22.23 and then the second host to read the second element 28.29.30.31.
Right now, ATL is always the first element and LAX the second.
ok: [ATL_isr_lab] => {
"msg": [
"20.21.22.23",
"24.25.26.27",
"24.25.26.28"
]
}
ok: [LAX_isr_lab] => {
"msg": [
"28.29.30.31",
"32.33.34.35",
"32.33.34.36"
]
}
------------------ Inventory Flipped -------------------------------
ok: [LAX_isr_lab] => {
"msg": [
"28.29.30.31",
"32.33.34.35",
"32.33.34.36"
]
}
ok: [ATL_isr_lab] => {
"msg": [
"20.21.22.23",
"24.25.26.27",
"24.25.26.28"
]
}
---
- hosts: test_hosts
vars:
region: east
_Hub_IP: [ 20.21.22.23, 28.29.30.31]
_Transit_IP: [ 24.25.26.27, 32.33.34.35]
_Neighbor_IP: [24.25.26.28, 32.33.34.36]
_idx: "{{ groups.all.index(inventory_hostname) }}"
#flips inventory if west
order: "{{ (region == 'east')|ternary('reverse_inventory', 'inventory') }}"
become: yes
ignore_unreachable: true
gather_facts: false
tasks:
- name: "Configure Router"
debug:
msg:
- "{{ _Hub_IP[_idx|int] }}"
- "{{ _Transit_IP[_idx|int] }}"
- "{{ _Neighbor_IP[_idx|int] }}"
Well, the issue is not coming with the reverse_inventory and inventory value of the order parameter like you seems to think it is.
The issue is to think that groups.all is indeed reversed when you do use the reverse_inventory value.
Here is an example of this, with the playbook:
- hosts: localhost
gather_facts: no
order: "{{ (region == 'east')|ternary('reverse_inventory', 'inventory') }}"
tasks:
- debug:
var: groups.all
Running it with, with the region as an extra-vars:
ansible-playbook play.yml --inventory inventory.yml --extra-vars "region=east"
Will yield:
ok: [localhost] =>
groups.all:
- LAX_isr_lab
- ATL_isr_lab
ansible-playbook play.yml --inventory inventory.yml --extra-vars "region=west"
Will yield:
ok: [localhost] =>
groups.all:
- LAX_isr_lab
- ATL_isr_lab
Still the sorting works, see:
- hosts: all
gather_facts: no
order: "{{ (region == 'east')|ternary('reverse_inventory', 'inventory') }}"
tasks:
- debug:
Run with:
ansible-playbook play.yml --inventory inventory.yml --extra-vars "region=east"
Will yield
ok: [ATL_isr_lab] =>
msg: Hello world!
ok: [LAX_isr_lab] =>
msg: Hello world!
ansible-playbook play.yml --inventory inventory.yml --extra-vars "region=west"
Will yield
ok: [LAX_isr_lab] =>
msg: Hello world!
ok: [ATL_isr_lab] =>
msg: Hello world!
So, what ends up being wrong is your _idx value.
To fix this, you could use the reverse filter of jinja with the same ternary as you are using in the order parameter, like this:
_idx: "{{ ((region == 'east')|ternary(groups.all|reverse, groups.all)).index(inventory_hostname) }}"
Working playbook:
- hosts: all
gather_facts: no
order: "{{ (region == 'east')|ternary('reverse_inventory', 'inventory') }}"
vars:
_Hub_IP: [20.21.22.23, 28.29.30.31]
_Transit_IP: [24.25.26.27, 32.33.34.35]
_Neighbor_IP: [24.25.26.28, 32.33.34.36]
_idx: "{{ ((region == 'east')|ternary(groups.all|reverse, groups.all)).index(inventory_hostname) }}"
tasks:
- debug:
msg:
- "{{ _Hub_IP[_idx|int] }}"
- "{{ _Transit_IP[_idx|int] }}"
- "{{ _Neighbor_IP[_idx|int] }}"
Running examples:
ansible-playbook play.yml --inventory inventory.yml --extra-vars "region=east"
Will yield:
ok: [ATL_isr_lab] =>
msg:
- 20.21.22.23
- 24.25.26.27
- 24.25.26.28
ok: [LAX_isr_lab] =>
msg:
- 28.29.30.31
- 32.33.34.35
- 32.33.34.36
ansible-playbook play.yml --inventory inventory.yml --extra-vars "region=west"
Will yield:
ok: [LAX_isr_lab] =>
msg:
- 20.21.22.23
- 24.25.26.27
- 24.25.26.28
ok: [ATL_isr_lab] =>
msg:
- 28.29.30.31
- 32.33.34.35
- 32.33.34.36
Got it working as posted originally. I had to upgrade to ansible version 2.11.6. I'm running Debian 10 and apt-get update/apt-get upgrade did not find a newer version.
My solution involved deleting the version and installing it again through pip. After that, I ran the code and it worked flawlessly.

Why is ansible creating a new host scoped list variables for each host?

I need to create a variable of ip addresses that are derived from the run state of all hosts. I expect to insert a new string value and have it appended to the list.
When I run the following ansible-playbook, it creates what looks to be a new list instance for each host instead of modifying the playbook level vars.
My understanding is the set_fact below should concatenate the lists and assign them back to the play scoped var ip_list_from_terraform. Why am I getting host scoped results?
---
- name: Bootstrap nodes
hosts: all
become: true
gather_facts: true
vars:
ip_list_from_terraform: ['Verify']
tasks:
- name: assign a list of all the physical network private IPs
set_fact:
ip_list_from_terraform: "{{ ip_list_from_terraform + [ item ] }}"
with_items: " {{ hostvars[inventory_hostname]['ansible_' + ansible_default_ipv4['interface']]['ipv4']['address'] }} "
register: ip_list_from_terraform_list
- name: Debug global var
debug:
var: ip_list_from_terraform
- name: Debug register result
debug:
var: ip_list_from_terraform_list
Expected:
ok: [shrimp-master-0] => {
"ip_list_from_terraform": [
"Verify",
"10.0.2.41",
"10.0.2.172",
"10.0.2.33",
"10.0.2.215",
"10.0.2.131",
"10.0.2.168",
"10.0.2.118"
]
}
Actual:
TASK [Debug global var] ********************************************************************************************************************************************************************************************************************
ok: [shrimp-master-0] => {
"ip_list_from_terraform": [
"Verify",
"10.0.2.12"
]
}
ok: [shrimp-master-1] => {
"ip_list_from_terraform": [
"Verify",
"10.0.2.33"
]
}
ok: [shrimp-master-2] => {
"ip_list_from_terraform": [
"Verify",
"10.0.2.215"
]
}
ok: [shrimp-worker-0-super-wallaby] => {
"ip_list_from_terraform": [
"Verify",
"10.0.2.131"
]
}
ok: [shrimp-gpu-worker-0-settled-wombat] => {
"ip_list_from_terraform": [
"Verify",
"10.0.2.151"
]
}
Let's simplify the case. Given the inventory
shell> cat hosts
host1 test_ip=10.0.2.41
host2 test_ip=10.0.2.172
host3 test_ip=10.0.2.33
the playbook
- hosts: host1,host2,host3
vars:
ip_list_from_terraform: ['Verify']
tasks:
- set_fact:
ip_list_from_terraform: "{{ ip_list_from_terraform + [ item ] }}"
with_items: "{{ hostvars[inventory_hostname]['test_ip'] }}"
- debug:
var: ip_list_from_terraform
gives
ok: [host2] =>
ip_list_from_terraform:
- Verify
- 10.0.2.172
ok: [host1] =>
ip_list_from_terraform:
- Verify
- 10.0.2.41
ok: [host3] =>
ip_list_from_terraform:
- Verify
- 10.0.2.33
As a side-note, with_items is needed because the argument is a string. loop would crash with the error 'Invalid data passed to ''loop'', it requires a list,.
Q: "set_fact should concatenate the lists"
A: Run once and loop ansible_play_hosts. For example
- set_fact:
ip_list_from_terraform: "{{ ip_list_from_terraform +
[hostvars[item]['test_ip']] }}"
loop: "{{ ansible_play_hosts }}"
run_once: true
gives
ip_list_from_terraform:
- Verify
- 10.0.2.41
- 10.0.2.172
- 10.0.2.33

How to pass extra variables to ansible as list of dictionary to ansible playbook by --extra-vars?

I want to pass variables to my ansible playbook by --extra-vars, and the type of variables is a list of dict like this:
list1:
- { key1: "val1", key2: "val2" }
- { key1: "val3", key2: "val4" }
My playbook is :
---
- name: main file
gather_facts: false
hosts: localhost
vars:
list1: "{{ lists }}"
tasks:
- name: echo item
shell: echo {{ item.key1 }}
with_items: list1
and I try to pass variables like this:
ansible-playbook build_and_cppcheck.yml -e "lists=[{ "key1": "val1", "key2":"val2" },{ "key1": "val3", "key2":"val4" }]"
But it doesn't work:
fatal: [localhost] => with_items expects a list or a set
Is there any advise?
Just use JSON string syntax: Ansible doc. For example:
$ play.yml
---
- hosts: localhost
gather_facts: no
tasks:
- debug:
msg: "This is {{ test[0] }}"
- debug:
msg: "This is {{ test[1] }}"
$ ansible-playbook play.yml -e '{"test":["1.23.45", "12.12.12"]}'
[3sky#t410 testing]$ ansible-playbook play.yml -e '{"test":["1.23.45", "12.12.12"]}'
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
PLAY [localhost] ********************************************************************************
TASK [debug] ********************************************************************************
ok: [localhost] => {
"msg": "This is 1.23.45"
}
TASK [debug] ********************************************************************************
ok: [localhost] => {
"msg": "This is 12.12.12"
}
PLAY RECAP ********************************************************************************
localhost

Resources