anible loops and subdictionaries - ansible

Question

I'm trying to loop through some parameters in Ansible using a jinja template and a yaml variable file.
The content of the jinja template isn't important here, but I can share a version of the playbook and the yaml variable file below.
What I want to be able to do is generate 4 files, each of them with the name of the peer in the filename. At the moment the playbook works, but generates one file called test-peers. I've tried with_nested and with_subelements - subelements only seems to want to work with a list rather than a dictionary, and I can't find a way to list the values within peers with with_nested. Is there a solution to this?
The YAML file:
customer: test_customer
routers:
router1:
router_number: 01
router_model: ISR4431
peers:
America:
hostname: America
priority: Primary
number: 1
Asia:
hostname: Asia
priority: Backup
number: 2
router2:
router_number: 02
router_model: ISR4431
peers:
America:
hostname: America
priority: Primary
number: 1
Asia:
hostname: Asia
priority: Backup
number: 2
The task in the playbook:
tasks:
- name: WAN gateway testing
template:
src: my.template.j2
dest: "{{ 'config/' + customer + '-' + item.1 }}"
with_nested:
- "{{ routers }}"
- peers

Answer 1

You have to transform your data structure to something you can use the intended way. Here is one possible way to do it in the below playbook:
---
- hosts: localhost
gather_facts: false
vars:
customer: test_customer
# Your orig data on a single json line for compactness
routers: {"router1": {"router_number": 1, "router_model": "ISR4431", "peers": {"America": {"hostname": "America", "priority": "Primary", "number": 1}, "Asia": {"hostname": "Asia", "priority": "Backup", "number": 2}}}, "router2": {"router_number": 2, "router_model": "ISR4431", "peers": {"America": {"hostname": "America", "priority": "Primary", "number": 1}, "Asia": {"hostname": "Asia", "priority": "Backup", "number": 2}}}}
tasks:
- name: Transform our dict to something easier to loop with subelements
vars:
current_router:
router_number: "{{ item.router_number }}"
router_model: "{{ item.router_model }}"
peers: "{{ item.peers | dict2items | map(attribute='value') }}"
set_fact:
routers_transformed: "{{ routers_transformed | default([]) + [current_router] }}"
loop: "{{ routers | dict2items | map(attribute='value') }}"
- name: Show the transformed data
debug:
var: routers_transformed
- name: Create a file name for each router/peer association in the structure as per requirement
debug:
msg: "Filename would be config/{{ customer }}/router{{ item.0.router_number }}/{{ item.1.hostname}}"
loop: "{{ routers_transformed | subelements('peers') }}"
loop_control:
label: "router {{ item.0.router_number }} for peer {{ item.1.hostname }}"
which gives:
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [Transform our dict to something easier to loop with subelements] *****************************************************************************************************************************************************************
ok: [localhost] => (item={'router_number': 1, 'router_model': 'ISR4431', 'peers': {'America': {'hostname': 'America', 'priority': 'Primary', 'number': 1}, 'Asia': {'hostname': 'Asia', 'priority': 'Backup', 'number': 2}}})
ok: [localhost] => (item={'router_number': 2, 'router_model': 'ISR4431', 'peers': {'America': {'hostname': 'America', 'priority': 'Primary', 'number': 1}, 'Asia': {'hostname': 'Asia', 'priority': 'Backup', 'number': 2}}})
TASK [Show the transformed data] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
"routers_transformed": [
{
"peers": [
{
"hostname": "America",
"number": 1,
"priority": "Primary"
},
{
"hostname": "Asia",
"number": 2,
"priority": "Backup"
}
],
"router_model": "ISR4431",
"router_number": "1"
},
{
"peers": [
{
"hostname": "America",
"number": 1,
"priority": "Primary"
},
{
"hostname": "Asia",
"number": 2,
"priority": "Backup"
}
],
"router_model": "ISR4431",
"router_number": "2"
}
]
}
TASK [Create a file name for each router/peer association in the structure as per requirement] *****************************************************************************************************************************************
ok: [localhost] => (item=router 1 for peer America) => {
"msg": "Filename would be config/test_customer/router1/America"
}
ok: [localhost] => (item=router 1 for peer Asia) => {
"msg": "Filename would be config/test_customer/router1/Asia"
}
ok: [localhost] => (item=router 2 for peer America) => {
"msg": "Filename would be config/test_customer/router2/America"
}
ok: [localhost] => (item=router 2 for peer Asia) => {
"msg": "Filename would be config/test_customer/router2/Asia"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Answer 2

Q: "Restructure the original yaml file."
A: Use template. For example
shell> cat templates/routers.j2
customer: {{ customer }}
routers:
{{ routers2|to_nice_yaml|indent(2) }}
The tasks below do the job
- set_fact:
routers2: "{{ routers2|default({})|
combine({item.0.key: item.0.value|
combine({'peers': item.1})}) }}"
loop: "{{ routers|dict2items|zip(peers)|list }}"
vars:
peers: "{{ routers|json_query('*.peers')|
map('dict2items')|
map('json_query', '[].value')|list }}"
- template:
src: routers.j2
dest: routers.yml
shell> cat routers.yml
customer: test_customer
routers:
router1:
peers:
- hostname: America
number: 1
priority: Primary
- hostname: Asia
number: 2
priority: Backup
router_model: ISR4431
router_number: '01'
router2:
peers:
- hostname: America
number: 1
priority: Primary
- hostname: Asia
number: 2
priority: Backup
router_model: ISR4431
router_number: '02'
Then the loop below
- include_vars: routers.yml
- debug:
msg: >-
{{ item.0.router_number }}
{{ item.0.router_model }}
{{ item.1.hostname }}
{{ item.1.number }}
{{ item.1.priority }}
with_subelements:
- "{{ routers }}"
- peers
gives (abridged)
msg: 01 ISR4431 America 1 Primary
msg: 01 ISR4431 Asia 2 Backup
msg: 02 ISR4431 America 1 Primary
msg: 02 ISR4431 Asia 2 Backup