Ansible iteration over nested dictionary - ansible

With Ansible 2.10, I am trying to iterate over a nested dictionary. The dictionary looks like that:
var1:
d1:
t1:
- p1
- p2
t2:
- p1
d2:
t1:
- p1
I would like the function return something like that:
d1, t1, p1
d1, t1, p2
d1, t2, p1
...
For any help, I will be grateful!

There are other (and possibly more convenient) ways to do this depending on your exact requirement. Changing your data structure might as well be an option.
Meanwhile, here is a solution to store the combinations in a list of 3 element tuples that you can reuse elsewhere.
---
- name: Product for nested dictionnary
hosts: localhost
gather_facts: false
vars:
var1:
d1:
t1:
- p1
- p2
t2:
- p1
d2:
t1:
- p1
tasks:
- name: Level 1 product (d* X t*)
set_fact:
level1: >-
{{
level1 | default([])
+
[item] | product(var1[item] | list)
}}
loop: "{{ var1 | list }}"
- name: Level 2 product (d* X t* X p*)
set_fact:
level2: >-
{{
level2 | default([])
+
[item] | product(var1[item.0][item.1]) | map('flatten')
}}
loop: "{{ level1 }}"
- name: Result
debug:
msg: "{{ item.0 }}, {{ item.1 }}, {{ item.2 }}"
loop: "{{ level2 }}"
Which gives:
PLAY [Product for nested dictionnary] **************************************************************************************************************************************************************************************************
TASK [Level 1 product (d* X t*)] *******************************************************************************************************************************************************************************************************
ok: [localhost] => (item=d1)
ok: [localhost] => (item=d2)
TASK [Level 2 product (d* X t* X p*)] **************************************************************************************************************************************************************************************************
ok: [localhost] => (item=['d1', 't1'])
ok: [localhost] => (item=['d1', 't2'])
ok: [localhost] => (item=['d2', 't1'])
TASK [Result] **************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item=['d1', 't1', 'p1']) => {
"msg": "d1, t1, p1"
}
ok: [localhost] => (item=['d1', 't1', 'p2']) => {
"msg": "d1, t1, p2"
}
ok: [localhost] => (item=['d1', 't2', 'p1']) => {
"msg": "d1, t2, p1"
}
ok: [localhost] => (item=['d2', 't1', 'p1']) => {
"msg": "d2, t1, p1"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

The "Jinja-free" version below
shell> cat next_loop.yml
- set_fact:
_batch: []
- set_fact:
_batch: "{{ _batch + [[ei.key, item.0.key, item.1]] }}"
with_subelements:
- "{{ ei.value|dict2items }}"
- value
- set_fact:
out: "{{ out|default([]) + _batch }}"
- include_tasks: next_loop.yml
loop: "{{ var1|dict2items }}"
loop_control:
loop_var: ei
- debug:
msg: "{{ item|join(',') }}"
loop: "{{ out }}"
gives the same result
msg: d1,t1,p1
msg: d1,t1,p2
msg: d1,t2,p1
msg: d2,t1,p1

Related

Assign values to variables using a loop (set_fact or other?)

Using an Ansible task, I'm trying to create variables and associated values from a returned loop object.
Ref: https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html
- name: Get values
someModule:
input_from_loop: "{{ item }}"
loop:
- "foo"
- "bar"
register: get_result
Gives
get_result:
changed: false
msg: All items completed
results:
- ansible_loop_var: item
item: foo
alpha:
beta:
content: someFooValue
- ansible_loop_var: item
item: bar
alpha:
beta:
content: someBarValue
With this get_result object, I'm trying to create variables and associated values such that:
Pseudocode:
- name: Assign values
set_fact:
"{{ item.item }}":"{{ item.alpha.beta.content }}"
loop: get_result.results
To result in:
foo:someFooValue
bar:someBarValue
I can't seem to get around the error
Implicit map keys need to be followed by map values"
I do not want to create a new object, just variables with values for later use (in existing tasks).
I've tried a few approaches to no avail.
[Or can it happen on each iteration of the initial loop calling the module?]
Now that I am rereading your your requirements, I am wondering if you are not looking for variables at the top level of your host that would be named
foo and contain someFooValue
bar and contain someBarValue
If this is the case, then most of that is above does still apply, the registering of the fact only change:
- set_fact: { "{{ item.0 }}": "{{ item.1 }}" }
loop: "{{
get_result.results | map(attribute='item') | zip(
get_result.results | map(attribute='alpha.beta.content')
)
}}"
Which gives you the two expected variables with the corresponding values.
ok: [localhost] =>
foo: someFooValue
ok: [localhost] =>
bar: someBarValue
What you can do in those kind of cases is to break down the dictionary in multiple lists, all containing one of the field you are interested into, with the map filter, then reconstruct a list of list with the help of the zip filter. And finally, join everything together.
So, with a task like:
- set_fact:
formatted_fact: "{{
get_result.results | map(attribute='item') | zip(
get_result.results | map(attribute='alpha.beta.content')
) | map('join', ':') | join('\n')
}}"
You get your expected output:
formatted_fact: |-
foo:someFooValue
bar:someBarValue
Given those couple of tasks, we have the two possibilities:
- set_fact:
formatted_fact: "{{
get_result.results | map(attribute='item') | zip(
get_result.results | map(attribute='alpha.beta.content')
) | map('join', ':') | join('\n')
}}"
vars:
get_result:
changed: false
msg: All items completed
results:
- ansible_loop_var: item
item: foo
alpha:
beta:
content: someFooValue
- ansible_loop_var: item
item: bar
alpha:
beta:
content: someBarValue
- debug:
var: formatted_fact
- set_fact: { "{{ item.0 }}": "{{ item.1 }}" }
loop: "{{
get_result.results | map(attribute='item') | zip(
get_result.results | map(attribute='alpha.beta.content')
)
}}"
vars:
get_result:
changed: false
msg: All items completed
results:
- ansible_loop_var: item
item: foo
alpha:
beta:
content: someFooValue
- ansible_loop_var: item
item: bar
alpha:
beta:
content: someBarValue
- debug:
var: foo
- debug:
var: bar
They would yield:
TASK [set_fact] **************************************************************
ok: [localhost]
TASK [debug] *****************************************************************
ok: [localhost] =>
formatted_fact: |-
foo:someFooValue
bar:someBarValue
TASK [set_fact] **************************************************************
ok: [localhost] => (item=['foo', 'someFooValue'])
ok: [localhost] => (item=['bar', 'someBarValue'])
TASK [debug] *****************************************************************
ok: [localhost] =>
foo: someFooValue
TASK [debug] *****************************************************************
ok: [localhost] =>
bar: someBarValue
For example
_query: '[].{key: item, value: alpha.beta.content}'
result: "{{ get_result.results|json_query(_query)|items2dict }}"
gives the expected result
result:
bar: someBarValue
foo: someFooValue
Example of a complete playbook (for testing)
- hosts: localhost
vars:
get_result:
changed: false
msg: All items completed
results:
- ansible_loop_var: item
item: foo
alpha:
beta:
content: someFooValue
- ansible_loop_var: item
item: bar
alpha:
beta:
content: someBarValue
_query: '[].{key: item, value: alpha.beta.content}'
result: "{{ get_result.results|json_query(_query)|items2dict }}"
tasks:
- debug:
var: result
Regarding the question in the headline
How to assign values to variables using a loop (set_fact)?
and the mentioned "pseudocode" I like to note that such approach might work, as the following test
---
- hosts: localhost
become: false
gather_facts: false
vars:
varKey: "TEST"
varVal: "testValue"
tasks:
- name: Create variable dynamic
set_fact:
"{{ varKey }}_{{ item }}": "{{ varVal }}_{{ item }}"
loop: [1, 2, 3]
- name: Show variable
debug:
var: TEST_{{ item }}
loop: ["1", "2", "3"]
results into an output of
TASK [Create variable dynamic] *********
ok: [localhost] => (item=1)
ok: [localhost] => (item=2)
ok: [localhost] => (item=3)
TASK [Show variable] *********
ok: [localhost] => (item=1) =>
TEST_1: testValue_1
ansible_loop_var: item
item: '1'
ok: [localhost] => (item=2) =>
TEST_2: testValue_2
ansible_loop_var: item
item: '2'
ok: [localhost] => (item=3) =>
TEST_3: testValue_3
ansible_loop_var: item
item: '3'
and also
- name: Create variable dynamic
set_fact:
"{{ item }}": "{{ item }}"
loop: [A1, A2, A3]
- name: Show variable
debug:
var: "{{ item }}"
loop: [A1, A2, A3]
into
TASK [Create variable dynamic] **********
ok: [localhost] => (item=A1)
ok: [localhost] => (item=A2)
ok: [localhost] => (item=A3)
TASK [Show variable] **********
ok: [localhost] => (item=A1) =>
A1: A1
ansible_loop_var: item
item: A1
ok: [localhost] => (item=A2) =>
A2: A2
ansible_loop_var: item
item: A2
ok: [localhost] => (item=A3) =>
A3: A3
ansible_loop_var: item
item: A3

Iterating list of dictionaries with dictionary

I'm having trouble grasping data manipulation in Ansible. Here's the setup:
- hosts: emirr01
gather_facts: no
vars:
setup:
- emirr01: { label: "label1" }
- emirr02: { label: "label2" }
- emirr03: { label: "label3" }
lookup: [ "emirr01", "emirr02"]
use_labels: [ ]
tasks:
- debug: msg="setup={{ setup }}"
- debug: msg="lookup={{ lookup }}"
- debug: msg="item0={{ item.0 }} item1={{ item.1 }}"
when: inventory_hostname == item.1
with_nested:
- "{{ setup }}"
- "{{ lookup }}"
- set_fact:
use_labels: "{{ use_labels + [ item.1.label ] }}"
when: item.0 == item.1.label
with_nested:
- "{{ setup }}"
- "{{ lookup }}"
- debug: msg="use_labels={{ use_labels }}"
What I need is to set a fact use_labels which is a list of labels as defined in setup list for each host found in lookup list. What I expect is a list like this:
[ "label1", "label2" ]
My problem is not being able to "reach" label in a list I get in item.1 which is (example):
item1=[{'emirr02': {'label': 'label2'}}
Here's is the error:
$ ansible-playbook test.yml -l emirr01
PLAY [emirr01] ****************************************************************************************************************************************************************************************************************************************************************************************************************************
TASK [debug] ******************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [emirr01] =>
msg: 'setup=[{''emirr01'': {''label'': ''label1''}}, {''emirr02'': {''label'': ''label2''}}, {''emirr03'': {''label'': ''label3''}}]'
TASK [debug] ******************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [emirr01] =>
msg: lookup=['emirr01', 'emirr02']
TASK [debug] ******************************************************************************************************************************************************************************************************************************************************************************************************************************
ok: [emirr01] => (item=[{'emirr01': {'label': 'label1'}}, 'emirr01']) =>
msg: 'item0={''emirr01'': {''label'': ''label1''}} item1=emirr01'
skipping: [emirr01] => (item=[{'emirr01': {'label': 'label1'}}, 'emirr02'])
ok: [emirr01] => (item=[{'emirr02': {'label': 'label2'}}, 'emirr01']) =>
msg: 'item0={''emirr02'': {''label'': ''label2''}} item1=emirr01'
skipping: [emirr01] => (item=[{'emirr02': {'label': 'label2'}}, 'emirr02'])
ok: [emirr01] => (item=[{'emirr03': {'label': 'label3'}}, 'emirr01']) =>
msg: 'item0={''emirr03'': {''label'': ''label3''}} item1=emirr01'
skipping: [emirr01] => (item=[{'emirr03': {'label': 'label3'}}, 'emirr02'])
TASK [set_fact] ***************************************************************************************************************************************************************************************************************************************************************************************************************************
fatal: [emirr01]: FAILED! =>
msg: |-
The conditional check 'item.0 == item.1.label' failed. The error was: error while evaluating conditional (item.0 == item.1.label): 'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'label'
The error appears to be in '/home/ansible/test.yml': line 21, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- set_fact:
^ here
It all boils down to "digging" for a value from a dictionary in a list. Does anyone have an idea of how to reach item.1.label? It's a little bit frustrating since this is trivial in Python or other programming languages but in Ansible... Or maybe my mistake is to use a wrong kind of with_* loop for this purpose?
Q: "Expect a list like this [label1, label2]"
setup:
- emirr01: { label: "label1" }
- emirr02: { label: "label2" }
- emirr03: { label: "label3" }
lookup: [ "emirr01", "emirr02"]
A: This is a typical example of a wrong data structure for this use-case. A list of dictionaries can't be used in mapping the filter extract. Moreover, the keys in the dictionaries don't have to be unique, because they are hidden in the items of the list. The solution is simple with the data in the dictionary
setup:
emirr01: { label: "label1" }
emirr02: { label: "label2" }
emirr03: { label: "label3" }
- set_fact:
use_labels: "{{ lookup|map('extract', setup, 'label')|list }}"
gives
use_labels:
- label1
- label2
One of the options is creating a dictionary from the list first. For example
- set_fact:
setup2: "{{ setup2|default({})|combine(item) }}"
loop: "{{ setup }}"
Then the dictionary setup2 can be used to get the same result
It's possible to filter the list directly. For example, the task below gives the same result too
- set_fact:
use_labels: "{{ setup|map('dict2items')|map('first')|
selectattr('key', 'in', lookup )|
map(attribute='value.label')|
list }}"
Your issue with the with_nested is actually coming from how those "two-lists" loops are working.
When you are using loops like with_nested, the item.0 is the item of the first list, while the item.1 is the item of the second list.
Because lookup is a list and not a list of dictionaries like
lookup:
- label: "emirr01"
- label: "emirr02"
You can use the value of item.1 right away.
So the way to have you issue fixed is to use it this way:
- set_fact:
use_labels: "{{ use_labels + [ item.0[item.1].label ] }}"
when: item.0[item.1].label is defined
with_nested:
- "{{ setup }}"
- "{{ lookup }}"
Here is a playbook demonstrating this:
- hosts: localhost
gather_facts: no
tasks:
- set_fact:
use_labels: "{{ use_labels | default([]) + [ item.0[item.1].label ] }}"
when: item.0[item.1].label is defined
with_nested:
- "{{ setup }}"
- "{{ lookup }}"
vars:
setup:
- emirr01:
label: "label1"
- emirr02:
label: "label2"
- emirr03:
label: "label3"
lookup:
- "emirr01"
- "emirr02"
- debug:
msg: "{{ use_labels }}"
Wich gives:
PLAY [localhost] ***************************************************************************************************
TASK [set_fact] ****************************************************************************************************
ok: [localhost] => (item=[{'emirr01': {'label': 'label1'}}, 'emirr01'])
skipping: [localhost] => (item=[{'emirr01': {'label': 'label1'}}, 'emirr02'])
skipping: [localhost] => (item=[{'emirr02': {'label': 'label2'}}, 'emirr01'])
ok: [localhost] => (item=[{'emirr02': {'label': 'label2'}}, 'emirr02'])
skipping: [localhost] => (item=[{'emirr03': {'label': 'label3'}}, 'emirr01'])
skipping: [localhost] => (item=[{'emirr03': {'label': 'label3'}}, 'emirr02'])
TASK [debug] *******************************************************************************************************
ok: [localhost] => {
"msg": [
"label1",
"label2"
]
}
PLAY RECAP *********************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

What is a better way to set a new list variable with cartesian products of several lists in Ansible?

I'm trying to combine data from different variables into a new list. I've used a combination of cartesian products and map/joins filters.
---
- name: Debug
hosts: localhost
gather_facts: no
vars:
uid: 'uid1'
product_types:
- product1
- product2
- product3
data_types:
- {data_name: data1, data_port: 111}
- {data_name: data2, data_port: 222}
- {data_name: data3, data_port: 333}
tasks:
- set_fact:
test_list: "{{ test_list | default([]) + ([item.data_name] | product(product_types) | map('join', '-') | product([uid]) | map('join', '-') | product([item.data_port]) | map('join', '_') | product(['pool']) | map('join', '_') | list) }}"
loop: "{{ data_types }}"
- debug: msg="{{ test_list | to_nice_yaml }}"
The new list shows the correct output but I would like to know if there was a better way for me to assemble the list/data.
$ ansible-playbook playbooks/test.yaml
Vault password:
PLAY [Debug] ************************************************************************************************************************************************************
TASK [debug] ************************************************************************************************************************************************************
ok: [localhost] => (item={'data_name': 'data1', 'data_port': 111}) => {}
$ ansible-playbook playbooks/test.yaml
PLAY [Debug] ************************************************************************************************************************************************************
TASK [set_fact] *********************************************************************************************************************************************************
ok: [localhost] => (item={'data_name': 'data1', 'data_port': 111})
ok: [localhost] => (item={'data_name': 'data2', 'data_port': 222})
ok: [localhost] => (item={'data_name': 'data3', 'data_port': 333})
TASK [debug] ************************************************************************************************************************************************************
ok: [localhost] => {}
MSG:
- data1-product1-uid1_111_pool
- data1-product2-uid1_111_pool
- data1-product3-uid1_111_pool
- data2-product1-uid1_222_pool
- data2-product2-uid1_222_pool
- data2-product3-uid1_222_pool
- data3-product1-uid1_333_pool
- data3-product2-uid1_333_pool
- data3-product3-uid1_333_pool
PLAY RECAP **************************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Q: "A better way for me to assemble the list/data?"
A: No. It's good. Formatting might improve readability
- set_fact:
test_list: "{{ test_list|default([]) +
[item.data_name]|
product(product_types)|map('join', '-')|
product([uid])|map('join', '-')|
product([item.data_port])|map('join', '_')|
product(['pool'])|map('join', '_')|
list }}"
loop: "{{ data_types }}"

How to use comparison operators with dictionary values?

I am trying to write a task that stores the dictionary values in variable based on the condition .
I'm new to this technology. Please anyone help on the below request.
I tried with the below code. Please check below.
- set_fact:
v1: "{{ v1|default([]) + item.keys() if item.values() == false else 1 }}"
loop: "{{ dv }}"
'dv' is a dictionary.
[{1A:True},{2A:True},{3A:False},{4A:False}]
Actually, here I'm trying to store false values in v1 by using comparison operators only.
Expected output:
v1 should contain following list:
[3A,4A]
Ansible Version: 2.5.15
Below works for me:
---
- hosts: localhost
vars:
dv:
1A: 'True'
2A: 'False'
3A: 'True'
4A: 'False'
tasks:
- name: debug
debug:
msg: "{{ item.value }}"
loop: "{{ dv | dict2items }}"
- set_fact:
v1: "{{ v1| default([]) + item.key if (item.value in 'False') else('') }}"
loop: "{{ dv | dict2items }}"
- debug:
var: v1
output -->
TASK [set_fact] *********************************************************************************************************
ok: [localhost] => (item={'key': u'1A', 'value': u'True'})
ok: [localhost] => (item={'key': u'3A', 'value': u'True'})
ok: [localhost] => (item={'key': u'2A', 'value': u'False'})
ok: [localhost] => (item={'key': u'4A', 'value': u'False'})
TASK [debug] ************************************************************************************************************
ok: [localhost] => {
"v1": "2A4A"
}
You can try the below code.
- hosts: localhost
connection: local
vars:
dv: [{1A:True},{2A:True},{3A:False},{4A:False}]
v2: []
v1: []
tasks:
- set_fact:
v1: "{{ v1|default([]) }} + [ {{ v1.append((item.keys()|first).split(':')[0]) if (item.keys()|first).split(':')[1] == 'False' else v2.append('1') }} ]"
with_items: "{{ dv }}"
- debug:
msg: "{{ v1 }}"
Here v2 is a variable declared to direct if the conditions are not met.
Output of the a above code is a below:
ok: [localhost] => {
"msg": [
"3A",
"4A"
]
}

How can I traverse nested lists in Ansible?

I have a list of devices, each of which has a varying list of attributes that must be created on the devices, one at a time. Is there a way to build a nested set of loops to do this? The with_nested construct applies every attribute to every device; I need only a single attribute per device per call.
This playbook demonstrates what I have tried (Ansible 2.7.1, Python 2.7.1):
- name: Test nested list traversal
hosts: localhost
connection: local
vars:
Stuff:
- Name: DeviceA
Info: AInfo
Values:
- ValueA1
- ValueA2
- Name: DeviceB
Info: BInfo
Values:
- ValueB1
- ValueB2
- ValueB3
tasks:
- name: Loop test
debug:
msg: "{{ item.Name }},{{ item.Info }}, {{ item.Values }}"
with_items:
- "{{ Stuff }}"
What I get: (One call per device, containing all attributes at once)
ok: [localhost] => (item={u'Info': u'AInfo', u'Values': [u'ValueA1', u'ValueA2'], u'Name': u'DeviceA'}) =>
msg: DeviceA,AInfo, [u'ValueA1', u'ValueA2']
ok: [localhost] => (item={u'Info': u'BInfo', u'Values': [u'ValueB1', u'ValueB2', u'ValueB3'], u'Name': u'DeviceB'}) =>
msg: DeviceB,BInfo, [u'ValueB1', u'ValueB2', u'ValueB3']
What I want (each msg represents a separate operation to be performed on the device with just that one attribute)
msg: DeviceA, AInfo, ValueA1
msg: DeviceA, AInfo, ValueA2
msg: DeviceB, BInfo, ValueB1
msg: DeviceB, BInfo, ValueB2
msg: DeviceB, BInfo, ValueB3
You can get what you want using the subelements filter:
---
- hosts: localhost
gather_facts: false
vars:
Stuff:
- Name: DeviceA
Info: AInfo
Values:
- ValueA1
- ValueA2
- Name: DeviceB
Info: BInfo
Values:
- ValueB1
- ValueB2
- ValueB3
tasks:
- debug:
msg: "{{ item.0.Name }}, {{ item.0.Info }}, {{ item.1 }}"
loop: "{{ Stuff|subelements('Values') }}"
loop_control:
label: "{{ item.0.Name }}"
Running the above playbook gets you:
PLAY [localhost] ******************************************************************************************************************************************************************************
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => (item=DeviceA) => {
"msg": "DeviceA, AInfo, ValueA1"
}
ok: [localhost] => (item=DeviceA) => {
"msg": "DeviceA, AInfo, ValueA2"
}
ok: [localhost] => (item=DeviceB) => {
"msg": "DeviceB, BInfo, ValueB1"
}
ok: [localhost] => (item=DeviceB) => {
"msg": "DeviceB, BInfo, ValueB2"
}
ok: [localhost] => (item=DeviceB) => {
"msg": "DeviceB, BInfo, ValueB3"
}
PLAY RECAP ************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0

Resources