Looping over a nested ansible dict - ansible

Question

I have a given vars as a list of kafka topics and possible configurations, like:
kafka_topics:
foo:
retentiontime: 3600
deletepolicy: delete
bar:
retentiontime: 3600
compression: gzip
I have figured multiple ways with dict2items to set them explicit like:
- name: Set RetentionTime for Topics
debug:
msg: "Topic {{ item.key }} get {{ item.value.retentiontime }} for retentiontime "
loop: "{{ lookup('dict', kafka_topics) }}"
when: item.value.retentiontime is defined
But is it possible to get a output like
Topic foo get 3600 for retentiontime
Topic foo get delete for deletepolicy
Topic bar get 3600 for retentiontime
Topic bar get gzip for compression
without defining the value name by name with ansible?
I also tried with
- name: Loop over subelements of the dictionary
debug:
msg: "Key={{ item.0.key }} value={{ item.1 }}"
loop: "{{ lookup('dict', kafka_topics) | list | subelements('value') }}"
which prints Key=bar value={'compression': gzip} but now i'm stuck at seperating those two. Is there a way to extract from item.1 the key and the value?

Answer 1

The simplest option is Jinja. For example
- debug:
msg: |-
{% for k1,v1 in kafka_topics.items() %}
{% for k2,v2 in v1.items() %}
Topic {{ k1 }} get {{ v2 }} for {{ k2 }}
{% endfor %}
{% endfor %}
gives
msg: |-
Topic foo get 3600 for retentiontime
Topic foo get delete for deletepolicy
Topic bar get 3600 for retentiontime
Topic bar get gzip for compression
Create a dictionary of lists first if the iteration is needed. For example the template
shell> cat templates/kafka_facts_list.j2
{% for k1,v1 in kafka_topics.items() %}
{{ k1 }}:
{% for k2,v2 in v1.items() %}
- {key: {{ k2 }}, val: {{ v2 }}}
{% endfor %}
{% endfor %}
- set_fact:
dict2: "{{ lookup('template', 'kafka_facts_list.j2')|from_yaml }}"
- debug:
var: dict2
gives
dict2:
bar:
- key: retentiontime
val: 3600
- key: compression
val: gzip
foo:
- key: retentiontime
val: 3600
- key: deletepolicy
val: delete
Then use subelements
- debug:
msg: "Topic {{ item.0.key }} get {{ item.1.val }} for {{ item.1.key }}"
with_subelements:
- "{{ dict2|dict2items }}"
- value
gives
msg: Topic foo get 3600 for retentiontime
msg: Topic foo get delete for deletepolicy
msg: Topic bar get 3600 for retentiontime
msg: Topic bar get gzip for compression

Answer 2

#Vladimir's answer is indeed the most straightforward one if you need to write some debugging info as above or create a configuration file from a template.
But this might get a little trickier if you actually need to loop over this data in a classic task where you have to pass each value separately to the corresponding moldule's option.
In such cases, here is an alternative that will transform your current dict to list.
The goal is to go from:
"first topic name":
"option 1 name": "option 1 value"
"option 2 name": "option 2 value"
"second topic name": ...
to
- topic_name: "first topic name"
topic_options:
- option_name: "option 1 name"
option_value: "option 1 value"
- option_name: "option 2 name"
option_value: "option 2 value"
- topic_name: "second topic name"
...
This transformed data is then easily loopable with subelements as demonstrated in the following playbook:
---
- hosts: localhost
gather_facts: false
vars:
kafka_topics: {"foo": {"retentiontime": 3600, "deletepolicy": "delete"}, "bar": {"retentiontime": 3600, "compression": "gzip"}}
tasks:
- name: Transform our list to something easier to loop with subelements
vars:
current_topic:
topic_name: "{{ item.topic_name }}"
topic_options: "{{ item.topic_options | dict2items(key_name='option_name', value_name='option_value') }}"
set_fact:
my_topic_list: "{{ my_topic_list | default([]) + [current_topic] }}"
loop: "{{ kafka_topics | dict2items(key_name='topic_name', value_name='topic_options') }}"
- name: Show the transformed var
debug:
var: my_topic_list
- name: Loop over topics and their options
debug:
msg: "Topic `{{ topic.0.topic_name }}` has option `{{ topic.1.option_name }}` with value `{{ topic.1.option_value }}`"
loop: "{{ my_topic_list | subelements('topic_options') }}"
loop_control:
label: "{{ topic.0.topic_name }} - {{ topic.1.option_name }} - {{ topic.1.option_value }}"
loop_var: topic
Which gives:
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [Transform our list to something easier to loop with subelements] *****************************************************************************************************************************************************************
ok: [localhost] => (item={'topic_name': 'foo', 'topic_options': {'retentiontime': 3600, 'deletepolicy': 'delete'}})
ok: [localhost] => (item={'topic_name': 'bar', 'topic_options': {'retentiontime': 3600, 'compression': 'gzip'}})
TASK [Show the transformed var] ********************************************************************************************************************************************************************************************************
ok: [localhost] => {
"my_topic_list": [
{
"topic_name": "foo",
"topic_options": [
{
"option_name": "retentiontime",
"option_value": 3600
},
{
"option_name": "deletepolicy",
"option_value": "delete"
}
]
},
{
"topic_name": "bar",
"topic_options": [
{
"option_name": "retentiontime",
"option_value": 3600
},
{
"option_name": "compression",
"option_value": "gzip"
}
]
}
]
}
TASK [Loop over topics and their options] **********************************************************************************************************************************************************************************************
ok: [localhost] => (item=foo - retentiontime - 3600) => {
"msg": "Topic `foo` has option `retentiontime` with value `3600`"
}
ok: [localhost] => (item=foo - deletepolicy - delete) => {
"msg": "Topic `foo` has option `deletepolicy` with value `delete`"
}
ok: [localhost] => (item=bar - retentiontime - 3600) => {
"msg": "Topic `bar` has option `retentiontime` with value `3600`"
}
ok: [localhost] => (item=bar - compression - gzip) => {
"msg": "Topic `bar` has option `compression` with value `gzip`"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0