Ansible: Loop over dict and filetree - ansible

Question

How to loop over dict and filetree? I want to recursively template files with .j2 suffix (key) to destination location (value), also basename should be renamed (remove .j2 suffix). Its a perfect use case. Unfortunatelly ansible is not good with complex data structures.
Input:
vars:
applications:
application1:
svcpaths:
localfolder/bardir1: remotefolder/bardir1
localfolder/bardir2: remotefolder/bardir2
localfolder/bardir3: remotefolder/bardir3
application2:
svcpaths:
localfolder/bardir5: remotefolder/bardir5
localfolder/bardir6: remotefolder/bardir6
My try:
- name: Files to template
template:
src: "{{ item.src }}"
dest: "{{ item.destination }}/{{ item.name | regex_replace('.j2','') }}"
loop: |
[
{% for c in applications %}
{% if applications[c]['svcpaths'] is defined and applications[c]['svcpaths'] |list|length >0 %}
{% for o,m in applications[c]['svcpaths'].items() %}
{% for i in lookup('filetree', o ) %}
{% if i.state == 'file' and i.path | regex_search('.\.j2') %}
{
"name": "{{ i.path }}",
"src": "{{ i.src }}",
"destination": "{{ m }}"
},
{% endif %}
{% endfor %}
{% endfor %}
{% endif %}
{% endfor %}
]
I know that using jinja in plays is not good and I want to avoid it, if its possible. Also input datastructure should not be changed.
Thannks

Answer 1

If I understand what you're trying to do, I think there is a reasonably simple solution. If you write a task file like this called template_files.yml:
---
- name: render templates to dest_dir
loop: "{{ query('filetree', src_dir) }}"
# we need this to avoid conflicts with the "item" variable in
# the calling playbook.
loop_control:
loop_var: template
when: template.src.endswith('.j2')
template:
src: "{{ template.src }}"
dest: "{{ dest_dir }}/{{ (template.src|basename)[:-3] }}"
Then you can write a playbook like this:
---
- hosts: localhost
gather_facts: false
vars:
applications:
application1:
svcpaths:
localfolder/bardir1: /tmp/remotefolder/bardir1
localfolder/bardir2: /tmp/remotefolder/bardir2
localfolder/bardir3: /tmp/remotefolder/bardir3
application2:
svcpaths:
localfolder/bardir5: /tmp/remotefolder/bardir5
localfolder/bardir6: /tmp/remotefolder/bardir6
tasks:
# generate a list of {key: src, value: destination}
# dictionaries from your data structure.
- set_fact:
templates: "{{ templates|default([]) + item|dict2items }}"
loop: "{{ applications|json_query('*.svcpaths')}}"
# show what the generated variable looks like
- debug:
var: templates
# template all the things
- include_tasks: template_files.yml
loop: "{{ templates }}"
vars:
src_dir: "{{ item.key }}"
dest_dir: "{{ item.value }}"
Given that I have a set of local files that look like this:
localfolder/bardir1/example.txt.j2
localfolder/bardir2/example.txt.j2
localfolder/bardir3/example.txt.j2
localfolder/bardir5/example.txt.j2
localfolder/bardir6/example.txt.j2
Running the playbook results in:
/tmp/remotefolder/bardir6/example.txt
/tmp/remotefolder/bardir5/example.txt
/tmp/remotefolder/bardir3/example.txt
/tmp/remotefolder/bardir2/example.txt
/tmp/remotefolder/bardir1/example.txt
I think that's probably easier to read and understand than the Jinja-template based solution you're using.