Nested templating with folders and files - ansible

Question

There is a variable which contains a name with value foo. Only for this name 'special' templating must be done. The folder name, the filename and content must be templated 2 times, and once without any additions. The remaining files must also be templated, but without the special treatment. Let me describe my issue:
This is my variable:
bar:
- name: something
- name: foo
This is my template directory in Ansible:
templates/foo/another_folder/foo.spec
templates/foo/folder_folder/folder2/some_file/foo.service
templates/foo/some_file.txt
templates/something/ignore.txt
The contents of foo.spec:
name: foo
The goal to have the directory structure on the target machine:
foo/another_folder/foo.spec
foo/folder_folder/folder2/some_file/foo.service
foo/some_file.txt
foo-1/another_folder/foo-1.spec
foo-1/folder_folder/folder2/some_file/foo-1.service
foo-1/some_file.txt
foo-2/another_folder/foo-2.spec
foo-2/folder_folder/folder2/some_file/foo-2.service
foo-2/some_file.txt
something/ignore.txt
The content for each foo.spec should be templated to:
# for foo:
name: foo
--
# for foo-1
name: foo-1
--
# for foo-2
name: foo-2
To solve my problem, I took a look at a similar question, which is with_filetree and loops. However, I simply can't figure out where to start. Is this possible to configure in Ansible?

Answer 1

This is quite an awkward requirement but was kind of fun crafting a solution so here we go.
The below solution takes for granted your templates have the exact same file names as the target files where you deploy them. If you ever want to add the j2 extension to them, see the examples in the filetree documentation to remove it while templating to target.
It is not possible in ansible to imbricate loops in the same task. The solution to this is to loop over an include_tasks and add more loops in the included file.
The basic operation is to template an entire file tree to a target dir so this will be our final included file. In between we just have to detect if we are in the foo situation where we want to loop X time over a range of integers, or in the default one where we only process the directory once.
I used the exact same file tree you introduced in your question for my tests. The only change is the content of the foo/another_folder/foo.spec which is now:
name: {{ spec_name }}
Let's start with the base playbook deploy.yml
---
- name: my bizare templating pattern
hosts: localhost
gather_facts: false
vars:
bar:
- name: something
- name: foo
target_base_dir: /tmp/example/
tasks:
- name: make sure target base dir exists
file:
path: "{{ target_base_dir }}"
state: directory
- name: load template pattern chooser file
include_tasks: "template_pattern_chooser.yml"
loop: "{{ bar }}"
loop_control:
loop_var: template_pattern
This is the included template_pattern_chooser.yml
---
- name: Select the corresponding templating pattern
vars:
find_me:
- "pattern_{{ template_pattern.name }}.yml"
- "pattern_default.yml"
include_tasks: "{{ lookup('first_found', find_me) }}"
As you can see, this will look for either a specific file named after the template pattern name or fallback to a default one.
Thi is the specific pattern_foo.yml. Note that this is where set the var spec_name which is used in the above template.
---
- name: "loop over our {{ template_pattern.name }} pattern"
vars:
target_dir_name: "{{ template_pattern.name }}-{{ pattern_iteration }}"
spec_name: "{{ target_dir_name }}"
include_tasks: "template_tree.yml"
loop: "{{ range(1,4) | list }}"
loop_control:
loop_var: pattern_iteration
pattern_default.yml:
---
- name: default templating pattern
vars:
target_dir_name: "{{ template_pattern.name }}"
include_tasks: template_tree.yml
Note that both files include the same template_tree.yml file. The only change is that we loop over it when we are dealing with the foo pattern. This is where the real job takes place:
---
- name: Get list of templates only once
set_fact:
template_tree: "{{ query('filetree', 'templates/' ~ template_pattern.name ~ '/') }}"
- name: Create needed target dir
file:
path: "{{ target_base_dir }}/{{ target_dir_name }}"
state: "directory"
- name: Create needed directories inside target
file:
path: "{{ target_base_dir }}/{{ target_dir_name }}/{{ item.path }}"
state: "{{ item.state }}"
loop: "{{ template_tree }}"
when: item.state == 'directory'
- name: Deploy templates
template:
src: "{{ item.src }}"
dest: "{{ target_base_dir }}/{{ target_dir_name }}/{{ item.path }}"
loop: "{{ template_tree }}"
when: item.state == 'file'
Running this as a test on my machine gives:
$ ansible-playbook deploy.yml
PLAY [my bizare templating pattern] *****************************************************************************
TASK [make sure target base dir exists] *****************************************************************************
changed: [localhost]
TASK [load template pattern chooser file] *****************************************************************************
included: /home/user/test/template_pattern_chooser.yml for localhost => (item={'name': 'something'})
included: /home/user/test/template_pattern_chooser.yml for localhost => (item={'name': 'foo'})
TASK [Select the corresponding templating pattern] *****************************************************************************
included: /home/user/test/pattern_default.yml for localhost
TASK [default templating pattern] *****************************************************************************
included: /home/user/test/template_tree.yml for localhost
TASK [Get list of templates only once] *****************************************************************************
ok: [localhost]
TASK [Create needed target dir] *****************************************************************************
changed: [localhost]
TASK [Create needed directories inside target] *****************************************************************************
skipping: [localhost] => (item={'root': '/home/user/test/templates/something/', 'path': 'ignore.txt', 'state': 'file', 'src': '/home/user/test/templates/something/ignore.txt', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 6, 'mtime': 1640280116.3296282, 'ctime': 1640280116.3296282})
TASK [Deploy templates] *****************************************************************************
changed: [localhost] => (item={'root': '/home/user/test/templates/something/', 'path': 'ignore.txt', 'state': 'file', 'src': '/home/user/test/templates/something/ignore.txt', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 6, 'mtime': 1640280116.3296282, 'ctime': 1640280116.3296282})
TASK [Select the corresponding templating pattern] *****************************************************************************
included: /home/user/test/pattern_foo.yml for localhost
TASK [loop over our foo pattern] *****************************************************************************
included: /home/user/test/template_tree.yml for localhost => (item=1)
included: /home/user/test/template_tree.yml for localhost => (item=2)
included: /home/user/test/template_tree.yml for localhost => (item=3)
TASK [Get list of templates only once] *****************************************************************************
ok: [localhost]
TASK [Create needed target dir] *****************************************************************************
changed: [localhost]
TASK [Create needed directories inside target] *****************************************************************************
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280024.0969715, 'ctime': 1640280024.0969715})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640279254.875486, 'ctime': 1640279254.875486})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'some_file.txt', 'state': 'file', 'src': '/home/user/test/templates/foo/some_file.txt', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280116.3376281, 'ctime': 1640280116.3376281})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280051.5691671, 'ctime': 1640280051.5691671})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file/foo.service', 'state': 'file', 'src': '/home/user/test/templates/foo/folder_folder/some_file/foo.service', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280051.561167, 'ctime': 1640280051.561167})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder/foo.spec', 'state': 'file', 'src': '/home/user/test/templates/foo/another_folder/foo.spec', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 21, 'mtime': 1640279254.871486, 'ctime': 1640279254.871486})
TASK [Deploy templates] *****************************************************************************
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280024.0969715, 'ctime': 1640280024.0969715})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640279254.875486, 'ctime': 1640279254.875486})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'some_file.txt', 'state': 'file', 'src': '/home/user/test/templates/foo/some_file.txt', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280116.3376281, 'ctime': 1640280116.3376281})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280051.5691671, 'ctime': 1640280051.5691671})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file/foo.service', 'state': 'file', 'src': '/home/user/test/templates/foo/folder_folder/some_file/foo.service', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280051.561167, 'ctime': 1640280051.561167})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder/foo.spec', 'state': 'file', 'src': '/home/user/test/templates/foo/another_folder/foo.spec', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 21, 'mtime': 1640279254.871486, 'ctime': 1640279254.871486})
TASK [Get list of templates only once] *****************************************************************************
ok: [localhost]
TASK [Create needed target dir] *****************************************************************************
changed: [localhost]
TASK [Create needed directories inside target] *****************************************************************************
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280024.0969715, 'ctime': 1640280024.0969715})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640279254.875486, 'ctime': 1640279254.875486})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'some_file.txt', 'state': 'file', 'src': '/home/user/test/templates/foo/some_file.txt', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280116.3376281, 'ctime': 1640280116.3376281})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280051.5691671, 'ctime': 1640280051.5691671})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file/foo.service', 'state': 'file', 'src': '/home/user/test/templates/foo/folder_folder/some_file/foo.service', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280051.561167, 'ctime': 1640280051.561167})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder/foo.spec', 'state': 'file', 'src': '/home/user/test/templates/foo/another_folder/foo.spec', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 21, 'mtime': 1640279254.871486, 'ctime': 1640279254.871486})
TASK [Deploy templates] *****************************************************************************
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280024.0969715, 'ctime': 1640280024.0969715})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640279254.875486, 'ctime': 1640279254.875486})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'some_file.txt', 'state': 'file', 'src': '/home/user/test/templates/foo/some_file.txt', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280116.3376281, 'ctime': 1640280116.3376281})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280051.5691671, 'ctime': 1640280051.5691671})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file/foo.service', 'state': 'file', 'src': '/home/user/test/templates/foo/folder_folder/some_file/foo.service', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280051.561167, 'ctime': 1640280051.561167})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder/foo.spec', 'state': 'file', 'src': '/home/user/test/templates/foo/another_folder/foo.spec', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 21, 'mtime': 1640279254.871486, 'ctime': 1640279254.871486})
TASK [Get list of templates only once] *****************************************************************************
ok: [localhost]
TASK [Create needed target dir] *****************************************************************************
changed: [localhost]
TASK [Create needed directories inside target] *****************************************************************************
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280024.0969715, 'ctime': 1640280024.0969715})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640279254.875486, 'ctime': 1640279254.875486})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'some_file.txt', 'state': 'file', 'src': '/home/user/test/templates/foo/some_file.txt', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280116.3376281, 'ctime': 1640280116.3376281})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280051.5691671, 'ctime': 1640280051.5691671})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file/foo.service', 'state': 'file', 'src': '/home/user/test/templates/foo/folder_folder/some_file/foo.service', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280051.561167, 'ctime': 1640280051.561167})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder/foo.spec', 'state': 'file', 'src': '/home/user/test/templates/foo/another_folder/foo.spec', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 21, 'mtime': 1640279254.871486, 'ctime': 1640279254.871486})
TASK [Deploy templates] *****************************************************************************
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280024.0969715, 'ctime': 1640280024.0969715})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640279254.875486, 'ctime': 1640279254.875486})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'some_file.txt', 'state': 'file', 'src': '/home/user/test/templates/foo/some_file.txt', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280116.3376281, 'ctime': 1640280116.3376281})
skipping: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file', 'state': 'directory', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0755', 'size': 3, 'mtime': 1640280051.5691671, 'ctime': 1640280051.5691671})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'folder_folder/some_file/foo.service', 'state': 'file', 'src': '/home/user/test/templates/foo/folder_folder/some_file/foo.service', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 4, 'mtime': 1640280051.561167, 'ctime': 1640280051.561167})
changed: [localhost] => (item={'root': '/home/user/test/templates/foo/', 'path': 'another_folder/foo.spec', 'state': 'file', 'src': '/home/user/test/templates/foo/another_folder/foo.spec', 'uid': 1000, 'gid': 100, 'owner': 'user', 'group': 'users', 'mode': '0644', 'size': 21, 'mtime': 1640279254.871486, 'ctime': 1640279254.871486})
PLAY RECAP *****************************************************************************
localhost : ok=24 changed=12 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
This is the result in the target dir:
$ tree /tmp/example/
/tmp/example/
├── foo-1
│   ├── another_folder
│   │   └── foo.spec
│   ├── folder_folder
│   │   └── some_file
│   │   └── foo.service
│   └── some_file.txt
├── foo-2
│   ├── another_folder
│   │   └── foo.spec
│   ├── folder_folder
│   │   └── some_file
│   │   └── foo.service
│   └── some_file.txt
├── foo-3
│   ├── another_folder
│   │   └── foo.spec
│   ├── folder_folder
│   │   └── some_file
│   │   └── foo.service
│   └── some_file.txt
└── something
└── ignore.txt
13 directories, 10 files
And as an example the content of one the foo.spec files:
$ cat /tmp/example/foo-2/another_folder/foo.spec
name: foo-2