ansible template custom src lookup.first_found - ansible

Question

How to change the value of src to use the first file found or use another path/file if none is found ignoring errors, for example, I want to have a file for a specific host, but default to another file if the first one is not found or skip, I can solve this by duplication the following block with a different src each:
- name: create file unique per host:
template:
src: "{{ ansible_host }}/{{ item }}.conf.j2"
dest: "/etc/wireguard/{{ item }}.conf"
loop: "{{ files }}"
- name: create file:
template:
src: "{{ item }}.conf.j2"
dest: "/etc/wireguard/{{ item }}.conf"
loop: "{{ files }}"
In the role directory, I have inside the templates a directory per host (IP), for example:
roles/
`templates/
| default.conf.j2
`10.1.2.3/
|wg.100.conf.j2
`wg.200.conf.j2
When the task runs against 10.1.2.3 I want to use the files wg.100.conf.j2 & wg.200.conf.j2 but for other hosts use default.conf.j2 or skip if nothing is defined
Any idea how to do it in a single task?
For example, I would like to call the playbook like this:
- roles:
- role: test_role
configs:
- wg.test-1
- wg.test-2
The expected result is to have the files wg.test-1 and wg.test-2 in all hosts:
/etc/wireguard/wg.test-1.conf
/etc/wireguard/wg.test-1.conf
But also include unique files per host, for example having the following structure within the role, host 10.1.2.3 will have the files wg.100 & wg.200 and host 10.1.2.4 the file wg.300
test_role
├── tasks
│ └── main.yml
└── templates
├── 10.1.2.3
│ ├── wg.100.conf.j2
│ └── wg.200.conf.j2
├── 10.1.2.4
│ └── wg.300.conf.j2
│
├── wg.test-2.conf.j2
└── wg.test-1.conf.j2
The reason I want to do all within a task is to simplify the restart of only changed services despite if the src is different and use something like:
- name: create config
template:
src: "{{ ansible_host }}/{{ item }}.conf.j2" || {{ item }}.conf.j2
dest: "/etc/wireguard/{{ item }}.conf"
loop: "{{ configs }}"
notify: restart service
register: changes
- set_fact:
restart_service: "{{ restart_service | default([]) + [item.item] }}"
when: item.changed
loop: "{{ changes.results }}"
no_log: true
This is the current working role that I have, but would like to find a better approach (prevent duplicating blocks)
---
- name: Create wg.file
template:
src: "{{ item }}.conf.j2"
dest: "/etc/wireguard/{{ item }}.conf"
loop: "{{ vpns }}"
notify: restart vpn
register: changes
- set_fact:
restart_service: "{{ restart_service | default([]) + [item.item] }}"
when: item.changed
loop: "{{ changes.results }}"
no_log: true
- name: find per host wg.files
set_fact:
per_host_wg_files: "{{ lookup('fileglob', 'templates/{{ ansible_host }}/*.j2', wantlist=True) }}"
- name: Create per host wg.files
template:
src: "{{ ansible_host }}/{{ item | basename }}"
dest: "/etc/wireguard/{{ (item | basename).split('.')[:3] | join('.') }}"
loop: "{{ per_host_wg_files }}"
notify: restart vpn
register: changes
- name: create fact with changes
set_fact:
restart_service: "{{ restart_service | default([]) + [(item.item | basename).split('.')[:2] | join('.')] }}"
when: item.changed
loop: "{{ changes.results }}"
no_log: true
Just in case this is the restart handler:
- name: restart vpn
systemd:
daemon_reload: true
state: restarted
enabled: true
name: "wg-quick#{{ item }}"
async: 10
poll: 5
loop: "{{ restart_service }}"
when: restart_service is defined

Answer 1

Example 1.
Given the tree for testing
shell> pwd
/scratch/tmp7/test-151
shell> tree .
.
├── ansible.cfg
├── hosts
├── pb.yml
└── roles
└── role_A
├── tasks
│   └── main.yml
└── templates
├── 10.1.2.3
│   ├── test-1.conf.j2
│   └── test-2.conf.j2
├── 10.1.2.4
│   └── test-2.conf.j2
└── default.conf.j2
6 directories, 8 files
shell> cat ansible.cfg
[defaults]
gathering = explicit
inventory = $PWD/hosts
roles_path = $PWD/roles
remote_tmp = ~/.ansible/tmp
retry_files_enabled = false
stdout_callback = yaml
shell> cat hosts
host1 ansible_host=10.1.2.3
host2 ansible_host=10.1.2.4
host3 ansible_host=10.1.2.5
shell> cat pb.yml
- hosts: all
vars:
files: [test-1, test-2, test-3]
findme: "{{ [ansible_host]|
product(files|
product(['.conf.j2'])|
map('join'))|
map('join', '/') + ['default.conf.j2'] }}"
found_file: "{{ lookup('ansible.builtin.first_found', found_params) }}"
found_params:
files: "{{ findme }}"
paths: ['templates']
skip: true
roles:
- role_A
shell> cat roles/role_A/tasks/main.yml
- debug:
var: findme
- debug:
var: found_file
- name: Do it in this single task
debug:
msg: "{{ lookup('template', found_file) }}"
shell> cat roles/role_A/templates/10.1.2.3/test-1.conf.j2
10.1.2.3 test-1
shell> cat roles/role_A/templates/10.1.2.3/test-2.conf.j2
10.1.2.3 test-2
shell cat roles/role_A/templates/10.1.2.4/test-2.conf.j2
10.1.2.4 test-2
shell> cat roles/role_A/templates/default.conf.j2
{{ ansible_host }} default
Create a list of files. Declare the variables
files: [test-1, test-2, test-3]
findme: "{{ [ansible_host]|
product(files|
product(['.conf.j2'])|
map('join'))|
map('join', '/') + ['default.conf.j2'] }}"
gives
TASK [role_A : debug] ************************************************************************
ok: [host3] =>
findme:
- 10.1.2.5/test-1.conf.j2
- 10.1.2.5/test-2.conf.j2
- 10.1.2.5/test-3.conf.j2
- default.conf.j2
ok: [host1] =>
findme:
- 10.1.2.3/test-1.conf.j2
- 10.1.2.3/test-2.conf.j2
- 10.1.2.3/test-3.conf.j2
- default.conf.j2
ok: [host2] =>
findme:
- 10.1.2.4/test-1.conf.j2
- 10.1.2.4/test-2.conf.j2
- 10.1.2.4/test-3.conf.j2
- default.conf.j2
Find the first file available.
The plugin first_found by default doesn't look for files in templates. You must add this subdirectory to the parameter paths when you put files into templates.
The lookup below always succeeds because each list of files is terminated by default.conf.j2.
skip: true means the lookup returns an empty list [] when no files are matched.
found_file: "{{ lookup('ansible.builtin.first_found', found_params) }}"
found_params:
files: "{{ findme }}"
paths: ['templates']
skip: true
gives
TASK [role_A : debug] ************************************************************************
ok: [host2] =>
found_file: /export/scratch/tmp7/test-151/roles/role_A/templates/10.1.2.4/test-2.conf.j2
ok: [host3] =>
found_file: /export/scratch/tmp7/test-151/roles/role_A/templates/default.conf.j2
ok: [host1] =>
found_file: /export/scratch/tmp7/test-151/roles/role_A/templates/10.1.2.3/test-1.conf.j2
Use the file.
- debug:
msg: "{{ lookup('template', found_file) }}"
If you want to allow an empty variable found_file test it
- debug:
msg: "{{ lookup('template', found_file) }}"
when: found_file|length > 0
gives
TASK [role_A : debug] ************************************************************************
ok: [host1] =>
msg: |-
10.1.2.3 test-1
ok: [host2] =>
msg: |-
10.1.2.4 test-2
ok: [host3] =>
msg: |-
10.1.2.5 default
Example 2.
Q: "Given the following structure within the role, the expected result is to have the files wg.test-1 and wg.test-2 in all hosts and also include unique files per host. Host 10.1.2.3 will have the files wg.100 and wg.200, and host 10.1.2.4 the file wg.300."
test_role
├── tasks
│ └── main.yml
└── templates
├── 10.1.2.3
│ ├── wg.100.conf.j2
│ └── wg.200.conf.j2
├── 10.1.2.4
│ └── wg.300.conf.j2
│
├── wg.test-2.conf.j2
└── wg.test-1.conf.j2
A: In this case the plugin first_found is not needed. Use the plugin fileglob instead.
Given the tree for testing
shell> tree .
.
├── ansible.cfg
├── hosts
├── pb.yml
└── roles
└── role_A
├── defaults
│   └── main.yml
├── tasks
│   └── main.yml
└── templates
├── 10.1.2.3
│   ├── wg.100.conf.j2
│   └── wg.200.conf.j2
├── 10.1.2.4
│   └── wg.300.conf.j2
├── wg.test-1.conf.j2
└── wg.test-2.conf.j2
shell> cat pb.yml
- hosts: all
roles:
- role_A
shell> cat roles/role_A/defaults/main.yml
files_global_pattern: templates/*.j2
files_global: "{{ q('ansible.builtin.fileglob', files_global_pattern) }}"
files_local_pattern: "templates/{{ ansible_host }}/*.j2"
files_local: "{{ q('ansible.builtin.fileglob', files_local_pattern) }}"
shell> cat roles/role_A/tasks/main.yml
- debug:
msg: |
files_global:
{{ files_global|to_nice_yaml }}
files_local:
{{ files_local|to_nice_yaml }}
- debug:
msg: |
src: {{ item }}
dest: {{ _dest }}
content: {{ lookup('template', item) }}
loop: "{{ files_global + files_local }}"
vars:
_dest: "{{ item|basename|splitext|first }}"
shell> find roles/role_A/templates -type f | xargs cat
{{ ansible_host }} wg.test-1
{{ ansible_host }} wg.test-2
10.1.2.3 wg.200
10.1.2.3 wg.100
10.1.2.4 wg.300
Declare the lists of global and local files
shell> cat roles/role_A/defaults/main.yml
files_global_pattern: templates/*.j2
files_global: "{{ q('ansible.builtin.fileglob', files_global_pattern) }}"
files_local_pattern: "templates/{{ ansible_host }}/*.j2"
files_local: "{{ q('ansible.builtin.fileglob', files_local_pattern) }}"
gives
TASK [role_A : debug] ************************************************************************
ok: [host1] =>
msg: |-
files_global:
- /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-1.conf.j2
- /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-2.conf.j2
files_local:
- /export/scratch/tmp7/test-152/roles/role_A/templates/10.1.2.3/wg.200.conf.j2
- /export/scratch/tmp7/test-152/roles/role_A/templates/10.1.2.3/wg.100.conf.j2
[WARNING]: Unable to find 'templates/10.1.2.5' in expected paths (use -vvvvv to see paths)
ok: [host3] =>
msg: |-
files_global:
- /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-1.conf.j2
- /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-2.conf.j2
files_local:
[]
ok: [host2] =>
msg: |-
files_global:
- /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-1.conf.j2
- /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-2.conf.j2
files_local:
- /export/scratch/tmp7/test-152/roles/role_A/templates/10.1.2.4/wg.300.conf.j2
Use the lists
- debug:
msg: |
src: {{ item }}
dest: {{ _dest }}
content: {{ lookup('template', item) }}
loop: "{{ files_global + files_local }}"
vars:
_dest: "{{ item|basename|splitext|first }}"
gives
TASK [role_A : debug] ************************************************************************
[WARNING]: Unable to find 'templates/10.1.2.5' in expected paths (use -vvvvv to see paths)
ok: [host1] => (item=/export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-1.conf.j2) =>
msg: |-
src: /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-1.conf.j2
dest: wg.test-1.conf
content: 10.1.2.3 wg.test-1
ok: [host2] => (item=/export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-1.conf.j2) =>
msg: |-
src: /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-1.conf.j2
dest: wg.test-1.conf
content: 10.1.2.4 wg.test-1
ok: [host1] => (item=/export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-2.conf.j2) =>
msg: |-
src: /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-2.conf.j2
dest: wg.test-2.conf
content: 10.1.2.3 wg.test-2
ok: [host2] => (item=/export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-2.conf.j2) =>
msg: |-
src: /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-2.conf.j2
dest: wg.test-2.conf
content: 10.1.2.4 wg.test-2
ok: [host3] => (item=/export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-1.conf.j2) =>
msg: |-
src: /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-1.conf.j2
dest: wg.test-1.conf
content: 10.1.2.5 wg.test-1
ok: [host2] => (item=/export/scratch/tmp7/test-152/roles/role_A/templates/10.1.2.4/wg.300.conf.j2) =>
msg: |-
src: /export/scratch/tmp7/test-152/roles/role_A/templates/10.1.2.4/wg.300.conf.j2
dest: wg.300.conf
content: 10.1.2.4 wg.300
ok: [host1] => (item=/export/scratch/tmp7/test-152/roles/role_A/templates/10.1.2.3/wg.200.conf.j2) =>
msg: |-
src: /export/scratch/tmp7/test-152/roles/role_A/templates/10.1.2.3/wg.200.conf.j2
dest: wg.200.conf
content: 10.1.2.3 wg.200
ok: [host3] => (item=/export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-2.conf.j2) =>
msg: |-
src: /export/scratch/tmp7/test-152/roles/role_A/templates/wg.test-2.conf.j2
dest: wg.test-2.conf
content: 10.1.2.5 wg.test-2
ok: [host1] => (item=/export/scratch/tmp7/test-152/roles/role_A/templates/10.1.2.3/wg.100.conf.j2) =>
msg: |-
src: /export/scratch/tmp7/test-152/roles/role_A/templates/10.1.2.3/wg.100.conf.j2
dest: wg.100.conf
content: 10.1.2.3 wg.100
Q: "How do you prevent the error: 'ansible_host' is undefined?"
A: Use default if you want to allow undefined variable. For example,
files_local_pattern: "templates/{{ ansible_host|d('udefined') }}/*.j2"
The result will be the warning below
[WARNING]: Unable to find 'templates/udefined' in expected paths