Jinja2 & Ansible - Loop over lines in several text files - ansible

Question

I'm using Ansible to retrieve a list of website we host over 50+ servers. I'm using a SQL query to store in a text file named as hostname_websites.txt using Ansible.
The next step is to build a prometheus (blackbox exporter) configuration file using that data.
Each line of each text file represent an URL. For the sake of representation I'll declare that the URL is $URL and the hostname is $HOSTNAME.
I need some jinja2 magic to generate a single yaml file as :
{% for site in *** each text file *** %}
- targets: ['{{ $URL }}']
labels: {'node': '{{ $HOSTNAME }}'}
{% endfor %}
Given that in the end I need to have a config line containing each time the URL which is a line in text file, and the hostname as a label which is in the txt filename.
Every hostname is in a group called production in Ansible, so I tried looping over that group using jinja2 as follow:
{% for host in group['production'] %}
{{ lookup('file', '{{ host }}_websites.txt).splitlines()' }}
{% endfor %}
That gave me some jinja2 parsing error, as if it wasn't rendering the for loop at all.
I stopped here, in the spirit that I would build by configuration skel around this statement.

Answer 1

There are two problems with your template that jump out immediately:
There is no variable group in Ansible (unless you've created one explicitly); you probably want groups.
You never nest Jinja {{...}} markers. If you want to generate a string containing a variable value, you can use string concatenation:
lookup('file', host ~ '_websites.txt')
Or string formatting:
lookup('file', '%s_websites.txt' % host)
Assuming that our inventory has a host node0 in the group production and that there exists a file node0_websites.txt with the following content:
http://stackoverflow.com/
http://ansible.com/
http://stackexchange.com/
Then running this playbook:
- hosts: localhost
gather_facts: false
tasks:
- copy:
dest: output.txt
content: |
{% for host in groups['production'] %}
# Host: {{ host }}
{% for url in lookup('file', host ~ '_websites.txt').splitlines() %}
- {{ url }}
{% endfor %}
{% endfor %}
Generates the following content in output.txt:
# Host: node0
- http://stackoverflow.com/
- http://ansible.com/
- http://stackexchange.com/