For lack of a better title, I have this dummy variable, this task and this j2 template:
# variable
my_list:
- filename: something.conf
content: hello
- filename: else.conf
content: asdf
# conf.j2
{% for item in my_list %}
{{ item.content }}
{% endfor %}
# task
- name: set config
template:
src: conf.j2
dest: "/tmp/{{ item.filename }}"
loop: "{{ my_list }}"
This results in having 2 files on /tmp, something.conf and else.conf, having the same content.
Q: How can I configure Ansible so that per filename, it only 'pastes/templates' the content for that filename. Note that the items in my_list will grow.
Expected output on target vm:
# cat /tmp/something.conf
hello
# cat /tmp/else.conf
asdf
Do I have to create an if clause in the template, or edit the loop in a certain way?
Related
I want to generate a list of dictionaries through the loop. I came across that we can use with_sequence to generate the integer sequences in ansible. I am getting the following error:
The offending line appears to be:
- { "dir": "{{ mat_cleaner_input_file_path_flow }}/{{ item }}" }
with_sequence: start=0 end={{ http_range }}
^ here
We could be wrong, but this one looks like it might be an issue with
missing quotes. Always quote template expression brackets when they
start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
My config.j2:
{{ cleaner_config | to_nice_yaml(indent=2) }}
My task.yml:
---
- name: Test dictionaries playbook
hosts: localhost
connection: local
vars:
mat_cleaner_input_file_path_flow: "/var/opt/miq/sftp/edr-flow"
mat_cleaner_input_file_path_http: "/var/opt/miq/sftp/edr-http"
mat_cleaner_input_retention_period: 21600
http_range: 5
cleaner_config:
- { "dir": "{{ mat_cleaner_input_file_path_flow }}/{{ item }}" }
with_sequence: start=0 end={{ http_range }}
tasks:
- name: Set Cleaner Config
template:
src: /home/osboxes/CleanerConfig/cleaner.config.j2
dest: /home/osboxes/CleanerConfig/output
delegate_to: localhost
with_items: cleaner_config
I want the final output file to be:
dir: /var/opt/miq/sftp/edr-flow/0
dir: /var/opt/miq/sftp/edr-flow/1
dir: /var/opt/miq/sftp/edr-flow/2
The format of the output file you want is text. It's not a list. You can create it by Jinja, e.g.
mat_cleaner_input_file_path_flow: "/var/opt/miq/sftp/edr-flow"
http_range: 3
cleaner_config: |
{% for i in range(http_range) %}
{{ mat_cleaner_input_file_path_flow }}/{{ i }}
{% endfor %}
gives
cleaner_config: |-
/var/opt/miq/sftp/edr-flow/0
/var/opt/miq/sftp/edr-flow/1
/var/opt/miq/sftp/edr-flow/2
Then, because it's not a list, you don't have to use to_nice_yaml in the template. Use indent instead if you want to indent the lines, e.g.
shell> cat cleaner.config.j2
{{ cleaner_config|indent(2) }}
- template:
src: cleaner.config.j2
dest: output
gives
shell> cat output
/var/opt/miq/sftp/edr-flow/0
/var/opt/miq/sftp/edr-flow/1
/var/opt/miq/sftp/edr-flow/2
Under normal circumstances, you would simplify the code, omit the cleaner_config variable, and put the Jinja iteration into the template, e.g.
mat_cleaner_input_file_path_flow: "/var/opt/miq/sftp/edr-flow"
http_range: 3
shell> cat cleaner.config.j2
{% for i in range(http_range) %}
{{ mat_cleaner_input_file_path_flow }}/{{ i }}
{% endfor %}
If you want to use a list
mat_cleaner_input_file_path_flow: "/var/opt/miq/sftp/edr-flow"
http_range: 3
cleaner_config_text: |
{% for i in range(http_range) %}
- {{ mat_cleaner_input_file_path_flow }}/{{ i }}
{% endfor %}
cleaner_config: "{{ cleaner_config_text|from_yaml }}
gives
cleaner_config:
- /var/opt/miq/sftp/edr-flow/0
- /var/opt/miq/sftp/edr-flow/1
- /var/opt/miq/sftp/edr-flow/2
Now you need to_nice_yaml to format the list, e.g.
shell> cat cleaner.config.j2
{{ cleaner_config|to_nice_yaml|indent(2) }}
gives
shell> cat output
- /var/opt/miq/sftp/edr-flow/0
- /var/opt/miq/sftp/edr-flow/1
- /var/opt/miq/sftp/edr-flow/2
I have an vendor Ansible playbook given to us and I will need add an new line to the j2 template and tweak the playbook for our env,
I will need to edit the template with an line -> retention_days: {{ xyz }}
This is how the orginal template looks like:
#cat cluster.j2
apiVersion: v1
metadata:
name: cluster
cluster_name: {{ my_name }}
data:
new_image: |+
baseImage: {{ FROM_repo }}
And here is my Ansible playbook to add the line.
---
- name: mydata
hosts: localhost
tasks:
- name: edit files
lineinfile:
dest: cluster.j2
line: " retention_days: {{ xyz }}"
insertafter: 'new_image'
My end result ie; my j2 template file should have the exact string like this
retention_days: {{ xyz }}
final - file should look like this ->
#cat cluster.j2
apiVersion: v1
metadata:
name: cluster
cluster_name: {{ my_name }}
data:
new_image: |+
retention_days: {{ xyz }}
baseImage: {{ FROM_repo }}
I don't want the {{ xyz }} to be treated as an variable by Ansible instead consider it a string and add them there... How can I escape the {{ and }} Please let me know .
Now ., I get an error: xyz is undefined..
MSG:
***The task includes an option with an undefined variable. The error was: 'xyz' is undefined***
As specified in the documentation, you can use {% raw %} to escape elements in a block, or add additional curly braces.
For example:
---
- name: mydata
hosts: localhost
tasks:
- name: edit files
lineinfile:
dest: cluster.j2
line: " retention_days: {% raw %}{{ xyz }}{% endraw %}"
# or
# line: " retention_days: {{ '{{ xyz }}' }}"
insertafter: 'new_image'
How to loop over dict and filetree? I want to recursively template files with .j2 suffix (key) to destination location (value), also basename should be renamed (remove .j2 suffix). Its a perfect use case. Unfortunatelly ansible is not good with complex data structures.
Input:
vars:
applications:
application1:
svcpaths:
localfolder/bardir1: remotefolder/bardir1
localfolder/bardir2: remotefolder/bardir2
localfolder/bardir3: remotefolder/bardir3
application2:
svcpaths:
localfolder/bardir5: remotefolder/bardir5
localfolder/bardir6: remotefolder/bardir6
My try:
- name: Files to template
template:
src: "{{ item.src }}"
dest: "{{ item.destination }}/{{ item.name | regex_replace('.j2','') }}"
loop: |
[
{% for c in applications %}
{% if applications[c]['svcpaths'] is defined and applications[c]['svcpaths'] |list|length >0 %}
{% for o,m in applications[c]['svcpaths'].items() %}
{% for i in lookup('filetree', o ) %}
{% if i.state == 'file' and i.path | regex_search('.\.j2') %}
{
"name": "{{ i.path }}",
"src": "{{ i.src }}",
"destination": "{{ m }}"
},
{% endif %}
{% endfor %}
{% endfor %}
{% endif %}
{% endfor %}
]
I know that using jinja in plays is not good and I want to avoid it, if its possible. Also input datastructure should not be changed.
Thannks
If I understand what you're trying to do, I think there is a reasonably simple solution. If you write a task file like this called template_files.yml:
---
- name: render templates to dest_dir
loop: "{{ query('filetree', src_dir) }}"
# we need this to avoid conflicts with the "item" variable in
# the calling playbook.
loop_control:
loop_var: template
when: template.src.endswith('.j2')
template:
src: "{{ template.src }}"
dest: "{{ dest_dir }}/{{ (template.src|basename)[:-3] }}"
Then you can write a playbook like this:
---
- hosts: localhost
gather_facts: false
vars:
applications:
application1:
svcpaths:
localfolder/bardir1: /tmp/remotefolder/bardir1
localfolder/bardir2: /tmp/remotefolder/bardir2
localfolder/bardir3: /tmp/remotefolder/bardir3
application2:
svcpaths:
localfolder/bardir5: /tmp/remotefolder/bardir5
localfolder/bardir6: /tmp/remotefolder/bardir6
tasks:
# generate a list of {key: src, value: destination}
# dictionaries from your data structure.
- set_fact:
templates: "{{ templates|default([]) + item|dict2items }}"
loop: "{{ applications|json_query('*.svcpaths')}}"
# show what the generated variable looks like
- debug:
var: templates
# template all the things
- include_tasks: template_files.yml
loop: "{{ templates }}"
vars:
src_dir: "{{ item.key }}"
dest_dir: "{{ item.value }}"
Given that I have a set of local files that look like this:
localfolder/bardir1/example.txt.j2
localfolder/bardir2/example.txt.j2
localfolder/bardir3/example.txt.j2
localfolder/bardir5/example.txt.j2
localfolder/bardir6/example.txt.j2
Running the playbook results in:
/tmp/remotefolder/bardir6/example.txt
/tmp/remotefolder/bardir5/example.txt
/tmp/remotefolder/bardir3/example.txt
/tmp/remotefolder/bardir2/example.txt
/tmp/remotefolder/bardir1/example.txt
I think that's probably easier to read and understand than the Jinja-template based solution you're using.
So I have an ansible playbook that is using a Jinja2 template to create a log file. Everytime I run the playbook it is pulling in customer information from customers.yml and outputting the completed template into a 'stunnel.conf' file. The template works fine but I am trying to find a way to append the previous 'stunnel.conf' rather than overwriting it using the Template module. I wish to add text to the beginning of the 'stunnel.conf' manually and not have it overwritten. Do you think this would be possible?
Stunnel.conf
; GFAM - PBSTP
[customer-GFAM-34074]
cert = /etc/stunnel/stunnel.pem
accept = 34094
connect = 35094
; GUANFABANK - FXSIM
[customer-GUANFABANK-34051]
cert = /etc/stunnel/stunnel.pem
accept = 34095
connect = 35095
; ONEZERO2 - TRADESTREAM
[customer-ONEZERO2-39124]
cert = /etc/stunnel/stunnel.pem
accept = 34096
connect = 35096
; BTG-VELOCITY - PBSTP
[customer-BTG-VELOCITY-42533]
cert = /etc/stunnel/stunnel.pem
accept = 34097
connect = 35097
Jinja2 Template
{#CONTEXT: {{ customers }}#}
{% set currentport = 34093%}
{% for cust, config in customers.items() %}
; {{ cust }} - {{ config['type'] }}
[customer-{{ cust }}-{{ config['accept'] }}]
cert = {{ "/etc/stunnel/stunnel.pem" }}
{#accept = {{ config['accept'] }}#}
{#connect = {{ config['connect'] }}#}
accept = {{ currentport + 1 }}
connect = {{ currentport + 1001 }}
{% set currentport = currentport + 1 %}
{% endfor %}
playbook.yml
- include_vars:
file: /home/vagrant/stunnelSimAnsPractice/roles/ns16/vars/customers.yml
name: customers
- template:
src: /home/vagrant/stunnelSimAnsPractice/roles/ns16/templates/stunnel.j2
dest: /home/vagrant/stunnelSimAnsPractice/roles/ns16/output/stunnel.conf
owner: root
group: root
You can use blockinfile module and template lookup to manage per-client blocks in your stunnel.conf:
- include_vars:
file: customers.yml
name: customers
- blockinfile:
dest: stunnel.conf
block: "{{ lookup('template', 'stunnel.j2') }}"
marker: "; {mark} ANSIBLE MANAGED BLOCK FOR {{ cust }}"
I've shortened file paths for readability.
This way Ansible will look for managed block for specific client ({{ cust }} variable) and add/replace with content from templated stunnel.j2.
I would like suggest to do it like this:
Save output of template to temporary file.
Append Stunnel.conf file with content of temporary file.
Delete temporary file.
In playbook it could look like:
- include_vars:
file: /home/vagrant/stunnelSimAnsPractice/roles/ns16/vars/customers.yml
name: customers
- template:
src: /home/vagrant/stunnelSimAnsPractice/roles/ns16/templates/stunnel.j2
dest: /home/vagrant/stunnelSimAnsPractice/roles/ns16/output/temp.conf
owner: root
group: root
- name: "Append stunnel.conf with content of temporary file"
shell: cat temp.conf >> stunnel.conf
args:
chdir: "/home/vagrant/stunnelSimAnsPractice/roles/ns16/output"
- name: "Delete temporary file"
file:
path: /home/vagrant/stunnelSimAnsPractice/roles/ns16/output/temp.conf
state: absent
My use case is the following :
I have a template file, and I would like to create 2 different files from that template, with the variables being filled by a different set of variables for each file.
For example, lets say I want to template the file containing the line:
mkdir -p {{myTemplateVariable}}
I would like to find a proper way to get this variable filled by "File1" and "File2". Something like :
- name: template test 1
template:
src=myTemplateFile
dest=result1
- name: template test 2
template:
src=myTemplateFile
dest=result2
where I could specify for the first templating that the variable to use is a = "File1" and for the second, b = "File2".
With Ansible 2.x you can use vars: with tasks.
Template test.j2:
mkdir -p {{myTemplateVariable}}
Playbook:
- template: src=test.j2 dest=/tmp/File1
vars:
myTemplateVariable: myDirName
- template: src=test.j2 dest=/tmp/File2
vars:
myTemplateVariable: myOtherDir
This will pass different myTemplateVariable values into test.j2.
For Ansible 2.x:
- name: template test
template:
src: myTemplateFile
dest: result1
vars:
myTemplateVariable: File1
- name: template test
template:
src: myTemplateFile
dest: result2
vars:
myTemplateVariable: File2
For Ansible 1.x:
Unfortunately the template module does not support passing variables to it, which can be used inside the template. There was a feature request but it was rejected.
I can think of two workarounds:
1. Include
The include statement supports passing variables. So you could have your template task inside an extra file and include it twice with appropriate parameters:
my_include.yml:
- name: template test
template:
src=myTemplateFile
dest=destination
main.yml:
- include: my_include.yml destination=result1 myTemplateVariable=File1
- include: my_include.yml destination=result2 myTemplateVariable=File2
2. Re-define myTemplateVariable
Another way would be to simply re-define myTemplateVariable right before every template task.
- set_fact:
myTemplateVariable: File1
- name: template test 1
template:
src=myTemplateFile
dest=result1
- set_fact:
myTemplateVariable: File2
- name: template test 2
template:
src=myTemplateFile
dest=result2
You can do this very easy, look my Supervisor recipe:
- name: Setup Supervisor jobs files
template:
src: job.conf.j2
dest: "/etc/supervisor/conf.d/{{ item.job }}.conf"
owner: root
group: root
force: yes
mode: 0644
with_items:
- { job: bender, arguments: "-m 64", instances: 3 }
- { job: mailer, arguments: "-m 1024", instances: 2 }
notify: Ensure Supervisor is restarted
job.conf.j2:
[program:{{ item.job }}]
user=vagrant
command=/usr/share/nginx/vhosts/parclick.com/app/console rabbitmq:consumer {{ item.arguments }} {{ item.job }} -e prod
process_name=%(program_name)s_%(process_num)02d
numprocs={{ item.instances }}
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor/{{ item.job }}.stderr.log
stdout_logfile=/var/log/supervisor/{{ item.job }}.stdout.log
Output:
TASK [Supervisor : Setup Supervisor jobs files] ********************************
changed: [loc.parclick.com] => (item={u'instances': 3, u'job': u'bender', u'arguments': u'-m 64'})
changed: [loc.parclick.com] => (item={u'instances': 2, u'job': u'mailer', u'arguments': u'-m 1024'})
Enjoy!
This is a solution/hack I'm using:
tasks/main.yml:
- name: parametrized template - a
template:
src: test.j2
dest: /tmp/templateA
with_items: var_a
- name: parametrized template - b
template:
src: test.j2
dest: /tmp/templateB
with_items: var_b
vars/main.yml
var_a:
- 'this is var_a'
var_b:
- 'this is var_b'
templates/test.j2:
{{ item }}
After running this, you get this is var_a in /tmp/templateA and this is var_b in /tmp/templateB.
Basically you abuse with_items to render the template with each item in the one-item list. This works because you can control what the list is when using with_items.
The downside of this is that you have to use item as the variable name in you template.
If you want to pass more than one variable this way, you can dicts as your list items like this:
var_a:
-
var_1: 'this is var_a1'
var_2: 'this is var_a2'
var_b:
-
var_1: 'this is var_b1'
var_2: 'this is var_b2'
and then refer to them in your template like this:
{{ item.var_1 }}
{{ item.var_2 }}
I did it in this way.
In tasks/main.yml
- name: template test
template:
src=myTemplateFile.j2
dest={{item}}
with_dict: some_dict
and in vars/main.yml
some_dict:
/path/to/dest1:
var1: 1
var2: 2
/path/to/dest2:
var1: 3
var2: 4
and in templates/myTemplateFile.j2
some_var = {{ item.value.var1 }}
some_other_var = {{ item.value.var2 }}
Hope this solves your problem.
I had a similar problem to solve, here is a simple solution of how to pass variables to template files, the trick is to write the template file taking advantage of the variable. You need to create a dictionary (list is also possible), which holds the set of variables corresponding to each of the file. Then within the template file access them.
see below:
the template file: test_file.j2
# {{ ansible_managed }} created by xbalaji#gmail.com
{% set dkey = (item | splitext)[0] %}
{% set fname = test_vars[dkey].name %}
{% set fip = test_vars[dkey].ip %}
{% set fport = test_vars[dkey].port %}
filename: {{ fname }}
ip address: {{ fip }}
port: {{ fport }}
the playbook
---
#
# file: template_test.yml
# author: xbalaji#gmail.com
#
# description: playbook to demonstrate passing variables to template files
#
# this playbook will create 3 files from a single template, with different
# variables passed for each of the invocation
#
# usage:
# ansible-playbook -i "localhost," template_test.yml
- name: template variables testing
hosts: all
gather_facts: false
vars:
ansible_connection: local
dest_dir: "/tmp/ansible_template_test/"
test_files:
- file_01.txt
- file_02.txt
- file_03.txt
test_vars:
file_01:
name: file_01.txt
ip: 10.0.0.1
port: 8001
file_02:
name: file_02.txt
ip: 10.0.0.2
port: 8002
file_03:
name: file_03.txt
ip: 10.0.0.3
port: 8003
tasks:
- name: copy the files
template:
src: test_file.j2
dest: "{{ dest_dir }}/{{ item }}"
with_items:
- "{{ test_files }}"
- name: copy vhosts
template: src=site-vhost.conf dest=/etc/apache2/sites-enabled/{{ item }}.conf
with_items:
- somehost.local
- otherhost.local
notify: restart apache
IMPORTANT: Note that an item does not have to be just a string, it can be an object with as many properties as you like, so that way you can pass any number of variables.
In the template I have:
<VirtualHost *:80>
ServerAdmin me#example.org
ServerName {{ item }}
DocumentRoot /vagrant/public
ErrorLog ${APACHE_LOG_DIR}/error-{{ item }}.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
Another real world example using a list
an extract for a template for php.ini
{% if 'cli/php.ini' in item.d %}
max_execution_time = 0
memory_limit = 1024M
{% else %}
max_execution_time = 300
memory_limit = 512M
{% endif %}
This is the var
php_templates:
- { s: 'php.ini.j2', d: "/etc/php/{{php_version}}/apache2/php.ini" }
- { s: 'php.ini.j2', d: "/etc/php/{{php_version}}/cli/php.ini" }
Then i deploy with this
- name: push templated files
template:
src: "{{item.s}}"
dest: "{{item.d}}"
mode: "{{item.m | default(0644) }}"
owner: "{{item.o | default('root') }}"
group: "{{item.g | default('root') }}"
backup: yes
with_items: "{{php_templates}}"