I am running into an issue where my template is rendered multiple times during a play. It seems like it is happening when the lookup plugin is called either from group or host vars. This is leading to slowness in my playbook execution.
I simplified my playbook and inventory to provide an example.
Below is the playbook I use to demonstrate the issue I am seeing:
---
- name: 1st play
hosts: mux01
connection: local
gather_facts: no
tasks:
- name: mux01 - set uplinks
set_fact:
uplinks: "{{ lookup('template', 'inventories/test_lookup/group_vars/mux_uplinks.j2') }}"
- name: mux01 - show uplink
debug:
msg: "{{ device[inventory_hostname].uplink }}"
- name: mux01 - show uplink again
debug:
msg: "{{ device[inventory_hostname].uplink }}"
- name: 2nd play
hosts: mux02
connection: local
gather_facts: no
tasks:
- name: mux02 - show uplink
debug:
msg: "{{ device[inventory_hostname].uplink }}"
- name: mux02 - show uplink again
debug:
msg: "{{ device[inventory_hostname].uplink }}"
For mux01, I am setting the uplinks variable using the template plugin and then print that variable twice. For mux02, the uplinks variable is set in the host vars file, so here I am just printing that variable twice as well.
Here is my mux01 host var file:
device:
mux01:
loopback: 1.1.1.1
uplink: "{{ uplinks.1 }}"
And here is my mux02 host var file:
uplinks: "{{ lookup('template', 'inventories/test_lookup/group_vars/mux_uplinks.j2') }}"
device:
mux02:
loopback: 1.1.1.1
uplink: "{{ uplinks.2 }}"
In this scenario, my mux_uplinks.j2 template is just updating the mux_uplinks variable in a for loop:
{% set mux_uplinks = {} %}
{% for i in range(1,100000) %}
{% set _ = mux_uplinks.update({i:i}) %}
{% endfor %}
{{ mux_uplinks }}
And finally when I run the playbook and profile it, I get the following:
mux01 - set uplinks ------------------ 8.45s
mux02 - show uplink ------------------ 7.83s
mux02 - show uplink again ------------ 7.71s
mux01 - show uplink again ------------ 0.64s
mux01 - show uplink ------------------ 0.63s
So it really seems to me that the template is being rendered multiple times if it is being called from the host/group vars. Anyone has any idea why?
Related
A webhook triggers an AWX job and I want to run the deployment on a certain host depending on the service, since they run on different servers. I need to know which server uses that service to set is as a var so it can be used as a host in the following play.
My variable inside vars.yaml looks like this:
staging_hosts:
server1: ['service1', 'service2', 'service3']
server2: ['service4', 'service5', 'service6']
server3: ['service7', 'service8', 'service9']
Playbook:
- name: write deployment hosts
hosts: localhost
vars:
deployment_hosts: absent
vars_files:
- ./group_vars/vars.yaml
tasks:
- set_fact:
modified_repos: (small regex filter to find modified repository)
- set_fact:
deployment_hosts: "{{ item }}"
when: '{{ modified_repos }} in {{ item }}'
with_list:
- "{{ staging_hosts }}"
- name: connect to Cluster
hosts: "{{ hostvars['localhost']['deployment_hosts'] }}"
What can I do against this warning and error?
[WARNING]: conditional statements should not include jinja2 templating
delimiters such as {{ }} or {% %}. Found: {{ modified_repos }} in {{ item }}
fatal: [localhost]: FAILED! => {"msg": "The conditional check '{{ modified_repos }} in {{ item }}' failed. True {% else %} False {% endif %}): unhashable type: 'list'
Oh I forgot to mention. It is important, that deployment_hosts could also contain two hosts if modified repos include for example service1 and service4.
Q: "deployment_hosts could also contain two hosts if modified repos include for example service1 and service4."
A: Use intersect filter. For example, the playbook
- hosts: localhost
vars:
staging_hosts:
server1: ['service1', 'service2', 'service3']
server2: ['service4', 'service5', 'service6']
server3: ['service7', 'service8', 'service9']
modified_repos: ['service1', 'service4']
tasks:
- set_fact:
deployment_hosts: "{{ deployment_hosts|default([]) + [item.key] }}"
loop: "{{ staging_hosts|dict2items }}"
when: modified_repos|intersect(item.value)|length > 0
- debug:
var: deployment_hosts
gives
deployment_hosts:
- server1
- server2
I have an Ansible playbook that gathers facts from Cisco switches.
---
- hosts: switches
gather_facts: False
connection: network_cli
vars:
backup_root: ./configs
cli:
host: "{{ inventory_hostname }}"
tasks:
- name: ensure device folder is created
file:
path: "{{ backup_root }}/{{ inventory_hostname }}"
state: directory
- name: Gather all facts
cisco.ios.ios_facts:
gather_subset: all
- name: Serial Number
debug: var=ansible_net_serialnum
- name: Model
debug: var=ansible_net_model
- name: Hostname
debug: var=ansible_net_hostname
- name: Version
debug: var=ansible_net_version
- name: CDP
debug: var=ansible_net_neighbors
- name: Config file
debug: var=ansible_net_config
- name: Stack SW Model Numbs
debug: var=ansible_net_stacked_models
- name: Stack SW Model Numbs
debug: var=ansible_net_stacked_serialnums
- name: Get VLAN Info
cisco.ios.ios_command:
commands: show vlan brief
register: show_vlan
- name: get timestamp
command: date +%Y%m%d
register: timestamp
- name: Generate configuration files
template:
src=roles/discovery/templates/ios_switches.j2
dest="{{ backup_root }}/{{ inventory_hostname }}/{{ inventory_hostname }}.txt" `
Here is the jinja file.
Hostname: {{ ansible_net_hostname }}
Model: {{ansible_net_model}}
Serial Number: {{ansible_net_serialnum}}
IOS Version: {{ansible_net_version}}
IOS Image: {{ansible_net_image}}
Switch Stack Models:
{{ansible_net_stacked_models | to_nice_yaml(indent=2)}}
Switch Stack Serials:
{{ansible_net_stacked_serialnums | to_nice_yaml(indent=2)}}
CDP Neighbors:
{{ansible_net_neighbors | to_nice_yaml(indent=2)}}
Configuration:
{{ansible_net_config}}
VLAN:
{{show_vlan.stdout[0] | to_nice_yaml(indent=2)}}
This all works fine until it hits a switch that cannot stack (e.g. chassis or VSS). When I run the playbook, I get the following-
msg: 'AnsibleUndefinedVariable: ''ansible_net_stacked_models'' is undefined
I've tried using if in Jinja2 like the following
...
Switch Stack Models:
{% if ansible_net_stacked_models is not defined %}
NOT A STACKABLE SWITCH
{% else %}
{{ansible_net_stacked_models | to_nice_yaml(indent=2)}}
{% endif %}
however it fails in the Jinja rendering and does not produce any output.
Is there a way to ignore missing variables in jinja or is there a better way to do this?
you can set a default value to your variable if is not defined
{{ ansible_net_stacked_models|default("NOT A STACKABLE SWITCH", true) | to_nice_yaml(indent=2) }}
Come to find out the error was stopping Ansible from even calling the jinja file. I dug around some more and found that I could set the variable to a default value in the inventory.
[switch:vars]
ansible_net_stacked_models='NOT A STACKABLE SWITCH'
When running the playbook if the device has valid info it will overwrite the default variable we defined in inventory. If the device doesn't have valid info, Ansible will simply pass the default variable we set in inventory, over to jinja2.
---
- name: Mikrotik info
hosts: mikrotik
connection: network_cli
remote_user: root
gather_facts: false
tasks:
- name: show info
routeros_command:
commands: /system routerboard print
register: rb_info
- name: Debug info
debug:
msg: "{{ rb_info.stdout_lines }}"
Output:
routerboard: yes
model: 751G-2HnD
serial-number: 3A6502B2A2E7
firmware-type: ar7240
factory-firmware: 3.0
current-firmware: 6.42.3
upgrade-firmware: 6.43.4
I need to filter it for "upgrade-firmware" string and get output like this:
upgrade-firmware: 6.43.4
I should use regex_replace? Or I can use grep or something like that?
Any thoughts are greatly appreciated.
Thank you
(update)
Use from_yaml and combine a dictionary. For example
- set_fact:
minfo: "{{ minfo|default({})|combine(item|from_yaml) }}"
loop: "{{ rb_info.stdout_lines }}"
- debug:
var: minfo['upgrade-firmware']
give
minfo['upgrade-firmware']: 6.43.4
(for the record)
Robust solution is to write the data to template and include_vars. The tasks below
- tempfile:
register: tempfile
- template:
src: minfo.j2
dest: "{{ tempfile.path }}"
- include_vars:
file: "{{ tempfile.path }}"
name: minfo
- debug:
var: minfo
with the template
shell> cat minfo.j2
{% for item in rb_info.stdout_lines %}
{{ item }}
{% endfor %}
should give
"minfo": {
"current-firmware": "6.42.3",
"factory-firmware": 3.0,
"firmware-type": "ar7240",
"model": "751G-2HnD",
"routerboard": true,
"serial-number": "3A6502B2A2E7",
"upgrade-firmware": "6.43.4"
}
The tasks below creates variable upgrade_firmware
- set_fact:
upgrade_firmware: "{{ item.split(':').1|trim }}"
loop: "{{ rb_info.stdout_lines|map('quote')|map('trim')|list }}"
when: item is search('^upgrade-firmware')
- debug:
var: upgrade_firmware
It is possible to put all the parameters into the dictionary
- set_fact:
minfo: "{{ minfo|default({})|
combine({item.split(':').0: item.split(':').1|trim}) }}"
loop: "{{ rb_info.stdout_lines|map('quote')|map('trim')|list }}"
- debug:
var: minfo['upgrade-firmware']
I'm trying to leverage ovirt-ansible (https://github.com/oVirt/ovirt-ansible) but it's vars are a bit verbose and I'd like to generate them off some simpler ones to reduce the chance of human error.
For example, something like this which obviously doesn't work:
vars:
cluster_name: test
domain_suffix: blah
subnet_mgmt: 1.2.3
myhosts:
- id: 001
ip_suffix: 101
hosts:
{% for host in myhosts %}
- name: "{{ cluster_name }}-{{ host[id] }}.{{ domain_suffix }}"
address: "{{ subnet_mgmt }}.{{ host[ip_suffix] }}"
{% endfor %}
Can anyone advise best way to go about doing this, without simply forking their repo and rewriting the playbook to read the variables in my own format? I'm hoping to avoid having to maintain a fork going forwards.
Is this the code that you're looking for?
> cat dynamic_var.yml
- hosts: localhost
vars:
cluster_name: test
domain_suffix: blah
subnet_mgmt: 1.2.3
myhosts:
- { id: "001", ip_suffix: "101" }
tasks:
- set_fact:
hosts:
- name: "{{ cluster_name }}-{{ item.id }}.{{ domain_suffix }}"
address: "{{ subnet_mgmt }}.{{ item.ip_suffix }}"
loop: "{{ myhosts }}"
- debug:
msg: "{{ hosts }}"
> ansible-playbook dynamic_var.ym
(abridged)
TASK [debug]
"address": "1.2.3.101",
"name": "test-001.blah"
Based on this question
Ansible recursive checks in playbooks
I have another one.
We need to go through this structure
Zone spec https://gist.github.com/git001/9230f041aaa34d22ec82eb17d444550c
Now I can adress the hostnames via the array index but can I also iterate over the array "hosts"?
playbook
--
- hosts: all
gather_facts: no
vars_files:
- "../doc/application-zone-spec.yml"
roles:
- { role: ingress_add, customers: "{{ application_zone_spec }}" }
role
- name: Print ingress hostnames
debug: msg="{{ item.hosts.0.hostname }} {{ item.hosts.1.hostname }}"
with_items: "{{ customers.ingress }}"
We use.
ansible-playbook --version
ansible-playbook 2.1.0.0
config file = /etc/ansible/ansible.cfg
configured module search path = Default w/o overrides
Use with_subelements:
- name: Print ingress hostnames
debug: msg="{{ item.0.type }} {{ item.1.hostname }}"
with_subelements:
- "{{ customers.ingress }}"
- "hosts"
There is quite a bit of examples for different loops in the Loops section of the documentation.