Using parse_xml in an Ansible playbook - ansible

I've been trying to parse XML data in Ansible. I can get it to work using the xml module but I think that using parse_xml would better suit my needs.
I don't seem to be able to match any of the data in the xml with my specs file.
Here is the xml data:
<data xmlns=\"urn:ietf:params:xml:ns:netconf:base:1.0\" xmlns:nc=\"urn:ietf:params:xml:ns:netconf:base:1.0\">
<ntp xmlns=\"http://cisco.com/ns/yang/Cisco-IOS-XR-ip-ntp-oper\">
<nodes>
<node>
<node>0/0/CPU0</node>
<associations>
<is-ntp-enabled>true</is-ntp-enabled>
<sys-leap>ntp-leap-no-warning</sys-leap>
<peer-summary-info>
<peer-info-common>
<host-mode>ntp-mode-client</host-mode>
<is-configured>true</is-configured>
<address>10.1.1.1</address>
<reachability>0</reachability>
</peer-info-common>
<time-since>-1</time-since>
</peer-summary-info>
<peer-summary-info>
<peer-info-common>
<host-mode>ntp-mode-client</host-mode>
<is-configured>true</is-configured>
<address>172.16.252.29</address>
<reachability>255</reachability>
</peer-info-common>
<time-since>991</time-since>
</peer-summary-info>
</associations>
</node>
</nodes>
</ntp>
</data>
This is what the spec file looks like:
---
vars:
ntp_peers:
address: "{{ item.address }}"
reachability: "{{ item.reachability}}"
keys:
result:
value: "{{ ntp_peers }}"
top: data/ntp/nodes/node/associations
items:
address: peer-summary-info/peer-info-common/address
reachability: peer-summary-info/peer-info-common/reachability
and the task in the yaml file:
- name: parse ntp reply
set_fact:
parsed_ntp_data: "{{ NTP_STATUS.stdout | parse_xml('specs/iosxr_ntp.yaml') }}"
but the data does not return any results:
TASK [debug parsed_ntp_data] **************************************************************************************************************************************************************************
ok: [core-rtr01] => {
"parsed_ntp_data": {
"result": []
}
}
ok: [dist-rtr01] => {
"parsed_ntp_data": {
"result": []
}
}

I had never seen parse_xml before, so that was a fun adventure
There appear to be two things conspiring against you: the top: key is evaluated from the root Element, and your XML (unlike the rest of the examples) uses XML namespaces (the xmlns= bit) which means your XPaths have to be encoded in the Element.findall manner
For the first part, since Element.findall is run while sitting on the <data> Element, that means one cannot reference data/... in an XPath because that would be applicable to a structure <data><data>. I tried being sneaky by just making the XPath absolute /data/... but Python's XPath library throws up in that circumstance. So, at the very least your top: key needs to not start with data anything
Then, the xmlns= in your snippet stood out to me because that means those element's names are actually NS+":"+localName for every element, and thus an XPath of ntp does NOT match ns0:ntp because they're considered completely separate names (that being the point of the namespace, after all). It may very well be possible to use enough //*[localname() = "ntp"] silliness to avoid having to specify the namespace over and over, but I didn't try it
Again, as a concession to Python's XPath library, they encode the fully qualified name in an xpath as {the-namespace}local-name and there does not seem to be any way short of modifying network.py to pass in namespaces :-(
Thus, the "hello world" version that I used to confirm my theory:
vars:
ntp_peers:
address: "{{ item.address }}"
keys:
result:
value: "{{ ntp_peers }}"
top: '{http://cisco.com/ns/yang/Cisco-IOS-XR-ip-ntp-oper}ntp/{http://cisco.com/ns/yang/Cisco-IOS-XR-ip-ntp-oper}nodes/{http://cisco.com/ns/yang/Cisco-IOS-XR-ip-ntp-oper}node'
items:
address: '{http://cisco.com/ns/yang/Cisco-IOS-XR-ip-ntp-oper}node'
cheerfully produced
ok: [localhost] => {
"msg": {
"result": [
{
"address": "0/0/CPU0"
}
]
}
}

Related

Ansible - Passing a dictionary to a module parameter

I'm using fortinet.fortios.system_global module as describe here: https://docs.ansible.com/ansible/latest/collections/fortinet/fortios/fortios_system_global_module.html#ansible-collections-fortinet-fortios-fortios-system-global-module
My goal is to pass a dictionary to the system_global parameter with the allowed sub-parameters. I have the dictionary as follows for example:
forti:
admin-concurrent: enable
admin-console-timeout: 0
admin-hsts-max-age: 15552000
<more key:value>
This dictionary lives in a separate file called forti.yml.
I then use include_vars to pull this yml file into my play as follows:
vars_files:
- /path/to/forti.yml
And then I use the system_global module:
- name: Configure system_global task
fortios_system_global:
access: "{{ access_token }}"
system_global: "{{ forti }}"
However, when I run the play it throws an error like so:
"msg": "Unsupported parameters for (fortios_system_global) module: system_global.admin-concurrent, system_global.admin-console-timeout, system_global.admin-hsts-max-age,<and so on>. Supported parameters include: member_path, member_state, system_global, vdom, enable_log, access_token."
I tried putting the key:value pairs in the vars: in the play level and passed it to the module the same way and it worked.
vars:
forti:
admin-concurrent: enable
admin-console-timeout: 0
admin-hsts-max-age: 15552000
<more key: value>
What am I missing? They're both type: dict, the data are exactly the same. Not sure what I'm missing here. Can someone please help?
You have - and the parameters are supposed to be _ so it is telling you the module parameter does not exist
vars:
forti:
admin-concurrent: enable
admin-console-timeout: 0
admin-hsts-max-age: 15552000
<more key: value>
should be
vars:
forti:
admin_concurrent: enable
admin_console_timeout: 0
admin_hsts_max_age: 15552000
<more key: value>
Keep on automating!
Just look at module examples here: https://docs.ansible.com/ansible/latest/collections/fortinet/fortios/fortios_system_global_module.html#ansible-collections-fortinet-fortios-fortios-system-global-module

Generate list of IP addresses from start and end values with Ansible

Is there a way to generate a list of IP addresses between two arbitrary IPs (not from a subnet/range) with Ansible (v2.9)?
I've searched and the ipaddr filter looks like a good candidate, but from the documentation I couldn't figure out if it supports this.
I'm looking for a solution that allows me to get a list like
[ '10.0.0.123', '10.0.0.124', ... , '10.0.1.23' ]
from a task like
- name: generate IP list
set_fact:
ip_list: "{{ '10.0.0.123' | ipaddr_something('10.0.1.23') }}"
Create a filter plugin. For example
shell> cat filter_plugins/netaddr.py
import netaddr
def netaddr_iter_iprange(ip_start, ip_end):
return [str(ip) for ip in netaddr.iter_iprange(ip_start, ip_end)]
class FilterModule(object):
''' Ansible filters. Interface to netaddr methods.
https://pypi.org/project/netaddr/
'''
def filters(self):
return {
'netaddr_iter_iprange' : netaddr_iter_iprange,
}
Then, the task below shall create the list
- set_fact:
ip_list: "{{ '10.0.0.123'|netaddr_iter_iprange('10.0.1.23') }}"

Override group_vars by external source

I have a pilote project keeping many common variables in group_vars.
group_vars/
group1.yml
group2.yml
group3.yml
For different implementations (usually per client), I'd like to maintain reserved file which overrides the content of group_vars, where the content of that file could have following format, i.e. client1.yml :
group1:
var11_to_override: "foo"
var12_to_override: "bar"
group2:
var21_to_override: "foo"
var22_to_override: "bar"
Is there a simple possibility to say to Ansible that file client1.yml overrides group_vars content?
The module include_vars could be certainly the first step together with set_facts within a loop, but it requires probably complicated jinja2 filter expressions ...
Have I to write a new module or filter updating hostvars?
Finally resolved by custom filter updating a dict by another:
filter_plugins/vars_update.py
import copy
import collections
class FilterModule(object):
def update_hostvars(self, _origin, overlay):
origin = copy.deepcopy(_origin)
for k, v in overlay.items():
if isinstance(v, collections.Mapping):
origin[k] = self.update_hostvars(origin.get(k, {}), v)
else:
origin[k] = v
return origin
def filters(self):
return {"update_hostvars": self.update_hostvars}
.. and using this filter when updating all variables:
- name: Include client file
include_vars:
file: "{{ client_file_path }}"
name: client_overlay
- name: Update group_vars by template client
set_fact:
"{{ item.key }}": "{{ hostvars[inventory_hostname][item.key] | update_hostvars(item.value) }}"
with_dict: "{{ client_overlay }}"
Using the examples given in this thread i made my own solution:
The "external source" feeds in an inventory item using --extra-vars "#". The file content itself is uploaded as base64 encoded content and then decoded/written to fs.
The external file has a list of overrides per role/group like so:
role_overrides: [{
"groups": [
"my-group"
],
"overrides": {
"foo": "value",
"bar": "value",
}
},
but then jsonified obviously...
The filter module
#!/usr/bin/env python
class FilterModule(object):
def filters(self):
return {
"filter_hostvars_overrides": self.filter_hostvars_overrides,
}
def filter_hostvars_overrides(self, role_overrides, group_names):
"""
filter the overrides for the ones to apply for this host
[
{
"groups": [
"my-group"
],
"overrides": {
"foo: 42,
}
},
:param group_names: List of groups this host is member of
:param role_overrides: document with all overrides; to be filtered using groups_names
:return: items to be set
"""
overrides = {}
for idx, per_group_overrides in enumerate(role_overrides):
groups = per_group_overrides.get("groups", [])
if set(groups).intersection(set(group_names)):
overrides.update(per_group_overrides.get("overrides", {}))
return overrides
The play code:
- name: Apply group overrides
set_fact:
"{{ item.key }}": "{{ item.value }}"
with_dict: "{{ role_overrides | filter_hostvars_overrides(group_names) }}"

Use list of dictionaries variable on Ansible Tower textare survey

I'm trying to develop a playbook were I have the following variable.
disk_vars:
- { Unit: C, Size: 50 }
- { Unit: D, Size: 50 }
With the variables defined on the playbook there is no problem but when I try to use a texarea survey on Ansible Tower I cannot manage to parse them as list of dictionaries.
I tried adding to the survey the following two lines which are already on yaml format.
- { Unit: C, Size: 50 }
- { Unit: D, Size: 50 }
And on my vars section I use test_var: "{{ test_var1.split('\n') }} which converts the output into a two line string. Without the split is just a single line string.
I could make my playbook work with a simple dictionary like
dict1: {{ Unit: C, Size: 50 }}
but I'm having issues parsing it as well.
EDIT
Changing it to the following as suggested by mdaniels works.
- set_fact:
test_var: "{{ test_var1 | from_yaml }}"
- name: test
debug: msg=" hostname is {{ item.Unit }} and {{ item.Size }}"
with_items:
- "{{ test_var }}"
I'm trying to figure a way to clear-up the data input as asking users to respect the format is not a very good idea.
tried changing the input date to the following but I could not figure out how to format that into a list of dictionaries.
disk_vars:
Unit: C, Size: 50
Unit: D, Size: 50
I tried with the following piece of code
- set_fact:
db_list: >-
{{ test_var1.split("\n") | select |
map("regex_replace", "^", "- {") |
map("regex_replace", "$", "}") |
join("\n") }}
But is putting it all on a single line.
"db_list": "- {dbid: 1, dbname: abc\ndbid: 2, dbname: xyz} "
I have tried to play with it but could not manage to make it work.
I believe you were very close; instead of "{{ test_var1.split('\n') }}" I believe you can just feed it to the from_yaml filter:
- set_fact:
test_var1: '{{ test_var1 | from_yaml }}'
# this is just to simulate the **str** that you will receive from the textarea
vars:
test_var1: "- { Unit: C, Size: 50 }\n- { Unit: D, Size: 50 }\n"
- debug:
msg: and now test_var1[0].Unit is {{ test_var1[0].Unit }}
I faced a similar dilemma, i.e. that I was bound to the survey format(s) available, and I was forced to use mdaniels suggested solution above with sending the data as text and then later parse it from YAML . Problem was however that controlling the format of the input (i.e. a YAML-string inside the text) would probably cause a lot of headache/errors, just like you describe.
Maybe you really need to use the Survey, but in my case I was more interested of calling the Job Template using the Tower REST API. For some reason I thought I then had to have a survey with all parameters defined. But it turned out I was wrong, when having a survey I was not able to provide dictionaries as input data (in the extra_vars). However, when removing the Survey, and also (not sure if required or not) enabling "Extra Variables -> prompt on launch", then things started to work!! Now I can provide lists / dictionaries as input to my Templates when calling them using REST API POST calls, see example below:
{
"extra_vars": {
"p_db_name": "MYSUPERDB",
"p_appl_id": "MYD32",
"p_admin_user": "myadmin",
"p_admin_pass": "mysuperpwd",
"p_db_state": "present",
"p_tablespaces": [
{
"name": "tomahawk",
"size": "10M",
"bigfile": true,
"autoextend": true,
"next": "1M",
"maxsize": "20M",
"content": "permanent",
"state": "present"
}
],
"p_users": [
{
"schema": "myschema",
"password": "Mypass123456#",
"default_tablespace": "tomahawk",
"state": "present",
"grants": "'create session', 'create any table'"
}
]
}
}

ERROR! Unexpected Exception, this is probably a bug: argument of type 'bool' is not iterable

I've got an error with Ansible, that I don't understand:
ERROR! Unexpected Exception, this is probably a bug: argument of type 'bool' is not iterable
It happens in in role where I call a var file:
- name: import pdt vars
include_vars:
file: "{{ pdt_type }}.yml"
The "{{ pdt_type }}.yml" contains:
pdt_pkg:
- { name: "zzz-libs" }
- { name: "zzz-core" }
What is wrong with that? Is it really a bug?
The { } is supposed to declare a dictionary, using a flow collection syntax
You seem to declare two dictionaries, each with the same key "name"
Check if the content if {{ pdt_type }}.yml is actually the issue, by using a simpler content:
pdt_pkg:
- name1: "zzz-libs"
- name2: "zzz-core"

Resources