Ansible: Updating values in a list of dictionaries - ansible

Question

In my payload, I have a variable that is actually a list of dictionaries, such as this one:
myvar:
- name: name1
ip_addresses:
- 10.10.10.10
- 11.11.11.11
nat_destination_addresses:
- host: 12.12.12.12
destination: 13.13.13.13
- host: 14.14.14.14
destination: 15.15.15.15
nat_source_address: 16.16.16.16
applications:
- protocol: tcp
port: 8302
- protocol: udp
port: 2000
- protocol: tcp
port: 2000-5600
- name: name2
ip_addresses:
- 17.17.17.17
- name: name3
ip_addresses:
- 18.18.18.18
- 19.19.19.19
All the values for each element in myvar are optional, except for the name, which is mandatory.
I am trying to pad the ip addresses (ip_addresses, nat_destination_addresses and nat_source_address) and ports. The ports should have a length of five characters with zeroes at the beginning (2000 becomes 02000 and 2000-5600 becomes 02000-05600) and the ip addresses should have three characters for each subsection (18.18.18.18 becomes 018.018.018.018).
The problem that I have is that I am not able to change only subsections of myvar.
I have read other questions here, such as:
merging dictionaries in ansible
Using set_facts and with_items together in Ansible
But to no avail. No matter what I do, I am not able to keep the original dictionary, I end up with a list of ip_addresses if I use the combine filter from the second StackOverflow link.
The expected result is the original myvar variable with updated ip addresses and ports.

Answer 1

This seems like a good time to throw your logic into a custom Ansible module. It doesn't have to be anything fancy, for example:
from ansible.module_utils.basic import AnsibleModule
def pad_addr(addr):
return '.'.join('%03d' % int(x) for x in addr.split('.'))
def main():
module_args = dict(
data=dict(type='list', required=True),
)
module = AnsibleModule(
argument_spec=module_args,
supports_check_mode=True
)
data = module.params['data']
for d in data:
if 'ip_addresses' in d:
d['ip_addresses'] = [pad_addr(x) for x in d['ip_addresses']]
if 'nat_destination_addresses' in d:
for dest in d['nat_destination_addresses']:
dest['host'] = pad_addr(dest['host'])
dest['destination'] = pad_addr(dest['destination'])
if 'nat_source_address' in d:
d['nat_source_address'] = pad_addr(d['nat_source_address'])
if 'applications' in d:
for service in d['applications']:
service['port'] = '%05d' % service['port']
module.exit_json(changed=False,
result=data)
if __name__ == '__main__':
main()
If I drop the above into library/pad_data.py and then run the following playbook:
- hosts: localhost
gather_facts: false
vars:
myvar:
- name: name1
ip_addresses:
- 10.10.10.10
- 11.11.11.11
nat_destination_addresses:
- host: 12.12.12.12
destination: 13.13.13.13
- host: 14.14.14.14
destination: 15.15.15.15
nat_source_address: 16.16.16.16
applications:
- protocol: tcp
port: 8302
- protocol: udp
port: 2000
- protocol: tcp
port: 2000
- name: name2
ip_addresses:
- 17.17.17.17
- name: name3
ip_addresses:
- 18.18.18.18
- 19.19.19.19
tasks:
- pad_data:
data: "{{ myvar }}"
register: padded
- debug:
var: padded.result
I get as the result:
TASK [debug] *******************************************************************
ok: [localhost] => {
"padded.result": [
{
"applications": [
{
"port": "08302",
"protocol": "tcp"
},
{
"port": "02000",
"protocol": "udp"
},
{
"port": "02000",
"protocol": "tcp"
}
],
"ip_addresses": [
"010.010.010.010",
"011.011.011.011"
],
"name": "name1",
"nat_destination_addresses": [
{
"destination": "013.013.013.013",
"host": "012.012.012.012"
},
{
"destination": "015.015.015.015",
"host": "014.014.014.014"
}
],
"nat_source_address": "016.016.016.016"
},
{
"ip_addresses": [
"017.017.017.017"
],
"name": "name2"
},
{
"ip_addresses": [
"018.018.018.018",
"019.019.019.019"
],
"name": "name3"
}
]
}

Answer 2

Larsks' answer was on point and is probably the best solution for most people, but my requirements are to limit the number of modules created with Python for this project, so here is my workaround for reference purposes.
Basically, what I do in this sample is:
Locally:
I take myvar, I output it to a yml file (with '---' at the top of the file and making sure that myvar is still set as the key.
Using regexp and the replace module, I replace the parts of the file that I want to replace.
On all of my hosts:
I reload the (now) properly formatted myvar and replace the old myvar variable using include_vars
---
- name: Customer {{ customer_id }} - Format the ip addresses and ports
hosts: localhost
gather_facts: no
connection: local
tags: [format_vars]
tasks:
- name: Copy the 'myvar' content to a local file to allow ip addresses
and ports formatting
copy:
content: "---\n{{ { 'myvar': myvar} | to_nice_yaml(indent=2) }}"
dest: "{{ formatted_myvar_file }}"
- name: Pad all ip addresses parts with two zeroes to ensure that all parts have at least three numbers
replace:
path: "{{ formatted_myvar_file }}"
regexp: '(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})'
replace: '00\1.00\2.00\3.00\4'
- name: Remove extra zeroes from ip addresses to ensure that all of their parts have exactly three numbers
replace:
path: "{{ formatted_myvar_file }}"
regexp: '\d{0,2}(\d{3})\.\d{0,2}(\d{3})\.\d{0,2}(\d{3})\.\d{0,2}(\d{3})'
replace: '\1.\2.\3.\4'
- name: Pad all ports with four zeroes to ensure that they all have at least five numbers
replace:
path: "{{ formatted_myvar_file }}"
regexp: 'port: (\d{1,5})'
replace: 'port: 0000\1'
- name: Remove extra zeroes from ports to ensure that they all have exactly five numbers
replace:
path: "{{ formatted_myvar_file }}"
regexp: 'port: \d{0,4}(\d{5})'
replace: 'port: \1'
- name: Pad all second parts of port ranges with four zeroes to ensure that they all have at least five numbers
replace:
path: "{{ formatted_myvar_file }}"
regexp: 'port: (\d{5})-(\d{1,5})'
replace: 'port: \1-0000\2'
- name: Remove extra zeroes from second parts of port ranges to ensure that they all have exactly five numbers
replace:
path: "{{ formatted_myvar_file }}"
regexp: 'port: (\d{5})-\d{0,4}(\d{5})'
replace: 'port: \1-\2'
- name: Customer {{ customer_id }} - Load the properly formatted ip addresses and ports
hosts: localhost:all-n7k:srx-clu:all-mx80:all-vsrx
gather_facts: no
connection: local
tags: [format_vars]
tasks:
- include_vars:
file: "{{ formatted_myvar_file }}"
ignore_errors: yes