Merge 2 hashes recursively with jinja2/ansible - ansible

Question

I'm trying to merge 2 hashes in ansible, but I'm running into problems. The hash is mostly merged, but the lists inside are not.
I'm using the combine filter for the merge:
config_hash: "{{ inventory_config | combine(host_config, recursive=True, list_merge='append_rp') }}"
Here are the two hashes/dicts I'd like to merge:
inventory_config:
exoscale:
inventory_name: test_inventory
security_groups:
- name: rancher
state: present
rules:
- port: 22
cidr: "1.2.3.4/32"
type: egress
- port: 80
cidr: "0.0.0.0/32"
type: egress
host_config:
exoscale:
name: host
disk_size: 25 GiB
security_groups:
- name: rancher
state: added
rules:
- port: 21
cidr: "1.2.3.4/32"
type: ingress
- port: 8080
cidr: "0.0.0.0/32"
type: ingress
The result:
TASK [debug] *******************************************************************
ok: [instance] => {
"config_hash": {
"exoscale": {
"disk_size": "25 GiB",
"inventory_name": "test_inventory",
"name": "host",
"security_groups": [
{
"name": "rancher",
"rules": [
{
"cidr": "1.2.3.4/32",
"port": 22,
"type": "egress"
},
{
"cidr": "0.0.0.0/32",
"port": 80,
"type": "egress"
}
],
"state": "present"
},
{
"name": "rancher",
"rules": [
{
"cidr": "1.2.3.4/32",
"port": 21,
"type": "ingress"
},
{
"cidr": "0.0.0.0/32",
"port": 8080,
"type": "ingress"
}
],
"state": "added"
}
]
}
}
}
The result I wanted:
"config_hash": {
"exoscale": {
"disk_size": "25 GiB",
"inventory_name": "test_inventory",
"name": "host",
"security_groups": [
{
"name": "rancher",
"rules": [
{
"cidr": "1.2.3.4/32",
"port": 22,
"type": "egress"
},
{
"cidr": "0.0.0.0/32",
"port": 80,
"type": "egress"
},
{
"cidr": "1.2.3.4/32",
"port": 21,
"type": "ingress"
},
{
"cidr": "0.0.0.0/32",
"port": 8080,
"type": "ingress"
}
],
"state": "added"
},
]
}
}
I tried some other options, and it seems that "list_merge='append_rp'" can combine lists of simple items but not hashes.
Any ideas? Thanks for the help!

Answer 1

Your question is complex, i suggest you to use a custom filter:
you create a folder filter_plugins in your playbook folder (i have named the file myfilters.py and the filter custom)
myfilters.py in folder filter_plugins:
#!/usr/bin/python
class FilterModule(object):
def filters(self):
return {
'custom': self.custom
}
def custom(self, invent, host):
isecu = invent['exoscale']['security_groups']
hsecu = host['exoscale']['security_groups']
inventory_name = invent['exoscale']['inventory_name']
name = host['exoscale']['name']
disk_size = host['exoscale']['disk_size']
security_groups = []
inames_present = [elem['name'] for elem in isecu]
# if you have present and added in inventory_config uncomment next lines
# inames_present = []
# inames_added = []
# for elem in isecu:
# if elem['state'] == 'present':
# inames_present.append(elem['name'])
# else:
# inames_added.append(elem['name'])
for it in hsecu:
secuname = it['name']
state = it['state']
if secuname in inames_present:
if state == 'present': #overwrite data
rules = it['rules']
else: #merge
rules = it['rules'] + isecu[inames_present.index(secuname)]['rules']
else:
rules = it['rules']
state = 'added'
security_groups.append({'name': secuname, 'rules': rules, 'state': state})
result = {'exoscale': {'disk_size': disk_size, 'name': name, 'inventory_name': inventory_name, 'security_groups': security_groups}}
#print(result)
return result
the playbook:
---
- hosts: localhost
vars:
inventory_config:
exoscale:
inventory_name: test_inventory
security_groups:
- name: rancher
state: present
rules:
- port: 22
cidr: "1.2.3.4/32"
type: egress
- port: 80
cidr: "0.0.0.0/32"
type: egress
host_config:
exoscale:
name: host
disk_size: 25 GiB
security_groups:
- name: rancher
state: added
rules:
- port: 21
cidr: "1.2.3.4/32"
type: ingress
- port: 8080
cidr: "0.0.0.0/32"
type: ingress
tasks:
- name: set variable
set_fact:
config_hash: "{{ inventory_config | custom(host_config) }}"
- name: Display
debug:
var: config_hash
result:
ok: [localhost] => {
"config_hash": {
"exoscale": {
"disk_size": "25 GiB",
"inventory_name": "test_inventory",
"name": "host",
"security_groups": [
{
"name": "rancher",
"rules": [
{
"cidr": "1.2.3.4/32",
"port": 21,
"type": "ingress"
},
{
"cidr": "0.0.0.0/32",
"port": 8080,
"type": "ingress"
},
{
"cidr": "1.2.3.4/32",
"port": 22,
"type": "egress"
},
{
"cidr": "0.0.0.0/32",
"port": 80,
"type": "egress"
}
],
"state": "added"
}
]
}
}
}
You have an idea how to use a custom filter in ansible.
You just adapt the code python.