Override group_vars by external source - ansible

Question

I have a pilote project keeping many common variables in group_vars.
group_vars/
group1.yml
group2.yml
group3.yml
For different implementations (usually per client), I'd like to maintain reserved file which overrides the content of group_vars, where the content of that file could have following format, i.e. client1.yml :
group1:
var11_to_override: "foo"
var12_to_override: "bar"
group2:
var21_to_override: "foo"
var22_to_override: "bar"
Is there a simple possibility to say to Ansible that file client1.yml overrides group_vars content?
The module include_vars could be certainly the first step together with set_facts within a loop, but it requires probably complicated jinja2 filter expressions ...
Have I to write a new module or filter updating hostvars?

Answer 1

Finally resolved by custom filter updating a dict by another:
filter_plugins/vars_update.py
import copy
import collections
class FilterModule(object):
def update_hostvars(self, _origin, overlay):
origin = copy.deepcopy(_origin)
for k, v in overlay.items():
if isinstance(v, collections.Mapping):
origin[k] = self.update_hostvars(origin.get(k, {}), v)
else:
origin[k] = v
return origin
def filters(self):
return {"update_hostvars": self.update_hostvars}
.. and using this filter when updating all variables:
- name: Include client file
include_vars:
file: "{{ client_file_path }}"
name: client_overlay
- name: Update group_vars by template client
set_fact:
"{{ item.key }}": "{{ hostvars[inventory_hostname][item.key] | update_hostvars(item.value) }}"
with_dict: "{{ client_overlay }}"

Answer 2

Using the examples given in this thread i made my own solution:
The "external source" feeds in an inventory item using --extra-vars "#". The file content itself is uploaded as base64 encoded content and then decoded/written to fs.
The external file has a list of overrides per role/group like so:
role_overrides: [{
"groups": [
"my-group"
],
"overrides": {
"foo": "value",
"bar": "value",
}
},
but then jsonified obviously...
The filter module
#!/usr/bin/env python
class FilterModule(object):
def filters(self):
return {
"filter_hostvars_overrides": self.filter_hostvars_overrides,
}
def filter_hostvars_overrides(self, role_overrides, group_names):
"""
filter the overrides for the ones to apply for this host
[
{
"groups": [
"my-group"
],
"overrides": {
"foo: 42,
}
},
:param group_names: List of groups this host is member of
:param role_overrides: document with all overrides; to be filtered using groups_names
:return: items to be set
"""
overrides = {}
for idx, per_group_overrides in enumerate(role_overrides):
groups = per_group_overrides.get("groups", [])
if set(groups).intersection(set(group_names)):
overrides.update(per_group_overrides.get("overrides", {}))
return overrides
The play code:
- name: Apply group overrides
set_fact:
"{{ item.key }}": "{{ item.value }}"
with_dict: "{{ role_overrides | filter_hostvars_overrides(group_names) }}"