Ansible : host in multiple groups - ansible

Question

I have a host in 2 groups : pc and Servers
I have 2 group_vars (pc and servers) with, in each the file packages.yml
These files define the list of packages to be installed on pc hosts and on servers hosts
I have a role to install default package
The problem is : only the group_vars/pc/packages.yml is take into account by the role task, packages from group_vars/servers/packages.yml are not installed
Of course what I want is installation of packages defined for pc and servers
I do not know if it is a bug or a feature ...
Thanks for your help
here is the configuration :
# file: production
[pc]
armen
kerbel
kerzo
[servers]
kerbel
---
# packages on servers
packages:
- lftp
- mercurial
---
# packages on pc
packages:
- keepassx
- lm-sensors
- hddtemp

Answer 1

It's not a bug. According to the docs about variable precedence, you shouldn't define a variable in multiple places and try to keep it simple. Michael DeHaan (Ansible's lead dev) responded to a similar question on this topic:
Generally I find the purpose of plays though to bind hosts to roles, so the individual roles should contain the package lists.
I would use roles as it's a bit cleaner IMO.
If you really want (and this is NOT the recommended way), you can set the hash_behaviour option in ansible.cfg:
[defaults]
hash_behaviour = merge
This will cause the merging of two values when a hash (dict) is redefined, instead of replacing the old value with the new one. This does NOT work on lists, though, so you'll need to create a hash of lists, like:
group_vars/all/package.yml:
packages:
all: [pkg1, pkg2]
group_vars/servers/package.yml:
packages:
servers: [pkg3, pkg4]
Looping though that in the playbook is a bit more complex though.

Answer 2

If you want to use such scheme. You should set the hash_behaviour option in ansible.cfg:
[defaults]
hash_behaviour = merge
In addition, you have to use dictionaries instead of lists. To prevent duplicates I recommend to use names as keys, for example:
group_vars/servers/packages.yml:
packages:
package_name1:
package_name2:
group_vars/pc/packages.yml:
packages:
package_name3:
package_name4:
And in a playbook task (| default({}) - for an absent "package" variable case):
- name: install host packages
yum: name={{ item.key }} state=latest
with_dict: packages | default({})

Answer 3

Create a dictionary of the variables per group and merge the lists on your own. For example, create a project for testing
shell> tree .
.
├── ansible.cfg
├── group_dict_create.yml
├── group_vars
│   ├── all
│   │   └── group_dict_vars.yml
│   ├── pc
│   │   └── packages.yml
│   └── servers
│   └── packages.yml
├── hosts
└── pb.yml
4 directories, 7 files
shell> cat ansible.cfg
[defaults]
gathering = explicit
collections_path = $HOME/.local/lib/python3.9/site-packages/
inventory = $PWD/hosts
roles_path = $PWD/roles
retry_files_enabled = false
stdout_callback = yaml
shell> cat hosts
[pc]
armen
kerbel
kerzo
[servers]
kernel
shell> cat group_vars/pc/packages.yml
packages:
- keepassx
- lm-sensors
- hddtemp
shell> cat group_vars/servers/packages.yml
packages:
- lftp
- mercurial
shell> cat pb.yml
- hosts: armen,kerbel,kerzo
pre_tasks:
- import_tasks: group_dict_create.yml
tasks:
- debug:
var: my_packages
Declare variables in group_vars/all/group_dict_vars.yml
shell> cat group_vars/all/group_dict_vars.yml
group_vars_dir: "{{ inventory_dir }}/group_vars"
group_names_all: "{{ ansible_play_hosts_all|
map('extract', hostvars, 'group_names')|
flatten|unique }}"
group_dict_str: |
{% for group in group_names_all %}
{{ group }}: {{ lookup('vars', 'groupvars_' ~ group) }}
{% endfor %}
_group_dict: "{{ group_dict_str|from_yaml }}"
my_packages: "{{ group_names|map('extract', group_dict, 'packages')|
flatten|unique }}"
group_vars_dir: The directory group_vars can be used either in the directory where the inventory or the playbook comes from. In this example, these directories are identical and we set it to inventory_dir. In the loop, the task include_vars will read all YAML and JSON files from group_vars/<group> and will store the variables in the dictionary groupvars_<group>, where <group> are items of group_names_all.
group_names_all: This is a list of all groups the hosts are members of. See group_names
group_dict_str: Create the string with the YAML structure of the dictionary
_group_dict: Convert the string to YAML
my_packages: Merge the lists of packages from the groups the host is a member of. If needed, use this variable as a pattern of how to merge other variables.
Create a block of tasks that creates the dictionary and writes the file
shell> cat group_dict_create.yml
- name: Create dictionary group_dict in group_vars/all/group_dict.yml
block:
- name: Create directory group_vars/all
file:
state: directory
path: "{{ group_vars_dir }}/all"
- include_vars:
dir: "{{ group_vars_dir }}/{{ item }}"
name: "groupvars_{{ item }}"
loop: "{{ group_names_all }}"
- debug:
var: _group_dict
when: debug|d(false)|bool
- name: Write group_dict to group_vars/all/group_dict.yml
copy:
dest: "{{ group_vars_dir }}/all/group_dict.yml"
content: |
group_dict:
{{ _group_dict|to_nice_yaml(indent=2)|indent(2) }}
- include_vars:
file: "{{ group_vars_dir }}/all/group_dict.yml"
delegate_to: localhost
run_once: true
when: group_dict is not defined or group_dict_refresh|d(false)|bool
If the dictionary group_dict does not exist (the file group_vars/all/group_dict.yml has not been created yet) it will create the dictionary, write it to the file group_vars/all/group_dict.yml, and include it in the play. You can refresh group_dict by setting group_dict_refresh=true if you change the variables in group_vars/<group>.
shell> cat group_vars/all/group_dict.yml
group_dict:
pc:
packages:
- keepassx
- lm-sensors
- hddtemp
servers:
packages:
- lftp
- mercurial
The results, stored in the variable my_packages, are merged lists of packages by groups
TASK [debug] *********************************************************
ok: [kerbel] =>
my_packages:
- keepassx
- lm-sensors
- hddtemp
- lftp
- mercurial
ok: [armen] =>
my_packages:
- keepassx
- lm-sensors
- hddtemp
ok: [kerzo] =>
my_packages:
- keepassx
- lm-sensors
- hddtemp
Notes:
Best practice is running group_dict_create.yml separately for all hosts and letting other playbooks use created group_vars/all/group_dict.yml
The framework described here is idempotent.
The framework should be easily extendable to other use cases of merging variables by groups.
Example. Add variable users to group_vars/<group>
shell> cat group_vars/pc/users.yml
users:
- alice
- bob
shell> cat group_vars/servers/users.yml
users:
- carol
- dave
, and add variable my_users to group_vars/all/group_dict_vars.yml
my_users: "{{ group_names|map('extract', group_dict, 'users')|flatten|unique }}"
Refresh the dictionary group_dict for all hosts
shell> ansible-playbook pb.yml -l all -e group_dict_refresh=true
gives
ok: [armen] =>
my_users:
- alice
- bob
ok: [kerbel] =>
my_users:
- alice
- bob
- carol
- dave
ok: [kerzo] =>
my_users:
- alice
- bob