ansible merge two lists based on an attribute - ansible

Question

I have a role in my playbook that generates two lists using set_fact.
The two facts are used in different tasks.
I then need to merge them for a final task.
list1:
- name: alice
roles: ['role1', 'role2']
- name: bob
roles: ['role1']
list2:
- name: alice
roles: ['role3']
- name: charlie
roles: ['role2']
For my final task I need the output to be:
list3:
- name: alice
roles: ['role1', 'role2', 'role3']
- name: bob
roles: ['role1']
- name: charlie
roles: ['role2']

Answer 1

I asked about lists vs dictionaries in the comment because of the impact it will have on the solution. If you were to restructure your data like this:
dict1:
alice: ['role1', 'role2']
bob: ['role1']
dict2:
alice: ['role3']
charlie: ['role2']
Then your solution becomes:
- set_fact:
dict3: >-
{{
dict3|default([])|combine({
item: (dict1[item]|default([]) + dict2[item]|default([]))|unique
})
}}
loop: "{{ (dict1.keys()|list + dict2.keys()|list)|unique }}"
- debug:
var: dict3
Which outputs:
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => {
"dict3": {
"alice": [
"role1",
"role2",
"role3"
],
"bob": [
"role1"
],
"charlie": [
"role2"
]
}
}
If you're stuck with using lists, we can improve upon the json_query solution that Zeitounator suggested:
- set_fact:
list3: >-
{{
list3|default([]) + [{
'name': item,
'roles': (list1|json_query('[?name==`' + item + '`].roles[]') + list2|json_query('[?name==`' + item + '`].roles[]'))|unique
}]
}}
loop: "{{ (list1|json_query('[].name') + list2|json_query('[].name'))|unique }}"
- debug:
var: list3
This produces your desired output:
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => {
"list3": [
{
"name": "alice",
"roles": [
"role1",
"role2",
"role3"
]
},
{
"name": "bob",
"roles": [
"role1"
]
},
{
"name": "charlie",
"roles": [
"role2"
]
}
]
}

Answer 2

This is a solution in plain ansible. If it becomes out of control because of your datastructure growth, you should consider writing your own filter (example)
---
- name: demo playbook for deeps dictionary merge
hosts: localhost
gather_facts: false
vars:
list1:
- name: alice
roles: ['role1', 'role2']
- name: bob
roles: ['role1']
list2:
- name: alice
roles: ['role3']
- name: charlie
roles: ['role2']
tasks:
- debug:
msg: >-
roles for {{ item }}:
{{
(list1 | json_query("[?name == '" + item +"'].roles")).0 | default([])
+
(list2 | json_query("[?name == '" + item +"'].roles")).0 | default([])
}}
loop: >-
{{
(
list1 | json_query("[].name")
+
list2 | json_query("[].name")
)
| unique
}}
Which gives:
PLAY [demo playbook for deeps dictionary merge] ************************************************************************************************************************************************************************************************************************
TASK [debug] ************************************************************************************************************************************************************************************************************************************************************
Wednesday 24 April 2019 16:23:07 +0200 (0:00:00.046) 0:00:00.046 *******
ok: [localhost] => (item=alice) => {
"msg": "roles for alice: ['role1', 'role2', 'role3']"
}
ok: [localhost] => (item=bob) => {
"msg": "roles for bob: ['role1']"
}
ok: [localhost] => (item=charlie) => {
"msg": "roles for charlie: ['role2']"
}