I have an inventory look like this
host:
vars:
fruit:
- {type: melon, id: 1}
- {type: apple, id: 2}
currently I used to get id like below
list: >-
{{groups['host']|map('extract',hostvars,'fruits')|first}}
- set_fact:
fruit_id: "{{list[2].id}}"
How can I get the id value by using type equal to apple?
The task
- debug:
msg: "{{ list|
selectattr('type', '==', 'apple')|
map(attribute='id')|
list|first }}"
gives
"msg": "2"
The same task in the loop
- debug:
msg: "{{ list|
selectattr('type', '==', item)|
map(attribute='id')|
list|first }}"
loop:
- melon
- apple
gives
"msg": "1"
"msg": "2"
Related
Is there find a key deep inside a nested dictionaries and show the path to that key?
Example:
---
isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
I want to find the initial variable and print the path to it like author.fistname.initial
I would do something like this using a custom filter. E.g.:
def _findkey_helper(data, want, path=None):
if path is None:
path = []
if not data:
return False, None, None
if want in data:
return True, path + [want], data[want]
for k, v in data.items():
if isinstance(v, dict):
found, val, foundat = _findkey_helper(v, want, path=path + [k])
if found:
return (found, foundat, val)
return False, None, None
def filter_findkey(data, want):
"""Search a nested dictionary for a given key.
This filter recursively searches the nested dictionary structure `data`
for the key `want`. It returns the 3-tuple (found, path, value), where:
- `found` is True if the named key was found, False otherwise
- `path` is a dotted path to the named key, or `None` if `found` is False
- `value` is the corresponding value, or `None` if `found` is False
"""
found, path, val = _findkey_helper(data, want)
if found:
return True, ".".join(path), val
return False, None, None
class FilterModule:
def filters(self):
return {"findkey": filter_findkey}
Drop this into filter_plugins/findkey.py adjacent to your playbook, and use it like this:
- hosts: localhost
gather_facts: false
vars:
data:
isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
tasks:
- debug:
msg: "{{ data | findkey('initial') }}"
The above outputs:
TASK [debug] ********************************************************************************************
ok: [localhost] => {
"msg": "(True, 'author.firstname.initial', 'D')"
}
This only works with nested dictionary structures (that is, it won't search through lists), but necessary changes to support lists would be pretty simple.
The following playbook does the job using the to_paths lookup
---
- hosts: localhost
gather_facts: false
vars:
isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
needle: initial
tasks:
# Note: this is a necessary step to "fix" the value from the lookup.
# If you set this value in play/task/host/group variable,
# you will get a recursive loop error any time you use the resulting variable
- name: extract the info
set_fact:
matches: "{{ lookup('ansible.utils.to_paths', vars) | dict2items | selectattr('key', 'search', needle) }}"
- name: Overall info
debug:
msg: "I found {{ matches | length }} entries having '{{ needle }}' in their path"
- name: details
debug:
msg: "item at path {{ item.key }} has value: {{ item.value }}"
loop: "{{ matches }}"
Example run:
$ ansible-playbook playbook.yml
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [extract the info] ****************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Overall info] ********************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "I found 1 entries having 'initial' in their path"
}
TASK [details] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'key': 'author.firstname.initial', 'value': 'D'}) => {
"msg": "item at path author.firstname.initial has value: D"
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
And to be ahead of some possible comments: this works with lists too. Replacing the var author above with:
authors:
- lastname: Doe
firstname:
name: John
initial: D
- lastname: Perez
firstname:
name: Juan
initial: J
gives with the exact same playbook:
TASK [Overall info] ********************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "I found 2 entries having 'initial' in their path"
}
TASK [details] *************************************************************************************************************************************************************************************************************************
ok: [localhost] => (item={'key': 'authors[0].firstname.initial', 'value': 'D'}) => {
"msg": "item at path authors[0].firstname.initial has value: D"
}
ok: [localhost] => (item={'key': 'authors[1].firstname.initial', 'value': 'J'}) => {
"msg": "item at path authors[1].firstname.initial has value: J"
}
Q: "Find the initial variable and print the path to it like author.fistname.initial"
A: Use the dictionaries below. There might be more paths. For example, given the data
data:
- isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
- isbn: 123-456-333
author:
lastname: Doe
firstname:
name: Alice
initial: C
- isbn: 123-456-444
author:
lastname: Doe
firstname:
name: Bob
initial: D
The task creates the dictionary initials
- name: Create dictionary initials
set_fact:
initials: "{{ initials|d({})|combine({item: authors}) }}"
loop: "{{ data|json_query('[].*.*.initial')|flatten|unique }}"
vars:
authors: "{{ data|
selectattr('author.firstname.initial', 'eq', item)|
json_query('[].join(`,`,[author.lastname,
author.firstname.name,
author.firstname.initial])') }}"
gives
initials:
C:
- Doe,Alice,C
D:
- Doe,John,D
- Doe,Bob,D
The task creates the dictionary paths
- name: Create dictionary paths
set_fact:
paths: "{{ paths|d({})|combine(_item, list_merge='append') }}"
loop: "{{ data }}"
vars:
_paths: "{{ lookup('ansible.utils.to_paths', item) }}"
_item: "{{ dict(_paths|dict2items|json_query('[].[value, [key]]')) }}"
gives
paths:
123-456-222:
- isbn
123-456-333:
- isbn
123-456-444:
- isbn
Alice:
- author.firstname.name
Bob:
- author.firstname.name
C:
- author.firstname.initial
D:
- author.firstname.initial
- author.firstname.initial
Doe:
- author.lastname
- author.lastname
- author.lastname
John:
- author.firstname.name
Filter unique paths. Create dictionary paths_unique
paths_keys: "{{ paths.keys()|list }}"
paths_vals: "{{ paths.values()|map('unique')|list }}"
paths_unique: "{{ dict(paths_keys|zip(paths_vals)) }}"
gives
paths_unique:
123-456-222:
- isbn
123-456-333:
- isbn
123-456-444:
- isbn
Alice:
- author.firstname.name
Bob:
- author.firstname.name
C:
- author.firstname.initial
D:
- author.firstname.initial
Doe:
- author.lastname
John:
- author.firstname.name
Example of a complete playbook for testing
- hosts: localhost
vars:
data:
- isbn: 123-456-222
author:
lastname: Doe
firstname:
name: John
initial: D
- isbn: 123-456-333
author:
lastname: Doe
firstname:
name: Alice
initial: C
- isbn: 123-456-444
author:
lastname: Doe
firstname:
name: Bob
initial: D
paths_keys: "{{ paths.keys()|list }}"
paths_vals: "{{ paths.values()|map('unique')|list }}"
paths_unique: "{{ dict(paths_keys|zip(paths_vals)) }}"
tasks:
- name: Create dictionary initials
set_fact:
initials: "{{ initials|d({})|combine({item: authors}) }}"
loop: "{{ data|json_query('[].*.*.initial')|flatten|unique }}"
vars:
authors: "{{ data|
selectattr('author.firstname.initial', 'eq', item)|
json_query('[].join(`,`,[author.lastname,
author.firstname.name,
author.firstname.initial])') }}"
- debug:
var: initials
- name: Create dictionary paths
set_fact:
paths: "{{ paths|d({})|combine(_item, list_merge='append') }}"
loop: "{{ data }}"
vars:
_paths: "{{ lookup('ansible.utils.to_paths', item) }}"
_item: "{{ dict(_paths|dict2items|json_query('[].[value, [key]]')) }}"
- debug:
var: paths
- debug:
var: paths_unique
I Have 2 dictionary:
- Test1:
1: pass
2: fail
3: pass
- Test2:
1.1.1.1: val1
2.2.2.2: val2
3.3.3.3: val3
Condition is when Test1.value contians fail
- name: test
debug:
msg: "{{item.1.value}} {{item.1.key}} {{item.0.key}} {{item.0.value}}"
with_together:
- "{{Test1}}"
- "{{Test2}}"
when: item.0.value == "fail"
This is not working as expected unable to get both key and value of 2 dict in one loop
In when statement you must to use item.0 or item.1 to evaluate the condition. And I recommend you use a list in with_together loop and if you are using a variable you have to use braces {{ variable }} .
Try as below:
- name: test
debug:
msg: "{{item.1 }}"
with_together:
- "{{ Test1.values() | list }}"
- "{{ Test2.values() | list }}"
when: item.0 == "fail"
You'll get
TASK [test] *******************************************************************************************************************************************************************************************************
skipping: [127.0.0.1] => (item=['pass', 'val1'])
ok: [127.0.0.1] => (item=['fail', 'val2']) => {
"msg": "val2"
}
skipping: [127.0.0.1] => (item=['pass', 'val3'])
I achieved this by :
converting dict to list using filter -> |list
since
both dict of same size I was able to get data of both dict in single loop:
- name: test
debug:
msg: "{{item.0}} {{item.1}} {{item.2}} {{item.3}}"
with_together:
- "{{ Test1.values() | list }}"
- "{{ Test2.values() | list }}"
- "{{ Test1.keys() | list }}"
- "{{ Test2.keys() | list }}"
when: item.0 == "fail"
Trying to create a dictionary per item and merge them
---
- name: TestingLab
hosts: localhost
gather_facts: False
tasks:
- name: Hello Vars
set_fact:
two_nums:
- 1
- 2
- name: create empty dict
set_fact:
ids: {}
- name: Merge all
vars:
single_entry: "{ '{{ item }}': {{ item }} }"
set_fact:
ids: "{{ ids | combine(single_entry) }}"
with_items: "{{ two_nums }}"
- name: Print Result
debug:
msg: "{{ ids }}"
I thought I followed the right guidelines but I seem to be getting only the last item afterwards:
ok: [localhost] => {
"msg": {
"2": 2
} }
I tried replacing the single_entry with the expression in vars but it does not run.
Is there a different syntax to get this done?
EDIT: version info
ansible-playbook 2.5.1
python version = 2.7.17 [GCC 7.5.0]
Try the filters dict and zip. The zip is available since 2.3, e.g.
- set_fact:
d2: "{{ dict(two_nums|zip(two_nums)) }}"
- debug:
var: d2
- debug:
var: d2|type_debug
gives
d2:
1: 1
2: 2
d2|type_debug: dict
If this does not work try Jinja and the filter from_yaml, e.g.
- hosts: localhost
vars:
two_nums:
- 1
- 2
l1: |-
{% for i in two_nums %}
{{ i }}: {{ i }}
{% endfor %}
tasks:
- set_fact:
d1: "{{ l1|from_yaml }}"
- debug:
var: d1
- debug:
var: d1|type_debug
gives the same result
d1:
1: 1
2: 2
d1|type_debug: dict
If you need the keys to be strings quote it, e.g.
l1: |-
{% for i in two_nums %}
"{{ i }}": {{ i }}
{% endfor %}
gives
d1:
'1': 1
'2': 2
In the first case, map the list's items to string, e.g.
- set_fact:
d2: "{{ dict(two_nums|map('string')|zip(two_nums)) }}"
gives the same result
d2:
'1': 1
'2': 2
I can't reproduce the behavior you're describing. Running your
playbook verbatim, I get as output:
TASK [Print Result] **************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": {
"1": 1,
"2": 2
}
}
I'm using Ansible core 2.11.2, but I've also tested your playbook with Ansible 2.9.20 and I get the same output.
I would probably drop the set_fact task, and also change how you're
setting single_entry:
- name: TestingLab
hosts: localhost
gather_facts: False
tasks:
- name: Hello Vars
set_fact:
two_nums:
- 1
- 2
- name: Merge all
vars:
ids: {}
single_entry: "{{ {item: item} }}"
set_fact:
ids: "{{ ids | combine(single_entry) }}"
with_items: "{{ two_nums }}"
- name: Print Result
debug:
msg: "{{ ids }}"
In this version, the template expression is returning a dictionary,
and only requires a single set of Jinja template markers. I'm curious
if this version behaves any differently for you.
Here is my json output:
{
"kind": [
{
"inventory": "",
"inventory_sources": "",
"job_templates": "",
"workflow_job_templates": "104"
},
{
"inventory": "",
"inventory_sources": "",
"job_templates": "114",
"workflow_job_templates": ""
},
{
"inventory": "24",
"inventory_sources": "",
"job_templates": "",
"workflow_job_templates": ""
},
{
"inventory": "",
"inventory_sources": "108",
"job_templates": "",
"workflow_job_templates": ""
}
]
}
I'd like to display all items name that contain a specific value. For example, for a search value of 104 I want to get the key name workflow_job_templates
I tested some syntaxes without any success:
- debug:
msg: "104 is {{kind|json_query(query)}}"
vars:
query: "[?*==`104`].workflow_job_templates"
I know it is wrong but can someone tell me how he'd do for himself?
json_query could be part of the equation for your solution but is really not needed here.
Explanation of the below piece of code:
Apply the dict2items filter to each element of your list. This transforms each mapping to a list of {key: "key", value: "value"} pairs
Flatten the given list so we get all those elements to a single top level
Select elements having a value of '104' only
Extract the key attribute of each element in a list
Make that list unique and sort it.
- name: Display all element having a value of 104
debug:
msg: "{{ kind | map('dict2items') | flatten
| selectattr('value', '==', '104')
| map(attribute='key') | unique | sort }}"
Please note that this solution will give you a result if the same key name has different values but one of them is `104. With your above data the result is:
TASK [Display all element having a value of 104] ***************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"workflow_job_templates"
]
}
(Update)
The task below
- debug:
msg: "{{ item }} {{ kind|
map('dict2items')|
map('json_query', query)|
flatten }}"
loop: [104, 114, 108, 24]
vars:
query: "[?to_string(value) == to_string('{{ item }}')].key"
gives
msg: 104 ['workflow_job_templates']
msg: 114 ['job_templates']
msg: 108 ['inventory_sources']
msg: 24 ['inventory']
(For the record. Brute-force approach)
Create a unique list of the keys
- set_fact:
my_keys: "{{ my_keys|default([]) + item.keys()|list }}"
loop: "{{ kind }}"
- set_fact:
my_keys: "{{ my_keys|unique }}"
gives
my_keys:
- inventory
- inventory_sources
- job_templates
- workflow_job_templates
Create a dictionary with all values
- set_fact:
my_dict: "{{ my_dict|default({})|combine({item: values}) }}"
loop: "{{ my_keys }}"
vars:
query: "[].{{ item }}"
values: "{{ kind|json_query(query) }}"
gives
my_dict:
inventory:
- ''
- ''
- '24'
- ''
inventory_sources:
- ''
- ''
- ''
- '108'
job_templates:
- ''
- '114'
- ''
- ''
workflow_job_templates:
- '104'
- ''
- ''
- ''
Then search the dictionary. For example
- debug:
msg: "{{ item }} {{ my_dict|dict2items|json_query(query) }}"
loop: [104, 114, 108, 24]
vars:
query: "[?value[?contains(#, '{{ item }}')]].key"
gives
msg: 104 ['workflow_job_templates']
msg: 114 ['job_templates']
msg: 108 ['inventory_sources']
msg: 24 ['inventory']
The correct alternative of selectattr with json_query is:
- debug:
msg: "{{ kind | map('dict2items') | flatten | json_query(query)}}"
vars:
- query: "[?value == `\"104\"`].key"
If you really just want to use json_query()
---
- name: PLAYBOOK Filtering
hosts: localhost # run locally
tasks:
- name: Create json
set_fact:
kind: '{{ lookup("file", "kind.json") }}'
- name: check the var was created properly
debug:
var: kind
- name: output the element that matches 104
debug:
msg: "{{ kind | json_query(\"kind[?workflow_job_templates=='104'].workflow_job_templates\") }}"
- name:
set_fact:
output: "{{ kind | json_query(\"kind[?workflow_job_templates=='104'].workflow_job_templates\") }}"
Output
TASK [output the element that matches 104] *************************************************************************************************************
ok: [localhost] => {
"msg": [
"104"
]
}
I have a list of dictionaries in an ansible variable. Some dictionaries have the same value in the field 'id and the field 'name' while they differ in other key value pairs (that are not important for me). I want to filter out all those dictionaries that are "duplicates" regarding the 'name' and 'id' fields.
Example:
[{
"name": "abc",
"id": "123456",
"other_key": "unimportant value"
},
{
"name": "abc",
"id": "123456",
"other_key": "another unimportant value"
},
{
"name": "bcd",
"id": "789012",
"other_key": "unimportant value"
}]
Desired result:
[{
"name": "abc",
"id": "123456"
},
{
"name": "bcd",
"id": "789012"
}]
How can I achieve this in Ansible? (the 'other_key' variable does not necessarily have to be discarded it could also be e.g. just the first occurrence, it just does not matter).
I already produced a list of unique ids with:
{{ mydictionaries | map(attribute='id') | unique | list }}
But how do I filter the list of dictionaries with this?
Q: "Filter out dictionaries that are "duplicates" regarding the 'name' and 'id' fields."
Update.
A: Given the list
my_list:
- {id: '123456', name: abc, other_key: unimportant value}
- {id: '123456', name: abc, other_key: another unimportant value}
- {id: '789012', name: bcd, other_key: unimportant value}
declare the variables
my_hash: "{{ my_list|json_query('[].[name, id]')|
map('join')|
map('hash')|
map('community.general.dict_kv', 'hash') }}"
my_list2: "{{ my_list|zip(my_hash)|map('combine') }}"
my_dict3: "{{ dict(my_list2|groupby('hash')|
map('last')|
map('first')|
json_query('[].[name, id]')) }}"
my_list3: "{{ my_dict3|dict2items(key_name='name', value_name='id') }}"
give
my_hash:
- {hash: 370194ff6e0f93a7432e16cc9badd9427e8b4e13}
- {hash: 370194ff6e0f93a7432e16cc9badd9427e8b4e13}
- {hash: f3814d5b43a4d67d7636ec64be828d82a92eedbb}
my_list2:
- hash: 370194ff6e0f93a7432e16cc9badd9427e8b4e13
id: '123456'
name: abc
other_key: unimportant value
- hash: 370194ff6e0f93a7432e16cc9badd9427e8b4e13
id: '123456'
name: abc
other_key: another unimportant value
- hash: f3814d5b43a4d67d7636ec64be828d82a92eedbb
id: '789012'
name: bcd
other_key: unimportant value
my_dict3:
abc: '123456'
bcd: '789012'
my_list3:
- {id: '123456', name: abc}
- {id: '789012', name: bcd}
Optionally, convert id from string to number
my_dict3: "{{ dict(my_list2|groupby('hash')|
map('last')|
map('first')|
json_query('[].[name, to_number(id)]')) }}"
my_list3: "{{ my_dict3|dict2items(key_name='name', value_name='id') }}"
give
my_dict3:
abc: 123456
bcd: 789012
my_list3:
- {id: 123456, name: abc}
- {id: 789012, name: bcd}
Example of a complete playbook for testing
- hosts: localhost
vars:
my_list:
- {id: '123456', name: abc, other_key: unimportant value}
- {id: '123456', name: abc, other_key: another unimportant value}
- {id: '789012', name: bcd, other_key: unimportant value}
my_hash: "{{ my_list|json_query('[].[name, id]')|
map('join')|
map('hash')|
map('community.general.dict_kv', 'hash') }}"
my_list2: "{{ my_list|zip(my_hash)|map('combine') }}"
my_dict3: "{{ dict(my_list2|groupby('hash')|
map('last')|
map('first')|
json_query('[].[name, to_number(id)]')) }}"
# json_query('[].[name, id]')) }}"
my_list3: "{{ my_dict3|dict2items(key_name='name', value_name='id') }}"
tasks:
- debug:
var: my_list|to_yaml
- debug:
var: my_hash|to_yaml
- debug:
var: my_list2
- debug:
var: my_dict3
- debug:
var: my_list3|to_yaml
Origin.
Given the data is stored in the variable my_list, let's add the hash attribute, created from the name and id attributes, to the list. For example,
- set_fact:
my_list2: "{{ my_list2|default([]) +
[item|combine({'hash': (item.name ~ item.id)|hash})] }}"
loop: "{{ my_list }}"
- debug:
var: my_list2
give
my_list2:
- hash: 370194ff6e0f93a7432e16cc9badd9427e8b4e13
id: '123456'
name: abc
other_key: unimportant value
- hash: 370194ff6e0f93a7432e16cc9badd9427e8b4e13
id: '123456'
name: abc
other_key: another unimportant value
- hash: f3814d5b43a4d67d7636ec64be828d82a92eedbb
id: '789012'
name: bcd
other_key: unimportant value
Subsequent use groupby filter and select the required attributes. For example,
- set_fact:
my_list3: "{{ my_list3|default([]) +
[{'name': item.1.0.name, 'id': item.1.0.id}] }}"
loop: "{{ my_list2|groupby('hash') }}"
- debug:
var: my_list3
give
my_list3:
- {id: '123456', name: abc}
- {id: '789012', name: bcd}
Example of a complete playbook for testing
- hosts: localhost
vars:
my_list:
- {id: '123456', name: abc, other_key: unimportant value}
- {id: '123456', name: abc, other_key: another unimportant value}
- {id: '789012', name: bcd, other_key: unimportant value}
tasks:
- set_fact:
my_list2: "{{ my_list2|default([]) +
[item|combine({'hash': (item.name ~ item.id)|hash})] }}"
loop: "{{ my_list }}"
- debug:
var: my_list2
- set_fact:
my_list3: "{{ my_list3|default([]) +
[{'name': item.1.0.name, 'id': item.1.0.id}] }}"
loop: "{{ my_list2|groupby('hash') }}"
- debug:
var: my_list3|to_yaml
You can filter only the desired keys from your map list with json_query and then apply the unique filter.
{{ mydictionnaries | json_query('[].{"name": name, "id": id}') | unique }}
Below a proof of concept playbook. Please note in the above doc that json_query requires pip install jmespath on the ansible controller.
---
- name: Unique filtered dictionaries list example
hosts: localhost
gather_facts: false
vars:
mydictionaries: [{"name": "abc","id": "123456","other_key": "unimportant value"},{"name": "abc","id": "123456","other_key": "another unimportant value"},{"name": "bcd","id": "789012","other_key": "unimportant value"}]
tasks:
- name: Filter out list as wanted
debug:
msg: >-
{{
mydictionaries
| json_query('[].{"name": name, "id": id}')
| unique
}}
which gives
PLAY [Unique filtered dictionaries list example] *************************************************************************************************************************************************************************************************************************************
TASK [Filter out list as wanted] ****************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"id": "123456",
"name": "abc"
},
{
"id": "789012",
"name": "bcd"
}
]
}
PLAY RECAP **************************************************************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0