Here's something I think should be straight-forward in Ansible, but I'm having trouble finding a canonical best practice.
In Ansible, I have a list of dicts in YAML:
config:
- id: x
value: X
- id: y
value: Y
I want to generate a list of strings that can be passed as a whole to the mysql_query module.
I've tried various elegant ways of doing this using Ansible filters, but that best I've come up with is to generate a list of newline-separate strings in a single template that iterates over config and then trim | split('\n') the resulting single string.
Template:
{% for item in config %}
{{ item.id }} => {{ item.value }}
{% endfor %}
Playbook task:
set_fact:
converted_items: "{{ lookup('template', './template.j2') | trim | split('\n') }}"
But this feels like a kludge.
What am I missing here?
[Note this is a canned example, to keep things simple.]
if you want to transform list of dicts to list of strings, just use loop and union:
- name: vartest
hosts: localhost
vars:
config:
- id: x
value: X
- id: y
value: Y
tasks:
- name: transform value
set_fact:
result: "{{ result | default([]) | union([item.id ~ ' => ' ~ item.value]) }}"
loop: "{{ config }}"
- name: display result
debug:
var: result
equivalent of union: result | default([]) + [item.id ~ ' => ' ~ item.value]
result:
ok: [localhost] => {
"result": [
"x => X",
"y => Y"
]
}
You don't have to use a file. Put the Jinja template into the local vars, e.g.
- set_fact:
converted_items: "{{ _converted_items|trim|split('\n') }}"
vars:
_converted_items: |
{% for item in config %}
{{ item.id }} => {{ item.value }}
{% endfor %}
gives
converted_items:
- x => X
- y => Y
There are many options of how to transform the data, e.g. the task below gives the same result
- set_fact:
converted_items: "{{ _keys|zip(_vals)|map('join', ' => ')|list }}"
vars:
_keys: "{{ config|map(attribute='id')|list }}"
_vals: "{{ config|map(attribute='value')|list }}"
Related
Trying to create a dictionary per item and merge them
---
- name: TestingLab
hosts: localhost
gather_facts: False
tasks:
- name: Hello Vars
set_fact:
two_nums:
- 1
- 2
- name: create empty dict
set_fact:
ids: {}
- name: Merge all
vars:
single_entry: "{ '{{ item }}': {{ item }} }"
set_fact:
ids: "{{ ids | combine(single_entry) }}"
with_items: "{{ two_nums }}"
- name: Print Result
debug:
msg: "{{ ids }}"
I thought I followed the right guidelines but I seem to be getting only the last item afterwards:
ok: [localhost] => {
"msg": {
"2": 2
} }
I tried replacing the single_entry with the expression in vars but it does not run.
Is there a different syntax to get this done?
EDIT: version info
ansible-playbook 2.5.1
python version = 2.7.17 [GCC 7.5.0]
Try the filters dict and zip. The zip is available since 2.3, e.g.
- set_fact:
d2: "{{ dict(two_nums|zip(two_nums)) }}"
- debug:
var: d2
- debug:
var: d2|type_debug
gives
d2:
1: 1
2: 2
d2|type_debug: dict
If this does not work try Jinja and the filter from_yaml, e.g.
- hosts: localhost
vars:
two_nums:
- 1
- 2
l1: |-
{% for i in two_nums %}
{{ i }}: {{ i }}
{% endfor %}
tasks:
- set_fact:
d1: "{{ l1|from_yaml }}"
- debug:
var: d1
- debug:
var: d1|type_debug
gives the same result
d1:
1: 1
2: 2
d1|type_debug: dict
If you need the keys to be strings quote it, e.g.
l1: |-
{% for i in two_nums %}
"{{ i }}": {{ i }}
{% endfor %}
gives
d1:
'1': 1
'2': 2
In the first case, map the list's items to string, e.g.
- set_fact:
d2: "{{ dict(two_nums|map('string')|zip(two_nums)) }}"
gives the same result
d2:
'1': 1
'2': 2
I can't reproduce the behavior you're describing. Running your
playbook verbatim, I get as output:
TASK [Print Result] **************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": {
"1": 1,
"2": 2
}
}
I'm using Ansible core 2.11.2, but I've also tested your playbook with Ansible 2.9.20 and I get the same output.
I would probably drop the set_fact task, and also change how you're
setting single_entry:
- name: TestingLab
hosts: localhost
gather_facts: False
tasks:
- name: Hello Vars
set_fact:
two_nums:
- 1
- 2
- name: Merge all
vars:
ids: {}
single_entry: "{{ {item: item} }}"
set_fact:
ids: "{{ ids | combine(single_entry) }}"
with_items: "{{ two_nums }}"
- name: Print Result
debug:
msg: "{{ ids }}"
In this version, the template expression is returning a dictionary,
and only requires a single set of Jinja template markers. I'm curious
if this version behaves any differently for you.
I am trying to run a nested for loop in order to retrieve a nested value.
I would like to retrieve some_value_4 when some_value_3 matches a criteria that's predefined.
{
"some_dict": {
"some_key_0": "value_0",
"some_key_1": "value_1"
},
"testval_dict": {
"test_key_0": "some_value_0",
"test_key_1": "some_value_1",
"test_key_2": "some_value_2",
"test_key_3": "some_value_3",
"test_key_4": "some_value_4"
}
}
The playbook:
- hosts: localhost
tasks:
- set_fact:
mydict: "{{ lookup('file', '/tmp/file.json' ) | from_json }}"
- debug:
msg: |
"{% for item in mydict %}
{{ item }}
{% endfor %}"
when run, it alreay looks like dict names are treated as string and nothing more:
ansible-playbook /tmp/test_playbook.yml -c local -i ', localhost'
TASK [debug] ******************************************************************
ok: [localhost] => {}
MSG:
" somee_dict
testval_dict
"
Then when I add an itme.key to the debug task, the playbook fails:
MSG:
The task includes an option with an undefined variable. The error was: 'str object' has no attribute 'value'
Thank you.
edit for clarification
In the real example, I will not know the names of the dicts, so I cannot use some_dict or testval_dict, that is why I'm trying to go over this data source in an item.key or item.value form.
Q: "{% for item in mydict %} ... dict names are treated as string and nothing more."
A: This is correct. A dictionary, when evaluated as a list, returns the list of its keys. See some examples below
- debug:
msg: "{{ mydict.keys()|list }}"
- debug:
msg: "{{ mydict[item] }}"
loop: "{{ mydict.keys()|list }}"
- debug:
msg: "{{ mydict|difference(['testval_dict']) }}"
give
msg:
- some_dict
- testval_dict
msg:
some_key_0: value_0
some_key_1: value_1
msg:
test_key_0: some_value_0
test_key_1: some_value_1
test_key_2: some_value_2
test_key_3: some_value_3
test_key_4: some_value_4
msg:
- some_dict
See How to iterate through a list of dictionaries in Jinja template?
If you need to loop over the dictionary, you can use with_dict loop functionality. This way, if you loop over mydict and get item.key you will get somee_dict and testval_dict.
tasks:
- set_fact:
mydict: "{{ lookup('file', '/tmp/file.json')|from_json }}"
# This will get the top level dictionaries somee_dict and testval_dict
- debug:
var: item.key
with_dict: "{{ mydict }}"
And if you get item.value of mydict you will get the sub-dictionary:
- debug:
var: item.value
with_dict: "{{ mydict }}"
Will produce (showing output only for testval_dict):
"item.value": {
"test_key_0": "some_value_0",
"test_key_1": "some_value_1",
"test_key_2": "some_value_2",
"test_key_3": "some_value_3",
"test_key_4": "some_value_4"
}
I have a json structure like this:
{
"group_a": {
"setting_one": "text one",
"setting_two": "text two"
},
"group_b": {
"setting_three": "text three",
"setting_four": "text four"
}
}
And I need to do iterate over the dicts inside the dicts and create tasks, kinda like this:
- name: Task group_a setting_one
...
- name: Task group_a setting_two
...
- name: Task group_b setting_three
...
- name: Task group_b setting_four
...
I was hoping to make something like this:
- name: Insert {{ group }} {{ setting }}
ini_file:
path: settings.ini
section: "{{ group }}"
option: "{{ setting }}"
value: "{{ setting_value }}"
with_nested:
- "{{ data.keys() }}" # Can I get this key and use in the next loop?
- "{{ data.key.keys() }}"
How can I do this? I have been messing about with loop, with_items, with_dict etc, but I can't manage to get it exactly the way I need. I also tried with_nested, but that does not seem to do what I need either.
Kind regards, Jonas
Let's reduce the structure to one with the same piece of information
my_data:
group_a:
- one
- two
group_b:
- three
- four
Then the play below
- debug:
msg: "{{ item.0.key }} setting_{{ item.1 }}"
with_subelements:
- "{{ my_data|dict2items }}"
- value
gives:
"msg": "group_b setting_three"
"msg": "group_b setting_four"
"msg": "group_a setting_one"
"msg": "group_a setting_two"
It's not straightforward to use the original data. The play below separates the keys and values
tasks:
- set_fact:
my_groups: "{{ my_data.keys()}}"
- debug:
msg: "{{ item }} {{ my_data|dict2items|json_query(my_query)|to_yaml }}"
loop: "{{ my_groups }}"
vars:
my_query: "[?key=='{{ item }}'].value"
and gives:
"msg": "group_b - - {setting_three: text three}\n - {setting_four: text four}\n"
"msg": "group_a - - {setting_one: text one}\n - {setting_two: text two}\n"
Multi-step loops are needed to proceed. For example the play below
- set_fact:
my_data1: |
{% for group, values in my_data.items() %}
{% for value in values %}
{% for item_key, item_val in value.items() %}
Task {{ group }} {{ item_key }},
{% endfor %}{% endfor %}{% endfor %}
- debug:
msg: "{{ my_data1.split('\n') }}"
gives:
"msg": [
"Task group_b setting_three,",
"Task group_b setting_four,",
"Task group_a setting_one,",
"Task group_a setting_two,",
""
]
I am trying to create a simple playbook task traversing over multiple dict items(v1) and create a new list var (list_var) with specific item in it, but it is not working list_var is only showing me one element, can anyone please suggest what am i missing?
if i do this:
set_fact:
list_var: "{{ v1.stdout }}"
with_items: "{{ v1.items }}"
Values in v1 are returned by simple shell output and have values from different hosts like hostname
Probably something list this?
- command: echo "{{ item }}"
register: v1
with_sequence: start=0 end=3
- set_fact:
list_var: "{{ list_var|default([]) }} + [ {{ item }} ]"
loop: "{{ v1.results | json_query('[].stdout') }}"
- debug: var=list_var
Which produces this list
TASK [debug] **********
ok: [localhost] => {
"list_var": [
0,
1,
2,
3
]
}
I want to add keys to a dictionary when using set_fact with with_items. This is a small POC which will help me complete some other work. I have tried to generalize the POC so as to remove all the irrelevant details from it.
When I execute following code it is shows a dictionary with only one key that corresponds to the last item of the with_items. It seems that it is re-creating a new dictionary or may be overriding an existing dictionary for every item in the with_items. I want a single dictionary with all the keys.
Code:
---
- hosts: localhost
connection: local
vars:
some_value: 12345
dict: {}
tasks:
- set_fact: {
dict: "{
{{ item }}: {{ some_value }}
}"
}
with_items:
- 1
- 2
- 3
- debug: msg="{{ dict }}"
This can also be done without resorting to plugins, tested in Ansible 2.2.
---
- hosts: localhost
connection: local
vars:
some_value: 12345
dict: {}
tasks:
- set_fact:
dict: "{{ dict | combine( { item: some_value } ) }}"
with_items:
- 1
- 2
- 3
- debug: msg="{{ dict }}"
Alternatively, this can be written without the complex one-liner with an include file.
tasks:
- include: append_dict.yml
with_items: [1, 2, 3]
append_dict.yml:
- name: "Append dict: define helper variable"
set_fact:
_append_dict: "{ '{{ item }}': {{ some_value }} }"
- name: "Append dict: execute append"
set_fact:
dict: "{{ dict | combine( _append_dict ) }}"
Output:
TASK [debug]
*******************************************************************
ok: [localhost] => {
"msg": {
"1": "12345",
"2": "12345",
"3": "12345"
}
}
Single quotes ' around {{ some_value }} are needed to store string values explicitly.
This syntax can also be used to append from a dict elementwise using with_dict by referring to item.key and item.value.
Manipulations like adding pre- and postfixes or hashes can be performed in the same step, for example
set_fact:
dict: "{{ dict | combine( { item.key + key_postfix: item.value + '_' + item.value | hash('md5') } ) }}"
Use a filter plugin.
First, make a new file in your ansible base dir called filter_plugins/makedict.py.
Now create a new function called "makedict" (or whatever you want) that takes a value and a list and returns a new dictionary where the keys are the elements of the list and the value is always the same.
class FilterModule(object):
def filters(self):
return { 'makedict': lambda _val, _list: { k: _val for k in _list } }
Now you can use the new filter in the playbook to achieve your desired result:
- hosts: 127.0.0.1
connection: local
vars:
my_value: 12345
my_keys: [1, 2, 3]
tasks:
- set_fact: my_dict="{{ my_value | makedict(my_keys) }}"
- debug: msg="{{ item.key }}={{ item.value }}"
with_dict: "{{my_dict}}"
You can customize the location of the filter plugin using the filter_plugins option in ansible.cfg.
this does not seems to work any more on ansible 2.5
---
- hosts: localhost
connection: local
vars:
some_value: 12345
dict: {}
tasks:
- set_fact:
dict: "{{ dict | combine( { item: some_value } ) }}"
with_items:
- 1
- 2
- 3
- debug: msg="{{ dict }}"
returns only last value {"dict":{"3": "some value"}}
I suggest you could do this :
- set_fact:
__dict: |
{% for item in [1,2,3] %}
{{item}}: "value"
{% endfor %}
- set_fact:
final_dict: "{{__dict|from_yaml}}"
- debug:
var: final_dict
Another solution could be this one, tested in Ansible 2.9.6.
This solutions adds the extra benefit that you do not have to declare _dict beforehand in vars section. This is achieved by the | default({}) pipe which ensures that the loop will not fail in the first iteration when _dict is empty.
In addition, the renaming of dict to _dict is necessary since the dict is a special keyword reserved for <class 'dict'>. Referenced (unfortunately only at devel branch yet) here.
---
- hosts: localhost
connection: local
vars:
some_value: 12345
tasks:
- set_fact:
_dict: "{{ _dict | default({}) | combine( { item: some_value } ) }}"
with_items:
- 1
- 2
- 3
- debug: msg="{{ _dict }}"