Building up a dictionary/hash with lists - ansible

I am attempting to build a dictionary but cannot grasp how jinja2 interpolates variables.
I want to set a specific item in the array (for example item[0]) to a specific key-value dictionary item.
- set_fact:
nodes:
- node1
- node2
- set_fact:
list_one:
- f-one
- f-two
- set_fact:
list_two:
- n-one
- n-two
what I want:
- set_fact:
**node_dict:
node1:
labels:
f-one: n-one
node2:
labels:
f-two: n-two**
When I run :
- name: check loop1
debug:
msg: '{{item[0]}} - {{item[1]}} - {{ item[2]}} '
with_nested:
- '{{ nodes }}'
- '{{ list_one }}'
- '{{ list_two }}'
item variable is availble. But doing this:
- set_fact:
final:
'{{item[0]}}':
labels:
"{{item[1] }}" : "{{item[2]}}"
with_nested:
- '{{ nodes }}'
- '{{ list_one }}'
- '{{ list_two }}'
results in an error.
Can someone explain why? How do I end up with my desired result?

Although your last piece of code above does not meet your requirement, It's perfectly valid: I'm not getting any error when running it.
As your are using it right now, set_fact is overwriting your final variable on each loop. To append element to a dict like your are trying to do, you need to initialize the var to an empty dict and combine it with the values you are calculating for each iteration. Since your calculated values are a dict themselves, you will need to use recursive=True if you have to write expressions deep inside the dict.
If I take into account your original data and your expected result, you want to relate the Nth element of each lists together. This is not what nested does (loop over nodes with a sub-loop on list_one sub-sub-loop on list_two....). In your case, you simply need to loop over an index of the length of your lists and combine the elements of same index together. My take below.
---
- name: test for SO
hosts: localhost
vars:
nodes:
- node1
- node2
list_one:
- f-one
- f-two
list_two:
- n-one
- n-two
tasks:
- name: Make my config
set_fact:
final: >-
{{
final
| default({})
| combine ({
nodes[item]: {
'labels': {
list_one[item]: list_two[item]
}
}
}, recursive=True)
}}
loop: "{{ range(nodes | length) | list }}"
- name: debug
debug:
var: final
which gives the following result
$ ansible-playbook test.yml
PLAY [test for SO] ******************************************************************
TASK [Gathering Facts] **************************************************************
ok: [localhost]
TASK [Make my config] ***************************************************************
ok: [localhost] => (item=0)
ok: [localhost] => (item=1)
TASK [debug] ************************************************************************
ok: [localhost] => {
"final": {
"node1": {
"labels": {
"f-one": "n-one"
}
},
"node2": {
"labels": {
"f-two": "n-two"
}
}
}
}
PLAY RECAP **************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0
Edit: The same result can be acheived using the zip filter (which I (re)discovered today reading an other contribution).
- name: Make my config
set_fact:
final: >-
{{
final
| default({})
| combine ({
item.0: {
'labels': {
item.1: item.2
}
}
}, recursive=True)
}}
loop: "{{ nodes | zip(list_one, list_two) | list }}"

An option would be to create the list of labels and then combine the dictionary. The play below
- hosts: localhost
vars:
nodes:
- node1
- node2
list_one:
- f-one
- f-two
list_two:
- n-one
- n-two
node_dict: {}
my_labels: []
tasks:
- set_fact:
my_labels: "{{ my_labels + [ {list_one[my_idx]:list_two[my_idx]} ] }}"
loop: "{{ nodes }}"
loop_control:
index_var: my_idx
- set_fact:
node_dict: "{{ node_dict | combine({item:{'labels':my_labels[my_idx]}}) }}"
loop: "{{ nodes }}"
loop_control:
index_var: my_idx
- debug:
var: node_dict
gives:
"node_dict": {
"node1": {
"labels": {
"f-one": "n-one"
}
},
"node2": {
"labels": {
"f-two": "n-two"
}
}
}

Related

Ansible conditionals with with_items/nested

Given the following tasks:
- name: Gather security group info
amazon.aws.ec2_group_info:
filters:
"tag:vpn_ports": "*"
register: sec_group_info_output
- name: Extract security groups and ports
set_fact:
vpn_groups: "{{ vpn_groups + [{ 'group_id': item.group_id, 'ports': item.tags.vpn_ports.split(',') }] }}"
with_items:
- "{{ sec_group_info_output | json_query('security_groups') }}"
vars:
vpn_groups: []
when: sec_group_info_output != []
- name: Generate list with CIDRs
set_fact:
vpn_rules: "{{ vpn_rules + [{ 'group_id': item.0.group_id , 'port': item.1, 'cidr': item.2 }] }}"
with_nested:
- "{{ vpn_groups|subelements('ports') }}"
- "{{ cidr_ranges }}"
vars:
vpn_rules: []
when: sec_group_info_output != []
I am trying to skip the last two tasks if the first task returns an empty set.
My understanding is that the when conditional is evaluated for every loop, and not just for the task as a whole.
The below therefor gives me:
TASK [security_groups : Gather security group info] ****************************
ok: [localhost]
TASK [security_groups : Extract security groups and ports] *********************
TASK [security_groups : Generate list with CIDRs] ******************************
fatal: [localhost]: FAILED! => {"msg": "obj must be a list of dicts or a nested dict"}
PLAY RECAP *********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=1 skipped=1 rescued=0 ignored=0
🚨 Error: The command exited with status 2
How would I go about fixing this error? I've tried putting |default([]) into my nested_items like below:
- name: Generate list with CIDRs
set_fact:
vpn_rules: "{{ vpn_rules + [{ 'group_id': item.0.group_id , 'port': item.1, 'cidr': item.2 }] |default([])}}"
with_nested:
- "{{ vpn_groups|subelements('ports') |default([])}}"
- "{{ cidr_ranges |default([])}}"
vars:
vpn_rules: []
when: sec_group_info_output != []
The error remains the same.
I've also tried putting both tasks in a block, but this had no effect and the error remains the same.
How would I be able to skip these tasks based on my condition?
Firstly, your condition is completely wrong. sec_group_info_output is a registered variable, so it can never be equal to an empty list. It will always be a dictionary containing information about the task execution. In order to have a chance of working as intended, it would need to be:
when: sec_group_info_output.security_groups != []
# more idiomatically, an empty list is false so you can just treat the value as a boolean
when: sec_group_info_output.security_groups
# or you can check the length
when: sec_group_info_output['security_groups'] | length > 0
However, in this case you don't need conditions at all. You're looping over the same list you're checking, and an empty loop will not execute any tasks. You just need a | default([]) in the loop definition on the third task in case the second didn't execute, and everything's fine.
- name: Gather security group info
amazon.aws.ec2_group_info:
filters:
"tag:vpn_ports": "*"
register: sec_group_info_output
- name: Extract security groups and ports
set_fact:
vpn_groups: "{{ vpn_groups | default([]) + [{ 'group_id': item.group_id, 'ports': item.tags.vpn_ports.split(',') }] }}"
loop: "{{ sec_group_info_output.security_groups }}"
- name: Generate list with CIDRs
set_fact:
vpn_rules: "{{ vpn_rules | default([]) + [{ 'group_id': item.0.0.group_id , 'port': item.0.1, 'cidr': item.1 }] }}"
loop: "{{ vpn_groups | default([]) | subelements('ports') | product(cidr_ranges) }}"
{{ vpn_groups | subelements('ports') | default([]) }} was headed in the right direction, but you put the default() in the wrong place. It needs to be before the subelements() filter so that that filter receives an empty list, not an undefined variable.

Iterate Over 2 dictionary in ansible

I Have 2 dictionary:
- Test1:
1: pass
2: fail
3: pass
- Test2:
1.1.1.1: val1
2.2.2.2: val2
3.3.3.3: val3
Condition is when Test1.value contians fail
- name: test
debug:
msg: "{{item.1.value}} {{item.1.key}} {{item.0.key}} {{item.0.value}}"
with_together:
- "{{Test1}}"
- "{{Test2}}"
when: item.0.value == "fail"
This is not working as expected unable to get both key and value of 2 dict in one loop
In when statement you must to use item.0 or item.1 to evaluate the condition. And I recommend you use a list in with_together loop and if you are using a variable you have to use braces {{ variable }} .
Try as below:
- name: test
debug:
msg: "{{item.1 }}"
with_together:
- "{{ Test1.values() | list }}"
- "{{ Test2.values() | list }}"
when: item.0 == "fail"
You'll get
TASK [test] *******************************************************************************************************************************************************************************************************
skipping: [127.0.0.1] => (item=['pass', 'val1'])
ok: [127.0.0.1] => (item=['fail', 'val2']) => {
"msg": "val2"
}
skipping: [127.0.0.1] => (item=['pass', 'val3'])
I achieved this by :
converting dict to list using filter -> |list
since
both dict of same size I was able to get data of both dict in single loop:
- name: test
debug:
msg: "{{item.0}} {{item.1}} {{item.2}} {{item.3}}"
with_together:
- "{{ Test1.values() | list }}"
- "{{ Test2.values() | list }}"
- "{{ Test1.keys() | list }}"
- "{{ Test2.keys() | list }}"
when: item.0 == "fail"

Parse yaml files in Ansible

I have got multiple yaml files on remote machine. I would like to parse those files in order to get information about names for each kind (Deployment, Configmap, Secret) of object, For example:
...
kind: Deployment
metadata:
name: pr1-dep
...
kind: Secret
metadata:
name: pr1
...
....
kind: ConfigMap
metadata:
name: cm-pr1
....
Ecpected result:
3 variables:
deployments = [pr1-dep]
secrets = [pr1]
configmaps = [cm-pr1]
I started with:
- shell: cat "{{ item.desc }}"
with_items:
- "{{ templating_register.results }}"
register: objs
but i have no idea how to correctly parse item.stdout from objs
Ansible has a from_yaml filter that takes YAML text as input and outputs an Ansible data structure. So for example you can write something like this:
- hosts: localhost
gather_facts: false
tasks:
- name: Read objects
command: "cat {{ item }}"
register: objs
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
- debug:
msg:
- "kind: {{ obj.kind }}"
- "name: {{ obj.metadata.name }}"
vars:
obj: "{{ item.stdout | from_yaml }}"
loop: "{{ objs.results }}"
loop_control:
label: "{{ item.item }}"
Given your example files, this playbook would output:
PLAY [localhost] ***************************************************************
TASK [Read objects] ************************************************************
changed: [localhost] => (item=deployment.yaml)
changed: [localhost] => (item=configmap.yaml)
changed: [localhost] => (item=secret.yaml)
TASK [debug] *******************************************************************
ok: [localhost] => (item=deployment.yaml) => {
"msg": [
"kind: Deployment",
"name: pr1-dep"
]
}
ok: [localhost] => (item=configmap.yaml) => {
"msg": [
"kind: ConfigMap",
"name: pr1-cm"
]
}
ok: [localhost] => (item=secret.yaml) => {
"msg": [
"kind: Secret",
"name: pr1"
]
}
PLAY RECAP *********************************************************************
localhost : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Creating the variables you've asked for is a little trickier. Here's
one option:
- hosts: localhost
gather_facts: false
tasks:
- name: Read objects
command: "cat {{ item }}"
register: objs
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
- name: Create variables
set_fact:
names: >-
{{
names|combine({
obj.kind.lower(): [obj.metadata.name]
}, list_merge='append')
}}
vars:
names: {}
obj: "{{ item.stdout | from_yaml }}"
loop: "{{ objs.results }}"
loop_control:
label: "{{ item.item }}"
- debug:
var: names
This creates a single variable named names that at the end of the
playbook will contain:
{
"configmap": [
"pr1-cm"
],
"deployment": [
"pr1-dep"
],
"secret": [
"pr1"
]
}
The key to the above playbook is our use of the combine filter, which can be used to merge dictionaries and, when we add list_merge='append', handles keys that resolve to lists by appending to the existing list, rather than replacing the existing key.
Include the dictionaries from the files into the new variables, e.g.
- include_vars:
file: "{{ item }}"
name: "objs_{{ item|splitext|first }}"
register: result
loop:
- deployment.yaml
- configmap.yaml
- secret.yaml
This will create dictionaries objs_deployment, objs_configmap, and objs_secret. Next, you can either use the dictionaries
- set_fact:
objs: "{{ objs|d({})|combine({_key: _val}) }}"
loop: "{{ query('varnames', 'objs_') }}"
vars:
_obj: "{{ lookup('vars', item) }}"
_key: "{{ _obj.kind }}"
_val: "{{ _obj.metadata.name }}"
, or the registered data
- set_fact:
objs: "{{ dict(_keys|zip(_vals)) }}"
vars:
_query1: '[].ansible_facts.*.kind'
_keys: "{{ result.results|json_query(_query1)|flatten }}"
_query2: '[].ansible_facts.*.metadata[].name'
_vals: "{{ result.results|json_query(_query2)|flatten }}"
Both options give
objs:
ConfigMap: cm-pr1
Deployment: pr1-dep
Secret: pr1

Customize dynamic inventory

Groups configured in Ansible inventory:
Group_A has 30 servers
Group_B has 40 Servers
Group_C has 15 Servers
I want to take 10 servers from each group and make a new group without editing the inventory manually.
These 10 servers is a variable that can change dynamically. If that works I got another question what if the inventory itself is dynamic
[Group_C]
server-1
server-2
server-3
...
server-10
''' New group created From 3 grouped servers now will be used in a playbook '''
(ansible 2.8.3)
If the inventory is dynamic we don't know the hosts. Let's assume we could choose any of them. Let's create the list of the selected hosts first and then loop add_hosts. With the inventory
[Group_A]
A-0
A-1
..
A-29
[Group_B]
B-0
B-1
..
B-39
[Group_C]
C-0
C-1
..
C-14
the plays below
- name: Create Group_X
hosts: localhost
vars:
no_of_servers: 2
my_groups:
- Group_A
- Group_B
- Group_C
tasks:
- set_fact:
my_list: "{{ my_list|default([]) +
groups[item][0:no_of_servers] }}"
loop: "{{ my_groups }}"
- add_host:
name: "{{ item }}"
groups: Group_X
loop: "{{ my_list }}"
- debug:
msg: "{{ groups['Group_X'] }}"
- name: Use Group_X
hosts: Group_X
gather_facts: false
tasks:
- debug:
msg: "{{ inventory_hostname }} is member of {{ group_names }}"
run_once: true
give
ok: [localhost] => {
"msg": [
"A-0",
"A-1",
"B-0",
"B-1",
"C-0",
"C-1"
]
}
ok: [A-0] => {
"msg": "A-0 is member of [u'Group_A', u'Group_X']"
}
Random choice.
It is possible to make the selection of the hosts random with the simple plugin below
$ cat filter_plugins/list_methods.py
import random
def list_sample(l,n):
return random.sample(l,n)
class FilterModule(object):
def filters(self):
return {
'list_sample' : list_sample
}
With the modification below
- set_fact:
my_list: '{{ my_list|default([]) +
groups[item]|list_sample(no_of_servers) }}'
the plays give for example
ok: [localhost] => {
"msg": [
"A-8",
"A-9",
"B-8",
"B-2",
"C-2",
"C-5"
]
}
ok: [A-8] => {
"msg": "A-8 is member of [u'Group_A', u'Group_X']"
}

Variable value for another variable ansible

Sorry if there are many posts about variables inside variable my use case is different.
Trying to access an element from a variable list "efs_list" based on the index-number of the current host. There are three hosts in the inventory
vars:
efs_list:
- efs1
- efs2
- efs3
sdb_index: "{{ groups['all'].index(inventory_hostname) }}"
The values should be as follows
host1- efs1
host2- efs2
host3- efs3
Tried accessing it through efs_list.{{ sdb_index }}
for - debug: var=efs_list.{{ sdb_index }} the output is as intended
ok: [10.251.0.174] => {
"efs_list.0": "efs1"
}
ok: [10.251.0.207] => {
"efs_list.1": "efs2"
}
ok: [10.251.0.151] => {
"efs_list.2": "efs3"
}
But for
- debug:
msg: "{{ efs_list.{{ sdb_index }} }}"
fatal: [10.251.0.174]: FAILED! => {"msg": "template error while templating string: expected name or number. String: {{ efs_list.{{ sdb_index }} }}"}
---
- name: SDB Snapshots Creation
hosts: all
remote_user: "centos"
become: yes
vars:
efs_list:
- efs1
- efs2
- efs3
sdb_index: "{{ groups['all'].index(inventory_hostname) }}"
tasks:
- debug: var=efs_list.{{ sdb_index }}
- debug:
msg: "{{ efs_list.{{ sdb_index }} }}"
- name: Get Filesystem ID
become: false
local_action: command aws efs describe-file-systems --creation-token "{{ efs_list.{{ sdb_index }} }}"
--region us-east-1 --query FileSystems[*].FileSystemId --output text
register: fs_id
It should attribute the element of list to current indexenter code here
extract filter will do the job. The input of the filter must be a list of indices and a container (array in this case). The tasks below
- set_fact:
sdb_index: "{{ [] + [ groups['all'].index(inventory_hostname) ] }}"
- debug:
msg: "{{ sdb_index|map('extract', efs_list)|list }}"
give
ok: [host1] => {
"msg": [
"efs1"
]
}
ok: [host2] => {
"msg": [
"efs2"
]
}
ok: [host3] => {
"msg": [
"efs3"
]
}
If the hosts are not sorted in the inventory it's necessary to sort them in the play
- set_fact:
my_hosts: "{{ groups['all']|sort }}"
- set_fact:
sdb_index: "{{ [] + [ my_hosts.index(inventory_hostname) ] }}"
- debug:
msg: "{{ sdb_index|map('extract', efs_list)|list }}"

Resources