Ansible: Create dictionary from an intent file - ansible

shell> cat myfile.yml
"ABC":
"ABC-C01":
- host: "a1"
prefixlen: "19"
- host: "a2"
prefixlen: "19"
"DEF":
"DEF-C01":
- host: "d1"
prefixlen: "19"
- host: "d2"
prefixlen: "19"
My expected answer below from the intent file :
ABC-C01:
- a1.domain
- a2.domain
DEF-C01:
- d1.domain
- d2.domain
I was able to get the list individually.
- include_vars:
file: myfile.yml
name: myfile
- set_fact:
dc: "{{ myfile.keys()|list }}"
Gives: 'ABC' and 'DEF'
- set_fact:
cl: "{{ cl|default([]) + myfile[item].keys()|list }}"
with_items: "{{ dc}}"
Gives: 'ABC-C01' and 'DEF-C01'

Put the below declarations into the vars
myfile: "{{ lookup('file', 'myfile.yml')|from_yaml }}"
myfile_groups: "{{ dict(myfile.values()|
map('dict2items')|
json_query(_query)) }}"
_query: '[].[key, value[].join(``,[host, `.domain`])]'
gives what you want
myfile_groups:
ABC-C01:
- a1.domain
- a2.domain
DEF-C01:
- d1.domain
- d2.domain

Related

Ansible with VMWare to create Dictionary using Ansible

I am trying to create a dictionary with lists of items for a VMWare details collection. I was able to create the list individually. But not sure how to merge this.
- name: Gather DC info
community.vmware.vmware_datacenter_info:
hostname: "{{ vcenter_hostname }}"
username: "{{ vcenter_username }}"
password: "{{ vcenter_password }}"
validate_certs: false
register: datacenter_infor
- name: Set DC_name variable
set_fact:
# dc_name: "{{ item.name }}"
dc_name: >-
{{ (dc_name | default([]))
+ [item.name]
}}
loop: "{{ datacenter_infor.datacenter_info }}"
- name: Gather cluster info
vmware_cluster_info:
hostname: "{{ vcenter_hostname }}"
username: "{{ vcenter_username }}"
password: "{{ vcenter_password }}"
validate_certs: false
datacenter: "{{ item }}"
register: cluster_info
loop: "{{ dc_name }}"
- name: Set Host_name variable
set_fact:
host_name_list: >-
{{ (host_name_list | default([]))
+ data
}}
vars:
data: "{{ item.clusters.values() }}"
loop: "{{ cluster_info.results }}"
This will result in below output:
TASK [Set Host_name variable] *********************************************************************************************************************************************************************************************
ok: [localhost] => (item={'changed': False, 'clusters': {'PQR-CLU01': {'hosts': [{'name': 'PQR-cn0001.myhost.com', 'folder': '/PQR/host/PQR-CLU01'}, {'name': 'PQR-cn0002.myhost.com', 'folder': '/PQR/host/PQR-CLU01'}})
ok: [localhost] => (item={'changed': False, 'clusters': {'ABC-CLU01': {'hosts': [{'name': 'ABC-cn0002.myhost.com', 'folder': '/ABC/host/ABC-CLU01'}, {'name': 'ABC-cn0001.myhost.com', 'folder': '/ABC/host/ABC-CLU01'}})
How can I create a dictionary with above items like using ansible:
{'PQR-CLU01': ['PQR-cn0001.myhost.com','PQR-cn0002.myhost.com'],'ABC-CLU01':['ABC-cn0002.myhost.com','ABC-cn0001.myhost.com']
Given the data
cluster_info:
results:
- changed: false
clusters:
PQR-CLU01:
hosts:
- folder: /PQR/host/PQR-CLU01
name: PQR-cn0001.myhost.com
- folder: /PQR/host/PQR-CLU01
name: PQR-cn0002.myhost.com
- changed: false
clusters:
ABC-CLU01:
hosts:
- folder: /ABC/host/ABC-CLU01
name: ABC-cn0002.myhost.com
- folder: /ABC/host/ABC-CLU01
name: ABC-cn0001.myhost.com
Q: "Create dictionary (below)."
cluster_dict:
ABC-CLU01:
- ABC-cn0002.myhost.com
- ABC-cn0001.myhost.com
PQR-CLU01:
- PQR-cn0001.myhost.com
- PQR-cn0002.myhost.com
A: Create the list of clusters
cluster_list: "{{ cluster_info.results|
map(attribute='clusters')|
map('dict2items')|
flatten }}"
gives
cluster_list:
- key: PQR-CLU01
value:
hosts:
- folder: /PQR/host/PQR-CLU01
name: PQR-cn0001.myhost.com
- folder: /PQR/host/PQR-CLU01
name: PQR-cn0002.myhost.com
- key: ABC-CLU01
value:
hosts:
- folder: /ABC/host/ABC-CLU01
name: ABC-cn0002.myhost.com
- folder: /ABC/host/ABC-CLU01
name: ABC-cn0001.myhost.com
Create a list of keys
cluster_keys: "{{ cluster_list|
map(attribute='key')|
list }}"
gives
cluster_keys:
- PQR-CLU01
- ABC-CLU01
Create a list of values
cluster_vals: "{{ cluster_list|
map(attribute='value.hosts')|
map('map', attribute='name')|
list }}"
gives
cluster_vals:
- - PQR-cn0001.myhost.com
- PQR-cn0002.myhost.com
- - ABC-cn0002.myhost.com
- ABC-cn0001.myhost.com
Crate the dictionary
cluster_dict: "{{ dict(cluster_keys|zip(cluster_vals)) }}"
gives
cluster_dict:
ABC-CLU01:
- ABC-cn0002.myhost.com
- ABC-cn0001.myhost.com
PQR-CLU01:
- PQR-cn0001.myhost.com
- PQR-cn0002.myhost.com
Example of a complete playbook
- hosts: localhost
vars:
cluster_info:
results:
- changed: false
clusters:
PQR-CLU01:
hosts:
- folder: /PQR/host/PQR-CLU01
name: PQR-cn0001.myhost.com
- folder: /PQR/host/PQR-CLU01
name: PQR-cn0002.myhost.com
- changed: false
clusters:
ABC-CLU01:
hosts:
- folder: /ABC/host/ABC-CLU01
name: ABC-cn0002.myhost.com
- folder: /ABC/host/ABC-CLU01
name: ABC-cn0001.myhost.com
cluster_list: "{{ cluster_info.results|
map(attribute='clusters')|
map('dict2items')|
flatten }}"
cluster_keys: "{{ cluster_list|
map(attribute='key')|
list }}"
cluster_vals: "{{ cluster_list|
map(attribute='value.hosts')|
map('map', attribute='name')|
list }}"
cluster_dict: "{{ dict(cluster_keys|zip(cluster_vals)) }}"
tasks:
- debug:
var: cluster_dict

How to parse text with ansible using a nested list/dict?

I have a text file that needs to be parsed so i can use it via a REST API with ansible.
10.0.0.0/16 Building-A
10.1.0.0/16 Building-A
10.2.0.0/16 Building-B
10.3.0.0/16 Building-B
I need to convert this text to something like this:
{
"parsed":[
{
"Building-A":[
"10.0.0.0/16",
"10.1.0.0/16"
]
},
{
"Building-B":[
"10.2.0.0/16",
"10.3.0.0/16"
]
}
]
}
Currently i run the following playbook to test the textparsing, without success. The list created is not unique.
- name: Test
hosts: localhost
tasks:
- name: Combine
ansible.builtin.set_fact:
parsed: "{{ (parsed | default([])) | union( [{item.split()[1]: item.split()[0] }] ) }}"
loop: "{{ lookup('file','hostgroups.txt').strip().splitlines() }}"
- name: Debug
ansible.builtin.debug:
var: parsed
ok: [localhost] => {
"parsed": [
{
"Building-A": "10.0.0.0/16"
},
{
"Building-A": "10.1.0.0/16"
},
{
"Building-B": "10.2.0.0/16"
},
{
"Building-B": "10.3.0.0/16"
}
]
}
Thanks for the tip with the helper variables and the groupby filter. Here is the final playbook:
- name: Test
hosts: localhost
# strategy: free
tasks:
- name: Get List
ansible.builtin.set_fact:
parsed_list: "{{ parsed_list | default([]) + [_item] }}"
loop: "{{ lookup('file','hostgroups.txt').strip().splitlines() }}"
vars:
_list: "{{ item.split() }}"
_item: "{{ {'Name': _list[1], 'Subnet': _list[0] } }}"
- name: Debug parsed_list
ansible.builtin.debug:
var: parsed_list
- name: Group parsed_list
ansible.builtin.set_fact.set_fact:
parsed_group: "{{ parsed_group | default([]) + [{_key: _value}] }}"
loop: "{{ parsed_list | groupby('Name') }}"
vars:
_key: "{{ item.0 }}"
_value: "{{ item.1 | map(attribute='Subnet') | list }}"
- name: Debug parsed_group
ansible.builtin.debug:
var: parsed_group
Parse the content, e.g.
- set_fact:
parsed_list: "{{ parsed_list|d([]) + [_item] }}"
loop: "{{ lookup('file','hostgroups.txt').splitlines() }}"
vars:
_array: "{{ item.split() }}"
_item: "{{ {'building': _array.1, 'ip': _array.0} }}"
gives
parsed_list:
- building: Building-A
ip: 10.0.0.0/16
- building: Building-A
ip: 10.1.0.0/16
- building: Building-B
ip: 10.2.0.0/16
- building: Building-B
ip: 10.3.0.0/16
Then, use filter groupby and create the list
- set_fact:
parsed: "{{ parsed|d([]) + [{_key: _val}] }}"
loop: "{{ parsed_list|groupby('building') }}"
vars:
_key: "{{ item.0 }}"
_val: "{{ item.1|map(attribute='ip')|list }}"
gives
parsed:
- Building-A:
- 10.0.0.0/16
- 10.1.0.0/16
- Building-B:
- 10.2.0.0/16
- 10.3.0.0/16
In some cases, a dictionary might be a better structure, e.g.
- set_fact:
parsed_dict: "{{ parsed_dict|d({})|combine({_key: _val}) }}"
loop: "{{ parsed_list|groupby('building') }}"
vars:
_key: "{{ item.0 }}"
_val: "{{ item.1|map(attribute='ip')|list }}"
gives
parsed_dict:
Building-A:
- 10.0.0.0/16
- 10.1.0.0/16
Building-B:
- 10.2.0.0/16
- 10.3.0.0/16

Ansible multiple vars/value

I need to role which adds records to my zones (bind9)
In hostvars I created vars like as below:
zones:
zone.name1:
- [ type: A, name: mike, ip: 192.168.1.10 ]
- [ type: A, name: bob, ip: 192.168.1.11 ]
zone.name2:
- [ type: A, name: alice, ip: 192.168.1.12 ]
- [ type: A, name: joanne, ip: 192.168.1.13 ]
In role/tasks/main.yaml
- lineinfile:
path: "/etc/bind/zones/{{ item }}.zone"
line: '# IN "{{ item.value.type }}" "{{ item.value.name }}" "{{ item.value.i }}"'
with_items: "{{ zones | dict2items }}"
How to get a result which adds a new record to the zone file?
There is couple of things wrong here.
You are using list notation for dict ([ type: A, name: mike, ip: 192.168.1.10 ] should be { type: A, name: mike, ip: 192.168.1.10 })
Your data structure requires two loops which you cannot do directly in the playbook.
You probably also want to have the freedom to remove records when they are not needed which doesn't work just like that when using lineinfile.
The following solution fixes all the above problems:
# main.yaml
---
- hosts: all
gather_facts: no
connection: local
vars:
zones:
zone.name1:
- { type: A, name: mike, ip: 192.168.1.10 }
# Remove this record
- { type: A, name: bob, ip: 192.168.1.11, state: absent }
zone.name2:
- { type: A, name: alice, ip: 192.168.1.12 }
- { type: A, name: joanne, ip: 192.168.1.13 }
tasks:
- include_tasks: lines.yaml
loop: "{{ zones | dict2items }}"
loop_control:
loop_var: records
Another task file which we loop through:
# lines.yaml
---
- lineinfile:
path: /tmp/{{ records.key }}.zone
line: >-
# IN "{{ item.type }}" "{{ item.name }}" "{{ item.ip }}"
regexp: >-
^#\s+IN\s+"{{ item.type }}"\s+"{{ item.name }}"\s+"{{ item.ip }}"$
state: >-
{{ 'present' if 'state' not in item or item.state == 'present' else 'absent' }}
loop: "{{ records.value }}"
Execute it with this command:
ansible-playbook -i localhost, --diff main.yaml

Ansible: Merge list to new list with item.key and item.value

Example:
file: storages.yml
---
storages:
- storageA
- storageB
file: storageA_list_vv.yml
---
vv:
- storageA_lun01
- storageA_lun02
file: storageB_list_vv.yml
---
vv:
- storageB_lun01
Expected new list: vvset.yml
---
vvset:
- key: serverA
value: serverA_lun01
- key: serverA
value: serverA_lun02
- key: serverB
value: serverB_lun01
Appreciate your guidance and advise on how to combine three list storages.yml, storageA_list_vv.yml and storageB_list_vv.yml into new list vvset.yml.
Thanks in advance.
Below is the step by step "brute-force" approach. Custom filter would make it much simpler.
Read the files and create the list of all items
- set_fact:
vv: "{{ vv|default([]) +
(lookup('file', item)|from_yaml).values()|list }}"
loop:
- storageA_list_vv.yml
- storageB_list_vv.yml
- debug:
var: vv|flatten
gives
vv|flatten:
- storageA_lun01
- storageA_lun02
- storageB_lun01
Extract the data
- set_fact:
vvset1: "{{ vvset1|default([]) +
[{'index': index, 'lun': lun}] }}"
loop: "{{ vv|flatten }}"
vars:
index: "{{ item.split('_').0|
regex_replace('^storage(.*)$', '\\1')}}"
lun: "{{ item.split('_').1 }}"
- debug:
var: vvset1
gives
vvset1:
- index: A
lun: lun01
- index: A
lun: lun02
- index: B
lun: lun01
Use groupby
- set_fact:
vvset2: "{{ vvset2|default([]) +
[{'index': item.0, 'lun': item.1|map(attribute='lun')|list}] }}"
loop: "{{ vvset1|groupby('index') }}"
- debug:
var: vvset2
gives
vvset2:
- index: A
lun:
- lun01
- lun02
- index: B
lun:
- lun01
Use subelements
- set_fact:
vvset3: "{{ vvset3|default([]) +
[{'key': key, 'value': val}] }}"
with_subelements:
- "{{ vvset2 }}"
- lun
vars:
key: "{{ 'server' ~ item.0.index }}"
val: "{{ 'server' ~ item.0.index ~ '_' ~ item.1 }}"
- debug:
var: vvset3
gives
vvset3:
- key: serverA
value: serverA_lun01
- key: serverA
value: serverA_lun02
- key: serverB
value: serverB_lun01

Ansible form the list or separate the dictionary

I have the below dicotionary value
{
"1application": "abc",
"2service": "def",
"3service": ghi",
"4application": "jkl",
"5service": "mno",
"6application": "abc",
"7service": "def",
"8service": ghi"
}
From this I have to form a list like below, starting from first application to next application I have to form one list, and so on
[
"1application": "abc",
"2service": "def",
"3service": ghi",
]
[
"4application": "jkl",
"5service": "mno",
]
[
"6application": "abc",
"7service": "def",
"8service": ghi"
]
The playbook below
shell> cat playbook.yml
- hosts: localhost
vars:
my_dict:
{"1application": "abc",
"2service": "def",
"3service": "ghi",
"4application": "jkl",
"5service": "mno",
"6application": "abc",
"7service": "def",
"8service": "ghi"}
tasks:
- set_fact:
my_list: "{{ my_list|default([]) +
[dict(my_range_keys|
zip(my_range_keys|
map('extract', my_dict)))] }}"
loop: "{{ my_idxs }}"
loop_control:
extended: true
vars:
my_keys: "{{ my_dict.keys()|list|sort }}"
my_len: "{{ my_keys|length }}"
my_apps: "{{ my_keys|select('match', '^(\\d+)application(.*)$')|list }}"
my_idxs: "{{ my_apps|map('regex_replace', '^(\\d+)(.*)$', '\\1')|list }}"
my_last: "{{ ansible_loop.nextitem|default(my_len|int + 1) }}"
my_range: "{{ range(item|int - 1, my_last|int - 1)|list }}"
my_range_keys: "{{ my_range|map('extract', my_keys)|list }}"
- debug:
var: my_list
gives
my_list:
- 1application: abc
2service: def
3service: ghi
- 4application: jkl
5service: mno
- 6application: abc
7service: def
8service: ghi
The previous solution works only with a single-digit prefix. With multiple digits, the sort function doesn't work anymore. In this case, we have to create the sorted list of the keys. For example, the playbook
shell> cat playbook.yml
- hosts: localhost
vars:
my_dict:
{"1application": "abc",
"2service": "def",
"3service": "ghi",
"4application": "jkl",
"5service": "mno",
"6application": "abc",
"7service": "def",
"8service": "ghi",
"9application": "prs",
"10service": "stu",
"11service": "vxy"}
tasks:
- set_fact:
my_keys_idx: "{{ my_keys_idx|default([]) +
[{'idx': my_idx|int, 'key': item}] }}"
loop: "{{ my_dict.keys()|list }}"
vars:
my_idx: "{{ item|regex_replace('^(\\d+)(.*)$', '\\1') }}"
- debug:
var: my_keys_idx
- set_fact:
my_keys: "{{ my_keys_idx|
sort(attribute='idx')|
map(attribute='key')|
list }}"
- debug:
var: my_keys
gives
my_keys:
- 1application
- 2service
- 3service
- 4application
- 5service
- 6application
- 7service
- 8service
- 9application
- 10service
- 11service
Then the task below works as expected
- set_fact:
my_list: "{{ my_list|default([]) +
[dict(my_range_keys|
zip(my_range_keys|
map('extract', my_dict)))] }}"
loop: "{{ my_idxs }}"
loop_control:
extended: true
vars:
# my_keys: "{{ my_dict.keys()|list|sort }}"
my_len: "{{ my_keys|length }}"
my_apps: "{{ my_keys|select('match', '^(\\d+)application(.*)$')|list }}"
my_idxs: "{{ my_apps|map('regex_replace', '^(\\d+)(.*)$', '\\1')|list }}"
my_last: "{{ ansible_loop.nextitem|default(my_len|int + 1) }}"
my_range: "{{ range(item|int - 1, my_last|int - 1)|list }}"
my_range_keys: "{{ my_range|map('extract', my_keys)|list }}"
- debug:
var: my_list
gives
my_list:
- 1application: abc
2service: def
3service: ghi
- 4application: jkl
5service: mno
- 6application: abc
7service: def
8service: ghi
- 10service: stu
11service: vxy
9application: prs

Resources