How to pull deeply nested data from a dictionary in ansible - ansible

I am trying to print nested data from an ansible dictionary. However, I cannot figure out how to pull the nested values. Specifically, in the data below, I am trying to print the two values returned under "ipv4". Currently, the output of item.value.ipv4 is the portion in brackets below:
"msg": "GigabitEthernet0/0/0,[{'address': '192.168.3.1', 'subnet': '28'}]"
I would like to just use the value like so:
"msg": "GigabitEthernet0/0/0, 192.168.3.1, 28"
I cannot figure out how to pull this nested data out of to do this. To put it simply, it would be nice if something like this worked: item.value.ipv4['address']. How is this done?
tasks:
- name: get config for Cisco IOS
ios_facts:
gather_subset: all
gather_network_resources: interfaces
- name: create dictionary with ansible_net_interfaces
set_fact:
foo_value: "{{ query('dict', ansible_net_interfaces) }}"
- name: display the results of foo_value
debug:
msg: "{{ foo_value }}"
- name: display certain detalys values from foo_value
debug:
msg: "{{ item.key }},{{ item.value.ipv4 }}"
with_items: "{{ foo_value }}"
These tasks produce the following:
TASK [display the results of foo] **********************************************************************************************************************
ok: [192.168.3.1] => {
"msg": [
{
"key": "GigabitEthernet0/0/0",
"value": {
"bandwidth": 1000000,
"description": "This is an interface description",
"duplex": "Full",
"ipv4": [
{
"address": "192.168.3.1",
"subnet": "28"
}
],
"lineprotocol": "up",
"macaddress": "50f7.123c.b0c0",
"mediatype": "RJ45",
"mtu": 1500,
"operstatus": "up",
"type": "ISR4331-3x1GE"
}
},
{
"key": "GigabitEthernet0/0/1",
"value": {
"bandwidth": 1000000,
"description": "This is another interface description",
"duplex": "Full",
"ipv4": [
{
"address": "192.168.3.33",
"subnet": "30"
}
],
"lineprotocol": "up",
"macaddress": "50f7.123c.b0c0",
"mediatype": "RJ45",
"mtu": 1500,
"operstatus": "up",
"type": "ISR4331-3x1GE"
}
},
]
}

ipv4 is a list of dictionaries. Assuming you need only the first dictionary,
- name: display certain detalys values from foo_value
debug:
msg: "{{ item.key }},{{ item.value.ipv4[0].address }},{{ item.value.ipv4[0].subnet }}"
when: item.value.ipv4 is defined and item.value.ipv4[0].subnet is defined and item.value.ipv4[0].address is defined
with_items: "{{ foo_value }}"

Related

Output of Ansibble task

I am using command hcloud to create cloud server in Hetzner. I get an output like this:
changed: [localhost] => (item={'name': 'TEST-VARIABLES', 'server_type': 'cx11', 'os_image': 'ubuntu-20.04', 'server_labels': 'Name=test-server', 'server_location': 'hel1'}) => {
"ansible_loop_var": "item",
"changed": true,
"hcloud_server": {
"backup_window": "None",
"datacenter": "hel1-dc2",
"delete_protection": false,
"id": "19461514",
"image": "ubuntu-20.04",
"ipv4_address": "11.111.111.111",
"ipv6": "1a71:7f9:c011:0b09::/64",
"labels": {
"Name": "test-server"
},
"location": "hel1",
"name": "TEST-VARIABLES",
"placement_group": null,
"rebuild_protection": false,
"rescue_enabled": false,
"server_type": "cx11",
"status": "running"
},
"invocation": {
"module_args": {
"allow_deprecated_image": false,
"api_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"backups": null,
"datacenter": null,
"delete_protection": null,
"endpoint": "https://api.SERVER.cloud/v1",
"firewalls": null,
"force": false,
"force_upgrade": false,
"id": null,
"image": "ubuntu-20.04",
"labels": {
"Name": "test-server"
},
"location": "hel1",
"name": "TEST-VARIABLES",
"placement_group": null,
"rebuild_protection": null,
"rescue_mode": null,
"server_type": "cx11",
"ssh_keys": null,
"state": "present",
"upgrade_disk": false,
"user_data": null,
"volumes": null
}
},
"item": {
"name": "TEST-VARIABLES",
"os_image": "ubuntu-20.04",
"server_labels": "Name=test-server",
"server_location": "hel1",
"server_type": "cx11"
},
"root_password": "DFLDJFLDFDLFKJDLFKJ"
}
I try to get line with "ipv4_address": "11.111.111.111", and "root_password": "DFLDJFLDFDLFKJDLFKJ", but when I use task:
---
- name: Create a basic server
hcloud_server:
api_token: "{{ token }}"
name: "{{ item.name }}"
server_type: "{{ item.server_type }}"
image: "{{ item.os_image }}"
labels: "{{ item.server_labels }}"
location: "{{ item.server_location }}"
state: present
register: server_info
with_items: "{{ server }}"
- name: Here IP
debug:
var: server_info.root_password
I got error:
TASK [/etc/ansible/roles/CREATE-server : Here IP.] *********************************************************************************************************
ok: [localhost] => {
"server_info.root_password": "VARIABLE IS NOT DEFINED!"
}
Could you please help, how I can get IP line and password line, to use them in the next task (for example to send via email). Thank you!
you register the content of loop, so your result is a list (results):
- name: display
debug:
msg: "ip: {{ item.hcloud_server.ipv4_address }} has password: {{ item.root_password }}"
loop: "{{ server_info.results }}"
result: here you have just one server declared, so just one item in the list results
"msg": "ip: 11.111.111.111 has password: DFDFDFDFDFDFDFDF"
if you want to limit the output of record, you could add loop_control parameter to loop with argument label:
loop: "{{ server_info.results }}"
loop_control:
label: "{{ item.hcloud_server.ipv4_address }}"
you could put another comment if you want with label or even empty string:
loop: "{{ server_info.results }}"
loop_control:
label: "result"

Ansible, how to query deeply nested json keys

I'm using Ansible to make an API call that returns a huge data set, and I need to be able to get a nested value to print to the screen. I tried using json_query but not sure what I'm doing wrong.
My task:
- name: Get certificate by CN name.
uri:
method: GET
url: "https://mayapi/api/1/certificates?filter=cn;{{ inventory_hostname }}"
headers:
Authorization: Bearer {{ login.json.token }}
Content-Type: application/json
validate_certs: no
register: certs
- name: Print certs for application
debug:
msg: "{{ certs.json | json_query(items) }}"
This is a small snippet of the output. I want to be able to print ID, and email.
{
"msg": {
"changed": false,
"connection": "close",
"content_length": "65833",
"content_type": "application/json",
"cookies": {},
"cookies_string": "",
"date": "Mon, 10 May 2021 21:33:29 GMT",
"elapsed": 0,
"failed": false,
"json": {
"items": [
{
"active": true,
"application": [
{
"director": {
"active": true,
"email": "user#domain.com",
"fullname": "John Doe",
"id": 1611,
"manager": "John Doe",
"managerEmail": "johndoe#email.com",
"username": "jdoe"
},
...
...
...
}
I get the following error indicating "certs.items" doesn't exist:
FAILED! => {"msg": "Error in jmespath.search in json_query filter plugin:\n'items' is undefined"}
I was expecting all of the items to get printed to the screen and then if I wanted something below items I would do items.active, items.application, etc... But this is not correct since I keep erroring out.
I also tried looping through cert.json and cert.json.items:
- name: Print certs for application
debug:
msg: "{{ item.application.name }}"
loop: "{{ certs.json}}"
But get this error message:
{"msg": "Invalid data passed to 'loop', it requires a list, got this instead: {u'items': [{u'status': u'Active-Pending Install'...shows all the data of the nested json
Then I tried this:
- name: Print certs for application
debug:
msg: "{{ item.application.name }}"
loop: "{{ certs.json.items}}"
But got this error message:
{"msg": "Invalid data passed to 'loop', it requires a list, got this instead: <built-in method items of dict object at 0x7f0c9ec43050>. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup."}
Made some progress with this:
- name: Print certs for application
debug:
msg: "KEY:::: {{ item.key }}, VALUE:::: {{ item.value.0 }}"
loop: "{{ lookup('dict', certs.json) }}"
when: "'items' in item.key"
ignore_errors: yes
But this only prints items in index 0 of the list:
"msg": "KEY:::: items, VALUE:::: {u'status': u'Active-Pending Install', u'serialHex': u'1111', u'validityStart': u'2021-05-10T21:01:36+00:00', u'cn': u'node2.test.corp.net', u'validityEnd': u'2023-05-10T21:11:36+00:00', u'application': [{u'uuid': u'2222', u'name': u'abc'}], u'certType': u'CertType.INTERNAL', u'id': 2582, u'issuer': u'server1'}"
I'm trying to print the 'cn', 'id', and 'serialHex' values from each list element for the key 'items'.
This is the data set that I'm trying to query with Ansible:
{
"total": 2,
"items": [
{
"application": [
{
"uuid": "111",
"name": "CDE"
}
],
"validityEnd": "2023-05-10T21:11:36+00:00",
"certType": "CertType.INTERNAL",
"issuer": "server1",
"id": 2582,
"validityStart": "2021-05-10T21:01:36+00:00",
"status": "Active-Pending Install",
"serialHex": "aaa",
"cn": "node2.corp.net"
},
{
"application": [
{
"uuid": "222",
"name": "CDE"
}
],
"validityEnd": "2023-05-10T21:05:26+00:00",
"certType": "CertType.INTERNAL",
"issuer": "server1",
"id": 2581,
"validityStart": "2021-05-10T20:55:26+00:00",
"status": "Active-Pending Install",
"serialHex": "bbbb",
"cn": "node1.corp.net"
}
]
}
You are regrettably stepping on a quirk of "objects in ansible are python dicts" in that .items() and .keys() and quite a few other attributes-which-are-methods cannot be referenced using the . notation since jinja2 believes you intend to call that method. Rather, one must use the __getitem__ syntax of ["items"] in order to make it abundantly clear that you mean the dict key, and not the method of the same name
tasks:
- name: use json_query as you were originally asking
debug:
msg: >-
{{ certs.json | json_query('items[*].{c: cn,i: id,s: serialHex}') }}
- name: or a jinja2 for loop as you separately attempted
debug:
msg: >-
[
{%- for i in certs.json["items"] -%}
{{ "" if loop.first else "," }}
{{ [i.cn, i.id, i.serialHex ]}}
{%- endfor -%}
]
produces the output from their respective steps:
TASK [debug] ******************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"c": "node2.corp.net",
"i": 2582,
"s": "aaa"
},
{
"c": "node1.corp.net",
"i": 2581,
"s": "bbbb"
}
]
}
TASK [debug] ******************************************************************************************************************************
ok: [localhost] => {
"msg": [
[
"node2.corp.net",
2582,
"aaa"
],
[
"node1.corp.net",
2581,
"bbbb"
]
]
}

Ansible set_fact is overwriting the items

Here is my main.yml
---
- name: Gathering VCenter facts
vmware_vm_info:
hostname: "{{ vcenter_server }}"
username: "{{ vcenter_user }}"
password: "{{ vcenter_pass }}"
validate_certs: false
register: vcenter_facts
delegate_to: localhost
- debug:
var: vcenter_facts.virtual_machines
- name: Find all test-vms to run IO
set_fact:
vm_ip: "{{ item.ip_address }}"
loop: "{{ vcenter_facts.virtual_machines }}"
when: item.guest_name is regex("test_vm*")
- name: print vm_ip variable value
debug:
var: vm_ip
- name: Mount 16TB dropbox in each test vm
shell: mount-16tb-dropbox.sh
args:
chdir: /usr/local/bin/
with_items: "{{ vm_ip }}"
And here is the recap:
ok: [localhost] => {
"vcenter_facts.virtual_machines": [
{
"attributes": {},
"cluster": "Compute Cluster",
"esxi_hostname": "100.80.90.179",
"guest_fullname": "CentOS 7 (64-bit)",
"guest_name": "test_vm4",
"ip_address": "192.168.202.13",
"mac_address": [
"00:50:56:9d:d2:99"
],
"power_state": "poweredOn",
"tags": [],
"uuid": "421d7b54-1359-14e8-3ec4-74b568cb96d2",
"vm_network": {
"00:50:56:9d:d2:99": {
"ipv4": [
"192.168.202.13"
],
"ipv6": [
"fe80::44f6:a395:cde3:4dd1",
"fe80::a357:a163:e44f:2086",
"fe80::cd0c:e7d7:1356:2830"
]
}
}
},
{
"attributes": {},
"cluster": "Compute Cluster",
"esxi_hostname": "100.80.90.178",
"guest_fullname": "CentOS 7 (64-bit)",
"guest_name": "test_vm3",
"ip_address": "192.168.202.12",
"mac_address": [
"00:50:56:9d:a9:e8"
],
"power_state": "poweredOn",
"tags": [],
"uuid": "421d9239-0980-80c1-bca4-540efd726452",
"vm_network": {
"00:50:56:9d:a9:e8": {
"ipv4": [
"192.168.202.12"
],
"ipv6": [
"fe80::cd0c:e7d7:1356:2830"
]
}
}
},
{
"attributes": {},
"cluster": "Compute Cluster",
"esxi_hostname": "100.80.90.178",
"guest_fullname": "CentOS 7 (64-bit)",
"guest_name": "Test_Automation_CentOS8_Linux_VM",
"ip_address": "192.168.202.6",
"mac_address": [
"00:50:56:9d:13:14"
],
"power_state": "poweredOn",
"tags": [],
"uuid": "421d53ba-4824-57e4-06fd-fba0f2b1dbea",
"vm_network": {
"00:50:56:9d:13:14": {
"ipv4": [
"192.168.202.6"
],
"ipv6": [
"fe80::cd0c:e7d7:1356:2830",
"fe80::44f6:a395:cde3:4dd1"
]
}
}
},
{
"attributes": {},
"cluster": "Compute Cluster",
"esxi_hostname": "100.80.90.180",
"guest_fullname": "CentOS 7 (64-bit)",
"guest_name": "test_vm5",
"ip_address": "192.168.202.14",
"mac_address": [
"00:50:56:9d:85:b6"
],
"power_state": "poweredOn",
"tags": [],
"uuid": "421d6855-e60e-cd80-f113-39f11927d63b",
"vm_network": {
"00:50:56:9d:85:b6": {
"ipv4": [
"192.168.202.14"
],
"ipv6": [
"fe80::44f6:a395:cde3:4dd1",
"fe80::cd0c:e7d7:1356:2830",
"fe80::a357:a163:e44f:2086"
]
}
}
}
]
}
I am not able to loop through all the ip_address variable (i.e. 192.168.202.12, 192.168.202.13, 192.168.202.14).
It just reads the last item (i.e. 192.168.202.14).
What am I possibly doing wrong with set_fact that it is not reading all the variable and performing the set of tasks that follows?
An alternate solution using json_query
---
- name: Gathering VCenter facts
vmware_vm_info:
hostname: "{{ vcenter_server }}"
username: "{{ vcenter_user }}"
password: "{{ vcenter_pass }}"
validate_certs: false
register: vcenter_facts
delegate_to: localhost
- name: Mount 16TB dropbox in each test vm
shell: mount-16tb-dropbox.sh
args:
chdir: /usr/local/bin/
vars:
query: >-
[?contains("guest_name", 'test_vm')].ip_address[]
with_items: "{{ vcenter_facts.virtual_machines | to_json | from_json | json_query(query) | list }}"
Note: to_json | from_json is a workaround for a bug between ansible and jmespath so that all values can be converted to real strings and can be used with the jmespath contains function.
This should give all the IP. You correctly assumed where the code may need correction. In the code, vm_ip variable was overwritten by each loop and the last IP remained. What you need is a list and then append each IP to the list.
- set_fact:
vm_ip: "{{ vm_ip | default([]) + [item.ip_address] }}"
loop: "{{ vcenter_facts.virtual_machines | flatten }}"
when: item.guest_name is regex("test_vm*")
- debug:
var: vm_ip
Alternative solution using Jinja2 filters.
- set_fact:
vm_ip: >-
{{ vcenter_facts.virtual_machines | flatten
| rejectattr('guest_name', 'match', '^(?!test_vm).*')
| map(attribute='ip_address') | list }}

Inner loop in loop over dict in Ansible

How to loop over dict in this example?
I can loop over list of strings (lorem:), but I can not loop over objects: dict.
I am using module which needs as input list and dict, so I have be able use both cases, but before module call I have to do some workaround with keys from dict so I have to be able reference it.
I am not sure what I am doing wrong. Can anyone show me a proper example?
Thanks
---
- name: test
hosts: localhost
connection: local
vars:
persons:
foo:
name: john
state: us
objects:
phone: samsung
color: black
capacity: 32
lorem:
- 1
- 2
- 3
bar:
name: helmut
state: de
objects:
phone: lg
color: red
capacity: 16
lorem:
- 4
- 5
- 6
tasks:
- name: List of strings is OK
debug:
msg: "{{ item.0.value.name }} and object: {{ item.1 }}"
loop: "{{ persons | dict2items |subelements('value.lorem',{ 'skip_missing': True }) }}"
- name: Dict referencing key:value is not OK
debug:
msg: "Name: {{ item.0.value.name }} and object: {{ item.1.[value] }} with key name: {{ item.1.[key]}}"
loop: "{{ persons | dict2items |subelements('value.objects',{ 'skip_missing': True }) }}"
Produces an error: fatal: [localhost]: FAILED! => {"msg": "the key 'objects' should point to a list, got {u'color': u'black', u'phone': u'samsung', u'capacity': 32}"}
Best I can think of is to do something like this. Using jinja to generate a list that returns what you need.
---
- name: test
hosts: localhost
connection: local
vars:
persons:
foo:
name: john
state: us
objects:
phone: samsung
color: black
capacity: 32
lorem:
- 1
- 2
- 3
bar:
name: helmut
state: de
objects:
phone: lg
color: red
capacity: 16
lorem:
- 4
- 5
- 6
tasks:
- debug:
msg: |
[
{% for p in persons %}
{% for o in persons[p].objects %}
{
"name": "{{ persons[p].name }}",
"key": "{{ o }}",
"value": "{{ persons[p].objects[o] }}"
},
{% endfor %}
{% endfor %}
]
Output
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": [
{
"key": "color",
"name": "john",
"value": "black"
},
{
"key": "phone",
"name": "john",
"value": "samsung"
},
{
"key": "capacity",
"name": "john",
"value": "32"
},
{
"key": "color",
"name": "helmut",
"value": "red"
},
{
"key": "phone",
"name": "helmut",
"value": "lg"
},
{
"key": "capacity",
"name": "helmut",
"value": "16"
}
]
}
Oh, and if you wanted to use that in a loop, just something like this.
- debug:
msg: "{{ item.name }} {{ item.key }} {{ item.value }}"
loop_control:
label: "{{ item.name }} {{ item.key }} {{ item.value }}"
loop: |
[
{% for p in persons %}
{% for o in persons[p].objects %}
{
"name": "{{ persons[p].name }}",
"key": "{{ o }}",
"value": "{{ persons[p].objects[o] }}"
},
{% endfor %}
{% endfor %}
]
Output
TASK [debug] *******************************************************************
ok: [localhost] => (item=john color black) => {
"msg": "john color black"
}
ok: [localhost] => (item=john phone samsung) => {
"msg": "john phone samsung"
}
ok: [localhost] => (item=john capacity 32) => {
"msg": "john capacity 32"
}
ok: [localhost] => (item=helmut color red) => {
"msg": "helmut color red"
}
ok: [localhost] => (item=helmut phone lg) => {
"msg": "helmut phone lg"
}
ok: [localhost] => (item=helmut capacity 16) => {
"msg": "helmut capacity 16"
}
You can use with_items as well.
Please refer below code:
playbook -->
---
- name: test hosts: localhost gather_facts: no tasks:
- include_vars: vars.yml
- name: debug
debug:
msg: "{{ item.vars.persons }}"
with_items:
- "{{ vars }}"
output --->
"msg": {
"bar": {
"lorem": [
4,
5,
6
],
"name": "helmut",
"objects": {
"capacity": 16,
"color": "red",
"phone": "lg"
},
"state": "de"
},
"foo": {
"lorem": [
1,
2,
3
],
"name": "john",
"objects": {
"capacity": 32,
"color": "black",
"phone": "samsung"
},
"state": "us"
}
}
}

set_fact create a list with items

I have task which calls an API and I register the o/p in a varaible;
- name: Get Object storage account ID
uri:
url: 'https://api.softlayer.com/rest/v3.1/SoftLayer_Network_Storage_Hub_Cleversafe_Account/getAllObjects.json?objectFilter={"username":{"operation":"{{ item }}"}}'
method: GET
user: abxc
password: 66c94c447a6ed8a0cf058774fe38
validate_certs: no
register: old_existing_access_keys_sl
with_items: '{{ info["personal"].sl_cos_accounts }}'
old_existing_access_keys_sl holds:
"old_existing_access_keys_sl.results": [
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "xyz-11"
}
]
},
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "abc-12"
}
]
}
I want to make a list of id's for further processing an tried the following task but this did not work:
- name: Create a list of account ids
set_fact:
admin_usernames = "{{ item.json[0].id | list }}"
with_items: old_existing_access_keys_sl.results
I am not sure if that's even possible. I also tried this:
- name: create a list
set_fact:
foo: "{% set foo = [] %}{% for i in old_existing_access_keys_sl.results %}{{ foo.append(i) }}{% endfor %}"
foo always comes as blank and as a string:
TASK [result] *****************************************************************************************************************************************
ok: [localhost] => {
"foo": ""
}
Given your example data, you can extract a list of ids using the json_query filter, like this:
---
- hosts: localhost
gather_facts: false
vars:
old_existing_access_keys_sl:
results:
[
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "xyz-11"
}
]
},
{
"json": [
{
"accountId": 12345,
"id": 70825621,
"username": "abc-12"
}
]
}
]
tasks:
- debug:
var: old_existing_access_keys_sl|json_query('results[*].json[0].id')
This will output:
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => {
"old_existing_access_keys_sl|json_query('results[*].json[0].id')": [
70825621,
70825621
]
}
If you want to store these in a new variable, you can replace that debug task with set_fact:
- set_fact:
admin_ids: "{{ old_existing_access_keys_sl|json_query('results[*].json[0].id') }}"
Update
For a list of dictionaries, just change the json_query expression:
- debug:
var: "old_existing_access_keys_sl|json_query('results[*].json[0].{id: id, username: username}')"
For more information, see the jmespath website for documentation and examples.

Resources