Complex JSON and Ansible JSON_QUERY syntax - ansible

i have complex result in JSON format and struggling to extract values out of it.
I tested the JSONpath online evaluator https://jsonpath.com/ where i can just simply type
$.[*].toPort
and get the list of requested variables.
I've tried to use the same syntax in Ansible but getting empty list.
- name: JSON test
hosts: localhost
gather_facts: no
vars:
jsoncontent:
{
"infraAccPortP": {
"attributes": {
"annotation": "",
"childAction": "",
"descr": "",
"nameAlias": "",
"ownerKey": "",
"ownerTag": "",
"status": "",
"uid": "15374"
},
"children": [
{
"infraHPortS": {
"attributes": {
"annotation": "",
"uid": "8927"
},
"children": [
{
"infraPortBlk": {
"attributes": {
"fromPort": "41",
"toPort": "41",
"uid": "8927"
}
}
}
]
}
},
{
"infraHPortS": {
"attributes": {
"annotation": "",
"uid": "8927"
},
"children": [
{
"infraPortBlk": {
"attributes": {
"fromPort": "42",
"toPort": "42",
"uid": "8927"
}
}
}
]
}
}
]
}
}
tasks:
- name: show jsoncontent
debug:
var: jsoncontent
- name: Show just toPort values
debug:
msg: "{{ jsoncontent | json_query(jmesquery) }}"
vars:
jmesquery: "[*].toPort"
Any help how to adjust the query to get the expected result? Its bit frustrating as there is not much documentation on how json_query is implemented in Ansible

The task below does the job
- name: Show just toPort values
debug:
msg: "{{ jsoncontent.infraAccPortP.children | json_query(jmesquery) }}"
vars:
jmesquery: "[].infraHPortS.children[].infraPortBlk.attributes.toPort"
gives
msg:
- '41'
- '42'
Q: "Ansible implementation doesn't allow wildcard searches?"
A: There doesn't seem to be anything special in the implementation. (This filter moved to ansible-collections/community.general since 2.10.)
Q: "Just confused why I got the same result with much shorter syntax in that online checker."
A: The question is how the checker is implemented, I think. jq doesn't work this way either. For example
shell> cat data.json | jq .jsoncontent.infraAccPortP.children[].infraHPortS.children[].infraPortBlk.attributes.toPort
"41"
"42"
The "checker" query crashes
shell> cat data.json | jq .[*].toPort
jq: error: syntax error, unexpected '*' (Unix shell quoting issues?) at <top-level>, line 1:
.[*].toPort
jq: 1 compile error
, or gives an empty result
shell> cat data.json | jq .[].toPort
null

Related

ansible json_query with character "-"

I have this json file and I need extract some data but problem is with this character -
{
"changed": true,
"failed": false,
"meta": {
"request_url": "pm/config/adom/AMIS/obj/dynamic/interface",
"response_code": 0,
"response_data": [
{
"color": 0,
"defmap-zonemember": [],
"description": "BYOD VLAN sub-interface.\nUsed in policy for firewalls maintaining BYOD traffic on LIBOs",
"dynamic_mapping": null,
"name": "BYOD",
"single-intf": "enable"
},
{
"color": 0,
"defmap-zonemember": [],
"dynamic_mapping": [
{
"_scope": [
{
"name": "ftg02",
"vdom": "dev"
}
],
"intrazone-deny": "enable",
"local-intf": [
"port8"
]
}
],
"name": "DMZ",
"single-intf": "disable"
},
{
"color": 0,
"defmap-zonemember": [],
"description": "Guest VLAN sub-interface. Used in policy for firewalls maintaining Guestnet traffic on LIBOs",
"dynamic_mapping": null,
"name": "Guest",
"single-intf": "enable"
}
]
}
}
I would like to get result of this "name": "DMZ" - DMZ with some condition
if local-intf == port8
I have got this error
FAILED! => {"msg": "template error while templating string: expected token ',', got 'local'. String: {{ item | json_query('dynamic_mapping[0]['local-intf'][0]') }}"}
here is my playbook which does not work \
- name: interface
debug:
msg: "{{ item.name }}"
loop: "{{ interface_info.meta.response_data }}"
when:
- item.dynamic_mapping != 'null'
- item | json_query('dynamic_mapping[0]._scope[0].name') == 'ftg02'
- item | json_query('dynamic_mapping[0]._scope[0].vdom') == 'dev'
- item | json_query('dynamic_mapping[0]['local-intf'][0]) == 'port8' #this line does not work
is some way to dont use loop ? Loop is to slow .
thank you for help
Try this
- debug:
msg: "{{ interface_info.meta.response_data|
selectattr('dynamic_mapping')|
selectattr('dynamic_mapping.0.local-intf.0', '==', 'port8')|
selectattr('dynamic_mapping.0._scope.0.name', '==', 'ftg02')|
selectattr('dynamic_mapping.0._scope.0.vdom', '==', 'dev')|
map(attribute='name')|first }}"
should give the first item
msg: DMZ

Usage of Ansible JMESPath in queries

I am using following JSON file:
sample.json:
{
"lldp_output['gathered']": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 12
},
"mode": "access",
"name": "GigabitEthernet0/1"
},
{
"name": "GigabitEthernet0/2"
}
]
}
And the playbook:
---
- hosts: localhost
gather_facts: no
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | community.general.json_query(lldp_output['gathered']) }}"
I get the following error:
TASK [Take 4] ********************************************************************************************
task path: /root/scripts/atest.yml:18
fatal: [localhost]: FAILED! => {
"msg": "Error in jmespath.search in json_query filter plugin:\n'lldp_output' is undefined"
}
How do I query the JSON shown so I get a list of all ports that have mode: trunk
When I run in a playbook:
---
- name: Find trunk ports
hosts: ios
tasks:
- name: Collect interface output
cisco.ios.ios_l2_interfaces:
config:
state: gathered
register:
"intf_output"
- debug:
var=intf_output
- name: Take 4
debug:
msg: "{{ intf_output | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"[?mode=='trunk']
The structure returned is like following:
{
"intf_output": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"failed": false,
"gathered": [
{
"name": "GigabitEthernet0/0"
},
{
"mode": "trunk",
"name": "GigabitEthernet0/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/2",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/3",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet1/0",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet1/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"name": "GigabitEthernet1/2"
},
{
"name": "GigabitEthernet1/3"
},
{
"name": "GigabitEthernet2/0"
},
{
"name": "GigabitEthernet2/1"
},
{
"name": "GigabitEthernet2/2"
},
{
"name": "GigabitEthernet2/3"
},
{
"name": "GigabitEthernet3/0"
},
{
"name": "GigabitEthernet3/1"
},
{
"name": "GigabitEthernet3/2"
},
{
"name": "GigabitEthernet3/3"
}
]
}
}
For each host I run against the playbook against.
The argument to json_query must be a string. Because you haven't quoted your argument, Ansible is looking for a variable named lldp_output. But you've got additonal problems, since you're trying to access a key named lldp_output['gathered'], but [ is a syntactically significant character in JSON (and JMESPath queries), so that needs to be escaped as well.
In order to avoid all sorts of quote escaping contortions, we can place the query itself into a variable, so that we have:
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"
Note that we are using the >- block quote operator, which means that the value of query is the literal string "lldp_output['gathered']", including the outer quotes.
That playbook outputs:
TASK [Take 4] *********************************************************************************
ok: [localhost] => {
"msg": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 12
},
"mode": "access",
"name": "GigabitEthernet0/1"
},
{
"name": "GigabitEthernet0/2"
}
]
}
To get just those systems with mode equal to trunk, just add that
criteria to your query:
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"[?mode=='trunk']
This will output:
TASK [Take 4] *********************************************************************************
ok: [localhost] => {
"msg": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
}
]
}
Update
Given the data you've shown in your updated question, things are much
simpler, because you don't have the weird quoting you had in the
original question. With intf_output defined as shown, you can
write:
tasks:
- name: Take 4
debug:
msg: "{{ intf_output | json_query(query) }}"
vars:
query: >-
gathered[?mode=='trunk']

Ansible, how to query deeply nested json keys

I'm using Ansible to make an API call that returns a huge data set, and I need to be able to get a nested value to print to the screen. I tried using json_query but not sure what I'm doing wrong.
My task:
- name: Get certificate by CN name.
uri:
method: GET
url: "https://mayapi/api/1/certificates?filter=cn;{{ inventory_hostname }}"
headers:
Authorization: Bearer {{ login.json.token }}
Content-Type: application/json
validate_certs: no
register: certs
- name: Print certs for application
debug:
msg: "{{ certs.json | json_query(items) }}"
This is a small snippet of the output. I want to be able to print ID, and email.
{
"msg": {
"changed": false,
"connection": "close",
"content_length": "65833",
"content_type": "application/json",
"cookies": {},
"cookies_string": "",
"date": "Mon, 10 May 2021 21:33:29 GMT",
"elapsed": 0,
"failed": false,
"json": {
"items": [
{
"active": true,
"application": [
{
"director": {
"active": true,
"email": "user#domain.com",
"fullname": "John Doe",
"id": 1611,
"manager": "John Doe",
"managerEmail": "johndoe#email.com",
"username": "jdoe"
},
...
...
...
}
I get the following error indicating "certs.items" doesn't exist:
FAILED! => {"msg": "Error in jmespath.search in json_query filter plugin:\n'items' is undefined"}
I was expecting all of the items to get printed to the screen and then if I wanted something below items I would do items.active, items.application, etc... But this is not correct since I keep erroring out.
I also tried looping through cert.json and cert.json.items:
- name: Print certs for application
debug:
msg: "{{ item.application.name }}"
loop: "{{ certs.json}}"
But get this error message:
{"msg": "Invalid data passed to 'loop', it requires a list, got this instead: {u'items': [{u'status': u'Active-Pending Install'...shows all the data of the nested json
Then I tried this:
- name: Print certs for application
debug:
msg: "{{ item.application.name }}"
loop: "{{ certs.json.items}}"
But got this error message:
{"msg": "Invalid data passed to 'loop', it requires a list, got this instead: <built-in method items of dict object at 0x7f0c9ec43050>. Hint: If you passed a list/dict of just one element, try adding wantlist=True to your lookup invocation or use q/query instead of lookup."}
Made some progress with this:
- name: Print certs for application
debug:
msg: "KEY:::: {{ item.key }}, VALUE:::: {{ item.value.0 }}"
loop: "{{ lookup('dict', certs.json) }}"
when: "'items' in item.key"
ignore_errors: yes
But this only prints items in index 0 of the list:
"msg": "KEY:::: items, VALUE:::: {u'status': u'Active-Pending Install', u'serialHex': u'1111', u'validityStart': u'2021-05-10T21:01:36+00:00', u'cn': u'node2.test.corp.net', u'validityEnd': u'2023-05-10T21:11:36+00:00', u'application': [{u'uuid': u'2222', u'name': u'abc'}], u'certType': u'CertType.INTERNAL', u'id': 2582, u'issuer': u'server1'}"
I'm trying to print the 'cn', 'id', and 'serialHex' values from each list element for the key 'items'.
This is the data set that I'm trying to query with Ansible:
{
"total": 2,
"items": [
{
"application": [
{
"uuid": "111",
"name": "CDE"
}
],
"validityEnd": "2023-05-10T21:11:36+00:00",
"certType": "CertType.INTERNAL",
"issuer": "server1",
"id": 2582,
"validityStart": "2021-05-10T21:01:36+00:00",
"status": "Active-Pending Install",
"serialHex": "aaa",
"cn": "node2.corp.net"
},
{
"application": [
{
"uuid": "222",
"name": "CDE"
}
],
"validityEnd": "2023-05-10T21:05:26+00:00",
"certType": "CertType.INTERNAL",
"issuer": "server1",
"id": 2581,
"validityStart": "2021-05-10T20:55:26+00:00",
"status": "Active-Pending Install",
"serialHex": "bbbb",
"cn": "node1.corp.net"
}
]
}
You are regrettably stepping on a quirk of "objects in ansible are python dicts" in that .items() and .keys() and quite a few other attributes-which-are-methods cannot be referenced using the . notation since jinja2 believes you intend to call that method. Rather, one must use the __getitem__ syntax of ["items"] in order to make it abundantly clear that you mean the dict key, and not the method of the same name
tasks:
- name: use json_query as you were originally asking
debug:
msg: >-
{{ certs.json | json_query('items[*].{c: cn,i: id,s: serialHex}') }}
- name: or a jinja2 for loop as you separately attempted
debug:
msg: >-
[
{%- for i in certs.json["items"] -%}
{{ "" if loop.first else "," }}
{{ [i.cn, i.id, i.serialHex ]}}
{%- endfor -%}
]
produces the output from their respective steps:
TASK [debug] ******************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"c": "node2.corp.net",
"i": 2582,
"s": "aaa"
},
{
"c": "node1.corp.net",
"i": 2581,
"s": "bbbb"
}
]
}
TASK [debug] ******************************************************************************************************************************
ok: [localhost] => {
"msg": [
[
"node2.corp.net",
2582,
"aaa"
],
[
"node1.corp.net",
2581,
"bbbb"
]
]
}

Parse / Loop ansible registered variable

Trying to figure out how to filter out the list of UserNames in the output of the following playbook.
- name: Get all users
ome_user_info:
hostname: "{{ dellome_hostname }}"
username: "{{ dellome_username }}"
password: "{{ dellome_password }}"
register: users
Now the output provides the following:
ok: [192.168.1.100] => {
"users": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python"
},
"changed": false,
"deprecations": [
{
"msg": "Distribution Ubuntu 18.04 on host 192.168.1.100 should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.8/reference_appendices/interpreter_discovery.html for more information",
"version": "2.12"
}
],
"failed": false,
"user_info": {
"192.168.1.100": {
"#odata.context": "/api/$metadata#Collection(AccountService.Account)",
"#odata.count": 3,
"value": [
{
"#odata.id": "/api/AccountService/Accounts('10066')",
"#odata.type": "#AccountService.Account",
"Description": "admin",
"DirectoryServiceId": 0,
"Enabled": true,
"Id": "10066",
"IsBuiltin": true,
"Locked": false,
"Name": "admin",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10066')/Permissions",
"RoleId": "10",
"UserName": "admin",
"UserTypeId": 1
},
{
"#odata.id": "/api/AccountService/Accounts('10102')",
"#odata.type": "#AccountService.Account",
"Description": null,
"DirectoryServiceId": 0,
"Enabled": true,
"Id": "10102",
"IsBuiltin": false,
"Locked": false,
"Name": "dell",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10102')/Permissions",
"RoleId": "10",
"UserName": "dell",
"UserTypeId": 1
},
{
"#odata.id": "/api/AccountService/Accounts('10233')",
"#odata.type": "#AccountService.Account",
"Description": null,
"DirectoryServiceId": 10232,
"Enabled": true,
"Id": "10233",
"IsBuiltin": false,
"Locked": false,
"Name": "Domain Users",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10233')/Permissions",
"RoleId": "10",
"UserName": "Domain Users",
"UserTypeId": 2
}
]
}
}
}
}
I was able to determine the following from the output of the registered variable users.
- debug:
var: "{{ users | length }}"
This provides me the length of 5. Which makes sense to an extent. If I start poking into the output I can then determine the following:
- debug:
var: "{{ users.user_info | length }}"
This shows me the length of 1 which makes sense. If I add the var using users.user_info I can then see the output below.
TASK [manage_users : debug] *************************************************************************************************************************************************************
ok: [192.168.1.100] => {
"users.user_info": {
"192.168.1.100": {
"#odata.context": "/api/$metadata#Collection(AccountService.Account)",
"#odata.count": 3,
"value": [
{
"#odata.id": "/api/AccountService/Accounts('10066')",
"#odata.type": "#AccountService.Account",
"Description": "admin",
"DirectoryServiceId": 0,
"Enabled": true,
"Id": "10066",
"IsBuiltin": true,
"Locked": false,
"Name": "admin",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10066')/Permissions",
"RoleId": "10",
"UserName": "admin",
"UserTypeId": 1
},
{
"#odata.id": "/api/AccountService/Accounts('10102')",
"#odata.type": "#AccountService.Account",
"Description": null,
"DirectoryServiceId": 0,
"Enabled": true,
"Id": "10102",
"IsBuiltin": false,
"Locked": false,
"Name": "dell",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10102')/Permissions",
"RoleId": "10",
"UserName": "dell",
"UserTypeId": 1
},
{
"#odata.id": "/api/AccountService/Accounts('10233')",
"#odata.type": "#AccountService.Account",
"Description": null,
"DirectoryServiceId": 10232,
"Enabled": true,
"Id": "10233",
"IsBuiltin": false,
"Locked": false,
"Name": "Domain Users",
"Password": null,
"Permissions#odata.navigationLink": "/api/AccountService/Accounts('10233')/Permissions",
"RoleId": "10",
"UserName": "Domain Users”,
"UserTypeId": 2
}
]
}
}
}
Trying to figure out how I can loop through and get an array of the following: value —> UserName. Essentially I am going to take the following value and loop through and delete users that don’t equal the following. Admin, dell, domain users.
Now one might say you would easily just say while not = to items - then that list would work - I first need to figure out how to search and get the values out. I have tried the following:
- debug:
var: users.user_info().value()
- debug:
var: users.user_info.find('UserName')
- debug:
msg: UserName
loop: users.user_info."192.168.1.100".value
#- debug:
# var: users.user_info."{{ dellome_hostname }}".UserName
#- debug:
# var: "(claims1 | from_json).value"
# msg: "{{ users.user_info.UserName | list }}"
# (output_text.stdout | from_json).ismaster
#- debug:
# msg: "{{ item }}"
#loop: "{{ users.user_info | from_json | list }}"
At the end of this once I understand how to get the data out i can then create a loop to execute the following:
---
- name: Delete a User in Dell OME
ome_user:
hostname: "{{ dellome_hostname }}"
username: "{{ dellome_username }}"
password: "{{ dellome_password }}"
state: "{{ requestedState }}"
name: "{{ requstedUserName }}"
This is where i can then add the loop to eliminate users that don't meet the list of names i provide. Any help would be greatly appreciated.
Here are some of the errors i have run into.
TASK [manage_users : debug] *************************************************************************************************************************************************************
fatal: [192.168.1.100]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{users.user_info().value()}}): 'dict' object is not callable"}
TASK [manage_users : debug] *************************************************************************************************************************************************************
fatal: [192.168.1.100]: FAILED! => {"msg": "template error while templating string: expected name or number. String: {{users.user_info.\"192.168.1.100\".value()}}"}
Or as i am looking at this - if i can figure out a way to create a loop that looks for the roleID and when it is not equal to 10 then delete the user.
Use json_query. The tasks below
- set_fact:
users_rm: "{{ users.user_info|
json_query('*.value[].UserName') }}"
- debug:
var: users_rm
give
users_rm:
- admin
- dell
- Domain Users
You are running the query at the host 192.168.1.100 and the dictionary users comprises the users from this single host only. If there are more hosts in the dictionary the asterisk '*' in the query above would select them all. It would be better to select users for the particular host the query is running at. For example, the task below gives the same result
- set_fact:
users_rm: "{{ users.user_info[inventory_hostname].value|
map(attribute='UserName')|
list }}"

Need to map json to list

I have register variable with below json as value and need to extract name and dn from this JSON to list. tried different options using set_fact without any luck.
{
"nodes": {
"status": -1,
"imdata": [
{
"fabricNode": {
"attributes": {
"status": "",
"dn": "topology/pod-1/node-1",
"name": "NOQCJALAB1"
}
}
},
{
"fabricNode": {
"attributes": {
"status": "",
"dn": "topology/pod-1/node-1",
"name": "NOQCJALAB2"
}
}
}
],
"totalCount": 2,
"changed": false,
"failed": false
},
"changed": false,
"_ansible_verbose_always": true,
"_ansible_no_log": false
}
You need to remove the extraneous calls to map('from_json') because the object is already a dict
- set_fact:
node_names: >-
{{ (nodes.stdout | from_json).data
| map(attribute='fabnode')
| map(attribute='attributes')
| map(attribute='dn')
| list
}}
You would only want those map('from_json') call if your data structure contained embedded JSON, like so:
- set_fact:
inner_value: >-
{{ (example_text | from_json).a_key
| map("from_json")
| map(attribute="inner_key")
| list
}}
vars:
example_text: |
{"a_key": ["{\"inner_key\": \"inner value\"}"]}
It's possible to use json_query. For example the tasks below
- set_fact:
my_list: "{{ nodes.imdata|
json_query('[].{dn: fabricNode.attributes.dn,
name: fabricNode.attributes.name}')
}}"
- debug:
var: my_list
give
"my_list": [
{
"dn": "topology/pod-1/node-1",
"name": "NOQCJALAB1"
},
{
"dn": "topology/pod-1/node-1",
"name": "NOQCJALAB2"
}
]

Resources