How to loop dictionary with nested dictionaries - ansible

from the json below I need to gather the interface name and the unit name value in a loop.
This is partial output of my json:
{
"configuration": {
"#": {
"junos:changed-localtime": "2021-04-30 12:47:05 PDT",
"junos:changed-seconds": "1619812025",
"xmlns": "http://xml.juniper.net/xnm/1.1/xnm"
},
"interfaces": {
"interface": [
{
"name": "irb",
"unit": [
{
"family": {
"inet": {
"address": [
{
"name": "100.111.10.4/24"
}
],
"mtu": 9100
}
},
"name": 4010
},
{
"family": {
"inet": {
"address": [
{
"name": "100.127.9.2/31"
}
]
}
},
"name": 4093
}
]
},
{
"name": "lo0",
"unit": [
{
"family": {
"inet": {
"address": [
{
"name": "100.127.0.32/32"
}
],
}
},
"name": 0
}
]
}
]
}
}
}
The playbook I'm using is below:
- name: Print response
debug:
msg: "{{item.name}}.{{item|json_query('unit[*].name')}}"
loop: "{{ test.parsed_output.configuration.interfaces.interface}}"
ignore_errors: yes
The problem that I'm having is this:
"msg": "lo0.[0]"
"msg": "irb.[4010, 4093]"
What I want to get is:
"msg": "lo0.0"
"msg": "irb.4010"
"msg": "irb.4093"
Same outside name with each unit.name, but I don't know how to do this.
Thanks.

Iterate with_subelements, e.g.
- debug:
msg: "{{ item.0.name }} {{ item.1.name }}"
with_subelements:
- "{{ test.parsed_output.configuration.interfaces.interface }}"
- unit
gives
msg: irb 4010
msg: irb 4093
msg: lo0 0

Related

ansible: Invalid data passed to 'loop'

I have this following playbook, which gather facts in google cloud and filter instance names we want.
tasks:
- name: get tags
delegate_to: localhost
gcp_compute_instance_info:
auth_kind: serviceaccount
service_account_file: "xxx"
zone: "xxx"
filters:
- "name:{{ names }}*"
project: "{{ project }}"
register: ilist
- name: Display output
debug: var=ilist['items']
- set_fact:
instances: "{{ instances|default([]) + [ item.name ] }}"
loop: "{{ ilist['items']}}"
when: item.name is regex(".*"+tag+"-[a-z0-9]{4}$")
- debug:
var: instances
I'm getting following error in the loop task
Invalid data passed to 'loop', it requires a list, got this instead:
<built-in method items of dict object at 0x7fe0f370aa70>. Hint: If you
passed a list/dict of just one element, try adding wantlist=True to
your lookup invocation or use q/query instead of lookup.
what I'm doing wrong here? thanks
and this is the output of ilist
{
"changed":false,
"failed":false,
"resources":[
{
"canIpForward":false,
"cpuPlatform":"Intel Haswell",
"creationTimestamp":"2020-09-14T04:05:58.316-07:00",
"deletionProtection":false,
"disks":[
{
"autoDelete":true,
"boot":true,
"deviceName":"persistent-disk-0",
"diskSizeGb":"10",
"guestOsFeatures":[
{
"type":"VIRTIO_SCSI_MULTIQUEUE"
}
],
"index":0,
"interface":"SCSI",
"kind":"compute#attachedDisk",
"licenses":[
"xxxx"
],
"mode":"READ_WRITE",
"source":"xxxx",
"type":"PERSISTENT"
}
],
"fingerprint":"3yOdLl6Hp8g=",
"id":"3503717950118097018",
"kind":"compute#instance",
"labelFingerprint":"IP52FxBvsW0=",
"labels":{
"block":"internal",
"component":"dev",
"function":"internal"
},
"lastStartTimestamp":"2020-09-14T04:06:26.016-07:00",
"machineType":"xxxx",
"metadata":{
"fingerprint":"sTp1wPeotCo=",
"items":[
{
"key":"instance-template",
"value":"xxxx
},
{
"key":"created-by",
"value":"xxxx"
}
],
"kind":"compute#metadata"
},
"name":"test-vm",
"networkInterfaces":[
{
"fingerprint":"ae26IqBpxVo=",
"kind":"compute#networkInterface",
"name":"nic0",
"network":"xxxx",
"networkIP":"xxxx"
}
],
"scheduling":{
"automaticRestart":true,
"onHostMaintenance":"MIGRATE",
"preemptible":false
},
"selfLink":"xxxxx",
"serviceAccounts":[
{
"email":"xxxx",
"scopes":[
"https://www.googleapis.com/auth/devstorage.read_only",
"https://www.googleapis.com/auth/logging.write",
"https://www.googleapis.com/auth/monitoring.write",
"https://www.googleapis.com/auth/pubsub",
"https://www.googleapis.com/auth/service.management.readonly",
"https://www.googleapis.com/auth/servicecontrol",
"https://www.googleapis.com/auth/trace.append"
]
}
],
"startRestricted":false,
"status":"RUNNING",
"tags":{
"fingerprint":"Oyf1u-BGqNA=",
"items":[
"no-ip-b"
]
},
"zone":"xxxx"
},
{
"canIpForward":false,
"cpuPlatform":"Intel Haswell",
"creationTimestamp":"2020-10-15T06:59:52.505-07:00",
"deletionProtection":false,
"disks":[
{
"autoDelete":true,
"boot":true,
"deviceName":"persistent-disk-0",
"diskSizeGb":"10",
"guestOsFeatures":[
{
"type":"VIRTIO_SCSI_MULTIQUEUE"
}
],
"index":0,
"interface":"SCSI",
"kind":"compute#attachedDisk",
"licenses":[
"xxxx"
],
"mode":"READ_WRITE",
"source":"xxxxx",
"type":"PERSISTENT"
}
],
"fingerprint":"-E0UpLFggow=",
"id":"5900118287465179960",
"kind":"compute#instance",
"labelFingerprint":"IP52FxBvsW0=",
"labels":{
"block":"internal",
"component":"dev",
"function":"internal"
},
"lastStartTimestamp":"2020-10-15T07:00:37.895-07:00",
"machineType":"xxxxx",
"metadata":{
"fingerprint":"05Oj3Bq6zb4=",
"items":[
{
"key":"instance-template",
"value":"xxxx"
},
{
"key":"created-by",
"value":"xxxx"
}
],
"kind":"compute#metadata"
},
"name":"test-vm",
"networkInterfaces":[
{
"fingerprint":"H24HFGkCFNg=",
"kind":"compute#networkInterface",
"name":"nic0",
"network":"xxxx",
"networkIP":"xxxx"
}
],
"scheduling":{
"automaticRestart":true,
"onHostMaintenance":"MIGRATE",
"preemptible":false
},
"selfLink":"xxxx",
"serviceAccounts":[
{
"email":"xxxx",
"scopes":[
"https://www.googleapis.com/auth/devstorage.read_only",
"https://www.googleapis.com/auth/logging.write",
"https://www.googleapis.com/auth/monitoring.write",
"https://www.googleapis.com/auth/pubsub",
"https://www.googleapis.com/auth/service.management.readonly",
"https://www.googleapis.com/auth/servicecontrol",
"https://www.googleapis.com/auth/trace.append"
]
}
],
"startRestricted":false,
"status":"RUNNING",
"tags":{
"fingerprint":"Oyf1u-BGqNA=",
"items":[
"no-ip-b"
]
},
"zone":"xxxx"
}
]
}
You can try with the query function
loop: {{ query('dict', ilist.items) }}

What the syntax would be to pull the "members" value out of this JSON where the name value is "PROD_poolgroup"

What the syntax would be to pull the "members" value out of this JSON where the name value is "PROD_poolgroup".
I think it should be [?config.name == "PROD_poolgroup"].config.members
I've tried other variations without success also.
"all_members": [
{
"config": {
"_last_modified": "1594434546441212",
"cloud_ref": "https://192.168.86.20/api/cloud/cloud-5a7a41cb-156e-43f4-ad28-031b27dac813",
"implicit_priority_labels": false,
"members": [{
"pool_ref": "https://192.168.86.20/api/pool/pool-cb436bba-000a-4d72-8ff5-7d6cdf1ac2fa",
"ratio": 1
}],
"min_servers": 0,
"name": "PROD_poolgroup",
"tenant_ref": "https://192.168.86.20/api/tenant/admin",
"url": "https://192.168.86.20/api/poolgroup/poolgroup-e08933c0-d142-4787-9990-45f94dfc6b89",
"uuid": "poolgroup-e08933c0-d142-4787-9990-45f94dfc6b89"
},
"uuid": "poolgroup-e08933c0-d142-4787-9990-45f94dfc6b89"},
{
"config": {
"_last_modified": "1594404797173635",
"cloud_ref": "https://192.168.86.20/api/cloud/cloud-5a7a41cb-156e-43f4-ad28-031b27dac813",
"implicit_priority_labels": false,
"members": [{
"pool_ref": "https://192.168.86.20/api/pool/pool-f98ed65c-00b2-4638-83df-8b89df79deec",
"ratio": 1
}],
"min_servers": 0,
"name": "QA_poolgroup",
"tenant_ref": "https://192.168.86.20/api/tenant/admin",
"url": "https://192.168.86.20/api/poolgroup/poolgroup-0b19b515-9ae6-41ef-9111-00e61470e615",
"uuid": "poolgroup-0b19b515-9ae6-41ef-9111-00e61470e615"},
"uuid": "poolgroup-0b19b515-9ae6-41ef-9111-00e61470e615"
}
]
Use either single quote or backquote for using literal expression to compare value of name. Below are two working examples:
Using single quote:
- debug:
msg: "{{ all_members | json_query(query) | list }}"
vars:
query: "[?config.name == 'PROD_poolgroup'].config.members"
Using literal expression:
- debug:
msg: "{{ all_members | json_query(query) | list }}"
vars:
query: '[?config.name == `"PROD_poolgroup"`].config.members'

Ansible JSON parsing empty string

I am trying to learn JSON parsing using jmesquery in Ansible.
Please consider the following play:
---
- name: GET ALL THE INTERFACES
junos_command:
commands: show configuration interfaces | display json
register: A
- name: DISPLAY VARIABLE A CONTENTS
debug:
var: A.stdout_lines
- name: JOSN QUERY TO STORE PORTS IN NEW VARIABLE ALL_PORTS
set_fact:
ALL_PORTS: "{{ A.stdout_lines | json_query(jmesquery) }}"
vars:
jmesquery: 'configuration.interfaces.interface[*].name'
- name: DISPLAY VARIABLE ALL_PORTS CONTENTS
debug:
var: ALL_PORTS
#
Based on the JSON query, the port ge-0/0/0 will be stored in ALL_PORT, but we do not see that when we run the playbook, debug on ALL_PORT shows it is empty.
PLAY [ROUTER-STIG-PLAYBOOK] ************************************************************************************************
TASK [STIG_ROUTER : GET ALL THE INTERFACES] ********************************************************************************
ok: [192.168.22.9]
TASK [STIG_ROUTER : DISPLAY VARIABLE A CONTENTS] ***************************************************************************
ok: [192.168.22.9] => {
"A.stdout_lines": [
{
"configuration": {
"#": {
"junos:changed-localtime": "2020-05-10 11:14:49 UTC",
"junos:changed-seconds": "1589109289",
"xmlns": "http://xml.juniper.net/xnm/1.1/xnm"
},
"interfaces": {
"interface": [
{
"name": "ge-0/0/0",
"unit": [
{
"family": {
"inet": {
"address": [
{
"name": "192.168.22.9/24"
}
]
}
},
"name": 0
}
]
}
]
},
"security": {
"policies": {
"global": {
"policy": [
{
"match": {
"application": [
"any"
],
"destination-address": [
"any"
],
"source-address": [
"any"
]
},
"name": "TEST",
"then": {
"permit": [
null
]
}
}
]
}
},
"screen": {
"ids-option": [
{
"icmp": {
"ping-death": [
null
]
},
"ip": {
"source-route-option": [
null
],
"tear-drop": [
null
]
},
"name": "untrust-screen",
"tcp": {
"land": [
null
],
"syn-flood": {
"alarm-threshold": 1024,
"attack-threshold": 200,
"destination-threshold": 2048,
"queue-size": 2000,
"source-threshold": 1024,
"timeout": 20
}
}
}
]
},
"zones": {
"security-zone": [
{
"name": "trust",
"tcp-rst": [
null
]
},
{
"name": "untrust",
"screen": "untrust-screen"
},
{
"host-inbound-traffic": {
"protocols": [
{
"name": "all"
}
],
"system-services": [
{
"name": "all"
}
]
},
"interfaces": [
{
"name": "ge-0/0/0.0"
}
],
"name": "A"
}
]
}
},
"system": {
"license": {
"autoupdate": {
"url": [
{
"name": "https://ae1.juniper.net/junos/key_retrieval"
}
]
}
},
"root-authentication": {
"encrypted-password": "$6$thcHCjAV$e3o5ZRNWv7WtysOxuKpBP2X0cA3QDNtWYyCSBAUkImSEsulEGTgfwEQBa12Wll0fegpwvZfTHLvCbDUIW1n211"
},
"services": {
"ftp": [
null
],
"netconf": {
"ssh": [
null
]
},
"ssh": {
"root-login": "allow"
}
},
"syslog": {
"file": [
{
"contents": [
{
"any": [
null
],
"name": "any"
},
{
"info": [
null
],
"name": "authorization"
}
],
"name": "messages"
},
{
"contents": [
{
"any": [
null
],
"name": "interactive-commands"
}
],
"name": "interactive-commands"
}
],
"user": [
{
"contents": [
{
"emergency": [
null
],
"name": "any"
}
],
"name": "*"
}
]
}
},
"version": "20191026.124700_builder.r1063854"
}
}
]
}
TASK [STIG_ROUTER : JOSN QUERY TO STORE PORTS IN NEW VARIABLE ALL_PORTS] ***************************************************
ok: [192.168.22.9]
TASK [STIG_ROUTER : DISPLAY VARIABLE ALL_PORTS CONTENTS] *******************************************************************
ok: [192.168.22.9] => {
"ALL_PORTS": ""
}
#
I used online JSON query tester.
https://jsonpath.com/
When I checked my my query "configuration.interfaces.interface[*].name" against A.stdout_lines, I found the expected response i.e
[
ge-0/0/0
]
Any feedback/guidance is appreciated!!
Have a good weekend!!
Your JMESPath may very well be correct for one object but as its name implies, and as your debug: var=A.stdout_lines shows, stdout_lines is a list
So, you can do one of several things:
recognize that your stdout_lines only contains one object and just feed that into json_query as {{ A.stdout_lines[0] | json_query(jmesquery) }}
use the map filter to apply that same filter to every list item, with something like {{ A.stdout_lines | map("json_query", jmesquery) | list }}
rewrite your JMESPath to apply that filter to the input list, akin to json_query("[*].configuration.AND THE REST HERE")
Those last two will naturally produce a different output shape, since they are lists of lists, and so it will look like [['ge-0/0/0']] when output, but they do have the advantage that if your stdout_lines ever does mysteriously start to contain more objects, they will be json_query-ied as expected

Ansible select sub list from list of dictionaries with matching regex values

I have a yml list that is similar to the following:
name_ips:
rr: [
{
"name": "db1-137.com",
"ip": "1.1.1.1"
},
{
"name": "core1-137.com",
"ip": "2.2.2.2"
},
{
"name": "core0-139.com",
"ip": "3.3.3.3"
},
{
"name": "db0-147.com",
"ip": "4.4.4.4"
}
]
Using ansible I want to select the items in this list with name values matching the regex:
regexp: ".*137.com"
My end result should look similar to the following, which I want to be able to set using the set_fact module:
[
{
"name": "db1-137.com",
"ip": "1.1.1.1"
},
{
"name": "core1-137.com",
"ip": "2.2.2.2"
}
]
How can I achieve this sub-list select operation?
Update:
I found I can return a single value if I state the exact name:
- debug: msg="{{ name_ips | json_query(\"rr[?name=='core0-139.com']\") }}"`
Returns:
{
"name": "core0-139.com",
"ip": "3.3.3.3"
}
This is a start, but I would still like to use a regex for this search.

Ansible : Accessing value using template

Here is my json :
{
"appId": "usecase-watchlist",
"instanceName": "usecase-watchlist-instance",
"node": "master",
"configuration": {
"videoSensorInstance": {
"node": "worker",
"configuration": {
"cameraSensor": 1
}
},
"faceDetectionInstance": {
"node": "worker",
"configuration": {
"cameraSensor": 1,
"database": {
"name": "facedetection_db",
"dbmsName": "mongodb",
"type": "mongodb",
"username": "fb719a05",
"password": "b5f64277"
}
}
},
"alerts-manager-instance": {
"node": "worker",
"configuration": {
"database": {
"name": "alerts_db",
"dbmsName": "mongodb",
"type": "mongodb",
"username": "fb719a05",
"password": "b5f64277"
}
}
}
}
}
I have a variable named : app_spec_contents where the content of this json is present.
Also, i have a dictionary app_instances which contains :
app_instances contains :
"app_instances": {
"alerts-manager": "alertsManagerInstance",
"video-sensor": "videoSensorInstance",...
}
Now i am writing below play that will create a map of given form :
#Get the configuration of instances one by one
- name: Get the configuration of instances one by one
set_fact:
app_configurations: "{{ app_configurations|default({}) | combine( { item.key:{ item.key + '_db_connection_string': app_spec_contents.configuration } } ) }}"
with_items : "{{ app_instances|dict2items }}"
I want a dict of below result :
{"alerts-manager" : {
"alerts-manager-instance_db_connection_string" :
"mongodb://fb719a05:b5f64277#172.17.0.2:27017/alerts_db",
"name":"alerts_db"
},
... } ---> for every item in dict if database is there, like in one instance.
Also, a when condition is to be written if the instance which is present in app_spec_contents.{{ item.value }} has configuration.database in it.
Kindly Help.
This has been resolved :
Below is the correct yaml :
- name: Get the configuration of instances one by one
set_fact:
app_configurations: "{{ app_configurations|default({}) | combine( { item.key:{ item.key + '_db_connection_string': app_spec_contents.configuration[item.value].configuration.database.dbmsName + '://' + app_spec_contents.configuration[item.value].configuration.database.username + ':' + app_spec_contents.configuration[item.value].configuration.database.password + '#172.17.0.2:27017/' + app_spec_contents.configuration[item.value].configuration.database.name , item.key + '_db_name' : app_spec_contents.configuration[item.value].configuration.database.name } } ) }}"
when: app_spec_contents.configuration[item.value].configuration.database is defined and app_spec_contents.configuration[item.value].configuration.database.dbmsName=="mongodb"
with_items : "{{ app_instances|dict2items }}"

Resources