Ansible JSON parsing empty string - ansible

I am trying to learn JSON parsing using jmesquery in Ansible.
Please consider the following play:
---
- name: GET ALL THE INTERFACES
junos_command:
commands: show configuration interfaces | display json
register: A
- name: DISPLAY VARIABLE A CONTENTS
debug:
var: A.stdout_lines
- name: JOSN QUERY TO STORE PORTS IN NEW VARIABLE ALL_PORTS
set_fact:
ALL_PORTS: "{{ A.stdout_lines | json_query(jmesquery) }}"
vars:
jmesquery: 'configuration.interfaces.interface[*].name'
- name: DISPLAY VARIABLE ALL_PORTS CONTENTS
debug:
var: ALL_PORTS
#
Based on the JSON query, the port ge-0/0/0 will be stored in ALL_PORT, but we do not see that when we run the playbook, debug on ALL_PORT shows it is empty.
PLAY [ROUTER-STIG-PLAYBOOK] ************************************************************************************************
TASK [STIG_ROUTER : GET ALL THE INTERFACES] ********************************************************************************
ok: [192.168.22.9]
TASK [STIG_ROUTER : DISPLAY VARIABLE A CONTENTS] ***************************************************************************
ok: [192.168.22.9] => {
"A.stdout_lines": [
{
"configuration": {
"#": {
"junos:changed-localtime": "2020-05-10 11:14:49 UTC",
"junos:changed-seconds": "1589109289",
"xmlns": "http://xml.juniper.net/xnm/1.1/xnm"
},
"interfaces": {
"interface": [
{
"name": "ge-0/0/0",
"unit": [
{
"family": {
"inet": {
"address": [
{
"name": "192.168.22.9/24"
}
]
}
},
"name": 0
}
]
}
]
},
"security": {
"policies": {
"global": {
"policy": [
{
"match": {
"application": [
"any"
],
"destination-address": [
"any"
],
"source-address": [
"any"
]
},
"name": "TEST",
"then": {
"permit": [
null
]
}
}
]
}
},
"screen": {
"ids-option": [
{
"icmp": {
"ping-death": [
null
]
},
"ip": {
"source-route-option": [
null
],
"tear-drop": [
null
]
},
"name": "untrust-screen",
"tcp": {
"land": [
null
],
"syn-flood": {
"alarm-threshold": 1024,
"attack-threshold": 200,
"destination-threshold": 2048,
"queue-size": 2000,
"source-threshold": 1024,
"timeout": 20
}
}
}
]
},
"zones": {
"security-zone": [
{
"name": "trust",
"tcp-rst": [
null
]
},
{
"name": "untrust",
"screen": "untrust-screen"
},
{
"host-inbound-traffic": {
"protocols": [
{
"name": "all"
}
],
"system-services": [
{
"name": "all"
}
]
},
"interfaces": [
{
"name": "ge-0/0/0.0"
}
],
"name": "A"
}
]
}
},
"system": {
"license": {
"autoupdate": {
"url": [
{
"name": "https://ae1.juniper.net/junos/key_retrieval"
}
]
}
},
"root-authentication": {
"encrypted-password": "$6$thcHCjAV$e3o5ZRNWv7WtysOxuKpBP2X0cA3QDNtWYyCSBAUkImSEsulEGTgfwEQBa12Wll0fegpwvZfTHLvCbDUIW1n211"
},
"services": {
"ftp": [
null
],
"netconf": {
"ssh": [
null
]
},
"ssh": {
"root-login": "allow"
}
},
"syslog": {
"file": [
{
"contents": [
{
"any": [
null
],
"name": "any"
},
{
"info": [
null
],
"name": "authorization"
}
],
"name": "messages"
},
{
"contents": [
{
"any": [
null
],
"name": "interactive-commands"
}
],
"name": "interactive-commands"
}
],
"user": [
{
"contents": [
{
"emergency": [
null
],
"name": "any"
}
],
"name": "*"
}
]
}
},
"version": "20191026.124700_builder.r1063854"
}
}
]
}
TASK [STIG_ROUTER : JOSN QUERY TO STORE PORTS IN NEW VARIABLE ALL_PORTS] ***************************************************
ok: [192.168.22.9]
TASK [STIG_ROUTER : DISPLAY VARIABLE ALL_PORTS CONTENTS] *******************************************************************
ok: [192.168.22.9] => {
"ALL_PORTS": ""
}
#
I used online JSON query tester.
https://jsonpath.com/
When I checked my my query "configuration.interfaces.interface[*].name" against A.stdout_lines, I found the expected response i.e
[
ge-0/0/0
]
Any feedback/guidance is appreciated!!
Have a good weekend!!

Your JMESPath may very well be correct for one object but as its name implies, and as your debug: var=A.stdout_lines shows, stdout_lines is a list
So, you can do one of several things:
recognize that your stdout_lines only contains one object and just feed that into json_query as {{ A.stdout_lines[0] | json_query(jmesquery) }}
use the map filter to apply that same filter to every list item, with something like {{ A.stdout_lines | map("json_query", jmesquery) | list }}
rewrite your JMESPath to apply that filter to the input list, akin to json_query("[*].configuration.AND THE REST HERE")
Those last two will naturally produce a different output shape, since they are lists of lists, and so it will look like [['ge-0/0/0']] when output, but they do have the advantage that if your stdout_lines ever does mysteriously start to contain more objects, they will be json_query-ied as expected

Related

Elastic \ Opensearch life cycle management - what is the difference between read_write & open actions

I want to use life cycle management, the goal is to delete messages after 14 days
What should be the action in the first stage? Open or Read_write
What is the difference between the two actions?
{
"policy": {
"policy_id": "delete_after14_days",
"description": "index delete"
"schema_version": 1,
"error_notification": null,
"default_state": "open",
"states": [
{
"name": "hot",
"actions": [
{
**"open": {} or "read_write": {}**
}
],
"transitions": [
{
"state_name": "delete",
"conditions": {
"min_index_age": "14d"
}
}
]
},
{
"name": "delete",
"actions": [
{
"delete": {}
}
],
"transitions": []
}
],
"ism_template": [
{
"index_patterns": [
"audit-*"
],
"priority": 0
}
]
}
}

jq How to filter array and add element

using jq i'm trying to add data to a specific element in my json below :
{
"users": [
{
"username": "karim",
"queue": [
"default"
]
},
{
"username": "admin",
"queue": [
"apps",
"prod"
]
}
]
}
what i want to do is to add items in queue[] of user admin like this
{
"users": [
{
"username": "hive",
"queue": [
"default"
]
},
{
"username": "admin",
"queue": [
"apps",
"prod",
"dev"
]
}
]
}
This is the command i used
jq '.users[] | select(.username == "admin").queue += ["dev"]' file.json
But the result is not as expected
{
"username": "hive",
"queue": [
"default"
]
}
{
"username": "admin",
"queue": [
"apps",
"prod",
"dev"
]
}
Why users array doesn't appear ? I need to keep it in the result
With the pipe you are changing the context down to an array element, which is what you want for the selection. If you put parentheses around the pipe and the selection, you will keep the assignment and thus the filter's output on top-level:
jq '(.users[] | select(.username == "admin")).queue += ["dev"]'
{
"users": [
{
"username": "karim",
"queue": [
"default"
]
},
{
"username": "admin",
"queue": [
"apps",
"prod",
"dev"
]
}
]
}
Demo

How to loop dictionary with nested dictionaries

from the json below I need to gather the interface name and the unit name value in a loop.
This is partial output of my json:
{
"configuration": {
"#": {
"junos:changed-localtime": "2021-04-30 12:47:05 PDT",
"junos:changed-seconds": "1619812025",
"xmlns": "http://xml.juniper.net/xnm/1.1/xnm"
},
"interfaces": {
"interface": [
{
"name": "irb",
"unit": [
{
"family": {
"inet": {
"address": [
{
"name": "100.111.10.4/24"
}
],
"mtu": 9100
}
},
"name": 4010
},
{
"family": {
"inet": {
"address": [
{
"name": "100.127.9.2/31"
}
]
}
},
"name": 4093
}
]
},
{
"name": "lo0",
"unit": [
{
"family": {
"inet": {
"address": [
{
"name": "100.127.0.32/32"
}
],
}
},
"name": 0
}
]
}
]
}
}
}
The playbook I'm using is below:
- name: Print response
debug:
msg: "{{item.name}}.{{item|json_query('unit[*].name')}}"
loop: "{{ test.parsed_output.configuration.interfaces.interface}}"
ignore_errors: yes
The problem that I'm having is this:
"msg": "lo0.[0]"
"msg": "irb.[4010, 4093]"
What I want to get is:
"msg": "lo0.0"
"msg": "irb.4010"
"msg": "irb.4093"
Same outside name with each unit.name, but I don't know how to do this.
Thanks.
Iterate with_subelements, e.g.
- debug:
msg: "{{ item.0.name }} {{ item.1.name }}"
with_subelements:
- "{{ test.parsed_output.configuration.interfaces.interface }}"
- unit
gives
msg: irb 4010
msg: irb 4093
msg: lo0 0

how to sort Data Sources in terraform based on arguments

I use following terraform code to get a list of available db resources:
data "alicloud_db_instance_classes" "resources" {
instance_charge_type = "PostPaid"
engine = "PostgreSQL"
engine_version = "10.0"
category = "HighAvailability"
zone_id = "${data.alicloud_zones.rds_zones.ids.0}"
multi_zone = true
output_file = "./classes.txt"
}
And the output file looks like this:
[
{
"instance_class": "pg.x4.large.2",
"storage_range": {
"max": "500",
"min": "250",
"step": "250"
},
"zone_ids": [
{
"id": "cn-shanghai-MAZ1(b,c)",
"sub_zone_ids": [
"cn-shanghai-b",
"cn-shanghai-c"
]
}
]
},
{
"instance_class": "pg.x8.medium.2",
"storage_range": {
"max": "250",
"min": "250",
"step": "0"
},
"zone_ids": [
{
"id": "cn-shanghai-MAZ1(b,c)",
"sub_zone_ids": [
"cn-shanghai-b",
"cn-shanghai-c"
]
}
]
},
{
"instance_class": "rds.pg.c1.xlarge",
"storage_range": {
"max": "2000",
"min": "5",
"step": "5"
},
"zone_ids": [
{
"id": "cn-shanghai-MAZ1(b,c)",
"sub_zone_ids": [
"cn-shanghai-b",
"cn-shanghai-c"
]
}
]
},
{
"instance_class": "rds.pg.s1.small",
"storage_range": {
"max": "2000",
"min": "5",
"step": "5"
},
"zone_ids": [
{
"id": "cn-shanghai-MAZ1(b,c)",
"sub_zone_ids": [
"cn-shanghai-b",
"cn-shanghai-c"
]
}
]
}
]
And I want to get the one that's cheapest.
One way to do so is by sorting with storage-range.min, but how do I sort this list based on 'storage_range.min'?
Or I can filter by 'instance_class', but "alicloud_db_instance_classes" doesn't seem to like filter as it says: Error: data.alicloud_db_instance_classes.resources: : invalid or unknown key: filter
Any ideas?
The sort() function orders lexicographical and you have no simple key here.
You can use filtering with some code like this (v0.12)
locals {
best_db_instance_class_key = "rds.pg.s1.small"
best_db_instance_class = element( alicloud_db_instance_classes.resources, index(alicloud_db_instance_classes.resources.*.instance_class, best_db_instance_class_key) )
}
(Untested code)

Ansible - How to combine list attributes?

I have two separate lists. The first is a list (base_list) with basic parameters, and the second is a list (dev_list) with parameters for a specific stand.
"base_list": [
{
"name": "kibana",
"path": "kibana/conf/kibana.xml",
"src": "/Users/ansible/inventories/_base/group_vars/kibana/conf/kibana.xml"
},
{
"name": "logstash",
"path": "logstash/conf/logstash.yml",
"src": "/Users/ansible/inventories/_base/group_vars/logstash/conf/logstash.yml"
},
{
"name": "grafana",
"path": "grafana/conf/grafana.json",
"src": "/Users/ansible/inventories/_base/group_vars/grafana/conf/grafana.json"
},
{
"name": "grafana",
"path": "grafana/conf/nginx.json",
"src": "/Users/ansible/inventories/_base/group_vars/grafana/conf/nginx.json"
},
{
"name": "grafana",
"path": "grafana/conf/config.json",
"src": "/Users/ansible/inventories/_base/group_vars/grafana/conf/config.json"
},
]
"dev_list": [
{
"name": "kibana",
"path": "kibana/conf/kibana.xml",
"src": "/Users/ansible/inventories/dev-st/group_vars/kibana/conf/kibana.xml"
},
{
"name": "logstash",
"path": "logstash/conf/jvm.options",
"src": "/Users/ansible/inventories/dev-st/group_vars/logstash/conf/jvm.options"
}
]
My goal is to combine these two lists to get one item.name with several item.path and item.src. Paths that look like this:
"end_list": [
{
"name": "kibana",
"path": "kibana/conf/kibana.xml",
"src": "/Users/ansible/inventories/dev-st/group_vars/kibana/conf/kibana.xml"
},
{
"name": "logstash",
"path": [
"logstash/conf/logstash.yml",
"logstash/conf/jvm.options"
],
"src": [
"/Users/ansible/inventories/_base/group_vars/logstash/conf/logstash.yml",
"/Users/ansible/inventories/dev-st/group_vars/logstash/conf/jvm.options"
]
},
{
"name": "grafana",
"path": [
"grafana/conf/grafana.json",
"grafana/conf/nginx.json",
"grafana/conf/config.json"
]
"src": [
"/Users/ansible/inventories/_base/group_vars/grafana/conf/grafana.json",
"/Users/ansible/inventories/_base/group_vars/grafana/conf/nginx.json",
"/Users/ansible/inventories/_base/group_vars/grafana/conf/config.json"
]
},
]
What would be the best way to do this?
This would probably be easier with a custom Python filter, but here's a solution using Ansible's built-in filters:
---
- hosts: localhost
gather_facts: false
vars:
"base_list": [
{
"name": "kibana",
"path": "kibana/conf/kibana.xml",
"src": "/Users/ansible/inventories/_base/group_vars/kibana/conf/kibana.xml"
},
{
"name": "logstash",
"path": "logstash/conf/logstash.yml",
"src": "/Users/ansible/inventories/_base/group_vars/logstash/conf/logstash.yml"
},
{
"name": "grafana",
"path": "grafana/conf/grafana.json",
"src": "/Users/ansible/inventories/_base/group_vars/grafana/conf/grafana.json"
},
]
"dev_list": [
{
"name": "kibana",
"path": "kibana/conf/kibana.xml",
"src": "/Users/ansible/inventories/dev-st/group_vars/kibana/conf/kibana.xml"
},
{
"name": "logstash",
"path": "logstash/conf/jvm.options",
"src": "/Users/ansible/inventories/dev-st/group_vars/logstash/conf/jvm.options"
}
]
tasks:
- set_fact:
end_list: >-
{{ end_list|default([]) + [
{
'name': item.0.name,
'path': item.1.path|ternary([item.0.path, item.1.path], item.0.path),
'src': item.1.src|ternary([item.0.src, item.1.src], item.1.src)
}
]}}
loop: >-
{{ base_list|zip_longest(dev_list,
fillvalue={'path': false, 'src': false})|list }}
- debug:
var: end_list
This was a little tricky to put together, so I'll try to describe the various parts:
The loop uses the zip_longest filter. Given the lists list1=[1, 2, 3] and list2=[11, 12], list1|zip_longest(list2) would produce [[1,11], [2,12], [3,None]] (that is, by default, zip_longest will use None as a fill value if one list is shorter than the other). By setting the fillvalue parameter, we can use a value other than None. In this case...
loop: >-
{{ base_list|zip_longest(dev_list,
fillvalue={'path': false, 'src': false})|list }}
...We're setting the fill value to a dictionary with stub values for path and src, since this makes the rest of the expression easier.
The meat of the solution is of course the set_fact action, which in simplified form looks like:
end_list: "{{ end_list|default([]) + [{...a dictionary...}] }}"
In other words, for each iteration of the loop, this will append a new dictionary to end_list.
We create the dictionary like this:
{
'name': item.0.name,
'path': item.1.path|ternary([item.0.path, item.1.path], item.0.path),
'src': item.1.src|ternary([item.0.src, item.1.src], item.1.src)
}
We're using the ternary filter here, which evaluates it's input as a boolean; if it's true, it selects the first argument, otherwise the second. Here we're taking advantage of the fillvalue we passed to the zip_longest filter: if dev_list is shorter than base_list, we'll have some items for which item.1.path and item.1.src are false, causing the ternary filter to select the second value (either item.0.path or item.1.src). In other cases, we build a list by combining the values from each of base_list and dev_list.
The result of running this playbook looks like:
ok: [localhost] => {
"end_list": [
{
"name": "kibana",
"path": [
"kibana/conf/kibana.xml",
"kibana/conf/kibana.xml"
],
"src": [
"/Users/ansible/inventories/_base/group_vars/kibana/conf/kibana.xml",
"/Users/ansible/inventories/dev-st/group_vars/kibana/conf/kibana.xml"
]
},
{
"name": "logstash",
"path": [
"logstash/conf/logstash.yml",
"logstash/conf/jvm.options"
],
"src": [
"/Users/ansible/inventories/_base/group_vars/logstash/conf/logstash.yml",
"/Users/ansible/inventories/dev-st/group_vars/logstash/conf/jvm.options"
]
},
{
"name": "grafana",
"path": "grafana/conf/grafana.json",
"src": false
}
]
}
Let me know if that helps, and whether or not the resulting data structure is what you were looking for. I had to make a few assumptions since your example end_list contained invalid syntax, so I took a guess at what you wanted.
Assuming you had well-formed json and those are properties on the root object, jq is perfectly suited for this. Group the contents of the arrays by name then generate the appropriate result objects.
$ jq '{
end_combine: (
.base_list + .dev_list
| group_by(.name)
| map({ name: .[0].name, path: map(.path), src: map(.src) })
)
}' input.json
{
"end_combine": [
{
"name": "grafana",
"path": [
"grafana/conf/grafana.json",
"grafana/conf/nginx.json",
"grafana/conf/config.json"
],
"src": [
"/Users/ansible/inventories/_base/group_vars/grafana/conf/grafana.json",
"/Users/ansible/inventories/_base/group_vars/grafana/conf/nginx.json",
"/Users/ansible/inventories/_base/group_vars/grafana/conf/config.json"
]
},
{
"name": "kibana",
"path": [
"kibana/conf/kibana.xml",
"kibana/conf/kibana.xml"
],
"src": [
"/Users/ansible/inventories/_base/group_vars/kibana/conf/kibana.xml",
"/Users/ansible/inventories/dev-st/group_vars/kibana/conf/kibana.xml"
]
},
{
"name": "logstash",
"path": [
"logstash/conf/logstash.yml",
"logstash/conf/jvm.options"
],
"src": [
"/Users/ansible/inventories/_base/group_vars/logstash/conf/logstash.yml",
"/Users/ansible/inventories/dev-st/group_vars/logstash/conf/jvm.options"
]
}
]
}

Resources