update user dict with list of values - ansible

Ansible 2.7, server and targets are running ubuntu.
At a point of my playbook, I have a users list of dicts:
ok: [virtual_tournesol] => {
"users": [
{
"key": "toto",
"value": [
"toto",
"/home/toto/.ssh/id_rsa"
]
},
{
"key": "riri",
"value": [
"www-data",
"/home/riri/.ssh/id_rsa"
]
}
]
}
I also fetch generated ssh keys within a pubkeys list:
ok: [virtual_tournesol] => {
"pubkeys": [
"ssh-rsa XXX1 ansible-generated on tournesol",
"ssh-rsa XXX2 ansible-generated on tournesol"
]
}
How could I merge both data such I can continue my tasks using users only ? I wish to get:
ok: [virtual_tournesol] => {
"users": [
{
"key": "toto",
"value": [
"toto",
"/home/toto/.ssh/id_rsa",
"ssh-rsa XXX1 ansible-generated on tournesol"
]
},
{
"key": "riri",
"value": [
"www-data",
"/home/riri/.ssh/id_rsa",
"ssh-rsa XXX2 ansible-generated on tournesol"
]
}
]
}
EDIT: In python, that would give something like:
for u, k in zip(users, pubkeys):
u['value'].append(k)

zip is a good choice. The code below
- set_fact:
user1: "{{ user1|default({}) | combine(
{ item.0.key: item.0.value + [ item.1 ] } ) }}"
loop: "{{ users|zip(pubkeys)|list }}"
- debug:
msg: "{{ user1|dict2items }}"
gives
"msg": [
{
"key": "riri",
"value": [
"www-data",
"/home/riri/.ssh/id_rsa",
"ssh-rsa XXX2 ansible-generated on tournesol"
]
},
{
"key": "toto",
"value": [
"toto",
"/home/toto/.ssh/id_rsa",
"ssh-rsa XXX1 ansible-generated on tournesol"
]
}
]

Related

How to Read and Output json Response using Ansible

How to read/display the following response (e.g: name, srcinft, dstinf) from FortiGate Firewall using Ansible. Or is there any way I can read this JSON output from file and display the fields i want.
{
"changed": false,
"meta": {
"http_method": "GET",
"size": 2,
"matched_count": 2,
"next_idx": 1,
"revision": "ac9c4e1d722b74695dee4fb3ce4fcd12",
"results": [
{
"policyid": 1,
"q_origin_key": 1,
"status": "enable",
"name": "test-policy01",
"uuid": "c4de3298-97ce-51ed-ccba-cafc556ba9e0",
"uuid-idx": 14729,
"srcintf": [
{
"name": "port2",
"q_origin_key": "port2"
}
],
"dstintf": [
{
"name": "port1",
"q_origin_key": "port1"
}
],
"action": "accept",
"ztna-status": "disable",
"srcaddr": [
{
"name": "all",
"q_origin_key": "all"
}
],
"dstaddr": [
{
"name": "all",
"q_origin_key": "all"
}
],
"policy-expiry": "disable",
"policy-expiry-date": "0000-00-00 00:00:00",
"service": [
{
"name": "ALL",
"q_origin_key": "ALL"
}
],
"tos": "0x00",
"sgt-check": "disable",
"sgt": []
},
{
"policyid": 2,
"q_origin_key": 2,
"status": "enable",
"name": "test-policy-02",
"uuid": "534b6c9c-97d1-51ed-7aa8-7544c628c7ea",
"uuid-idx": 14730,
"srcintf": [
{
"name": "port1",
"q_origin_key": "port1"
}
],
"dstintf": [
{
"name": "port2",
"q_origin_key": "port2"
}
],
"action": "accept",
"nat64": "disable",
"nat46": "disable",
"ztna-status": "disable",
"srcaddr": [
{
"name": "all",
"q_origin_key": "all"
}
],
"dstaddr": [
{
"name": "login.microsoft.com",
"q_origin_key": "login.microsoft.com"
}
],
"srcaddr6": [],
"reputation-direction6": "destination",
"policy-expiry-date": "0000-00-00 00:00:00",
"service": [
{
"name": "ALL_ICMP6",
"q_origin_key": "ALL_ICMP6"
}
],
"tos": "0x00",
"webcache": "disable",
"webcache-https": "disable",
"sgt-check": "disable",
"sgt": []
}
],
"vdom": "root",
"path": "firewall",
"name": "policy",
"version": "v7.2.3",
"build": 1262
},
"invocation": {
"module_args": {
"vdom": "root",
"access_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"selector": "firewall_policy",
"selectors": null
}
},
"_ansible_no_log": false
}
Expected result:
result:
test-policy-02:
dstintf:
- name: port2
q_origin_key: port2
srcintf:
- name: port1
q_origin_key: port1
test-policy01:
dstintf:
- name: port1
q_origin_key: port1
srcintf:
- name: port2
q_origin_key: port2
Unfortunately there is absolute no description or any further information.
However, regarding
How to read/display the following response (e.g: name, srcinft, dstinf) from FortiGate Firewall using Ansible.
you may have a look into the following simple and lazy approach with loop
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Include vars of stuff.yaml into the 'stuff' variable (2.2).
ansible.builtin.include_vars:
file: stuff.json
name: stuff
- name: Show list of dict
debug:
msg: "{{ stuff.meta.results }}"
- name: Print key:value
debug:
msg:
- "name: {{ item.name }}"
- "{{ item.srcintf }}"
- "{{ item.dstintf }}"
loop_control:
label: "policyid: {{ item.policyid }}"
loop: "{{ stuff.meta.results }}"
resulting into an output of
TASK [Print key:value] *****************
ok: [localhost] => (item=policyid: 1) =>
msg:
- 'name: test-policy01'
- - name: port2
q_origin_key: port2
- - name: port1
q_origin_key: port1
ok: [localhost] => (item=policyid: 2) =>
msg:
- 'name: test-policy-02'
- - name: port1
q_origin_key: port1
- - name: port2
q_origin_key: port2
Further Documentation
include_vars module – Load variables from files, dynamically within a task
Loops
Or is there any way I can read this JSON output from file and display the fields I want?
To get familiar with data structure, respective JSON response you've provided in your example, you could start with something like a JSONPathFinder. It will result into an path of
x.meta.results[0].name
x.meta.results[0].srcintf
x.meta.results[0].dstintf
x.meta.results[1].name
x.meta.results[1].srcintf
x.meta.results[1].dstintf
for the provided keys.
It is also possible to use jq on CLI
jq keys stuff.json
[
"_ansible_no_log",
"changed",
"invocation",
"meta"
]
and just proceed further with
jq '.meta.results' stuff.json
jq '.meta.results[0]' stuff.json
jq '.meta.results[1]' stuff.json
Further Q&A
How to get key names from JSON using jq?

How to remove null and replace with a string value in ansible

How to remove null and replace with a string novlaue?
I have below output stored in variable out1
{
"out1": [
[
{
"destination": "dest-a",
"interface": "e1/1",
"metric": "10",
"name": "A"
},
{
"destination": "dest-b",
"interface": "e1/2",
"metric": "10",
"name": "B"
},
{
"destination": "dest-c",
"interface": null,
"metric": "10",
"name": "C"
},
{
"destination": "dest-d",
"interface": null,
"metric": "10",
"name": "B"
}
]
]
}
I have a json_query in my code:
- debug: msg="{{out1 |json_query(myquery)}}"
vars:
myquery: "[].{dest: destination ,int: interface}"
register: out2
Above code will print the following:
{
"msg": [
{
"dest": "dest-a",
"int": "e1/1"
},
{
"dest": "dest-b",
"int": "e/12"
},
{
"dest": "dest-c",
"int": null
},
{
"dest": "dest-d",
"int": null
}
]
}
I want to replace or remove null with the string novalue.
I looked into some posts and found default("novalue") can do the trick but in my case it is not working. I tried following added default("novalue") to my debug task, but I am getting an error.
I am sure that the error resides in myquery, the way I interpret/understand default() might be wrong and might be used wrongly.
Can anyone help me here please?
- debug: msg="{{out1 |json_query(myquery)}}"
vars:
myquery: "[].{dest: destination ,int: interface|default("novalue")}"
register: out2
Otherwise, with another JMESPath expression to achieve this, you can use an or expression ||, that will display the value of interface or a string that you are free to define.
So, given your JSON:
[
{
"destination": "dest-a",
"interface": "e1/1",
"metric": "10",
"name": "A"
},
{
"destination": "dest-b",
"interface": "e1/2",
"metric": "10",
"name": "B"
},
{
"destination": "dest-c",
"interface": null,
"metric": "10",
"name": "C"
},
{
"destination": "dest-d",
"interface": null,
"metric": "10",
"name": "B"
}
]
And the JMESPath query
[].{dest: destination ,int: interface || 'novalue'}
This yields
[
{
"dest": "dest-a",
"int": "e1/1"
},
{
"dest": "dest-b",
"int": "e1/2"
},
{
"dest": "dest-c",
"int": "novalue"
},
{
"dest": "dest-d",
"int": "novalue"
}
]
And your task ends up being:
- debug:
msg: "{{ out1 | json_query(_query) }}"
vars:
_query: "[].{dest: destination ,int: interface || 'novalue')}"
register: out2
You are using the jinja2 default filter inside a jmespath (i.e. json_query) expression. This can't work.
You can use the jmespath function not_null in this case
The playbook:
---
- hosts: localhost
gather_facts: false
vars:
"out1": [
[
{
"destination": "dest-a",
"interface": "e1/1",
"metric": "10",
"name": "A"
},
{
"destination": "dest-b",
"interface": "e1/2",
"metric": "10",
"name": "B"
},
{
"destination": "dest-c",
"interface": null,
"metric": "10",
"name": "C",
},
{
"destination": "dest-d",
"interface": null,
"metric": "10",
"name": "B"
}
]
]
tasks:
- debug:
msg: "{{ out1 | json_query(myquery) }}"
vars:
myquery: >-
[].{dest: destination ,int: not_null(interface, 'no value')}
Gives:
PLAY [localhost] **************************************************************************************************************************************************************************************************************
TASK [debug] ******************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"dest": "dest-a",
"int": "e1/1"
},
{
"dest": "dest-b",
"int": "e1/2"
},
{
"dest": "dest-c",
"int": "no value"
},
{
"dest": "dest-d",
"int": "no value"
}
]
}
PLAY RECAP ********************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Q: "How to remove 'null' and replace it with a string 'no value'?"
A: The variable out1 is a list of lists. Let's iterate it and create out2 with null replaced by 'no value' string. In each loop create the list of interface attributes with null replaced. Combine this list with the item and add it to the new list, e.g.
- set_fact:
out2: "{{ out2|d([]) + [item|zip(_item)|map('combine')] }}"
loop: "{{ out1 }}"
vars:
_item: "{{ item|json_query(_query) }}"
_query: |
[].{interface: not_null(interface, 'no value')}
gives
out2:
- - destination: dest-a
interface: e1/1
metric: '10'
name: A
- destination: dest-b
interface: e1/2
metric: '10'
name: B
- destination: dest-c
interface: no value
metric: '10'
name: C
- destination: dest-d
interface: no value
metric: '10'
name: B

Ansible search and query

Updated with suggestions from larsks.
With the following structure
"intf_output_ios": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"failed": false,
"gathered": [
{
"name": "GigabitEthernet0/0"
},
{
"mode": "trunk",
"name": "GigabitEthernet0/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"99",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/2",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"99",
"100"
],
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 30
},
"mode": "access",
"name": "GigabitEthernet0/3"
},
{
"name": "GigabitEthernet1/0"
},
{
"name": "GigabitEthernet1/1"
},
{
"name": "GigabitEthernet1/2"
},
{
"name": "GigabitEthernet1/3"
},
{
"name": "GigabitEthernet2/0"
},
{
"name": "GigabitEthernet2/1"
},
{
"name": "GigabitEthernet2/2"
},
{
"name": "GigabitEthernet2/3"
},
{
"name": "GigabitEthernet3/0"
},
{
"name": "GigabitEthernet3/1"
},
{
"name": "GigabitEthernet3/2"
},
{
"access": {
"vlan": 99
},
"mode": "access",
"name": "GigabitEthernet3/3"
}
]
}
To print only the ports in VLAN 30 use the following?
- name: "P901T6: Set fact to include only access ports - IOS"
set_fact:
access_ports_ios_2: "{{ intf_output_ios | json_query(query) }}"
vars:
query: >-
gathered[?access.vlan==`30`]
- name: "P901T7: Dump list of access ports - IOS"
debug:
var=access_ports_ios_2
NOTE: It is important to use 30 (with backticks) and not '30'
I have gone through https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#managing-list-variables without really understanding how to fix this. If someone has some good link that would be very useful
With a structure like
ok: [access01] => {
"access_ports_ios": [
{
"access": {
"vlan": 30
},
"mode": "access",
"name": "GigabitEthernet0/3"
},
{
"access": {
"vlan": 99
},
"mode": "access",
"name": "GigabitEthernet3/3"
}
]
}
To get ports in vlan 30 use:
- debug:
var: access_ports_ios|json_query(query)
vars:
query: >-
[?access.vlan==`30`]
Note:
If you want to use a variable for vlan instead of hard-coding it. I had to do as follows:
- name: Debug 4
debug:
var: access_ports_ios|json_query('[?access.vlan==`{{ src_vlan | int}}`]')
You're asking for gathered.access, but gathered is a list and does not have an access attribute. You want "all items from gathered for which access.vlan is 30 (and note that the value of access.vlan is an integer, not a string):
- debug:
var: intf_output_ios|json_query(query)
vars:
query: >-
gathered[?access.vlan==`30`]
Which given you example input produces:
TASK [debug] *******************************************************************
ok: [localhost] => {
"intf_output_ios|json_query(query)": [
{
"access": {
"vlan": 30
},
"mode": "access",
"name": "GigabitEthernet0/3"
}
]
}
I'm going to reiterate advice I often give for json_query questions: use something like jpterm or the JMESPath website to test JMESPath expressions against your actual data. This makes it much easier to figure out where an expression might be going wrong.

Usage of Ansible JMESPath in queries

I am using following JSON file:
sample.json:
{
"lldp_output['gathered']": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 12
},
"mode": "access",
"name": "GigabitEthernet0/1"
},
{
"name": "GigabitEthernet0/2"
}
]
}
And the playbook:
---
- hosts: localhost
gather_facts: no
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | community.general.json_query(lldp_output['gathered']) }}"
I get the following error:
TASK [Take 4] ********************************************************************************************
task path: /root/scripts/atest.yml:18
fatal: [localhost]: FAILED! => {
"msg": "Error in jmespath.search in json_query filter plugin:\n'lldp_output' is undefined"
}
How do I query the JSON shown so I get a list of all ports that have mode: trunk
When I run in a playbook:
---
- name: Find trunk ports
hosts: ios
tasks:
- name: Collect interface output
cisco.ios.ios_l2_interfaces:
config:
state: gathered
register:
"intf_output"
- debug:
var=intf_output
- name: Take 4
debug:
msg: "{{ intf_output | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"[?mode=='trunk']
The structure returned is like following:
{
"intf_output": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"failed": false,
"gathered": [
{
"name": "GigabitEthernet0/0"
},
{
"mode": "trunk",
"name": "GigabitEthernet0/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/2",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/3",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet1/0",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet1/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"name": "GigabitEthernet1/2"
},
{
"name": "GigabitEthernet1/3"
},
{
"name": "GigabitEthernet2/0"
},
{
"name": "GigabitEthernet2/1"
},
{
"name": "GigabitEthernet2/2"
},
{
"name": "GigabitEthernet2/3"
},
{
"name": "GigabitEthernet3/0"
},
{
"name": "GigabitEthernet3/1"
},
{
"name": "GigabitEthernet3/2"
},
{
"name": "GigabitEthernet3/3"
}
]
}
}
For each host I run against the playbook against.
The argument to json_query must be a string. Because you haven't quoted your argument, Ansible is looking for a variable named lldp_output. But you've got additonal problems, since you're trying to access a key named lldp_output['gathered'], but [ is a syntactically significant character in JSON (and JMESPath queries), so that needs to be escaped as well.
In order to avoid all sorts of quote escaping contortions, we can place the query itself into a variable, so that we have:
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"
Note that we are using the >- block quote operator, which means that the value of query is the literal string "lldp_output['gathered']", including the outer quotes.
That playbook outputs:
TASK [Take 4] *********************************************************************************
ok: [localhost] => {
"msg": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 12
},
"mode": "access",
"name": "GigabitEthernet0/1"
},
{
"name": "GigabitEthernet0/2"
}
]
}
To get just those systems with mode equal to trunk, just add that
criteria to your query:
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"[?mode=='trunk']
This will output:
TASK [Take 4] *********************************************************************************
ok: [localhost] => {
"msg": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
}
]
}
Update
Given the data you've shown in your updated question, things are much
simpler, because you don't have the weird quoting you had in the
original question. With intf_output defined as shown, you can
write:
tasks:
- name: Take 4
debug:
msg: "{{ intf_output | json_query(query) }}"
vars:
query: >-
gathered[?mode=='trunk']

Accessing the value of every key of a JSON formatted list in Ansible

I have the following list, which I have queried with the help of json_query from cisco controller.
"aci_access_port_list": [
[
{
"description": [
"NOMON KASTOM"
],
"port_name": "E1-17"
},
{
"description": [
"ASDF NOMON K"
],
"port_name": "E1-16"
},
{
"description": [
"NOMON K9876"
],
"port_name": "E1-15"
},
{
"description": [
"LAN5002 TC "
],
"port_name": "E1-46"
},
{
"description": [
"LAN5002 TC "
],
"port_name": "E1-47"
},
{
"description": [
"NOMON LAN5001 TC "
],
"port_name": "E1-45"
},
{
"description": [
"LAN5001 TC "
],
"port_name": "E1-48"
},
{
"description": [
"NOMON No Cabling - TEST-4-AUTOMATION"
],
"port_name": "E1-30"
},
{
"description": [
"NOMON No Cabling - TEST-6-AUTOMATION"
],
"port_name": "E1-31"
}
]
]
I want to check every description key and if it has "NOMON K" written save it into a new list. For this I wrote the following:
- set_fact:
clean_list: [\n]
- name: getting the clean list of ports and descr.
set_fact:
clean_list: "{{clean_list }} \n {{item.0.port_name}} - {{item.1}}"
loop: "{{ aci_access_port_list[0]|subelements('description') }}"
when: item.1 is search ("NOMON K")
- debug:
msg: "{{ clean_list }}"
and this returns:
ok: [acitst-apic.net.dsh.at] => {
"msg": "[u'\\\\n'] \n E1-17 - NOMON KASTOM \n E1-16 - ASDF NOMON K \n E1-15 - NOMON K9876"
}
What I want to have is same formatting as aci_access_port_lists but I couldn't figure it out. It should look like:
"clean_list":[
{
"description": [
"NOMON KASTOM"
],
"port_name": "E1-17"
},
{
"description": [
"ASDF NOMON K"
],
"port_name": "E1-16"
},
{
"description": [
"NOMON K9876"
],
"port_name": "E1-15"
}
]
I assume there is a better way but I couldn't find it and if somebody can help me I will be glad. Thank you in advance.
This is easilly feasible using the selectattr filter and the contains test, targeting the first element of the description list since it always has a single element. Of course, this approach will break if you have several elements in that list.
- name: Show me a clean list
vars:
clean_list: >-
{{
aci_access_port_list.0 |
selectattr('description.0', 'contains', 'NOMON K') |
list
}}
debug:
var: clean_list

Resources