Ansible: How to modify a list of dicts - ansible

I want to modify dicts in a list within hostvars with a new entry for the IP address I get from the IPAM.
{
"vm_guest_networks": [
{
"device_type": "vmxnet3",
"state": "present",
"subnet": "10.91.1.0/24"
},
{
"device_type": "vmxnet3",
"state": "present",
"subnet": "10.91.0.0/24"
}
]
}
Within a loop I have the subnet to identify the right dict and the IP address I want to add with the ipv4_address key so the result should look like:
{
"vm_guest_networks": [
{
"device_type": "vmxnet3",
"state": "present",
"subnet": "10.91.1.0/24",
"ipv4_address": "10.91.1.216"
},
{
"device_type": "vmxnet3",
"state": "present",
"subnet": "10.91.0.0/24",
"ipv4_address": "10.91.0.21"
}
]
}
The current WIP ansible code is at https://pastebin.com/bFc1Ww2K

Let's assume the list of IPs is available. For example
ip4: [10.91.1.216, 10.91.0.21]
Let's use combine filter and Extended loop variables to create a new list where each dictionary will be updated. For example
- set_fact:
mydata: "{{ mydata|default([]) +
[item|combine({'ipv4_address': ip4[ansible_loop.index0]})] }}"
loop: "{{ vm_guest_networks }}"
loop_control:
extended: yes
- set_fact:
vm_guest_networks: "{{ mydata }}"
- debug:
var: vm_guest_networks
give
"vm_guest_networks": [
{
"device_type": "vmxnet3",
"ipv4_address": "10.91.1.216",
"state": "present",
"subnet": "10.91.1.0/24"
},
{
"device_type": "vmxnet3",
"ipv4_address": "10.91.0.21",
"state": "present",
"subnet": "10.91.0.0/24"
}
]

Related

How to Read and Output json Response using Ansible

How to read/display the following response (e.g: name, srcinft, dstinf) from FortiGate Firewall using Ansible. Or is there any way I can read this JSON output from file and display the fields i want.
{
"changed": false,
"meta": {
"http_method": "GET",
"size": 2,
"matched_count": 2,
"next_idx": 1,
"revision": "ac9c4e1d722b74695dee4fb3ce4fcd12",
"results": [
{
"policyid": 1,
"q_origin_key": 1,
"status": "enable",
"name": "test-policy01",
"uuid": "c4de3298-97ce-51ed-ccba-cafc556ba9e0",
"uuid-idx": 14729,
"srcintf": [
{
"name": "port2",
"q_origin_key": "port2"
}
],
"dstintf": [
{
"name": "port1",
"q_origin_key": "port1"
}
],
"action": "accept",
"ztna-status": "disable",
"srcaddr": [
{
"name": "all",
"q_origin_key": "all"
}
],
"dstaddr": [
{
"name": "all",
"q_origin_key": "all"
}
],
"policy-expiry": "disable",
"policy-expiry-date": "0000-00-00 00:00:00",
"service": [
{
"name": "ALL",
"q_origin_key": "ALL"
}
],
"tos": "0x00",
"sgt-check": "disable",
"sgt": []
},
{
"policyid": 2,
"q_origin_key": 2,
"status": "enable",
"name": "test-policy-02",
"uuid": "534b6c9c-97d1-51ed-7aa8-7544c628c7ea",
"uuid-idx": 14730,
"srcintf": [
{
"name": "port1",
"q_origin_key": "port1"
}
],
"dstintf": [
{
"name": "port2",
"q_origin_key": "port2"
}
],
"action": "accept",
"nat64": "disable",
"nat46": "disable",
"ztna-status": "disable",
"srcaddr": [
{
"name": "all",
"q_origin_key": "all"
}
],
"dstaddr": [
{
"name": "login.microsoft.com",
"q_origin_key": "login.microsoft.com"
}
],
"srcaddr6": [],
"reputation-direction6": "destination",
"policy-expiry-date": "0000-00-00 00:00:00",
"service": [
{
"name": "ALL_ICMP6",
"q_origin_key": "ALL_ICMP6"
}
],
"tos": "0x00",
"webcache": "disable",
"webcache-https": "disable",
"sgt-check": "disable",
"sgt": []
}
],
"vdom": "root",
"path": "firewall",
"name": "policy",
"version": "v7.2.3",
"build": 1262
},
"invocation": {
"module_args": {
"vdom": "root",
"access_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
"selector": "firewall_policy",
"selectors": null
}
},
"_ansible_no_log": false
}
Expected result:
result:
test-policy-02:
dstintf:
- name: port2
q_origin_key: port2
srcintf:
- name: port1
q_origin_key: port1
test-policy01:
dstintf:
- name: port1
q_origin_key: port1
srcintf:
- name: port2
q_origin_key: port2
Unfortunately there is absolute no description or any further information.
However, regarding
How to read/display the following response (e.g: name, srcinft, dstinf) from FortiGate Firewall using Ansible.
you may have a look into the following simple and lazy approach with loop
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Include vars of stuff.yaml into the 'stuff' variable (2.2).
ansible.builtin.include_vars:
file: stuff.json
name: stuff
- name: Show list of dict
debug:
msg: "{{ stuff.meta.results }}"
- name: Print key:value
debug:
msg:
- "name: {{ item.name }}"
- "{{ item.srcintf }}"
- "{{ item.dstintf }}"
loop_control:
label: "policyid: {{ item.policyid }}"
loop: "{{ stuff.meta.results }}"
resulting into an output of
TASK [Print key:value] *****************
ok: [localhost] => (item=policyid: 1) =>
msg:
- 'name: test-policy01'
- - name: port2
q_origin_key: port2
- - name: port1
q_origin_key: port1
ok: [localhost] => (item=policyid: 2) =>
msg:
- 'name: test-policy-02'
- - name: port1
q_origin_key: port1
- - name: port2
q_origin_key: port2
Further Documentation
include_vars module – Load variables from files, dynamically within a task
Loops
Or is there any way I can read this JSON output from file and display the fields I want?
To get familiar with data structure, respective JSON response you've provided in your example, you could start with something like a JSONPathFinder. It will result into an path of
x.meta.results[0].name
x.meta.results[0].srcintf
x.meta.results[0].dstintf
x.meta.results[1].name
x.meta.results[1].srcintf
x.meta.results[1].dstintf
for the provided keys.
It is also possible to use jq on CLI
jq keys stuff.json
[
"_ansible_no_log",
"changed",
"invocation",
"meta"
]
and just proceed further with
jq '.meta.results' stuff.json
jq '.meta.results[0]' stuff.json
jq '.meta.results[1]' stuff.json
Further Q&A
How to get key names from JSON using jq?

Ansible Filtering with Values within Dictionary of the List (FortiOS Facts)

I am tyring to use filter with Values within Dictionary of the List (FortiOS Facts). Sample Resp onse from FortiGate (Facts)
{
"changed": false,
"meta": {
"http_method": "GET",
"size": 2,
"matched_count": 2,
"next_idx": 1,
"revision": "ac9c4e1d722b74695dee4fb3ce4fcd12",
"results": [
{
"policyid": 1,
"q_origin_key": 1,
"status": "enable",
"name": "test-policy01",
"uuid": "c4de3298-97ce-51ed-ccba-cafc556ba9e0",
"uuid-idx": 14729,
"srcintf": [
{
"name": "port2",
"q_origin_key": "port2"
}
],
"dstintf": [
{
"name": "port1",
"q_origin_key": "port1"
}
],
"action": "accept",
"ztna-status": "disable",
"srcaddr": [
{
"name": "all",
"q_origin_key": "all"
}
],
Ansible Filter works fine for the following code
- name: To Fetch Existing Firewall Polices Based on Selector firewall_policy
fortios_configuration_fact:
vdom: "{{ vdom }}"
access_token: "{{ fortigate_access_token }}"
selector: "firewall_policy"
filters:
- action=="accept"
register: existing_fw_policy_object
when: existing_fw_name is defined
- name: Display Existing Firewall Policy
debug:
msg: "{{ existing_fw_policy_object }}"
What is the way to filter on srcintf. Wanted to Filter Firewall Policy with port2.

How to remove null and replace with a string value in ansible

How to remove null and replace with a string novlaue?
I have below output stored in variable out1
{
"out1": [
[
{
"destination": "dest-a",
"interface": "e1/1",
"metric": "10",
"name": "A"
},
{
"destination": "dest-b",
"interface": "e1/2",
"metric": "10",
"name": "B"
},
{
"destination": "dest-c",
"interface": null,
"metric": "10",
"name": "C"
},
{
"destination": "dest-d",
"interface": null,
"metric": "10",
"name": "B"
}
]
]
}
I have a json_query in my code:
- debug: msg="{{out1 |json_query(myquery)}}"
vars:
myquery: "[].{dest: destination ,int: interface}"
register: out2
Above code will print the following:
{
"msg": [
{
"dest": "dest-a",
"int": "e1/1"
},
{
"dest": "dest-b",
"int": "e/12"
},
{
"dest": "dest-c",
"int": null
},
{
"dest": "dest-d",
"int": null
}
]
}
I want to replace or remove null with the string novalue.
I looked into some posts and found default("novalue") can do the trick but in my case it is not working. I tried following added default("novalue") to my debug task, but I am getting an error.
I am sure that the error resides in myquery, the way I interpret/understand default() might be wrong and might be used wrongly.
Can anyone help me here please?
- debug: msg="{{out1 |json_query(myquery)}}"
vars:
myquery: "[].{dest: destination ,int: interface|default("novalue")}"
register: out2
Otherwise, with another JMESPath expression to achieve this, you can use an or expression ||, that will display the value of interface or a string that you are free to define.
So, given your JSON:
[
{
"destination": "dest-a",
"interface": "e1/1",
"metric": "10",
"name": "A"
},
{
"destination": "dest-b",
"interface": "e1/2",
"metric": "10",
"name": "B"
},
{
"destination": "dest-c",
"interface": null,
"metric": "10",
"name": "C"
},
{
"destination": "dest-d",
"interface": null,
"metric": "10",
"name": "B"
}
]
And the JMESPath query
[].{dest: destination ,int: interface || 'novalue'}
This yields
[
{
"dest": "dest-a",
"int": "e1/1"
},
{
"dest": "dest-b",
"int": "e1/2"
},
{
"dest": "dest-c",
"int": "novalue"
},
{
"dest": "dest-d",
"int": "novalue"
}
]
And your task ends up being:
- debug:
msg: "{{ out1 | json_query(_query) }}"
vars:
_query: "[].{dest: destination ,int: interface || 'novalue')}"
register: out2
You are using the jinja2 default filter inside a jmespath (i.e. json_query) expression. This can't work.
You can use the jmespath function not_null in this case
The playbook:
---
- hosts: localhost
gather_facts: false
vars:
"out1": [
[
{
"destination": "dest-a",
"interface": "e1/1",
"metric": "10",
"name": "A"
},
{
"destination": "dest-b",
"interface": "e1/2",
"metric": "10",
"name": "B"
},
{
"destination": "dest-c",
"interface": null,
"metric": "10",
"name": "C",
},
{
"destination": "dest-d",
"interface": null,
"metric": "10",
"name": "B"
}
]
]
tasks:
- debug:
msg: "{{ out1 | json_query(myquery) }}"
vars:
myquery: >-
[].{dest: destination ,int: not_null(interface, 'no value')}
Gives:
PLAY [localhost] **************************************************************************************************************************************************************************************************************
TASK [debug] ******************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"dest": "dest-a",
"int": "e1/1"
},
{
"dest": "dest-b",
"int": "e1/2"
},
{
"dest": "dest-c",
"int": "no value"
},
{
"dest": "dest-d",
"int": "no value"
}
]
}
PLAY RECAP ********************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Q: "How to remove 'null' and replace it with a string 'no value'?"
A: The variable out1 is a list of lists. Let's iterate it and create out2 with null replaced by 'no value' string. In each loop create the list of interface attributes with null replaced. Combine this list with the item and add it to the new list, e.g.
- set_fact:
out2: "{{ out2|d([]) + [item|zip(_item)|map('combine')] }}"
loop: "{{ out1 }}"
vars:
_item: "{{ item|json_query(_query) }}"
_query: |
[].{interface: not_null(interface, 'no value')}
gives
out2:
- - destination: dest-a
interface: e1/1
metric: '10'
name: A
- destination: dest-b
interface: e1/2
metric: '10'
name: B
- destination: dest-c
interface: no value
metric: '10'
name: C
- destination: dest-d
interface: no value
metric: '10'
name: B

Usage of Ansible JMESPath in queries

I am using following JSON file:
sample.json:
{
"lldp_output['gathered']": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 12
},
"mode": "access",
"name": "GigabitEthernet0/1"
},
{
"name": "GigabitEthernet0/2"
}
]
}
And the playbook:
---
- hosts: localhost
gather_facts: no
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | community.general.json_query(lldp_output['gathered']) }}"
I get the following error:
TASK [Take 4] ********************************************************************************************
task path: /root/scripts/atest.yml:18
fatal: [localhost]: FAILED! => {
"msg": "Error in jmespath.search in json_query filter plugin:\n'lldp_output' is undefined"
}
How do I query the JSON shown so I get a list of all ports that have mode: trunk
When I run in a playbook:
---
- name: Find trunk ports
hosts: ios
tasks:
- name: Collect interface output
cisco.ios.ios_l2_interfaces:
config:
state: gathered
register:
"intf_output"
- debug:
var=intf_output
- name: Take 4
debug:
msg: "{{ intf_output | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"[?mode=='trunk']
The structure returned is like following:
{
"intf_output": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"failed": false,
"gathered": [
{
"name": "GigabitEthernet0/0"
},
{
"mode": "trunk",
"name": "GigabitEthernet0/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/2",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/3",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet1/0",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet1/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"name": "GigabitEthernet1/2"
},
{
"name": "GigabitEthernet1/3"
},
{
"name": "GigabitEthernet2/0"
},
{
"name": "GigabitEthernet2/1"
},
{
"name": "GigabitEthernet2/2"
},
{
"name": "GigabitEthernet2/3"
},
{
"name": "GigabitEthernet3/0"
},
{
"name": "GigabitEthernet3/1"
},
{
"name": "GigabitEthernet3/2"
},
{
"name": "GigabitEthernet3/3"
}
]
}
}
For each host I run against the playbook against.
The argument to json_query must be a string. Because you haven't quoted your argument, Ansible is looking for a variable named lldp_output. But you've got additonal problems, since you're trying to access a key named lldp_output['gathered'], but [ is a syntactically significant character in JSON (and JMESPath queries), so that needs to be escaped as well.
In order to avoid all sorts of quote escaping contortions, we can place the query itself into a variable, so that we have:
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"
Note that we are using the >- block quote operator, which means that the value of query is the literal string "lldp_output['gathered']", including the outer quotes.
That playbook outputs:
TASK [Take 4] *********************************************************************************
ok: [localhost] => {
"msg": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 12
},
"mode": "access",
"name": "GigabitEthernet0/1"
},
{
"name": "GigabitEthernet0/2"
}
]
}
To get just those systems with mode equal to trunk, just add that
criteria to your query:
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"[?mode=='trunk']
This will output:
TASK [Take 4] *********************************************************************************
ok: [localhost] => {
"msg": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
}
]
}
Update
Given the data you've shown in your updated question, things are much
simpler, because you don't have the weird quoting you had in the
original question. With intf_output defined as shown, you can
write:
tasks:
- name: Take 4
debug:
msg: "{{ intf_output | json_query(query) }}"
vars:
query: >-
gathered[?mode=='trunk']

Q: Ansible - How can I merge 2 lists of hashes with a common key/value pair

How can I merge 2 lists of hashes based on a key/value pair, using Ansible 2.4.4
"foo": [
{
"hostname": "web1.example.com",
"guid": "73e85eb2-2ad5-4699-8a09-adf658a11ff2"
},
{
"hostname": "web2.example.com",
"guid": "827025fe-f13c-4fc8-ba51-7ff582596bbd"
},
{
"hostname": "web3.example.com",
"guid": "bba27304-c1bb-4889-aa44-125626be8488"
}
]
"bar": [
{
"ipaddress": "1.1.1.1",
"guid": "73e85eb2-2ad5-4699-8a09-adf658a11ff2"
},
{
"ipaddress": "2.2.2.2",
"guid": "827025fe-f13c-4fc8-ba51-7ff582596bbd"
},
{
"ipaddress": "3.3.3.3",
"guid": "bba27304-c1bb-4889-aa44-125626be8488"
}
]
I want something like:
"foobar" : [
{
"hostname": "web1.example.com",
"guid": "73e85eb2-2ad5-4699-8a09-adf658a11ff2",
"ipaddress": "1.1.1.1"
},
{
"hostname": "web2.example.com",
"guid": "827025fe-f13c-4fc8-ba51-7ff582596bbd",
"ipaddress": "2.2.2.2"
},
{
"hostname": "web3.example.com",
"guid": "bba27304-c1bb-4889-aa44-125626be8488",
"ipaddress": "3.3.3.3"
}
]
I've looked into several Ansible / Jinja2 filters including combine, union, map, custom plugins, but not having much success. I need to be able to match on the guid.
not sure if there is a smarter way, but to achieve what you need you can use the nested query plugin to loop over the combinations of the elements from the 2 list variables, find the combinations that have the common field equal, and then construct a new dictionary element and append it to the "final" list variable.
playbook:
- hosts: localhost
gather_facts: false
vars:
foo:
- hostname: web1.example.com
guid: 73e85eb2-2ad5-4699-8a09-adf658a11ff2
- hostname: web2.example.com
guid: 827025fe-f13c-4fc8-ba51-7ff582596bbd
- hostname: web3.example.com
guid: bba27304-c1bb-4889-aa44-125626be8488
bar:
- ipaddress: 1.1.1.1
guid: 73e85eb2-2ad5-4699-8a09-adf658a11ff2
- ipaddress: 2.2.2.2
guid: 827025fe-f13c-4fc8-ba51-7ff582596bbd
- ipaddress: 3.3.3.3
guid: bba27304-c1bb-4889-aa44-125626be8488
tasks:
- name: merge lists
set_fact:
merged_list: "{{ merged_list|default([]) + [{ 'hostname': item[0].hostname, 'guid': item[0].guid, 'ipaddress': item[1].ipaddress }] }}"
when: "item[0].guid == item[1].guid"
loop: "{{ query('nested', foo, bar) }}"
- name: print results
debug:
var: merged_list
result:
TASK [print results] ************************************************************************************************************************************************************************************************
ok: [localhost] => {
"merged_list": [
{
"guid": "73e85eb2-2ad5-4699-8a09-adf658a11ff2",
"hostname": "web1.example.com",
"ipaddress": "1.1.1.1"
},
{
"guid": "827025fe-f13c-4fc8-ba51-7ff582596bbd",
"hostname": "web2.example.com",
"ipaddress": "2.2.2.2"
},
{
"guid": "bba27304-c1bb-4889-aa44-125626be8488",
"hostname": "web3.example.com",
"ipaddress": "3.3.3.3"
}
]
}

Resources