How to remove null and replace with a string value in ansible - ansible

How to remove null and replace with a string novlaue?
I have below output stored in variable out1
{
"out1": [
[
{
"destination": "dest-a",
"interface": "e1/1",
"metric": "10",
"name": "A"
},
{
"destination": "dest-b",
"interface": "e1/2",
"metric": "10",
"name": "B"
},
{
"destination": "dest-c",
"interface": null,
"metric": "10",
"name": "C"
},
{
"destination": "dest-d",
"interface": null,
"metric": "10",
"name": "B"
}
]
]
}
I have a json_query in my code:
- debug: msg="{{out1 |json_query(myquery)}}"
vars:
myquery: "[].{dest: destination ,int: interface}"
register: out2
Above code will print the following:
{
"msg": [
{
"dest": "dest-a",
"int": "e1/1"
},
{
"dest": "dest-b",
"int": "e/12"
},
{
"dest": "dest-c",
"int": null
},
{
"dest": "dest-d",
"int": null
}
]
}
I want to replace or remove null with the string novalue.
I looked into some posts and found default("novalue") can do the trick but in my case it is not working. I tried following added default("novalue") to my debug task, but I am getting an error.
I am sure that the error resides in myquery, the way I interpret/understand default() might be wrong and might be used wrongly.
Can anyone help me here please?
- debug: msg="{{out1 |json_query(myquery)}}"
vars:
myquery: "[].{dest: destination ,int: interface|default("novalue")}"
register: out2

Otherwise, with another JMESPath expression to achieve this, you can use an or expression ||, that will display the value of interface or a string that you are free to define.
So, given your JSON:
[
{
"destination": "dest-a",
"interface": "e1/1",
"metric": "10",
"name": "A"
},
{
"destination": "dest-b",
"interface": "e1/2",
"metric": "10",
"name": "B"
},
{
"destination": "dest-c",
"interface": null,
"metric": "10",
"name": "C"
},
{
"destination": "dest-d",
"interface": null,
"metric": "10",
"name": "B"
}
]
And the JMESPath query
[].{dest: destination ,int: interface || 'novalue'}
This yields
[
{
"dest": "dest-a",
"int": "e1/1"
},
{
"dest": "dest-b",
"int": "e1/2"
},
{
"dest": "dest-c",
"int": "novalue"
},
{
"dest": "dest-d",
"int": "novalue"
}
]
And your task ends up being:
- debug:
msg: "{{ out1 | json_query(_query) }}"
vars:
_query: "[].{dest: destination ,int: interface || 'novalue')}"
register: out2

You are using the jinja2 default filter inside a jmespath (i.e. json_query) expression. This can't work.
You can use the jmespath function not_null in this case
The playbook:
---
- hosts: localhost
gather_facts: false
vars:
"out1": [
[
{
"destination": "dest-a",
"interface": "e1/1",
"metric": "10",
"name": "A"
},
{
"destination": "dest-b",
"interface": "e1/2",
"metric": "10",
"name": "B"
},
{
"destination": "dest-c",
"interface": null,
"metric": "10",
"name": "C",
},
{
"destination": "dest-d",
"interface": null,
"metric": "10",
"name": "B"
}
]
]
tasks:
- debug:
msg: "{{ out1 | json_query(myquery) }}"
vars:
myquery: >-
[].{dest: destination ,int: not_null(interface, 'no value')}
Gives:
PLAY [localhost] **************************************************************************************************************************************************************************************************************
TASK [debug] ******************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"dest": "dest-a",
"int": "e1/1"
},
{
"dest": "dest-b",
"int": "e1/2"
},
{
"dest": "dest-c",
"int": "no value"
},
{
"dest": "dest-d",
"int": "no value"
}
]
}
PLAY RECAP ********************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Q: "How to remove 'null' and replace it with a string 'no value'?"
A: The variable out1 is a list of lists. Let's iterate it and create out2 with null replaced by 'no value' string. In each loop create the list of interface attributes with null replaced. Combine this list with the item and add it to the new list, e.g.
- set_fact:
out2: "{{ out2|d([]) + [item|zip(_item)|map('combine')] }}"
loop: "{{ out1 }}"
vars:
_item: "{{ item|json_query(_query) }}"
_query: |
[].{interface: not_null(interface, 'no value')}
gives
out2:
- - destination: dest-a
interface: e1/1
metric: '10'
name: A
- destination: dest-b
interface: e1/2
metric: '10'
name: B
- destination: dest-c
interface: no value
metric: '10'
name: C
- destination: dest-d
interface: no value
metric: '10'
name: B

Related

Getting a value out of registered variable in Ansible

So, I'm provisioning an EC2 instance using the ec2 module. At the end I'm using register: ec2info, so, I can later reference values inside, such as instance ID and Public DNS.
- name: Print the results
debug:
var: ec2info
Gets me
TASK [Print the results] ***********************************************************************************************************************
ok: [localhost] => {
"ec2info": {
"changed": true,
"deprecations": [
{
"collection_name": "amazon.aws",
"msg": "The 'ec2' module has been deprecated and replaced by the 'ec2_instance' module'",
"version": "4.0.0"
}
],
"failed": false,
"instance_ids": [
"i-0e9de15eb82eda3ad"
],
"instances": [
{
"ami_launch_index": "0",
"architecture": "x86_64",
"block_device_mapping": {
"/dev/sda1": {
"delete_on_termination": true,
"status": "attached",
"volume_id": "vol-066bb2dd4d14bdcac"
}
},
"dns_name": "ec2-35-173-126-60.compute-1.amazonaws.com",
"ebs_optimized": false,
"groups": {
"sg-0a732c6cbeb3f1025": "launch-wizard-39"
},
"hypervisor": "xen",
"id": "i-0e9de15eb82eda3ad",
"image_id": "ami-06644055bed38ebd9",
"instance_type": "t2.micro",
"kernel": null,
"key_name": "daro.io",
"launch_time": "2022-02-01T02:05:22.000Z",
"placement": "us-east-1c",
"private_dns_name": "ip-172-31-90-87.ec2.internal",
"private_ip": "172.31.90.87",
"public_dns_name": "ec2-35-173-126-60.compute-1.amazonaws.com",
"public_ip": "35.173.126.60",
"ramdisk": null,
"region": "us-east-1",
"root_device_name": "/dev/sda1",
"root_device_type": "ebs",
"state": "running",
"state_code": 16,
"tags": {
"Name": "new_demo_template"
},
"tenancy": "default",
"virtualization_type": "hvm"
}
],
"tagged_instances": []
}
}
I would like to get the value of Public DNS I have tried
- name: Print the results
debug:
var: ec2info.instances.public_dns_name
## AND
var: ec2info.instances['public_dns_name']
## AND
var: '{{ec2info.instances.public_dns_name}}'
If you could also point out to the part of the documentation that covers this?
So, I figured it out
var: ec2info.instances[0].public_dns_name

Looping through Ansible Register output and printing the result as list

This is the sample task which give me the output which contains a list of self IPs. I want to capture this in Result and just print only the name of self IP.
I was able to print if I give a list index, but I can't loop through it to capture it as a separate list to use it in a different task.
- name: Port Lock Down - Allow Default
bigip_device_info:
gather_subset:
- self-ips
provider: "{{provider}}"
register: Result
- name : Display the selfips
debug:
var: Result
TASK [Display the Selfip config] **************************************************************************************************************************************************************************
ok: [LD-F5-01] => {
"Result": {
"ansible_facts": {
"ansible_net_queried": true,
"ansible_net_self_ips": [
{
"address": "3.3.3.3",
"allow_access_list": [
"tcp:8080",
"udp:53",
"gre:0",
"default"
],
"floating": "no",
"full_path": "/Common/test-self-ip-1",
"name": "test-self-ip-1",
"netmask": "255.255.255.0",
"netmask_cidr": 24,
"traffic_group": "/Common/traffic-group-local-only",
"traffic_group_inherited": "no",
"vlan": "/Common/test-vlan-1"
},
{
"address": "4.4.4.4",
"allow_access_list": "all",
"floating": "no",
"full_path": "/Common/test-self-ip-2",
"name": "test-self-ip-2",
"netmask": "255.255.255.0",
"netmask_cidr": 24,
"traffic_group": "/Common/traffic-group-local-only",
"traffic_group_inherited": "no",
"vlan": "/Common/test-vlan-1"
}
]
},
"changed": false,
"failed": false,
"queried": true,
"self_ips": [
{
"address": "3.3.3.3",
"allow_access_list": [
"tcp:8080",
"udp:53",
"gre:0",
"default"
],
"floating": "no",
"full_path": "/Common/test-self-ip-1",
"name": "test-self-ip-1",
"netmask": "255.255.255.0",
"netmask_cidr": 24,
"traffic_group": "/Common/traffic-group-local-only",
"traffic_group_inherited": "no",
"vlan": "/Common/test-vlan-1"
},
{
"address": "4.4.4.4",
"allow_access_list": "all",
"floating": "no",
"full_path": "/Common/test-self-ip-2",
"name": "test-self-ip-2",
"netmask": "255.255.255.0",
"netmask_cidr": 24,
"traffic_group": "/Common/traffic-group-local-only",
"traffic_group_inherited": "no",
"vlan": "/Common/test-vlan-1"
}
]
}
}
- name : Display the selfips
debug:
var: Result.self_ips[0]['name']
TASK [Display the selfips] ********************************************************************************************************************************************************************************
ok: [LD-F5-01] => {
"Result.self_ips[0]['name']": "test-self-ip-1"
}
Can someone help me to use the for loop to get all the self IPs names as list instead of one single output?
You actually do not need a loop to achieve this, you can use the map filter of Jinja instead:
Result.self_ips | map(attribute='name')
Given the task:
- debug:
var: Result.self_ips | map(attribute='name')
vars:
Result:
self_ips:
- address: 3.3.3.3
allow_access_list:
- 'tcp:8080'
- 'udp:53'
- 'gre:0'
- default
floating: 'no'
full_path: /Common/test-self-ip-1
name: test-self-ip-1
- address: 4.4.4.4
allow_access_list: all
floating: 'no'
full_path: /Common/test-self-ip-2
name: test-self-ip-2
This yields the result:
Result.self_ips | map(attribute='name'):
- test-self-ip-1
- test-self-ip-2

Ansible search and query

Updated with suggestions from larsks.
With the following structure
"intf_output_ios": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"failed": false,
"gathered": [
{
"name": "GigabitEthernet0/0"
},
{
"mode": "trunk",
"name": "GigabitEthernet0/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"99",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/2",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"99",
"100"
],
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 30
},
"mode": "access",
"name": "GigabitEthernet0/3"
},
{
"name": "GigabitEthernet1/0"
},
{
"name": "GigabitEthernet1/1"
},
{
"name": "GigabitEthernet1/2"
},
{
"name": "GigabitEthernet1/3"
},
{
"name": "GigabitEthernet2/0"
},
{
"name": "GigabitEthernet2/1"
},
{
"name": "GigabitEthernet2/2"
},
{
"name": "GigabitEthernet2/3"
},
{
"name": "GigabitEthernet3/0"
},
{
"name": "GigabitEthernet3/1"
},
{
"name": "GigabitEthernet3/2"
},
{
"access": {
"vlan": 99
},
"mode": "access",
"name": "GigabitEthernet3/3"
}
]
}
To print only the ports in VLAN 30 use the following?
- name: "P901T6: Set fact to include only access ports - IOS"
set_fact:
access_ports_ios_2: "{{ intf_output_ios | json_query(query) }}"
vars:
query: >-
gathered[?access.vlan==`30`]
- name: "P901T7: Dump list of access ports - IOS"
debug:
var=access_ports_ios_2
NOTE: It is important to use 30 (with backticks) and not '30'
I have gone through https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#managing-list-variables without really understanding how to fix this. If someone has some good link that would be very useful
With a structure like
ok: [access01] => {
"access_ports_ios": [
{
"access": {
"vlan": 30
},
"mode": "access",
"name": "GigabitEthernet0/3"
},
{
"access": {
"vlan": 99
},
"mode": "access",
"name": "GigabitEthernet3/3"
}
]
}
To get ports in vlan 30 use:
- debug:
var: access_ports_ios|json_query(query)
vars:
query: >-
[?access.vlan==`30`]
Note:
If you want to use a variable for vlan instead of hard-coding it. I had to do as follows:
- name: Debug 4
debug:
var: access_ports_ios|json_query('[?access.vlan==`{{ src_vlan | int}}`]')
You're asking for gathered.access, but gathered is a list and does not have an access attribute. You want "all items from gathered for which access.vlan is 30 (and note that the value of access.vlan is an integer, not a string):
- debug:
var: intf_output_ios|json_query(query)
vars:
query: >-
gathered[?access.vlan==`30`]
Which given you example input produces:
TASK [debug] *******************************************************************
ok: [localhost] => {
"intf_output_ios|json_query(query)": [
{
"access": {
"vlan": 30
},
"mode": "access",
"name": "GigabitEthernet0/3"
}
]
}
I'm going to reiterate advice I often give for json_query questions: use something like jpterm or the JMESPath website to test JMESPath expressions against your actual data. This makes it much easier to figure out where an expression might be going wrong.

Usage of Ansible JMESPath in queries

I am using following JSON file:
sample.json:
{
"lldp_output['gathered']": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 12
},
"mode": "access",
"name": "GigabitEthernet0/1"
},
{
"name": "GigabitEthernet0/2"
}
]
}
And the playbook:
---
- hosts: localhost
gather_facts: no
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | community.general.json_query(lldp_output['gathered']) }}"
I get the following error:
TASK [Take 4] ********************************************************************************************
task path: /root/scripts/atest.yml:18
fatal: [localhost]: FAILED! => {
"msg": "Error in jmespath.search in json_query filter plugin:\n'lldp_output' is undefined"
}
How do I query the JSON shown so I get a list of all ports that have mode: trunk
When I run in a playbook:
---
- name: Find trunk ports
hosts: ios
tasks:
- name: Collect interface output
cisco.ios.ios_l2_interfaces:
config:
state: gathered
register:
"intf_output"
- debug:
var=intf_output
- name: Take 4
debug:
msg: "{{ intf_output | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"[?mode=='trunk']
The structure returned is like following:
{
"intf_output": {
"ansible_facts": {
"discovered_interpreter_python": "/usr/bin/python3"
},
"changed": false,
"failed": false,
"gathered": [
{
"name": "GigabitEthernet0/0"
},
{
"mode": "trunk",
"name": "GigabitEthernet0/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/2",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet0/3",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet1/0",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"mode": "trunk",
"name": "GigabitEthernet1/1",
"trunk": {
"allowed_vlans": [
"10",
"20",
"30",
"100"
],
"encapsulation": "dot1q"
}
},
{
"name": "GigabitEthernet1/2"
},
{
"name": "GigabitEthernet1/3"
},
{
"name": "GigabitEthernet2/0"
},
{
"name": "GigabitEthernet2/1"
},
{
"name": "GigabitEthernet2/2"
},
{
"name": "GigabitEthernet2/3"
},
{
"name": "GigabitEthernet3/0"
},
{
"name": "GigabitEthernet3/1"
},
{
"name": "GigabitEthernet3/2"
},
{
"name": "GigabitEthernet3/3"
}
]
}
}
For each host I run against the playbook against.
The argument to json_query must be a string. Because you haven't quoted your argument, Ansible is looking for a variable named lldp_output. But you've got additonal problems, since you're trying to access a key named lldp_output['gathered'], but [ is a syntactically significant character in JSON (and JMESPath queries), so that needs to be escaped as well.
In order to avoid all sorts of quote escaping contortions, we can place the query itself into a variable, so that we have:
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"
Note that we are using the >- block quote operator, which means that the value of query is the literal string "lldp_output['gathered']", including the outer quotes.
That playbook outputs:
TASK [Take 4] *********************************************************************************
ok: [localhost] => {
"msg": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
},
{
"access": {
"vlan": 12
},
"mode": "access",
"name": "GigabitEthernet0/1"
},
{
"name": "GigabitEthernet0/2"
}
]
}
To get just those systems with mode equal to trunk, just add that
criteria to your query:
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
tasks:
- name: Take 4
debug:
msg: "{{ tmpdata | json_query(query) }}"
vars:
query: >-
"lldp_output['gathered']"[?mode=='trunk']
This will output:
TASK [Take 4] *********************************************************************************
ok: [localhost] => {
"msg": [
{
"mode": "trunk",
"name": "GigabitEthernet0/0",
"trunk": {
"encapsulation": "dot1q"
}
}
]
}
Update
Given the data you've shown in your updated question, things are much
simpler, because you don't have the weird quoting you had in the
original question. With intf_output defined as shown, you can
write:
tasks:
- name: Take 4
debug:
msg: "{{ intf_output | json_query(query) }}"
vars:
query: >-
gathered[?mode=='trunk']

Q: Ansible - How can I merge 2 lists of hashes with a common key/value pair

How can I merge 2 lists of hashes based on a key/value pair, using Ansible 2.4.4
"foo": [
{
"hostname": "web1.example.com",
"guid": "73e85eb2-2ad5-4699-8a09-adf658a11ff2"
},
{
"hostname": "web2.example.com",
"guid": "827025fe-f13c-4fc8-ba51-7ff582596bbd"
},
{
"hostname": "web3.example.com",
"guid": "bba27304-c1bb-4889-aa44-125626be8488"
}
]
"bar": [
{
"ipaddress": "1.1.1.1",
"guid": "73e85eb2-2ad5-4699-8a09-adf658a11ff2"
},
{
"ipaddress": "2.2.2.2",
"guid": "827025fe-f13c-4fc8-ba51-7ff582596bbd"
},
{
"ipaddress": "3.3.3.3",
"guid": "bba27304-c1bb-4889-aa44-125626be8488"
}
]
I want something like:
"foobar" : [
{
"hostname": "web1.example.com",
"guid": "73e85eb2-2ad5-4699-8a09-adf658a11ff2",
"ipaddress": "1.1.1.1"
},
{
"hostname": "web2.example.com",
"guid": "827025fe-f13c-4fc8-ba51-7ff582596bbd",
"ipaddress": "2.2.2.2"
},
{
"hostname": "web3.example.com",
"guid": "bba27304-c1bb-4889-aa44-125626be8488",
"ipaddress": "3.3.3.3"
}
]
I've looked into several Ansible / Jinja2 filters including combine, union, map, custom plugins, but not having much success. I need to be able to match on the guid.
not sure if there is a smarter way, but to achieve what you need you can use the nested query plugin to loop over the combinations of the elements from the 2 list variables, find the combinations that have the common field equal, and then construct a new dictionary element and append it to the "final" list variable.
playbook:
- hosts: localhost
gather_facts: false
vars:
foo:
- hostname: web1.example.com
guid: 73e85eb2-2ad5-4699-8a09-adf658a11ff2
- hostname: web2.example.com
guid: 827025fe-f13c-4fc8-ba51-7ff582596bbd
- hostname: web3.example.com
guid: bba27304-c1bb-4889-aa44-125626be8488
bar:
- ipaddress: 1.1.1.1
guid: 73e85eb2-2ad5-4699-8a09-adf658a11ff2
- ipaddress: 2.2.2.2
guid: 827025fe-f13c-4fc8-ba51-7ff582596bbd
- ipaddress: 3.3.3.3
guid: bba27304-c1bb-4889-aa44-125626be8488
tasks:
- name: merge lists
set_fact:
merged_list: "{{ merged_list|default([]) + [{ 'hostname': item[0].hostname, 'guid': item[0].guid, 'ipaddress': item[1].ipaddress }] }}"
when: "item[0].guid == item[1].guid"
loop: "{{ query('nested', foo, bar) }}"
- name: print results
debug:
var: merged_list
result:
TASK [print results] ************************************************************************************************************************************************************************************************
ok: [localhost] => {
"merged_list": [
{
"guid": "73e85eb2-2ad5-4699-8a09-adf658a11ff2",
"hostname": "web1.example.com",
"ipaddress": "1.1.1.1"
},
{
"guid": "827025fe-f13c-4fc8-ba51-7ff582596bbd",
"hostname": "web2.example.com",
"ipaddress": "2.2.2.2"
},
{
"guid": "bba27304-c1bb-4889-aa44-125626be8488",
"hostname": "web3.example.com",
"ipaddress": "3.3.3.3"
}
]
}

Resources