I am trying to query the following Infoblox data with Ansible and JMESPath json_query:
{
"ip_records.json": {
"result": [
{
"_ref": "fixedaddress/blabla",
"ipv4addr": "10.10.10.10",
"network_view": "Bla"
},
{
"_ref": "record:host/blabla",
"ipv4addrs": [
{
"_ref": "record:host_ipv4addr/blabla",
"host": "bla.bla.com",
"ipv4addr": "10.10.10.10"
}
],
"name": "bla.bla.com",
"view": " "
},
{
"_ref": "record:a/blabla",
"ipv4addr": "10.10.10.10",
"name": "bla.bla.com",
"view": "bla"
}
]
}
}
I want to get only the _ref value for the item with fixedaddress in the _ref value.
Forgot to add that there might also be multiple records with fixedaddress but different IP's. So I also want to filter on a specific IP as the same time.
I have created queries to filter
only on IP address given as input
the string fixedaddress
a combination of both
The first two work as expected. But, I want to combine both conditions and would expect to get the single item as output, but I get nothing. I tried using && and | to combine both, as showed below.
- name: "Search IP Record: Task 2.2: Filter Results."
vars:
jmesquery: "[] | [?ipv4addr==`{{ infoblox_ip }}`]._ref"
set_fact:
ip_records_refs: "{{ ip_records.json.result | json_query(jmesquery) }}"
- name: "Search IP Record: Task 2.4: Filter Results."
vars:
jmesquery: "[] | [?_ref.contains(#,`fixedaddress`)]._ref"
set_fact:
ip_records_refs: "{{ ip_records.json.result | to_json | from_json | json_query(jmesquery) }}"
- name: "Search IP Record: Task 2.6: Filter Results."
vars:
# jmesquery: "[] | ([?ipv4addr==`{{ infoblox_ip }}` && _ref.contains(#,`fixedaddress`)])._ref"
jmesquery: "[] | [?ipv4addr==`{{ infoblox_ip }}`].ref | [?_ref.contains(#,`fixedaddress`)]._ref"
set_fact:
ip_records_refs: "{{ ip_records.json.result | to_json | from_json | json_query(jmesquery) }}"
Output:
TASK [Search IP Record: Task 2.3 Dump variable Content] ***********
ok: [localhost] => {
"ip_records_refs": [
"fixedaddress/blabla",
"record:a/blabla"
]
}
TASK [Search IP Record: Task 2.5 Dump variable Content] ***********
ok: [localhost] => {
"ip_records_refs": [
"fixedaddress/blabla"
]
}
TASK [Search IP Record: Task 2.7 Dump variable Content] ***********
ok: [localhost] => {
"ip_records_refs": []
}
You are misusing the pipe expression.
From your trial, it is hard to tell exactly what you think it does, but here is a simple explanation: you might not see it, but a JMESPath filter on an array does not return you a JSON array, rather it returns you a projection.
You cannot chain a filter on top of projection, you need to reset it first, in order to get the resulting JSON array, and this is what the pipe expression is meant for.
In your case, you do not want to have a filter on top of a projection, you want a filter with multiple conditions, so, your last set_fact query should read:
jmesquery: >-
[?
_ref.contains(#,`fixedaddress`)
&& ipv4addr == `{{ infoblox_ip }}`
]._ref
And your two first queries should be simplified to:
jmesquery: "[?_ref.contains(#,`fixedaddress`)]._ref"
and
jmesquery: "[?ipv4addr == `{{ infoblox_ip }}`]._ref"
Q: "Get _ref value for the item with 'fixedaddress' in the _ref key."
A: The query below
jmesquery: "[?_ref.contains(#,`fixedaddress`)]._ref"
ip_records_refs: "{{ ip_records.json.result|json_query(jmesquery) }}"
gives the expected result
ip_records_refs:
- fixedaddress/blabla
Example of a complete playbook for testing
- hosts: localhost
vars:
ip_records:
json:
result:
- _ref: fixedaddress/blabla
ipv4addr: 10.10.10.10
network_view: Bla
- _ref: record:host/blabla
ipv4addrs:
- _ref: record:host_ipv4addr/blabla
host: bla.bla.com
ipv4addr: 10.10.10.10
name: bla.bla.com
view: ' '
- _ref: record:a/blabla
ipv4addr: 10.10.10.10
name: bla.bla.com
view: bla
# Get _ref value for the item with 'fixedaddress' in the _ref key
jmesquery: "[?_ref.contains(#,`fixedaddress`)]._ref"
ip_records_refs: "{{ ip_records.json.result|json_query(jmesquery) }}"
tasks:
- debug:
var: ip_records
- debug:
var: ip_records_refs
Related
I am trying to match certain key from an ansible output/vars, and return the value of another key.
Here is my vars.
{
"_meta": {
"hostvars": {
"ansibletower1_422c3aed-780c-8c33-3054-d32e330c9285": {
"guest.hostName": "ansibletower1.rh.pro",
"name": "ansibletower1"
},
"child domain_422c4cd1-d644-7eeb-df7c-c32a2a05c030": {
"guest.hostName": null,
"name": "child domain"
}
}
}
}
My non-working playbook
- hosts: ansibletower1.rh.pro
tasks:
- debug:
msg: "{{ _meta | json_query(querystr) }}"
vars:
querystr: "[?hostvars.*.\"guest.hostName\"=='{{inventory_hostname}}'].name"
I am trying to follow the method here, https://blog.networktocode.com/post/ansible-filtering-json-query/. However in my scenario is not array, which is different than the example in the link.
My end goal is to match guest.hostName with {{ inventory_hostname }}, and return the value of the name by using - set_fact: to register it to another variable.
I'd try with:
tasks:
- name: Loop over data and continue if string was found.
debug:
msg: "{{ _meta | json_query(querystr) }}"
vars:
querystr: "hostvars.* | [?\"guest.hostName\"==`{{inventory_hostname}}`].name"
when you select all keys with .* you get an array back, so it is piped into another query that filters and returns name.
I want to combine two attribute into single string separated by delimiter using the json_query in ansible
Sample data
{
"locations": [
{"name": "Seattle", "state": "WA"},
{"name": "New York", "state": "NY"},
{"name": "Bellevue", "state": "WA"},
{"name": "Olympia", "state": "WA"}
]
}
As shown in above data set i'm trying to filter the state "WA" and execpted output is:
[
"Seattle-WA",
"Bellevue-WA",
"Olympia-WA"
]
What i have tried as of now:
- debug:
msg: "{{ chart_list.HELM_CHARTS | json_query(\"[?state == 'WA'].{name:name,state:state}\") }}"
Output:
[
{
"name": "Seattle",
"state": "WA"
},
{
"name": "Bellevue",
"state": "WA"
},
{
"name": "Olympia",
"state": "WA"
}
]
Updated :
I was able to get the expected result by trial and error method and these are my findings:
[?state == 'WA'].[join('-',[name,state])][]
Output:
[
"Seattle-WA",
"Bellevue-WA",
"Olympia-WA"
]
Also if the input which you give is in unicode format, i suggest you to add to_json | from_json expressions as mentioned below:
selected_cities: "{{ test.locations| to_json | from_json | json_query(\"[?state == 'WA'].[join('-',[name,state])][]\") }}"
Using above expression will eliminate unicode error whil using the values or in any condition.
Check JMESPath site for more detail on the json_query, it was really helpful in resolving the issue.
For example
- debug:
msg: "{{ locations|
json_query('[?state == `WA`].[name,state]')|
map('join', '-')|list }}"
gives
msg:
- Seattle-WA
- Bellevue-WA
- Olympia-WA
The same result gives the task below using Jinja2 filters only
- debug:
msg: "{{ _names|zip(_states)|map('join', '-')|list }}"
vars:
_locations: "{{ locations|selectattr('state', 'eq', 'WA')|list }}"
_names: "{{ _locations|map(attribute='name')|list }}"
_states: "{{ _locations|map(attribute='state')|list }}"
json_query issue (fixed in 2.10 and later)
There is JMESPath join. Unfortunately
- debug:
msg: "{{ locations|
json_query('[].join(`-`, [name,state])') }}"
fails
msg: |-
JMESPathError in json_query filter plugin:
In function join(), invalid type for value: Seattle, expected one of: ['array-string'], received: "AnsibleUnicode"
to_json|from_json workaround
Quoting from json_query: Add examples for starts_with and contains #72821
data structure returned from register variables needs to be parsed using to_json | from_json in order to get a correct result. Fixes: ansible-collections/community.general#320
- debug:
msg: "{{ locations|to_json|from_json|
json_query('[].join(`-`, [name,state])') }}"
gives
msg:
- Seattle-WA
- New York-NY
- Bellevue-WA
- Olympia-WA
Just for the sake of a pure JMESPath way of doing it, as your trial and error solution still have an unneeded extra layer of complexity.
When you are doing
[?state == 'WA'].[join('-', [name, state])][]
You are creating an array [join('-', [name, state])] then flattening it [] for no reason.
You can just go to the solution with a shorter approach:
[?state == `WA`].join(`-`, [name, state])
Also mind that you can overcome the quotes in quotes (simple or double) complication for JMESPath queries using:
YAML multilines string: How do I break a string in YAML over multiple lines?
Backticks in your JMESPath query, as pointed in the documentation:
In the example above, quoting literals using backticks avoids escaping quotes and maintains readability.
Source: https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#selecting-json-data-json-queries
So you end up with (see note below if you are on an Ansible version < 2.10):
- debug:
msg: >-
{{ test.locations
| json_query('[?state == `WA`].join(`-`, [name, state])') }}
Please note: as raised by #Vladimir Botka on the versions prior to 2.10, you will be affected by this issue: https://github.com/ansible/ansible/issues/27299#issuecomment-331068246, forcing you to add a | to_json | from_json filter on the list.
Given the playbook:
- hosts: all
gather_facts: yes
tasks:
- debug:
msg: >-
{{ test.locations
| json_query('[?state == `WA`].join(`-`, [name, state])')
}}
vars:
test:
locations:
- name: Seattle
state: WA
- name: New York
state: NY
- name: Bellevue
state: WA
- name: Olympia
state: WA
This yields:
[
"Seattle-WA",
"Bellevue-WA",
"Olympia-WA"
]
Here is my json output:
{
"kind": [
{
"inventory": "",
"inventory_sources": "",
"job_templates": "",
"workflow_job_templates": "104"
},
{
"inventory": "",
"inventory_sources": "",
"job_templates": "114",
"workflow_job_templates": ""
},
{
"inventory": "24",
"inventory_sources": "",
"job_templates": "",
"workflow_job_templates": ""
},
{
"inventory": "",
"inventory_sources": "108",
"job_templates": "",
"workflow_job_templates": ""
}
]
}
I'd like to display all items name that contain a specific value. For example, for a search value of 104 I want to get the key name workflow_job_templates
I tested some syntaxes without any success:
- debug:
msg: "104 is {{kind|json_query(query)}}"
vars:
query: "[?*==`104`].workflow_job_templates"
I know it is wrong but can someone tell me how he'd do for himself?
json_query could be part of the equation for your solution but is really not needed here.
Explanation of the below piece of code:
Apply the dict2items filter to each element of your list. This transforms each mapping to a list of {key: "key", value: "value"} pairs
Flatten the given list so we get all those elements to a single top level
Select elements having a value of '104' only
Extract the key attribute of each element in a list
Make that list unique and sort it.
- name: Display all element having a value of 104
debug:
msg: "{{ kind | map('dict2items') | flatten
| selectattr('value', '==', '104')
| map(attribute='key') | unique | sort }}"
Please note that this solution will give you a result if the same key name has different values but one of them is `104. With your above data the result is:
TASK [Display all element having a value of 104] ***************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"workflow_job_templates"
]
}
(Update)
The task below
- debug:
msg: "{{ item }} {{ kind|
map('dict2items')|
map('json_query', query)|
flatten }}"
loop: [104, 114, 108, 24]
vars:
query: "[?to_string(value) == to_string('{{ item }}')].key"
gives
msg: 104 ['workflow_job_templates']
msg: 114 ['job_templates']
msg: 108 ['inventory_sources']
msg: 24 ['inventory']
(For the record. Brute-force approach)
Create a unique list of the keys
- set_fact:
my_keys: "{{ my_keys|default([]) + item.keys()|list }}"
loop: "{{ kind }}"
- set_fact:
my_keys: "{{ my_keys|unique }}"
gives
my_keys:
- inventory
- inventory_sources
- job_templates
- workflow_job_templates
Create a dictionary with all values
- set_fact:
my_dict: "{{ my_dict|default({})|combine({item: values}) }}"
loop: "{{ my_keys }}"
vars:
query: "[].{{ item }}"
values: "{{ kind|json_query(query) }}"
gives
my_dict:
inventory:
- ''
- ''
- '24'
- ''
inventory_sources:
- ''
- ''
- ''
- '108'
job_templates:
- ''
- '114'
- ''
- ''
workflow_job_templates:
- '104'
- ''
- ''
- ''
Then search the dictionary. For example
- debug:
msg: "{{ item }} {{ my_dict|dict2items|json_query(query) }}"
loop: [104, 114, 108, 24]
vars:
query: "[?value[?contains(#, '{{ item }}')]].key"
gives
msg: 104 ['workflow_job_templates']
msg: 114 ['job_templates']
msg: 108 ['inventory_sources']
msg: 24 ['inventory']
The correct alternative of selectattr with json_query is:
- debug:
msg: "{{ kind | map('dict2items') | flatten | json_query(query)}}"
vars:
- query: "[?value == `\"104\"`].key"
If you really just want to use json_query()
---
- name: PLAYBOOK Filtering
hosts: localhost # run locally
tasks:
- name: Create json
set_fact:
kind: '{{ lookup("file", "kind.json") }}'
- name: check the var was created properly
debug:
var: kind
- name: output the element that matches 104
debug:
msg: "{{ kind | json_query(\"kind[?workflow_job_templates=='104'].workflow_job_templates\") }}"
- name:
set_fact:
output: "{{ kind | json_query(\"kind[?workflow_job_templates=='104'].workflow_job_templates\") }}"
Output
TASK [output the element that matches 104] *************************************************************************************************************
ok: [localhost] => {
"msg": [
"104"
]
}
I have 2 lists as set_fact and want to create a dict
I am running ansible 2.8
I have list1 as below
"inventory_devices": [
"device0",
"device1"
]
and list2 as below
"inventory_ips": [
"10.1.1.1",
"10.1.1.2"
]
I want to get an output shows like
"inventory_dict": [
"device0": "10.1.1.1",
"device1": "10.1.1.2"
]
Thanks.
You can do it entirely with jinja2 using the zip filter built into ansible.
To get a list combining the elements of other lists use zip
- name: give me list combo of two lists
debug:
msg: "{{ [1,2,3,4,5] | zip(['a','b','c','d','e','f']) | list }}"
...
Similarly to the output of the items2dict filter mentioned above, these filters can be
used to contruct a dict:
{{ dict(keys_list | zip(values_list)) }}
The zip filter sequentially combines items from pairs of lists and the dict construct creates a dictionary from a list of pairs.
inventory_dict: "{{ dict(inventory_devices | zip(inventory_ips)) }}"
here is the task to do it, populate combined var in the PB below:
---
- hosts: localhost
gather_facts: false
vars:
inventory_devices:
- device0
- device1
inventory_ips:
- 10.1.1.1
- 10.1.1.2
tasks:
- name: populate combined var
set_fact:
combined_var: "{{ combined_var|default({}) | combine({ item.0: item.1 }) }}"
loop: "{{ query('together', inventory_devices, inventory_ips) }}"
- name: print combined var
debug:
var: combined_var
result:
TASK [print combined var] **********************************************************************************************************************************************************************************************
ok: [localhost] => {
"combined_var": {
"device0": "10.1.1.1",
"device1": "10.1.1.2"
}
}
hope it helps
When should I use the from_json filter in Ansible?
I found out that using it sometimes has and sometimes have no effect.
Please consider the following example which illustrates the inconsistency I am getting.
Included in reverse order are: the questions - expected result - actual result - the playbook - the data. The data is taken from this question and the playbook is based on this answer.
The question(s):
Why storing the left part (before json_query) of the following expression in a variable and then using json_query on the variable causes the expression to be evaluated differently?
"{{ lookup('file','test.json') | json_query(query) }}"
Why does adding from_json filter alter the results (but does not if processing a variable):
"{{ lookup('file','test.json') | from_json | json_query(query) }}"
Expected result:
Last four tasks should give the same result. Alternatively, last two tasks should give the same result as previous two tasks.
Actual result (last four tasks only):
One task result differs.
TASK [This query is run against lookup value with from_json stored in a variable] ***
ok: [localhost] => {
"msg": [
678
]
}
TASK [This query is run against lookup value without from_json stored in a variable] ***
ok: [localhost] => {
"msg": [
678
]
}
TASK [This query is run directly against lookup value with from_json] **********
ok: [localhost] => {
"msg": [
678
]
}
TASK [This query is run directly against lookup value without from_json - the result is empty - why?] ***
ok: [localhost] => {
"msg": ""
}
The playbook:
---
- hosts: localhost
gather_facts: no
connection: local
tasks:
- set_fact:
from_lookup_with_from_json: "{{ lookup('file','test.json') | from_json }}"
- set_fact:
from_lookup_without_from_json: "{{ lookup('file','test.json') }}"
- name: Save the lookup value stored in a variable in a file for comparison
copy: content="{{ from_lookup_with_from_json }}" dest=./from_lookup_with_from_json.txt
- name: Save the lookup value stored in a variable in a file for comparison (they are the same)
copy: content="{{ from_lookup_without_from_json }}" dest=./from_lookup_without_from_json.txt
- name: This query is run against lookup value with from_json stored in a variable
debug: msg="{{ from_lookup_with_from_json | json_query(query) }}"
vars:
query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
- name: This query is run against lookup value without from_json stored in a variable
debug: msg="{{ from_lookup_without_from_json | json_query(query) }}"
vars:
query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
- name: This query is run directly against lookup value with from_json
debug: msg="{{ lookup('file','test.json') | from_json | json_query(query) }}"
vars:
query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
- name: This query is run directly against lookup value without from_json - the result is empty - why?
debug: msg="{{ lookup('file','test.json') | json_query(query) }}"
vars:
query: "Foods[].{id: Id, for: (Tags[?Key=='For'].Value)[0]} | [?for=='Tigger'].id"
The data (test.json):
{ "Foods" :
[ { "Id": 456
, "Tags":
[ {"Key":"For", "Value":"Heffalump"}
, {"Key":"Purpose", "Value":"Food"}
]
}
, { "Id": 678
, "Tags":
[ {"Key":"For", "Value":"Tigger"}
, {"Key":"Purpose", "Value":"Food"}
]
}
, { "Id": 911
, "Tags":
[ {"Key":"For", "Value":"Roo"}
, {"Key":"Purpose", "Value":"Food"}
]
}
]
}
json_query requires Python object (dict) as input, if you feed it with string, it gives empty string as result.
You get different result because of Ansible templating engine tricky work.
I should definitely write a post about it on my site...
After evaluating jijna2 expression Ansible try to cast complex types to Python objects (like dict or list). See my other answer.
In your case:
1.
- set_fact:
from_lookup_with_from_json: "{{ lookup('file','test.json') | from_json }}"
from_lookup_with_from_json is a dict, because you manually convert JSON-string from file to dict with from_json filter.
2.
- set_fact:
from_lookup_without_from_json: "{{ lookup('file','test.json') }}"
from_lookup_with_from_json becomes dict, because Ansible converts it when jinja2 expression ends with }}. So from_json is actually unnecessary as the last filter in chain.
3.
debug: msg="{{ lookup('file','test.json') | from_json | json_query(query) }}"
Again, you manually convert JSON-string here. So json_query get dict as input.
4.
debug: msg="{{ lookup('file','test.json') | json_query(query) }}"
In this case you feed JSON-string (not dict) as input to json_query filter. As everything happens inside one jinja2 expression, Ansible doesn't attempt to convert anything in between.
You can also get empty string result with a variable this way:
- set_fact:
from_lookup_force_string: "{{ lookup('file','test.json') | string }}"
In this case from_lookup_force_string will not be converted by Ansible tempating engine, and json_query will give you empty response on it.