There is a simple JSON file, sample.json with the following content:
{
"test": {
"domain": [
{
"name": "cluster1"
}
]
}
}
With Ansible, I want to query over the test key, which works with the following Ansible playbook.
---
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
- debug:
msg: "{{ tmpdata | json_query('test') }}"
The play
ok: [localhost] => {
"msg": {
"domain": [
{
"name": "cluster1"
}
]
}
}
However, when they key in the JSON file is changed, from test to test/something, and the ansible json_query from test to test/something as well, Ansible/JMESPath produces an error.
fatal: [localhost]: FAILED! => {"msg": "JMESPathError in json_query filter plugin:\nBad jmespath expression: Unknown token /:\ntest/something\n ^"}
I've looked into the JMESpath documentation, but it does not make sense to me.
How can I ensure JMESpath works with forward slashes in the Ansible query.
JMESPath defines identifier as unquoted-string / quoted-string.
unquoted-string is A-Za-z_. Anything else should be quoted.
In your case:
- debug:
msg: "{{ tmpdata | json_query('\"test/something\"') }}"
Here we escape \" because we are inside YAML double quotes msg: "...".
Related
I am trying to query the following Infoblox data with Ansible and JMESPath json_query:
{
"ip_records.json": {
"result": [
{
"_ref": "fixedaddress/blabla",
"ipv4addr": "10.10.10.10",
"network_view": "Bla"
},
{
"_ref": "record:host/blabla",
"ipv4addrs": [
{
"_ref": "record:host_ipv4addr/blabla",
"host": "bla.bla.com",
"ipv4addr": "10.10.10.10"
}
],
"name": "bla.bla.com",
"view": " "
},
{
"_ref": "record:a/blabla",
"ipv4addr": "10.10.10.10",
"name": "bla.bla.com",
"view": "bla"
}
]
}
}
I want to get only the _ref value for the item with fixedaddress in the _ref value.
Forgot to add that there might also be multiple records with fixedaddress but different IP's. So I also want to filter on a specific IP as the same time.
I have created queries to filter
only on IP address given as input
the string fixedaddress
a combination of both
The first two work as expected. But, I want to combine both conditions and would expect to get the single item as output, but I get nothing. I tried using && and | to combine both, as showed below.
- name: "Search IP Record: Task 2.2: Filter Results."
vars:
jmesquery: "[] | [?ipv4addr==`{{ infoblox_ip }}`]._ref"
set_fact:
ip_records_refs: "{{ ip_records.json.result | json_query(jmesquery) }}"
- name: "Search IP Record: Task 2.4: Filter Results."
vars:
jmesquery: "[] | [?_ref.contains(#,`fixedaddress`)]._ref"
set_fact:
ip_records_refs: "{{ ip_records.json.result | to_json | from_json | json_query(jmesquery) }}"
- name: "Search IP Record: Task 2.6: Filter Results."
vars:
# jmesquery: "[] | ([?ipv4addr==`{{ infoblox_ip }}` && _ref.contains(#,`fixedaddress`)])._ref"
jmesquery: "[] | [?ipv4addr==`{{ infoblox_ip }}`].ref | [?_ref.contains(#,`fixedaddress`)]._ref"
set_fact:
ip_records_refs: "{{ ip_records.json.result | to_json | from_json | json_query(jmesquery) }}"
Output:
TASK [Search IP Record: Task 2.3 Dump variable Content] ***********
ok: [localhost] => {
"ip_records_refs": [
"fixedaddress/blabla",
"record:a/blabla"
]
}
TASK [Search IP Record: Task 2.5 Dump variable Content] ***********
ok: [localhost] => {
"ip_records_refs": [
"fixedaddress/blabla"
]
}
TASK [Search IP Record: Task 2.7 Dump variable Content] ***********
ok: [localhost] => {
"ip_records_refs": []
}
You are misusing the pipe expression.
From your trial, it is hard to tell exactly what you think it does, but here is a simple explanation: you might not see it, but a JMESPath filter on an array does not return you a JSON array, rather it returns you a projection.
You cannot chain a filter on top of projection, you need to reset it first, in order to get the resulting JSON array, and this is what the pipe expression is meant for.
In your case, you do not want to have a filter on top of a projection, you want a filter with multiple conditions, so, your last set_fact query should read:
jmesquery: >-
[?
_ref.contains(#,`fixedaddress`)
&& ipv4addr == `{{ infoblox_ip }}`
]._ref
And your two first queries should be simplified to:
jmesquery: "[?_ref.contains(#,`fixedaddress`)]._ref"
and
jmesquery: "[?ipv4addr == `{{ infoblox_ip }}`]._ref"
Q: "Get _ref value for the item with 'fixedaddress' in the _ref key."
A: The query below
jmesquery: "[?_ref.contains(#,`fixedaddress`)]._ref"
ip_records_refs: "{{ ip_records.json.result|json_query(jmesquery) }}"
gives the expected result
ip_records_refs:
- fixedaddress/blabla
Example of a complete playbook for testing
- hosts: localhost
vars:
ip_records:
json:
result:
- _ref: fixedaddress/blabla
ipv4addr: 10.10.10.10
network_view: Bla
- _ref: record:host/blabla
ipv4addrs:
- _ref: record:host_ipv4addr/blabla
host: bla.bla.com
ipv4addr: 10.10.10.10
name: bla.bla.com
view: ' '
- _ref: record:a/blabla
ipv4addr: 10.10.10.10
name: bla.bla.com
view: bla
# Get _ref value for the item with 'fixedaddress' in the _ref key
jmesquery: "[?_ref.contains(#,`fixedaddress`)]._ref"
ip_records_refs: "{{ ip_records.json.result|json_query(jmesquery) }}"
tasks:
- debug:
var: ip_records
- debug:
var: ip_records_refs
So, I have this list:
ip_range:
- "1.x.x.x/24"
- "2.x.x.x/24"
And I'm trying to pass it on to an AWS cli command as a valid JSON:
- name: list of IPs
set_fact:
allowed_cidrs: "{{ ip_range | ipaddr('network/prefix') }}"
- debug:
msg: "{{ allowed_qualys_cidrs | to_json }}"
- name: send command
command: >
aws wafv2 update-ip-set
--addresses "{{ allowed_cidrs | to_json }}"
That debug prints:
["1.x.x.x/24", "2.x.x.x/24"]
But what the command tries to send is:
"cmd": [
"aws",
"wafv2",
"update-ip-set",
"--addresses",
"[1.x.x.x/24, 2.x.x.x/24]",
]
Which, because the double quotes are gone, is not a valid JSON.
I already tried every possible approach with Ansible and Jinja2, but I can't figure out a way to send that command with a valid JSON.
Use single quotes in your command rather than double quotes:
- name: list of IPs
set_fact:
allowed_cidrs: "{{ ip_range | ipaddr('network/prefix') }}"
vars:
ip_range:
- "1.1.1.1/24"
- "2.2.2.2/24"
- name: send command
command: >-
aws wafv2 update-ip-set
--addresses '{{ allowed_cidrs | to_json }}'
register: _cmd
- debug:
var: _cmd.cmd
Yields:
TASK [list of IPs] **********************************************************
ok: [localhost]
TASK [send command] *********************************************************
changed: [localhost]
TASK [debug] ****************************************************************
ok: [localhost] => {
"_cmd.cmd": [
"aws",
"wafv2",
"update-ip-set",
"--addresses",
"[\"1.1.1.0/24\", \"2.2.2.0/24\"]"
]
}
I have the following jinja2 template
[
{% for items in hosts %}
{
"name":"{{ items.name }}",
"display_name":"{{ items.display_name }}",
"services": {{ host_group | from_json | json_query('[*].services[0]') | to_json }},
}
{% endfor %}
]
i need to replace services[0] by a variable {{ loop.index0 }}, i tried this syntax
"services": {{ host_group | from_json | json_query('[*].services[loop.index0]') | to_json }}
but i'm getting an error :
AnsibleFilterError: JMESPathError in json_query filter plugin:
Expecting: star, got: unquoted_identifier: Parse error at column 13, token "loop" (UNQUOTED_IDENTIFIER), for expression:
"[*].services[loop.index0]"
I tried another syntax:
"services": {{ host_group | from_json | json_query('[*].services[' + {{ loop.index0 }} +']') | to_json }},
and it gives also an error :
AnsibleError: template error while templating string: expected token ':', got '}'. String: [
There are two things to keep in mind when working with Jinja:
You never nest the {{...}} template markers.
If you put something in quotes, it's a literal string.
So when you write:
json_query('[*].services[loop.index0]')
You are passing json_query the literal string [*].services[loop.index0], which isn't a valid JMESPath query. If you want to substitute the value of a variable in a string, you need to either build the string via concatenation, or use string formatting logic.
Concatenation
Using concatenation might look like:
json_query('[*].services[' ~ loop.index0 ` ']')
Here, ~ is the string concatenation operator -- it's like +, but it makes sure to convert everything into a string. Compare this:
ansible localhost -m debug -a 'msg={{ "there are " + 4 + " lights" }}'
To this:
ansible localhost -m debug -a 'msg={{ "there are " ~ 4 ~ " lights" }}'
String formatting
Using string formatting might look like:
json_query('[*].services[%s]' % (loop.index0))
Or:
json_query('[*].services[{}]'.format(loop.index0))
These are two forms of string formatting available in Python; for more details, start here.
Using Ansible 2.9.23.
For the string placeholders {} and %s I had to use backticks, otherwise it wouldn't work:
json_query('results[?name==`{}`].version'.format(query))
json_query('results[?name==`%s`].version' % (query))
Given the example json:
{
(...)
"results": [
{
"envra": "0:ntp-4.2.6p5-29.el7_8.2.x86_64",
"name": "ntp",
"repo": "installed",
"epoch": "0",
"version": "4.2.6p5",
"release": "29.el7_8.2",
"yumstate": "installed",
"arch": "x86_64"
},
(...)
],
(...)
}
created by:
- name: list installed packages
yum:
list: installed
register: installed_list
with a defined variable query: ntp, pass this variable to json_query in two ways:
- name: pass var to json_query - string formatting (1)
debug:
msg: "{{ installed_list|json_query('results[?name==`{}`].version'.format(query))|first }}"
- name: pass var to json_query - string formatting (2)
debug:
msg: "{{ installed_list|json_query('results[?name==`%s`].version' % (query))|first }}"
Results:
TASK [pass var to json_query - string formatting (1)] *****************************************************************************************************
ok: [host] => {
"msg": "4.2.6p5"
}
TASK [pass var to json_query - string formatting (2)] *****************************************************************************************************
ok: [host] => {
"msg": "4.2.6p5"
}
I have the following json structure.
"results": [
{
"ltm_pools": [
{
"members": [
{
"full_path": "/Common/10.49.128.185:8080",
},
{
"full_path": "/Common/10.49.128.186:8080",
}
"name": "Staging-1-stresslab",
},
{
"members": [
{
"full_path": "/Common/10.49.128.187:0",
},
{
"full_path": "/Common/10.49.128.188:0",
}
],
"name": "Staging-2-lab",
},
I get an error when trying to do something like this
- debug:
msg: "{{item[0].host}} --- {{ item[1] }} --- {{ item[2] }}"
with_nested:
- "{{F5_hosts}}"
- "{{bigip_facts | json_query('[results[0].ltm_pools[*].name]') | flatten }}"
- "{{bigip_facts | json_query('[results[0].ltm_pools[?name.contains(#,'Staging'].members[::2].full_path]') | flatten }}"
I am unable to get the third array working.
I want to print the even members full_path variable from all objects where name contains staging.
I hope someone can help me I've been struggling with this for days.
From what I see/read/tried myself, you fell in this bug: https://github.com/ansible/ansible/issues/27299
This is the problem of "contains" JMESPath function as it is run by Ansible, to quote:
https://github.com/ansible/ansible/issues/27299#issuecomment-331068246
The problem is related to the fact that Ansible uses own types for strings: AnsibleUnicode and AnsibleUnsafeText.
And as long as jmespath library has very strict type-checking, it fails to accept this types as string literals.
There is also a suggested workaround, if you convert the variable to json and back, the strings in there have the correct type. Making long story short, this doesn't work:
"{{bigip_facts | json_query('results[0].ltm_pools[?name.contains(#,`Staging`)==`true`].members[::2].full_path') }}"
but this does:
"{{bigip_facts | to_json | from_json | json_query('results[0].ltm_pools[?name.contains(#,`Staging`)==`true`].members[::2].full_path') }}"
I've managed to run such a code:
- hosts: all
gather_facts: no
tasks:
- set_fact:
bigip_facts:
results:
- ltm_pools:
- members:
- full_path: "/Common/10.49.128.185:8080"
- full_path: "/Common/10.49.128.186:8080"
name: "Staging-1-stresslab"
- members:
- full_path: "/Common/10.49.128.187:0"
- full_path: "/Common/10.49.128.188:0"
name: "Staging-2-stresslab"
- name: "Debug ltm-pools"
debug:
msg: "{{ item }}"
with_items:
- "{{bigip_facts | to_json | from_json | json_query('results[0].ltm_pools[?name.contains(#,`Staging`)==`true`].members[::2].full_path') }}"
And it works as you wanted:
PLAY [all] *****************************************************************************************
TASK [set_fact] ************************************************************************************
ok: [localhost]
TASK [Debug ltm-pools] *****************************************************************************
ok: [localhost] => (item=[u'/Common/10.49.128.185:8080']) => {
"msg": [
"/Common/10.49.128.185:8080"
]
}
ok: [localhost] => (item=[u'/Common/10.49.128.187:0']) => {
"msg": [
"/Common/10.49.128.187:0"
]
}
PLAY RECAP *****************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0
I have an input as such
ok: [localhost] => {
"static_plugin_versions": [
{
"name": "ace-editor",
"version": "1.1"
},
{
"name": "analysis-core",
"version": "1.95"
},
{
"name": "ant",
"version": "1.9"
},
{
"name": "antisamy-markup-formatter",
"version": "1.5"
},
{
"name": "apache-httpcomponents-client-4-api",
"version": "4.5.5-3.0"
}
]
}
My aim to to print out the version by specifying a particular name. In this case specifically looking for the version of analysis-core
What I have tried are the following
- debug:
var: static_plugin_versions['analysis-core']['version']
- debug:
var: static_plugin_versions['analysis-core'].version
- debug:
var: static_plugin_versions[analysis-core.version]
The only thing that works is
- debug:
var: static_plugin_versions[1].version
But this is not feasible since if more entries get added to the dictionary, it will break.
Any indication what Im doing wrong here. Am looking for a way that does not rely on looping.
EDIT
Just tried this
- set_fact:
analysis_core_version: "{{ item.version }}"
when: "'analysis-core' in item.name"
with_items: "{{ static_plugin_versions }}"
- debug:
var: analysis-core-version
But I get:
TASK [copy : set_fact] *******************************************************************************************************************************************************************************************************************************************************************************************************
skipping: [localhost] => (item={u'version': u'1.1', u'name': u'ace-editor'})
ok: [localhost] => (item={u'version': u'1.95', u'name': u'analysis-core'})
skipping: [localhost] => (item={u'version': u'1.9', u'name': u'ant'})
skipping: [localhost] => (item={u'version': u'1.5', u'name': u'antisamy-markup-formatter'})
skipping: [localhost] => (item={u'version': u'4.5.5-3.0', u'name': u'apache-httpcomponents-client-4-api'})
TASK [copy : debug] **********************************************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"analysis-core-version": "VARIABLE IS NOT DEFINED!"
}
The easiest way to do this is with the selectattr filter, which lets you apply a filter to a list of objects. For example, if I have this playbook:
---
- hosts: localhost
gather_facts: false
vars:
"static_plugin_versions": [
{
"name": "ace-editor",
"version": "1.1"
},
{
"name": "analysis-core",
"version": "1.95"
},
{
"name": "ant",
"version": "1.9"
},
{
"name": "antisamy-markup-formatter",
"version": "1.5"
},
{
"name": "apache-httpcomponents-client-4-api",
"version": "4.5.5-3.0"
}
]
tasks:
- debug:
msg: "version of {{ item }} is {{ (static_plugin_versions|selectattr('name', 'eq', item)|first).version }}"
loop:
- ace-editor
The output will be:
TASK [debug] **********************************************************************************************************************************************************************************
ok: [localhost] => (item=ace-editor) => {
"msg": "version of ace-editor is 1.1"
}
Or, using your example:
- set_fact:
analysis_core_version: "{{ (static_plugin_versions|selectattr('name', 'eq', 'analysis-core')|first).version }}"
- debug:
var: analysis-core-version
Which produces:
ok: [localhost] => {
"analysis_core_version": "1.95"
}
If necessary, the json_query filter allows for substantially more complex queries.
Solution 1 (not optimized)
As suggested by Illias comment, you can use a loop to go other each element in the list, match its name value and print its version when condition is met.
- name: print version of analysis-core
debug:
msg: "{{ item.version }}"
when: item.name == 'analysis-core'
loop: "{{ static_plugin_versions }}"
Meanwhile this will go other each element and skip task whenever there is no match. If you have hundreds of plugins, this will soon become unreadable in your ansible execution log.
Solution 2 (my preferred)
Query your data stucture to get exactly what you need. Your friend here is the json_query filter (and you should read jmespath doc if you want to go further). For your particular example
- name: print version of analysis-core
debug:
msg: >-
{{ (static_plugin_versions | json_query("[?name == 'analysis-core'].version")).0 }}
Notes:
I inferred that your plugin names where unique in your list. Therefore querying for a particular plugin name returns a single element and I can print the first one in list as the expected result.
The order of single and double quotes matters in the above example (see jmespath specification: you'll get an empty string result if you switch them around.
I used a yaml folded block in my example so I didn't have to escape the double quotes (because I'm a very lazy B#sT#rD :D), but you can use a normal string and escape double quotes if you wish.