I have the following jinja2 template
[
{% for items in hosts %}
{
"name":"{{ items.name }}",
"display_name":"{{ items.display_name }}",
"services": {{ host_group | from_json | json_query('[*].services[0]') | to_json }},
}
{% endfor %}
]
i need to replace services[0] by a variable {{ loop.index0 }}, i tried this syntax
"services": {{ host_group | from_json | json_query('[*].services[loop.index0]') | to_json }}
but i'm getting an error :
AnsibleFilterError: JMESPathError in json_query filter plugin:
Expecting: star, got: unquoted_identifier: Parse error at column 13, token "loop" (UNQUOTED_IDENTIFIER), for expression:
"[*].services[loop.index0]"
I tried another syntax:
"services": {{ host_group | from_json | json_query('[*].services[' + {{ loop.index0 }} +']') | to_json }},
and it gives also an error :
AnsibleError: template error while templating string: expected token ':', got '}'. String: [
There are two things to keep in mind when working with Jinja:
You never nest the {{...}} template markers.
If you put something in quotes, it's a literal string.
So when you write:
json_query('[*].services[loop.index0]')
You are passing json_query the literal string [*].services[loop.index0], which isn't a valid JMESPath query. If you want to substitute the value of a variable in a string, you need to either build the string via concatenation, or use string formatting logic.
Concatenation
Using concatenation might look like:
json_query('[*].services[' ~ loop.index0 ` ']')
Here, ~ is the string concatenation operator -- it's like +, but it makes sure to convert everything into a string. Compare this:
ansible localhost -m debug -a 'msg={{ "there are " + 4 + " lights" }}'
To this:
ansible localhost -m debug -a 'msg={{ "there are " ~ 4 ~ " lights" }}'
String formatting
Using string formatting might look like:
json_query('[*].services[%s]' % (loop.index0))
Or:
json_query('[*].services[{}]'.format(loop.index0))
These are two forms of string formatting available in Python; for more details, start here.
Using Ansible 2.9.23.
For the string placeholders {} and %s I had to use backticks, otherwise it wouldn't work:
json_query('results[?name==`{}`].version'.format(query))
json_query('results[?name==`%s`].version' % (query))
Given the example json:
{
(...)
"results": [
{
"envra": "0:ntp-4.2.6p5-29.el7_8.2.x86_64",
"name": "ntp",
"repo": "installed",
"epoch": "0",
"version": "4.2.6p5",
"release": "29.el7_8.2",
"yumstate": "installed",
"arch": "x86_64"
},
(...)
],
(...)
}
created by:
- name: list installed packages
yum:
list: installed
register: installed_list
with a defined variable query: ntp, pass this variable to json_query in two ways:
- name: pass var to json_query - string formatting (1)
debug:
msg: "{{ installed_list|json_query('results[?name==`{}`].version'.format(query))|first }}"
- name: pass var to json_query - string formatting (2)
debug:
msg: "{{ installed_list|json_query('results[?name==`%s`].version' % (query))|first }}"
Results:
TASK [pass var to json_query - string formatting (1)] *****************************************************************************************************
ok: [host] => {
"msg": "4.2.6p5"
}
TASK [pass var to json_query - string formatting (2)] *****************************************************************************************************
ok: [host] => {
"msg": "4.2.6p5"
}
Related
I want to combine two attribute into single string separated by delimiter using the json_query in ansible
Sample data
{
"locations": [
{"name": "Seattle", "state": "WA"},
{"name": "New York", "state": "NY"},
{"name": "Bellevue", "state": "WA"},
{"name": "Olympia", "state": "WA"}
]
}
As shown in above data set i'm trying to filter the state "WA" and execpted output is:
[
"Seattle-WA",
"Bellevue-WA",
"Olympia-WA"
]
What i have tried as of now:
- debug:
msg: "{{ chart_list.HELM_CHARTS | json_query(\"[?state == 'WA'].{name:name,state:state}\") }}"
Output:
[
{
"name": "Seattle",
"state": "WA"
},
{
"name": "Bellevue",
"state": "WA"
},
{
"name": "Olympia",
"state": "WA"
}
]
Updated :
I was able to get the expected result by trial and error method and these are my findings:
[?state == 'WA'].[join('-',[name,state])][]
Output:
[
"Seattle-WA",
"Bellevue-WA",
"Olympia-WA"
]
Also if the input which you give is in unicode format, i suggest you to add to_json | from_json expressions as mentioned below:
selected_cities: "{{ test.locations| to_json | from_json | json_query(\"[?state == 'WA'].[join('-',[name,state])][]\") }}"
Using above expression will eliminate unicode error whil using the values or in any condition.
Check JMESPath site for more detail on the json_query, it was really helpful in resolving the issue.
For example
- debug:
msg: "{{ locations|
json_query('[?state == `WA`].[name,state]')|
map('join', '-')|list }}"
gives
msg:
- Seattle-WA
- Bellevue-WA
- Olympia-WA
The same result gives the task below using Jinja2 filters only
- debug:
msg: "{{ _names|zip(_states)|map('join', '-')|list }}"
vars:
_locations: "{{ locations|selectattr('state', 'eq', 'WA')|list }}"
_names: "{{ _locations|map(attribute='name')|list }}"
_states: "{{ _locations|map(attribute='state')|list }}"
json_query issue (fixed in 2.10 and later)
There is JMESPath join. Unfortunately
- debug:
msg: "{{ locations|
json_query('[].join(`-`, [name,state])') }}"
fails
msg: |-
JMESPathError in json_query filter plugin:
In function join(), invalid type for value: Seattle, expected one of: ['array-string'], received: "AnsibleUnicode"
to_json|from_json workaround
Quoting from json_query: Add examples for starts_with and contains #72821
data structure returned from register variables needs to be parsed using to_json | from_json in order to get a correct result. Fixes: ansible-collections/community.general#320
- debug:
msg: "{{ locations|to_json|from_json|
json_query('[].join(`-`, [name,state])') }}"
gives
msg:
- Seattle-WA
- New York-NY
- Bellevue-WA
- Olympia-WA
Just for the sake of a pure JMESPath way of doing it, as your trial and error solution still have an unneeded extra layer of complexity.
When you are doing
[?state == 'WA'].[join('-', [name, state])][]
You are creating an array [join('-', [name, state])] then flattening it [] for no reason.
You can just go to the solution with a shorter approach:
[?state == `WA`].join(`-`, [name, state])
Also mind that you can overcome the quotes in quotes (simple or double) complication for JMESPath queries using:
YAML multilines string: How do I break a string in YAML over multiple lines?
Backticks in your JMESPath query, as pointed in the documentation:
In the example above, quoting literals using backticks avoids escaping quotes and maintains readability.
Source: https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#selecting-json-data-json-queries
So you end up with (see note below if you are on an Ansible version < 2.10):
- debug:
msg: >-
{{ test.locations
| json_query('[?state == `WA`].join(`-`, [name, state])') }}
Please note: as raised by #Vladimir Botka on the versions prior to 2.10, you will be affected by this issue: https://github.com/ansible/ansible/issues/27299#issuecomment-331068246, forcing you to add a | to_json | from_json filter on the list.
Given the playbook:
- hosts: all
gather_facts: yes
tasks:
- debug:
msg: >-
{{ test.locations
| json_query('[?state == `WA`].join(`-`, [name, state])')
}}
vars:
test:
locations:
- name: Seattle
state: WA
- name: New York
state: NY
- name: Bellevue
state: WA
- name: Olympia
state: WA
This yields:
[
"Seattle-WA",
"Bellevue-WA",
"Olympia-WA"
]
I want to filter this variable hp but it getting print as square bracket with "".
how do i remove square bracket with "" just to get the only value. can someone please help here ?
I was looking as regex but not able to find the exact syntax.
srv_make1: '{{ basic_params | from_json | json_query("servers.server_details[*].srv_make") }}'
Thanks
I had something similar.
Was getting
["abc"]
to overcome it, had to do 2 things:
append | [0] to the json query
use replace to get rid of "
so in your case instead of
srv_make1: '{{ basic_params | from_json | json_query("servers.server_details[*].srv_make") }}'
it will look something like
srv_make1: '{{ basic_params | from_json | json_query("servers.server_details[*].srv_make | [0]") | replace('\"','') }}'
Q: "How to remove square bracket & double quote?"
json_query always returns a list. It depends on the debug task how a list is displayed. For example
vars:
srv_make1: [a,b,c]
tasks:
- debug:
var: srv_make1
- debug:
msg: "{{ srv_make1|to_yaml }}"
give
TASK [debug] ***
ok: [localhost] => {
"srv_make1": [
"a",
"b",
"c"
]
}
TASK [debug] ***
ok: [localhost] => {
"msg": "[a, b, c]\n"
}
It's possible to use template and write the list into a file without brackets and quotes. For example the template
shell> cat srv_make1.conf.j2
{% for item in srv_make1 %}{{ item }} {% endfor %}
with the task
- template:
src: srv_make1.conf.j2
dest: srv_make1.conf
gives
shell> cat srv_make1.conf
a b c
There is a simple JSON file, sample.json with the following content:
{
"test": {
"domain": [
{
"name": "cluster1"
}
]
}
}
With Ansible, I want to query over the test key, which works with the following Ansible playbook.
---
- hosts: localhost
vars:
tmpdata: "{{ lookup('file','sample.json') | from_json }}"
- debug:
msg: "{{ tmpdata | json_query('test') }}"
The play
ok: [localhost] => {
"msg": {
"domain": [
{
"name": "cluster1"
}
]
}
}
However, when they key in the JSON file is changed, from test to test/something, and the ansible json_query from test to test/something as well, Ansible/JMESPath produces an error.
fatal: [localhost]: FAILED! => {"msg": "JMESPathError in json_query filter plugin:\nBad jmespath expression: Unknown token /:\ntest/something\n ^"}
I've looked into the JMESpath documentation, but it does not make sense to me.
How can I ensure JMESpath works with forward slashes in the Ansible query.
JMESPath defines identifier as unquoted-string / quoted-string.
unquoted-string is A-Za-z_. Anything else should be quoted.
In your case:
- debug:
msg: "{{ tmpdata | json_query('\"test/something\"') }}"
Here we escape \" because we are inside YAML double quotes msg: "...".
I have key of an object variable I can't escape in Ansible playbook, possibly because of containing dots,
Here's the data structure of variable:
"results":[
{
//snip//
"changed": false,
"hostvars[item].commandResult.stdout": "abc",
//snip//
},
{
//snip//
"changed": true,
"hostvars[item].commandResult.stdout": "xyz",
//snip//
}
]
I'm unable to extract "hostvars[item].commandResult.stdout" inside it with this playbook,
- debug:
msg: "{{variable.results | map(attribute='hostvars[item].commandResult.stdout') }}"
While I can get other value just fine,
- debug:
msg: "{{variable.results | map(attribute='changed') }}"
I tried with \ , '.', and {{...}} to escape . (dot) but still no luck.
I suspect it's . because of this error message:
msg: |-
The task includes an option with an undefined variable. The error was: 'dict object' has no attribute 'hostvars[item]'
When running ansible-playbook -vvv command
How I can map "hostvars[item].commandResult.stdout" ?
Turned out I work it around by wrapping inside Ansible set_fact first:
- set_fact:
variable: "{{hostvars[item].commandResult.stdout_lines}}"
with_items: "{{ groups['servers'] }}"
register: all_result
- debug:
msg: "{{all_result.results | map(attribute='ansible_facts') | list | to_nice_json }}
The original question to escape dots remain unanswered, though.
To address a key with dots, use array notation with single quotes instead of dot notation, i.e.:
- debug:
msg: "{{variable.results | map(attribute=['hostvars[item].commandResult.stdout']) }}"
This returns the value.
cf.: Ansible FAQ
I have variable named "network" registered in Ansible:
{
"addresses": {
"private_ext": [
{
"type": "fixed",
"addr": "172.16.2.100"
}
],
"private_man": [
{
"type": "fixed",
"addr": "172.16.1.100"
},
{
"type": "floating",
"addr": "10.90.80.10"
}
]
}
}
Is it possible to get the IP address ("addr") with type="floating" doing something like this?
- debug: var={{ network.addresses.private_man | filter type="fixed" | get "addr" }}
I know the syntax is wrong but you get the idea.
To filter a list of dicts you can use the selectattr filter together with the equalto test:
network.addresses.private_man | selectattr("type", "equalto", "fixed")
The above requires Jinja2 v2.8 or later (regardless of Ansible version).
Ansible also has the tests match and search, which take regular expressions:
match will require a complete match in the string, while search will require a match inside of the string.
network.addresses.private_man | selectattr("type", "match", "^fixed$")
To reduce the list of dicts to a list of strings, so you only get a list of the addr fields, you can use the map filter:
... | map(attribute='addr') | list
Or if you want a comma separated string:
... | map(attribute='addr') | join(',')
Combined, it would look like this.
- debug: msg={{ network.addresses.private_man | selectattr("type", "equalto", "fixed") | map(attribute='addr') | join(',') }}
I've submitted a pull request (available in Ansible 2.2+) that will make this kinds of situations easier by adding jmespath query support on Ansible. In your case it would work like:
- debug: msg="{{ addresses | json_query(\"private_man[?type=='fixed'].addr\") }}"
would return:
ok: [localhost] => {
"msg": [
"172.16.1.100"
]
}
Not necessarily better, but since it's nice to have options here's how to do it using Jinja statements:
- debug:
msg: "{% for address in network.addresses.private_man %}\
{% if address.type == 'fixed' %}\
{{ address.addr }}\
{% endif %}\
{% endfor %}"
Or if you prefer to put it all on one line:
- debug:
msg: "{% for address in network.addresses.private_man if address.type == 'fixed' %}{{ address.addr }}{% endfor %}"
Which returns:
ok: [localhost] => {
"msg": "172.16.1.100"
}