I have a variable in the inventory that contains a JSON formatted data.
I want to extract a specific part of the data with json_query.
The variable contains a list of domains with related IP addresses (the JSON is valid):
DOMAIN={"domain1.net": {"addr": ["10.10.10.10", "10.10.10.20"]}, "domain2.net": {"addr": ["172.16.10.1", "172.16.20.1", "172.16.30.1"]}}
With an ansible-playbook, using json_query, I want to extact only the domain2.net IP addresses.
I've used https://api.gopipeline.io/jmespath-tester to validate the JMESPath query.
With this filter: "domain2.net".addr in the jmespath-tester, I got the following (expected) output:
[
"172.16.10.1",
"172.16.20.1",
"172.16.30.1"
]
When I apply the same json_query with this ansible-playbook, I have no output:
Task
---
- name: Extract addr for domain2.net
tags: test
debug: msg="{{ DOMAIN | to_json | from_json | json_query("domain2.net".addr) }}"
Output:
ok: [domain-lab-1] => {
"msg": ""
}
I've tested also another query, by filtering only domain2.net in JMESPath online tester:
https://api.gopipeline.io/jmespath-tester and I get this expected output:
{
"addr": [
"172.16.10.1",
"172.16.20.1",
"172.16.30.1"
]
}
But, when I try to do the same within an Ansible playbook, still no output:
Task
---
- name: Extract addr for domain2.net
tags: test
debug: msg="{{ DOMAIN | to_json | from_json | json_query("domain2.net") }}"
Output:
ok: [domain-lab-1] => {
"msg": ""
}
If I try to print only the DOMAIN var, I can see the whole JSON output.
So, the variable is correctly read.
I'm using ansible 2.9.14.
I've read that the to json|from json from here:
Ansible : filter elements containing string with JMESPath
I'm not sure if is needed in my case, anyway adding or removing them does not make any difference.
You don't need json_query. Simply reference the attribute. You can't use the dot notation because the attribute domain2.net is not a valid name of the variable in Ansible. Put it into the brackets instead. For example
- name: Extract addr for domain2.net
debug:
msg: "{{ DOMAIN['domain2.net'].addr }}"
gives
msg:
- 172.16.10.1
- 172.16.20.1
- 172.16.30.1
Notes
See Referencing key:value dictionary variables.
Any string is a valid key in the YAML dictionary(mapping).
Ansible variable name can only include letters, numbers, and underscores.
Related
I have a simple list with strings. I want to select the item in the list based on the presence of a predefined string 'fixedaddress'. I created a query and I expect the query to return the first item/string from the example input list, but it is retruning the following error:
fatal: [localhost]: FAILED! => {"msg": "JMESPathError in json_query filter plugin:\nIn function contains(), invalid type for value: fixedaddress/ZG5zLmZpeGVkX2FkZHJlc3MkMTAuMjM5LjEyLja, expected one of: ['array', 'string'], received: \"unknown\""}
Input (strings are shortened because of sensitive info, rest of the string contains another : and /, not sure if that is related to the error):
{
[
"fixedaddress/ZG5zLmZpeGVkX2FkZHJlc3MkMTAuMjM5LjEyLja",
"record:a/ZG5zLmJpbmRfYSQuMjMubmwubW9kLG1hcmMwMDAxLW1"
]
}
Ansible code:
- name: "Delete IP Record: Task 2.2a: Filter Results."
vars:
jmesquery: "[?contains(#,`fixedaddress`)]"
set_fact:
ip_record_ref: "{{ ip_record_refs | json_query(jmesquery) }}"
when: ip_record_refs | length > 1
I found the solution. Apparently the Ansible version we are using has a bug. See also:
https://github.com/ansible/ansible/issues/27299
So workaround was to use the following as JMESpath query:
ip_record_ref: "{{ ip_record_refs | to_json | from_json | json_query(jmesquery) }}"
I want to get an array of values of the specific variable from all hosts. For example: let's say I have host dc1.com with variable test_var = "value1" and host dc2.com with variable test_var = "value2". I want to get an array of these values that looks like this [ "value1", "value2" ]. Is it possible?
You can fetch any existing var from any available host through the hostvars dict availab as a magic var
e.g.:
hostvars["dc1.com"].test_var
If you want a list of all found values, the following will got through all hosts in your inventory and extract the defined values in a list. Check the documentation on filters for more details and to arrange to your exact requirements.
- name: Show a list of all test_var values in my inventory
debug:
msg: "{{ hostvars | dict2items | selectattr('value.test_var', 'defined') | map(attribute='value.test_var') }}"
Bonus (once more study the documentation above for more explanation), almost the same as above but only for your two example hosts. Note I dropped the attribute defined filter taking for granted the var will exist.
- name: Show list of test_var valuse for dc1 and dc2 hosts
vars:
host_list:
- dc1.com
- dc2.com
debug:
msg: "{{ host_lists | map('extract', hostvars) | map(attribute='test_var') }}"
I have a list of dict mydicts=[{name: foo, data: 1}, {name: foo1, data: 3}, {name: bar, data: 2}] and a list of names names=[foo, foo1, mars, jonh]
I want to create the list of dict only contains names in the list. I know if I want to select single dict I can do jq="[?(name=='foo')]" then mydicts | json_query(jq). However I cannot make the contains version work so far. I need something like [?(contains(names, name))]. Can any one show me an example about how to do this?
It seems json_query correctly get the value name with contains in jq2, but it think it is a variable instead of string. I just found out there is a bug in ansible we need this | to_json | from_json to handle it.
In jq3, it seems it never get the value names I guess I need to wrap both mydicts and names into a dict and pass to jsn_query
- hosts: localhost
gather_facts: False
vars:
mydicts:
- name: foo
data: 1
- name: bar
data: 2
names:
- foo
- foo1
- mars
- jonh
jq1: "[?(name == 'foo')]"
jq2: "[?(contains(name, 'f'))]"
jq3: "[?(contains(names, name))]"
tasks:
- debug:
var: json
- name: JMEPath test equal
debug:
msg: "{{mydicts | json_query(jq1)}}"
- name: JMEPath test str contain str does not work
debug:
msg: "{{mydicts | json_query(jq2)}}"
- name: JMEPath test another list contain str
debug:
msg: "{{mydicts | json_query(jq3)}}"
ignore_errors: yes
TASK [JMEPath test equal] ************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
{
"data": 1,
"name": "foo"
}
]
}
TASK [JMEPath test str contain str] **************************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "JMESPathError in json_query filter plugin:\nIn function contains(), invalid type for value: foo, expected one of: ['array', 'string'], received: \"unknown\""}
...ignoring
TASK [JMEPath test another list contain str] *****************************************************************************************************************************
fatal: [localhost]: FAILED! => {"msg": "JMESPathError in json_query filter plugin:\nIn function contains(), invalid type for value: None, expected one of: ['array', 'string'], received: \"null\""}
...ignoring
For this simple example, there is a work around: using jinja filter "{{mydicts | selectattr('name', 'in', names) | list}}". But I still need the json_query functionality for deep nested keys.
JMESPath does not have visibility into the jinja2 variables in the same way that the jinja2 filters do, which is why the reference to names doesn't do what you are expecting: JMESPath thinks that is a field on the current-node named names, rather than a reference to that jinja2 variable
AFAIK you can either construct an artificial input structure just to make both of those pieces of data available to JMESPath (akin to {"names": [...], "mydicts": ...} or you can inline the names list into the JMESPath query:
- debug:
msg: '{{ mydicts | json_query(jq) }}'
vars:
jq: '[? contains(`{{ names | to_json }}`, name) ]'
Q: "[?(contains(names, name))]. Can anyone show me an example of how to do this?"
A: IMHO. It's not possible. The function contains takes the item from the input and searches if the item contains the parameter.
boolean contains(array|string $subject, any $search)
You need a function with the same functionality but switched parameters. Hypothetically,
boolean in(any $search, array|string $subject)
which could be translated to a (hypothetical) query
jq: "[?(in(name, names))]"
There is no such function.
Notes:
selectattr can handle nested attributes
I have an ansible playbook, which I need to compare values returned from a task to a variable loaded from metadata file.
This metadata can be in any format, and I decided to go with YAML.
What I'm trying to achieve is to build a variable name from another variable + extra stuff and then lookup this value.
I've searched for answers over the web but I couldn't find any. There are also some similar questions here on SO, but they don't address exactly my issue.
Here's the code:
temp_task.yml
---
- name: Temp task
hosts: xenservers
gather_facts: no
vars_files:
- vars/xenservers_metadata.yml
tasks:
- command: ls /home # just a dummy task..
ignore_errors: yes
- set_fact: nic={{ inventory_hostname }}.network
- debug: msg={{ nic }}
- debug: msg={{ xen_perf.network }}
xenservers_metadata.yml
---
- xen:
network:
- xenbr0: "9b8be49c-....-....-...-..."
I'm trying to get the two debug messages print the same thing. One was constructed dynamically by {{ inventory_hostname }}.network while the other is explicit variable I loaded.
TASK [debug] ********************************************************************************************************************************************************
ok: [xen_perf] => {
"msg": "xen.network"
}
TASK [debug] ********************************************************************************************************************************************************
ok: [xen] => {
"msg": [
{
"xenbr0": "9b8be49c-....-....-...-..."
}
]
}
The first debug just prints the string. The second prints the actual data I need. How can I achieve the second data output by constructing the variable/attribute dynamically?
You don’t build the variable name dynamically in your example.
All variables (not facts) are stored in vars structure and you can access them this way:
- debug:
msg: "{{ vars[inventory_hostname].network }}"
I am writing a playbook in which I need to execute mysql query and pass
output in json format. playbook code working fine just I want facing error in string concatenate part. If I am passing sample json string it is working fine.
- name: Execute php script with json parameter
command: php -f /path/to/php/test.php "{'colors':{'red','blue','yellow'}}"
register: php_output
output.stdout_lines is variable already set in my playbook which contains output in {'red','blue','yellow'} format.
- name: Execute php script with json parameter
command: php -f /path/to/php/test.php '{"stdout_lines": {{output.stdout_lines}} }'
register: php_output
So how can I concatenate output.stdout_lines variable in '{"stdout_lines": {{output.stdout_lines}} }' ? any suggestions
stdout_lines was created for convenience. Back in the day we only had stdout. Which is what you want, I think:
command: php -f /path/to/php/test.php '{"stdout_lines": {{output.stdout}} }'
and if you want to really concat yourself, say because you have your own list of strings then you can use the Jinja2 built-in filter join:
- hosts: localhost
gather_facts: False
tags: so9
vars:
- str_list: ['Hello', 'world', '!', ]
tasks:
- debug: msg="orig list={{ str_list }}"
- debug: msg="concated = {{ str_list | join(' ') }}"
- set_fact: concated_str="{{ str_list | join(' ') }}"
- debug: msg="assigned to a variable concated_str = {{ concated_str }}"
Try like this.
vars:
- img_path: "/var/lib/libvirt/images/{{ instance_name }}-disk2.img"
Where instance name is a another variable
This will do
- name: Generate JSON output based on template
template: src=colors.json.j2 dest=/tmp/colors.json
with_items: colors
It will generate a file like
{'colors':
{
'blue',
'red',
'green',
}
}
The to_json filter is what you want.
Ansible doesn't store variables as JSON strings, it stores them as Python objects. When you're debugging e.g. with -vvv, Ansible displays these Python objects as JSON strings, but that's for your convenience.
To produce the desired command:
- name: Execute php script with json parameter
vars:
# define my desired object
my_object:
colors: "{{ output.stdout_lines }}"
# my_object contains a dict with a single key
# "colors" who's value is the list contained
# in output.stdoutlines
command: "php -f /path/to/php/test.php {{ my_object | to_json | quote }}"
register: php_output
to_json takes the object stored in my_object and outputs a JSON string.
quote automatically handles any shell quoting that might be required for the JSON string. e.g. If my JSON string contains double-quotes, quote will wrap the whole JSON string in single-quotes.