Is it possible to include a JSON query in the actual task? All of the examples show using an additional var for the query.
Taking the example from Ansible Filters
- name: "Display all ports from cluster1"
debug:
var: item
loop: "{{ domain_definition | json_query(server_name_cluster1_query) }}"
vars:
server_name_cluster1_query: "domain.server[?cluster=='cluster1'].port"
Converted to:
- name: "Display all ports from cluster1"
debug:
var: item
loop: "{{ domain_definition | json_query(domain.server[?cluster=='cluster1'].port) }}"
as is it returns:
FAILED! => {"reason": "Syntax Error while loading YAML.\n found unknown escape character '?'
I have tried to add an escape backslash before the question mark, but it still fails with:
"template error while templating string: unexpected char '?'
It's possible to use back-ticks ` . For example
- name: "Display all ports from cluster1"
debug:
var: item
loop: "{{ domain_definition | json_query('domain.server[?cluster==`cluster1`].port') }}"
(not tested)
Ok it appears you can escape double quote the query and it will work!
- name: "Display all ports from cluster1"
debug:
var: item
loop: "{{ domain_definition | json_query(\"domain.server[?cluster=='cluster1'].port\") }}"
To step a little bit aside from escape hell, I like to take advantage of yaml scalar blocks
- name: "Display all ports from cluster1"
vars:
my_query: >-
domain.server[?cluster=='cluster1'].port
debug:
var: item
loop: "{{ domain_definition | json_query(my_query) }}"
You don't have to escape anything this way (works for quotes and backslash as well, nice for regexps)
Related
I am trying to remove the last character of the below output but I am having trouble doing it:
- debug:
msg: "{{ ansible_mounts|json_query('[?mount == `/`].device') }}"
register: rootpart
The below works with simple text:
- debug:
msg: "{{ '/dev/sda4'[:-1] }}"
But not with a variable:
- debug:
msg: "{{ rootpart[:-1] }}"
Error:
msg: Unexpected templating type error occurred on ({{ rootpart[:-1] }}): unhashable type: 'slice'
There is a big "don't" in your current attempt: you should not register on a debug task.
If you want to create a new variable, then use the set_fact module.
This is not only a better way to do it, it is also saving you from having a dictionary with the keys changed, failed and msg to dive into in order to get the variable you were expecting out of the msg property.
Then, your json_query is going to return you a list of devices, no matter if there is only one match thanks to your filter. So, you also need to get the first element of this list.
So, with all this, here are your two tasks:
- set_fact:
root_part: "{{ ansible_mounts | json_query(_query) }}"
vars:
_query: "[?mount == `/`].device | [0]"
- debug:
var: root_part[:-1]
I'm trying write role to install MySql 8, and get problem with this:
- name: Extract root password from logs into {{ mysql_root_old_password }} variable
ansible.builtin.slurp:
src: "{{ mysql_logfile_path }}"
register: mysql_root_old_password
#when: "'mysql' in ansible_facts.packages"
- name: Extract root password from logs into {{ mysql_root_old_password }} variable
set_fact:
mysql_root_old_password: "{{ mysql_root_old_password.content | b64decode | regex_findall('generated for root#localhost: (.*)$', 'multiline=true') }}"
#when: "'mysqld' in ansible_facts.packages"
- name: Get Server template
ansible.builtin.template:
src: "{{ item.name }}.j2"
dest: "{{ item.path }}"
loop:
- { name: "my.cnf", path: "/root/.my.cnf" }
notify:
- Restart mysqld
on the .my.cnf I get password with quotes and brackets:
[client]
user=root
password=['th6k(gZeJSt4']
How to trim that?
What I try:
- name: trim password
set_fact:
mysql_root_old_password2: "{{ mysql_root_old_password | regex_findall('[a-zA-Z0-9,()!##$%^&*]{12}')}}"
Thanks.
The result of regex_findall is a list because there might be more matches. Take the last item
- set_fact:
mysql_root_old_password: "{{ mysql_root_old_password.content|
b64decode|
regex_findall('generated for root#localhost: (.*)$', 'multiline=true')|
last }}"
From your description
on the .my.cnf I get password with quotes and brackets ... How to trim that
I understand that you like to read a INI file like my.cnf.ini
[client]
user=root
password=['A1234567890B']
where the value of the key password looks like a list with one element in YAML and the structure doesn't change, but you are interested in the value without leading and trailing square brackets and single quotes only.
To do so there are several possibilities.
Via Ansible Lookup plugins
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Extract root password from INI file
debug:
msg: "{{ lookup('ini', 'password section=client file=my.cnf.ini') }}"
register: result
- name: Show result with type
debug:
msg:
- "{{ result.msg }}"
- "result.msg is of type {{ result.msg | type_debug }}"
- "Show password only {{ result.msg[0] }}" # the first list element
Such approach will work on the Control Node.
Like all templating, lookups execute and are evaluated on the Ansible control machine.
Further Q&A
How to read a line from a file into an Ansible variable
What is the difference between .ini and .conf?
Further Documentation
ini lookup – read data from an INI file
Via Ansible shell module, sed and cut.
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Extract root password from INI file
shell:
cmd: echo $(sed -n 's/^password=//p' my.cnf.ini | cut -d "'" -f 2)
register: result
- name: Show result
debug:
msg: "{{ result.stdout }}"
Please take note regarding "I get password with quotes and brackets ... ['<pass>'] ... How to trim that?" that from perspective of the SQL service, .cnf file, [' and '] are actually part of the password!
- name: Server template
template:
src: "my.cnf.ini"
dest: "/root/.my.cnf"
If that is correct they will be necessary for proper function of the service.
this
- name: ugly
hosts: localhost
vars:
badstr: "asdf{%jkl"
mydir: "."
mydict:
filea:
Value: "blue!42!"
fileb:
Value: "a{%isbad"
tasks:
- copy:
dest: "{{ item.key }}"
content: "{{ item.value.Value }}"
loop: "{{ mydict|default({})|dict2items }}"
gives me this error:
fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while templating 'asdf{%jkl'. Error was a <class 'ansible.errors.AnsibleError'>, original message: template error while templating string: Encountered unknown tag 'jkl'.. String: asdf{%jkl"}
The 'mydict' structure is returned from a plugin and I do not get to define the members. One of the 'Value's contains a "{%". Any reference to it will cause an error, whether as a variable, file content or in a template.
I have tried all kinds of quoting and combinations of unsafe, {{, %raw, etc. It either gives me the error or puts the name of the variable in the file.
How can I write the value to a file? Or just use it as a variable?
Ansible 2.8.4 on MacOS 11.3, also ansible 2.9 on RHEL 7.
You can use !unsafe for the variables expected to have these chars. Check this documentation. when !unsafe is used, the string/variable will never get templated.
- name: ugly
hosts: localhost
vars:
badstr: !unsafe "asdf{%jkl"
mydir: "."
mydict:
filea:
Value: !unsafe "blue!42!"
fileb:
Value: !unsafe "a{%isbad"
tasks:
- copy:
dest: "{{ item.key }}"
content: "{{ item.value.Value }}"
loop: "{{ mydict|default({})|dict2items }}"
When handling values returned by lookup plugins, Ansible uses a data
type called unsafe to block templating. Marking data as unsafe
prevents malicious users from abusing Jinja2 templates to execute
arbitrary code on target machines. The Ansible implementation ensures
that unsafe values are never templated. It is more comprehensive than
escaping Jinja2 with {% raw %} ... {% endraw %} tags.
You can use the same unsafe data type in variables you define, to
prevent templating errors and information disclosure. You can mark
values supplied by vars_prompts as unsafe. You can also use unsafe in
playbooks. The most common use cases include passwords that allow
special characters like { or %, and JSON arguments that look like
templates but should not be templated. For example:
---
mypassword: !unsafe 234%234{435lkj{{lkjsdf
The problem here is not in the copy task where the values are
evaluated; the problem is how they are being set. For example, if I
create a simple ansible module named example.sh that looks like
this:
#!/bin/sh
cat <<EOF
{
"files": {
"filea": {
"Value": "blue!42!"
},
"fileb": {
"Value": "a{%isbad"
}
}
}
EOF
I can write a playbook like this:
- name: ugly
hosts: localhost
tasks:
- example:
register: mydict
- copy:
dest: "{{ item.key }}"
content: "{{ item.value.Value }}"
loop: "{{ mydict.files|dict2items }}"
And this runs as expected, creating without any errors a file fileb with the content:
a{%isbad
Similarly, if I read the data from a JSON file and pass it through from_json, it also works fine:
- name: ugly
hosts: localhost
tasks:
- set_fact:
mydict: "{{ lookup('file', 'data.json')|from_json }}"
- copy:
dest: "{{ item.key }}"
content: "{{ item.value.Value }}"
loop: "{{ mydict.files|dict2items }}"
The problem only happens if you define the variables in a context in
which Ansible is looking for Jinja templating -- so, as the values of
variables in a playbook, a vars file, the arguments to set_fact,
etc.
You can potentially work around the problem by changing how you are
consuming these values.
I'm working on a playbook to modify a sql script, I need to create a string with a concatenation of conditionals.
this is the conditional structure:
(server.primary_name||'.'||server.arpa_domain like '%server_name%')
This is the ansible task:
- name: Query Conditionals
set_fact:
query_conditionals_list: "{{ query_conditionals_list + [ '(server.primary_name||.||server.arpa_domain like %' + item + '%)' ] }}"
with_items: "{{ server_names }}"
when: "item not in query_conditionals"
loop_control:
label: "{{item}}"
- name: Join names
set_fact:
query_conditionals: "{{query_conditionals_list | join (' OR ')}}"
I believe that I'm not escaping correctly the parentesis and the single qoutes.
Is it possible to escape them?
Thanks
EDIT
I made some modifications.
As suggested by mdaniel, instead of creating the string from the get go, I'm creating first a list and then doing a join with the OR. So that solves my OR problem.
Although, I was not able to escape the single quotes yet
I was able to find a workaround using vars on the ansible task.
While using vars, I was able to maintain the structure I wanted to use and save the single quotes.
- name: Query Conditionals
set_fact:
query_conditionals_list: "{{ query_conditionals_list + [ server_item ] }}"
when: "item not in query_conditionals_list"
loop_control:
label: "{{item}}"
vars:
server_item: "(server.primary_name||'.'||server.arpa_domain like '%{{item}}%')"
with_items: "{{ server_names }}"
I was a little bit surprised to discover that his piece of code fails with an IOError exception instead of defaulting to omitting the value.
#!/usr/bin/env ansible-playbook -i localhost,
---
- hosts: localhost
tasks:
- debug: msg="{{ lookup('ini', 'foo section=DEFAULT file=missing-file.conf') | default(omit) }}"
How can I load a value without raising an exception?
Please note that the lookup module supports a default value parameter but this one is useless to me because it works only when it can open the file.
I need a default value that works even when the it fails to open the file.
As far as I know Jinja2 unfortunately doesn't support any try/catch mechanism.
So you either patch ini lookup plugin / file issue to Ansible team, or use this ugly workaround:
---
- hosts: localhost
gather_facts: no
tasks:
- debug: msg="{{ lookup('first_found', dict(files=['test-ini.conf'], skip=true)) | ternary(lookup('ini', 'foo section=DEFAULT file=test-ini.conf'), omit) }}"
In this example first_found lookup return file name if file exists or empty list otherwise. If file exists, ternary filter calls ini lookup, otherwise omit placeholder is returned.
In case people like me stumble upon this question in 2022,
Ansible now supports rescue blocks, which is similar to try-catch-finally in programming languages.
Examples can be found in the official documentation Error handling with blocks.
You can use block/rescue as follows:
- hosts: localhost
tasks:
- block:
- debug: msg="{{ lookup('ini', 'foo section=DEFAULT file=missing-file.conf') }}"
rescue:
- debug: msg="omit"
You can also convert your input file with a from_yaml filter before using the default filter
- name: "load a yaml file or a default value"
set_fact:
myvar: "{{ lookup('file', 'myfile.yml', errors='ignore') | from_yaml | default(mydefaultObject, true) }}"
To avoid the error when the path doesn't exist, use a condition to check for the path before attempting the lookup:
---
- hosts: localhost
tasks:
- debug: msg="{{ lookup('ini', 'foo section=DEFAULT file=missing-file.conf') }}"
when: missing-file.conf | exists
You can use this with set_fact as well, then omit the undefined var when using it if required:
- hosts: localhost
tasks:
- set_fact:
foo: "{{ lookup('ini', 'foo section=DEFAULT file=missing-file.conf') }}"
when: missing-file.conf | exists
- debug:
var: foo # undefined
msg: "{{ foo | default(omit) }}" # omitted
Note that lookups and Jinja2 tests run on the controller. If you need to check the path on the host, use the stat and either slurp or fetch modules:
- stat:
file: missing-remote-file-with-text-i-want
register: file
- slurp:
src: missing-remote-file-with-text-i-want
register: slurp
when: file.stat.exists
- set_fact:
foo: "{{ slurp.content | b64decode }}"
when: file.stat.exists
- fetch:
src: missing-file.conf
dest: /tmp/fetched
fail_on_missing: False
- set_fact:
bar: "{{ lookup('ini', 'foo section=DEFAULT file=/tmp/fetched/' + inventory_hostname + '/missing-file.conf') }}"
when: ('/tmp/fetched/' + inventory_hostname + '/missing-file.conf') | exists
Second note, in Ansible v2.5 the grammar for using the path tests was changed, the format is now:
- set_fact:
foo: "{{ lookup('ini', 'foo section=DEFAULT file=missing-file.conf') }}"
when: missing-file.conf is exists