Is there any way in Ansible to set a variable to some structured value (e.g. a list or a dictionary) that is constructed from an expression?
For example, I have these variables:
all_flavours: ["vanilla", "chocolate", "strawberry", "banana"]
flavours_to_keep: ["chocolate", "banana"]
I want to evaluate the expression all_flavours | difference(flavours_to_keep) and store the resulting structured collection (list or indeed set) in the variable flavours_to_drop, so that for these values it's as if I had defined:
flavours_to_drop: ["vanilla", "strawberry"]
However, I can't find a syntax to do this. Here are my attempts:
Attempt 1
flavours_to_drop: all_flavours | difference(flavours_to_keep)
This is simply interpreted as a string:
flavours_to_drop: "all_flavours | difference(flavours_to_keep)"
Attempt 2
flavours_to_drop: {{ all_flavours | difference(flavours_to_keep) }}
This is an error because YAML interprets the "{" to mean a dictionary.
Attempt 3
flavours_to_drop: "{{ all_flavours | difference(flavours_to_keep) }}"
This does the set difference correctly, but then converts it to a string, so I end up with:
flavours_to_drop: "set([\"vanilla\", \"strawberry\"])"
How can I evaluate the expression, but store the structured object straight back into a variable instead of converting it into a string?
I should add that I'm using Ansible version 1.6.2. Is this behaviour that has changed between Ansible versions?
The formatting, etc. can be a little tricky. You need to quote the filter in order for it to be parsed properly. You can do it either as a var or in set_fact. You want something like this:
vars:
all_flavours: ["vanilla", "chocolate", "strawberry", "banana"]
flavours_to_keep: ["chocolate", "banana"]
flavours_to_drop_1: "{{ all_flavours | difference(flavours_to_keep) }}"
tasks:
- debug: var=all_flavours
- debug: var=flavours_to_keep
- set_fact:
flavours_to_drop_2: "{{ all_flavours | difference(flavours_to_keep) }}"
- debug: var=flavours_to_drop_1
- debug: var=flavours_to_drop_2
The result of the above is:
TASK: [debug var=all_flavours] ************************************************
ok: [localhost] => {
"all_flavours": [
"vanilla",
"chocolate",
"strawberry",
"banana"
]
}
TASK: [debug var=flavours_to_keep] ********************************************
ok: [localhost] => {
"flavours_to_keep": [
"chocolate",
"banana"
]
}
TASK: [set_fact ] *************************************************************
ok: [localhost]
TASK: [debug var=flavours_to_drop_1] ********************************************
ok: [localhost] => {
"flavours_to_drop_1": [
"vanilla",
"strawberry"
]
}
TASK: [debug var=flavours_to_drop_2] ********************************************
ok: [localhost] => {
"flavours_to_drop_2": [
"vanilla",
"strawberry"
]
}
Related
I have this playbook:
- name: "This works"
hosts: localhost
tasks:
- debug:
msg: "{{ lookup('dict', foo) | map(attribute='key') | list}}"
vars:
foo:
bar:
type: v1
baz:
type: v2
- name: "This does not work"
hosts: localhost
tasks:
- debug:
msg: "{{ lookup('dict', foo) | map(attribute='key') | list}}"
vars:
foo:
bar:
type: v1
When running this, I get the following output:
PLAY [This works] *******************************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [debug] ************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"bar",
"baz"
]
}
PLAY [This does not work] ***********************************************************************************************************************************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [debug] ************************************************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "[AnsibleUndefined, AnsibleUndefined]"
You see, it prints out bar and baz as list in the first example, but instead of a list containing just bar in the second example, I get some AnsibleUndefined output.
What do I have to change to make these filters also work with a single-item dict?
This is because lookup does not always return a list. And in your second case, if you debug, you'll see it returns one single object which is not inside a list:
{
"key": "bar",
"value": {
"type": "v1"
}
}
2 solutions to get around the problem:
instruct lookup you want a list
msg: {{ lookup('dict', foo, wantlist=true) | map(attribute='key') | list }}
use query in place of lookup which always returns a list and is better suited for this kind of processing (loops, mapping)
msg: {{ query('dict', foo) | map(attribute='key') | list }}
Reference:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html#ensuring-list-input-for-loop-using-query-rather-than-lookup
The idea is to add an item into the array if the item does not exist. The select attribute function returns an empty list even though there are items. Therefore the length of an empty is always 0, which in turn will create a new list every time in my case. So the union function in add a new comment section is returning a list with the only new comment erasing all the old ones.
I understand that the issue is with t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list|length == 0 conditional statement but I am not sure what I am doing wrong. I have tried many variations of this command. Few of them are commented. Any advice/suggestion is appreciated :)
main.yml:
- name: Get comment array from facts
set_fact:
common_motd_qsc_comments_array: "{{ ansible_local['snps']['motd']['comment_array'] }}"
register: t_array_exists
when:
- ansible_local['snps']['motd'] is defined
- ansible_local['snps']['motd']['comment_array'] is defined
- debug:
var: t_array_exists
- debug:
var: t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list
# var: ansible_facts|selectattr("common_motd_qsc_comments_array", "defined")|list
# var: t_array_exists|selectattr("common_motd_qsc_comments_array", "defined")|list
- name: Create an empty array if there is no array
set_fact:
common_motd_qsc_comments_array: []
when:
- t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list|length == 0
- debug:
var: t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list|length
- name: Deleting a comment if it exists
set_fact:
common_motd_qsc_comments_array: "{{ common_motd_qsc_comments_array | difference([t_new_entry]) }}"
loop: "{{ common_motd_qsc_delete_comment }}"
when: t_new_entry in common_motd_qsc_comments_array
vars:
t_new_entry: "{{ item | trim }}"
- name: Add a new comment if it doesn't exist
set_fact:
common_motd_qsc_comments_array: "{{ common_motd_qsc_comments_array | union([t_new_entry]) }}"
loop: "{{ common_motd_qsc_add_comment }}"
when: t_new_entry not in common_motd_qsc_comments_array
vars:
t_new_entry: "{{ item | trim }}"
- name: Saving comments to snps.fact file
ini_file:
dest: "/etc/ansible/facts.d/snps.fact"
section: 'motd' # header
option: 'comment_array' # key
value: "{{ common_motd_qsc_comments_array }}" # value
Ansible output:
TASK [common/motd_scratch/v1 : Get comment array from facts] ***************************************************************************************
ok: [ansible-poc-cos6]
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"t_array_exists": {
"ansible_facts": {
"common_motd_qsc_comments_array": [
"new comment 1"
]
},
"changed": false,
"failed": false
}
}
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"ansible_facts|selectattr(\"common_motd_qsc_comments_array\", \"defined\")|list": []
}
TASK [common/motd_scratch/v1 : Create an empty array if there is no array] *************************************************************************
ok: [ansible-poc-cos6]
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"t_array_exists[\"ansible_facts\"]|selectattr(\"common_motd_qsc_comments_array\", \"defined\")|list|length": "0"
}
TASK [common/motd_scratch/v1 : Add a new comment if it doesn't exist] ******************************************************************************
ok: [ansible-poc-cos6] => (item=new comment 22)
TASK [common/motd_scratch/v1 : Saving comments to snps.fact file] **********************************************************************************
changed: [ansible-poc-cos6]
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"common_motd_qsc_add_comment": [
"new comment 22"
]
}
snps.fact output:
Before:
[motd]
comment_array = ['new comment 1']
After passing new_comment as "new comment 2":
[motd]
comment_array = ['new comment 2']
Expected:
[motd]
comment_array = ['new comment 1', 'new comment 2']
You can do all of the above in one single easy step. Here is an MCVE playbook to illustrate.
Prior to running the playbook, I added the following file on my machine:
/etc/ansible/facts.d/snps.fact
{
"motd_example1": [
"One message",
"A message",
"Message to delete"
],
"motd_example2": [
"Some messages",
"Mandatory message",
"Other message"
]
}
The playbook
---
- name: Add/Remove list elements demo
hosts: localhost
gather_facts: true
vars:
# The (list of) custom message(s) I want to add if not present
my_custom_mandatory_messages:
- Mandatory message
# The (list of) custom message(s) I want to remove if present
my_custom_messages_to_delete:
- Message to delete
# The list of custom vars I'm going to loop over for demo
# In your case, you can simply use your single motd var
# directly in the below task without looping. I added this
# for convenience as the expression is exactly the same
# in all cases. Only the input data changes
my_motd_example_vars:
- motd_example1
- motd_example2
- motd_example3 # This one does not even exist in ansible_local.snps
tasks:
- name: Show result using local facts for each demo test var
debug:
msg: "{{ ansible_local.snps[item] | default([])
| union(my_custom_mandatory_messages)
| difference(my_custom_messages_to_delete) }}"
loop: "{{ my_motd_example_vars }}"
- name: Proove it works with a totaly undefined var
debug:
msg: "{{ totally.undefined.var | default([])
| union(my_custom_mandatory_messages)
| difference(my_custom_messages_to_delete) }}"
The result
TASK [Gathering Facts] **********************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Show result using local facts for each demo test var] *********************************************************************************************************************************************************************************
ok: [localhost] => (item=motd_example1) => {
"msg": [
"One message",
"A message",
"Mandatory message"
]
}
ok: [localhost] => (item=motd_example2) => {
"msg": [
"Some messages",
"Mandatory message",
"Other message"
]
}
ok: [localhost] => (item=motd_example3) => {
"msg": [
"Mandatory message"
]
}
TASK [Simple proof that it works with a totaly undefined var] *******************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"Mandatory message"
]
}
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Not sure I understood exactly what the problem is, but if you are trying to merge two dictionaries (with one overwriting the other if the key already exists), then you might want to take a look at:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_advanced_syntax.html#yaml-anchors-and-aliases-sharing-variable-values
The very first example shows exactly that, with just YAML. If that doesn't do the trick, then you might want to try the combine filter:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#combining-hashes-dictionaries
HTH
Alex
I was able to fix it with the help of my colleague. I just had to check if comment_array is not defined instead of trying to find its length. A simple alternate idea which works :)
- name: Create an empty array if there is no array
set_fact:
common_motd_qsc_comments_array: []
when:
- ansible_local['snps']['motd']['comment_array'] is not defined
Thank you all for your help :)
I've got a list of string element in a ansible var. I'm looking how to append to each element of the list with a defined string.
Do you know how I can do? I didn't find a way to do so.
Input:
[ "a", "b", "c" ]
Output:
[ "a-Z", "b-Z", "c-Z" ]
I really didn't like using the add-on filters or the loop. However, I stumbled across this blog post https://www.itix.fr/blog/ansible-add-prefix-suffix-to-list/ that used a different method that worked in Ansible 2.9.x.
- set_fact:
output: "{{ list_to_suffix | product(['-Z']) | map('join') | list }}"
You can use join for this. Please see the code below:
playbook -->
---
- hosts: localhost
vars:
input: [ "a", "b", "c" ]
tasks:
- name: debug
set_fact:
output: "{{ output | default([]) + ['-'.join((item,'Z'))] }}"
loop: "{{ input | list}}"
- debug:
var: output
output -->
PLAY [localhost] ********************************************************************************************************
TASK [Gathering Facts] **************************************************************************************************
ok: [localhost]
TASK [debug] ************************************************************************************************************
ok: [localhost] => (item=a)
ok: [localhost] => (item=b)
ok: [localhost] => (item=c)
TASK [debug] ************************************************************************************************************
ok: [localhost] => {
"output": [
"a-Z",
"b-Z",
"c-Z"
]
}
PLAY RECAP **************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0
With simple filters
shell> cat filter_plugins/string_filters.py
def string_prefix(prefix, s):
return prefix + s
def string_postfix(postfix, s):
return s + postfix
class FilterModule(object):
''' Ansible filters. Python string operations.'''
def filters(self):
return {
'string_prefix' : string_prefix,
'string_postfix' : string_postfix
}
the tasks below
- set_fact:
output: "{{ input|map('string_prefix', '-Z')|list }}"
- debug:
var: output
give
"output": [
"a-Z",
"b-Z",
"c-Z"
]
The same output gives the loop below
- set_fact:
output: "{{ output|default([]) + [item + '-Z'] }}"
loop: "{{ input }}"
- debug:
var: output
Below is how both prefix and suffix can be done in one line
- debug:
var: result
vars:
prefix: foo1
suffix: foo2
a_list: [ "bar", "bat", "baz" ]
result: "{{ [prefix] | product(a_list) | map('join') | list | product([suffix]) | map('join') | list }}"
Yet another solution, for pre-fixing and post-fixing, without custom filters (which make a very elegant code, by the way):
- set_fact:
input: ['a', 'b', 'c']
suffix: '-Z'
prefix: 'A-'
- debug:
var: suffixed
vars:
suffixed: "{{ input | zip_longest([], fillvalue=suffix) | map('join') | list }}"
- debug:
var: prefixed
vars:
prefixed: "{{ [] | zip_longest(input, fillvalue=prefix) | map('join') | list }}"
"suffixed": [
"a-Z",
"b-Z",
"c-Z"
]
"prefixed": [
"A-a",
"A-b",
"A-c"
]
A lot of the other answers felt a bit cumbersome when I wanted to both append and prepend data, so I ended up using regex_replace with map instead, which simplifies things quite considerably in my opinion:
- name: Create some data to play with
set_fact:
domains:
- example-one
- example-two
- example-three
tld: com
- name: Demonstrate the method
debug:
msg: >-
{{
domains
| map('regex_replace', '^(.*)$', 'www.\1.' + tld)
}}
Outputs:
TASK [example : Create some data to play with] ****************************
ok: [server]
TASK [example : Demonstrate the method] ***********************************
ok: [server] => {
"msg": [
"www.example-one.com",
"www.example-two.com",
"www.example-three.com"
]
}
In writing this answer, I actually found this documented in the Ansible docs, so it appears this is a recommended method too.
I have a variable in my playbook that's derived from a list. In some instances this variable contains a "-" to separate two values. For example,
Numbers:
- 2211
- 2211-2212
When this is the case I would like to replace the "-" with a "_" based on a conditional: If the number is 4 characters long, do this. Else, replace the "-" with a " _ " and do that.
I've already tried to fiddle around with jinja2 ans regex in my playbooks but so far no luck. Here's what I tried,
number: {% if length(item) == 4 %} {{ item | regex_replace("^(.*)$", "Number_\1") | string }} {% else %} {{ item | regex_replace("^(.*)$", "Number_\1") |replace("-", "_") | string }}
The result that I would like to have,
Number is four characters long:
number: Number_2211
Number is more then 4 characters long:
number: Number_2211_2212
Some of the Error messages I have received are,
ERROR! Syntax Error while loading YAML.
did not find expected key
ERROR! Syntax Error while loading YAML.
found character that cannot start any token
Is there a way to achieve this within the Ansible playbook?
Thanks in advance!
It's not really clear how you're trying to use this data. Ansible isn't great at modifying complex data structures in place, but it has lots of way of transforming data when you access it. So for example, this playbook:
---
- hosts: localhost
gather_facts: false
vars:
numbers:
- "2211"
- "2211-2212"
tasks:
- debug:
msg: "number: {{ item.replace('-', '_') }}"
loop: "{{ numbers }}"
Will output:
TASK [debug] **********************************************************************************
ok: [localhost] => (item=2211) => {
"msg": "number: 2211"
}
ok: [localhost] => (item=2211-2212) => {
"msg": "number: 2211_2212"
}
If you really need to make the transformation conditional on the length (and it's not clear that you do), you could do something like:
- debug:
msg: "{{ item.replace('-', '_') if item|length > 4 else item }}"
loop: "{{ numbers }}"
Update
I see you've selected the other answer. The solution presented here seems substantially simpler (there is no "incomprehensible sequence of filters, regex expressions, and equality checks"), and produces almost identical output:
TASK [debug] **********************************************************************************
ok: [localhost] => (item=445533) => {
"msg": "445533"
}
ok: [localhost] => (item=112234-538) => {
"msg": "112234_538"
}
ok: [localhost] => (item=11) => {
"msg": "11"
}
ok: [localhost] => (item=1111) => {
"msg": "1111"
}
ok: [localhost] => (item=1111-1111) => {
"msg": "1111_1111"
}
ok: [localhost] => (item=11-11) => {
"msg": "11_11"
}
It's not clear, given 11-11, whether you expect 11_11 or 11-11 as output. If you expect the former, this answer is more correct.
You can use an incomprehensible sequence of filters, regex expressions, and equality checks to do this.
#!/usr/bin/env ansible-playbook
- name: Lets munge some data
hosts: localhost
gather_facts: false
become: false
vars:
array:
- 445533
- 112234-538
- 11
- 1111
- 1111-1111
- 11-11
tasks:
- name: Replace hypens when starting with 4 numbers
debug:
msg: "{{ ((item | string)[0:4] | regex_search('[0-9]{4}') | string != 'None')
| ternary((item | regex_replace('-', '_')), item) }}"
loop: "{{ array }}"
PLAY [Lets munge some data] *****************************************************************************************************************************************************************************************************
TASK [Replace hypens when starting with 4 numbers] ******************************************************************************************************************************************************************************
ok: [localhost] => (item=445533) => {
"msg": "445533"
}
ok: [localhost] => (item=112234-538) => {
"msg": "112234_538"
}
ok: [localhost] => (item=11) => {
"msg": "11"
}
ok: [localhost] => (item=1111) => {
"msg": "1111"
}
ok: [localhost] => (item=1111-1111) => {
"msg": "1111_1111"
}
ok: [localhost] => (item=11-11) => {
"msg": "11-11"
}
PLAY RECAP **********************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0
Explanation
Assuming I have a dictionary mydict set to { "key1": "value1" }:
The result of dictsort filter (mydict|dictsort) in Ansible seems to be a list containing another list:
[
[
"key1",
"value1"
]
]
However, when accessing the first element of this list directly in Jinja2 template (mydict|dictsort)[0], it renders to a strangely looking:
(u'key1', u'value1')
Then, if I set a fact with the value of (mydict|dictsort), it behaves like a regular list - accessing the first element with [0] results in:
[
"key1",
"value1"
]
Accessing its [0] element returns key1.
But if I set a fact with the value of (mydict|dictsort)[0], it behaves like a string - accessing [0] element returns the first character, i.e. (.
On the other hand, if I access subelements directly, for example (mydict|dictsort)[0][0], it behaves like a list, i.e. returns key1.
Questions
What is (u'key1', u'value1')? What kind of object does dictsort produce?
How to access the dictsort results in a consistent, reliable way?
Full playbook:
---
- hosts: localhost
gather_facts: no
connection: local
vars:
mydict:
key1: value1
tasks:
- name: show dict
debug:
msg: "{{ mydict }}"
- name: show mydict|dictsort
debug:
msg: "{{ mydict|dictsort }}"
- set_fact:
mydict_dictsorted: "{{ mydict|dictsort }}"
- name: show (mydict|dictsort)[0]
debug:
msg: "{{ (mydict|dictsort)[0] }}"
- name: show mydict_dictsorted[0]
debug:
msg: "{{ mydict_dictsorted[0] }}"
- name: show (mydict|dictsort|list)[0]
debug:
msg: "{{ (mydict|dictsort|list)[0] }}"
- name: show (mydict_dictsorted|list)[0]
debug:
msg: "{{ (mydict_dictsorted|list)[0] }}"
- set_fact:
mydict_dictsorted_element: "{{ (mydict|dictsort)[0] }}"
- name: mydict_dictsorted_element
debug:
msg: "{{ mydict_dictsorted_element }}"
- name: mydict_dictsorted_element[0]
debug:
msg: "{{ mydict_dictsorted_element[0] }}"
- name: (mydict|dictsort)[0][0]
debug:
msg: "{{ (mydict|dictsort)[0][0] }}"
Full transcript:
PLAY [localhost] ********************************************************************************************
TASK [show dict] ********************************************************************************************
ok: [localhost] => {
"msg": {
"key1": "value1"
}
}
TASK [show mydict|dictsort] *********************************************************************************
ok: [localhost] => {
"msg": [
[
"key1",
"value1"
]
]
}
TASK [set_fact] *********************************************************************************************
ok: [localhost]
TASK [show (mydict|dictsort)[0]] ****************************************************************************
ok: [localhost] => {
"msg": "(u'key1', u'value1')"
}
TASK [show mydict_dictsorted[0]] ****************************************************************************
ok: [localhost] => {
"msg": [
"key1",
"value1"
]
}
TASK [show (mydict|dictsort|list)[0]] ***********************************************************************
ok: [localhost] => {
"msg": "(u'key1', u'value1')"
}
TASK [show (mydict_dictsorted|list)[0]] *********************************************************************
ok: [localhost] => {
"msg": [
"key1",
"value1"
]
}
TASK [set_fact] *********************************************************************************************
ok: [localhost]
TASK [mydict_dictsorted_element] ****************************************************************************
ok: [localhost] => {
"msg": "(u'key1', u'value1')"
}
TASK [mydict_dictsorted_element[0]] *************************************************************************
ok: [localhost] => {
"msg": "("
}
TASK [(mydict|dictsort)[0][0]] ******************************************************************************
ok: [localhost] => {
"msg": "key1"
I checked the values with copy/content and they are the same as debug's (except indentation), so posting debug results for clarity.
dictsort produces a list of tuples. It uses dict.items() under the hood.
So when you access it as (mydict|dictsort)[0], you access Python's tuple.
Whereas if you access it after it is templated, you get generic list, because JSON doesn't make difference between tuples and lists, it has only lists.
Update: how to test – insert print into _dump_results here, like this:
print("Unaltered: {}".format(abridged_result))
return json.dumps(abridged_result, indent=indent, ensure_ascii=False, sort_keys=sort_keys)
And see this as the output:
TASK [show mydict|dictsort] ***************************
Unaltered: {'msg': [(u'key1', u'value1')]}
ok: [localhost] => {
"msg": [
[
"key1",
"value1"
]
]
}
Update2: why list of tuples becomes list of lists, but single tuple becomes string repr?
This is because of the fact that Jinja2 expression inside {{...}} can produce only string as its output, and there's some Ansible template magic done to try to type-cast it back to some complex type. But this magic only works with strings that looks like dicts or lists and not tuples. So if you have dict with tuples inside or list of tuples, you'll get it evaluated, but if you have a single tuple, it will remain a string. Here's demo of this:
- name: results in a string
debug:
msg: "{{ test_str }}"
vars:
test_str: "(u'a', u'b')"
- name: results in a list of tuples/lists
debug:
msg: "{{ test_str }}"
vars:
test_str: "[(u'a', u'b')]"
Output:
TASK [results in a string] ******************************************
ok: [localhost] => {
"msg": "(u'a', u'b')"
}
TASK [results in a list of tuples/lists] ****************************
ok: [localhost] => {
"msg": [
[
"a",
"b"
]
]
}