Ansible: parsing a dictionary recursively - ansible

I'm struggling to implement something like a "recursive function" in Ansible, but it seems to be a slightly tricky task.
Let's say, I have a tree-like dictionary:
tree:
alpha: 1
foo/:
bravo: 2
bar/:
charlie: 3
baz/:
delta: 4
What I need is to parse it, create some entities (let's say, a sub-directory or sub-domain, it doesn't matter what exactly) for each element which name ends with the / character and create some other entity (let's say a file or a DNS-record, it doesn't matter as well) for each element which name ends with anything else.
Here's a minimal example of the solution I'm trying to implement.
The playbook.yaml file:
- hosts:
- localhost
tasks:
- include_tasks: proceed_branch.yaml
vars:
tree:
alpha: 1
foo/:
bravo: 2
bar/:
charlie: 3
baz/:
delta: 4
The proceed_branch.yaml file:
- debug:
msg: "{{ tree }}"
- debug:
msg: "Got {{ item.key }} = {{ item.value }}"
loop: "{{ tree | dict2items }}"
when: item.key is not regex('/$')
- include_tasks: proceed_branch.yaml
vars:
tree: "{{ item.value }}"
loop: "{{ tree | dict2items }}"
when: item.key is regex('/$')
And here's what I get as the output:
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'
PLAY [localhost] ***************************************************************
TASK [include_tasks] ***********************************************************
Included: /tmp/ansible/proceed_branch.yaml for localhost
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": {
"foo/": {
"bar/": {
"baz/": {
"delta": 4
},
"charlie": 3
},
"bravo": 2
},
"alpha": 1
}
}
TASK [debug] *******************************************************************
ok: [localhost] => (item={'key': 'alpha', 'value': 1}) => {
"msg": "Got alpha = 1"
}
skipping: [localhost] => (item={'key': 'foo/', 'value': {'bravo': 2, 'bar/': {'charlie': 3, 'baz/': {'delta': 4}}}})
TASK [include_tasks] ***********************************************************
fatal: [localhost]: FAILED! => {"msg": "Unable to look up a name or access an attribute in template string ({{ tree | dict2items }}).\nMake sure your variable name does not contain invalid characters like '-': dict2items requires a dictionary, got <class 'ansible.template.AnsibleUndefined'> instead.. dict2items requires a dictionary, got <class 'ansible.template.AnsibleUndefined'> instead.. Unable to look up a name or access an attribute in template string ({{ tree | dict2items }}).\nMake sure your variable name does not contain invalid characters like '-': dict2items requires a dictionary, got <class 'ansible.template.AnsibleUndefined'> instead.. dict2items requires a dictionary, got <class 'ansible.template.AnsibleUndefined'> instead."}
PLAY RECAP *********************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
What do you think, why dict2items gets the "undefined value" at some point?
Thanks in advance!
P.S. Here's what I expect to get as the result:
PLAY [localhost] ***************************************************************
TASK [include_tasks] ***********************************************************
included: /tmp/ansible/proceed_branch.yaml for localhost
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": {
"alpha": 1,
"foo/": {
"bar/": {
"baz/": {
"delta": 4
},
"charlie": 3
},
"bravo": 2
}
}
}
TASK [debug] *******************************************************************
ok: [localhost] => (item={'key': 'alpha', 'value': 1}) => {
"msg": "Got alpha = 1"
}
skipping: [localhost] => (item={'key': 'foo/', 'value': {'bravo': 2, 'bar/': {'charlie': 3, 'baz/': {'delta': 4}}}})
TASK [include_tasks] ***********************************************************
skipping: [localhost] => (item={'key': 'alpha', 'value': 1})
included: /tmp/ansible/proceed_branch.yaml for localhost => (item={'key': 'foo/', 'value': {'bravo': 2, 'bar/': {'charlie': 3
, 'baz/': {'delta': 4}}}})
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": {
"bar/": {
"baz/": {
"delta": 4
},
"charlie": 3
},
"bravo": 2
}
}
TASK [debug] *******************************************************************
ok: [localhost] => (item={'key': 'bravo', 'value': 2}) => {
"msg": "Got bravo = 2"
}
skipping: [localhost] => (item={'key': 'bar/', 'value': {'charlie': 3, 'baz/': {'delta': 4}}})
TASK [include_tasks] ***********************************************************
skipping: [localhost] => (item={'key': 'bravo', 'value': 2})
included: /tmp/ansible/proceed_branch.yaml for localhost => (item={'key': 'bar/', 'value': {'charlie': 3, 'baz/': {'delta': 4}}})
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": {
"baz/": {
"delta": 4
},
"charlie": 3
}
}
TASK [debug] *******************************************************************
ok: [localhost] => (item={'key': 'charlie', 'value': 3}) => {
"msg": "Got charlie = 3"
}
skipping: [localhost] => (item={'key': 'baz/', 'value': {'delta': 4}})
TASK [include_tasks] ***********************************************************
skipping: [localhost] => (item={'key': 'charlie', 'value': 3})
included: /tmp/ansible/proceed_branch.yaml for localhost => (item={'key': 'baz/', 'value': {'delta': 4}})
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": {
"delta": 4
}
}
TASK [debug] *******************************************************************
ok: [localhost] => (item={'key': 'delta', 'value': 4}) => {
"msg": "Got delta = 4"
}
TASK [include_tasks] ***********************************************************
skipping: [localhost] => (item={'key': 'delta', 'value': 4})
skipping: [localhost]

Found a pretty ugly solution, though it seems to be working. The point is to add a temporary variable (by setting a fact):
- hosts:
- localhost
tasks:
- include_tasks: proceed_branch.yaml
vars:
tree:
alpha: 1
foo/:
bravo: 2
bar/:
charlie: 3
baz/:
delta: 4
- debug:
msg: "{{ tree }}"
- set_fact:
__t: "{{ tree }}"
- debug:
msg: "Got {{ item.key }} = {{ item.value }}"
loop: "{{ __t | dict2items }}"
when: item.key is not regex('/$')
- include_tasks: proceed_branch.yaml
vars:
tree: "{{ item.value }}"
loop: "{{ __t | dict2items }}"
when: item.key is regex('/$')
I hate how it looks like, though it works.
If anyone have a better idea, please, be so kind as to share.

Related

for each host need to print only one item

I have a list of descriptions that will be added one per host. I would like to print one item per host and it is becoming harder than I expected. The same item is being printed per each host.
tasks:
- name: items in list
debug:
msg: "{{ item }}"
with_items:
- ['a', 'b']
THIS IS WHAT I GET:
TASK [items in list] *****************************************************************************************************************************************************************************************************************************
ok: [Host1] => (item=a) => {
"msg": "a"
}
ok: [Host2] => (item=a) => {
"msg": "a"
}
ok: [Host1] => (item=b) => {
"msg": "b"
}
ok: [Host2] => (item=b) => {
"msg": "b"
}
THIS IS WHAT I WANT:
TASK [items in list] *****************************************************************************************************************************************************************************************************************************
ok: [Host1] => (item=a) => {
"msg": "a"
}
ok: [Host2] => (item=b) => {
"msg": "b"
}
There are many options. For example,
Create lists for the hosts
- name: items in list
debug:
msg: "{{ item }}"
loop: "{{ _list[inventory_hostname] }}"
vars:
_list:
host1: ['a']
host2: ['b']
gives
ok: [host1] => (item=['a']) =>
msg: a
ok: [host2] => (item=['b']) =>
msg: b
Calculate the index of the host in the group, e.g.
- name: items in list
debug:
msg: "{{ _list[_idx|int] }}"
vars:
_list: ['a', 'b']
_idx: "{{ groups.all.index(inventory_hostname) }}"
gives
ok: [host2] =>
msg: b
ok: [host1] =>
msg: a
Run the task once and iterate the sequence, e.g.
- name: items in list
debug:
msg: "{{ _list[item|int] }}"
with_sequence: start=0 end="{{ groups.all|length -1 }}"
vars:
_list: ['a', 'b']
run_once: true
delegate_to: localhost
gives
ok: [host1 -> localhost] => (item=0) =>
msg: a
ok: [host1 -> localhost] => (item=1) =>
msg: b

Trying to set conditional based on var type in ansible

I can't seem to find a filter for matching a string versus a list versus a dictionary. My goal is to run different task blocks based on a given variable is a certain variable type. Here is how it would be wrote up:
- name: BLOCK of stuff based on our var is a string
when: var | type == 'string'
block:
# bunch of tasks
- name: BLOCK of stuff based on our var is a list
when: var | type == 'array'
block:
# bunch of tasks
- name: BLOCK of stuff based on our var is a dictionary
when: var | type == 'dict'
block:
# bunch of tasks
You can have a look at the built-in jinja2 tests. With those available, the only difficulty is that there is no test to specifically find lists. So you have to check for vars being sequences and exclude strings and dicts.
Here is an example playbook:
--
- hosts: localhost
gather_facts: false
vars:
test_vars:
- "I'm a string"
- ['I', 'am', 'a', 'list']
- {this: is, a: dict}
- 10 # This is an integer
- 2.34 # and here we have a float
tasks:
- name: Check if var is a string
debug:
msg: this is a string
when: item is string
loop: "{{ test_vars }}"
- name: Check if var is a list
debug:
msg: this is a list
when:
- item is sequence
- item is not string
- item is not mapping
loop: "{{ test_vars }}"
- name: Check if var is a dict
debug:
msg: this is a dict
when: item is mapping
loop: "{{ test_vars }}"
which gives:
PLAY [localhost] ***********************************************************************************************************************************************************************************************************************
TASK [Check if var is a string] ********************************************************************************************************************************************************************************************************
ok: [localhost] => (item=I'm a string) => {
"msg": "this is a string"
}
skipping: [localhost] => (item=['I', 'am', 'a', 'list'])
skipping: [localhost] => (item={'this': 'is', 'a': 'dict'})
skipping: [localhost] => (item=10)
skipping: [localhost] => (item=2.34)
TASK [Check if var is a list] **********************************************************************************************************************************************************************************************************
skipping: [localhost] => (item=I'm a string)
ok: [localhost] => (item=['I', 'am', 'a', 'list']) => {
"msg": "this is a list"
}
skipping: [localhost] => (item={'this': 'is', 'a': 'dict'})
skipping: [localhost] => (item=10)
skipping: [localhost] => (item=2.34)
TASK [Check if var is a dict] **********************************************************************************************************************************************************************************************************
skipping: [localhost] => (item=I'm a string)
skipping: [localhost] => (item=['I', 'am', 'a', 'list'])
ok: [localhost] => (item={'this': 'is', 'a': 'dict'}) => {
"msg": "this is a dict"
}
skipping: [localhost] => (item=10)
skipping: [localhost] => (item=2.34)
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Check the defined variable is empty or not in Ansible

- hosts: localhost
vars:
files_list:
#it may contain the file_list like
#file_list:
# - "file*"
tasks:
- name: copy
copy:
src: "{{item}}"
dest: "/tmp/"
with_fileglob: "{{files_list}}"
when: files != None
I want to copy some multiple files with a specific pattern from the files_list. but sometimes the file_list may be empty. so how to check if the file_list is empty I have tried above code but it doesn't work. it is giving me following error
The full traceback is:<br>
Traceback (most recent call last):<br>
File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 104, in run<br>
items = self._get_loop_items()<br>
File "/usr/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 245, in _get_loop_items<br>
items = wrap_var(mylookup.run(terms=loop_terms, variables=self._job_vars, wantlist=True))<br>
File "/usr/lib/python2.7/site-packages/ansible/plugins/lookup/fileglob.py", line 60, in run<br>
term_file = os.path.basename(term)<br>
File "/usr/lib64/python2.7/posixpath.py", line 121, in basename<br>
i = p.rfind('/') + 1<br>
AttributeError: 'NoneType' object has no attribute 'rfind'<br>
fatal: [machine1.kirusa.com]: FAILED! => {<br>
"msg": "Unexpected failure during module execution.", <br>
"stdout": ""<br>
}
Can you also explain whats this mean.
thank you in advance.
Q: "Check whether the defined variable is empty or not in Ansible."
A: Simply test the variable. An empty list evaluates to False. This also covers the case when the variable is not defined. YAML None is Python null. None also evaluates to False. For example
- debug:
msg: The variable files is an empty list or None.
when: not files|default(None)
In the loop, it's not necessary to test whether the list is empty or not. An empty list will be skipped anyway.
YAML string is a list of characters. An empty string evaluates to False the same way as an empty list does.
Notes
The evaluation of 'not item' works in the context of this question if the variable is a string, list, dictionary, or None. if the variable is a boolean or number this test does not work. See the self-explaining test below
- debug:
msg: "{{ not item }}"
loop:
- ''
- []
- {}
-
- null
- true
- false
- 1
- 0
gives
TASK [debug] **************************************************
ok: [localhost] => (item=) =>
msg: true
ok: [localhost] => (item=[]) =>
msg: true
ok: [localhost] => (item={}) =>
msg: true
ok: [localhost] => (item=None) =>
msg: true
ok: [localhost] => (item=None) =>
msg: true
ok: [localhost] => (item=True) =>
msg: false
ok: [localhost] => (item=False) =>
msg: true
ok: [localhost] => (item=1) =>
msg: false
ok: [localhost] => (item=0) =>
msg: true
To fix this problem test the variable is iterable
- debug:
msg: "{{ item is iterable }}"
loop:
- ''
- []
- {}
-
- null
- false
- 0
gives
TASK [debug] ******************************************************
ok: [localhost] => (item=) =>
msg: true
ok: [localhost] => (item=[]) =>
msg: true
ok: [localhost] => (item={}) =>
msg: true
ok: [localhost] => (item=None) =>
msg: false
ok: [localhost] => (item=None) =>
msg: false
ok: [localhost] => (item=False) =>
msg: false
ok: [localhost] => (item=0) =>
msg: false
Complete test: The variable is empty or None
- debug:
msg: The variable is empty or None.
when:
- not item
- item is iterable or item is none
loop:
- ''
- []
- {}
-
- null
- true
- false
- 1
- 0
gives
TASK [debug] ***************************************************
ok: [localhost] => (item=) =>
msg: The variable is empty or None.
ok: [localhost] => (item=[]) =>
msg: The variable is empty or None.
ok: [localhost] => (item={}) =>
msg: The variable is empty or None.
ok: [localhost] => (item=None) =>
msg: The variable is empty or None.
ok: [localhost] => (item=None) =>
msg: The variable is empty or None.
skipping: [localhost] => (item=True)
skipping: [localhost] => (item=False)
skipping: [localhost] => (item=1)
skipping: [localhost] => (item=0)
A dictionary is also iterable. A list of the keys is used
- debug:
var: item
loop: "{{ files|list }}"
vars:
files:
a: 1
b: 2
c: 3
gives
TASK [debug] ********************************************************
ok: [localhost] => (item=a) =>
ansible_loop_var: item
item: a
ok: [localhost] => (item=b) =>
ansible_loop_var: item
item: b
ok: [localhost] => (item=c) =>
ansible_loop_var: item
item: c
To check if it is empty you need to give as below
when: not files_list
See Default rules here: https://docs.ansible.com/ansible-lint/rules/default_rules.html
It states: Don’t compare to empty string, use when: var rather than when: var != "" (or conversely when: not var rather than when: var == "")
You can first assert that it is indeed a list and then assert that it has more than one element:
when: (files_list is defined) and (files_list | type_debug == 'list') and (files_list | count > 0)

In Ansible, how can I iterate over stdout with an array?

Ansible v2.6.3
I have the simple task, which gets the AWS ARNs in my jenkins ECS cluster
tasks:
- command: aws ecs list-container-instances --cluster jenkins
register: jenkins_ecs_containers
- debug: var=jenkins_ecs_containers.stdout
and has the following output
TASK [debug] *******************************************************************
ok: [localhost] => {
"jenkins_ecs_containers.stdout": {
"containerInstanceArns": [
"arn:aws:ecs:us-east-1:arn0",
"arn:aws:ecs:us-east-1:arn1"
]
}
}
How can I iterate over the ARNs? I tried
- debug: var=item
with_items: jenkins_ecs_containers.stdout.containerInstanceArns
gives
TASK [debug] *******************************************************************
ok: [localhost] => (item=jenkins_ecs_containers.stdout.containerInstanceArns) => {
"item": "jenkins_ecs_containers.stdout.containerInstanceArns"
}
or
- debug: var=item
with_items: "{{ jenkins_ecs_containers.stdout.containerInstanceArns }}"
gives
TASK [debug] *******************************************************************
fatal: [localhost]: FAILED! => {"msg": "'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'containerInstanceArns'"}
to retry, use: --limit #/Users/cfouts/git-repos/ansible/playbooks/loop.retry
Thanks!
I created a file with your output. So I used set_fact. Otherwise, it's just a string, not a JSON object:
tasks:
- command: cat files/stdout.txt
register: result
- debug: var=result.stdout
- set_fact:
jenkins_ecs_containers: "{{ result.stdout }}"
- debug:
msg: "{{ item }}"
with_items: "{{ jenkins_ecs_containers.containerInstanceArns }}"
This gave me the following output:
PLAY [localhost] ***************************************************************
TASK [Gathering Facts] *********************************************************
ok: [localhost]
TASK [command] *****************************************************************
changed: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => {
"result.stdout": {
"containerInstanceArns": [
"arn:aws:ecs:us-east-1:arn0",
"arn:aws:ecs:us-east-1:arn1"
]
}
}
TASK [set_fact] ****************************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => (item=None) => {
"msg": "arn:aws:ecs:us-east-1:arn0"
}
ok: [localhost] => (item=None) => {
"msg": "arn:aws:ecs:us-east-1:arn1"
}
PLAY RECAP *********************************************************************
localhost : ok=5 changed=1 unreachable=0 failed=0
You can iterate over like this:
- debug:
msg: "{{ item[1] }}"
with_subelements:
- "{{ jenkins_ecs_containers }}"
- containerInstanceArns
Go through this link, it will make it clearer.

ansible find first undefined value in list

I have a list in ansible and I want to find the first "unused" item in that list.
Example list:
item001
item002
item004
item005
item100
item101
The prefix numbers in the items could be up to 999
In the example above the result should be item003.
Here is an example playbook of what I want
---
- name: Test
connection: local
hosts: localhost
vars:
my_list: []
list1:
- item107
- item002
- item004
- item001
- item007
- item101
- item604
tasks:
- name: Initialize a dummy list
set_fact:
my_list: "{{ my_list|sort }} + [ '{{ item }}' ]"
with_sequence: start=1 end=19 format=item%.3d
- name: print first unused value in my_list
debug:
msg: "{{ (my_list | difference(list1))[0] }}"
output:
PLAY [Test] *******************************************************************************************************************************************************************************************************
TASK [Gathering Facts] ********************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Initialize a dummy list] ************************************************************************************************************************************************************************************
ok: [localhost] => (item=item001)
ok: [localhost] => (item=item002)
ok: [localhost] => (item=item003)
ok: [localhost] => (item=item004)
ok: [localhost] => (item=item005)
ok: [localhost] => (item=item006)
ok: [localhost] => (item=item007)
ok: [localhost] => (item=item008)
ok: [localhost] => (item=item009)
ok: [localhost] => (item=item010)
ok: [localhost] => (item=item011)
ok: [localhost] => (item=item012)
ok: [localhost] => (item=item013)
ok: [localhost] => (item=item014)
ok: [localhost] => (item=item015)
ok: [localhost] => (item=item016)
ok: [localhost] => (item=item017)
ok: [localhost] => (item=item018)
ok: [localhost] => (item=item019)
TASK [print first unused value in my_list] ************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": "item003"
}
PLAY RECAP ********************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0
But if I change with_sequence: start=1 end=19 to with_sequence: start=1 end=999 the tasks takes very long time and also prints 999 lines in the output I really would love to not see.
This actually worked better, didn't print any large output and was much quicker:
- name: Initialize a dummy list
set_fact:
my_list: "{{ lookup('sequence', 'start=1 count=999 format=item%.3d', wantlist=True) }}"

Resources