How to include files one by one using dictionary key in ansible? - ansible

I am trying to invoke test cases one by one from ansible. Each test case is written in a yml file.
---
- hosts: localhost
tasks:
- set_fact:
k8s_tests:
san-1.yml: "Scale replica to 1"
san-2.yml: "Scale replica to 2"
- name: display local tests
debug: var=k8s_tests
- include: "{{ k8s_test_item }}"
vars:
local_test: "{{ k8s_test_item }}"
with_items: "{{ k8s_tests }}"
loop_control:
loop_var: k8s_test_item
When i execute it , the order of the file is wrong.
The include should be running in the order of san-1.yml first then san-2.yml
but it was opposite.
TASK [include] *********************************************************************************
included: /root/oc310/jenkinsrun/containers/K8S/san-2.yml for localhost
included: /root/oc310/jenkinsrun/containers/K8S/san-1.yml for localhost
So i added dictsort to sort the dictionary.
- include: "{{ k8s_test_item }}"
vars:
local_test: "{{ k8s_test_item }}"
with_items: "{{ k8s_tests | dictsort }}"
loop_control:
loop_var: k8s_test_item
but it tries to include both key and value and fails.
TASK [include] *********************************************************************************
included: /root/oc310/jenkinsrun/containers/K8S/san-1.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 1' on the Ansible Controller."}
included: /root/oc310/jenkinsrun/containers/K8S/san-2.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 2' on the Ansible Controller."}
The values 'Scale replica to 1' and 'Scale replica to 2' are not files and they should not be included. I tried to sort by key but still it tries to include based on value as well and fails.
- include: "{{ k8s_test_item }}"
vars:
local_test: "{{ k8s_test_item }}"
with_items: "{{ k8s_tests | dictsort(false,'key') }}"
loop_control:
loop_var: k8s_test_item
below is the same output
TASK [include] *********************************************************************************
included: /root/oc310/jenkinsrun/containers/K8S/san-1.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 1' on the Ansible Controller."}
included: /root/oc310/jenkinsrun/containers/K8S/san-2.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 2' on the Ansible Controller."}
I need to include the file in order one by one. How to achieve this?

Here is a solution to make sure your file names are always sorted correctly. You need to:
Transform your dictionary to a list with the dict2items filter
Extract only the key attribute for each element in the list with the map filter
sort the resulting list
Below is an implementation with a debug to show each described steps.
Notes:
I had to reverse the order in var declaration to get the same initial result as yours and have a successful demo
loop is the new keyword for defining loops and is equivalent here to with_list. See loop documentation
---
- name: Looping demo
hosts: localhost
gather_facts: false
vars:
k8s_tests:
san-2.yml: "Scale replica to 2"
san-1.yml: "Scale replica to 1"
tasks:
- name: Show initial var
debug:
var: k8s_tests
- name: Show transforming dict to list
debug:
msg: "{{ k8s_tests | dict2items }}"
- name: Show attribute extraction
debug:
msg: "{{ k8s_tests | dict2items | map(attribute='key') | list }}"
- name: Show final sorted result
debug:
msg: "{{ k8s_tests | dict2items | map(attribute='key') | sort }}"
- name: Actually looping over the data
debug:
var: item
loop: "{{ k8s_tests | dict2items | map(attribute='key') | sort }}"

Related

List name server from resolv.conf with hostname in one line per host

I need to get the DNS server(s) from my network, I tried using:
- hosts: localhost
gather_facts: no
tasks:
- name: check resolv.conf exists
stat:
path: /etc/resolv.conf
register: resolv_conf
- name: check nameservers list in resolv.conf
debug:
msg: "{{ contents }}"
vars:
contents: "{{ lookup('file', '/etc/resolv.conf') | regex_findall('\\s*nameserver\\s*(.*)') }}"
when: resolv_conf.stat.exists == True
But this does not quite gives the result I need.
Will it be possible to write a playbook in such a way that the result looks like the below?
hostname;dns1;dns2;dnsN
The declaration below gives the list of nameservers
nameservers: "{{ lookup('file', '/etc/resolv.conf').splitlines()|
select('match', '^nameserver.*$')|
map('split', ' ')|
map('last')|list }}"
You can join the hostname and the items on the list
msg: "{{ inventory_hostname }};{{ nameservers|join(';') }}"
Notes
Example of a complete playbook for testing
- hosts: localhost
vars:
nameservers: "{{ lookup('file', '/etc/resolv.conf').splitlines()|
select('match', '^nameserver.*$')|
map('split', ' ')|
map('last')|list }}"
tasks:
- debug:
var: nameservers
- debug:
msg: |
{{ inventory_hostname }};{{ nameservers|join(';') }}
The simplified declaration below works fine if there is no nameserver.* in the comments
nameservers: "{{ lookup('file', '/etc/resolv.conf')|
regex_findall('\\s*nameserver\\s*(.*)') }}"
Unfortunately, the Linux default file /etc/resolv.conf contains the comment:
| # run "systemd-resolve --status" to see details about the actual nameservers.
This regex will match nameservers.
nameservers:
- s.
You can solve this problem by putting at least one space behind the keyword nameserver.
regex_findall('\\s*nameserver\\s+(.*)') }}"
However, this won't help if there is the keyword nameserver in the comment.
Q: "No filter named 'split'"
A: There is no filter split in Ansible less than 2.11. Use regex_replace instead
nameservers: "{{ lookup('file', '/etc/resolv.conf').splitlines()|
select('match', '^nameserver.*$')|
map('regex_replace', '^(.*) (.*)$', '\\2')|list }}"
Since your regex_findall already creates you a list with all DNS servers, you just need to add the hostname to that list and join the whole list with a semicolon.
- name: check nameservers list in resolv.conf
debug:
msg: >-
{{
(
[ ansible_hostname ] +
lookup('file', '/etc/resolv.conf', errors='ignore')
| regex_findall('\s*nameserver\s*(.*)')
) | join(';')
}}
Which will result in something like (b176263884e6 being the actual hostname of a container):
TASK [check nameservers list in resolv.conf] *****************************
ok: [localhost] =>
msg: b176263884e6;1.1.1.1;4.4.4.4;8.8.8.8
Note that you don't even need the stat task, as you can ignore errors of the lookup with errors='ignore'.
This will, then, give you only the hostname, along with a warning:
TASK [check nameservers list in resolv.conf] *****************************
[WARNING]: Unable to find '/etc/resolv.conf' in expected paths
(use -vvvvv to see paths)
ok: [localhost] =>
msg: b176263884e6

How to match/search a substring from a dict attribute that is a list

Here's the scenario:
a playbook that calls a role to create users in multiple servers, including a VM Scale Set (where ansible_hostnames can't be predicted) - inventory is already being dynamically generated and works fine and not the issue
a users dict variable will provide the user list as well as a series of attributes for each
one of these attributes is a server list named target_servers - this variable's attribute is the actual issue
target_servers is used by the playbook to decide if the user will be present/absent on that particular server - it complements ansible's inventory
target_servers might include only the starting name of a particular target host, a sub-string, like "vmss" as a "vmss*" wildcard, but also fixed hostnames server12345, server12346, etc.
so, dynamic inventory tells ansible which servers to connect to, but the variable tells it whether the user should be created or removed from that particular servers (i.e. servers have different users)
Objective(s):
Have a conditional that checks if a target_server list element content matches the ansible_hostname (i.e. if the substring found in the target_servers list (from the users dict) matches, then we provision the user; additionally, off course, if the list provides the entire hostname, it should match and the users also be provisioned)
Here's the code:
---
- hosts: all
become: yes
vars:
users:
user1:
is_sudo: no
is_chrooted: yes
auth_method: hvault
sa_homedir: firstname1lastname1
state: present
target_servers:
- vmss
- ubuntu
user2:
is_sudo: no
is_chrooted: yes
auth_method: hvault
sa_homedir: firstname2lastname2
state: present
target_servers:
- vmss
- ubuntu18
tasks:
- debug:
msg: "{{ ansible_hostname }}"
- debug:
msg: "{{ item.value.target_servers }}"
loop: "{{ lookup('dict', users|default({})) }}"
# This is just to exemplify what I'm trying to achieve as it is not supposed to work
- debug:
msg: "ansible_hostname is in target_servers of {{ item.key }}"
loop: "{{ lookup('dict', users|default({})) }}"
when: ansible_hostname is match(item.value.target_servers)
Here's the output showing that the match string test cannot be applied to a list (as expected):
TASK [debug] ************************************************************************************************************************************************
ok: [ubuntu18] =>
msg: ubuntu18
TASK [debug] ************************************************************************************************************************************************
ok: [ubuntu18] => (item={'key': 'user1', 'value': {'is_sudo': False, 'is_chrooted': True, 'auth_method': 'hvault', 'sa_homedir': 'firstname1lastname1', 'state': 'present', 'target_servers': ['vmss', 'ubuntu']}}) =>
msg:
- vmss
- ubuntu
ok: [ubuntu18] => (item={'key': 'user2', 'value': {'is_sudo': False, 'is_chrooted': True, 'auth_method': 'hvault', 'sa_homedir': 'firstname2lastname2', 'state': 'present', 'target_servers': ['vmss', 'ubuntu18']}}) =>
msg:
- vmss
- ubuntu18
TASK [debug] ************************************************************************************************************************************************
fatal: [ubuntu18]: FAILED! =>
msg: |-
The conditional check 'ansible_hostname is match(item.value.target_servers)' failed. The error was: Unexpected templating type error occurred on ({% if ansible_hostname is match(item.value.target_servers) %} True {% else %} False {% endif %}): unhashable type: 'list'
The error appears to be in 'test-play-users-core.yml': line 32, column 5, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- debug:
^ here
Already tried researching about selectattr, json_query and subelements but I currently lack the understanding on how to make them work to match a substring inside a dict attribute that is a list.
In the example above, by changing from is match() to in, exact hostnames work fine, but that is not the goal. I need to match both exact hostnames and sub-strings for these hostnames.
Any help on how to accomplish this or suggestions about alternate methods will be greatly appreciated.
The example here might work if I could find a way to run it against a list (target_servers) after having already looped through the entire dictionary (are nested loops possible?): https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html#testing-strings
I guess I've just found what I needed: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/subelements_lookup.html
Will try and provide an update soon.
Update: yes, subelements work! Here's the code needed:
- name: test 1
debug:
msg: "{{ item.1 }} matches {{ ansible_hostname }}"
with_subelements:
- "{{ users }}"
- target_servers
when: >
ansible_hostname is match(item.1)
You can use the select filter to apply the in test to all the elements of your users' target_servers list.
This would be your debug task:
- debug:
msg: "hostname is in target_servers of {{ item.key }}"
loop: "{{ users | dict2items }}"
loop_control:
label: "{{ item.key }}"
when: >-
item.value.target_servers
| select('in', inventory_hostname)
| length > 0
Given the playbook:
- hosts: all
gather_facts: false
vars:
_hostname: ubuntu18
users:
user1:
target_servers:
- vmss
- ubuntu
user2:
target_servers:
- vmss
- ubuntu18
tasks:
- debug:
msg: "hostname is in target_servers of {{ item.key }}"
loop: "{{ users | dict2items }}"
loop_control:
label: "{{ item.key }}"
when: >-
item.value.target_servers
| select('in', inventory_hostname)
| length > 0
This yields:
ok: [ubuntu18] => (item=user1) =>
msg: hostname is in target_servers of user1
ok: [ubuntu18] => (item=user2) =>
msg: hostname is in target_servers of user2
Doing it with subelements instead:
- hosts: all
gather_facts: false
vars:
_hostname: ubuntu18
users:
user1:
target_servers:
- vmss
- ubuntu
user2:
target_servers:
- vmss
- ubuntu18
tasks:
- debug:
msg: "hostname is in target_servers of {{ item.0.key }}"
loop: "{{ users | dict2items | subelements('value.target_servers') }}"
loop_control:
label: "{{ item.0.key }}"
when: item.1 in inventory_hostname
Will yield:
skipping: [ubuntu18] => (item=user1)
ok: [ubuntu18] => (item=user1) =>
msg: hostname is in target_servers of user1
skipping: [ubuntu18] => (item=user2)
ok: [ubuntu18] => (item=user2) =>
msg: hostname is in target_servers of user2

In Ansible, how to query hostvars to get a specific value of a key from a list item based on the value of a different key?

EDIT-UPDATE:
I found a way to achieve what was trying to do, using the index_of plugin. The following code outputs what I need.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
mac_address: "{{ hostvars[inventory_hostname]['interfaces'][int_idx|int]['mac_address'] }}"
vars:
int_name: 'PCI1.1'
int_idx: "{{ lookup('ansible.utils.index_of', hostvars[inventory_hostname]['interfaces'], 'eq', int_name, 'name') }}"
- debug:
var: mac_address
Output:
PLAY [CASPOSR1BDAT003] ***********************************************************************************************************************************************************************************************
TASK [ansible.builtin.set_fact] **************************************************************************************************************************************************************************************
ok: [CASPOSR1BDAT003]
TASK [debug] *********************************************************************************************************************************************************************************************************
ok: [CASPOSR1BDAT003] =>
mac_address: 20:67:7C:00:36:A0
What I am trying to do:
Use the Netbox dynamic inventory plugin (this works, brings back all the info I need)
Query hostvars for a particular host, and get the value of the MAC address for a particular interface called PCI1.1
What I have tried:
Converting the hostvars to JSON and using json_query: this hasn't worked, and having looked at some issues on GitHub, hostvars isn't a "normal" dictionary. I've logged a couple of issues anyway (https://github.com/ansible/ansible/issues/76289 and https://github.com/ansible-collections/community.general/issues/3706).
Use a sequence loop and conditional "when" to get the value - this sort of works when using the debug module, but still not just returning the value
What works:
I have tried the following, which outputs the mac_address variable as expected. The length of the list is found, and then the conditional matches the name. I do get an warning about using jinja2 templating delimiters but that's not the target of this question.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- debug:
var: hostvars[inventory_hostname]['interfaces'][{{ item }}]['mac_address']
with_sequence: start=0 end="{{ end_at }}"
vars:
- end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
The result is:
TASK [debug] *************************************************************************************************************************************
[WARNING]: conditional statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found:
hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
skipping: [CASPOSR1BDAT003] => (item=0)
skipping: [CASPOSR1BDAT003] => (item=1)
skipping: [CASPOSR1BDAT003] => (item=2)
skipping: [CASPOSR1BDAT003] => (item=3)
skipping: [CASPOSR1BDAT003] => (item=4)
ok: [CASPOSR1BDAT003] => (item=5) =>
ansible_loop_var: item
hostvars[inventory_hostname]['interfaces'][5]['mac_address']: 20:67:7C:00:36:A0
item: '5'
skipping: [CASPOSR1BDAT003] => (item=6)
skipping: [CASPOSR1BDAT003] => (item=7)
skipping: [CASPOSR1BDAT003] => (item=8)
skipping: [CASPOSR1BDAT003] => (item=9)
I'm trying to use set_fact to store this mac_address variable as I need to use it in a couple of different ways. However, I am unable to use set_fact on this (or any other hostvars data, it seems). For example, the following:
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item]['mac_address'] }}"
with_sequence: start=0 end="{{ end_at }}"
vars:
- end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
- debug:
var: interfaces
results in:
fatal: [CASPOSR1BDAT003]: FAILED! =>
msg: |-
The task includes an option with an undefined variable. The error was: 'list object' has no attribute '5'
The error appears to be in '/Users/kivlint/Documents/GitHub/vmware-automation/ansible/prepare-pxe.yml': line 19, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
# when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
- ansible.builtin.set_fact:
^ here
If I hard-code the number 5 in, it works fine:
TASK [ansible.builtin.set_fact] ******************************************************************************************************************
ok: [CASPOSR1BDAT003]
TASK [debug] *************************************************************************************************************************************
ok: [CASPOSR1BDAT003] =>
interfaces: 20:67:7C:00:36:A0
If I use '5' as a var for the task, it also works.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][int_index]['mac_address'] }}"
vars:
- int_index: 5
So I'm wondering, is this a "bug/feature" in how set_fact does or doesn't work with loops (meaning, the same loop worked fine with debug? Or do I need to re-think the approach and consider trying to use set_fact to set a variable with the index of the list (e.g. 5 in the above example)? Or something else?
There's a lot going on in your code, and achieving the result you want is simpler than you've made it.
Firstly, don't use hostvars[inventory_hostname]; plain variables are the ones belonging to the current host, and going through hostvars introduces some exciting opportunities for things to go wrong. hostvars is for accessing variables belonging to other hosts.
Secondly, using Jinja's built-in filtering capabilities avoids the need to worry about the index of the item that you want.
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
vars:
int_name: PCI1.1
mac_address: "{{ interfaces | selectattr('name', 'eq', int_name) | map(attribute='mac_address') | first }}"
tasks:
- debug:
var: mac_address
there is a confusion between the [5] (6th item of a list) and ['5'] (a key named "5") ,
you see in your error: The error was: 'list object' has no attribute '5'.
with the module debug you have not error because [{{item}}] is replaced by [5] and not by ['5']. Its not the same thing with set_fact.
its the reason you have to use filter int to clarify the situation.
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item|int]['mac_address'] }}"
with_sequence: start=0 end="{{ end_at }}"
vars:
end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][item|int]['name'] == "PCI1.1"
so i suggest you to use loop instead with_sequence:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item]['mac_address'] }}"
loop: "{{ range(0, end_at|int, 1)|list }}"
vars:
end_at: "{{ hostvars[inventory_hostname]['interfaces'] | length }}"
when: hostvars[inventory_hostname]['interfaces'][item]['name'] == "PCI1.1"
set_fact works with loops, but not in a way you expect.
This example constructs list with loop from lists of dicts:
- set_fact:
foo: '{{ foo|d([]) + [item.value] }}'
loop:
- value: 1
- value: 2
Basically, each execution of set_fact creates a fact. You may refer to the same fact in jinja expression for set_fact, but you can't expect it to automatically build lists or something like that.

Conditional set_fact only with selectattr?

I am trying to use set_fact, but only when an item is defined. I'm having trouble figuring out how to do this.
Playbook:
---
- hosts: localhost
vars:
apps:
- name: app1
role: "app1-role"
- name: app2
role: "app2-role"
- name: app4
role: "app4-role"
tasks:
- name: "Find a matching app, and print the values"
set_fact:
app: "{{ apps | selectattr('name', 'match', app_name) | first }}"
The task fails when there is no match, for example: app_name=app3, with this message:
FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: No first item, sequence was empty.
I have tried a few different conditionals, but I'm not quite sure how to structure this.
when: (apps | selectattr('name', 'match', app_name)) is undefined
This condition always evaluates as False - skipping: [localhost] => {"changed": false, "skip_reason": "Conditional result was False"}.
I tried this :
when: "{{ apps | selectattr('name', 'match', app_name) | list }}"
It seems that without the list filter, selectattr returning a generator object does not seem to allow converting to a boolean for the test evaluation. I'm not 100% sure about this interpretation though.
Note that having is defined or not would not change anything, it would be totally equivalent.
Try
- set_fact:
app: "{{ mylist|first }}"
vars:
mylist: "{{ apps|selectattr('name', 'match', app_name)|list }}"
when: mylist|length > 0
Next option is json_query instead of selectattr
- set_fact:
app: "{{ mylist|first }}"
vars:
query: "[?name=='{{ app_name }}']"
mylist: "{{ apps|json_query(query) }}"
when: mylist|length > 0

Variable with_items and hostvars

I need to pass a dynamic group name info with_items so that i can access a specific fact that is ran from another host. I cannot hard code the group name
I tried to set a generic variable that is passed as 'GroupName' a few different ways. Including
with_items: "{{ groups['{{GROUPNAME}}'] }}"
- name: Name of task
debug:
msg: "{{ hostvars[item]['ansible_check_mode'] }}"
with_items: "{{ groups['GROUPNAME'] }}"
fatal: [localhost]: FAILED! => {"msg": "'dict object' has no attribute '{{ GROUPNAME }}'"}
Get the list of the hosts in the group and loop them
vars:
my_group: GROUPNAME
tasks:
- set_fact:
my_hosts: "{{ groups|
dict2items|
selectattr('key', 'match', my_group)|
map(attribute='value')|
list|
flatten }}"
- debug:
msg: "{{ hostvars[item]['ansible_check_mode'] }}"
loop: "{{ my_hosts }}"
(not tested)

Resources