Ansible compare two list variables - ansible

I have to check if a list of mount points are available on the system.
So, I defined a variable with the list of mount points then extracted the available mount points from Ansible facts.
---
- hosts: all
vars:
required_mounts:
- /prom/data
- /prom/logs
tasks:
- name: debug mountpoint
set_fact:
mount_points: "{{ ansible_mounts|json_query('[].mount') }}"
- name: check fs
fail:
msg: 'mount point not found'
when: required_mounts not in mount_points
I am stuck here, I don't know how to compare the variable required_mounts with existing mount points.
If any item in required_mounts is not in the existing mount points the task should fail.
The task check fs always fail, even if the mount points are present.
Do I have to loop one by one? And compare item by item? If so, how can I achieve this?

You can use the set theory for this, since what you are looking for is simply the difference between the required_mounts and the ansible_mounts.
Also, there is no need for a JMESPath query here, this simple requirement can be achieved with a simple map.
So, this can be achieved with the task alone:
- fail:
msg: "Missing mounts: `{{ missing_mounts | join(', ') }}`"
when: missing_mounts | length > 0
vars:
missing_mounts: >-
{{
required_mounts
| difference(
ansible_mounts | map(attribute='mount')
)
}}
Given the playbook:
- hosts: localhost
gather_facts: yes
vars:
required_mounts:
- /etc/hostname
- /etc/hosts
- /tmp/not_an_actual_mount
- /tmp/not_a_mount_either
tasks:
- fail:
msg: "Missing mounts: `{{ missing_mounts | join(', ') }}`"
when: missing_mounts | length > 0
vars:
missing_mounts: >-
{{
required_mounts
| difference(
ansible_mounts | map(attribute='mount')
)
}}
This yields:
TASK [Gathering Facts] *******************************************************
ok: [localhost]
TASK [fail] ******************************************************************
fatal: [localhost]: FAILED! => changed=false
msg: 'Missing mounts: `/tmp/not_an_actual_mount, /tmp/not_a_mount_either`'

Related

List name server from resolv.conf with hostname in one line per host

I need to get the DNS server(s) from my network, I tried using:
- hosts: localhost
gather_facts: no
tasks:
- name: check resolv.conf exists
stat:
path: /etc/resolv.conf
register: resolv_conf
- name: check nameservers list in resolv.conf
debug:
msg: "{{ contents }}"
vars:
contents: "{{ lookup('file', '/etc/resolv.conf') | regex_findall('\\s*nameserver\\s*(.*)') }}"
when: resolv_conf.stat.exists == True
But this does not quite gives the result I need.
Will it be possible to write a playbook in such a way that the result looks like the below?
hostname;dns1;dns2;dnsN
The declaration below gives the list of nameservers
nameservers: "{{ lookup('file', '/etc/resolv.conf').splitlines()|
select('match', '^nameserver.*$')|
map('split', ' ')|
map('last')|list }}"
You can join the hostname and the items on the list
msg: "{{ inventory_hostname }};{{ nameservers|join(';') }}"
Notes
Example of a complete playbook for testing
- hosts: localhost
vars:
nameservers: "{{ lookup('file', '/etc/resolv.conf').splitlines()|
select('match', '^nameserver.*$')|
map('split', ' ')|
map('last')|list }}"
tasks:
- debug:
var: nameservers
- debug:
msg: |
{{ inventory_hostname }};{{ nameservers|join(';') }}
The simplified declaration below works fine if there is no nameserver.* in the comments
nameservers: "{{ lookup('file', '/etc/resolv.conf')|
regex_findall('\\s*nameserver\\s*(.*)') }}"
Unfortunately, the Linux default file /etc/resolv.conf contains the comment:
| # run "systemd-resolve --status" to see details about the actual nameservers.
This regex will match nameservers.
nameservers:
- s.
You can solve this problem by putting at least one space behind the keyword nameserver.
regex_findall('\\s*nameserver\\s+(.*)') }}"
However, this won't help if there is the keyword nameserver in the comment.
Q: "No filter named 'split'"
A: There is no filter split in Ansible less than 2.11. Use regex_replace instead
nameservers: "{{ lookup('file', '/etc/resolv.conf').splitlines()|
select('match', '^nameserver.*$')|
map('regex_replace', '^(.*) (.*)$', '\\2')|list }}"
Since your regex_findall already creates you a list with all DNS servers, you just need to add the hostname to that list and join the whole list with a semicolon.
- name: check nameservers list in resolv.conf
debug:
msg: >-
{{
(
[ ansible_hostname ] +
lookup('file', '/etc/resolv.conf', errors='ignore')
| regex_findall('\s*nameserver\s*(.*)')
) | join(';')
}}
Which will result in something like (b176263884e6 being the actual hostname of a container):
TASK [check nameservers list in resolv.conf] *****************************
ok: [localhost] =>
msg: b176263884e6;1.1.1.1;4.4.4.4;8.8.8.8
Note that you don't even need the stat task, as you can ignore errors of the lookup with errors='ignore'.
This will, then, give you only the hostname, along with a warning:
TASK [check nameservers list in resolv.conf] *****************************
[WARNING]: Unable to find '/etc/resolv.conf' in expected paths
(use -vvvvv to see paths)
ok: [localhost] =>
msg: b176263884e6

Ansible | delete files from a directory if the filename doesn't contains any of the strings from a list

I'm creating vm-s with libvirt, and I would like to do a housekeeping, if I delete a host (in this example a VM) from my inventory, at the next run of the playbook, it should delete that VM's qcow2 disk from the disk pool.
I don't really get, how could I create a nested loop that iterates through the file list of that specific directory and the list of vms in my inventory, checks if the name of the vm is part of any file in the filelist, and deletes the files whose have no connection into the inventory.
Here is an example from the many things I already tried:
- name: "Housekeeping: list qcow2 disks in libvirt-pool"
find:
paths: /mnt/hdd/libvirt-pool
depth: 1
patterns:
- "*.qcow2"
register: qcow_disks
- name: debug
debug:
msg: "{{item[0]}}"
with_nested:
- "{{ qcow_disks.files | map(attribute='path') | list }}"
- "{{ groups.vm }}"
when: item[1] in item[0]
register: valid_disks
- name: debug1
debug:
msg: "invalid disks: {{ valid_disks.results | difference(all_disk) }}"
variable:
all_disk: "{{ qcow_disks.files | map(attribute='path') | list }}"
Hope you can help me out!
Thanks in advance!
I assume you have in groups.vm a list of names of VMs, without the extension .qcow2.
So the list groups.vm could looks like e.g:
['vm1', 'vm5', 'test']
The find command returns files like:
[
"/mnt/hdd/libvirt-pool/bob.qcow2",
"/mnt/hdd/libvirt-pool/daniel.qcow2",
"/mnt/hdd/libvirt-pool/test.qcow2",
"/mnt/hdd/libvirt-pool/vm1.qcow2",
"/mnt/hdd/libvirt-pool/vm5.qcow2"
]
With the following command you can reduce this list to the name without extension, then you can easily compare the lists.
{{ qcow_disks.files | map(attribute='path') | map('basename') | map('splitext') | map('first') }}
basename returns the filename, without preceding path
splitext splits the filename into a list: [name, extension]
first takes the first element from the list, i.e. the name
More on basename and splitext in the Ansible docs.
{{ found_disks | reject('in', current_vms) }}
With the reject filter you can then discard the current elements, so that you contain a list with all old VMs.
The following tasks:
- name: "Housekeeping: list qcow2 disks in libvirt-pool"
find:
paths: /mnt/hdd/libvirt-pool
depth: 1
patterns:
- "*.qcow2"
register: qcow_disks
- debug:
msg: "{{ qcow_disks.files | map(attribute='path') }}"
- debug:
msg: "{{ old_disks }}"
vars:
current_vms: ['vm1', 'vm5', 'test']
found_disks: "{{ qcow_disks.files | map(attribute='path') | map('basename') | map('splitext') | map('first') }}"
old_disks: "{{ found_disks | reject('in', current_vms) }}"
Note: current_vms corresponds to the list you have via groups.vm.
return this result:
TASK [Housekeeping: list qcow2 disks in libvirt-pool] ************************
ok: [localhost]
TASK [debug] *****************************************************************
ok: [localhost] => {
"msg": [
"/mnt/hdd/libvirt-pool/bob.qcow2",
"/mnt/hdd/libvirt-pool/daniel.qcow2",
"/mnt/hdd/libvirt-pool/test.qcow2",
"/mnt/hdd/libvirt-pool/vm1.qcow2",
"/mnt/hdd/libvirt-pool/vm5.qcow2"
]
}
TASK [debug] *****************************************************************
ok: [localhost] => {
"msg": [
"bob",
"daniel"
]
}
I hope this helps you.

Line too long for setting up variable

This is my Ansible task:
- name: get the custom job id
ansible.builtin.set_fact:
custom_job_id: >
"{{ train_custom_image_unmanaged_response.stderr_lines |select('search', 'describe') |list |regex_search('.*/customJobs/(\\d+)', '\\1') |first }}"
when: "(gcs_model_list.stdout is not defined) or ('saved_model.pb' not in gcs_model_list.stdout)"
I am getting "line too long" as Ansible lint error for custom_job_id line.
Any idea how can I break it down in smaller parts?
You can do it using YAML multi lines syntaxes, as you started doing it.
With this syntax, the indentation is what is defining a block, so, as long as you are indented inward of the fact name custom_job_id, all the following code is considered as being the expression that is going to be assigned to that fact.
For example:
- name: get the custom job id
ansible.builtin.set_fact:
custom_job_id: >-
{{
train_custom_image_unmanaged_response.stderr_lines
| select('search', 'describe')
| list
| regex_search('.*/customJobs/(\d+)', '\1')
| first
}}
when: >-
gcs_model_list.stdout is not defined
or 'saved_model.pb' not in gcs_model_list.stdout
Here is a playbook complying with the Ansible linting demonstrating this:
- hosts: localhost
gather_facts: true
tasks:
- name: Get the custom job id
ansible.builtin.set_fact:
custom_job_id: >-
{{
train_custom_image_unmanaged_response.stderr_lines
| select('search', 'describe')
| list
| regex_search('.*/customJobs/(\d+)', '\1')
| first
}}
when: >-
gcs_model_list.stdout is not defined
or 'saved_model.pb' not in gcs_model_list.stdout
vars:
train_custom_image_unmanaged_response:
stderr_lines:
- foo
- bar
- describe - /customJobs/123
- baz
gcs_model_list:
- name: Display `custom_job_id`
ansible.builtin.debug:
var: custom_job_id
Which yields:
PLAY [localhost] **********************************************************
TASK [Get the custom job id] **********************************************
ok: [localhost]
TASK [Display `custom_job_id`] ********************************************
ok: [localhost] =>
custom_job_id: '123'

In Ansible, how to query hostvars to get a specific value of a key from a list item based on the value of a different key?

EDIT-UPDATE:
I found a way to achieve what was trying to do, using the index_of plugin. The following code outputs what I need.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
mac_address: "{{ hostvars[inventory_hostname]['interfaces'][int_idx|int]['mac_address'] }}"
vars:
int_name: 'PCI1.1'
int_idx: "{{ lookup('ansible.utils.index_of', hostvars[inventory_hostname]['interfaces'], 'eq', int_name, 'name') }}"
- debug:
var: mac_address
Output:
PLAY [CASPOSR1BDAT003] ***********************************************************************************************************************************************************************************************
TASK [ansible.builtin.set_fact] **************************************************************************************************************************************************************************************
ok: [CASPOSR1BDAT003]
TASK [debug] *********************************************************************************************************************************************************************************************************
ok: [CASPOSR1BDAT003] =>
mac_address: 20:67:7C:00:36:A0
What I am trying to do:
Use the Netbox dynamic inventory plugin (this works, brings back all the info I need)
Query hostvars for a particular host, and get the value of the MAC address for a particular interface called PCI1.1
What I have tried:
Converting the hostvars to JSON and using json_query: this hasn't worked, and having looked at some issues on GitHub, hostvars isn't a "normal" dictionary. I've logged a couple of issues anyway (https://github.com/ansible/ansible/issues/76289 and https://github.com/ansible-collections/community.general/issues/3706).
Use a sequence loop and conditional "when" to get the value - this sort of works when using the debug module, but still not just returning the value
What works:
I have tried the following, which outputs the mac_address variable as expected. The length of the list is found, and then the conditional matches the name. I do get an warning about using jinja2 templating delimiters but that's not the target of this question.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- debug:
var: hostvars[inventory_hostname]['interfaces'][{{ item }}]['mac_address']
with_sequence: start=0 end="{{ end_at }}"
vars:
- end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
The result is:
TASK [debug] *************************************************************************************************************************************
[WARNING]: conditional statements should not include jinja2 templating delimiters such as {{ }} or {% %}. Found:
hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
skipping: [CASPOSR1BDAT003] => (item=0)
skipping: [CASPOSR1BDAT003] => (item=1)
skipping: [CASPOSR1BDAT003] => (item=2)
skipping: [CASPOSR1BDAT003] => (item=3)
skipping: [CASPOSR1BDAT003] => (item=4)
ok: [CASPOSR1BDAT003] => (item=5) =>
ansible_loop_var: item
hostvars[inventory_hostname]['interfaces'][5]['mac_address']: 20:67:7C:00:36:A0
item: '5'
skipping: [CASPOSR1BDAT003] => (item=6)
skipping: [CASPOSR1BDAT003] => (item=7)
skipping: [CASPOSR1BDAT003] => (item=8)
skipping: [CASPOSR1BDAT003] => (item=9)
I'm trying to use set_fact to store this mac_address variable as I need to use it in a couple of different ways. However, I am unable to use set_fact on this (or any other hostvars data, it seems). For example, the following:
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item]['mac_address'] }}"
with_sequence: start=0 end="{{ end_at }}"
vars:
- end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
- debug:
var: interfaces
results in:
fatal: [CASPOSR1BDAT003]: FAILED! =>
msg: |-
The task includes an option with an undefined variable. The error was: 'list object' has no attribute '5'
The error appears to be in '/Users/kivlint/Documents/GitHub/vmware-automation/ansible/prepare-pxe.yml': line 19, column 7, but may
be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
# when: hostvars[inventory_hostname]['interfaces'][{{ item }}]['name'] == "PCI1.1"
- ansible.builtin.set_fact:
^ here
If I hard-code the number 5 in, it works fine:
TASK [ansible.builtin.set_fact] ******************************************************************************************************************
ok: [CASPOSR1BDAT003]
TASK [debug] *************************************************************************************************************************************
ok: [CASPOSR1BDAT003] =>
interfaces: 20:67:7C:00:36:A0
If I use '5' as a var for the task, it also works.
---
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
tasks:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][int_index]['mac_address'] }}"
vars:
- int_index: 5
So I'm wondering, is this a "bug/feature" in how set_fact does or doesn't work with loops (meaning, the same loop worked fine with debug? Or do I need to re-think the approach and consider trying to use set_fact to set a variable with the index of the list (e.g. 5 in the above example)? Or something else?
There's a lot going on in your code, and achieving the result you want is simpler than you've made it.
Firstly, don't use hostvars[inventory_hostname]; plain variables are the ones belonging to the current host, and going through hostvars introduces some exciting opportunities for things to go wrong. hostvars is for accessing variables belonging to other hosts.
Secondly, using Jinja's built-in filtering capabilities avoids the need to worry about the index of the item that you want.
- hosts: CASPOSR1BDAT003
connection: local
gather_facts: no
become: false
vars:
int_name: PCI1.1
mac_address: "{{ interfaces | selectattr('name', 'eq', int_name) | map(attribute='mac_address') | first }}"
tasks:
- debug:
var: mac_address
there is a confusion between the [5] (6th item of a list) and ['5'] (a key named "5") ,
you see in your error: The error was: 'list object' has no attribute '5'.
with the module debug you have not error because [{{item}}] is replaced by [5] and not by ['5']. Its not the same thing with set_fact.
its the reason you have to use filter int to clarify the situation.
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item|int]['mac_address'] }}"
with_sequence: start=0 end="{{ end_at }}"
vars:
end_at: "{{ (hostvars[inventory_hostname]['interfaces'] | length) - 1 }}"
when: hostvars[inventory_hostname]['interfaces'][item|int]['name'] == "PCI1.1"
so i suggest you to use loop instead with_sequence:
- ansible.builtin.set_fact:
interfaces: "{{ hostvars[inventory_hostname]['interfaces'][item]['mac_address'] }}"
loop: "{{ range(0, end_at|int, 1)|list }}"
vars:
end_at: "{{ hostvars[inventory_hostname]['interfaces'] | length }}"
when: hostvars[inventory_hostname]['interfaces'][item]['name'] == "PCI1.1"
set_fact works with loops, but not in a way you expect.
This example constructs list with loop from lists of dicts:
- set_fact:
foo: '{{ foo|d([]) + [item.value] }}'
loop:
- value: 1
- value: 2
Basically, each execution of set_fact creates a fact. You may refer to the same fact in jinja expression for set_fact, but you can't expect it to automatically build lists or something like that.

How to include files one by one using dictionary key in ansible?

I am trying to invoke test cases one by one from ansible. Each test case is written in a yml file.
---
- hosts: localhost
tasks:
- set_fact:
k8s_tests:
san-1.yml: "Scale replica to 1"
san-2.yml: "Scale replica to 2"
- name: display local tests
debug: var=k8s_tests
- include: "{{ k8s_test_item }}"
vars:
local_test: "{{ k8s_test_item }}"
with_items: "{{ k8s_tests }}"
loop_control:
loop_var: k8s_test_item
When i execute it , the order of the file is wrong.
The include should be running in the order of san-1.yml first then san-2.yml
but it was opposite.
TASK [include] *********************************************************************************
included: /root/oc310/jenkinsrun/containers/K8S/san-2.yml for localhost
included: /root/oc310/jenkinsrun/containers/K8S/san-1.yml for localhost
So i added dictsort to sort the dictionary.
- include: "{{ k8s_test_item }}"
vars:
local_test: "{{ k8s_test_item }}"
with_items: "{{ k8s_tests | dictsort }}"
loop_control:
loop_var: k8s_test_item
but it tries to include both key and value and fails.
TASK [include] *********************************************************************************
included: /root/oc310/jenkinsrun/containers/K8S/san-1.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 1' on the Ansible Controller."}
included: /root/oc310/jenkinsrun/containers/K8S/san-2.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 2' on the Ansible Controller."}
The values 'Scale replica to 1' and 'Scale replica to 2' are not files and they should not be included. I tried to sort by key but still it tries to include based on value as well and fails.
- include: "{{ k8s_test_item }}"
vars:
local_test: "{{ k8s_test_item }}"
with_items: "{{ k8s_tests | dictsort(false,'key') }}"
loop_control:
loop_var: k8s_test_item
below is the same output
TASK [include] *********************************************************************************
included: /root/oc310/jenkinsrun/containers/K8S/san-1.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 1' on the Ansible Controller."}
included: /root/oc310/jenkinsrun/containers/K8S/san-2.yml for localhost
fatal: [localhost]: FAILED! => {"reason": "Could not find or access '/root/oc310/jenkinsrun/containers/K8S/Scale replica to 2' on the Ansible Controller."}
I need to include the file in order one by one. How to achieve this?
Here is a solution to make sure your file names are always sorted correctly. You need to:
Transform your dictionary to a list with the dict2items filter
Extract only the key attribute for each element in the list with the map filter
sort the resulting list
Below is an implementation with a debug to show each described steps.
Notes:
I had to reverse the order in var declaration to get the same initial result as yours and have a successful demo
loop is the new keyword for defining loops and is equivalent here to with_list. See loop documentation
---
- name: Looping demo
hosts: localhost
gather_facts: false
vars:
k8s_tests:
san-2.yml: "Scale replica to 2"
san-1.yml: "Scale replica to 1"
tasks:
- name: Show initial var
debug:
var: k8s_tests
- name: Show transforming dict to list
debug:
msg: "{{ k8s_tests | dict2items }}"
- name: Show attribute extraction
debug:
msg: "{{ k8s_tests | dict2items | map(attribute='key') | list }}"
- name: Show final sorted result
debug:
msg: "{{ k8s_tests | dict2items | map(attribute='key') | sort }}"
- name: Actually looping over the data
debug:
var: item
loop: "{{ k8s_tests | dict2items | map(attribute='key') | sort }}"

Resources