Ansible, iterate over multiple registers and make extended use of build in methods - ansible

To start off I have all my variables defined in YAML
app_dir: "/mnt/{{ item.name }}"
app_dir_ext: "/mnt/{{ item.0.name }}"
workstreams:
- name: tigers
service_workstream: tigers-svc
app_sub_dir:
- inbound
- inbound/incoming
- inbound/error
- inbound/error/processed
- name: lions
service_workstream: lions-svc
app_sub_dir:
- inbound
- inbound/incoming
- inbound/error
- inbound/error/processed
You may note app_dir: "/mnt/{{ item.name }}" and app_dir_ext: "/mnt/{{ item.0.name }}" looking odd, so I originally had my variables set as below in YAML but decided to use the above mainly due to less lines in YAML when I have a large amount of workstreams.
workstreams:
- name: tigers
service_workstream: tigers-svc
app_dir: /mnt/tigers
...
I then have Ansible code to check if the directories exists, if not create them and apply permissions (!note, have taken this approach due to a ssh timeout on operation when using the file: module on a number of very big NFS mounted shares).
- name: Check workstreams app_dir
stat:
path: "{{ app_dir }}"
register: app_dir_status
with_items:
- "{{ workstreams }}"
- name: Check workstreams app_sub_dir
stat:
path: "{{ app_dir_ext }}/{{ item.1 }}/"
register: app_sub_dir_status
with_subelements:
- "{{ workstreams }}"
- app_sub_dir
- name: create workstreams app_dir
file:
path: "/mnt/{{ item.0.name }}"
state: directory
owner: "ftp"
group: "ftp"
mode: '0770'
recurse: yes
with_nested:
- '{{ workstreams }}'
- app_dir_status.results
when:
- '{{ item.1.stat.exists }} == false'
This is a little hacky but works, however I have a 2nd, 3rd, 4th path to check..etc
My question here is how to I update/refactor the above code to use <register_name>.stat.exists == false from both app_dir_status and app_sub_dir_status to control my task ?

You don't need to make nested loop! There's all required data inside app_sub_dir_status – just strip unnecessary items.
Here's simplified example:
---
- hosts: localhost
gather_facts: no
vars:
my_list:
- name: zzz1
sub:
- aaa
- ccc
- name: zzz2
sub:
- aaa
- bbb
tasks:
- stat:
path: /tmp/{{ item.0.name }}/{{ item.1 }}
register: stat_res
with_subelements:
- "{{ my_list }}"
- sub
- debug:
msg: "path to create '{{ item }}'"
with_items: "{{ stat_res.results | rejectattr('stat.exists') | map(attribute='invocation.module_args.path') | list }}"
You can iterate over stat_res.results | rejectattr('stat.exists') | list as well, but will have to construct path again as /tmp/{{ item.item.0.name }}/{{ item.item.1 }} – note double item, because first item is an element of stat_res.results which contain another item as element of your original loop for stat task.
P.S. Also I see no reason for your first task, as subdir task can detect all missing directories.

Related

Ansible - loop over multiple items in stdout_lines

I am performing a grep with multiple items.
---
- hosts: my_host
gather_facts: false
vars:
my_list:
- whatever
- something
tasks:
- name: grep for item in search path
shell: "grep -rIL {{ item }} /tmp"
register: the_grep
loop: "{{ my_list }}"
- debug:
msg: "{{ item.stdout_lines }}"
loop: "{{ the_grep.results }}"
Depending on the result, multiple files could match.
msg:
- /tmp/something.conf
- /tmp/folder/file.txt
Q: How would I configure Ansible to loop over the items in stdout_lines?
The use case I'm solving is to delete .ini sections based on the item, but in this case, Ansible doesn't loop over the stdout_lines.
- name: remove stanza from ini file
ini_file:
path: "{{ item.stdout_lines }}"
section: "{{ item.item }}"
mode: '0600'
state: absent
loop: "{{ the_grep.results }}"
when: item.stdout_lines | length > 0
It seems that this doesn't work, but configuring item.stdout_lines[0] gives the partially expected result, since Ansible will use only the first item in that list. But ofc, not the 2nd and so on.
Perhaps there's a prettier answer, but solved it by using with_nested and creating a json_query:
- name: remove stanza from ini file
ini_file:
path: "{{ item.0 }}"
section: "{{ item.1.item }}"
mode: '0600'
state: absent
with_nested:
- "{{ the_grep | json_query('results[].stdout_lines[]') }}"
- "{{ the_grep.results }}"

Ansible line-in-file re-adding entries

What I'm trying to solve: I have several servers (in this example 3, 1 nfs server, 2 clients - they all resolve back to localhost for this minimal example), and the clients need to access shares on the server, which are created using the playbook.
The IP addresses of the clients need to be added to the respective entries in /etc/exports as they go - the list is not predefined at any given time. (In my actual playbook I use ansible facts, for this example I've added them as a variable)
In Ansible lineinfile regexp to manage /etc/exports Vladimir was so kind as to help with an initial thing, which works, but doesn't seem to be idempotent. The entries / ip addresses are added correctly the first time round, but the second run the IP addresses get re-added to the entries in /etc/exports, causing nfs to bomb out at that time (double entries)
Correct /etc/exports:
bar 192.168.34.47(rw,sync,no_root_squash,no_subtree_check) 192.168.34.46(rw,sync,no_root_squash,no_subtree_check)
foo 192.168.34.47(rw,sync,no_root_squash,no_subtree_check) 192.168.34.46(rw,sync,no_root_squash,no_subtree_check)
What I'm getting:
bar 192.168.34.47(rw,sync,no_root_squash,no_subtree_check) 192.168.34.46(rw,sync,no_root_squash,no_subtree_check) 192.168.34.47(rw,sync,no_root_squash,no_subtree_check) 192.168.34.46(rw,sync,no_root_squash,no_subtree_check)
foo 192.168.34.47(rw,sync,no_root_squash,no_subtree_check) 192.168.34.46(rw,sync,no_root_squash,no_subtree_check) 192.168.34.47(rw,sync,no_root_squash,no_subtree_check) 192.168.34.46(rw,sync,no_root_squash,no_subtree_check)
I've been butting my head around it but I can't come up with anything that works. I've distilled it down to the following playbook:
$ ansible-playbook main.yml
Content of the main.yml:
- hosts: localhost
become: yes
vars:
client_ip: 192.168.34.46
nfs_server: localhost
shares:
- bar
- foo
tasks:
- name: Mountpoint management
ansible.builtin.include_tasks: task.yml
loop: "{{ shares | default([]) }}"
loop_control:
loop_var: volume
args:
apply:
delegate_to: "{{ nfs_server }}"
- hosts: localhost
become: yes
vars:
client_ip: 192.168.34.47
nfs_server: localhost
shares:
- bar
- foo
tasks:
- name: Mountpoint management
ansible.builtin.include_tasks: task.yml
loop: "{{ shares | default([]) }}"
loop_control:
loop_var: volume
args:
apply:
delegate_to: "{{ nfs_server }}"
Second file task.yml which is needed to do a loop:
---
---
- name: Ensure /etc/exports exists
ansible.builtin.file:
path: /etc/exports
owner: root
group: root
mode: '0644'
state: touch
changed_when: False
- name: Add host {{ client_ip }} to {{ volume }}
ansible.builtin.lineinfile:
path: "/etc/exports"
regex: '^{{ volume }}(\s+)({{ ip_regex }})*({{ mount_opts_regex }})*(\s*)(.*)$'
line: '{{ volume }}\g<1>{{ ip }}{{ mount_opts }} \g<5>'
backrefs: true
vars:
ip: "{{ client_ip }}"
ip_regex: '{{ client_ip | regex_escape() }}'
mount_opts: '(rw,sync,no_root_squash,no_subtree_check)'
mount_opts_regex: '\(.*?\)'
- name: Read /etc/exports
command: "cat {{ item }}"
register: result
check_mode: no
loop:
- /etc/exports
changed_when: False
- ansible.builtin.set_fact:
content: "{{ dict(_files|zip(_lines)) }}"
vars:
_lines: "{{ result.results|map(attribute='stdout_lines')|list }}"
_files: "{{ result.results|map(attribute='item')|list }}"
- name: Add new line to /etc/exports
ansible.builtin.lineinfile:
path: "/etc/exports"
line: '{{ volume }} {{ client_ip }}{{ mount_opts }}'
vars:
mount_opts: '(rw,sync,no_root_squash,no_subtree_check)'
loop: "{{ content }}"
when: content[item] | select('search', volume)|length == 0
Well, thinking it through I finally went with constructing two lists, mapping those into a dict and checking if the values are already present. If they are, don't do anything, if not, add it.
main.yaml:
- hosts: localhost
become: yes
vars:
client_ip: 192.168.34.46
nfs_server: localhost
shares:
- bar
- foo
tasks:
- name: Mountpoint management
ansible.builtin.include_tasks: task.yml
loop: "{{ shares | default([]) }}"
loop_control:
loop_var: volume
args:
apply:
delegate_to: "{{ nfs_server }}"
- hosts: localhost
become: yes
vars:
client_ip: 192.168.34.47
nfs_server: localhost
shares:
- bar
- foo
tasks:
- name: Mountpoint management
ansible.builtin.include_tasks: task.yml
loop: "{{ shares | default([]) }}"
loop_control:
loop_var: volume
args:
apply:
delegate_to: "{{ nfs_server }}"
task.yaml:
---
- name: Ensure /etc/exports exists
ansible.builtin.file:
path: /etc/exports
owner: root
group: root
mode: '0644'
state: touch
changed_when: false
- name: Read /etc/exports
command: "cat /etc/exports"
register: result
check_mode: no
changed_when: false
- ansible.builtin.set_fact:
share_names: "{{ share_names | default([]) + [item.split(' ')[0]] }}"
share_values: "{{ share_values | default([]) + [item.split(' ')[1:]] }}"
loop:
"{{ result.stdout_lines }}"
- ansible.builtin.set_fact:
share_content: "{{ dict(share_names | zip(share_values)) }}"
- name: Add host {{ client_ip }} to {{ volume }}
ansible.builtin.lineinfile:
path: "/etc/exports"
regex: '^{{ volume }}(\s+)({{ ip_regex }})*({{ mount_opts_regex }})*(\s*)(.*)$'
line: '{{ volume }}\g<1>{{ client_ip }}{{ mount_opts }} \g<5>'
backrefs: true
vars:
ip_regex: '{{ client_ip | regex_escape() }}'
mount_opts: '(rw,sync,no_root_squash,no_subtree_check)'
mount_opts_regex: '\(.*?\)'
str: '{{ client_ip }}{{ mount_opts }}'
when: volume in share_content and str not in share_content[volume]
- name: Add new line to /etc/exports
ansible.builtin.lineinfile:
path: "/etc/exports"
line: '{{ volume }} {{ client_ip }}{{ mount_opts }}'
vars:
mount_opts: '(rw,sync,no_root_squash,no_subtree_check)'
loop: "{{ share_content }}"
when: volume not in share_content

In Ansible loop, test existence of files from registered results

I have several files that I need to backup in different directories. I have tried the code below and not working for me.
vars:
file_vars:
- {name: /file1}
- {name: /etc/file2}
- {name: /etc/file/file3}
tasks:
- name: "Checking if config files exists"
stat:
path: "{{ item.name }}"
with_items: "{{ file_vars }}"
register: stat_result
- name: Backup Files
copy: src={{ item.name }} dest={{ item.name }}{{ ansible_date_time.date }}.bak
with_items: "{{ file_vars }}"
remote_src: yes
when: stat_result.stat.exists == True
The problem is the condition
when: stat_result.stat.exists == True
There is no attribute stat_result.stat. Instead, the attribute stat_result.results is a list of the results from the loop. It's possible to create a dictionary of files and their statuses. For example
- set_fact:
files_stats: "{{ dict(my_files|zip(my_stats)) }}"
vars:
my_files: "{{ stat_result.results|json_query('[].item.name') }}"
my_stats: "{{ stat_result.results|json_query('[].stat.exists') }}"
Then simply use this dictionary in the condition
when: files_stats[item.name]
Below is a shorter version which creates the dictionary more efficiently
- set_fact:
files_stats: "{{ dict(stat_result.results|
json_query('[].[item.name, stat.exists]')) }}"
Please try using below worked for me:
---
- name: Copy files
hosts: localhost
become: yes
become_user: root
vars_files:
- files.yml
tasks:
- name: "Checking if config files exists"
stat:
path: "{{ item }}"
with_items: "{{ files }}"
register: stat_result
- name: Ansible
debug:
msg: "{{ stat_result }}"
- name: Backup Files
copy:
src: "{{ item }}"
dest: "{{ item.bak }}"
with_items: "{{ files }}"
when: stat_result == "True"
and files.yml will look like:
---
files:
- /tmp/file1
- /tmp/file2
you can check you playbook syntax using below command:
ansible-playbook copy.yml --syntax-check
Also you do dry run your playbook before actual execution.
ansible-playbook -i localhost copy.yml --check

ansible looping through the result and finding files

I am using ansible to configire 10-20 linux systems. I have a set of tools that I define in my invetory files with versions, as:
tools:
- tool: ABC
version: 7.8
- tool: XYZ
version: 8.32.1
Now, in my playback yml file, I would like to loop through them and have the necessay installation logic. Such as:
DEBUG tools loop
- name: Find installer files
copy:
src=
with_items:
- "{{ tools }}"
when:
tools.tool == "ABC"
In my case, {{tools.tool}}/{{tools.version}} has a tgz file which I need to unarchive at a remote location. Do you know how to do this? I have tried these:
- name: Find installer files
vars:
files: {{ lookup("fileglob",'tools/{{item.tool}}/linux/{{item.version}}/*') }}
unarchive:
src: "{{ files }}"
dest: "tools/{{item.tool}}/{{item.version}}/"
with_items:
- "{{ tools }}"
when:
item.tool == "ABC"
- name: Find installer files
debug:
msg: "{{ item}}"
with_items:
- "{{ tools }}"
with_fileglob:
- "tools/{{item.tool}}/linux/{{item.version}}/*"
when:
item.toolchain == "ABC"
But none worked. Thanks for the help.
It's not that simple as Your own solution breaks if there are multiple files in the directory, I assume.
So if You only have one file in the directory I wouldn't use fileglob at all but define a fixed name for it that You can generate knowing tool and version.
I also see the need for such sort of things often but did not found any nice solution for that. Only such ugly thing as:
- name: example book
hosts: localhost
become: false
gather_facts: false
vars:
tools:
- tool: ABC
version: 7.8
- tool: XYZ
version: 8.32.1
tools_files: []
tasks:
- name: prepare facts
set_fact:
tools_files: "{{ tools_files + [{'tool': item.tool | string, 'version': item.version | string, 'files': lookup('fileglob', 'tools/' ~ item.tool ~ '/linux/' ~ item.version ~ '/*', wantlist=True)}] }}"
with_items:
- "{{ tools }}"
- name: action loop
debug:
msg: "{{ {'src': item[1], 'dest': 'tools/' ~ item[0].tool ~ '/' ~ item[0].version ~ '/'} }}"
with_subelements:
- "{{ tools_files }}"
- files
when:
item[0].tool == "ABC"
or
- name: example book
hosts: localhost
become: false
gather_facts: false
vars:
tools:
- tool: ABC
version: 7.8
- tool: XYZ
version: 8.32.1
tools_files: []
tasks:
- name: prepare facts
set_fact:
tools_files: "{{ tools_files + [{'tool': item.tool | string, 'version': item.version | string, 'files': lookup('fileglob', 'tools/' ~ item.tool ~ '/linux/' ~ item.version ~ '/*', wantlist=True)}] }}"
with_items:
- "{{ tools }}"
- name: action loop
debug:
msg: "{{ {'src': item[1], 'dest': 'tools/' ~ item[0].tool ~ '/' ~ item[0].version ~ '/'} }}"
with_items:
- "{{ tools_files | subelements('files') }}"
when:
item[0].tool == "ABC"
Maybe I miss somthing because such things are a very basic feature (looping throug an array generating a result array beeing able to use all functions available and not just map using filters where some important things are just not available or cannot be used because map gives the inport as first argument to filter always).
That was actually simple. This worked for me:
- name: Find installer files
unarchive:
src:
"lookup('fileglob','tools/item.tool/linux/item.version/*') }}"
dest: "tools/{{item.tool}}/{{item.version}}/"
with_items:
- "{{ tools }}"
when:
item.tool == "ABC"

win_find FAILED! => {"msg": "'dict object' has no attribute 'files'"}

I am trying to run win_find with variables for "paths" to find temp files (e.g. c:\users\public\appdata\local\temp) but when I pass it to debug I get the error message
"FAILED! => {"msg": "'dict object' has no attribute 'files'"}"
I have tried swapping the "register" and "with_items" lines but that seems to make no difference at all.
- hosts: windows-02
vars:
dir: \AppData\Local\Temp\
tasks:
- name: find user directories
win_find:
paths: c:\users
recurce: yes
file_type: directory
get_checksum: no
register: user_dir
- name: debug 1
debug:
msg: "{{ item.path + dir }}"
loop: "{{ user_dir.files }}"
- name: find temp files
win_find:
paths: "{{ item.path + dir }}"
recurce: yes
hidden: yes
get_checksum: no
register: files_to_delete
with_items: "{{ user_dir.files }}"
- name: debug
debug:
msg: "{{ item }}"
loop: "{{ files_to_delete.files }}"
- name: remove
win_file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"
I expect to get a list of files to be deleted which will be passed to the "win_files" module. Instead I'm getting the error message
"FAILED! => {"msg": "'dict object' has no attribute 'files'"}"
There is a typo in win_find module, it is recurse: yes
Please find the below code whcih worked for me
---
- name: Find files
win_find:
paths: "{{paths}}"
age: "{{duration}}"
register: log_files
- name: Delete the files
win_file:
path: "{{item.path}}"
state: absent
with_items: "{{log_files.files}}"
Also files_to_delete will not have files_to_delete.files as it is in loop. It will have files_to_delete.results
So, it appears that win_find, at least in my case, does not return files when using "when_items". I got around this by creating a list of paths and passing it into the "paths" parameter of the module.
Here is my code:
- hosts: "{{ host }}"
vars:
dir: \AppData\Local\Temp\
temp_paths: []
line_break: \n
tasks:
- name: find user directories
win_find:
paths: c:\users
recurse: no
file_type: directory
get_checksum: no
register: user_dir
- name: debug 1
debug:
msg: "{{ item.path + dir }}"
loop: "{{ user_dir.files }}"
- name: set temp path
set_fact: temp_paths="{{temp_paths + [ item.path + dir ] }}"
when: item.path != 'C:\\users\\Public'
with_items: "{{ user_dir.files }}"
- name: find temp files
win_find:
paths: "{{ temp_paths }}"
recurse: True
patterns: '*'
hidden: False
get_checksum: False
register: files_to_delete
# ignore_errors: yes
- name: debug
debug:
msg: "{{ item.path }}"
loop: "{{ files_to_delete.files }}"
- name: remove
win_file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"

Resources