The below code only deletes the first file it gets inside the web dir. I want to remove all the files and folders inside the web directory and retain the web directory. How can I do that?
- name: remove web dir contents
file: path='/home/mydata/web/{{ item }}' state=absent
with_fileglob:
- /home/mydata/web/*
Note: I've tried rm -rf using command and shell, but they don't work. Perhaps I am using them wrongly.
Any help in the right direction will be appreciated.
I am using ansible 2.1.0.0
- name: Delete content & directory
file:
state: absent
path: /home/mydata/web/
Note: this will delete the directory too.
Remove the directory (basically a copy of https://stackoverflow.com/a/38201611/1695680), Ansible does this operation with rmtree under the hood.
- name: remove files and directories
file:
state: "{{ item }}"
path: "/srv/deleteme/"
owner: 1000 # set your owner, group, and mode accordingly
group: 1000
mode: '0777'
with_items:
- absent
- directory
If you don't have the luxury of removing the whole directory and recreating it, you can scan it for files, (and directories), and delete them one by one. Which will take a while. You probably want to make sure you have [ssh_connection]\npipelining = True in your ansible.cfg on.
- block:
- name: 'collect files'
find:
paths: "/srv/deleteme/"
hidden: True
recurse: True
# file_type: any # Added in ansible 2.3
register: collected_files
- name: 'collect directories'
find:
paths: "/srv/deleteme/"
hidden: True
recurse: True
file_type: directory
register: collected_directories
- name: remove collected files and directories
file:
path: "{{ item.path }}"
state: absent
with_items: >
{{
collected_files.files
+ collected_directories.files
}}
Using shell module (idempotent too):
- shell: /bin/rm -rf /home/mydata/web/*
If there are dot/hidden files:
- shell: /bin/rm -rf /home/mydata/web/* /home/mydata/web/.*
Cleanest solution if you don't care about creation date and owner/permissions:
- file: path=/home/mydata/web state=absent
- file: path=/home/mydata/web state=directory
I really didn't like the rm solution, also ansible gives you warnings about using rm.
So here is how to do it without the need of rm and without ansible warnings.
- hosts: all
tasks:
- name: Ansible delete file glob
find:
paths: /etc/Ansible
patterns: "*.txt"
register: files_to_delete
- name: Ansible remove file glob
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"
source: http://www.mydailytutorials.com/ansible-delete-multiple-files-directories-ansible/
try the below command, it should work
- shell: ls -1 /some/dir
register: contents
- file: path=/some/dir/{{ item }} state=absent
with_items: {{ contents.stdout_lines }}
That's what I come up with:
- name: Get directory listing
find:
path: "{{ directory }}"
file_type: any
hidden: yes
register: directory_content_result
- name: Remove directory content
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ directory_content_result.files }}"
loop_control:
label: "{{ item.path }}"
First, we're getting directory listing with find, setting
file_type to any, so we wouldn't miss nested directories and links
hidden to yes, so we don't skip hidden files
also, do not set recurse to yes, since it is not only unnecessary, but may increase execution time.
Then, we go through that list with file module. It's output is a bit verbose, so loop_control.label will help us with limiting output (found this advice here).
But I found previous solution to be somewhat slow, since it iterates through the content, so I went with:
- name: Get directory stats
stat:
path: "{{ directory }}"
register: directory_stat
- name: Delete directory
file:
path: "{{ directory }}"
state: absent
- name: Create directory
file:
path: "{{ directory }}"
state: directory
owner: "{{ directory_stat.stat.pw_name }}"
group: "{{ directory_stat.stat.gr_name }}"
mode: "{{ directory_stat.stat.mode }}"
get directory properties with the stat
delete directory
recreate directory with the same properties.
That was enough for me, but you can add attributes as well, if you want.
Using file glob also it will work. There is some syntax error in the code you posted. I have modified and tested this should work.
- name: remove web dir contents
file:
path: "{{ item }}"
state: absent
with_fileglob:
- "/home/mydata/web/*"
Following up on the most upvoted answer here (which I cannot edit since "edit queue is full"):
- name: Delete content & directory
file:
state: absent
path: /home/mydata/web/
- name: Re-create the directory
file:
state: directory
path: /home/mydata/web/
While Ansible is still debating to implement state = empty
https://github.com/ansible/ansible-modules-core/issues/902
my_folder: "/home/mydata/web/"
empty_path: "/tmp/empty"
- name: "Create empty folder for wiping."
file:
path: "{{ empty_path }}"
state: directory
- name: "Wipe clean {{ my_folder }} with empty folder hack."
synchronize:
mode: push
#note the backslash here
src: "{{ empty_path }}/"
dest: "{{ nl_code_path }}"
recursive: yes
delete: yes
delegate_to: "{{ inventory_hostname }}"
Note though, with synchronize you should be able to sync your files (with delete) properly anyway.
Created an overall rehauled and fail-safe implementation from all comments and suggestions:
# collect stats about the dir
- name: check directory exists
stat:
path: '{{ directory_path }}'
register: dir_to_delete
# delete directory if condition is true
- name: purge {{directory_path}}
file:
state: absent
path: '{{ directory_path }}'
when: dir_to_delete.stat.exists and dir_to_delete.stat.isdir
# create directory if deleted (or if it didn't exist at all)
- name: create directory again
file:
state: directory
path: '{{ directory_path }}'
when: dir_to_delete is defined or dir_to_delete.stat.exist == False
Below code worked for me :
- name: Get directory listing
become: yes
find:
paths: /applications/cache
patterns: '*'
hidden: yes
register: directory_content_result
- name: Remove directory content
become: yes
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ directory_content_result.files }}"
There is an issue open with respect to this.
For now, the solution works for me: create a empty folder locally and synchronize it with the remote one.
Here is a sample playbook:
- name: "Empty directory"
hosts: *
tasks:
- name: "Create an empty directory (locally)"
local_action:
module: file
state: directory
path: "/tmp/empty"
- name: Empty remote directory
synchronize:
src: /tmp/empty/
dest: /home/mydata/web/
delete: yes
recursive: yes
I want to make sure that the find command only deletes everything inside the directory and leave the directory intact because in my case the directory is a filesystem. The system will generate an error when trying to delete a filesystem but that is not a nice option. Iam using the shell option because that is the only working option I found so far for this question.
What I did:
Edit the hosts file to put in some variables:
[all:vars]
COGNOS_HOME=/tmp/cognos
find=/bin/find
And create a playbook:
- hosts: all
tasks:
- name: Ansible remove files
shell: "{{ find }} {{ COGNOS_HOME }} -xdev -mindepth 1 -delete"
This will delete all files and directories in the COGNOS_HOME variable directory/filesystem. The "-mindepth 1" option makes sure that the current directory will not be touched.
I have written an custom ansible module to cleanup files based on multiple filters like age, timestamp, glob patterns, etc.
It is also compatible with ansible older versions. It can be found here.
Here is an example:
- cleanup_files:
path_pattern: /tmp/*.log
state: absent
excludes:
- foo*
- bar*
Just a small cleaner copy & paste template of ThorSummoners answer, if you are using Ansible >= 2.3 (distinction between files and dirs not necessary anymore.)
- name: Collect all fs items inside dir
find:
path: "{{ target_directory_path }}"
hidden: true
file_type: any
changed_when: false
register: collected_fsitems
- name: Remove all fs items inside dir
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ collected_fsitems.files }}"
when: collected_fsitems.matched|int != 0
Isn't it that simple ... tested working ..
eg.
---
- hosts: localhost
vars:
cleandir: /var/lib/cloud/
tasks:
- shell: ls -a -I '.' -I '..' {{ cleandir }}
register: ls2del
ignore_errors: yes
- name: Cleanup {{ cleandir }}
file:
path: "{{ cleandir }}{{ item }}"
state: absent
with_items: "{{ ls2del.stdout_lines }}"
- name: Files to delete search
find:
paths: /home/mydata/web/
file_type: any
register: files_to_delete
- name: Deleting files to delete
file:
path: '{{ item.path }}'
state: absent
with_items: "{{ files_to_delete.files }}"
I like the following solution:
- name: remove web dir contents
command:
cmd: "find . -path '*/*' -delete -print"
chdir: "/home/mydata/web/"
register: web_files_list
changed_when: web_files_list.stdout | length > 0
because it is:
simple
idempotent
fast
Assuming you are always in Linux, try the find cmd.
- name: Clean everything inside {{ item }}
shell: test -d {{ item }} && find {{ item }} -path '{{ item }}/*' -prune -exec rm -rf {} \;
with_items: [/home/mydata/web]
This should wipe out files/folders/hidden under /home/mydata/web
- name: delete old data and clean cache
file:
path: "{{ item[0] }}"
state: "{{ item[1] }}"
with_nested:
- [ "/data/server/{{ app_name }}/webapps/", "/data/server/{{ app_name }}/work/" ]
- [ "absent", "directory" ]
ignore_errors: yes
Below worked for me,
- name: Ansible delete html directory
file:
path: /var/www/html
state: directory
Related
I want to take a list in a directory like: ls /root/test and then have a list variable that deletes the files in another directory like: /root/test2
- hosts: all
tasks:
- name: take a list in test dir
find:
paths: /root/test
register: files_to_delete
- name: Ansible delete files in test2 path
file:
path: /root/test2/{{item}}
state: absent
with_items: "{{ files_to_delete.files }}"
I couldn't find a guide anywhere and this method has not worked, I would appreciate it if someone can help me fix this issue.
Thanks to #Zeitounator I read docs and solved it:
- name: Ansible delete file glob
find:
paths: /root/test
pattern: "*.txt"
register: files_to_delete
- debug: var={{item}}
with_items: files_to_delete
- name: Ansible remove file glob
file:
path: /root/test2/{{ item.path | basename }}
state: absent
with_items: "{{ files_to_delete.files }}"
AM in a process of achieving below list of tasks, and could someone please rectify the playbook or suggest a way to get the requirement done.
High level purpose of the activity is below:
find previous day's log files in multiple paths and archive them under a date wise folder (folder has to be created for particular date) in a different path.
My approach is:
Create a date wise directory and then search the previous day's log files and then copy them in to the newly created directory and then archive it.
I am having an issue when defining paths and variables in copy section. Can someone help with this?
- name: Purge old spider logs
become: true
hosts: node1
vars:
date: "{{ lookup('pipe', 'date +%Y-%m-%d') }}"
tasks:
- name: create a directory
file:
path: /path/{{ date }}
state: directory
mode: '777'
register: logdir
- name: Find log files
find:
path: /test/logs
age: 3600
patterns:
- "name.log.*"
recurse: yes
register: testlogs
- debug:
var: testlogs.path
- debug:
var=item.files
with_items: '{{ testlogs.files }}'
- name: Copy files in to backup location
copy:
src: "{{ item.files }}"
dest: "{{ item.path }}"
with_items:
- '{{ item.files.testlog.files }}'
- '{{ item.path.logdir.path }}'
if i understand your problem you want to copy all remote log files to another destination with a folder dated:
- name: Purge old spider logs
become: true
hosts: node1
vars:
date: "{{ lookup('pipe', 'date +%Y-%m-%d') }}"
tasks:
- name: create a remote directory
file:
path: /path/{{ date }}
state: directory
mode: '777'
register: logdir
- name: Find log files
find:
path: logs
age: 3600
patterns:
- "name.log.*"
recurse: yes
register: testlogs
- name: Copy (remote) files in to backup location (remote)
copy:
remote_src: yes
src: "{{ item.path }}"
dest: "{{logdir.path}}/"
with_items:
- '{{ testlogs.files }}'
Goal:
Create multiple directories if they don't exist.
Don't change permissions of existing folder
Current playbook:
- name: stat directories if they exist
stat:
path: "{{ item }}"
with_items:
- /data/directory
- /data/another
register: myvar
- debug: var=myvar.results
- name: create directory if they don't exist
file:
path: "{{ item.invocation.module_args.path }}"
state: directory
owner: root
group: root
mode: 0775
loop: "{{ stat.results }}"
# with_items: "{{ stat.results }}" # for older versions of Ansible
# when: myvar.results.stat.exists == false
The when statement is wrong.
I looked at the example provided; http://docs.ansible.com/ansible/stat_module.html. But this only works for a single folder.
Using Ansible modules, you don't need to check if something exist or not, you just describe the desired state, so:
- name: create directory if they don't exist
file:
path: "{{ item }}"
state: directory
owner: root
group: root
mode: 0775
loop:
- /data/directory
- /data/another
Ansible - Creating multiple folders without changing permissions of previously existing.
Working fine for me. Hope this works for you as well just try.
---
- name: "Creating multiple by checking folders"
hosts: your_host_name
tasks:
- block:
- name: "Checking folders"
stat:
path: "{{item}}"
register: folder_stats
with_items:
- ["/var/www/f1","/var/www/f2","/var/www/f3","/var/www/f4"]
- name: "Creating multiple folders without disturbing previous permissions"
file:
path: "{{item.item}}"
state: directory
mode: 0755
group: root
owner: root
when: item.stat.exists == false
loop:
- "{{folder_stats.results}}"
...
Starting from Ansible 2.5, loop should be used to iterate over a list, see https://docs.ansible.com/ansible/latest/user_guide/playbooks_loops.html#standard-loops
As the Ansible file module is idempotent, you do not have to check if the folders already exist.
For example:
- name: create backup directories
file:
path: "{{ item }}"
state: directory
owner: backup
group: backup
mode: 0775
loop:
- /backupdisk/certificates
- /backupdisk/mysql
- /backupdisk/wordpress
How can I copy more than a single file into remote nodes by Ansible in a task?
I've tried to duplicate the copy module line in my task to define files but it only copies the first file.
You can use the with_fileglob loop for this:
- copy:
src: "{{ item }}"
dest: /etc/fooapp/
owner: root
mode: 600
with_fileglob:
- "/playbooks/files/fooapp/*"
- name: copy multiple items
copy:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
loop:
- src: containerizers
dest: /etc/mesos/containerizers
- src: another_file
dest: /etc/somewhere
- src: dynamic
dest: "{{ var_path }}"
Since Ansible 2.5 the with_* constructs are not recommended, and loop syntax should be used. A simple practical example:
- name: Copy CA files
copy:
src: '{{item}}'
dest: '/etc/pki/ca-trust/source/anchors'
owner: root
group: root
mode: 0644
loop:
- symantec-private.crt
- verisignclass3g2.crt
You can use with_together for this purpose:
- name: Copy multiple files to multiple directories
copy: src={{ item.0 }} dest={{ item.1 }}
with_together:
- [ 'file1', 'file2', 'file3' ]
- [ '/dir1/', '/dir2/', '/dir3/' ]
If you need more than one location, you need more than one task. One copy task can copy only from one location (including multiple files) to another one on the node.
- copy: src=/file1 dest=/destination/file1
- copy: src=/file2 dest=/destination/file2
# copy each file over that matches the given pattern
- copy: src={{ item }} dest=/destination/
with_fileglob:
- /files/*
You can use a find, and then copy those files.
---
- hosts: lnx
tasks:
- find:
paths: /appl/scripts/inq
recurse: true
patterns: "inq.Linux*"
register: file_to_copy
- copy:
src: "{{ item.path }}"
dest: /usr/local/sbin/
owner: root
mode: 0775
loop: "{{ files_to_copy.files }}"
Or you can use with_items:
- copy:
src: "{{ item }}"
dest: /etc/fooapp/
owner: root
mode: 600
with_items:
- dest_dir
- name: find inq.Linux*
find: paths="/appl/scripts/inq" recurse=yes patterns="inq.Linux*"
register: find_files
- name: set fact
set_fact:
all_files:
- "{{ find_files.files | map(attribute='path') | list }}"
when: find_files > 0
- name: copy files
copy:
src: "{{ item }}"
dest: /destination/
with_items: "{{ all_files }}"
when: find_files > 0
copy module is a wrong tool for copying many files and/or directory structure, use synchronize module instead which uses rsync as backend. Mind you, it requires rsync installed on both controller and target host. It's really powerful, check ansible documentation.
Example - copy files from build directory (with subdirectories) of controller to /var/www/html directory on target host:
synchronize:
src: ./my-static-web-page/build/
dest: /var/www/html
rsync_opts:
- "--chmod=D2755,F644" # copy from windows - force permissions
You can loop through variable with list of directories:
- name: Copy files from several directories
copy:
src: "{{ item }}"
dest: "/etc/fooapp/"
owner: root
mode: "0600"
loop: "{{ files }}"
vars:
files:
- "dir1/"
- "dir2/"
Use the following source code for copy multiple files on your client machine.
- name: Copy data to the client machine
hosts: hostname
become_method: sudo
become_user: root
become: true
tasks:
# Copy twice as sometimes files get skipped (mostly only one file skipped from a folder if the folder does not exist)
- name: Copy UFO-Server
copy:
src: "source files path"
dest: "destination file path"
owner: root
group: root
mode: 0644
backup: yes
ignore_errors: true
Note:
If you are passing multiple paths by using variable then
src: "/root/{{ item }}"
If you are passing path by using a variable for different items then
src: "/root/{{ item.source_path }}"
Copy files from multiple directories to multiple directories with Ansible
I found the guenhter answer helpful but needed to change also the remote files' mode. I don't have enough reputation to put this as a comment, which would be a more appropriate place for this. In the example, I copy two files from two directories into /tmp and /tmp/bin, which I create first and modify remote files mode.
- name: cpldupd
hosts: test
remote_user: root
become: true
vars:
- rpth: /tmp
tasks:
- name: Create '{{rpth}}/bin'
file:
path: '{{rpth}}/bin'
state: directory
- name: Transfer
copy: src={{ item.src }} dest={{ item.dest }} mode=0775
with_items:
- { src: '../utils/cpldupd', dest: '{{rpth}}/cpldupd' }
- { src: '../utils/bin/cpldupd', dest: '{{rpth}}/bin/cpldupd' }
Here is a generic solution for copying files:
...
- name: Find files you want to move
ansible.builtin.find:
paths: /path/to/files/
file_type: file
excludes: "*.txt" # Whatever pattern you want to exclude
register: files_output
- name: Copy the files
ansible.builtin.copy:
src: "{{ item.path }}"
dest: /destination/directory/
loop: "{{ files_output.files }}"
...
This is more powerful than using with_fileglob as you can match using regexes. Here is this play in action:
$ ls /path/to/files
demo.yaml test.sh ignore.txt
$ ls /destination/directory
file.h
$ ansible-playbook playbook.yaml
...[some output]...
$ ls /destination/directory
file.h demo.yaml test.sh
As you can see from the above example, ignore.txt was not copied over to the destination directory because of the excludes regex in the playbook. Ignoring files like this is not possible as simply using with_fileglob.
Additionally, you can move files from multiple directories with relative ease:
...
- name: Find files you want to move
ansible.builtin.find:
paths: /path/to/files/
# ... the rest of the task
register: list1
- name: Find more files you want to move
ansible.builtin.find:
paths: /different/path/
# ... the rest of the task
register: list2
- name: Copy the files
ansible.builtin.copy:
src: "{{ item.path }}"
dest: /destination/directory/
loop: "{{ list1.files + list2.files }}"
...
Here is a sample Ansible Script to copy multiple Files on remote Hosts
- name: Copy Multiple Files on remote Hosts
ansible.windows.win_copy:
src: "{{ srcPath }}/{{ item }}" # Remeber to us {{item}}
# as a postfix to source path
dest: "{{ destPath }}"
remote_src: yes # if source path is available on remote Host
with_items:
- abc.txt
- abc.properties
hosts: test
gather_facts: false
become: true
vars:
path: '/home/ansibm/playbooks'
remote_path: '/home/{{ansible_ssh_user}}'
dir: 'yml_files'
tasks:
name: "creating directory for backup file"
file:
path: '{{ remote_path }}/{{ dir }}'
state: directory
owner: '{{ansible_ssh_user}}'
group: '{{ansible_ssh_user}}'
mode: 0700
name: "copying yml files"
copy:
src: '{{item}}'
dest: '{{ remote_path }}/{{ dir }}'
owner: '{{ansible_ssh_user}}'
group: '{{ansible_ssh_user}}'
mode: 0644
loop:
- '{{ path }}/ab.html'
- '{{ path }}/cp.yml'
I want to recursively copy over a directory and render all .j2 files in there as templates. For this I am currently using the following lines:
- template: >
src=/src/conf.d/{{ item }}
dest=/dest/conf.d/{{ item|replace('.j2','') }}
with_lines: find /src/conf.d/ -type f -printf "%P\n"
Now I'm looking for a way to remove unmanaged files from this directory. For example if I remove a file/template from /src/conf.d/ I want Ansible to remove it from /dest/conf.d/ as well.
Is there some way to do this? I tried fiddling around with rsync --delete, but there I got a problem with the templates which get their suffix .j2 removed.
I'd do it like this, assuming a variable defined as 'managed_files' up top that is a list.
- shell: ls -1 /some/dir
register: contents
- file: path=/some/dir/{{ item }} state=absent
with_items: contents.stdout_lines
when: item not in managed_files
We do this with our nginx files, since we want them to be in a special order, come from templates, but remove unmanaged ones this works:
# loop through the nginx sites array and create a conf for each file in order
# file will be name 01_file.conf, 02_file.conf etc
- name: nginx_sites conf
template: >
src=templates/nginx/{{ item.1.template }}
dest={{ nginx_conf_dir }}/{{ '%02d' % item.0 }}_{{ item.1.conf_name|default(item.1.template) }}
owner={{ user }}
group={{ group }}
mode=0660
with_indexed_items: nginx_sites
notify:
- restart nginx
register: nginx_sites_confs
# flatten and map the results into simple list
# unchanged files have attribute dest, changed have attribute path
- set_fact:
nginx_confs: "{{ nginx_sites_confs.results|selectattr('dest', 'string')|map(attribute='dest')|list + nginx_sites_confs.results|selectattr('path', 'string')|map(attribute='path')|select|list }}"
when: nginx_sites
# get contents of conf dir
- shell: ls -1 {{ nginx_conf_dir }}/*.conf
register: contents
when: nginx_sites
# so we can delete the ones we don't manage
- name: empty old confs
file: path="{{ item }}" state=absent
with_items: contents.stdout_lines
when: nginx_sites and item not in nginx_confs
The trick (as you can see) is that template and with_items have different attributes in the register results. Then you turn them into a list of files you manage and then get a list of the the directory and removed the ones not in that list.
Could be done with less code if you already have a list of files. But in this case I'm creating an indexed list so need to create the list as well with map.
I want to share my experience with this case.
Ansible from 2.2 is had with_filetree loop provides simple way to upload dirs, links, static files and even (!) templates. It's best way to keep my config dir synchronized.
- name: etc config - Create directories
file:
path: "{{ nginx_conf_dir }}/{{ item.path }}"
state: directory
mode: 0755
with_filetree: etc/nginx
when: item.state == 'directory'
- name: etc config - Creating configuration files from templates
template:
src: "{{ item.src }}"
dest: "{{ nginx_conf_dir }}/{{ item.path | regex_replace('\\.j2$', '') }}"
mode: 0644
with_filetree: etc/nginx
when:
- item.state == "file"
- item.path | match('.+\.j2$') | bool
- name: etc config - Creating staic configuration files
copy:
src: "{{ item.src }}"
dest: "{{ nginx_conf_dir }}/{{ item.path }}"
mode: 0644
with_filetree: etc/nginx
when:
- item.state == "file"
- not (item.path | match('.+\.j2$') | bool)
- name: etc config - Recreate symlinks
file:
src: "{{ item.src }}"
dest: "{{ nginx_conf_dir }}/{{ item.path }}"
state: link
force: yes
mode: "{{ item.mode }}"
with_filetree: etc/nginx
when: item.state == "link"
Next we may want delete unused files from config dir. It's simple.
We gather list of uploaded files and files exist on remote server, next remove diffrence.
But we may want to have unmanaged files in config dir.
I've used -prune functionality of find to avoid clearing folders with unmanaged files.
PS _(Y)_ sure after I have deleted some unmanaged files
- name: etc config - Gathering managed files
set_fact:
__managed_file_path: "{{ nginx_conf_dir }}/{{ item.path | regex_replace('\\.j2$', '') }}"
with_filetree: etc/nginx
register: __managed_files
- name: etc config - Convert managed files to list
set_fact: managed_files="{{ __managed_files.results | map(attribute='ansible_facts.__managed_file_path') | list }}"
- name: etc config - Gathering exist files (excluding .ansible_keep-content dirs)
shell: find /etc/nginx -mindepth 1 -type d -exec test -e '{}/.ansible_keep-content' \; -prune -o -print
register: exist_files
changed_when: False
- name: etc config - Delete unmanaged files
file: path="{{ item }}" state=absent
with_items: "{{ exist_files.stdout_lines }}"
when:
- item not in managed_files
Here's something I came up with:
- template: src=/source/directory{{ item }}.j2 dest=/target/directory/{{ item }}
register: template_results
with_items:
- a_list.txt
- of_all.txt
- templates.txt
- set_fact:
managed_files: "{{ template_results.results|selectattr('invocation', 'defined')|map(attribute='invocation.module_args.dest')|list }}"
- debug:
var: managed_files
verbosity: 0
- find:
paths: "/target/directory/"
patterns: "*.txt"
register: all_files
- set_fact:
files_to_delete: "{{ all_files.files|map(attribute='path')|difference(managed_files) }}"
- debug:
var: all_files
verbosity: 0
- debug:
var: files_to_delete
verbosity: 0
- file: path={{ item }} state=absent
with_items: "{{ files_to_delete }}"
This generates the templates (however which way you want), and records the results in 'template_results'
The the results are mangled to get a simple list of the "dest" of each template. Skipped templates (due to a when condition, not shown) have no "invocation" attribute, so they're filtered out.
"find" is then used to get a list of all files that should be absent unless specifically written.
this is then mangled to get a raw list of files present, and then the "supposed to be there" files are removed.
The remaining "files_to_delete" are then removed.
Pros: You avoid multiple 'skipped' entries showing up during deletes.
Cons: You'll need to concatenate each template_results.results if you want to do multiple template tasks before doing the find/delete.
There might be a couple of ways to handle this, but would it be possible to entirely empty the target directory in a task before the template step? Or maybe drop the templated files into a temporary directory and then delete+rename in a subsequent step?
Usually I do not remove files but I add -unmanaged suffix to its name.
Sample ansible tasks:
- name: Get sources.list.d files
shell: grep -r --include=\*.list -L '^# Ansible' /etc/apt/sources.list.d || true
register: grep_unmanaged
changed_when: grep_unmanaged.stdout_lines
- name: Add '-unmanaged' suffix
shell: rename 's/$/-unmanaged/' {{ item }}
with_items: grep_unmanaged.stdout_lines
EXPLANATION
Grep command uses:
-r to do recursive search
--include=\*.list - only take files
with .list extension during recursive search
-L '^# Ansible' - display file names that are not having line starting with '# Ansible'
|| true - this is used to ignore errors. Ansible's ignore_errors also works but before ignoring the error ansible will show it in red color during ansible-playbook run
which is undesired (at least for me).
Then I register output of grep command as a variable. When grep displays any output I set this task as changed (the line changed_when is responsible for this).
In next task I iterate grep output (i.e. file names returned by grep) and run rename command to add suffix to each file.
That's all. Next time you run the command first task should be green and second skipped.
I am using Ansible version 2.9.20
---
# tasks file for delete_unmanaged_files
- name: list files in dest
shell: ls -1 dest/conf.d
register: files_in_dest
- name: list files in src
shell: ls -1 src/conf.d
register: files_in_src
- name: Managed files - dest
command: echo {{ item|replace('.j2','') }}
with_items: "{{ files_in_dest.stdout_lines }}"
register: managed_files_dest
- name: Managed files - src
command: echo {{ item|replace('.j2','') }}
with_items: "{{ files_in_src.stdout_lines }}"
register: managed_files_src
- name: Convert src managed files to list
set_fact: managed_files_src_list="{{ managed_files_src.results | map(attribute='stdout') | list }}"
- name: Delete unmanaged files in dest
file: path=dest/conf.d/{{ item.stdout }} state=absent
with_items: "{{ managed_files_dest.results }}"
when: item.stdout not in managed_files_src_list
I think depending on the usecase of this issue, I found above solution might help you. Here, I have created 6 tasks.
Explanation:
Task-1 & Task-2 will help storing file names in variable "files_in_dest" and "files_in_src" in it.
Task-3 & Task-4 will inherit the output coming from Task-1 & Task-2 and then replace the j2 file (required for the usecase). Then these tasks will store the output in "managed_files_dest" and "managed_files_src" variables.
Task-5 will convert the output of "managed_files_src" to list, so that we can have all the present files (at current state) stored in src directory in a proper or single list and then we can use this list in next task to know the unmanaged files in dest directory.
Task-6 will delete the unmanaged files in dest.
Apparently this isn't possible with ansible at the moment. I had a conversation with mdehaan on IRC and it boils down to ansible not having a directed acyclic graph for resources, making things like this very hard.
Asking mdehaan for an example e.g. authoritatively managing a sudoers.d directory he came up with these things:
14:17 < mdehaan> Robe: http://pastebin.com/yrdCZB0y
14:19 < Robe> mdehaan: HM
14:19 < Robe> mdehaan: that actually looks relatively sane
14:19 < mdehaan> thanks :)
14:19 < Robe> the problem I'm seeing is that I'd have to gather the managed files myself
14:19 < mdehaan> you would yes
14:19 < mdehaan> ALMOST
14:20 < mdehaan> you could do a fileglob and ... well, it would be a little gross
[..]
14:32 < mdehaan> eh, theoretical syntax, nm
14:33 < mdehaan> I could do it by writing a lookup plugin that filtered a list
14:34 < mdehaan> http://pastebin.com/rjF7QR24
14:34 < mdehaan> if that plugin existed, for instance, and iterated across lists in A that were also in B
Building on #user2645850's answer I came up with this improved version, in this case it manages the vhost configuration of Apache. It doesn't use shell and thus works also in --check mode.
# Remove unmanged vhost configs left over from renaming or removing apps
# all managed configs need to be added to "managed_sites" in advance
- find:
paths: /etc/apache2/sites-available
patterns: '*.conf'
register: sites_available_contents
- name: Remove unmanaged vhost config files
file:
path: /etc/apache2/sites-available/{{ item }}
state: absent
with_items: "{{ sites_available_contents.files | map(attribute='path') | map('basename') | list }}"
when: item not in managed_sites
# links may differ from files, therefore we need our own find task for them
- find:
paths: /etc/apache2/sites-enabled
file_type: any
register: sites_enabled_contents
- name: Remove unmanaged vhost config links
file:
path: /etc/apache2/sites-enabled/{{ item }}
state: absent
with_items: "{{ sites_enabled_contents.files | map(attribute='path') | map('basename') | list }}"
when: item not in managed_sites
Examples on how to build managed_sites:
# Add single conf and handle managed_sites being unset
- set_fact:
managed_sites: "{{ (managed_sites | default([])) + [ '000-default.conf' ] }}"
# Add a list of vhosts appending ".conf" to each entry of vhosts
- set_fact:
managed_sites: "{{ managed_sites + ( vhosts | map(attribute='app') | product(['.conf']) | map('join') | list ) }}"