I want to take a list in a directory like: ls /root/test and then have a list variable that deletes the files in another directory like: /root/test2
- hosts: all
tasks:
- name: take a list in test dir
find:
paths: /root/test
register: files_to_delete
- name: Ansible delete files in test2 path
file:
path: /root/test2/{{item}}
state: absent
with_items: "{{ files_to_delete.files }}"
I couldn't find a guide anywhere and this method has not worked, I would appreciate it if someone can help me fix this issue.
Thanks to #Zeitounator I read docs and solved it:
- name: Ansible delete file glob
find:
paths: /root/test
pattern: "*.txt"
register: files_to_delete
- debug: var={{item}}
with_items: files_to_delete
- name: Ansible remove file glob
file:
path: /root/test2/{{ item.path | basename }}
state: absent
with_items: "{{ files_to_delete.files }}"
How to register a variable when using loop with stat module?
I am working on a project where I wish to run comparisons against the known value of a collection of files (checksum), which I will then take action if a change is detected (EG: notify someone, have not written this part yet).
If this were purely a CLI matter, I would have this sorted with some easy SH scripting.
That said, I have Ansible (2.7.5) available within my ENV and am keen to use it!
In reading the vendor documents, using the stat module felt the "Ansible way" to go on this one.
Currently just *NIX servers (Linux, Solaris, and possibly AIX) are in scope, but eventually this might also apply to Windows, where I expect I would use win_stat instead with suitable parameters.
At present I plan to dump the results of the scan to a file (EG: CSV), which I would then iterate / match against, for the purposes of a comparison (to detect if a file has been somehow changed).
This is another part I have not written yet (the read a file and compare portions), but expect to hit those once I get this present matter sorted.
My current challenge, is that I can get "one-off" stat checks to work fine.
However, I expect to be targeting a whole directory worth of files, and thus want to presumably:
"discover" the contents of the target directory, and retain this in memory
iterate (loop) through the list in memory
performing a stat check upon each file
retaining the checksum of each file
building some sort of dict or list?
write the collective results (or one line at a time) out to a log file of sorts (CSV.log: file_path,file_checksum)
I would welcome your feedback on what I might be missing (aside from some hair at this point)?
I have tried a few different approaches to looping within the playbook (loop, with_items, etc.), however the challenge remains the same.
The stat loop runs fine, but the trailing register statement fails to commit the output to memory (resulting in a variety of "undefined variable" errors).
Am I somehow missing something in my loop definition?
Looking at the vendor docs on "Using register with a loop", it would appear I am doing this correctly (in my view anyway).
Simple "target files" I am checking against within a directory.
/tmp/app/targets/file1.txt
Some text.
/tmp/app/targets/file2.cfg
cluster=0
cluster_id=app_pool_00
/tmp/app/targets/file3.sh
#!/bin/sh
printf "Hello world\n"
exit 0
My prototyping playbook as it exists currently.
---
- name: check file integrity
hosts: localhost
become: no
vars:
TARGET: /tmp/app/targets
LOG: /tmp/app/archive/scan_results.log
tasks:
- name: discover target files
find:
paths: "{{ TARGET }}"
recurse: yes
file_type: file
register: TARGET_FILES
- name: scan target
stat:
path: "{{ item.path }}"
get_checksum: yes
loop: "{{ TARGET_FILES.files }}"
register: TARGET_RESULTS
- name: DEBUG
debug:
var: "{{ TARGET_RESULTS }}"
- name: write findings to log
copy:
content: "{{ TARGET_RESULTS.stat.path }},{{ TARGET_RESULTS.stat.checksum }}"
dest: "{{ LOG }}"
...
My "one-off" playbook that worked.
---
- name: check file integrity
hosts: localhost
become: no
vars:
TARGET: /tmp/app/targets/file1.txt
LOG: /tmp/app/archive/scan_results.log
tasks:
- name: scan target
stat:
path: '{{ TARGET }}'
checksum_algorithm: sha1
follow: no
get_attributes: yes
get_checksum: yes
get_md5: no
get_mime: yes
register: result
- name: write findings to log
copy:
content: "{{ result.stat.path }},{{ result.stat.checksum }}"
dest: "{{ LOG }}"
...
The output was not exciting, but useful.
Would expect to build this up with multi-line output (one line per file stat checked) if I could figure out how to loop / register loop output correctly.
/tmp/app/archive/scan_results.log
/tmp/app/targets/file1.txt,8d06cea05d408d70c59b1dbc5df3bda374d869a4
You can use the set_fact module to register a variable like you want.
I don't use it in my test for you, it maybe useless in your case :
---
- name: check file integrity
hosts: localhost
vars:
TARGET: /tmp/app/targets
LOG: /tmp/app/archive/scan_results.log
tasks:
- name: 'discover target files'
find:
paths: "{{ TARGET }}"
recurse: yes
file_type: file
register: TARGET_FILES
- debug:
var: TARGET_FILES
- name: 'scan target'
stat:
path: "{{ item.path }}"
get_checksum: yes
loop: "{{ TARGET_FILES.files }}"
register: TARGET_RESULTS
- debug:
var: TARGET_RESULTS
- name: 'write findings to log'
lineinfile:
line: "{{ item.stat.path }},{{ item.stat.checksum }}"
path: "{{ LOG }}"
create: yes
loop: '{{ TARGET_RESULTS.results }}'
result:
# cat /tmp/app/archive/scan_results.log
/tmp/app/targets/file3.sh,bb4b0ffe4b5d26551785b250c38592b6f482cab4
/tmp/app/targets/file1.txt,8d06cea05d408d70c59b1dbc5df3bda374d869a4
/tmp/app/targets/file2.cfg,fb23292e06f91a0e0345f819fdee34fac8a53e59
Best Regards
I have a bit of problem I can't seem overcome. I have a folder with a lot of folders that are generated. I want to delete all folders that are older than three days, but I want to keep a minimum of 10 folders.
I came up with this half-working code and I'd like some suggestions how to tackle this.
---
- hosts: all
tasks:
# find all files that are older than three
- find:
paths: "/Users/asteen/Downloads/sites/"
age: "3d"
file_type: directory
register: dirsOlderThan3d
# find all files that are in the directory
- find:
paths: "/Users/asteen/Downloads/sites/"
file_type: directory
register: allDirs
# delete all files that are older than three days, but keep a minimum of 10 files
- file:
path: "{{ item.path }}"
state: absent
with_items: "{{ dirsOlderThan3d.files }}"
when: allDirs.files > 10 and not item[0].exists ... item[9].exists
You just have to filter your list of files older than 3 days:
---
- hosts: all
tasks:
- name: find all files that are older than three
find:
paths: "/Users/asteen/Downloads/sites/"
age: "3d"
file_type: directory
register: dirsOlderThan3d
- name: remove older than 3 days but first ten newest
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ (dirsOlderThan3d.files | sort(attribute='ctime'))[:-10] | list }}"
I've been previously using a cronjob with find and decided to move to AWX and after checking here and other articles, I've come up with the following. Tested and working as we speak. First task registers all files older than 3 days as being matched_files_dirs. Second task removes them. Does the job but is slower than just running cron on linux:
find /opt/system*/target_directories -type f -mtime +3 -exec rm {} \;
---
- name: Cleanup
hosts: linux
gather_facts: false
tasks:
- name: Collect files
shell: find /opt/system*/target_directories -type f -mtime +3
register: matched_files_dirs
- name: Remove files
become_user: root
file:
path: "{{ item }}"
state: absent
with_items: "{{ matched_files_dirs.stdout_lines }}"
I'm trying to write a playbook that will rsync the folders from source to target after a database refresh. Our Peoplesoft HR application also requires a filesystem refresh along with database. I'm new to ansible and not an expert with python. I've written this but my playbook fails if any of the with_items doesn't exist. I'd like to use this playbook for all apps and the folders may differ between apps. How can I skip the folders that doesn't exist in source. I'm passing {{ target }} at command line.
---
- hosts: '<hostname>'
remote_user: <user>
tasks:
- shell: ls -l /opt/custhome/prod/
register: folders
- name: "Copy PROD filesystem to target"
synchronize:
src: "/opt/custhome/prod/{{ item }}"
dest: "/opt/custhome/dev/"
delete: yes
when: "{{ folders == item }}"
with_items:
- 'src/cbl/'
- 'sqr/'
- 'bin/'
- 'NVISION/'
In this case, NVISION doesn't exist in HR app but it does in FIN app. But the playbook is failing coz that folder doesn't exist in source.
You can use find module to find and store paths to source folders and then to iterate over results. Example playbook:
- hosts: '<hostname>'
remote_user: <user>
tasks:
- name: find all directories
find:
file_type: directory
paths: /opt/custhome/prod/
patterns:
- "src"
- "sqr"
- "bin"
register: folders
#debug to understand contents of {{ folders }} variable
# - debug: msg="{{ folders }}"
- name: "Copy PROD filesystem to target"
synchronize:
src: "{{ item.path }}"
dest: "/opt/custhome/dev/"
delete: yes
with_items: "{{ folders.files }}"
You may want to use recurse to descend into subdirectories and use_regex to use the power of python regex instead of shell globbing.
The below code only deletes the first file it gets inside the web dir. I want to remove all the files and folders inside the web directory and retain the web directory. How can I do that?
- name: remove web dir contents
file: path='/home/mydata/web/{{ item }}' state=absent
with_fileglob:
- /home/mydata/web/*
Note: I've tried rm -rf using command and shell, but they don't work. Perhaps I am using them wrongly.
Any help in the right direction will be appreciated.
I am using ansible 2.1.0.0
- name: Delete content & directory
file:
state: absent
path: /home/mydata/web/
Note: this will delete the directory too.
Remove the directory (basically a copy of https://stackoverflow.com/a/38201611/1695680), Ansible does this operation with rmtree under the hood.
- name: remove files and directories
file:
state: "{{ item }}"
path: "/srv/deleteme/"
owner: 1000 # set your owner, group, and mode accordingly
group: 1000
mode: '0777'
with_items:
- absent
- directory
If you don't have the luxury of removing the whole directory and recreating it, you can scan it for files, (and directories), and delete them one by one. Which will take a while. You probably want to make sure you have [ssh_connection]\npipelining = True in your ansible.cfg on.
- block:
- name: 'collect files'
find:
paths: "/srv/deleteme/"
hidden: True
recurse: True
# file_type: any # Added in ansible 2.3
register: collected_files
- name: 'collect directories'
find:
paths: "/srv/deleteme/"
hidden: True
recurse: True
file_type: directory
register: collected_directories
- name: remove collected files and directories
file:
path: "{{ item.path }}"
state: absent
with_items: >
{{
collected_files.files
+ collected_directories.files
}}
Using shell module (idempotent too):
- shell: /bin/rm -rf /home/mydata/web/*
If there are dot/hidden files:
- shell: /bin/rm -rf /home/mydata/web/* /home/mydata/web/.*
Cleanest solution if you don't care about creation date and owner/permissions:
- file: path=/home/mydata/web state=absent
- file: path=/home/mydata/web state=directory
I really didn't like the rm solution, also ansible gives you warnings about using rm.
So here is how to do it without the need of rm and without ansible warnings.
- hosts: all
tasks:
- name: Ansible delete file glob
find:
paths: /etc/Ansible
patterns: "*.txt"
register: files_to_delete
- name: Ansible remove file glob
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"
source: http://www.mydailytutorials.com/ansible-delete-multiple-files-directories-ansible/
try the below command, it should work
- shell: ls -1 /some/dir
register: contents
- file: path=/some/dir/{{ item }} state=absent
with_items: {{ contents.stdout_lines }}
That's what I come up with:
- name: Get directory listing
find:
path: "{{ directory }}"
file_type: any
hidden: yes
register: directory_content_result
- name: Remove directory content
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ directory_content_result.files }}"
loop_control:
label: "{{ item.path }}"
First, we're getting directory listing with find, setting
file_type to any, so we wouldn't miss nested directories and links
hidden to yes, so we don't skip hidden files
also, do not set recurse to yes, since it is not only unnecessary, but may increase execution time.
Then, we go through that list with file module. It's output is a bit verbose, so loop_control.label will help us with limiting output (found this advice here).
But I found previous solution to be somewhat slow, since it iterates through the content, so I went with:
- name: Get directory stats
stat:
path: "{{ directory }}"
register: directory_stat
- name: Delete directory
file:
path: "{{ directory }}"
state: absent
- name: Create directory
file:
path: "{{ directory }}"
state: directory
owner: "{{ directory_stat.stat.pw_name }}"
group: "{{ directory_stat.stat.gr_name }}"
mode: "{{ directory_stat.stat.mode }}"
get directory properties with the stat
delete directory
recreate directory with the same properties.
That was enough for me, but you can add attributes as well, if you want.
Using file glob also it will work. There is some syntax error in the code you posted. I have modified and tested this should work.
- name: remove web dir contents
file:
path: "{{ item }}"
state: absent
with_fileglob:
- "/home/mydata/web/*"
Following up on the most upvoted answer here (which I cannot edit since "edit queue is full"):
- name: Delete content & directory
file:
state: absent
path: /home/mydata/web/
- name: Re-create the directory
file:
state: directory
path: /home/mydata/web/
While Ansible is still debating to implement state = empty
https://github.com/ansible/ansible-modules-core/issues/902
my_folder: "/home/mydata/web/"
empty_path: "/tmp/empty"
- name: "Create empty folder for wiping."
file:
path: "{{ empty_path }}"
state: directory
- name: "Wipe clean {{ my_folder }} with empty folder hack."
synchronize:
mode: push
#note the backslash here
src: "{{ empty_path }}/"
dest: "{{ nl_code_path }}"
recursive: yes
delete: yes
delegate_to: "{{ inventory_hostname }}"
Note though, with synchronize you should be able to sync your files (with delete) properly anyway.
Created an overall rehauled and fail-safe implementation from all comments and suggestions:
# collect stats about the dir
- name: check directory exists
stat:
path: '{{ directory_path }}'
register: dir_to_delete
# delete directory if condition is true
- name: purge {{directory_path}}
file:
state: absent
path: '{{ directory_path }}'
when: dir_to_delete.stat.exists and dir_to_delete.stat.isdir
# create directory if deleted (or if it didn't exist at all)
- name: create directory again
file:
state: directory
path: '{{ directory_path }}'
when: dir_to_delete is defined or dir_to_delete.stat.exist == False
Below code worked for me :
- name: Get directory listing
become: yes
find:
paths: /applications/cache
patterns: '*'
hidden: yes
register: directory_content_result
- name: Remove directory content
become: yes
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ directory_content_result.files }}"
There is an issue open with respect to this.
For now, the solution works for me: create a empty folder locally and synchronize it with the remote one.
Here is a sample playbook:
- name: "Empty directory"
hosts: *
tasks:
- name: "Create an empty directory (locally)"
local_action:
module: file
state: directory
path: "/tmp/empty"
- name: Empty remote directory
synchronize:
src: /tmp/empty/
dest: /home/mydata/web/
delete: yes
recursive: yes
I want to make sure that the find command only deletes everything inside the directory and leave the directory intact because in my case the directory is a filesystem. The system will generate an error when trying to delete a filesystem but that is not a nice option. Iam using the shell option because that is the only working option I found so far for this question.
What I did:
Edit the hosts file to put in some variables:
[all:vars]
COGNOS_HOME=/tmp/cognos
find=/bin/find
And create a playbook:
- hosts: all
tasks:
- name: Ansible remove files
shell: "{{ find }} {{ COGNOS_HOME }} -xdev -mindepth 1 -delete"
This will delete all files and directories in the COGNOS_HOME variable directory/filesystem. The "-mindepth 1" option makes sure that the current directory will not be touched.
I have written an custom ansible module to cleanup files based on multiple filters like age, timestamp, glob patterns, etc.
It is also compatible with ansible older versions. It can be found here.
Here is an example:
- cleanup_files:
path_pattern: /tmp/*.log
state: absent
excludes:
- foo*
- bar*
Just a small cleaner copy & paste template of ThorSummoners answer, if you are using Ansible >= 2.3 (distinction between files and dirs not necessary anymore.)
- name: Collect all fs items inside dir
find:
path: "{{ target_directory_path }}"
hidden: true
file_type: any
changed_when: false
register: collected_fsitems
- name: Remove all fs items inside dir
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ collected_fsitems.files }}"
when: collected_fsitems.matched|int != 0
Isn't it that simple ... tested working ..
eg.
---
- hosts: localhost
vars:
cleandir: /var/lib/cloud/
tasks:
- shell: ls -a -I '.' -I '..' {{ cleandir }}
register: ls2del
ignore_errors: yes
- name: Cleanup {{ cleandir }}
file:
path: "{{ cleandir }}{{ item }}"
state: absent
with_items: "{{ ls2del.stdout_lines }}"
- name: Files to delete search
find:
paths: /home/mydata/web/
file_type: any
register: files_to_delete
- name: Deleting files to delete
file:
path: '{{ item.path }}'
state: absent
with_items: "{{ files_to_delete.files }}"
I like the following solution:
- name: remove web dir contents
command:
cmd: "find . -path '*/*' -delete -print"
chdir: "/home/mydata/web/"
register: web_files_list
changed_when: web_files_list.stdout | length > 0
because it is:
simple
idempotent
fast
Assuming you are always in Linux, try the find cmd.
- name: Clean everything inside {{ item }}
shell: test -d {{ item }} && find {{ item }} -path '{{ item }}/*' -prune -exec rm -rf {} \;
with_items: [/home/mydata/web]
This should wipe out files/folders/hidden under /home/mydata/web
- name: delete old data and clean cache
file:
path: "{{ item[0] }}"
state: "{{ item[1] }}"
with_nested:
- [ "/data/server/{{ app_name }}/webapps/", "/data/server/{{ app_name }}/work/" ]
- [ "absent", "directory" ]
ignore_errors: yes
Below worked for me,
- name: Ansible delete html directory
file:
path: /var/www/html
state: directory