In my Ansible Playbook I'd like to have a task that removes old files and folders from the application's directory. The twist to this otherwise simple task is that a few files or folders need to remain. Imagine something like this:
/opt/application
- /config
- *.properties
- special.yml
- /logs
- /bin
- /var
- /data
- /templates
Let's assume I'd like to keep /logs completely, /var/data and from /config I want to keep special.yml.
(I cannot provide exact code at the moment because I left work frustrated by this and, after cooling down, I am now writing up this question at home)
My idea was to have two lists of exclusions, one holding the folders and one the file. Then I use the find module to first get the folders in the application's directory into a variable and the same for the remaining files into another variable. Afterwards I wanted to remove every folder and file that are not in the lists of exclusions using the file module.
(Pseudo-YML because I'm not yet fluent enough in Ansible that I can whip up a properly structured example; it should be close enough though)
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ found_files_list.files }}"
when: well, that is the big question
What I can't figure out is how to properly construct the when clause. Is it even possible like this?
I don't believe there is a when clause with the file module.
But you can probably achieve what you need as follows:
- name: Find /opt/application all directories, exclude logs, data, and config
find:
paths: /opt/application
excludes: 'logs,data,config'
register: files_to_delete
- name: Ansible remove file glob
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"
I hope this is what you need.
First use the find module like you said to get a total list of all files and directories. Register to a variable like all_objects.
- name: Get list of all files recursively
find:
path: /opt/application/
recurse: yes
register: all_objects
Then manually make a list of things you want to keep.
vars:
keep_these:
- /logs
- /var/data
- /config/special.yml
Then this task should delete everything except things in your list:
- name: Delete all files and directories except exclusions
file:
path: "{{ item.path }}"
state: absent
recurse: true
with_items: "{{ all_objects.files }}"
when: item.path not in keep_these
I think this general strategy should work... only thing I'm not sure about is the exact nesting heiararchy of the registered variable from the find module. You might have to play around with the debug module to get it exactly right.
Related
A couple of tasks in my playbook keep finding and modifying files in hidden directories--I do not want the hidden files to be altered.
ansible-core 2.13.7
- name: Find all script files
find:
recurse: yes
paths: /var/bbb
patterns:
- "*.sh"
- "*.py"
- "*.env"
register: script_files
- name: Make all scripts executable
file:
dest: "{{ item.path }}"
mode: "a+rx"
with_items: "{{ script_files.files }}"
Sample of output from second task:
ok: [localhost] => (item={'path': '/var/bbb/.ansible/collections/ansible_collections/community/docker/tests/unit/plugins/module_utils/test_util.py', 'mode': '0755', ...
I've gone through the docs and they say hidden files are left alone by default in the current and former versions. I suspect it may be the patterns, but again the docs say the pattern is compared to the file base name and excludes the directory.
Regarding your observation
Ansible find module collecting files in hidden directories
that's right and the expected behavior.
... they say hidden files are left alone by default in the current and former versions.
I interpret the documentation about Parameter: hidden that it applies to files only and not to directories.
A minimal test setup
mdkir .hidden
touch .hidden/script.sh
touch .hidden/.script.sh
with an example playbook
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Find all script files
find:
hidden: false
recurse: true
paths: "/home/{{ ansible_user }}/"
patterns:
- "*script.sh"
register: result
- name: Show result
debug:
msg: "{{ result.files }}"
will result into an output of
...
path: /home/user/.hidden/script.sh
...
or if hidden: true
...
path: /home/user/.hidden/script.sh
...
path: /home/user/.hidden/.script.sh
...
Similar Q&A
Using Ansible find module to get hidden folders
To summarize, your question seems to be about
How to exclude hidden directories from Ansible find module recursive search?
Since with the parameter hidden: false and file_type: any still the /.hidden/script.sh is found, the excludes parameter operates on basenames of files, I recommend to adjust the search path.
Given the useful explanation in the previous answer and the complexity of the search path solution in my use case, I looked beyond the parameters of find and developed a separate task that removes results that contain hidden directories:
- name: filter hidden directories
set_fact:
filtered_list: "{{result.files | map(attribute='path') | reject('search', '/\\.')}}"
Be aware that this will remove all paths which contain a hidden directory: it should not be used when the find task is performed on paths that contain a hidden directory.
How to delete the oldest directory with ansible.
suppose I have the following tree structure
Parent Directory
-Dir2020-05-20
-Dir2020-05-21
-Dir2020-05-22
-Dir2020-05-23
now every time an ansible playbook is run, it should delete the oldest directory, For e.g it should delete Dir2020-05-20 in its first run if we consider its creation date to be 2020-05-20.
age attribute of file module does not seen helpful as i have to run this playbook very randomly and i want to keep limited no. of these directories.
Just assign dirpath to the path of your "Parent Directory" where all these directories are present
---
- hosts: localhost
vars:
dir_path: "/home/harshit/ansible/test/" ##parent directory path, make sure it ends with a slash
tasks:
- name: find oldest directory
shell:
cmd: "ls `ls -tdr | head -n 1 `"
chdir: "{{dir_path}}"
register: dir_name_to_delete
- name: "delete oldest directory: {{dir_path}}{{dir_name_to_delete.stdout}}"
file:
state: absent
path: "{{dir_path}}{{dir_name_to_delete.stdout}}"
Considering a recommended practice is not to use shell or command modules wherever possible I suggest a pure ansible solution for this case:
- name: Get directory list
find:
paths: "{{ target_directory }}"
file_type: directory
register: found_dirs
- name: Get the oldest dir
set_fact:
oldest_dir: "{{ found_dirs.files | sort(attribute='mtime') | first }}"
- name: Delete oldest dir
file:
state: absent
path: "{{ oldest_dir.path }}"
when:
- found_dirs.files | count > 3
There are two ways to know how many files were found with find module - either using its return value matched like this when: found_dirs.matched > 3 or using count filter. I prefer the latter method because I just use this filter in a lot of other cases so this is just a habit.
For your reference, ansible has a whole bunch of useful filters (e.g. I used count and sort here, but there are dozens of them). One does not need to remember those filter names, of course, just keep in mind they exist and might be useful in many cases.
I'm currently trying to get used to Ansible but I'm failing to achieve what seems to be a common use-case:
Lets say I have have a role nginx in roles/nginx and and one task is to setup a custom default page:
- name: install nginx default page
copy:
src: "index.html"
dest: /var/www/html/
owner: root
mode: 0644
Ansible will look for the file in:
roles/nginx/files
roles/nginx
roles/nginx/tasks/files
roles/nginx/tasks
files
./
Now for some reason a single host should receive a completely different file.
I know I could alter the file src path to src: "{{ inventory_hostname }}/index.html" but then it would search in host-specific directories only.
Is there a way to alter the search paths so that Ansible will look for files in host-specific directories first but fall-back to common directories?
I don't want to decide if files might need to be host-specific when writing roles. I'd rather like to overwrite the role default files without altering the base role at all.
Q: "Is there a way to alter the search paths so that Ansible will look for files in host-specific directories first but fall back to common directories?"
A: In general, it is not possible to change the search paths. But, with first_found it is possible to define how a specific file shall be searched. For example,
- copy:
src: "{{ lookup('first_found', findme) }}"
dest: /scratch/tmp/
owner: root
mode: 0644
vars:
findme:
- "{{ inventory_hostname }}/index.html"
- "{{ role_path }}/files/index.html"
- "{{ role_path }}/files/defaults/index.html"
I have an ansible playbook that I need to add a task to. Essentially I have a directory that on each deploy adds another subdirectory. When there are more than 3 subdirectories I want to delete all of the older directories past the 3 most recent directories. I am having a very hard time trying to write a play for it.
I first try to get all the subdirectories
- name: Get all install artifacts
find:
paths: "{{ home_path }}/install/"
file_type: directory
recurse: no
register: install_artifacts
But after this I am trying to check if there are more than 3 items, and if so get the 3rd items modified date, and delete everything with a modified date earlier than this.
- block:
- name: Determine old directories
set_fact:
old_dirs: "{{ (install_artifacts.files|sort(attribute='mtime', reverse=True))[3:] }}"
- name: Remove old directories
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ old_dirs }}"
when: install_artifacts.matched > 3
First of all, install_artifacts.files|sort(attribute='mtime', reverse=True) will sort the list of dicts by mtime in descending order. [3:] means removing the first 3 items from the list, which are the 3 most recent directories. So old_dirs now contains all the old directories.
For some weird reasons I'm having troubles with a simple task which is copying a content of the folder myfiles (few files in there) to the dist/myfiles location. Task looks like this:
name: Deploy config files like there is no tomorrow
copy:
src: "{{ item }}"
dest: "/home/{{ ansible_user_id }}/dist/{{ item }}"
with_items:
- 'config'
- 'myfiles/'
myfiles folder exist under the dist and config file is copied to the dist folder.
Is this possible in Ansible or I should copy each file separately? Am I doing it completely wrong?
Your task copies both: the config file and the myfiles on Debian and CentOS targets properly.
If for some reason you have a problem, you might have a look at Looping over Fileglobs.
You need to split the task into two, with the second one looking like:
- name: Deploy multiple config files
copy:
src: "{{ item }}"
dest: "/home/{{ ansible_user_id }}/dist/myfiles/{{ item | basename }}"
with_fileglob:
- /path/to/myfiles/*
For a recursive copy, check this question on SeverFault
Alternatively, you could use the synchronize module, but pay special attention when using become. See this question on SuperUser.