How to delete the oldest directory with ansible.
suppose I have the following tree structure
Parent Directory
-Dir2020-05-20
-Dir2020-05-21
-Dir2020-05-22
-Dir2020-05-23
now every time an ansible playbook is run, it should delete the oldest directory, For e.g it should delete Dir2020-05-20 in its first run if we consider its creation date to be 2020-05-20.
age attribute of file module does not seen helpful as i have to run this playbook very randomly and i want to keep limited no. of these directories.
Just assign dirpath to the path of your "Parent Directory" where all these directories are present
---
- hosts: localhost
vars:
dir_path: "/home/harshit/ansible/test/" ##parent directory path, make sure it ends with a slash
tasks:
- name: find oldest directory
shell:
cmd: "ls `ls -tdr | head -n 1 `"
chdir: "{{dir_path}}"
register: dir_name_to_delete
- name: "delete oldest directory: {{dir_path}}{{dir_name_to_delete.stdout}}"
file:
state: absent
path: "{{dir_path}}{{dir_name_to_delete.stdout}}"
Considering a recommended practice is not to use shell or command modules wherever possible I suggest a pure ansible solution for this case:
- name: Get directory list
find:
paths: "{{ target_directory }}"
file_type: directory
register: found_dirs
- name: Get the oldest dir
set_fact:
oldest_dir: "{{ found_dirs.files | sort(attribute='mtime') | first }}"
- name: Delete oldest dir
file:
state: absent
path: "{{ oldest_dir.path }}"
when:
- found_dirs.files | count > 3
There are two ways to know how many files were found with find module - either using its return value matched like this when: found_dirs.matched > 3 or using count filter. I prefer the latter method because I just use this filter in a lot of other cases so this is just a habit.
For your reference, ansible has a whole bunch of useful filters (e.g. I used count and sort here, but there are dozens of them). One does not need to remember those filter names, of course, just keep in mind they exist and might be useful in many cases.
Related
This question already has answers here:
How to move/rename a file using an Ansible task on a remote system
(13 answers)
Closed 1 year ago.
So i have been trying to fix a mistake i did in all the servers by using a playbook. Basicly i launched a playbook with logrotate to fix the growing logs problem, and in there is a log named btmp, which i wasnt supposed to rotate but did anyway by accident, and now logrotate changed its name to add a date to it and therefore braking the log. Now i want to use a playbook that will find a log named btmp in /var/log directory and rename it back, problem is that the file atm is different in each server for example 1 server has btmp-20210316 and the other has btmp-20210309, so in bash command line one would use wildcard "btmp*" to bypass thos problem, however this does not appear to work in playbook. So far i came up with this:
tasks:
- name: stat btmp*
stat: path=/var/log
register: btmp_stat
- name: Move btmp
command: mv /var/log/btmp* /var/log/btmp
when: btmp_stat.stat.exists
However this results in error that the file was not found. So my question is how does one get the wildcard working in playbook or is there an equivalent way to find all files that have "btmp" in their names and rename them ? BTW all servers are Centos 7 servers.
So i will add my own solution aswell, even tho the answer solution is better.
Make a bash script with a single line, anywhere in you ansible VM.
Line is : mv /var/log/filename* /var/log/filename
And now create a playbook to operate this in target VM:
---
- hosts: '{{ server }}'
remote_user: username
become: yes
become_method: sudo
vars_prompt:
- name: "server"
prompt: "Enter server name or group"
private: no
tasks:
- name: Move the script to target host VM
copy: src=/anywhereyouwant/bashscript.sh dest=/tmp mode=0777
- name: Execute the script
command: sh /tmp/bashscript.sh
- name: delete the script
command: rm /tmp/bashscript.sh
There's more than one way to do this in Ansible, and using the shell module is certainly a viable way to do it (but you would need the shell module in place of command as the latter does not support wildcards). I would solve the problem as follows:
First create a task to find all matching files (i.e. /var/log/btmp*) and store them in a variable for later processing - this would look like this:
- name: Find all files named /var/log/btmp*
ansible.builtin.find:
paths: /var/log
patterns: 'btmp*'
register: find_btmp
This task uses the find module to locate all files called btmp* in /var/log - the results are stored in a variable called find_btmp.
Next create a task to copy the btmp* file to btmp. Now you may very well have more than 1 file pathing the above pattern, and logically you don't want to rename them all to btmp as this simply keeps overwriting the file every time. Instead, let's assume you want only the newest file that you matched - we can use a clever Jinja2 filter to get this entry from the results of the first task:
- name: Copy the btmp* to the required filename
ansible.builtin.copy:
src: "{{ find_btmp.files | sort(attribute='mtime',reverse=true) | map(attribute='path') | first }}"
dest: /var/log/btmp
remote_src: yes
when: find_btmp.failed == false
This task uses Ansible's copy module to copy our chosen source file to /var/log/btmp. The remote_src: yes parameter tells the copy module that the source file exists on the remote machine rather than the Ansible host itself.
We use a when clause to ensure that we don't run this copy operation if we failed to find any files.
Now let's break down that Jinja2 filter:
find_btmp.files - this is all of the files listed in our find_btmp variable
sort(attribute='mtime',reverse=true) - here we are sorting our list of files using the mtime (modification time) attribute - we're reverse sorting so that the newest entry is at the top of the list.
map(attribute='path') - we're using map to "extract" the path attribute of the files dictionary, as this is the only data we actually want to pass to the copy module - the path of the file itself
first - this selects only the first element in the list (i.e. the newest file as they were reverse sorted)
Finally, you asked for a move operation - there's no native "move" module in Ansible so you will want to remove the source file after the copy - this can be done as follows (the Jinja2 filter is the same as before:
- name: Delete the original file
ansible.builtin.file:
path: "{{ find_btmp.files | sort(attribute='mtime',reverse=true) | map(attribute='path') | first }}"
state: absent
when: find_btmp.failed == false
Again we use a when clause to ensure we don't delete anything if we didn't find it in the first place.
I have tested this on Ansible 3.1.0/ansible-base 2.10.7 - if you're running Ansible 2.9 or earlier, remove the ansible.builtin. from the module names (i.e. ansible.builtin.copy becomes copy.)
Hope this helps you out!
In my Ansible Playbook I'd like to have a task that removes old files and folders from the application's directory. The twist to this otherwise simple task is that a few files or folders need to remain. Imagine something like this:
/opt/application
- /config
- *.properties
- special.yml
- /logs
- /bin
- /var
- /data
- /templates
Let's assume I'd like to keep /logs completely, /var/data and from /config I want to keep special.yml.
(I cannot provide exact code at the moment because I left work frustrated by this and, after cooling down, I am now writing up this question at home)
My idea was to have two lists of exclusions, one holding the folders and one the file. Then I use the find module to first get the folders in the application's directory into a variable and the same for the remaining files into another variable. Afterwards I wanted to remove every folder and file that are not in the lists of exclusions using the file module.
(Pseudo-YML because I'm not yet fluent enough in Ansible that I can whip up a properly structured example; it should be close enough though)
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ found_files_list.files }}"
when: well, that is the big question
What I can't figure out is how to properly construct the when clause. Is it even possible like this?
I don't believe there is a when clause with the file module.
But you can probably achieve what you need as follows:
- name: Find /opt/application all directories, exclude logs, data, and config
find:
paths: /opt/application
excludes: 'logs,data,config'
register: files_to_delete
- name: Ansible remove file glob
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"
I hope this is what you need.
First use the find module like you said to get a total list of all files and directories. Register to a variable like all_objects.
- name: Get list of all files recursively
find:
path: /opt/application/
recurse: yes
register: all_objects
Then manually make a list of things you want to keep.
vars:
keep_these:
- /logs
- /var/data
- /config/special.yml
Then this task should delete everything except things in your list:
- name: Delete all files and directories except exclusions
file:
path: "{{ item.path }}"
state: absent
recurse: true
with_items: "{{ all_objects.files }}"
when: item.path not in keep_these
I think this general strategy should work... only thing I'm not sure about is the exact nesting heiararchy of the registered variable from the find module. You might have to play around with the debug module to get it exactly right.
I have an ansible playbook that I need to add a task to. Essentially I have a directory that on each deploy adds another subdirectory. When there are more than 3 subdirectories I want to delete all of the older directories past the 3 most recent directories. I am having a very hard time trying to write a play for it.
I first try to get all the subdirectories
- name: Get all install artifacts
find:
paths: "{{ home_path }}/install/"
file_type: directory
recurse: no
register: install_artifacts
But after this I am trying to check if there are more than 3 items, and if so get the 3rd items modified date, and delete everything with a modified date earlier than this.
- block:
- name: Determine old directories
set_fact:
old_dirs: "{{ (install_artifacts.files|sort(attribute='mtime', reverse=True))[3:] }}"
- name: Remove old directories
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ old_dirs }}"
when: install_artifacts.matched > 3
First of all, install_artifacts.files|sort(attribute='mtime', reverse=True) will sort the list of dicts by mtime in descending order. [3:] means removing the first 3 items from the list, which are the 3 most recent directories. So old_dirs now contains all the old directories.
Is possible to edit find module command to find X latest folders inside remotebackup dir? Thanks
- find: paths="{{ remotebackupdir }}" age="**find_oldest_folders_except_X_newer**" recurse=no
register: result
- debug: var=result.files
- file: path: "{{ remotebackupdir }}" state: absent
with_items: result.files
Ive done something similar.
Try something like this: (to be executed in the directory where the files are that you want to delete)
- name: Get first 4 files
command: ls -trd -1 {{path}}/* | head -4
register: filenames
- name: actually delete files
file: {{path}}/{{ item }} state=absent
with_items: filenames.stdout_lines
Basically -tr lists things in reverse date order, and -1 ensures everything is on one line, and -d says directories only
I have a VPS sever where I deploy frequently releases and the dir structure is that I have a current dir, what is a symlink to an actual release under a releases dir.
How can I achieve, that only X ( in my case 3) releases stay in the releases dir, the rest can be deleted, to spare free HDD and because I don't need them any more.
This setup is what capifony uses.
Something along this lines should work, where bin would be your symlinked directory :
- name: Set timestamp
set_fact: release_timestamp="{{ansible_date_time.epoch}}"
- name: Deploy code from repository
action: git repo={{repo_url}} dest={{app_dir}}/releases/{{release_timestamp}} remote={{repo_remote}} version={{branch}}
- name: Create symlink to bin folder
file: src={{app_dir}}/releases/{{release_timestamp}} dest={{app_dir}}/bin state=link
- name: List old releases and clean them up
shell: "ls -t {{app_dir}}/releases | tail -n +{{releases_to_keep + 1}}"
register: ls_output
- file: name={{app_dir}}/releases/{{ item }} state=absent
with_items: ls_output.stdout_lines
If you want to do rollback:
- name: Find the previous release
shell: "ls -t {{app_dir}}/releases | head -2 | tail -1"
register: ls_output
- name: Create symlink to previous release folder
file: src={{app_dir}}/releases/{{item}} dest={{app_dir}}/bin state=link
with_items: ls_output.stdout_lines
This is an old topic but I think this answer will be helpful.
- name: Find all directories in releases folder
find:
paths="{{ app_dir }}/{{ releases }}"
file_type=directory
register: dirs
- name: Only last three releases
file:
path="{{ item.path }}"
state=absent
with_items:
- "{{ (dirs.files | sort(attribute='ctime'))[:-3] }}"
Perhaps a bit off-topic, but it will give you a place to start:
There is an Ansible role in the galaxy specifically pointed at replacing Capistrano/Capifony. With an example for a Symfony2 project even :-)
https://galaxy.ansible.com/list#/roles/732
Example usage:
https://github.com/SweetLakePHP/SweetLakePHP/blob/master/ansible/deploy.yml
To answer your question more specifically:
The loop required to clean the releases folder would be either a shell script or a composition of Ansible tasks (saving ls output, sorting and removing). To make this a smoother process, we decided to place this logic in an Ansible module. This module is used in the project_deploy role linked above. But if you prefer to wrote your own role, the module was extracted and placed in ot's own galaxy role:
https://galaxy.ansible.com/list#/roles/2266
Finally, if you're interested in a deeper look at why we came to the current construction:
http://future500.nl/articles/2014/07/thoughts-on-deploying-with-ansible/