I am new to Ansible.
I have such a directory:
home/etc/directory/
Which contains the following files:
application.yml
application.yml_new
application.yml.12345
etc.yml
What I want to do is to delete/remove the files
application.yml_new
application.yml.12345
I do not want to explicitly name these files but I want to delete all files with a suffix additional to application.yml (since the names are slightly different depending on the application).
Now I am wondering how to do this?
I found the file but I am not sure if I can do this with it or can it be only done with the shell-module?
Q: "Delete all files with a suffix additional to application.yml"
A: Find the files first. For example, given the tree
shell> tree etc/directory
etc/directory/
├── application.yml
├── application.yml.12345
├── application.yml_new
└── etc.yml
0 directories, 4 files
The module find does the job
- find:
paths: etc/directory
patterns: '^application\.yml.+$'
use_regex: true
register: result
gives
result.files|map(attribute='path')|list:
- etc/directory/application.yml.12345
- etc/directory/application.yml_new
Iterate the list and delete the files
- file:
state: absent
path: "{{ item }}"
loop: "{{ result.files|map(attribute='path')|list }}"
gives
shell> tree etc/directory
etc/directory/
├── application.yml
└── etc.yml
0 directories, 2 files
Explanation of the Python regex
patterns: '^application\.yml.+$'
^ ............. matches the beginning of the string
application ... matches the string 'application'
\. ............ matches dot '.'; must be escaped because of dot matches any character
yml ........... matches the string 'yml'
.+ ............ matches one or more of any characters
$ ............. matches the end of the string
One way to achieve this is to use the find command in combination with the file module in ansible.
Like this:
- name: getfiles
shell: "find * -name 'application.yml*' ! -name application.yml"
register: files_to_delete
args:
chdir: "/home/admin"
- name: delete files
file:
path: "/home/admin/{{item}}"
state: absent
with_items: "{{files_to_delete.stdout_lines}}"
First, we search for the files by using the find command and register the results in a variable.
Then we are able to use the stdout lines which is an array of the filenames we want to delete.
NOTE: Because the find command also includes the application.yml I exclude it. As seen in the example above
At last, we can loop over the list and delete the files.
Related
A couple of tasks in my playbook keep finding and modifying files in hidden directories--I do not want the hidden files to be altered.
ansible-core 2.13.7
- name: Find all script files
find:
recurse: yes
paths: /var/bbb
patterns:
- "*.sh"
- "*.py"
- "*.env"
register: script_files
- name: Make all scripts executable
file:
dest: "{{ item.path }}"
mode: "a+rx"
with_items: "{{ script_files.files }}"
Sample of output from second task:
ok: [localhost] => (item={'path': '/var/bbb/.ansible/collections/ansible_collections/community/docker/tests/unit/plugins/module_utils/test_util.py', 'mode': '0755', ...
I've gone through the docs and they say hidden files are left alone by default in the current and former versions. I suspect it may be the patterns, but again the docs say the pattern is compared to the file base name and excludes the directory.
Regarding your observation
Ansible find module collecting files in hidden directories
that's right and the expected behavior.
... they say hidden files are left alone by default in the current and former versions.
I interpret the documentation about Parameter: hidden that it applies to files only and not to directories.
A minimal test setup
mdkir .hidden
touch .hidden/script.sh
touch .hidden/.script.sh
with an example playbook
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Find all script files
find:
hidden: false
recurse: true
paths: "/home/{{ ansible_user }}/"
patterns:
- "*script.sh"
register: result
- name: Show result
debug:
msg: "{{ result.files }}"
will result into an output of
...
path: /home/user/.hidden/script.sh
...
or if hidden: true
...
path: /home/user/.hidden/script.sh
...
path: /home/user/.hidden/.script.sh
...
Similar Q&A
Using Ansible find module to get hidden folders
To summarize, your question seems to be about
How to exclude hidden directories from Ansible find module recursive search?
Since with the parameter hidden: false and file_type: any still the /.hidden/script.sh is found, the excludes parameter operates on basenames of files, I recommend to adjust the search path.
Given the useful explanation in the previous answer and the complexity of the search path solution in my use case, I looked beyond the parameters of find and developed a separate task that removes results that contain hidden directories:
- name: filter hidden directories
set_fact:
filtered_list: "{{result.files | map(attribute='path') | reject('search', '/\\.')}}"
Be aware that this will remove all paths which contain a hidden directory: it should not be used when the find task is performed on paths that contain a hidden directory.
How to delete the oldest directory with ansible.
suppose I have the following tree structure
Parent Directory
-Dir2020-05-20
-Dir2020-05-21
-Dir2020-05-22
-Dir2020-05-23
now every time an ansible playbook is run, it should delete the oldest directory, For e.g it should delete Dir2020-05-20 in its first run if we consider its creation date to be 2020-05-20.
age attribute of file module does not seen helpful as i have to run this playbook very randomly and i want to keep limited no. of these directories.
Just assign dirpath to the path of your "Parent Directory" where all these directories are present
---
- hosts: localhost
vars:
dir_path: "/home/harshit/ansible/test/" ##parent directory path, make sure it ends with a slash
tasks:
- name: find oldest directory
shell:
cmd: "ls `ls -tdr | head -n 1 `"
chdir: "{{dir_path}}"
register: dir_name_to_delete
- name: "delete oldest directory: {{dir_path}}{{dir_name_to_delete.stdout}}"
file:
state: absent
path: "{{dir_path}}{{dir_name_to_delete.stdout}}"
Considering a recommended practice is not to use shell or command modules wherever possible I suggest a pure ansible solution for this case:
- name: Get directory list
find:
paths: "{{ target_directory }}"
file_type: directory
register: found_dirs
- name: Get the oldest dir
set_fact:
oldest_dir: "{{ found_dirs.files | sort(attribute='mtime') | first }}"
- name: Delete oldest dir
file:
state: absent
path: "{{ oldest_dir.path }}"
when:
- found_dirs.files | count > 3
There are two ways to know how many files were found with find module - either using its return value matched like this when: found_dirs.matched > 3 or using count filter. I prefer the latter method because I just use this filter in a lot of other cases so this is just a habit.
For your reference, ansible has a whole bunch of useful filters (e.g. I used count and sort here, but there are dozens of them). One does not need to remember those filter names, of course, just keep in mind they exist and might be useful in many cases.
I have a following directory structure:
parent_dir/
├── subdir_1
├── subdir_2
└── subdir_3
The subdirs don't have a fixed name and there can be an arbitrary number of them.
How to make ansible run a task for each sub directory?
(any task will do, eventually every dir will be a python package to install, but that isn't important for the context of this question)
This is the solution I managed to come up with, perhaps there is a cleaner way with lookups to achieve this in a single task.
Copy pasting the following code will create a directory structure with a minimal ansible playbook that does the required. (tested on Ubuntu/dash)
mkdir action_per_dir
cd action_per_dir
mkdir -p parent_dir/subdir_1 parent_dir/subdir_2 parent_dir/subdir_3
cat > action_per_dir.yml << "EOF"
---
# Gets all the directories and stores all the return values of `find`
# into the results_of_find
# The return value will consist of:
# https://docs.ansible.com/ansible/latest/modules/find_module.html#return-values
- hosts: localhost
tasks:
- name: Get all dirs
find:
paths: parent_dir
file_type: directory
register: result_of_find
# We're interested only in the `files` part of results of find.
# In pseudo code what's happening here is:
# for each item in result_of_find.files:
# print item.path
#
# The output will be very verbose but for debugging purposes it can be filtered
# ansible-playbook action_per_dir.yml | grep msg
- name: Print all the dirs
debug:
msg: "{{ item.path }}"
with_items: "{{ result_of_find.files }}"
EOF
After that it just needs to be run:
ansible-playbook action_per_dir.yml | grep msg
I have directory structure as below, wants to fetch the specific sub directory files using ansible rather than all.
/mnt/server1 ->
----> yyy.deb
----> /mnt/server1/All/tttsss.deb
----> /mnt/server1/HS-CLONE/gggg.deb
----> /mnt/server1/HS-TEST/kkkk.deb
I need to find only files present under /mnt/server1/All/tttsss.deb and /mnt/server1/HS-CLONE/gggg.deb directories. I don't require all other files.
When i trying using below logic, the parent directory file yyy.deb is also coming as output.
- name: Ansible find files in subdirectory examples
find:
paths: /mnt/server1
file_type: file
recurse: yes
use_regex: yes
patterns:
- 'All'
- "HS-CLONE"
- '.*deb$'
register: files_matched_subdirectory
With the above logic output as:
Output:
yyy.deb
/mnt/server1/All/tttsss.deb
/mnt/server1/HS-CLONE/gggg.deb
Expected output should be:
/mnt/server1/All/tttsss.deb
/mnt/server1/HS-CLONE/gggg.deb
This is happening because you have below specific pattern in your find command which will match yyy.deb in the directory /mnt/server1
patterns:
- '.*deb$'
you can use excludes parameter to explicitly exclude this particular file :
- name: Ansible find files in subdirectory examples
find:
paths: /mnt/server1
file_type: file
recurse: yes
use_regex: yes
patterns:
- 'All'
- "HS-CLONE"
- '.*deb$'
excludes:
- 'yyy.deb'
register: files_matched_subdirectory
Or may be you can try something like below in your pattern parameter :
patterns:
- 'All'
- "HS-CLONE"
- '*/.*deb$'
i am working with ansible and using a playbook.
In this playbook I am performing a download (from a web url) and unarchive of a file into hosts (using unarchive module), and after that I am copying some files from control machine into hosts (using copy module).
What is happening is that every time I use unarchive module, although every file is the same, ansible is overwriting files in hosts.
How can I make it so that it does not overwrite if contents are the same?
My playbook:
---
- hosts: group1
sudo: yes
tasks:
- name: Download and Extract apache
unarchive:
src: http://mirrors.up.pt/pub/apache/tomcat/tomcat-9/v9.0.1/bin/apache-tomcat-9.0.1.tar.gz
dest: /opt/
remote_src: yes
- name: Copy file to host
copy: src=/etc/ansible/files/myfile.xml dest=/opt/apache-tomcat-9.0.1/conf/myfile.xml
Add a creates option referencing something the unarchive places.
c.f. the documentation page here (check the version against what you are using.)
e.g.:
- unarchive:
remote_src : yes
src : "{{ url }}"
dest : "{{ install_dir }}/"
creates : "{{ flagFile }}"
If the unarchive creates a /tmp/foo directory with a file named bar, then flagFile can be /tmp/foo/bar, and unarchive won't run again if it's already there.
I've handled this multiple ways, depending on the situation.
Option 1:
One of the files created by the archive has a specific name, such as foo-version_no
In this case, I can add the option:
creates: 'foo=version_no'
If this file exists, ansible will not extract it.
Caveat:
If the extracted directory is supposed to contain 12 files, and at least 1 of them has been removed or altered, the unarchive module will not replace them.
Option 2:
If there is no version number in the file name, examine one of the files in the extract directory and asses if it is the correct file.
Perhaps a checksum on the file; a grep for a unique parameter; or executing a command with the '--version' option.
Register the results into a variable:
register: correct_file
The unarchive command can then have the when parameter:
unarchive:
. . .
. . .
when: (correct_file.stdout != My_Required_Version and correct_file is defined)
The last part, about being defined, is there because, if you determine the extracted files don;t exist at all, you would skip checking the version, so the variable 'correct_file' will be undefined.
Caveat:
See Caveat from Option 1.
Option 3:
Use an argument, or extract command that will not over write files if they have not changed.
Caveat:
There will still be a little over head, because the extract command needs to asses each file, but will not make a new one.
Option 4:
Have another way to assess the quality of the files in the extracted directory, and set a variable based upon that result.
A simple example would be to run a checksum on all files, the run a checksum on that output, yielding a "Checksum of Checksums"
Ex:
curr_sum=$( cksum $( cksum /path/to/extracted/files/* ) )
if [ $curr_sum -eq $Correct_Value ]
then
echo "This is OK"
exit 0
else
echo "This is not ok"
exit 1
fi
In ansible, you would run this command and register the result. Then compare the output to the pre-set value:
cmd: "script_name"
register: cksum_answer
failed_when: failed
. . .
unarchive:
. . .
. . .
when: cksum_answer.rc == 1