There are a large number of servers with different connected partitions.
It is necessary to find 1 specific file on all partitions.
I am performing a file search
- name: search .file
stat:
path: " ???????/.file"
register: stat_result
- name: print server name
debug:
msg: "{{inventory_hostname}}"
when: stat_result.stat.exists
How do I find all the disks on the servers and substitute them in the search?
Need a list of servers where file found.
How do I find all the disks on the servers and substitute them in the search?
Since you like to search within all partitions and do not like to specify anything further here, it might be a better job for the find module – Return a list of files based on specific criteria.
A minimal example playbook
---
- hosts: localhost
become: true
gather_facts: false
tasks:
- find:
paths: /
recurse: yes
file_type: file
pattern: ".file"
use_regex: true
hidden: "true"
register: result
- debug:
msg: "{{ result.files }}"
when: result.matched != 0
will result for server and if the file was found into an output of
TASK [debug] *******************
ok: [localhost] =>
msg:
- atime: 1676487500.000000
ctime: 1676487500.000000
dev: 1000
gid: 1000
gr_name: ansible_users
inode: 1234567
isblk: false
ischr: false
isdir: false
isfifo: false
isgid: false
islnk: false
isreg: true
issock: false
isuid: false
mode: '0644'
mtime: 1676487500.000000
nlink: 1
path: /home/user/test/.file
...
will start a search recursive from the given root \ and if correct access rights are given on all mounted partitions, directories, etc.
Please Note
Since that might cause a significant amount of I/O operations (IOps), maybe even over the wire (network), it is strongly recommended to decrease the search path and pattern as much as possible. The same is recommended because of the access right as it might trigger an unwanted amount of access denied messages.
Related
A couple of tasks in my playbook keep finding and modifying files in hidden directories--I do not want the hidden files to be altered.
ansible-core 2.13.7
- name: Find all script files
find:
recurse: yes
paths: /var/bbb
patterns:
- "*.sh"
- "*.py"
- "*.env"
register: script_files
- name: Make all scripts executable
file:
dest: "{{ item.path }}"
mode: "a+rx"
with_items: "{{ script_files.files }}"
Sample of output from second task:
ok: [localhost] => (item={'path': '/var/bbb/.ansible/collections/ansible_collections/community/docker/tests/unit/plugins/module_utils/test_util.py', 'mode': '0755', ...
I've gone through the docs and they say hidden files are left alone by default in the current and former versions. I suspect it may be the patterns, but again the docs say the pattern is compared to the file base name and excludes the directory.
Regarding your observation
Ansible find module collecting files in hidden directories
that's right and the expected behavior.
... they say hidden files are left alone by default in the current and former versions.
I interpret the documentation about Parameter: hidden that it applies to files only and not to directories.
A minimal test setup
mdkir .hidden
touch .hidden/script.sh
touch .hidden/.script.sh
with an example playbook
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Find all script files
find:
hidden: false
recurse: true
paths: "/home/{{ ansible_user }}/"
patterns:
- "*script.sh"
register: result
- name: Show result
debug:
msg: "{{ result.files }}"
will result into an output of
...
path: /home/user/.hidden/script.sh
...
or if hidden: true
...
path: /home/user/.hidden/script.sh
...
path: /home/user/.hidden/.script.sh
...
Similar Q&A
Using Ansible find module to get hidden folders
To summarize, your question seems to be about
How to exclude hidden directories from Ansible find module recursive search?
Since with the parameter hidden: false and file_type: any still the /.hidden/script.sh is found, the excludes parameter operates on basenames of files, I recommend to adjust the search path.
Given the useful explanation in the previous answer and the complexity of the search path solution in my use case, I looked beyond the parameters of find and developed a separate task that removes results that contain hidden directories:
- name: filter hidden directories
set_fact:
filtered_list: "{{result.files | map(attribute='path') | reject('search', '/\\.')}}"
Be aware that this will remove all paths which contain a hidden directory: it should not be used when the find task is performed on paths that contain a hidden directory.
I am trying to find and copy files with pattern *-chrony-configuration.yaml from the machineconfig directory to manifests directory.
The location of the machineconfig varies depending on the user
if the user is root, the folder is /root/machineconfig
if the user is non-root, the folder is /home/machineconfig
- name: Find machineconfig files generated from helpernode
find:
paths: "machineconfig/"
patterns: "*-chrony-configuration.yaml"
register: machine_file
- name: Copy machineconfig files generated from helpernode
copy:
src: "{{ item.path }}"
dest: "{{ workdir }}/manifests"
remote_src: yes
with_items:
- "{{ machine_file.files }}"
The above code errors out giving
"msg": "machineconfig/ was skipped as it does not seem to be a valid directory or it cannot be accessed\n"
Is there a way in ansible where it finds the path of the file and uses that to produce its copy?
What you could do, if your path is either /root/machineconfig or /home/machineconfig, as it seems, is to feed the paths parameter with a list, as the documentation propose it.
Given the task:
- find:
paths:
- /root/machineconfig
- /home/machineconfig
patterns: "*-chrony-configuration.yaml"
register: machine_file
This will list you the files you are looking for and raise a simple warning for folder that does not exists.
With the playbook:
- hosts: localhost
gather_facts: yes
gather_subset:
- min
tasks:
- find:
paths:
- /root/machineconfig
- /home/machineconfig
patterns: "*-chrony-configuration.yaml"
register: machine_file
- debug:
var: machine_file.files | map(attribute='path')
This yields:
TASK [find] ****************************************************************
[WARNING]: Skipped '/root/machineconfig' path due to this access issue:
'/root/machineconfig' is not a directory
ok: [localhost]
TASK [debug] ***************************************************************
ok: [localhost] =>
machine_file.files | map(attribute='path'):
- /home/machineconfig/foobar-chrony-configuration.yaml
If the remote_user of the playbook is the user for which the machineconfig folder is created, then you can use the ansible_env.HOME fact in order to get the home directory of that user.
So, that would make your copy task looks like:
- name: Find machineconfig files generated from helpernode
find:
paths: "{{ ansible_env.HOME }}/machineconfig/"
patterns: "*-chrony-configuration.yaml"
register: machine_file
become: no
Please mind: that you need to gather some minimal facts for this, to work:
- hosts: foobar
gather_facts: yes
gather_subset:
- min
I am working on small playbook that will search for files in specified directories and then delete them if they met certain conditions. I have following playbook so far
- hosts:
- localhost
gather_facts: false
tasks:
- name: find logs
find:
paths: "{{ item.0 }}"
file_type: file
patterns: "{{ item.1 }}"
register: find_logs
with_nested:
- ["/var/log/apache2", "/var/log/nginx"]
- ["access.log", "error.log"]
- debug:
var: item
loop:
- "{{ find_logs }}"
So this will obviously look into /var/log/apache2 and /var/log/nginx directories and search for access.log and error.log files. Now, following ansible's documentation I want to access files return values and their paths. The issue I'm having right now is with nested loop and registered find_logs variable which holds list of dictionaries in results key. If I do find_logs.results then I will get a list of dictionaries and each of these dictionaries will have another list of files kept in files section. How can I 'flatten' this list even more to be able to retrieve files.path for every element produced by nested loop? To be honest I also tried find_logs | json_query('results[*].files[*]') but that gives me another list and I can't seem to iterate over it to get what I want (which is path of file). Any idea on how to make this work?
I completely misunderstood documentation for find module, Nested loop can be excluded in this case and replaced with following syntax
- hosts:
- localhost
gather_facts: false
tasks:
- name: find logs
find:
paths:
- /var/log/apache2
- /var/log/nginx
file_type: file
patterns:
- "access.log"
- "error.log"
- "other_vhosts_access.log"
register: find_logs
- name: check what was registered
debug:
msg: "my path -> {{ item.path }}"
with_items:
- "{{ find_logs.files }}"
I am running an Ansible playbook that looks into a directory and if the size is greater than 1 megabyte, then it has found a match. I just want to verify that it found the correct file. However, I am having a hard time capturing the output. I used the debug module in the documentation, but I still haven't solved it.
- hosts: node2
tasks:
- name: Recursively find /tmp files with last access time greater than 3600 seconds
find:
paths: /tmp/myFile/outputs
size: 1m
recurse: yes
- debug:
matched:
I get this error:
TASK [debug] *******************************************************************************************************************************************************************************************
fatal: [10.245.61.245]: FAILED! => {"msg": "Invalid options for debug: matched"}
You can save the return value of a task to a variable and after that you can print out the variable.
(I slightly modified your code)
- hosts: node2
tasks:
- name: Recursively find /tmp files with last access time greater than 3600 seconds
find:
paths: /tmp
size: 1k
recurse: yes
register: result
- debug:
var: result
Here are some useful links, if you want to go more in depth:
https://docs.ansible.com/ansible/latest/modules/find_module.html
https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html
https://docs.ansible.com/ansible/latest/modules/debug_module.html
I hope I could help you :)
How to register a variable when using loop with stat module?
I am working on a project where I wish to run comparisons against the known value of a collection of files (checksum), which I will then take action if a change is detected (EG: notify someone, have not written this part yet).
If this were purely a CLI matter, I would have this sorted with some easy SH scripting.
That said, I have Ansible (2.7.5) available within my ENV and am keen to use it!
In reading the vendor documents, using the stat module felt the "Ansible way" to go on this one.
Currently just *NIX servers (Linux, Solaris, and possibly AIX) are in scope, but eventually this might also apply to Windows, where I expect I would use win_stat instead with suitable parameters.
At present I plan to dump the results of the scan to a file (EG: CSV), which I would then iterate / match against, for the purposes of a comparison (to detect if a file has been somehow changed).
This is another part I have not written yet (the read a file and compare portions), but expect to hit those once I get this present matter sorted.
My current challenge, is that I can get "one-off" stat checks to work fine.
However, I expect to be targeting a whole directory worth of files, and thus want to presumably:
"discover" the contents of the target directory, and retain this in memory
iterate (loop) through the list in memory
performing a stat check upon each file
retaining the checksum of each file
building some sort of dict or list?
write the collective results (or one line at a time) out to a log file of sorts (CSV.log: file_path,file_checksum)
I would welcome your feedback on what I might be missing (aside from some hair at this point)?
I have tried a few different approaches to looping within the playbook (loop, with_items, etc.), however the challenge remains the same.
The stat loop runs fine, but the trailing register statement fails to commit the output to memory (resulting in a variety of "undefined variable" errors).
Am I somehow missing something in my loop definition?
Looking at the vendor docs on "Using register with a loop", it would appear I am doing this correctly (in my view anyway).
Simple "target files" I am checking against within a directory.
/tmp/app/targets/file1.txt
Some text.
/tmp/app/targets/file2.cfg
cluster=0
cluster_id=app_pool_00
/tmp/app/targets/file3.sh
#!/bin/sh
printf "Hello world\n"
exit 0
My prototyping playbook as it exists currently.
---
- name: check file integrity
hosts: localhost
become: no
vars:
TARGET: /tmp/app/targets
LOG: /tmp/app/archive/scan_results.log
tasks:
- name: discover target files
find:
paths: "{{ TARGET }}"
recurse: yes
file_type: file
register: TARGET_FILES
- name: scan target
stat:
path: "{{ item.path }}"
get_checksum: yes
loop: "{{ TARGET_FILES.files }}"
register: TARGET_RESULTS
- name: DEBUG
debug:
var: "{{ TARGET_RESULTS }}"
- name: write findings to log
copy:
content: "{{ TARGET_RESULTS.stat.path }},{{ TARGET_RESULTS.stat.checksum }}"
dest: "{{ LOG }}"
...
My "one-off" playbook that worked.
---
- name: check file integrity
hosts: localhost
become: no
vars:
TARGET: /tmp/app/targets/file1.txt
LOG: /tmp/app/archive/scan_results.log
tasks:
- name: scan target
stat:
path: '{{ TARGET }}'
checksum_algorithm: sha1
follow: no
get_attributes: yes
get_checksum: yes
get_md5: no
get_mime: yes
register: result
- name: write findings to log
copy:
content: "{{ result.stat.path }},{{ result.stat.checksum }}"
dest: "{{ LOG }}"
...
The output was not exciting, but useful.
Would expect to build this up with multi-line output (one line per file stat checked) if I could figure out how to loop / register loop output correctly.
/tmp/app/archive/scan_results.log
/tmp/app/targets/file1.txt,8d06cea05d408d70c59b1dbc5df3bda374d869a4
You can use the set_fact module to register a variable like you want.
I don't use it in my test for you, it maybe useless in your case :
---
- name: check file integrity
hosts: localhost
vars:
TARGET: /tmp/app/targets
LOG: /tmp/app/archive/scan_results.log
tasks:
- name: 'discover target files'
find:
paths: "{{ TARGET }}"
recurse: yes
file_type: file
register: TARGET_FILES
- debug:
var: TARGET_FILES
- name: 'scan target'
stat:
path: "{{ item.path }}"
get_checksum: yes
loop: "{{ TARGET_FILES.files }}"
register: TARGET_RESULTS
- debug:
var: TARGET_RESULTS
- name: 'write findings to log'
lineinfile:
line: "{{ item.stat.path }},{{ item.stat.checksum }}"
path: "{{ LOG }}"
create: yes
loop: '{{ TARGET_RESULTS.results }}'
result:
# cat /tmp/app/archive/scan_results.log
/tmp/app/targets/file3.sh,bb4b0ffe4b5d26551785b250c38592b6f482cab4
/tmp/app/targets/file1.txt,8d06cea05d408d70c59b1dbc5df3bda374d869a4
/tmp/app/targets/file2.cfg,fb23292e06f91a0e0345f819fdee34fac8a53e59
Best Regards