Hoping with some help on the following. I'm trying to make reference to an xml file within the work_dir location:
---
- include_vars:
file: var_file.yml
- name: Locate audit results file
find:
paths: "{{ item }}"
recurse: no
patterns: '*.xml'
with_items: "{{ work_dir }}"
register: audit_file
- name: Copy audit results file to local destination
fetch:
src: "{{ item }}/{{ audit_file }}"
dest: /home/bob/audit_results/
flat: yes
validate_checksum: no
with_items: "{{ work_dir }}"
var_file.yml:
---
work_dir:
- /var/tmp/audit
However the above code keeps erroring with:
"msg": "unable to calculate the checksum of the remote file"}
You find files on a loop of paths and register the result. Therefore your registered var audit_file contains a results attribute which is a list.
Each element of the results list contains a files attribute which is again a list. Each files element contains a result of your find run with all the info about the found file. In those info, there is a path attribute pointing to the exact path of the file on the remote server.
Before going further, I strongly suggest your read the above documentation and issue a debug of your var to understand its structure and content
- debug:
var: audit_file
If you want to fetch each found file, you need to make a loop on each results with a sub-loop on each files element. This can be done with a subelements lookup although we will only use here the sub element (i.e. item.1) of the loop.
This is how you can fix your second task:
- name: Copy audit results file to local destination
fetch:
src: "{{ item.1.path }}"
dest: /tmp/test/
flat: yes
validate_checksum: no
with_subelements:
- "{{ audit_file.results }}"
- files
Note: this is not the only solution, I went to the easiest and most obvious.
Related
Using the Ansible win_find module, I want to output just the path to the found files and I am trying this:
- name: Find files in directory while searching recursively
win_find:
paths: C:\Downloads
recurse: yes
register: dwnlds_list
- name: Display win_find file list result
debug: var=item.path
with_items: "{{ dwnlds_list.files }}"
However, the output I get is every return value for every found file.
What I would like is just:
"C:\\Downloads\\file0"
"C:\\Downloads\\file1"
"C:\\Downloads\\file2"
This is the solution that worked for me:
- name: Find files
win_find:
paths: C:\Downloads
recurse: yes
register: dwnlds_list
- name: Output file list
debug: msg="{{ dwnlds_list | json_query('files[].path') }}"
I am working on small playbook that will search for files in specified directories and then delete them if they met certain conditions. I have following playbook so far
- hosts:
- localhost
gather_facts: false
tasks:
- name: find logs
find:
paths: "{{ item.0 }}"
file_type: file
patterns: "{{ item.1 }}"
register: find_logs
with_nested:
- ["/var/log/apache2", "/var/log/nginx"]
- ["access.log", "error.log"]
- debug:
var: item
loop:
- "{{ find_logs }}"
So this will obviously look into /var/log/apache2 and /var/log/nginx directories and search for access.log and error.log files. Now, following ansible's documentation I want to access files return values and their paths. The issue I'm having right now is with nested loop and registered find_logs variable which holds list of dictionaries in results key. If I do find_logs.results then I will get a list of dictionaries and each of these dictionaries will have another list of files kept in files section. How can I 'flatten' this list even more to be able to retrieve files.path for every element produced by nested loop? To be honest I also tried find_logs | json_query('results[*].files[*]') but that gives me another list and I can't seem to iterate over it to get what I want (which is path of file). Any idea on how to make this work?
I completely misunderstood documentation for find module, Nested loop can be excluded in this case and replaced with following syntax
- hosts:
- localhost
gather_facts: false
tasks:
- name: find logs
find:
paths:
- /var/log/apache2
- /var/log/nginx
file_type: file
patterns:
- "access.log"
- "error.log"
- "other_vhosts_access.log"
register: find_logs
- name: check what was registered
debug:
msg: "my path -> {{ item.path }}"
with_items:
- "{{ find_logs.files }}"
Using Ansible 2.9.12
Question: How do I configure Ansible to ensure the contents of a file is equal amongst at least 3 hosts, when the file is present at at least one host?
Imagine there are 3 hosts.
Host 1 does not has /file.txt.
Host 2 has /file.txt with contents hello.
Host 3 has /file.txt with contents hello.
Before the play is run, I am unaware whether the file is present or not. So the file could exist on host1, or host2 or host3. But the file exists on at least one of the hosts.
How would I ensure each time Ansible runs, the files across the hosts are equal. So in the end, Host 1 has the same file with the same contents as Host 2 or Host 3.
I'd like this to be dynamically set, instead of specifying the host names or group names, e.g. when: inventory_hostname == host1.
I am not expecting a check to see whether the contents of host 2 and 3 are equal
I do however, want this to be setup in an idempotent fashion.
The play below does the job, I think
shell> cat pb.yml
- hosts: all
tasks:
- name: Get status.
stat:
path: /file.txt
register: status
- block:
- name: Create dictionary status.
set_fact:
status: "{{ dict(keys|zip(values)) }}"
vars:
keys: "{{ ansible_play_hosts }}"
values: "{{ ansible_play_hosts|
map('extract', hostvars, ['status','stat','exists'])|
list }}"
- name: Fail. No file exists.
fail:
msg: No file exists
when: status.values()|list is not any
- name: Set reference to first host with file present.
set_fact:
reference: "{{ status|dict2items|
selectattr('value')|
map(attribute='key')|
first }}"
- name: Fetch file.
fetch:
src: /file.txt
dest: /tmp
delegate_to: "{{ reference }}"
run_once: true
- name: Copy file if not exist
copy:
src: "/tmp/{{ reference }}/file.txt"
dest: /file.txt
when: not status[inventory_hostname]
But, this doesn't check the existing files are in sync. It would be safer to sync all hosts, I think
- name: Synchronize file
synchronize:
src: "/tmp/{{ reference }}/file.txt"
dest: /file.txt
when: not status[inventory_hostname]
Q: "FATAL. could not find or access '/tmp/test-multi-01/file.txt on the Ansible controller. However, folder /tmp/test-multi-03 is present with the file.txt in it."
A: There is a problem with the fetch module when the task is delegated to another host. When the TASK [Fetch file.] is delegated to test-multi-01 which is localhost in this case changed: [test-multi-03 -> 127.0.0.1] the file will be fetched from test-multi-01 but will be stored in /tmp/test-multi-03/file.txt. The conclusion is, the fetch module ignores delegate_to when it comes to creating host-specific directories (not reported yet).
As a workaround, it's possible to set flat: true and store the files in a specific directory. For example, add the variable sync_files_dir with the directory, set fetch flat: true, and use the directory to both fetch and copy the file
- hosts: all
vars:
sync_files_dir: /tmp/sync_files
tasks:
- name: Get status.
stat:
path: /file.txt
register: status
- block:
- name: Create dir for files to be fetched and synced
file:
state: directory
path: "{{ sync_files_dir }}"
delegate_to: localhost
- name: Create dictionary status.
set_fact:
status: "{{ dict(keys|zip(values)) }}"
vars:
keys: "{{ ansible_play_hosts }}"
values: "{{ ansible_play_hosts|
map('extract', hostvars, ['status','stat','exists'])|
list }}"
- debug:
var: status
- name: Fail. No file exists.
fail:
msg: No file exists
when: status.values()|list is not any
- name: Set reference to first host with file present.
set_fact:
reference: "{{ status|dict2items|
selectattr('value')|
map(attribute='key')|
first }}"
- name: Fetch file.
fetch:
src: /file.txt
dest: "{{ sync_files_dir }}/"
flat: true
delegate_to: "{{ reference }}"
run_once: true
- name: Copy file if not exist
copy:
src: "{{ sync_files_dir }}/file.txt"
dest: /file.txt
when: not status[inventory_hostname]
We can achieve it by fetching the file from hosts where the file exists. The file(s) will be available on the control machine. However if the file which will be the source, exists on more than 1 node, then there will be no single source of truth.
Consider an inventory:
[my_hosts]
host1
host2
host3
Then the below play can fetch the file, then use that file to copy to all nodes.
# Fetch the file from remote host if it exists
- hosts: my_hosts
tasks:
- stat:
path: /file.txt
register: my_file
- fetch:
src: /file.txt
dest: /tmp/
when: my_file.stat.exists
- find:
paths:
- /tmp
patterns: file.txt
recurse: yes
register: local_file
delegate_to: localhost
- copy:
src: "{{ local_file.files[0].path }}"
dest: /tmp
If multiple hosts had this file then it would be in /tmp/{{ ansible_host }}. Then as we won't have a single source of truth, our best estimate can be to use the first file and apply on all hosts.
Well i believe the get_url module is pretty versatile - allows for local file paths or paths from a web server. Try it and let me know.
- name: Download files in all host
hosts: all
tasks:
- name: Download file from a file path
get_url:
url: file:///tmp/file.txt
dest: /tmp/
Edited ans:
(From documentation: For the synchronize module, the “local host” is the host the synchronize task originates on, and the “destination host” is the host synchronize is connecting to)
- name: Check that the file exists
stat:
path: /etc/file.txt
register: stat_result
- name: copy the file to other hosts by delegating the task to the source host
synchronize:
src: path/host
dest: path/host
delegate_to: my_source_host
when: stat_result.stat.exists
How to register a variable when using loop with stat module?
I am working on a project where I wish to run comparisons against the known value of a collection of files (checksum), which I will then take action if a change is detected (EG: notify someone, have not written this part yet).
If this were purely a CLI matter, I would have this sorted with some easy SH scripting.
That said, I have Ansible (2.7.5) available within my ENV and am keen to use it!
In reading the vendor documents, using the stat module felt the "Ansible way" to go on this one.
Currently just *NIX servers (Linux, Solaris, and possibly AIX) are in scope, but eventually this might also apply to Windows, where I expect I would use win_stat instead with suitable parameters.
At present I plan to dump the results of the scan to a file (EG: CSV), which I would then iterate / match against, for the purposes of a comparison (to detect if a file has been somehow changed).
This is another part I have not written yet (the read a file and compare portions), but expect to hit those once I get this present matter sorted.
My current challenge, is that I can get "one-off" stat checks to work fine.
However, I expect to be targeting a whole directory worth of files, and thus want to presumably:
"discover" the contents of the target directory, and retain this in memory
iterate (loop) through the list in memory
performing a stat check upon each file
retaining the checksum of each file
building some sort of dict or list?
write the collective results (or one line at a time) out to a log file of sorts (CSV.log: file_path,file_checksum)
I would welcome your feedback on what I might be missing (aside from some hair at this point)?
I have tried a few different approaches to looping within the playbook (loop, with_items, etc.), however the challenge remains the same.
The stat loop runs fine, but the trailing register statement fails to commit the output to memory (resulting in a variety of "undefined variable" errors).
Am I somehow missing something in my loop definition?
Looking at the vendor docs on "Using register with a loop", it would appear I am doing this correctly (in my view anyway).
Simple "target files" I am checking against within a directory.
/tmp/app/targets/file1.txt
Some text.
/tmp/app/targets/file2.cfg
cluster=0
cluster_id=app_pool_00
/tmp/app/targets/file3.sh
#!/bin/sh
printf "Hello world\n"
exit 0
My prototyping playbook as it exists currently.
---
- name: check file integrity
hosts: localhost
become: no
vars:
TARGET: /tmp/app/targets
LOG: /tmp/app/archive/scan_results.log
tasks:
- name: discover target files
find:
paths: "{{ TARGET }}"
recurse: yes
file_type: file
register: TARGET_FILES
- name: scan target
stat:
path: "{{ item.path }}"
get_checksum: yes
loop: "{{ TARGET_FILES.files }}"
register: TARGET_RESULTS
- name: DEBUG
debug:
var: "{{ TARGET_RESULTS }}"
- name: write findings to log
copy:
content: "{{ TARGET_RESULTS.stat.path }},{{ TARGET_RESULTS.stat.checksum }}"
dest: "{{ LOG }}"
...
My "one-off" playbook that worked.
---
- name: check file integrity
hosts: localhost
become: no
vars:
TARGET: /tmp/app/targets/file1.txt
LOG: /tmp/app/archive/scan_results.log
tasks:
- name: scan target
stat:
path: '{{ TARGET }}'
checksum_algorithm: sha1
follow: no
get_attributes: yes
get_checksum: yes
get_md5: no
get_mime: yes
register: result
- name: write findings to log
copy:
content: "{{ result.stat.path }},{{ result.stat.checksum }}"
dest: "{{ LOG }}"
...
The output was not exciting, but useful.
Would expect to build this up with multi-line output (one line per file stat checked) if I could figure out how to loop / register loop output correctly.
/tmp/app/archive/scan_results.log
/tmp/app/targets/file1.txt,8d06cea05d408d70c59b1dbc5df3bda374d869a4
You can use the set_fact module to register a variable like you want.
I don't use it in my test for you, it maybe useless in your case :
---
- name: check file integrity
hosts: localhost
vars:
TARGET: /tmp/app/targets
LOG: /tmp/app/archive/scan_results.log
tasks:
- name: 'discover target files'
find:
paths: "{{ TARGET }}"
recurse: yes
file_type: file
register: TARGET_FILES
- debug:
var: TARGET_FILES
- name: 'scan target'
stat:
path: "{{ item.path }}"
get_checksum: yes
loop: "{{ TARGET_FILES.files }}"
register: TARGET_RESULTS
- debug:
var: TARGET_RESULTS
- name: 'write findings to log'
lineinfile:
line: "{{ item.stat.path }},{{ item.stat.checksum }}"
path: "{{ LOG }}"
create: yes
loop: '{{ TARGET_RESULTS.results }}'
result:
# cat /tmp/app/archive/scan_results.log
/tmp/app/targets/file3.sh,bb4b0ffe4b5d26551785b250c38592b6f482cab4
/tmp/app/targets/file1.txt,8d06cea05d408d70c59b1dbc5df3bda374d869a4
/tmp/app/targets/file2.cfg,fb23292e06f91a0e0345f819fdee34fac8a53e59
Best Regards
I'm using the module stat for checking if some files exists in a remote server and registering their result in a variable. i.e. config files I'm looking for: conf_dev.conf, conf_pred.conf, conf_pro.conf.
Later I'm using the copy module for transfering only the files that are missing using a conditional based on an attribute stored in
variable.results.{{(ITEM)index}}.stat.exists.
This returnsTRUE if file exists and FALSE if does not.
For running properly As the register variable is an array there are an index storing the result for each file I'm asking so I don't know how to convert the item in a index (0,1,2,3)
Does anyone know how to get the index of an item? I've tried this (look at the last line):
- name: Checking if common configuration files exists
stat:
path: "{{HOME_COMUN_CONFIG}}/{{item}}"
with_items: "{{LIST_COMMON_CONFIGURATION_ARTIFACTS}}"
register: store_results
- name: debug existe_app_comun
debug:
var: store_results
- name: Deploying missing files
copy:
src: "{{DIRTEMP_COMUN_CONFIG}}/{{item}}"
dest: "{{HOME_COMUN_CONFIG}}/{{item}}"
with_items: "{{LIST_COMMON_CONFIGURATION_ARTIFACTS}}"
when: existe_app_comun.results.{{index(item)}}.stat.exists is defined
Why do you loop over LIST_COMMON_CONFIGURATION_ARTIFACTS if you want to loop over registered variable?
- name: Deploying missing files
copy:
src: "{{DIRTEMP_COMUN_CONFIG}}/{{ item.item }}"
dest: "{{HOME_COMUN_CONFIG}}/{{ item.item }}"
with_items: "{{ existe_app_comun.results }}"
when: item.stat.exists
Here item is an element of results and item.item is an element of original loop.
P.S. if files at destination folder are not modified and should be same as in DIRTEMP_COMUN_CONFIG, then you must not use stat+copy, but just use copy – because it's idempotent and will not copy the same file twice.