I want to list all files of a directory with the owner of the file her name start with 'a',
i have a ansible playbook like this
- name: Recursively find
find:
paths: "/tmp"
age: "6d"
age_stamp: ctime
file_type: any
patterns: "*"
recurse: no
register: output
Q: "List all files of a directory with the owner of the file her name starts with 'a'."
A: For example, given the tree
shell> tree -u tmp
tmp
├── [admin ] file1
├── [admin ] file2
└── [user1 ] file3
Find all files in the first task and select the owner in the second task
- find:
paths: tmp
register: output
- debug:
msg: "{{ item.path }}"
loop: "{{ output.files|selectattr('pw_name', 'match', '^a.*$')|list }}"
give
msg: tmp/file2
msg: tmp/file1
Related
I need to find folder names in /home/test directory. Once I have the list of folder names I need to loop over a task. I am able to retrieve the complete path, but not able to retrieve the folder name. is my approach wrong, or is there any easy way to retrieve folder names?
--
- hosts: localhost
gather_facts: no
vars:
Files: []
namespaces: []
tasks:
- name: Recursively find folders under /home/test
ansible.builtin.find:
paths: /home/test
file_type: directory
recurse: no
register: output
- name: Adding Files to the LIST
set_fact:
Files: "{{ Files + [item.path] }}"
with_items: "{{ output.files }}"
- name: Remove the path to only and retrieve folder name only
set_fact:
namespaces: "{{ namespaces + item | split('/') | last}}"
with_items: "{{ Files }}"
Error:
fatal: [localhost]: FAILED! => {"msg": "Unexpected templating type error occurred on ({{ namespaces + item | split('/') | last}}): can only concatenate list (not \"str\") to list"}
Put the below declaration into the vars
namespaces: "{{ output.files|map(attribute='path')|map('basename')|list }}"
Example of a complete playbook for testing
Given the tree
shell> tree /tmp/test/
/tmp/test/
├── dir_A
├── dir_B
├── file_1
└── file_2
the playbook
- hosts: localhost
vars:
namespaces: "{{ output.files|map(attribute='path')|map('basename')|list }}"
tasks:
- find:
paths: /tmp/test
file_type: directory
register: output
- debug:
var: namespaces
gives (abridged)
namespaces:
- dir_B
- dir_A
Your expression item | split('/') | last currently returns a string that you try to concatenate to the namespaces list. However, this is not allowed and thus you need to cast it to a list as well by simply put it into braces []:
- name: Remove the path to only and retrieve folder name only
set_fact:
namespaces: "{{namespaces + [item | split('/') | last]}}"
with_items: "{{ Files }}"
I have a playbook that copy files from different local folders to different remote paths and it works, configured to overwrite the files with backup (with copy module).
How to get the list of files that will be copied and overwrite before doing it?
I'd like to prevent to copy wrong files.
Example of the copy:
name: Copy all
copy:
src: scenarios/all_machines/
dest: /path/to/dest
backup: yes
force: yes
owner: root
group: root
mode: 0644
when: ansible_facts.services['crowdsec.service'] is defined
UPDATE
With this local folder:
scenarios/all_machines/
├── wg-sshd-more-logs.yaml
└── wg-x00.yaml
and this remote dir:
test_ansible/
└── wg-x00.yaml
where the wg-x00.yaml file has the same content.
With this playbook (where I changed the '>' char with '<'):
- name: TEST
synchronize:
src: scenarios/all_machines/
dest: test_ansible/
rsync_opts:
- --dry-run
register: result
- debug:
var: result.stdout_lines
- name: TEST 2
set_fact:
overwrite: "{{ file_stat|
selectattr('stat', 'match', '<f.*')|
selectattr('stat', 'ne', '<f+++++++++')|
map(attribute='file')|
list }}"
vars:
file_stat: "{{ result.stdout_lines|
map('regex_replace', regex, replace)|
map('from_yaml')|
list }}"
regex: '^(.*?) (.*)$'
replace: '{file: "\2", stat: "\1"}'
- debug:
var: overwrite
I have this output:
"result.stdout_lines": [
".d..tp..... ./",
"<f+++++++++ wg-sshd-more-logs.yaml",
"<f..tp..... wg-x00.yaml"
]
"overwrite": [
"wg-x00.yaml"
]
So:
I changed the the '>' char with '<' because my output is different from your
The overwrite variable tells me that the file will be overwritten but they are equal...
Given the directories below for testing
shell> tree scenarios/all_machines/
scenarios/all_machines/
├── file1
├── file2
└── file3
0 directories, 3 files
shell> tree dest/
dest/
└── file1
0 directories, 1 file
the file file1 at dest/ will be replaced. The question is how to find it.
Use synchronize and set 'rsync_opts: --dry-run'
- synchronize:
src: scenarios/all_machines/
dest: dest
rsync_opts:
- --dry-run
register: result
The result will display the status of the files
result.stdout_lines:
- .d..t...... ./
- '>f..t...... file1'
- '>f+++++++++ file2'
- '>f+++++++++ file3'
The task below
- set_fact:
overwrite: "{{ file_stat|
selectattr('stat', 'match', '>f.*')|
selectattr('stat', 'ne', '>f+++++++++')|
map(attribute='file')|
list }}"
vars:
file_stat: "{{ result.stdout_lines|
map('regex_replace', regex, replace)|
map('from_yaml')|
list }}"
regex: '^(.*?) (.*)$'
replace: '{file: "\2", stat: "\1"}'
will create the list
file_stat:
- {file: ./, stat: .d..t......}
- {file: file1, stat: '>f..t......'}
- {file: file2, stat: '>f+++++++++'}
- {file: file3, stat: '>f+++++++++'}
and will select names of the existing files only
overwrite:
- file1
See Understanding the output of rsync --itemize-changes.
Notes
Use debug to display the variables. For example, use to_yaml/to_nice_yaml filters
- debug:
var: file_stat|to_yaml
- debug:
var: overwrite|to_nice_yaml
Use yaml callback.
See DEFAULT_STDOUT_CALLBACK on how to configure the callback.
I am trying to set permission on directories from a list.
Below is an example list where I would like dirA, dirB, dirC and dirD to get o-w permission.
"__cronjob_paths": [
"/dirA/dirB",
"/dirC/dirD",
]
file module has recursive option but it also sets the permissions on the files within the directories so it's not an option.
Would a regex or filter be the best way to solve this? Maybe filter out the /and save the directories into another list which i can loop through and set permissions?
I also need to have in mind that there could be other directories within the system that has a directory with the same name as the one from the list (which should not get o-w)
Create lists of the subdirs and indexes, e.g.
- set_fact:
_paths: "{{ _paths|d([]) + [{'dir': _dir, 'idx': _idx}] }}"
loop: "{{ __cronjob_paths }}"
vars:
_dir: "{{ item.split('/')|select() }}"
_idx: "{{ range(1, _dir|length + 1) }}"
gives
_paths:
- dir:
- dirA
- dirB
idx:
- 1
- 2
- dir:
- dirC
- dirD
idx:
- 1
- 2
Now, you can iterate idx subelements and join the subdirs, e.g.
- debug:
msg: "chmod o-w /{{ item.0.dir[:item.1]|join('/') }}"
with_subelements:
- "{{ _paths }}"
- idx
gives
msg: chmod o-w /dirA
msg: chmod o-w /dirA/dirB
msg: chmod o-w /dirC
msg: chmod o-w /dirC/dirD
Proceed either with command or file module if this is what you want.
I have folders like this in my windows machine. using ansible want to sort this app-* pattern folder and get the last element. from this example I want to get d:\app-9.7.8 in ansible variable
d:\app-1.0.3
d:\app-1.0.7
d:\app-2.0.4
d:\app-7.0.4
d:\app-9.7.8
This code list the folder, but not sure how to sort and get the last element.
- name: Find dir
win_find:
paths: D:\
recurse: no
file_type: directory
register: result
- name: Find dir
debug:
msg: "output {{ result.files }}"
I am getting the last element like this:
- name: set dir
set_fact:
mule_dir_list: "{{ result.files | sort(attribute='path') | map(attribute='path') | list }}"
when: "( result.files |length > 0 )"
- name: print dir
debug:
msg: "dir {{ mule_dir_list[-1] }}"
What I am trying to achieve here is as below
Unarchive the code from tar.gz - working
Find file names in unarchived directory - working
Find file names in code directory based on file names fetched in step 2 - failed
Copy files from source:(Step 2) destination:(Step3) - working if I use the hardcoded file names in the pattern section of step 3
Below mentioned is the Ansible role I have used:
- name: Unarchive config files to server
unarchive:
src: "{{ config_dir }}/config.tar.gz"
dest: /tmp
list_files: yes
register: tar_path
- name: Find file names in unarchived config files
find:
paths: "{{ tar_path.dest }}"
file_type: file
recurse: yes
register: tmp_file_path
- name: Find file names in code base
find:
paths: /opt
file_type: file
recurse: yes
patterns:
#Search for file names with the values in tmp_file_path
register: code_file_path
- set_fact:
code_files: "{{ code_files|default([]) +
[{'path': item, 'name': item|basename}] }}"
loop: "{{ code_file_path.files|map(attribute='path')|list }}"
- name: copy files
command: cp "{{ item.0 }}" "{{ item.1.path }}"
with_together:
- "{{ tmp_file_path.files|map(attribute='path')|list|sort }}"
- "{{ code_files|sort(attribute='name') }}"
Here I need to use find to locate files in /opt directory based on pattern(filename) exactly which I have unarchived in /tmp
And finally, replace files from /tmp to /opt based on the file names and paths(This I am able to do). The directory structure is as follows:
shell> tree tmp
tmp
├── file1
├── file2
└── file3
shell> tree opt
opt
├── bar
│ └── file2
├── baz
│ └── file3
└── foo
└── file1
Here if I use the below code wherein I manually mention the file names, then it works. However, I don't want to do that
- name: Find file names in code base
find:
paths: /opt
file_type: file
recurse: yes
patterns:
- file1
- file2
- file3
register: code_file_path
I need a solution to replace the hardcoding for patterns: file1, file2 and file3 and use some variable to do that. The file names in /tmp and /opt where I need to replace is exactly the same
If I understood correctly, here is a possible way to handle what you're trying to do. In the below example, I took away the unarchive job as it's not on the critical path.
Playbook walkthrough
I created two sample directories. The first two tasks are only there to further show you this test structure:
an archive directory containing 4 files in random directories. One of them is not present in the target
a code directory containing several files random directories. 3 files have the same basenames as other files found in archive.
The first find task is identical to yours and registers a result with details of all files in the archive dir.
For the second find task in the code directory, the key point is to pass as patterns parameter the list of basenames from the first search which you can get with the expression:
{{ search_archive.files | map(attribute='path') | map('basename') | list }}
We can detail this one as: get files list from our archive find result, extract only the path attribute, apply the basename filter on each list element and return a list.
For the last task, I used the copy module. My example runs on localhost but since yours will probably run on a remote target, the remote_src has to be set (or files will be fetched from the controller).
The loop is done on the result of the previous task so we only get the matching files in code directory as dest. To select the src, we look for corresponding files in the archive folder with the following expression:
{{ search_archive.files | map(attribute='path') | select('match', '^.*/' + item | basename + '$') | first }}
The select filter select is applying the match test to each path in the list selecting only the elements ending with the current code path basename. The first filter get only the first (and only in your case) matching element. loop_control.label is used to get a better output of task result.
Demo playbook
The first two task are only for debugging/demo purpose.
---
- name: Update files from package in code wherever they are
hosts: localhost
gather_facts: false
tasks:
- name: Capture sample data structure
command: tree archive code
register: structure
changed_when: false
- name: Show sample data structure
debug:
msg: "{{ structure.stdout_lines}}"
- name: Find files in archive
find:
paths: archive
file_type: file
recurse: yes
register: search_archive
- name: Find files in code matching names in archive
find:
paths: code
file_type: file
recurse: yes
patterns: >-
{{
search_archive.files |
map(attribute='path') |
map('basename') |
list
}}
register: search_code
- name: Copy files from archive to code
vars:
archive_source: >-
{{
search_archive.files |
map(attribute='path') |
select('match', '^.*/' + item | basename + '$') |
first
}}
copy:
remote_src: yes
src: "{{ archive_source }}"
dest: "{{ item }}"
loop: "{{ search_code.files | map(attribute='path') | list }}"
loop_control:
label:
Source: "{{ archive_source }}"
Destination: "{{ item }}"
Result
PLAY [Update files from package in code wherever they are] *****************************************************************************************************************************************************************************
TASK [Capture sample data structure] ***************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Show sample data structure] ******************************************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"archive",
"├── a_dir",
"│ └── file2",
"├── file1.txt",
"├── file3",
"└── other_dir",
" └── bla",
" └── fileX",
"code",
"├── dir1",
"│ └── file1.txt",
"├── dir2",
"│ ├── file2",
"│ ├── pipo",
"│ └── toto",
"└── dir3",
" └── subdir",
" └── file3",
"",
"7 directories, 9 files"
]
}
TASK [Find files in archive] ***********************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Find files in code matching names in archive] ************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Copy files from archive to code] *************************************************************************************************************************************************************************************************
changed: [localhost] => (item={'Source': 'archive/file1.txt', 'Destination': 'code/dir1/file1.txt'})
changed: [localhost] => (item={'Source': 'archive/a_dir/file2', 'Destination': 'code/dir2/file2'})
changed: [localhost] => (item={'Source': 'archive/file3', 'Destination': 'code/dir3/subdir/file3'})
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=5 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0