I'm trying to specify part of an archive not to extract via Ansible's unarchive module using the exclude option.
I believe the syntax should be roughly as shown here...
- name: Extract files from discovered zip file
unarchive:
src: "{{ base_path }}/weblogic-deployment/environments/{{ client_environment }}/discovered_domain.zip"
dest: "{{ base_path }}/weblogic-deployment/environments/{{ client_environment }}/tmp"
exclude:
- ./wlsdeploy/applications/
remote_src: yes
I have tried a great many slight differences but the excluded directory is always output. Any suggestions?
The exclude option is expected to be a list of paths to exclude. So most likely it doesn't support directories alone.
List the directory and file entries that you would like to exclude from the unarchive action.
So try something like:
exclude:
- ./wlsdeploy/applications/*
For some syntax examples, see: unarchive/tasks/main.yml. Also check the source code.
To exclude the folder entirely:
exclude:
- "wlsdeploy/applications"
To extract the folder, but exclude the contents, you can use a glob:
exclude:
- "wlsdeploy/applications/*"
As the paths are relative to the archive, you don't need the preceding ./.
If you need to look inside the archive first (to find the paths), you can use less:
less discovered_domain.zip
From the unarchive tests for exclude:
- name: Unpack archive file excluding regular and glob files.
unarchive:
src: "{{ remote_tmp_dir }}/unarchive-00.{{item}}"
dest: "{{ remote_tmp_dir }}/exclude-{{item}}"
remote_src: yes
exclude:
- "exclude/exclude-*.txt"
- "other/exclude-1.ext"
with_items:
- zip
- tar
Related
A couple of tasks in my playbook keep finding and modifying files in hidden directories--I do not want the hidden files to be altered.
ansible-core 2.13.7
- name: Find all script files
find:
recurse: yes
paths: /var/bbb
patterns:
- "*.sh"
- "*.py"
- "*.env"
register: script_files
- name: Make all scripts executable
file:
dest: "{{ item.path }}"
mode: "a+rx"
with_items: "{{ script_files.files }}"
Sample of output from second task:
ok: [localhost] => (item={'path': '/var/bbb/.ansible/collections/ansible_collections/community/docker/tests/unit/plugins/module_utils/test_util.py', 'mode': '0755', ...
I've gone through the docs and they say hidden files are left alone by default in the current and former versions. I suspect it may be the patterns, but again the docs say the pattern is compared to the file base name and excludes the directory.
Regarding your observation
Ansible find module collecting files in hidden directories
that's right and the expected behavior.
... they say hidden files are left alone by default in the current and former versions.
I interpret the documentation about Parameter: hidden that it applies to files only and not to directories.
A minimal test setup
mdkir .hidden
touch .hidden/script.sh
touch .hidden/.script.sh
with an example playbook
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Find all script files
find:
hidden: false
recurse: true
paths: "/home/{{ ansible_user }}/"
patterns:
- "*script.sh"
register: result
- name: Show result
debug:
msg: "{{ result.files }}"
will result into an output of
...
path: /home/user/.hidden/script.sh
...
or if hidden: true
...
path: /home/user/.hidden/script.sh
...
path: /home/user/.hidden/.script.sh
...
Similar Q&A
Using Ansible find module to get hidden folders
To summarize, your question seems to be about
How to exclude hidden directories from Ansible find module recursive search?
Since with the parameter hidden: false and file_type: any still the /.hidden/script.sh is found, the excludes parameter operates on basenames of files, I recommend to adjust the search path.
Given the useful explanation in the previous answer and the complexity of the search path solution in my use case, I looked beyond the parameters of find and developed a separate task that removes results that contain hidden directories:
- name: filter hidden directories
set_fact:
filtered_list: "{{result.files | map(attribute='path') | reject('search', '/\\.')}}"
Be aware that this will remove all paths which contain a hidden directory: it should not be used when the find task is performed on paths that contain a hidden directory.
As per my requirement I want to exclude the few folders recursively by using the find module. I tried the below method to exclude the folders, but unfortunately its excludes only in the base path not sub folders.
- name: Find all files
find:
path: "{{ src_path }}"
patterns: "*.ksh,*.src,*.par,*.ctl"
file_type: file
recurse: yes
excludes: ["log","dat","archive","backup","history","outfiles","_hist","tmp","data","bak"]
register: list_of_files
Need to unarchive a .zip file then list the unpacked files. This is to select some pattern of files and use them in rest of the code
My playbook yaml snippet is
- name: uarchive the opar zip
unarchive:
src: "{{opar_download_path}}/opar.zip"
dest: "{{opar_download_path}}"
remote_src: yes
list_files: yes
I am not able to find details of how to use the result from "list_files",ie how to store the list of files to a variable. I was referring below document
https://docs.ansible.com/ansible/latest/modules/unarchive_module.html
The list_files: yes includes additional response attribute called files. For better understanding, try to print the output unarchived_list.files, as shown below:
- name: unarchive the opar zip
unarchive:
src: "{{opar_download_path}}/opar.zip"
dest: "{{opar_download_path}}"
remote_src: yes
list_files: yes
register: unarchived_list
- name: print unarchived folder list of files
debug: msg="{{unarchived_list.files}}"
More information on registering variables can be found here: https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html#registering-variables
In my Ansible Playbook I'd like to have a task that removes old files and folders from the application's directory. The twist to this otherwise simple task is that a few files or folders need to remain. Imagine something like this:
/opt/application
- /config
- *.properties
- special.yml
- /logs
- /bin
- /var
- /data
- /templates
Let's assume I'd like to keep /logs completely, /var/data and from /config I want to keep special.yml.
(I cannot provide exact code at the moment because I left work frustrated by this and, after cooling down, I am now writing up this question at home)
My idea was to have two lists of exclusions, one holding the folders and one the file. Then I use the find module to first get the folders in the application's directory into a variable and the same for the remaining files into another variable. Afterwards I wanted to remove every folder and file that are not in the lists of exclusions using the file module.
(Pseudo-YML because I'm not yet fluent enough in Ansible that I can whip up a properly structured example; it should be close enough though)
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ found_files_list.files }}"
when: well, that is the big question
What I can't figure out is how to properly construct the when clause. Is it even possible like this?
I don't believe there is a when clause with the file module.
But you can probably achieve what you need as follows:
- name: Find /opt/application all directories, exclude logs, data, and config
find:
paths: /opt/application
excludes: 'logs,data,config'
register: files_to_delete
- name: Ansible remove file glob
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"
I hope this is what you need.
First use the find module like you said to get a total list of all files and directories. Register to a variable like all_objects.
- name: Get list of all files recursively
find:
path: /opt/application/
recurse: yes
register: all_objects
Then manually make a list of things you want to keep.
vars:
keep_these:
- /logs
- /var/data
- /config/special.yml
Then this task should delete everything except things in your list:
- name: Delete all files and directories except exclusions
file:
path: "{{ item.path }}"
state: absent
recurse: true
with_items: "{{ all_objects.files }}"
when: item.path not in keep_these
I think this general strategy should work... only thing I'm not sure about is the exact nesting heiararchy of the registered variable from the find module. You might have to play around with the debug module to get it exactly right.
I am using ansible to move .js files from my local machine to an ec2 development environment and am having an issue copying the entire folder structure.
I am using the following task to move the files and seem to be running into an issue where only the files directly in the dist folder are getting copied. I need to copy the entire folder including the child files and folders to the destination folder.
- name: Copy each file over that matches the given pattern
copy:
src: "{{ item }}"
dest: "/home/admin/microservice/dist"
owner: "admin"
group: "admin"
force: "yes"
recurse: "true"
mode: 0755
with_fileglob:
- "/Users/myfolder/WebStormProjects/project/microservice/dist/*.js"
I need to copy the entire folder contents from the source to the destination, including subfolders and files? What can I do to fix this task to make this happen?
With the copy module, the solution to your problem would be much more complicated than you think, because:
you can't match a directory and *.js files in a single globbing operation,
even if you could, you can't use the same "copy" operation to copy the file as well as create a directory (notice: create directory, not copy! as the latter would imply copying with all the files).
You'd need to handle the directories and files separately (see an implementation in the first revision of this answer).
With rsync, the solution is much more concise and requires only setting appropriate filters --include='*/' --include='*.js' --exclude='*'.
The synchronize task implementing this in Ansible:
- synchronize:
src: /source/Users/myfolder/WebStormProjects/project/microservice/dist/
dest: /home/admin/microservice/dist/
rsync_opts:
- --include=*/
- --include=*.js
- --exclude=*
Note 1: it is important not to add quotes for the filtered values in rsync_opts.
Note 2: you might still need to set the appropriate ownership and permissions.
First using copy module here shouldn't be ideal as "The copy module recursively copy facility does not scale to lots (>hundreds) of files. For alternative, see synchronize module, which is a wrapper around rsync."
copy module documentation
synchonize module documentation
However you can perform as below using copy module::
copy:
src: "{{ item }}"
dest: /home/admin/microservice/dist
with_lines: "find /home/admin/microservice/dist -type f -name *.js "
Similarly you can try with "synchronize" module as below::
synchronize:
src: "{{ item }}"
dest: /home/admin/microservice/dist
with_lines: "find /home/admin/microservice/dist -type f -name *.js "
If you want to retain the Directory layout, You can do it as below::
In Step1=> You will copy the necessary pattern files in parent directory structure into a temp directory.
Step2=>Then you need to copy the temp directory into the destination. After wards you can delete the temp directory or whatever your use case is.
- name: copy pattern files and directory into a temp directory
shell: find . -type f -name "*.js" | cpio -pvdmB /temp/dir/
args:
chdir: "/Users/myfolder/WebStormProjects/project/microservice/dist/"
- name: Copy the temp directory recursively to destination directory
copy:
src: "/temp/dir/"
dest: "/home/admin/microservice/dist/"
owner: "admin"
group: "admin"
force: "yes"
mode: 0755