I am trying to copy the content of dist directory to nginx directory.
- name: copy html file
copy: src=/home/vagrant/dist/ dest=/usr/share/nginx/html/
But when I execute the playbook it throws an error:
TASK [NGINX : copy html file] **************************************************
fatal: [172.16.8.200]: FAILED! => {"changed": false, "failed": true, "msg": "attempted to take checksum of directory:/home/vagrant/dist/"}
How can I copy a directory that has another directory and a file inside?
You could use the synchronize module. The example from the documentation:
# Synchronize two directories on one remote host.
- synchronize:
src: /first/absolute/path
dest: /second/absolute/path
delegate_to: "{{ inventory_hostname }}"
This has the added benefit that it will be more efficient for large/many files.
EDIT: This solution worked when the question was posted. Later Ansible deprecated recursive copying with remote_src
Ansible Copy module by default copies files/dirs from control machine to remote machine. If you want to copy files/dirs in remote machine and if you have Ansible 2.0, set remote_src to yes
- name: copy html file
copy: src=/home/vagrant/dist/ dest=/usr/share/nginx/html/ remote_src=yes directory_mode=yes
To copy a directory's content to another directory you CAN use ansibles copy module:
- name: Copy content of directory 'files'
copy:
src: files/ # note the '/' <-- !!!
dest: /tmp/files/
From the docs about the src parameter:
If (src!) path is a directory, it is copied recursively...
... if path ends with "/", only inside contents of that directory are copied to destination.
... if it does not end with "/", the directory itself with all contents is copied.
Resolved answer:
To copy a directory's content to another directory I use the next:
- name: copy consul_ui files
command: cp -r /home/{{ user }}/dist/{{ item }} /usr/share/nginx/html
with_items:
- "index.html"
- "static/"
It copies both items to the other directory. In the example, one of the items is a directory and the other is not. It works perfectly.
The simplest solution I've found to copy the contents of a folder without copying the folder itself is to use the following:
- name: Move directory contents
command: cp -r /<source_path>/. /<dest_path>/
This resolves #surfer190's follow-up question:
Hmmm what if you want to copy the entire contents? I noticed that * doesn't work – surfer190 Jul 23 '16 at 7:29
* is a shell glob, in that it relies on your shell to enumerate all the files within the folder before running cp, while the . directly instructs cp to get the directory contents (see https://askubuntu.com/questions/86822/how-can-i-copy-the-contents-of-a-folder-to-another-folder-in-a-different-directo)
Ansible remote_src does not support recursive copying.See remote_src description in Ansible copy docs
To recursively copy the contents of a folder and to make sure the task stays idempotent I usually do it this way:
- name: get file names to copy
command: "find /home/vagrant/dist -type f"
register: files_to_copy
- name: copy files
copy:
src: "{{ item }}"
dest: "/usr/share/nginx/html"
owner: nginx
group: nginx
remote_src: True
mode: 0644
with_items:
- "{{ files_to_copy.stdout_lines }}"
Downside is that the find command still shows up as 'changed'
the ansible doc is quite clear https://docs.ansible.com/ansible/latest/collections/ansible/builtin/copy_module.html for parameter src it says the following:
Local path to a file to copy to the remote server.
This can be absolute or relative.
If path is a directory, it is copied recursively. In this case, if path ends with "/",
only inside contents of that directory are copied to destination. Otherwise, if it
does not end with "/", the directory itself with all contents is copied. This behavior
is similar to the rsync command line tool.
So what you need is skip the / at the end of your src path.
- name: copy html file
copy: src=/home/vagrant/dist dest=/usr/share/nginx/html/
I found a workaround for recursive copying from remote to remote :
- name: List files in /usr/share/easy-rsa
find:
path: /usr/share/easy-rsa
recurse: yes
file_type: any
register: find_result
- name: Create the directories
file:
path: "{{ item.path | regex_replace('/usr/share/easy-rsa','/etc/easy-rsa') }}"
state: directory
mode: "{{ item.mode }}"
with_items:
- "{{ find_result.files }}"
when:
- item.isdir
- name: Copy the files
copy:
src: "{{ item.path }}"
dest: "{{ item.path | regex_replace('/usr/share/easy-rsa','/etc/easy-rsa') }}"
remote_src: yes
mode: "{{ item.mode }}"
with_items:
- "{{ find_result.files }}"
when:
- item.isdir == False
I got involved whole a day, too! and finally found the solution in shell command instead of copy: or command: as below:
- hosts: remote-server-name
gather_facts: no
vars:
src_path: "/path/to/source/"
des_path: "/path/to/dest/"
tasks:
- name: Ansible copy files remote to remote
shell: 'cp -r {{ src_path }}/. {{ des_path }}'
strictly notice to:
1. src_path and des_path end by / symbol
2. in shell command src_path ends by . which shows all content of directory
3. I used my remote-server-name both in hosts: and execute shell
section of jenkins, instead of remote_src: specifier in playbook.
I guess it is a good advice to run below command in Execute Shell section in jenkins:
ansible-playbook copy-payment.yml -i remote-server-name
Below worked for me,
-name: Upload html app directory to Deployment host
copy: src=/var/lib/jenkins/workspace/Demoapp/html dest=/var/www/ directory_mode=yes
This I found an ideal solution for copying file from Ansible server to remote.
copying yaml file
- hosts: localhost
user: {{ user }}
connection: ssh
become: yes
gather_facts: no
tasks:
- name: Creation of directory on remote server
file:
path: /var/lib/jenkins/.aws
state: directory
mode: 0755
register: result
- debug:
var: result
- name: get file names to copy
command: "find conf/.aws -type f"
register: files_to_copy
- name: copy files
copy:
src: "{{ item }}"
dest: "/var/lib/jenkins/.aws"
owner: {{ user }}
group: {{ group }}
remote_src: True
mode: 0644
with_items:
- "{{ files_to_copy.stdout_lines }}"
How to copy directory and sub dirs's and files from ansible server to remote host
- name: copy nmonchart39 directory to {{ inventory_hostname }}
copy:
src: /home/ansib.usr.srv/automation/monitoring/nmonchart39
dest: /var/nmon/data
Where:
copy entire directory: src: /automation/monitoring/nmonchart39
copy directory contents src: nmonchart39/
Related
I have an integration where I download one or more ZIP files. Within those ZIP files, there are dozens of GZ files that also need to be uncompressed. Below is an example of the file structure:
metrics.zip
-> 239238923323.gz
-> 839389239232.gz
-> 928392892839.gz
metrics-001.zip
-> 29389238923.gz
-> 39828393822.gz
-> 09320930323.gz
(etc)
I was struggling to write the playbook needed to loop through the ZIP file(s), then all of the GZ files and uncompress them all.
Created an answer from the author's original post:
- hosts: localhost
gather_facts: no
tasks:
# Download the report into a temporary directory on the Ansible Playbook host
- name: Create temporary directory
ansible.builtin.tempfile:
state: directory
suffix: unique_suffix
register: temp_dir
- name: Download File
ansible.builtin.get_url:
url: "https://path.to/file/download.zip"
dest: "{{ temp_dir.path }}/download_file_name.zip"
# Unzip the ZIP files
- name: Extract all ZIP files
ansible.builtin.unarchive:
src: "{{ item }}"
dest: "{{ temp_dir.path }}"
with_fileglob:
- "{{ temp_dir.path }}/*.zip"
# Unzip the GZ files
- name: Extract all of the GZ files
ansible.builtin.command: find "{{ temp_dir.path }}" -name '*.gz' -exec gzip -d {} \;
- name: Merge CSVs into a single file
ansible.builtin.assemble:
src: "{{ temp_dir.path }}"
dest: "{{ temp_dir.path }}/extract.x"
regexp: '\.csv$'
# Use a Windows host to copy the file over to a file share (copying from Linux to a Windows file share requires mounting the FS. Using a Windows host is easier)
- hosts: "{{ windows_host_name_in_inventory }}"
tasks:
- name: Copy Extract file to File Share
win_copy:
src: "{{ hostvars['localhost']['temp_dir'].path }}/extract.x"
dest: "{{ Extract_To }}\\unique_name.csv"
# Remove the Temporary folder
- hosts: localhost
tasks:
- name: Remove Temporary Directory
ansible.builtin.file:
path: "{{ hostvars['localhost']['temp_dir'].path }}"
state: absent
when: hostvars['localhost']['temp_dir'].path is defined
In my files directory I have various files, with a similar name structure:
data-example.zip
data-precise.zip
data-arbitrary.zip
data-collected.zip
I would like to transfer all of these files in the /tmp directory of my remote machine using Ansible without specifying each file name explicitly.
In other words I would like to transfer every file that stars with "data-".
What is the correct way to do that? In a similar thread, someone suggested the with_fileglob keyword, - but I couldn't get that to work.
Can someone provide me an example on how to accomplish said task?
Method 1: Find all files, store them in a variable and copy them to destination.
- hosts: lnx
tasks:
- find: paths="/source/path" recurse=yes patterns="data*"
register: file_to_copy
- copy: src={{ item.path }} dest=/dear/dir
owner: root
mode: 0775
with_items: "{{ files_to_copy.files }}"
Use remote_src: yes to copy file in remote machine from one path to another.
Ansible documentation
Method 2: Fileglob
- name: Copy each file over that matches the given pattern
copy:
src: "{{ item }}"
dest: "/etc/fooapp/"
owner: "root"
mode: 0600
with_fileglob:
- "/playbooks/files/fooapp/*"
Ansible documentation
Shortly after posting the question I actually figured it out myself. The with_fileglob keyword is the way to do it.
- name: "Transferring all data files"
copy:
src: "{{ item }}"
dest: /tmp/
with_fileglob: "data-*"
I have a task , where should I copy a file from its source , to its destination while renaming it in the destination .
My task looks like this :
- name: Go to the target folder
shell: ls
args:
chdir: "{{pathTest}}/target"
register: resultLS
- debug:
msg: "{{resultLS}}"
- name: copy jar file
copy:
src: "{{resultLS.stdout}}"
dest: "{{pathTest}}"
mode: 0777
But, like this it copies the jar file with its same name , my purpose is how to rename it in the dest (ideally with the copy action)
Ideas ?
rename it to: renamed.jar
Here you are:
- name: Ensure the first matched file from {{ pathTest }}/target is present on the target
copy:
src: "{{ lookup('fileglob', pathTest + '/target/*') | first }}"
dest: "{{ pathTest }}/renamed.jar"
mode: 0777
Remarks:
Don't parse ls output!
Think how you should handle multiple files.
in the example above - copy only the first one
I'm trying to write a playbook that will rsync the folders from source to target after a database refresh. Our Peoplesoft HR application also requires a filesystem refresh along with database. I'm new to ansible and not an expert with python. I've written this but my playbook fails if any of the with_items doesn't exist. I'd like to use this playbook for all apps and the folders may differ between apps. How can I skip the folders that doesn't exist in source. I'm passing {{ target }} at command line.
---
- hosts: '<hostname>'
remote_user: <user>
tasks:
- shell: ls -l /opt/custhome/prod/
register: folders
- name: "Copy PROD filesystem to target"
synchronize:
src: "/opt/custhome/prod/{{ item }}"
dest: "/opt/custhome/dev/"
delete: yes
when: "{{ folders == item }}"
with_items:
- 'src/cbl/'
- 'sqr/'
- 'bin/'
- 'NVISION/'
In this case, NVISION doesn't exist in HR app but it does in FIN app. But the playbook is failing coz that folder doesn't exist in source.
You can use find module to find and store paths to source folders and then to iterate over results. Example playbook:
- hosts: '<hostname>'
remote_user: <user>
tasks:
- name: find all directories
find:
file_type: directory
paths: /opt/custhome/prod/
patterns:
- "src"
- "sqr"
- "bin"
register: folders
#debug to understand contents of {{ folders }} variable
# - debug: msg="{{ folders }}"
- name: "Copy PROD filesystem to target"
synchronize:
src: "{{ item.path }}"
dest: "/opt/custhome/dev/"
delete: yes
with_items: "{{ folders.files }}"
You may want to use recurse to descend into subdirectories and use_regex to use the power of python regex instead of shell globbing.
I'm trying to write an Ansible role that moves a number of files on the remote system. I found a Stack Overflow post about how to do this, which essentially says "just use the command module with 'mv'". I have a single task defined with a with_items statement like this where each item in dirs is a dictionary with src and dest keys:
- name: Move directories
command: mv {{ item.src }} {{ item.dest }}
with_items: dirs
This is good and it works, but I run into problems if the destination directory already exists. I don't want to overwrite it, so I thought about trying to stat each dest directory first. I wanted to update the dirs variable with the stat info, but as far as I know, there isn't a good way to set or update variables once they're defined. So I used stat to get the info on each directory and then saved the data with register:
- name: Check if directories already exist
stat: path={{ item.dest }}
with_items: dirs
register: dirs_stat
Is there a way to tie the registered stat info to the mv commands? This would be easy if it were a single directory. The looping is what makes this tricky. Is there a way to do this without unrolling this loop into two tasks per directory?
This is not the simplest solution by any means, but if you wanted to use Ansible and not "unroll":
---
- hosts: all
vars:
dirs:
- src: /home/ubuntu/src/test/src1
dest: /home/ubuntu/src/test/dest1
- src: /home/ubuntu/src/test/src2
dest: /home/ubuntu/src/test/dest2
tasks:
- stat:
path: "{{item.dest}}"
with_items: dirs
register: dirs_stat
- debug:
msg: "should not copy {{ item.0.src }}"
with_together:
- dirs
- dirs_stat.results
when: item.1.stat.exists
Simply adapt the debug task to run the appropriate command task instead and when: to when: not ....
You can use stat keyword in your playbook to check if it exists or not if it doesn't then move.
---
- name: Demo Playbook
hosts: all
become: yes
tasks:
- name: check destination
stat:
path: /path/to/dest
register: p
- name: copy file if not exists
command: mv /path/to/src /path/to/src
when: p.stat.exists == False