In my files directory I have various files, with a similar name structure:
data-example.zip
data-precise.zip
data-arbitrary.zip
data-collected.zip
I would like to transfer all of these files in the /tmp directory of my remote machine using Ansible without specifying each file name explicitly.
In other words I would like to transfer every file that stars with "data-".
What is the correct way to do that? In a similar thread, someone suggested the with_fileglob keyword, - but I couldn't get that to work.
Can someone provide me an example on how to accomplish said task?
Method 1: Find all files, store them in a variable and copy them to destination.
- hosts: lnx
tasks:
- find: paths="/source/path" recurse=yes patterns="data*"
register: file_to_copy
- copy: src={{ item.path }} dest=/dear/dir
owner: root
mode: 0775
with_items: "{{ files_to_copy.files }}"
Use remote_src: yes to copy file in remote machine from one path to another.
Ansible documentation
Method 2: Fileglob
- name: Copy each file over that matches the given pattern
copy:
src: "{{ item }}"
dest: "/etc/fooapp/"
owner: "root"
mode: 0600
with_fileglob:
- "/playbooks/files/fooapp/*"
Ansible documentation
Shortly after posting the question I actually figured it out myself. The with_fileglob keyword is the way to do it.
- name: "Transferring all data files"
copy:
src: "{{ item }}"
dest: /tmp/
with_fileglob: "data-*"
I am using unarchive module in ansible to extract tar.gz files to a folder. After that, i want to copy the extracted folder to another location. Is there a ansible way of doing it? I tried below , it does not work
- name : Extract Files
unarchive:
src: "/home/app/{{ item }}"
dest: "/home/app/"
mode: 0777
copy: no
with_items:
- "{{ sharef }}"
delegate_to: localhost
register: extractedfiles
- name : Copy Files
copy:
src: "{{extractedfiles.results}}"
dest: "{{ location }}"
delegate_to: localhost
How do i extract the filie names from the previously executed task extractedfiles.results does not work. Any help would be appreciated.
How would you resolve this small script in ansible playbook ?
Files to copy are named [ServerName].[extension]
The destination server is ServeName
for file in $(ls /var/tmp)
do
ServerName=$(echo $file | awk -F. 'NF{NF--};1'
scp /var/tmp/$file $ServerName:/var/tmp/
scp /var/tmp/pkg.rpm $ServerName:/var/tmp/
ssh $ServerName "cd /var/tmp; yum -y localinstall pkg.rpm "
done
Thanks for your help
The idea would be to have something like this (but working, of course)
- name: main loop
copy:
src: "{{ item }}"
dest: "/var/tmp/myfile.json"
- name: Install package
yum:
name: "packageToInstall"
state: present
delegate_to: "{{ item.split('/')[-1][:-5] }}"
with_fileglob:
- "/var/temp/*json"
If you were to write this in yaml, you should use ansible_hostname, ansible already has several informations about the host via setup.
- name: copying files
copy:
src: /mine/file.ext
dest: /etc/{{ ansible_hostname }}.ext
More about copy in module description.
All facts gather during setup are available here.
With mode=pull, I want to fetch and save remote files to the "dest" directory per hostname under the same top-level directory tree.
This is what I want:
src=/proc/cpuinfo (of every Ansible inventory host)
dest=/tmp/host1/cpuinfo, /tmp/host2/cpuinfo, /tmp/host3/cpuinfo, etc. (of the Ansible master)
If I do,
ansible all -m synchronize 'src=/proc/cpuinfo dest=/tmp/cpuinfo mode=pull'
/tmp/cpuinfo file on the Ansible master (= dest) gets overwritten by every remote host's cpuinfo file and I get to see only the very last one.
That is, I want a similar behavior as if I run
ansible all -m fetch -a 'src=/proc/cpuinfo dest=/tmp/cpuinfo'
Thank you in advance!
Steve
I doubt you can do this with single ad-hoc command.
ansible all -m synchronize -a 'src=/proc/cpuinfo dest=/tmp/{{inventory_hostname}}/cpuinfo mode=pull'
could do the thing, but you must create /tmp/<hostname> directories in advance, because rsync doesn't create non-existent directories for you. And you can't use ansible facts (like ansible_hostname and ansible_fqdn) as parameters for ad-hoc module execution - only "predefined" variables (like inventory_hostname).
Update: playbook code
- file:
path: "/tmp/{{ inventory_hostname }}"
state: directory
delegate_to: localhost
- synchronize:
src: /proc/cpuinfo
dest: "/tmp/{{ inventory_hostname }}/cpuinfo"
mode: pull
(Original poster)
Another way to do it using the synchronize module only:
- synchronize:
src: "{{ item.src }}"
dest: "{{ item.dest }}"
mode: pull
with_items:
- { src: '/proc/cpuinfo', dest: '/tmp/testing/{{ inventory_hostname }}/proc' }
- { src: '/proc/meminfo', dest: '/tmp/testing/{{ inventory_hostname }}/proc' }
- { src: '/etc/services', dest: '/tmp/testing/{{ inventory_hostname }}/etc' }
I am trying to copy the content of dist directory to nginx directory.
- name: copy html file
copy: src=/home/vagrant/dist/ dest=/usr/share/nginx/html/
But when I execute the playbook it throws an error:
TASK [NGINX : copy html file] **************************************************
fatal: [172.16.8.200]: FAILED! => {"changed": false, "failed": true, "msg": "attempted to take checksum of directory:/home/vagrant/dist/"}
How can I copy a directory that has another directory and a file inside?
You could use the synchronize module. The example from the documentation:
# Synchronize two directories on one remote host.
- synchronize:
src: /first/absolute/path
dest: /second/absolute/path
delegate_to: "{{ inventory_hostname }}"
This has the added benefit that it will be more efficient for large/many files.
EDIT: This solution worked when the question was posted. Later Ansible deprecated recursive copying with remote_src
Ansible Copy module by default copies files/dirs from control machine to remote machine. If you want to copy files/dirs in remote machine and if you have Ansible 2.0, set remote_src to yes
- name: copy html file
copy: src=/home/vagrant/dist/ dest=/usr/share/nginx/html/ remote_src=yes directory_mode=yes
To copy a directory's content to another directory you CAN use ansibles copy module:
- name: Copy content of directory 'files'
copy:
src: files/ # note the '/' <-- !!!
dest: /tmp/files/
From the docs about the src parameter:
If (src!) path is a directory, it is copied recursively...
... if path ends with "/", only inside contents of that directory are copied to destination.
... if it does not end with "/", the directory itself with all contents is copied.
Resolved answer:
To copy a directory's content to another directory I use the next:
- name: copy consul_ui files
command: cp -r /home/{{ user }}/dist/{{ item }} /usr/share/nginx/html
with_items:
- "index.html"
- "static/"
It copies both items to the other directory. In the example, one of the items is a directory and the other is not. It works perfectly.
The simplest solution I've found to copy the contents of a folder without copying the folder itself is to use the following:
- name: Move directory contents
command: cp -r /<source_path>/. /<dest_path>/
This resolves #surfer190's follow-up question:
Hmmm what if you want to copy the entire contents? I noticed that * doesn't work – surfer190 Jul 23 '16 at 7:29
* is a shell glob, in that it relies on your shell to enumerate all the files within the folder before running cp, while the . directly instructs cp to get the directory contents (see https://askubuntu.com/questions/86822/how-can-i-copy-the-contents-of-a-folder-to-another-folder-in-a-different-directo)
Ansible remote_src does not support recursive copying.See remote_src description in Ansible copy docs
To recursively copy the contents of a folder and to make sure the task stays idempotent I usually do it this way:
- name: get file names to copy
command: "find /home/vagrant/dist -type f"
register: files_to_copy
- name: copy files
copy:
src: "{{ item }}"
dest: "/usr/share/nginx/html"
owner: nginx
group: nginx
remote_src: True
mode: 0644
with_items:
- "{{ files_to_copy.stdout_lines }}"
Downside is that the find command still shows up as 'changed'
the ansible doc is quite clear https://docs.ansible.com/ansible/latest/collections/ansible/builtin/copy_module.html for parameter src it says the following:
Local path to a file to copy to the remote server.
This can be absolute or relative.
If path is a directory, it is copied recursively. In this case, if path ends with "/",
only inside contents of that directory are copied to destination. Otherwise, if it
does not end with "/", the directory itself with all contents is copied. This behavior
is similar to the rsync command line tool.
So what you need is skip the / at the end of your src path.
- name: copy html file
copy: src=/home/vagrant/dist dest=/usr/share/nginx/html/
I found a workaround for recursive copying from remote to remote :
- name: List files in /usr/share/easy-rsa
find:
path: /usr/share/easy-rsa
recurse: yes
file_type: any
register: find_result
- name: Create the directories
file:
path: "{{ item.path | regex_replace('/usr/share/easy-rsa','/etc/easy-rsa') }}"
state: directory
mode: "{{ item.mode }}"
with_items:
- "{{ find_result.files }}"
when:
- item.isdir
- name: Copy the files
copy:
src: "{{ item.path }}"
dest: "{{ item.path | regex_replace('/usr/share/easy-rsa','/etc/easy-rsa') }}"
remote_src: yes
mode: "{{ item.mode }}"
with_items:
- "{{ find_result.files }}"
when:
- item.isdir == False
I got involved whole a day, too! and finally found the solution in shell command instead of copy: or command: as below:
- hosts: remote-server-name
gather_facts: no
vars:
src_path: "/path/to/source/"
des_path: "/path/to/dest/"
tasks:
- name: Ansible copy files remote to remote
shell: 'cp -r {{ src_path }}/. {{ des_path }}'
strictly notice to:
1. src_path and des_path end by / symbol
2. in shell command src_path ends by . which shows all content of directory
3. I used my remote-server-name both in hosts: and execute shell
section of jenkins, instead of remote_src: specifier in playbook.
I guess it is a good advice to run below command in Execute Shell section in jenkins:
ansible-playbook copy-payment.yml -i remote-server-name
Below worked for me,
-name: Upload html app directory to Deployment host
copy: src=/var/lib/jenkins/workspace/Demoapp/html dest=/var/www/ directory_mode=yes
This I found an ideal solution for copying file from Ansible server to remote.
copying yaml file
- hosts: localhost
user: {{ user }}
connection: ssh
become: yes
gather_facts: no
tasks:
- name: Creation of directory on remote server
file:
path: /var/lib/jenkins/.aws
state: directory
mode: 0755
register: result
- debug:
var: result
- name: get file names to copy
command: "find conf/.aws -type f"
register: files_to_copy
- name: copy files
copy:
src: "{{ item }}"
dest: "/var/lib/jenkins/.aws"
owner: {{ user }}
group: {{ group }}
remote_src: True
mode: 0644
with_items:
- "{{ files_to_copy.stdout_lines }}"
How to copy directory and sub dirs's and files from ansible server to remote host
- name: copy nmonchart39 directory to {{ inventory_hostname }}
copy:
src: /home/ansib.usr.srv/automation/monitoring/nmonchart39
dest: /var/nmon/data
Where:
copy entire directory: src: /automation/monitoring/nmonchart39
copy directory contents src: nmonchart39/