How to copy first file to first host, second file to second host and so on via ansible? [closed] - ansible

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 13 days ago.
Improve this question
I have 10 hosts and 10 files. I want to copy one of 10 files to corresponding host:
1_file.txt -> 1_host
2_file.txt -> 2_host
...
10_file.txt -> 10_host
I've tried to do it with zip filter, like
- name: Copy splitted files to instances
copy:
src: "{{ item.0 }}"
dest: "{{ remote_ips_file }}"
delegate_to: "{{ item.1 }}"
loop: "{{ splitted_files.files | zip(instances) | list }}"
but got an error:
{"msg": "Unexpected failure during module execution.", "stdout": ""}
Another way (from ChatGPT) was:
- name: Copy files to hosts
hosts: "{{ hosts }}"
tasks:
- name: Copy file to host
copy:
src: "{{ item.0 }}"
dest: "/tmp/{{ item.1 }}"
loop: "{{ zip(files, hosts) }}"
vars:
files: ['/path/to/file1', '/path/to/file2']
hosts: ['host1', 'host2']
But also got an error:
zip is undefined
How to do it with proper way?
Edit
I've found workaround, but it works not in parallel:
- name: Copy splitted files to instances
copy:
src: "{{ item.0.path }}"
dest: "{{ remote_ips_file }}"
delegate_to: "{{ item.1 }}"
with_together:
- "{{ splitted_files.files }}"
- "{{groups['all']}}"

Given the lists
my_files: [file1, file2]
my_hosts: [host1, host2]
Create the dictionary
hosts_files: "{{ dict(my_hosts|zip(my_files)) }}"
gives
hosts_files:
host1: file1
host2: file2
Use the dictionary in the task
- copy:
src: "{{ hosts_files[inventory_hostname] }}"
dest: /tmp
Example of a complete project for testing
shell> tree .
.
├── ansible.cfg
├── files
│   ├── file1
│   └── file2
├── hosts
└── pb.yml
1 directory, 5 files
shell> cat ansible.cfg
[defaults]
gathering = explicit
collections_path = $HOME/.local/lib/python3.9/site-packages/
inventory = $PWD/hosts
roles_path = $PWD/roles
remote_tmp = ~/.ansible/tmp
retry_files_enabled = false
stdout_callback = yaml
shell> cat hosts
[test]
host1 ansible_host=10.1.0.61
host2 ansible_host=10.1.0.63
[test:vars]
ansible_connection=ssh
ansible_user=admin
ansible_become=yes
ansible_become_user=root
ansible_become_method=sudo
ansible_python_interpreter=/usr/local/bin/python3.8
ansible_perl_interpreter=/usr/local/bin/perl
shell> cat pb.yml
- hosts: host1,host2
vars:
my_files: [file1, file2]
my_hosts: [host1, host2]
hosts_files: "{{ dict(my_hosts|zip(my_files)) }}"
tasks:
- debug:
var: hosts_files
run_once: true
- copy:
src: "{{ hosts_files[inventory_hostname] }}"
dest: /tmp
shell> ansible-playbook pb.yml
PLAY [host1,host2] ***************************************************************************
TASK [debug] *********************************************************************************
ok: [host1] =>
hosts_files:
host1: file1
host2: file2
TASK [copy] **********************************************************************************
changed: [host1]
changed: [host2]
PLAY RECAP ***********************************************************************************
host1: ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
host2: ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
shell> ssh admin#10.1.0.61 ls -1 /tmp/file1
/tmp/file1
shell> ssh admin#10.1.0.63 ls -1 /tmp/file2
/tmp/file2

Related

With ansible.posix.synchronize, is there a way to a source folder that may not exist?

tasks:
- name: sync folders
loop: "{{ folder_list | list }}"
ansible.posix.synchronize:
src: "/path/folder/{{ item }}"
dest: "/other_node/folders/"
archive: false
recursive: true
perms: true
checksum: true
delete: true
The list of folders folder_list is defined somewhere else.
I do not have control of this, and cannot change it. I also do not know the folder list up front, so cannot set it statically.
It may contain items that do not exist on "this" machine.
Is there a way to have the sync task skip such items? I found stat. Looks like it can be used to check for the existence of a file or folder, but I couldn't figure out how to use it, set_fact, and synchronize together within the task to accomplish this.
What I'm trying to do is something like:
Loop through folder list > if source folder exists > sync folder to destination.
PS: Please let me know if this belongs on ServerFault instead.
The testing paths on the controller is simple. For example, given the tree
shell> tree /tmp/export/
/tmp/export/
├── dir1
│   ├── a
│   └── b
└── dir2
└── c
2 directories, 3 files
The playbook below skips missing folders
shell> cat pb.yml
- hosts: test_11
vars:
folder_list:
- /tmp/export/dir1
- /tmp/export/dir2
- /tmp/export/dir3
tasks:
- debug:
msg: "synchronize {{ item }}"
loop: "{{ folder_list }}"
when: item is directory
gives
shell> ansible-playbook pb.yml
PLAY [test_11] ***********************************************************************************************
TASK [debug] *************************************************************************************************
ok: [test_11] => (item=/tmp/export/dir1) =>
msg: synchronize /tmp/export/dir1
ok: [test_11] => (item=/tmp/export/dir2) =>
msg: synchronize /tmp/export/dir2
skipping: [test_11] => (item=/tmp/export/dir3)
PLAY RECAP ***************************************************************************************************
test_11: ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You can combine the conditions. For example,
when: item is directory or item is link
, or simply test the existence only
when: item is exists
Note: The module stat examines the files at the current host. You have to delegate this task to localhost if you want to use it here. Register the results and declare the variable below
folder_exists: "{{ dict(folder_list_stat.results|
json_query('[].[item, stat.exists]')) }}"
- block:
- stat:
path: "{{ item }}"
loop: "{{ folder_list }}"
register: folder_list_stat
- debug:
var: folder_list_stat
when: debug|d(false)|bool
- debug:
var: folder_exists
when: debug|d(false)|bool
delegate_to: localhost
run_once: true
gives the dictionary
folder_exists:
/tmp/export/dir1: true
/tmp/export/dir2: true
/tmp/export/dir3: false
Then, the condition is trivial. See the debug on what other attributes are available in folder_list_stat and create other dictionaries for testing if you want to.
Example of a complete playbook for testing
- hosts: test_11
vars:
folder_list:
- /tmp/export/dir1
- /tmp/export/dir2
- /tmp/export/dir3
folder_exists: "{{ dict(folder_list_stat.results|
json_query('[].[item, stat.exists]')) }}"
tasks:
- block:
- stat:
path: "{{ item }}"
loop: "{{ folder_list }}"
register: folder_list_stat
- debug:
var: folder_list_stat
when: debug|d(false)|bool
- debug:
var: folder_exists
when: debug|d(false)|bool
delegate_to: localhost
run_once: true
- debug:
msg: "synchronize {{ item }}"
loop: "{{ folder_list }}"
when: folder_exists[item]

Display Ansible playbook with lookups interpolated

I have an Ansible playbook that looks, in part, like this:
...
environment:
F2B_DB_PURGE_AGE: "{{ lookup('env','F2B_DB_PURGE_AGE') }}"
F2B_LOG_LEVEL: "{{ lookup('env','F2B_LOG_LEVEL') }}"
SSMTP_HOST: "{{ lookup('env','SSMTP_HOST') }}"
SSMTP_PORT: "{{ lookup('env','SSMTP_PORT') }}"
SSMTP_TLS: "{{ lookup('env','SSMTP_TLS') }}"
...
Is there any way to run ansible-playbook so that it will show the results of the YAML file after replacing the lookups with their values? That is, I would like to be able to run something like ansible-playbook file.yaml --dry-run and see on standard output (assuming the environment variables were set appropriately):
...
environment:
F2B_DB_PURGE_AGE: "20"
F2B_LOG_LEVEL: "debug"
SSMTP_HOST: "smtp.example.com"
SSMTP_PORT: "487"
SSMTP_TLS: "true"
...
Set the environment for testing
shell> cat env.sh
#!/usr/bin/bash
export F2B_DB_PURGE_AGE="20"
export F2B_LOG_LEVEL="debug"
export SSMTP_HOST="smtp.example.com"
export SSMTP_PORT="487"
export SSMTP_TLS="true"
shell> source env.sh
Given the inventory
shell> cat hosts
localhost ansible_connection=local
Q: "Run something like ansible-playbook file.yaml --dry-run and see environment"
A: The below playbook does the job
shell> cat file.yml
- hosts: all
vars:
my_environment:
F2B_DB_PURGE_AGE: "{{ lookup('env','F2B_DB_PURGE_AGE') }}"
F2B_LOG_LEVEL: "{{ lookup('env','F2B_LOG_LEVEL') }}"
SSMTP_HOST: "{{ lookup('env','SSMTP_HOST') }}"
SSMTP_PORT: "{{ lookup('env','SSMTP_PORT') }}"
SSMTP_TLS: "{{ lookup('env','SSMTP_TLS') }}"
tasks:
- block:
- debug:
msg: |
my_environment:
{{ my_environment|to_nice_yaml|indent(2) }}
- meta: end_play
when: dry_run|d(false)|bool
- debug:
msg: Continue ...
Set dry_run=true
shell> ansible-playbook file.yml -e dry_run=true
PLAY [all] ***********************************************************************************
TASK [debug] *********************************************************************************
ok: [localhost] =>
msg: |-
my_environment:
F2B_DB_PURGE_AGE: '20'
F2B_LOG_LEVEL: debug
SSMTP_HOST: smtp.example.com
SSMTP_PORT: '487'
SSMTP_TLS: 'true'
TASK [meta] **********************************************************************************
PLAY RECAP ***********************************************************************************
localhost: ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
By default, the playbook will execute tasks
shell> ansible-playbook file.yml
PLAY [all] ***********************************************************************************
TASK [debug] *********************************************************************************
skipping: [localhost]
TASK [meta] **********************************************************************************
skipping: [localhost]
TASK [debug] *********************************************************************************
ok: [localhost] =>
msg: Continue ...
PLAY RECAP ***********************************************************************************
localhost: ok=1 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
Optionally, let the playbook gather facts and use the dictionary ansible_env. Use the filer ansible.utils.keep_keys to select your variables
- hosts: all
gather_facts: true
vars:
my_environment_vars:
- F2B_DB_PURGE_AGE
- F2B_LOG_LEVEL
- SSMTP_HOST
- SSMTP_PORT
- SSMTP_TLS
my_environment: "{{ ansible_env|
ansible.utils.keep_keys(target=my_environment_vars) }}"
tasks:
- block:
- debug:
msg: |
my_environment:
{{ my_environment|to_nice_yaml|indent(2) }}
- meta: end_play
when: dry_run|d(false)|bool
- debug:
msg: Continue ...

How to execute on multiple hosts in ansible

I have a script that will execute in two parts. First it will execute on localhost and query a database table to get a hostname. second part of the script should run on the host which was registered in the query before. I am not able to set the host with the set_fact I did in the first part of the code.
this is what iam trying to do:
- hosts: localhost
gather_facts: false
become: yes
become_user: oracle
vars_files:
- vars/main.yml
tasks:
- name: Get new hostname
tempfile:
state: file
register: tf
- name: create sql file
template:
src: get_hostname.sql.j2
dest:"{{ tf.path }}"
mode: 0775
- name: login
command:
argv:
- "sqlplus"
- -s
- "#{{ tf.path }}"
environment:
ORACLE_HOME: "oracle/home"
register: command_out
- set_fact:
NEW_HOST: "{{ command_out.stdout }}"
- hosts: "{{ NEW_HOST }}"
gather_facts: false
become: yes
become_user: oracle
vars_file:
- vars/main.yml
tasks:
- name: debug
command: hostname
register: new_host_out
- debug:
msg: "new host is {{ new_host_out.stdout }}"
Everything works fine in the first part of the code, but errors out at the second part saying it cannot find the NEW_HOST.
Use hostvars to reference such a variable. Create a dummy host to keep this variable. For example, given the inventory
shell> cat hosts
dummy
[test]
test_11
test_12
test_13
The playbook creates the variable. See Delegated facts
shell> cat pb.yml
- hosts: localhost
tasks:
- set_fact:
NEW_HOST: test_12
delegate_to: dummy
delegate_facts: true
- debug:
var: hostvars.dummy.NEW_HOST
- hosts: "{{ hostvars.dummy.NEW_HOST }}"
gather_facts: false
tasks:
- debug:
var: inventory_hostname
gives
shell> ansible-playbook pb.yml
PLAY [localhost] ****************************************************************************
TASK [set_fact] *****************************************************************************
ok: [localhost -> dummy]
TASK [debug] ********************************************************************************
ok: [localhost] =>
hostvars.dummy.NEW_HOST: test_12
PLAY [test_12] ******************************************************************************
TASK [debug] ********************************************************************************
ok: [test_12] =>
inventory_hostname: test_12
PLAY RECAP **********************************************************************************
localhost: ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
test_12 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You can use localhost for this purpose as well. The playbook below works as expected
- hosts: localhost
tasks:
- set_fact:
NEW_HOST: test_12
- hosts: "{{ hostvars.localhost.NEW_HOST }}"
gather_facts: false
tasks:
- debug:
var: inventory_hostname

Looping through a product of two arrays

I have this vars file in Ansible:
for_create:
client: ["VK","SB"]
folders: ["toula","tina"]
for_delete:
client: ["VK","SB"]
folders: ["invoices","scripts"]
for_rename:
client: ["VK", "SB"]
old_name: ["home"]
new_name: ["town"]
Is it possible to do something like that in YAML? The following code in Ruby
clients = ["VK", "SB"]
folders = ["toula","tina"]
clients.each do |client|
folders.each do |folder|
puts "folder #{folder} for client #{client} created"
end
end
To sum up I want each client to create the folders ["toula", "tina"]
I have tried a lot but I can't manage to make it loop twice for each folder.
The output of the above code
folder toula for client VK created
folder tina for client VK created
folder toula for client SB created
folder tina for client SB created
Iterate the lists with_nested. For example,
- debug:
msg: "folder {{ item.1 }} for client {{ item.0 }} created"
with_nested:
- "{{ for_create.client }}"
- "{{ for_create.folders }}"
vars:
for_create:
client: [VK, SB]
folders: [toula, tina]
gives (abridged)
msg: folder toula for client VK created
msg: folder tina for client VK created
msg: folder toula for client SB created
msg: folder tina for client SB created
To rename folders zip the lists
- debug:
msg: "folder {{ item.1 }} renamed to {{ item.2 }} for client {{ item.0 }}"
with_nested:
- "{{ for_rename.client }}"
- "{{ for_rename.old_name|zip(for_rename.new_name) }}"
vars:
for_rename:
client: [VK, SB]
old_name: [home]
new_name: [town]
gives (abridged)
msg: folder home renamed to town for client VK
msg: folder home renamed to town for client SB
Q: "Is it possible to check if the folder exists inside the loop?"
A: Yes. It is. Use the parameters creates or removes. For example, given the tree
shell> tree /tmp/home/
/tmp/home/
├── SB
│   └── home
└── VK
└── home
The playbook below
shell: cat pb.yml
- hosts: localhost
tasks:
- command:
cmd: "mv {{ main_path }}/{{ item.1 }} {{ main_path }}/{{ item.2 }}"
removes: "{{ main_path }}/{{ item.1 }}"
with_nested:
- "{{ for_rename.client }}"
- "{{ for_rename.old_name|zip(for_rename.new_name) }}"
vars:
main_path: "/tmp/home/{{ item.0 }}"
for_rename:
client: [VK, SB]
old_name: [home]
new_name: [town]
moves the files only if exist
shell> ansible-playbook pb.yml
PLAY [localhost] *****************************************************************************
TASK [command] *******************************************************************************
changed: [localhost] => (item=['VK', 'home', 'town'])
changed: [localhost] => (item=['SB', 'home', 'town'])
PLAY RECAP ***********************************************************************************
localhost: ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
shell> tree /tmp/home/
/tmp/home/
├── SB
│   └── town
└── VK
└── town
2 directories, 2 files
The playbook is idempotent. The commands will not execute if the files are missing
shell> ansible-playbook pb.yml
PLAY [localhost] *****************************************************************************
TASK [command] *******************************************************************************
ok: [localhost] => (item=['VK', 'home', 'town'])
ok: [localhost] => (item=['SB', 'home', 'town'])
PLAY RECAP ***********************************************************************************
localhost: ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You can use the module command the same way also to create or remove files. The next option is using the module file. For example, the playbook below
shell> cat pb.yml
- hosts: localhost
tasks:
- file:
state: touch
path: "{{ main_path }}/{{ item.1 }}"
with_nested:
- "{{ for_create.client }}"
- "{{ for_create.folders }}"
vars:
main_path: "/tmp/home/{{ item.0 }}"
for_create:
client: [VK, SB]
folders: [toula, tina]
creates the files
shell> ansible-playbook pb.yml
PLAY [localhost] *****************************************************************************
TASK [file] **********************************************************************************
changed: [localhost] => (item=['VK', 'toula'])
changed: [localhost] => (item=['VK', 'tina'])
changed: [localhost] => (item=['SB', 'toula'])
changed: [localhost] => (item=['SB', 'tina'])
PLAY RECAP ***********************************************************************************
localhost: ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
shell> tree /tmp/home/
/tmp/home/
├── SB
│   ├── tina
│   ├── toula
│   └── town
└── VK
├── tina
├── toula
└── town
This task is not idempotent because the files will receive updated file access and modification times (similar to the way touch works from the command line). Preserve access_time and modification_time to make the task idempotent
- file:
state: touch
path: "{{ main_path }}/{{ item.1 }}"
access_time: preserve
modification_time: preserve
...

Running Environment variable at Ansible Play level question

I have following commands declared in main playbook file:
---
- name: Install config
hosts: all
become: yes
become_method: sudo
become_user: root
roles:
- initial_config
environment:
http_proxy: http://{{ProxyHost}}:{{ProxyPort}}
https_proxy: http://{{ProxyHost}}:{{ProxyPort}}
ftp_proxy: http://{{ProxyHost}}:{{ProxyPort}}
Inside my group_vars/ all: I have declared these parameter:
ProxyHost: proxy.test.com
ProxyPort: 9999
no_proxy: 'test.com'
But I am receiving error when i run main playbook locally saying:
ERROR! Syntax Error while loading YAML.
found unexpected end of stream
The play below
shell> cat group_vars/all
ProxyHost: proxy.test.com
ProxyPort: 9999
no_proxy: test.com
shell> cat pb.yml
- hosts: test_01
environment:
http_proxy: "http://{{ ProxyHost }}:{{ ProxyPort }}"
https_proxy: "http://{{ ProxyHost }}:{{ ProxyPort }}"
ftp_proxy: "http://{{ ProxyHost }}:{{ ProxyPort }}"
tasks:
- shell: set | grep proxy
register: result
- debug:
var: result.stdout_lines
gives
shell> ansible-playbook pb.yml
PLAY [test_01] **********
TASK [shell] ************
changed: [test_01]
TASK [debug] ************
ok: [test_01] => {
"result.stdout_lines": [
"SUDO_COMMAND='/bin/sh -c echo BECOME-SUCCESS-hhojceebldohrrfvivcndrtjtvtrzgfg ; http_proxy=http://proxy.test.com:9999 https_proxy=http://proxy.test.com:9999 ftp_proxy=http://proxy.test.com:9999 /usr/local/bin/python3.7 /home/admin/.ansible/tmp/ansible-tmp-1588775688.2160842-43227888427690/AnsiballZ_command.py'",
"ftp_proxy=http://proxy.test.com:9999",
"http_proxy=http://proxy.test.com:9999",
"https_proxy=http://proxy.test.com:9999"
]
}
PLAY RECAP **********
test_01: ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Resources