I am currently taking two hosts and dynamically adding them to a group, followed by a synchronize task using with_together to use 3 lists of 2 elements in parallel to copy the specified files between two remote servers.
Here's an example based on the idea:
---
- name: Configure Hosts for Copying
hosts: localhost
gather_facts: no
tasks:
- name: Adding given hosts to new group...
add_host:
name: "{{ item }}"
groups: copy_group
with_items:
- ["remoteDest1", "remoteDest2"]
- name: Copy Files between servers
hosts: copy_group
gather_facts: no
tasks:
- name: Copying files...
synchronize:
src: "{{ item[1] }}"
dest: "{{ item[2] }}"
with_together:
- ["remoteSrc1", "remoteSrc2"]
- ["/tmp/remote/source/one/", "/tmp/remote/source/two/"]
- ["/tmp/remote/dest/one/", "/tmp/remote/dest/two/"]
delegate_to: "{{ item[0] }}"
Currently, it does both operations for both servers, resulting in 4 operations.
I need it to synchronize like so:
copy /tmp/remote/source/one/ from remoteSrc1 to /tmp/remote/dest/one/ on remoteDest1
copy /tmp/remote/source/two/ from remoteSrc2 to /tmp/remote/dest/two/ on remoteDest2
Which would mean it's a 1:1 ratio; essentially acting on the hosts in the same manner as with_together does for the lists.
The hosts are obtained dynamically, so I can't just make a different play for each host.
Since synchronize is essentially simplified version of rsync, then if there's a simple solution for this using rsync directly, then it would be much appreciated.
There isn't native functionality for this, so this is how I solved it:
Given the original task, add the following two lines:
- "{{ groups['copy_group'] }}"
when: inventory_hostname == item[3]
To get:
- name: Copying files...
synchronize:
src: "{{ item[1] }}"
dest: "{{ item[2] }}"
with_together:
- ["remoteSrc1", "remoteSrc2"]
- ["/tmp/remote/source/one/", "/tmp/remote/source/two/"]
- ["/tmp/remote/dest/one/", "/tmp/remote/dest/two/"]
- "{{ groups['copy_group'] }}"
delegate_to: "{{ item[0] }}"
when: inventory_hostname == item[3]
Essentially, by adding the hosts as a list, they can be used in the when statement to execute the task only when the current host (inventory_hostname) matches the host currently being indexed in the list.
The result is that the play only runs against each host once in a serial manner with the other list items that have the same index.
Related
I have a code to backup config using the ios_config module. I used ios_facts to get the hostname of devices and I want to use it to put the backup file in a similarly named folder and also use it in the file name itself.
In the last task of my code, I need to loop through two items - the sequence from 0 to 1(or how many items are in my inventory) as I need to access the hostname in the results and use it in the backup options, and also loop through my inventory of devices which I extracted from a csv file. I am aware of the rule of double curly braces but I do not know how to get around it.
---
- hosts: localhost
gather_facts: false
tasks:
- name: Block
block:
- name: Use CSV
csv_to_facts:
src: '{{playbook_dir}}/NEW/Inventory.csv'
vsheets:
- INFO:
- IP
- OS
- debug:
msg: '{{item.IP}}'
loop: '{{INFO}}'
- name: Create Inventory
add_host:
hostname: '{{item.IP}}'
ansible_network_os: '{{item.OS}}'
ansible_user: cisco
ansible_ssh_pass: cisco
ansible_connection: network_cli
ansible_become: yes
ansible_become_method: enable
groups: group_01
loop: '{{INFO}}'
- name: Gather Facts (IOS)
ios_facts:
register: ios_facts_loop
delegate_to: '{{item}}'
loop: "{{groups['group_01']}}"
- name: Backup Switch (IOS)
ios_config:
backup: yes
backup_options:
dir_path: "tmp/backups/{{ ios_facts_loop.results.{{item[0]}}.ansible_facts.ansible_net_hostname }}"
filename: "{{ios_facts_loop.results.item{{[0]}}.ansible_facts.ansible_net_hostname}} {{ lookup('pipe','date +%Y-%m-%d#%H:%M:%S')}}"
register: backup_ios_location
delegate_to: '{{item[1]}}'
loop:
- with_sequence: "0-{{output|length - 3}}"
- "{{groups['group_01']}}"
TLDR; for vars notation
You cannot add double curly braces inside double curly braces like in your above code. You current var reference:
ios_facts_loop.results.{{item[0]}}.ansible_facts.ansible_net_hostname
should be turned to
ios_facts_loop.results[item[0]].ansible_facts.ansible_net_hostname
# or equivalent
ios_facts_loop.results[item.0].ansible_facts.ansible_net_hostname
Meanwhile, this will only fix your current syntax error (that you didn't share in your question) as the first element in your loop is a string 'with_sequence: "0-X"' which therefore has no index 0.
Attempt to fix the logic
If I understand correctly, for your last task, you just need to loop over the results of your ios_facts register and delegate the task to the server it was taken from. Luckilly, you should already have all the info you need in ios_facts_loop.results
It is a list so you can directly loop over it
Each element should contain an item key with the actual item that was used in the previous run at time of register (i.e. one of your groups['group_01'] element).
So I would try to write your last task like this. Disclaimer this is a pure guess as I didn't see your exact datastructure.
- name: Backup Switch (IOS)
ios_config:
backup: yes
backup_options:
dir_path: "tmp/backups/{{ item.ansible_facts.ansible_net_hostname }}"
filename: "{{ item.ansible_facts.ansible_net_hostname}}{{ lookup('pipe','date +%Y-%m-%d#%H:%M:%S')}}"
register: backup_ios_location
delegate_to: '{{item.item}}'
loop: "{{ ios_facts_loop.results }}"
Going further.
I'm not really familiar with the ios_* modules but they should be really close to other stuff I use daily and I think you could really simplify your playbook taking advantage of more ansible feature (e.g. multiple plays in a playbook). I believe the following should actually do the job:
---
- name: Construct inventory from CSV
hosts: localhost
gather_facts: false
tasks:
- name: Use CSV
csv_to_facts:
src: '{{playbook_dir}}/NEW/Inventory.csv'
vsheets:
- INFO:
- IP
- OS
- name: Create Inventory
add_host:
hostname: '{{item.IP}}'
ansible_network_os: '{{item.OS}}'
ansible_user: cisco
ansible_ssh_pass: cisco
ansible_connection: network_cli
ansible_become: yes
ansible_become_method: enable
groups: group_01
loop: '{{INFO}}'
- name: Backup switches from created inventory
hosts: group_01
gather_facts: false
tasks:
- name: Get facts from network os
ios_facts:
gather_subset: all
- name: Backup Switch (IOS)
ios_config:
backup: yes
backup_options:
dir_path: "tmp/backups/{{ ansible_net_hostname }}"
filename: "{{ ansible_net_hostname }}{{ lookup('pipe','date +%Y-%m-%d#%H:%M:%S') }}"
More background on dot and brackets notation for vars
You can basically navigate a yaml datastructure with two notation which are equivalent.
the dot notation
a_list_var.index_number
a_hasmap_var.keyname
the brackets notation
a_list_var[index_number]
a_hashmap_var['key_name']
If we take the following example:
my_servers:
hostA:
ips:
- x.x.x.x
- y.y.y.y
env:
shell: bash
home: somewhere
hostB:
ips:
- a.a.a.a
- b.b.b.b
env:
shell: sh
home: elsewhere
The following notation are all strictly equivalent:
# all vars of hostA
hostA_vars: "{{ my_servers.hostA }}"
hostA_vars: "{{ my_server['hostA'] }}"
# first IP of hostB
hostB_ip: "{{ my_servers.hostB.0 }}"
hostB_ip: "{{ my_servers.hostB[0] }}"
hostB_ip: "{{ my_servers['hostB'].0 }}"
hostB_ip: "{{ my_servers['hostB'][0] }}"
As you can see, the dot notation tends to be less verbose and more readable. Meanwhile, you cannot use a variable identifier with the dot notation. So If you want to ave the home env of a variable server you would have to use:
# set a var for server
server: hostA
# all equivalent again
server_home: "{{ my_servers[server].env.home }}"
server_home: "{{ my_servers[server]['env'].home }}"
server_home: "{{ my_servers[server].env['home'] }}"
server_home: "{{ my_servers[server]['env']['home'] }}"
I want to run an Ansible playbook on multiple hosts and register outputs to a variable. Now using this variable, I want to copy output to single file. The issue is that in the end there is output of only one host in the file. How can I add output of all the hosts in a file one after the other. I don't want to use serial = 1 as it slows down execution considerably if we have multiple hosts.
-
hosts: all
remote_user: cisco
connection: local
gather_facts: no
vars_files:
- group_vars/passwords.yml
tasks:
- name: Show command collection
ntc_show_command:
connection: ssh
template_dir: /ntc-ansible/ntc-templates/templates
platform: cisco_ios
host: "{{ inventory_hostname }}"
username: "{{ ansible_ssh_user }}"
password: "{{ ansible_ssh_pass }}"
command: "{{commands}}"
register: result
- local_action:
copy content="{{result.response}}" dest='/home/user/show_cmd_ouput.txt'
result variable will be registered as a fact on each host the task ntc_show_command was run, thus you should access the value through hostvars dictionary.
- local_action:
module: copy
content: "{{ groups['all'] | map('extract', hostvars, 'result') | map(attribute='response') | list }}"
dest: /home/user/show_cmd_ouput.txt
run_once: true
You also need run_once because the action would still be run as many times as hosts in the group.
I'm trying to load a list of hosts in my playbook to provision a KVM. I need to key this off of the hosts.yml because another playbook is going to take the hosts and connect to them once they're up.
This my hosts.yml:
kvm:
hosts:
kvm01
dcos:
dcos-bootstrap:
hosts:
dcos-bootstrap
vars:
lv_size: "10g"
dcos-masters:
hosts:
dcos-master-1
vars:
lv_size: "50g"
dcos-agents:
hosts:
dcos-agent-1
dcos-agent-2
vars:
lv_size: "50g"
So on a single KVM I run this playbook to create the logical volumes for each of the Virtual Machines corresponding to the dcos hosts:
---
- hosts: kvm
tasks:
- name: Get a list of vm's to create
include_vars:
file: "../hosts.yml"
- name: Verify the host list
debug: var=dcos
when: dcos is defined
- name: Provision Volume Groups
lvol:
vg: vg01
lv: "{{ item.value.hosts }}"
size: "{{ item.value.vars.lv_size }}"
with_dict: "{{ dcos }}"
This works fine until you include more than one host for a group. I've tried other loops but I'm not sure how to continue. How can I iterate over the hash while working on each host in a group?
I'm new to Ansible so I really wasn't sure how to accomplish what I wanted. I initially tried to parse the hosts file myself, not knowing that Ansible does this for me. Now I know...
All of the host and group data is stored in host_vars and groups. All I needed to do what use this like so:
vars:
dcoshosts: "{{ groups['dcos'] }}"
tasks:
- name: List groups
debug:
msg: "{{ groups }}"
- name: Get All DCOS hosts
debug:
msg: "Host: {{ item }}"
with_items:
- "{{ dcoshosts }}"
- name: Provision Volume Groups
lvol:
vg: vg01
lv: "{{ item }}"
size: "{{ hostvars[item].lv_size }}"
with_items: "{{ dcoshosts }}"
I ended up using a hosts.ini file instead of yaml because the ini worked and the yaml didn't. Here it is to complete the picture:
[dcos-masters]
dcos-master-1
dcos-master-2
[dcos-masters:vars]
lv_size="50g"
[dcos-agents]
dcos-agent-1
dcos-agent-2
[dcos-agents:vars]
lv_size="50g"
[dcos-bootstraps]
dcos-bootstrap
[dcos-bootstraps:vars]
lv_size="10g"
[dcos:children]
dcos-masters
dcos-agents
dcos-bootstraps
Thanks to everybody for the help and pushing me to my solution :)
You try to reinvent what Ansible provides already.
This is the Ansbile way of life:
Define your hosts and groups in an inventory file dcos.ini:
[dcos-bootstraps]
dcos-bootstrap
[dcos-masters]
dcos-master-1
[dcos-agents]
dcos-agent-1
dcos-agent-2
[dcos:children]
dcos-bootstraps
dcos-masters
dcos-agents
Then you customize the group parameters in group variables.
Bootstrap parameters in groups_vars/dcos-bootstraps.yml:
---
lv_size: 10g
Masters parameters in group_vars/dcos-masters.yml:
---
lv_size: 50g
Agents parameters in group_vars/dcos-agents.yml:
---
lv_size: 50g
And your playbooks becomes quite simple:
---
- hosts: dcos
tasks:
- name: Provision Volume Groups
lvol:
vg: vg01
lv: "{{ inventory_hostname }}"
size: "{{ lv_size }}"
When you run this each host gets its lv_size parameter based on the group membership of the host.
First of all, you I suspect you have incorrect data.
If you want your hosts to be a list, you should make it so:
kvm:
hosts:
kvm01
dcos:
dcos-bootstrap:
hosts:
- dcos-bootstrap
vars:
lv_size: "10g"
dcos-masters:
hosts:
- dcos-master-1
vars:
lv_size: "50g"
dcos-agents:
hosts:
- dcos-agent-1
- dcos-agent-2
vars:
lv_size: "50g"
Note the hyphens.
And to your question: if you are not going to use "group" names in your loop (like dcos-bootstrap, etc.), you can use with_subelements:
- name: Provision Volume Groups
lvol:
vg: vg01
lv: "{{ item.1 }}"
size: "{{ item.0.vars.lv_size }}"
with_subelements:
- "{{ dcos }}"
- hosts
I have a need to know the index of host names in the inventory. I am using the below code to create a variable file that I can use in a subsequent play book
- name: Debug me
hosts: hosts
tasks:
- debug: msg="{{ inventory_hostname }}"
- debug: msg="{{ play_hosts.index(inventory_hostname) }}"
- local_action: 'lineinfile create=yes dest=/tmp/test.conf
line="host{{ play_hosts.index(inventory_hostname) }}=
{{ inventory_hostname }}"'
I have the following inventory file
[hosts]
my.host1.com
my.host2.com
Now when I run this, the test.conf that gets generated under /tmp sometimes has both hostnames like this
host1= my.host2.com
host0= my.host1.com
when I run the same playbook a few times each time emptying the test.conf before running. quite a few times the file only has one entry
host1= my.host2.com
or
host0= my.host1.com
how come the same ansible playbook behaving differently?
I believe the issue is your running two threads against different hosts, and using local_action is not thread safe.
Try using the serial keyword:
- name: Debug me
hosts: hosts
serial: 1
tasks:
- debug: msg="{{ inventory_hostname }}"
- debug: msg="{{ play_hosts.index(inventory_hostname) }}"
- local_action: 'lineinfile create=yes dest=/tmp/test.conf
line="host{{ play_hosts.index(inventory_hostname) }}=
{{ inventory_hostname }}"'
Edit: A better way to do this if just trying to operate on the list of host in inventory on the localhost, would be to avoid doing the action on the host and using local_action in the first place.
- name: Debug me
hosts: localhost
tasks:
- lineinfile:
create: yes
dest: /tmp/test.conf
line: "host{{ groups['hosts'].index(item)}}={{ item }}"
with_items: " {{ groups['hosts'] }}"
This will get you the results you desire. Then you can add another play to do operations against the hosts themselves.
The solution I am trying to avoid problems with race conditions with non-thread safe Local_action: lineinfile to write gathered data to local file. Split it across 2 different plays in the same file.
eg:
- name: gather_date
hosts: all
any_errors_fatal: false
gather_facts: no
tasks:
- name: get_Aptus_device_count_list
shell: gather_data.sh
become: true
register: Aptus_device_count_list
changed_when: false
- name: Log_gathered_date
hosts: all
any_errors_fatal: false
gather_facts: no
tasks:
- name: log_gathered_info
local_action:
module: lineinfile
dest: /home/rms-mit/MyAnsible/record_Device_count_collection.out
line: "\n--- {{ inventory_hostname }} --- \n
{{ Aptus_device_count_list.stdout }} \n.\n---\n"
changed_when: false
I needed to be able to invert variables stored in a JSON file that is passed to the playbook from the command line.
These are the tasks that I set up (they are identical except for vars), this is a fragment of a playbook:
- name: Prepare a .sql file
delegate_to: 127.0.0.1
mysql_db:
name: "{{ source['database']['db_name'] }}"
state: dump
login_host: "{{ source['database']['host'] }}"
login_user: "{{ source['database']['user'] }}"
login_password: "{{ source['database']['password'] }}"
target: test_db.sql
when: invert is not defined
- name: Prepare a .sql file (inverted)
delegate_to: 127.0.0.1
mysql_db:
name: "{{ target['database']['db_name'] }}"
state: dump
login_host: "{{ target['database']['host'] }}"
login_user: "{{ target['database']['user'] }}"
login_password: "{{ target['database']['password'] }}"
target: test_db.sql
when: invert is defined
So consequently when I execute
ansible-playbook -i hosts playbook.yml --extra-vars "#dynamic_vars.json"
the first task is executed. If I execute
ansible-playbook -i hosts playbook.yml --extra-vars "#dynamic_vars.json" --extra-vars "invert-yes"
the second task is executed that takes the same hash as parameters, but only swaps source for target (which essentially becomes a source in my playbook).
As you can see, this is a very simplistic approach, there is a lot of unnecessary duplication, I just do not like it. However, I cannot think of a better way to be able to revert variables at the command line without building some more complex include logic.
Perhaps you can advice me on how I can do it better? Thanks!
I'm a big fan of YAMLs anchors and references when it comes to the topic of avoiding repetition. Since the content is dynamic, you could take advantage of with_items, which can be used to pass a parameter like so:
- &sqldump
name: Prepare a .sql file
delegate_to: 127.0.0.1
mysql_db:
name: "{{ item['database']['db_name'] }}"
state: dump
login_host: "{{ item['database']['host'] }}"
login_user: "{{ item['database']['user'] }}"
login_password: "{{ item['database']['password'] }}"
target: test_db.sql
when: invert is not defined
with_items:
- source
- <<: *sqldump
name: Prepare a .sql file (inverted)
when: invert is defined
with_items:
- target
The 2nd task is a perfect clone of the first one, you then override the name, condition and the loop with_items to pass the target instead of the source.
After reading your answer to #ydaetskcoR it sounds like you have quite some cases where you need to use the data from one or the other dict. Maybe in that case it then would make sense to just define the var globally depending on the invert parameter. Your vars file could look like that:
---
source:
database: ...
db_name: ...
target:
database: ...
db_name: ...
data: "{{ target if invert is defined else source }}"
You then simply can use data in all your tasks without dealing with conditions any further.
- name: Prepare a .sql file
delegate_to: 127.0.0.1
mysql_db:
name: "{{ data['database']['db_name'] }}"
state: dump
login_host: "{{ data['database']['host'] }}"
login_user: "{{ data['database']['user'] }}"
login_password: "{{ data['database']['password'] }}"
target: test_db.sql
Of course, this way you have a fixed task name which does not change with the param you pass.
If you are attempting to do the same thing but just want to specify different variables depending on the host/group then a better approach may be to simply set these as host/group vars and run it as a single task.
If we set up our inventory file a bit like this:
[source_and_target-nodes:children]
source-nodes
target-nodes
[source-nodes]
source database_name='source_db' database_login_user='source_user' database_login_pass='source_pass'
[target-nodes]
target database_name='target_db' database_login_user='target_user' database_login_pass='target_pass'
Then we can target the task at the source_and_target-nodes like so:
- name: Prepare a .sql file
hosts: source_and_target-nodes
mysql_db:
name: "{{ database_name }}"
state: dump
login_host: "{{ inventory_hostname }}"
login_user: "{{ database_login_user }}"
login_password: "{{ database_login_pass }}"
target: test_db.sql
You won't be able to access the host vars of a different host this easily if you need to use delegate_to as you are in your question but if you are simply needing to run the play locally you can instead set ansible_connection to local in your host/group vars or setting connection: local in the play.