Im trying to run a playbook for n host groups serially (but 100% parallel within the host group). How do I achieve this?
I've tried things like:
- name: Test
hosts: group1:group2
serial: 100%
and even
- name: Test
hosts: group1:group2
serial: 1
Thinking it would do group by group, however these do not work.
How do I get it to run over all of group1, then after, all of group2 (but fail if anything in group1 fails)?
Also, how do I get it to run over n groups? (There are many hostgroups, which might be tough to define in the hosts key)
You can't control a playbook from another playbook. You'll have to control the playbook from outside, for example by a script. Given the inventory
shell> cat hosts-497
[group1]
srv1
[group2]
srv2
srv3
[group3]
srv4
srv5
srv6
and the playbook
shell> cat test-497.yml
- name: Test
hosts: all
gather_facts: false
tasks:
- debug:
msg: "{{ '%H:%M:%S'|strftime }}: {{ inventory_hostname }}"
the debug task is executed in parallel by all hosts
shell> ansible-playbook -i hosts-497 test-497.yml
PLAY [Test] ***************************************************************
TASK [debug] **************************************************************
ok: [srv3] =>
msg: '20:51:30: srv3'
ok: [srv1] =>
msg: '20:51:30: srv1'
ok: [srv4] =>
msg: '20:51:30: srv4'
ok: [srv2] =>
msg: '20:51:30: srv2'
ok: [srv5] =>
msg: '20:51:30: srv5'
ok: [srv6] =>
msg: '20:51:30: srv6'
If you want to control the hosts create a script and iterate the groups, e.g.
shell> cat test-497.sh
#!/usr/bin/sh
for i in group1 group2 group3; do
ansible-playbook -i hosts-497 --limit $i test-497.yml
done
gives (abridged)
shell> ./test-497.sh
PLAY [Test] *************************************************************
TASK [debug] ************************************************************
ok: [srv1] =>
msg: '20:56:41: srv1'
PLAY [Test] *************************************************************
TASK [debug] ************************************************************
ok: [srv3] =>
msg: '20:56:45: srv3'
ok: [srv2] =>
msg: '20:56:45: srv2'
PLAY [Test] *************************************************************
TASK [debug] ************************************************************
ok: [srv5] =>
msg: '20:56:52: srv5'
ok: [srv6] =>
msg: '20:56:52: srv6'
ok: [srv4] =>
msg: '20:56:53: srv4'
Related
I run a playbook, which contains a role, and executes so it runs once against the first server in the host file, it then executes against 25% of servers in the host file, finally, it executes against the rest of the servers.
What I want to achieve is:
If I run the playbook without any tags or task1, it should run against all servers in one go.
If I run the playbook with tags task2 it runs the playbook in batches (as per the serial).
At the moment, it will always run in batches regardless.
I have the following playbooks:
Playbook:
- hosts: all
serial:
- 1
- "25%"
- "100%"
roles:
- playbook2
playbook2 role:
- name: task 1
shell: [do something]
tags: task1
- name: task 2
shell: [do more]
tags: task2
You can template serial and omit it based on the content of the tags, and the special variable ansible_run_tags.
Your serial parameter, then should be
serial: "{{ ['1', '25%', '100%'] if 'task2' in ansible_run_tags else omit }}"
Given the playbook:
- hosts: _nodes
gather_facts: no
serial: "{{ ['1', '25%', '100%'] if 'task2' in ansible_run_tags else omit }}"
tasks:
- debug:
tags:
- task2
- debug:
tags:
- task2
When run with no tags, on three hosts it will yield:
PLAY [_nodes] ***********************************************************
TASK [debug] ************************************************************
ok: [node2] =>
msg: Hello world!
ok: [node3] =>
msg: Hello world!
ok: [node1] =>
msg: Hello world!
TASK [debug] ************************************************************
ok: [node2] =>
msg: Hello world!
ok: [node3] =>
msg: Hello world!
ok: [node1] =>
msg: Hello world!
When run with --tags task2, it will, then, yield:
PLAY [_nodes] ***********************************************************
TASK [debug] ************************************************************
ok: [node2] =>
msg: Hello world!
TASK [debug] ************************************************************
ok: [node2] =>
msg: Hello world!
PLAY [_nodes] ***********************************************************
TASK [debug] ************************************************************
ok: [node3] =>
msg: Hello world!
TASK [debug] ************************************************************
ok: [node3] =>
msg: Hello world!
PLAY [_nodes] ***********************************************************
TASK [debug] ************************************************************
ok: [node1] =>
msg: Hello world!
TASK [debug] ************************************************************
ok: [node1] =>
msg: Hello world!
The special tag 'never' is not applied in my second use case. I can't find out in the Ansible document if this is normal behavior, or if it is a bug.
playbook-parent.yml
- name: parent playbook
import_playbook: playbook-child.yml
tags:
- toto
playbook-child.yml
- hosts: local
roles:
- role: test
tags:
- never
roles/test/tasks/main.yml
- name: hello word
copy:
content: hello world
dest: testfile.txt
Test Case 1: Tag never is applied with the command below ==> OK :)
shell> ansible-playbook playbook-parent.yml
PLAY [local] **********************************
TASK [Gathering Facts] ************************
ok: [127.0.0.]
PLAY RECAP ************************************
...
Test Case 2: Tag 'never' is not applied with the command below ==> Normal? Bug?
ansible-playbook playbook-parent.yml --tags toto
PLAY [local] **********************************
TASK [Gathering Facts] ************************
ok: [127.0.0.]
TASK [test: help word] ************************
changed: [127.0.0.]
PLAY RECAP ************************************
...
Q: "The special tag 'never' is not applied in my second use case, I can't find out in the Ansible document if this is normal behavior."
A: Quoting from Special tags: always and never
If you assign the never tag to a task or play, Ansible will skip that task
or play unless you specifically request it (--tags never).
Given the role
shell> cat roles/test/tasks/main.yml
- debug:
var: ansible_run_tags
The play
shell> cat playbook-child.yml
- hosts: localhost
roles:
- role: test
tags:
- never
works as expected
shell> ansible-playbook playbook-child.yml -t never
displays the debug message
TASK [test : debug] ***************************************************
ok: [localhost] =>
ansible_run_tags:
- never
and the play without the tag does nothing
shell> ansible-playbook playbook-child.yml
The next playbook applies the tag toto to all tasks in the playbook playbook-child.yml
shell> cat playbook-parent.yml
- name: parent playbook
import_playbook: playbook-child.yml
tags:
- toto
As expected, without any tag the play does nothing
shell> ansible-playbook playbook-parent.yml
But if any of the tags toto or never or both are applied the play displays the debug message
shell> ansible-playbook playbook-parent.yml -t toto
TASK [test : debug] ********************************************************
ok: [localhost] =>
ansible_run_tags:
- toto
shell> ansible-playbook playbook-parent.yml -t never
TASK [test : debug] ********************************************************
ok: [localhost] =>
ansible_run_tags:
- never
shell> ansible-playbook playbook-parent.yml -t never,toto
TASK [test : debug] ********************************************************
ok: [localhost] =>
ansible_run_tags:
- never
- toto
You appear to be misunderstanding the function of the never tag. As the documentation says:
If you assign the never tag to a task or play, Ansible will skip that task or play unless you specifically request it
You have specifically requested that play to run by passing --tags toto, so it ran. Your example is overly complex; you can demonstrate the behaviour clearly in a single playbook.
- hosts: localhost
gather_facts: false
tasks:
- debug:
msg: task 1
tags:
- never
- debug:
msg: task 2
tags:
- never
- foo
- debug:
msg: task 3
tags:
- foo
- debug:
msg: task 4
ansible-playbook test.yml:
PLAY [localhost] ***************************************************************
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "task 3"
}
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "task 4"
}
ansible-playbook test.yml --tags foo:
PLAY [localhost] ***************************************************************
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "task 2"
}
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "task 3"
}
ansible-playbooks --tags foo --skip-tags never:
PLAY [localhost] ***************************************************************
TASK [debug] *******************************************************************
ok: [localhost] => {
"msg": "task 3"
}
My ansible inventory file has the following entry
[non_prod_servers]
oracle[1:13]
How can I eliminate hosts "oracle7" and "oracle10" from the above specification, without having to create the following entries?
[non_prod_servers]
oracle[1:6]
oracle[8:9]
oracle[11:13]
Essentially, looking for an elegant solution than what I have come up with.
Thanks in advance.
FR
IMHO, the functionality, you're looking for, is not available. You can use Python slicing inside the inventory. The inventory patterns do not apply inside the inventory file.
If you need it for automation, i.e. you want to control the process by a couple of variables the inventory and playbook below create the group dynamically
shell> cat hosts
[non_prod_servers]
localhost
[non_prod_servers:vars]
_name=oracle
_from=1
_to=13
_deny=[7,10]
shell> cat playbook.yml
---
- hosts: non_prod_servers
gather_facts: false
tasks:
- add_host:
hostname: "{{ _name }}{{ item }}"
groups: non_prod_servers_dyn
loop: "{{ range(_from, _to + 1)|difference(_deny) }}"
- hosts: non_prod_servers_dyn
gather_facts: false
tasks:
- debug:
var: ansible_play_hosts_all
run_once: true
gives
shell> ansible-playbook -i hosts playbook.yml
PLAY [non_prod_servers] *************************************************
TASK [add_host] *********************************************************
ok: [localhost] => (item=1)
ok: [localhost] => (item=2)
ok: [localhost] => (item=3)
ok: [localhost] => (item=4)
ok: [localhost] => (item=5)
ok: [localhost] => (item=6)
ok: [localhost] => (item=8)
ok: [localhost] => (item=9)
ok: [localhost] => (item=11)
ok: [localhost] => (item=12)
ok: [localhost] => (item=13)
PLAY [non_prod_servers_dyn] **********************************************
TASK [debug] *************************************************************
ok: [oracle1] =>
ansible_play_hosts_all:
- oracle1
- oracle2
- oracle3
- oracle4
- oracle5
- oracle6
- oracle8
- oracle9
- oracle11
- oracle12
- oracle13
I'm using the latest version of Ansible, and I am trying to use a default variable in role-one used on host one, in role-two, used on host two, but I can't get it to work.
Nothing I have found in the documentation or on StackOverflow has really helped. I'm not sure what I am doing wrong. Ideally I want to set the value of the variable once, and be able to use it in another role for any host in my playbook.
I've broken it down below.
In my inventory I have a hosts group called [test] which has two hosts aliased as one and two.
[test]
one ansible_host=10.0.1.10 ansible_connection=ssh ansible_user=centos ansible_ssh_private_key_file=<path_to_key>
two ansible_host=10.0.1.20 ansible_connection=ssh ansible_user=centos ansible_ssh_private_key_file=<path_to_key>
I have a single playbook with a play for each of these hosts and I supply the hosts: value as "{{ host_group }}[0]" for host one and "{{ host_group }}[1]" for host two.
The play for host one uses a role called role-one and the play for host two uses a role called role-two.
- name: Test Sharing Role Variables
hosts: "{{ host_group }}[0]"
roles:
- ../../ansible-roles/role-one
- name: Test Sharing Role Variables
hosts: "{{ host_group }}[1]"
roles:
- ../../ansible-roles/role-two
In role-one I have set a variable variable-one.
---
# defaults file for role-one
variable_one: Role One Variable
I want to use the value of variable_one in a template in role-two but I haven't had any luck. I'm using the below as a task in role-two to test and see if the variable is getting "picked-up".
---
# tasks file for role-two
- debug:
msg: "{{ variable_one }}"
When I run the playbook with ansible-playbook test.yml --extra-vars "host_group=test" I get the below failure.
TASK [../../ansible-roles/role-two : debug] ***********************************************************************************************************************************************************************************************
fatal: [two]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: \"hostvars['test']\" is undefined\n\nThe error appears to be in 'ansible-roles/role-two/tasks/main.yml': line 3, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n# tasks file for role-two\n- debug:\n ^ here\n"}
Variables declared in roles are scoped to the play. If you want to access a variable from role-one in role-two, they would both need to be in the same play. For example, you could write:
- name: Test Sharing Role Variables
hosts: "{{ host_group }}"
tasks:
- import_role:
name: role-one
when: inventory_hostname == "one"
- import_role:
name: role-two
when: inventory_hostname == "two"
Alternatively, you could restructure your roles so that the variables can be imported separately from your actions. That is, have a role_one_vars role that does nothing but define variables, and then you can import that in both role-one and role-two. That is, you would have a structure something like:
playbook.yml
hosts
roles/
role-one/
tasks/
main.yml
role-one-vars/
variables/
main.yml
role-two/
tasks/
main.yml
And role-one/tasks/main.yml would look like:
- import_role:
name: role-one-vars
- debug:
msg: "in role-one: {{ variable_one }}"
role-two/tasks/main.yml would look like:
---
- import_role:
name: role-one-vars
- debug:
msg: "in role-two: {{ variable_one }}"
And role-one-vars/vars/main.yml would look like:
---
variable_one: role one variable
Putting this all together, the output looks like:
PLAY [Test Sharing Role Variables] *****************************************************************************************************************************************
TASK [Gathering Facts] *****************************************************************************************************************************************************
ok: [one]
TASK [role-one : debug] ****************************************************************************************************************************************************
ok: [one] => {
"msg": "in role-one: role one variable"
}
PLAY [Test Sharing Role Variables] *****************************************************************************************************************************************
TASK [Gathering Facts] *****************************************************************************************************************************************************
ok: [two]
TASK [role-two : debug] ****************************************************************************************************************************************************
ok: [two] => {
"msg": "in role-two: role one variable"
}
PLAY RECAP *****************************************************************************************************************************************************************
one : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
two : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Q: "Access variable from one role in another role in an Ansible playbook with multiple hosts"
A: Short answer: Use set_fact and put the variable into the hostvars.
Details: Given the roles
shell> cat roles/role-one/defaults/main.yml
variable_one: Role One Variable
shell> cat roles/role-one/tasks/main.yml
- debug:
var: variable_one
shell> cat roles/role-two/tasks/main.yml
- debug:
var: variable_one
The playbook
- hosts: one
roles:
- role-one
tasks:
- debug:
var: variable_one
- hosts: two
roles:
- role-two
- hosts: one
tasks:
- debug:
var: variable_one
gives (abridged)
PLAY [one] ************************************************
TASK [role-one : debug] ****
ok: [one] =>
variable_one: Role One Variable
TASK [debug] ****
ok: [one] =>
variable_one: Role One Variable
PLAY [two] ************************************************
TASK [role-two : debug] ****
ok: [two] =>
variable_one: VARIABLE IS NOT DEFINED!
PLAY [one] ************************************************
TASK [debug] ****
ok: [one] =>
variable_one: VARIABLE IS NOT DEFINED!
As expected, the variable variable_one is visible to the tasks in the first play. But, there is no reason the variable should be visible to the host two in the second play. The variable is not visible also to the same host in the third play because it hasn't been stored in the hostvars aka "instantiated". The playbook below
- hosts: one
roles:
- role-one
tasks:
- debug:
var: variable_one
- set_fact:
variable_one: "{{ variable_one }}"
- hosts: two
roles:
- role-two
- hosts: one
tasks:
- debug:
var: variable_one
gives (abridged)
PLAY [one] ************************************************
TASK [role-one : debug] ****
ok: [one] =>
variable_one: Role One Variable
TASK [debug] ****
ok: [one] =>
variable_one: Role One Variable
TASK [set_fact] ****
ok: [one]
PLAY [two] ************************************************
TASK [role-two : debug] ****
ok: [two] =>
variable_one: VARIABLE IS NOT DEFINED!
PLAY [one] ************************************************
TASK [debug] ****
ok: [one] =>
variable_one: Role One Variable
Now, the variable is visible to the host one in the whole playbook and can be visible to other hosts using hostvars as well. For example, the playbook below
- hosts: one
roles:
- role-one
tasks:
- debug:
var: variable_one
- set_fact:
variable_one: "{{ variable_one }}"
- hosts: two
tasks:
- set_fact:
variable_one: "{{ hostvars.one.variable_one }}"
- include_role:
name: role-two
gives (abridged)
PLAY [one] ************************************************
TASK [role-one : debug] ****
ok: [one] =>
variable_one: Role One Variable
TASK [debug] ****
ok: [one] =>
variable_one: Role One Variable
TASK [set_fact] ****
ok: [one]
PLAY [two] ************************************************
TASK [set_fact] ****
ok: [two]
TASK [include_role : role-two] ****
TASK [role-two : debug] ****
ok: [two] =>
variable_one: Role One Variable
The problem with the above setting is that the host referencing hostvars is hardcoded. A better approach is to "instantiate" the variable in the first play for all hosts. For example, add a dummy task to the role
shell> cat roles/role-one/tasks/noop.yml
- meta: noop
Then, in the first play, include all hosts, run_once import the role, run the dummy task only, and "instantiate" the variable for all hosts. For example
- hosts: all
tasks:
- import_role:
name: role-one
tasks_from: noop.yml
run_once: true
- set_fact:
variable_one: "{{ variable_one }}"
run_once: true
- hosts: two
roles:
- role-two
- hosts: one
roles:
- role-two
gives (abridged)
PLAY [all] ************************************************
TASK [set_fact] ****
ok: [one]
PLAY [two] ************************************************
TASK [role-two : debug] ****
ok: [two] =>
variable_one: Role One Variable
PLAY [one] ************************************************
TASK [role-two : debug] ****
ok: [one] =>
variable_one: Role One Variable
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I do have an inventory file as below
[ParentGroup]
ChildrenGroup1
ChildrenGroup2
[ChildrenGroup1]
host1
host2
host3
[ChildrenGroup2]
host4
host5
host6
Now i want to iterate Children wise..
i.e. Perform my task in parallel on host1,host2, host3 i.e only on hosts exists inChildrenGroup1 and once this is success, i Need to go with ChildrenGroup2 i.e on host4, host5, host6
Points to be taken care ?
if there is any failure on any one of the childrengroup hosts then we need to wait/pause before proceeding with next children group
I shall have many children groups on my inventory
I need to action my task only on one chidlrengroup at a time.
I shall make sure all the childrengroups are addressed in one-shot too.
Can you suggest on how to take this forward ?
The critical limitation here is the fact that a playbook can't start another playbook. The only option is import_playbook. Imported files must be available when a playbook starts. As a result, the solution is a two-step process. Create the playbooks in the first step and then run them. For example, given the inventory
shell> cat hosts
[ParentGroup:children]
ChildrenGroup1
ChildrenGroup2
[ChildrenGroup1]
host1
host2
host3
[ChildrenGroup2]
host4
host5
host6
you want to run the playbook pb.yml as described in the question. Take the playbook and create the template by putting {{ item }} to hosts:
shell> cat pb.yml.j2
- hosts: "{{ item }}"
gather_facts: false
tasks:
- debug:
msg: "{{ inventory_hostname }}: Playbook started."
1. Create playbooks
The playbook below creates the list of the groups my_groups in the first task. Then the template task iterates this list and creates playbooks for the groups. The next template task imports these playbooks into the playbook pb-groups.yml
shell> cat pb-init.yml
- hosts: localhost
vars:
groups_other: [ParentGroup, all, ungrouped]
tasks:
- set_fact:
my_groups: "{{ groups.keys()|difference(groups_other) }}"
- template:
src: pb.yml.j2
dest: "pb-{{ item }}.yml"
loop: "{{ my_groups }}"
- template:
src: pb-groups.yml.j2
dest: pb-groups.yml
shell> cat pb-groups.yml.j2
- hosts: localhost
gather_facts: false
{% for group in my_groups %}
- import_playbook: pb-{{ group }}.yml
{% endfor %}
See created files
shell> cat pb-ChildrenGroup1.yml
- hosts: "ChildrenGroup1"
gather_facts: false
tasks:
- debug:
msg: "localhost: Playbook started."
shell> cat pb-ChildrenGroup2.yml
- hosts: "ChildrenGroup2"
gather_facts: false
tasks:
- debug:
msg: "localhost: Playbook started."
shell> cat pb-groups.yml
- hosts: localhost
gather_facts: false
- import_playbook: pb-ChildrenGroup1.yml
- import_playbook: pb-ChildrenGroup2.yml
2. Run created playbooks
shell> ansible-playbook pb-groups.yml
PLAY [localhost] ****
PLAY [ChildrenGroup1] ****
TASK [debug] ****
ok: [host1] =>
msg: 'localhost: Playbook started.'
ok: [host2] =>
msg: 'localhost: Playbook started.'
ok: [host3] =>
msg: 'localhost: Playbook started.'
PLAY [ChildrenGroup2] ****
TASK [debug] ****
ok: [host4] =>
msg: 'localhost: Playbook started.'
ok: [host5] =>
msg: 'localhost: Playbook started.'
ok: [host6] =>
msg: 'localhost: Playbook started.'
PLAY RECAP ****
...
Many children groups on my inventory
Change the inventory. For example
shell> cat hosts
[ParentGroup:children]
ChildrenGroup1
ChildrenGroup2
ChildrenGroup3
[ChildrenGroup1]
host1
host2
[ChildrenGroup2]
host4
host5
[ChildrenGroup3]
host3
host6
The commands below work as expected
shell> ansible-playbook pb-init.yml
...
shell> ansible-playbook pb-groups.yml
PLAY [localhost] ****
PLAY [ChildrenGroup1] ****
TASK [debug] ****
ok: [host1] =>
msg: 'localhost: Playbook started.'
ok: [host2] =>
msg: 'localhost: Playbook started.'
PLAY [ChildrenGroup2] ****
TASK [debug] ****
ok: [host4] =>
msg: 'localhost: Playbook started.'
ok: [host5] =>
msg: 'localhost: Playbook started.'
PLAY [ChildrenGroup3] ****
TASK [debug] ****
ok: [host3] =>
msg: 'localhost: Playbook started.'
ok: [host6] =>
msg: 'localhost: Playbook started.'
PLAY RECAP ****
...