We have inherited quite a lot of ansible playbooks and roles/tasks underneath
One of the ansible playbook is triggered by a shell script after setting few variables
ansible-playbook -i inventory.yml install_multiple_nodes.yml
In the install_multiple_nodes.yml, it contains as following
- hosts: all_nodes
run_once: true
roles:
- set_keys
- download_rpms
become: true
- hosts: node1
roles:
- install_rpms_node1
- custom_actions_node1
become_user: node1_user
- hosts: node2
roles:
- install_rpms_node2
- custom_actions_node2
become_user: node2_user
... This continues for multiple nodes ...
The entire playbook takes about 1.5hours to run.
But many of the 'plays' within the playbook can be run in parallel. For instance in the above snippet, the "node1" and "node2" can be run in parallel, but later down in the chain some have to wait for another etc.
Is there a way we can
put a flag to say the node1 and node2 can run in parallel?
What's the best practice to have these type of playbooks? i.e. parallellise options and put dependency after a set of 'plays' , start next etc.
Following my comment, one thing you can try to work around the poor design:
- hosts: all_nodes
run_once: true
roles:
- set_keys
- download_rpms
become: true
tasks:
- name: Run machine specific roles
ansible.builtin.include_role:
name: "{{ item }}_{{ inventory_hostname }}"
loop:
- install_rpms
- custom_actions
Notes:
Untested, see in you own envitonment
This will surely break as soon as there is no specific role for a given node. Add conditions/tests if this is the case.
Related
I’ll tell you about my scenario. I have a host file with two servers and two playbooks.
I need to perform the following, first I have to run a playbook on server number 1 and if there is no errors, I need to run the second playbook on server number 2.
What condition should I use?
- name: Execute the First Play
import_playbook: first-playbook-to-run.yml
- name: Run the Second Playbook
import_playbook: second-playbook-to-run.yml
Any helps?
Regards,
Try the below
- name: Execute the First Play
import_playbook: first-playbook-to-run.yml
delegate_to: server1
run_once: true
- name: Run the Second Playbook
import_playbook: second-playbook-to-run.yml
delegate_to: server2
run_once: true
The calling playbook has:
- hosts: ssh_servers
tasks:
- import_tasks: create_files.yml
Then, in create_files.yml, I'd like to run some tasks on hosts other than ssh_servers, such as:
- Hosts: other_servers
tasks:
- file:
I get: ERROR! conflicting action statements: hosts, tasks .
Is this because I'm trying to run against hosts that were never included in the calling task ?
Is there a way to accomplish this other than in the calling playbook have:
- hosts:
- ssh_servers
- other_servers
tasks:
- import_tasks: create_files.yml
Thank you.
Is this because I'm trying to run against hosts that were never included in the calling task ? Is there a way to accomplish this other than in the calling playbook
I believe the answer is yes, although it'll be weird and could cause subsequent folks who interact with your playbook some confusion
given a hypothetical create_files.yml of:
- name: create /tmp/hello_world on hosts "not_known_at_launch_time"
file:
path: /tmp/hello_world
state: present
delegate_to: '{{ item }}'
with_items: '{{ groups["not_known_at_launch_time"] }}'
then the glue needed to bridge them together is the dynamic creation of a group and that delegate_to: keyword
- hosts: ssh_hosts
tasks:
- add_host:
groups: not_known_at_launch_time
name: secret-host-0
ansible_host: 192.168.1.1 # or whatever
# ... other hostvars here ...
- include_tasks: create_files.yml
it may be possible to combine those inside create_files.yml, via some shared vars: that say which host-and-ip should be added to the magic group name, which also has the benefit of keeping the magic group name localized to the file that consumes it.
BE AWARE, I did actually test this, but not extensively, so there may be some weird things such as the need to run_once: yes on them to keep the tasks from being run groups.ssh_hosts|length times or similar stuff
As Vladimir correctly pointed out, what you actually want to happen is to make that relationship formal:
- hosts: ssh_hosts
tasks:
... whatever tasks you had before
- add_host: ... as before ...
- hosts: anonymous_group_name_goes_here
tasks:
- include_tasks: create_files.yml
- hosts: ssh_hosts
tasks:
- debug:
msg: and now you are back to the ssh_hosts to pick up what they were supposed to be doing when you stopped to post on SO
I have a playbook with this structure:
---
- hosts: foo-servers
roles:
- foo_setup
become: yes
tags: tweaks
- hosts: bar-servers
roles:
- bar_setup
become: yes
tags: tweaks
[a few more server groups with a similar pattern]
I have a somewhat similar feature to deploy in all servers but each server group has it's own small differences, so I need to keep separate roles for each group.
I also want to run just a select group of tasks from each role, the ones tagged 'tweaks', in all hosts.
And all hosts should be run with raised privileges, but that is not true for all playbooks, so I want that setting to apply just to this playbook (no global vars).
I would like to move all the repeated parameters - become: yes and tags: tweaks outside of host: plays where they will be indicated to apply to all roles bellow. Something to the effect of
--
- all_hosts_this_playbook:
become: yes
tags: tweaks
- hosts: foo-servers
roles:
- foo_setup
- hosts: bar-servers
roles:
- bar_setup
I suppose this is possible in the command line. Like ansible-playbook setup_tweaks.yml --tags "tweak" --become? But is there a playbook equivalent? I'd rather have these in the file than in the command line, where I often forget to add stuff.
And looping doesn't work...
ERROR! 'loop' is not a valid attribute for a Play
- name: Make tweaks in many servers
become: yes
tags: tweaks
hosts: "{{ item.host }}"
roles:
- "{{ item.role }}"
loop:
- { host: 'foo-servers', role: 'foo_setup' }
- { host: 'bar-servers', role: 'bar_setup' }
I also want to add post_tasks: to be run in all servers (maybe they also need to be tagged?):
post_tasks_all_hosts:
- name: Upgrade system
apt:
autoremove: yes
autoclean: yes
update_cache: yes
upgrade: dist
tags: tweaks
- name: Reboot
shell: sleep 2 && reboot
async: 3
poll: 0
tags: tweaks
Is it possible to define playbook-wide pre_tasks or post_tasks?
Here Ansible: How to declare global variable within playbook? it is indicated that one 'cannot define a variable accessible on a playbook level', but in my case it's not variables - it's task parameters and post_tasks:.
Maybe the parameters and the 'pre/post tasks' are different problems with different solutions, but I decided to ask in the same place because they both fall on the same category of parameters that I'd like to set for the whole playbook, outside of host: plays.
Q: "I suppose this is possible in the command line. Like ansible-playbook setup_tweaks.yml --tags "tweak" --become? But is there a playbook equivalent?"
A: No. There is no such playbook equivalent.
Isn't this is a misunderstanding of the command-line option --tags
only run plays and tasks tagged with these values
versus
tag inheritance ?
Adding tags: to a play, or to statically imported tasks and roles, adds those tags to all of the contained tasks...When you apply tags: attributes to structures other than tasks, Ansible processes the tag attribute to apply ONLY to the tasks they contain. Applying tags anywhere other than tasks is just a convenience so you don’t have to tag tasks individually.
Details
In the play below tag "tweaks" is added to all of the contained tasks
- hosts: foo-servers
roles:
- foo_setup
tags: tweaks
The command below selects only tasks tagged "tweaks"
ansible-playbook setup_tweaks.yml --tags "tweak"
Q: "Is it possible to define playbook-wide pre_tasks or post_tasks?"
A: No. It's not possible. The scope of pre/post_tasks is the play.
I have the following set up for Ansible, and I would like to parameterize a filter that will loop, and filter out specific hosts.
- name: run on hosts
hosts: "{{ item }}"
roles:
- directory/role-name
with_items:
- us-east-1a
- us-east-1b
- us-east-1c
The result would be that the role called role-name would be first run on us-east-1a hosts, then us-east-1b... etc.
The above simple errors out with
ERROR! 'with_items' is not a valid attribute for a Play
Is there a way to accomplish what I am trying to do, which is chunking my host list into groups, and running the same role against them, one at a time?
The following achieves the result I am looking for, but is clunky, and not dynamic in length.
- name: run on us-east-1a
hosts: "us-east-1a"
roles:
- my-role
- name: run on us-east-1b
hosts: "us-east-1b"
roles:
- my-role
- name: run on us-east-1c
hosts: "us-east-1c"
roles:
- my-role
I think the only way to (1) have a common code and (2) serialise play execution per group of hosts (with targets inside a group running in parallel) would be to split your playbook into two:
playbook-main.yml
---
- import_playbook: playbook-sub.yml
vars:
host_group_to_run: us-east-1a
- import_playbook: playbook-sub.yml
vars:
host_group_to_run: us-east-1b
- import_playbook: playbook-sub.yml
vars:
host_group_to_run: us-east-1c
playbook-sub.yml
- hosts: "{{ host_group_to_run }}"
roles:
- my-role
# other common code
If you wanted to serialise per host, then there is a serial declaration that might be used in conjunction with this suggestion, but despite your comments and edit, it's unclear because once you refer to us-east-1a as a "host" in singular form, other times as a "group of hosts" or an "availability zone".
Will host patterns do the job?:
- name: run on us-east-1a
hosts: us-east-1a,us-east-1b,us-east-1c
roles:
- my-role
Update: #techraf has opened my eyes with his comment – host pattern alone will not do the job.
It will just concatenate all hosts from all groups.
But in a predictable way, which in some cases can be used to iterate hosts in every group separately.
See this answer for details.
- name: "API"
hosts: api
vars:
platform: "{{ application.api }}"
vars_files:
- vars/application-vars.yml
tasks:
- include: tasks/application-install.yml
- name: "JOBS"
hosts: jobs
vars:
platform: "{{ application.jobs }}"
vars_files:
- vars/application-vars.yml
tasks:
- include: tasks/application-install.yml
playbook like before described, can I execute this difference tasks on difference hosts in the same time as parallel way?
No sure what do you actually want, but I'd combine it into single play:
- hosts: api:jobs
tasks:
- include: tasks/application-install.yml
And add group vars to inventory:
[api:vars]
platform="{{ application.api }}"
[jobs:vars]
platform="{{ application.jobs }}"
This way you can run your playbook on all hosts at once, and also you can use --limit option to choose api or jobs group only.
you can run more playbooks using "ansible-playbook [OPTIONS] *.yml" command. This will execute all the playbooks NOT IN PARALLEL WAY, but in serial way, so first one playbook and after the execution, another playbook. This command can be helpful if you have many playbooks.
I don't know a way to execute more playbooks in parallel.